Skip to content

irb in a Windows console: Fiddle finds A exports, pack("p") writes a real address - #39

Merged
Largo merged 2 commits into
masterfrom
fix/pack-pointer
Oct 28, 2026
Merged

Largo merged 2 commits into
masterfrom
fix/pack-pointer

Conversation

@Largo

@Largo Largo commented Oct 28, 2026 •

Copy link
Copy Markdown
Owner

irb starts and runs in a Windows console again, with reline exactly as Ruby ships it. Both bugs were in IronRuby, not in reline.

1. Fiddle looks a Win32 name up as CRuby does

reline asks Fiddle for FillConsoleOutputCharacter, but kernel32 exports only the ...A and ...W variants. In 4.0.2, irb therefore died at startup with unknown symbol "FillConsoleOutputCharacter", piped or not.

On Windows, Handle#[] now tries the name as given, then with the A suffix, then the stdcall decorations @0 to @252, plain and after the A. That is the order of fiddle 1.1.8's fiddle_handle_find_func.

This change is the same as #38's 920e6ca8: FiddleHandle.cs and package-release.sh are byte for byte identical. The test files are not: both PRs add Fiddle_AnsiSuffix1 at the same spot, and this one adds the PackPointer_* tests next to it. So merge this PR before #38. #38 then takes this PR's two test files, which contain everything its own do, reverts its reline change, and updates its 4.0.3 notes. Merged the other way round, #38's reline change would land on master.

2. pack("p") writes a real address

[str].pack("p") and ("P") wrote an index into the packed string's list of Strings, not an address. reline's Win32API hands the console API every buffer as [str].pack("p").unpack("q*") and reads what the call wrote from str. So in a real console, irb crashed with an access violation in fiddle_calli, and so would any gem that passes buffers that way.

  • Native block per String: each String that pack points at gets its own native block (PackedPointers), holding its bytes and a NUL for as long as the String lives. pack writes the block's address.
  • Copy in and back: Fiddle::Function#call recognises the address among its pointer arguments. It copies the String into the block before the call and the block back into the String after it. In CRuby, the String's own buffer gives the same result.
  • unpack("p"): still finds only Strings packed into that same string ("no associated pointer"), now by address.

reline/io/windows.rb is not touched; it stays as upstream has it. #38 changed it to pass Strings to Fiddle instead, and it reverts that once this PR is merged.

Tests

  • Fiddle_AnsiSuffix1: the A suffix, and the message for an unknown symbol.
  • PackPointer_RealAddress1, all platforms: C reads and writes through pack("p") (strlen, memset, Fiddle::Pointer#to_s); the unpack round trip, nil, and the error for a foreign address. It checks the address before C touches it, so a regression fails the test instead of killing the run. Without the fix it reports the old indexes, [:not_an_address, 1, 1].
  • PackPointer_RelineWin32Api1, Windows: reline's own, unmodified Win32API fills a buffer through GetModuleFileNameA. That is the path that crashed.
  • CI smoke test and release check: irb is started with piped input on every platform, and must answer 42.
  • CI test step: a run that did not finish now fails it. An access violation ends the test process and leaves no > FAILED line to count, so until now such a run passed.

Checked

  • ruby/spec core/array/pack and core/string/unpack: 3193 examples, 0 failures. The full C# suite passes.
  • A probe that reads, writes and unpacks through pack("p") gives output byte-identical to CRuby 4.0.1.
  • Real console (a ConPTY terminal tab): a script queues key events for p 6 * 7, Enter, exit, Enter with WriteConsoleInputW, then runs irb, which reads them through ReadConsoleInputW. CRuby 4.0.1 and this branch show the same thing: the prompt, 42, and a clean exit. On master, irb crashes with 0xC0000005.

Summary by CodeRabbit

  • New Features
    • Packing strings as pointers now provides their native memory addresses, allowing packed strings to be used with Fiddle and unpacked back into associated strings.
    • Fiddle symbol lookup on Windows now checks common ANSI-suffixed and stdcall-decorated symbol names.
  • Bug Fixes
    • Changes made through Fiddle to eligible packed string buffers are reflected in the associated string. Frozen strings are not modified.

Largo added 2 commits October 28, 2026 16:25
…n packaging

On Windows, Handle#[] tries a name as given, then with the ANSI "A" suffix,
then with the stdcall decorations @0..@252, plain and after the A - the
order of fiddle 1.1.8's fiddle_handle_find_func. reline asks for
FillConsoleOutputCharacter, which kernel32 exports only as ...A and ...W,
so irb died at startup in 4.0.2. The same change is in #38 (920e6ca).

Fiddle_AnsiSuffix1 tests it; CI's smoke test and the release check start
irb with piped input on every platform.
[str].pack("p") and ("P") wrote an index into the packed string's list of
Strings, not an address. reline's Win32API hands the console API every
buffer as [str].pack("p").unpack("q*") and reads what the call wrote from
str, so irb in a Windows console crashed with an access violation; so
would any gem that passes buffers that way.

Now each String pack points at gets a native block of its own
(PackedPointers), holding its bytes and a NUL for as long as the String
lives, and pack writes the block's address. Fiddle::Function#call knows
the address among its pointer arguments: it copies the String into the
block before the call and the block back into the String after it, which
is what the String's own buffer gives in CRuby. unpack("p") still finds
only Strings packed into that same string ("no associated pointer"), now
by address. reline/io/windows.rb stays as upstream has it.

- Tests: PackPointer_RealAddress1 (C reads and writes through pack("p"),
  strlen and memset, all platforms) and PackPointer_RelineWin32Api1
  (reline's own Win32API filling a buffer, Windows). Both check the
  address before C touches it, so a regression fails them rather than
  killing the run; without this change they report the old indexes.
- CI fails a test run that did not finish: an access violation ends the
  process and leaves no "> FAILED" line to count.

ruby/spec core/array/pack and core/string/unpack (3193 examples) and the
C# suite pass; reading, writing and unpacking through pack("p") gives
byte-identical output to CRuby 4.0.1; and irb in a real console, with
typed input, answers as CRuby's does.
@coderabbitai

coderabbitai Bot commented Oct 28, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

No actionable comments were generated in the recent review.

Recent review info
Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 9f7335c7-6f7e-4a03-bdf5-3c529abe5798
Commits

Reviewing files that changed from the base of the PR and between 99711fd and 3b486b6.

Files selected for processing: 8
  • .github/workflows/ci.yml
  • Src/IronRuby.Tests/RubyTests.cs
  • Src/IronRuby.Tests/Runtime/MiscTests.cs
  • Src/Libraries/Builtins/PackedPointers.cs
  • Src/Libraries/Builtins/RubyEncoder.cs
  • Src/Libraries/Fiddle/FiddleCall.cs
  • Src/Libraries/Fiddle/FiddleHandle.cs
  • Util/package-release.sh

Included review availability: This review used your included allowance. Your plan provides up to 1 included review per hour; 0 remain after this review.

Walkthrough

The changes add native addresses for packed Ruby strings, update Fiddle pointer handling and Windows symbol lookup, and add regression tests and startup checks. CI also detects test runs without terminal output.

Changes: Fiddle and runtime integration

Layer / File(s) Summary
Packed string addresses
Src/Libraries/Builtins/PackedPointers.cs, Src/Libraries/Builtins/RubyEncoder.cs, Src/IronRuby.Tests/Runtime/MiscTests.cs
Packed strings now use registered native buffers. Packing stores the buffer address, and unpacking resolves it against associated strings. Regression tests cover pointer reads, writes, round-tripping, nil, and unassociated addresses.
Fiddle packed-pointer calls
Src/Libraries/Fiddle/FiddleCall.cs, Src/IronRuby.Tests/RubyTests.cs, Src/IronRuby.Tests/Runtime/MiscTests.cs
Fiddle tracks packed string blocks during native calls and copies eligible changes back. A regression test covers Reline's Windows API call with a packed string buffer.
Windows symbol lookup
Src/Libraries/Fiddle/FiddleHandle.cs, Src/IronRuby.Tests/Runtime/MiscTests.cs
Windows symbol lookup tries ANSI-suffixed and stdcall-decorated names. Tests check lookup results and missing-symbol errors.
Startup and CI checks
.github/workflows/ci.yml, Util/package-release.sh
Startup checks run an expression through irb on Windows and non-Windows platforms. CI reports an error and prints the last 40 output lines if test output has neither a terminal PASSED line nor a Repro: line.

Priority: Normal

Estimated code review effort: 3 (Moderate) | ~25 minutes

Change: Bug fix

Sequence Diagram(s)

sequenceDiagram
  participant RubyEncoder
  participant PackedPointers
  participant FiddleCall
  participant FiddleTarget
  RubyEncoder->>PackedPointers: Get the string buffer address
  RubyEncoder->>RubyEncoder: Pack the native address
  FiddleCall->>PackedPointers: Find the block for the address
  FiddleCall->>FiddleTarget: Pass the pointer to the native call
  FiddleTarget-->>FiddleCall: Return from the native call
  FiddleCall->>PackedPointers: Copy eligible buffer changes back
Loading

Possibly related PRs

  • Largo/ironruby#35: Adds the launcher contract used by the irb smoke checks in this pull request.

Suggested reviewers: claude

Merge Risk: Minimal · up to 3b486

No established issue prevents merging after normal checks.

Pre-merge checks: Passed 5
Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title clearly identifies the two main changes: Windows Fiddle symbol lookup for A exports and real native addresses from pack("p").
Docstring Coverage Passed Docstring coverage is 81.82% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentati…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Docstring Coverage

Explanation

Docstring coverage is 81.82% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentation-only edits and deletions within surviving functions are included; unchanged diff context and fully deleted functions are excluded. Analyzed 11 functions across 7 files. (1 skipped: 1 unsupported.)

Finishing Touches
📝 Generate docstrings
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@Largo
Largo merged commit 79aec55 into master Oct 28, 2026
12 of 13 checks passed
Largo added a commit that referenced this pull request Oct 28, 2026
…upstream ships it

#39 makes [str].pack("p") a real address, so reline's Win32API works as Ruby ships it.
This branch's change to Src/StdLib/ruby/4.0/reline/io/windows.rb goes back to upstream's
file, as on master. RubyTests.cs and MiscTests.cs are #39's, which hold this branch's
Fiddle_AnsiSuffix1 and #39's PackPointer tests; the release note says how irb was fixed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant