Skip to content

An open file is written and stated through its paged object, one write at a time, against credit - #1258

Merged
Japabu merged 18 commits into
mainfrom
fileops-step1
Nov 12, 2026
Merged

Japabu merged 18 commits into
mainfrom
fileops-step1

Conversation

@Japabu

@Japabu Japabu commented Nov 11, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

Every write and fstat of a served file is a request to its file server. On the T14 the round trip alone is about
7 µs with nothing in it (the split on metal, PR #1225's evidence), a 4 KiB write copies its bytes three times, and on
DATA each write call reserves room on the volume, 4 to 14 µs more. fs.mem.fstat reads 7.6 µs against Linux's
0.21, fs.disk.write_4k_at 22.4 µs against 0.72, fs.mem.seq_write.4096 438 MB/s against 3,605.

Change

A holder of a file's paged object, which an open already hands the client for reads, now also writes the file and
reads its stat without a request: SYS_PAGED_WRITE, SYS_PAGED_APPEND and SYS_PAGED_STAT. Every open of a
regular file on a server with a pager is answered with a holder, minted to write where the file is opened to write
and holders can write it.

  • One write at a time, as POSIX 2.9.7 requires. Each object has one write side. A holder's write copies its
    bytes into frames held aside first; then, in one hold of the pager's lock, it takes the side, is checked whole
    (its span at most 64 pages and ending within a holder's reach, every page present or reading as zeros,
    the credit and the dirty ceiling covering each page it turns dirty) and is admitted; its copy into the object is
    the kernel's own, and the side is given back. It never waits: while anyone holds or waits for the side, or any
    check fails, it is refused with nothing written, and the SDK sends the write to the server. A write longer than
    64 pages therefore goes to the server, which writes it whole up to its window, as before this change; no write
    answers a short count. The server takes the side with Hold for its own writes, truncations and reads, however
    many calls and waits they take, and is told the size the other writers left: its append lands there, and a
    write it cuts back after a refusal loses only its own growth.
  • An append is its own call, and every holder's write ends within a file's largest size. SYS_PAGED_APPEND
    reads the end under the side and checks it in that same hold of the lock; no offset value means "append". No
    holder's write, positioned or appended, ends past PAGED_HOLDER_END (16 TiB, the largest file the file system
    has, which the SDK asserts): one that would is refused as a long one is, and the server writes or refuses it as
    it would any other.
  • Reads and statements see each write whole, bounded. A holder's read waits out any write under way, the
    server's included, and reads again where one began meanwhile. At the third write it meets it answers
    WouldBlock, and the SDK reads through the server, whose read runs under its hold; a waiting server goes before
    any new holder's write, so the bound holds. While the server holds the side, a holder's fstat answers the size
    and time of the moment it took it, and the server stamps its writes' time inside its hold. Whoever waits for the
    side waits only on a holder's copy inside the kernel, never on anything a holder controls; the server holds it
    across its own disk I/O.
  • Credit is room. The server grants credit only for pages whose write-back the volume holds room for, so a full
    volume is still answered at the write and never at a later write-back. On DATA that room is a pool of the
    engine's blocks, 64 pages a file, placed where each page is written back (a new engine reservation bound to no
    range of the file, write_placed); it is made up after each of the server's own writes and taken back when the
    last holder goes. A page a holder paid for gives the credit back where the server writes it itself or the page
    goes before its write-back, so the pool holds one bulk beside the pages owed. FAT grants none, so its holders
    read and stat and its writes go to the server; MEMORY's objects are anonymous and need none. A sealed object
    refuses a holder's write as TextBusy, a kind of its own (ETXTBSY, ExecutableFileBusy).
  • Attributes. The object keeps the size and the modification time holders' writes move, and a 64-byte record
    the server writes (kind, links, volume, node, the other times) that the kernel never reads. While a holder may
    have written a file, the server answers its stat, listing and seek from the end from the object; the volume
    adopts the object's size and time at each write-back and truncate, so a persisted size never covers a byte not
    yet written back, and once no holder is left and nothing waits for a write-back they are the volume's again.
  • Write-back. The first page a holder dirties after the server last took the range posts one notice (not one
    per write); a write-back, a sync, an fsync, a write-back for room and the last holder's going take the range from
    the object, so nothing a holder wrote waits on a notice being read. The last holder's notice waits for a holder's
    write still under way.

Main's #1233, merged here, ends a holder's wait for an absent page where the object's version moved. A read here
then begins again rather than answering, so a restarted read now asks for its pages anew: the range it named
before may have been supplied and then made absent again by the write it met, and the pager would otherwise never
hear of it.

The manual's memory and files pages and the interface page say the design and the guarantee.

Checks

Round 8 at d1a8554, and main merged at 3581821.

  • The seal answers TextBusy (above). A holder's write that begins once the object's last holder went is refused Gone, and the no-holders notice is re-checked against writes under way as well as holders. owe_dirtied went; its two callers mark and unmark the list. The slot table's test marks slot 3 on both lists.
  • Every kernel library test (510) with sched-check, the kernel's three clippy shapes, the SDK's tests (215) and two clippy shapes, the ABI's tests and clippy, and the testcases image, whose sysroot builds libc's and std's new arms for both architectures: rc 0 at d1a8554. The control for the slot table's test, a removal that unmarks only list 0, builds and reds it on the list-1 assertion. Logs: r8b/checks.
  • Main 3c5c555 (The shared boots run as guest rows, each member judged by its own exit #1271, the shared boots as guest rows) merged as 3581821 with no file changed on both sides; cargo test --test toyos-checks rc 0 (r8b/merge).
  • The refusal of a write begun after the last holder went rests on inspection: no host test reaches the kernel object, and a guest test would need a close timed inside another thread's write.

Main merged again at a2e4dfd (main d1cefba, with #1273's on-demand slots). Five files conflicted:

At the merge: every kernel library test (510), the kernel's three clippy shapes, bcachefs (208) and the file server (132 + 3) with their clippy (main's four findings), the SDK, the ABI, and the testcases image, rc 0 (r8).

Main merged again at c809599 (64 commits). Four files conflicted, and one call stopped compiling.

  • The paged read: A paged read takes two passes of its pager's lock and populates nothing ahead #1245's read returns once a copy reaches the read's end, after its second pass of the lock. Here that return sits where no write met the read. A copy that met a write counts it and begins again, as before, and nothing populates ahead.
  • The paged system calls: A thread carries its process's handles, and a lookup takes no lock but the table's #1246 looks a holder up under the handle table's own lock. The lookup keeps that form, with the right each call needs.
  • The engine's tests and the memory filesystem's record: both sides kept.
  • A reserved write's bytes go to the device at once, at the session's depth #1216 split write_run and gave reserved writes their own path: plan every piece, send each stretch of contiguous sectors once, then point the extents, eight fresh ones of a bucket a transaction. The pool's placed writes now take that same path (round 6, 66de9f9): they plan their pieces onto the pool's lowest unwritten blocks with a cursor and share the reserved write's send-then-point phase, which counts blocks written by sector. write_run is main's. A placed write checks the volume is writable before its bytes go, because they now go before any transaction. A test shows a placed write over contiguous pool blocks sends each stretch once before any pointer, and a refused send leaves the file and the pool as they were. Logs: ea5a9d8c/r6.

The kernel's three clippy shapes and pure tests, the engine's and the file server's tests and clippy, and the SDK's and ABI's checks, all at the merge: rc 0 (the file server's clippy findings are main's four). Logs: 7700bf33/r2/step1-merge.

Host, every crate the change touches. The round-2 fixes at a5a4ea5, main merged at 16a50e2 (its kernel, file
server and engine checked before #1225 joined it, which touches only the SDK's files module), the read's count
at 1b46c51, and a holder's reach at b55f2b5; 1ee3e6d adds only issue records, 4a2d78d pagerprobe's
holder writes and comments, 230a2ad pagerprobe's labels and records, 695146e a record line, c809599 main
merged, 66de9f9 the pool's writes on the reserved write's path (above), 4240a3d the data slot's
contract restated and the pool flag passed by its caller, and the head, a2e4dfd, main merged with #1273. Logs:
00bdb14a/r2 and
de2e2055/r3, each ending
with its exit code, the README saying which tree each set ran on.

check why it covers the change result
cargo test -p kernel --lib paged:: the write side, the hold's recorded size and time, the holder's span (whole, refused past 64 pages and past a holder's reach, up to the last offsets a u64 holds), credit a holder paid for, main's wait tests and model with a read that begins again, and a model driving a read, two holders' writes and the pager's through every interleaving rc 0, 31 passed at b55f2b5
cargo test --manifest-path system/fileserver/Cargo.toml the server's hold around its writes, reads and truncations, its stamp inside the hold, the append at the end holders left, the write-back for room, credit given back, holders' writes to the volume through the pool rc 0, 132 + 3 passed at b55f2b5
cargo test -p bcachefs --lib the engine's placed pool, a short pool, and its share of the reserved write's send-then-point phase rc 0, 208 passed at 4240a3d's tree (34 in tests::data:: at the merge)
cargo test -p toyos --lib fs:: and files::, cargo test -p toyos-abi the record, the descriptor paths, the calls rc 0 at b55f2b5
clippy's shapes for kernel (host, x86-64, aarch64), toyos (two), bcachefs, toyos-abi, warnings denied rc 0 each
the same shapes for system/fileserver, every target four findings, the same at main e10b6fd, in lines this PR does not change (Risks)

Negative controls, each a patch applied, built and run, then restored, every one red on its test's assertion:
an append's end left unchecked (B1, the pure span helper); stat answering the live size while the server holds
(B2); the server stamping after its hold (B2); a long span clamped rather than refused (B3); a holder's write
taking the side while the server waits for it (S1, the interleaving model); a restarted read that does not ask
anew (main's wait model, which then leaves a read waiting for a page nobody was asked for); and a holder's span
checked against the last page rather than its reach. Round 1's controls stand for what this round did not change.

Guest, all passed (run 43152496728, at cb8701b, whose code the head 001f812 keeps; log; earlier runs
were at superseded heads or ended in startup_failure while Actions was degraded):
a_row_reaches_its_own_state_and_no_other, the_data_partition_beside_the_stick_is_designated,
the_log_lands_on_the_boot_stick (LOG on FAT: its holders cannot write, so every write goes to the server),
nvme_disk_boots, virt_user_mode (aarch64: the paged object is shared kernel code), and
testcases_members_exit_0 (below). 43142627680 passed the five rows at a2e4dfd; 43146950193, at 3581821,
was cancelled while queued once the control showed the stale member it would red. Runs at superseded heads
(43076343465, 43090603584, 43092641282) were cancelled before they ran; 43092689328's cancel is refused by GitHub
as already complete though it reads queued. 43096096727 ended in startup_failure with no job.

That holders' writes are admitted is pagerprobe's claim. Its helper writes through write holders its pager
mints. On anonymous memory it checks:

  • a page's count, an append's new size, the size and moved time a statement answers, and the bytes read back;
  • WouldBlock with nothing changed for a write past a holder's reach;
  • PermissionDenied from a read-only holder.

The 64-page span limit is the pure holder_span's, host-tested with its control red; pagerprobe has no room
that would let the limit alone refuse a longer span.

It then writes a page to an object granted a page of credit. Once the pager has that object's Dirtied notice,
it checks take_dirtied and the size Hold answers, and reads both objects back as their owner.

a_pager_s_end_answers_its_waiting_holders_gone, which starts pagerprobe, and the served-file tests
(a_served_files_read_never_tears_against_its_write, a_served_files_writes_land_whole) are members of the
testcases shared boot, which #1271 made the guest row testcases_members_exit_0, each member judged by its own
exit. Run 43152496728 at cb8701b, the head's code (001f812 adds only issue text), passed it: all 242
members exited 0 and the list's reboot ended the boot (log). A green run keeps no console, so the order of
pagerprobe's lines and the absence of a paged overdue report are not shown; the T14's shared boot owes the latter.

Its negative control is the kernel refusing every holder write: branch fileops-step1-ctl-refuse, c97966e (the
head 3581821 and that one patch). Run 43146956481 there went red on two of 242 members
(r9/control-run):

  • a_pager_s_end_answers_its_waiting_holders_gone: pagerprobe exited 101 after 30.49 s of system calls, its
    30-second wait for the helper's dirtying of the credited object. The panic text was not kept
    (issues/a-red-run-keeps-no-serial-for-a-boot-whose-uart-is-qemus-stdio.md, which takes this run as an
    instance), so the reason, the helper's first holder write answered WouldBlock, is inferred from that timing
    and the code, not read.
  • a_served_file_reads_through_its_paged_object, at its first assertion, "an open to write alone carries no
    holder", before it writes anything; it writes only through the server. Not the patch's effect: the assertion
    was main's rule, and this change answers an open to write with a holder too, so the head would red it as well. Round
    9 (cb8701b) has it take the writer's holder and drop it before the writer's close; the member type-checks
    against the ToyOS sysroot (r9/member), and the run at cb8701b passed it.
    What the run must show and what the control showed are recorded in
    issues/a-file-server-request-costs-three-to-twelve-times-a-kernel-file-call.md.
    So is the gap no test closes yet: that opening a file on /tmp or /home mints a write holder, and that the SDK
    writes through it before it asks the server, rests on inspection and on the owed osbench_disk rows.

Metal: none. The benches are down, so this change is not measured on metal: its speed-up is not yet measured, and
no metal row has run it. Owed on metal, recorded in the issue records:

  • the T14's whole testcases shared boot on main after this change: every member passes, the two served-file
    tests and a_pager_s_end_answers_its_waiting_holders_gone among them; no paged overdue report; --fat32-check
    clean (issues/a-file-server-request-costs-three-to-twelve-times-a-kernel-file-call.md);
  • osbench_disk on the T14, main from before this change and after, two boots a side in A, B, B, A order: fstat
    about 1 µs or below on both volumes, write_4k_at a few µs, seq_write.4096 up (on /tmp well above 438 MB/s),
    DATA's 1 MiB sequential write, listings and path calls within main before's spread, no row beyond twice main
    before's (the same record, and issues/a-memory-filesystem-call-costs-toyos-up-to-35-times-linuxs.md);
  • the bulk: osbench_disk on the T14 at bulk 16 and 256 pages beside main's 64, scratch builds that never land, as
    the measurement behind the bulk, not a bar
    (issues/a-reserved-data-block-holds-a-transactions-metadata-credit-until-written.md).

Risks and what is left

  • The kernel's write side is exercised only in guests. The host tests reach the protocol on the pure object,
    through every interleaving of the model, and the file server against a fake; nothing on the host runs the kernel
    object's holder_write, read or hold. On the guest, the two served-file tests and the pager member
    (testcases_members_exit_0, 43152496728, green) show whole writes and reads, whole appends and a holder's write
    admitted against credit, under QEMU's concurrency on two vCPUs. Nothing has run them on metal, and a refused
    write begun after the last holder went rests on inspection.

  • Copies that touch a program's memory while holding pager frames, for map7b's step 2. Two places:

    • a holder's write copies the caller's bytes in one place (holder_write's staging loop) while holding up to
      128 frames drawn from the pager's core, one for each 4 KiB of its bytes and one for each page it would
      install; it holds no lock, pin or write side there;
    • a holder's read copies out to the caller in one place (read's copy step, as before this change) while
      holding pins on up to 16 of the object's frames.

    Once step 2 lets a caller's buffer be an absent page of a mapped file, either copy can park on that page while
    holding those frames. Step 2's pinned-copy mode (stop, give back or unpin, fault in holding nothing, retry) is for
    these two places. Before step 2 no user page is a parked paged page, so nothing here waits so.

  • A holder's write draws its frames from the pager's core: under memory pressure it is refused and goes to the
    server, which answers as before.

  • The SDK cannot tell a holder minted only to read from one that writes, so on FAT, and on a file opened to write
    where holders do not write, each write still makes one refused system call before it goes to the server.

  • Each DATA file a holder may write holds 64 reserved blocks and their metadata credit, 512 KiB, while it is open;
    recorded in issues/a-reserved-data-block-holds-a-transactions-metadata-credit-until-written.md.

  • system/fileserver has four clippy findings under the lint shapes, all in lines this PR does not change and the
    same at main (listing.rs, two in pages/tests.rs from earlier commits, writeback.rs); not fixed here.

  • Path calls (open, stat by path, rename, unlink) are unchanged; they are the next steps (the server's resolution by
    node, then the round trip itself). The per-capability turnstile no longer serialises a process's writes and
    fstat, and stays for the rest (issues/a-process-s-threads-take-turns-..., recorded here, to be removed next).

🤖 Generated with Claude Code

@Japabu

Japabu commented Nov 11, 2026

Copy link
Copy Markdown
Collaborator Author

Review of #1258 at ee1f814 (base 67713cb)

Architecture

The ownership matches the owner's decision:

  • Kernel. The paged object admits a holder's write against credit the pager grants. It keeps the size, the modification time and 64 opaque bytes the pager writes, so fstat needs no request. Only the kernel can admit a write without the server, so the kernel addition is justified.
  • Types. The object's record is opaque bytes at the ABI. The file protocol's Record type stays in toyos::fs.
  • Server. While a write holder exists, the server reads size and time from the object (Files::meta). The volume takes the object's values at write-back and truncate.
  • Admission. Pages::admit matches admission-interface.md: refusals in the order Gone, Sealed, NoCredit, DirtyFull; the seal check in admit, dirty supply and resize both ways; the Busy arm in word().
  • write_placed (engine) is correct. It always takes the lowest unwritten block, so the written blocks of a pool are always a prefix and cut_blocks only ever cuts unwritten ones (the assert in cut holds). Runs written whole are dropped while the pool lives on to be extended. Metadata credit is held for each unwritten block. Its test covers out-of-order placement, a write across two blocks, a spent pool, extension, the cut, and fsck.
  • Size. About +1,500 production lines, +440 test lines and −219 lines.

Merges. This PR merges cleanly with #1225, with #1259 (map7b step 1, which does not touch the paged object) and with #1245 (git merge-tree).

Required

1. BLOCKER kernel/src/syscall/paged.rs:102: any process can panic the kernel.

  • (offset / page + PAGED_RANGE_PAGES) * page overflows for any offset in the last 512 pages below LAST_END. For example, offset = LAST_END - 4096 with len = 1 passes within_reach, and then 2^52 + 510 pages times 4096 overflows.
  • The kernel builds with overflow-checks = true ([profile.toyos]), and this arithmetic runs before the handle and rights check. Any process can therefore trigger the panic with any handle value.
  • Fix: len.min(PAGED_RANGE_PAGES * page - offset % page).

2. BLOCKER system/fileserver/src/pages.rs:519: write_back_some sees only f.dirty, so holder-written pages trigger false ENOSPC and EIO.

  • Pages a holder dirtied stay in the object until take_dirtied. A file with only holder-dirtied pages has dirty == None, so write_back_some skips it. Its callers at :469, :493 and :591 then answer NoSpace for a server write that hits the dirty ceiling, and, once the table fills, fail a read's supply: EIO for a holder, ENOSPC through the server.
  • Example: a build that writes 128 or more files of up to 256 KiB each within the 2 s write-back delay fills DATA's 8,192-page ceiling. The next refused holder write goes to the server and fails with ENOSPC. That holder write was refused only because of the ceiling.
  • Fix: use may_owe(), as sync and fsync already do.
  • Cheapest check: a pages/tests.rs test. Fill a small Fake ceiling with holder_write across two files, then write a third through the server.

3. BLOCKER pages.rs:538,544-556: a failed server write deletes another writer's completed write.

  • A server write that fails partway resizes the object to old.max(landed), where old is the object's size when the write started.
  • The server's own write never grows the object past landed, because sizes grow only at unpin and dirty supply. So the only thing this resize can remove is growth from a concurrent holder write, one that already returned success to its caller. On main nothing else could write concurrently, so the resize was harmless.
  • Finding 2 makes such failures reachable: put fails with NoSpace at the ceiling.
  • Fix: keep unreserve and do not shrink the object below the size other writers left it.

4. BLOCKER kernel/src/object/paged.rs:576-583,593-598,648-668: a reader can wait forever, and nothing reports it.

  • The object has one version for all its writes, so any holder write anywhere in the file restarts a read.
  • A writer calling in a tight loop can therefore keep a large read restarting forever.
  • Each individual wait_written ends quickly, so Overdue never fires. The livelock is silent, and the PR records it nowhere in issues/.
  • Fix: bound it. For example, once a read has had to restart, refuse new holder writes with WouldBlock, so they go to the server, until the read completes. The read then waits only for the copies already under way.
  • If the owner accepts the unbounded wait instead, report the read's total wait as overdue and record it as a hang defect with owner and exit.

5. BLOCKER Atomicity is narrower than the manual says, and narrower than on main.

  • docs/src/architecture/memory.md:56 says "A holder's read sees each write whole or not at all, whoever wrote it." That is false on the fallback path. A holder read that meets a pager write gets WouldBlock, and the SDK asks the server. The server's own read (owner_read, waits = false) is not ordered against holder writes, so it can return a holder's write half done.
  • Reads through the server's offset (shared descriptions) lost the same guarantee against holder writes.
  • Writes are no longer atomic with respect to each other. On main the server serialised every write. Now overlapping holder writes copy into the same pinned frames at once, and holder writes interleave with the server's.
  • O_APPEND loses its guarantee too: the server reads the object's end (main.rs:1331), and a concurrent holder extension can land on the same bytes before the append does.
  • XSH 2.9.7 requires all of these, and ToyOS's read path is built to honour it. Giving it up is a change to the owner's requirements, not an implementation detail.
  • Fix: deliver the guarantees, or take the relaxation to the owner. Either way, record it, and have the manual state exactly what holds.

6. BLOCKER Known compromises that are not in issues/ (No silent debt):

  • (a) The pool keeps room for pages the server overwrote. A holder spends a credit on page P, the server then writes P with its own ticket, and write-back uses the ticket. The pool's block for P stays held, and top_up adds another. a_write_past_the_credit_goes_to_the_server_which_makes_it_up does exactly this.
    • This is unbounded for the life of a write holder, and it breaks the plan's bound of "at most one bulk per object open to write".
    • On DATA, each leaked block also holds K_TX * 8 bytes of metadata credit.
    • The result is a full-volume answer while room is actually free.
  • (b) O_APPEND is still a request. The PR body gives "the server owns the end" as the reason, which contradicts the decided rule that the object owns the size while writable. The real reason is that the kernel has no atomic way to reserve the file's end.
  • (c) issues/a-reserved-data-block-holds-a-transactions-metadata-credit-until-written.md is now out of date. Every DATA file open to write now holds a pool of 64 blocks plus their metadata credit (512 KiB) for as long as it is open, not only until write-back. That record must say so.

7. BLOCKER sdk/abi/src/syscall.rs:3184-3194: the paged_write contract is false in three places. The module header at kernel/src/object/paged.rs:14 makes the same first claim.

  • It says "Every page is admitted before the first byte lands". In fact the whole write is checked first and pages are admitted one chunk at a time. A concurrent holder can spend the credit or hit the dirty ceiling between chunks, or the pager can withdraw a page.
  • It says "Io for a failed page". A failed page is refused as Absent, which becomes WouldBlock, and no path returns Io.
  • It omits the 512-page cap that sys_paged_write applies.
  • kernel/pure/paged.rs's header states this correctly ("only a race cuts it short").

8. BLOCKER The bulk-size measurement the brief requires is missing.

  • The brief says 64 pages "with 16/256 measured". The metal request stages only 64.
  • At 64, any write over 256 KiB on DATA can never take the holder path. seq_write.1048576 pays a refused call that scans 256 pages under the pager lock, plus Status calls, on every write.

Optional

  • S1 Have a holder read wait out a pager write as it waits out a holder write. The reader already waits on its pager for supply, so this adds no new dependency, and the pager's writes end within the request that started them. It removes the fallback in finding 5, holder_writing, pager_writing() and the Written { pager } split.
  • S2 Fix 6(a) at its source: one bit per page records that a holder's credit covers it, and a pager write that lands on such a page returns that credit to the object. The pool then stays at one bulk.
  • S3 Mint a holder that can write only where it can actually write. Not where !volume.grants() and the object is not anonymous: on FAT that saves a refused system call per write. Not for opens with only O_APPEND: that saves 64 pages of room plus metadata credit per appending file, and a pwrite on such a descriptor would go to the server.
  • S4 may_owe() counts held, which includes read-only holders. Every sync therefore makes two kernel calls for every file a reader holds open. For files that are not anonymous, granting is enough.
  • S5 No production caller reads holder_fits's count of fresh pages (kernel/pure/paged.rs:736). Return Result<(), Refused>.
  • S6 NoHolders can be posted while a holder write is still under way, because the write holds the object, not a handle. After that, ungrant(u64::MAX) can cut the pool that the write's page needs at write-back, which then fails with Io. The SDK keeps the handle alive across the call, so only a client making raw system calls on its own file can trigger this. Defer the notice until holder_writing drains.
  • S7 Put the span arithmetic from finding 1 into a pure helper with a unit test that sits at LAST_END.

Evidence

  • Host: 13 steps rc 0. I checked them against the tree they ran on: 549c11c differs from ee1f814 only by a comment in owner.rs and the file server change, and the file server tests were rerun at ee1f814 (126 + 3 passed). The four controls are real patches, and each turned its test red. No host test reaches holder_write, holder_pages, wait_written or the system call layer. Findings 1 and 4 sit exactly there.
  • Guest: run 43061175625 is still queued. Since findings 1–3 change the write path these rows exercise, dispatch the same rows again at the fixed head. All five must PASS. the_log_lands_on_the_boot_stick must also show that FAT holders' writes are refused and the server takes them.
  • Metal, at the fixed head, B and A on the same day:
    • testcases: every member passes, including a_served_files_read_never_tears_against_its_write and a_pager_s_end_answers_its_waiting_holders_gone. No paged overdue report. --fat32-check clean.
    • fs.mem.fstat and fs.disk.fstat: from 7.6 µs to about 1 µs or less. It is one system call, and the server's own work was 0.36 µs, so a value near 7 µs means the holder path is not taken.
    • fs.disk.write_4k_at (and pwrite_4k on both volumes): from 22.4 µs to a few µs. There is no longer a reservation per call.
    • fs.mem.seq_write.4096: well above 438 MB/s. On DATA, seq_write.4096 should have its p50 on the holder path and a tail of one server write per 64 pages.
    • seq_write.1048576 on DATA within noise of A. readdir.1000 and the path rows (stat, open, rename) unchanged.
    • The 16 and 256 bulk sizes beside 64 (finding 8).
    • No row beyond 2× its recorded value.
  • After metal: update the current-state rows of the three issue records the handoff names, in this PR.

The PR description's "admits … before any byte lands, or refuses it with nothing written" and "Reads stay atomic" are inaccurate (findings 5 and 7). Correcting them is the orchestrator's job.

SEND BACK

Japabu added a commit that referenced this pull request Nov 11, 2026
…ting them is bounded

The round's ruling keeps POSIX's atomicity: writes to one file are atomic
with respect to each other and to reads, a holder's, the server's or an
append. Each object now has one write side. A holder's write copies its
bytes into frames held aside first, then takes the side in one hold of the
pager's lock, refused at once while anyone holds or waits for it, and is
checked and admitted whole there; its copy into the object is the kernel's
own. The server takes the side with a hold for its writes, truncations and
own reads, which also answers the size the other writers left: an append
through the server lands there, and a write refused part way cuts back only
its own growth, never a holder's (finding 3). A holder's append reads the
end under the side. A holder's read waits out any write, the server's too,
and after three restarts queues for the side, which no writer takes before
it, copies under it into frames held aside and copies out after giving it
back: bounded, and nothing holding a file's writes off waits on a program's
memory (finding 4). A host model drives a reader, two holders and the pager
through every interleaving of the protocol.

The span of a holder's write is clamped inside its 64 pages without
overflow at the last pages a u64 reaches (finding 1). The server's
write-back for room reaches files whose only dirty pages are holders'
(finding 2). A page a holder's credit paid for gives the credit back where
the server writes it or it goes before its write-back, so the pool holds
one bulk (finding 6). Write holders are minted only where holders write,
the last holder's notice waits for a holder's write under way, and the
contracts say what holds (finding 7). The metadata-credit record says what
a file open to write now holds.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Japabu and others added 2 commits November 11, 2026 20:16
…credit

Every write and fstat of a served file was a request to its file server: a
round trip of about 7 us on the T14 before any work, three copies of a
4 KiB write, and on DATA a reservation per call of 4 to 14 us. A holder of
the file's paged object now writes it and reads its size, time and the
rest of its stat with one system call, as it already read it.

The kernel admits a holder's write whole or refuses it before any byte
lands: every page of it present, or reading as zeros past the size or in
an anonymous hole, and the credit and the dirty ceiling covering each page
it turns dirty. Credit is pages the pager granted for which it holds room
on the volume, so a full volume is still answered at the write: any
refusal sends the holder to the server, whose write supplies, writes back
or answers "no space". A holder's write never waits on its pager, so a
read that meets one waits it out and reads again rather than ask the
server, whose own read is not ordered against it. A fresh page whose copy
faulted is removed rather than left dirty past the size.

The object keeps the size and the modification time holders' writes move,
and 64 bytes the pager writes and the kernel never reads. While a holder
may have written a file, the file server answers its stat, listing, seek
from the end and append from the object, and the volume adopts the
object's size and time at each write-back and truncate, so a persisted
size never covers a byte not yet written back; once no holder is left and
nothing waits for a write-back, they are the volume's again. The first page a holder dirties after the pager last took
the range posts one notice; a write-back, a sync, an fsync and the last
holder's going take the range from the object, so a notice not yet read
loses nothing.

On DATA the credit is a bulk of 64 pages, held as a pool of the engine's
blocks placed where each page is written back (`write_placed`, new: a
reservation bound to no range of the file), made up after each of the
server's own writes and taken back when the last holder goes. FAT grants
none, and its holders' writes go to the server; MEMORY's objects are
anonymous and need none. A sealed object refuses a holder's write as Busy
(ETXTBSY), the agreed seal check.

Records the per-capability turnstile, which still serialises a process's
path calls, measured with this work's diagnosis.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ting them is bounded

The round's ruling keeps POSIX's atomicity: writes to one file are atomic
with respect to each other and to reads, a holder's, the server's or an
append. Each object now has one write side. A holder's write copies its
bytes into frames held aside first, then takes the side in one hold of the
pager's lock, refused at once while anyone holds or waits for it, and is
checked and admitted whole there; its copy into the object is the kernel's
own. The server takes the side with a hold for its writes, truncations and
own reads, which also answers the size the other writers left: an append
through the server lands there, and a write refused part way cuts back only
its own growth, never a holder's (finding 3). A holder's append reads the
end under the side. A holder's read waits out any write, the server's too,
and after three restarts queues for the side, which no writer takes before
it, copies under it into frames held aside and copies out after giving it
back: bounded, and nothing holding a file's writes off waits on a program's
memory (finding 4). A host model drives a reader, two holders and the pager
through every interleaving of the protocol.

The span of a holder's write is clamped inside its 64 pages without
overflow at the last pages a u64 reaches (finding 1). The server's
write-back for room reaches files whose only dirty pages are holders'
(finding 2). A page a holder's credit paid for gives the credit back where
the server writes it or it goes before its write-back, so the pool holds
one bulk (finding 6). Write holders are minted only where holders write,
the last holder's notice waits for a holder's write under way, and the
contracts say what holds (finding 7). The metadata-credit record says what
a file open to write now holds.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Japabu

Japabu commented Nov 11, 2026

Copy link
Copy Markdown
Collaborator Author

Review r1, at 66702d6 (rebased onto main 45935f4):

  1. Fixed: holder_span clamps to HOLDER_PAGES * PAGE - offset % PAGE, a pure helper with a test at every one of the last 128 pages below LAST_END (the old arithmetic panics there: control f1).
  2. Fixed: write_back_some takes every file that may_owe(). Test a_server_write_at_the_ceiling_writes_back_what_holders_dirtied, control f2.
  3. Fixed by the ruling's serialization: the server's writes run under the object's Hold, which answers the size the other writers left. A holder's write meanwhile is refused and comes to the server, so the cut-back after a refusal only removes the write's own growth. Test a_refused_server_write_holds_holders_off_and_cuts_back_only_its_own, control f3.
  4. Fixed: after READ_RESTARTS (3) a read queues for the write side, which no writer takes before it. It copies at most 64 pages into frames held aside and copies them out after giving the side back. A pure model drives a read, two holders' writes and the pager's write through every interleaving: every read whole, no two writes at once, no deadlock, and nothing overtakes a queued read. Controls f4 and s1.
  5. Per the owner's ruling, POSIX 2.9.7 is kept. Each object has one write side: a holder's write (staged first, refused if the side is busy, never waiting), the server's Hold, or a queued read. Holder reads wait out the server's writes (S1); pager_writing, holder_writing and the WouldBlock fallback are gone. Appends: a holder's goes through the object (PAGED_APPEND), the end read under the side; the server's lands at the size Hold answers (test and control s2). Nothing holding the side touches a program's memory, so the server waits only on kernel copies. The manual states the guarantee, the 256 KiB parts and the short read under contention.
  6. (a) Fixed at the source (S2): a page carries the credit a holder paid; the pager's write of it, or its removal before write-back, gives it back (pure test, control f6; server test a_page_the_server_writes_gives_its_holders_credit_back). (b) Gone: appends go through the object. (c) The metadata-credit issue now records the 512 KiB a file open to write holds.
  7. Fixed: the paged_write and paged_read contracts and the module header say what holds; the 64-page cap is PAGED_HOLDER_PAGES in the ABI.
  8. Staged: diskbenchcase at bulk 16 and 256 beside 64; request below in the orchestrator's queue.

S3: write holders are minted only where holders write (anonymous objects, or a volume that grants). The SDK still tries one refused write on a read-only holder; recorded under Risks. S4: may_owe counts granting. S5: kept, since the fresh-page count now sizes the frames drawn up front. S6: fixed, the last holder's notice waits for a holder's write under way. S7: done (1).

New guest member: a_served_files_writes_land_whole (two writers and a reader over one file, two appenders, on DATA and /tmp).

🤖 Generated with Claude Code

@Japabu

Japabu commented Nov 11, 2026

Copy link
Copy Markdown
Collaborator Author

Review round 2 of #1258 at 66702d6 (base 45935f4)

Architecture

The write side is the right mechanism.

  • One owner, the kernel object, serializes every writer. The server's Hold makes its multi-call writes, truncations and reads count as one operation each.
  • The kernel refuses Write, a dirty Supply and Resize without the hold, so the round-1 finding 3 cannot happen.

Deadlock freedom holds. I traced every wait for the side:

  • hold waits only on two things, both bounded copies inside the kernel. One is a holder's write, whose bytes are staged before it takes the side and which copies between kernel frames under it. The other is a queued read, which pins and copies between kernel frames and gives the side back before any supply wait.
  • The server is single-threaded and never nests holds. Its write_back, fill, make_room and forget take no hold, and the kernel refuses a second hold of the same object as InvalidArgument.
  • A reader already waited on its pager for supply, so also waiting out the pager's hold adds no new dependency.
  • Lock order is unchanged: the pager's lock, then the allocator. No user copy happens under the lock.

Size of this round.

  • Production code grows by about +420 lines net: kernel object +276, the pure module +59, the server +26, and the SDK, ABI and system call layer about +48.
  • Tests grow by about +500 lines: the pure module +240, including the model; the server +139; the guest +121.
  • The round deletes EndWrite, Put and round 1's fallback from pager writes.

S1 below would remove the largest new mechanism.

Round-1 findings

  1. Resolved.

    • holder_span clamps the span without overflow.
    • The test covers the last 128 pages a u64 reaches.
    • Control f1 is red with the old arithmetic.
  2. Resolved.

    • write_back_some uses may_owe().
    • a_server_write_at_the_ceiling_writes_back_what_holders_dirtied covers it, and control f2 is red.
  3. Resolved.

    • The cut-back goes to old.max(landed), where old is the size Hold answered. No other writer can move the size while the hold lasts.
    • There is a test, and control f3 is red: the fake's hold assertion fires.
  4. Resolved as a bound.

    • After READ_RESTARTS, a read queues for the side.
    • take_holder refuses while a read is queued, and take_pager waits behind it.
    • The model and controls f4 and s1 are red as expected. See B4 for a defect in how the queued read holds frames.
  5. Mostly resolved. These hold now:

    • Writes are serialized for every pair.
    • Appends work through a holder, under the side, and through the server, at the size Hold answered.
    • Truncation runs under the hold.
    • A holder's read sees no torn write.
    • The server's reads run under the hold.

    fstat is still not atomic with writes, and XSH 2.9.7 lists it alongside read and write (B2).

  6. Resolved.

    • 6a: the credited bit. A pager write refunds the credit, and so does removal before write-back. There is a test, and control f6 is red.
    • 6b: a holder can append, but see B1.
    • 6c: the issue record now states the pool.
  7. Resolved for paged_write. Its contract now matches the code: the write is checked and admitted whole in one hold of the lock, BadAddress means nothing was written, and the 64-page limit is stated. This round made paged_read's contract incomplete (B5).

  8. Pending: bulk sizes 16 and 256 are queued on metal.

S3's remainder (Risks). Accepted. It affects only FAT: one refused system call before a request that costs far more. The SDK cannot see a handle's rights.

S6. Adopted through the writers count.

Required

B1. BLOCKER kernel/src/object/paged.rs:1330,1359-1361,1395, kernel/src/syscall/paged.rs:104-105, sdk/toyos/src/files/owner.rs:611: any process can panic the kernel through an append.

The bug.

  • An append takes offset = object.size() and never checks offset + len against LAST_END.
  • holder_span checks reach only for a positioned write, and append_fits checks only len.

The size can reach LAST_END.

  • A positioned holder write may end exactly there.
  • toyos_fs_pwrite passes the caller's offset straight to the holder.

Two calls in safe Rust do it on /tmp, which is anonymous and needs no credit:

  1. write_at(b"x", LAST_END - 1).
  2. Open the same file with .append(true) and call write(b"y").

The append's page is then LAST_END / PAGE. (page + 1) * PAGE at :1360 is 2^64, and with overflow-checks = true it panics while the pager's lock is held. offset + len at :1360 and :1395 overflows the same way for any append that crosses u64::MAX.

The same cause makes PAGED_APPEND = u64::MAX an in-band value.

  • FileExt::write_at(buf, u64::MAX) on a held file becomes an append that answers success.
  • Through the server, the same call is refused by bounds::in_a_file.
  • libc's pwrite refuses negative offsets; Rust's write_at does not.
  • Holder writes at offsets above fs::MAX_FILE_BYTES also skip the limit the server enforces.

Fix.

  • Kernel: refuse an append whose offset + len is not within_reach, in the same hold of the lock that reads the size. Put this in a pure helper with a unit test at LAST_END, as finding 1's fix did.
  • SDK: hand a positioned write to the holder only for offsets the file may hold. Alternatively, mark an append with a flag instead of an offset value.

B2. BLOCKER kernel/src/object/paged.rs:1244-1251, system/fileserver/src/pages.rs:580-592: a holder's fstat is not atomic with the server's writes, which the ruling requires.

The mechanism.

  • XSH 2.9.7 lists fstat with read and write.
  • fstat on a held file is now SYS_PAGED_STAT. It reads the size under the pager's lock without regard to the write side.
  • The server's write grows the size with each 16-page chunk of write_pages and of a dirty supply, across several calls under its hold.

What a holder can see.

  • A size partway through a server write.
  • After a refused write is cut back (:611), a size that then disappears.
  • On main, fstat was a request, serialized with every write.
  • A holder's own write is fine, because its size moves in one hold of the lock.
  • The server stamps the time after under_hold returns (:589). Between unhold and the stamp, fstat sees the new size with the old time.

Fix.

  • While the pager holds the side, have stat answer the size recorded at take_pager. Reads wait meanwhile, so that answer is ordered before the write.
  • Stamp inside the hold.
  • A pure test of the recorded size is enough.

B3. BLOCKER kernel/pure/paged.rs:466-468, kernel/src/syscall/paged.rs:106-109, docs/src/architecture/memory.md:60-61: write() now returns short counts from 256 KiB. On main it wrote whole up to the 2 MiB window.

What changed.

  • Round 1 clamped a holder's write at 512 pages, the same as the server's window. This round clamps it at 64.
  • The SDK returns the clamped count (owner.rs:138, :611), and libc's write passes it straight to C.

The effect.

  • A 1 MiB write() to /tmp returns 262,144.
  • On DATA it does the same whenever the credit covers 64 pages.
  • Rust's write_all and libc's stdio loop over short counts. A C program that checks write(fd, buf, n) != n now fails for any n between 256 KiB and 2 MiB.
  • The existing a_served_files_read_never_tears_against_its_write writes exactly 64 pages and asserts == BYTES. With one more page that assertion would fire on this branch and not on main.

The manual's sentence. "A write of more than 256 KiB may be taken in parts, each whole" suggests the system takes the parts. In fact the program does.

The owner's ruling and the brief do not cover this.

Fix (preferred).

  • Refuse a positioned span longer than 64 pages with WouldBlock, as an append longer than 63 is already refused.
  • The server then writes it whole under its hold, up to the window, as on main.
  • holder_span becomes a predicate, and the clamp goes.

Otherwise. Take short writes to the owner, and have the manual say plainly that such a write answers fewer bytes.

B4. BLOCKER kernel/src/object/paged.rs:645-672,763-766: a queued read holds up to 64 frames from its pager's core across two waits on that same pager.

The holding. read_queued draws its bounce frames before copy_under_side queues, and keeps them through two waits:

  • while it waits for the side, which the pager holds across disk I/O and room-making;
  • in wait_supplied, while it waits for the pager to supply a page.

Why that matters.

  • The core is what keeps the pager's own supplies independent of the allocator.
  • For MEMORY's pager, the core is only the 512-page headroom, because its dirty ceiling is 0.
  • So a few contended readers waiting on a pager hold the frames that the pager's supplies and its holders' writes need. Those calls then fall back on pmm::promise.
  • Under memory pressure the promise is refused: ResourceExhausted becomes Refused::NoRoom. The server then evicts other files, or answers ENOSPC to a write or EIO to a read, while readers waiting on it hold the frames it lacks.
  • This breaks the rule that no wait holds a pin, applied to frames instead of pins.

Fix.

  • Draw the frames in the same hold of the lock that takes the side, and answer WouldBlock there if none can be had.
  • Give them back before wait_supplied.
  • S1 removes this entirely.

B5. BLOCKER: four comments and contracts are now false or redundant.

  • sdk/toyos/src/files/owner.rs:85: "Every write lands at the end, which only the server knows." A holder's append now reads the end in the kernel. The clause "since only it knows the size" at :164 is false as well, because holders read the size. Give the real reason a seek from the end asks the server, or drop the clause.
  • sdk/abi/src/syscall.rs:3257-3268: paged_read's contract omits WouldBlock. A queued read answers it when it gets no frames (object/paged.rs:658). The SDK's doc mentions it; the kernel boundary's contract must too.
  • kernel/src/object/paged.rs:39-42 repeats what :14-23 already says about restarts and the queue. Delete it.
  • docs/src/architecture/memory.md:61-62: "Nothing that holds a file's writes off waits on a program's memory" is false for the server, which holds the side across copies from its own buffers and window, and across disk I/O. What is true is that nothing a holder controls keeps the side's holder waiting. Say that. The 256 KiB sentence is covered under B3.

Optional

S1. After READ_RESTARTS, answer WouldBlock and let the SDK read through the server, as read_held already does on any refusal.

  • The server's read runs under its hold, so it is whole, up to the window as on main.
  • Through pager_waiting, it goes before any new holder write. The bound holds.
  • It deletes Side::Reader, readers_waiting, queue_reader, take_reader, unqueue_reader, read_queued, copy_under_side, Queued, zero_flat, copy_flat's FromFrame direction and the bounce frames, which is most of this round's kernel growth.
  • With them go B4 and the "may answer fewer bytes" semantics of a read.
  • The cost is one round trip for a read that has already met three writes.

Evidence

Host. Checked against 4e8524bf/r1, and it holds.

  • checks-2's diff equals the commit's, apart from the new guest binary, which is untracked there.
  • checks-3 ran at 66702d6 after the rebase.
  • git diff 67713cbc1 45935f4b9 touches none of kernel/pure, kernel/src/object/paged.rs, kernel/src/syscall/paged.rs, system/fileserver or lib/bcachefs, so the results from before the rebase stand:
    • 28 paged:: tests, the model included;
    • 130 + 3 file server tests;
    • 29 engine tests.
  • Gap: no clippy run of system/fileserver, a touched package, appears in the evidence. Run it with the fixes.

Controls. All seven are real patches, and each is red on its test's assertion. s2 failed to build at first; after a rebuild it is red.

What no host test reaches. Nothing on the host reaches the kernel object's holder_write, read_queued, copy_under_side or hold. Three things stand in for that:

  • the model, which covers the protocol on Object;
  • pagerprobe, which checks an unheld write is refused;
  • the served-file tests on metal.

B1's and B2's fixes each need a pure test.

Model. It explores every interleaving of:

  • a two-page read;
  • two holders' writes, one of them twice;
  • the pager's write.

It asserts that no read is torn, at most one write holds the side, no interleaving deadlocks, and a queued read goes first. It does not model a queued read giving the side back at an absent page, which inspection covers.

a_served_files_writes_land_whole.

  • Scope: it runs on DATA and /tmp, with two holders writing over each other while a third thread reads, plus two appenders.
  • Size: its 32 pages stay inside both limits, so its full-count assertions are sound.
  • Premise check: it checks its own premise with seen.len() > 2.
  • Header: the header says why it needs several CPUs.
  • Where it runs: it is in the T14 shared boot by membership.

Pending:

  • guest run 43076343465, at 66702d6, still queued;
  • the T14 metal run.

Landing conditions

  • Guest 43076343465: all five rows pass, and the_log_lands_on_the_boot_stick shows FAT writes going to the server. The fixes do not touch what those rows cover.
  • testcases on the T14 at the fixed head, because B1, B2 and B4 change holder_write, stat and read_queued, which these exercise:
    • every member passes, including a_served_files_read_never_tears_against_its_write, a_served_files_writes_land_whole, pagerprobe and a_pager_s_end_answers_its_waiting_holders_gone;
    • no paged overdue report;
    • --fat32-check is clean.
  • diskbenchcase in A, B, B, A order:
    • fs.mem.fstat and fs.disk.fstat: about 1 µs or less, from 7.6 µs.
    • fs.disk.write_4k_at, and pwrite_4k on both volumes: a few µs, from 22.4 µs.
    • fs.mem.seq_write.4096: well above 438 MB/s.
    • DATA's seq_write.1048576, readdir.1000 and the path rows (stat, open, rename): within A's noise.
    • No row beyond twice its recorded value.
    • The run at 66702d6 stands for these. The exception: if B3 is fixed by refusing long spans, re-measure the seq_write rows of 256 KiB and up at the fixed head.
  • Bulk sizes 16 and 256 beside 64: recorded as the measurement behind BULK, not as a pass bar.
  • After metal: update the current-state rows of the three issue records.

For the orchestrator (PR description).

  • The map7b note says a holder's write stages "holding no pin, lock or write side" and that a queued read "copies out holding nothing". Both hold frames drawn from the pager's core across their user copy: up to 128 for the write, 64 for the read. For step 2, those are two more places where a park would hold pager memory.
  • "a longer one is taken in parts, each whole": the program receives a short count (B3).

SEND BACK

…the server's writes, and long writes go whole

An append took the object's size and never checked the end against the
last page, so a file grown to it by a positioned write let any process
panic the kernel with an append; and an offset of u64::MAX meant append,
so a positioned write there answered success. An append is now its own
call, SYS_PAGED_APPEND, whose end is checked, as a positioned write's is,
in the hold of the lock that reads the size; the SDK hands a holder only
writes inside what a file holds. A holder's span is taken whole or
refused: a write of more than 64 pages goes to the server, which writes
it whole up to its window, as before this change, and no write answers a
short count. While the pager holds the writes, a holder states the size
and time of the moment it took them, and the server stamps its writes'
time inside its hold, so fstat is whole against the server's writes too.
A read that met three writes now asks the server, whose read holds them
off; the queued read, its frames and its short answers go. The contracts,
the module header and the manual say what holds.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Japabu and others added 4 commits November 11, 2026 21:45
Main's #1233 ends a holder's wait for an absent page where the object's
version moved, and the read there answered WouldBlock. Here a holder's
read that meets a write begins again instead, until it met
READ_RESTARTS writes; its want of the page would then be the range it
already named, which the pager answered before the write made the page
absent again, so no notice would name it and the read would wait for
ever. A read that begins again now asks anew (Want::anew), and main's
model of the wait plays the restart. The pager's write is its hold, so
main's tests take and give the write side where they began and ended a
write.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A read that copied across the start of a write counted it, and then
counted it again when it found the write still under way, so two writes
could send a read to the server where the contract says three. Every
write a read meets now takes one path: it is counted, and the read
waits for its end and begins again, asking anew, or at the third asks
the server. The bound is named for what it counts, and the protocol
model counts as the kernel does.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A positioned holder's write was kept inside the file system's largest
file by the SDK, but an append's end is known only in the kernel, which
checked it against the last page a u64 reaches: a file grown to the
largest size by a positioned write took an append past it, which every
other path refuses and whose size the SDK then reads as malformed. The
kernel now ends every holder's write at PAGED_HOLDER_END, 16 TiB, and
refuses one past it WouldBlock as it does a long one, so the server
writes or refuses it as it would any other. The SDK asserts the bound
lies within the file system's largest file, and its own offset check
goes: the kernel's covers positioned writes and appends alike.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The benches are down, so the T14 boots #1258 was to land on are owed
instead: its testcases shared boot and the osbench_disk A/B of main from
before and after it, in the records of the file server's request cost
and the memory filesystem's, and the bulk measurement in the pool's.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Japabu Japabu changed the title An open file is written and stated through its paged object, against credit An open file is written and stated through its paged object, one write at a time, against credit Nov 11, 2026
@Japabu

Japabu commented Nov 11, 2026

Copy link
Copy Markdown
Collaborator Author

Round 2 response

Fixed in a5a4ea5; main merged at 16a50e2; two more fixes at 1b46c51 and b55f2b5; the head 1ee3e6d adds
issue records only. Logs and controls:
00bdb14a/r2.

B1, fixed. An append is its own call, SYS_PAGED_APPEND, so no offset value means "append" and write_at(buf, u64::MAX) is an ordinary positioned write. The kernel reads the end and checks it with the pure holder_span in the
same hold of the lock that takes the write side.

a5a4ea5 bounded positioned writes in the SDK at fs::MAX_FILE_BYTES, but an append's end is known only in the
kernel, which checked it against the last page a u64 reaches. A file grown to 16 TiB by a positioned write still
took an append past it. b55f2b5 ends every holder's write at PAGED_HOLDER_END, 16 TiB, in the kernel, and the SDK
asserts that this lies within fs::MAX_FILE_BYTES. A write past it is refused OutOfReach, answered WouldBlock as
a long write is, and the server writes or refuses it as it would any other (FileTooLarge, as on main). The SDK's
own offset check and its test are gone, because the kernel's check covers positioned writes and appends alike.

The pure test walks the pages below the reach and the last offsets a u64 holds. Controls, each red on its assertion:
holder_span without the reach check (a5a4ea5), and the span checked against the last page rather than the reach
(b55f2b5).

B2, fixed. take_pager records the size and time; while the pager holds the side, Object::stat answers them,
so a statement is ordered before the server's write as a read is. The server stamps its writes' and truncations' time
inside under_hold. Pure test: a_holder_states_the_moment_the_pager_took_the_writes. Server test:
a_server_write_stamps_its_time_inside_its_hold. Controls: a live stat, and the stamp after the hold. Both are red.

B3, fixed the preferred way. holder_span takes a span whole or refuses it. A span longer than 64 pages is
TooLong, answered WouldBlock, and the SDK sends the write to the server, which writes it whole up to its window,
as on main. The clamp is gone and no write answers a short count. The manual's "taken in parts" sentence is gone.
Control: the clamp restored, red on the span test.

B4 and S1, S1 taken. At the third write a holder's read meets, it answers WouldBlock, and the SDK reads through
the server, under its hold. Side::Reader, the queue, read_queued, copy_under_side, Queued, zero_flat,
copy_flat's direction and the bounce frames are deleted, and B4 goes with them. A waiting server still goes before
any new holder's write (pager_waiting). The model sends its read to the pager, and asserts that no holder's write
takes the side while the pager waits. Control: a holder taking the side over a waiting pager, red in the model. The
manual says what holds: "a holder's read that keeps meeting writes goes to the server, whose reads hold the writes
off".

B5, fixed.

  • owner.rs's append: "Every write lands at the file's end, read where the write is made."
  • The seek from the end: "From the end, the server is asked: it reads the end as of the writes it orders."
  • paged_read's contract states WouldBlock after three writes.
  • The kernel header's repeated paragraph is deleted.
  • The manual says "nothing a holder does keeps the server's writes waiting" in place of the false sentence.

One more fix, 1b46c51. A read that copied across the start of a write counted it twice: once at the copy, and
again on finding the write still under way. So two writes could send a read to the server. Every write a read meets
now takes one path and counts once. The bound is named for what it counts, WRITES_A_READ_MEETS, and the model
counts as the kernel does.

Main's #1233, merged. It ends a holder's wait for an absent page where the object's version moved, and there the
read answered WouldBlock. Here such a read begins again instead. Its want would still be the range it already
named, which the pager answered before the write made the page absent again, so no notice would name it, and the read
would wait forever. A read that begins again now asks anew (Want::anew). Main's wait model plays the restart;
without anew it is red: after Report, Answer, Supply, Supply, Shrink, Grow, "the read waits for ever". Main's tests
take and give the write side where they began and ended a write.

Evidence gap, closed. system/fileserver's clippy, with the same shapes, is in the logs at the head and at main.
Its four findings are the same at both, in lines this PR does not change, and are recorded in the PR's Risks.

Description. The map7b note now names both places that hold pager frames across a user copy: a holder's write
staging, up to 128 drawn frames, and a holder's read copy-out, up to 16 pinned frames.

Guest. Run 43092689328 at b55f2b5, code-identical to the head, covers the five rows. Runs at superseded heads
were cancelled before they ran. a_served_files_writes_land_whole is a member of the T14 testcases shared boot,
which no QEMU row selects.

Metal. The benches are down, so nothing was staged. These are recorded as owed, in the PR body and as
Owed on metal: bullets in the three issue records:

  • the T14 testcases boot on main after this change;
  • the osbench_disk A/B of main before and after, two boots a side in A, B, B, A order, with the seq_write rows of
    256 KiB and up among its rows;
  • bulk 16 and 256 beside 64.

The speed-up is not yet measured on metal.

@Japabu

Japabu commented Nov 11, 2026

Copy link
Copy Markdown
Collaborator Author

Review round 3 of #1258 at 1ee3e6d (code head b55f2b5, base e10b6fd)

Architecture

The round makes the mechanism smaller, and the design is sound.

  • S1 removed the queued read. Side::Reader, the read queue, read_queued, copy_under_side, Queued and the bounce frames are gone (a5a4ea5: object/paged.rs +40/−209). The write side now has two takers: a holder's write, which never waits, and the pager's hold. A holder's read only waits for the side or goes to the server.
  • An append is its own call (SYS_PAGED_APPEND), so no offset value means "append". The end is read in the same hold of the lock that takes the side and checks the span.
  • The holder's reach is a kernel constant (HOLDER_END, 2^44). The kernel asserts that it equals the ABI's PAGED_HOLDER_END, and the SDK asserts that it does not exceed fs::MAX_FILE_BYTES, which is 2^44 too. This is simple. S1 below raises one point about who owns the value.
  • Size. Against main: 28 files, +3,135/−508 (round 2: +3,177/−481). This round's own code commits remove about 120 lines net, almost all from the kernel object.

Round-2 findings

  • B1. Resolved. holder_write reads at.unwrap_or(object.size()) after take_holder, in the same hold of the lock (kernel/src/object/paged.rs:1206-1216). holder_span then checks offset + len ≤ HOLDER_END with checked_add. Once the span is accepted, page arithmetic stays below 2^44. write_at(buf, u64::MAX) is refused OutOfReach, the SDK sends it to the server, and the server answers FileTooLarge, as on main. The pure test covers the pages below the reach and the last offsets a u64 holds. Controls b1 and reach are red.
  • B2. Resolved.
    • take_pager records held (kernel/pure/paged.rs:479), stat answers it (:440), and give clears it.
    • The server stamps inside under_hold for a write (pages.rs:586), a truncate (:704) and an open that truncates (:320).
    • Every size change outside a holder's write needs the hold: Write, dirty Supply and Resize. The utimes stamp still runs outside a hold, but it is one SetRecord and moves no size, so a statement still sees one moment.
    • Controls b2-stat-live and b2-stamp-after-hold are red.
  • B3. Resolved. holder_span refuses a span of more than 64 pages before any frame is drawn. The SDK sends that write to the server whole, up to the window. No clamp remains, and the round-2 control (the clamp restored) is red.
  • S1, taken. B4 is gone with it. The model now sends a read to the pager and asserts that no holder's write takes the side while the pager waits for it. Control s1 is red.
  • B5. The four named items are fixed. Two comments made false by this round's own changes are B6 below.

Changes since round 2

  • 1b46c51 is correct. Every write a read meets reaches the single met += 1 site at object/paged.rs:620, from three paths:

    • a copy found written falls through to it;
    • Step::Written reaches it directly;
    • an absent-page wait continues, and its next look finds the moved version and reaches it as Written.

    Each wait then ends only when read_begin is Some, so the write it waited out cannot be counted again. The model's meets matches this.

  • The merge (16a50e2) against A withdraw names again every waiting read it takes a page from #1233 is correct.

    • On main, a read whose version moved while it waited answers WouldBlock. Here it begins again, so it needs Want::anew: Want::ask reports a repeated range as no news. Without anew, a page supplied for the read's earlier notice and then removed by a shrink and a grow would never be named again. The anew control makes main's wait model red on exactly that sequence.
    • Main's tests now take and give the side where they began and ended a write.
    • Wake-ups survive the merge. Every give posts changed: a holder write's end (:1275), a holder write refused after taking the side (:1225), and unhold (:789). A reader whose version moved while it slept is therefore woken by the unhold at the latest.
  • b55f2b5 is correct.

    • The word and holder_word arms are exhaustive.
    • The SDK's offset check is gone. The kernel bound covers positioned writes and appends alike, and a write past it reaches the server's in_a_file.
  • 1ee3e6d follows the benches-down rule. The Owed on metal: bullets name the rows, the bench and what main must show.

Required

B6. BLOCKER: two comments are false after this round's changes.

  • sdk/abi/src/syscall.rs:3080-3081 (PagedOp::Hold) says: "It waits for a holder's write or a holder's read already holding them, which wait on nothing."
    • Since S1, no holder's read holds the side, because Side is only Free, Holder or Pager.
    • The kernel's hold waits only for a holder's write, as its own doc at object/paged.rs:735-736 says.
    • This is the pager's contract at the kernel boundary. Change it to "It waits for a holder's write under way, which waits on nothing."
  • kernel/pure/paged.rs:739-741 (Pages::wait_over) ends: "…or a write or resize moved the version, which the read answers WouldBlock."
    • That is main's behaviour. Since the merge, the read counts the write and begins again, asking anew, and it answers WouldBlock only at the third write (object/paged.rs:618-634).
    • End the sentence at "moved the version", or say that the read begins again.

Optional

S1. HOLDER_END's doc says the value is "a served file's largest size, which its pager keeps for every other write". That is file-protocol policy fixed in the kernel. It holds today:

  • only DATA and MEMORY mint write holders, and both stop at MAX_FILE_BYTES;
  • FAT's smaller format limit stays out of reach only because FAT grants no credit.

A reach the pager sets for each object, at creation or with Grant, would make that constraint structural. Keeping the constant is acceptable. If you keep it, word the pure doc as the ABI's is: the kernel's reach for holders, which a pager minting write holders must accept.

Evidence

  • My host run at 1ee3e6d. Detached worktree, now removed:
    • cargo test -p kernel --lib paged::: 31 passed.
    • cargo test --manifest-path system/fileserver/Cargo.toml: 132 + 3 passed.
    • cargo test -p toyos --lib fs::: 13 passed. cargo test -p toyos --lib files::: 11 passed.
    • cargo test -p toyos-abi: 50 + 4 passed.
    • cargo test -p bcachefs --lib tests::data::: 29 passed.
    • Clippy, every shape rc 0: kernel on the host with sched-check, on x86-64 and on aarch64; toyos with all targets and with fd-table; toyos-abi.
    • Clippy on system/fileserver with all targets: the same four findings the PR records. They are at listing.rs:94, pages/tests.rs:792 and :879 (both from 2026-11-01 commits) and writeback.rs:91, in files and lines this PR does not change.
  • The author's evidence holds.
    • checks-reach/tree.txt is 1b46c51 plus a diff whose stat matches b55f2b5's.
    • checks-merge ran on the merge's tree.
    • I read the controls anew, b2-stat-live, s1 and reach. Each is a real patch, red on its test's assertion.
  • Guest run 43092689328 was still queued at the time of this review. It is at b55f2b5; 1ee3e6d adds only issue records, so the run stands for the head. All five rows must pass, and the_log_lands_on_the_boot_stick must show FAT's writes refused by the holder and taken by the server.
  • Gap: no evidence shows the kernel admits a holder's write at all. No host test reaches holder_write or the object's stat. Every integration claim above would still pass if every holder write were refused and sent to the server. The served-file tests and pagerprobe are members of testcases only. Not a blocker, per the orchestrator, but when they get a guest it needs several vCPUs, on x86-64, and order and contents only, no timing. There they must show:
    • Both a_served_files_read_never_tears_against_its_write and a_served_files_writes_land_whole pass on /home and /tmp. The reader meets more than two generations, no read is torn, the final file is one generation, and all 1,000 appended records land whole and in each writer's order.
    • The log has no paged overdue report.
    • A premise that holders' writes were admitted. The cheapest check: pagerprobe's --hold helper writes through a write holder and asserts:
      • on an anonymous object, which needs no credit: write's count, append's returned size, stat's new size and moved time, and the bytes the owner reads back;
      • WouldBlock for a 65-page span and for one ending past PAGED_HOLDER_END;
      • PermissionDenied from a read-only holder;
      • a Dirtied notice on a non-anonymous object granted credit.
    • a_pager_s_end_answers_its_waiting_holders_gone still passes.

For the orchestrator (PR description)

  • "Whoever holds the side waits only on a holder's copy inside the kernel" should say whoever waits for the side. The server holds the side across disk I/O.

SEND BACK

…ts say what holds

Nothing showed the kernel admits a holder's write at all: every claim
would hold if each were refused and sent to the server. pagerprobe's
helper now writes through write holders its pager mints. On anonymous
memory it checks a page's count, an append's new size, the size and
moved time a statement answers, the bytes read back, WouldBlock with
nothing changed for a 65-page span and for a write past a holder's
reach, and PermissionDenied from a read-only holder; it then writes a
page to an object granted a page of credit, whose Dirtied notice the
pager waits for before it reads both objects back as their owner. The
pager's quota grows by the three pages those writes install, with a
dirty ceiling of one.

Hold's contract says it waits only for a holder's write under way, and
wait_over that a read whose version moved begins again. The kernel's
reach for holders is worded as the ABI words it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Japabu

Japabu commented Nov 11, 2026 •

Copy link
Copy Markdown
Collaborator Author

Round 3 response

Fixed at 4a2d78d. Evidence:
de2e2055/r3.

B6, fixed.

  • PagedOp::Hold now says: "It waits for a holder's write under way, which waits on nothing."
  • Pages::wait_over now ends: "…or a write or resize moved the version, and the read begins again."

S1, constant kept. The pure doc is now worded as the ABI's: "the kernel's reach for holders: one past it is refused and is the pager's to write, and a pager that mints write holders accepts that no holder grows an object past it."

Holders' writes admitted (the orchestrator's requirement). The pagerprobe helper (--hold) now writes through write holders the pager mints.

On anonymous memory, which needs no credit, the helper asserts:

  • a page's write count;
  • the size an append answers;
  • the size stat reports and the time it shows moved;
  • the bytes it reads back;
  • WouldBlock with nothing changed, both for a 65-page span and for a write past PAGED_HOLDER_END;
  • PermissionDenied from the read-only holder it already had.

It then writes a page to a non-anonymous object the pager granted one page of credit.

The pager waits for that object's Dirtied notice. As owner, it then checks:

  • take_dirtied is (0, 1);
  • the sizes Hold reports, one for each object;
  • the bytes of both objects.

To fit the writes, the pager's quota grows by those three pages, with a dirty ceiling of one and five objects. The probe's room assertions now expect those three pages.

Built and controlled.

  • The testcases image built with the change, and pagerprobe compiled with no warning (r3/build-head, rc 0).
  • The negative control has the kernel refuse every holder write at holder_write's entry. It builds into the same
    image and was restored (r3/control-refuse, rc 0). A local scratch branch holds it, ready to dispatch.
  • Expected under the control, not yet run: pagerprobe's helper fails its first write assertion, and the pager,
    never seeing Dirtied, fails its 30 s bound saying the helper never dirtied the credited object.

Guest.

  • Run 43096096727 at 4a2d78d covers the five rows. Run 43092689328, at b55f2b5, was cancelled while still
    queued.
  • No guest row runs pagerprobe today. It is a member of the testcases shared boot, as is
    a_pager_s_end_answers_its_waiting_holders_gone, which launches it.
  • Its run at the head, and its control's red, wait for the shared-boot guest row now in work. I dispatch both once
    that row exists.

Description. The body now says that whoever waits for the side waits only on a holder's copy, and that the server holds the side across its own disk I/O.

@Japabu

Japabu commented Nov 11, 2026

Copy link
Copy Markdown
Collaborator Author

Review round 4 of #1258 at 4a2d78d (base e10b6fd)

Architecture

No mechanism changes this round. The head commit touches three files (+114/−20): two contract comments, the pure doc for the holder reach, and pagerprobe (about +100 net).

pagerprobe is the right home for the admission evidence. It is a program with the pager role, so it mints holders whose writes only the kernel answers, with no file server and no SDK fallback in the way. Paged::write and Paged::append are bare system calls (sdk/toyos/src/paged.rs:290-298), so the helper sees the kernel's own answer, and nothing can redirect it.

Round-3 findings

  • B6. Resolved.
    • PagedOp::Hold (sdk/abi/src/syscall.rs:3081) now says it waits for a holder's write under way, which matches hold (kernel/src/object/paged.rs:735-736).
    • wait_over (kernel/pure/paged.rs:740-742) now says the read begins again, which matches read (object/paged.rs:610-635).
  • S1. Taken as offered. The pure HOLDER_END doc (kernel/pure/paged.rs:513-515) now states the kernel's reach, which a pager that mints write holders accepts. It agrees with PAGED_HOLDER_END's.
  • PR body. It now says "whoever waits for the side", and that the server holds the side across its own disk I/O.
  • Nothing else changed: 1ee3e6da9..4a2d78dd3 touches only these three files.

Does the probe establish admission?

Yes, once it runs green. I traced every check against the kernel at the head.

State when the helper writes.

  • The capacity is LARGE+19 pages. In use: LARGE pages for left and 16 for anonymous; large's and dirty_supply_prefix's pages went when they were released. So room() is 3, as asserted.
  • Five objects are alive (left, anonymous, object, written, credited), within objects = 5. A released object's slot goes with its last reference.
  • dirty_held is 0, because anonymous dirty supplies are untracked (pure/paged.rs:785). The dirty ceiling of 1 therefore admits the credited page.

What each check catches.

check admitted path at the head what makes it fail
a page written to written: Ok(4096) span 1. holder_fits passes anonymous memory without credit or dirty checks and finds 1 fresh page; admit_fresh installs it, and write_unpin grows the size to PAGE any refusal, including the control: Err(WouldBlock)
the append returns PAGE+100 the end is read under the lock at PAGE, and page 1 is fresh the end not read under the lock, or the append treated as positioned
stat: the size, and a moved mtime stamp(now()) at the write's end no stamp: mtime stays 0. The object's mtime starts at 0, so this check needs firmware's wall clock, which QEMU's firmware and metal both give
the holder's read-back, and the owner's read-back under Hold bytes dropped, written to the wrong place, or answered Ok without landing
a write past PAGED_HOLDER_END: WouldBlock holder_span refuses it with no reach check, its single fresh page fits the one free slot, so it would be admitted (Ok(1))
a read-only holder: PermissionDenied holder() checks WRITE (syscall/paged.rs:134) with no rights check, it would be admitted into anonymous's present page 0
credited: Ok, then Dirtied, take_dirtied = (0,1), Hold = PAGE, and the bytes credit 1, ceiling 1 and a free slot admit the page. holder_dirtied owes the notice, and it is posted in the same hold of the lock that gives the side back no notice: the pager's 30 s bound fails. A wrong range, size or bytes: their assertions fail
a 65-page span: WouldBlock refused for lack of room whether or not the span limit exists. See B7 nothing

The control. c2f8d737a returns WouldBlock at the entry of holder_write, before anything else. Every holder write, positioned or appended, reaches the kernel only there: PagedHolderObject::write, and nothing maps a paged object. The control therefore disables admission completely and leaves the rights check, stat and reads as they are. Under it:

  • the helper fails its first assertion;
  • the pager fails its 30 s bound with "the helper never dirtied the credited object";
  • every other member of the boot should pass as at the head, because the SDK sends refused writes to the server.

So the red is specific to admission. The control is built (r3/control-refuse, rc 0, the file restored after), but it has not run.

Required

B7. BLOCKER: tests/guest/pagerprobe/src/main.rs:248-249. The 65-page check cannot fail for the defect it names.

  • When the check runs, the pager has one free slot: capacity LARGE+19, minus LARGE for left, 16 for anonymous and 2 for written.
  • A 65-page write at 0 to written would need 63 fresh pages, so holder_fits refuses it NoRoom (kernel/pure/paged.rs:911-912). That happens before any frame is drawn, and holder_word answers WouldBlock, which is the same answer the check expects.
  • The result is the same with the span limit removed from holder_span, and with round 2's B3 defect restored (the span clamped to 64 pages). The stat comparison after it holds as well.
  • So the assertion passes under exactly the defect it names, while the header (lines 21-22) and the PR body present it as evidence that the kernel refuses a long span.
  • Fix, either way:
    • Delete it. The span limit is the pure holder_span, host-tested with B3's control red, and holder_write calls it before it draws anything (object/paged.rs:1163-1166). Inspection suffices.
    • Or make room for it. Grow the pager's capacity so that 63 fresh pages fit when the check runs (62 more), and update the room() assertions. Then only the span limit can refuse the write. Without the limit, the kernel would admit it, or panic when it stages 65 pages into 64 (object/paged.rs:1191); either is red.
  • Update the header and the PR body to match.

Optional

S1. The probe now carries claims that its launcher and labels do not name:

  • the launcher's header (tests/guest/rust-tests/src/bin/a_pager_s_end_answers_its_waiting_holders_gone.rs:5-6, "The probe also asserts…");
  • pagerprobe's description (tests/guest/pagerprobe/Cargo.toml:3);
  • tests/images/testcases.toml:84.

A holder-write regression will be reported as red in a test named after the pager's end. Either drop the inventory sentence from the launcher, since pagerprobe's header owns it, or name the holder writes in all three places.

S2. The helper paragraph of the header (lines 18-25) lists each assertion, which the code and its messages already state. One sentence would do: "its writes through write holders land, are stated and read back; one past a holder's reach, or through a holder minted to read, is refused with nothing written."

Evidence

  • My host run at 4a2d78d. Detached worktree, now removed:
    • cargo test -p kernel --lib paged::: 31 passed.
    • cargo test -p toyos-abi: 50 + 4 passed.
    • Clippy with the adopted lints and warnings denied, rc 0 each: kernel on the host with sched-check, on x86_64-unknown-none and on aarch64-unknown-none-softfloat; toyos-abi with all targets.
    • cargo run -- --build-only --boot-config tests/images/testcases.toml: rc 0. pagerprobe compiled with no warning, and the image holds bin/pagerprobe.
  • The author's r3 evidence holds.
    • build-head/tree.txt is 1ee3e6d plus a diff whose stat matches 4a2d78d's (3 files, +114/−20).
    • The testcases image built with rc 0, and pagerprobe compiled with no warning.
    • control-refuse/refuse-holder-writes.patch is c2f8d737a's diff exactly. It built with rc 0, and the file was restored.
  • Guest run 43096096727 at 4a2d78d is still queued. No row in it runs pagerprobe. What its five rows must show stands as round 3 stated it.
  • Not run: pagerprobe and its control. Metal is not a landing condition while the benches are down.

What pagerprobe's run must show

This applies once the testcases shared boot has its x86-64 guest row, run at the head or at a revision whose kernel/src/object/paged.rs, kernel/pure/paged.rs and pagerprobe are the head's. The row is several vCPUs, and it establishes order and contents only.

  • At that revision:
    • a_pager_s_end_answers_its_waiting_holders_gone passes.
    • Its log shows, in this order:
      • pagerprobe --hold: a holder's writes are admitted whole, …;
      • pagerprobe --hold: a page written against credit;
      • pagerprobe: the owner reads back what the helper's holders wrote;
      • pagerprobe --hold: the pager's end answered the wait Gone;
      • pagerprobe: ok.
    • There is no paged overdue report.
    • The two served-file tests pass as round 3 stated.
  • With c2f8d737a's patch applied to the same revision:
    • That test is red. The helper fails on "a page written through a holder" with Err(WouldBlock), and the pager fails with "the helper never dirtied the credited object".
    • Every other member passes as at the head.

For the orchestrator

  • The PR body says "this PR lands on that row green at its head". If it lands before that row exists, the sentence is false. The owed guest run and its control then belong in issues/a-file-server-request-costs-three-to-twelve-times-a-kernel-file-call.md, which today owes only the metal boot.
  • The PR body's pagerprobe list includes the 65-page item (B7).
  • Remaining gap, not a blocker under your scope.
    • pagerprobe shows that the kernel admits a holder's write.
    • That an open of a file on /tmp or /home mints a write holder, and that the SDK writes through it first, is shown only by inspection (system/fileserver/src/main.rs:1199-1201, sdk/toyos/src/files/owner.rs:374) and by the owed osbench rows. No guest assertion shows it.

SEND BACK

…n is owed

The 65-page write could not fail for a missing span limit: with one
free slot left, the pager's lack of room refused it first, answering
the same WouldBlock. It goes; the limit is the pure holder_span's,
host-tested with its control red. pagerprobe's helper says in one
sentence what its writes show, and its launcher, its description and
its image name the holders' writes too.

The record of the file server's request cost now owes pagerprobe's run
and its control on the testcases shared boot's guest row, with what
each must show, and names what no test shows yet: that an open on /tmp
or /home mints a write holder that the SDK writes through first.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Japabu

Japabu commented Nov 11, 2026

Copy link
Copy Markdown
Collaborator Author

Round 4 response

Fixed at 230a2ad.

B7, deleted, as the orchestrator decided. The 65-page write and its message are gone. The span limit is the pure holder_span's, host-tested, with B3's control red. The header and the PR body no longer claim it.

S1, S2, taken.

  • The helper paragraph of pagerprobe's header is now one sentence: its writes through write holders land, are stated and read back, and one past a holder's reach, or through a holder minted to read, is refused with nothing written.
  • The holder writes are named in the launcher's header, in pagerprobe's description, and at testcases.toml's pagerprobe entry.

The gap and the owed run. These are recorded in issues/a-file-server-request-costs-three-to-twelve-times-a-kernel-file-call.md, under #1258:

  • pagerprobe's run on the testcases shared boot's x86-64 guest row, with what it must show, in your order;
  • its control, which must go red as you describe while every other member stays green;
  • the gap: no test yet shows that an open on /tmp or /home mints a write holder, or that the SDK writes through it first.

The body's landing paragraph now says the run is owed and recorded, and no longer says the PR lands on it.

Control branch. fileops-step1-ctl-refuse is rebuilt as the head plus the one patch (2ee4899). kernel/src/object/paged.rs is the same at 4a2d78d and at the head, so the patch is unchanged.

Checks.

  • The testcases image built at the head's tree, with pagerprobe compiled and no warning: rc 0.
  • The launcher's change is a comment.
  • Nothing here changes what the five guest rows run. The orchestrator re-dispatches them at the head once Actions recovers.

Japabu added a commit that referenced this pull request Nov 11, 2026
B4. The memory track's step 2 bullets name every boot and group owed on
metal and what main must show: the shared boots with `mapped_file` and the
pager-end test, the A/B of the groups the fault and copy paths run under
with the process-exit cost every end now pays, and the `map` group beside
Linux with each row's ratio. It records the pager-wide wakes (a departure
from §3/S4) and the revocation visit's cost with the rows that price them,
names step 3 as R4's, keeps the cold DATA miss as a row still to write, and
records the pinned-copy test owed with #1258. libc's `msync` reason is what
is true: a file is mapped read-only only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Japabu

Japabu commented Nov 11, 2026

Copy link
Copy Markdown
Collaborator Author

Review round 5 of #1258 at 230a2ad (since 4a2d78d)

Scope. 4a2d78dd3..230a2ad3d is one commit, 230a2ad, touching five files (+26/−14):

  • pagerprobe's main.rs and Cargo.toml;
  • its launcher;
  • tests/images/testcases.toml;
  • the request-cost record.

Nothing else changed.

B7 resolved. The 65-page check is deleted, and so is its PAGED_HOLDER_PAGES import. Nothing else in the helper used either, so the helper still compiles by inspection. The check was refused with nothing written, so deleting it changes no state that a later assertion reads.

S1 and S2 taken.

  • The helper's paragraph (tests/guest/pagerprobe/src/main.rs:18-23) is now the offered sentence, and it is true at the head: each write and refusal it names is asserted in writes.
  • The launcher's header, pagerprobe's description and testcases.toml:84-85 now name the holder writes.

PR body. True:

  • The landing paragraph says the shared-boot run and its control are owed, and where they are recorded.
  • It says the span limit's evidence is holder_span's host test.
  • fileops-step1-ctl-refuse (2ee4899) is the head plus one commit, which adds 3 lines to kernel/src/object/paged.rs only.

Record. The control, the remaining gap and the metal rows are stated truly, with one exception.

Required

B8. issues/a-file-server-request-costs-three-to-twelve-times-a-kernel-file-call.md:100-101 — the record requires an order the run does not guarantee.

  • "then the page written against credit, then the owner's read-back" orders lines printed by two processes.

  • The kernel posts Dirtied inside the helper's credited write, and owner_reads runs on that notice. The pager can therefore print the owner reads back what the helper's holders wrote before the helper returns from its system call and prints a page written against credit.

  • A green run can show either order, and a judge following the record would call one of them red.

  • The wording is mine from round 4; the author copied it.

  • Fix. Require only the orders that hold:

    • admitted whole before the owner's read-back, since the credited write follows that line in the helper's own order;
    • the read-back before the pager's end answered the wait Gone;
    • Gone before pagerprobe: ok.

    a page written against credit falls anywhere between admitted whole and Gone.

Evidence

There has been no run since round 4, and none was needed: the change deletes a self-contained check and edits text.

SEND BACK (one line of the record)

…cess orders

The record required the helper's credited write to print before the
owner's read-back, but the pager reads back on the Dirtied notice the
kernel posts inside that write, so either process may print first. The
record now requires only the orders that hold, and places the credited
write's line anywhere between the helper's admitted writes and the
pager's end.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Japabu

Japabu commented Nov 11, 2026

Copy link
Copy Markdown
Collaborator Author

Round 5 response

B8, fixed at 695146e. The record now requires only the orders that hold:

  • the helper's writes admitted, before the owner's read-back;
  • the read-back, before the pager's end answers the wait Gone;
  • Gone, before pagerprobe: ok.

The helper's "a page written against credit" can fall anywhere between its writes admitted and Gone, because the two processes print in either order. This is a change to the record's text only.

@Japabu

Japabu commented Nov 11, 2026

Copy link
Copy Markdown
Collaborator Author

Orchestrator verification of round 5 at 695146e: only issues/a-file-server-request-costs-three-to-twelve-times-a-kernel-file-call.md changed (+3/−2), removing the false order between two processes' lines (my wording). Review r4's other points were settled. #1258 lands on guest run 43099717034 (five rows, at 230a2ad; code-identical) plus pagerprobe and its control on the shared-boot guest row once #1271 exists.

@Japabu

Japabu commented Nov 12, 2026

Copy link
Copy Markdown
Collaborator Author

Review r8 of #1258 at a2e4dfd (main d1cefba merged into 4240a3d)

Scope.

Architectural assessment

The composition is the right one. #1258's two flags, two counters and two walks give way to a second list in #1273's table.

The guarantees, traced.

  • No lost notice.
    • take_marked unmarks whatever it answers.
    • remove unmarks every list before freeing the slot (slots.rs:141-144), so no freed slot stays linked. A linked free slot would end take_marked early and drop the rest of its list.
    • DIRTIED is marked only for an owned object (paged.rs:1256). It is unmarked by take_dirtied (:1102) and by let_go (:393-394).
  • No duplicate. A mark already set answers false, so the watch is posted once per mark, as the flags did.
  • A slot is never collected while marked. A slot goes only with its last reference. The pager's own reference goes only in let_go, which unmarks both lists first. An entry that is not owned is never marked, so the end's walk loses nothing by skipping it.
  • Release never allocates. let_go calls two unmarks, then unref_locked, then remove. None of them allocates.
  • Order. NO_HOLDERS is still taken before DIRTIED. Within a list the order is now last-marked first rather than by slot. The server handles each notice by object, so nothing depends on that order.
  • Holder bookkeeping. It is An open file is written and stated through its paged object, one write at a time, against credit #1258's split, unchanged:
    • holder_gone counts a holder down (:410-414);
    • say_no_holders marks only at holders == 0 && writers == 0 && owned && !gone (:418-424);
    • HolderWriting's drop asks again (:1616-1621).
    • It is correct on every path the SDK takes, because the SDK holds a descriptor's handle until its call returns. S2 covers two windows that predate the merge and that only raw calls reach.
  • The file server. A pager's objects are charged as they are made, and /tmp holds its files by that #1273's keeps and drop_oldest both pick through droppable, which includes An open file is written and stated through its paged object, one write at a time, against credit #1258's !held && !holder_dirty (pages.rs:176-178). So a file a holder may have dirtied is never let go to make room.

Slot size. The log (r8/slot-sizes.log) shows object 112, entry 192, mark 12, and a slot with two lists 216.

  • What the entry holds:
    • the holder-write state: size, version, credit, mtime, and the held and dirtied pairs at 24 bytes each;
    • the 64-byte record, which is the agreed design for a stat without a request;
    • three counts and owned.
  • No field is unused. Moving any of them off the slot would take an allocation per object, or a second table under the same id, which costs as much.
  • About 24 to 32 bytes could go by encoding Mark's tag in its links and dropping the tags of the two Option pairs, at some cost to clarity. That is not worth a finding at 216 bytes.

Busy: my recommendation is a kind of its own (B2).

Required

B1. kernel/pure/slots.rs:303-318 — the new test cannot fail for half of what its name and doc claim.

  • What it misses. The slot it removes, 3, is marked only on list 0 (:312). :313's !t.unmark(1, 3) even asserts that it was never on list 1. So a remove that unmarks only list 0 passes, and that is exactly the bug a port of A pager's objects are charged as they are made, and /tmp holds its files by that #1273's self.unmark(slot) invites.
  • Why it matters.
    • The property is material: a freed slot left on DIRTIED ends take_marked early and loses every notice behind it.
    • The PR description cites this test as covering "removal clearing both".
  • Fix. Mark slot 3 on both lists: add && t.mark(1, 3) at :312 and drop && !t.unmark(1, 3) at :313.
    • A removal that unmarks only list 0 then reds :316 ([] against [2]).
    • One that unmarks only list 1 reds :315.
  • Host test only.

B2. sdk/abi/src/syscall.rs:677-679, kernel/src/object/paged.rs:1407,1417, sdk/toyos/src/files/owner.rs:345 — one error kind now carries two unrelated meanings, and the merge dropped the ETXTBSY this PR had set.

  • Recommendation. Give the seal its own kind, for example TextBusy = 12:
    • libc maps it to ETXTBSY, and std to ErrorKind::ExecutableFileBusy;
    • Refused::Sealed maps to it in both word and holder_word;
    • held_write matches it;
    • Busy's doc goes back to main's one meaning.
  • Why.
    • The conditions differ.
      • A poll belongs to the handle it was made through, and a ring to its one submitter #1261's Busy is momentary contention between a process's own threads over its inbox. That is EBUSY, "device or resource busy".
      • The seal is a state of the file: a program runs from it. POSIX has an errno for exactly that, ETXTBSY, "text file busy", which open gives for opening such a file to write. std has ExecutableFileBusy for it.
      • With one kind, libc can answer only one errno, and a program that tests for ETXTBSY gets EBUSY.
    • map7b needs the same kind. Under the agreed admission interface, the seal's other refusal is map7b's image mint while the object is being written: execve's ETXTBSY.
      • Both of the seal's refusals are the text-busy condition, and neither is the inbox's.
      • Fixed now, before map7b builds on it, this costs one ABI variant and four mapping lines. Later it is an ABI change across two tracks. The interface's wording ("refused Busy") should follow.
    • The doc shows the problem. It now has to list unrelated uses ("a write to a file a program runs from, or a call on an inbox another thread submits to").
  • What to run. Nothing raises Object::seals until map7b, so Refused::Sealed is reachable today only in host tests, as the interface says. Host checks suffice: the ABI, the SDK, libc's mapping and the kernel's clippy shapes.
  • For map7b's record, not this PR. Once the seal is live, the file server answers its own refused write with Io (system/fileserver/src/pages.rs:137-142, Refused::Kernel(_)).
    • A holder refuses a write over 256 KiB before it reaches the seal, so such a write goes to the server and answers EIO.
    • A short write answers the seal's kind.
    • The server needs a protocol word for it when map7b raises the count.

Suggestions

S1. kernel/src/object/paged.rs:221-231 — owe_dirtied no longer earns its place.

  • It now only forwards to mark or unmark, its bool picks between the two, and its false arm always answers false.
  • Call state.entries.mark(DIRTIED, slot) at :1256 and state.entries.unmark(DIRTIED, slot) at :1102, and delete the function.

S2. Two windows in round 1's S6, as adopted, both from before the merge. The composition keeps them exactly.

  • The stale mark. notices re-checks only holders == 0 when it takes a NO_HOLDERS mark (:472).
    • The sequence: the mark is set with no holder and no write; a holder is minted; a write begins through it; its last handle closes under the write. The notice is then said with writers == 1, and said again when the write ends.
    • && entry.writers == 0 there loses nothing, because the write's end marks the object again.
  • A write counted after its holder's last close. HolderWriting::begin (:1610-1613) counts a write whose holder's last handle has already closed, because the system call holds the holder's Arc, not a handle. NO_HOLDERS may already have been said by then.
    • Refusing begin while holders == 0 closes this window. Such a write raced its own close.
  • The effect. The server's no-holders handling (pages.rs:879-897) can run while a write is still being admitted.
    • On DATA, a write admitted between that handling's take_dirtied and its take_back leaves the file droppable until the server reads the next notice. drop_oldest could release the dirty page meanwhile.
    • Only a client that closes, with raw calls, its own last handle to a file nobody else has open, while it writes through it, can reach this.
  • Fix it or record it.

PR description (the orchestrator's)

  • The Change section still says a sealed object refuses a holder's write "as Busy (ETXTBSY)". At this head that is EBUSY, and it should follow B2's resolution.

Evidence

The author's r8 logs (07e063af), all rc 0 except the file server's clippy:

  • cargo test -p kernel --lib --features sched-check: 510 passed, each_list_marks_apart_and_a_removal_unmarks_every_list among them;
  • the kernel's three clippy shapes (host with sched-check, x86-64, aarch64), with the adopted lints and -D warnings;
  • bcachefs: 208 passed; the file server: 132 + 3 passed;
  • the file server's clippy: rc 101, main's four findings at their merged lines (pages/tests.rs:793, :880);
  • the SDK (215), the ABI, and the testcases image.

The tree tested.

  • tree.txt records 4240a3d with d1cefba staged, at 12,023 insertions. The commit has 12,026.
  • The three lines are most likely the /tmp record's: it cites the evidence commit that holds these logs, so it was necessarily written after they ran. That is my inference, and the record is markdown, so the logs stand for the code.

The control branch. fileops-step1-ctl-refuse is 1273504, which is a2e4dfd plus three lines in paged.rs. Its testcases image builds (rc 0).

Guest.

  • Run 43142627680 at a2e4dfd is still queued.
  • 43117177604, which round 7 counted on, ended cancelled.
  • So no guest row has a result for this PR at any head. Host tests reach neither the kernel object's notices nor its holder paths, both of which this merge rewrote.
  • The five rows are therefore owed before landing, beside the owed testcases row with pagerprobe and its control.

Metal: none. It is not a landing condition while the benches are down.

SEND BACK (B1: two tokens in a test; B2: one ABI kind and its mappings. Neither needs a guest run.)

Japabu and others added 2 commits November 12, 2026 05:07
…he last holder is refused

The seal refused a holder's write as `Busy`, which main's #1261 had made
an inbox submitter's word, so one kind said two things and libc mapped
them to one errno. The seal now answers `TextBusy` (12): libc gives
ETXTBSY and std `ExecutableFileBusy`, and `Busy` keeps its one meaning.

A holder's write that begins once the object's last holder went is
refused as `Gone`: the pager may already have been told no holder is
left, and the write would dirty what it was told no one holds. The
no-holders notice is re-checked against writes under way as well as
holders when it is taken.

`owe_dirtied` goes; its two callers mark and unmark the dirtied list
directly. The slot table's test marks slot 3 on both lists, so a
removal that unmarks only one list fails it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Main 3c5c555 (#1271) makes the testcases list a guest row,
`testcases_members_exit_0`, which judges each member by its own exit;
pagerprobe and the two served-file members this branch adds are members
of that list. No file is changed on both sides.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Japabu

Japabu commented Nov 12, 2026

Copy link
Copy Markdown
Collaborator Author

Round 8, at d1a8554; main 3c5c555 merged as 3581821.

  • B1. The test marks slot 3 on both lists. Control: a remove that unmarks only list 0 builds and reds the test on its list-1 assertion (slot 3 still marked) (checks/control-one-list.log).
  • B2. TextBusy = 12. libc maps it to ETXTBSY (26) and std to ExecutableFileBusy. Refused::Sealed maps to it in word and holder_word, and held_write matches it. Busy's doc is main's again. The SDK and ABI docs, the agreed seal interface ("refused TextBusy") and the PR body follow.
  • S1. owe_dirtied is gone. take_dirtied calls unmark(DIRTIED, ..), and the end of a holder's write calls mark(DIRTIED, ..).
  • S2. notices re-checks holders == 0 && writers == 0. HolderWriting::begin returns None while holders == 0, and holder_write answers Gone. paged_write's doc names that case. This rests on inspection: no host test reaches the kernel object, and a guest test would need a close timed inside another thread's write.

Checks at d1a8554, all rc 0: every kernel library test (510, sched-check), the kernel's three clippy shapes, the SDK's tests (215) and two clippy shapes, the ABI's tests and clippy, and the testcases image. That image's sysroot builds libc's and std's new arms on both architectures (r8b/checks). At the merge: cargo test --test toyos-checks, rc 0 (r8b/merge).

Guest runs:

  • 43146950193 at 3581821: the five rows and testcases_members_exit_0.
  • 43146956481 at fileops-step1-ctl-refuse c97966e (3581821 plus the refusal patch, image built: r8b/control): testcases_members_exit_0.

43142627680 at a2e4dfd was already in progress, so it was not cancelled.

Japabu added a commit that referenced this pull request Nov 12, 2026
B4. The memory track's step 2 bullets name every boot and group owed on
metal and what main must show: the shared boots with `mapped_file` and the
pager-end test, the A/B of the groups the fault and copy paths run under
with the process-exit cost every end now pays, and the `map` group beside
Linux with each row's ratio. It records the pager-wide wakes (a departure
from §3/S4) and the revocation visit's cost with the rows that price them,
names step 3 as R4's, keeps the cold DATA miss as a row still to write, and
records the pinned-copy test owed with #1258. libc's `msync` reason is what
is true: a file is mapped read-only only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Japabu

Japabu commented Nov 12, 2026

Copy link
Copy Markdown
Collaborator Author

Review r9 of #1258 at 3581821

Scope. Only what round 8 changed:

  • d1a8554 in full (seven files, +39 −31);
  • the merge 3581821 against its two parents;
  • the PR body and the issue records at this head.

I ran no tests or builds. The results below are the author's r8b logs. For each I checked the command, the exit code and the tree it ran on.

Architectural assessment

Round 8 is small, and each change sits where it belongs.

  • The seal's kind. The seal now answers its own ABI kind, TextBusy. That costs one variant, one from_u64 arm, one Display arm and one mapping each in libc and std. Busy keeps its one meaning, the inbox submitter's.
  • The no-holders window. It closes in the kernel object, the only place that sees both counts under one lock. The fix adds no state: one condition in notices, and one refusal in HolderWriting::begin.
  • owe_dirtied. The forwarding function is gone.
  • Net size. Production grows by about eight lines. The test's change is two tokens.

Round 8's findings

B1: resolved.

  • slots.rs:312 now marks slot 3 on both lists, and the assertion that it was never on list 1 is gone.
  • The control (r8b/checks/control-one-list.patch) makes remove unmark only list 0. It builds (rc 0) and reds the test at slots.rs:314 of the patched file, which is :316 here, the list-1 assertion: left: [], right: [2], rc 101. The freed slot left on list 1 ends take_marked at once, which is the lost-notice failure this test exists to catch.
  • The patch was reverted, and the script checked git diff --quiet before the full run.
  • A removal that unmarks only list 1 reds :315, the list-0 assertion, by inspection.

B2: resolved, as the orchestrator decided.

  • ABI. TextBusy = 12 has its from_u64 arm (syscall.rs:707) and its Display arm (:728). Busy's doc is main's, word for word.
  • Kernel. word and holder_word map Refused::Sealed to TextBusy (paged.rs:1399, :1409). word also covers the pager's own supply and resize on a sealed object, and ETXTBSY is also POSIX's answer to truncating a running program, so that is consistent.
  • SDK. held_write (owner.rs:345) returns TextBusy to the caller and sends any other refusal to the server, as before.
  • libc. of gives ETXTBSY (26). That matches errno.h:35, and errtext.rs:35 already says "Text file busy".
  • std. to_io_error gives ExecutableFileBusy. It is exhaustive, so the compiler would have caught a missing arm.
  • Remaining uses. The other matches on SyscallError (toyos::fs::Error::kernel, paged::Refused::of, pci-claim, the compositor, endow) have wildcard arms and never see the seal. SyscallError::Busy remains only in the inbox (inbox/mod.rs:427) and its guest test.
  • Docs. The ABI's paged_write doc and the SDK's Paged::write doc say TextBusy.
  • Build. The testcases build log shows the sysroot built for both x86_64-unknown-toyos and aarch64-unknown-toyos, with toyos-libc compiled for each. sdk/std is compiled into that std through the relative path build/toolchain.rs:859 names.
  • Not checked. The agreed seal interface the response cites ("refused TextBusy") is not in the tree, so I could not check its wording.

S1: adopted.

  • take_dirtied calls unmark(DIRTIED, ..) (:1092), and a holder's write ends with mark(DIRTIED, ..) (:1248).
  • owe_dirtied has no other reference left.

S2: adopted, and the reasoning holds. I traced each order of a close against another thread's write under PagerObject::state, the one lock that holder_gone, begin, HolderWriting::drop and notices all take.

  • Close first.
    • holder_gone brings holders to 0. say_no_holders sees writers == 0, marks NO_HOLDERS and posts.
    • The write's begin then reads holders == 0 and answers Gone.
    • It has drawn no frame, taken no side and dirtied nothing, so the pager was told the truth.
  • Begin first.
    • writers is 1, so the close's say_no_holders does not mark.
    • The write ends, whether admitted or refused. HolderWriting::drop counts it down and marks.
    • So the pager hears no-holders only after the write. Its no-holders handling calls take_dirtied, which collects whatever the write dirtied.
  • The stale mark. The sequence: marked at (holders 0, writers 0), then a holder minted, a write begun, the close.
    • notices takes the mark at (0, 1) and drops it.
    • The write's end marks again. No notice is lost, and none is said early.
  • Why holders == 0 is a sound test.
    • sys_paged_holder counts a holder (held()) under the same hold of the handle table as its install (syscall/paged.rs:66-70).
    • The lookup in holder() takes that same lock. So no thread can resolve a new holder's handle while holders still reads 0. A write racing a mint is never refused by mistake.
  • A second holder. The check counts the object's holders, not the calling holder's own handles. So a write through a holder whose handles have closed is still admitted while another holder exists, for example one minted for a later open. That is harmless:
    • the server counts the file held while any holder exists;
    • the write is reported through DIRTIED;
    • the holder's Named keeps the slot from reuse.
  • Gone is the right answer.
    • A call issued after the close is a handle fault that ends the process (process.rs:1240). This call resolved its handle legitimately, so a reported refusal fits better than a fault.
    • Gone already means the holder can no longer write ("the object or the pager ended"), and the ABI doc now names this case.
    • The SDK keeps its descriptor's handle until the call returns, so it cannot reach this. If it did, held_write would send the write to the server.
  • Evidence. Inspection is enough here. The rule is two conditions under one lock, and a guest test would need a close timed inside another thread's write.
  • Comments. The new comments in begin and notices are short and accurate. The writers field's doc is still accurate.

The merge 3581821

It composes.

  • From the merge base d1cefba, d1a8554 and 3c5c555 change disjoint sets of files. Main's nine are two issue records, tests/checks.rs, tests/common/{compile,libcchildren,metal,qemu}.rs, readdir_bound.rs and tests/toyos.rs. None is a kernel, SDK or server file.
  • So the r8b checks at d1a8554 stand for 3581821. cargo test --test toyos-checks at the merge is rc 0, 55 passed.
  • Membership. testcases_members_exit_0 takes its members from discover_rust_tests(), which reads tests/guest/rust-tests/src/bin. So a_served_files_writes_land_whole, a_served_files_read_never_tears_against_its_write and a_pager_s_end_answers_its_waiting_holders_gone are members without any list edit. The last of them starts pagerprobe (testcases.toml's starts).
  • The control branch. fileops-step1-ctl-refuse c97966e is 3581821 plus exactly the three-line refusal in holder_write. Its image build is rc 0 at that commit, with a clean tree.

Required

B1. issues/a-file-server-request-costs-three-to-twelve-times-a-kernel-file-call.md:96 — the record still owes a guest row "once it exists". This merge brought that row in.

  • What is wrong. At this head the row exists as testcases_members_exit_0, and runs 43146950193 (head) and 43146956481 (control) are queued on it.
  • Consequence. As written, the record will claim a missing row and owed work after this PR lands, whatever those runs show.
  • Fix. Name testcases_members_exit_0 and drop "once it exists". Once the orchestrator has judged the two runs, the record says what they showed, with their run ids. If they do not return before landing, the bullet stays owed, against the named row.
  • Markdown only. Nothing to rerun.

Suggestions

None.

PR description and history (the orchestrator's)

  • Checks, pagerprobe paragraph. It lists pagerprobe among the testcases members. pagerprobe is the program a_pager_s_end_answers_its_waiting_holders_gone starts, not a member.
  • Risks, first bullet. "No row runs the two served-file tests" is now stale. The row exists and has not yet returned.
  • The merge commit's message. It says "the two served-file members this branch adds". This branch adds one. a_served_files_read_never_tears_against_its_write is main's. That is history, not worth a rewrite.

Evidence

r8b at d1a8554. tree.txt holds the commit, and both porcelain snapshots, before and after, are empty. All rc 0:

  • cargo test -p kernel --lib --features sched-check: 510 passed, the B1 test among them;
  • the kernel's clippy in three shapes (host with sched-check, x86-64, aarch64), with the adopted lints and -D warnings;
  • toyos: 215 tests, plus clippy with --all-targets and with fd-table;
  • toyos-abi: 50 + 4 tests, plus clippy;
  • the testcases image.

Guest. Both runs above are still queued. They are the orchestrator's to judge. This round's kernel change (the refusal in begin, the re-check in notices) is on the holder path those rows boot through. No earlier guest result exists at any head of this PR.

Metal. None. It is not a landing condition while the benches are down.

SEND BACK (B1: one record bullet, markdown only, finished once the orchestrator has judged the queued runs. Round 8's B1, B2, S1 and S2 are resolved.)

Japabu added a commit that referenced this pull request Nov 12, 2026
B4. The memory track's step 2 bullets name every boot and group owed on
metal and what main must show: the shared boots with `mapped_file` and the
pager-end test, the A/B of the groups the fault and copy paths run under
with the process-exit cost every end now pays, and the `map` group beside
Linux with each row's ratio. It records the pager-wide wakes (a departure
from §3/S4) and the revocation visit's cost with the rows that price them,
names step 3 as R4's, keeps the cold DATA miss as a row still to write, and
records the pinned-copy test owed with #1258. libc's `msync` reason is what
is true: a file is mapped read-only only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Japabu added a commit that referenced this pull request Nov 12, 2026
…record says who measures again

B19. The `Mappers` doc pointed at `Object::epoch`, which r8 replaced with
`Object::link`. `Pages::object` said a released object's id may be reused
because `release` takes its pages, but `release` went in the composition with
#1273. The docs now say a node is seen at the epoch `link` answers, and an id
is reused once its object is empty: `invalidate` marks its pages and the
walk's end takes them.

B20. The /tmp record now says that whichever of #1269 and #1258 lands second
measures the combined slot again, updates the figures and deletes the
sentence. #1258 rewrites the same paragraph for its own slot growth.

S1, S2: `Entry.refs` says only what it keeps: the slot, free with the last
reference. It also counts mappings, walks, queued removals and writes, which
the old list missed. The record says every entry carries the 64 bytes of
mapping state, not only a mapped object's.

Doc comments and a record only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…t, and the control's run is recorded

`a_served_file_reads_through_its_paged_object` asserted that an open to
write alone carries no holder. That was main's rule, an open to read
alone being answered with one; #1258 answers every open file with a
holder, minted to write where opened to write, and the member never
followed. The control's run of `testcases_members_exit_0`
(guest-rows 43146956481) reddened it at that assertion, before it writes
anything, so the head reds it as well. It now takes the writer's holder
and drops it before the writer's close, so its reopen still follows the
last close.

The served-file record names the row and states the control's result:
the pager member red, pagerprobe exiting 101 after 30.49 s of system
calls, its 30-second wait for the helper's dirtying, and every member but
the stale one green. The panic text was not kept; the serial record takes
that run as its second instance.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Japabu

Japabu commented Nov 12, 2026

Copy link
Copy Markdown
Collaborator Author

Round 9, at cb8701b.

  • The control's second red (run 43146956481): a_served_file_reads_through_its_paged_object failed at its first assertion, "an open to write alone carries no holder". That comes before it writes anything, and it writes only through the server, so the refusal patch never reaches it. The assertion was main's rule. This PR answers an open to write with a holder too, so the head would red it as well. The member now takes the writer's holder and drops it before the writer's close, so its reopen still follows the last close. It type-checks for x86_64-unknown-toyos against the ToyOS sysroot (r9/member).
  • The control's pager red: pagerprobe exited 101 after 30.49 s of system calls, its 30-second wait for the helper's dirtying. The panic text was not kept, so the WouldBlock reason is inferred from that timing and the code (r9/control-run). This is recorded in issues/a-file-server-request-costs-three-to-twelve-times-a-kernel-file-call.md, which also names the row testcases_members_exit_0 now.
  • The lost panic text is the case issues/a-red-run-keeps-no-serial-for-a-boot-whose-uart-is-qemus-stdio.md already records: a console-port boot's console is held only in memory. Its exit, every red boot leaving its whole console, covers it. I added this run as its second instance rather than filing a duplicate.
  • PR body: pagerprobe is the program the pager member starts, not a member, and the first Risks bullet names the row.

Guest: 43152496728 at cb8701b runs the five rows and testcases_members_exit_0. 43146950193 was cancelled while queued.

Japabu added a commit that referenced this pull request Nov 12, 2026
B4. The memory track's step 2 bullets name every boot and group owed on
metal and what main must show: the shared boots with `mapped_file` and the
pager-end test, the A/B of the groups the fault and copy paths run under
with the process-exit cost every end now pays, and the `map` group beside
Linux with each row's ratio. It records the pager-wide wakes (a departure
from §3/S4) and the revocation visit's cost with the rows that price them,
names step 3 as R4's, keeps the cold DATA miss as a row still to write, and
records the pinned-copy test owed with #1258. libc's `msync` reason is what
is true: a file is mapped read-only only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Japabu added a commit that referenced this pull request Nov 12, 2026
…record says who measures again

B19. The `Mappers` doc pointed at `Object::epoch`, which r8 replaced with
`Object::link`. `Pages::object` said a released object's id may be reused
because `release` takes its pages, but `release` went in the composition
with #1273. The docs now say a node is seen at the epoch `link` answers, and an id
is reused once its object is empty: `invalidate` marks its pages and the
walk's end takes them.

B20. The /tmp record now says that whichever of #1269 and #1258 lands second
measures the combined slot again, updates the figures and deletes the
sentence. #1258 rewrites the same paragraph for its own slot growth.

S1, S2: `Entry.refs` says only what it keeps: the slot, free with the last
reference. It also counts mappings, walks, queued removals and writes, which
the old list missed. The record says every entry carries the 64 bytes of
mapping state, not only a mapped object's.

Doc comments and a record only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Japabu

Japabu commented Nov 12, 2026

Copy link
Copy Markdown
Collaborator Author

Review r10 of #1258 at cb8701b

Scope. The delta 3581821..cb8701b: one commit, three files, +26 −10. It changes one guest member, adds one record bullet, adds one paragraph to the serial record, and corrects the PR body.

What I checked:

  • the member, against the file server's open path and the SDK's handle types;
  • the record and the body, against the control run's job log (43146956481), its serial artifact and the cited evidence.

I ran nothing.

The member: right against #1258's holder semantics

  • The assertion. Opened.holder is an Option<OwnedHandle> (sdk/toyos/src/fs.rs:663). The server's open mints a holder for every regular file on a server with a pager (system/fileserver/src/main.rs:1225-1228), with writes set where the open writes. So O_WRITE | O_CREATE | O_TRUNCATE on /home must carry one, and expect("an open to write alone carries a holder") (:41) states An open file is written and stated through its paged object, one write at a time, against credit #1258's rule.
  • The old red. The control's failure confirms it: :41:5, "an open to write alone carries no holder", 3 ms of system calls, before any write. The control patch touches only holder_write, so this red was the stale assertion and not the patch. The head would have failed the same way.
  • The drop. writer.holder.take() moves the handle out, and drop(writing) (:60) closes it, because OwnedHandle's Drop calls close.
    • Without the drop, the holder would live inside writer until main returns, so the reopen at :62 would come while the file still had a holder. The test's claim, "reopened after its last close", would then be false.
    • With it, both holders and both fids are closed before the reopen.
    • The member asserts only bytes after the reopen. So whether the server has handled the no-holders notice by then cannot change the result: as it was on main for the reader's holder.
  • The writer holder's effect. The writer's holder now lives through the server-side writes and the truncation. That is exactly An open file is written and stated through its paged object, one write at a time, against credit #1258's state for an open-to-write file: the server takes the write side for its own writes and answers from the object. The member's later assertions read through the reader's holder and std, and both paths are An open file is written and stated through its paged object, one write at a time, against credit #1258's.
  • The doc. The doc line (:5-6) now matches.

The records and the body

The file-server record: true, and r9's B1 is resolved.

  • The record names testcases_members_exit_0, and "once it exists" is gone. The head's run stays owed against the named row.
  • I checked each fact of the control's result in the job log:
    • "2 of testcases's 242 member(s) did not exit 0", so 240 passed;
    • pagerprobe's exit line, code=101 … syscall_wall=30493ms;
    • PARK_BOUND is Duration::from_secs(30) (pagerprobe/src/main.rs:53);
    • the served-file member's panic at :41:5.
  • The record labels the WouldBlock reason as inferred. That is honest:
    • the 40-line window starts inside pagerprobe's backtrace;
    • PARK_BOUND's panic covers both "never dirtied" and "never waited";
    • under the control, only a holder write could dirty the credited object, so "never dirtied" is the one that fits.

The PR body: true where I checked.

  • 43142627680 passed the five rows at a2e4dfd (5 passed, 5 total).
  • 43146950193 is cancelled.
  • The control paragraph matches the log.
  • Risks now names the row and the pending run.
  • pagerprobe is described as the program the pager member starts.

Required

B1. issues/a-red-run-keeps-no-serial-for-a-boot-whose-uart-is-qemus-stdio.md (the added paragraph) — two of its statements about the control run's output are false.

  • "before the first member started".
    • The serial artifact's uart-0.log carries the whole first member: ===TEST_START test_rs_a_held_files_fstat_is_its_servers_stat=== at 3.960, its pass, and TEST_END … exit=0 at 3.971.
    • It then carries TEST_START of the pager member and that member's spawn at 3.979.
    • It stops at 3.984 ("console: the log is carried elsewhere from here"). That is 5 ms after the pager member spawned and before pagerprobe printed anything.
  • "those held only the backtrace of pagerprobe's panic".
    • The window holds frames 3-17 of pagerprobe's backtrace and pagerprobe's exit line.
    • It also holds the member's own panic, "the pager probe ended ExitStatus(…Exited(101))", with its backtrace.
    • What it lacks is pagerprobe's panic message.
  • Why it matters. The paragraph's conclusion, that neither kept pagerprobe's message, is true. These two details are evidence recorded in the repository, and they are wrong.
  • Fix. "…which stops carrying the log once the console port takes it, 5 ms after the pager member started and before pagerprobe printed anything." And: "…held the end of pagerprobe's backtrace and the member's own panic, not pagerprobe's message."
  • Markdown only. Nothing to rerun.

Suggestions

S1. a_served_file_reads_through_its_paged_object.rs:41 — the member now holds the writer's holder but never checks that it was minted to write.

  • One line would close half of the record's "Shown by no test yet" for /home: assert_eq!(Paged::received(writing).unwrap().write(&[], 0), Ok(0), "minted to write").
  • Why it discriminates:
    • a zero-length write passes the kernel's WRITE rights check and then answers 0 without touching the object;
    • a read-only holder answers PermissionDenied.
  • It needs the row again, so it is fine to defer.

Evidence

  • The member type-check. r9/member/check-head.log (rc 0) records neither its command nor its tree. The tree file beside it (tree-1272.txt, 2e7dd79) belongs to the other log, which ran in a different worktree. So the body's "type-checks against the ToyOS sysroot" does not show which revision it checked. The pending row compiles the member at cb8701b anyway, so I am not asking for the check again.
  • Guest. 43152496728 at cb8701b is queued, and it is the orchestrator's to judge. It is the member's first guest run, and the first head run of the pager member and the two served-file members. If it returns before landing, the record's run bullet should state its result. If it does not, the bullet stays owed.

SEND BACK (B1: two phrases in the serial record. Otherwise the delta is right, and nothing beyond the pending head run needs running.)

Japabu added a commit that referenced this pull request Nov 12, 2026
B4. The memory track's step 2 bullets name every boot and group owed on
metal and what main must show: the shared boots with `mapped_file` and the
pager-end test, the A/B of the groups the fault and copy paths run under
with the process-exit cost every end now pays, and the `map` group beside
Linux with each row's ratio. It records the pager-wide wakes (a departure
from §3/S4) and the revocation visit's cost with the rows that price them,
names step 3 as R4's, keeps the cold DATA miss as a row still to write, and
records the pinned-copy test owed with #1258. libc's `msync` reason is what
is true: a file is mapped read-only only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Japabu added a commit that referenced this pull request Nov 12, 2026
…record says who measures again

B19. The `Mappers` doc pointed at `Object::epoch`, which r8 replaced with
`Object::link`. `Pages::object` said a released object's id may be reused
because `release` takes its pages, but `release` went in the composition
with #1273. The docs now say a node is seen at the epoch `link` answers, and an id
is reused once its object is empty: `invalidate` marks its pages and the
walk's end takes them.

B20. The /tmp record now says that whichever of #1269 and #1258 lands second
measures the combined slot again, updates the figures and deletes the
sentence. #1258 rewrites the same paragraph for its own slot growth.

S1, S2: `Entry.refs` says only what it keeps: the slot, free with the last
reference. It also counts mappings, walks, queued removals and writes, which
the old list missed. The record says every entry carries the 64 bytes of
mapping state, not only a mapped object's.

Doc comments and a record only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…erial record says what its tail kept

Guest-rows 43152496728 ran `testcases_members_exit_0` at cb8701b: all
242 members exited 0, the pager member and the two served-file tests
among them. The served-file cost record states that run where it listed
the run as owed, and what a green run does not show: the order of the
probe's lines and the absence of a paged overdue report, which the T14's
shared boot still owes.

The serial record's instance said the verdict's 40 lines held only
pagerprobe's backtrace and that `uart-0.log` stopped before the first
member. They held the end of that backtrace and the pager member's own
panic and backtrace, and `uart-0.log` stopped at 3.984 s, after the pager
member started and before pagerprobe printed anything. What neither kept
is pagerprobe's own panic message.

Text only: the code is cb8701b's.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Japabu

Japabu commented Nov 12, 2026

Copy link
Copy Markdown
Collaborator Author

Round 10, at 001f812: text only over cb8701b.

  • B1: the serial record's instance now says the verdict's 40 lines held the end of pagerprobe's backtrace and then the pager member's own panic message and backtrace. It says uart-0.log stopped at 3.984 s, after the pager member started and before pagerprobe printed anything. Neither kept pagerprobe's own panic message.
  • S1: deferred. The record still lists it as shown by no test.
  • The head run: 43152496728 at cb8701b passed all six rows, and all 242 testcases members exited 0 (log). The cost record now states this. A green run keeps no console, so it does not show the order of pagerprobe's lines or the absence of a paged overdue report; the T14 shared boot still owes the latter. The PR body's guest line and first Risks bullet follow.

@Japabu

Japabu commented Nov 12, 2026

Copy link
Copy Markdown
Collaborator Author

Orchestrator verification at 001f812: a text-only commit over cb8701b that touches two issue records and nothing else (git diff --stat).

  • r10's B1: the red-run record now says that the 40-line excerpt holds the end of pagerprobe's backtrace and then the member's own panic and backtrace, and that uart-0.log stops at 3.984 s, after the pager member started and before pagerprobe printed. That matches the reviewer's reading of 43146956481.
  • Run 43152496728: the file-server cost record now gives its result.
  • Landing run: 43152496728 at cb8701b, the same code as this head, is green on all six rows, including testcases_members_exit_0 (242 members exited 0, ended by the list's reboot).
  • Control: 43146956481 went red on the pager member, with its cause inferred from timing, as the record says.
  • Merge: the PR merges cleanly onto main 302fb36 (A child's handles are a list its parent names: labels and namespaces leave the kernel #1252), and the merge queue compiles the composition.

Queueing. #1272 follows once this merges and it is retargeted to main.

@Japabu
Japabu marked this pull request as ready for review November 12, 2026 05:38
@Japabu
Japabu added this pull request to the merge queue Nov 12, 2026
Merged via the queue into main with commit ee15a2e Nov 12, 2026
10 checks passed
@Japabu
Japabu deleted the fileops-step1 branch November 12, 2026 05:52
Japabu added a commit that referenced this pull request Nov 12, 2026
#1258)

#1258 gave a paged object a write side, holders' writes against credit, a
seal, a pager's record and a notice of holders' dirtying. This branch makes a
removed page revoking until no mapping reaches it. The resolution keeps both
sides, and makes them meet where a page is revoking:

- A revoking page takes no write. `admit` refuses it as it refuses an
  absent page. `holder_fits` finds it `Absent`, so a holder's write over it
  goes to the pager and `admit_fresh`'s checked install never meets it.
- The room a holder's credit paid for a page comes back as the page is
  marked revoking, as #1258 gives it back for a page gone before its
  write-back. The marking resets the page's state, so the walk's end gives
  nothing back twice. `a_revoking_page_takes_no_write_and_gives_its_credit_back_once`
  holds this.
- `resize` keeps #1258's seal and its trimming of the dirtied range, and
  marks the pages past the size revoking rather than giving their frames
  out. `fail` and `invalidate` keep this branch's revocation. #1258's tests
  of shrinks, fails and releases settle their walks as this branch's do.
- `Want::anew` is #1258's, a fresh `NONE`. The model's fault that asks
  afresh asks its run after it.
- The slot table's two lists: a release unmarks both, and a mapping's end
  goes through `say_no_holders`, which waits out holders' writes too.
- `PagedHolderObject::write` and `stat` sit beside `maps` and `link`, and
  `Paged::map` beside `write`, `append` and `stat`. A map takes the holder
  through `syscall::paged::holder` with `READ`. #1258's ABI range of dirtied
  pages is named `abi::PagedRange` beside the kernel's `PagedRange`.
- libc's map of a served file goes through `on_fd_held`, whose holder
  closure #1258 made answer a `Result` of the call's own: `toyos_fs_map`'s
  does.
- A truncation and the pager's writes need its hold now. `mapprobe` holds
  around its write in `coherent` and its shrink in `shrunk`, mints its
  holders to read, and makes its pager with a dirty ceiling of 2. At 0, the
  `coherent` case's own write was refused `DirtyFull`.
- The kernel's overdue report says "wait overdue" since #1258, and the
  mapped-file row reads it so.

The `/tmp` record measures the combined slot, since this lands after #1258:
- A slot is 272 bytes: a 248-byte entry and two 12-byte marks.
- The object's 152 bytes include the 48 of mapping's walk state.
- It was 216 on main.
- The sentence on whichever lands second goes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Japabu added a commit that referenced this pull request Nov 12, 2026
…rames while it waits

#1258's holder write copies its bytes from user memory into frames it drew
from its pager's core, before it takes the write side. The source may be a
page of a mapping that is absent. The copy then parked on the page's pager
while it still held the frames, and frames are what that pager's supply
draws. If holders' writes sourced from a mapping of the same pager drew the
core dry, the pager could not supply the page they wait for, and none would
end.

This is the orchestrator's ruling for pinned copies, that no wait holds
what its pager needs, and the staging copy now takes the same mode. It runs
under `user_ptr::Pinned`, so an absent page stops it. The write gives every
frame back and waits for the source pages holding nothing (`wait_in`). It
then draws and copies again, against the object as it stands then. A copy
that faults answers its own cause, `BadAddress` or `Io`, as every other copy
since B5 does, where #1258 flattened it to `BadAddress`.

`Pinned`, `Copies` and the fault handler's docs say what the mode guards: a
copy that holds what no wait may hold, page pins or a pager's frames.

`mapprobe`'s `written` case is the duty the memory record kept for
whichever of #1258 and #1269 landed second. The case:
- endows a mapper with a write holder of an empty object granted a page of
  credit;
- the mapper writes it from the first page of its mapping, which no one has
  supplied;
- the pager hears the request and supplies the run;
- the mapper's write lands whole, and the pager reads the page back as the
  supplied bytes.

The record's paragraph on the owed test goes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Japabu added a commit that referenced this pull request Nov 12, 2026
#1258 gave the pager one wait that reports itself overdue past the report
time, `wait_changed`, and one draw of frames, `draw` with `materialize`.
A fault's wait and a touched hole's zeroed frame each had a copy of those.
They now call the pager's. The wait still requests the fault's run and
lasts until the page is no longer absent, the version moved or the pager
is gone. The hole's frame still comes from the pager's core, with the same
refusal where there is no room.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Japabu added a commit that referenced this pull request Nov 12, 2026
…say what holds

B1. In the combined layout, the walk state mapping needs adds 40 bytes to the
object: 152 against main's 112. The record's 48 was r9's figure on the layout
before #1258. The record now says 40 to the object, and with the list of
mappers 56 to a slot. It also says that main's 216 was measured on x86-64.

B2. `draw`'s doc no longer says its frames are completed outside the lock:
`Index::zeroed` materializes its frame under it. The module header says a
holder's write copies into frames drawn first, not held aside.

S1. The comment over the holder write's staging copy repeated its doc and
the module header, so it goes. `faulted()`'s doc and panic name a copy
outside `Pinned`, which is what they mean now that `Pinned` guards a pager's
frames as well as pins.

S2. `mapprobe`'s `written` case says what it establishes: the write stops at
the absent page, waits, draws and copies again, and lands whole with the
supplied bytes. That the frames go back while it waits is the code's to show.

Doc comments, a panic message and a record only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant