Skip to content

A path is routed through the view's own names, asking the kernel for none - #1225

Merged
Japabu merged 2 commits into
mainfrom
fileops-route
Nov 11, 2026
Merged

Japabu merged 2 commits into
mainfrom
fileops-route

Conversation

@Japabu

@Japabu Japabu commented Nov 11, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

Routing an absolute path tried each of its first four prefixes as a capability name, deepest first, and
remembered every answer. A prefix the process's view does not hold cost, the first time, a 2 MiB window made and
dropped and a namespace open the kernel refused, and then an entry kept for the life of the process. A program
naming many files up to four components deep, as anything writing into /tmp does, grew that table by one entry
a file without bound. Every later path paid two allocations and a locked lookup per prefix tried.

Change

A process's namespace is immutable and the SDK already lists it for its directories. The held directories are now
kept beside them, and a path goes to the deepest held directory it lies in: no capability the view does not hold is
asked for, and nothing is allocated but the path beneath the capability. That directory's refusal to connect is the
call's error; nothing falls through to a shallower directory's server. Only connections are kept, so the table is
bounded by the view.

The routing helper's host test asserts that no unheld name is ever asked for, and the tests pin the deepest match,
the root's capability for what no other holds, and a name that only shares a prefix (/homestead).

Evidence

Metal, T14, one boot with the change (B-1) against two without it (A-1, A-2), each the osbench_fileops job
(all three verdict passed, judged offline from the readbacks). The images carried scratch stage timers that do
not land; A and B differ only in this change. Readbacks, extracted rows, p50 tables and judge logs:
https://github.com/ToyOSOrg/evidence/tree/33f7f469ebba9f0b82c8d4df8c59ea5aaa86a44c/fileops-diagnosis

T14 p50 A-1 A-2 B-1 (this change)
SDK client time, stat in /tmp 1.59 µs 1.30 0.91
SDK client time, stat on DATA 1.21 1.24 0.92
SDK client time, open and close in /tmp 0.80 0.85 0.34
SDK client time, rename on DATA 2.27 1.98 1.30
SDK client time, create-write-unlink in /tmp 2.23 2.31 1.55
build-shaped rounds, 1,280 compiles, j1 3,446 ms 3,433 3,029 (-12%)
build-shaped rounds, j8 1,112.5 ms 1,112.5 827.4 (-26%)

The whole-row latencies move less and within more noise (stat in /tmp 9.88 / 9.51 to 9.27 µs). The larger
gain is in programs that start: each compile is a new process naming fresh /tmp paths, and paid the refused
opens and windows for each. B ran before the A pair on the bench; the order does not bear on these readings
(each boot flashes its image, and these rows touch only /tmp and /system). B's process also made 531 fewer of
each of two namespace system calls and 539 fewer allocations over the job: the refused connects that are gone.

The harness's offline judge refused all three readbacks for one reason outside this change: the runner's
toyos-metal wrote no back_unix_secs in boot.txt, which the judge at this base requires. Every boot's own log
ends ===TEST_END osbench_fileops exit=0===.

Host, at 5386a47 (review round 1; logs:
https://github.com/ToyOSOrg/evidence/tree/c20541dd31d1e8b3e5b52e0e00d37a6cafb19b6d/host-checks-r1): the change is confined to the SDK's files module, so
the checks are that module's host tests and the SDK's lint. cargo test -p toyos --lib files:: exit 0 (11 passed);
cargo clippy -p toyos --all-targets and cargo clippy -p toyos --lib --features fd-table, each with the adopted
lints and warnings denied, exit 0. The round only removed the fall-through no measured boot reached, so the guest
row and the metal measurement below still hold for it. The full host matrix runs in the merge queue.

Guest, at 6f2ea34 (run 43054675576, log):
a_row_reaches_its_own_state_and_no_other, a nested /state/<row> capability held beside /, which exercises the
deepest-held match: PASS, --ci rows all green. x86-64 only: an SDK change with no architecture-specific code. The
file-call binaries (state_view, hierarchy_paths and the rest) are members of the metal shared boot, not QEMU rows;
two earlier dispatches named them or osbench_files, a metal bench job, and failed at selection (runs 43036474857
and 43042985719).

Risks

None known. As before, a capability more than four components deep (MAX_CAPABILITY_DEPTH) is not routed to;
the view's listing already skips it.

🤖 Generated with Claude Code

…none

Routing an absolute path tried each of its first four prefixes as a
capability name, deepest first, and remembered every answer: a path whose
prefix the view does not hold cost a 2 MiB window made and dropped and a
namespace open the kernel refused, the first time, and an entry kept for
the life of the process ever after. A program that names many files four
components deep or less, as anything writing into /tmp does, grew that
table by one entry a file. Each later path paid two allocations and a
locked lookup per prefix tried.

A process's namespace is immutable and already listed for its
directories, so the held directories are kept beside them and a path is
matched against those: no allocation but the path beneath the
capability, and no capability is asked for that the view does not hold.

On the T14 the SDK's own time for a path call falls by 0.3 to 0.9 µs,
and the build-shaped rounds, whose every compile is a new process naming
fresh /tmp paths, take 12% less wall time at one job and 26% less at
eight.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Japabu

Japabu commented Nov 11, 2026

Copy link
Copy Markdown
Collaborator Author

Review of #1225 at 6f2ea34

Architecture

The fix sits in the right place. A process's namespace is immutable and the SDK already lists it, so routing a path against the held directories in that listing removes a kernel round trip for each prefix the view doesn't hold, and it removes the unbounded negative cache. The change touches only the SDK. Production code grows by a net 22 lines and tests by 22. Routing keeps the same "longest held prefix" meaning (/homestead does not match /home), so the manual's files page stays correct. It merges cleanly with #1258.

Required

BLOCKER sdk/toyos/src/files/namespace.rs:29-34, sdk/toyos/src/files/owner.rs:23-25,227-239, test namespace.rs:156-163: the fall-through this change makes unreachable is now pinned by a test.

  • After this change, capability() is only asked for names the view lists. In an immutable namespace, opening a listed name never returns NotFound (Namespace::open). The file server never answers a hello with NotHeld: its only hello refusals are TooManyConnections or a drop (system/fileserver/src/main.rs answer). So three things are now dead: the Ok(None) arm, the "found or not" negative cache, and route's loop on to the next held directory.
  • One case is still reachable: a server that answers NotHeld (0x81) to the hello. Its path is then quietly resolved on a different server, the parent capability's. The new test a_directory_that_answers_nothing_leaves_the_path_to_the_one_above pins that as intended behaviour, which is the concealing fallback the principles forbid.
  • Fix: route to the deepest held directory and let its connect failure be the call's error. capability() returns Result<Arc<Capability>, Error>, the cache holds only connections, and the loop and that test go. The change is a net deletion.
  • In the same area, route's doc says "so a path costs the kernel nothing". That is false: the first path into each held directory still connects (namespace open, a 2 MiB window, a hello). State what is true: no capability the view doesn't hold is asked for.

Optional

None.

Evidence

Checked rather than taken from the PR:

  • Host: logs at 6f2ea34: files:: 12 passed, and both clippy shapes rc 0 with the adopted lints and -D warnings. The changed routed helper now asserts that no unheld name is asked for. Run against the old prefix walk, that assertion fails in every routing test, so the tests are their own negative control.
  • Guest: run 43054675576 is at 6f2ea34 and passed 1 of 1.
  • Metal: B-1 measured fileops-diagnosis. I diffed its routing commit against the revert pair (fileops-timers-A..fileops-diagnosis). It differs from 6f2ea34 only in the two doc comments, so the −12% at j1 and −26% at j8 apply to this code. A-1 and A-2 agree within 0.4%.

After the fix, rerun cargo test -p toyos --lib files:: and the two clippy shapes. No guest or metal rerun is needed, because the removed path cannot be reached in the measured system.

A note for the orchestrator, not a finding: the judge refused all three readbacks because the bench runner's toyos-metal writes no back_unix_secs, which main's judge requires (tests/common/metal.rs:243). Refresh the runner before #1258's boots, or their readbacks will be refused the same way.

SEND BACK

…refusal is the call's

Routing still tried a shallower held directory where the deepest would not
connect, kept every failed connect, and a test pinned that fall-through as
intended. A view's names are immutable and a server never refuses a hello
as NotHeld, so only a misbehaving server reached it, and then a path was
quietly resolved on another server. A path now goes to the deepest held
directory it lies in, a refused connect is the call's error, and only
connections are kept. The routing comment no longer says a path costs the
kernel nothing: the first path into each held directory connects.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Japabu

Japabu commented Nov 11, 2026

Copy link
Copy Markdown
Collaborator Author

Review r1, at 5386a47: a path goes to the deepest held directory, and that directory's refused connect is the call's error. capability() returns Result<Arc<Capability>, Error>, the table keeps only connections, and the fall-through loop and its test are gone. The routing comment no longer claims a path costs the kernel nothing. Rerun: files:: 11 passed, both clippy shapes rc 0.

🤖 Generated with Claude Code

@Japabu

Japabu commented Nov 11, 2026

Copy link
Copy Markdown
Collaborator Author

Review round 2 of #1225 at 5386a47

Architecture

No change since round 1. Routing stays in the SDK, where the view's listing already lives. The round removes the fall-through: a path goes to the deepest held directory it lies in, and that directory's connect is the only one tried. This round deletes 13 lines net: 38 removed, 25 added.

Round-1 blocker

Resolved.

  • route takes an FnOnce(&str) -> Result<C, Error> and calls it once, for holding's single deepest match (namespace.rs:24-36, :67-78). A refused connect is the call's error, and nothing is tried at a shallower directory.
  • capability() returns Result<Arc<Capability>, Error>, and CAPABILITIES keeps only connections (owner.rs:23-25, :224-237). The negative cache is gone, so the table is bounded by the view.
  • The test that pinned the fall-through is deleted. The remaining routing tests still assert, through routed, that no unheld name is asked for.
  • The doc comment now says what is true: no capability the view does not hold is asked for. The false claim that a path costs the kernel nothing is gone.

Optional

  • owner.rs:232 maps endow::namespace() returning None to NotHeld. That branch cannot be reached: directories() builds an empty Directories from the same None, so holding never matches and capability is never called. An expect would state that invariant instead of returning an answer for it.

Evidence

  • Host: checked against the evidence at c20541dd/host-checks-r1. The round's diff was applied to 6f2ea34, and tree.txt's stat matches the commit's 2 files, +25/−38. cargo test -p toyos --lib files:: exited 0 with 11 passed, which is the 12 from round 1 minus the deleted test. Both clippy shapes exited 0 with the adopted lints and -D warnings.
  • Guest and metal: reusing them is justified. The only code removed is a path that no measured boot reached, because no server answers a hello with NotHeld.

LAND

@Japabu

Japabu commented Nov 11, 2026

Copy link
Copy Markdown
Collaborator Author

Orchestrator: follow-up review r2 LAND at 5386a47. A path routes only to the deepest held directory; the fall-through loop and cached failures are gone. The reused guest run 43054675576 and the build-round metal are valid, since the only removed code is a path no measured boot reached. Queueing.

@Japabu
Japabu marked this pull request as ready for review November 11, 2026 18:53
@Japabu
Japabu enabled auto-merge November 11, 2026 18:53
@Japabu
Japabu added this pull request to the merge queue Nov 11, 2026
Merged via the queue into main with commit e10b6fd Nov 11, 2026
10 checks passed
@Japabu
Japabu deleted the fileops-route branch November 11, 2026 19:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant