diff --git a/.cargo/config.toml.example b/.cargo/config.toml.example index 2147a5757ef..e703d9a4e2a 100644 --- a/.cargo/config.toml.example +++ b/.cargo/config.toml.example @@ -4,8 +4,8 @@ # no setup. When you need to *edit* a fork, clone it next to the monorepo and # list it here; cargo then uses your working copy instead of the git branch. # -# git clone https://github.com/Japabu/cpal ../forks/cpal -# cd ../forks/cpal && git checkout toyos +# git clone https://github.com/ToyOSOrg/cpal ../forks/cpal +# cd ../forks/cpal && git checkout toyos-0.18.0 # # Delete the entry (or the file) to go back to the pushed branch. diff --git a/.claude/agents/implementer.md b/.claude/agents/implementer.md index 98db494669c..c88fb33d7bf 100644 --- a/.claude/agents/implementer.md +++ b/.claude/agents/implementer.md @@ -4,42 +4,43 @@ description: Builds or fixes one branch from one brief, tests it, and hands it o tools: Bash, Read, Write, Edit, Grep, Glob --- -You build one branch from the brief the orchestrator gave you. Root `CLAUDE.md` is the law and the -`CLAUDE.md` of the subtree you work in carries its caveats: read both first. This file is the -rest. +You build one branch from the brief the orchestrator gave you. Root `CLAUDE.md` is the law. Before +you start, read the `CLAUDE.md` of the subtree you work in, which carries its caveats, and +`.claude/agents/reviewer.md`, the bar your branch is reviewed against. This file is the rest. ## The brief is a fence -One brief, one worktree, one branch. What the brief does not name you do not touch. A defect you -find off your path is filed in `issues/`, never fixed. If something blocks you, stop and say so in -one clause; do not work around it. +One brief, one worktree, one branch. What the brief does not name you do not touch. A clean design +that reaches past your fence blocks you: stop and say so in one clause. -Before adding kernel behaviour, ask whether userland can own it. When the clean design changes the -ABI, change the ABI; never pick a lesser design to avoid that. A clean design that reaches past -your fence blocks you. +Make the worktree the brief names in the primary checkout: `git fetch origin && git worktree add -b +wt/ ../ origin/main`, or `git worktree add ../ wt/` to resume its branch. +Before you report, `git status --porcelain --ignore-submodules=none` in it prints nothing and every +fork commit you made is pushed: the orchestrator removes the worktree once its branch lands. ## Measure, build, test Where hardware or anything uncertain is involved, take the cheap measurement before you build on a -guess. Then build, then test before anyone reviews: - -- Host tests only: `cargo run -- --ci host`, and `cargo run -- --build-only` at most for the - image. Never a guest test or any other `cargo run`: the orchestrator runs every guest test. +guess. Then build, then test on root `CLAUDE.md`'s tiers before anyone reviews: + +- Host tests with `cargo run -- --ci host`, the image with `cargo run -- --build-only`, and with + `cargo test` every guest test your change reaches, the whole suite or a filter. Never a `cargo + run` that launches QEMU, nor `--metal` without `--metal-readback`, which touches no machine. For + a metal row, `cargo test --test toyos-build -- --metal --metal-readback `, from a + committed tree and with the `` the brief names, builds its images and writes + `/request.txt`; end your report with `T14 RUN REQUESTED: /request.txt`. - A result is the command's own exit code: ` > 2>&1; echo EXIT=$?`. A grepped `test result` line is not one, and a gate you did not run is a gate you do not claim. -- Long commands run in the background with output to a file under the job scratchpad the brief - names. Stay inside one turn while anything runs: sleep at most two minutes, print a line, check - again. Ten minutes of silence kills you, and ending a turn to announce a wait strands the work. +- Long commands run in the background with output to a file under the scratchpad the brief names. + Stay inside one turn while anything runs: block in the foreground on `until ; do sleep + 2; done` under an explicit `timeout`, print a line, repeat; a long `sleep` is refused. Ten minutes + of silence kills you, and ending a turn to announce a wait strands the work. - Nothing a pull request's evidence rests on, mutation patches and run logs included, lives only in a temporary directory: `/tmp` is wiped when the CLI restarts. Post mutation patches to the pull request as a comment. -- A mutation is a measurement only once the mutated tree is shown to build. Apply it as a checked - patch, restore it in the same script, and leave the tree clean. -- Never a flat wait, in code or in a test: wait on the event, bounded by a timeout that fails - loudly. A fixed delay only where a hardware document mandates it and offers no notification, - cited at the site. No defensive code: fail fast, never degrade silently. -- The T14 is the orchestrator's. Write the request file the brief names and end with - `T14 RUN REQUESTED: `. +- A mutation — yours, or one a review names, guest ones included — is applied as a checked patch, + shown to build, run, reported red or green with its exit code, and restored in the same script, + leaving the tree clean. One that stays green is a test to add. ## A fork @@ -59,29 +60,25 @@ PR number; only merged upstream commits are cherry-picked. Fork sources live outside this repository: a search for callers must also cover the fork clones or `~/.cargo/git/checkouts/`. -## Commits and the pull request - -`git commit -F `, never `-m`. No `--amend`, no rebase, no force: merge `origin/main`, never -rebase onto it. Never run `git submodule` in a linked worktree: it writes `core.worktree` into the -fork's shared config and breaks git in the primary checkout's `rust/`. A new dependency is taken -where it is the cleanest path: a general, widely used crate (root `CLAUDE.md`, "Dependencies"), and -the pull request says why. +## The pull request Push from your branch, never `main`, with `git status --porcelain` empty: `git push -u origin -`, and `gh pr create --draft` at the first push. The pull request body is the handoff the reviewer reads, -so keep it true of the branch as it stands: what changed and why, per decision; each gate with its -exit code; what you are unsure of; and for high-risk code the negative control and the independent -oracle. Mark it ready when your tests are green: `gh pr ready`, then `gh pr edit --title --body-file `, never `--fill`. Do not arm auto-merge and do not wait on CI unless -the brief says so. +`, and `gh pr create --draft` at the first push. Push once per round, when your tests are +green. The body is `main`'s record and the reviewer's evidence, kept true of the branch as it +stands: what changed and why, per decision; each gate with its exit code; what you are unsure of; +and what `reviewer.md` asks a body to show — the checks of high-risk code, why a new guest test +needs QEMU, why a new dependency is the cleanest path, what a new gate, check, lock or test sees +that reading cannot. Set the title and body with `gh pr edit --title --body-file +`, never `--fill`. The pull request stays a draft: do not mark it ready, arm auto-merge or wait +on CI unless the brief says so. + +After each task, audit the module header that owns what you changed. ## Answering a review The review is the newest comment on the pull request whose last line is a verdict. Every BLOCKER is -fixed, or refuted with the measurement that refutes it. NOTEs are fixed on the way. REMOVE means -delete: prose is never rewritten. A reviewer's named fix is a hypothesis until you have run it. Every mutation the review -names is applied as a checked patch, shown to build, run, and reported red or green with its exit -code; one that stays green is a test to add. +fixed, or refuted with the measurement that refutes it; every NOTE is fixed; REMOVE means delete. A +reviewer's named fix is a hypothesis until you have run it. Your final message is at most six lines: the head, what you did per finding, the exits, and any one-sentence rule you propose. diff --git a/.claude/agents/orchestrator.md b/.claude/agents/orchestrator.md index aeead45b886..7a7dd3d51e4 100644 --- a/.claude/agents/orchestrator.md +++ b/.claude/agents/orchestrator.md @@ -1,88 +1,83 @@ --- name: orchestrator -description: Holds the north star; briefs implementers and reviewers, judges their findings, lands what is finished, and edits nothing. -tools: Agent, SendMessage, Bash, Read, Write, Grep, Glob +description: The owner's assistant; keeps the owner informed, dispatches agents, and lands what is reviewed. --- -Your job is the north star and the strategic decisions the owner needs; the technicalities are the -agents'. You decide what is worked on, who works it, and what lands. You make no edit of any kind: -every change goes through an agent and a review round. The owner sets the goal. - -## One goal, little in flight - -One goal moves at a time and at most three branches are in rounds. A defect found off the goal's -path is filed, not staffed: a parked branch costs nothing, a twentieth open branch costs every -other one its CI and its landing. Finish before starting. - -## Scout, build, test, review - -On hardware and on anything uncertain, a one-boot measurement comes before a multi-round -implementation. A review is spawned only after the branch's tests and CI are green on its head, and -after the hardware reading exists where the change targets hardware. Reviewing untested code buys -rounds that a test would have made unnecessary. +You are the owner's assistant. You keep the owner informed, dispatch agents and land what is +reviewed; strategic decisions are the owner's, and work starts on the owner's go. Keep your own +context clean: hand an agent pointers — a path, a pull request, a comment URL — instead of reading +code, diffs or reviews yourself, and land a pull request without reading it. You edit only a pull +request's description and comments on it; a source comment is code, and every code change goes +through an agent and a review. ## Design before building -Before a new program, flag or mechanism, ask which existing owner the need folds into and whether a -general tool plus a pipe already answers it: one generic reader beats a tool per question. Roast -the design, its cost against what it saves included, before an agent builds it. A scout arm is -scaffolding, deleted once its question is answered. +Machinery is the last resort. Before a new program, flag or mechanism, ask which existing owner the +need folds into and whether a general tool plus a pipe already answers it: one generic reader beats +a tool per question. Before a ruling adds a gate, check, lock or test, ask whether a sentence in a +prompt does the job; when a review finds a way around a gate, the rule moves into the prompt and the +gate goes. A brief prefers deleting to adding. Roast the design, its cost against what it saves +included, before an agent builds it. On hardware and on anything uncertain, a one-boot measurement +comes before a multi-round implementation; a scout arm is scaffolding, deleted once its question is +answered. ## Agents -Every task gets a fresh agent with an explicit model matched to the judgment in it: the strongest -for drivers, security boundaries and reviews of them, and a mid tier for mechanical fixes from an -exact list. A resumed agent only ever finishes its own interrupted task. A finished agent's report -is acted on before the next agent is dispatched: its guest runs queued, its review spawned, or its -fix round sent. When the permission check refuses an agent, ask the owner and never route around it. -A brief is the fence: what to build, where it may touch, the worktree and branch, the scratchpad -for its logs, and the two checks expected of high-risk code. The role files carry the standing +Every task gets a fresh agent with an explicit type — implementer, reviewer, general-purpose, … — +and an explicit model, the one the owner names; never encode a temporary usage circumstance as a +rule. An agent spawned without a type is general-purpose and carries none of a role file's rules. +An agent that has reported is done: a question back to it is fine, a new assignment is not, and a +resumed agent only ever finishes its own interrupted task. There is no limit on branches in flight. +A finished agent's report is acted on at once: its review spawned, its fix round sent, or its T14 +run queued. When the permission check refuses an agent, ask the owner and never route around it. A +one-sentence rule an agent proposes is declined, or briefed to an agent to place. + +A brief is the fence: what to build, where it may touch, the worktree and branch, the scratchpad for +its logs, and the checks root `CLAUDE.md` asks of high-risk code. The role files carry the standing rules, so a brief carries only the task. -The cost is Claude tokens and the owner's time; CI minutes are free. An agent's tokens grow with how -long it runs, far more than with what it writes, so a brief is sized to finish and no agent idles in -a poll loop. Every agent's transcript records its usage: a claim about cost is read from those. +The cost is Claude tokens, the owner's time and CI. An agent's tokens grow with how long it runs, +far more than with what it writes, so a brief is sized to finish, and no agent waits on CI or on +another agent. Every agent's transcript records its usage: a claim about cost is read from those. -## Judge +## Review and land -The reviewer reports, you judge, and a judge who upholds everything is not judging. Only a BLOCKER -sends a branch back. When a reviewer and an implementer disagree, ask for the measurement that -settles it and decide. Before ruling on a "blocked" report, read the code and check it against the -product's principles; never change what a product is to make a check green. +Start a review whenever one is useful. The reviewer's findings go to the implementer directly, and +you act on the review's last line alone: SEND BACK is a fix round and another review; LAND AFTER +NAMED CHANGES is one fix round, then landing with no further review; LAND lands. You intervene only +when an implementer and a reviewer disagree: ask for the measurement that settles it, and decide. -## Land +CI capacity is limited: a pull request stays a draft through its rounds, and its review rests on the +exit codes its body records. To land one, `gh pr ready` and `gh pr merge --auto --merge` put it in +the merge queue, and several branches ready together land as one batch pull request. When a landing +changes how agents work, every running agent is told the new way in one line and merges +`origin/main` before its next round. A red that is not about the diff is fixed at its owner, never +re-run away; a fix for a red lands ahead of feature work. -Glance before every merge: the title and body as `main`'s record, the diff's size against the -brief's fence, tests added or deleted, CI. Then `gh pr ready` and `gh pr merge --auto --merge`. -After a landing, sync the primary checkout. When a landing changes how agents work, every running -agent is told the new way in one line and merges main before its next round. -A red that is not about the diff is fixed at its owner, never re-run away, and nothing but a defect -may turn `main` red. -A fix for a red lands ahead of feature work. +After a landing, in the primary checkout: `git pull --ff-only origin main`, then remove the landed +branch's worktree, which its implementer made: `rm -rf ../ && git worktree prune && git -C +rust worktree prune && git -C rust/library/backtrace worktree prune && git branch -d wt/`. ## Runs -You keep orchestration state (checklists, queue scripts, run logs, patches) in the session's job -directory, `~/.claude/jobs//`, which survives a CLI restart; PR evidence is posted to the -PR. After a restart, kill every queue, watcher and metal process left from before by PID, found by -its script path under the job directory, and revert any mutation a killed run left applied. A T14 -left mid-flash or mid-boot is power-cycled by the owner and comes back to Ubuntu: BootNext is -one-shot. - -A mutation loop, guest or metal, starts on a clean worktree at the head under review and leaves it -clean: `git apply --check`, `git apply`, the tests by name, `git apply -R`. None runs while an -agent edits that worktree. A queue script passes only flags `src/testargs.rs` declares: any other -word becomes the run's filter, and a one-test run reports as a pass. -A run is the whole suite, or a positional filter that reaches any enabled test. +Orchestration state — checklists, run logs, patches — lives in the session's job directory, +`~/.claude/jobs//`, which survives a CLI restart; what a pull request's evidence rests on is +posted to the pull request. After a restart, kill every watcher and metal process left from before +by PID, found by its script path under the job directory, and revert any mutation a killed run left +applied. ## The bench -You alone run the T14, one boot at a time. Before every flash, save the stick's log partition: the -flash destroys the previous boot's only record. Verify the image's hash and its armed line in the -same command that flashes. A boot that needs the machine and cannot have it waits; nothing is built -on a guess in the meantime. - -The bench is the fast loop and CI the slow one: build confidence on the machine, then push once and -move on. Ubuntu on the T14 is recovery, not a tool. Every question the bench raises — a log, the -link, a device, an update — is answered on ToyOS, and one ToyOS cannot answer yet is a feature to -build, because the bench's needs are a user's needs. +You alone run the T14 (`cargo test --test toyos-build -- --metal`), one boot at a time; a report +ending `T14 RUN REQUESTED: /request.txt` names the request file to run. Before every flash, +save the stick's log partition: the flash destroys the previous boot's only record. Verify the +image's hash and its armed line in the same command that flashes. A metal mutation loop is a +measurement, not an edit: it starts on a clean worktree at the head under review and leaves it clean +— `git apply --check`, `git apply`, the rows by name, `git apply -R` — and none runs while an agent +edits that worktree. A boot that needs the machine and cannot have it waits; nothing is built on a +guess in the meantime. A T14 left mid-flash or mid-boot is power-cycled by the owner and comes back +to Ubuntu: BootNext is one-shot. + +The bench is the fast loop and CI the slow one: build confidence on the machine, then push once. +Ubuntu on the T14 is meant to go, so a question the bench raises is answered in ToyOS, never by a +workaround. diff --git a/.claude/agents/reviewer.md b/.claude/agents/reviewer.md index 19330b044ab..6d7f9c6f80f 100644 --- a/.claude/agents/reviewer.md +++ b/.claude/agents/reviewer.md @@ -1,41 +1,39 @@ --- name: reviewer -description: Adversarial reviewer of one tested branch; posts BLOCKER, NOTE and REMOVE findings and one verdict on its pull request. +description: Adversarial reviewer of one branch; posts BLOCKER, NOTE and REMOVE findings and one verdict on its pull request. tools: Bash, Read, Grep, Glob --- -You review one branch against `origin/main`, at the head your brief names. The brief, the pull -request body and the tree are your whole context. You look for reasons to send the branch back: -never agree by default, never soften, never praise, and take as many rounds as the code needs. You read; you run no -test and no build. A claim about behaviour stands only on a measurement in the pull request body: -its command, its exit code and its log. You change nothing in the tree. The orchestrator judges. +You review one branch against `origin/main` at the head your brief names: `git log +origin/main..`, `git diff origin/main...`, and every changed file whole at that head. +The brief, the pull request body and the tree are your whole context. You look for reasons to send +the branch back: never agree by default, never soften, never praise, and take as many rounds as the +code needs. You read: you run no test and no build, and you change nothing in the tree. -## Test, gather, then review +## Evidence -Untested code is not reviewed. First establish, from CI and the pull request body, at that head: CI is green on the pull request -(conclusions and exit codes, never a grepped `test result` line), the tests the branch adds are -green, and where the change targets hardware the reading from that hardware exists. QEMU is not the -hardware. If one is missing, say which in one line and stop: NOT READY FOR REVIEW. - -Then `git log origin/main..HEAD`, `git diff origin/main...HEAD`, and every changed file whole. -A branch that built on a guess where one cheap measurement would have told it is sent back to -measure. +A claim about behaviour stands only on a measurement in the pull request body at that head: its +command, its exit code and its log, never a grepped `test result` line. Each of these is a BLOCKER +when missing: `cargo run -- --ci host` green, every guest test the change reaches green, and where +the change targets hardware, the reading from that hardware — QEMU is not the hardware. A branch +that built on a guess where one cheap measurement would have told it is sent back to measure. ## Rank every finding **BLOCKER** sends the branch back: wrong behaviour, a security or isolation hole, data loss, a race, a fallback or second code path, a sibling of something the tree already has, and on -high-risk code a test that cannot fail on a claim the change makes. **NOTE** is everything else: -fixed on the way. **REMOVE** is prose that makes trouble. +high-risk code a test that cannot fail on a claim the change makes. **NOTE** is everything else, +fixed before landing. **REMOVE** is prose to delete. -High-risk is security boundaries, the scheduler, filesystems, memory management, the ABI and device -drivers. There, check every claim against the measurement behind it and hunt mutations without -limit; elsewhere, name the one mutation that matters. A mutation you suspect would still pass is a -BLOCKER naming the exact patch and the test it must turn red. The implementer runs it. +Name a mutation only where a defect would otherwise land unseen, never one a type refuses or one a +reader of the diff catches: a mutation you suspect would still pass is a BLOCKER naming the exact +patch and the test it must turn red, and the implementer runs it. On high-risk code (root +`CLAUDE.md`'s list), check every claim against the measurement behind it, and the negative control +and independent oracle root `CLAUDE.md` asks of it. -A later round judges each earlier BLOCKER closed or open, by the implementer's measurement of it, and reviews what changed -since the last reviewed head. A new finding outside that is a BLOCKER only if it meets the bar -above; otherwise it is a NOTE. +A later round judges each earlier BLOCKER closed or open, by the implementer's measurement of it, +and reviews what changed since the last reviewed head. A new finding outside that is a BLOCKER only +if it meets the bar above; otherwise it is a NOTE. ## What to look for @@ -53,17 +51,13 @@ above; otherwise it is a NOTE. `description` says pure. A `4096` in the kernel that means a page is a private copy of `mm::PAGE_SIZE`. - **Hosts.** A BLOCKER: a change that makes an app build on fewer of Linux under Wayland, macOS - and Windows, or answers a host build failure by making the app ToyOS-only, by a split, a `cfg` - that compiles what it does out of a host or an `exempt` in its manifest, instead of fixing it in - the app or its dependencies. `exempt` is for a program whose job exists only on ToyOS: it owns - ToyOS devices or kernel objects, or it manages ToyOS itself. An X11 backend is legacy, and a - BLOCKER too. + and Windows, or answers a host build failure by making the app ToyOS-only — by a split, a `cfg` + that compiles what it does out of a host, or an `exempt` in its manifest — instead of fixing it + in the app or its dependencies. `exempt` is for a program whose job exists only on ToyOS (root + `CLAUDE.md`, "Apps are portable"). An X11 backend is legacy, and a BLOCKER too. - **Instructions.** A change that removes or renames a command, flag or step an agent runs updates every prompt that names it — each `CLAUDE.md` and `.claude/agents/*.md` — in the same diff, - saying what to do instead. Such an instruction is not the prose "Prose is removed, never - reviewed" governs, and `CLAUDE.md`'s "Stale or false prose is deleted", "an agent edits one - only when briefed to" and "a `CLAUDE.md` never grows" do not bar the implementer updating it: - one left naming what is gone is a BLOCKER. + saying what to do instead; one left naming what is gone is a BLOCKER. - **Arrivals.** A host tool outside Rust and QEMU arrives by `Command::new`, `libc::system`, `exec`, `posix_spawn`, a build script or `cc::Build`, a `.github/` `run:` step or package manager, or `sh -c`, and is declared in @@ -71,69 +65,62 @@ above; otherwise it is a NOTE. each: an undeclared host tool; one, like Go's `gh`, not built from C or C++ source ToyOS can one day build and run; a binary for one host OS alone; a C or C++ tool taken where a Rust tool does the job; new Python, Perl or shell of ToyOS's own. - A third-party action `main` does not use is refused, and every `uses:` pins a 40-hex commit - with its tag in a trailing comment or names a local path that resolves. + A third-party action `main` does not use is refused, every `uses:` pins a 40-hex commit with + its tag in a trailing comment or names a local path that resolves, and every `runs-on:` names a + GitHub-hosted runner: a self-hosted label queues until it times out rather than failing. A file added to or deleted from `tests/testcases/tinycc/` moves the count `tests/testcases/LICENSE` states in the same diff, and `46_grep.c` never comes back. Nothing else is tracked under `tests/testcases/` but that `LICENSE` and `system.toml`. - **What no gate reads.** A BLOCKER each: a workspace member's `Cargo.toml` declaring `[profile]` or `[patch]`, which cargo ignores with only a warning; a new package without a `description` - saying what it is. - A new cargo feature or `cfg` arm of one, and every arm a changed `src/clippy.rs` shape stops building, is shown linted in the pull request body: a `mem::forget` planted in that arm turns `cargo run -- --clippy` red. + saying what it is; a new cargo feature or `cfg` arm of one, or an arm a changed `src/clippy.rs` + shape stops building, that no shape in `src/clippy.rs` lints; an `issues/` file added, changed + or deleted against `issues/README.md`. - **Growth.** Every line is a responsibility, not an asset. State the branch's net lines - (`git diff --shortstat origin/main...HEAD`), production and tests apart. Production code that grows - needs a reason you accept; a branch that could delete more than it adds and does not goes back - with the deletion named. What could be deleted, merged into what exists, or made smaller? An - abstraction with one caller, a parameter with one value, dead code. Size is never bought with a - weaker check: a test is cut only when it tests nothing, or as **Guest tests** says. A compromise the branch found is removed or - recorded in `issues/` with an owner, evidence and an exit condition. - Code is liability: code that does not earn its keep is deleted or simplified, and code kept - "just in case", or because nobody knows whether it is needed, is an instant delete. Doubt is - not a reason to keep; confidence decides. -- **Tests.** The refusals and the boundary, not the happy path. Write down the partial fix or - one-field mutation that would still pass, as a patch the implementer can apply. High-risk code names a negative control, the whole change reverted onto a named commit - and red there, and one oracle independent of the author. + (`git diff --shortstat origin/main...`), production and tests apart. Production code that + grows needs a reason you accept; a branch that could delete more than it adds and does not goes + back with the deletion named, and so does a new gate, check, lock or test that guards what a + reader can check: that rule is a sentence in a prompt. What could be deleted, merged into what + exists, or made smaller? An abstraction with one caller, a parameter with one value, dead code, + code kept "just in case" or because nobody knows whether it is needed. Size is never bought with + a weaker check: a test is cut only when it tests nothing, or as **Guest tests** says. A + compromise the branch found is removed or recorded in `issues/` with an owner, evidence and an + exit condition. +- **Tests.** The refusals and the boundary, not the happy path. - **Edges.** Untrusted input never panics the kernel; it is refused. Check-then-act races. A lock held across a user copy or a device wait. Arithmetic on a value the caller chooses. A short read, an exit status nobody reads. An `at_most(::MAX)` or `index(usize::MAX)` on an `Untrusted` is an unwrap wearing a check's name. -- **Guest tests.** A behaviour is tested on the cheapest tier that reaches it: a type that makes - the bug unrepresentable, then a host test, then a metal row on the T14, and a QEMU guest test - last. A new guest test, or one whose behaviour changes, whose pull request body does not say - why a type, a host test and a metal row cannot reach its behaviour is a BLOCKER, and so is one - whose reason a cheaper tier answers. A guest test is cut for a cheaper tier only where that - tier already holds its behaviour, named in the pull request body, or where a stage of a track - names it, in the same diff, with the behaviour it guarded and an exit a build or test can fail; - any other cut is a BLOCKER. -- **Waits.** A flat wait — sleep, then assume it happened — is a BLOCKER, in code and in tests, - unless a hardware document mandates that time and offers no notification, cited at the site. - Wait on the event itself, bounded by a timeout that fails loudly. Defensive code that hides a - failure instead of failing fast is a BLOCKER too. +- **Guest tests.** A behaviour is tested on root `CLAUDE.md`'s cheapest tier that reaches it. A new + guest test, or one whose behaviour changes, whose pull request body does not say why a type, a + host test and a metal row cannot reach its behaviour is a BLOCKER, and so is one whose reason a + cheaper tier answers. A guest test is cut only where a cheaper tier already holds its behaviour, + named in the pull request body; where a stage of a track names it, in the same diff, with the + behaviour it guarded and an exit a build or test can fail; or as root `CLAUDE.md` deletes a red + or flaky test. Any other cut is a BLOCKER. +- **Waits.** A flat wait, or defensive code that hides a failure instead of failing fast (root + `CLAUDE.md`, "Fail fast"), is a BLOCKER, in code and in tests. - **Actuators.** A kernel static of any kind, atomic or `Lock`-wrapped, that an `actuator::` guard's arm reads or writes is touched only under that guard or inside an item compiled only with an actuator feature (`#[cfg(feature = "…-actuators")]`); a `cfg(not(feature = …))` item compiles into shipping and does not count, and any other touch runs on the shipping kernel. -- **Forks.** A fork change that is not upstream-mergeable is a BLOCKER: ToyOS enters as a new platform, a - cross-platform change is written as upstream would accept it, and a path dependency on a ToyOS crate - is never mergeable. A pull request that changes a fork's consumed commit changes it in every lockfile and - gitlink that names that branch; one it leaves behind is a BLOCKER. In `rust/`: any `library/alloc` or `library/core` delta, a cross-platform semantic change, a - `change_tracker` entry with no upstream PR number, a copied unmerged upstream PR. A search for +- **Forks.** A broken rule of `implementer.md`'s "A fork" is a BLOCKER, and so is a lockfile or + gitlink left naming a fork branch whose consumed commit the pull request changes. A search for callers that skipped the fork clones and `~/.cargo/git/checkouts/` searched part of the tree. -## Prose is removed, never reviewed - -A wrong line number, a stale run id, a count, a date, a citation: never a send-back, never -corrected, never checked for its own sake. Prose that is false, will rot or misleads is flagged -REMOVE, one line, and the implementer deletes it. Nobody rewrites prose. +## Prose -Reducing prose is part of the review. Every comment, doc line, issue line and PR-body line a -branch adds or rewrites must be load-bearing — the code or the record needs it — or it is REMOVE. -Prose rewritten or "corrected" instead of deleted is REMOVE. +In a source comment or a doc, a wrong line number, a stale run id, a count, a date, a citation: +never a send-back, never corrected, never checked for its own sake. Every comment, doc line, issue +line and PR-body line a branch adds or rewrites is load-bearing — the code or the record needs it — +or it is REMOVE, one line, and the implementer deletes it. So is a source comment that is not one +of root `CLAUDE.md`'s three kinds, and a comment or doc line corrected instead of deleted. ## Output -Post the report as a comment on the pull request (`gh pr comment --body-file`) and return the -same text. A later round opens with each earlier BLOCKER, CLOSED or OPEN, and the measurement that -says so. Then findings, one line each, `path:line — what — why`, under BLOCKER, NOTE, REMOVE. The -last line is the verdict alone: NOT READY FOR REVIEW, LAND, LAND AFTER NAMED CHANGES, or SEND BACK. -SEND BACK exactly when a BLOCKER is open. No summary of what the branch does. +Post the report as a comment on the pull request (`gh pr comment --body-file `) and return +the same text. A later round opens with each earlier BLOCKER, CLOSED or OPEN, and the measurement +that says so. Then findings, one line each, `path:line — what — why`, under BLOCKER, NOTE, REMOVE. +The last line is the verdict alone: SEND BACK exactly when a BLOCKER is open, LAND AFTER NAMED +CHANGES when only a NOTE or a REMOVE is, and LAND when nothing is. No summary of what the branch +does. diff --git a/.claude/agents/upstreamer.md b/.claude/agents/upstreamer.md deleted file mode 100644 index 8ae9aee56d4..00000000000 --- a/.claude/agents/upstreamer.md +++ /dev/null @@ -1,54 +0,0 @@ ---- -name: upstreamer -description: Turns one fork's ToyOS change into the smallest pull request its upstream maintainers can review and merge in minutes. -tools: Bash, Read, Write, Edit, Grep, Glob ---- - -You carry one fork's change to its upstream project. A fork exists only to carry a change being -upstreamed, and it goes when upstream has it. The measure of your work is the maintainer's time: -a pull request they can read in a minute and merge without a question. - -## The change is theirs, not ours - -- Start from upstream's current default branch, never from our fork's history. Re-derive the - change on it; our fork is a reference, not a base. -- The smallest diff that does the job. No refactors, no drive-by fixes, no renames, no new - abstractions, no comments upstream would not write. A change our fork carries that upstream does - not need, or that serves only ToyOS's tooling, stays out. Two independent changes are two pull - requests. -- Their style, exactly. Read `CONTRIBUTING.md`, the PR template, the changelog convention and the - last ten merged pull requests that added a platform or target, and copy them: file placement, - `cfg` spelling and ordering, naming, import order, error types, doc tone, commit subject form. - Where a similar platform exists (another hobby or tier-3 OS), mirror its shape line for line. -- Their checks, green. Run the project's own formatter with its config, its clippy and lint - settings, its tests and its CI's check commands, from its CI files, and record each exit code. - Nothing of ours (ToyOS gates, conventions, prose rules) applies inside their repository. -- One commit unless their history shows otherwise, with a subject in their form. - -## Prose is a cost to them - -- The PR title is their convention's one line. The body is their template, filled tersely: what - it adds in one sentence, how it was tested in one line. No background, no story, no ToyOS - architecture, no marketing, no lists of what was not changed. -- End commit messages and the PR body with the attribution lines the session gives; add nothing - else of ours. - -## Before anything is published - -Opening a pull request on someone else's project is outward-facing. You prepare, you do not -publish: - -- Check that upstream can accept the change at all: whether `target_os = "toyos"` is known to the - compiler they build with (`unexpected_cfgs`, their MSRV), whether they take tier-3 targets, and - whether another project must land first (the Rust target, `target-lexicon`, `libc`). If it - cannot be accepted yet, say what must land first and stop. -- Push the branch to our fork of the project under a name that says what it is, and report: the - compare URL, the diff stat, the title and body you would open with, every check with its exit - code, and the one upstream PR you modelled it on. The orchestrator opens it. -- Answering review is the same discipline: the smallest change that answers the comment, in - their style, with a one-line reply. - -## Report - -Say plainly whether the change is upstreamable now, what blocks it if not, and what ToyOS still -needs from its fork until upstream releases it. diff --git a/.github/pull_request_template.md b/.github/pull_request_template.md index 4db02e7fd67..beee6196b5b 100644 --- a/.github/pull_request_template.md +++ b/.github/pull_request_template.md @@ -1,7 +1,7 @@ +Each gate with its exit code. Numbers come from a command that was actually +run, with the run id or the command. An estimate says it is one. --> + + +## What is unsure ## Anything a reader of `main` must not miss diff --git a/.gitignore b/.gitignore index 09df96c737f..33b9e87c42a 100644 --- a/.gitignore +++ b/.gitignore @@ -5,7 +5,6 @@ target/ !.claude/agents/reviewer.md !.claude/agents/implementer.md !.claude/agents/orchestrator.md -!.claude/agents/upstreamer.md # A SoundFont is a licence question, and this repository answers it for exactly # one: `assets/soundfont.sf2`, which `NOTICE` accounts for. The pattern stays # for every other one, because a bank somebody drops in to try is a bank nobody diff --git a/CLAUDE.md b/CLAUDE.md index 3c5f2ec6c0e..91220c75994 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -1,46 +1,43 @@ # ToyOS -A general-purpose operating system built from scratch in Rust, held to a production-grade engineering bar — the bar is the changes, not yet the product. Modern x86-64 hardware (2020+), UEFI only; ARM64 planned — keep the architecture portable. Its test machines decide its feature set, never its design. The quality bar is shipping software: correct, efficient, minimal, zero silent debt. A tracked weakness is still a weakness: the honest answer about current state is "known, tracked, still true" — never "we have an issue for that." +A general-purpose operating system built from scratch in Rust, held to a production-grade engineering bar — the bar is the changes, not yet the product. Modern x86-64 hardware (2020+) and AArch64 under QEMU `virt`, UEFI only; keep the architecture portable. Its test machines decide its feature set, never its design. The quality bar is shipping software: correct, efficient, minimal. A tracked weakness is still a weakness: the honest answer about current state is "known, tracked, still true" — never "we have an issue for that." ## Where the rest of this lives -**This file is what every agent needs before it knows which subsystem it is in.** Detail lives below it and loads when you go there: +This file is what every agent needs before it knows which subsystem it is in. A subdirectory `CLAUDE.md` loads when a file in that subtree is `Read`, and not from `Bash`. | | | |---|---| | `kernel/CLAUDE.md` | the caveats that bite kernel work | | `userland/CLAUDE.md` | the server doctrine, and the caveats that bite userland work | | `tests/CLAUDE.md` | the caveats that bite the harness | -| `src/CLAUDE.md` | boot modes, the locks, worktrees — the operational file | +| `src/CLAUDE.md` | the caveats that bite build-system work | | `issues/README.md` | the issue tracker: one file per issue, typed by kind; `ls` is the index | -| `.claude/agents/reviewer.md` | the review prompt the orchestrator spawns a reviewer with | +| `.claude/agents/` | each role's prompt; `reviewer.md` is the bar a branch lands against | -There are no spec documents. Rules live where they are enforced — a gate, a -module header, the review prompt — and everything else is an -issue. Free text that merely describes the tree rots and is deleted, not -maintained. A `CLAUDE.md` never holds a list that a manifest, a directory or a -gate already answers; it points at that source instead. - -A subdirectory `CLAUDE.md` loads when a file in that subtree is `Read`, and not from `Bash`. A rule whose violation is unrecoverable or invisible stays here; everything else lives where the work is. +There are no spec documents. A rule a reader can check lives in an agent's prompt or at its site; code enforces only what reading cannot see — runtime behaviour, bytes, measurements — and everything else is an issue. A `CLAUDE.md` never holds a list that a manifest, a directory or a gate already answers; it points at that source instead. ## Principles - **Zero legacy.** No backwards compatibility, no fallbacks, no workarounds, no BIOS, no 32-bit. Research state-of-the-art OS design instead of replicating older OSes. - **Zero silent debt.** Dead code is deleted; every abstraction earns its place. A discovered compromise has exactly two legal outcomes: remove it, or record it with ownership, evidence and an exit condition — and it stays a present-state weakness until removed. -- **Fail fast, trust nothing.** Panics over silent degradation; exhaustive matches; the unimplemented dies loudly. Input that crossed a trust boundary is never trusted and never panics the kernel — it is refused. +- **Code is liability.** If code can be deleted, it is; code that does not earn its keep is deleted or simplified, and not knowing whether it is needed is no reason to keep it. Tooling is code too: a gate, lock, check or test is added only for what reading cannot see. +- **Fail fast, trust nothing.** Panics over silent degradation; exhaustive matches; the unimplemented dies loudly; no defensive code. Input that crossed a trust boundary is never trusted and never panics the kernel — it is refused. Never a flat wait, in code or in a test: wait on the event, bounded by a timeout that fails loudly; a fixed delay only where a hardware document mandates it and offers no notification, cited at the site. - **The kernel never crashes from userland.** A kernel bug crashes loudly; a userland bug never reaches it. +- **Userland first.** The kernel takes on only what userland cannot. - **No kernel threads.** The kernel creates no thread but the per-CPU idle loop: kernel work runs, bounded, on the thread or interrupt that caused it and is charged to it; long-running work with no owner is a userland server's. -- **Rust is first class.** Not POSIX, not C. Unrepresentable is best: prefer compile-time safety over runtime checks over tests. -- **Existing Rust just works.** A program that builds for other operating systems builds and runs on ToyOS unchanged; the ecosystem gains ToyOS support through forks carried upstream, never through ToyOS-specific replacement crates. -- **Development ergonomics above all.** Iteration speed beats feature count; tooling comes first. +- **Rust is first class.** Not POSIX, not C. Unrepresentable is best: compile-time safety over runtime checks over tests, and nothing a type refuses is tested. +- **Existing Rust just works.** A program that builds for other operating systems builds and runs on ToyOS unchanged; the ecosystem gains ToyOS support through forks, never through ToyOS-specific replacement crates. +- **Apps are portable.** An app builds and runs on Linux under Wayland (never X11, which is legacy), macOS and Windows from the same source as on ToyOS; a host build that fails is fixed in the app or its dependencies, never by making the app ToyOS-only. A program is ToyOS-only only when its job exists only on ToyOS — it owns ToyOS devices or kernel objects, or manages ToyOS itself — and its manifest's `exempt` says which, and why (`src/userlandhost.rs`). +- **Development ergonomics above all.** Iteration speed beats feature count. ## Architecture > A snapshot, deliberately shallow — always read the code. -**Kernel** — takes on only what userland cannot. 2 MB pages, demand paging, PIE binaries, full SMP. +**Kernel** — 2 MB pages, demand paging, PIE binaries, full SMP. -**Userspace daemons** — compositor, netd, soundd, sshd, logd. Each claims a device or capability from the kernel and serves its function; crash one and the kernel is fine. +**Userspace servers** — `system.toml` names them. Each claims a device or capability from the kernel and serves its function; crash one and the kernel is fine. **The log is a userland file.** `/system/bin/logd` reads records on a cursor and owns `/log`; the kernel keeps the record ring, the console and the panel, and writes no file. `SYS_FSYNC` reaches the device's cache flush because logd's durability claim rests on it. @@ -58,51 +55,44 @@ A subdirectory `CLAUDE.md` loads when a file in that subtree is `Read`, and not ## Dependencies -**Rust** and **QEMU** for development, on any host OS and architecture — the development machine is nothing special. Beside them, where no Rust tool does the job, only C or C++ tools ToyOS can one day build and run (Python, Perl, CMake, make), each declared. No binary for one host OS alone: a macOS binary is a hard no, and "only for tests" does not soften it. ToyOS's own code is Rust; it writes no Python, Perl or shell of its own. Only general and widely used crates — one that does *our* job we write ourselves, and a driver crate never; third-party crates are used as published, and a fork carries a change written to upstream quality and goes when upstream has it. No upstream pull requests are sent for now: ToyOS needs more attention and more contributors before upstream projects take it seriously, and upstreams tend to refuse AI-first projects and their contributions. A third-party source ToyOS cannot build without changing it is carried as an unmodified-source packaging mirror with a byte-identity gate, not as a fork. The north star is **self-hosting**: nothing — build, test, or verification — rests on a host binary. Ask of anything new: could this ever run inside ToyOS? Self-hosting means ToyOS rebuilds itself on ToyOS and reproduces the host's bytes; a bootstrap from source with no binary seed is out of scope. - -Vendor firmware a device or CPU verifies by its maker's signature may be shipped: pinned by version and hash, redistributable unmodified, recorded in `NOTICE`. A device's is loaded only by its own driver through its IOMMU domain and never executes on the CPU; CPU microcode is loaded by the kernel. +**Rust** and **QEMU** for development, on any host OS and architecture — the development machine is nothing special. Beside them, where no Rust tool does the job, only C or C++ tools ToyOS can one day build and run (Python, Perl, CMake, make), each declared where `reviewer.md`'s Arrivals says. No binary for one host OS alone: a macOS binary is a hard no, and "only for tests" does not soften it. ToyOS's own code is Rust; it writes no Python, Perl or shell of its own. Only general and widely used crates — one that does *our* job we write ourselves, and a driver crate never; third-party crates are used as published, and a fork carries a change written to upstream quality and goes when upstream has it. No upstream pull request is sent for now. A third-party source ToyOS cannot build without changing it is carried as an unmodified-source packaging mirror with a byte-identity gate, not as a fork. The north star is **self-hosting**: ToyOS rebuilds itself on ToyOS and reproduces the host's bytes, and nothing — build, test, or verification — rests on a host binary; a bootstrap from source with no binary seed is out of scope. Ask of anything new: could this ever run inside ToyOS? -The bar is not yet the tree: `.claude/agents/reviewer.md`, "Arrivals", says where every host tool and every standing failure is declared. `NOTICE` names every committed third-party file with its hash, upstream and licence; an image carrying `DOOM1.WAD` may not be sold. +Vendor firmware a device or CPU verifies by its maker's signature may be shipped: pinned by version and hash, redistributable unmodified, recorded in `NOTICE`. A device's is loaded only by its own driver through its IOMMU domain and never executes on the CPU; CPU microcode is loaded by the kernel. `NOTICE` names every committed third-party file with its hash, upstream and licence. - **toyos-ld** — frozen: everything links with rust-lld, and toyos-ld stays only as the linker inside ToyOS until lld runs there, then goes. - **rust/** — Rust compiler/std fork with ToyOS platform support (submodule). Auto-bootstraps; kept current with upstream. Its rules: `.claude/agents/implementer.md`, "A fork". ## Build & test -The testing rules live where they are enforced: the PR gate and the nightly in `.github/workflows/`. Operationally: - -- `cargo run` builds everything (toolchain, kernel, bootloader, userland, image) and launches QEMU; `--build-only` skips the launch. `cargo test` runs the QEMU harness; `cargo run -- --ci host` runs every host suite, as the PR gate's required `host` check does. -- **Agents never run QEMU.** An agent verifies with host tests and builds the image at most; the orchestrator runs every guest test, one suite at a time. -- **Both produce large output**: run them in the background and read the output file — `[N characters truncated]` means data was lost. A full boot is under a second; incremental builds finish in seconds. -- **Leave the machine as you found it.** The development machine is shared: every agent stops what it started, killing only by PID and waiting out a build that holds the global lock, and removes the worktrees and scratch build output it no longer needs. +- `cargo run` builds everything (toolchain, kernel, bootloader, userland, image) and launches QEMU; `--build-only` skips the launch. `cargo test` runs the QEMU harness, and `cargo test --test toyos-build -- --metal` the T14's. `cargo run -- --ci host` runs every host suite: it is the `host` check a ready pull request and the merge queue run. Guests run nightly. +- **A behaviour is tested on the cheapest tier that reaches it**: a type that makes the bug unrepresentable, then a host test, then a metal row on the T14, and a QEMU guest test last. +- **Agents run their own guest tests**, the whole suite or a filter, side by side. **The T14 is the orchestrator's alone: no other agent runs `--metal` without `--metal-readback`, which touches no machine.** +- **Timing and audio verdicts come only from metal.** A QEMU test asserts order, completion, content and counts, never how long something took, and plays no audio; its only clock is a hang ceiling. +- **A red test is a defect**: it is fixed, or deleted with its issue recording the commit that restores it; a flaky test is deleted at once, never re-run. A red seen only under load is no flake: it is a defect, recorded with the host's load. +- **A high-risk change names its checks.** Security boundaries, the scheduler, the ABI, filesystems, devices, memory management, concurrency primitives: a negative control where a defect would otherwise land unseen — the *whole* change reverted onto the base the green arm was measured on — and an independent oracle where one exists: an external specification, a differential implementation, real hardware, a third-party checker, a formal model, or a recorded real failure. A second agent is not independence. +- **Never truncate command output.** No `| head`, `| tail`, `| grep` to reduce it: long output runs in the background and is read from its file — `[N characters truncated]` means data was lost. +- **Leave the machine as you found it.** The development machine is shared: every agent stops what it started and never another's process, killing only by PID and waiting out a build that holds the global lock, and removes the scratch output it made once it no longer needs it. ## Repository layout The root `Cargo.toml`'s `[workspace]` `members` and `exclude` lists account for every crate in the tree, and `src/hostws.rs` reds on one in neither; every package they name says what it is in its `description`. -## Workflow +## Working here -**One agent, one worktree, one branch.** `cargo run -- --worktree add ` makes one; never `git worktree add` by hand — the naive path clones the rust fork's history and takes the machine-global toolchain name from every other checkout. The primary checkout is not a workspace: it owns `rust/`, the rustup link and `main`; `cargo run -- --sync` moves it onto whatever GitHub merged. +- **Stay on the task.** What you find off it is filed in `issues/`, one file per issue, and not fixed on the way. If something blocks, stop and report it; never work around it. What ToyOS cannot do yet is filed, and work on it starts only on the owner's go. +- **Never degrade audible or visual quality** — even temporarily, even for a big win elsewhere — without the owner's explicit sign-off. +- **Always be empirical.** Read actual output; run the code; investigate root causes instead of guessing. Every written number comes from a command that was run; an estimate or datasheet bound says so. +- **Never put the owner's email or any other personal data in a network request or its headers**; any `User-Agent` is `toyos-build (https://github.com/ToyOSOrg/ToyOS)`. +- **One agent, one worktree, one branch.** The primary checkout owns `rust/`, the rustup link and `main`, and is no workspace; `.claude/agents/implementer.md` makes a worktree and `orchestrator.md` removes it. Never make one with `git clone`, and never run `git submodule` in one: either fetches the fork's history again, a clone builds a toolchain that takes the rustup link, and `git submodule` writes `core.worktree` into the fork's shared config, which breaks git in the primary's `rust/`. +- **Commit freely on your branch; never rewrite history.** `git commit -F `, never `-m`. No `--amend`, no `rebase`, no `--force` — merge `origin/main` instead: a pushed hash may already be cited. +- **Never touch `main`.** It moves only through a merged pull request, by its required merge queue, and is protected — no push, force-push, deletion or bypass. A pull request's title and body become the merge commit's: write them as `main`'s record. A branch lands after a review against `.claude/agents/reviewer.md`. A modify/delete conflict is resolved by accounting for every hunk of the modified side, never by checking its headings survived. A merge that deletes a document also deletes every citation to it in the same merge, found by searching the bare name as well as the path. An ABI change lands with the work that needs it: every worktree builds the toolchain its own sources name, so branches that change the ABI run side by side. Every merge leaves `main`'s tip compiling. -- Stay on the current task. File what you find in `issues/` and do not go fix it; one file per issue, its README has the shape. -- If something blocks, stop and report it. Don't work around it. -- Never degrade audible or visual quality — even temporarily, even for a big win elsewhere — without the owner's explicit sign-off. -- **Never truncate command output.** No `| head`, `| tail`, `| grep` to reduce it; long output runs in the background and is read from the file. -- **Always be empirical.** Read actual output; run the code; investigate root causes instead of guessing. -- **Every written number comes from a command that was run.** An estimate or datasheet bound says so. Write commit messages with `git commit -F `, never `-m` — a double-quoted `-m` substitutes backticks and the shell runs them. -- **Commit freely on your branch; land through a pull request.** `main` moves only through a merged PR. `gh pr create --draft` at the first push — CI runs on PRs and nothing else; `gh pr ready` plus a written `--title`/`--body-file` when finished (never `--fill`); `gh pr merge --auto --merge` enqueues on `main`'s required merge queue, which builds each merge's exact composition and runs the required checks on it before `main` moves; `cargo run -- --sync` after it lands. Never merge into `main` by hand. The PR's title and body become the merge commit's: write them as main's record. A modify/delete conflict is resolved by accounting for every hunk of the modified side, never by checking its headings survived. A merge that deletes a document also deletes every citation to it in the same merge, checked by searching the bare name as well as the path. An ABI change lands with the work that needs it: every worktree builds the toolchain its own sources name, so branches that change the ABI run side by side and none waits on another. Every merge leaves `main`'s tip compiling. A branch lands after a review against `.claude/agents/reviewer.md`, spawned by the orchestrator with its brief and judged by it. -- **Never rewrite history, and never touch `main`.** No `--amend`, no `rebase`, no `--force` — on your own branch as much as anywhere: a pushed hash may already be cited. `main` is protected — PR required, no force-push, no deletion, no bypass. -- **A red test is a defect: it is fixed, or deleted with its issue recording the commit that restores it; a flaky test is deleted at once, never re-run.** -- **A high-risk change names its two checks.** Security boundaries, the scheduler, the ABI, filesystems, devices, memory management, concurrency primitives: the PR names the negative control or mutation that fails if the implementation is wrong, and one epistemically independent oracle — an external specification, a differential implementation, real hardware, a third-party checker, a formal model, or a recorded real failure. A second agent is not independence: five artifacts from one wrong model still agree. A mutation is a negative control only if it reverts the *whole* change onto the base the green arm was measured on — a one-line revert of a change that moved two things measures neither. -- **Timing and audio verdicts come only from metal.** A QEMU test asserts order, completion, content and counts, never how long something took, and plays no audio; its only clock is a hang ceiling. -- **Subagents wait in the foreground** — background notifications do not reliably re-wake them: explicit `timeout`s, and for longer work background once and block with a few long foreground waits, polling before each sleep. -- **An agent never waits on CI.** It arms auto-merge, reports, and exits. Sequencing across landings belongs to the orchestrator, done in passes on its own wake-ups; several finished branches land as one batch PR rather than as one agent babysitting N cycles. -- **Subagents get an explicit model, never the session default.** The orchestrator scopes, dispatches and verifies; it edits nothing. Match the tier to the judgment in the task: judgment-bearing coding gets a frontier model, mechanical execution from an exact brief a mid tier, and non-coding mechanical work the cheapest. Never encode a temporary usage circumstance as a rule. -- **An agent is spawned fresh for every task, never resumed into the next one.** A long transcript rots: a resumed agent carries the last task's assumptions, stale facts and a filled context into the new brief. Once it has reported it is done; a question back to it is fine, a new assignment is not. -- **Durable facts go in the module header at the site — never in private agent memory, and almost never in a `CLAUDE.md`.** A `CLAUDE.md` is pointers and caveats of the most general kind — it never cites an individual issue file, because it is not an issue tracker; **an agent edits one only when briefed to.** A rule that truly has no better home and whose violation is invisible or unrecoverable is *proposed as one sentence in the final report*, and the orchestrator declines it or briefs an agent to place it. The story of a change goes in its commit message; after each task, audit the module header that owns what you changed. A comment never restates a count that somebody else's landing moves. -- **Code is the product, not prose.** Prose exists only where it is load-bearing — where the code needs it to be read correctly. Stale or false prose is deleted, never corrected or rewritten, and the reviewer cuts every line that is not load-bearing. -- **Code is liability.** If code can be deleted, it is; code that does not earn its keep is deleted or simplified. Keeping code "just in case" or for want of knowing whether it is needed is not a reason — it is deleted. We are confident, and the reviewer enforces it. -- **A comment is one of three kinds or it goes** — the one-clause invariant at the edit site, the boundary contract, or the refusal-reason at a surprising decision, over a module doc that is the contract and nothing else. Chronology, measurements' provenance, past implementations, investigation stories and narration of the obvious live in commit messages and the tracker, never in source — a date in a source comment is the tell, and a `CLAUDE.md` carries none either. Moving a durable fact to the site means moving the invariant, never the investigation. `.claude/agents/reviewer.md` holds this; a `CLAUDE.md` never grows. +## Prose + +- **Code is the product, not prose.** Prose exists only where it is load-bearing — where the code or the record needs it to be read correctly. Stale or false prose in source and docs is deleted, never corrected or rewritten; a record — a pull request's body, an issue, a prompt — is kept true of what it describes. +- **Durable facts go in the module header at the site** — never in private agent memory, and almost never in a `CLAUDE.md`. A `CLAUDE.md` never grows: it holds pointers and caveats of the most general kind, and never cites an individual issue file. The story of a change goes in its commit message. A comment never restates a count that somebody else's landing moves. +- **An agent edits a `CLAUDE.md` or a role prompt only when briefed to**, or to update a command, flag or step its own change removes or renames. A rule that truly has no better home and whose violation is invisible or unrecoverable is proposed as one sentence in the agent's final report. +- **A comment is one of three kinds or it goes** — the one-clause invariant at the edit site, the boundary contract, or the refusal-reason at a surprising decision, over a module doc that is the contract and nothing else. Chronology, measurements' provenance, past implementations, investigation stories and narration of the obvious live in commit messages and the tracker, never in source — a date in a source comment is the tell, and a `CLAUDE.md` carries none either. ## Planned work diff --git a/Cargo.toml b/Cargo.toml index 318e42c520e..9fbbcf0c5ca 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -112,9 +112,8 @@ default-run = "toyos-build" fatfs = "0.3.6" fontdue = "0.9" gpt = "3.1.0" -# `statvfs` for `worktree::free_bytes`, and the unprivileged ICMP datagram -# socket `icmp::echo` asks a metal boot over, which is the platform call that -# replaces a `ping` binary. +# The unprivileged ICMP datagram socket `icmp::echo` asks a metal boot over, +# which is the platform call that replaces a `ping` binary. libc = "0.2" toml = "0.8" uuid = { version = "1", features = ["v4"] } diff --git a/issues/README.md b/issues/README.md index a8a12f28438..88dd02eec2a 100644 --- a/issues/README.md +++ b/issues/README.md @@ -1,14 +1,7 @@ # Issues -`issues/` at the repository root is the only prose this project maintains. -Everything else that could be silently false — a spec, a plan, an assessment — -was deleted, because prose is the one artifact nothing checks. An issue is -different in kind: it reproduces or it does not, and closing it costs evidence. - One file per issue, `issues//.md`. There is no index and no -numbering: **`ls` is the index and the frontmatter is the query.** A number -encodes a position, and every insertion moved one — which is what made a -reference to an issue a reference that rots. +numbering: **`ls` is the index and the frontmatter is the query.** `ls issues/*/` lists everything. To ask a question of the set: @@ -16,7 +9,6 @@ reference to an issue a reference that rots. rg -l '^status: open' issues/ # every unheld piece of work rg -l '^status: assigned' issues/ # what somebody is holding rg -l '^status: owner' issues/ # what is waiting on the owner -rg -c '' issues/audio/ # how much audio owes ``` ## Frontmatter @@ -31,16 +23,13 @@ Four fields, three required, no defaults. | | `none` | nothing is owed | | `kind` | `defect` | real, reproducible, someone should fix it | | | `tooling` | the development machine — the harness, a gate, a price, CI, the tracker, the build system, a measurement owed | -| | `finding` | noticed in passing — and bounded: at its next review it is promoted to a `defect` or folded into the owning module header and closed (owner ruling 2026-08-25) | +| | `finding` | noticed in passing — and bounded: at its next review it is promoted to a `defect` or folded into the owning module header and closed | | | `track` | staged work — something to build that nobody has built | | | `question` | blocked on the owner, and nobody else can decide it | | | `rejected` | considered and declined, recorded so nobody re-proposes it | -| `opened` | `YYYY-MM-DD` | the first commit whose issue tracker carried this heading. Before 2026-08-08 that is derived from the single file this directory replaced, so a reworded heading dates from the rewording | +| `opened` | `YYYY-MM-DD` | the first commit whose issue tracker carried this heading | | `task` | a number | optional; present only where the issue names one | -`kind: defect` is the OS; `kind: tooling` is the development machine, worked -only when it blocks a landing. - **`status` and `kind` are not free of each other.** `kind` says what the entry is; `status` says what is owed. Two of the kinds answer that second question by themselves, so they may not contradict it: @@ -52,18 +41,13 @@ answer that second question by themselves, so they may not contradict it: | `question` | `owner` | | `rejected` | `none` | -That rule is what makes `rg -l '^status: open'` mean *unheld work* rather than -"every file that was not assigned to somebody" — the eleven `question` and -`rejected` files all said `open` before it existed, so the query over-reported -by eleven and nothing could tell. - **`kind: rejected` is not work.** It is here so the next agent does not spend a day re-deriving an answer the owner already gave. Nothing in a `rejected` file is owed — and if the body says otherwise, the *kind* is what is wrong. A ruling that declared a standing failure rather than removing it deferred the work; it did not decline it, so the entry is a `defect` and stays open. -**`kind: finding` does not accumulate** (owner ruling 2026-08-25). A finding +**`kind: finding` does not accumulate.** A finding has a bounded life: whoever next reviews it either promotes it to a `defect` (something real that someone should act on — a fix, a measurement, an instrument) or moves its one durable line to the module header or doc comment @@ -84,16 +68,11 @@ defect is. A `track` says what is to be built, what it is blocked on, and any constraint a reader would otherwise pay to re-derive — a hardware bound, a number somebody measured, a design line the owner already drew. It does not carry a design, a stage table, a rationale or a review history: a design that is -right is written as code, and one that is not yet written is not yet known. A -`track` that has grown past a screen is a plan again, and is cut back. +right is written as code, and one that is not yet written is not yet known. ## Areas -`isolation` · `panic-path` · `kernel` · `audio` · `diagnostics` · `build` · -`design-debt` · `hardware` · `filesystem` · `boot-media` - -That list is closed. An area is a -directory because it makes every cross-reference a path that resolves. Moving +An area is a directory because it makes every cross-reference a path that resolves. Moving an issue between areas is a `git mv`; the **slug** is its identity — unique across every area — so `rg ` finds every pointer at it wherever it has been put. A slug is a claim like any sentence here: one the tree has refuted is @@ -138,17 +117,3 @@ and `.github/` holds citations too. Then read where the hits are. One in a comme under `toyos-abi/src`, `toyos/src` or a published crate changes no identity (`src/identity.rs`), so it owes no version and builds no sysroot. -## Two area notes, carried over from the file this replaced - -**`filesystem`** — `toyos-fat32/` is new (host tests: `cargo test` inside it) and -`userland/fsd/src/fat.rs` serves it; `boot-media` carries what the kernel's -adapter it replaced found. Most of what is filed here is not a defect found later but a -residual the crate's own gate identified while it was being written, recorded so -the adapter's author did not have to rediscover it. - -**`boot-media`** — `/boot` and `/log` are both served by `/system/bin/fsd` over -`toyos-fat32`, off the partitions the loader names by unique GUID; -the kernel writes no log file — `/system/bin/logd` does, an ordinary user process that -owns "every policy about files — where they go, what they are called, how many -there are, what happens when the stick stops answering" -(`userland/logd/src/main.rs:1-10`). diff --git a/issues/build/a-loaded-suite-reds-a-volume-checker-on-both-arms.md b/issues/build/a-loaded-suite-reds-a-volume-checker-on-both-arms.md index 3fb7de7f953..073a13022d0 100644 --- a/issues/build/a-loaded-suite-reds-a-volume-checker-on-both-arms.md +++ b/issues/build/a-loaded-suite-reds-a-volume-checker-on-both-arms.md @@ -48,8 +48,7 @@ is that this name cannot be waved through on a scheduler branch. **A decisive A/B needs a quiet host and this one stopped being quiet.** Six more runs an arm were started and abandoned: the first took **417.4 s** against the 58-79 s of every run above, and `pgrep` found three other worktrees' suites on -the box. `tests/CLAUDE.md`'s rule is that a block which gains company mid-run is -discarded and re-run, never corrected, so it was discarded. +the box, so it was discarded. ## What to do with it diff --git a/issues/build/a-worktree-cannot-build-a-hosted-rustc-of-its-own.md b/issues/build/a-worktree-cannot-build-a-hosted-rustc-of-its-own.md index a1938ad124f..29f1b4b9a5e 100644 --- a/issues/build/a-worktree-cannot-build-a-hosted-rustc-of-its-own.md +++ b/issues/build/a-worktree-cannot-build-a-hosted-rustc-of-its-own.md @@ -10,7 +10,7 @@ The ToyOS-hosted rustc is built by the primary checkout alone (`src/toolchain.rs`'s `ensure`, under `Owner::Us`), from the primary's `rust/` under the primary's `write_config`. A worktree's image with `hosted-rustc = true` carries that one: refused by name when the worktree builds a compiler of -its own (`src/build.rs`, `env.primary_compiler`), and taken without a word when +its own (`src/build.rs`, `env.sysroot.primary_compiler`), and taken without a word when only the worktree's `write_config` differs, so the image then carries a rustc another tree's recipe built. diff --git a/issues/build/apps-that-build-or-run-on-toyos-alone.md b/issues/build/apps-that-build-or-run-on-toyos-alone.md index a842e0799e4..c1b30838c11 100644 --- a/issues/build/apps-that-build-or-run-on-toyos-alone.md +++ b/issues/build/apps-that-build-or-run-on-toyos-alone.md @@ -7,7 +7,7 @@ opened: 2026-10-01 # Apps that build or run on ToyOS alone An app builds and runs on Linux under Wayland, macOS and Windows from the same -source as on ToyOS (`userland/CLAUDE.md`). These build on none of the three: +source as on ToyOS (root `CLAUDE.md`). These build on none of the three: each manifest's `[package.metadata.toyos.host] fails` names all three, so `cargo run -- --ci host` checks none of them there. diff --git a/issues/build/cargo-package-fails-in-a-linked-worktree.md b/issues/build/cargo-package-fails-in-a-linked-worktree.md index 69b8987674f..df886e46a30 100644 --- a/issues/build/cargo-package-fails-in-a-linked-worktree.md +++ b/issues/build/cargo-package-fails-in-a-linked-worktree.md @@ -7,20 +7,19 @@ opened: 2026-09-26 # `cargo package`/`cargo publish --dry-run` cannot run inside a linked worktree `cargo package -p ` (tried `toyos-abi` and `toyos-ld`) -reds with a bare `error: No such file or directory (os error 2)` in a worktree -made by `cargo run -- --worktree add`, right after cargo's own trace logs +reds with a bare `error: No such file or directory (os error 2)` in a linked +worktree, right after cargo's own trace logs `found a git repo` and `found (git) Cargo.toml`, inside `cargo::ops::cargo_package::vcs::check_repo_state` — before it prints anything about a dirty tree, and regardless of `--allow-dirty` or a fully clean working tree (confirmed with `git stash`). The identical command against the identical crate exits 0 in the **primary** checkout. Every worktree carries dozens of submodule entries (`userland/*`, `rust`) registered in the shared `.git/config` -but not checked out on disk (`rust`'s own empty stub included, per -`src/worktree.rs`) — a repo shape only a linked worktree has, and the likely +but not checked out on disk (`rust`'s own empty stub included) — a repo shape only a linked worktree has, and the likely reason cargo's git-repo-state walk (`git2`) chokes there and not in the primary. -Reproduce: from any `--worktree add` checkout, `cargo package -p toyos-abi +Reproduce: from any linked worktree, `cargo package -p toyos-abi --no-verify --allow-dirty` exits 101 with that message; the same command in the primary checkout exits 0. diff --git a/issues/build/sync-from-a-worktree-never-moves-main-because-the-primary-reads-dirty.md b/issues/build/sync-from-a-worktree-never-moves-main-because-the-primary-reads-dirty.md deleted file mode 100644 index 9ac307c8845..00000000000 --- a/issues/build/sync-from-a-worktree-never-moves-main-because-the-primary-reads-dirty.md +++ /dev/null @@ -1,15 +0,0 @@ ---- -status: open -kind: tooling -opened: 2026-09-29 ---- - -# `--sync` run from a worktree never moves main, because the primary always reads dirty - -`sync` skips the fast-forward when the primary checkout has uncommitted work and -the asking checkout is not the primary. The primary's `git status --porcelain` -always reads ` M rust`, so from any worktree `main` is left where it is and -only a `--sync` run in the primary itself moves it. - -True on `origin/main` before the `--pr` removal too. Not fixed by the branch -that renamed `pr.rs` to `sync.rs`. diff --git a/issues/build/the-build-runs-host-tools-outside-rust-and-qemu.md b/issues/build/the-build-runs-host-tools-outside-rust-and-qemu.md index b0239d20584..2db25ab2343 100644 --- a/issues/build/the-build-runs-host-tools-outside-rust-and-qemu.md +++ b/issues/build/the-build-runs-host-tools-outside-rust-and-qemu.md @@ -17,8 +17,8 @@ arrives and is not one. M4 and M5 are stages of `issues/build/toyos-builds-itsel | Python | LLVM's CMake, whenever this host builds an LLVM (`src/llvm.rs`); the C++ runtime's CMake, in every sysroot build (`src/libcxx.rs`), and its build, which runs `libcxx/utils/generate_iwyu_mapping.py` for a header it installs | admitted: no Rust tool does the job, LLVM's CMake requires one (`find_package(Python3 … REQUIRED)` in `rust/src/llvm-project/llvm/CMakeLists.txt`), and so does the runtimes', unconditionally (`runtimes/CMakeLists.txt`): configured as `src/libcxx.rs` does with no Python on `PATH` or in CMake's search, it found none and stopped, exit 1, and with only `python3` added it configured, exit 0 | M5 runs it in the guest | | CMake | rustc's bootstrap, for LLVM and clang; `src/llvm.rs`, for the LLVM's key; `src/libcxx.rs`, for the C++ runtime in every sysroot build | admitted: no Rust tool does the job, LLVM, clang and LLD are described in CMake, and upstream's only other descriptions are a GN overlay it does not support and a Bazel one | M5 runs it in the guest | | `sh` running LLVM's `config.guess`, and the POSIX tools and `cc` it runs | LLVM's CMake, whenever this host builds an LLVM, and the C++ runtime's, in every sysroot build, ask it the host's triple, unconditionally (`get_host_triple` in `rust/src/llvm-project/llvm/cmake/modules/GetHostTriple.cmake`, which runs `sh` by name) | refused: a Rust tool does the shell's part, brush 0.4.0: on the development host (macOS, arm64) `config.guess` printed `/bin/sh`'s triple under it, `arm64-apple-darwin27.0.0`, exit 0 each. The script runs `sed`, `uname`, `mktemp`, `grep`, `rm`, `rmdir` and `cc` there under either shell, and that `cc` is the `cc` rows'. Five of the other six are refused, uutils' doing each: under brush with sed 0.2.0, grep 0.2.0 and coreutils 0.12.0's `mktemp`, `rm` and `rmdir`, and nothing else on `PATH` but the host's `uname` and `cc`, it printed that triple, exit 0. `uname` is admitted: coreutils 0.12.0's answers `-p` with `unknown` where macOS's answers `arm`, and `config.guess` reads that as PowerPC, `powerpc-apple-darwin27.0.0`, exit 0 | CMake finds brush as its `sh`, uutils' `sed`, `grep`, `mktemp`, `rm` and `rmdir`, and a Rust `uname` that answers `-p` as the host's does; or M5 runs it in the guest | -| `git` for worktrees, submodules, checkouts, fixtures and rustc's bootstrap | adds, removes and prunes worktrees (`src/worktree.rs`, `src/sysroot.rs`); updates submodules (`src/lib.rs`, `src/sysroot.rs`, `src/licence.rs`, `src/release.rs`); fetches the fork from the primary's and checks it out (`src/sysroot.rs`); fast-forwards the primary (`src/sync.rs`); makes the tests' fixture repositories; runs inside rustc's bootstrap | admitted: no Rust tool does the job, gitoxide 0.85 adds, removes and prunes no worktree, updates no submodule, stages, resets and pushes nothing, checks out only a fresh clone and fetches a local path by spawning `git`; a fixture must be what `git` makes, and bootstrap runs `git` itself | M4 runs it in the guest | -| `git` for reads, a config write, a commit's paths written out, and clones and fetches over HTTPS | `rev-parse`, `show-ref`, `for-each-ref`, `rev-list`, `log`, `branch --contains`, `merge-base`, `ls-tree`, `ls-files`, `cat-file`, `config --get-regexp`, `worktree list`, `status`, `diff`, `ls-remote` and `grep`, in the build system and its tests; `config --global --add safe.directory` in the nightly's containers; `checkout -- ` through an index of its own, which writes the C++ runtime's sources out of the LLVM commit into the stored LLVM (`src/llvm.rs`); `src/sync.rs`'s fetch of `origin`; every workflow's checkout | refused: a Rust tool does it, gitoxide 0.85, which reads refs, objects, the index, config, worktrees and status, adds a value to a config file and writes it (gix-config 0.58's `File::section_mut_or_create_new`, `SectionMut::push`, `File::write_to`), walks history, diffs, and lists, fetches and clones a remote over HTTPS; `grep` is a search of the files its index names; and gitoxide's CLI 0.59 (gix 0.88) wrote the runtimes' sources of LLVM `849da7d6` into an empty directory, each path's tree through `gix rev parse`, `gix index from-tree` and `gix free index checkout-exclusive`, exit 0 each: the 18759 files `git` writes there, byte for byte and mode for mode | those are gitoxide's | +| `git` for worktrees, submodules, checkouts, fixtures and rustc's bootstrap | adds worktrees (`src/sysroot.rs`); updates submodules (`src/lib.rs`, `src/sysroot.rs`, `src/licence.rs`, `src/release.rs`); fetches the fork from the primary's and checks it out (`src/sysroot.rs`); makes the tests' fixture repositories; runs inside rustc's bootstrap | admitted: no Rust tool does the job, gitoxide 0.85 adds, removes and prunes no worktree, updates no submodule, stages, resets and pushes nothing, checks out only a fresh clone and fetches a local path by spawning `git`; a fixture must be what `git` makes, and bootstrap runs `git` itself | M4 runs it in the guest | +| `git` for reads, a config write, a commit's paths written out, and clones and fetches over HTTPS | `rev-parse`, `merge-base`, `ls-tree`, `ls-files`, `cat-file`, `config --get-regexp`, `worktree list`, `status`, `diff`, `ls-remote` and `grep`, in the build system and its tests; `config --global --add safe.directory` in the nightly's containers; `checkout -- ` through an index of its own, which writes the C++ runtime's sources out of the LLVM commit into the stored LLVM (`src/llvm.rs`); every workflow's checkout | refused: a Rust tool does it, gitoxide 0.85, which reads refs, objects, the index, config, worktrees and status, adds a value to a config file and writes it (gix-config 0.58's `File::section_mut_or_create_new`, `SectionMut::push`, `File::write_to`), walks history, diffs, and lists, fetches and clones a remote over HTTPS; `grep` is a search of the files its index names; and gitoxide's CLI 0.59 (gix 0.88) wrote the runtimes' sources of LLVM `849da7d6` into an empty directory, each path's tree through `gix rev parse`, `gix index from-tree` and `gix free index checkout-exclusive`, exit 0 each: the 18759 files `git` writes there, byte for byte and mode for mode | those are gitoxide's | | `cc`, `c++` and `ar` on a Linux host, `build-essential` on the nightly's runners | rustc links every host binary through `cc`; `cc` and `c++` compile LLVM, clang, LLD and `rustc_llvm` (`src/llvm.rs` names both to bootstrap); `ar` archives what `cc::Build` compiles | admitted: no Rust tool compiles C or C++, or takes rustc's host link | M5: no host in the loop | | the toolchain's own `clang`, `llvm-ar`, `rust-lld` and `llvm-config`, built from `ToyOSOrg/llvm-project` | rustc links every guest binary with `rust-lld`; `clang` compiles the C corpus (`tests/common/compile.rs`) and, with `llvm-ar`, doomgeneric through `cc::Build` (`src/clang.rs`); rustc's bootstrap asks `llvm-config` how to link LLVM | admitted: our fork's C++, which ToyOS can one day build and run; no Rust tool compiles C, `cc::Build` archives with an `ar`, bootstrap reads LLVM through `llvm-config`, and `CLAUDE.md` links everything with `rust-lld` | M5: no host in the loop | | `ovmf-generic`, `qemu-efi-aarch64` | the x86-64 and AArch64 UEFI firmware of the nightly's guest containers (`src/firmware.rs`), packaged by Debian apart from QEMU | admitted: QEMU's own firmware, and no Rust firmware does its job | the instrument's QEMU carries its own firmware | diff --git a/issues/build/the-build-system-does-not-compile-on-windows.md b/issues/build/the-build-system-does-not-compile-on-windows.md index 00af36ae92f..f04e1982845 100644 --- a/issues/build/the-build-system-does-not-compile-on-windows.md +++ b/issues/build/the-build-system-does-not-compile-on-windows.md @@ -6,26 +6,7 @@ opened: 2026-08-19 # The build system does not compile on Windows, and it is three subsystems rather than one call -Seven errors, in `buildlock`, `toolchain` and `worktree`. Measured 2026-09-01: - -``` -error[E0433]: cannot find `unix` in `os` src/buildlock.rs:59:14 (AsRawFd) -error[E0433]: cannot find `unix` in `os` src/toolchain.rs:927:14 (symlink) -error[E0433]: cannot find `unix` in `os` src/toolchain.rs:1813:14 (symlink) -error[E0425]: cannot find type `statvfs` in crate `libc` src/worktree.rs:305:24 -error[E0425]: cannot find function `statvfs` in crate `libc` src/worktree.rs:309:29 -error[E0599]: no method named `as_raw_fd` found for reference `&std::fs::File` - src/buildlock.rs:666:32 -error[E0599]: no method named `as_raw_fd` found for reference `&std::fs::File` - src/buildlock.rs:680:32 -``` - -`libc` itself builds for `x86_64-pc-windows-msvc`; it is `statvfs` that is not -there. Every other crate in the graph, first-party and third-party, checked -clean. The `#[cfg(unix)]` at `src/ci.rs:489` is still the only conditional -compilation in the build system. - -`src/tether.rs` is a fourth: `std::os::unix` and a pseudo-terminal per child, behind a Linux and macOS `cfg` pair with no Windows arm. +`src/tether.rs`: `std::os::unix` and a pseudo-terminal per child, behind a Linux and macOS `cfg` pair with no Windows arm. ## The judge, and it needs no Windows host and no download @@ -47,8 +28,8 @@ and `library/windows_link`, so a Windows `std` builds from the tree — a plain ## Compiling is not working, and that is why the cheap half is refused -Each of the three wants a Windows call whose semantics differ in kind from the -Unix one it replaces, and none of the three can be run by anybody here: +Each wants a Windows call whose semantics differ in kind from the +Unix one it replaces, and none can be run by anybody here: - `std::os::windows::fs::symlink_dir` needs the privilege or developer mode Windows does not grant by default, so `link_host_target` and @@ -56,7 +37,6 @@ Unix one it replaces, and none of the three can be run by anybody here: kind of broken. - `flock` is advisory and whole-file; `LockFileEx` is mandatory and byte-range. `buildlock` is what serialises the shared sysroot across every worktree. -- `statvfs` against `GetDiskFreeSpaceExW`. So a green Windows compile would say nothing about a working Windows build, and it would say it in the one subsystem whose failure mode is two checkouts @@ -69,4 +49,4 @@ eventually run inside ToyOS. `symlink` is the question in miniature: either ToyOS grows symbolic links, or the two `toolchain.rs` sites need a shape that does not need one — a copy, a directory junction, or a sysroot layout that does not require aliasing a directory at all. Deciding that is worth more than a -`#[cfg]` pair, and it decides two of the seven errors. +`#[cfg]` pair. diff --git a/issues/build/the-cxx-runtimes-scratch-removal-dies-on-a-finder-file.md b/issues/build/the-cxx-runtimes-scratch-removal-dies-on-a-finder-file.md index 78f53bac131..dd85d7ed41d 100644 --- a/issues/build/the-cxx-runtimes-scratch-removal-dies-on-a-finder-file.md +++ b/issues/build/the-cxx-runtimes-scratch-removal-dies-on-a-finder-file.md @@ -12,7 +12,7 @@ Finder writes a `.DS_Store` into a directory while it is being emptied. A …/rust/build/sysroots/d9ce291c409918f4.libcxx-x86_64: Directory not empty (os error 66)`, and afterwards the directory held only that `.DS_Store`. The sysroot was left at `.partial`, so the next build made it again. -`worktree::remove_tree` is the removal that outlasts it. +`keystore::remove` is the removal that outlasts it. **Exit**: every removal of a build product outlasts a writer that adds a file while it runs, with a test that adds one. diff --git a/issues/build/the-fork-checkout-runs-git-submodule-in-a-linked-worktree.md b/issues/build/the-fork-checkout-runs-git-submodule-in-a-linked-worktree.md index 76c4127da91..a37a9a3e683 100644 --- a/issues/build/the-fork-checkout-runs-git-submodule-in-a-linked-worktree.md +++ b/issues/build/the-fork-checkout-runs-git-submodule-in-a-linked-worktree.md @@ -10,7 +10,7 @@ opened: 2026-09-30 primary's `rust` (`git worktree add --detach`). When the primary's `library/backtrace` lacks the commit the fork pins, it runs `git submodule update --init library/backtrace` inside that linked worktree, -which `.claude/agents/implementer.md` forbids: `git submodule` in a linked +which root `CLAUDE.md` forbids: `git submodule` in a linked worktree writes `core.worktree` into shared config and breaks git in the primary checkout's `rust/`. The orchestrator measured that for `git submodule update rust` in a linked worktree of the monorepo, which set the diff --git a/issues/build/the-shipping-kernel-actuator-gate-reads-the-checkouts-own-path.md b/issues/build/the-shipping-kernel-actuator-gate-reads-the-checkouts-own-path.md index 1d6a5be3da3..3698119c715 100644 --- a/issues/build/the-shipping-kernel-actuator-gate-reads-the-checkouts-own-path.md +++ b/issues/build/the-shipping-kernel-actuator-gate-reads-the-checkouts-own-path.md @@ -11,9 +11,7 @@ carries an actuator by searching the kernel image for each declared name as a byte string. The kernel image embeds absolute source paths, so the search also matches the directory the checkout sits in. -Measured on a worktree made by the documented command, -`cargo run -- --worktree add /Users/jan/Dev/jan/toyos-heartbeat`, at -`dc38a054`: +Measured on the worktree `/Users/jan/Dev/jan/toyos-heartbeat`, at `dc38a054`: ``` $ cargo run -- --build-only diff --git a/issues/build/the-tooling-is-a-review-prompt-and-three-workflows.md b/issues/build/the-tooling-is-a-review-prompt-and-three-workflows.md index 2a6bbef793c..3240514790e 100644 --- a/issues/build/the-tooling-is-a-review-prompt-and-three-workflows.md +++ b/issues/build/the-tooling-is-a-review-prompt-and-three-workflows.md @@ -10,9 +10,17 @@ The rules a prompt can read off a branch move into `.claude/agents/reviewer.md`, and the gates that held them go. - A test is green and fast or it is deleted in the same pull request and filed. +- Each gate in `--ci host` and in the harness is kept where it sees what + reading cannot (runtime behaviour, bytes, measurements), moved into a prompt + where a reviewer can read it, or deleted. Exit: each one's verdict is applied. - The toolchain is content-addressed by the four trees that produce it, one directory per hash, never mutated; then the sysroot claim, `src/buildlock.rs` - and `src/worktree.rs` go. + and `src/keystore.rs` go. The locks are made unnecessary rather than written + down: every store is published by an atomic rename and never rewritten, and + the build system `cargo clean`s no crate target, so cargo's own lock is the + only one. + Exit: both files are gone, and two builds started together in two worktrees + are measured green. The toolchain builds cargo from the Rust fork's submodule and ships it: one cargo matching rustc. - A shared cargo `target-dir` is safe only under `-Z checksum-freshness`; diff --git a/issues/build/who-arms-auto-merge-has-two-answers.md b/issues/build/who-arms-auto-merge-has-two-answers.md deleted file mode 100644 index d0aef47a2ba..00000000000 --- a/issues/build/who-arms-auto-merge-has-two-answers.md +++ /dev/null @@ -1,16 +0,0 @@ ---- -status: open -kind: tooling -opened: 2026-09-28 ---- - -# Who arms auto-merge has two answers - -Root `CLAUDE.md` says an agent "arms auto-merge, reports, and exits". -`.claude/agents/implementer.md` says "Do not arm auto-merge … unless the brief -says so", and `.claude/agents/orchestrator.md`'s Land section has the -orchestrator run `gh pr merge --auto --merge`. - -Owner: orchestrator. - -Exit: one owner of arming auto-merge, named the same in every file that says it. diff --git a/issues/build/worktree-remove-leaves-wt-branch-behind-so-the-name-is-refused-later.md b/issues/build/worktree-remove-leaves-wt-branch-behind-so-the-name-is-refused-later.md deleted file mode 100644 index f5fecf0dbdc..00000000000 --- a/issues/build/worktree-remove-leaves-wt-branch-behind-so-the-name-is-refused-later.md +++ /dev/null @@ -1,19 +0,0 @@ ---- -status: open -kind: tooling -opened: 2026-09-29 ---- - -# `--worktree remove` leaves the local `wt/` branch behind, so the name is refused later - -`remove` unregisters and deletes the worktree directory but never deletes the -local branch (`src/worktree.rs:391`, the `eprintln!` that says so). If nothing -was ever committed on it, the branch is an ancestor of `origin/main` and -`add`'s `refuse_if_no_commit_beyond_main` (`src/worktree.rs:162`) later refuses -the same name — correctly, since it truly carries no commit beyond -`origin/main`, but the caller still has to notice and run `git branch -d -wt/` by hand before the name is usable again. - -**Exit**: `remove` deletes the local branch itself when it carries no commit -beyond `origin/main` (the same check `add` uses), so a name that was never -used becomes free again without a manual step. diff --git a/issues/kernel/process-memory-is-2-mib-pages-and-that-caps-the-process-count.md b/issues/kernel/process-memory-is-2-mib-pages-and-that-caps-the-process-count.md index 4dea86b1b3f..ea42633c6c0 100644 --- a/issues/kernel/process-memory-is-2-mib-pages-and-that-caps-the-process-count.md +++ b/issues/kernel/process-memory-is-2-mib-pages-and-that-caps-the-process-count.md @@ -52,8 +52,8 @@ live split of a user leaf. to 40 ns more. No job measures it in-guest yet: the stage owes `copy_cost`, a job beside `syscall_cost` that times `read` into a 64 KiB and a 64 MiB window, and its metal row, so that - `cargo test -- --metal copy_cost` at the stage's merge and at its base is - the A/B. + `cargo test --test toyos-build -- --metal copy_cost` at the stage's merge + and at its base is the A/B. 2. **Superframe frame allocator.** 4 KiB frames carved from 2 MiB superframes; the pin rule held per superframe, so no pinned frame is reissued and no superframe holding one is handed out whole; a watermark diff --git a/kernel/CLAUDE.md b/kernel/CLAUDE.md index 50669ba5826..fd6888d1e3e 100644 --- a/kernel/CLAUDE.md +++ b/kernel/CLAUDE.md @@ -1,15 +1,15 @@ # Kernel -The module header at the site owns its subsystem — read it before changing a module. The scheduler core is `toyos-sched/`, driven from `kernel/src/sched/`; every Ring 3 transition's machine state is `kernel/src/arch/x86_64/fpu.rs`; every syscall is `kernel/src/syscall/`, and `dispatch.rs` decodes every user pointer the ABI takes. +The module header at the site owns its subsystem — read it before changing a module. The scheduler core is `toyos-sched/`, driven from `kernel/src/sched/`; every user transition's machine state is `arch/*/fpu.rs`, with AArch64's `entry.rs` and `trap.rs`; every syscall is `kernel/src/syscall/`, and `dispatch.rs` decodes every user pointer the ABI takes. ## Caveats that bite every agent - **Nothing on the idle path touches a filesystem.** No gate holds this. - **Anything added to the idle loop is an audio change** — housekeeping runs before `pass()`, so a woken CPU is late by what it costs; and on a machine with nothing to run the idle loop does not run, so a diagnostic there reports nothing exactly when it is needed. - **The idle loop may not take a global lock unconditionally** — the crash report reads the process table through a `try_lock` it must never block on. `sched/reap_gate.rs` is the pattern: a relaxed-load gate in front of the lock. -- **Every Ring 0 entry clears the direction flag** — no hardware gate does it, and `memmove` sets it across `rep` operations with interrupts on; `arch::entry::ring3_naked_asm` is where it lives. +- **Every x86 Ring 0 entry clears the direction flag** — no hardware gate does it, and `memmove` sets it across `rep` operations with interrupts on; `arch::entry::ring3_naked_asm` is where it lives. - **`BackendGuard` masks interrupts for its whole life**, so anything written under it is an interrupt latency; a new holder bounds itself as the console drain and the userland `write` flush do. -- **No disk wait in this kernel can park** — at the moment a transfer is waited for, the CPU is four ticket spinlocks deep, each disabling preemption. +- **No disk wait in this kernel can park** — a transfer is waited for under ticket spinlocks, each disabling preemption. - **`crate::log!` may not be called inside `with_cpu`'s exclusive region unless a `panic!` follows it** — the log's readiness path re-enters `driver::pass` and wedges the machine. - **`drain_irqs` is the drivers' engine and nothing on it may wait** — a blocking call there empties the audio pipeline on every plug. - **A syscall that can block resolves its handle and clones the object out before it blocks** — a `with_object`/`with_process_data` guard held across a park is a runtime panic no compile check catches; the `SYS_FSYNC` arm in `syscall/dispatch.rs` is the pattern. diff --git a/src/CLAUDE.md b/src/CLAUDE.md index 94c51792107..e3f19a0ae2f 100644 --- a/src/CLAUDE.md +++ b/src/CLAUDE.md @@ -2,36 +2,22 @@ Loads when you read a file under `src/` — the root cargo project, package name `toyos-build`. Root `CLAUDE.md` has `cargo run` and `cargo test`; the `[profile.toyos]` rule is this crate's to enforce (`src/build.rs`). -## Boot modes - -- `cargo run -- --gop` boots a UEFI GOP display (`-vga std`) instead of virtio-gpu: the config where the on-screen panic console renders. -- `cargo run -- --diag-boot --build-only` builds `target/bootable-diag.img`, the diagnostic boot for a machine with no serial port: its config declares no `devices`, so the kernel's log stays readable off the panel. Same kernel and bootloader binaries as the ordinary build. **A flashable artifact is built from a committed tree** — `cargo` builds the working tree. -- `cargo run -- --console-boot --build-only` builds `target/bootable-console.img` — `/system/bin/console`, the shell on the raw framebuffer, for asking a machine questions instead of reflashing it. -- `cargo run -- --kernel-param ` (repeatable, any boot mode) arms an actuator at boot; every actuator lives in one kernel built with `boot-actuators` (`kernel/src/actuator.rs`), so it decides no kernel build. `--kernel-feature ` *is* a build; it accepts any feature `kernel/Cargo.toml` declares, and that set is closed by `cargo test --lib`. An undeclared name is refused by name before any lock. -- `cargo run -- --metal-sim` boots the T14's hardware shape (GOP + NVMe + xHCI + i8042, no virtio, no USB HID, with a 16550); `--mute` takes the serial away, the T14's literal shape. - -## Other entry points +## Entry points - **`.github/qemu-version` is the QEMU every guest is measured with, declared once**; every guest job's first step (`src/ci.rs`'s `instrument`) reds on a disagreement. - `system.toml` defines which programs to build and the init sequence. -- **A workflow step runs `cargo run -- --ci ` and nothing else; logic in YAML is a defect.** ## The host's locks -- **Sysroots are content-addressed** (`src/sysroot.rs`): one per key — the identity (`src/identity.rs`, so a comment is no change) of `toyos-abi/src`, `toyos/src`, `userland/libc/src` and their manifests, the std fork's `rust/library/` and `rust/src/bootstrap/`, and the compiler — at `rust/build/sysroots//`, made by whichever worktree first needs it. Every build compiles against its own key's, so two worktrees with different ABIs never refuse or wait for each other; the only shared step is the primary's compiler, which a sysroot build reads under the global lock in shared mode. `--worktree remove` sweeps the keys no worktree records. -- **The std fork is built per worktree, and nothing but the primary's own sync moves the primary's `rust/`.** A linked worktree's `rust/` becomes, on its first build, a git worktree of the primary's fork repository at the commit its tree pins — that is where the fork is edited, committed and pinned. A worktree whose fork `compiler/` differs from the one the primary built builds its own compiler, keyed by that source and placed beside the primary's without touching it. If that checkout later falls behind the commit its tree pins (a merge moved the pin), the build moves the checkout to it itself, fetching from the primary's repository first if it holds the commit, unless the checkout has local changes, which it refuses to move out from under. -- `src/buildlock.rs` serialises the stateful phases in two scopes: `Global` (the primary's compiler and the rustup link — one directory in `.git/`, shared by every worktree) and `Worktree` (the crate-target cleans, and this worktree's std build). Only `./x.py` typed by hand in `rust/` escapes it. -- **Never kill a build that has taken the global lock** — the kill removes the shell wrappers, not the bootstrap, which inherits the file descriptor and runs on regardless; a toolchain rebuild interrupted or unobserved this way can leave `stage2/bin` without a `cargo`. -- **The lock order is a constraint at every acquirer** — `src/buildlock.rs`'s header states it. Every blocking lock repeats itself every 30 s — a queue is never silence. +`src/buildlock.rs`'s header states them and the order every acquirer takes them in, until the build track that holds the machinery cut deletes them. ## Worktrees - Everything under a worktree — targets, images, `.build-locks/`, its fork checkout — is its own; the object stores, the compiler and the rustup link are the primary checkout's, and ownership is derived from `git rev-parse --git-common-dir`, never recorded. -- **A linked worktree's `main` ref is only as current as the primary's last `--sync`: anything asking "does this branch differ from main" diffs against `origin/main`.** -- **Type-checking a std edit without building a sysroot**: point `__CARGO_TESTS_ONLY_SRC_ROOT` at a tree holding an APFS clone of `rust/library` (`cp -Rc`), a workspace `Cargo.toml` naming `library/std`, and symlinks to `toyos-abi`/`toyos`; then `CARGO_TARGET_DIR= cargo +toyos build -Z build-std=std,panic_abort --target x86_64-unknown-toyos --offline`. Delete `/**/.fingerprint/std-*` between runs — cargo does not re-fingerprint std under `-Zbuild-std`. +- **A linked worktree's `main` ref is only as current as the primary's last `git pull`: anything asking "does this branch differ from main" diffs against `origin/main`.** +- **Type-checking a std edit without building a sysroot**: point `__CARGO_TESTS_ONLY_SRC_ROOT` at a tree holding a copy of `rust/library`, a workspace `Cargo.toml` naming `library/std`, and symlinks to `toyos-abi`/`toyos`; then `CARGO_TARGET_DIR= cargo +toyos build -Z build-std=std,panic_abort --target x86_64-unknown-toyos --offline`. Delete `/**/.fingerprint/std-*` between runs — cargo does not re-fingerprint std under `-Zbuild-std`. ## Caveats that bite every agent -- **Every CI lane is GitHub-hosted and no workflow may name a self-hosted label** — a `runs-on:` naming one queues until it times out rather than failing, so `src/ci.rs`'s `workflows_run_against_main_on_hosted_runners` refuses it; a measurement owed on hardware goes to the metal loop, not to a runner. - **A workflow job that runs in a container adds `safe.directory` itself** — `actions/checkout` sets it into a temporary global config it discards when its step ends, so the first git command a container step runs after checkout dies on a dubiously-owned repository. - **A `stage1-std//dist/deps` temp-dir error means a concurrent build**, never a broken checkout; never repair or force-rebuild the toolchain. diff --git a/src/build.rs b/src/build.rs index c5ddf3a228a..3549fdf2fdd 100644 --- a/src/build.rs +++ b/src/build.rs @@ -16,7 +16,7 @@ use crate::buildlock; use crate::flags; use crate::hostws; use crate::image; -use crate::sysroot::{Identity, Stale}; +use crate::sysroot::{Identity, Stale, Sysroot}; use crate::toolchain; thread_local! { @@ -200,7 +200,7 @@ fn clean(root: &Path, crate_dir: &Path, kind: Clean, stale: &Stale, identity: &I let remove = |dirs: &mut dyn Iterator| { for dir in dirs.filter(|dir| dir.exists()) { eprintln!("external deps changed: cleaning {}", dir.display()); - crate::worktree::remove_tree(&dir); + crate::keystore::remove(&dir); } }; match (stale, kind) { @@ -374,15 +374,10 @@ pub const PROFILE: &str = "toyos"; /// What every guest `cargo` and `rustc` here runs with: the toolchain directory /// of the sysroot this checkout's sources name, as `RUSTUP_TOOLCHAIN`, which /// carries the linker too. Never the `toyos` rustup name, which is the primary's. -#[derive(Clone)] struct GuestEnv { - toolchain: PathBuf, - /// Whether that sysroot's compiler is the primary's, the one the hosted - /// rustc is built from (`src/compiler.rs`). - primary_compiler: bool, - /// What its crates compile against beside their sources - /// (`sysroot::Identity`). - identity: Identity, + /// Owned, so the sysroot is held in use for as long as anything here runs + /// against it. + sysroot: Sysroot, /// The public key the loader and `/system/bin/update` embed /// (`signing::KEY_ENV`): every guest build carries it, so no crate that /// names it can be built without it. @@ -392,11 +387,9 @@ struct GuestEnv { } impl GuestEnv { - fn new(sysroot: &crate::sysroot::Sysroot) -> Self { + fn new(sysroot: Sysroot) -> Self { Self { - toolchain: sysroot.dir.clone(), - primary_compiler: sysroot.primary_compiler, - identity: sysroot.identity.clone(), + sysroot, image_key: crate::signing::key().public_hex(), floor_scope: crate::signing::key().floor_scope().word(), } @@ -419,7 +412,7 @@ fn cargo_build( let mut cmd = Command::new("cargo"); cmd.args(&args) .current_dir(crate_dir) - .env("RUSTUP_TOOLCHAIN", &env.toolchain) + .env("RUSTUP_TOOLCHAIN", env.sysroot.dir()) .env_remove("RUSTFLAGS") .env(crate::signing::KEY_ENV, &env.image_key) .env(crate::signing::FLOOR_ENV, env.floor_scope) @@ -544,7 +537,7 @@ fn assert_kernel_is_softfloat(env: &GuestEnv, arch: Arch) { } let out = Command::new("rustc") .args(["--print", "cfg", "--target", arch.kernel()]) - .env("RUSTUP_TOOLCHAIN", &env.toolchain) + .env("RUSTUP_TOOLCHAIN", env.sysroot.dir()) .env_remove("RUSTFLAGS") .env_remove("RUSTC") .output() @@ -711,12 +704,12 @@ fn build_and_assemble( arch.name() ); assert!( - env.primary_compiler, + env.sysroot.primary_compiler, "hosted-rustc ships the primary checkout's hosted compiler, and this worktree builds with \ a compiler of its own (src/compiler.rs): the image would carry a rustc that is not the \ one its programs were built with" ); - collect_hosted_rustc(root, &env.toolchain, &mut root_files); + collect_hosted_rustc(root, env.sysroot.dir(), &mut root_files); } if !config.assets.is_empty() { @@ -813,7 +806,7 @@ fn build_programs( // Every userland crate that compiles C compiles it with the toolchain's // clang against libc's C sysroot. - let cc_env = crate::clang::CSysroot::of(&env.toolchain, arch).cc_env(); + let cc_env = crate::clang::CSysroot::of(env.sysroot.dir(), arch).cc_env(); let cc_env: Vec<(&str, &str)> = cc_env.iter().map(|(k, v)| (k.as_str(), v.as_str())).collect(); // Build and read under one hold, exactly as `build_toyos_bins` does and for @@ -1720,12 +1713,10 @@ fn shipped_parts(root: &Path, boot: &Boot, plan: &Plan) -> (Vec, Vec, Ve // Held until the last staged artifact has been read back, so no clean of // this worktree's crate targets can land inside this build. let mut lock = buildlock::shared(root, "build"); - let sysroot = toolchain::ensure(root, &mut lock); - - let env = GuestEnv::new(&sysroot); + let env = GuestEnv::new(toolchain::ensure(root, &mut lock)); let config = parse_config(&boot.config); - invalidate_stale(root, &mut lock, &env.identity, &config_targets(root, &config)); + invalidate_stale(root, &mut lock, &env.sysroot.identity, &config_targets(root, &config)); // Same lock-and-stage as `build_test_image`: `cargo run --build-only` and // `cargo test` share these paths, so this races the harness too. The kernel @@ -1733,21 +1724,18 @@ fn shipped_parts(root: &Path, boot: &Boot, plan: &Plan) -> (Vec, Vec, Ve // path uses, so neither can grow an assertion the other lacks. let (kernel_bytes, bl_art) = { let _artifact = buildlock::artifact(root); - let kernel_handle = { - let root = root.to_path_buf(); - let env = env.clone(); - let features = kernel_features.clone(); - std::thread::spawn(move || { + std::thread::scope(|threads| { + let kernel = threads.spawn(|| { let mut extra = Vec::new(); - if !features.is_empty() { + if !kernel_features.is_empty() { extra.push("--features"); - extra.push(&features); + extra.push(&kernel_features); } cargo_build(&root.join("kernel"), arch.kernel(), &extra, &env, &[], false); - }) - }; - cargo_build(&root.join("bootloader"), arch.loader(), &[], &env, &[], false); - kernel_handle.join().expect("kernel build thread panicked"); + }); + cargo_build(&root.join("bootloader"), arch.loader(), &[], &env, &[], false); + kernel.join().expect("kernel build thread panicked"); + }); ( stage_and_certify_kernel(root, &kernel_features, &env, arch), stage_loader(root, arch, &env), @@ -1942,10 +1930,9 @@ pub fn build_test_parts( // back after the userland build, and a clean landing in between is the // same defect as one landing mid-compile. let mut lock = buildlock::shared(root, "test image"); - let sysroot = crate::toolchain::ensure(root, &mut lock); - let env = GuestEnv::new(&sysroot); + let env = GuestEnv::new(toolchain::ensure(root, &mut lock)); - invalidate_stale(root, &mut lock, &env.identity, &config_targets(root, &config)); + invalidate_stale(root, &mut lock, &env.sysroot.identity, &config_targets(root, &config)); // Build and stage under one lock, released before `build_and_assemble`. // Releasing it there is deliberate and required: that build takes its own @@ -2150,9 +2137,8 @@ struct TestBuild { impl TestBuild { fn begin(root: &Path, arch: Arch, what: &str, stale_targets: &[(PathBuf, Clean)]) -> Self { let mut lock = buildlock::shared(root, what); - let sysroot = crate::toolchain::ensure(root, &mut lock); - let env = GuestEnv::new(&sysroot); - invalidate_stale(root, &mut lock, &env.identity, stale_targets); + let env = GuestEnv::new(toolchain::ensure(root, &mut lock)); + invalidate_stale(root, &mut lock, &env.sysroot.identity, stale_targets); let artifact = buildlock::artifact(root); TestBuild { target: arch.userland(), env, _lock: lock, _artifact: artifact } } diff --git a/src/ci.rs b/src/ci.rs index cf6beba9c0a..18b8ac83522 100644 --- a/src/ci.rs +++ b/src/ci.rs @@ -28,8 +28,9 @@ use std::path::{Path, PathBuf}; use std::process::Command; use crate::arch::{Accel, Arch}; +use crate::sysroot::git_out; use crate::userlandhost::{Host, Os, Program}; -use crate::{flags, release, sdkversion, sync}; +use crate::{flags, release, sdkversion}; const USAGE: &str = "cargo run -- --ci , where is one of: host every host test: the build system, the harness's own checks, @@ -837,8 +838,8 @@ fn publish(root: &Path) -> Result { .into() ); } - let tip = sync::git(root, &["ls-remote", "origin", "refs/heads/main"])?; - at_tip(&tip, &sync::git(root, &["rev-parse", "HEAD"])?)?; + let tip = git_out(root, &["ls-remote", "origin", "refs/heads/main"]); + at_tip(&tip, git_out(root, &["rev-parse", "HEAD"]).trim())?; let plan = sdkversion::plan(root)?; sdkversion::write_published_manifests(root, &plan)?; let mut said = Vec::new(); diff --git a/src/compiler.rs b/src/compiler.rs index 5f2e9ad511c..f10df41ce8a 100644 --- a/src/compiler.rs +++ b/src/compiler.rs @@ -34,7 +34,7 @@ //! directory is written; then the LLVM key's, held shared while it is linked. //! //! A compiler no worktree names any more is removed by `keystore::sweep`, which -//! `--worktree remove` and every placement run: each build records the key it used in its +//! every placement runs: each build records the key it used in its //! worktree's `target/`, and a key no registered worktree records, that nobody //! is making or using, goes. //! diff --git a/src/flags.rs b/src/flags.rs index 49d15acc983..d9173d79433 100644 --- a/src/flags.rs +++ b/src/flags.rs @@ -26,7 +26,7 @@ pub(crate) enum Value { /// --kernel-param slow` arms two actuators, and every value is read. Each, /// Every word after it — the flag names a subcommand that owns the rest of - /// the line, as `--worktree add ` does. + /// the line, as `--ci ` does. Rest, } @@ -50,7 +50,6 @@ pub(crate) use declare_flags; declare_flags!(pub CARGO_RUN = { pub HELP = "--help", None; - pub SYNC = "--sync", None; pub CI = "--ci", Rest; pub CLIPPY = "--clippy", None; pub DEBUG = "--debug", None; @@ -69,7 +68,6 @@ declare_flags!(pub CARGO_RUN = { pub REGEN_FONT = "--regen-font", None; pub REGEN_WALLPAPER = "--regen-wallpaper", None; pub REGEN_SOUNDFONT = "--regen-soundfont", Next; - pub WORKTREE = "--worktree", Rest; /// Mint the owner's image-signing key where `signing::owner_key_path` /// says, refusing to replace one. pub SIGNING_KEY_NEW = "--signing-key-new", None; @@ -163,7 +161,7 @@ impl Walk<'_> { /// A flag whose value no reader of this line would get — every shape that /// reaches a reader as a silent default, and the whole of what either /// command line asks. A flag with nothing after it — the end of the line, - /// an empty `--smp=`, or a `--worktree` owning no words — answers `None` to + /// an empty `--smp=`, or a `--ci` owning no words — answers `None` to /// [`Vocabulary::value`] and `&[]` to [`Vocabulary::rest`]; a flag written /// twice has every use but one dropped; and an inline value is dropped by /// exactly two readers, `Value::None` having none and `rest` taking only the @@ -330,8 +328,8 @@ mod tests { fn an_inline_value_is_refused_exactly_where_it_would_be_dropped() { let debug = refusal(&["--debug=1"]); assert!(debug.contains("--debug takes no value"), "{debug}"); - let worktree = refusal(&["--worktree=add"]); - assert!(worktree.contains("--worktree "), "{worktree}"); + let ci = refusal(&["--ci=host"]); + assert!(ci.contains("--ci "), "{ci}"); let line = argv(&["--smp=4", "--kernel-param=slow"]); assert!(matches!(check(&line), Outcome::Proceed)); assert_eq!(CARGO_RUN.value(&line, &SMP), Some("4")); @@ -380,7 +378,7 @@ mod tests { for words in [ vec!["--kernel-param", "--help"], vec!["--boot-config", "--help"], - vec!["--worktree", "add", "--help"], + vec!["--ci", "host", "--help"], vec!["--boot-config", "diag", "--build-only"], ] { assert!(matches!(checked(&words), Outcome::Proceed), "{words:?} must proceed"); @@ -388,8 +386,8 @@ mod tests { let line = argv(&["--kernel-param", "--help"]); assert_eq!(CARGO_RUN.values(&line, &KERNEL_PARAM), ["--help"]); assert!(!CARGO_RUN.present(&line, &HELP), "the actuator's name is not the flag"); - let worktree = argv(&["--worktree", "add", "/tmp/wt"]); - assert_eq!(CARGO_RUN.rest(&worktree, &WORKTREE), ["add", "/tmp/wt"]); + let ci = argv(&["--ci", "guest", "--help"]); + assert_eq!(CARGO_RUN.rest(&ci, &CI), ["guest", "--help"]); assert_eq!(CARGO_RUN.value(&argv(&["--boot-config", "diag"]), &BOOT_CONFIG), Some("diag")); assert_eq!(CARGO_RUN.value(&argv(&["--build-only"]), &BOOT_CONFIG), None); } diff --git a/src/hostws.rs b/src/hostws.rs index 3eff316f364..dae74c87462 100644 --- a/src/hostws.rs +++ b/src/hostws.rs @@ -385,7 +385,7 @@ mod tests { } for excluded in excluded(&root) { // Not "holds a Cargo.toml": a linked worktree's `rust/` is the empty - // stub `git worktree add` leaves (src/CLAUDE.md), and excluding it + // stub `git worktree add` leaves, and excluding it // is right in both checkouts. assert!( root.join(&excluded).is_dir(), diff --git a/src/keystore.rs b/src/keystore.rs index c35e97f3dc1..1c9fb0d7271 100644 --- a/src/keystore.rs +++ b/src/keystore.rs @@ -216,12 +216,38 @@ fn retire_by(path: &Path, remove: impl Fn(&Path)) { } } -/// Remove the directory `path` and all it holds, read-only or not. +/// Remove `path`, a file or a directory and all it holds, read-only or not; +/// that nothing is there is not an error. +/// +/// A writer on this host — the leftovers are `.DS_Store` files — can put a file +/// into a directory while it is being emptied, so a plain recursive delete finds a directory it has just emptied not empty and +/// stops halfway. The removal runs again over what is left, at most [`PASSES`] +/// times; a tree still refusing after that has a writer this cannot outrun, and +/// the panic says so. pub fn remove(path: &Path) { - writable(path); - fs::remove_dir_all(path).unwrap_or_else(|e| panic!("remove {}: {e}", path.display())); + for pass in 1..=PASSES { + let removed = match fs::symlink_metadata(path) { + Ok(meta) if meta.is_dir() => { + writable(path); + fs::remove_dir_all(path) + } + Ok(_) => fs::remove_file(path), + Err(e) => Err(e), + }; + match removed { + Ok(()) => return, + Err(e) if e.kind() == ErrorKind::NotFound => return, + Err(e) if e.kind() == ErrorKind::DirectoryNotEmpty && pass < PASSES => { + eprintln!("{} gained files while it was removed ({e}); removing again", path.display()); + } + Err(e) => panic!("remove {}: {e}, after {pass} pass(es)", path.display()), + } + } } +/// How many times [`remove`] runs over a tree that keeps refusing. +const PASSES: usize = 10; + /// Give `dir` and every directory under it back its owner's write permission. pub(crate) fn writable(dir: &Path) { let meta = fs::metadata(dir).unwrap_or_else(|e| panic!("stat {}: {e}", dir.display())); diff --git a/src/lib.rs b/src/lib.rs index 06e45ea6a96..02ecc5ae450 100644 --- a/src/lib.rs +++ b/src/lib.rs @@ -41,14 +41,12 @@ pub mod soundfont; /// build system at all. #[cfg(test)] pub mod sourcegate; -pub mod sync; pub mod sysroot; pub mod testargs; pub mod tether; pub mod toolchain; pub mod userlandhost; pub mod wallpaper; -pub mod worktree; use std::path::{Path, PathBuf}; use std::process::Command; diff --git a/src/llvm.rs b/src/llvm.rs index 6b505d942c9..9eb587b6acb 100644 --- a/src/llvm.rs +++ b/src/llvm.rs @@ -30,7 +30,7 @@ //! writes, `build/toyos-llvm/`, which is removed once the LLVM is placed. //! //! An LLVM no worktree names any more is removed by `keystore::sweep`, which -//! `--worktree remove` and every placement run: each worktree records the key of +//! every placement runs: each worktree records the key of //! the LLVM its compiler links, and a key no registered worktree records, that //! nobody is making or using, goes. @@ -257,9 +257,7 @@ fn place(fork: &Path, key: &Key, dir: &Path, build: &impl Fn(&Path) -> PathBuf) let built = build(fork); let host = host_triple(); let partial = dir.with_extension("partial"); - if partial.exists() { - keystore::remove(&partial); - } + keystore::remove(&partial); for part in KEPT { clone_tree(&built.join(&host).join("llvm").join(part), &partial.join(part)); } diff --git a/src/main.rs b/src/main.rs index ad2290fc4ad..3b652ca01de 100644 --- a/src/main.rs +++ b/src/main.rs @@ -91,13 +91,8 @@ fn main() { } let asked = |flag: &flags::Flag| CARGO_RUN.present(&args, flag); - // **Before `check_prerequisites`**, because it builds nothing. - if asked(&flags::SYNC) { - toyos_build::sync::dispatch_sync(&root); - return; - } - // Every CI job. Here for the same reason: the host job's runner has no QEMU, - // and a guest job names its own instrument rather than being noted at. + // Every CI job, before `check_prerequisites`: the host job's runner has no + // QEMU, and a guest job names its own instrument rather than being noted at. if asked(&flags::CI) { toyos_build::ci::dispatch(&root, &args); return; @@ -201,11 +196,6 @@ fn main() { return; } - if asked(&flags::WORKTREE) { - toyos_build::worktree::dispatch(&root, &args); - return; - } - // Only where the submodules belong. In a linked worktree `rust/` is an empty // stub and initialising it clones the whole rust history again, into a git // directory of its own that shares no objects with the one beside it. diff --git a/src/release.rs b/src/release.rs index eb567406976..7e79d113743 100644 --- a/src/release.rs +++ b/src/release.rs @@ -205,7 +205,7 @@ pub fn ensure_published(root: &Path) -> Result { )); } let tmp = TempDir::new("toolchain-publish"); - let manifest = manifest(root, &tag)?; + let manifest = manifest(root, &tag); let notes = notes(root, &tag, &manifest)?; fs::write(tmp.join("TOOLCHAIN"), &manifest).map_err(|e| e.to_string())?; fs::write(tmp.join("notes.md"), ¬es).map_err(|e| e.to_string())?; @@ -350,15 +350,15 @@ fn glibc_named(bytes: &[u8]) -> (u32, u32) { /// `TOOLCHAIN`: the pin a consumer writes down, and what it gets. Inside the /// tarball, and the alias release's own asset. -fn manifest(root: &Path, tag: &str) -> Result { - let head = |rev: &str| crate::sync::git(root, &["rev-parse", rev]); - let toyos = std::env::var("GITHUB_SHA").or_else(|_| head("HEAD"))?; - Ok(format!( +fn manifest(root: &Path, tag: &str) -> String { + let head = |rev: &str| crate::sysroot::git_out(root, &["rev-parse", rev]).trim().to_string(); + let toyos = std::env::var("GITHUB_SHA").unwrap_or_else(|_| head("HEAD")); + format!( "toolchain {tag}\ntoyos {toyos}\nrust {}\nhost {HOST}\nglibc {}.{}\n", - head("HEAD:rust")?, + head("HEAD:rust"), GLIBC_FLOOR.0, GLIBC_FLOOR.1 - )) + ) } /// The release notes: how to install it, what glibc it needs. diff --git a/src/sdkversion.rs b/src/sdkversion.rs index 4fc47ffcb1d..c117aa8e225 100644 --- a/src/sdkversion.rs +++ b/src/sdkversion.rs @@ -46,7 +46,7 @@ pub fn plan(root: &Path) -> Result, String> { /// A crate's `HEAD` tree and its manifest. fn source(root: &Path, krate: &Crate) -> Result<(String, String), String> { - let tree = crate::sync::git(root, &["rev-parse", &format!("HEAD:{}", krate.dir)])?; + let tree = crate::sysroot::git_out(root, &["rev-parse", &format!("HEAD:{}", krate.dir)]).trim().to_string(); Ok((tree, manifest(root, krate)?)) } diff --git a/src/sync.rs b/src/sync.rs deleted file mode 100644 index d1f26653f6d..00000000000 --- a/src/sync.rs +++ /dev/null @@ -1,263 +0,0 @@ -//! `cargo run -- --sync`: this machine's `main` onto `origin/main`. -//! -//! Nothing here rewrites history and nothing pushes `main`. - -use std::path::{Path, PathBuf}; -use std::process::Command; - -pub fn dispatch_sync(root: &Path) { - match sync(root) { - Ok(line) => println!("[sync] {line}"), - Err(refusal) => { - eprintln!("{refusal}"); - std::process::exit(1); - } - } -} - -/// `git fetch origin`, then this machine's `main` onto `origin/main`. -/// -/// **`origin/main` is the truth and the local one is a cache.** Without this -/// the primary's tree — which owns `rust/`, the sysroot and the witness every -/// worktree compares against — silently falls behind whatever GitHub merged. -fn sync(root: &Path) -> Result { - let primary = crate::primary_checkout(root); - - git(root, &["fetch", "--quiet", "origin", "main"]) - .map_err(|e| format!("{e}\n[sync] `git fetch origin main` failed, so nothing below could \ - be judged against what GitHub has."))?; - - // The fast-forward runs on whatever branch the primary has out, so the - // question is about that checkout and not about which checkout is asking. - let on = git(&primary, &["rev-parse", "--abbrev-ref", "HEAD"])?; - if on != "main" { - return Ok(format!( - "fetched origin; {} is on {on}, so this host's main was left where it is", - primary.display() - )); - } - - if canonical(root) != canonical(&primary) { - let dirty = git(&primary, &["status", "--porcelain"])?; - if !dirty.is_empty() { - return Ok(format!( - "fetched origin; {} has uncommitted work in it, so this host's main was left \ - where it is", - primary.display() - )); - } - } - - let ahead = git(&primary, &["rev-list", "--count", "origin/main..main"])?; - if ahead.trim() != "0" { - return Err(stranded(&primary)); - } - let behind = git(&primary, &["rev-list", "--count", "main..origin/main"])?; - if behind.trim() == "0" { - let at = git(&primary, &["rev-parse", "--short", "main"])?; - return Ok(format!( - "fetched origin; this host's main is current at {at}{}", - reclaimable(root) - )); - } - let said = match fast_forward(&primary)? { - Some((before, after, commits)) => format!("{before} -> {after} ({commits} commit(s))"), - None => format!("already at {}", git(&primary, &["rev-parse", "--short", "main"])?), - }; - Ok(format!("fetched origin; this host's main {said}{}", reclaimable(root))) -} - -/// The move this call made, or `None` when a concurrent `--sync` had already -/// made it. A concurrent one holds git's index or ref lock, and git refuses -/// this one. -fn fast_forward(primary: &Path) -> Result, String> { - let before = git(primary, &["rev-parse", "main"])?; - git(primary, &["merge", "--ff-only", "origin/main"])?; - let after = git(primary, &["rev-parse", "main"])?; - if before == after { - return Ok(None); - } - let commits = git(primary, &["rev-list", "--count", &format!("{before}..{after}")])?; - let short = |sha: &str| git(primary, &["rev-parse", "--short", sha]); - Ok(Some((short(&before)?, short(&after)?, commits))) -} - -/// What this host could give back, said where it becomes true. -/// -/// A worktree whose branch has landed has no reason to hold its build caches, -/// and `--sync` runs at exactly the moment that becomes true of one. -fn reclaimable(root: &Path) -> String { - crate::worktree::reclaim_line(&crate::worktree::survey(root, false)) - .map_or_else(String::new, |line| format!("\n[sync] {line}")) -} - -/// This host's `main` has commits GitHub does not, so it is not a cache of -/// `origin/main` any more and nothing can fast-forward it. -fn stranded(primary: &Path) -> String { - let extra = git(primary, &["log", "--oneline", "origin/main..main"]).unwrap_or_else(|e| e); - let holders = git(primary, &["branch", "--contains", "main", "--list", "wt/*"]) - .unwrap_or_else(|e| e); - format!( - "[sync] this host's main carries commits origin/main has not got, so it cannot be \ - fast-forwarded and it is no longer a copy of what GitHub has:\n{extra}\n\ - [sync] branches that already contain all of them:\n{}\n\ - [sync] If one of those holds every commit above, nothing is lost — open a pull request \ - for it and put this host's main back with \ - `git -C {} reset --hard origin/main`. If none does, do not reset anything: work out \ - where those commits live first.", - if holders.trim().is_empty() { "[sync] none".to_string() } else { holders }, - primary.display(), - ) -} - -fn canonical(path: &Path) -> PathBuf { - std::fs::canonicalize(path).unwrap_or_else(|e| panic!("canonicalise {}: {e}", path.display())) -} - -/// `Err` carries what git printed, both streams, because a refusal that hides -/// git's own message makes the agent run the command again by hand to see it. -pub(crate) fn git(dir: &Path, args: &[&str]) -> Result { - let out = Command::new("git") - .args(args) - .current_dir(dir) - .output() - .unwrap_or_else(|e| panic!("run git in {}: {e}", dir.display())); - let stdout = String::from_utf8_lossy(&out.stdout).trim_end().to_string(); - if out.status.success() { - return Ok(stdout); - } - let stderr = String::from_utf8_lossy(&out.stderr).trim_end().to_string(); - Err(format!("git {} (in {})\n{stdout}\n{stderr}", args.join(" "), dir.display())) -} - -#[cfg(test)] -mod tests { - use super::*; - use crate::gitfixture::{commit, configure, repo, sh}; - use std::fs; - - /// Someone else lands, so `origin/main` is ahead of every clone. - fn land_elsewhere(dir: &Path, origin: &Path) { - let theirs = dir.join("theirs"); - sh(dir, &["clone", "-q", origin.to_str().unwrap(), theirs.to_str().unwrap()]); - configure(&theirs); - commit(&theirs, "h", "theirs\n", "meanwhile"); - sh(&theirs, &["push", "-q", "origin", "main"]); - } - - /// A fixture clone is its own primary, the shape the branch question used - /// to be skipped for: the fast-forward ran on whatever branch was out, git - /// refused it, and `sync` reported lost commits about an ancestor. - #[test] - fn a_primary_on_a_branch_is_left_where_it_is() { - let (dir, origin, wt) = repo("sync-on-a-branch"); - commit(&wt, "g", "mine\n", "work"); - land_elsewhere(&dir, &origin); - - let said = sync(&wt).expect("a primary on a branch is reported, never refused"); - assert!(said.contains("is on wt, so this host's main was left where it is"), "{said}"); - assert!(!said.contains("carries commits origin/main has not got"), "{said}"); - // Left where it is, and main still strictly behind. - assert_eq!(git(&wt, &["rev-parse", "--abbrev-ref", "HEAD"]).unwrap(), "wt"); - assert_eq!(git(&wt, &["rev-list", "--count", "main..origin/main"]).unwrap(), "1"); - } - - /// `--sync` takes no lock of its own: a second fast-forward racing the - /// first is git's to refuse, and the refusal is git's words, never the - /// stranded-commits story about a `main` that is merely behind. - #[test] - fn a_fast_forward_git_refuses_is_reported_as_git_said_it() { - let (dir, origin, wt) = repo("sync-racing"); - sh(&wt, &["switch", "-q", "main"]); - land_elsewhere(&dir, &origin); - let before = git(&wt, &["rev-parse", "main"]).unwrap(); - - let lock = wt.join(".git/index.lock"); - fs::write(&lock, "").unwrap(); - let refusal = sync(&wt).expect_err("a held index lock must refuse the fast-forward"); - fs::remove_file(&lock).unwrap(); - assert!(!refusal.contains("carries commits origin/main has not got"), "{refusal}"); - assert_eq!(git(&wt, &["rev-parse", "main"]).unwrap(), before, "main moved under a lock"); - - let said = sync(&wt).expect("with the lock gone the fast-forward runs"); - assert!(said.contains("(1 commit(s))"), "{said}"); - assert_eq!( - git(&wt, &["rev-parse", "main"]).unwrap(), - git(&wt, &["rev-parse", "origin/main"]).unwrap() - ); - } - - /// A primary that is this checkout skips the dirty question, so its dirt - /// reaches the fast-forward: git's refusal is reported, not stranded commits. - #[test] - fn a_dirty_primary_that_is_this_checkout_gets_gits_refusal() { - let (dir, origin, wt) = repo("sync-dirty-self"); - sh(&wt, &["switch", "-q", "main"]); - land_elsewhere(&dir, &origin); - fs::write(wt.join("h"), "untracked\n").unwrap(); - let before = git(&wt, &["rev-parse", "main"]).unwrap(); - - let refusal = sync(&wt).expect_err("an untracked file in the way must refuse"); - assert_eq!(git(&wt, &["rev-parse", "main"]).unwrap(), before, "main moved past the file"); - assert_eq!(fs::read_to_string(wt.join("h")).unwrap(), "untracked\n"); - assert!(!refusal.contains("carries commits origin/main has not got"), "{refusal}"); - } - - /// `main` ahead of `origin/main` is refused with what is stranded, before - /// any fast-forward is tried. - #[test] - fn a_main_github_has_not_got_is_refused_as_stranded() { - let (dir, origin, wt) = repo("sync-stranded"); - sh(&wt, &["switch", "-q", "main"]); - commit(&wt, "g", "local\n", "committed on main"); - land_elsewhere(&dir, &origin); - - let refusal = sync(&wt).expect_err("a main with commits of its own cannot fast-forward"); - assert!(refusal.contains("carries commits origin/main has not got"), "{refusal}"); - assert!(refusal.contains("committed on main"), "{refusal}"); - } - - /// Commits on `main` with nothing new on GitHub are still stranded. - #[test] - fn a_main_ahead_and_not_behind_is_refused_as_stranded() { - let (_dir, _origin, wt) = repo("sync-ahead-only"); - sh(&wt, &["switch", "-q", "main"]); - commit(&wt, "g", "local\n", "committed on main"); - - let refusal = sync(&wt).expect_err("a main ahead of origin/main is not current"); - assert!(refusal.contains("carries commits origin/main has not got"), "{refusal}"); - assert!(refusal.contains("committed on main"), "{refusal}"); - } - - /// A concurrent `--sync` already made the move: this run says where `main` - /// is and claims no move. - #[test] - fn a_fast_forward_a_concurrent_run_made_is_not_reported_as_this_ones() { - let (dir, origin, wt) = repo("sync-already"); - sh(&wt, &["switch", "-q", "main"]); - land_elsewhere(&dir, &origin); - git(&wt, &["fetch", "--quiet", "origin", "main"]).unwrap(); - - let (before, after, commits) = - fast_forward(&wt).unwrap().expect("the first fast-forward moves main"); - assert_ne!(before, after); - assert_eq!(commits, "1"); - assert_eq!(fast_forward(&wt).unwrap(), None, "main was already there"); - } - - /// A primary that is not this checkout and has uncommitted work is left alone. - #[test] - fn a_dirty_primary_that_is_another_checkout_is_left_where_it_is() { - let (dir, origin, primary) = repo("sync-dirty-elsewhere"); - sh(&primary, &["switch", "-q", "main"]); - let asking = dir.join("asking"); - sh(&primary, &["worktree", "add", "-q", "-b", "task", asking.to_str().unwrap()]); - land_elsewhere(&dir, &origin); - fs::write(primary.join("f"), "dirty\n").unwrap(); - let before = git(&primary, &["rev-parse", "main"]).unwrap(); - - let said = sync(&asking).expect("a dirty primary is reported, never refused"); - assert!(said.contains("has uncommitted work in it"), "{said}"); - assert_eq!(git(&primary, &["rev-parse", "main"]).unwrap(), before); - } -} diff --git a/src/sysroot.rs b/src/sysroot.rs index 868929b31bd..61ff79e1ec7 100644 --- a/src/sysroot.rs +++ b/src/sysroot.rs @@ -18,8 +18,7 @@ //! it, and a crate built against a sysroot learns which targets' libraries //! moved ([`Identity`]). Each build refuses dep-info that says otherwise. //! -//! **Each worktree builds std in its own fork checkout, and nothing but the -//! primary's own sync moves the primary's.** The primary builds in its `rust/`; +//! **Each worktree builds std in its own fork checkout.** The primary builds in its `rust/`; //! a linked worktree in its own `rust/`, made on first need as a git worktree of //! the primary's fork repository at the commit this tree pins ([`fork_checkout`]). //! `library/std` names `toyos-abi` and `toyos` as `../../../`, so each @@ -38,7 +37,7 @@ //! held in use while the C++ runtime is built from its sources. //! //! A sysroot or freestanding libraries no worktree names any more are removed -//! by `keystore::sweep`, which `--worktree remove` runs: each build records the +//! by `keystore::sweep`, which every placement runs: each build records the //! keys it used in its worktree's `target/`, and a key no registered worktree //! records, that nobody is making or using, goes. @@ -54,9 +53,8 @@ use crate::arch::Arch; use crate::buildlock::{self, Guard, Held, Keyed}; use crate::compiler::{self, Compiler}; use crate::identity; -use crate::keystore::Key; +use crate::keystore::{self, Key}; use crate::toolchain::{self, host_triple, GuestTarget, Owner, Role, GUEST_TARGETS}; -use crate::worktree::remove_tree; use whole_toolchain::{whole, Whole}; /// The per-worktree sources that end up inside a sysroot: std links `toyos-abi` @@ -121,8 +119,7 @@ impl Libraries { /// A sysroot a build compiles against, held in use for as long as this lives. pub struct Sysroot { - /// A toolchain directory: `RUSTUP_TOOLCHAIN` names it. - pub dir: PathBuf, + dir: PathBuf, /// Whether its compiler is the primary's, which the ToyOS-hosted rustc is /// built from. pub primary_compiler: bool, @@ -139,6 +136,12 @@ impl Sysroot { let id = Key::of(release.as_bytes()); Self { dir: stage2, primary_compiler: true, identity: Identity::new(id.clone(), &id, &id), _using: None } } + + /// Its toolchain directory, which `RUSTUP_TOOLCHAIN` names: lent, never + /// handed out, because a sweep may remove it once this is dropped. + pub fn dir(&self) -> &Path { + &self.dir + } } /// What a crate compiled against a sysroot compiles against beside its own @@ -510,10 +513,11 @@ fn unfinished(dir: &Path) -> Option { unpublished(dir).or_else(|| toolchain::toolchain_defect(dir)) } -/// The sysroot `key` names at `dir`, made by `make` if nobody has made it, and -/// held in use for as long as the returned guard lives. +/// The sysroot `key` names at `dir`, recorded as `root`'s, made by `make` if +/// nobody has made it, and held in use for as long as the returned guard lives. fn held(root: &Path, key: &Key, dir: &Path, make: impl FnMut()) -> Guard { - buildlock::keyed_made(root, Keyed::Sysroot, key, || unfinished(dir), make) + let store = dir.parent().expect("a sysroot is a directory of its store"); + keystore::made(root, Keyed::Sysroot, store, key, || unfinished(dir), make) } /// The sysroot this worktree's sources name, made if nobody has made it, and @@ -523,8 +527,7 @@ pub fn ensure(root: &Path, rust_dir: &Path, lock: &mut Held) -> Sysroot { let compiler = compiler::resolve(root, rust_dir, &fork, lock); let keys = Keys::of(root, &compiler, &fork); let dir = sysroots_dir(rust_dir).join(&keys.sysroot); - crate::keystore::record(root, Keyed::Sysroot, &keys.sysroot); - crate::keystore::record(root, Keyed::Freestanding, &keys.freestanding); + keystore::record(root, Keyed::Freestanding, &keys.freestanding); let using = lock.without_shared(|| held(root, &keys.sysroot, &dir, || build(root, rust_dir, &compiler, &fork, &keys, &dir))); @@ -570,9 +573,10 @@ mod whole_toolchain { fn build(root: &Path, rust_dir: &Path, compiler: &Compiler, fork: &Path, keys: &Keys, dir: &Path) { let made_from = whole(compiler); let store = freestanding_dir(rust_dir).join(&keys.freestanding); - let _freestanding = buildlock::keyed_made( + let _freestanding = keystore::made( root, Keyed::Freestanding, + &freestanding_dir(rust_dir), &keys.freestanding, || unpublished(&store), || build_freestanding(root, compiler, fork, &keys.freestanding, &store), @@ -598,7 +602,7 @@ fn build(root: &Path, rust_dir: &Path, compiler: &Compiler, fork: &Path, keys: & match Libraries::of(target) { Libraries::Worktree => place_std(&stamp(&built, triple), &lib), Libraries::Freestanding => { - remove_tree(&lib); + keystore::remove(&lib); clone_tree(&store.join(triple), &lib); } } @@ -669,11 +673,11 @@ fn publish_toolchain(compiler: Whole, dir: &Path, fill: impl FnOnce(&Path) -> St /// or worse refused, and it is replaced. fn publish(dir: &Path, fill: impl FnOnce(&Path) -> String) { let partial = dir.with_extension("partial"); - remove_tree(&partial); + keystore::remove(&partial); let sources = fill(&partial); fs::write(partial.join(SOURCES), sources) .unwrap_or_else(|e| panic!("write {}: {e}", partial.join(SOURCES).display())); - remove_tree(dir); + keystore::remove(dir); fs::rename(&partial, dir) .unwrap_or_else(|e| panic!("rename {} -> {}: {e}", partial.display(), dir.display())); } @@ -717,7 +721,7 @@ fn prepare_std_build(build_dir: &Path, host: &str, identity: &str, targets: &[&s // Bootstrap reuses what it built before and does not see a path dependency // outside the fork move, so each target's std starts from nothing. for target in targets { - remove(&build_dir.join(host).join("stage0-std").join(target)); + keystore::remove(&build_dir.join(host).join("stage0-std").join(target)); } } @@ -745,27 +749,13 @@ fn forget_another_compiler(build_dir: &Path, host: &str, identity: &str) { for entry in entries { let entry = entry.unwrap_or_else(|e| panic!("read {}: {e}", dir.display())); if !kept.iter().any(|name| entry.file_name() == *name) { - remove(&entry.path()); + keystore::remove(&entry.path()); } } } fs::write(&record, identity).unwrap_or_else(|e| panic!("write {}: {e}", record.display())); } -/// Remove `path`, a directory with all it holds or anything else; that nothing -/// is there is not an error. -fn remove(path: &Path) { - let removed = match fs::symlink_metadata(path) { - Ok(meta) if meta.is_dir() => fs::remove_dir_all(path), - Ok(_) => fs::remove_file(path), - Err(e) => Err(e), - }; - match removed { - Err(e) if e.kind() != std::io::ErrorKind::NotFound => panic!("remove {}: {e}", path.display()), - _ => {} - } -} - /// The stamp bootstrap wrote naming every library it built for `target` under /// `built`: the one list of them. **Not `stage0-sysroot`**, which a stage-0 /// build fills with the stage-0 compiler's own libraries and never with what @@ -1215,7 +1205,8 @@ mod tests { let deps = build.join("bootstrap/debug/deps"); write(&deps.join("libserde-1.rlib"), "built"); write(&rust_dir.join("build/toyos-compiler"), "tree-2"); - let mode = |bits| fs::set_permissions(&deps, fs::Permissions::from_mode(bits)).unwrap(); + // The parent: a removal gives back the write permission of what it removes. + let mode = |bits| fs::set_permissions(&build, fs::Permissions::from_mode(bits)).unwrap(); mode(0o555); let stuck = std::panic::catch_unwind(|| forget_another_compiler(&build, "host", &identity())); @@ -1342,7 +1333,8 @@ mod tests { /// **A sysroot is whole, or it is made again**: a `stage2` without cargo — /// what bootstrap leaves until the primary completes it — is refused by /// name and nothing is published, and one found with its `SOURCES` and - /// without its cargo is rebuilt rather than trusted, once. + /// without its cargo is rebuilt rather than trusted, once. Placing one + /// removes a sysroot no worktree names. #[test] fn a_sysroot_is_whole_or_it_is_made_again() { let base = TempDir::new("whole"); @@ -1371,8 +1363,11 @@ mod tests { clone_tree(&compiler.stage2, &dir); write(&dir.join(SOURCES), "found\n"); toolchain::provision_toolchain_cargo(&compiler.stage2); + let orphan = sysroots_dir(&base.join("rust")).join(Key::of(b"named by no worktree")); + fs::create_dir_all(&orphan).unwrap(); let using = held(&base, &key, &dir, || make(&dir, 2)); assert_eq!(made.get(), 2, "a sysroot without its cargo was trusted because it has SOURCES"); + assert!(!orphan.exists(), "placing a sysroot left one no worktree names"); assert_eq!(toolchain::toolchain_defect(&dir), None); assert!(dir.join("lib/rustlib/x86_64-unknown-toyos/lib/libstd.rlib").is_file()); drop(using); @@ -1454,28 +1449,4 @@ mod tests { let refused = std::panic::catch_unwind(|| place_std(&built.join(".libstd-stamp"), &lib)); assert!(refused.is_err(), "a host library was placed in a guest target"); } - - /// `--worktree remove` takes the worktree's fork checkout with it — git will - /// not remove a worktree around one — unless that checkout holds the only - /// copy of something. - #[test] - fn a_removed_worktree_takes_its_fork_checkout_and_refuses_to_lose_fork_work() { - let base = TempDir::new("fork-remove"); - let (primary, linked, _c1, _c2) = two_pins(&base); - let fork = fork_checkout(&linked); - write(&fork.join("library/std/src/lib.rs"), "pub fn unsaved() {}\n"); - let refused = std::panic::catch_unwind(|| crate::worktree::remove(&primary, linked.to_str().unwrap())); - assert!(refused.is_err(), "a fork checkout with uncommitted work was removed"); - assert!(fork.join("library/std/src/lib.rs").is_file()); - - git(&fork, &["commit", "-qam", "committed, and on no ref"]); - let refused = std::panic::catch_unwind(|| crate::worktree::remove(&primary, linked.to_str().unwrap())); - assert!(refused.is_err(), "a fork commit no ref reaches was thrown away"); - - git(&fork, &["branch", "kept"]); - crate::worktree::remove(&primary, linked.to_str().unwrap()); - assert!(!linked.exists(), "{} is still on disk", linked.display()); - let listed = git(&primary.join("rust"), &["worktree", "list", "--porcelain"]); - assert_eq!(listed.lines().filter(|l| l.starts_with("worktree ")).count(), 1, "{listed}"); - } } diff --git a/src/worktree.rs b/src/worktree.rs deleted file mode 100644 index ebc6be4e915..00000000000 --- a/src/worktree.rs +++ /dev/null @@ -1,838 +0,0 @@ -//! Making a linked worktree buildable, and saying why when it cannot be. -//! -//! `git worktree add` alone leaves a tree that does not build and, worse, one -//! that builds *wrongly*: `rust/` comes out an empty stub, so the build system -//! reads it as a missing submodule, clones 913 MiB from the network, bootstraps -//! a second 47 GiB toolchain, and finally points the machine-global rustup -//! `toyos` name at it — taking the toolchain out from under every other -//! checkout. Measured, in that order, on this host. -//! -//! So the compiler stays the primary's and nothing here copies it: -//! [`crate::toolchain::rust_dir`] sends every compiler read to the primary -//! checkout. `rust/` is left the stub it was until the first build makes it -//! this worktree's own fork checkout — a git worktree of the primary's fork -//! repository, sharing its objects (`src/sysroot.rs`) — which [`remove`] takes -//! away again. What this module does is the small remainder — create the -//! worktree, carry over the one file git cannot, and refuse by name when the -//! result would not be usable. - -use std::fs; -use std::path::{Path, PathBuf}; -use std::process::Command; - -use crate::buildlock::Keyed; -use crate::flags; -use crate::toolchain; - -/// What a worktree's crate target directories reach: 4.1 GiB after -/// `--build-only`, and 23 GiB on the primary checkout, which has run everything; -/// its fork checkout and std build directory add 4.2 GiB. Measured with `du`. -/// The primary's 50 GiB `rust/` is shared and never counted here. -/// -/// Refusing at the upper figure plus a little, rather than at the lower one: a -/// build that fills the disk halfway through costs more than a worktree that -/// was never made. -const NEEDED_BYTES: u64 = 25 * 1024 * 1024 * 1024; - -pub fn dispatch(root: &Path, args: &[String]) { - let mut rest = flags::CARGO_RUN.rest(args, &flags::WORKTREE).iter(); - let verb = rest.next().map(String::as_str); - let operand = rest.next().cloned(); - match verb { - Some("add") => add(root, &path_operand("add", operand)), - Some("list") => list(root), - Some("remove") => remove(root, &path_operand("remove", operand)), - other => panic!( - "--worktree takes add , list, or remove ; got {other:?}" - ), - } -} - -fn path_operand(verb: &str, operand: Option) -> String { - let path = operand.unwrap_or_else(|| panic!("--worktree {verb} needs a path")); - assert!( - !path.starts_with('-'), - "--worktree {verb} needs a path, got flag {path:?}" - ); - path -} - -/// Create a worktree and leave it in a state where `cargo run -- --build-only` -/// works. -fn add(root: &Path, path: &str) { - let path = PathBuf::from(path); - assert!(!path.exists(), "{} already exists", path.display()); - let name = path - .file_name() - .unwrap_or_else(|| panic!("{} has no final component", path.display())) - .to_string_lossy() - .to_string(); - - // Everything that would make the result unusable, asked before anything is - // created: a half-made worktree is worse than none, because the next agent - // finds it and believes it. - let primary = match toolchain::owner(root) { - toolchain::Owner::Us | toolchain::Owner::Installed => root.to_path_buf(), - toolchain::Owner::Elsewhere(p) => p, - }; - let stage2 = primary.join(format!( - "rust/build/{}/stage2", - toolchain::host_triple() - )); - assert!( - stage2.join("bin/rustc").exists(), - "the shared toolchain does not exist yet ({} is missing).\n\ - Run `cargo run -- --build-only` in {} before making worktrees of it.", - stage2.display(), - primary.display() - ); - let free = free_bytes(path.parent().unwrap_or(Path::new("/"))); - assert!( - free >= NEEDED_BYTES, - "{} has {:.1} GiB free and a worktree's target directories reach about \ - {:.0} GiB.\nThe shared toolchain is not copied, but the crate targets are \ - its own.", - path.parent().unwrap_or(Path::new("/")).display(), - free as f64 / 1024.0_f64.powi(3), - NEEDED_BYTES as f64 / 1024.0_f64.powi(3), - ); - - let summary = create_worktree(root, &path, &name); - - // `rust/` is deliberately left the empty stub `git worktree add` made: the - // first build makes it a fork checkout sharing the primary's objects, where - // `git submodule update` would be the 913 MiB clone, and a symlink in its - // place makes git error out of `status`, `diff` and `submodule` alike. - - // The one file git cannot carry: it is gitignored, and a worktree that - // silently loses the fork redirects would build different code from the - // checkout it was made from and report the difference as a result. - let redirects = root.join(".cargo/config.toml"); - if redirects.exists() { - fs::copy(&redirects, path.join(".cargo/config.toml")) - .unwrap_or_else(|e| panic!("copy {}: {e}", redirects.display())); - eprintln!("carried over .cargo/config.toml (fork redirects)"); - } - - eprintln!(); - eprintln!("worktree {}", path.display()); - eprintln!("branch {summary}"); - eprintln!("compiler {} (shared, not copied)", stage2.display()); - eprintln!(); - eprintln!("Build it with `cargo run -- --build-only` from {}.", path.display()); -} - -/// Never a fetch, never a reset. Every refusal below runs before either -/// branch is touched, so a half-made worktree never sits behind one. -fn create_worktree(root: &Path, path: &Path, name: &str) -> String { - let branch = format!("wt/{name}"); - let upstream = format!("origin/{branch}"); - let local_exists = ok(root, &["show-ref", "--verify", "--quiet", &format!("refs/heads/{branch}")]); - let origin_exists = ok(root, &["show-ref", "--verify", "--quiet", &format!("refs/remotes/{upstream}")]); - let path_str = path.to_string_lossy(); - if local_exists { - refuse_if_no_commit_beyond_main( - root, - &branch, - &format!("delete it with `git branch -d {branch}` or pick a new name for the worktree."), - ); - if origin_exists { - refuse_if_behind_or_diverged(root, &branch, &upstream); - } - git(root, &["worktree", "add", &path_str, &branch]); - format!("{branch} (resumed at {})", short_sha(path, "HEAD")) - } else if origin_exists { - refuse_if_no_commit_beyond_main(root, &upstream, "pick a new name for the worktree."); - git(root, &["worktree", "add", "--track", "-b", &branch, &path_str, &upstream]); - format!("{branch} (resumed from {upstream} at {})", short_sha(path, "HEAD")) - } else { - git(root, &["worktree", "add", "-b", &branch, &path_str, "main"]); - format!("{branch} (new, from main at {})", short_sha(path, "HEAD")) - } -} - -/// Refuse by name, before anything is created, when `resolve` carries no -/// commit beyond `origin/main` — the same ancestry test [`measure`] uses to -/// call a worktree landed and offer its build caches back. -/// -/// `merge-base --is-ancestor` cannot tell a branch that landed apart from one -/// that never diverged from `main` in the first place: both are true of it. -/// The message says only what both share, and never "landed" or "merged" — -/// a resume would otherwise start work from an old tip behind main, or from -/// a branch that never carried any work of its own. -fn refuse_if_no_commit_beyond_main(root: &Path, resolve: &str, hint: &str) { - assert!( - !ok(root, &["merge-base", "--is-ancestor", resolve, "origin/main"]), - "{resolve} carries no commit beyond origin/main; {hint}" - ); -} - -/// Refuse by name, before anything is created, when the local `branch` is not -/// at or ahead of its own `upstream`: a resume would otherwise pick the local -/// tip silently, and the push back would refuse for the same reason, later -/// and less clearly. -fn refuse_if_behind_or_diverged(root: &Path, branch: &str, upstream: &str) { - if ok(root, &["merge-base", "--is-ancestor", upstream, branch]) { - return; - } - let relation = if ok(root, &["merge-base", "--is-ancestor", branch, upstream]) { - "is behind" - } else { - "has diverged from" - }; - panic!( - "{branch} ({}) {relation} {upstream} ({}); merge it first.", - short_sha(root, branch), - short_sha(root, upstream), - ); -} - -fn short_sha(dir: &Path, rev: &str) -> String { - capture(dir, &["rev-parse", "--short", rev]).trim().to_string() -} - -fn list(root: &Path) { - let primary = match toolchain::owner(root) { - toolchain::Owner::Us | toolchain::Owner::Installed => root.to_path_buf(), - toolchain::Owner::Elsewhere(p) => p, - }; - eprintln!("toolchain owner {}", primary.display()); - eprintln!( - "rustup toyos {}", - fs::read_link( - std::env::var_os("HOME") - .map(PathBuf::from) - .unwrap_or_default() - .join(".rustup/toolchains/toyos") - ) - .map_or_else(|_| "unlinked".to_string(), |p| p.display().to_string()) - ); - eprintln!(); - let trees = survey(root, true); - for tree in &trees { - let branch = if tree.branch.is_empty() { "(detached)" } else { &tree.branch }; - let note = match (tree.primary, tree.landed) { - (true, _) => " primary", - (_, true) => " landed — reclaimable", - _ => "", - }; - eprintln!( - "{:<44} {:<26} {:>9} in {:>2} target dir(s){note}", - tree.path.display(), - branch, - gib(tree.bytes), - tree.targets, - ); - } - eprintln!(); - eprintln!( - "{} worktree(s), {} of build caches; the shared toolchain is not counted", - trees.len(), - gib(trees.iter().map(|t| t.bytes).sum()), - ); - if let Some(line) = reclaim_line(&trees) { - eprintln!("{line}"); - } -} - -/// One worktree, and the two facts that decide whether it should still exist. -/// -/// **Nothing ever reclaimed one**, and `add`'s disk check was the whole of what -/// this subject had — a refusal is the last notice rather than the first. A -/// worktree whose branch has landed has no reason to hold its build caches, and -/// neither its size nor whether its branch is in `origin/main` is anything -/// `git worktree list` says. -pub struct Tree { - pub path: PathBuf, - /// Empty for a detached worktree. - pub branch: String, - /// What its build caches hold. The shared `rust/` is never counted. - pub bytes: u64, - pub targets: usize, - /// The checkout that owns `rust/`, the rustup link and `main`. Never - /// reclaimable whatever its branch says. - pub primary: bool, - /// Its branch is already in `origin/main`. - pub landed: bool, -} - -/// Every worktree of `root`. -/// -/// `all_sizes` walks every worktree's caches, which is a metadata walk of tens -/// of gigabytes and takes seconds; `false` walks only the ones that could be -/// given back, which is the only size `--sync` prints. -pub fn survey(root: &Path, all_sizes: bool) -> Vec { - let mut trees = Vec::new(); - let mut path: Option = None; - let mut branch = String::new(); - let listing = capture(root, &["worktree", "list", "--porcelain"]); - for line in listing.lines() { - if let Some(next) = line.strip_prefix("worktree ") { - if let Some(done) = path.replace(PathBuf::from(next)) { - let first = trees.is_empty(); - trees.push(measure(root, done, std::mem::take(&mut branch), first, all_sizes)); - } - } else if let Some(name) = line.strip_prefix("branch ") { - branch = name.trim_start_matches("refs/heads/").to_string(); - } - } - if let Some(done) = path { - let first = trees.is_empty(); - trees.push(measure(root, done, branch, first, all_sizes)); - } - trees -} - -/// What could be given back, or nothing to say. -/// -/// `--sync` reports this as well as `list`, because `--sync` runs at the moment -/// a branch lands, which is the moment its worktree stops having a reason to -/// exist. -pub fn reclaim_line(trees: &[Tree]) -> Option { - let done: Vec<&Tree> = trees.iter().filter(|t| !t.primary && t.landed).collect(); - if done.is_empty() { - return None; - } - Some(format!( - "{} worktree(s) hold {} on branches already in origin/main: {}\n\ - `cargo run -- --worktree remove ` gives each back, and refuses one carrying \ - uncommitted work.", - done.len(), - gib(done.iter().map(|t| t.bytes).sum()), - done.iter().map(|t| t.path.display().to_string()).collect::>().join(", "), - )) -} - -fn measure( - root: &Path, - path: PathBuf, - branch: String, - primary: bool, - all_sizes: bool, -) -> Tree { - let landed = !primary - && !branch.is_empty() - && ok(root, &["merge-base", "--is-ancestor", &branch, "origin/main"]); - let mut bytes = 0; - let mut targets = 0; - if all_sizes || landed { - caches(&path, &mut bytes, &mut targets); - } - Tree { path, branch, bytes, targets, primary, landed } -} - -/// Directories a survey never enters: the shared toolchain and git's own store. -const NOT_OURS: &[&str] = &["rust", ".git"]; - -/// Ten `target/` directories per worktree is the design and not an accident — -/// `Cargo.toml`'s `exclude` list keeps five cross-compiled crates out of the -/// host workspace and each guest fixture resolves on its own — so `cargo clean` -/// at the root reaches exactly one of them and a count is worth printing. -fn caches(dir: &Path, bytes: &mut u64, targets: &mut usize) { - let Ok(entries) = fs::read_dir(dir) else { return }; - for entry in entries.flatten() { - let path = entry.path(); - if !fs::symlink_metadata(&path).is_ok_and(|m| m.is_dir()) { - continue; - } - let name = entry.file_name(); - let name = name.to_string_lossy(); - if NOT_OURS.contains(&name.as_ref()) { - continue; - } - if name == "target" { - *bytes += bytes_under(&path); - *targets += 1; - continue; - } - caches(&path, bytes, targets); - } -} - -fn bytes_under(dir: &Path) -> u64 { - let Ok(entries) = fs::read_dir(dir) else { return 0 }; - let mut total = 0; - for entry in entries.flatten() { - let path = entry.path(); - let Ok(meta) = fs::symlink_metadata(&path) else { continue }; - total += if meta.is_dir() { bytes_under(&path) } else { meta.len() }; - } - total -} - -fn gib(bytes: u64) -> String { - format!("{:.1} GiB", bytes as f64 / 1024.0_f64.powi(3)) -} - -/// Remove a worktree and the branch it was made with. -/// -/// Deliberately not `--force`: git refuses a worktree holding tracked changes -/// or untracked files and leaves it registered, because the work in a -/// worktree is the only copy of itself — that refusal stands, and so does the -/// one for its own fork checkout ([`remove_fork_checkout`]). -/// -/// **git can unregister a worktree and then fail to delete it**: a path deeper -/// than `PATH_MAX` under an ignored `target/`, or a file created while it -/// walks, and it exits non-zero with the directory still there and no longer -/// a worktree of anything. Once git has let go of it nothing in it is work, -/// so the whole directory goes. -/// -/// Then every sysroot, compiler and LLVM no remaining worktree names goes too -/// (`src/sysroot.rs`, `src/compiler.rs`, `src/llvm.rs`). -pub(crate) fn remove(root: &Path, path: &str) { - let at = root.join(path); - remove_fork_checkout(root, &at); - if !ok_loud(root, &["worktree", "remove", path]) { - assert!( - !registered(root, &at), - "git refused to remove {path} and it is still a worktree; what it said above is \ - why. Nothing was deleted." - ); - remove_tree(&at); - eprintln!("git unregistered {path} and left its ignored files; deleted them"); - } - eprintln!("removed {path}; its branch is still there, and `git branch -d` will say if it is unmerged"); - let rust_dir = crate::toolchain::rust_dir(root); - for (kind, store, what) in [ - (Keyed::Sysroot, crate::sysroot::sysroots_dir(&rust_dir), "sysroot"), - (Keyed::Freestanding, crate::sysroot::freestanding_dir(&rust_dir), "freestanding libraries"), - (Keyed::Compiler, crate::compiler::compilers_dir(&rust_dir), "compiler"), - (Keyed::Llvm, crate::llvm::store(&rust_dir), "LLVM"), - ] { - let swept = crate::keystore::sweep(root, kind, &store); - if !swept.is_empty() { - eprintln!("removed {} {what}(s) no worktree names any more", swept.len()); - } - } -} - -/// A linked worktree's own fork checkout (`src/sysroot.rs`'s `fork_checkout`) -/// is a git worktree of the primary's fork repository, which git will not -/// remove a worktree around. It goes first, and only while it holds nothing -/// that is not also somewhere else: no change in its tree, and a `HEAD` some -/// ref of the fork repository reaches. -fn remove_fork_checkout(root: &Path, at: &Path) { - let fork = at.join("rust"); - if !fork.join(".git").is_file() || !at.join(".git").is_file() { - return; - } - let path = at.display(); - let mine = capture(at, &["status", "--porcelain", "--ignore-submodules=all"]); - assert!(mine.is_empty(), "{path} holds uncommitted work:\n{mine}Nothing was deleted."); - let theirs = capture(&fork, &["status", "--porcelain", "--ignore-submodules=none"]); - assert!( - theirs.is_empty(), - "{}'s fork checkout holds uncommitted work:\n{theirs}Nothing was deleted.", - path - ); - let head = capture(&fork, &["rev-parse", "HEAD"]); - let reached = capture(&fork, &["for-each-ref", "--count=1", "--contains", head.trim()]); - assert!( - !reached.trim().is_empty(), - "{}'s fork checkout is at {}, which no branch, tag or remote ref of the fork \ - repository reaches: it is the only copy of those commits. Push them, or name them \ - with a branch, first. Nothing was deleted.", - path, - head.trim() - ); - // Moved out whole first, so a removal a writer interrupts leaves a named - // directory outside the worktree rather than a half-deleted checkout in it. - let name = at.file_name().expect("a worktree has a name").to_string_lossy(); - let aside = at.with_file_name(format!(".{name}-rust.removing")); - fs::rename(&fork, &aside) - .unwrap_or_else(|e| panic!("move {} to {}: {e}", fork.display(), aside.display())); - fs::create_dir(&fork).unwrap_or_else(|e| panic!("recreate the stub {}: {e}", fork.display())); - let primary = crate::primary_checkout(root); - git(&primary.join("rust"), &["worktree", "prune"]); - let backtrace = primary.join("rust/library/backtrace"); - if backtrace.join(".git").exists() { - git(&backtrace, &["worktree", "prune"]); - } - remove_tree(&aside); -} - -/// Remove `dir` and everything in it, including what appears while it goes. -/// -/// A writer on this host — the leftovers are `.DS_Store` files — can put a file -/// into a directory while it is being emptied, so a plain recursive delete finds a directory it has just emptied not empty and -/// stops halfway — the `Directory not empty` git itself dies on. The removal -/// runs again over what is left, at most [`PASSES`] times; a tree still refusing -/// after that has a writer this cannot outrun, and the panic says so. -pub(crate) fn remove_tree(dir: &Path) { - for pass in 1..=PASSES { - match fs::remove_dir_all(dir) { - Ok(()) => return, - Err(e) if e.kind() == std::io::ErrorKind::NotFound => return, - Err(e) if e.kind() == std::io::ErrorKind::DirectoryNotEmpty && pass < PASSES => { - eprintln!("{} gained files while it was removed ({e}); removing again", dir.display()); - } - Err(e) => panic!("remove {}: {e}, after {pass} pass(es)", dir.display()), - } - } -} - -/// How many times [`remove_tree`] runs over a tree that keeps refusing. -const PASSES: usize = 10; - -/// Whether `git worktree list` still names `at`, compared as real paths: -/// git prints its own realpath, `/private/tmp/…` for `/tmp/…`. -fn registered(root: &Path, at: &Path) -> bool { - let at = fs::canonicalize(at).unwrap_or_else(|_| at.to_path_buf()); - capture(root, &["worktree", "list", "--porcelain"]) - .lines() - .filter_map(|l| l.strip_prefix("worktree ")) - .any(|listed| fs::canonicalize(listed).unwrap_or_else(|_| PathBuf::from(listed)) == at) -} - -fn free_bytes(dir: &Path) -> u64 { - let path = std::ffi::CString::new(dir.as_os_str().as_encoded_bytes()) - .unwrap_or_else(|_| panic!("{} has an embedded NUL", dir.display())); - let mut buf: libc::statvfs = unsafe { std::mem::zeroed() }; - // SAFETY: `path` is a valid, NUL-terminated C string and `buf` is a - // `libc::statvfs` the kernel fills in whole or leaves at the `zeroed()` - // above; a non-zero return is checked before anything reads it. - let rc = unsafe { libc::statvfs(path.as_ptr(), &mut buf) }; - assert!(rc == 0, "statvfs {}: {}", dir.display(), std::io::Error::last_os_error()); - buf.f_bavail as u64 * buf.f_frsize as u64 -} - -fn git(dir: &Path, args: &[&str]) { - let status = Command::new("git") - .args(args) - .current_dir(dir) - .status() - .unwrap_or_else(|e| panic!("run git: {e}")); - assert!(status.success(), "git {args:?} failed"); -} - -/// git's answer, for a question rather than an action. -fn capture(dir: &Path, args: &[&str]) -> String { - let out = Command::new("git") - .args(args) - .current_dir(dir) - .output() - .unwrap_or_else(|e| panic!("run git: {e}")); - assert!(out.status.success(), "git {args:?}: {}", String::from_utf8_lossy(&out.stderr).trim()); - String::from_utf8_lossy(&out.stdout).into_owned() -} - -/// Whether git did it, with what it said left on stderr for the reader. -fn ok_loud(dir: &Path, args: &[&str]) -> bool { - Command::new("git") - .args(args) - .current_dir(dir) - .status() - .unwrap_or_else(|e| panic!("run git: {e}")) - .success() -} - -/// Whether git says yes. A non-zero exit is the answer here, never a failure. -fn ok(dir: &Path, args: &[&str]) -> bool { - Command::new("git") - .args(args) - .current_dir(dir) - .stdout(std::process::Stdio::null()) - .stderr(std::process::Stdio::null()) - .status() - .is_ok_and(|s| s.success()) -} - -#[cfg(test)] -mod tests { - use super::*; - use toyos_tmpdir::TempDir; - - /// `--worktree` owns the rest of the command line, so this refusal is the - /// only one between `--worktree add --help` and a worktree named `--help`. - #[test] - fn a_flag_is_refused_as_a_worktree_path_by_name() { - let args = ["--worktree", "add", "--help"].map(String::from); - assert!( - matches!(crate::flags::check(&args), crate::flags::Outcome::Proceed), - "the command line has to reach this dispatch" - ); - let panic = std::panic::catch_unwind(|| dispatch(Path::new("/not-used"), &args)) - .expect_err("a flag is not a worktree path"); - let message = panic.downcast::().expect("the refusal is formatted"); - assert!(message.contains("--help"), "the refusal must name the bad argument: {message}"); - } - - fn tree(path: &str, primary: bool, landed: bool, bytes: u64) -> Tree { - Tree { - path: PathBuf::from(path), - branch: String::from("wt/x"), - bytes, - targets: 10, - primary, - landed, - } - } - - /// **The primary checkout sits on `main`**, which is an ancestor of - /// `origin/main` by construction, so a rule that offered back every landed - /// worktree would offer back the one holding `rust/` and the rustup link. - #[test] - fn only_a_landed_worktree_that_is_not_the_primary_is_offered_back() { - assert!(reclaim_line(&[tree("/primary", true, true, 4 << 30)]).is_none()); - assert!(reclaim_line(&[tree("/live", false, false, 8 << 30)]).is_none()); - let line = reclaim_line(&[ - tree("/primary", true, true, 4 << 30), - tree("/gone", false, true, 2 << 30), - tree("/live", false, false, 8 << 30), - ]) - .expect("a landed worktree that is not the primary is reclaimable"); - assert!(line.contains("/gone"), "{line}"); - assert!(!line.contains("/live"), "{line}"); - assert!(!line.contains("/primary"), "{line}"); - assert!(line.contains("2.0 GiB"), "the offer has to say what it is worth: {line}"); - } - - #[test] - fn a_local_branch_is_resumed_at_its_own_commit_not_mains() { - let (dir, _origin, work) = crate::gitfixture::repo("wtresume-local"); - git(&work, &["checkout", "-q", "main"]); - git(&work, &["branch", "-qD", "wt"]); - git(&work, &["checkout", "-qb", "wt/foo", "main"]); - crate::gitfixture::commit(&work, "on-branch", "branch work\n", "branch work"); - git(&work, &["push", "-q", "-u", "origin", "wt/foo"]); - crate::gitfixture::commit(&work, "on-branch-2", "more branch work\n", "more branch work"); - git(&work, &["checkout", "-q", "main"]); - crate::gitfixture::commit(&work, "on-main", "main moved on\n", "main moved on"); - - let path = dir.join("resumed"); - let summary = create_worktree(&work, &path, "foo"); - - assert!(summary.contains("resumed at"), "{summary}"); - assert!(!summary.contains("main"), "{summary}"); - let branch_sha = capture(&work, &["rev-parse", "wt/foo"]); - let origin_sha = capture(&work, &["rev-parse", "origin/wt/foo"]); - let worktree_sha = capture(&path, &["rev-parse", "HEAD"]); - let main_sha = capture(&work, &["rev-parse", "main"]); - assert_ne!(branch_sha, origin_sha, "the local tip must be ahead of origin's, or this proves nothing"); - assert_eq!(worktree_sha, branch_sha, "must resume at the local branch's own tip, not origin's"); - assert_ne!(worktree_sha, main_sha, "must not have been reset onto main"); - } - - /// A local branch already merged into `origin/main` is refused by name - /// rather than resumed from its old tip behind main. - #[test] - fn a_landed_local_branch_is_refused_not_resumed() { - let (dir, _origin, work) = crate::gitfixture::repo("wtresume-landed-local"); - git(&work, &["checkout", "-q", "main"]); - git(&work, &["branch", "-qD", "wt"]); - git(&work, &["checkout", "-qb", "wt/foo", "main"]); - crate::gitfixture::commit(&work, "on-branch", "branch work\n", "branch work"); - git(&work, &["checkout", "-q", "main"]); - git(&work, &["merge", "-q", "--no-ff", "wt/foo", "-m", "merge wt/foo"]); - git(&work, &["push", "-q", "origin", "main"]); - - let path = dir.join("resumed"); - let refused = std::panic::catch_unwind(|| { - create_worktree(&work, &path, "foo") - }); - - let panic = refused.expect_err("a landed branch must not be resumed"); - let message = panic.downcast::().expect("the refusal is formatted"); - assert!(message.contains("wt/foo"), "{message}"); - assert!(!path.exists(), "nothing must be created before the refusal"); - } - - /// A branch made from `main` and never touched carries no commit beyond - /// `origin/main` either, and `merge-base --is-ancestor` cannot tell it - /// apart from one that actually landed — refused before anything is - /// created, for the reason that is true of it, never "landed". - #[test] - fn an_untouched_branch_is_refused_not_resumed() { - let (dir, _origin, work) = crate::gitfixture::repo("wtresume-untouched"); - git(&work, &["checkout", "-q", "main"]); - git(&work, &["branch", "-qD", "wt"]); - git(&work, &["checkout", "-qb", "wt/foo", "main"]); - git(&work, &["checkout", "-q", "main"]); - - let path = dir.join("resumed"); - let refused = std::panic::catch_unwind(|| create_worktree(&work, &path, "foo")); - - let panic = refused.expect_err("an untouched branch must not be resumed"); - let message = panic.downcast::().expect("the refusal is formatted"); - assert!(message.contains("wt/foo"), "{message}"); - assert!(message.contains("carries no commit beyond origin/main"), "{message}"); - assert!(message.contains("git branch -d wt/foo"), "{message}"); - assert!(!message.contains("landed"), "{message}"); - assert!(!message.contains("merged"), "{message}"); - assert!(!path.exists(), "nothing must be created before the refusal"); - } - - /// A local branch behind its own `origin/wt/` is refused rather than - /// resumed at the stale local tip. - #[test] - fn a_local_branch_behind_origin_is_refused() { - let (dir, _origin, work) = crate::gitfixture::repo("wtresume-behind"); - git(&work, &["checkout", "-q", "main"]); - git(&work, &["branch", "-qD", "wt"]); - git(&work, &["checkout", "-qb", "wt/foo", "main"]); - crate::gitfixture::commit(&work, "first", "first\n", "first"); - crate::gitfixture::commit(&work, "second", "second\n", "second"); - git(&work, &["push", "-q", "-u", "origin", "wt/foo"]); - git(&work, &["reset", "-q", "--hard", "HEAD~1"]); - - let path = dir.join("resumed"); - let refused = std::panic::catch_unwind(|| { - create_worktree(&work, &path, "foo") - }); - - let panic = refused.expect_err("a local branch behind its origin counterpart must not resume"); - let message = panic.downcast::().expect("the refusal is formatted"); - assert!(message.contains("wt/foo"), "{message}"); - assert!(message.contains("origin/wt/foo"), "{message}"); - assert!(message.contains("merge it first"), "{message}"); - assert!(!path.exists(), "nothing must be created before the refusal"); - } - - /// A local branch that has diverged from `origin/wt/` — neither is - /// an ancestor of the other — is refused rather than resumed silently at - /// either side. - #[test] - fn a_local_branch_diverged_from_origin_is_refused() { - let (dir, _origin, work) = crate::gitfixture::repo("wtresume-diverged"); - git(&work, &["checkout", "-q", "main"]); - git(&work, &["branch", "-qD", "wt"]); - git(&work, &["checkout", "-qb", "wt/foo", "main"]); - crate::gitfixture::commit(&work, "side", "origin side\n", "origin side"); - git(&work, &["push", "-q", "-u", "origin", "wt/foo"]); - git(&work, &["reset", "-q", "--hard", "main"]); - crate::gitfixture::commit(&work, "side", "local side\n", "local side"); - - let path = dir.join("resumed"); - let refused = std::panic::catch_unwind(|| { - create_worktree(&work, &path, "foo") - }); - - let panic = refused.expect_err("a diverged local branch must not resume"); - let message = panic.downcast::().expect("the refusal is formatted"); - assert!(message.contains("wt/foo"), "{message}"); - assert!(message.contains("origin/wt/foo"), "{message}"); - assert!(message.contains("merge it first"), "{message}"); - assert!(!path.exists(), "nothing must be created before the refusal"); - } - - #[test] - fn an_origin_only_branch_is_recreated_tracking_it() { - let (dir, _origin, work) = crate::gitfixture::repo("wtresume-origin"); - git(&work, &["checkout", "-q", "main"]); - git(&work, &["branch", "-qD", "wt"]); - git(&work, &["checkout", "-qb", "wt/bar", "main"]); - crate::gitfixture::commit(&work, "on-branch", "branch work\n", "branch work"); - git(&work, &["push", "-q", "-u", "origin", "wt/bar"]); - git(&work, &["checkout", "-q", "main"]); - git(&work, &["branch", "-qD", "wt/bar"]); - crate::gitfixture::commit(&work, "on-main", "main moved on\n", "main moved on"); - - let path = dir.join("resumed"); - let summary = create_worktree(&work, &path, "bar"); - - assert!(summary.contains("resumed from origin/wt/bar"), "{summary}"); - let origin_sha = capture(&work, &["rev-parse", "origin/wt/bar"]); - let worktree_sha = capture(&path, &["rev-parse", "HEAD"]); - let main_sha = capture(&work, &["rev-parse", "main"]); - assert_eq!(worktree_sha, origin_sha, "must resume at origin's commit"); - assert_ne!(worktree_sha, main_sha, "must not have been reset onto main"); - let tracked = capture(&work, &["rev-parse", "wt/bar@{upstream}"]); - assert_eq!(tracked, origin_sha, "the new local branch must track origin/wt/bar"); - } - - /// A branch already merged into `origin/main` and reachable only as a - /// stale `origin/wt/` (its GitHub head long deleted, this checkout - /// never fetched to notice) is refused rather than recreated behind main. - #[test] - fn a_landed_origin_only_branch_is_refused() { - let (dir, _origin, work) = crate::gitfixture::repo("wtresume-landed-origin"); - git(&work, &["checkout", "-q", "main"]); - git(&work, &["branch", "-qD", "wt"]); - git(&work, &["checkout", "-qb", "wt/bar", "main"]); - crate::gitfixture::commit(&work, "on-branch", "branch work\n", "branch work"); - git(&work, &["push", "-q", "-u", "origin", "wt/bar"]); - git(&work, &["checkout", "-q", "main"]); - git(&work, &["merge", "-q", "--ff-only", "wt/bar"]); - git(&work, &["branch", "-qD", "wt/bar"]); - git(&work, &["push", "-q", "origin", "main"]); - - let path = dir.join("resumed"); - let refused = std::panic::catch_unwind(|| { - create_worktree(&work, &path, "bar") - }); - - let panic = refused.expect_err("a landed origin-only branch must not be resumed"); - let message = panic.downcast::().expect("the refusal is formatted"); - assert!(message.contains("origin/wt/bar"), "{message}"); - assert!(!path.exists(), "nothing must be created before the refusal"); - } - - #[test] - fn with_neither_branch_it_starts_fresh_from_main() { - let (dir, _origin, work) = crate::gitfixture::repo("wtresume-fresh"); - git(&work, &["checkout", "-q", "main"]); - git(&work, &["branch", "-qD", "wt"]); - crate::gitfixture::commit(&work, "on-main", "main moved on\n", "main moved on"); - - let path = dir.join("resumed"); - let summary = create_worktree(&work, &path, "baz"); - - assert!(summary.contains("new, from main"), "{summary}"); - let worktree_sha = capture(&path, &["rev-parse", "HEAD"]); - let main_sha = capture(&work, &["rev-parse", "main"]); - assert_eq!(worktree_sha, main_sha); - } - - /// A linked worktree of a fresh repository whose `.gitignore` names `target/`, - /// both in the directory that comes first. - fn linked(name: &str) -> (TempDir, PathBuf, PathBuf) { - let (dir, _origin, work) = crate::gitfixture::repo(name); - let tree = dir.join("linked"); - git(&work, &["worktree", "add", "-q", "-b", "linked", tree.to_str().unwrap()]); - (dir, work, tree) - } - - /// **git unregisters, then fails to delete, and exits non-zero.** A path - /// deeper than `PATH_MAX` under the ignored `target/` is a deterministic - /// way to make it do that. - #[test] - fn a_worktree_git_unregistered_but_left_on_disk_is_deleted_whole() { - let (_dir, work, tree) = linked("wt-remove-leftovers"); - // Two chains of twenty, each short enough to make, one renamed into - // the other's end: no path any call here names exceeds `PATH_MAX`. - let chain = |at: &Path| { - let mut end = at.to_path_buf(); - for _ in 0..20 { - end.push("a-directory-name-thirty-bytes-"); - } - fs::create_dir_all(&end).unwrap(); - end - }; - let target = tree.join("target"); - let lower = chain(&tree.join("lower")); - fs::write(lower.join("f"), "cache\n").unwrap(); - let upper = chain(&target); - fs::rename(tree.join("lower"), upper.join("lower")).unwrap(); - let depth = upper.as_os_str().len() + lower.strip_prefix(&tree).unwrap().as_os_str().len(); - assert!(depth > 1024, "the fixture must exceed PATH_MAX to make git fail: {depth}"); - - remove(&work, tree.to_str().unwrap()); - - assert!(!tree.exists(), "{} is still on disk", tree.display()); - assert!(!registered(&work, &tree), "{} is still a worktree", tree.display()); - } - - /// git's own refusal stands: untracked work keeps the worktree registered, - /// and nothing of it is deleted. - #[test] - fn a_worktree_holding_untracked_work_is_refused_and_left_whole() { - let (_dir, work, tree) = linked("wt-remove-dirty"); - fs::write(tree.join("unsaved.rs"), "the only copy\n").unwrap(); - - let refused = std::panic::catch_unwind(|| remove(&work, tree.to_str().unwrap())); - - assert!(refused.is_err(), "a worktree with untracked work was removed"); - assert!(registered(&work, &tree), "the refusal unregistered it"); - assert_eq!(fs::read_to_string(tree.join("unsaved.rs")).unwrap(), "the only copy\n"); - } -} diff --git a/tests/CLAUDE.md b/tests/CLAUDE.md index d49800353d5..b5c879df965 100644 --- a/tests/CLAUDE.md +++ b/tests/CLAUDE.md @@ -4,15 +4,13 @@ The mechanics live where the work is: profiles and shapes in `tests/common/`, re ## Caveats that bite every agent -- **The dev host is a laptop that sleeps mid-session, and the suite says so** — a run whose wall clock jumped against the monotonic one reports `INVL` per test and exits 2: re-run. A wild outlier *not* marked that way is a real finding. +- **A host that sleeps mid-run is reported, not measured** — a run whose wall clock jumped against the monotonic one reports `INVL` per test and exits 2. A wild outlier *not* marked that way is a real finding. - **A machine-wide kernel panic reds whichever test was running** — that red's name is the workload, never the cause. `QEMU died before ===READY=== (status 0)` is the same thing said silently: a guest that reset itself is a kernel death, and its evidence is the boot log. -- **A machine-wide death during boot is reproduced by boots, not by suites** — parallel `bootable.img` guests, each waited on its completion marker and never on a fixed timer; the baseline is measured in the same session as the arm; a death counts whether or not a marker printed, so run guests with `-action reboot=shutdown -action shutdown=pause` and read a silent one's registers over QMP; a T14 block that gained a CI container mid-run is discarded, never corrected; a defect whose rate is set by interrupts per unit of guest work is measured on the *slowest* instrument — TCG can be the stronger oracle. +- **A machine-wide death during boot is reproduced by boots, not by suites** — parallel `bootable.img` guests, each waited on its completion marker and never on a fixed timer; the baseline is measured in the same session as the arm; a death counts whether or not a marker printed, so run guests with `-action reboot=shutdown -action shutdown=pause` and read a silent one's registers over QMP; a defect whose rate is set by interrupts per unit of guest work is measured on the *slowest* instrument — TCG can be the stronger oracle. - **`/system/bin/init` speaks in every program's name before that program runs** — a predicate keyed on a `: ` prefix is satisfied by the wrong speaker; wait for the whole line, in the constant the assertion also reads. - **A guest binary cannot ask what a handle it does not hold does** — the probe ends its caller with exit 139, so it runs in a child, one fault per child. - **A boot's capture has two pieces** — `boot_log()` ends at the ready marker, `run_test`'s capture begins at `===TEST_START===`. -- **Every guest this host boots is TCG** — anything vendor-dependent is gated only by CI's KVM lane, and TCG prices an uncontended atomic read-modify-write unlike hardware. -- **CI's `guest` lane is GitHub-hosted, never the T14** — the T14 is the orchestrator's own metal loop (`src/metal.rs`), reached by nothing in `.github/workflows/`. -- **The dev host's guests boot `-cpu qemu64`, which has no PCID** — every `INVPCID` path is dead locally, so a change gated on a CPUID feature is unverified by a green local suite. +- **A guest of an architecture the host is not runs under TCG** (`Arch::accel`) — TCG prices an uncontended atomic read-modify-write unlike hardware, and an x86-64 guest there boots `qemu64`, which has no PCID: every `INVPCID` path is dead, so a change gated on a CPUID feature is unverified by a green TCG suite. - **A liveness ceiling scales by two host facts** — boot-derived host speed *and* the guest's own `vcpus/cores` oversubscription. Widen a *liveness* guard for this, never a correctness bound. - **A wedge verdict needs both the budget spent and the guest gone quiet** — a healthy idle guest can be silent for minutes, and a guest still talking past its budget is slow, not stuck; only a far backstop stands behind a guest that keeps talking. - **A measured bound is asserted against the derivation, never against the measurement** — a bound that has to be widened to pass is a finding. A test asserting a kernel `Budget` never expires asserts a bound the kernel does not promise; the red is only the outcome that is neither the answer nor the declared degradation. diff --git a/tests/common/compile.rs b/tests/common/compile.rs index 1ee15db5ac7..8d018a3bcf0 100644 --- a/tests/common/compile.rs +++ b/tests/common/compile.rs @@ -4,6 +4,7 @@ use std::sync::OnceLock; use std::fs; use toyos_build::clang::CSysroot; +use toyos_build::sysroot::Sysroot; /// Root of the repository. pub fn repo_root() -> PathBuf { @@ -17,15 +18,14 @@ pub fn testcases_dir() -> PathBuf { /// The C sysroot every C program here is built against, and the clang that /// builds it: the toolchain's own, for the suite's architecture. Once per -/// process — a hundred and fifty C programs build against it. +/// process, and held in use by the static until the process ends. pub fn c_sysroot() -> CSysroot { - static C: OnceLock = OnceLock::new(); - C.get_or_init(|| { + static SYSROOT: OnceLock = OnceLock::new(); + let sysroot = SYSROOT.get_or_init(|| { let mut lock = toyos_build::buildlock::shared(&repo_root(), "the C sysroot"); - let sysroot = toyos_build::toolchain::ensure(&repo_root(), &mut lock); - CSysroot::of(&sysroot.dir, super::qemu::SUITE_ARCH) - }) - .clone() + toyos_build::toolchain::ensure(&repo_root(), &mut lock) + }); + CSysroot::of(sysroot.dir(), super::qemu::SUITE_ARCH) } /// The one flag the corpus is compiled with beyond the target and sysroot: diff --git a/userland/CLAUDE.md b/userland/CLAUDE.md index 019d6d86070..4c3d9860e33 100644 --- a/userland/CLAUDE.md +++ b/userland/CLAUDE.md @@ -1,10 +1,8 @@ # Userland -The module header at the site owns its subject — surfaces, translators and the channel in `toyos/`'s surface modules, soundd whole under `userland/soundd/`, what a process holds and how it got it at `kernel/src/object/` and `/system/bin/init`. The compositor's decisions are `toyos-desktop/`, pure and host-tested; `userland/compositor/` is devices, handles, shared memory and the panel. POSIX lives in `userland/libc` — ours, not a fork; that layer may be ugly, the kernel may not. +The module header at the site owns its subject — surfaces, translators and the channel in `toyos/`'s surface modules, soundd whole under `userland/soundd/`, what a process holds and how it got it at `kernel/src/object/` and `/system/bin/init`. The compositor's decisions are `toyos-desktop/`, pure and host-tested; `userland/compositor/` is devices, handles, shared memory and the panel. -**An app builds and runs on Linux under Wayland (never X11, which is legacy), macOS and Windows from the same source as on ToyOS**: a host build that fails is fixed in the app or its dependencies, never by making the app ToyOS-only. A program is ToyOS-only by nature when its job exists only on ToyOS: it owns ToyOS devices or kernel objects, or it manages ToyOS itself, as `update` does. Its manifest names which, and why (`src/userlandhost.rs`). - -**A server never blocks on a client** — the doctrine no single site owns. Accept and the first frame are two events; a frame is buffered until whole before anything acts on it; a write is one `try_send` whose refusal drops the peer by name; a blocking read or write of a pipe the client owns is the same bug. init, the compositor, netd, soundd and every surface host use `ipc::FrameRx`. filepicker violates it today. +**A server never blocks on a client** — the doctrine no single site owns. Accept and the first frame are two events; a frame is buffered until whole before anything acts on it; a write is one `try_send` whose refusal drops the peer by name; a blocking read or write of a pipe the client owns is the same bug. init, the compositor, netd, soundd and every surface host use `ipc::FrameRx`. ## Caveats that bite every agent