Repository navigation
fork fleet: seed verb and seeds in the fleet config (D11) - #236
scuttletoby wants to merge 7 commits into
Conversation
442e00e to
01b61e4
Compare
cb91fdd to
91abd6e
Compare
01b61e4 to
0781efb
Compare
91abd6e to
f877ff9
Compare
0781efb to
ca286df
Compare
A desk pool may list warm caches ("seeds") in its fleet config entry. The
always-on host's forced command answers a desktop's `seeds` (the pool's list)
and `seed NAME` (that one ready-made tar, with a size and SHA-256 trailer) in
an ssh session beside the agent's. Only names of the list, never a path;
exit 3 while another stream for the pool runs, 4 when the file may not be
sent. The desktop's fetcher follows in the next change.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
f877ff9 to
f18fbda
Compare
ca286df to
57e1cb5
Compare
|
Review round 1 at 57e1cb5: two independent lenses (security, correctness), both SAFE WITH NITS. Nothing serves a path the desktop picks; the existing
State question: runs again (read-only, lock reused), killed mid-stream (flock dies, no trailer), broken pipe (exit 1, files closed), missing/broken config (exit 2), pool not in config (exit 2), no |
- seeds live in FILES/POOL/NAME.tar: pools never share a file - a file dated more than five minutes in the future is refused (exit 4) - the stream stops at the size that was checked; ctime and inode join the changed-file check - the forced command sends every command that starts with `seed` to the seed handler, so `seeds x` and `seed` exit 2 and never reach a session - a desktop that takes nothing for STALL seconds is cut off (exit 1) and the pool's lock let go - the lock file must be this user's; a clear refusal otherwise Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ool (D11) The lock check after a stall sent 1 MB into a pipe nobody read until the call returned, and the second pool reused the first one's label and beacon, which the config check refuses. The pool test also puts a tar at the old shared path, so the old code fails it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Fix round 1 (commits 7026afd and 099f438, the second a test-only fix; D10's round 1 merged in; head 099f438). Tests: desk-seed 12 OK, desk 42 OK.
Fails before the fix: the seed tests run against the round-0 code give 10 failures and 1 error (every test above). Some fail there partly because the per-pool path moved the tar; PR body updated. Still draft. |
|
Re-check lens 236s r1 (security) at 099f438: SAFE WITH NITS Earlier findings: 1 pools sharing a seed FIXED; 2 future-dated file FIXED, but its test passes on round-0 (round-0 never reads New:
def test_a_rename_while_sent_is_fine(self):
old = self.make('m2', {'a': b'x' * 3000}); send = seed.send
def renaming(fd, out, size):
self.make('m2', {'a': b'y' * 3000})
return send(fd, out, size)
with mock.patch.object(seed, 'send', renaming):
code, out, err = self.offer('seed m2')
self.assertEqual((code, out), (0, old), err)
Checked safe: exact-key parsing of |
|
Re-check lens 236c r1 (correctness) at 099f438: SAFE WITH NITS Earlier findings 1-7 FIXED (round-0 seed tests: 10 failures + 1 error; each fix also removed in a copy to see its test catch it). Exceptions: the future-date case passes on round-0 too (round-0 never reads the per-pool path), though it does catch removing the check at head; the lock-owner check ( New:
State question: rerun (lock reused |
…) (D11) - A seed replaced by rename while it streams is sent whole: when the open file has no links left (st_nlink 0) it was replaced, not rewritten, so its ctime is not compared; size and mtime still are. The st_ino compare (fstat of the same fd, it never differed) is gone. - The stall wait restarts per 64 KiB piece written, not per 1 MiB, and a stream has an overall deadline: 600 s plus one second per 256 KiB. - max_mb is at most 16384 (fork-fleet.py SEED_MAX_MB). - Tests: the lens rename test, the lock-owner refusal, every refused file must say its own reason (the future-dated case now fails on round-0's path), the deadline, and a slow but steady reader that is kept. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Fix round 2 at 3fb8617 (D10 round 2 merged in first) Lens 236s r1 (security):
Lens 236c r1 (correctness):
State question:
Local run (round 2):
Red-check: the new seed tests run against round-1 code give 3 failures and 2 errors. |
Row D11 of the desktop pool plan, part 1 of 2: the host side of seeds, the warm caches for a desk pool's jobs. It adds
seedsto a pool's fleet config entry and aseeds/seed NAMErequest on the desk link's forced command. Stacked on #230 (D10). Critical: it adds a second request to the desk link's forced command.The D11 row came to 518 non-test lines, over the 400 limit, so it is split:
Why
A job's copy starts empty, so every job on a desktop downloads its tools and dependencies again. That makes it slow and wastes the PC's network. With seeds, the always-on host offers ready-made caches and the desktop keeps a recent copy that its jobs start from.
pools[].seedsis a list of{"name", "every_hours", "max_age_hours", "max_mb"}.check_configrefuses bad names, values that are not positive whole numbers (bools too),max_age_hoursbelowevery_hours,max_mbabove 16384, missing fields and duplicate names.fork-fleet-desk-seed.py(standard library only) answers two requests.seedsprints the pool's list as one JSON object: for each seed,bytesandmadewhen it is offered now, andwhywhen it is not.seed NAMEstreams/var/lib/fork-fleet/seed-files/POOL/NAME.tar(a folder per pool: two pools never share afile), followed by a trailer line on stderr:
seed NAME BYTES SHA256 MADE.fork-fleet-desk.py serve POOLalready runs as the desk-link user for the desk key. WhenSSH_ORIGINAL_COMMANDstarts withseed(soseeds,seed NAME, and alsoseeds x,seed,seedsx), it hands the request to the seed module instead of serving a session; the module refuses anything butseedsandseed NAMEwith exit 2. Every other request keeps the old path.Safety (critical)
seedslist is served. The name never becomes part of a path unless it is on that list, and names match[a-z0-9_-](the POOL pattern).seed ../m2,seed /etc/passwd,seed M2,seed m2 x,seeds xand the like all exit 2 and send nothing (tested, also through the real forced command:seeds x,seed,seedsxandseedsexit 2 and make no socket). A command that does not start withseedkeeps the old path (a session request).O_NOFOLLOW. It must be a plain file that no group or other user may write, in a pool folder and a files folder (not links) that no group or other user may write, at mostmax_mb, younger thanmax_age_hours, and not dated more than 5 minutes in the future. Otherwise it exits 4 and sends nothing. The folders' owner is not checked, only their mode (documented).st_nlink0), so only its ctime (changed by the unlink) is not compared, and the old whole file is sent (tested). The trailer's SHA-256 is of exactly the bytes sent.POOL.seed.lock, undersafe_dir). A second stream exits 3, so a misbehaving desktop cannot open many. A desktop that takes less than 64 KiB in 120 s, or not the whole file within 600 s plus one second per 256 KiB, is cut off (exit 1), so it cannot hold the lock for long. A lock file that cannot be opened or that another user owns is a clear refusal (exit 1, tested).What happens when this runs again, stops half-way, or meets existing, old or empty state?
flock, so it goes with the process, and the next stream gets it (tested).max_age_hoursis not offered. A file replaced by rename while being sent is fine, because the open fd still reads the old whole file. A file written in place is refused. A fleet config withoutseedsmeans an empty list. Older desktops that never ask are unaffected.why: not made yetin the list, andseed NAMEexits 4. A pool with no seeds lists[].General vs project
General. The verb, the config check and the offer name no project. Which caches a pool has, and how the host makes each tar, are the project's own config and its own timer. The doc example (
gradle) is only an example.Upstream fit
Only the fork-only
.github/scripts/fork-fleet*files,.github/fork-fleet.mdand the fork CI test list change. Nothing in upstream T3 changes.Tests
.github/scripts/fork-fleet-desk-seed.test.py(new, in.github/fork-ci-tests.txt). It uses temp folders only: no root, no ssh, no network.max_mbat its cap and 11 bad ones.seeds,seed NAMEand every otherseed-prefixed command. It serves no session for them.The existing
fork-fleet-desk.test.pyandfork-fleet.test.pywere run too.Local run (round 2):
.github/scripts/fork-fleet-desk-seed.test.py: 16 OK.github/scripts/fork-fleet-desk.test.py: 42 OK.github/scripts/fork-fleet.test.py: 201 OK (7 skipped)seed/seedsdivert, the stall deadline). Some of these fail on the round-0 code only because the per-pool path changed where the test puts the tar.max_mbcap). Mutations of the round-2 code: without the future-date check, without the lock-owner check, and with round-0's path (no pool folder), the matching tests fail (the future-dated case now says why it fails: "not made yet" instead of "dated in the future").Red-checks
Each rule was broken in a copy, one at a time, and the named test failed (all 11 red):
test_other_commands_and_names_are_refused_and_send_nothingO_NOFOLLOW) ->test_a_file_that_may_not_be_sent_is_exit_4max_age_hoursis not sent -> the same testtest_a_file_changed_while_sent_is_exit_4test_a_second_stream_for_the_pool_is_exit_3test_a_lock_of_another_user_is_refusedtest_a_file_that_may_not_be_sent_is_exit_4test_never_as_root_and_a_bad_config_is_exit_2max_age_hoursat leastevery_hours->Configtest_the_forced_command_diverts_seed_requestsThe first run found one gap: with
O_NOFOLLOWremoved the link case still passed, because the link's target wasgroup-writable and refused for that reason. The test now makes the target 0644, so only the link can be the reason.
🤖 Generated with Claude Code