Skip to content

T017: A dir with a TAB or a backslash is listed, indexed and reached as recorded, a stored sub-field reads back as written, and request-handoff refuses a host or container the fence rejects (opensoft/openRepoTools#301) (part of opensoft/openRepoTools#186) - #4

Merged
brettheap merged 1 commit into
mainfrom
fix/subfield-misreads-301
Nov 8, 2026

Conversation

@brettheap

@brettheap brettheap commented Nov 7, 2026 •

Copy link
Copy Markdown
Contributor

What this is

This is the T017 continuation, in this leg (opensoft/openRepoTools-code), of the opensoft/openRepoTools branch fix/subfield-misreads-301 at f68030f466ad, for opensoft/openRepoTools#301 (part of opensoft/openRepoTools#186). The original tree and branch stay as the archived original (lane 4's decision, opensoft/openRepoTools#186 comment 6548343086). One commit here, a04c937f1b96, carries it.

It carries that branch's own change against e0970a8, the head of fix/dir-space-listing-295 it is stacked on (so against the previous PR of this chain), for the code leg's paths: lanes-edit.sh (+147 −3), lane (+30), lanes (+30) and the new tests/test_subfield_misreads_301.py (+1084). Its one other path, docs/README-lanes.md (+1 −1, the lanes row of the reads table, which names the escape on column 7), is the spec leg's, in the paired PR. lane-start and lane-handoff have no diff.

What was wrong and what changed

What was wrong

Three ways a stored lane sub-field was misread. A TAB in a recorded dir shifted the listing's tab-separated row, so the lane dropped out of worktrees --all and the inventory index. payload_subfield passed its payload through awk -v, which expands backslash escapes, so a stored \t, \\, \n, \101 or \" read back as something else. And request-handoff wrote the requesting host's by host sub-field unquoted and unfenced, so my host read back as my and was typed into a session's pane.

Found by the independent review of the opensoft/openRepoTools#295 fix, which listed the three under "older problems this branch does not touch"; the third was first raised by that branch's writer. The opensoft/openRepoTools#295 fix causes none of them and makes none worse. This change is stacked on it.

  1. A TAB in a recorded dir shifted the listing's row. lanes_rows prints the dir into column 7 of the seventeen-column, TAB-separated row, and every reader splits that row with a tab-IFS read. A TAB inside the dir was one separator more: the row had 18 columns, and each column after the dir arrived one place early. In a sandbox estate (mawk 1.3.4) the lane left worktrees --all and the inventory index with no UNREAD and no PARTIAL (a sync upserted 2 trees instead of 3), worktrees <lane> said 8, and the lanes command showed the lane's objects as its AGE and printed a false DEFECT: ... live fork(s) of this lane transcript, because the fork-count column then held the lane's home. lane's go_available took R_DIR cut at the TAB, which may be a different existing directory.
  2. payload_subfield expanded backslash escapes. It handed the payload to awk -v pay=..., and awk -v processes escapes in the value it is given. A stored dir of back\tslash read back holding a TAB, \\ as one backslash, \n as a newline that cut the answer in two, \101 and \x41 as A, \" as ". Every read of a lane's sub-fields goes through it: lane-dir answered a path that does not exist, and the writers filed the lane's lifecycle under $PROJECTS_ROOT/.lane-state or, for oct\101x, in a sibling directory that belongs to another checkout. worktrees --all still listed those lanes, because it made the same wrong pass as the writers. A host spelled a\tb also read back with a TAB.
  3. request-handoff wrote its host unquoted and unfenced. With LANES_HOST='my host' it wrote and pushed by host my host; container none; window none; wait 0s, which the bound session's guard reads back with payload_subfield, up to its first space, as my, and types into that session's pane in /handoff --exit requested by .... A ;, a " or a control byte would have been typed into the pane too. The forced release wrote forced by <uuid>@<h>/<container> the same way, and its container was just as unfenced: LANES_HOST=plainhost LANES_CONTAINER='a b' with request-handoff ... --force exited 0 and wrote ...@plainhost/a b, where the request path refused the same container, and only at the write. write_event's binding_subfields_check fences the sub-fields named host, os and container, and neither of these is one.

On the real register (brett-wip e27b539, read on 2026-10-22), no recorded dir holds a backslash, a TAB or a space, and there is one by host, Eagle. Nothing there moves under any fix below.

What changed

  • Item 1, a TAB in a dir (lane 3's ruling 1, option C with its additions).
    • row_field_escape in lanes-edit.sh writes a value with pesc's four escapes, the alphabet lane-worktrees escapes a porcelain field with: a backslash as \\, a TAB as \t, a newline as \n, a carriage return as \r. It is an awk loop over the value's bytes read through ENVIRON under LC_ALL=C, because awk -v would process escapes and Apple's awk's gsub missed non-ASCII bytes. If the encoder prints nothing, column 7 is none and the listing says that the dir could not be escaped.
    • row_field_unescape_into decodes left to right in the shell and keeps a backslash before anything else. lanes-edit.sh, lane and lanes each carry one (lane and lanes are lane 3's own files, and run under set -euo pipefail; the decoder is pure shell and runs no utility through command).
    • lanes_rows escapes column 7 at the print only, and only where the dir holds one of the four bytes, so the --dir filter still compares the stored dir and an ordinary dir costs no process. worktree_rows decodes it before it hands the dir on, lane's split_row decodes it so that go_available tests and enters the directory the log records, and the lanes renderer decodes it before it shows it.
    • The manual names the escape in the lanes row of the reads table (the paired spec PR).
  • Item 2, backslash escapes. Only payload_subfield changes. The payload goes in as PSF_PAY and is read as ENVIRON["PSF_PAY"], as lane_sidecar_schema_ok already reads its id. want and mode stay on -v: they are the callers' own words. The function stays self-contained, because the test cuts it out of the file and runs it alone. Its callers' answers change with it where a stored value holds a backslash, and only there: lane-dir, lane-profile, lane-agent, lane-transcript, both control roots, lane_reconcile, session_start_block, lane_state_follow, holder_is_dead and guard_run read the stored text; lane-start and lane-handoff read the recorded dir as written, with no source change of their own (accepted in lane 3's ruling 5). binding_subfields_add, binding_subfields_check and pause_subfields_check judge the text that is written rather than its expansion, so host a\101b is no longer passed as aAb: stricter, as Brett Heap accepted with ruling 3.
  • The root move. A lane whose recorded dir holds an escape awk -v expanded now finds its control root beside its checkout. Its records filed under the old root, $PROJECTS_ROOT/.lane-state or a sibling directory, stay where they are, unread, with no migration (Brett Heap's ruling 3, "Accept, no migration"; no recorded dir on the register holds a backslash). lanes-index sync holds such a lane's rows while the generation read at the new root is below the stored one (in sync only, and only where the stored generation is at least 1; not in reconcile). A test names, and does not fix, the old-root restart intent that lane_restart_control_root does not find.
  • Item 3, the requesting host and container (lane 3's ruling 2, and its answer that the rule holds on both paths). The request-handoff arm holds LANES_HOST_NAME and LANES_CONTAINER_NAME to lanes_binding_value_ok, the fence binding_subfields_add holds the binding's own host to, once, after the binding is read and before the --force branch. A value that fails it is refused with exit 2 and "Nothing was written" on the request path, on the --force path and on either one's --dry-run. The refusal says where the value came from (LANES_HOST_SOURCE or LANES_CONTAINER_SOURCE: the LANES_HOST or LANES_CONTAINER seam, the workstation name inside a container, hostname -s, or the container's own hostname) and names the setting to put right. Outside every container the container value is none, which the fence takes. Only the request-handoff arm changes: no lock function and no call site of one. The cost: a host or a container outside letters, digits, ., _, - and : can never ask.

Not in this PR

Old to new: the receipt

The original's own non-merge commits in opensoft/openRepoTools (e0970a8..f68030f), 10 in all, are carried by one commit in this leg, a04c937f1b96. The list is the commit's own body, and it equals the original's non-merge commits read from the chain's refs/root/ ref (checked).

773a562bccd1 Add the tests that a recorded dir holding a TAB or a backslash, and a requesting host holding a space, are read back as they were written (#301)
dd361f3839cf The #301 tests split the listings they read on a newline alone, not on every line boundary Python knows (#301)
430ac8f0804b The lane listing escapes a lane's directory in its seventh column as lane-worktrees' porcelain escapes a field, and lanes-edit.sh, lane and lanes decode it, so a TAB in a recorded dir no longer shifts the row (#301)
647f4511b210 request-handoff refuses a requesting host that lanes_binding_value_ok rejects, on the request path and the --force path alike, before anything is planned or written (#301)
ae14da06a53f payload_subfield hands the payload to awk through ENVIRON instead of awk -v, so a stored sub-field holding a backslash reads back as it was written (#301)
c0491f1db6f6 Pin what item 2 moves: lane-start reaches a backslash lane's own checkout, lane-handoff writes its dir back byte for byte, and its lifecycle starts again at the new root while the index holds the old rows (#301)
567035fec28b request-handoff refuses a container that lanes_binding_value_ok rejects, on the request path and the --force path alike, before anything is planned or written (#301)
27253fe78a10 The #301 tests pin a carriage return in a recorded dir and the decoders' pass-through arm, and the lanes_rows contract comment says column 7 is escaped (#301)
e1d26e135cec The #301 tests now pin the fallback for a dir the encoder cannot escape, where a refused name came from, the free-lane answer before a bad host, and a container of the fence's alphabet (#301)
f68030f466ad The #301 tests now pin a capital letter in the binding fence's whole-alphabet row and pin that a free lane's request leaves the published register where it was (#301)

The subjects are the original's, byte for byte; a bare (#N) in them names the root issue.

Resolutions

None. The patch applied clean, with no offset.

The pairing

The manual's part of this change (docs/README-lanes.md, +1 −1) is the commit 454f062c13a5 on fix/subfield-misreads-301 in opensoft/openRepoTools-spec, its own PR, opensoft/openRepoTools-spec#7 (base main). The code leg's checks read the manual through OPENREPOTOOLS_SPEC_ROOT: tests/test_repo_hygiene.py reads docs/README-lanes.md (call sites at lines 1373, 1404, 1423, 3139 and 3572 of the code leg's main; 1404 is the tuple of surfaces that a loop reads through doc()) and tests/test_test_roots.py pins which manual a run reads. The code leg's own CI (OPENREPOTOOLS_COMPOSED: '0', no spec leg beside the checkout) skips those checks, so they bind only in a composed run or with the variable set to a spec clone. A search of the manual lines this change touches for the strings those checks pin (the rung-count heading, the two settlement quotes, every lane on this workstation, the exit-code section's two markers and lane-end <lane> --retire) finds none: read, not run. So the two PRs are one change in two repositories, and they land together, as a pair. The code PR goes first, so that the manual never describes behaviour the code lacks: this PR first, then its spec PR, opensoft/openRepoTools-spec#7. The state that order passes through, the code chain's tip beside the spec leg's main, passed -k test_repo_hygiene in the independent review (run (e) under Tests: 223 passed, 14 skipped; that run read the chains' tips, not this commit alone). The other order's state, the spec tip beside the code leg's main, was not run. I found no check that turns either order red; that is a reading, not a run.

Tests

Run by the CODE writer, in a sandbox clone, with env -i (HOME, XDG_* and TMPDIR inside the sandbox), OPENREPOTOOLS_CODE_ROOT set and OPENREPOTOOLS_COMPOSED=0, as the leg's CI sets it. Each targeted run read this branch's own commit in its own worktree; that commit was 8acd8f66374c before the message amend, and the amend kept its tree, so the results stand for a04c937f1b96.

run exit result time
bash -n on lanes-edit.sh, lane and lanes, at every commit of the chain 0 all ok -
py_compile on every changed .py file and on lanes-index 0 all ok -
run.sh -k test_subfield_misreads_301 0 118 passed 68.01 s

shellcheck, over the whole chain (main to the tip): errors 0 to 0 and warnings 108 to 108 (lane 14 to 14, lanes 8 to 8); notes up by 3, 2 and 2, all SC1003 on the literal '\' patterns of opensoft/openRepoTools#301's commit.

The pair runs below are of the two chains' tips, so they cover this commit with every later commit of its chain; apart from the targeted run above, no run was made at this commit alone. They are lane 4's independent paired review, 2026-10-31 (the letters are the review report's own), with OPENREPOTOOLS_SPEC_ROOT set to the spec clone shown, env -i, TMPDIR=/tmp and a 54-character XDG_STATE_HOME.

run code + spec exit counts time
(a) -k test_repo_hygiene code tip 5f2995d43893 + spec tip 8104f2cf965c 0 223 passed, 14 skipped, 2944 deselected 11.1 s (33 s wall)
(b) -k test_repo_hygiene, baseline code main 2a057a6 + spec main 9f8b21a 0 223 passed, 14 skipped, 2772 deselected 9.2 s (10 s wall)
(c) -k lane_helpers code tip + spec tip 0 42 passed, 6 skipped; 0 FAIL in the shell suite (the wrapper asserts " passed, 0 failed"), with a 54-character state root 2188.6 s (36:28)
(e) -k test_repo_hygiene code tip 5f2995d43893 + spec main 9f8b21a 0 223 passed, 14 skipped 5.5 s

The pair runs read the chains' tips as they were before the message amend (code 5f2995d43893, spec 8104f2cf965c). The amend kept every tree (git diff from each old commit to its new one is empty), so the runs stand for the amended tips, code 2c15aa6613c3 and spec 430ea20784ac. No failure is new at the tips. Run (d), the manual-reading tests at each branch, was skipped under the review's brief, because none of the five branches' test files reads the manual.

The CODE writer's first run of -k lane_helpers at the chain's tip, with a 212-character run root, gave 5 FAILs in the shell suite (4870 ok, 5 FAIL, 3 skipped), because rc_tree_id in tests/test_lane_helpers.sh lacks the 180-character cap that tree_id_for in lanes-edit.sh has, so at that length the two names differ and the planted records are never read; the chain changes neither, and run (c) above, with the 54-character state root, gave 0 FAIL.

Not run: the suite at the leg's main for comparison, bash 3.2, macOS and the assembly's composed check; the submodule was not initialized, so 6 tests skip.

Test files this commit adds or changes:

  • tests/test_subfield_misreads_301.py, new (+1084): 118 cases from 23 test functions at the root head, on an estate of eight recorded dirs (a plain one, one with a space, one with a TAB, one with a carriage return, and four with a backslash: \t, \\, \n and an octal \101). Item 1: a row keeps its seventeen columns, column 7 is the dir escaped as pesc escapes a field, lanes shows each dir as recorded with no DEFECT line, lane <name> --dry-run reaches each dir, and the listing, the named read and the inventory index keep every lane's trees; a round trip of 226 values runs the encoder against each decoder. Item 2: lane-dir and the writers read each backslash lane as recorded, payload_subfield alone gives back what was stored, and three pins for ruling 5 and the root move. Item 3: a host or container the fence refuses is never written, on all four invocations, with the setting named; the whole alphabet is written whole.

The original's own record, from the root repository, at the root heads and not repeated in this leg: the file was red at e0970a8 (23 failed and 35 passed for phase 1's 58 cases), 118 of 118 at f68030f by lane 3's direct calls, and 355 passed for -k "subfield_misreads_301 or repo_hygiene" through tests/run.sh. Of lane 4's 20 scripted mutants, 18 were killed; guard-drop-nl is an equivalent, and esc-no-lcall is unkillable on that workstation's mawk (see Not in this PR).

Release notes (v1.2.0)

Proposed by lane 4's writer, for lane 3's 785(b) ("the release-notes line should name the stricter gates Brett accepted, as ae14da0's message does"). Not established: whether lane 3 accepts the line; it has seen only a two-bullet summary of it, not the line itself. Lane 3 ruled (opensoft/openRepoTools#301 comment 6352342484) that each line goes in the PR description under "Release notes (v1.2.0)" and that no docs/release-notes/v1.2.0.md is created.

A lane whose recorded directory holds a TAB or a backslash is now listed, indexed and reached as recorded.

  • The listing's directory column is escaped the way lane-worktrees escapes a porcelain field, and decoded by lanes, lane and worktrees.
  • Every read of a stored sub-field gives back what was written, because payload_subfield no longer passes it through awk -v.
  • Root move: such a lane's lifecycle and trees are now looked for beside its checkout. Records the old read filed under $PROJECTS_ROOT/.lane-state or a sibling directory stay there unread, with no migration, since no recorded dir on the register holds a backslash.
  • Stricter gates: binding_subfields_add, binding_subfields_check and pause_subfields_check now judge the text that is written rather than its awk -v expansion, so host a\101b is no longer passed as aAb (accepted by Brett Heap with ruling 3, "Accept, no migration"). request-handoff refuses a requesting host or container outside letters, digits, ., _, - and :, on the request and --force paths and on either one's --dry-run, with exit 2 and "Nothing was written" (lane 3's ruling 2). (Three ways a stored lane sub-field is misread: a TAB in a recorded dir drops the lane from worktrees --all, payload_subfield expands backslash escapes through awk -v, and request-handoff writes its by host sub-field unquoted openRepoTools#301)

The line is here, in the code PR of the pair, and only here: the paired spec PR, opensoft/openRepoTools-spec#7, repeats none.

Review

READY, lane 4's independent paired review of both chains, 2026-10-31. It found no MUST and no SHOULD against the commits, and three NITs. Report sha256: 4ac9e5441affc6aeac48e6d73419580f6786440dc0e5dd32722cc40b8070a682. The review read the chains before the message amend (tips 5f2995d43893 and 8104f2cf965c, the same trees as the amended tips).

The original's recorded review, from the root repository; the comment ids are copied from the root draft, which lists them as the comments of opensoft/openRepoTools#301, and were not re-read (no network):

Landing

It lands only on Brett Heap's word, PR by PR; nothing in this text is a landing.

Lane: openRepoTools-4
Claim: opensoft/openRepoTools#301 (comment 6349895849)

🤖 Generated with Claude Code

Summary by Sourcery

Ensure lane metadata with special characters is preserved end to end and prevent unsafe handoff identities from being written.

Bug Fixes:

  • Preserve recorded directories containing tabs, newlines, carriage returns, or backslashes across lane listings, indexing, path resolution, and lifecycle operations.
  • Read stored sub-fields literally without expanding backslash escape sequences.
  • Reject request-handoff hosts and containers that violate the binding-value character fence before planning or writing.

Enhancements:

  • Keep legacy control records at their existing roots without migration while moving future lifecycle reads to the recorded checkout root for affected lanes.

Documentation:

  • Document the escaping used for the lane listing directory column.

Tests:

  • Add comprehensive regression coverage for directory escaping, literal sub-field reads, root movement, and host/container validation across request and forced handoff paths.

…ndexed and reached as recorded, a stored sub-field reads back as it was written, and request-handoff refuses a host or container the binding fence rejects

T017 continuation, in opensoft/openRepoTools-code, of the
opensoft/openRepoTools branch fix/subfield-misreads-301 at
f68030f466ad49c81f02edf8f21a58423a7f8173 (issue
opensoft/openRepoTools#301). It carries that branch's own change
against e0970a8, the head of fix/dir-space-listing-295 it is stacked
on, for the code leg's paths: lanes-edit.sh, lane, lanes and
tests/test_subfield_misreads_301.py. Its one other path,
docs/README-lanes.md (+1 -1, the lanes row of the reads table naming
column 7's escape), belongs to the spec leg and is not in this commit.
lanes_rows escapes column 7 as lane-worktrees' porcelain escapes a
field and lanes-edit.sh, lane and lanes decode it; payload_subfield
reads its payload through ENVIRON instead of awk -v; and
request-handoff refuses a host or container that
lanes_binding_value_ok rejects.

The subjects below are quoted from the originals; a bare #N in them is
opensoft/openRepoTools#N.

Old-to-new receipt (the original's own non-merge commits):
773a562bccd1 Add the tests that a recorded dir holding a TAB or a backslash, and a requesting host holding a space, are read back as they were written (#301)
dd361f3839cf The #301 tests split the listings they read on a newline alone, not on every line boundary Python knows (#301)
430ac8f0804b The lane listing escapes a lane's directory in its seventh column as lane-worktrees' porcelain escapes a field, and lanes-edit.sh, lane and lanes decode it, so a TAB in a recorded dir no longer shifts the row (#301)
647f4511b210 request-handoff refuses a requesting host that lanes_binding_value_ok rejects, on the request path and the --force path alike, before anything is planned or written (#301)
ae14da06a53f payload_subfield hands the payload to awk through ENVIRON instead of awk -v, so a stored sub-field holding a backslash reads back as it was written (#301)
c0491f1db6f6 Pin what item 2 moves: lane-start reaches a backslash lane's own checkout, lane-handoff writes its dir back byte for byte, and its lifecycle starts again at the new root while the index holds the old rows (#301)
567035fec28b request-handoff refuses a container that lanes_binding_value_ok rejects, on the request path and the --force path alike, before anything is planned or written (#301)
27253fe78a10 The #301 tests pin a carriage return in a recorded dir and the decoders' pass-through arm, and the lanes_rows contract comment says column 7 is escaped (#301)
e1d26e135cec The #301 tests now pin the fallback for a dir the encoder cannot escape, where a refused name came from, the free-lane answer before a bad host, and a container of the fence's alphabet (#301)
f68030f466ad The #301 tests now pin a capital letter in the binding fence's whole-alphabet row and pin that a free lane's request leaves the published register where it was (#301)

Resolutions: none. The patch applied clean, with no offset.

Lane: openRepoTools-4
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@sourcery-ai

sourcery-ai Bot commented Nov 7, 2026

Copy link
Copy Markdown

Reviewer's Guide

API

Escape directory values in listing column 7 and decode them in each row consumer, preserving the tab-separated interface when paths contain control characters.

  • lane L507-535, L544
  • lanes L108-136, L713
  • lanes-edit.sh L4293-4353, L7968-7979, L8800-8817 (+1 small edits)

Pass stored payloads to awk through the environment so sub-field reads preserve literal backslashes.

  • lanes-edit.sh L6796-6808

Refuse handoff requests and forced releases when the requesting host or container fails the binding-value fence, before planning or writing.

  • lanes-edit.sh L17270-17309

Tests

Add regression coverage for escaped listing fields, exact sub-field reads, lifecycle root behavior, and rejected handoff host and container values.

  • tests/test_subfield_misreads_301.py L1-1084

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@sonarqubecloud

sonarqubecloud Bot commented Nov 7, 2026

Copy link
Copy Markdown

@brettheap
brettheap changed the base branch from fix/dir-space-listing-295 to main November 8, 2026 13:42
@brettheap
brettheap marked this pull request as ready for review November 8, 2026 13:42
@sourcery-ai

sourcery-ai Bot commented Nov 8, 2026

Copy link
Copy Markdown

Sorry @brettheap, you've used your own review budget of 250,000 diff characters for the last 7 days.

You can request another review in 6 days and 7 hours by commenting @sourcery-ai review. Upgrade to get a review now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant