Skip to content

fix(gate): a Python string prefix defeated every agent-memory destination match - #93

Merged
garymike merged 1 commit into
mainfrom
fix/memory-dest-string-prefix
Oct 30, 2026
Merged

garymike merged 1 commit into
mainfrom
fix/memory-dest-string-prefix

Conversation

@garymike

Copy link
Copy Markdown
Owner

dev-exec-writes-agent-memory required the quote to follow open( immediately, so every Python string prefix walked straight through the gate.

Found because NVIDIA/SkillSpector#893 was sent back in review for the same class of bug in its own analyzer. Its cause was different (it stripped the backslashes; we rejected the prefix), so pointing that finding at our own rule is what surfaced ours. Same method that produced the 1.7.1 fixes: run the upstream boundary case against our gate, not just against theirs.

Measured before this change

Ten cases wrong: eight false negatives, two false positives.

case before after
open(r"MEMORY.md", "a") no finding blocked
open(r"C:\Users\alice\MEMORY.md", "a") no finding blocked
open(rf"{base}\MEMORY.md", "a") no finding blocked
open(f"{base}/MEMORY.md", "a") no finding blocked
open(R"MEMORY.md", "a") no finding blocked
Path(r"C:\Users\MEMORY.md").write_text(x) no finding blocked
open(r".claude\settings.json", "w") no finding blocked
appendFileSync(".claude\settings.json", x) no finding blocked
open("memory.md.bak", "a") blocked quiet
appendFileSync("MY_AGENTS.md", x) blocked quiet

Three causes

The string prefix. open( had to be followed immediately by the quote. This was never about backslashes on our side: open("C:\Users\MEMORY.md", "a") already matched and still does. A raw string is the natural way to write a Windows path, because it is the spelling that does not need every separator doubled, so the uncovered spelling was the likely one in a real carrier.

Windows separators. The agent config directories were spelled .claude/ with a hardcoded forward slash. The same rule covers the memory files and the config directories, so the gap applied to both.

Near-miss filenames. The basename matched anywhere inside the literal. memory.md.bak is a backup that nothing loads into a session, and MY_AGENTS.md is a team roster. A basename now has to start the literal or follow a separator. Upstream lists memory.md.bak as a quiet case in its own tests for the same reason.

Verification

A 34-case matrix, run against the rule as parsed back out of the YAML rather than from the version in the editor:

  • the verbatim source lines of all four existing memory fixtures, so nothing previously covered regressed
  • every one of the eight gaps
  • the shapes already covered (w mode, ./ prefix, escaped backslashes, write_text, write_bytes, fs.promises, createWriteStream, template literals, shell redirects)
  • eleven false-positive controls, including the arrow-function read that 1.7.1 fixed, docs/claude/notes.md (no leading dot), and a bare assignment with no sink

0 mismatches. Then the full proof against a freshly built image carrying the new rule:

PROOF-FIXTURE PASSED
16 assertions, 16 PASS, 0 FAIL

Assertion 4's table shows real scores rather than blanks, which is how you can tell the containers actually ran:

  gecko-demo               64/100    exit 1, BLOCKS
  gecko-hook-demo          76/100    exit 1, BLOCKS
  config-injection-demo    100/100   exit 1, BLOCKS
  sibling-config-demo      51/100    exit 1, BLOCKS
  memory-poisoning-demo    0/100     exit 0, does NOT block

Added

Four fixtures and four assertions, taking gate-proof.sh from 12 checks to 16. Three block (memory-poisoning-raw-path-demo, memory-poisoning-fstring-demo, memory-poisoning-win-config-demo) and one must not (memory-near-miss-benign). Each of the three defects has a fixture that fails without its fix.

Scope

The documented residual in docs/threats/developer-execution.md is now stated more precisely: the line is the basename, not the path. A computed directory with a literal basename is covered. An assembled or indirect basename, where no literal filename appears at the call site, still passes and is still recorded as a limit rather than asserted as expected behaviour.

…tion match

`dev-exec-writes-agent-memory` required the quote to follow `open(` immediately, so
open(r"C:\Users\alice\MEMORY.md", "a") produced no finding, and neither did rf"", f"" or
R"". A raw string is the natural spelling for a Windows path, so the uncovered spelling was
the likely one rather than an exotic one.

This was never a backslash problem here: open("C:\Users\MEMORY.md", "a") already matched.
The prefix was the whole cause. NVIDIA/SkillSpector#893 shipped the same class of bug in its
own analyzer, where the cause was stripping the backslashes rather than rejecting the prefix,
and being sent back in review for it is what prompted measuring ours. Pointing an upstream
finding at our own gate is the method that produced the 1.7.1 fixes too.

Two more defects measured at the same time:

  The agent config directories were spelled with a hardcoded forward slash, so
  .claude\settings.json was uncovered on the one platform where a backslash is the normal
  separator. All three branches now accept either.

  A basename matched anywhere inside the string literal, so memory.md.bak (a backup nothing
  loads into a session) and MY_AGENTS.md (a team roster) both blocked. A basename now has to
  start the literal or follow a separator. Upstream lists memory.md.bak as a quiet case in
  its own tests for the same reason.

Ten cases measured as wrong before this change: eight false negatives and two false
positives. Verified against a 34-case matrix that includes the four existing fixtures'
verbatim source lines, so none of the previously covered shapes regressed, and eleven
false-positive controls including the arrow-function read that 1.7.1 fixed.

gate-proof.sh goes from 12 assertions to 16. Each of the three defects now has a fixture
that fails without its fix.
@garymike
garymike merged commit 774ab22 into main Oct 30, 2026
9 checks passed
@garymike
garymike deleted the fix/memory-dest-string-prefix branch October 30, 2026 18:31
garymike added a commit that referenced this pull request Oct 30, 2026
… never passed (#94)

#88 compared each reusable-workflow image pin against the digest the build had just
published. It failed on 4 of 4 runs after it landed, including the 2026-10-26 SCHEDULED
rebuild where nothing in the repo had changed and all six image jobs passed.

The premise was mine and it was wrong: a rebuild with unchanged inputs does not produce a
byte-identical digest. The Dockerfiles apt-get install current packages, so a rebuild days
apart publishes a new digest. The check could never stay green past the next rebuild, and
because the weekly cron rebuilds on its own, "pins behind" stopped carrying information.
That is the cry-wolf signal this repo refuses elsewhere.

So it now compares something that only moves when a human moves it. build-toolbox.yml
stamps every image it publishes with a `git ls-files -s` fingerprint of that image's build
inputs: toolbox/base, the image's own context, and toolbox/tools.lock. The check reads that
label back off the PINNED image and compares it with the repo's inputs now. Four small
registry calls per image, resolving the attestation-bearing index to its one real platform
manifest and reading only the config blob, so no layers are pulled.

Verified against the live registry before shipping: the chain resolves for all five images,
and the shell fingerprint in the workflow and the Python one in the script agree on all five.

More precise, too. #93 changed only toolbox/skill-audit/rules/, which under the old check
turned all five pins red; under this one only skill-audit.yml is affected.

A pinned image with no stamp is reported UNVERIFIABLE rather than passing, so this cannot
report success on absent input, which is the failure mode it replaces.

Moved out of build-toolbox.yml into its own workflow. A red build should mean the images are
broken; the pins being behind is a different fact and should read as one, the same separation
tool-drift.yml already uses. Not run on pull_request on purpose: the digest to pin does not
exist until the publish has happened, so a PR that bumps a tool could never be green, and a
check that cannot pass on the PR that causes it is one people learn to ignore.

Also corrects the reproducibility claim in both cosign sign steps. Tolerating a Rekor 409 is
still right, but a rebuild is byte-identical only when it hits the buildx cache, not whenever
its inputs are unchanged.
garymike added a commit that referenced this pull request Oct 30, 2026
…blished (#95)

The second half of #91 and #93, and the first pins that scripts/check-image-pins.py can
actually verify: 02c62e4 is the first build that stamps each image with a fingerprint of its
build inputs, so before this every pin reported UNVERIFIABLE.

Resolved from each image's `sha-02c62e46ac16` tag rather than `latest`. Measured reason: the
#91 and #93 builds started 25 seconds apart and ran concurrently, both pushing `latest`, and
for iac-toolbox the earlier run's push landed last. `latest` resolved to 7f5292725111 while
the 774ab22 build had published c1d133bb0e97. The per-commit tag is race-free. build-toolbox.yml
has no concurrency guard, which is a separate thing to fix.

Each digest was verified before being written: its build-inputs label was read back out of the
registry and compared with the fingerprint this repo computes now, and nothing would have been
written if any of the five disagreed. scripts/check-image-pins.py then reports all five current.
garymike added a commit that referenced this pull request Oct 30, 2026
… at once (#98)

build-toolbox.yml had no concurrency guard, and that was not a theoretical gap: the #91 and
#93 builds ran 25 seconds apart, both pushed `latest`, and for iac-toolbox the EARLIER run's
push landed last. `latest` named the older build while the newer digest existed only under
its `sha-` tag.

The group omits github.ref deliberately. The registry tags are global, so a
workflow_dispatch from another branch has to queue behind main rather than get its own lane.

cancel-in-progress is false, matching tool-bump.yml and for the same reason. These jobs push
and sign six images; killing a run mid-flight can leave the base published and the five
domain images not, which is a worse state than a slower queue. Nothing in flight is
interrupted, runs serialize, and the newest commit's run finishes last so it wins `latest`.
GitHub keeps one pending run per group, so a burst of pushes collapses to the one running
plus the newest, which is the right pair to keep.

Found while moving the image pins, by comparing what the registry served against what the
build log said it published, rather than trusting either on its own. Nothing broke because of
it: scripts/check-image-pins.py and the pin mover both resolve the per-commit `sha-` tag
precisely so a raced `latest` cannot be pinned. `latest` is still what a human pulling the
image by tag gets, which is why this is worth fixing rather than routing around.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant