Skip to content

feat(agents): shared AX target and result contract with a read-only inspector - #346

Merged
roodboi merged 5 commits into
nextfrom
wf/f17-ax-contracts-b16
Oct 20, 2026
Merged

roodboi merged 5 commits into
nextfrom
wf/f17-ax-contracts-b16

Conversation

@roodboi

@roodboi roodboi commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Linear: HACK-1144 (WU31, shared AX target and result contracts).

This adds a provisional target/result decision record and a read-only target inspector that behaves the same through the CLI and MCP.

  • CLI: hack project target [--project=…|--path=…] [--branch=…] [--service=…] [--env=…] [--expect-generation=…] [--json] resolves the exact project, checkout (primary, linked or non-Git), runtime instance (explicit, worktree-derived or default, plus its Compose project name), optional service, env overlay selection and a Hack config generation. It reads local files only. It does not write the registry (not even the last-seen touch), call the engine or Docker, start or repair anything, or change Git. It reports runtime: { observed: false, reason: "not_probed" } and authority: "none".
  • MCP: hack.project.target (read-only) spawns the same command and passes projectName/path through without resolving them first. Every resolution refusal therefore comes from the CLI. data is the CLI envelope, re-validated against the contract: off-contract output becomes null and foreign fields are dropped. stdout is withheld because data carries the same envelope.
  • Refusals: a closed catalogue of 15 reasons. Each has a fixed, value-free message, a stage and a retry disposition, and reuses an existing HACK_ERROR_CODES code, so no new code or guidance render is needed. --expect-generation refuses stale config with observed_generation. If the config changes during the read, the inspector refuses instead of returning a mixed snapshot.
  • Malformed Compose file: a Compose file that is not YAML, whose top level is not a mapping, or whose services is not a mapping is refused as compose_parse_failed (E_CONFIG_PARSE, stage project, retry fix_project). It is no longer reported as a target with no services, and --service no longer turns it into service_not_found. An absent Compose file or an empty services: key still gives services: [].
  • Instance slug: an explicit --branch is slugged like hack up --branch, so Feature/X gives <base>--feature-x.
  • Generation: a fingerprint of .hack/hack.config.json, hack.toml and docker-compose.yml only. Ordinary source edits do not change it.
  • Equal names: registration ownership is matched by exact root or a recorded linked worktree, never by name. Selecting a name that two registrations share is refused as ambiguous_project, and so is a checkout that two registrations own.
  • The Compose project name resolver moved from src/commands/project.ts to src/lib/compose-project-name.ts, so the inspector and the commands share one implementation. Behavior is unchanged.
  • Docs: docs/reference/ax-target-result-contract.md (decision record, schema, refusal table, compatibility matrix, known gaps), a hack.project.target section and row in docs/reference/mcp-tools.md, and a regenerated docs/reference/cli.md.

All names, fields, reasons and retry values are marked provisional for the compatibility review.

Rework after verification (head bc49c11a)

  • Added the compose_parse_failed refusal and documented it, together with a "Compose validity" known gap: the inspector does not run docker compose config, so a file that parses but that Compose would reject is still reported.
  • Added paired CLI/MCP fixtures for the paths no test covered before: env selection (project default, explicit overlay, base), explicit instance slugging, two registrations owning one checkout, config_parse_failed, compose_parse_failed (with and without --service, and a list-valued services) and generation_input_too_large.
  • Added a stubbed-CLI MCP fixture: a second MCP server whose HACK_MCP_COMMAND prints a stored envelope. It proves the tool re-validates the payload instead of passing it on. Added direct validator tests for the code/message consistency check.
  • Moved the buildEnvExplainArgs JSDoc back above its function in src/mcp/server.ts.

Validation

  • bun test tests/project-target-inspect.test.ts: 18 pass. Each paired case runs the source CLI directly and through the MCP tool, whose child is the same source CLI via HACK_MCP_COMMAND. Both use privateCliEnv, a private HOME/HACK_HOME/registry and an isolated Docker client config. The envelopes must be equal apart from observed_at. Cases:
    • two projects, by session cwd and by name
    • a linked worktree's derived instance (alpha--feature-login), an explicit instance overriding it with the Git branch unchanged, and Feature/X slugged to alpha--feature-x
    • env selection: project default overlay, explicit overlay and base
    • conflicting selectors
    • stale generation, and a source edit that keeps the generation
    • ambiguous equal names, two registrations owning one checkout, a moved (deleted) registered root, an unknown project, a detached worktree, no project, a missing path and an unknown service
    • a malformed Hack config, a malformed or list-valued Compose file, and an oversized Compose file
    • malformed selectors: CLI invalid_selector, MCP invalid_arguments with no spawn
    • no-write checks: registry bytes and mtime, state dir contents, .hack contents and the Git worktree list are unchanged
    • MCP re-validation against a stubbed CLI: drifted data becomes null, foreign fields are dropped, and a mismatched message becomes null
    • domain: config changed during the read, selector validation, value-free messages, and validator drift rejection (including code/message consistency)
  • Required and related suites: tests/integration-ownership.test.ts, tests/cli-spawn-env-isolation.test.ts, tests/mcp-inspect-tools.test.ts, tests/agent-instruction-source.test.ts, tests/cli-help.test.ts and tests/experimental-gating.test.ts (78 pass together with the focused file). tests/mcp.test.ts: 11 pass.
  • bun run --cwd packages/cli typecheck: clean. bun run --cwd packages/cli check: 0 errors. The 34 warnings it reports are in existing files. ultracite check on the changed files reports nothing.
  • bun run docs:cli-reference: no drift (the CLI surface did not change in the rework). bun run privacy:check: ok. No Rust changed.

Negative controls (mutation proofs)

Each mutation was applied alone and then reverted. The focused suite failed every time.

First round:

Mutation Failing test
Drop the --project/--path conflict check conflicting selectors
MCP drops path when projectName is set (pre-resolution) conflicting selectors (parity)
Skip the expected-generation comparison stale generation
Skip the generation re-read config change during the read
Ignore an explicit instance explicit instance wins
Include a source file in the generation source edit keeps generation
Validator stops checking stage/retry drifted envelope rejection
Match registrations by name 7 paired cases
Accept undeclared services closed refusals

Rework (all 12 killed):

Mutation Failing test
resolveEnvSelection returns base for an explicit overlay env selection
resolveEnvSelection ignores the project default env selection
Delete the --env argv entry from buildTargetInspectArgs env selection
Use the raw explicit branch instead of sanitizeBranchSlug explicit instance slug
Disable the two-owner check (candidates.length > 1) two registrations own one checkout
Disable the config_parse_failed refusal malformed or oversized inputs
Raise the generation input bound to 64 MiB malformed or oversized inputs
Validator drops the code check validator drift rejection
Validator drops the message check validator drift rejection, MCP re-validation
MCP returns parseJson(stdout) without re-validation MCP re-validation
A Compose YAML error returns no services malformed or oversized inputs
A non-mapping services returns no services malformed or oversized inputs

Remaining gaps (host-bound or out of scope)

  • No runtime readback. The snapshot never observes containers, so it does not prove an instance exists or is healthy. Container and env readback for two projects and two branches needs a host with an engine and is not covered.
  • Not exercised: concurrent session isolation, slow or failed probes, unknown provenance and capability-limited clients. The inspector has no runtime probe yet.
  • Env: selection only. Overlay existence is not checked.
  • Compose validity: only YAML/mapping shape is checked; docker compose config is not run.
  • Generation scope: worktree-local overrides and env files are excluded. Native hack.project.json projects are refused as project_inputs_unsupported.
  • The CLI parser's shared --project/--env grammar refusals stay E_USAGE with no detail.reason, as for every command.
  • Hosted CI has not reported on the new head yet.
  • WU20, WU21 and WU28 runtime and AX gates remain open. This draft does not claim compatibility-review sign-off.

Rework 2

Fixes the three open verifier findings from round 1.

  • Registry no-write proof (should-fix). inspection writes nothing now compares projects.json against the bytes and mtime that beforeAll wrote, not a baseline read inside the test after earlier paired cases had already run the CLI. Mutant: inject touchProjectRegistration({ project: context }) after selectProject in resolveTarget. Round 1: survived. Now: killed (registry text differs).

  • Unreadable inputs leaked OS text and paths (blocking). A filesystem fault while reading inputs used to escape inspectTarget as E_UNEXPECTED with the raw message, for example EACCES: permission denied, open '<abs path>/.hack/docker-compose.yml'. It is now one of two closed, value-free refusals:

    • input_permission_denied (E_PERMISSION, project, fix_project) for EACCES/EPERM
    • input_unreadable (E_CONFIG_INVALID, project, fix_project) for any other errno error. New repro found in this round: when .hack/docker-compose.yml is a directory, the generation read treats it as absent, and the Compose read then threw EISDIR.

    The adapter validator now accepts a detail-less error only when its code is E_USAGE (the parser refusals). Any other detail-less error, such as an E_UNEXPECTED raised by a defect, becomes data: null. The contract doc and mcp-tools.md now say so.

  • Validator data checks untested (minor). New tests mutate a real inspector success envelope: authority set to something other than "none" or removed, runtime.observed: true, a wrong runtime.reason, and runtime: null. The authority case is also checked through the stubbed MCP CLI.

New paired CLI/MCP fixtures: dir-compose, which is deterministic and works as root, and locked-compose (mode 000; skipped when running as root, because root can read the file). Both also assert that the CLI envelope contains neither the fixture root nor the errno name.

Mutations, all killed by the named test:

Mutation Failing test
touchProjectRegistration after project selection inspection writes nothing
Drop the filesystem-fault mapping unreadable input, permission-denied input
Map EACCES to input_unreadable permission-denied input
Accept a detail-less error for any code parser E_USAGE only, drifted CLI output
Drop authority === "none" claims authority or readiness, drifted CLI output
Drop runtime.observed === false claims authority or readiness

Gates run locally on head 5352415d:

  • bun run typecheck and bun run check pass, as do bun run --cwd packages/cli typecheck and check.
  • gitleaks 8.30.1 over origin/next..HEAD finds no leaks.
  • The target-inspect, mcp and env-command-matrix suites pass: 37 pass, 0 fail.
  • Full bun run test (7369 tests, run on a host under load average 17–20) had 28 failures in 18 files. Serial reruns with --timeout 60000 passed in 10 of those files. The other 8 files are host-native tests (native-authored-*, native-compose-adoption-*, native-compose-proxy-probe, native-compose-retained-storage-command, native-compose-storage-activation-command), and they keep failing serially. The same native failures reproduce on the unmodified base a3eb34e9 (checked: proxy-probe, retained-storage-command, process-incarnation). They are pre-existing host-environment failures, not caused by this change.
  • There are no Rust changes.

Still open:

  • A non-filesystem defect inside the inspector still reaches the CLI as E_UNEXPECTED with free text. MCP returns data: null for it. Mapping every thrown value to a closed refusal would hide defects, so it was not done.
  • An errno error from a read-only git child, if one were thrown, would also be reported as input_unreadable.
  • The gaps listed under "Remaining gaps" above are unchanged.

Rework 2: independent re-check (head 5352415d, no new commits)

A second rework pass started from the current remote head, 5352415d. All three round-1 findings are already fixed there, so this pass pushes nothing.

  • bun test tests/project-target-inspect.test.ts: 22 pass, 0 fail. This includes the permission-denied case, which runs because the host user is not root.
  • Every mutant was re-run after the Biome reflow, and each was killed by the named test:
    • touchProjectRegistration after project selection: "inspection writes nothing"
    • filesystem-fault mapping removed: unreadable input, permission-denied input
    • EACCES mapped to input_unreadable: permission-denied input
    • detail-less error accepted for any code: parser E_USAGE only, drifted CLI output
    • authority === "none" removed: authority/readiness, drifted CLI output
    • runtime.observed === false removed: authority/readiness
    • runtime.reason === "not_probed" removed: authority/readiness
  • The argv-helper gap (the direct CLI run reuses buildTargetInspectArgs) does not hide a dropped flag. Removing any one of --project, --path, --branch, --service, --env or --expect-generation from the helper fails 1 to 8 paired tests through their per-field assertions.
  • Hosted CI on 5352415d: all 16 checks pass, including test, secret-scan and every docker-e2e shard.
  • The branch is behind next (099877a0). It was not updated, because force-pushing and merging next were both outside this pass.

Open (not fixed; each needs a product decision):

  • Silent registry degradation. When projects.json is present but unreadable (mode 000) or is not valid JSON, the shared lenient reader returns an empty registry. --path then reports registration: null, and --project refuses with project_not_found. Lifecycle commands read the registry the same way, so the answer matches what they would do. The caller just cannot tell this case from a project that is genuinely unregistered. The options are a closed refusal, an explicit registration state, or keeping parity with the commands.
  • Retry advice for transient errno. Transient errno errors (EMFILE, EAGAIN, EIO, EINTR) map to input_unreadable, whose retry is fix_project. For these errors, reinspect is the more accurate advice.

Rework 2: re-confirmation on merged head 8395b342 (no new commits)

The branch now includes next (b431f0e7, merged by the main session). That merge does not touch the inspector, its tests or its docs.

  • Focused and related suites (project-target-inspect, env-command-matrix, mcp, mcp-inspect-tools, cli-spawn-env-isolation, integration-ownership): 69 pass, 0 fail. packages/cli typecheck and check: 0 errors. docs:cli-reference: no drift. gitleaks 8.30.1 over the origin/next..HEAD patches: no leaks.
  • The six round-1 mutants were re-run on 5352415d, whose inspector files are identical on 8395b342 (registry touch, detail-less pass-through for any code, authority check removed, runtime.observed check removed, filesystem-fault mapping removed, EACCES mapped to input_unreadable). Each one was killed by the named test.
  • The two open items above (silent registry degradation and retry advice for transient errno) are unchanged. Both still need a product decision.

🤖 Generated with Claude Code

roodboi and others added 2 commits October 19, 2026 11:57
…nspector

Add a provisional target/result decision record and a read-only target
inspector with CLI and MCP parity (HACK-1144, WU31).

- `hack project target [--json]` resolves the exact project, checkout,
  runtime instance, optional service, env overlay selection and a Hack
  config generation from local files only. It performs no registry write,
  engine call or Git mutation, and reports runtime as not probed.
- `hack.project.target` (read-only MCP tool) spawns the same command and
  passes selection through unresolved, so every resolution refusal is the
  CLI's. Its data is the CLI envelope re-validated against the contract.
- Refusals are a closed, value-free catalogue (reason, stage, retry) that
  reuses existing error codes. `--expect-generation` refuses stale config;
  a config change during the read is refused instead of snapshotted.
- Extract the Compose project name resolver from commands/project.ts so
  the inspector and commands share one implementation.
- Paired CLI/MCP fixtures: two projects, a linked-worktree branch and an
  explicit instance, conflicting selectors, stale generation, ambiguous
  equal names, a moved checkout, detached worktree and no-write checks.

Names, fields and reasons are provisional pending compatibility review.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…n gaps

The read-only target inspector now refuses a Compose file that is not YAML,
whose top level is not a mapping, or whose services key is not a mapping, as
compose_parse_failed, instead of reporting a target with no services.

Paired CLI/MCP fixtures now cover env selection (project default, explicit
overlay, base), explicit instance slugging (Feature/X -> <base>--feature-x),
two registrations owning one checkout, config_parse_failed,
compose_parse_failed and generation_input_too_large. A stubbed CLI proves the
MCP tool re-validates the envelope, and the validator's code/message
consistency check has direct tests. Restores the buildEnvExplainArgs JSDoc.

Refs HACK-1144

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@blacksmith-sh

This comment has been minimized.

roodboi and others added 3 commits October 19, 2026 13:04
The new read-only target inspector accepts --env and --service, so the env
matrix source check requires a row for it. It reports only the overlay
selection and never reads env files or values.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A filesystem fault while reading target inputs (a mode-000 or directory
Compose file) escaped as E_UNEXPECTED with the raw OS message and absolute
path. Map errno errors to the closed input_permission_denied and
input_unreadable refusals, and let the adapter validator pass a detail-less
error only for parser E_USAGE refusals.

Strengthen the no-registry-write proof by comparing against the registry
bytes written in beforeAll, and cover the validator's authority and
runtime.observed checks.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@roodboi
roodboi marked this pull request as ready for review October 20, 2026 17:59
@roodboi
roodboi merged commit 8c79000 into next Oct 20, 2026
16 checks passed
@roodboi
roodboi deleted the wf/f17-ax-contracts-b16 branch October 20, 2026 17:59
roodboi added a commit that referenced this pull request Oct 21, 2026
Brings #355 (G4, HACK-1272: retire authored starts refused before journal
reservation), #352, #341, #342 and #346 into the G1 closed-refusal branch.
Both sides' behavior is kept.

Conflict resolutions:

- docs/reference/native-authored-commands.md: keep G1's explanation of
  `pending` (in-progress start or failed start with retained evidence; only
  the owner's cleanup proof may retire it) and append G4's pointer that
  `down --recover` retires an attempt that never reserved runtime state when
  no startup is running.

- src/lib/native-authored-command.ts: the `pending` status note keeps G1's
  wording and adds G4's `hack down --recover` hint. Auto-merged code keeps
  G4's canceled `unreserved` exit 130 ahead of G1's stopped-runtime
  classification. Semantic fix: G1's stoppedRuntimeStartup message map had
  no entry for G4's new `unreserved` outcome, so a runtime_not_running start
  that was retired as unreserved would have produced an undefined message.
  It now reports E_DOCKER_UNAVAILABLE with an `unreserved` message (intent
  retired, `hack ps` reports not_started). G4's `down --recover` pending-start
  path already refuses through commandFailure as a closed coded HackCliError
  (E_LIFECYCLE_FAILED, values omitted), and unreserved startup failures map
  to E_STARTUP_INCOMPLETE or E_DOCKER_UNAVAILABLE, so no stack or help is
  printed.

- tests/native-authored-project-start.test.ts: the fixture keeps G1's
  `stoppedDuringServe` and G4's `failBeforeReceiptCode` and
  `unreservedInspection` options. The stub serve emits G4's configurable
  failBeforeReceipt code and G1's wrapped stopped-runtime refusal. The stub
  inspect answers both failure modes, with `native_graph_unreserved` only
  when requested. The duplicate native-runtime-client import is merged into
  one. Semantic fix: G4's HACK-1272 call-log expectation now includes G1's
  read-only `runtime status --json` preflight between plan and serve.
  Both sides' tests are kept.

- tests/native-authored-command.test.ts (auto-merged): add an `unreserved`
  case to the stopped-runtime startup table.

Verification (Bun 1.4.2, macOS):
- packages/cli typecheck and check: 0 errors.
- native-authored-command, native-refusal-cli,
  native-authored-command-dispatch and cli-spawn-env-isolation: 124 pass.
- Other native-authored-* and native-runtime-* suites: 409 pass, 5 fail.
  2 are load-timing failures that pass when rerun alone. 3 are host
  live-session census failures in the recovery and incarnation suites,
  which fail identically on origin/next and ead8383 on this host.
- native-authored-project-start (--timeout 60000): 48 pass. The 3
  host-metadata cases are intermittent (HACK-1299). They also fail
  intermittently on origin/next and ead8383 and pass when run alone.
- Six scripts/check-native-*-plan-cli.ts: all pass after build and
  build:config-compiler.
- docs:cli-reference and generate:agent-plugins: no drift.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant