Skip to content

feat(map): stage 1b, each shipped build's own index and three-valued reachability - #1051

Merged
cryptskii merged 1 commit into
mainfrom
feat/code-map-conformance
Sep 29, 2026
Merged

cryptskii merged 1 commit into
mainfrom
feat/code-map-conformance

Conversation

@cryptskii

Copy link
Copy Markdown
Collaborator

What

This is stage 1b of the code map. Each shipped build now gets its own real index, and every definition is reached, dead or indeterminate per build, with a stable reason code and the source of every fact. Stage 1a landed in #1046.

More chunks are coming on this PR:

  • Stage 1c: the adversarial verification layer (fixtures, mutation harness, contradiction checks, sentinels).
  • The intent layer: specs/requirements/INTENT_MANIFEST.tsv and a comparator that derives an action from intended versus actual state, per the owner's rulings of 2026-09-28.
  • Stages 2–4: conformance checks with BLAKE3 pins on manifest rows, canonical evidence references, and the code-to-spec report.

Profiles are committed configs under ci/. They are hashed into the tree fingerprint together with the lockfile, the toolchain and the analyzer version.

Profile Target Built
android the real aarch64-linux-android, with the NDK at the Gradle pins and jni,bluetooth everywhere
node x86_64-unknown-linux-gnu Linux only (CI). Elsewhere its items read IND_PROFILE_NOT_BUILT, never a Darwin stand-in
tests host, cfg(test) everywhere

Reachability is computed per artifact, over that artifact's edges only.

  • Reached: a path of proven edges leads from an entry point. A trait-dispatch or self-type step counts only with evidence that the impl can run:
    • a method taking self needs its Self type constructed in reached code;
    • an associated function needs the type dispatched on (a type argument, a qualified path, a path call).
  • Indeterminate: only an uncertain step leads there. The codes are IND_DISPATCH_SELF_TYPE, IND_NOT_CONSTRUCTED, IND_AMBIGUOUS_TRAIT, IND_AMBIGUOUS_SELF_TYPE, IND_UNLINKED_IMPL, IND_MACRO_GROUP, IND_SCOPE_UNDECIDED, IND_UNRESOLVED_CALL, IND_MACRO_BODY, IND_UNINDEXED_ARTIFACT, IND_CFG_UNDECIDED, IND_UNREAD_DECLARATION and IND_PROFILE_NOT_BUILT.
  • Dead: DEAD_NO_ROOT_PATH.
  • Not compiled into the build: NOT_IN_ARTIFACT.

Uncertainty never becomes an edge.

Entry points follow one rule. An exported symbol is a root only when a Kotlin external fun declares it.

  • JNI_OnLoad and JNI_OnUnload are VM roots, ctor/dtor are load roots, and a shipped binary's main is a main root.
  • An export nothing declares is a dead-root candidate.
  • The Kotlin reader follows JNI naming: nested and companion classes, @JvmStatic, @JvmName, @file:JvmName/JvmPackageName, the file facade, generic and extension natives, and qualified and use-site annotations.
  • A declaration whose symbol it cannot spell is kept with the prefix of every symbol it could be. Examples: an internal member's module suffix, a multifile part class, a Java native. Any export it could be is IND_UNREAD_DECLARATION, never a root or a dead root.
  • Every declaration the Android library does not export is named. Today there are five, all anchor natives that live in the unindexed crates/dsm-android-anchor.

Source is read as Rust, parsed with syn. Tokens are scanned only where nothing parses. This covers:

  • function extents and receivers, and impl headers;
  • how each name in a path is used: a pattern names, a literal or unit value constructs, and a turbofish or qualified self type dispatches;
  • calls and macro heads;
  • what a cfg covers: the attributed node itself, doc comments included, with cfg_attr read as any(not(C), …), nested ones included.

A macro's tokens are parsed as items, expressions or statements. Tokens that parse as none of them, such as macro_rules! bodies, are read token by token, and only in the direction that adds uncertainty. Each such invocation is listed in token-read.tsv with its parse errors: 72 on Android, 76 in tests.

Nothing is silently dropped or merged:

  • Every definition occurrence is exactly one of: a node, a repeat, a local, a module or a parameter. A check refuses any remainder.
  • A symbol that several definitions share stays several nodes, resolved by Rust's scoping. Two module-level definitions with one symbol stop the map.
  • #[async_trait] extents are repaired from the parsed source. Extents that still cross, or that share an extent outside a macro invocation, stop the map.
  • Every ERROR/WARN line of every analyzer log is a known kind. Any of these stops the map: a build-script failure, a proc-macro failure, a config error, an unexplained unplaceable definition, or an unknown line.
  • Assembly, #[link_section], and cfgs set by build scripts stop the map.

Found and fixed by this stage

  • Test-only code counted as production. dsm_sdk's dev-dependency on itself turned test-utils on in the "production" index, so stage 1a counted economic_fixtures and other test-only code as production. Each build's real features now come from cargo tree, and whatever a differing feature's cfg turns off is taken out of the build: 39 definitions on Android.
  • Test-only functions read Dead. Three of them read Dead instead of not-in-build, because the gate's extent started at the cfg line while the definition starts at its doc comment: reset_sdk_context_for_testing, SdkContext::reset_for_testing and CoreSDK::forget_process_startup_admissions_for_testing.
  • False construction evidence. Match-arm patterns (Self::Invalid(_) =>) were read as constructing the type, which could have established a dispatch. 1,895 such edges are gone. Unit enum variants used as values (Err(MarketLegRefusal::NotTransferable)) were missed, and 982 edges are added.
  • Calls from attributes. Calls were fabricated from inner attributes (#![…]) and are removed.

No reachability state moved as a result of the last three; the evidence under the states is now true.

Accounting (this Mac; CI adds the node profile)

Profile Definition occurrences Nodes Locals Modules Collisions Repaired extents Macro groups Unplaceable
android 45,126 12,417 32,222 487 8 16 1 113
tests 63,164 16,400 45,973 791 11 16 4 144
Android Count
entry points 63 (61 declared JNI exports, JNI_OnLoad, one ctor)
dead-root candidates 7 (six Bitcoin exports no Kotlin declares, and dsm_init_runtime)
not compiled into the build (test-utils) 39
reached / indeterminate / dead 8,627 / 40 / 1,544
unresolved call tokens in reached code 240, none of which names a workspace candidate
macro-body / anchor-crate candidates 0 / 2
Kotlin declarations read / unread / not exported 66 / 0 / 5 (all in the unindexed anchor crate)

Analyzer logs, per profile (android / tests):

Kind Count
path-search limits (display only) 2,314
unnamed enclosing modules 272 / 430
duplicate-symbol reports 118 / 121
self dev-dependency 1
unplaceable: derive 74 / 89
unplaceable: include! 19 / 37
unplaceable: macro invocation 3 / 1
unplaceable: shared test module 17 / 17
build-script failures, proc-macro failures, config errors 0

Queries for agents

python3 ci/requirement_map.py --map target/requirement-map explain <symbol | Rust path | file>
python3 ci/requirement_map.py --map target/requirement-map impact  <symbol | Rust path | file>

Verification

  • cargo test -p requirement_map --release: 81 passed. Each new rule is covered by a test with a constructed input, and each fix in the last rounds was mutation-checked by removing it and watching its test go red.
  • make requirement-map-fixture: all 12 readings as expected. This is the end-to-end dispatch fixture through the real rust-analyzer pipeline.
  • make requirement-map on the full tree.
  • make lint, ci/production_safety_checks.sh and scripts/real_code_guard.py all pass.
  • The Gemini push gate is satisfied, after 14 review rounds.

Not in this chunk

  • Stage 1c. Adversarial fixtures across receivers, UFCS, trait objects, local impls, macros, collisions, FFI roots and exclusions; the mutation harness, including two real-tree cases; contradiction checks; sentinels; and the adversarial CI stage.
  • The intent manifest and comparator.
  • Stages 2–4.

…reachability

The code map now indexes each shipped build as it is built: the Android
library for the real aarch64-linux-android target with the NDK Gradle
pins, the storage node for Linux (in CI), and the host test build, all
with release cfgs (rust-analyzer's default debug_assertions and miri
off). Every definition is reached, dead or indeterminate in each build
separately, with a stable reason code and the source of every fact;
reachability is never computed over two builds' edges together.

- a trait impl or an outside-trait impl (Drop, Display, From) runs only
  with evidence: a method taking self needs a value of its Self type
  constructed in reached code; an associated function needs the type
  dispatched on (a type argument, a qualified path, a path call, a
  value). A type only named constructs and dispatches nothing
- uncertainty (an ambiguous type or trait, an unexpanded macro body, an
  unresolved call, the unindexed anchor crate, a node profile not built
  here) is Indeterminate, never an edge
- entry points: exports a Kotlin `external fun` declares, JNI_OnLoad and
  JNI_OnUnload, load-time constructors, a binary's main; an export
  nothing declares is a dead-root candidate. The Kotlin reader follows
  JNI naming (nested and companion classes, @JvmStatic, @JvmName,
  @file:JvmName, the file facade class, generic and extension natives,
  qualified annotations); every declaration the Android library does
  not export is named, with where a function of that name is written
  (today: the five anchor natives, in the unindexed anchor crate)
- every definition occurrence is accounted for; symbols several
  definitions share stay several nodes, resolved by scope; async_trait
  extents are repaired from the source, which is parsed as Rust (syn):
  function extents and receivers, impl headers, how each name in a path
  is used (a pattern names, a literal or unit value constructs, a
  turbofish or qualified self type dispatches), calls, macro heads;
  a macro's tokens that parse as no Rust are read token by token only to
  add uncertainty, and counted in token-read.tsv; impl headers inside functions
  declare, not use; crossing extents, module-level collisions,
  unexplained analyzer log lines, asm, link sections and build-script
  cfgs stop the map
- dsm_sdk's dev-dependency on itself turned test-utils on in the index:
  each build's real features are read from cargo tree, and what a
  differing feature's cfg turns off is taken out of the build. What a
  cfg covers is read by parsing the file as Rust (syn): the node the
  attribute is written on, doc comments included; a cfg a macro writes
  whose items do not parse is undecided
- an end-to-end fixture (make requirement-map-fixture) pins static
  dispatch through a type argument and a qualified path, instance
  dispatch on a constructed value, and a type only named
- `explain` and `impact` queries for agents
@cryptskii
cryptskii merged commit 632e238 into main Sep 29, 2026
25 of 26 checks passed
cryptskii added a commit that referenced this pull request Sep 29, 2026
CI sets CARGO_TERM_COLOR=always, so cargo tree wrapped its `(*)` repeat
marker in terminal escape codes and the feature reader refused the line
(Code map job on #1051). Every cargo tree the map runs now asks for
`--color never`, and a line holding escape codes is refused by name,
saying so.

Reproduced locally with CARGO_TERM_COLOR=always (314 colored lines);
with --color never the output is byte-identical to the uncolored run,
and the full map builds with unchanged readings.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant