Skip to content

Chained hops folded: mask-risc GatherChain, Quack Filter::Chain, glove out(r1).out(r2) (D-CMH-G1-7) - #1522

Closed
AdaWorldAPI wants to merge 5 commits into
mainfrom
ccr-9f3b8075-74u47l-chain2
Closed

AdaWorldAPI wants to merge 5 commits into
mainfrom
ccr-9f3b8075-74u47l-chain2

Conversation

@AdaWorldAPI

@AdaWorldAPI AdaWorldAPI commented Oct 12, 2026 •

Copy link
Copy Markdown
Owner

Depends on AdaWorldAPI/ndarray#358 (mask_gather_chain_u32[_under]). CI checks out ndarray master with no pin, so this PR's CI goes green only after #358 merges.

What

Chained hops are folded hops. out(r1).out(r2)… over functional relations becomes ONE op. Each hop is one more aperture in the same pass, and the only thing materialized is the op's destination over the anchor table's rows. No mask over any hop table exists, and no program feeds another.

mask-risc

  • New op MaskOp::GatherChain { lane, chain, under, dst }. The chain it names lives in the new field Program::chains: Vec<Vec<Hop>>.

  • Each Hop names a resident foreign plane (live), up to 8 HopTests read in place at the row the key reached, and a next key lane (present on every hop except the last).

  • The executor runs the op through ndarray::simd::mask_gather_chain_u32[_under].

  • The facade chain is built on the stack, bounded by MAX_CHAIN_HOPS/MAX_HOP_TESTS = 8, so L1 holds: execute still allocates nothing.

  • Validation runs before the facade and refuses:

    • a missing chain (ChainOutOfRange);
    • a bad shape (ChainShape): empty, over the caps, or next placed wrong;
    • a plane or lane that is missing or too short.

    This makes the facade's shape panics unreachable.

  • The oracle walks the chain from the IR and never calls the facade.

Quack

  • New leaf Filter::Chain { fk, hops }.
  • Both lowerings emit exactly one GatherChain, with the gate baked in.
  • A NULL fk follows Semijoin: the leaf is FALSE.

Glove / composer (plan §11)

  • New Path type (1..=MAX_PATH relations) and new conditions Cond::ReachesPath / Cond::ViaPath. One-hop conditions are unchanged.
  • Preparation walks each relation on the previous one's target glove.
  • Every condition on a maximal path, and on each of its prefixes, binds into one chain. Hop-1 conditions fold into hop 1 instead of becoming a separate semijoin.
  • The next key must be a proven Code field the call supplies; otherwise preparation returns NoForeignLane.
  • Every target universe along the path is checked at the call.
  • R2IL names the missing GATHER.
  • Gremlin: out after out extends the path; past MAX_PATH it is refused with PathTooDeep, which replaces SecondHop. has after that is a ViaPath. dedup, values and min/max after a path of two are refused.
  • The text form and the awareness graph carry the path.

Why a new op: Gather can test a target only through a plane over the target's table. A depth-2 predicate would therefore need a depth-1 mask built first, with one program's Keep feeding another program's Gather. That is the shape the ScatterOrU32 survival condition forbids. #1521 built exactly that shape and is closed.

Tests

  • mask-risc/tests/chain.rs: the executor, the oracle and a fixture row loop all agree, gated and ungated, across tile boundaries; one bare hop equals Gather; eight refusals behave the same on both paths.
  • no_alloc.rs: 1000 gated two-hop executes allocate 0 bytes.
  • Quack: both lowerings emit one GatherChain and match a per-row reading of the filter.
  • glove-parity/tests/two_hop.rs covers line -billed_to-> partner -located_in-> region:
    • against a row loop over the text at n ∈ {1, 64, 65, 4097}, with tests at either hop, negated, and with a local condition;
    • the lowering is exactly one chain;
    • the composer, the text form and the awareness graph;
    • the refusals: unknown relation, missing next key, R2IL, missing universe.
  • One existing pin changed in tests/relation.rs: the old SecondHop refusal now composes.
  • 13 disable runs, all red, then restored:
    • Executor and oracle: executor drops the gate; executor drops the tests; oracle ignores negate; validation no longer checks the next shape.
    • Quack lowering: drops negate; emits a plain Gather; oracle stops at depth 1.
    • Glove: prefix conditions not folded into the chain; hop tests dropped; tests attached to the wrong hop; next key not followed; only the first universe checked; composer refuses the second hop.
  • cargo clippy -D warnings is clean on mask-risc, Quack and glove-parity, and cargo fmt is clean. The full suites of all three are green locally.

Board: STATUS_BOARD row D-CMH-G1-7, entry 2026-10-12-chained-hops-folded.md, plan §11.

Open

The M:N hop through two edge tables (run-level OR) still needs a decision.

🤖 Generated with Claude Code

https://claude.ai/code/session_011BMJnDBAfzcuQh5HeT39Rh

MaskOp::GatherChain { lane, chain, under, dst } with Program::chains:
dst[i] holds when the key lane[i] leads through every hop of the named
chain, each hop's reached row live in its resident foreign plane, passing
its HopTests read in place, and the hop's next lane keying the following
hop. No mask over any hop table is built and no program feeds another;
the op realises through ndarray::simd::mask_gather_chain_u32[_under].

Gather could only test a target through a plane over the target's table,
so a depth-2 predicate needed a depth-1 mask built first and a Keep fed
into another program's Gather, the shape the ScatterOrU32 survival
condition forbids.

- validate: chain exists, non-empty, <= MAX_CHAIN_HOPS hops,
  <= MAX_HOP_TESTS tests, next on every hop but the last, planes and
  U32 lanes long enough (ChainOutOfRange / ChainShape / LenMismatch),
  so the facade's shape panics are unreachable.
- executor builds the facade chain on the stack (L1 holds; test).
- oracle walks the chain from the IR, never through the facade.
- tests/chain.rs: executor, oracle and a fixture row loop agree, gated
  and ungated, across tile boundaries; one bare hop equals Gather;
  eight refusals identical on both paths.
- exhaustive matches in quack and glove-parity extended.

Needs AdaWorldAPI/ndarray#358 (mask_gather_chain_u32).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011BMJnDBAfzcuQh5HeT39Rh
Filter::Chain { fk, hops: Vec<ChainHop> } — the chained semijoin, hops
each a RESIDENT plane, in-place tests and the key onward. gate_walk turns
it into Node::GatherChain; index_chains moves the hops into the program's
chain table; both emitters (in place and fused) emit one
MaskOp::GatherChain carrying the gate. NULL fk follows Semijoin (FALSE,
not UNKNOWN).

Test: two hops, a test at each depth, an out-of-range key and a dead
partner row; both lowerings emit exactly one GatherChain with both hops
and no Gather, and the count matches a per-row reading of the filter.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011BMJnDBAfzcuQh5HeT39Rh
@coderabbitai

coderabbitai Bot commented Oct 12, 2026 •

Copy link
Copy Markdown

Warning

Review limit reached

Your organization has reached its usage spending cap. Adjust your spending cap in the billing tab.

Next included review available in 19 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available. Your 62 included PR review attempts over the past 7 days set your current allowance at 1 review per hour.

Learn how review limits work.

Review configuration:

⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: CHILL
  • Plan: Essentials
  • Run ID: 8d869999-c7c7-45c0-9cba-01b505c2aa82

📥 Commits

Reviewing files that changed from the base of the PR and between 8ae5bce and 3eaf15f.


📒 Files selected for processing (19)
  • .claude/board/STATUS_BOARD.md
  • .claude/board/entries/2026-10-12-chained-hops-folded.md
  • .claude/board/entries/README.md
  • .claude/plans/gremlin-composable-algebra-v1.md
  • crates/lance-graph-glove-parity/src/awareness.rs
  • crates/lance-graph-glove-parity/src/decl.rs
  • crates/lance-graph-glove-parity/src/gremlin.rs
  • crates/lance-graph-glove-parity/src/method.rs
  • crates/lance-graph-glove-parity/tests/relation.rs
  • crates/lance-graph-glove-parity/tests/two_hop.rs
  • crates/lance-graph-mask-risc/examples/program_collapse_probe.rs
  • crates/lance-graph-mask-risc/src/exec.rs
  • crates/lance-graph-mask-risc/src/ir.rs
  • crates/lance-graph-mask-risc/src/lib.rs
  • crates/lance-graph-mask-risc/src/reference.rs
  • crates/lance-graph-mask-risc/src/value.rs
  • crates/lance-graph-mask-risc/tests/chain.rs
  • crates/lance-graph-mask-risc/tests/no_alloc.rs
  • crates/lance-graph-quack/src/lib.rs

  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Warning

Billing warning: we have not been able to collect payment for this subscription for more than 72 hours. Please update the payment method or pay any pending invoices in Billing to avoid service interruption.


Comment @coderabbitai help to get the list of available commands.

Path (1..=MAX_PATH relations, Copy) and Cond::ReachesPath / ViaPath for a
path two relations or longer. Preparation walks each relation on the
previous one's target glove; every condition on a maximal path and on its
prefixes binds into ONE BoundCond::Chain (Filter::Chain, one
MaskOp::GatherChain): hop k's live plane, the tests the conditions put
on it, and the next key, which must be a proven Code field the call
supplies as a foreign lane (NoForeignLane otherwise). One-relation paths
no chain extends keep their semijoin / EqU32Via leaves unchanged. Every
target universe along the path is checked at the call. R2IL names the
missing GATHER.

Gremlin: out after out extends the path (SecondHop is gone; PathTooDeep
past MAX_PATH); has after it is a ViaPath; dedup, values, min/max after a
path of two are refused. The text form and the awareness graph carry the
path.

tests/two_hop.rs: line -billed_to-> partner -located_in-> region against
a row loop over the text at n in {1,64,65,4097}, tests at either hop,
negated, with a local condition; the lowering is one chain with the hop-1
test folded in; composer, text form and awareness; refusals. The
relation.rs SecondHop pin is re-pinned: a second hop now composes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011BMJnDBAfzcuQh5HeT39Rh
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011BMJnDBAfzcuQh5HeT39Rh
@AdaWorldAPI AdaWorldAPI changed the title mask-risc + Quack: chained hops folded into one GatherChain (D-CMH-G1-7) Chained hops folded: mask-risc GatherChain, Quack Filter::Chain, glove out(r1).out(r2) (D-CMH-G1-7) Oct 12, 2026

Copy link
Copy Markdown
Owner Author

CI on this PR (clippy, member-tests, parity, cats, dir-sim) is red because this branch calls ndarray::simd::mask_gather_chain_u32[_under], which exists only in AdaWorldAPI/ndarray#358. CI checks out ndarray master with no pin, so it compiles against an ndarray that doesn't have those functions. The code itself is not at fault.

ndarray#358 is green on every check. Once it merges, a re-run here builds against the new primitive; locally the full suites already pass against it. Nothing to fix in this PR until then.


Generated by Claude Code

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 12, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-12T21:53:06.635760Z 3eaf15f Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 3eaf15f370

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +1505 to +1510
if let Cond::Via {
field,
literal,
negate,
..
} = *c

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Handle one-hop ViaPath conditions

When a caller constructs Cond::ViaPath with Path::of(&[rel]), this branch emits only the relation-liveness check because it recognizes Cond::Via but not Cond::ViaPath, silently dropping the field comparison and returning extra rows. This is a representable public input—Path explicitly accepts lengths starting at one—so bind a one-hop ViaPath like Via or reject it during preparation.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Owner Author

I'm closing this without merging. It models chained hops as per-row key reads, and the graph has no foreign keys: a hop is a coordinate range, and chained hops are the intersection of ranges. D-TELEPORT-0 (#1484) had already measured this gather shape as speeding up work rather than eliminating it.

The branch is kept as backup/session-011BMJ-2026-10-12-chain2. The rebuild is tracked from #1524.


Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants