Skip to content

fix(engine): a gather in a loop counts only the current pass's lanes - #194

Merged
M3gA-Mind merged 1 commit into
tinyhumansai:mainfrom
M3gA-Mind:fix/gather-current-pass-lanes
Nov 11, 2026
Merged

M3gA-Mind merged 1 commit into
tinyhumansai:mainfrom
M3gA-Mind:fix/gather-current-pass-lanes

Conversation

@M3gA-Mind

Copy link
Copy Markdown
Collaborator

Summary

A gather inside a loop body no longer collects lanes left over from an earlier pass. This also corrects the validator's stated reason for refusing a gather with no upstream scatter.

Root cause

Lane slots (nodes.<node>.lanes.<scatter>#<i>) are never cleared between loop passes; the reducer only adds keys. The gather counted every slot under its predecessors against the scatter's lane_count, and it took that count from the first lane_count found anywhere in state. So a pass with 3 lanes followed by a pass with 2 released [20, 21, 12], where 12 is pass 1's third lane: wrong results, silently.

Fix

  • The scatter stamps each lane envelope with its activation super-step (LaneContext::scatter_step). The step travels along the lane.
  • Every lane slot records _scatter and _scatter_step. They are written after the node's own meta, so a node can't forge them.
  • The gather counts only slots from its own scatter's current activation (that scatter's _activation_step), and reads lane_count from that same scatter.

Counting on read rather than clearing on re-fire stays correct whatever writes a slot late. It also lets a pass with no lanes yet wait instead of releasing stale ones.

  • The orphan-gather refusal now states the real harm: the gather releases empty and silently drops upstream output, rather than "waiting out its poll budget".

Compatibility

Slots and envelopes from checkpoints written by older binaries carry no stamp. Unstamped slots all count when no stamped slot exists, so an in-flight run resumed across the upgrade still completes. Once stamped slots exist, unstamped ones are treated as stale.

LaneContext gains a public field (scatter_step), so outside code that builds it as a struct literal must set it (CHANGELOG).

Validation

On a Linux build host, merged onto main 8b22179:

  • cargo test --workspace --all-features --locked: 2863 passed, 0 failed, 35 ignored (no new ignores).
  • Red control: tests/loop_scatter_lanes_tests.rs copied onto unmodified main fails with left: [20, 21, 12] right: [20, 21]. It passes with this change.
  • clippy -D warnings, fmt, cargo +1.88.0 check --all-features, rustdoc -D warnings.

Tests

  • tests/loop_scatter_lanes_tests.rs: a loop whose second pass is narrower gathers only [20, 21]. It also asserts the stale slot is still in state, so the test shows the slot being ignored, not removed.
  • Gather unit tests:
    • a stale activation is ignored;
    • nothing counts before the current pass lands;
    • the expected count comes from the gather's own scatter;
    • both legacy unstamped cases.
  • lane_context_tests: an old envelope decodes with no step; the step round-trips to the lane slot, and meta cannot forge it.

Found while writing #46's fuzz targets.

Lane slots are never cleared between loop passes, and the gather
counted every slot under its predecessors against the scatter's
lane_count, taken from the first lane_count anywhere in state. A pass
with 3 lanes followed by one with 2 released [20, 21, 12], where 12 is
the first pass's third lane.

The scatter now stamps each lane envelope with its activation step,
every lane slot records `_scatter` and `_scatter_step`, and the gather
counts only slots from its own scatter's current activation, reading
lane_count from that scatter. Slots from checkpoints written before
this carry no stamp and count as before while no stamped slot exists.

Also corrects the reason given for refusing a gather with no scatter:
it releases empty and silently drops its predecessors' output.
@coderabbitai

coderabbitai Bot commented Nov 11, 2026

Copy link
Copy Markdown

Warning

Review limit reached

  • Run on-demand review

This review includes 12 billable files and costs up to $3.00.

Or wait 4 minutes for your next included review.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: c0686769-512f-40b1-be79-a9ea58490991
Commits

Reviewing files that changed from the base of the PR and between 8b22179 and 96d24e9.

Files selected for processing: 12

  • CHANGELOG.md
  • crates/tinyflows/src/engine.rs
  • crates/tinyflows/src/engine/build/activation/routing.rs
  • crates/tinyflows/src/engine/build/outcome.rs
  • crates/tinyflows/src/engine/state.rs
  • crates/tinyflows/src/lane_context_tests.rs
  • crates/tinyflows/src/nodes/control_flow/gather.rs
  • crates/tinyflows/src/nodes/control_flow/gather_tests.rs
  • crates/tinyflows/src/nodes/mod.rs
  • crates/tinyflows/src/validate/scatter.rs
  • crates/tinyflows/tests/loop_scatter_lanes_tests.rs
  • crates/tinyflows/tests/scatter_gather_e2e.rs
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

@tinysweeper

tinysweeper Bot commented Nov 11, 2026 •

Copy link
Copy Markdown

Tiny Sweeper review

Reviewed head: 96d24e943b518ccca997ca961c5e25e716534e7e

Verdict: Ready for maintainer review · Priority: none
Coverage: Reviewed 12 of 12 files.
Pull request purpose (title): fix(engine): a gathe…

Diagnostics and supporting details

Full summary

Pull request purpose (title): fix(engine): a gather in a loop counts only the current pass's lanes. Tiny Sweeper completed its review; deterministic results follow.

Review snapshot

Change surface Files Review signal Count
Production 7 Active findings 0
Tests 4 Noted findings 0
Documentation 1 Resolved findings 0
Configuration 0 Pending checks/questions 0

Completeness: Complete
Test assessment: Test coverage is assessed from changed tests and lane evidence; execution is not claimed without trusted check data.

Features

  • Modified — Gather counts only the current scatter activation's lanes: A gather inside a loop no longer counts slots left by an earlier pass: it filters slots to its own scatter's current `_activation_step`, so a narrower pass waits for its own lanes instead of releasing with the previous pass's extras. Slots written from pre-stamp envelopes all count when no stamped slot exists, preserving in-flight runs resumed across the upgrade; once a stamped slot exists, unstamped ones are treated as stale. (crates/tinyflows/src/nodes/control_flow/gather.rs fn arrivals(ctx: &NodeContext<'_>, predecessors: &[String]) -> Vec<Arrived> {, crates/tinyflows/src/nodes/control_flow/gather.rs impl NodeExecutor for GatherNode {, crates/tinyflows/src/nodes/control_flow/gather.rs struct Arrived {)
  • Modified — Gather reads its expected lane count from its own scatter: The expected total comes from the scatter that opened the lanes rather than the first `lane_count` found anywhere in state, so an unrelated earlier scatter's count can no longer make a gather wait out its poll budget against the wrong total. (crates/tinyflows/src/nodes/control_flow/gather.rs fn arrivals(ctx: &NodeContext<'_>, predecessors: &[String]) -> Vec<Arrived> {, crates/tinyflows/src/nodes/control_flow/gather.rs impl NodeExecutor for GatherNode {)
  • Modified — Corrected validation reason for an orphan gather: Refusing a gather with no upstream scatter now explains the actual harm — it releases with no items and silently drops its predecessors' output — instead of claiming it would wait out its poll budget. (crates/tinyflows/src/validate/scatter.rs pub(super) fn validate_scatter_regions(graph: &WorkflowGraph, errors: &mut Vec<V, crates/tinyflows/tests/scatter_gather_e2e.rs async fn a_scatter_without_a_gather_is_refused() {)
  • Added — Public `LaneContext::scatter_step` field: `LaneContext` gains a public `Option<usize>` field, so out-of-tree code building it as a struct literal must set it; this is documented in the CHANGELOG. (crates/tinyflows/src/nodes/mod.rs pub struct LaneContext {, CHANGELOG.md and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0)

Tests

  • unit — Decodes a lane envelope from before the stamp, asserting `scatter_step` is `None` so legacy envelopes still decode as lanes.: Present in the diff and matched to the claimed compatibility behavior; not executed here. (crates/tinyflows/src/lane_context_tests.rs fn a_well_formed_envelope_decodes() {)
  • unit — Round-trips an envelope with step 7 to a lane slot and asserts forged meta carrying `_scatter_step`/`_scatter` cannot overwrite the real stamps.: Directly exercises the write-after-meta ordering claim; not executed here. (crates/tinyflows/src/lane_context_tests.rs fn a_well_formed_envelope_decodes() {, crates/tinyflows/src/engine/state.rs pub(super) fn lane_items_update()
  • unit — Covers a stale earlier activation being ignored, a pass with no lanes landed yet counting none, the expected count coming from the gather's own scatter, and both legacy unstamped cases.: Present in the diff and aligned with the described filtering and fallback rules; not executed here. (crates/tinyflows/src/nodes/control_flow/gather_tests.rs fn on_lane_error_defaults_to_collect() {)
  • integration — Runs a refinement loop whose first pass scatters 3 candidates and second pass 2, asserting the gather releases `[20, 21]` while the stale pass-1 slot is still present in state.: Present in the diff and reflects the bug scenario from the description; not executed here. (crates/tinyflows/tests/loop_scatter_lanes_tests.rs, crates/tinyflows/src/nodes/control_flow/gather.rs fn arrivals(ctx: &NodeContext<'_>, predecessors: &[String]) -> Vec<Arrived> {)

Findings

No active actionable findings.

Before merge

None.

Agent review details

critique

  • Conclusion: Success
  • Scope reviewed: all assigned evidence
  • Lane summary: Reviewed 12 files; 0 findings. _The code index is behind this pull request (indexed at `0221482f1599`), so retrieved context may be out of date._

security

  • Conclusion: Success
  • Scope reviewed: all assigned evidence
  • Positive: The security lane answered with no findings across the 9 reviewed files; CHANGELOG.md and two engine build files were out of its review scope.
  • Lane summary: Reviewed 9 files; 0 findings. 3 files were not security-reviewed: CHANGELOG.md (prose or tabular data), crates/tinyflows/src/engine/build/activation/routing.rs (vendored or build output), crates/tinyflows/src/engine/build/outcome.rs (vendored or build output). _The code index is behind this pull request (indexed at `0221482f1599`), so retrieved context may be out of date._

tests

  • Conclusion: Success
  • Scope reviewed: all assigned evidence
  • Positive: The tests lane answered with no findings, describing the change as stamping each lane slot with scatter identity and activation step, filtering stale lanes out of arrivals, and reading the expected lane count from the correct scatter.
  • Lane summary: This pull request fixes a bug where a gather inside a loop body could collect lanes from an earlier pass's scatter activation, by stamping each lane slot with the scatter identity and activation step. The change adds the `scatter_step` field to the lane envelope and slot, filters stale lanes out of arrivals, and reads the expected lane count from the correct scatter. New unit tests and an end-to-end test verify the fix. The code is sound and safe to merge. _The code index is behind this pull request (indexed at `0221482f1599`), so retrieved context may be out of date._

commits

  • Conclusion: Success
  • Scope reviewed: all assigned evidence
  • Positive: The commits lane answered reporting no sensitive content in committed files.
  • Lane summary: No sensitive content in committed files. _The code index is behind this pull request (indexed at `0221482f1599`), so retrieved context may be out of date._

description

  • Conclusion: Success
  • Scope reviewed: all assigned evidence
  • Lane summary: The description matches the diff: the scatter stamps the envelope with its activation step, `lane_items_update` writes `_scatter`/`_scatter_step` after the node's own meta, the gather filters to the scatter's current `_activation_step` and reads `lane_count` from that same scatter, the validator wording and CHANGELOG are updated, and the tests named in the body (loop integration test, gather unit tests, `lane_context_tests`) are all present and test exactly the claimed behaviour, including the stay-waiting pass and both legacy unstamped cases. I found no defect in the changed code; the reported `cargo test`/clippy numbers are claims I cannot verify from the diff, but nothing in the diff contradicts them. _The code index is behind this pull request (indexed at `0221482f1599`), so retrieved context may be out of date._
Evidence and run details
  • Models: ladder/vectors-oai3, deepseek-v4.1-flash, deepseek-v4-flash
  • Spend: $0.033618
  • Completion usage: incomplete; some provider completion costs are unavailable.
  • Tokens: 918659 input · 117258 output · 407552 cached · 758 embedding
  • Updated: 2026-11-11T11:59:23Z
Head State Pass summary
96d24e943b51 ready for maintainer review 0 active finding(s), 0 resolved finding(s) (at 2026-11-11T11:59:23Z)

tinysweeper 0.1.2

@tinysweeper tinysweeper Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tinysweeper found nothing blocking. Approving.

Diagnostics

Total spend: $0.0336

@tinysweeper tinysweeper Bot added the priority: p3 Whenever. Cosmetic, a nicety, or a cleanup with no user visible effect. label Nov 11, 2026
@M3gA-Mind
M3gA-Mind merged commit d718768 into tinyhumansai:main Nov 11, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

priority: p3 Whenever. Cosmetic, a nicety, or a cleanup with no user visible effect.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant