Skip to content

test: write #46's remaining fuzz properties and complex fixtures - #193

Merged
M3gA-Mind merged 1 commit into
tinyhumansai:mainfrom
M3gA-Mind:test/fuzz-remaining-46
Nov 11, 2026
Merged

M3gA-Mind merged 1 commit into
tinyhumansai:mainfrom
M3gA-Mind:test/fuzz-remaining-46

Conversation

@M3gA-Mind

Copy link
Copy Markdown
Collaborator

Summary

The remaining fuzz properties and complex fixtures from #46. Test-only: no engine code changes.

  • Lane isolation, write by write (tests/fuzz_lane_isolation_tests.rs): across every checkpoint boundary of a scatter run (run_with_checkpointer), a lane slot is written once and never changes, and no lane-body node gets a top-level items. Across StepPhase::After frames, each (node, lane) activates exactly once. It varies rows, body length, the lane cap, and on_error: continue / route endings.
  • Validator soundness (tests/fuzz_validator_soundness_tests.rs): each refused graph is checked to fail compile with the expected reason, then forced in through a CompiledWorkflow { graph } literal and shown to misbehave. Cases: an off-cycle merge in a loop, condition labels on to_port, a gather with no scatter, a scatter with no gather, a nested scatter, a loop inside a lane, an edge to an unknown node, and no trigger. A proptest mutates generated branches into InvalidConditionRouting.
  • Timing schedules over generated shapes (tests/fuzz_timing_tests.rs): a per-node yield-count schedule plus injected failures. Two different schedules with the same failure set give the same stable output. Scheduler stamps and runner-numbered spawn tickets are excluded.
  • Fixtures (tests/complex_graphs/loop_scatter_and_lane_switch_tests.rs): a refinement loop whose body scatters, and a lane with a switch and an on_error: route arm under every on_lane_error policy. Both assert final state and the RunObserver order.

Closes #46

#46 checklist

  • Fixture: scatter → child → gather(quorum) → loop → second scatter. Landed earlier (tests/complex_graphs/composition_tests.rs:6).
  • Fixture: refinement loop whose body scatters. Added.
  • Fixture: nested spawn/gate across a sub_workflow with an approval resume. Landed earlier (composition_tests.rs:122).
  • Fixture: lane with a switch and an on_error: route arm under every on_lane_error, with observer order. Added. Partial coverage landed earlier in concurrency_and_failures_tests.rs:134, :181.
  • Fixture: diamond in a loop. Landed earlier (tests/loop_e2e/loop_e2e_part_01_tests.rs:350).
  • Fixture: wide scatter with low max_concurrency. Landed earlier (concurrency_and_failures_tests.rs:27, parallel_e2e.rs:225).
  • Cancellation property. Landed earlier (tests/fuzz_cancellation_tests.rs:170, :214).
  • Reducer associativity and the $replace round-trip. Landed earlier (src/engine_merge_tests.rs:71, :88).
  • Lane isolation. Added, write by write; the final-state version is fuzz_scatter_tests.rs:98.
  • Timing schedules. Added over generated shapes; the flat fan-out version is fuzz_timing_tests.rs:132.
  • Validator soundness. Added.
  • Shrunk counterexamples checked in, and spawn/gate reference workflows. Landed earlier (proptest-regressions/, tests/reference_workflows_tests.rs:30).

Found while writing these, fixed in their own PRs

  • A gather in a loop gathered an earlier pass's lanes: fix/gather-current-pass-lanes.
  • A failing parallel run reported a completion-order-dependent error: fix/deterministic-parallel-error.
  • Lane activations wrote the node's top-level slot on cancel and re-entry paths: fix/lane-never-writes-top-level.

Validation

On a Linux build host, merged onto main 8b22179:

  • cargo test --workspace --all-features --locked: 2869 passed, 0 failed, 35 ignored (main: 2856 / 35; no new ignores).
  • The new targets ran 3× on the branch, with no failures and no proptest regression written. fuzz_timing_tests 3/3, fuzz_lane_isolation_tests, fuzz_validator_soundness_tests (9 tests), complex_graphs_tests (7 tests).
  • clippy -D warnings, fmt, cargo +1.88.0 check --all-features (MSRV), rustdoc -D warnings.
  • Every file is ≤500 lines.

…ixtures

- Lane isolation, write by write: across every checkpoint a lane slot is
  written once and never changes, no lane-body node gets a top-level
  `items`, and each (node, lane) activates exactly once.
- Validator soundness: graphs the validator refuses are forced past it
  and shown to misbehave as each refusal says they would.
- Timing schedules over generated shapes: two different per-node latency
  schedules with the same failures give the same output.
- The refinement-loop-with-a-scatter and lane-switch fixtures.

The two engine bugs these surfaced are fixed in their own PRs.
Closes tinyhumansai#46
@coderabbitai

coderabbitai Bot commented Nov 11, 2026

Copy link
Copy Markdown

Warning

Review limit reached

  • Run on-demand review

This review includes 5 billable files and costs up to $1.25.

Or wait 4 minutes for your next included review.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 6ffd5aed-694e-48f8-9cf6-0ab97dd578ad
Commits

Reviewing files that changed from the base of the PR and between 8b22179 and 0221482.

Files selected for processing: 5

  • crates/tinyflows/tests/complex_graphs/loop_scatter_and_lane_switch_tests.rs
  • crates/tinyflows/tests/complex_graphs_tests.rs
  • crates/tinyflows/tests/fuzz_lane_isolation_tests.rs
  • crates/tinyflows/tests/fuzz_timing_tests.rs
  • crates/tinyflows/tests/fuzz_validator_soundness_tests.rs
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

@tinysweeper

tinysweeper Bot commented Nov 11, 2026 •

Copy link
Copy Markdown

Tiny Sweeper review

Reviewed head: 0221482f159982764ced8c71155f6b4d3043c6db

Verdict: Ready for maintainer review · Priority: none
Coverage: Reviewed 5 of 5 files.
Pull request purpose (title): test: write #46's re…

Diagnostics and supporting details

Full summary

Pull request purpose (title): test: write #46's remaining fuzz properties and complex fixtures. This pull-request is titled "test: write #46's remaining fuzz properties and complex fixtures" and its description says it lands the remaining fuzz properties and complex fixtures from issue #46, with no engine code changes.

Review snapshot

Change surface Files Review signal Count
Production 0 Active findings 0
Tests 5 Noted findings 0
Documentation 0 Resolved findings 0
Configuration 0 Pending checks/questions 0

Completeness: Complete
Test assessment: Test coverage is assessed from changed tests and lane evidence; execution is not claimed without trusted check data.

What changed

Five test files change with 1201 additions and 1 deletion: two new complex-graph fixture bodies folded into the existing suite through a new `include!` line, and three new or extended fuzz targets. A trusted base-to-head comparison is not available for this push, so no push delta is described.

Features

  • Added — Lane isolation verified write by write across boundaries and activations: Adds a proptest that varies rows, lane-body length, the scatter lane cap, and `on_error: continue` / `on_error: route` endings, then checks that a lane slot goes absent to present once and never changes at any super-step boundary, that no lane-body node ever writes its top-level `items`, that each (node, lane) pair produces exactly one `After` frame, and that the lane slots seen at boundaries agree with the recorded activations. This closes the gap a final-state-only check cannot see, such as a slot written twice within a step. (crates/tinyflows/tests/fuzz_lane_isolation_tests.rs)
  • Added — Validator soundness: refusals tied to demonstrable misbehaviour: Adds fixed cases plus a proptest that first require `compile` to fail with the expected validation error, then construct `CompiledWorkflow { graph }` directly to run the refused graph unvalidated and assert the specific wrong behavior the refusal prevents: an off-cycle merge stalling a loop, condition labels on `to_port` running both fan-out arms, a gather without a scatter losing its input, a scatter without a gather stranding its lanes, a nested scatter failing inside its lane, a loop inside a lane not counting passes, an edge to an unknown node, a graph with no trigger, and a mutation of generated branches into `InvalidConditionRouting`. (crates/tinyflows/tests/fuzz_validator_soundness_tests.rs)
  • Modified — Timing determinism over generated graph shapes: Extends the existing latency-schedule property with a second property that generates graph shapes, marks selected leaf nodes as `on_error: continue`, and injects per-node latency as a count of `yield_now` calls plus a shared failure set through a `StepInterceptor`. Two independent schedules with the same failure set must produce the same stable output or the same error, with scheduler stamps and runner-numbered spawn tickets stripped before comparison. (crates/tinyflows/tests/fuzz_timing_tests.rs, crates/tinyflows/tests/fuzz_timing_tests.rs proptest! {)
  • Added — Refinement-loop scatter and lane-switch complex fixtures: Adds a fixture where a loop body scatters candidate proposals per pass, gathers them, folds the best score into loop state and exits on an `until` condition, asserting exit reason, iteration count, final state, the gathered lane items in lane order, and the observed activation order. Adds a second fixture with a scatter into a lane-local switch, an `on_error: route` arm, and a gather, run under `collect`, `skip` and `fail_fast` policies, asserting that lane-body nodes never write top-level slots, that routed follow-ups occur after their arm tool, and the differing gather contents per policy. (crates/tinyflows/tests/complex_graphs/loop_scatter_and_lane_switch_tests.rs, crates/tinyflows/tests/complex_graphs_tests.rs)

Tests

  • property test — Runs a generated scatter graph through `run_with_checkpointer` and a second identical run through `run_intercepted`, then asserts lane slots are written once and frozen, top-level `items` never appears for lane-body nodes, each (node, lane) has exactly one `After` frame, and boundary lane keys match activation keys.: Added by this pull-request; not executed as part of this review. (crates/tinyflows/tests/fuzz_lane_isolation_tests.rs)
  • property test — For generated graph shapes with marked failures, runs two different per-node yield-count schedules and requires the stable output or the error to be identical between them.: Added by this pull-request; not executed as part of this review. (crates/tinyflows/tests/fuzz_timing_tests.rs, crates/tinyflows/tests/fuzz_timing_tests.rs proptest! {)
  • unit test — Asserts forced, unvalidated graphs are refused by `validate_all` for the expected reason and then misbehave in the specific way the refusal exists to prevent, including a mutated generated branch that must be rejected only for `InvalidConditionRouting`.: Added by this pull-request; not executed as part of this review. (crates/tinyflows/tests/fuzz_validator_soundness_tests.rs)
  • integration test — Asserts the refinement loop exits via `until` after three passes with the expected accumulated state and gathered lane items, and that the lane-switch fixture produces the expected per-policy gather contents and observer ordering.: Added by this pull-request; not executed as part of this review. (crates/tinyflows/tests/complex_graphs/loop_scatter_and_lane_switch_tests.rs, crates/tinyflows/tests/complex_graphs_tests.rs)

Findings

No active actionable findings.

Before merge

None.

Agent review details

critique

  • Conclusion: Success
  • Scope reviewed: all assigned evidence
  • Lane summary: Reviewed 5 files; 0 findings. _The code index is behind this pull request (indexed at `337370b4cfaf`), so retrieved context may be out of date._

security

  • Conclusion: Success
  • Scope reviewed: all assigned evidence
  • Lane summary: Reviewed 5 files; 0 findings. _The code index is behind this pull request (indexed at `337370b4cfaf`), so retrieved context may be out of date._

tests

  • Conclusion: Neutral
  • Scope reviewed: all assigned evidence
  • Lane summary: No behavioural change: nothing outside documentation, configuration and tests.

commits

  • Conclusion: Success
  • Scope reviewed: all assigned evidence
  • Positive: The commits lane reported no sensitive content in committed files.
  • Lane summary: No sensitive content in committed files. _The code index is behind this pull request (indexed at `337370b4cfaf`), so retrieved context may be out of date._

description

  • Conclusion: Success
  • Scope reviewed: all assigned evidence
  • Positive: The description lane found the change well-structured and test-only, following the line-count, module-organisation and Rust-idiom rules, with no reportable defects.
  • Lane summary: This pull request adds fuzz properties for lane isolation, validator soundness, and timing determinism over generated shapes, plus two complex fixtures. The code is well-structured and test-only; no engine changes. The line-count, module-organisation, and Rust-idiom rules are followed. No reportable defects were found. _The code index is behind this pull request (indexed at `337370b4cfaf`), so retrieved context may be out of date._
Evidence and run details
  • Models: ladder/vectors-oai3, deepseek-v4-flash, deepseek-v4.1-flash
  • Spend: $0.039907
  • Completion usage: incomplete; some provider completion costs are unavailable.
  • Tokens: 908986 input · 135426 output · 324351 cached · 695 embedding
  • Updated: 2026-11-11T12:00:54Z
Head State Pass summary
0221482f1599 ready for maintainer review 0 active finding(s), 0 resolved finding(s) (at 2026-11-11T12:00:54Z)

tinysweeper 0.1.2

@tinysweeper tinysweeper Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tinysweeper found nothing blocking. Approving.

Diagnostics

Total spend: $0.0399

@tinysweeper tinysweeper Bot added the priority: p3 Whenever. Cosmetic, a nicety, or a cleanup with no user visible effect. label Nov 11, 2026
@M3gA-Mind
M3gA-Mind merged commit b428b08 into tinyhumansai:main Nov 11, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

priority: p3 Whenever. Cosmetic, a nicety, or a cleanup with no user visible effect.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Test coverage: remaining fuzz targets and complex graph fixtures

1 participant