refactor(blockchain): keep the aggregation worker running at all times

[MegaRedHand]

What

Aggregation stops being a per-slot session. One worker thread starts with the blockchain actor and lives as long as it does, holding its own Store clone: rank the pool's candidates, prove the best job, hand it to the actor, rank again. The actor buffers each aggregate and gossips the lot at the vote-aggregation interval.

before:  interval 2 (or up to 600ms early)  ->  snapshot pool, spawn worker,
                                                <= MAX_AGGREGATION_JOBS jobs,
                                                cancel at a soft deadline
after:   always                              ->  rank, prove best, send, repeat
         interval 2                          ->  publish what the worker produced

Why

The session existed to fit XMSS proving into one interval. The prover sat idle for most of the slot and then had ~800ms to do everything, which is what the early-start window and the job caps were compensating for. Splitting proving from publishing removes the deadline without moving what the network sees: aggregates still only go out at interval 2.

Changes

Area	Change
aggregation.rs	spawn_aggregation_worker + a loop over select_best_job, with a JobPolicy derived from the wall clock; sessions, session ids, AGGREGATION_DEADLINE, EarlyAggregationCheck, AggregationDeadline, AggregationDone, MAX_AGGREGATION_JOBS and the publish-alignment all gone. EARLY_AGGREGATION_WINDOW survives with a new job: it is now the stretch in which the worker keeps the prover free, not a licence to start early
lib.rs	started() spawns the worker, stopped() joins it; pending_aggregates buffer drained at interval 2; pause flag raised around propose_block
storage/store.rs	max_gossip_group_count_for_slot dropped with its only caller (the early-start check)
docs	architecture.md, slots_and_intervals.md, spec_deviations.md

What replaces the session machinery

What the worker may take up is a function of where the slot is (JobPolicy):

phase	eligible
start of slot → T2 − EARLY_AGGREGATION_WINDOW	backlog work (stale groups, merges of proofs already held), plus a current-slot group at the two-thirds floor
the last EARLY_AGGREGATION_WINDOW before T2	only a current-slot group at the floor
T2 onwards	everything, however few signatures back it

The floor (min_current_slot_group_sigs) is the old early-start threshold: two thirds of the votes this node's own subnets are expected to carry. It keeps a slot's votes going out as one wide aggregate rather than several thin ones.

The window is the part worth arguing about. A backlog job is a recursive proof merge that can run well past the boundary, and the prover is single-threaded, so starting one there would delay the very aggregate the slot is waiting on. Idling costs little by comparison: the backlog is not going anywhere, and this is the window where the committee's signatures usually cross the floor anyway.

Separately, the actor raises a pause flag around its own block build, which is what the max_jobs = 1 proposer cap used to buy. A proof already in flight is not interrupted.

Two details worth a look

• Apply on arrival, publish later. The actor applies each aggregate to the store the moment the message lands, because the pool the worker re-reads has to account for it. Only the gossip publication waits for interval 2.
• Worker-side coverage memory. Between send and the actor's apply, the store still shows the job as pending, so the next round would prove it again. The worker remembers the coverage it emitted per slot (EmittedCoverage) to close that window without becoming a store writer. Coverage is recorded on attempt, so a failing job is not retried at full prover cost either.

A plain thread, not spawn_blocking

The worker runs for the life of the process and spends it in leanVM proofs, so a blocking-pool thread would be parked permanently for nothing: the loop awaits nothing and reaches the actor through an unbounded channel that needs no reactor. Shutdown polls is_finished() instead of blocking a runtime thread on the proof in flight, and leaves the thread detached past WORKER_JOIN_TIMEOUT.

Metrics

• Removed: lean_aggregation_early_starts_total, lean_aggregation_early_start_lead_seconds — the window they measured is gone. Neither was documented in docs/metrics.md.
• lean_committee_signatures_aggregation_time_seconds now times one aggregate's proof instead of a session's total.
• lean_gossip_aggregation_arrival_* samples our own aggregates at publication rather than production, keeping them comparable with the peers' aggregates they share a histogram with.
• lean_aggregator_skipped_total{reason="other"} now counts a job whose proof failed.

Test status

• make fmt, make lint (clippy -D warnings): clean
• cargo test --workspace --lib: 250 passed, 0 failed
• Spec tests not run: they are red on main already (fixtures pull rolling latest)
• Local 3-node devnet, 1 aggregator: clean, finality trailing head by 3 slots, proving at t+1024ms and publication at t+1604ms (results)

[github-actions]

🤖 Kimi Code Review

I'll review this PR which refactors the aggregation system from per-slot sessions to a continuous always-on worker thread. This is a significant architectural change with security and correctness implications for consensus.

High-Level Assessment

The refactoring moves from a session-based model (start at interval 2, deadline, early-start trigger) to a continuous worker that picks jobs greedily. This is cleaner architecturally but introduces new concurrency patterns that need careful review.

---

Critical Issues

1. Race Condition in EmittedCoverage::covers — False Negative on Superset Coverage

File: crates/blockchain/src/aggregation.rs, lines 189-193

fn covers(&self, data_root: &H256, coverage: &HashSet<u64>) -> bool {
    self.by_data_root
        .get(data_root)
        .is_some_and(|emitted| coverage.is_subset(emitted))
}

Bug: This check is backwards. coverage.is_subset(emitted) asks "is the candidate's coverage contained in what we already emitted?" But the candidate's coverage is the union of raw signatures + children proofs. After the actor applies an aggregate, those raw signatures are deleted from the store (see apply_aggregated_group). The next select_best_job call will re-resolve the same AttestationData and find different coverage — the remaining un-aggregated signatures, not a superset.

The actual race: Two consecutive worker loops for the same AttestationData:

• Loop 1: raw sigs {0,1,2}, children {} → emits aggregate → actor deletes {0,1,2}
• Loop 2: re-resolves same data_root → raw sigs {3,4} (new arrivals), children {} → coverage = {3,4}
• emitted for this data_root is {0,1,2} → {3,4}.is_subset({0,1,2}) is false → worker re-proves

This is actually correct behavior for disjoint sets! But consider:

• Loop 1: raw {0,1}, child proof covering {2,3} → coverage {0,1,2,3}
• Actor applies, deletes raw {0,1}, keeps child proof (it's in the store's aggregated payloads)
• Loop 2: re-resolves → raw {4,5}, children includes the proof for {2,3} → coverage {2,3,4,5}
• emitted is {0,1,2,3} → {2,3,4,5}.is_subset({0,1,2,3}) is false → re-proves {2,3}

This re-proves validators {2,3} unnecessarily, but worse: the emitted aggregate's proof may have been built on a different child proof structure than what the store now presents, leading to potential double-counting or invalid proof construction if the child proofs changed.

Severity: Medium — causes redundant work, not incorrectness, but wastes prover time and could amplify under load.

Fix: The EmittedCoverage should track which validators were covered by the emitted proof's new contribution, not the total coverage. Or better: since the actor applies immediately and the store is re-read, the natural deduplication should come from resolve_job finding no new material. The EmittedCoverage mechanism seems to be papering over a deeper issue: why does the store still show work to do after the actor applied?

Wait — re-reading: apply_aggregated_group deletes keys from gossip_signatures and updates new_aggregated_payloads. But resolve_job also considers new_payload_keys for merges. The re-prove scenario is: same AttestationData, new raw signatures arrived + existing child proofs in new_payload_keys. The worker already emitted an aggregate for some coverage, but new signatures came in.

The EmittedCoverage intent seems to be: "don't re-prove the exact same coverage while the prior proof is in flight to the actor." But it doesn't handle the case where new signatures arrive for the same data. The is_subset check being backwards would actually be: emitted.is_subset(coverage) — "did we already emit a subset of what's now available?" No, that's also wrong for "don't re-prove."

Actually re-reading the doc comment on EmittedCoverage:

The actor applies an aggregate (which deletes the group's gossip signatures) only once the message reaches it, so between sending and that apply the store still shows the job as pending and the very next selection round would prove it a second time.

So the window is: worker sent AggregateProduced, actor hasn't processed it yet. In this window, the store still has the raw signatures. The worker's next select_best_job would re-resolve and find the same job. EmittedCoverage prevents this exact re-prove.

But the check coverage.is_subset(emitted) asks: is the new candidate's coverage a subset of what we already emitted? If we emitted coverage {0,1,2}, and the new candidate (from re-resolving the unchanged store) also has {0,1,2}, then yes, it's a subset (equal), and we skip. Correct.

If new sig {3} arrived in the tiny window: new coverage {0,1,2,3}, emitted {0,1,2}, {0,1,2,3}.is_subset({0,1,2}) is false, so we don't skip. We re-prove {0,1,2,3}. But the actor will soon apply {0,1,2}, deleting those. Then the next round sees {3} alone, which may not be viable (needs 2+ sigs for payload-only merge or raw+child mix).

This seems acceptable — we might over-prove slightly in the race window, but correctness is preserved. The is_subset direction is actually correct for "skip if candidate doesn't add new validators beyond what we already emitted."

Retracting as critical — after deeper analysis, this is correct. But the logic is subtle; a comment explaining the subset direction would help.

---

2. Unbounded Growth of pending_aggregates if Publication Fails Repeatedly

File: crates/blockchain/src/lib.rs, lines 1318-1320

self.pending_aggregates.push(SignedAggregatedAttestation {
    data: msg.output.hashed.data().clone(),
    proof: msg.output.proof,
});

In Handler<AggregateProduced>, aggregates are pushed to pending_aggregates unconditionally. If publish_pending_aggregates never runs (e.g., node loses aggregator role, or interval 2 ticks are missed due to overload), this vector grows without bound.

Mitigation: The publish_pending_aggregates does drain via std::mem::take even when is_aggregator is false, with a debug log. But if the node is never an aggregator (started without flag), the buffer still fills — the worker is spawned unconditionally in on_started and runs if aggregator.is_enabled().

Wait: next_job checks aggregator.is_enabled(). If false, worker sleeps. So no aggregates produced. But if role is toggled off after being on, pending aggregates drain at next interval 2.

Severity: Low — bounded by role toggle frequency, but worth a hard cap or metric.

---

3. Pause Guard Not Held Across await Point in propose_block

File: crates/blockchain/src/lib.rs, lines 449-453

let _pause = self
    .aggregation_worker
    .as_ref()
    .map(AggregationWorker::pause);
self.propose_block(next_slot, validator_id).await;

The PauseGuard is held across an .await, but PauseGuard is Send (contains Arc<AtomicBool>) and not !Send. However, the guard is a local variable in an async fn, so it's held across yield points. This is fine for memory safety, but...

The pause is advisory — the worker only checks between jobs. If propose_block is fast (< one proof time), the pause is effective. If propose_block is slow, the worker may start a new job before the pause flag is seen (due to memory ordering lag) or if the worker is mid-proof, it continues.

The comment says "bounds contention rather than eliminating it" — this is accurate, but the Ordering::Release/Acquire pair is correct for the flag.

Issue: PauseGuard::drop uses Ordering::Release. If the actor panics during propose_block, the guard drops and flag clears. Good. But if the actor's stopped() runs while paused, the worker is cancelled — but the pause flag stays? No, shutdown cancels the token, worker exits loop, PauseGuard is irrelevant.

Minor: The map on Option creates a temporary Option<PauseGuard>. The guard is dropped at the end of the statement... no, it's bound to _pause which lives until end of block. Correct.

---

4. Worker Thread Panic Handling

File: crates/blockchain/src/aggregation.rs, lines 160-167

match self.handle.join() {
    Ok(()) => info!("Aggregation worker joined on shutdown"),
    Err(_) => warn!("Aggregation worker panicked"),
}

If the worker panics, shutdown logs and returns. The actor continues stopping. But the worker thread is the only aggregation path — a panicked worker means no more aggregation for this process lifetime. The actor does not restart it.

Severity: Medium — silent loss of functionality. Should metric + potentially panic the actor (which would restart the whole node if supervised) or at least surface prominently.

---

5. current_slot_gate Returns Some(usize::MAX) When No Validators Expected

File: crates/blockchain/src/aggregation.rs, lines 384-394

Some(
    min_current_slot_group_sigs(
        validator_count,
        config.attestation_committee_count,
        &config.subscribed_subnets,
    )
    .unwrap_or(usize::MAX),
)

When min_current_slot_group_sigs returns None (no subscribed subnets in range, or no committees), the gate becomes Some(usize::MAX). This means every current-slot group is held back, which is correct (unreachable floor). But usize::MAX could theoretically be compared against a group with usize::MAX signatures (malicious or bug). Use None consistently instead of the unwrap_or hack.

Actually, the Some/None distinction in current_slot_gate is: None = gate lifted, Some(floor) = gate active. The unwrap_or(usize::MAX) encodes "gate active with impossible floor" as a shortcut. But min_current_slot_group_sigs already returns None for "no expectation"; propagating that None would mean "gate lifted" which is wrong — we want "gate active, nothing passes."

Better: Keep Some(usize::MAX) but document, or change current_slot_gate return type to an enum Gate::Open | Gate::Closed(floor) | Gate::NoExpectation.

---

6. select_best_job Omits max_jobs — No Cap on Work Per Slot

The old snapshot_aggregation_inputs took max_jobs: usize, capping work per session. The new select_best_job returns a single job. The worker loop calls it repeatedly, with no per-slot limit.

Implication: In a busy network, the worker could prove continuously, producing many aggregates per slot. The old design capped at MAX_AGGREGATION_JOBS = 2 per session. Now, with WORKER_IDLE_POLL = 100ms and proofs taking ~200-500ms, a slot (4 seconds, 4 intervals) could see 8-40 proofs.

Is this intended? The doc says "proving runs whenever there is work." But publication is batched at interval 2. So many proofs, one publication. The block builder's select_attestations will pick from the pool at interval 4.

Potential issue: Excessive proving wastes CPU and could starve the block build even with the pause flag (which only covers interval 4). The vote-propagation gate helps for current-slot, but stale groups are ungated.

Missing: A per-slot budget or rate limit on the worker. The old deadline + max_jobs provided backpressure.

---

7. Store Clone Shares In-Memory Buffers — Potential for Stale Reads

File: crates/blockchain/src/aggregation.rs, lines 789-799

pub(crate) fn spawn_aggregation_worker(
    store: Store,  // cloned from actor's store
    ...
)

The doc says "same backend, same in-memory buffers." If Store uses Arc<Mutex<...>> internally, this is fine. But if any caching layers assume single-writer, the worker's reads may see partially-updated state or cache incoherence.

Need to verify: Store implementation. The diff doesn't show Store internals. Assuming it's Clone-safe, but worth confirming no RefCell or non-thread-safe caching inside.

---

8. Metrics: inc_aggregator_skipped_other Counts Failed Proofs, Not Skipped Jobs

File: crates/blockchain/src/aggregation.rs, lines 856-857

metrics::inc_aggregator_skipped_other(1);

In the old code, this counted "jobs dropped because deadline cancelled worker before reaching them." Now it counts "proof failed." The metric name and label "other" are misleading.

Also in metrics.rs, lines 1009-1014:

/// Aggregation jobs the worker attempted but could not turn into an
/// aggregate, i.e. the proof itself failed.
pub fn inc_aggregator_skipped_other(count: u64) {

The doc is updated but the metric name lean_aggregator_skipped_total{reason="other"} still suggests "skipped" not "failed." Consider renaming or adding a new reason label.

---

Security Considerations

A. Vote-Propagation Gate Bypass via Stale Group Manipulation

A malicious validator could craft attestations with old target.slot (stale groups) to flood the worker, since stale groups are never gated. The worker would prioritize current-slot before stale, but within stale, it proves greedily.

Mitigation: The ranking puts current-slot first, so stale only runs when no current-slot work. But a slot with few current-slot attestations could see stale work. Not a direct attack vector, but resource exhaustion possibility.

B. No Signature Validation Before Aggregation

Not new in this PR, but worth noting: resolve_job trusts signatures in the store. The store's insert_gossip_signature presumably validates. Ensure this path isn't bypassed.

C. Double-Publish Risk from pending_aggregates Buffer

If AggregateProduced arrives twice for the same aggregate (message duplication bug), it gets pushed to pending_aggregates twice, then published twice. Gossip should deduplicate, but wasteful.

The old design published immediately on message receipt, so duplicates would also double-publish. Not new.

---

Rust Best Practices

9. Unbounded Channel to Actor — Potential for OOM

File: crates/blockchain/src/aggregation.rs, implied

The worker sends AggregateProduced via actor.send(...). If the actor is blocked (e.g., processing a large block import), the channel backs up. With an unbounded channel, memory grows.

The spawned_concurrency crate's ActorRef::send — need to check if bounded. The comment says "unbounded channel" in module docs. If truly unbounded, this is a memory safety issue under backpressure.

10. std::thread::sleep in Worker Loop

File: crates/blockchain/src/aggregation.rs, line 851

std::thread::sleep(WORKER_IDLE_POLL);

Blocks the OS thread. Acceptable for a dedicated thread, but prevents fast shutdown (up to 100ms delay). The cancel token is checked before sleep, so shutdown after sleep start waits.

Minor: Could use park_timeout or a channel recv with timeout for interruptible sleep.

11. Ordering::Relaxed Would Suffice for paused Flag

File: crates/blockchain/src/aggregation.rs, lines 141, 181

self.paused.store(true, Ordering::Release);
// ...
self.0.store(false, Ordering::Release);

The Release/Acquire pairing is technically correct but overkill for a boolean flag with no associated data. Relaxed would suffice since there's no memory to synchronize beyond the flag itself. Not a bug, just unnecessary barrier.

---

Documentation & Comments

12. Inconsistent Term: "Vote-Propagation Gate" vs "Vote-Aggregation Interval"

The code uses both terms. Standardize: the offset is VOTE_AGGREGATION_OFFSET_MS, the function is current_slot_gate, the docs say "vote-propagation gate." Pick one.

13. Stale Comment in store.rs

File: crates/blockchain/src/store.rs, lines 358-363

The comment still refers to BlockChainServer::start_aggregation_session which no longer exists. Update to publish_pending_aggregates.

---

Tests

14. Missing Test: Worker Loop Cancellation

No test verifies that cancel.cancel() breaks the worker loop between jobs.

15. Missing Test: Pause Flag Behavior

No test verifies that paused.load(Ordering::Acquire) causes next_job to return None.

16. Missing Test: EmittedCoverage Roll Behavior

roll_to is tested implicitly, but no explicit test for slot advancement clearing coverage.

---

Summary Table

Priority	Item	File:Line	Action
High	Worker panic = silent aggregation death	aggregation.rs:166	Metric + consider actor panic
High	Unbounded channel to actor	implied	Verify boundedness or add backpressure
Medium	No per-slot work cap on worker	aggregation.rs:run_aggregation_worker	Add MAX_PROOFS_PER_SLOT or similar
Medium	pending_aggregates unbounded growth	lib.rs:1318	Add hard cap or ring buffer
Low	usize::MAX sentinel in gate	aggregation.rs:394	Use enum for clarity
Low	Metrics name mismatch	metrics.rs:1009	Rename or re-label
Low	Stale comment	store.rs:363	Update reference
Low	`Ordering

---

Automated review by Kimi (Moonshot AI) · kimi-k2.6 · custom prompt

[github-actions]

🤖 Codex Code Review

1. The new pause mechanism does not actually protect local block production from an in-flight aggregation proof. The worker can start a fresh aggregate_mixed immediately before interval 4, and pause() only stops the next job; the current proof is explicitly uninterruptible. That means a proposer can still enter propose_block() while the worker is consuming the same prover/CPU budget, which is exactly the case the old “cap work before our proposal slot” logic avoided. For consensus code this is a real liveness risk: missed or late proposals under load. See crates/blockchain/src/lib.rs, crates/blockchain/src/aggregation.rs, and crates/blockchain/src/aggregation.rs.

2. EmittedCoverage is cleared on wall-clock slot rollover, not when the actor has actually applied the previously produced aggregate. If the actor is blocked across a slot boundary (most obviously in propose_block()), the worker can forget an already-sent-but-not-yet-applied group and prove it again once unpaused. That reopens the duplicate-proof window this structure is supposed to close, leading to wasted XMSS work and duplicate aggregate gossip. The problematic reset is in crates/blockchain/src/aggregation.rs and is exercised from crates/blockchain/src/aggregation.rs; the actor only applies the result later in crates/blockchain/src/lib.rs.

I could not run the Rust tests here because the environment could not fetch the leansig git dependency/toolchain artifacts.

---

Automated review by OpenAI Codex · gpt-5.4 · custom prompt

[github-actions]

🤖 Claude Code Review

Review: PR #603 — always-on aggregation worker

Overall this is a well-reasoned redesign and the accompanying docs/tests are thorough. The vote-propagation gate and the apply-on-arrival/publish-later split are both correctly implemented and match the documented boundary semantics (verified via current_slot_gate_lifts_at_the_vote_aggregation_boundary and the emitted-coverage tests). Two concerns stood out, one of them significant enough to warrant a fix before merge.

1. No crash recovery for the long-lived worker thread (reliability regression)

spawn_aggregation_worker (crates/blockchain/src/aggregation.rs:601) starts a single std::thread that is expected to live for the entire process. If anything inside its loop panics — aggregate_job's job.slot.try_into().expect("slot exceeds u32") (aggregation.rs:645), any of the several other .expect()/.unwrap() invariants in select_best_job/resolve_job, or a future bug touching the shared store — the thread dies silently. Nothing detects this at runtime:

• is_finished() is only ever checked inside AggregationWorker::shutdown() (aggregation.rs:162-175), which runs solely on actor stop.
• There's no periodic health check or respawn logic.
• No metric increments to signal the loss (the AGGREGATOR_SKIP_REASONS counters all require the loop to still be running to fire).

Under the old per-slot-session design, a panicking proof only cost that one slot's session — the next slot spawned a brand-new worker, so aggregation self-healed. With this change, a single panic anywhere in the loop permanently and silently disables aggregation for the rest of the node's uptime (a node that's the network's sole --is-aggregator would stop finalizing with no operator-visible signal until someone happens to grep logs for "Aggregation worker panicked" — which only prints at shutdown, not when the panic actually occurs).

Suggest either:

• Wrapping the per-job body (proof + send) in std::panic::catch_unwind, or
• Having the actor periodically observe worker.handle.is_finished() (e.g. on a tick) and respawn the worker if it died unexpectedly, logging loudly when that happens.

2. current_slot_gate clones the full head state on every poll (efficiency)

current_slot_gate (aggregation.rs:547) calls store.head_state().validators.len() every time next_job runs while ms_into_slot < VOTE_AGGREGATION_OFFSET_MS — i.e., on every proof completion and every 100ms idle poll (WORKER_IDLE_POLL) for roughly the first 40% of every slot. Store::head_state() clones the entire reconstructed State (crates/storage/src/store.rs:1716), including the full validators list, just to read its length.

Previously this same computation (self.store.head_state().validators.len() in the old maybe_start_early_aggregation) only ran on a gossip-signature-insert event plus once at window-open — i.e., driven by attester traffic, not a fixed 10Hz poll. The new code runs it far more often, and the clone competes for the shared state_cache mutex (crates/storage/src/store.rs:548) with the actor thread's own state reads. For larger validator sets this is a non-trivial, easily avoidable cost.

Consider caching validator_count (invalidated when store.head() changes) instead of re-deriving it from a full state clone on every round, or add a lightweight accessor that returns just the count without materializing the whole State.

Minor / non-blocking

• min_current_slot_group_sigs/current_slot_gate correctly preserve the old semantics (gate only applies before the interval-2 boundary, usize::MAX floor when no committees/subnets expected), confirmed against both the old maybe_start_early_aggregation logic and the new unit tests — no behavioral regression there.
• The EmittedCoverage send-to-apply race guard is sound: covers() correctly uses subset-checking so a job whose coverage grows (new signature arrived) is still eligible, while an unchanged/failing job isn't re-proven every 100ms.
• PauseGuard's drop-based reset is a good defensive pattern for propose_block's early returns.

No fork-choice, justification/finalization, or SSZ-encoding correctness issues found — this PR only touches aggregation orchestration, and the aggregate content/selection logic (select_best_job, pick_best_candidate, resolve_job) is carried over unchanged in behavior, just restructured from a batch to a single-pick call.

---

Automated review by Claude (Anthropic) · sonnet · custom prompt

[MegaRedHand]

Local devnet: 3 ethlambda nodes, 1 aggregator, 20 slots

attestation_committee_count = 1, so the single aggregator sees every vote and 2 of 3 validators are enough to justify.

	ethlambda_0 (aggregator)	ethlambda_1	ethlambda_2
blocks imported	11	11	12
aggregates proved	18	0	0
aggregates gossiped	16	0	0
attestations	19	18	18
WARN	0	0	0
ERROR / panic	0	0	0

Finality kept up: finalized_slot=15 with head_slot=18 at the end, so finalization trailed the head by 3 slots throughout.

The split between proving and publishing is visible in the timing

Offsets into the slot, from the aggregator's log (interval boundaries at t+0 / 800 / 1600 / 2400 / 3200):

duty	n	median	min	max
attestation published	19	t+815ms	t+806ms	t+859ms
aggregate proved	18	t+1024ms	t+951ms	t+3617ms
aggregates gossiped	16	t+1604ms	t+1602ms	t+1607ms
block published	12	t+232ms	t+194ms	t+328ms

The proof lands ~200 ms after the votes do — as soon as the group crosses two thirds, well inside interval 1 — while publication sits on the interval-2 boundary to the millisecond. That is exactly the intent: proving off the grid, publication on it.

The gate does what it is for

Every aggregate was raw_sigs=3 children=0 participants=3, one per slot, at 134-170 ms of prover time. One wide aggregate per slot rather than the several thin ones an ungated always-on worker would produce.

The single t+3617ms outlier is a backlog job taken late in a slot, which is the intended use of the otherwise-idle stretch.

How it was run

DOCKER_TAG=pr603 make docker-build
.claude/skills/devnet-runner/scripts/run-devnet-with-timeout.sh 140

Genesis keys came from blockblaz/hash-sig-cli:latest (52-byte pubkeys). The lean-quickstart checkout defaults to ghcr.io/lambdaclass/hash-sig-cli:0.5.0, which emits the 32-byte leanVM-main keys and makes any main-based image fail with failed to parse genesis config. Worth knowing before running a local devnet off main.