[kixelated]

Finishes quest/m1/unbounded-groups/log.md: kio gains an append-only log whose readers take committed entries by index without a lock, and GroupState's frames move onto it.

Summary

• kio::log (new): Writer<T> appends and evicts from the front. Any number of Reader<T>s read by absolute index without a lock and get Err(Evicted) below the base rather than a stale entry.
  • Entries live in segments whose slots never move, doubling from 1 to 64 slots. Each segment is one allocation (header and slots together), and a one-segment log keeps its segment list inline.
  • Appends publish with a Release store of the segment's fill count and the log's end. A reader reuses the segment it last read when that segment holds the index, on one Acquire load. Otherwise it follows links from there, or from the log's oldest segment.
  • Reclamation is epoch-based (crossbeam-epoch). Reader::read() returns a short-lived Read handle that pins the log for a run of reads. A reader holds nothing between reads: its last segment is kept only as an unowned hint, which the next read() trusts only if the base shows it wasn't retired. Eviction advances the base, moves the head, and retires each segment wholly below the base. The collector frees a retired segment once every pin taken before the retirement is gone, so eviction frees memory whatever readers do, with no size constant. A reader that stopped reading gets Evicted when it comes back. Dropping the writer evicts everything.
  • The log doesn't park. Callers append under a paired channel's write guard and park on that channel; the module docs describe the protocol.
• moq-net group storage: GroupState's frames, committed, the relay objects table, and Flate::sizes collapse into one Entry { frame, fields, inflated } per frame on a kio::log::Writer. Adopted head frames (adopt_head) go in a front deque below the log's base, as Front eviction will need. release() evicts everything.
• Cursors read committed frames, and refill their prefetch batch, from the log without the group lock, pinning once per batch. They take the lock only to park at the edge, for adopted frames below the base, for a plain read of a compressed group (which inflates under the guard as before), and to resolve terminal state.
• Charging: CACHE_OVERHEAD covers the log's shared state and its first one-slot segment, which replace the four Frame slots VecDeque rounded up to. Later segments, and the segment list once it spills, are charged as they're allocated and released with the frames. Evicted bytes are no longer held by readers, so nothing outlives its charge beyond reads already in progress.
• Benches:
  • contended_log_lockfree in rs/kio/benches/contended.rs, next to the mutex log in the same binary.
  • group_contended in rs/moq-net/benches/group.rs: one live multi-frame group read by 16 to 1,024 readers over 1, 4, or 16 threads, with bursts of 1 and 8 frames.
• Loom: just rs loom now also sets --cfg crossbeam_loom, so loom permutes the collector's own atomics.
  • One model checks that an append under the channel guard either reaches a reader or wakes it when it parks.
  • Another races a reader holding a stale hint against eviction and collection. Under loom, every reader access to a segment reads a marker cell that the free writes, so a read not ordered before the free fails the model. Disabling the hint check makes this model fail with a causality violation, so it does catch a use-after-free.
  • That model runs with a preemption bound of 3, like moq-net's models: with the collector under loom, the full search is intractable.
• Drive-by, needed to run just rs loom: rs/moq-net/tests/loom.rs still called Subscription::with_groups, which main renamed to with_range. I found no open PR fixing it.

Impact

• kio public API (additive): new kio::log module with Writer, Reader, Read, and Evicted. kio gains a crossbeam-epoch dependency, which is already in the lockfile through other crates and builds for wasm32.
• moq-net public API: no signature changes. One behavior change: group::Producer::start_at called again before the first frame can still raise the start but now returns Error::Closed if it would lower it, because the log's base only advances. Every in-tree caller calls it once. I didn't mark the title !.
• Wire: none. js/net needs no sync.

Benchmarks

The host was oversubscribed during every run (load average 17 to 46 on 24 cores), so numbers are noisy. The tables compare bench binaries interleaved and run 2 or 3 times, and report the lowest median of each.

kio contended_log (mutex) vs contended_log_lockfree, same binary, per round of one append plus every reader reading it. This was measured with the earlier refcounted design; epoch costs are compared further down.

threads	readers	mutex	log	change
1	16	2.2 us	1.2 us	-46%
1	256	13.2 us	11.8 us	-10%
1	4096	276 us	165 us	-40%
4	256	1461 us	55 us	-96%
4	4096	3506 us	815 us	-77%
16	256	5149 us	240 us	-95%
16	4096	2290 us	1260 us	-45%

The mutex numbers above 1 thread are inflated by lock-holder preemption on an oversubscribed host. Treat those as an upper bound on the win.

Epoch reclamation vs the earlier refcounted segments. Three binaries ran in the same rounds: main (the mutex VecDeque), the refcounted design that held its last segment between reads (91b3bc7), and this head:

bench	vs main: refcounted	vs main: epoch	epoch vs refcounted
group_contended, 1 thread	-23% to +5%	-1% to +4%	+4% to +28%
group_contended, 4 threads	-14% to -41%	-13% to -41%	0% to +4%
group_contended, 16 threads (except 8/16)	-15% to -64%	-14% to -61%	-39% to +8%
group_read_frames, 512 to 8,192 frames	-10% to -2%	-1% to +13%	+5% to +21%
group_relay_frames/plain, 512 to 8,192 frames	-11% to -16%	-5% to -7%	+5% to +13%
kio contended_log_lockfree, 1 thread	n/a	n/a	+2% to +13%
kio contended_log_lockfree, 4 and 16 threads	n/a	n/a	-61% to +3%

• group_contended with bursts of 8 on 16 threads and 16 readers was erratic in both runs (+57% and +123% against main) and is left out of the 16-thread row.
• The single-thread cost is the pin: about 4 ns per frame on large groups, since a cursor pins once per prefetch batch of 8. Across cores, epoch matches or beats the refcounted design, because readers no longer bump a shared segment refcount on each hop.
• On the write path (group_write_frames, track_recv_groups, track_stamp_groups), epoch and refcounted segments are within noise of each other: -15% to +9%, apart from single-sample outliers at 2,048 and 4,096. Both cost a few ns more per append than main, plus about +100 ns per 4-frame group for the 1-, 2-, then 4-slot segment allocations.

Not run: just bench-profile lock video-heavy before and after. It needs kernel.perf_event_paranoid=-1, and this host has 4. The log lock profile quest added here covers it.

Choices made without asking

• Segments start at 1 slot rather than the prototype's 4. A one-frame group (chat, catalog, stats) then holds one 72-byte slot instead of four. The cost is two extra allocations by the 4th frame.
• Retirement calls flush(), so a retired segment reaches the global collector right away instead of waiting for the evicting thread's local bag to fill. Collection still needs some thread to pin, and every log read and eviction does.
• crossbeam-epoch's global collector, rather than one per log. Pins stay cheap and per-thread, and no log carries collector state.
• I kept the frame log a separate structure beside GroupState's other fields, as the quest asked. Scalar fields stay under the channel lock.

Decisions

1. ✅ Name: keep kio::log::{Writer, Reader, Evicted} (maintainer, 2026-10-25).
2. ✅ Pinning: a stalled reader must not keep evicted memory alive, with no size constant (maintainer, 2026-10-25). Implemented with epoch-based reclamation, as described above.
   • Copy-out under a version check was rejected: cloning T (a Bytes refcount bump) from memory that may already be freed is unsound.
   • A reader refcount held only for the read was rejected: it puts a contended atomic on every read.
   • Epoch was measured against the earlier design (see Benchmarks).
3. ✅ Lock profile: don't wait for it; land this PR and validate later (maintainer, 2026-10-25). Planned as quest/m1/unbounded-groups/log-lock-profile.md, which is blocked on the perf-host quest from moq-dev/moq#6796.

Checks

• just check (Nix dev shell): lint and every other module pass. check flake fails locally because this host can't unshare to build the github-runner image, and that failure cancels test rs. Run on its own, just rs test-changed passes: 10,324 tests, with one SLOW test in moq-cli that is unrelated.
• just rs loom (now with crossbeam_loom) passes: kio's 18 models, including both log models; moq-net's loom (19) and expiry-loom (2).
• cargo nextest for kio and moq-net: 2,744 tests pass. kio builds for wasm32-unknown-unknown.

Suggested follow-ups

• None beyond the planned log lock profile quest.

(Written by Claude Opus 5.5)

🤖 Generated with Claude Code

[kixelated]

Decisions recorded (maintainer, 2026-10-25):

1. ✅ Name: keep kio::log::{Writer, Reader, Evicted}.
2. ✅ Pinning: a stalled reader must not keep evicted memory alive, with no size constant. Implemented in 2fabcb3 with crossbeam-epoch: reads go through a short-lived Read pin, readers hold nothing between reads, and eviction retires segments to the collector. Loom (with crossbeam_loom) proves no read touches a freed segment, and the model fails if the hint check is removed. Measured against the earlier refcounted design: about +4 ns per frame on one thread (one pin per prefetch batch), and equal or faster across cores. The PR body has the tables.
3. Hold as draft until just bench-profile lock video-heavy can run on a host with perf_event_paranoid=-1 (the quest/m1/perf-host.md condition in quest: plan the overnight run's follow-ups #6796).

(Written by Claude Opus 5.5)

[kixelated]

Decision recorded (maintainer, 2026-10-25):

3. ✅ Lock profile: don't wait for it. This PR lands now, and the profile moves to quest/m1/unbounded-groups/log-lock-profile.md (XS, added in this PR, linked from the unbounded-groups epic). That quest is blocked on the perf-host quest planned in quest: plan the overnight run's follow-ups #6796.

## Decisions needed is gone from the body, and the PR is marked ready.

(Written by Claude Opus 5.5)

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-10-25T19:44:27.522824Z	9c7a5e0	New commits

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[coderabbitai]

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 2 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used all 4 included reviews currently available.

Learn how review limits work.

Review configuration:

Run configuration

• Source mode: Auto
• Repository: Unavailable
• Branch: Unavailable
• File: No configuration file from this repository applied
• Commit: Unavailable
• Layers: Organization UI
• Review profile: CHILL
• Plan: Advanced
• Run ID: 8ae14498-c6bb-4efe-8304-b1d079a1e192

Commits

Reviewing files that changed from the base of the PR and between 8b628d5 and 9c7a5e0.

Files ignored due to path filters: 1

• Cargo.lock is excluded by !**/*.lock

Files selected for processing: 19

• quest/m1/unbounded-groups/README.md
• quest/m1/unbounded-groups/front-eviction.md
• quest/m1/unbounded-groups/log-lock-profile.md
• quest/m1/unbounded-groups/log.md
• quest/m2/perf/kio-channel-contention/README.md
• quest/m2/perf/kio-channel-contention/log-adoption.md
• quest/m2/perf/kio-channel-contention/version.md
• quest/m3/middle-eviction.md
• rs/justfile
• rs/kio/Cargo.toml
• rs/kio/README.md
• rs/kio/benches/contended.rs
• rs/kio/src/lib.rs
• rs/kio/src/log.rs
• rs/kio/src/loom.rs
• rs/kio/src/sync.rs
• rs/moq-net/benches/group.rs
• rs/moq-net/src/model/group.rs
• rs/moq-net/tests/loom.rs

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[kixelated]

Automated review by review (OpenAI)

Reviewed commit: d812f84

P2 — Restart traversal when eviction disconnects a cached segment (rs/kio/src/log.rs:502–503,523). A single Read can read entry 0, then observe writer.evict(1); writer.push(1);, and incorrectly return Err(Evicted) for entry 1. Its pinned old tail is still safe to access, but is disconnected from the newly created head; reusing that hint reaches a null successor despite index 1 being live. Retry from the current head with a fresh base check when the hinted chain ends. Add a regression that retains the same Read across full eviction and reappend; existing tests repin between accesses.

Overall direction is coherent: keeping scalar state locked while consolidating committed frame storage simplifies the group integration. The custom unsafe storage warrants maintaining focused reclamation and traversal tests.

Verification: static inspection of the log, group integration, dependency changes and Loom models, including an independent unsafe/concurrency pass. No tests or benchmarks were run. No memory-safety defect was demonstrated.

[kixelated]

Re the P2 in #6782 (review): agreed, and fixed in 9c7a5e0. When the hinted chain ends short of the index, Read::get now retries from the head loaded under the same pin. a_read_spans_a_full_eviction keeps one Read open across evict(1); push(1) and fails without the fix. The kio loom models still pass.

(Written by Claude Opus 5.5)

[kixelated]

Automated review by review (OpenAI)

Reviewed commit: 9c7a5e0

Fixed: the stale-hint false-eviction finding from #6782 (review). Read::get now falls back to the current head when its cached chain cannot reach the requested entry (rs/kio/src/log.rs:502–515). The new same-Read regression exercises full eviction followed by reappend (717–729).

No new actionable bugs found in this delta. The shared traversal helper and fallback are a focused fix without expanding the reclamation protocol.

Verification: static comparison against d812f84 and surrounding read/publication/reclamation paths. Tests and Loom models were not run.

[kixelated]

Landing summary:

• Change: kio::log, a lock-free append-only log with epoch-based reclamation (crossbeam-epoch), now holds GroupState's frames. Cursors read without the group lock.
• Decisions (maintainer, 2026-10-25):
  • The name stays kio::log.
  • Evicted memory is freed whatever readers do, with no size constant.
  • The lock profile moves to the new XS quest quest/m1/unbounded-groups/log-lock-profile.md, blocked on the perf-host quest in quest: plan the overnight run's follow-ups #6796.
• Review: an OpenAI review of d812f84 found that a Read spanning a full eviction misreported a live entry as Evicted. It's fixed in 9c7a5e0 with a regression test, and the re-review on that head is clean. Codex gave the final head a thumbs-up.
• Local checks:
  • Lint and test-changed pass (10,335 tests).
  • kio's loom models pass under crossbeam_loom.
  • quest check --base origin/main passes.
• Landing: queued 9c7a5e0 through the merge queue.

(Written by Claude Opus 5.5)