Skip to content

feat(session): flag-gated append-only session event log - #1406

Merged
TheGreatAxios merged 3 commits into
mainfrom
cl-9459-event-log
Oct 13, 2026
Merged

TheGreatAxios merged 3 commits into
mainfrom
cl-9459-event-log

Conversation

@TheGreatAxios

@TheGreatAxios TheGreatAxios commented Oct 12, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Prototype for CL-9459: a single append-only session event stream that coexists with the segmented turns.jsonl writer behind a flag. No read-path change — nothing in the existing write/read path imports this module, and the writer is inert unless the flag is on.

Slice covered

  • src/session/event-log.ts: createEventStreamWriter(dir, opts) → { append(event), flush(), readFrom(cursor), latestCursor(), projectTurns(), projectPrompt(), backfillFromLegacy() }. Append-only, one JSON line per event in event-log.jsonl, UUIDv7 id (via the existing generateSessionId pattern) + optional parentId linkage, monotonic seq assigned in append order; a reopened writer continues the on-disk line count.
  • Flag: settings.sessionEventLog with env override CORBITS_SESSION_EVENT_LOG=1; explicit "off" env values (0, false, no, off, empty) win over settings. openEventStreamWriter is the gated seam — returns null when off, so nothing is written.
  • Unit tests: append, order, linkage, reopen, cursor reads, projections, legacy backfill, and flag gating (append/order/linkage/reopen).

Follow-ups (not in this PR)

  • Wiring the writer into the session lifecycle (event emission points) behind the flag.
  • Read path / Hub consumption of the event stream; full turn/prompt projections.
  • Compaction interaction with an append-only log.
  • Registering sessionEventLog in the shared settings schema (src/config/settings.ts).

Design doc is tracked on the Linear issue (the writer lands behind a flag; nothing 0.4.x-adjacent is force-migrated).

Verification

  • bun test src/session/event-log.test.ts: 18 pass, 0 fail — the 14 writer-suite tests plus the 4 edge tests (dir-lock serialization, torn-tail drop, backfill idempotency, flag gating)
  • bun run typecheck passes; oxfmt --check and oxlint pass on the two changed files
  • Flag-off inertness: openEventStreamWriter returns null when the flag is off and creates nothing on disk; explicit "off" env values win over settings.sessionEventLog
  • Dir-lock serialization: 20 concurrent appends from two writers on one dir produce unique, contiguous seqs with no interleaving
  • Torn-tail drop: an unterminated final line is dropped on read and truncated before the next append, so seq stays contiguous
  • Backfill idempotency: re-running backfillFromLegacy after a partial backfill appends no duplicates (content-hash keying)

@linear-code

linear-code Bot commented Oct 12, 2026

Copy link
Copy Markdown

CL-9459

@corbits-triage

corbits-triage Bot commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor

No changes needed from you. A maintainer will review this pull request.

@TheGreatAxios TheGreatAxios changed the title feat(session): flag-gated append-only event log prototype (CL-9459) feat(session): flag-gated append-only session event log Oct 12, 2026
Single append-only session event stream (UUIDv7 id + parentId linkage,
one JSON line per event) that coexists with the segmented turns.jsonl
writer behind the settings.sessionEventLog / CORBITS_SESSION_EVENT_LOG
flag. No read-path change: the module is inert unless the flag is on, and
nothing in the existing write/read path imports it.

- createEventStreamWriter: append-only writer with append, flush,
  readFrom, latestCursor, projectTurns, projectPrompt, backfillFromLegacy;
  a reopened writer continues the on-disk line count
- appends serialized with withResolvedDirLock; seq derives from the
  on-disk complete line count inside the lock, so concurrent writers
  cannot interleave lines or duplicate seqs
- torn tails (unterminated partial lines) are dropped on read and
  truncated before the next append, keeping seq contiguous
- backfillFromLegacy is idempotent via a per-line content hash
- openEventStreamWriter: flag-gated seam returning null unless the flag
  is enabled; explicit "off" env values win over settings
- unit tests: append, order, linkage, reopen, cursor reads, concurrency,
  torn-tail recovery, projections, legacy backfill, flag gating

@TheGreatAxios TheGreatAxios left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review · Request changes

Flag-gated append-only session event log, unwired from the read path. ASCII tests pass. Torn-tail repair corrupts any prior non-ASCII line and then reuses seq.

Findings

  • src/session/event-log.ts:154 — dropTornTail passes a UTF-16 string index to fs.truncate, which takes a byte length. A complete record whose payload contains é, followed by a partial line with no newline, loses the previous record's newline. The next append is glued onto that line, readFrom throws a JSON parse error, and the following append is also assigned seq: 1. The existing torn-tail test is ASCII, so it stays green. Truncate by byte length. A permanent test should use a non-ASCII complete line plus a torn tail and assert two parseable lines and distinct seqs.

  • src/session/event-log.ts:120 — backfill idempotency hashes only the raw line. Two identical turns.jsonl lines backfill as one event, so a repeated user turn is dropped and the parent chain skips it. Key by hash plus occurrence index.

  • src/session/event-log.ts:257 — backfill reads only turns.jsonl. A rolled segment (turns-0001.jsonl) is skipped. Walk listSegmentFiles in order.

  • src/session/event-log.ts:287 — backfill stores the legacy object at payload.record, while projectPrompt only reads payload.text. After backfilling {"role":"user","text":"one"}, projectPrompt() is "". Store the same shape projectTurns already emits for turn.user events.

  • src/session/event-log.ts:205 — append records pending with both handlers swallowing rejection, so flush resolves when the append it covers rejects. append of a circular payload rejects on JSON.stringify, and await flush() still resolves. Surface that rejection from flush.

No production file imports the module. That matches the flag-gated, inert design and is not a defect. The commit body walks the implementation and two lines exceed 72 characters. Not a behavior defect.

@TheGreatAxios
TheGreatAxios merged commit 86d7487 into main Oct 13, 2026
13 checks passed
@TheGreatAxios
TheGreatAxios deleted the cl-9459-event-log branch October 13, 2026 01:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant