Skip to content

docs(adr): ADR-017 object events (draft) - #292

Draft
alukach wants to merge 4 commits into
mainfrom
worktree-adr-017-object-events
Draft

alukach wants to merge 4 commits into
mainfrom
worktree-adr-017-object-events

Conversation

@alukach

@alukach alukach commented Nov 21, 2026

Copy link
Copy Markdown
Contributor

Draft ADR-017: emit read/write events from the proxy and consume them for analytics, per-product audit, usage tracking, and a public subscription API.

  • Write events go to a Queue and a router Worker. Read events go to a Pipelines stream and are not offered to external subscribers.
  • Audit is one Iceberg table (R2 Data Catalog) partitioned by day, account and product. Retention becomes ours to set.
  • Usage is tracked in per-principal Durable Objects. Limit enforcement is out of scope.
  • External users pull write events from GET /.events/{account}/{product} with a client-held cursor, using the same SigV4/STS credentials as reads, so service accounts work unchanged.
  • Delivery is at-least-once and best effort at the source; subscribers are told to reconcile by listing.

The ADR ends with eight open questions. The main one is whether best-effort capture is enough for the source/metadata-catalog indexer or whether it needs an outbox. Retention figures (13 months audit, 7 days replay) are placeholders. The Pipelines cost estimate is inferred from the request rate implied by the observability config and should be measured.

Docs only; no code changes.

🤖 Generated with Claude Code

Proposes emitting read/write events from the proxy, with an Iceberg audit
ledger, usage tracking, and a pull-based /.events subscription for writes.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@claude

claude Bot commented Nov 21, 2026 •

Copy link
Copy Markdown
Contributor

Claude finished @alukach's task in 22s —— View job


✅ No blocking issues — safe to merge.

This is a docs-only draft ADR (adrs/017-object-events.md). I found nothing that blocks merging a draft. The points below are worth fixing before the status moves from Proposed.

Non-blocking findings

  • PR description is out of date. It says "eight open questions" and that audit is "partitioned by day, account and product". The ADR has seven open questions (:244-252). It also says Pipelines can partition only by ingest day (:153). Please update the PR body.
  • The ADR says readers dedupe on id, but read rows have none (:70, :82, :157). Only write rows carry id. Either give read events an id too, or say that reads are not deduped. The proxy sends each event once (:80), so the ADR also doesn't show where duplicates in the proxy-to-stream path would come from.
  • The 2 s response cache needs a stated key (:129). The cache is described only for anonymous pulls, and its key is product and after. principal is visible only to callers with write permission (:131). If the cache is ever applied to signed requests, a response for a writer could be served to a reader. State that the cache applies to anonymous requests only, or add the caller's visibility class to the key.
  • A new subscriber can see old events from before they had access (:118, :131). Within the 7-day replay window, someone who gains read permission can see deleted key names that were never listable to them. The ADR's reasoning is that "the subscriber had read access". That doesn't hold for later grants. Consider starting new private-product subscribers at the current sequence number, or document the exposure.
  • Open question 2 should carry a decision deadline (:247). The indexer's reliance on best-effort capture is the main risk, and the nightly list (:176) is the only backstop.

Simplify (ponytail)

  • :135-149 Push delivery (webhooks, workload identity federation, subscription registry, ownership checks, SSRF controls): cut from v1. Pull already covers source/metadata-catalog and external users. Add push when a real subscriber needs it.
  • :160-168 Per-principal usage Durable Objects and the 15-minute audit-table job: cut. Usage isn't enforced, and a query on the audit table gives the same numbers when someone needs them.
  • :50-63 CloudEvents envelope: use the flat audit row for the Queue and for /.events. Wrap in CloudEvents only when push ships.
  • :93 The 5M-event per-product cap duplicates the 7-day alarm. Keep only the time-based prune.
    · Branch: worktree-adr-017-object-events

💰 Estimated review cost: $0.13 · 0m22s · 4 turns

@github-actions

github-actions Bot commented Nov 21, 2026 •

Copy link
Copy Markdown

🚀 Latest commit deployed to https://source-data-proxy-pr-292.source-coop.workers.dev

  • Date: 2026-11-21T19:05:02Z
  • Commit: 548bf7c

Audit only signed reads, refusals and a 1% anonymous sample; short-poll
/.events instead of holding a Durable Object; day-partitioned audit table;
authorize /.events as ListBucket; define cursor, ordering and idempotency;
state the new wait_until plumbing; add a cost section.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…ations

Add webhook (CloudEvents, Standard Webhooks signing) and OIDC-federated
cloud-queue destinations beside the pull API; compare the two strategies.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…uthenticated

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

This branch was successfully deployed

1 active deployment
preview — fac92432 Deployed Nov 21, 2026 by alukach via Deploy & Test / Deploy #512
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant