docs(scribe): P/A/I for granola-tools (CL-8775)

ARCHITECTURE.md

@@ -1,48 +1,159 @@
-# Architecture
+# @corbits/granola — Architecture
 
-## Three faces, one dependency direction
+## Shape
 
-This package has three independent faces, each with its own source directory:
+Three faces, one dependency direction.
 
-- `src/tools/` — the Granola API client and the tools an agent calls (fetch a note,
-  search notes). Published as the package root, `@corbits/granola`.
-- `src/ingress/` — the extension that receives Granola webhooks, verifies signatures,
-  and dispatches notes to handlers. Published as the subpath export
+```
+host
+  │
+  ├─ grant tools ──▶  @corbits/granola          src/tools
+  │                         ▲
+  │                         │ client + note/bucket shapes
+  ├─ mount webhook ─▶  @corbits/granola/ingress  src/ingress
+  │                         ▲
+  │                         │ binding store + webhook payload
+  └─ after ack ─────▶  @corbits/granola/ingest   src/ingest
+```
+
+- `src/tools/` — Granola REST client and the two grantable tool definitions.
+  Published as the package root, `@corbits/granola`.
+- `src/ingress/` — signature verification, Hono mount, durable folder
+  bindings, and webhook-endpoint reconciliation. Published as
   `@corbits/granola/ingress`.
-- `src/ingest/` — the host-agnostic pipeline that turns an acked webhook event into a
-  persisted transcript and a dispatched bucket handler: fetch note, resolve bucket,
-  persist, capture knowledge, hand off. Published as the subpath export
-  `@corbits/granola/ingest`.
-
-`src/ingress` depends on `src/tools` for the client. `src/ingest` depends on both —
-`src/tools` for the client and note shapes, `src/ingress` for the binding store and
-webhook payload it processes. `src/tools` must never depend on `src/ingress` or
-`src/ingest`, or on any hub, mounting, extension or webhook machinery at all — the
-operator requirement is that someone can import the tools and grant them to any agent
-like any other plain Interchange tool, and nothing hub-shaped comes along for the ride.
+- `src/ingest/` — host-agnostic pipeline after ack: fetch note, resolve
+  bucket, persist transcript, capture knowledge, dispatch handler.
+  Published as `@corbits/granola/ingest`.
+
+`src/ingress` depends on `src/tools` for the client and bucket shapes.
+`src/ingest` depends on both — `src/tools` for the client and note shapes,
+`src/ingress` for the binding store and the webhook payload it processes.
+`src/tools` must never depend on `src/ingress` or `src/ingest`, or on any
+hub, mounting, extension, or webhook machinery. Someone can import the
+tools and grant them like any other Interchange tool; nothing hub-shaped
+comes along.
 
 ## The rule is enforced, not just documented
 
-A convention nobody checks gets violated. `scripts/check-deps.ts` greps `src/tools` for
-any reference to `ingress` and fails the build if it finds one. It runs in CI as its own
-job, separate from typecheck and test, so a dependency-direction leak fails loudly and
-specifically rather than getting buried in an unrelated job's log.
+`scripts/check-deps.ts` greps `src/tools` for any import of `ingress` and
+fails the build if it finds one. It runs in CI as its own job, separate
+from typecheck and test.
+
+The intended rule is stronger: tools must not import ingest either.
+`check-deps` does not yet grep for `ingest`. Reviewing a change to
+`src/tools/`: if it needs anything from `src/ingress/` or `src/ingest/`,
+the boundary is wrong, not a reason to import across it.
+
+## Why subpath exports, not three packages
+
+The client, the webhook, and the pipeline version and release together — a
+Granola API surface change affects all three. Separate packages would let
+them drift for no benefit here. One package with three `exports` entries
+keeps them versioned together while still letting a consumer install
+`@corbits/granola` and pull nothing from ingress or ingest into the import
+graph until they ask for those subpaths.
+
+## Tools
+
+The client is Bearer-auth `fetch` against Granola's public API. Responses
+are arktype-validated. Folder `name` from the API is published as `title`
+so notes, summaries, and folders share field names. `folder_membership` is
+left `unknown`; ingest, not the client, interprets it.
+
+Tool definitions are plain `{ name, description, handler }`. Handlers are
+placeholders (`not implemented`). The catalog is `GRANOLA_TOOL_DEFINITIONS`;
+callers dispatch on `name`.
+
+## Ingress
+
+Verification is pure: Standard Webhooks HMAC-SHA256 over
+`{webhook-id}.{webhook-timestamp}.{raw body}`, keyed on the base64-decoded
+signing secret (`whsec_` prefix stripped). Multiple space-separated `v1,`
+signatures are accepted (rotation). Stale timestamps and short keys fail
+closed.
+
+`mountGranolaWebhook` is the only Hono concretion. Granola is not a
+principal; the signature is the only authentication. No secret, or an
+unusable secret, means the route is not mounted — that is a valid
+configuration, not an error.
+
+Granola gives the endpoint 15 seconds and retries only on non-2xx. The
+route acks `202` before `onEvent` resolves. A post-ack failure is logged,
+not retried by the vendor. In-memory `event_id` dedupe (bounded FIFO)
+guards Granola's at-least-once duplicates; it remembers an id only after
+`onEvent` succeeds. A host restart forgets the set.
+
+`GranolaBindingsPort` is the persistence seam. `createGranolaBindingStore`
+is seed-on-first-write: `list()` never writes; seed bindings remain until
+an explicit `replaceAll`. Cache is process-local — a binding edit on one
+replica is invisible to others until restart.
+
+`ensureGranolaWebhook` reconciles the vendor subscription at boot.
+`signing_secret` is returned only on create and cannot be fetched later.
+A matching endpoint with no env secret leaves the webhook unmounted rather
+than delete-and-recreate (which would loop every restart). A redacted URL
+in the list with no exact match does not create (duplicate risk) and falls
+back to the env secret. API failures never crash host boot.
+
+`reconcileGranolaWebhookFolders` is the `onChange` target so a binding
+edit can converge `folder_ids` without a reboot.
+
+## Ingest
+
+`createGranolaIngest` returns the `onEvent` function the mount calls after
+verify, dedupe, and ack. The pipeline is generic over the host:
+
+| Seam | Role |
+| --- | --- |
+| `client` | Fetch the note (with transcript). |
+| `bindingStore` | Folder id → `{ type, channel }`. |
+| `transcripts` | `hasTranscript` / `persist`; `TRef` is opaque. |
+| `captureKnowledge` | Enrichment after persist; failures are swallowed. |
+| `lifecycle` | One hook per human-visible event; `TAnchor` is opaque. |
+| `handlers` | Per-bucket-type behavior (`diligence` / `internal`). |
+
+```
+event ─▶ fetch note ─▶ resolve bucket ─▶ already-processed?
+                                      │ no
+                                      ▼
+                         onProcessingStarted (anchor)
+                                      │
+                         not ready? ──retry once ── give up
+                                      │ ready
+                                      ▼
+                         persist ─▶ capture ─▶ handler
+```
+
+A note with no bound folder is ignored (nowhere to notify). Multiple
+folder matches take the first. `folder_membership` is tolerated as an id,
+a list of ids, or objects with `folderId` / `id`; unexpected shapes become
+"no bucket" rather than a crash.
+
+The durable already-processed gate is a persisted transcript, not the
+in-flight map. A re-added note does not silently skip or blindly rerun;
+`onAlreadyProcessed` asks the human. `reprocess` / `reprocessPinned`
+bypass that gate. Concurrent events for the same `note_id` drop with
+`onDuplicateEvent` rather than queue.
 
-Reviewing a change to `src/tools/`: if it needs anything from `src/ingress/`, that is a
-sign the boundary is wrong, not a reason to import across it.
+In-flight state and the not-ready retry timer are process-local. A restart
+inside the retry window loses the timer; the next webhook event starts
+from scratch.
 
-## Why subpath exports, not two packages
+## Failure modes
 
-The tools and the ingress extension version and release together — a Granola API
-surface change affects both. Two `package.json` files (as some sibling repositories use
-for genuinely independent packages) would let them drift out of sync for no benefit
-here. One package with two `exports` entries keeps them versioned together while still
-letting a consumer install `@corbits/granola` alone and get nothing from `src/ingress/`
-in their dependency graph.
+- Hung Granola HTTP must not hold ingest's in-flight guard forever (client
+  timeout) or wedge host boot (registration timeout).
+- `note.access_granted` + 404 means the note is not generated yet, not a
+  fetch failure. Recovery is `note.generated` after a human generates
+  notes in Granola.
+- Knowledge-capture errors must not skip the bucket handler.
+- Handler errors are reported through `onHandlerFailed`; they do not
+  un-ack the webhook.
+- Lifecycle hooks that throw are logged; they do not rewrite pipeline
+  control flow except `onProcessingStarted`, which aborts the run if it
+  cannot establish an anchor.
 
-## What's not here yet
+## What is still placeholder
 
-This is a skeleton. `src/tools/client.ts` and `src/tools/tools.ts` have no real Granola
-API calls; `src/ingress/webhook.ts` has no real signature verification or dispatch. That
-extraction is scoped and lands separately — this repository establishes the structure
-and the CI gate it has to pass.
+Tool handlers. The client, webhook verification, registration, binding
+store, and ingest pipeline are real.

CONTRIBUTING.md

@@ -2,7 +2,7 @@
 
 A small, deliberately boring codebase: strict TypeScript, no magic.
 
-Setup and the commands are in the [README](./README.md#working-on-it). `bun run
+Setup and the commands are in the [README](./README.md#development). `bun run
 typecheck` must be clean — it is its own CI step, and `any` is not a way past it.
 
 ## Dependency rule

IMPLEMENTATION.md

@@ -0,0 +1,155 @@
+# @corbits/granola — Implementation
+
+## Package
+
+- Name: `@corbits/granola` `0.1.0`
+- License: LGPL-2.1-only
+- Engines: Node `>= 24` consumes built `dist/`. Bun loads TypeScript source
+  via the `bun` export condition.
+- Public exports:
+
+  | Subpath | Bun | Node / types |
+  | --- | --- | --- |
+  | `.` | `src/tools/index.ts` | `dist/tools/index.js` + `.d.ts` |
+  | `./ingress` | `src/ingress/index.ts` | `dist/ingress/index.js` + `.d.ts` |
+  | `./ingest` | `src/ingest/index.ts` | `dist/ingest/index.js` + `.d.ts` |
+
+- npm tarball: `dist/`, `src/` (tests excluded), `LICENSE`, `README.md`.
+  These design docs live at the repository root and are not in `files`.
+- `sideEffects: false`. The package never reads `process.env`.
+
+## Runtime dependencies
+
+- `arktype` — response, payload, and binding validation
+- `@intx/log` — ingress mount, registration, binding store, ingest
+- `hono` `^4` — **peer**, required only for `@corbits/granola/ingress`
+- `node:crypto` — HMAC-SHA256 and `timingSafeEqual` in webhook verify
+
+## Install
+
+```sh
+npm add @corbits/granola
+pnpm add @corbits/granola
+yarn add @corbits/granola
+bun add @corbits/granola
+```
+
+Ingress also needs `hono` in the host.
+
+## Public surface
+
+From `@corbits/granola`:
+
+- `createGranolaClient({ apiKey, baseUrl?, fetchImpl? })` —
+  `{ getNote, listNotes, listFolders }`
+- `transcriptText`, `speakerLabel`, `GranolaApiError`, `GranolaNote`
+- `GranolaBucketType`, `GranolaBucketsArray`
+- `fetchNoteTool`, `searchNotesTool`, `GRANOLA_TOOL_DEFINITIONS`
+
+From `@corbits/granola/ingress`:
+
+- `verifyGranolaSignature`, `decodeSigningSecret`, `signGranolaPayload`,
+  `parseGranolaPayload`, `MIN_SIGNING_KEY_BYTES`, `KNOWN_GRANOLA_EVENT_TYPES`
+- `createGranolaBindingStore`
+- `ensureGranolaWebhook`, `reconcileGranolaWebhookFolders`
+- `mountGranolaWebhook`
+
+From `@corbits/granola/ingest`:
+
+- `createGranolaIngest` — callable `onEvent` plus `reprocess` /
+  `reprocessPinned`
+
+## Client
+
+Default `baseUrl`: `https://public-api.granola.ai/v1`. Docs:
+https://docs.granola.ai (OpenAPI at
+https://docs.granola.ai/api-reference/openapi.json).
+
+| Method | HTTP |
+| --- | --- |
+| `getNote(id, { includeTranscript? })` | `GET /notes/:id` (`include=transcript`) |
+| `listNotes({ folderId, cursor?, pageSize? })` | `GET /notes?folder_id&page_size&cursor` |
+| `listFolders({ cursor?, pageSize? })` | `GET /folders?page_size&cursor` |
+
+Auth: `Authorization: Bearer <apiKey>`. Also sends `Accept` and
+`Content-Type: application/json` on GET (Granola otherwise answers
+non-JSON on some routes). Never sets `User-Agent`.
+
+Timeout: `AbortSignal.timeout(30_000)` on every client request. Non-2xx
+throws `GranolaApiError` with `status` and raw `body`. `null` cursors
+normalize to absent. Default `pageSize` is 30. Folder API `name` maps to
+public `title`.
+
+## Tools
+
+| Name | Args (intended) | Handler today |
+| --- | --- | --- |
+| `granola_fetch_note` | `{ noteId }` | throws `not implemented` |
+| `granola_search_notes` | `{ query }` | throws `not implemented` |
+
+## Webhook protocol
+
+Standard Webhooks (https://docs.granola.ai/webhooks). Signed content:
+`{webhook-id}.{webhook-timestamp}.{raw body}`. Secret is base64, commonly
+prefixed `whsec_`; HMAC key is the decoded bytes. Reject if decoded key is
+shorter than `MIN_SIGNING_KEY_BYTES` (16). Timestamp skew:
+`SIGNATURE_TOLERANCE_SECONDS` = 300.
+
+Payload (arktype): `event_id`, `event_type` (any string), `note_id`,
+`occurred_at`, optional `data.changed_fields`. Known event names (not a
+closed union on the wire): `note.generated`, `note.regenerated`,
+`note.edited`, `note.access_granted`.
+
+## Mount
+
+Absolute route: `POST /api/granola/webhook`. Body cap: 1 MiB (413
+`payload_too_large`). Missing/bad signature: 401 with `missing_headers` /
+`invalid_timestamp` / `stale_timestamp` / `bad_signature`. Verified but
+malformed JSON: 202 (terminal for Granola; logged). Duplicate `event_id`:
+202. Success: 202, then `onEvent` in the background. Default dedupe
+capacity: 1000.
+
+## Registration
+
+Granola REST, 10s timeout, no `User-Agent`.
+
+| | |
+| --- | --- |
+| List | `GET /webhook-endpoints` |
+| Create | `POST /webhook-endpoints` (`url`, `scopes: ["public"]`, `events`, `folder_ids`) |
+| Update | `PATCH /webhook-endpoints/:id` |
+
+Create is the only call that returns `signing_secret`. Target URL is
+`{publicUrl}/api/granola/webhook`. Host env names in comments/logs:
+`GRANOLA_WEBHOOK_SECRET`, `GRANOLA_PUBLIC_URL` (or a portal origin) — the
+library still takes them as function arguments.
+
+## Bindings
+
+`GranolaBucket`: `{ folderId, type: "diligence" \| "internal", channel }`.
+Port: `load({ tenantId })` → `{ bindings, version } | undefined`;
+`save({ tenantId, principalId, bindings })`.
+
+## Ingest
+
+Default not-ready retry: 60s, once, via `setTimeout` (overridable
+`retryDelayMs`). Persist text is transcript lines (`Speaker: text`), else
+`summary_markdown`, else `summary_text`. Synthetic event types for
+operator re-entry: `operator.reprocess`, `operator.pinned-diligence`.
+
+## Development
+
+```sh
+bun install
+bun run typecheck
+bun run test
+bun run test:tools
+bun run test:ingress
+bun run test:ingest
+bun run check-deps
+bun run build
+```
+
+CI (`.github/workflows/test.yml`): `check-deps`, `typecheck`, `test-tools`,
+`test-ingress`, `test-ingest`, `build` as separate jobs; a Node 24 job
+runs `npm install`, `typecheck`, and `build`. Bun in CI is `1.3.14`.