From f7df6d745645a4feed7d933248ddb48a979d5602 Mon Sep 17 00:00:00 2001 From: Kurt Overmier Date: Thu, 15 Oct 2026 18:46:18 -0500 Subject: [PATCH 1/2] docs: sync documentation Co-Authored-By: Claude Opus 5.5 --- README.md | 4 ++-- docs/maintenance/issue-sweep.md | 6 +++--- 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/README.md b/README.md index 5a06c01..3268db2 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ # @stackbilt/llm-providers -Development status and resume notes: [issue sweep checkpoint](docs/maintenance/issue-sweep.md). The checkpoint identifies APIs and fixes that are still unreleased. +Development status and resume notes: [issue sweep checkpoint](docs/maintenance/issue-sweep.md). The checkpoint records what has shipped and the next targets. A Cloudflare-first LLM router: Workers AI models are the primary route, and OpenAI, Anthropic, Cerebras, Groq and NVIDIA NIM serve as secondary providers behind the same interface, with automatic failover, graduated circuit breakers, cost tracking, and intelligent retry. When Workers AI is configured and healthy, catalog routing ranks a Workers AI model first for every use case; breaker state, ledger pressure and request requirements (for example vision) can still move a request to another provider. Built for Cloudflare Workers but runs anywhere with a standard `fetch` API. Extracted from a production orchestration platform handling 80K+ LOC across multiple services. @@ -375,7 +375,7 @@ Model selection is driven by a declarative catalog rather than a hardcoded fallb - circuit breaker state (`CLOSED`, `DEGRADED`, `RECOVERING`, `OPEN`) - CreditLedger utilization and projected burn/depletion pressure -The catalog also distinguishes active, compatibility, and retired models. Retired IDs can remain exported for compatibility, but they are not recommendation targets. +The catalog also distinguishes active, compatibility, and retired models. Retired IDs can remain exported for compatibility, but they are not recommendation targets. Entries marked `explicitOnly` (for example the Claude 5.x and GPT-5.6/6 models until #124 lands) are served only when a request names them; recommendations and automatic routing skip them. ```typescript import { diff --git a/docs/maintenance/issue-sweep.md b/docs/maintenance/issue-sweep.md index 242f447..218020a 100644 --- a/docs/maintenance/issue-sweep.md +++ b/docs/maintenance/issue-sweep.md @@ -4,7 +4,7 @@ Checkpoint: 2026-10-15. Repository: `Stackbilt-dev/llm-providers`. ## Resume here -The sweep below is **merged to `main`** and is versioned as 1.22.0 by release PR [#121](https://github.com/Stackbilt-dev/llm-providers/pull/121) (npm publish is a separate operator step): [#119](https://github.com/Stackbilt-dev/llm-providers/pull/119) (sweep plus review fixes) and [#120](https://github.com/Stackbilt-dev/llm-providers/pull/120) (#116 hook isolation). Each went through adversarial review rounds; review-driven compatibility fixes are recorded in the 1.22.0 CHANGELOG entry. The issues stay open for the remaining scope in the table. No live or billable vendor requests were made. +The sweep below is **merged to `main`** and was released as 1.22.0 by release PR [#121](https://github.com/Stackbilt-dev/llm-providers/pull/121) and published to npm; 1.22.1 (#122) and 1.23.0 (#123) followed. Sweep PRs: [#119](https://github.com/Stackbilt-dev/llm-providers/pull/119) (sweep plus review fixes) and [#120](https://github.com/Stackbilt-dev/llm-providers/pull/120) (#116 hook isolation). Each went through adversarial review rounds; review-driven compatibility fixes are recorded in the 1.22.0 CHANGELOG entry. The issues stay open for the remaining scope in the table. No live or billable vendor requests were made. 1. Inspect `git status --short`. Preserve the pre-existing `src/__tests__/factory.test.ts` modification; its diff is line-ending-only when checked with `--ignore-space-at-eol`. 2. #112 phase 1 shipped in 1.22.1; phase 2 (current models, Workers AI deprecations, reasoning-model request parameters, `explicitOnly`) shipped in 1.23.0 via [#123](https://github.com/Stackbilt-dev/llm-providers/pull/123). Next: #124 (Anthropic `tool_use_id`, tool error text and Claude 5.x thinking blocks in tool loops; prerequisite for routing the 5.x models), then #109 atomic spend reservations (needs a design pass first). @@ -49,13 +49,13 @@ Main changes: [factory](../../src/factory.ts), [public exports](../../src/index. New utilities: [response cache identity/validation](../../src/utils/response-cache.ts), [provider SSE lifecycle](../../src/utils/provider-stream.ts), [resilience context](../../src/utils/resilience-context.ts). -New regression suites: [tenant safety](../../src/__tests__/tenant-safety.test.ts) (45 tests), [stream lifecycle](../../src/__tests__/stream-lifecycle.test.ts) (35), [context ownership/recovery](../../src/__tests__/resilience-context.test.ts) (12), [canonical identity/routing](../../src/__tests__/canonical-identity-routing.test.ts) (9). Existing schema-drift tests now reset breaker history between cases; gateway diagnostic tests check failed-plan executability. The pre-existing factory test file was preserved. +New regression suites: [tenant safety](../../src/__tests__/tenant-safety.test.ts) (45 tests), [stream lifecycle](../../src/__tests__/stream-lifecycle.test.ts) (35), [context ownership/recovery](../../src/__tests__/resilience-context.test.ts) (12), [canonical identity/routing](../../src/__tests__/canonical-identity-routing.test.ts) (9). Existing schema-drift tests now reset breaker history between cases; gateway diagnostic tests check failed-plan executability. The pre-existing factory test file was preserved. Later releases added [catalog accuracy](../../src/__tests__/catalog-accuracy.test.ts) (1.22.1) and [model refresh](../../src/__tests__/model-refresh.test.ts) (1.23.0, 125 tests). ## Validation evidence Host: Windows/PowerShell, Node `v24.19.0`, npm `11.17.0`. -- `npm test`: **605 passing tests across 27 files**. +- `npm test`: **789 passing tests across 30 files** at 1.23.0 (605 across 27 when the sweep landed). - `npm run typecheck`: passed. - `npm run test:worker-import`: build and browser/Worker-targeted bundle smoke passed. - Packed artifact: unpacked in a temporary directory, verified 33 JavaScript maps have complete nonempty `sourcesContent`, and required the packed entrypoint to verify all new public exports and client methods. The Linux-oriented `npm run test:package` shell script itself was not run on this Windows host; its artifact checks were exercised manually. From 86e1bd1946dbe058c6cd80c1c761f52112135377 Mon Sep 17 00:00:00 2001 From: Kurt Overmier Date: Thu, 15 Oct 2026 19:31:52 -0500 Subject: [PATCH 2/2] docs: correct explicit-only reasons, test counts and validation evidence Addresses the PR #125 review. - README and checkpoint: only the Claude 5.x models wait on #124; the OpenAI models are explicit-only to keep the gpt-4o-mini default. List every explicit-only model, including gpt-5.5 and SEA-LION. - Checkpoint: per-suite test counts measured at 1.23.0, add the missing hook-isolation suite, drop the unverified historical total, and split validation evidence into what was re-run at 1.23.0 and what dates from the original sweep. Co-Authored-By: Claude Opus 5.5 --- README.md | 2 +- docs/maintenance/issue-sweep.md | 15 ++++++++++----- 2 files changed, 11 insertions(+), 6 deletions(-) diff --git a/README.md b/README.md index 3268db2..b7da182 100644 --- a/README.md +++ b/README.md @@ -375,7 +375,7 @@ Model selection is driven by a declarative catalog rather than a hardcoded fallb - circuit breaker state (`CLOSED`, `DEGRADED`, `RECOVERING`, `OPEN`) - CreditLedger utilization and projected burn/depletion pressure -The catalog also distinguishes active, compatibility, and retired models. Retired IDs can remain exported for compatibility, but they are not recommendation targets. Entries marked `explicitOnly` (for example the Claude 5.x and GPT-5.6/6 models until #124 lands) are served only when a request names them; recommendations and automatic routing skip them. +The catalog also distinguishes active, compatibility, and retired models. Retired IDs can remain exported for compatibility, but they are not recommendation targets. Entries marked `explicitOnly` are served only when a request names them; recommendations and automatic routing skip them. In 1.23.0 these are the Claude 5.x models (until #124 fixes Anthropic tool loops), `gpt-6-astra`, `gpt-5.6-sol`, `gpt-5.6-terra`, `gpt-5.6-luna` and `gpt-5.5` (so the OpenAI default stays `gpt-4o-mini`), and `@cf/aisingapore/gemma-sea-lion-v4-27b-it`. ```typescript import { diff --git a/docs/maintenance/issue-sweep.md b/docs/maintenance/issue-sweep.md index 218020a..dfa6faf 100644 --- a/docs/maintenance/issue-sweep.md +++ b/docs/maintenance/issue-sweep.md @@ -28,7 +28,7 @@ The sweep below is **merged to `main`** and was released as 1.22.0 by release PR | [#116](https://github.com/Stackbilt-dev/llm-providers/issues/116) | Quota observers and streaming terminal/schema observers are guarded; stream body deadlines, cancellation, fault/race tests and awaited stream settlement were added. | Hook replay fixed in #120: every factory/adapter hook is guarded. Remaining: non-streaming body deadlines, retry sleep aborts, Retry-After/vendor error fixtures, unproduced hook events, floating non-streaming cache/quota writes, request/tool-loop signals, workerd and coverage baselines | | [#118](https://github.com/Stackbilt-dev/llm-providers/issues/118) | Coordinating roadmap receives this checkpoint and priorities. | Track reviewed/merged implementation phases separately from local progress. | -Other open items are unchanged: #112 (phase 2 shipped in 1.23.0; remaining: Claude 5.x thinking-block round trip and `tool_use_id` in tool loops, tracked in #124 (then lift `explicitOnly` on the 5.x and GPT-5.6 models), Cerebras/Groq/NVIDIA prices and IDs, GPT-6 Luna / 6.1 Sol tool-calling restrictions, OpenAI cached-input pricing, custom-model escape hatch, unknown-cost semantics, provenance canary), #114 (failure demo/README restructuring/repo cleanup), #117 (optional AI SDK adapter), #77 (supply-chain workflow), #78/#79/#80 (new provider proposals). #90/#95 have existing code surfaces but were not re-audited for closure during this sweep. +Other open items are unchanged: #112 (phase 2 shipped in 1.23.0; remaining: Claude 5.x thinking-block round trip, `tool_use_id` and tool error text in tool loops, tracked in #124 (then lift `explicitOnly` on the Claude 5.x models); deciding when the explicit-only OpenAI models (`gpt-6-astra`, `gpt-5.6-sol`/`-terra`/`-luna`, `gpt-5.5`) become routable, which would change the `gpt-4o-mini` default; Cerebras/Groq/NVIDIA prices and IDs, GPT-6 Luna / 6.1 Sol tool-calling restrictions, OpenAI cached-input pricing, custom-model escape hatch, unknown-cost semantics, provenance canary), #114 (failure demo/README restructuring/repo cleanup), #117 (optional AI SDK adapter), #77 (supply-chain workflow), #78/#79/#80 (new provider proposals). #90/#95 have existing code surfaces but were not re-audited for closure during this sweep. ## Public behavior to retain @@ -49,15 +49,20 @@ Main changes: [factory](../../src/factory.ts), [public exports](../../src/index. New utilities: [response cache identity/validation](../../src/utils/response-cache.ts), [provider SSE lifecycle](../../src/utils/provider-stream.ts), [resilience context](../../src/utils/resilience-context.ts). -New regression suites: [tenant safety](../../src/__tests__/tenant-safety.test.ts) (45 tests), [stream lifecycle](../../src/__tests__/stream-lifecycle.test.ts) (35), [context ownership/recovery](../../src/__tests__/resilience-context.test.ts) (12), [canonical identity/routing](../../src/__tests__/canonical-identity-routing.test.ts) (9). Existing schema-drift tests now reset breaker history between cases; gateway diagnostic tests check failed-plan executability. The pre-existing factory test file was preserved. Later releases added [catalog accuracy](../../src/__tests__/catalog-accuracy.test.ts) (1.22.1) and [model refresh](../../src/__tests__/model-refresh.test.ts) (1.23.0, 125 tests). +New regression suites: [tenant safety](../../src/__tests__/tenant-safety.test.ts), [stream lifecycle](../../src/__tests__/stream-lifecycle.test.ts), [context ownership/recovery](../../src/__tests__/resilience-context.test.ts), [canonical identity/routing](../../src/__tests__/canonical-identity-routing.test.ts). Existing schema-drift tests now reset breaker history between cases; gateway diagnostic tests check failed-plan executability. The pre-existing factory test file was preserved. Later work added [hook isolation](../../src/__tests__/hook-isolation.test.ts) (#120, 1.22.0), [catalog accuracy](../../src/__tests__/catalog-accuracy.test.ts) (1.22.1) and [model refresh](../../src/__tests__/model-refresh.test.ts) (1.23.0). Test counts at 1.23.0: tenant safety 46, stream lifecycle 43, context ownership/recovery 14, canonical identity/routing 10, hook isolation 20, catalog accuracy 27, model refresh 125. ## Validation evidence -Host: Windows/PowerShell, Node `v24.19.0`, npm `11.17.0`. +Host: Windows, Node `v24.19.0`, npm `11.17.0`. -- `npm test`: **789 passing tests across 30 files** at 1.23.0 (605 across 27 when the sweep landed). +Re-run at 1.23.0 (2026-10-15), and again by the publish workflow on Ubuntu with Node 22: + +- `npm test`: **789 passing tests across 30 files**. - `npm run typecheck`: passed. -- `npm run test:worker-import`: build and browser/Worker-targeted bundle smoke passed. +- `npm run build`, `npm run test:package` and `npm run test:worker-import`: passed. + +From the original sweep (before 1.22.0), not re-run since: + - Packed artifact: unpacked in a temporary directory, verified 33 JavaScript maps have complete nonempty `sourcesContent`, and required the packed entrypoint to verify all new public exports and client methods. The Linux-oriented `npm run test:package` shell script itself was not run on this Windows host; its artifact checks were exercised manually. - Whitespace check passed for the sweep's modified tracked files with CR-at-EOL allowed, excluding the preserved factory-test diff. - No live providers, workerd execution, minimum Node version, NodeNext matrix, measured bundle size, coverage baseline or published-release validation was performed.