CI: native SQLite install check, cached apt packages for e2e, and a date-proof upgrade test by irparent · Pull Request #899 · iris-eval/mcp-server

irparent · 2026-10-25T19:13:47Z

Three ways CI turned red without anything wrong in the code it tests.

1. A dropped native SQLite module now fails as an install problem

better-sqlite3 is an optional dependency. When its prebuilt download and its source build both fail, npm drops it and reports success. On 2026-10-24 the nightly worker exit run's macOS / Node 22 cell did exactly that ("added 310 packages"), then failed all 50 of its search-worker runs on "The native SQLite module (better-sqlite3) is not installed".

Every job that needs the native driver (worker exit, test, and the native cells of search index and stall guard) now runs scripts/ci/native-sqlite-installed.sh after npm ci. It opens a database with the module. If that fails, it installs once more with the install scripts' output in the log, and fails with that cause if the module still does not load.

2. e2e installs the browsers' system libraries from a cache

On 2026-10-22 the Ubuntu mirror served apt's package index at 9.7 MB/s but the 54 MB of packages at about 50 KB/s. The step hit its 10-minute bound on main and on #874, #880 and #881. apt now downloads into a folder that actions/cache keeps per runner image and Playwright version. With that folder restored, apt reads only the index from the mirror. A new runner image restores the previous image's packages and downloads only what changed.

Checked in an ubuntu:24.04 container: the packages land in the configured folder, and with the mirror's hosts blocked a reinstall reports Need to get 0 B and installs every package from it.

3. The upgrade test's retention sweep no longer depends on the day it runs

tests/upgrade/from-previous-release.test.ts has the previous release sweep traces older than 30 days from a fixture dated 2025-09-24 and 2026-09-24. From 2026-10-24 the sweep also took a 2026-09-24 trace, and swept: 2 became 3 on every OS. The cutoff is now 2026-01-01 whatever the day: past the 2025 trace, before the 2026 ones. npx vitest run --config tests/upgrade/vitest.config.ts: 10 of 10 pass.

actionlint (the image scripts/preflight.mjs pins) passes on every workflow, and shellcheck on the new script.

🤖 Generated with Claude Code

…2e installs its system libraries from a cache Two ways CI went red for reasons outside the code under test: - better-sqlite3 is optional, so when its prebuilt download and its source build both fail, npm drops it and reports success. On 2026-10-24 the nightly worker-exit run's macOS Node 22 cell did exactly that, then failed all 50 search-worker runs on the module's absence. Every job that needs the native driver now runs scripts/ci/native-sqlite-installed.sh after npm ci: it opens a database with the module and, if that fails, installs once more with the install scripts' output in the log, then fails with that cause if the module still does not load. - On 2026-10-22 the Ubuntu mirror served apt's index at 9.7 MB/s and the 54 MB of browser system libraries at about 50 KB/s, so the e2e step hit its 10-minute bound on main and on three pull requests. apt now downloads into a folder actions/cache keeps per runner image and Playwright version; with it restored, apt reads only the index from the mirror. Checked in an ubuntu:24.04 container with the mirror's hosts blocked: "Need to get 0 B", every package installed from the folder. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…day the test runs The upgrade test has the previous release sweep traces older than 30 days from a fixture whose traces are dated 2025-09-24 and 2026-09-24. From 2026-10-24 the sweep also took a 2026-09-24 trace, so "swept: 2" became 3 on every operating system. The cutoff is now 2026-01-01 whatever the day: past the 2025 trace, before the 2026 ones. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

vercel · 2026-10-25T19:13:48Z

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment

Project	Deployment	Actions	Updated
website	Ignored	Preview	Oct 25, 2026 7:20pm UTC

…l step, not a check the preflight mirrors The preflight mirrors the jobs whose checks can run locally, and a test requires it to run every script those jobs run. The new install check is not a check of the tree: it confirms a runner's fresh npm ci kept the optional native module and, when it did not, runs npm ci again, which must never run in a checkout whose node_modules other worktrees share. It is now named with that reason. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

github-actions · 2026-10-25T19:22:41Z

Iris gate — 1 of 2 tripped `--fail-on detector_veto`

iris-eval ingest: 3 stored, 1 tripped --fail-on detector_veto (2 of 3 evaluated in dataset "release-gate")

Trace	Verdict	Basis	Rules, classes or missing inputs	Evidence
`bc1b09f63c5f626071d7abf697b3e1f8`	failed	`detector_veto` + `risk_over_loss`	no_pii, pii_leak, credential_leak	no_pii: AWS Access Key (output 45–65)

Verdict basis	Traces
`detector_veto`	2
`clean`	1

Unjudged questions: task_completed (3), tool_use_correct (3) — a trace that did not carry what a rule needs.

_{tests/fixtures/ci-gate/traces.ndjson · 3 evaluated · dataset release-gate: 2 in the gate · exit 1 · what the bases mean}

github-actions · 2026-10-25T19:22:43Z

Iris gate — 1 stored, nothing tripped `--fail-on any`

iris-eval ingest: 1 stored, 0 tripped --fail-on any

Verdict basis	Traces
`clean`	1

Unjudged questions: task_completed (1), tool_use_correct (1) — a trace that did not carry what a rule needs.

_{tests/fixtures/ci-gate/clean.ndjson · 1 evaluated · exit 0 · what the bases mean}

The same pins those PRs proposed (commit sha and version comment), applied to every workflow as it is now, ci.yml included after #899. The four codeql-action steps in codeql.yml and scorecard.yml move together, as CodeQL requires. 73 lines in 12 workflows; no old pin is left. - actions/setup-node v7.0.0 (8207627) -> v7.1.0 (949feb2), 43 lines - actions/upload-artifact v7.0.1 (043fb46) -> v7.0.2 (cf430e0), 13 lines - actions/download-artifact v8.0.1 (3e5f45b) -> v8.0.2 (9000827), 12 lines - github/codeql-action init, autobuild, analyze, upload-sarif v4.38.2 (2892aa5) -> v4.39.0 (6f2ac81), 4 lines - ossf/scorecard-action v2.4.4 (2d11466) -> v2.5.0 (fa7117d), 1 line Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

… updates (#901) * build: pnpm 11 and later build Iris from a clone again; two dependency updates unblocked - pnpm 11 stopped reading settings from package.json, so `pnpm install` refused the install scripts of better-sqlite3 and esbuild (ERR_PNPM_IGNORED_BUILDS) and exited 1 on pnpm 11 and 12, and `pnpm install && pnpm run build`, the command Glama runs, never built. The allow-list moves back to pnpm-workspace.yaml, as `allowBuilds` (pnpm 11 and later) and `onlyBuiltDependencies` (pnpm 10). Checked in node:22 from a clean tree on pnpm 10.34.9, 11.28.8 and 12.15.0: install and build exit 0, better-sqlite3 loads, and check-tools-listed lists all 12 tools. The fresh-clone-build job now runs on pnpm 10 and on the newest pnpm, and requires the native module to have been built. - jsdom 30.1.3 reports the computed `background` shorthand in full, so two dashboard badge tests compare `backgroundColor` instead (unblocks #893). - openai 3 is ignored in examples/otel-recipes: llama-index-llms-openai 0.8.2 and 0.8.3 both declare openai<3 (#851). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * deps: the root lockfile, Dependabot #869 #880 #881 #882 in one Lockfile-only updates within the existing package.json ranges. vitest and @vitest/coverage-v8 move together (they pin each other exactly), as do the typescript-eslint parser and plugin. Each package is at the newest patch on the line its PR proposed that is at least five days old (the repo's Dependabot cooldown); @modelcontextprotocol/sdk 1.32.2 (published 2026-10-23) is left for the next run. vite stays 8.3.5: #880's own lockfile had written it back to 8.3.4. - @eslint-community/eslint-utils 4.9.1 -> 4.10.1 - @modelcontextprotocol/sdk 1.32.0 -> 1.32.1 - @playwright/test 1.63.0 -> 1.64.1 - @types/node 26.6.3 -> 26.6.5 - @typescript-eslint/eslint-plugin 8.71.0 -> 8.71.2 - @typescript-eslint/parser 8.71.0 -> 8.71.2 - @typescript-eslint/project-service 8.71.0 -> 8.71.2 - @typescript-eslint/scope-manager 8.71.0 -> 8.71.2 - @typescript-eslint/tsconfig-utils 8.71.0 -> 8.71.2 - @typescript-eslint/type-utils 8.71.0 -> 8.71.2 - @typescript-eslint/types 8.71.0 -> 8.71.2 - @typescript-eslint/typescript-estree 8.71.0 -> 8.71.2 - @typescript-eslint/utils 8.71.0 -> 8.71.2 - @typescript-eslint/visitor-keys 8.71.0 -> 8.71.2 - @vitest/coverage-v8 5.0.3 -> 5.0.5 - @vitest/istanbul-lib-coverage 1.0.1 -> 1.0.2 - @vitest/istanbul-lib-report 1.0.1 -> 1.0.2 - @vitest/mocker 5.0.3 -> 5.0.5 - @vitest/spy 5.0.3 -> 5.0.5 - acorn 8.16.0 -> 8.19.0 - ast-v8-to-istanbul 1.0.6 -> 1.0.7 - content-disposition 1.0.1 -> 2.0.1 - eslint 10.11.0 -> 10.13.0 - eslint > ajv 6.14.0 -> 6.15.0 - express 5.2.1 -> 5.3.0 - express-rate-limit 8.7.0 -> 8.7.1 - express > content-type 2.1.0 (new nested copy) - magic-string 1.4.2 -> 1.4.3 - minimatch 10.2.5 -> 10.2.6 - nanoid 3.3.19 -> 3.3.20 - obug 2.2.1 -> 3.0.0 - pino 10.3.1 -> 10.4.0 - pino > real-require 1.0.0 (new nested copy) - playwright 1.63.0 -> 1.64.1 - playwright-core 1.63.0 -> 1.64.1 - postcss 8.5.28 -> 8.5.31 - prettier 3.9.9 -> 3.9.10 - safe-regex2 5.1.1 -> 5.1.2 - tinybench 6.2.0 -> 6.2.2 - tinyrainbow 3.1.1 -> 3.2.0 - vitest 5.0.3 -> 5.0.5 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * deps(dashboard): Dependabot #891 #893 in one package.json takes the ranges #891 and #893 set, plus @typescript-eslint/parser ^8.71.2 to match the plugin it pairs with. The storybook addons lock at 10.6.3, the version storybook itself is already on. - @asamuzakjp/css-color 7.1.2 -> 7.1.7 - @asamuzakjp/dom-selector 9.2.2 -> 10.0.4 - @csstools/css-calc 3.4.1 -> 3.4.3 - @csstools/css-color-parser 4.2.4 -> 4.2.6 - @csstools/css-parser-algorithms 4.0.1 -> 4.0.2 - @csstools/css-syntax-patches-for-csstree 1.1.14 -> 1.1.15 - @storybook/addon-docs 10.6.1 -> 10.6.3 - @storybook/builder-vite 10.6.1 -> 10.6.3 - @storybook/react 10.6.1 -> 10.6.3 - @storybook/react-dom-shim 10.6.1 -> 10.6.3 - @storybook/react-vite 10.6.1 -> 10.6.3 - @testing-library/user-event 14.6.7 -> 14.6.8 - @typescript-eslint/eslint-plugin 8.71.0 -> 8.71.2 - @typescript-eslint/parser 8.71.0 -> 8.71.2 - @typescript-eslint/project-service 8.71.0 -> 8.71.2 - @typescript-eslint/scope-manager 8.71.0 -> 8.71.2 - @typescript-eslint/tsconfig-utils 8.71.0 -> 8.71.2 - @typescript-eslint/type-utils 8.71.0 -> 8.71.2 - @typescript-eslint/types 8.71.0 -> 8.71.2 - @typescript-eslint/typescript-estree 8.71.0 -> 8.71.2 - @typescript-eslint/utils 8.71.0 -> 8.71.2 - @typescript-eslint/visitor-keys 8.71.0 -> 8.71.2 - @vitejs/plugin-react 6.1.1 -> 6.1.2 - @vitest/mocker 5.0.3 -> 5.0.5 - @vitest/spy 5.0.3 -> 5.0.5 - data-urls 7.0.0 -> 8.0.0 - eslint 10.11.0 -> 10.13.0 - jsdom 30.1.1 -> 30.1.3 - jsdom > whatwg-url 17.1.2 (nested copy removed; the hoisted copy now satisfies it) - lucide-react 1.52.0 -> 1.58.0 - magic-string 1.4.2 -> 1.4.3 - obug 2.2.1 -> 3.0.0 - react 19.3.0 -> 19.3.1 - react-dom 19.3.0 -> 19.3.1 - resolve 1.22.12 -> 1.22.13 - tinybench 6.2.0 -> 6.2.2 - tr46 6.0.0 -> 7.0.0 - vitest 5.0.3 -> 5.0.5 - whatwg-url 16.0.1 -> 17.3.0 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * deps(website): Dependabot #872 #874 #895 in one package.json takes the ranges #874 and #895 set; eslint-config-next stays an exact 16.4.2, the version next is on. - @babel/generator 7.29.7 -> 7.29.8 - @babel/helpers 7.29.7 -> 7.29.10 - @babel/parser 7.29.7 -> 7.29.9 - @babel/traverse 7.29.7 -> 7.29.10 - @babel/types 7.29.7 -> 7.29.8 - @next/eslint-plugin-next 16.3.8 -> 16.4.2 - @radix-ui/primitive 1.1.7 -> 1.2.0 - @radix-ui/react-collection 1.1.15 -> 1.2.0 - @radix-ui/react-collection > @radix-ui/react-compose-refs 1.1.5 (nested copy removed; the hoisted copy now satisfies it) - @radix-ui/react-compose-refs 1.1.5 (new nested copy) - @radix-ui/react-context 1.2.2 -> 1.3.0 - @radix-ui/react-direction 1.1.4 -> 1.2.0 - @radix-ui/react-presence 1.1.10 -> 1.1.11 - @radix-ui/react-primitive 2.1.10 -> 2.2.0 - @radix-ui/react-roving-focus 1.1.19 -> 1.2.1 - @radix-ui/react-roving-focus > @radix-ui/react-compose-refs 1.1.5 (nested copy removed; the hoisted copy now satisfies it) - @radix-ui/react-slot 1.3.3 -> 1.5.0 - @radix-ui/react-slot > @radix-ui/react-compose-refs 1.1.5 (nested copy removed; the hoisted copy now satisfies it) - @radix-ui/react-tabs 1.1.21 -> 1.2.1 - @radix-ui/react-toggle 1.1.18 -> 1.2.1 - @radix-ui/react-use-callback-ref 1.1.4 -> 1.1.5 - @radix-ui/react-use-controllable-state 1.2.6 -> 1.3.0 - @radix-ui/react-visually-hidden 1.2.11 -> 1.2.13 - @types/node 26.6.3 -> 26.6.5 - baseline-browser-mapping 2.11.20 -> 2.11.30 - browserslist 4.28.8 -> 4.29.4 - caniuse-lite 1.0.30001810 -> 1.0.30001817 - electron-to-chromium 1.5.420 -> 1.5.462 - eslint-config-next 16.3.8 -> 16.4.2 - eslint-plugin-react-hooks 7.0.1 -> 7.1.1 - node-releases 2.0.54 -> 2.0.59 - update-browserslist-db 1.3.2 -> 1.3.4 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * deps(sdk): Dependabot #878 devDependencies take the ranges #878 set; the open peer ranges are unchanged. ai locks at 7.0.147, the newest patch at least five days old (7.0.148 to 7.0.153 are newer). - @ai-sdk/gateway 4.0.103 -> 4.0.120 - @ai-sdk/provider 4.0.21 -> 4.0.31 - @ai-sdk/provider-utils 5.0.53 -> 5.0.64 - @anthropic-ai/sdk 0.131.0 -> 0.132.1 - ai 7.0.127 -> 7.0.147 - openai 7.27.0 -> 7.30.1 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * deps(langchain): Dependabot #879 devDependencies take the ranges #879 set; the peer range is unchanged. @langchain/core locks at 1.2.19, the newest patch at least five days old (1.2.20 is newer); langsmith 0.11 is what core 1.2.19 requires. - @langchain/core 1.2.14 -> 1.2.19 - @langchain/langgraph 1.4.18 -> 1.4.21 - @langchain/langgraph-checkpoint 1.1.5 -> 1.2.0 - @langchain/langgraph-sdk 1.12.0 -> 1.12.5 - @langchain/langgraph-sdk > p-queue 9.3.3 -> 9.3.5 - langsmith 0.10.5 -> 0.11.0 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * ci: Dependabot #873 #877 #883 in one The same pins those PRs proposed (commit sha and version comment), applied to every workflow as it is now, ci.yml included after #899. The four codeql-action steps in codeql.yml and scorecard.yml move together, as CodeQL requires. 73 lines in 12 workflows; no old pin is left. - actions/setup-node v7.0.0 (8207627) -> v7.1.0 (949feb2), 43 lines - actions/upload-artifact v7.0.1 (043fb46) -> v7.0.2 (cf430e0), 13 lines - actions/download-artifact v8.0.1 (3e5f45b) -> v8.0.2 (9000827), 12 lines - github/codeql-action init, autobuild, analyze, upload-sarif v4.38.2 (2892aa5) -> v4.39.0 (6f2ac81), 4 lines - ossf/scorecard-action v2.4.4 (2d11466) -> v2.5.0 (fa7117d), 1 line Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * deps(examples): Dependabot #902, zod 4.6.5 -> 4.6.7 in the OpenTelemetry JS recipe (GHSA-mrhx-c7xc-m3w5) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

irparent and others added 2 commits October 25, 2026 12:13

irparent merged commit 799b17b into main Oct 25, 2026
79 checks passed

irparent deleted the ci/install-and-apt-cache branch October 25, 2026 19:30

Navigation Menu

Repository navigation

CI: native SQLite install check, cached apt packages for e2e, and a date-proof upgrade test - #899

CI: native SQLite install check, cached apt packages for e2e, and a date-proof upgrade test#899
irparent merged 3 commits into
mainfrom
ci/install-and-apt-cache

irparent commented Oct 25, 2026

Uh oh!

vercel Bot commented Oct 25, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Oct 25, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Oct 25, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

irparent commented Oct 25, 2026

1. A dropped native SQLite module now fails as an install problem

2. e2e installs the browsers' system libraries from a cache

3. The upgrade test's retention sweep no longer depends on the day it runs

Uh oh!

vercel Bot commented Oct 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Oct 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Iris gate — 1 of 2 tripped --fail-on detector_veto

Uh oh!

github-actions Bot commented Oct 25, 2026

Iris gate — 1 stored, nothing tripped --fail-on any

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

vercel Bot commented Oct 25, 2026 •

edited

Loading

github-actions Bot commented Oct 25, 2026 •

edited

Loading

Iris gate — 1 of 2 tripped `--fail-on detector_veto`

Iris gate — 1 stored, nothing tripped `--fail-on any`