Skip to content

docs: rebuild the docs bot index before the release, re-record its evals - #628

Merged
anurag629 merged 1 commit into
devfrom
docs/docs-index-before-release
Nov 19, 2026
Merged

anurag629 merged 1 commit into
devfrom
docs/docs-index-before-release

Conversation

@anurag629

Copy link
Copy Markdown
Member

docs-index:check warns about 14 pages that changed since the index was last built in #517. Three of them are new: the site indexing guide (#537), the test-before-launch guide (#570) and the React page (#583). Until they're in the index, the docs bot answers from the old pages and doesn't know @chatcops/react exists, so this goes in before the release (#627).

This rebuilds the index (32 pages, 347 chunks, openai:text-embedding-3-small@512) and re-records both eval files, since the new chunks change what the bot gets as context.

How I tested this

  • pnpm docs-index:check: up to date.
  • pnpm --filter @chatcops/website evals (replay, as in CI): 9 of 9 tests, 12 of 12 assertions.
  • pnpm --filter @chatcops/website evals:citations: 48 of 50 cite the right page (96%, needs 85%), the same as in docs: rebuild the docs bot index for the 1.0 snippets, re-record its evals #501. The misses are the Cloudflare cleanup question (cited /server/storage/, which covers it) and the lead callback question (cited /server/express/ and /examples/cloudflare-full/).
  • CI's key scan pattern finds nothing in the recordings. The only Bearer and ?key= matches are docs text the bot quoted.

@anurag629
anurag629 merged commit 2b2a3c0 into dev Nov 19, 2026
5 checks passed
@anurag629
anurag629 deleted the docs/docs-index-before-release branch November 19, 2026 08:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant