Skip to content

chore(reporters): preview every PR comment shape without a CI run - #239

Draft
veksen wants to merge 1 commit into
mainfrom
chore-comment-preview-harness
Draft

veksen wants to merge 1 commit into
mainfrom
chore-comment-preview-harness

Conversation

@veksen

@veksen veksen commented Oct 12, 2026

Copy link
Copy Markdown
Member

Parked work, opened as a draft so it is not lost. It is not finished and should not be merged as-is.

Goal

A change to the PR comment template should be visible before it reaches a real pull request. Today the only way to see success.md.j2 rendered is to push a branch, wait for a CI run, and read the comment GitHub stored. Three analyzer PRs in the gates-first round (#210, #211, #213) were layout fixes found that way, after shipping.

What

node --import tsx tools/comment-preview/preview.ts writes tools/comment-preview/out/index.html: the 16 shapes the comment can take, each rendered twice, light theme beside dark. It needs an authenticated gh and no database, no CI run and no network beyond GitHub.

The shapes are a clean run, a cost regression with and without table growth, a new query with an index recommendation, schema drift, every gate firing at once, 12 regressions, one of everything, assumed statistics, no baseline, comparison unavailable, comparison branch unset, two ingest failures, untested data access, and the modeled-tables notice.

How

preview.ts reads src/reporters/github/success.md.j2 and renders it with the same nunjucks configuration GithubReporter uses, because trimBlocks and lstripBlocks decide most of the whitespace this tool exists to show. It posts each result to POST /markdown through gh api, so the HTML is GitHub's own: constructs the sanitiser strips (style=, inline <svg>) are stripped here too, and the spacing comes from GitHub's stylesheet rather than an approximation of it.

fixtures.ts holds view models, which is the limit worth knowing: they are buildViewModel output, so a defect in buildViewModel itself does not show up in the preview. Only the template is under test.

github-markdown-light.css and github-markdown-dark.css are vendored from github-markdown-css.

Unfinished

  • The header comment advertises npm run preview. No such script exists in package.json.
  • Nothing asserts anything. A reader compares panes by eye; there is no snapshot, so a regression is caught only if someone looks.
  • The fixtures were written against the template as of 2026-07-31 and have no owner keeping them current when a gate is added.
  • out/ is gitignored rather than committed, so the rendered page has to be generated to be read.

Tests

None, by design at this stage. I ran the harness against the current template on 2026-10-12: it renders all 16 fixtures and writes the page. tsc --noEmit has not been run over tools/, which is outside the shipped src/ tree.

🤖 Generated with Claude Code

Renders the 16 shapes success.md.j2 can take through GitHub's own
/markdown endpoint, light and dark side by side, so a spacing or
sanitiser regression is visible before it reaches a real PR.

Parked work from the gates-first comment redesign: the harness is what
would have caught the 16px mark scale error that shipped instead.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Query Doctor — 6 successful checks

 Cost regression — No query went up more than 5%
 Untested data access — No changed data-access file without a test
 New query — No new queries
 New query with index recommendation — No new query ships an index recommendation
 Schema drift — No schema changes
 High-value nudge — No index or rewrite past the threshold


More details via MCP → get_ci_run({ runId: "01a13112-74da-7ee5-b5d0-6a5349a7caed" }) · view run · docs
3 queries read against main on assumed statistics of 10,000,000 rows per table. Sync production stats for costs measured against your real data.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant