Skip to content

feat!: name the case ground truth expectedOutput - #24

Closed
TheGreatAxios wants to merge 1 commit into
cl-11496-rename-the-eval-entrypoint-to-runevaluationfrom
cl-11502-hard-cut-expected-to-expectedoutput
Closed

TheGreatAxios wants to merge 1 commit into
cl-11496-rename-the-eval-entrypoint-to-runevaluationfrom
cl-11502-hard-cut-expected-to-expectedoutput

Conversation

@TheGreatAxios

Copy link
Copy Markdown
Contributor

Summary

  • Hard-cut the case ground-truth field from expect to expectedOutput.
  • JSONL rows that still use expect or expected throw. There is no alias.
  • Stored experiment format is unchanged.

Verification

  • bun run check exited 0 (typecheck, oxlint, oxfmt, 2 unit tests, 53 e2e tests).
  • Review of dad00da found no defects.

Stacked on #23.

Fixes CL-11502

JSONL rows that still say expect or expected are rejected.
@linear-code

linear-code Bot commented Oct 26, 2026

Copy link
Copy Markdown

CL-11502

@corbits-triage

corbits-triage Bot commented Oct 26, 2026 •

Copy link
Copy Markdown

Triage: Pass

No changes needed from you. A maintainer will review this pull request.

Stacked on #23: only this pull request's own changes were checked.

@TheGreatAxios TheGreatAxios left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review · Approve

Hard-cuts case ground truth to expectedOutput and rejects JSONL rows that still say expect or expected.

Findings

No findings.

toCase in src/datasets.ts:24 throws before copying a legacy key. runOnce, runCase, matches, Jev toState, and calibrate read only expectedOutput. bun test src/datasets.test.ts exited 0 (1 pass). A probe also confirmed falsy expect / expected values and a run that still has expect and no expectedOutput do not pass matches().

@TheGreatAxios

Copy link
Copy Markdown
Contributor Author

Superseded by the 0.2.0 release: #32

Do not merge this pull request separately. The release branch cherry-picks the feature stack (except the 0.1.3 bump on #22) and sets @corbits/evals to 0.2.0 with @corbits/system-one ^0.4.0.

@TheGreatAxios

Copy link
Copy Markdown
Contributor Author

Superseded by the 0.2.0 release: #32

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant