Repository navigation
Conversation
Knowledge contributions declaring gpt-6.2-astra or gpt-6.2-sol currently fail model admission. Add both exact identities at xhigh and max, with no aliases, as requested by the owner. The official model pages confirm the canonical IDs and the official model catalog lists xhigh and max for both (checked 2026-11-18). Move the admission tests' default author to gpt-6.2-sol before any older model is removed, so failure-asserting tests keep a compliant baseline, and assert that baseline directly. Cover both new IDs at xhigh and max, every other effort including case and whitespace variants, undeclared short names and near-miss IDs, suspension, and a mixed declaration in which one author is noncompliant. The evaluator, exception process, protected paths and trusted-main policy selection are unchanged. Agent-assisted-by: Claude Code / claude-opus-5-5 [effort=xhigh] (author) Signed-off-by: Dong1017 <55723162+Dong1017@users.noreply.github.com>
The owner moved GPT knowledge authorship to the 6.2 generation and chose to delete the older entries rather than suspend them. Remove gpt-6-astra, gpt-6.1-sol and gpt-6.1-terra from the allowlist, the owner table and the verification text, and record that this is a repository admission decision, not a statement about vendor support. Merged contributions are not re-evaluated; an open PR that still declares a removed model fails at its next refresh unless the affected work is re-authored with an admitted model or an explicit exception is approved. Move the effort and suspension cases that named the removed models to admitted ones, so none of them turns vacuous or indexes a missing entry, and assert that each removed ID is rejected at xhigh and max, alone and inside a mixed declaration. The evaluator, exception process, protected paths and trusted-main policy selection are unchanged. Agent-assisted-by: Claude Code / claude-opus-5-5 [effort=xhigh] (author) Signed-off-by: Dong1017 <55723162+Dong1017@users.noreply.github.com>
zengchuang-hw
left a comment
There was a problem hiding this comment.
PR review verdict — DO NOT PASS yet: execution and decision-trace evidence incomplete. Formal COMMENT, not a Request changes vote.
Initial independent review of base cf671a880ac8037e01922e1be0b701771fb9922f → exact head 49c5453c63f7b86e4c466b804f427fec524f1e37.
Verified source behavior. All three changed files were reviewed. The two new canonical IDs have no aliases and admit only xhigh/max; the three retired IDs are removed rather than suspended. Unchanged model_errors performs exact identity lookup and explicit effort-set intersection. The policy's protected paths, approval accounts and exception evaluator are not expanded. The tests load the actual JSON model table (only maintainer identities are substituted), move the default fixture to an admitted model, add an explicit default-positive guard, and cover retired IDs, ambiguous aliases/case/whitespace, suspension and mixed compliant/noncompliant authors. The positive guard matters: it prevents unrelated negative tests passing merely because the default fixture became inadmissible.
I independently checked the official OpenAI model catalog, which lists gpt-6.2-astra and gpt-6.2-sol with xhigh and max. This supports the documented ID/effort claims, not comparative model quality. Keeping old model identifiers in release-notes request-format tests is not an admission bypass: those tests check a separate client payload rather than this knowledge allowlist.
Before approval.
- Obtain completed current-content policy-test/full-check evidence and the applicable ready checks. At inspection, full-check jobs 134786431885/134786431884 and browser/native-host jobs were running or queued. I read the actual lint/smoke job 134784897106: Ruff passed and 8 smoke/claim tests passed on this exact head. That job does not execute the changed policy test module. Installed-package checks are successful metadata, not a substitute for these policy regressions. No failing product test is asserted.
- Replace the literal
DECISION_LINKin the governing-scope paragraph with the actual source-account decision record resolving the cutover questions. Issue #487's normative scope calls for the allowlist update but leaves already-merged contributions and an in-flight transition window as explicit questions. The new doc specifies immediate cutover/no transition and re-authoring or exception for affected open work. Those consequences need an auditable decision link; the PR's statement that such a decision exists is not itself independent verification. This is not a demand for an invented machine policy-change approval: the existing evaluator correctly treats policy_paths as classification, not a separate approval gate.
No new concrete implementation defect was established in this bounded static review. Merged-contribution history must not be relabeled or rewritten, and admission compliance remains distinct from content quality. No model was invoked and no repository tests/code were executed locally. I read AGENTS/review guidance, the actual evaluator and test sources, isolated governing requirements, official model evidence and current check records; I did not read other reviewers' opinions, edit sources, change policy settings or merge.
zengchuang-hw
left a comment
There was a problem hiding this comment.
LGTM
PR review verdict — PASS for this bounded policy-change slice.
Acting independent reviewer: dot, under zengchuang-hw. Direct standing delegation from this account's owner covers independent review of others' InferRSI PRs and submitting APPROVE after the substantive review and necessary evidence pass; it does not authorize merging. I did not author or materially contribute to this slice.
Revision: base cf671a880ac8037e01922e1be0b701771fb9922f → head 49c5453c63f7b86e4c466b804f427fec524f1e37. Since my previous review, the head-to-head code delta is empty. This is an incremental closure of the decision-trace and execution-evidence gaps, not a claim of new source changes.
I re-read all three changed-file diffs, the actual policy JSON, changed test fixtures/cases, unchanged model_errors evaluator and ready workflow, alongside AGENTS and repository review guidance. The exact new IDs admit only xhigh/max without inferred aliases; the three retired entries are absent; Claude entries and protected paths/approval ownership/exception evaluation are not expanded. The actual JSON-backed tests retain a compliant default and explicitly guard it, exercise the two new IDs at both allowed efforts, reject retired/near-miss/case/whitespace identities and disallowed efforts, and test suspension plus either position of a noncompliant mixed author. These tests avoid the vacuous-negative-fixture regression. No blocking implementation defect was found.
Resolved findings:
- The governing paragraph now links the actual D1–D6 decision, rather than DECISION_LINK. Its normative requirements support deletion, prospective treatment of merged work, immediate trusted-main cutover and truthful re-authoring or an existing explicit exception. This closes the source-trace finding already recorded in my incremental Issue review. The previous independent official model-catalog verification remains applicable to this unchanged policy content; model quality is not established by admission.
- I inspected the actual completed Python 3.10 full-check log and Python 3.11 full-check log. Both execute Ruff and
pytest -rs, report Ruff clean and 5194 passed, 37 skipped, 672 subtests passed, 1 warning. The logged checkout is synthetic merge0c075679d485cf097d51332538f9139d6f1e6506of the stated base/head. Git tree API verification shows this merge and the reviewed head have exactly the same tree,91954f6d2d9a35c70467cabcdc1a4a2b90f04e1b, establishing tested-content equivalence. The changed admission module is included in this suite and is not among the reported skips. - Current-head metadata reports all 11 check runs successful: both full-checks, browser, both native-host checks, both installed-package checks, pr-references, declarations and lint/smoke. The separate knowledge-model-policy status is success, declaration compliance only. I did not treat the latter, which uses trusted main, as proof of the candidate allowlist; the candidate-content tests supply that evidence.
Before approval: no remaining blocker for this reviewed slice. Non-blocking follow-up: after an independently authorized merge, verify the refreshed per-PR admission status for affected open work, particularly #480. A successful sweep alone does not prove its individual outcome, and this approval neither changes truthful authorship nor closes #487.
Limits: I inspected source and existing hosted evidence only; I executed no repository code, local test suite or model call. The 37 full-suite skips concern optional dependencies, browser/native/environment-specific or opt-in integrations; they are not claimed passed. Separate browser/native/package jobs are verified as successful check records, not independently rerun here. This is PR readiness, not campaign/E2E quality acceptance, deployment, Issue closure or permission to merge. Other reviewers' opinions are not a basis for this verdict.
Governing Issue and delivery slice
Refs #487
Accepted scope / delegation: the owner's RFC policy: admit GPT-6.2 Astra / GPT-6.2 Sol, retire GPT-6-Astra, GPT-6.1 Sol and GPT-6.1 Terra #487 and the owner's decision record on it (decision record). Retired entries are deleted, not suspended. The cutover is immediate, with no transition window. Merged contributions are not re-evaluated.
What this PR delivers: two commits on
maincf671a88, in this order:fd80c4afadmitsgpt-6.2-astraandgpt-6.2-sol(exact IDs, no aliases,xhighandmaxonly, the same shape as policy: admit GPT-6.1 Terra at xhigh and max #298 and policy: admit Claude Fable 5.2 at xhigh and max #439). It also moves the admission tests' default author fromgpt-6.1-soltogpt-6.2-solwhile the old entries still exist. A new test asserts that this default is admitted.49c5453cremovesgpt-6-astra,gpt-6.1-solandgpt-6.1-terrafrom.github/knowledge-model-policy.json, the owner table and the verification text indocs/knowledge-model-policy.md. It moves the effort and suspension cases that named them to admitted models and adds explicit rejection cases for the removed IDs.The policy doc now records the 2026-11-18 doc check for the new IDs. It also adds one paragraph saying four things. The removal is a repository admission decision, not a statement about the vendor's support for those models. Merged contributions are not re-evaluated. An open PR still declaring a removed model fails at its next refresh. It then needs re-authoring with an admitted model or an explicit exception.
What it does not deliver: this PR does not change the evaluator, the exception process, protected paths, trusted-main policy selection, or the Claude Opus 5.5 and Claude Fable 5.2 entries. It does not run a model and does not qualify knowledge content. The other references to the retired IDs are left unchanged on purpose; see "Retired-ID references" below.
Knowledge authorship declaration
This PR changes
policy_pathsonly (the policy JSON and its document) and no protected knowledge path, so the declaration is disclosure only. Checks use the target branch's policy, so this PR's own status does not exercise the new list; the dry-run below does.Changed artifacts and evidence
gpt-6.2-astra,gpt-6.2-solatxhigh/max, no aliases.github/knowledge-model-policy.json, owner table and check paragraph indocs/knowledge-model-policy.mdxhigh,max} passesassessxhigh,max} gets exactlymodel is unapproved or suspendedattribution()default isgpt-6.2-solin commit 1;test_default_attribution_is_admittedKeyErrorin the suspension case, whiletest_every_material_author_must_qualifystays green for the wrong reason. With this PR: all passtests/test_knowledge_model_policy.pyxhigh/max. Rejected: all three removed IDs at both efforts;gpt-6.2,astra,sol; display names; case variants (GPT-6.2-SOL,gpt-6.2-Astra); leading/trailing spaces; near misses (gpt-6.2-terra,gpt-6.2-astra-latest,gpt-6-sol); effortslow,medium,high,none,minimal,ultra,maximum,unknown,XHIGH,MAX,xhigh,max; suspension of each new ID; a mixed declaration with one bad author in either position, where only that author is reportedDoc check (2026-11-18). Pages were fetched with
curlfrom a Linux host, because the local network blocks these sites.gpt-6.2-astra, "Model IDs: gpt-6.2-astra", and "The API rejects requests that set reasoning effort to none."gpt-6.2-sol, "Model IDs: gpt-6.2-sol", and the samenonerejection. Neither page lists aliases or snapshots.gpt-6.2-astraandgpt-6.2-soleach list Reasoninglowmediumhighxhighmax. The two model pages do not list effort values themselves, so the catalog is the source forxhighandmax.gpt-6.2-astra/gpt-6.2-solrelease entry.Open PRs, re-scanned 2026-11-18 right before publishing (14 open). Only #480 declares a removed model (
gpt-6-astra). #457 declaresclaude-opus-5-5; its Sol mention is reviewer-only. No open PR declares agpt-6.2-*ID. #456 (gpt-6.1-sol) merged at 2026-11-18T03:41Z, so it is outside the sweep. A read-only dry-run used the checker with no token,--policy-shaset tomain76ca44a8(the policy is unchanged on the currentmaincf671a88) and--dry-run:mainpolicy20851a9c)0d6ab1f8)Full refresh on merge. Changing the policy JSON and the policy document forces a full refresh of open PRs that target
main(docs/knowledge-model-policy.md, "Bounded impact routing"), even if their heads do not change. That is expected and needs no change here. A green sweep job does not show that each PR is green. After the merge push, read #480's ownknowledge-model-policystatus on its current head.Retired-ID references kept as they are:
docs/reviews/issue91-*.md(10 files),evidence/indexer-255-raw-string-20261030/plan/publication.md, andevidence/skill-eval-v1-20261021/(README and two protocol files).tests/fixtures/repo_to_skill/*/reports/integration/(7 fixtures).tools/skill_eval_pr_cases.jsonandtests/test_skill_eval_pr.py:91pin the Skill Eval executor model.tests/test_release_notes.py:507-515checks that an explicitly given model ID is sent unchanged and that noreasoningfield is added.CHANGELOG.mdis unchanged. policy: admit GPT-6.1 Terra at xhigh and max #298 and policy: admit Claude Fable 5.2 at xhigh and max #439 did not add entries either.Checks and dependencies
Tests ran on the CPU-only H200 (Linux, Python 3.12.13, pytest 9.0.3, git 2.34.1,
CUDA_VISIBLE_DEVICES=empty). The command waspython3 -m pytest -o addopts= -q. Each tree was extracted from agit archiveofmain76ca44a8, with this branch's patches applied. The candidate's changed files were verified by SHA-256 against the commit.main76ca44a84acff558)a84f2cf8)tests/test_knowledge_model_policy.pytests/test_release_notes.pyAfter rebasing both commits onto
maincf671a88(feat: 增加逐轮检索、备选、选择与执行结果展示 #495, which touches none of these files), the head49c5453cwas re-run the same way on H200: the first two files 346 passed; the other six 107 passed and the same 2 git-fixture tests failed, and those 2 pass locally on49c5453c. Lint was re-run on49c5453c: all checks passed.The 8 files are the two above plus
test_knowledge_policy_{comment_routing,http,impact_routing,push_health,reporting}andtest_knowledge_declaration_paths. On all three trees, the same 2 tests intest_knowledge_declaration_paths.pyfail because git 2.34.1 rejects--no-lazy-fetch(exit 129). That file passes locally on the head: 2 passed, with git 2.53.0 and Python 3.13.5.Mutation check: each of six deliberately broken
model_errorsvariants makes the head'stest_knowledge_model_policy.pyfail:The head's tests also fail against commit 1's policy JSON, where the removed entries are still present: 7 failures.
uvx ruff@0.17.5 check src tests tools(the CI version): all checks passed.git diff --check: clean.tests/test_skill_eval_pr.pywas not run on H200, because an installed third-partytestspackage there shadowstests.fixtures. This PR does not change that file or its inputs. The full suite was not run; hostedfull-checkandhost-compatibilityare authoritative.main. The policy takes effect for open PRs only once it is onmain, because the workflow evaluates trustedmain.Remaining work and Issue closure
Provenance
Two commits, written on
main76ca44a8and rebased ontomaincf671a88before the first push (no content change). No merge, amend, force push or skipped hooks. The policy JSON, document and tests were written in a Claude Code session reported asclaude-opus-5-5, effortxhigh. This is client/session attribution, not a backend attestation. Passing tests is not policy approval.Agent-assisted-by: Claude Code / claude-opus-5-5 [effort=xhigh] (author)