Skip to content

policy: admit GPT-6.2 Astra and GPT-6.2 Sol, stop admitting GPT-6-Astra, GPT-6.1 Sol and GPT-6.1 Terra - #499

Open
Dong1017 wants to merge 2 commits into
mainfrom
policy/gpt-6.2-admission
Open

Dong1017 wants to merge 2 commits into
mainfrom
policy/gpt-6.2-admission

Conversation

@Dong1017

@Dong1017 Dong1017 commented Nov 18, 2026 •

Copy link
Copy Markdown
Collaborator

Governing Issue and delivery slice

Refs #487

  • Accepted scope / delegation: the owner's RFC policy: admit GPT-6.2 Astra / GPT-6.2 Sol, retire GPT-6-Astra, GPT-6.1 Sol and GPT-6.1 Terra #487 and the owner's decision record on it (decision record). Retired entries are deleted, not suspended. The cutover is immediate, with no transition window. Merged contributions are not re-evaluated.

  • What this PR delivers: two commits on main cf671a88, in this order:

    1. fd80c4af admits gpt-6.2-astra and gpt-6.2-sol (exact IDs, no aliases, xhigh and max only, the same shape as policy: admit GPT-6.1 Terra at xhigh and max #298 and policy: admit Claude Fable 5.2 at xhigh and max #439). It also moves the admission tests' default author from gpt-6.1-sol to gpt-6.2-sol while the old entries still exist. A new test asserts that this default is admitted.
    2. 49c5453c removes gpt-6-astra, gpt-6.1-sol and gpt-6.1-terra from .github/knowledge-model-policy.json, the owner table and the verification text in docs/knowledge-model-policy.md. It moves the effort and suspension cases that named them to admitted models and adds explicit rejection cases for the removed IDs.

    The policy doc now records the 2026-11-18 doc check for the new IDs. It also adds one paragraph saying four things. The removal is a repository admission decision, not a statement about the vendor's support for those models. Merged contributions are not re-evaluated. An open PR still declaring a removed model fails at its next refresh. It then needs re-authoring with an admitted model or an explicit exception.

  • What it does not deliver: this PR does not change the evaluator, the exception process, protected paths, trusted-main policy selection, or the Claude Opus 5.5 and Claude Fable 5.2 entries. It does not run a model and does not qualify knowledge content. The other references to the retired IDs are left unchanged on purpose; see "Retired-ID references" below.

Knowledge authorship declaration

This PR changes policy_paths only (the policy JSON and its document) and no protected knowledge path, so the declaration is disclosure only. Checks use the target branch's policy, so this PR's own status does not exercise the new list; the dry-run below does.

{"schema_version":1,"head_sha":"49c5453c63f7b86e4c466b804f427fec524f1e37","human_only":false,"material_ai_authors":[{"tool":"Claude Code","model_id":"claude-opus-5-5","effort":"xhigh","role":"author"}]}

Changed artifacts and evidence

Requirement Actual change Check / evidence and result Remaining gap
Admit gpt-6.2-astra, gpt-6.2-sol at xhigh/max, no aliases .github/knowledge-model-policy.json, owner table and check paragraph in docs/knowledge-model-policy.md Official docs checked 2026-11-18 (below); each new ID × {xhigh, max} passes assess None
Remove the three older GPT entries (delete, not suspend) Same two files; the 2026-10-21 paragraph now cites only the Claude docs; the Terra paragraph is removed Each removed ID × {xhigh, max} gets exactly model is unapproved or suspended None
Default test author moved before retirement attribution() default is gpt-6.2-sol in commit 1; test_default_attribution_is_admitted Base tests run against the new policy JSON: 51 failures, including a KeyError in the suspension case, while test_every_material_author_must_qualify stays green for the wrong reason. With this PR: all pass None
Identity matrix (zengchuang-hw) tests/test_knowledge_model_policy.py Accepted: both new IDs × xhigh/max. Rejected: all three removed IDs at both efforts; gpt-6.2, astra, sol; display names; case variants (GPT-6.2-SOL, gpt-6.2-Astra); leading/trailing spaces; near misses (gpt-6.2-terra, gpt-6.2-astra-latest, gpt-6-sol); efforts low, medium, high, none, minimal, ultra, maximum, unknown, XHIGH, MAX, xhigh , max; suspension of each new ID; a mixed declaration with one bad author in either position, where only that author is reported None
Merged contributions and in-flight PRs Policy doc paragraph Open-PR scan and dry-run below Post-merge status check of #480

Doc check (2026-11-18). Pages were fetched with curl from a Linux host, because the local network blocks these sites.

  • GPT-6.2 Astra model page: title "GPT-6.2 Astra", ID gpt-6.2-astra, "Model IDs: gpt-6.2-astra", and "The API rejects requests that set reasoning effort to none."
  • GPT-6.2 Sol model page: title "GPT-6.2 Sol", ID gpt-6.2-sol, "Model IDs: gpt-6.2-sol", and the same none rejection. Neither page lists aliases or snapshots.
  • Model catalog: gpt-6.2-astra and gpt-6.2-sol each list Reasoning low medium high xhigh max. The two model pages do not list effort values themselves, so the catalog is the source for xhigh and max.
  • Reasoning guide: "Supported values are model-dependent and can include none, minimal, low, medium, high, xhigh, and max." and "GPT-6.2 Astra and GPT-6.2 Sol reject requests that set reasoning.effort (Responses) or reasoning_effort (Chat Completions) to none."
  • Developer changelog, Nov 17: the gpt-6.2-astra / gpt-6.2-sol release entry.
  • The announcement returned HTTP 403 (a bot-challenge page) from both hosts and was not read.
  • No paid API call or model run was made.

Open PRs, re-scanned 2026-11-18 right before publishing (14 open). Only #480 declares a removed model (gpt-6-astra). #457 declares claude-opus-5-5; its Sol mention is reviewer-only. No open PR declares a gpt-6.2-* ID. #456 (gpt-6.1-sol) merged at 2026-11-18T03:41Z, so it is outside the sweep. A read-only dry-run used the checker with no token, --policy-sha set to main 76ca44a8 (the policy is unchanged on the current main cf671a88) and --dry-run:

PR main policy This PR's policy
#480 (head 20851a9c) success, "Declaration compliance only" failure, "Material AI author 1: model is unapproved or suspended"
#457 (head 0d6ab1f8) success, "Not applicable" success, "Not applicable"

Full refresh on merge. Changing the policy JSON and the policy document forces a full refresh of open PRs that target main (docs/knowledge-model-policy.md, "Bounded impact routing"), even if their heads do not change. That is expected and needs no change here. A green sweep job does not show that each PR is green. After the merge push, read #480's own knowledge-model-policy status on its current head.

Retired-ID references kept as they are:

  • Historical records: the material-author lines in docs/reviews/issue91-*.md (10 files), evidence/indexer-255-raw-string-20261030/plan/publication.md, and evidence/skill-eval-v1-20261021/ (README and two protocol files).
  • Fixtures recording past integration runs: tests/fixtures/repo_to_skill/*/reports/integration/ (7 fixtures).
  • Not authorship admission:
    • tools/skill_eval_pr_cases.json and tests/test_skill_eval_pr.py:91 pin the Skill Eval executor model.
    • tests/test_release_notes.py:507-515 checks that an explicitly given model ID is sent unchanged and that no reasoning field is added.
  • CHANGELOG.md is unchanged. policy: admit GPT-6.1 Terra at xhigh and max #298 and policy: admit Claude Fable 5.2 at xhigh and max #439 did not add entries either.

Checks and dependencies

  • Checks actually run (result and tested scope):
    • Tests ran on the CPU-only H200 (Linux, Python 3.12.13, pytest 9.0.3, git 2.34.1, CUDA_VISIBLE_DEVICES= empty). The command was python3 -m pytest -o addopts= -q. Each tree was extracted from a git archive of main 76ca44a8, with this branch's patches applied. The candidate's changed files were verified by SHA-256 against the commit.

      File main 76ca44a8 Commit 1 (pre-rebase 4acff558) Head (pre-rebase a84f2cf8)
      tests/test_knowledge_model_policy.py 171 passed 229 passed 205 passed
      tests/test_release_notes.py 141 passed 141 passed 141 passed
      8 policy-related files 419 passed, 2 failed 477 passed, 2 failed 453 passed, 2 failed

      After rebasing both commits onto main cf671a88 (feat: 增加逐轮检索、备选、选择与执行结果展示 #495, which touches none of these files), the head 49c5453c was re-run the same way on H200: the first two files 346 passed; the other six 107 passed and the same 2 git-fixture tests failed, and those 2 pass locally on 49c5453c. Lint was re-run on 49c5453c: all checks passed.

      The 8 files are the two above plus test_knowledge_policy_{comment_routing,http,impact_routing,push_health,reporting} and test_knowledge_declaration_paths. On all three trees, the same 2 tests in test_knowledge_declaration_paths.py fail because git 2.34.1 rejects --no-lazy-fetch (exit 129). That file passes locally on the head: 2 passed, with git 2.53.0 and Python 3.13.5.

    • Mutation check: each of six deliberately broken model_errors variants makes the head's test_knowledge_model_policy.py fail:

      Broken variant Failures
      Only the first author is checked 6
      Case-folded model ID 3
      Whitespace-stripped model ID 2
      Loose effort matching 12
      Prefix model match 5
      Suspension ignored 4

      The head's tests also fail against commit 1's policy JSON, where the removed entries are still present: 7 failures.

    • uvx ruff@0.17.5 check src tests tools (the CI version): all checks passed. git diff --check: clean.

  • Failed, skipped, timed-out or not-run checks (if applicable): tests/test_skill_eval_pr.py was not run on H200, because an installed third-party tests package there shadows tests.fixtures. This PR does not change that file or its inputs. The full suite was not run; hosted full-check and host-compatibility are authoritative.
  • Dependencies / shared interfaces / integration assumptions (if applicable): none beyond main. The policy takes effect for open PRs only once it is on main, because the workflow evaluates trusted main.

Remaining work and Issue closure

Provenance

Two commits, written on main 76ca44a8 and rebased onto main cf671a88 before the first push (no content change). No merge, amend, force push or skipped hooks. The policy JSON, document and tests were written in a Claude Code session reported as claude-opus-5-5, effort xhigh. This is client/session attribution, not a backend attestation. Passing tests is not policy approval.

Agent-assisted-by: Claude Code / claude-opus-5-5 [effort=xhigh] (author)

Knowledge contributions declaring gpt-6.2-astra or gpt-6.2-sol currently
fail model admission. Add both exact identities at xhigh and max, with no
aliases, as requested by the owner. The official model pages confirm the
canonical IDs and the official model catalog lists xhigh and max for both
(checked 2026-11-18).

Move the admission tests' default author to gpt-6.2-sol before any older
model is removed, so failure-asserting tests keep a compliant baseline, and
assert that baseline directly. Cover both new IDs at xhigh and max, every
other effort including case and whitespace variants, undeclared short
names and near-miss IDs, suspension, and a mixed declaration in which one
author is noncompliant. The evaluator, exception process, protected paths
and trusted-main policy selection are unchanged.

Agent-assisted-by: Claude Code / claude-opus-5-5 [effort=xhigh] (author)
Signed-off-by: Dong1017 <55723162+Dong1017@users.noreply.github.com>
The owner moved GPT knowledge authorship to the 6.2 generation and chose
to delete the older entries rather than suspend them. Remove gpt-6-astra,
gpt-6.1-sol and gpt-6.1-terra from the allowlist, the owner table and the
verification text, and record that this is a repository admission decision,
not a statement about vendor support. Merged contributions are not
re-evaluated; an open PR that still declares a removed model fails at its
next refresh unless the affected work is re-authored with an admitted model
or an explicit exception is approved.

Move the effort and suspension cases that named the removed models to
admitted ones, so none of them turns vacuous or indexes a missing entry,
and assert that each removed ID is rejected at xhigh and max, alone and
inside a mixed declaration. The evaluator, exception process, protected
paths and trusted-main policy selection are unchanged.

Agent-assisted-by: Claude Code / claude-opus-5-5 [effort=xhigh] (author)
Signed-off-by: Dong1017 <55723162+Dong1017@users.noreply.github.com>

@zengchuang-hw zengchuang-hw left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR review verdict — DO NOT PASS yet: execution and decision-trace evidence incomplete. Formal COMMENT, not a Request changes vote.

Initial independent review of base cf671a880ac8037e01922e1be0b701771fb9922f → exact head 49c5453c63f7b86e4c466b804f427fec524f1e37.

Verified source behavior. All three changed files were reviewed. The two new canonical IDs have no aliases and admit only xhigh/max; the three retired IDs are removed rather than suspended. Unchanged model_errors performs exact identity lookup and explicit effort-set intersection. The policy's protected paths, approval accounts and exception evaluator are not expanded. The tests load the actual JSON model table (only maintainer identities are substituted), move the default fixture to an admitted model, add an explicit default-positive guard, and cover retired IDs, ambiguous aliases/case/whitespace, suspension and mixed compliant/noncompliant authors. The positive guard matters: it prevents unrelated negative tests passing merely because the default fixture became inadmissible.

I independently checked the official OpenAI model catalog, which lists gpt-6.2-astra and gpt-6.2-sol with xhigh and max. This supports the documented ID/effort claims, not comparative model quality. Keeping old model identifiers in release-notes request-format tests is not an admission bypass: those tests check a separate client payload rather than this knowledge allowlist.

Before approval.

  1. Obtain completed current-content policy-test/full-check evidence and the applicable ready checks. At inspection, full-check jobs 134786431885/134786431884 and browser/native-host jobs were running or queued. I read the actual lint/smoke job 134784897106: Ruff passed and 8 smoke/claim tests passed on this exact head. That job does not execute the changed policy test module. Installed-package checks are successful metadata, not a substitute for these policy regressions. No failing product test is asserted.
  2. Replace the literal DECISION_LINK in the governing-scope paragraph with the actual source-account decision record resolving the cutover questions. Issue #487's normative scope calls for the allowlist update but leaves already-merged contributions and an in-flight transition window as explicit questions. The new doc specifies immediate cutover/no transition and re-authoring or exception for affected open work. Those consequences need an auditable decision link; the PR's statement that such a decision exists is not itself independent verification. This is not a demand for an invented machine policy-change approval: the existing evaluator correctly treats policy_paths as classification, not a separate approval gate.

No new concrete implementation defect was established in this bounded static review. Merged-contribution history must not be relabeled or rewritten, and admission compliance remains distinct from content quality. No model was invoked and no repository tests/code were executed locally. I read AGENTS/review guidance, the actual evaluator and test sources, isolated governing requirements, official model evidence and current check records; I did not read other reviewers' opinions, edit sources, change policy settings or merge.

@Dong1017 Dong1017 added kind:task Bounded delivery task with a delegated slice area:knowledge Knowledge line: packs, skill library, knowledge admission labels Nov 18, 2026

@zengchuang-hw zengchuang-hw left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

PR review verdict — PASS for this bounded policy-change slice.

Acting independent reviewer: dot, under zengchuang-hw. Direct standing delegation from this account's owner covers independent review of others' InferRSI PRs and submitting APPROVE after the substantive review and necessary evidence pass; it does not authorize merging. I did not author or materially contribute to this slice.

Revision: base cf671a880ac8037e01922e1be0b701771fb9922f → head 49c5453c63f7b86e4c466b804f427fec524f1e37. Since my previous review, the head-to-head code delta is empty. This is an incremental closure of the decision-trace and execution-evidence gaps, not a claim of new source changes.

I re-read all three changed-file diffs, the actual policy JSON, changed test fixtures/cases, unchanged model_errors evaluator and ready workflow, alongside AGENTS and repository review guidance. The exact new IDs admit only xhigh/max without inferred aliases; the three retired entries are absent; Claude entries and protected paths/approval ownership/exception evaluation are not expanded. The actual JSON-backed tests retain a compliant default and explicitly guard it, exercise the two new IDs at both allowed efforts, reject retired/near-miss/case/whitespace identities and disallowed efforts, and test suspension plus either position of a noncompliant mixed author. These tests avoid the vacuous-negative-fixture regression. No blocking implementation defect was found.

Resolved findings:

  • The governing paragraph now links the actual D1–D6 decision, rather than DECISION_LINK. Its normative requirements support deletion, prospective treatment of merged work, immediate trusted-main cutover and truthful re-authoring or an existing explicit exception. This closes the source-trace finding already recorded in my incremental Issue review. The previous independent official model-catalog verification remains applicable to this unchanged policy content; model quality is not established by admission.
  • I inspected the actual completed Python 3.10 full-check log and Python 3.11 full-check log. Both execute Ruff and pytest -rs, report Ruff clean and 5194 passed, 37 skipped, 672 subtests passed, 1 warning. The logged checkout is synthetic merge 0c075679d485cf097d51332538f9139d6f1e6506 of the stated base/head. Git tree API verification shows this merge and the reviewed head have exactly the same tree, 91954f6d2d9a35c70467cabcdc1a4a2b90f04e1b, establishing tested-content equivalence. The changed admission module is included in this suite and is not among the reported skips.
  • Current-head metadata reports all 11 check runs successful: both full-checks, browser, both native-host checks, both installed-package checks, pr-references, declarations and lint/smoke. The separate knowledge-model-policy status is success, declaration compliance only. I did not treat the latter, which uses trusted main, as proof of the candidate allowlist; the candidate-content tests supply that evidence.

Before approval: no remaining blocker for this reviewed slice. Non-blocking follow-up: after an independently authorized merge, verify the refreshed per-PR admission status for affected open work, particularly #480. A successful sweep alone does not prove its individual outcome, and this approval neither changes truthful authorship nor closes #487.

Limits: I inspected source and existing hosted evidence only; I executed no repository code, local test suite or model call. The 37 full-suite skips concern optional dependencies, browser/native/environment-specific or opt-in integrations; they are not claimed passed. Separate browser/native/package jobs are verified as successful check records, not independently rerun here. This is PR readiness, not campaign/E2E quality acceptance, deployment, Issue closure or permission to merge. Other reviewers' opinions are not a basis for this verdict.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:knowledge Knowledge line: packs, skill library, knowledge admission kind:task Bounded delivery task with a delegated slice

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants