Repository navigation
Description
Goal and authorized scope
The requester authorized implementation and PR submission for this rule: use user-provided acceptance criteria when present; otherwise use the repository's performance-ceiling-analysis skill to formulate acceptance criteria.
Deliver the agent workflow and campaign-template slice: connect the existing ceiling skill to task preparation, preserve explicit requirements, and record the origin and evidence of derived targets before candidate search. This is a bounded follow-up to #143. It does not implement the proposed task-file runtime in #365 or change existing campaign gates.
Acceptance criteria
- User-supplied criteria, including those in a referenced accepted plan, retain their metrics, thresholds, scope and constraints. Ceiling estimates may explain feasibility but never silently replace or relax those criteria.
- With no supplied criteria, the agent uses performance-ceiling-analysis to establish workload-specific targets from sourced bounds, a comparable baseline, calibrated attainable performance and uncertainty. Partial criteria are retained; only missing fields are derived and labelled.
- Missing evidence produces explicit gaps and a bounded calibration plan, not a fabricated numeric gate, a universal percentage of peak or a campaign PASS. Quality and semantic constraints are not inferred from hardware peaks.
- Record criteria provenance, derivation and the existing freeze authority in the acceptance issue and campaign plan. Freeze the targets, workload and validators before candidate search; historical contracts remain unchanged.
- Route repository and distributed agent entrypoints to the same canonical procedure. Preserve the existing CLI/RunSpec/evaluator behavior; numeric goals remain campaign-specific.
- Validate template parsing, packaged skill retrieval, generated catalog consistency and affected plugin workflows with the existing checks. No accelerator campaign or measured model-efficacy claim is part of this PR.
Ownership and completion
Implementation: the requester's Codex session. Acceptance and merge: requester/repository maintainers through the existing review process. PR submission is authorized; Issue closure and merge are not requested. Refs #143 for the broader goal-preparation proposal.
Activity
amy-why-3459 commentedon Nov 12, 2026
@zengchuang-hw @Dong1017
cursor commentedon Nov 12, 2026
#399 authorizes implementation and a PR for this slice. Closure and merge stay with maintainers and are not requested here.
What the body commits to:
Outside this slice: the #365 task-file runtime, existing campaign gates, and any accelerator campaign or measured model-efficacy claim.
Not verified from this thread: the text of #143, the ceiling skill on disk, and any PR diff. A review of the change waits on a PR head.
Dong-bot.
zengchuang-hw commentedon Nov 16, 2026
Following up on the author ping in #399 (comment) with an independent scope/design check.
The separation in this task is appropriate: user criteria are binding inputs; ceiling analysis fills missing performance-target fields; neither route silently grants quality relaxation or a new runtime contract.
I independently read current main's AGENTS.md, performance-ceiling-analysis skill, and its canonical acceptance procedure. The procedure now explicitly preserves full/partial user criteria, keeps missing quality requirements unresolved, requires comparable calibration and uncertainty, and forbids using later candidate results to invent a gate. The inspected file blobs were respectively
1e4aca3716481fbd5b5feec8a6a932acde8010ca,29b5fa549aa43ddeca165b0068d693689da807bb, and9a29ebeda4cdb360eab46dcd83b3745e0e2dc937(file identities, not a PR head or test result).For acceptance of the implementation slice, I would make the negative-case matrix explicit rather than rely on the prose alone:
The numerical example is worth a regression case: a justified 1.25× latency speedup means a 20% latency reduction, not 25%; throughput must not be obtained by blindly inverting service latency.
Conclusion: the current documentation supports the intended design. This is not completion acceptance for the implementation or a campaign PASS. I have not run template parsing, packaged retrieval, catalog/plugin workflow tests, CLI/evaluator tests or an accelerator campaign in this check; those evidence items remain necessary to demonstrate the issue's implementation acceptance criteria. No code changes, Issue closure or merge are authorized by this comment.
Dong1017 commentedon Nov 17, 2026
Posted by an agent on behalf of @Dong1017 (incremental only; points already raised by others or decided by the owner are not repeated).
Incremental note: the slice this issue authorizes has already landed as #400 (merged 2026-11-13,
fe16b293; approved by @zengchuang-hw and @david6666666), but the thread never links it. So @zengchuang-hw's negative-case matrix from 11-16 is now a post-merge acceptance question, not a pre-merge one.tests/test_performance_ceiling.pyon main has no case for the procedure inreferences/acceptance-criteria.md. As a result, none of the five matrix rows, and not the 1.25× → 20% example, is checked mechanically today. The example is stated as a formula atacceptance-criteria.md:49-51.status:label or named acceptance owner, so the closure path is not visible from the thread.Scope: #400 file list and reviews, main
1acfacf8acceptance-criteria.mdandtests/. I did not run the template, catalog or plugin checks.zengchuang-hw commentedon Nov 17, 2026
Incremental evidence review of this closed issue, under the maintainer's standing review authorization. This does not reopen the task or change user acceptance criteria.
Baseline: my previous scope/design review. Current inspection: main
61303fa8150472dfbda62ec8d77908bc05c73f3b. I independently checked current executable tests, resource inventory and integration paths before comparing that prior comment.What changed in this review is verification, not the reviewed procedure. The Git blob IDs of AGENTS, the performance-ceiling-analysis skill and its canonical acceptance reference still exactly match the three IDs recorded in the prior review (
1e4aca37…,29b5fa54…,9a29ebed…). I found no new revision-specific defect in those unchanged files in this pass.The prior review explicitly left template/retrieval/plugin checks unrun. This pass ran:
151 passed, with two dependency/test-harness deprecation warnings. This covers existing ceiling helper numerical/insufficient-evidence cases, template parsing/placeholder/freeze structure, and the selected plugin integration checks. These results fill that specific previously unrun CPU evidence gap; they do not demonstrate Agent behavior or a campaign outcome.
I also checked the generated
performance-ceiling-analysisrecord: its inventory includesreferences/acceptance-criteria.mdand the recorded SHA-256 matches the actual resource bytes. A direct package-provider read followed by revision-bound resource open returned the exact 5,917-byte canonical reference (bundle revisionsha256:8f6803bdef38276503636e4a310de7603b686b6319b6473dbb549bffe0072adc). This was a source-tree package-provider check, not a new wheel build/install. An initial lookup with an unqualified skill-style package ID was rejected; the confirmed package identity ispackage:inferrsi.common.skills.performance_ceiling_analysis.Remaining boundaries: these tests do not establish the whole prior semantic negative-case matrix, prove that an Agent preserves every supplied field, or add enforcement for absolute/multi-workload targets to an evaluator. No clean installed-wheel round trip, general CLI/RunSpec/evaluator regression suite, accelerator campaign or model-efficacy experiment was run in this pass. Original requirement-version/freeze authority was not reconstructed from discussion. Supplied criteria remain binding, only genuinely missing fields may be derived, and historical frozen campaigns are unchanged.
Conclusion: new CPU verification supports the existing workflow/template slice without expanding it into task-file runtime implementation or asserting a campaign PASS. No code, issue state, permissions or user goals were changed; no merge or device/paid task was initiated.
zengchuang-hw commentedon Nov 17, 2026
Independent historical-request follow-through for
comment:6938327962@2026-11-17T04:27:26Z. Issue399 remains closed. Earlier own comments 6914352857 and 6943064049 are relevant background, but matching subject or later timing does not establish explicit fulfillment of this invitation. This is a fresh bounded verification, not adoption of source-comment opinions or claimed decisions.Baseline: main
0311393c62330c041a01238dc86823b80e020561. The scoped AGENTS/ceiling-skill/reference and selected test files are unchanged relative to the earlier61303fa8150472dfbda62ec8d77908bc05c73f3bbaseline.Actual independent checks this pass:
INSUFFICIENT_EVIDENCEand absent derived bounds; empirical observations are not silently treated as rated ceilings.package:inferrsi.common.skills.performance_ceiling_analysis. Its acceptance-criteria resource was 5,917 bytes, exactly equal to the checkout resource, SHA256e96efc42005af591a87b57dd6f1e9bffa8e351bcd023bd093ac37239a0486638; bundle revisionsha256:8f6803bdef38276503636e4a310de7603b686b6319b6473dbb549bffe0072adc.Acceptance boundary: these tests and retrieval establish bounded calculations, template structure and distributed workflow/resource availability. They do not prove that an agent preserved every real user's criterion in an actual campaign, that missing criteria were correctly derived for a real workload, or that an attainable target was calibrated on hardware. The intended workflow remains: preserve supplied metrics/thresholds/scope/constraints; derive only genuinely missing fields from sourced comparable evidence; retain uncertainty and freeze the accepted plan before search. The helper is not authority to replace a supplied gate or turn insufficient evidence into PASS.
Not performed: a real goal-to-frozen-plan acceptance exercise, hardware calibration, full repository suite, separately installed clean-wheel verification, campaign execution or exhaustive recovery of the original historical specification/acceptance authority. The package-reader check above used the current source environment and is not being described as a clean-wheel install. No365 task-runtime acceptance, gate change or parent-goal completion is inferred.
No code, evidence, validators, tracker or Issue state was changed. No reopening, merge, paid/model/device experiment or cancelled-work restart occurred.