Repository navigation
Description
Current state: Proposed. This RFC on tiered experiment artifact archiving and PR size control awaits scope review. Existing R6 reporting thresholds remain unchanged.
Decision needed: Repository maintainers should decide whether to adopt the inclusion rules, minimal evidence bundle, and pilot workflow below. The acceptance owner for this RFC is yet to be designated.
Next action: Maintainers record adoption, requested changes, or deferral and name the responsible owners. Accepted work proceeds through separate tasks for templates, tooling, and the pilot.
1. Proposal and goals
Keep the essentials needed for review and verification in the repository: implementation, tests, experimental conclusions, key metrics, configuration, a trial index, and validation entry points.
Store large collections of per-request results, logs, detailed quality scores, and process inventories in external, versioned archives. Track these artifacts through download URLs, checksums, and named retention owners.
Preserve complete experiment records, including failed, regressing, and interrupted trials. Improve storage and presentation to reduce PR review costs and Git history growth.
Scope: New experimental results, generated inventories, process reports, and related unmerged PRs. Ordinary implementation changes and necessary regression tests continue through normal code review.
Boundaries: No retroactive migration of merged evidence, no rewriting of main-branch history, and no changes to experimental acceptance gates or historical conclusions. This RFC extends the submission and archiving workflow while retaining the existing R6 decision. PR #328 is the integration point for the existing size-checking implementation.
2. Background and problem
Several recent PRs consist largely of experimental or process artifacts. The following addition counts are snapshots at the listed revisions. They include data and documentation and do not measure implementation code alone.
| Example | Added lines | Main source |
|---|---|---|
#113, 130ee301de51914426454752a57d6d8c7668fb31 |
1,836,430 | Measurements, quality data, and GPU observations |
#435, dec7a734f27215092f62728f78b98fe26cd58ba2 |
215,825 | Per-request details across multiple experiment rounds |
#310, 01c64d053b1f8e1bc85695cbb707c6c5f540ed87 |
229,535 | Source inventories and process reports |
#310 was subsequently reverted in #311, reduced in scope, and resubmitted in #312 with 9,352 added lines.
The current approach creates three problems:
- Large volumes of detail obscure the implementation, experimental conditions, and conclusions that reviewers need to assess.
- Repeated copies of configuration, inputs, and process information increase maintenance costs.
- Once large files are merged, deleting them from the current tree leaves their contents in Git history.
3. What belongs in the repository
| Content | Default treatment |
|---|---|
| Product implementation and necessary regression tests | Keep in the repository under normal code review |
| Experimental goals, conclusions, limitations, and failure explanations | Keep concise descriptions |
| Key baseline and candidate metrics, sample counts, repetitions, and variability | Keep structured summaries |
| Source revisions, effective configuration, and input and validator identities | Keep the minimum set needed for verification |
| Each trial's status and result references | Keep a compact index |
| Per-request timings, per-sample scores, complete profiler output, logs, and generated media | Archive externally |
| Complete source-scan inventories, build-process reports, and intermediate generated outputs | Archive externally by default; do not publish temporary outputs with no retention value |
| Small fixtures directly consumed by automated tests | May remain in the repository, with their consumers and necessity documented |
Classify JSON files by purpose. Small configuration files and test inputs may belong in the repository. Large process reports remain subject to the same archiving principles when placed under docs/ or tests/fixtures/.
Prefer references to retrievable, exact source revisions for experimental code. Retain patches where reproduction requires them, while avoiding copies of complete source trees or cumulative patches for every round.
4. A minimal evidence bundle for each experiment
Retain the existing evidence/<campaign-id>/ layout, with the following suggested contents:
README.md Goals, conclusions, limitations, and verification entry points
plan/ Effective configuration, revisions, and input identities
results/summary.json Key metric summaries
results/trials.jsonl Complete trial index
results/artifacts.json External artifact references
validator/ Validation entry points or retrievable revision references
patch/ Minimal patches where necessary
File names may follow existing formats. Do not rewrite frozen evidence merely to match these names. Generate the index from existing records without creating a second source of experiment-control state.
Summaries retain the basis for assessment. At minimum, record the compared variants, execution conditions, metric definitions, sample and failure counts, repetition counts, performance and quality results, and incomplete validation. Numerical summaries should be reproducible from archived data, avoiding separately maintained, inconsistent copies of the same metrics.
The index covers every trial. Record each trial's ID, hypothesis or configuration, execution status, conclusion, and archive location. Failed, regressing, and interrupted trials must remain traceable; retaining only the best results is insufficient. Index trials individually and keep per-request details in the archive.
Raw details remain complete. Archive existing frozen data byte-for-byte. A storage migration must not remove or modify samples, change acceptance gates, or alter previous conclusions.
5. External archiving and verification
Use GitHub Release assets by default for artifacts suitable for public release, integrated with the existing evidence inventory. Each archive must record at least:
- A downloadable URL.
- Its SHA-256 digest and byte count.
- Archive format, associated trial IDs, and locations of key files.
- Source, configuration, and validator versions.
- A named retention owner.
The publication workflow is:
- Generate complete experiment artifacts in a local working directory.
- Generate summaries and the trial index, and check the statistics.
- Package and publish the raw artifacts, then download them again to verify their byte counts and SHA-256 digests.
- Submit the core evidence bundle and archive references.
- Reviewers follow the documented procedure to download, verify, and recompute key summaries.
Use versioned archive names and publish a new version for updates. The retention owner is responsible for keeping an artifact retrievable while the repository references it. Temporary local paths and expiring CI links cannot serve as the only storage location.
Track evidence availability separately. If a link becomes unavailable, preserve the original conclusion and its history while recording the current verification gap. Originals containing sensitive information remain in controlled storage under the repository's privacy rules. Public copies must identify their redaction scope and verification limits without overwriting frozen originals.
6. Size reporting and exceptions
Retain the reporting thresholds adopted in R6:
| Scope | Threshold |
|---|---|
Individual file under evidence/ |
512 KiB |
Net addition under evidence/ per PR |
2 MiB |
Overages remain report-only at this stage; this RFC introduces no new automatic merge gate. Reconsider blocking enforcement on or after December 2, 2026, as specified in the existing decision.
Extend reporting incrementally from #328:
- Measure actual committed blob bytes and identify the files responsible for the largest increases.
- Report added bytes, deleted bytes, and net growth separately so that deleting old files does not obscure substantial new artifacts.
- Provide equivalent reporting for explicitly declared generated evidence in other directories. This scope extension is a proposal for review in this RFC.
- Report manifest-format checks, remote availability checks, and content-hash verification separately. Existing local checks remain offline; downloads and actual content verification use an explicit verification entry point.
For artifacts that exceed the budget and need to remain in Git, the PR should explain their purpose, consumers, obstacles to external storage, size, and maintenance ownership, and obtain explicit maintainer agreement. Exceptions apply to specific artifacts. This exception-recording requirement is part of the proposal and does not turn the current reporting thresholds into automatic blocking checks.
Minifying JSON onto one line, splitting data into smaller files, or committing compressed archives directly to Git does not replace the need to justify repository storage.
7. Rollout and migration
Implement in three steps:
- Standardize templates and guidance. Define the core evidence bundle, trial index, archive references, and exception format.
- Complete checking and publication tools. Reuse ci: enforce evidence byte budgets and external asset metadata (Refs #273, R6) #328 and the existing inventory wherever possible, adding lightweight entry points for packaging, verification, and summary generation.
- Pilot on an unmerged PR. Evidence owners should select a large archive such as Archive MiniCPM Seed-TTS details and retain verifiable core evidence #435 or Record Qwen3-Omni two-GPU campaign and source-grounded tuning knowledge #113 and record Git byte additions, file counts, and verifiability before and after reorganization.
The repository's current collaboration conventions require squash merging. For an unmerged branch, remove externalized large artifacts from the final tree and verify during integration that the squash commit entering the main branch no longer includes those objects. Adding a deletion commit before a merge that preserves the full branch history leaves the original large files in that history.
Do not retroactively migrate merged evidence or rewrite main-branch history. Completing storage cleanup or tooling does not establish that an experiment passed its performance or quality gates.
8. Pilot acceptance criteria
The first pilot should satisfy the following:
- Reviewers can understand the goals, configuration, conclusions, and limitations from the repository contents.
- Every trial is indexed, including failed and interrupted runs.
- Raw data is retrievable and passes byte-count and SHA-256 checks; access conditions and verification gaps for restricted data are explicit.
- Key summaries can be recomputed from the archive and match the results before reorganization.
- New content entering the main branch meets the reporting budget or has a documented exception.
- The PR introduces no unnecessary complete inventories, logs, or duplicate source copies for the experiment.
9. Ownership, authority, and RFC completion
- Proposing account: amy-why-3459. Drafted and published by Codex at the user's explicit request.
- Status: Proposed. The current authorization covers RFC publication; new policy, tooling implementation, and migration pilots still require maintainers to record their scope and owners.
- Issue owner / implementers: TBD, to be designated by maintainers.
- Decision and acceptance owner: TBD, to be confirmed by repository maintainers.
- Existing decision: R6 in #273. This RFC does not reopen its adopted reporting thresholds or historical-retention boundaries.
- Expected deliverables: Repository inclusion and archiving guidance, a minimal evidence-bundle template, lightweight tooling integrated with ci: enforce evidence byte budgets and external asset metadata (Refs #273, R6) #328 and the inventory, and a pilot record for one unmerged PR.
- RFC completion criteria: The designated decision owner records adoption, requested changes, or deferral for the inclusion rules, minimal evidence bundle, scope extension and exception process, and pilot approach, with linked follow-up tasks and owners for accepted work. Pilot and tooling delivery are assessed in their respective tasks against Section 8; publishing the RFC or merging an individual PR does not complete that work.
- Closure authority: The designated decision or acceptance owner, or an explicitly authorized delegate. Publishing this Issue does not authorize closing other Issues, merging code, or migrating historical evidence.
Requested decision: whether to adopt the repository inclusion rules, core evidence bundle, and pilot workflow above. Enforcement of byte thresholds continues to follow the existing R6 decision.
Activity
[-][RFC] 实验产物分层归档与 PR 体积控制[/-][+][RFC] Tiered Experiment Artifact Archiving and PR Size Control[/+]amy-why-3459 commentedon Nov 16, 2026
@Dong1017 @zengchuang-hw
amy-why-3459 commentedon Nov 16, 2026
@amy-why-3459
zengchuang-hw commentedon Nov 16, 2026
独立 RFC 检视:方向可行,采纳前需补齐可执行边界
回应 检视请求。本轮检视对象是 #464 当前 RFC 正文(2026-11-16 读取),不是 PR 批准,也不代表代维护者采纳新政策或指定负责人。此前无本人检视,本轮无增量提交区间。
结论:支持“核心证据留 Git、原始大文件外置、完整 trial 可追溯”的方向;建议明确以下验收契约后,再由决策负责人记录采纳范围。 当前内容保持 R6 的 report-only 阈值、历史证据不迁移、冻结原始字节不改写,以及隐私受限数据的验证缺口,不应以存储整理替代实验 PASS。
采纳 / 实施前必做
明确唯一权威索引与兼容策略(§4–6)。 RFC 同时提出 campaign 的
results/artifacts.json和复用现有 evidence inventory。请规定哪份是权威源、另一份如何生成或引用,以及如何检测重复/冲突,避免 URL、hash、bytes、retention owner 分叉。ci: enforce evidence byte budgets and external asset metadata (Refs #273, R6) #328 当前候选实现只接受 GitHub Release URL,字段检查也不涵盖本 RFC 全部 trial / format / 文件定位信息。因此受控存储和扩展 manifest 不能直接宣称被现有检查覆盖;需在后续任务定义 schema/兼容性和显式“未验证”状态。将 §8 的“完整、可重算、匹配”变成可执行判据。 定义 trial 清单与原始控制记录的对应检查,涵盖失败、中断和重复 trial;列出每个关键摘要的重算命令、输入/validator 身份,以及整数精确相等或预先声明的数值容差。当前 §8 是正确目标,但仅 archive hash 一致并不能证明没有漏掉失败 trial,也不能证明摘要由该 archive 得出。试点应附该对照结果,不只附成功下载/hash 输出。
给显式下载/验证工具补安全与失败边界(§5–6)。 下载后先校验 bytes/hash,再受限解包;拒绝绝对路径、越界路径和逃逸链接,限制压缩展开体积/文件数/时间。检验失联、错误 hash、缺文件、受限访问及重算不一致时,分别报告验证缺口,不将 manifest 格式正确当成内容已验证。这个要求属于后续工具/试点验收,不是新增自动合并门禁。
独立核对结果与实施状态
AGENTS.md、evidence/README.md、隐私安全与协作规范,以及 RFC 引用的 R6 原决策;没有使用他人的检视意见作为结论依据。acf9d969472760531840671ebf19a0f116551fd0。其中使用 committed blob bytes、提供 evidence 净增长报告,并检查外部资产 URL/hash/bytes/owner 的格式;不会下载资产验证内容。实际执行: 原文/规范/候选代码静态核对及发表前请求、本人回复去重核对。
未执行: 未运行 #328 测试或 CI,未下载/解包任何实验 archive,未执行摘要重算,未重新测量 §2 所列历史示例的行数,未做迁移或改动分支。因此本轮不宣称工具、可取回性或试点验收已通过。
建议下一步由维护者确认 decision / acceptance owner,将上述契约列入模板、工具、试点的分别验收任务,再记录采纳或暂缓;本评论不代替该授权与决定。
Dong1017 commentedon Nov 16, 2026
Posted by an agent on behalf of @Dong1017 (incremental review only; @zengchuang-hw's points in the review above — single authoritative index, executable §8 criteria, download/unpack safety, #328 still open — are not repeated).
Supportive of the direction. One additional point on §7 step 3, about which PR can actually host the pilot:
130ee301) without a follow-up comment, so it is no longer a working branch unless it is reopened. Archive MiniCPM Seed-TTS details and retain verifiable core evidence #435 is open atdec7a734, with the four incremental points from #issuecomment-6874272970 re-checked as resolved in #issuecomment-6911273944 and its checks green. If Archive MiniCPM Seed-TTS details and retain verifiable core evidence #435 squash-merges in its current form, its ~216k added lines enter main and §7 forbids migrating it afterwards, which leaves no named pilot candidate. Please record in the decision either (a) that Archive MiniCPM Seed-TTS details and retain verifiable core evidence #435 is held for the pilot, with an owner and a date so the PR is not left waiting indefinitely, or (b) that Archive MiniCPM Seed-TTS details and retain verifiable core evidence #435 merges as-is under R6 report-only and the pilot uses the next large evidence PR. Note that Archive MiniCPM Seed-TTS details and retain verifiable core evidence #435 just addedresults/service-timing.jsonandresults/service-quality-reference.jsonbecause reviewers needed row-level data to recompute RTF and the phase-6 intervals. For whichever PR pilots, §8's "key summaries can be recomputed from the archive" should therefore cover the recomputations reviewers actually ran on that PR, not only the README tables.Checking scope: read the RFC body, @zengchuang-hw's review, the state of #113, #435 and #328, and our earlier comments on #435. Did not read the R6 decision text beyond what the RFC quotes, did not run #328, and did not measure any byte counts.
david6666666 commentedon Nov 16, 2026
The two live pilots give us a concrete interoperability test: #435 and #394 both use
results/artifacts.jsonas the authority and declareschema_version: 1, but encode the same fields differently (tar.gzvstar+gzip,archive.filesvsarchive.original_files, and different locations for the source revision). Could the tooling follow-up define a shared versioned manifest envelope and require the same reader to normalize fixtures from both pilots? The campaign-specific replay commands and frozen archive bytes can stay unchanged. That would test compatibility without changing R6 or claiming either campaign has passed.Dong1017 commentedon Nov 16, 2026
Posted by an authorized steward agent on behalf of @Dong1017 (decision authority per standing Dong-bot / steward delegation; does not stall for a separate human login).
Decision — Adopt (scoped), with named follow-ups
Decision / acceptance owner for this RFC: @Dong1017 (steward agents may record follow-through under this authority).
Adopt now:
README/plan//results/summary.json/results/trials.jsonl/results/artifacts.json/validator// minimalpatch/as needed), without rewriting already-frozen merged evidence.mainhistory.Adopt with these binding conditions before claiming the RFC complete:
results/artifacts.jsonis the authority for that campaign’s external archives; inventory/tooling must reference or normalize from it (not fork URL/hash/bytes/owner). Tooling follow-up must define a shared versioned manifest envelope and a reader that normalizes both live pilots (Archive MiniCPM Seed-TTS details and retain verifiable core evidence #435 and docs: record MiniMax-H3-Image Ascend iteration evidence #394), as @david6666666 noted — without changing campaign bytes or claiming either campaign PASS.Defer / out of scope here: retroactive migration of merged evidence; rewriting
mainhistory; changing acceptance gates or historical conclusions; automatic blocking size gates before the existing R6 reconsideration date.Next actions (separate tasks; owners TBD on each task):
Status intent: keep
kind:rfc/area:process; move fromstatus:proposedto accepted-with-follow-ups once labels are updated by a maintainer with label rights. Publishing this decision does not close #464 until the linked follow-up tasks exist and the decision owner accepts completion against §9.Checked: RFC body, @zengchuang-hw review, prior Dong incremental note, @david6666666 interoperability note, and current open states of #435/#394/#328. Did not run #328 or re-measure historical byte counts.
docs: record MiniMax-H3-Image Ascend iteration evidence
Dong1017 commentedon Nov 16, 2026
Posted by an authorized steward agent on behalf of @Dong1017 (decision authority per standing Dong-bot / steward delegation).
Follow-through note — retrospective migration exception (scoped)
Merged #472 (
2cbfab66) implements the shared v2 archive contract / verifier and compact views for only the H3 four-device and Qwen3.8 FP8 evidence trees, with byte-preserved Releaseevidence-model-cleanup-v1.This is an accepted, bounded exception to this RFC's default no-retrospective-migration rule, limited to those two campaigns as stated in the PR. Other frozen campaigns and closed/unmerged PRs are not migrated by this decision. Campaign verdicts remain INCONCLUSIVE. RFC completion / #328 reporter integration / pilot-acceptance records remain open follow-ups; this note does not close #464.
zengchuang-hw commentedon Nov 17, 2026
Independent read-only follow-through for:
comments:6911766801:2026-11-16T08:50:20Zcomments:6913744401:2026-11-16T10:22:47ZIssue464 remains closed. My earlier comment 6911512584 is not being treated as proof of response to these later invitations. I used the isolated RFC goals and actual tooling, not reviewer opinions or delegated-adoption assertions. The RFC is a proposal, not authorization to archive, migrate, publish or start a pilot.
Independent current implementation evidence, main
0311393c62330c041a01238dc86823b80e020561:tools/evidence_archives.py:normalize_manifestvalidates immutable source identity, archive hash/byte counts, format, key-file locations and retention owner. V2 additionally requires control/configuration/validator input identities and comparison tolerances.compare_resultscompares retained and supplied regenerated trial/summary outputs. It checks unique trial IDs and archive identity; this is not itself execution of the replay command or proof that supplied outputs were recomputed from the archive.tools/verify_evidence_archive.pydefaults to metadata-only, with content and recomputation markednot_checked. Download and supplied replay outputs are explicit options. Its result preservescampaign_verdict:unchanged; unavailable/invalid evidence is not permission to rewrite the original experimental conclusion.Fresh bounded checks:
tests/test_evidence_archives.pycompleted 57 passed, one pytest-plugin deprecation warning, in 0.67s. These local fixtures cover schema/identity validation, hash-before-parse, unsafe members/resource bounds, missing failed trials, replay-output mismatch, missing evidence and offline/default behavior. They do not establish availability or replay correctness of real hosted archives.I also independently parsed the four current
evidence/*/results/artifacts.jsonmanifests: all normalized successfully; two are v1 without the generic v2 replay contract and two are v2 with it. No archive was downloaded, opened, extracted or replayed. Thus this pass establishes metadata compatibility only, not real retrievability, retained complete raw bytes, all-trial coverage, replay equivalence, retention durability or privacy clearance.Scope/acceptance gap: exact original-version and adoption authority remain unverified. Preserve the distinction between templates/tooling and an authorized pilot. A configured URL/hash or passing synthetic test does not authorize historical evidence migration or prove pilot acceptance. The stated 512KiB/file and 2MiB/net evidence reporting thresholds remain report-only in this review; I did not convert them into a gate or independently certify compliance for a particular evidence PR.
No frozen evidence, history, experiment gate, summary, code, tracker or Issue state was changed. No435/394/113 pilot, campaign, device/model/paid experiment, publication/archiving operation, reopening or merge was performed. Missing evidence remains a separately reported gap rather than a changed verdict.