diff --git a/docs/contributing.md b/docs/contributing.md index 738456bab..003057fd4 100644 --- a/docs/contributing.md +++ b/docs/contributing.md @@ -127,11 +127,18 @@ research spikes. ```text evidence// plan/ Hypothesis, RunSpec, frozen gates and environment - results/ Raw measurements, outputs, profiler data and verdict rationale + results/ Metric summaries, complete trial index, archive references and rationale patch/ Candidate diff or exact revision pointer validator/ Frozen validation code, versions and inputs ``` +Follow the [archiving contract](evidence-archiving.md) and +[bundle template](../evidence/_template/README.md). Keep large raw measurements, +logs, profiler data and generated inventories in versioned external archives. +The campaign's `results/artifacts.json` owns metadata; index every attempt and +retain executable replay and comparison criteria. Existing frozen evidence is +not rewritten to match the template. + | Verdict | Meaning | | --- | --- | | PASS | Pinned comparison, complete evidence and every required gate met | diff --git a/docs/evidence-archiving.md b/docs/evidence-archiving.md new file mode 100644 index 000000000..4fdef8bee --- /dev/null +++ b/docs/evidence-archiving.md @@ -0,0 +1,187 @@ +# Experiment evidence and archive verification + +Implements the inclusion/template/tooling slice of the +[scoped archiving decision](https://github.com/InferMatrixLab/InferRSI/issues/464#issuecomment-6913744401). +The RFC's completion and campaign acceptance remain separate owner decisions. + +## Repository contents + +Retain implementation, necessary tests, goals, conclusions, limitations, key +metrics, effective configuration, input/validator identities, a complete trial +index and verification entry points. Start from [the bundle template](../evidence/_template/README.md). +Small test fixtures stay only when their consumers and necessity are clear. + +Archive per-request results, per-sample scores, profiler output, logs, media, +complete source inventories and generated process reports outside Git. Classify +JSON by its purpose; changing directories, minifying, splitting or committing +compressed archives does not justify retaining large generated artifacts. +Do not retain temporary outputs with no review or reproduction value. + +Generate summaries and trial indexes from the original control/evidence records. +Record all attempts, including failed, regressing, interrupted and repeated +trials. Retain sample/failure counts, repetitions, variability, metric definitions +and incomplete validation. Preserve frozen raw bytes and historical conclusions. + +## Archive authority and schema + +Each campaign's `results/artifacts.json` owns its archive metadata. The inventory +references it via `schema: archive-sha256`, `field: archive`, and the expected +local artifact path. Do not copy URL/hash/bytes/owner into inventory entries. +The offline inventory rejects such duplicate `external_assets` metadata. + +New documents use **schema_version 2**, illustrated by +[artifacts.example.json](../evidence/_template/results/artifacts.example.json). +Replace placeholders with measured values, including integer byte/file counts. +The executable format contract is `normalize_manifest` in +[tools/evidence_archives.py](../tools/evidence_archives.py). + +| Field | Contract | +| --- | --- | +| `archive.id`, `file` | Stable version identity and basename | +| `archive.url` | Credential-free HTTPS download URL; no query or fragment | +| `archive.sha256`, `bytes` | Lowercase SHA-256 and positive compressed byte count | +| `archive.format` | `tar+gzip`; regular nonsparse files only | +| `archive.source_revision` | Immutable 40–64 hexadecimal source revision | +| `archive.root` | Safe relative member prefix, or empty for the whole archive | +| `archive.files`, `uncompressed_bytes` | File count and sum of file bytes under root | +| `archive.key_files` | Named archive-relative files or directory prefixes ending `/` | +| `archive.retention_owner` | Named owner responsible while references remain | +| `verification.inputs` | Archive-relative SHA-256 identities with at least control, configuration and validator roles | +| `verification.command` | Reviewed replay argv, recorded but never executed by the generic tool | +| `verification.trials`, `summary` | Paths relative to retained/regenerated bundle roots | +| `verification.absolute_tolerance`, `relative_tolerance` | Finite nonnegative float-comparison tolerances; integers/structure exact | + +Both frozen v1 pilots remain readable through the same normalization function: +`tar.gz`/`tar+gzip`, `files`/`original_files`, +`uncompressed_bytes`/`original_bytes`, and the two source-revision/key-file +locations normalize into one view. Conflicting aliases fail validation. H3's +`snapshot.json` is outside its original-file root; it is still inspected and +counted against safety limits. V1 replay remains with each frozen campaign +checker; normalization does not rewrite either pilot's bytes or interpretation. + +## Publication and explicit verification + +1. Generate the complete raw artifact locally. Review privacy before publication. +2. Generate summaries and the index from those raw/control records; test coverage. +3. Package regular files, publish a versioned Release asset, and record measured + metadata. Controlled originals remain in authorized storage under the + [privacy rules](privacy-security.md). +4. Download again and verify the published bytes and contents with the explicit + tool. Run the reviewed replay command and compare outputs. +5. Submit the compact bundle, verification results and availability limits. + +From the repository root (replace angle-bracket placeholders): + +```bash +# Offline metadata only. Never downloads or certifies content. +python tools/verify_evidence_archive.py evidence//results/artifacts.json +python tools/validate_evidence.py + +# Explicit network operation, bounded bytes/hash/member verification. +python tools/verify_evidence_archive.py evidence//results/artifacts.json --download + +# Controlled storage: retrieve through its authorized client, then verify offline. +python tools/verify_evidence_archive.py evidence//results/artifacts.json --archive + +# Separately run the reviewed, pinned replay command from the manifest, then: +python tools/verify_evidence_archive.py evidence//results/artifacts.json \ + --archive --recomputed +``` + +The common tool authenticates compressed bytes before tar parsing, then streams +members without filesystem extraction or archived code execution. It rejects +absolute/traversing paths, duplicate names, links, devices and sparse members. +Defaults cap compressed data at 256 MiB, expanded tar bytes (including metadata) +at 512 MiB, files at 10,000, and each download/verification operation at 120 +seconds. Downloads have a 10-second socket timeout and permit only GitHub +Release hosts and their HTTPS asset redirects. These are tool resource limits, +not experiment acceptance gates. Larger archives need an explicitly reviewed +tool configuration. Split campaigns only for meaningful storage reasons. + +Failures distinguish metadata, unavailable/restricted access, bad bytes/hash, +unsafe paths/members/redirects, resource limits, missing files, bad contents, +trial mismatch and summary recomputation mismatch. Output always states which +scope was checked. Metadata success or verified archive bytes alone leaves +`recomputation: not_checked`; successful comparison reports `outputs_match`. +The caller must review the replay's derivation and coverage; comparing two +identical files cannot prove they came from independent recomputation. + +Offline CI does not download, unpack or run a replay. A broken link records a +verification gap without changing the historical verdict. Do not use expiring +CI links or local paths as the only archive source. New bytes require a new +version; keep the old digest/version retrievable. Redacted public copies have +their own identity and explicit limits, never overwrite frozen originals. + +## R6 reporting and exceptions + +Existing R6 thresholds remain report-only: 512 KiB per `evidence/` file and +2 MiB net evidence growth per PR. This change adds no blocking size gate. +Reconsideration remains on or after 2026-12-02 under the existing decision. + +[PR #328](https://github.com/InferMatrixLab/InferRSI/pull/328) owns the pending byte +reporter. Its legacy input can be generated without maintaining duplicate +metadata (after that checker is integrated): + +```bash +python tools/validate_evidence.py --budget-assets > +python tools/check_evidence_budget.py --base --report-only \ + --inventory +``` + +The generated view is disposable, not committed. #328's Release-only URL +format check does not support controlled-store URLs; use the shared reader and +an authorized local archive for those and report that compatibility gap. The +view does not claim remote availability. Added/deleted/net byte reporting +remains in #328's follow-up scope. + +Use [the exception record](../evidence/_template/plan/storage-exception.md) when +a specific oversized artifact must stay in Git. Record purpose, consumers, +externalization obstacle, byte count, maintenance owner and explicit maintainer +agreement. An exception record does not create automatic enforcement. + +## Cleanup and acceptance + +Before removing redundant generated data, identify its consumers and preserve +an exact, retrievable source with digest/size and a concise summary. Convert +behavioral tests to small synthetic inputs when they do not need a full source +scan. Verify links and affected consumers after removal. This removes the file +from the current tree, not existing Git history. + +This workflow does not automatically migrate frozen merged campaign evidence. +An explicitly requested, scoped cleanup may replace the current presentation +only after every original file is preserved in a verified versioned archive; +original frozen bytes, historical conclusions and Git history remain unchanged. +For new/unmerged work, externalize before the required squash merge so large +blobs do not enter main. The two pilot archive/replay checks and their +independent reviews remain campaign-specific acceptance records. Completing +this tool or removing redundant data neither establishes experiment PASS nor +closes the parent RFC. + +### Scoped legacy record projections + +The [H3 four-device bundle](../evidence/h3-ref2va-910c-4logical-20261101-297/README.md) +and [Qwen3.8 bundle](../evidence/qwen38-h20-fp8-single-request-vllm032/README.md) +use v2 for their explicitly requested cleanup. Their full originals, including +superseded cumulative checkpoints and repeated diagnostic code, remain in the +archives. This is not blanket authorization to migrate other frozen campaigns. + +`tools/rebuild_model_evidence.py` regenerates compact views from authenticated +originals. `archive.key_files.snapshot` identifies the original per-file hash map; +`archive.key_files.projection` identifies the declarative source/metric selection. +Both and the replay code are pinned by `verification.inputs`. The replayer uses +the common bounded reader to capture authenticated bytes, checks original frozen +manifests, and requires its local implementation to match the pinned code. It +never executes code from an archive and never overwrites an existing output. + +Index coverage is deliberately labelled: every published result JSON document +and named/nested attempt observation, including failures and cancellations. +Identical observations of the same named attempt share one row and retain all +source pointers; changed observations, retries and anonymous records remain +distinct. `configuration.record_sha256` hashes canonical JSON for the source +observation. Entry counts are not independent trial counts. Narrative-only +history and unpublished controller records cannot become fully structured +trial evidence through a storage migration; those limits remain explicit. + +Declared sample distributions are recomputed; other frozen reported statistics +and verdicts are labelled as recorded values. Replaying the projection does not +rerun inaccessible quality checks, change gates or complete campaign acceptance. diff --git a/docs/operations.md b/docs/operations.md index b5031ebe6..547900e6a 100644 --- a/docs/operations.md +++ b/docs/operations.md @@ -43,6 +43,9 @@ Effect classes used below: | Amend repository-owned knowledge | Scoped repository PR | — | Repository review and integration authority apply; this is not a runtime store Update | | Inspect/prepare experiments | CLI `doctor`; `plan` or `--dry-run` | Read-only inspection / preview | `doctor` may invoke bounded probes and does not certify target compatibility; `plan` previews a specification without persisting a campaign or authorizing execution | | Execute/inspect experiments | CLI `run`; `status`; `report` | Experiment-executing / read-only / report-generating | `run` executes caller-supplied commands and records evidence; `status` reads recorded progress; `report` generates HTML from recorded evidence | +| Inspect/verify archived evidence | `python tools/verify_evidence_archive.py`; [archive contract](evidence-archiving.md) | Read-only by default; explicit network/local scratch with `--download` | Default validates metadata offline; `--archive`/`--download` authenticates and bounds archive reading; `--recomputed` compares separately generated v2 results; never executes archived code or changes verdicts | +| Rebuild scoped legacy evidence views | `python tools/rebuild_model_evidence.py --archive --output ` | CPU-only; creates a new output directory | Authenticates original bytes and pinned projection/code; no extraction, network, archived-code execution or campaign qualification | +| Inspect evidence inventory / generate budget input | `python tools/validate_evidence.py [--budget-assets]` | Read-only | Offline layout/hash/schema checks; optional disposable #328 metadata view derived from campaign authority; no downloads | | Execute one ordinary local CPU command with a fixed record | Python `inferrsi.harness.api.Harness.allocate` / `execute` / `read` | Experiment-executing / record-writing | One explicit RunStore root and one trusted local CPU target; ordinary command mode only; a record holds execution facts, so exit 0 is not a verdict; no device, model or network fallback | | Visualize a recorded run or campaign | CLI `dashboard`; [dashboard](dashboard.md) | Report-generating / local read-only serving | Writes an offline HTML snapshot or serves an optional live view on loopback; does not run an agent or validate a claim | | Capture a knowledge context | CLI `campaign capture-context`; [offline dashboard](dashboard.md#record-the-optimization-story-campaign-trace) | Local evidence-writing | Runs `collect` with a context budget and keeps the full envelope and resource hashes in a new `knowledge/.json`; proves what was prepared, not what a model read | diff --git a/docs/reviews/vllm-ascend-0271rc1/README.md b/docs/reviews/vllm-ascend-0271rc1/README.md new file mode 100644 index 000000000..d84676e82 --- /dev/null +++ b/docs/reviews/vllm-ascend-0271rc1/README.md @@ -0,0 +1,30 @@ +# Historical source-scan inventory + +The complete generated inventory is retained at its [exact source revision](https://github.com/InferMatrixLab/InferRSI/blob/a590ccb25596396a592f237105166ade4aedeaaa/docs/reviews/vllm-ascend-0271rc1/history-inventory.json). +It is omitted from the current tree under the [archiving contract](../../evidence-archiving.md). +Runtime knowledge readers use the compact reviewed records and screening script; +they do not consume this full scan. + +- Original bytes: 1,401,157 +- SHA-256: `5df28360760a6d5cb513b7a4627eeb5a365d2d557b0fc2ff3d5c84cff925b72c` +- Upstream repository: https://github.com/vllm-project/vllm-ascend +- Upstream revision: `3b31886237ed65c435e0965b10e8002ff5769766` +- Reachable commits: 5,029 +- Keyword/path/diff candidates: 1,381 +- Topic counts (overlapping): {"communication": 148, "deepseek-v4-dspark": 275, "fusion": 973, "priority-models": 251} + +Before removal, the pinned copy was compared byte-for-byte; unique candidate +count, topic totals and inclusion of the separately reviewed commits were +checked. These are historical screening observations, not whole-commit semantic +review or native validation. The immutable source remains available in history. +This cleanup removes a generated file from the current tree only, not Git history. + +For local retrieval from a checkout containing that revision: + +```bash +git show a590ccb25596396a592f237105166ade4aedeaaa:docs/reviews/vllm-ascend-0271rc1/history-inventory.json > +``` + +Check its byte count and SHA-256 above before use. Re-run the published screening +script against the pinned upstream revision for an independent reconstruction; +behavioral regression tests use a small synthetic Git history, not this scan. diff --git a/docs/reviews/vllm-ascend-0271rc1/history-inventory.json b/docs/reviews/vllm-ascend-0271rc1/history-inventory.json deleted file mode 100644 index d082163f4..000000000 --- a/docs/reviews/vllm-ascend-0271rc1/history-inventory.json +++ /dev/null @@ -1,30368 +0,0 @@ -{ - "schema_version": 1, - "repository": "https://github.com/vllm-project/vllm-ascend", - "commit": "3b31886237ed65c435e0965b10e8002ff5769766", - "reachable_commits": 5029, - "candidate_commits": 1381, - "topic_counts": { - "deepseek-v4-dspark": 275, - "communication": 148, - "priority-models": 251, - "fusion": 973 - }, - "screening": { - "messages_and_paths": { - "deepseek-v4-dspark": "deepseek[ _-]?v?4|dsv4|deepseek_v4|dspark|indexcache|indexshare|hcpre", - "communication": "dsa.?cp|dsacp|flashcomm.?1|fused.?mc2", - "priority-models": "kimi[ _-]?k3|kimi_k3|minimax[ _-]?m3|minimax_m3|glm[ _-]?5", - "fusion": "fus(?:e|ed|ion)|gmm.?situ|swiglu.?quant|norm.?quant|qknorm.?rope" - }, - "runtime_diff_pattern": "fused|fusion|fuse_|_fuse|npu_.*(norm.*quant|matmul.*(reduce|gather|swiglu))", - "runtime_diff_paths": [ - "vllm_ascend", - "csrc" - ], - "boundary": "Keyword/path/diff candidates only; semantic review is recorded separately." - }, - "commits": [ - { - "commit": "3b31886237ed65c435e0965b10e8002ff5769766", - "subject": "[v0.27.1][Doc][Misc] Prepare v0.27.1rc1 release (#16295)", - "topics": [ - "deepseek-v4-dspark", - "communication", - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/schedule_image_build_and_push.yaml", - ".github/workflows/schedule_release_code_and_wheel.yml", - "README.md", - "README.zh.md", - "docs/source/community/versioning_policy.md", - "docs/source/faqs.md", - "docs/source/user_guide/release_notes.md", - "mkdocs.yml" - ] - }, - { - "commit": "e9335912e4991c2066f6b9e25a88e41062561c21", - "subject": "[0.27.1rc][Doc][Test] Add v0.27.1rc1 model validation and release conclusion report (#17517)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "tests/0.27.1rc_vllm-ascend_overall_conclusion_release_notes_EN.md" - ] - }, - { - "commit": "fa5b960733ffd610a0b2d4fd8b9615556f4a1d8b", - "subject": "[Doc][Model] Update Kimi-K3 PD KV transfer config (#17056)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/tutorials/models/Kimi-K3.md", - "docs/source/user_guide/support_matrix/supported_models.md" - ] - }, - { - "commit": "1862d58c095f513e237b5c51d9f3efe07b148da1", - "subject": "[Doc]Add Ascend950DT DeepseekV4 Flash Doc (#17435)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/tutorials/models/DeepSeek-V4-Flash.md" - ] - }, - { - "commit": "9b8bf62449a158a52d559974fc09fa97fb65dae7", - "subject": "[Doc][Model] Remove A2-related descriptions from MiniMax-M3 tutorial (#17411)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/tutorials/models/MiniMax-M3.md", - "docs/source/user_guide/support_matrix/supported_models.md" - ] - }, - { - "commit": "4ece41f5aa6cc06aa521782863b6c29682184d71", - "subject": "[Doc] Optimize the chinese description of ascend (#17316)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/locale/zh_CN/LC_MESSAGES/getting_started/installation/cann_image/atlas-a3.inc.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Cohere-Transcribe.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/MiniMax-M3.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Minitron-8B-Base.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Mixtral-8x7B-Instruct-v0.1.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-30B-A3B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-ASR-1.7B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3.5-397B-A17B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3.6-35B-A3B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3.8-27B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/dynamic_chunk_pipeline_parallel.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/flash_attention.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/release_notes.po" - ] - }, - { - "commit": "a8cdf421c7c7cd6be1343b8a3d946116877dc615", - "subject": "[v0.27.1rc][Doc] Translated Doc files 2026-09-22 (#17254)", - "topics": [ - "deepseek-v4-dspark", - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/Design_Documents/context_parallel.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/Design_Documents/dynamic_chunked_pipeline_parallel.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/DeepSeek-V4-Pro.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Kimi-K3.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/MiniMax-M3.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/configuration/additional_config.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/context_parallel.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/dynamic_chunk_pipeline_parallel.po" - ] - }, - { - "commit": "785ae3bff4f5aa369681a037f30ab38cd1ba687a", - "subject": "[Doc] A3 PD enforce_eager:true (#17242)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/tutorials/models/MiniMax-M3.md" - ] - }, - { - "commit": "47082f9ccf020c1080d6ef95fdf836dba7953fc4", - "subject": "[Doc] Disable jemalloc preloading for Kimi K3 on Atlas 800 A3 (#17155)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/tutorials/models/Kimi-K3.md" - ] - }, - { - "commit": "d95acdd8884aa8dd142be9f95a479fa910edc147", - "subject": "[v0.27.1rc][BugFix][SpecDecode] Skip over-limit drafting with DP alignment (#16513)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/worker/test_model_runner_v1.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "79044c4369e3f30ef86e6042952a3c9ee8a8dd13", - "subject": "[Cherry-pick][releases/v0.27.1rc][Feature][KVTransfer] Support multi-main-cache-spec layers in layerwise reuse layout (from #15442) (#17081)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/distributed/ascend_store/test_layerwise_cache_layout.py", - "tests/ut/kv_transfer/test_layerwise_multi_cache_spec.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/ascend_store/layerwise_cache_layout.py" - ] - }, - { - "commit": "fb7f60f4ce34bef69d59de4521f803c22324dd29", - "subject": "[Doc][Model] Add chunked-prefill/prefix-caching and fix 5.2 config consistency for MiniMax-M3 (#16987)", - "topics": [ - "communication", - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/MiniMax-M3.po", - "docs/source/tutorials/models/MiniMax-M3.md" - ] - }, - { - "commit": "6f9e86a714ffbc2d5cfd6740bccf3252b7c81d30", - "subject": "[BugFix][SpecDecode][v0.27] Handle empty DSpark context (#16720)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_copy_and_expand_dflash_dspark.py", - "vllm_ascend/ops/triton/spec_decode/utils.py" - ] - }, - { - "commit": "d3a45210d880860832b2d4e6a5e5e8cffaa854ab", - "subject": "[Doc] Add enforce_eager to EAGLE3 speculative-config in MiniMax-M3 PD disaggregation (#16722)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/MiniMax-M3.po", - "docs/source/tutorials/models/MiniMax-M3.md" - ] - }, - { - "commit": "93b7942540c53d8d4486c2cc4579f356f520d0e0", - "subject": "[BugFix][Attention] Restore device-local DSA-CP QLI max length (#16480)", - "topics": [ - "communication" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/attention/test_dsa_v1.py", - "vllm_ascend/attention/context_parallel/dsa_cp.py" - ] - }, - { - "commit": "65bb3688b49213bce80132cfb1488bbffc9b9c60", - "subject": "[doc]: add EAGLE3 weight link and KV cache pool PD guide for MiniMax-M3 and modifyed max model len (#16509)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/tutorials/models/MiniMax-M3.md", - "docs/source/user_guide/support_matrix/supported_models.md" - ] - }, - { - "commit": "67dd255260e531e159c9de0103b70613ed140107", - "subject": "[v0.27.1rc][BugFix][KDA] Revert #16046 to fix Kimi-K3 accuracy on Ascend A3 (#16445)", - "topics": [ - "deepseek-v4-dspark", - "priority-models", - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "csrc/attention/recurrent_kda/op_host/op_api/aclnn_recurrent_kda.cpp", - "csrc/attention/recurrent_kda/op_host/recurrent_kda_def.cpp", - "csrc/attention/recurrent_kda/op_host/recurrent_kda_tiling.cpp", - "csrc/attention/recurrent_kda/op_host/recurrent_kda_tiling_processor.h", - "csrc/attention/recurrent_kda/op_kernel/arch35/recurrent_kda.h", - "csrc/attention/recurrent_kda/op_kernel/recurrent_kda.h", - "csrc/attention/recurrent_kda/op_kernel/recurrent_kda_struct.h", - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_kimi_kda_recurrent_ascendc_npu.py", - "tests/ut/ops/test_kimi_kda.py", - "vllm_ascend/ops/kimi_kda.py" - ] - }, - { - "commit": "9272234d7d7bb41a5a2a58213d0a20540e5f6365", - "subject": "[v0.27.1rc][Performance][Kimi K3] Avoid attention-residual token-count specialization (#16006)", - "topics": [ - "priority-models", - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_kimi_k3_fusions.py", - "vllm_ascend/ops/triton/kimi_k3/attention_residual.py" - ] - }, - { - "commit": "78bf96a55bc91466eae7ac5fe080596fac7b9d5a", - "subject": "[v0.27.1rc][BugFix] Fix DSACP with SP for Deepseek V4 (#16021)", - "topics": [ - "deepseek-v4-dspark", - "communication", - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "docs/source/developer_guide/Design_Documents/context_parallel.md", - "docs/source/tutorials/models/DeepSeek-V4-Pro.md", - "docs/source/user_guide/configuration/additional_config.md", - "docs/source/user_guide/feature_guide/context_parallel.md", - "tests/e2e/pull_request/four_card/context_parallel/test_accuracy.py", - "tests/ut/attention/test_dsa_cp_o_proj_tp.py", - "tests/ut/ops/test_fused_moe.py", - "tests/ut/ops/test_prepare_finalize.py", - "tests/ut/patch/platform/test_patch_parallel_config.py", - "tests/ut/spec_decode/test_eagle_proposer.py", - "tests/ut/test_ascend_config.py", - "tests/ut/test_ascend_forward_context.py", - "tests/ut/test_profiling_config.py", - "vllm_ascend/ascend_config.py", - "vllm_ascend/attention/context_parallel/dsa_cp.py", - "vllm_ascend/device/device_op.py", - "vllm_ascend/models/common/__init__.py", - "vllm_ascend/models/common/ops/__init__.py", - "vllm_ascend/models/common/ops/sequence_parallel.py", - "vllm_ascend/models/deepseek_v4/dspark.py", - "vllm_ascend/models/deepseek_v4/model.py", - "vllm_ascend/models/deepseek_v4/mtp.py", - "vllm_ascend/ops/dsa.py", - "vllm_ascend/ops/fused_moe/prepare_finalize.py", - "vllm_ascend/ops/fused_moe/router/fused_topk_router.py", - "vllm_ascend/patch/platform/__init__.py", - "vllm_ascend/patch/platform/patch_parallel_config.py", - "vllm_ascend/platform.py", - "vllm_ascend/profiling_config.py", - "vllm_ascend/spec_decode/llm_base_proposer.py", - "vllm_ascend/spec_decode/step3p5.py" - ] - }, - { - "commit": "2295ef002de030df621a276317648925114aef03", - "subject": "[0.27.1rc][Feature][Config] Warn about upcoming DSA CP deprecation (#15890)", - "topics": [ - "communication" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/user_guide/feature_guide/context_parallel.md", - "vllm_ascend/ascend_config.py" - ] - }, - { - "commit": "3da432751b879aa529f2399d020eaf25e3793e7f", - "subject": "[v0.27.1rc][Performance][KDA] Reduce preprocessing copies and redundant output masks (#16046)", - "topics": [ - "priority-models", - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "csrc/attention/recurrent_kda/op_host/op_api/aclnn_recurrent_kda.cpp", - "csrc/attention/recurrent_kda/op_host/recurrent_kda_def.cpp", - "csrc/attention/recurrent_kda/op_host/recurrent_kda_tiling.cpp", - "csrc/attention/recurrent_kda/op_host/recurrent_kda_tiling_processor.h", - "csrc/attention/recurrent_kda/op_kernel/arch35/recurrent_kda.h", - "csrc/attention/recurrent_kda/op_kernel/recurrent_kda.h", - "csrc/attention/recurrent_kda/op_kernel/recurrent_kda_struct.h", - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_kimi_kda_recurrent_ascendc_npu.py", - "tests/ut/ops/test_kimi_kda.py", - "vllm_ascend/ops/kimi_kda.py" - ] - }, - { - "commit": "215043676e00696c0c0c2f89984c57ba495ee532", - "subject": "[BugFix][Model] Fix Kimi K3 dense MLP sequence parallelism (#16354)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/models/test_kimi_k3_adapter.py", - "vllm_ascend/models/kimi_k3.py" - ] - }, - { - "commit": "1c4b2768bd3c3362387286431bbea70106765eb9", - "subject": "[BugFix][KDA] Allocate Post-WU workspace for padded chunks (#16180)", - "topics": [ - "deepseek-v4-dspark", - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "csrc/attention/chunk_kda_fwd/op_host/chunk_kda_fwd_tiling.cpp", - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_chunk_kda_aclnn.py" - ] - }, - { - "commit": "39ea041bd87116719a380b24b93bd98614c895c4", - "subject": "[0.27.1rc][BugFix][Attention] Support native non-power-of-two MLA heads (#15882)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/attention/test_mla_v1.py", - "vllm_ascend/attention/mla_v1.py" - ] - }, - { - "commit": "84815632457e86391cff65003b1ff226a99e8aa5", - "subject": "[Doc][Model] Migrate Kimi-K3 guide to v0.27.1 (#16008)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "docs/hooks/nav_titles.py", - "docs/source/tutorials/models/Kimi-K3.md", - "docs/source/user_guide/support_matrix/supported_models.md" - ] - }, - { - "commit": "9e2ba22f767e6d3deef757c77f6e91f226d37d7c", - "subject": "[v0.27.1rc][Feature] support Kimi K3 DSpark with DCP (#16276)", - "topics": [ - "deepseek-v4-dspark", - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "tests/e2e/pull_request/one_card/test_model_runner_v1_with_device.py", - "tests/ut/attention/test_mla_cp.py", - "tests/ut/attention/test_mla_cp_mask_slice.py", - "tests/ut/ops/test_gdn_attn_builder.py", - "tests/ut/patch/platform/test_patch_speculative_config_dspark.py", - "tests/ut/patch/platform/test_prefix_cache_cp_patches.py", - "tests/ut/spec_decode/test_dspark_proposer.py", - "tests/ut/worker/test_dcp_manager.py", - "vllm_ascend/attention/context_parallel/mla_cp.py", - "vllm_ascend/ops/gdn_attn_builder.py", - "vllm_ascend/ops/triton/spec_decode/utils.py", - "vllm_ascend/patch/platform/patch_mamba_manager.py", - "vllm_ascend/patch/platform/patch_speculative_config.py", - "vllm_ascend/spec_decode/dspark_proposer.py", - "vllm_ascend/spec_decode/llm_base_proposer.py", - "vllm_ascend/worker/dcp_utils.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "8ecaac3d413f4fdf735e6f393386028426f738f5", - "subject": "[v0.27.1rc][Doc] Translated Doc files 2026-09-11 (#16301)", - "topics": [ - "deepseek-v4-dspark", - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/locale/zh_CN/LC_MESSAGES/community/versioning_policy.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/Design_Documents/balance_schedule_refactor.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/Design_Documents/speculative_kv_sliding_window.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/contribution/e2e_ci_test.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/contribution/testing.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/performance_and_debug/performance_benchmark.po", - "docs/source/locale/zh_CN/LC_MESSAGES/getting_started/installation.po", - "docs/source/locale/zh_CN/LC_MESSAGES/getting_started/installation/base_environment.inc.po", - "docs/source/locale/zh_CN/LC_MESSAGES/getting_started/installation/cann_image/atlas-200i-pro.inc.po", - "docs/source/locale/zh_CN/LC_MESSAGES/getting_started/installation/cann_image/atlas-300i-duo.inc.po", - "docs/source/locale/zh_CN/LC_MESSAGES/getting_started/installation/cann_image/atlas-950dt.inc.po", - "docs/source/locale/zh_CN/LC_MESSAGES/getting_started/installation/cann_image/atlas-a2.inc.po", - "docs/source/locale/zh_CN/LC_MESSAGES/getting_started/installation/cann_image/atlas-a3.inc.po", - "docs/source/locale/zh_CN/LC_MESSAGES/getting_started/installation/existing_cann_environment.inc.po", - "docs/source/locale/zh_CN/LC_MESSAGES/getting_started/installation/install_vllm_ascend.inc.po", - "docs/source/locale/zh_CN/LC_MESSAGES/getting_started/installation/prebuilt_image.inc.po", - "docs/source/locale/zh_CN/LC_MESSAGES/getting_started/installation/supported_hardware.inc.po", - "docs/source/locale/zh_CN/LC_MESSAGES/getting_started/overview.po", - "docs/source/locale/zh_CN/LC_MESSAGES/getting_started/quick_start.po", - "docs/source/locale/zh_CN/LC_MESSAGES/getting_started/quick_start/ascend_image/atlas-200i-pro.inc.po", - "docs/source/locale/zh_CN/LC_MESSAGES/getting_started/quick_start/ascend_image/atlas-300i-duo.inc.po", - "docs/source/locale/zh_CN/LC_MESSAGES/getting_started/quick_start/ascend_image/atlas-950dt.inc.po", - "docs/source/locale/zh_CN/LC_MESSAGES/getting_started/quick_start/ascend_image/atlas-a2.inc.po", - "docs/source/locale/zh_CN/LC_MESSAGES/getting_started/quick_start/ascend_image/atlas-a3.inc.po", - "docs/source/locale/zh_CN/LC_MESSAGES/getting_started/quick_start/ascend_image/image_download_mirror.inc.po", - "docs/source/locale/zh_CN/LC_MESSAGES/getting_started/quick_start/ascend_image/verify_container.inc.po", - "docs/source/locale/zh_CN/LC_MESSAGES/getting_started/quick_start/offline/qwen3-0.6b-310p.inc.po", - "docs/source/locale/zh_CN/LC_MESSAGES/getting_started/quick_start/offline/qwen3-0.6b.inc.po", - "docs/source/locale/zh_CN/LC_MESSAGES/getting_started/quick_start/online/qwen3-0.6b-310p.inc.po", - "docs/source/locale/zh_CN/LC_MESSAGES/getting_started/quick_start/online/qwen3-0.6b.inc.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/features/dynamic_chunked_pipeline_parallel.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Cohere-Transcribe.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/DeepSeek-R1.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/DeepSeek-V3.1.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/DeepSeek-V3.2.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/DeepSeek-V4-Flash.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/DeepSeek-V4-Pro.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/DeepSeekOCR2.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/GLM4.x.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/GLM5.2.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/GLM5.3.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/GLM5.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Gemma4.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Hy3-preview.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Hy4-preview.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/InternVL3.5.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Kimi-K2-Thinking.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Kimi-K2.5.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Kimi-K2.6.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Kimi-K3.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/LLaVA-OneVision-Qwen2-0.5B-OV.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/MiniMax-M2.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/MiniMax-M3.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Minitron-8B-Base.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Mixtral-8x7B-Instruct-v0.1.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/PaddleOCR-VL.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen-VL-Dense.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen2.5-Math-RM-72B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-235B-A22B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-30B-A3B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-ASR-1.7B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-Coder-30B-A3B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-Dense.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-Embedding.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-Next.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-Omni-30B-A3B-Thinking.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-Reranker.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-VL-235B-A22B-Instruct.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-VL-30B-A3B-Instruct.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-VL-Embedding.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-VL-Reranker.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3.5-27B-Qwen3.6-27B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3.5-397B-A17B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3.5-Dense.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3.6-35B-A3B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3.8-2.4T-A95B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3.8-27B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/gpt-oss-120b.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/configuration/additional_config.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/configuration/encoder_cache_manager.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/batch_invariance.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/context_parallel.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/flash_attention.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/kv_pool.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/quantization.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/rl.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/short_request_first.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/speculative_decoding.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/ucm_deployment.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/index.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/release_notes.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/support_matrix/supported_models.po" - ] - }, - { - "commit": "2669e5b6f2d7d91855de8e1e94d90407c2fc55ff", - "subject": "[Performance][LoRA] Improve LoRA performance on A2 & A3 (#16190)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_lora_linear.py", - "tests/ut/lora/test_lora.py", - "tests/ut/lora/test_punica_npu.py", - "tests/ut/lora/test_utils.py", - "vllm_ascend/lora/lora_ops.py", - "vllm_ascend/lora/punica_npu.py", - "vllm_ascend/lora/utils.py", - "vllm_ascend/ops/triton/docs/lora_linear.md", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "bc7c2fe869c09a4db4dc14cf2b5d29b9aa3b254d", - "subject": " [doc] Add PD separation guidance and standalone layout for the Minimax M3 with 950DT. For releases/v0.27.1 (#16173)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/MiniMax-M3.po", - "docs/source/tutorials/models/MiniMax-M3.md", - "docs/source/user_guide/support_matrix/supported_models.md" - ] - }, - { - "commit": "19ee63ec9518475cbdce309352ccfaeb7e70e061", - "subject": "[Feature][Model] Enable MiniMax-M3 FP8 MSA index score on A5 for v0.27.1rc (#15926)", - "topics": [ - "priority-models", - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "csrc/attention/msa_index_score/README.md", - "csrc/attention/msa_index_score/docs/aclnnMsaIndexScore.md", - "csrc/attention/msa_index_score/examples/test_aclnn_msa_index_score.cpp", - "csrc/attention/msa_index_score/msa_index_score_torch_adpt.h", - "csrc/attention/msa_index_score/op_host/CMakeLists.txt", - "csrc/attention/msa_index_score/op_host/msa_index_score_def.cpp", - "csrc/attention/msa_index_score/op_host/msa_index_score_infershape.cpp", - "csrc/attention/msa_index_score/op_host/msa_index_score_tiling.cpp", - "csrc/attention/msa_index_score/op_host/msa_index_score_tiling.h", - "csrc/attention/msa_index_score/op_kernel/arch22/msa_block_mmad.h", - "csrc/attention/msa_index_score/op_kernel/arch22/msa_index_score_kernel.h", - "csrc/attention/msa_index_score/op_kernel/arch22/msa_index_score_task.h", - "csrc/attention/msa_index_score/op_kernel/arch22/msa_seg_row_max_epilogue.h", - "csrc/attention/msa_index_score/op_kernel/arch35/msa_block_mmad.h", - "csrc/attention/msa_index_score/op_kernel/arch35/msa_index_score_kernel.h", - "csrc/attention/msa_index_score/op_kernel/arch35/msa_seg_row_max_epilogue.h", - "csrc/attention/msa_index_score/op_kernel/catlass/arch/msa_arch.hpp", - "csrc/attention/msa_index_score/op_kernel/catlass/arch/msa_cross_core_sync.hpp", - "csrc/attention/msa_index_score/op_kernel/catlass/arch/msa_local_tensor_buffer.hpp", - "csrc/attention/msa_index_score/op_kernel/catlass/arch/msa_resource.hpp", - "csrc/attention/msa_index_score/op_kernel/catlass/detail/msa_alignment.hpp", - "csrc/attention/msa_index_score/op_kernel/catlass/detail/msa_dependent_false.hpp", - "csrc/attention/msa_index_score/op_kernel/catlass/detail/msa_macros.hpp", - "csrc/attention/msa_index_score/op_kernel/catlass/gemm/msa_dispatch_policy.hpp", - "csrc/attention/msa_index_score/op_kernel/catlass/gemm/msa_gemm_helper.hpp", - "csrc/attention/msa_index_score/op_kernel/catlass/gemm/msa_gemm_type.hpp", - "csrc/attention/msa_index_score/op_kernel/catlass/gemm/tile/msa_cast_fp8_to_fp16.hpp", - "csrc/attention/msa_index_score/op_kernel/catlass/gemm/tile/msa_cast_int4_to_int8.hpp", - "csrc/attention/msa_index_score/op_kernel/catlass/gemm/tile/msa_cast_int8_to_fp16.hpp", - "csrc/attention/msa_index_score/op_kernel/catlass/gemm/tile/msa_copy_gm_to_l1.hpp", - "csrc/attention/msa_index_score/op_kernel/catlass/gemm/tile/msa_copy_gm_to_ub.hpp", - "csrc/attention/msa_index_score/op_kernel/catlass/gemm/tile/msa_copy_l0c_to_gm.hpp", - "csrc/attention/msa_index_score/op_kernel/catlass/gemm/tile/msa_copy_l1_to_bt.hpp", - "csrc/attention/msa_index_score/op_kernel/catlass/gemm/tile/msa_copy_l1_to_fp.hpp", - "csrc/attention/msa_index_score/op_kernel/catlass/gemm/tile/msa_copy_l1_to_l0a.hpp", - "csrc/attention/msa_index_score/op_kernel/catlass/gemm/tile/msa_copy_l1_to_l0b.hpp", - "csrc/attention/msa_index_score/op_kernel/catlass/gemm/tile/msa_copy_ub_to_gm.hpp", - "csrc/attention/msa_index_score/op_kernel/catlass/gemm/tile/msa_tile_copy.hpp", - "csrc/attention/msa_index_score/op_kernel/catlass/gemm/tile/msa_tile_mmad.hpp", - "csrc/attention/msa_index_score/op_kernel/catlass/layout/msa_layout.hpp", - "csrc/attention/msa_index_score/op_kernel/catlass/layout/msa_matrix.hpp", - "csrc/attention/msa_index_score/op_kernel/catlass/layout/msa_tensor.hpp", - "csrc/attention/msa_index_score/op_kernel/catlass/layout/msa_vector.hpp", - "csrc/attention/msa_index_score/op_kernel/catlass/msa_catlass.hpp", - "csrc/attention/msa_index_score/op_kernel/catlass/msa_conv_coord.hpp", - "csrc/attention/msa_index_score/op_kernel/catlass/msa_coord.hpp", - "csrc/attention/msa_index_score/op_kernel/catlass/msa_gemm_coord.hpp", - "csrc/attention/msa_index_score/op_kernel/catlass/msa_matrix_coord.hpp", - "csrc/attention/msa_index_score/op_kernel/catlass/msa_numeric_size.hpp", - "csrc/attention/msa_index_score/op_kernel/msa_index_score.cpp", - "csrc/attention/msa_index_score/op_kernel/msa_index_score_common.h", - "csrc/attention/msa_index_score/op_kernel/msa_index_score_debug.h", - "csrc/attention/msa_index_score/tests/README.md", - "csrc/attention/msa_index_score/tests/golden/msa_index_score_golden.py", - "csrc/build_aclnn.sh", - "tests/ut/models/minimax_m3/test_msa_m3.py", - "vllm_ascend/models/minimax_m3/msa_m3.py", - "vllm_ascend/models/minimax_m3/ops/msa_m3_npu.py" - ] - }, - { - "commit": "05841f71a6c29e6e0d9752a8adacbe681bd0ab78", - "subject": "[0.27.1][BugFix] Fix kimi k3 dspark kv grouping when --no-enable-prefix-caching (#16147)", - "topics": [ - "deepseek-v4-dspark", - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/patch/platform/test_prefix_cache_cp_patches.py", - "vllm_ascend/patch/platform/patch_kv_cache_utils.py" - ] - }, - { - "commit": "0934bc1ece53534af043119eb1ad952733ff287d", - "subject": "[BugFix][Frontend] Align DeepSeek V4 frontend behavior for v0.27 (#16115)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/patch/platform/test_deepseek_v4_thinking.py", - "tests/ut/patch/platform/test_deepseek_v4_tool_streaming.py", - "vllm_ascend/patch/__init__.py", - "vllm_ascend/patch/platform/__init__.py", - "vllm_ascend/patch/platform/patch_deepseek_v4_frontend.py", - "vllm_ascend/patch/platform/patch_deepseek_v4_tool_streaming.py" - ] - }, - { - "commit": "0b129c7f1deed4637836cf0f0e9a8a2bdd53e155", - "subject": "[BugFix][v0.27.1]Fix eagle external hit trim condition to apply without layerwise mode (#16132)", - "topics": [ - "priority-models", - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/distributed/kv_transfer/kv_pool/ascend_store/pool_scheduler.py" - ] - }, - { - "commit": "cf0617597b97f341044e08073213eb01c038546c", - "subject": "[0.27.1][BugFix][Feat] Fix _prepare_prefill_topk_scores_kernel op (#16122)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/models/minimax_m3/ops/msa_m3_triton.py" - ] - }, - { - "commit": "7ba437c5f6fdb08fc71bd4bb9864eb654cabedfd", - "subject": "[v0.27.1][Feature][MoE] Integrate GMM-SiTU operator (#15871)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "CMakeLists.txt", - "csrc/build_aclnn.sh", - "csrc/moe/grouped_matmul_situ_quant/CMakeLists.txt", - "csrc/moe/grouped_matmul_situ_quant/README.md", - "csrc/moe/grouped_matmul_situ_quant/grouped_matmul_situ_quant_torch_adpt.h", - "csrc/moe/grouped_matmul_situ_quant/op_host/CMakeLists.txt", - "csrc/moe/grouped_matmul_situ_quant/op_host/grouped_matmul_situ_quant_def.cpp", - "csrc/moe/grouped_matmul_situ_quant/op_host/grouped_matmul_situ_quant_infershape.cpp", - "csrc/moe/grouped_matmul_situ_quant/op_host/grouped_matmul_situ_quant_tiling.cpp", - "csrc/moe/grouped_matmul_situ_quant/op_kernel/gmsq_vcv_controller.h", - "csrc/moe/grouped_matmul_situ_quant/op_kernel/grouped_matmul_situ_quant.cpp", - "csrc/moe/grouped_matmul_situ_quant/op_kernel/grouped_matmul_situ_quant_tiling.h", - "csrc/moe/grouped_matmul_situ_quant/op_kernel/situ_epilogue.h", - "csrc/moe/grouped_matmul_situ_quant/op_kernel/vendor/gmsq2/basic_block_config.h", - "csrc/moe/grouped_matmul_situ_quant/op_kernel/vendor/gmsq2/basic_block_vf_mx.h", - "csrc/moe/grouped_matmul_situ_quant/op_kernel/vendor/gmsq2/gmmsq_quant_mx_vf.h", - "csrc/moe/grouped_matmul_situ_quant/op_kernel/vendor/gmsq2/gmmsq_weight_quant_cube_compute.h", - "csrc/moe/grouped_matmul_situ_quant/op_kernel/vendor/gmsq2/gmmsq_weight_quant_cube_compute_tools.h", - "csrc/moe/grouped_matmul_situ_quant/op_kernel/vendor/gmsq2/gmmsq_weight_quant_vcv_basic_block.h", - "csrc/moe/grouped_matmul_situ_quant/op_kernel/vendor/gmsq2/gmmsq_weight_quant_vec_compute.h", - "csrc/moe/grouped_matmul_situ_quant/op_kernel/vendor/wqbmm/anti_quant_y_vf.h", - "csrc/moe/grouped_matmul_situ_quant/op_kernel/vendor/wqbmm/basic_api/weight_quant_basic_api_v1.h", - "csrc/moe/grouped_matmul_situ_quant/op_kernel/vendor/wqbmm/basic_block_config.h", - "csrc/moe/grouped_matmul_situ_quant/op_kernel/vendor/wqbmm/basic_block_vf_mx.h", - "csrc/moe/grouped_matmul_situ_quant/op_kernel/vendor/wqbmm/basic_block_vf_nd.h", - "csrc/moe/grouped_matmul_situ_quant/op_kernel/vendor/wqbmm/basic_block_vf_nz.h", - "csrc/moe/grouped_matmul_situ_quant/op_kernel/vendor/wqbmm/custom_policy/wqbmm_copy_cube_out.h", - "csrc/moe/grouped_matmul_situ_quant/op_kernel/vendor/wqbmm/custom_policy/wqbmm_custom_policy.h", - "csrc/moe/grouped_matmul_situ_quant/op_kernel/vendor/wqbmm/grouped_matmul_tiling_data_apt.h", - "csrc/moe/grouped_matmul_situ_quant/op_kernel/vendor/wqbmm/grouped_matmul_weight_quant_basic_controller.h", - "csrc/moe/grouped_matmul_situ_quant/op_kernel/vendor/wqbmm/grouped_matmul_weight_quant_resplit_controller.h", - "csrc/moe/grouped_matmul_situ_quant/op_kernel/vendor/wqbmm/mx_a8w4_scale_trans_id_constant.h", - "csrc/moe/grouped_matmul_situ_quant/op_kernel/vendor/wqbmm/weight_quant_basic_block.h", - "csrc/moe/grouped_matmul_situ_quant/op_kernel/vendor/wqbmm/weight_quant_basic_block_base.h", - "csrc/moe/grouped_matmul_situ_quant/op_kernel/vendor/wqbmm/weight_quant_cube_compute.h", - "csrc/moe/grouped_matmul_situ_quant/op_kernel/vendor/wqbmm/weight_quant_tiling_key.h", - "csrc/moe/grouped_matmul_situ_quant/op_kernel/vendor/wqbmm/weight_quant_tool.h", - "csrc/moe/grouped_matmul_situ_quant/op_kernel/vendor/wqbmm/weight_quant_vcv_basic_block.h", - "csrc/moe/grouped_matmul_situ_quant/op_kernel/vendor/wqbmm/weight_quant_vcv_basic_block_base.h", - "csrc/moe/grouped_matmul_situ_quant/op_kernel/vendor/wqbmm/weight_quant_vec_compute.h", - "csrc/moe/grouped_matmul_situ_quant/ops.h", - "csrc/moe/grouped_matmul_situ_quant/sync_from_p0.sh", - "csrc/moe/grouped_matmul_situ_quant/utils/torch_kernel_helper.h", - "csrc/torch_binding.cpp", - "csrc/torch_binding_meta.cpp", - "tests/ut/device/test_device_op.py", - "tests/ut/ops/test_moe_mlp.py", - "tests/ut/quantization/methods/test_w4a8_mxfp4.py", - "vllm_ascend/device/device_op.py", - "vllm_ascend/ops/fused_moe/moe_mlp.py", - "vllm_ascend/quantization/methods/w4a8/w4a8_mxfp4.py" - ] - }, - { - "commit": "e36b303778cd0aa319c521ee8734ab5ea8418ab4", - "subject": "[v0.27.1rc][Performance][MoE] Overlap shared-expert collectives and routed projections (#15974)", - "topics": [ - "deepseek-v4-dspark", - "priority-models", - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/_310p/fused_moe/test_shared_fused_moe_310.py", - "tests/ut/models/test_kimi_k3_adapter.py", - "tests/ut/ops/test_fused_moe.py", - "vllm_ascend/_310p/fused_moe/fused_moe.py", - "vllm_ascend/ops/fused_moe/fused_moe.py", - "vllm_ascend/ops/fused_moe/shared_experts.py" - ] - }, - { - "commit": "e4c9372abb76da32096b9d32c14ec396637115d0", - "subject": "[v0.27.1rc][Bugfix][MLA] Detect draft models from runner type (#15977)", - "topics": [ - "deepseek-v4-dspark", - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "tests/e2e/pull_request/eight_card/model_runner_v2/test_glm5_2.py", - "tests/ut/attention/test_mla_v1.py", - "tests/ut/spec_decode/test_llm_base_proposer.py", - "vllm_ascend/attention/mla_v1.py", - "vllm_ascend/patch/worker/patch_v2/patch_dspark.py", - "vllm_ascend/spec_decode/llm_base_proposer.py" - ] - }, - { - "commit": "cc3537bd59ccb1297bc21619624d1cd7d2ac04c3", - "subject": "[v0.27.1rc][Performance][KDA] Compose and overlap gate projections (#15978)", - "topics": [ - "priority-models", - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/workflows/pr_test.yaml", - "tests/ut/models/test_kimi_k3_adapter.py", - "tests/ut/ops/test_kimi_kda.py", - "tests/ut/test_platform.py", - "vllm_ascend/models/kimi_k3.py", - "vllm_ascend/ops/kimi_kda.py", - "vllm_ascend/platform.py" - ] - }, - { - "commit": "e61a5d7204fe4a0fc329471df8e4c3c90e9bb2f1", - "subject": "[0.27.1rc][Feature][Model] Enable MiniMax-M3 FP8 MSA and index top-k on A5 (#15897)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "csrc/attention/sparse_attention_score/sparse_attention_score_torch_adpt.h", - "csrc/torch_binding.cpp", - "csrc/torch_binding_meta.cpp", - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_rope.py", - "tests/ut/models/minimax_m3/test_msa_m3.py", - "tests/ut/ops/test_register_custom_ops.py", - "tests/ut/ops/test_rotary_embedding.py", - "tests/ut/test_platform.py", - "tests/ut/worker/test_worker_v1.py", - "vllm_ascend/_310p/ops/rotary_embedding.py", - "vllm_ascend/models/minimax_m3/minimax_m3.py", - "vllm_ascend/models/minimax_m3/msa_m3.py", - "vllm_ascend/models/minimax_m3/ops/msa_m3_npu.py", - "vllm_ascend/models/minimax_m3/ops/msa_m3_triton_a5.py", - "vllm_ascend/ops/register_custom_ops.py", - "vllm_ascend/ops/rotary_embedding.py", - "vllm_ascend/ops/triton/docs/rope.md", - "vllm_ascend/ops/triton/rope.py", - "vllm_ascend/platform.py", - "vllm_ascend/worker/worker.py" - ] - }, - { - "commit": "4a24c5f18348f09f45fbf329a013e83ce4b58b47", - "subject": "[0.27.1rc][Doc][Misc] Standardize DeepSeek-V4 served model names (#15945)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/tutorials/models/DeepSeek-V4-Flash.md", - "docs/source/tutorials/models/DeepSeek-V4-Pro.md" - ] - }, - { - "commit": "0e52c4a121ee9fe6f6ec1e474c44bcc37c2bc2a9", - "subject": "[Doc][Misc] Standardize unit abbreviation to lowercase 'k' in documentation (#15735)", - "topics": [ - "deepseek-v4-dspark", - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "csrc/attention/compressor/README.md", - "docs/source/_templates/Model-Deployment-Tutorial-Template.md", - "docs/source/_templates/Model-Deployment-Tutorial-Template.zh.md", - "docs/source/developer_guide/Design_Documents/dynamic_chunked_pipeline_parallel.md", - "docs/source/developer_guide/Design_Documents/speculative_kv_sliding_window.md", - "docs/source/tutorials/features/dynamic_chunked_pipeline_parallel.md", - "docs/source/tutorials/models/DeepSeek-V3.1.md", - "docs/source/tutorials/models/DeepSeek-V4-Flash.md", - "docs/source/tutorials/models/DeepSeek-V4-Pro.md", - "docs/source/tutorials/models/GLM5.2.md", - "docs/source/tutorials/models/GLM5.md", - "docs/source/tutorials/models/Hy4-preview.md", - "docs/source/tutorials/models/Kimi-K2-Thinking.md", - "docs/source/tutorials/models/Kimi-K2.5.md", - "docs/source/tutorials/models/Kimi-K2.6.md", - "docs/source/tutorials/models/Minitron-8B-Base.md", - "docs/source/tutorials/models/Qwen3-Coder-30B-A3B.md", - "docs/source/tutorials/models/Qwen3.5-397B-A17B.md", - "docs/source/user_guide/feature_guide/kv_pool.md", - "docs/source/user_guide/feature_guide/speculative_decoding.md", - "docs/source/user_guide/feature_guide/ucm_deployment.md", - "docs/source/user_guide/release_notes.md" - ] - }, - { - "commit": "53f698ff7a48ce1f2b6c6ae5d26811883001bcd5", - "subject": "[0.27.1rc][Performance][Worker] Build DeepSeek V4 metadata asynchronously on v0.27 (#15776)", - "topics": [ - "deepseek-v4-dspark", - "communication" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/configs/nightly_config.yaml", - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_compressor_metadata.py", - "tests/ut/attention/test_dsa_v1.py", - "tests/ut/attention/test_dsv4_block_geometry.py", - "tests/ut/models/test_deepseek_v4_compressor.py", - "tests/ut/spec_decode/test_dspark_proposer.py", - "tests/ut/worker/test_device_metadata.py", - "tests/ut/worker/test_model_runner_v1.py", - "vllm_ascend/ascend_forward_context.py", - "vllm_ascend/attention/context_parallel/dsa_cp.py", - "vllm_ascend/attention/dsa_v1.py", - "vllm_ascend/models/deepseek_v4/compressor.py", - "vllm_ascend/models/deepseek_v4/indexer.py", - "vllm_ascend/spec_decode/dspark_proposer.py", - "vllm_ascend/spec_decode/llm_base_proposer.py", - "vllm_ascend/worker/device_metadata.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "d8c9e3a673af918c94a7794ef6f1f69c024d9c62", - "subject": "[BugFix][GDN] Skip recurrent state updates for idle DP dummy runs (#15655)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/worker/test_model_runner_v1.py", - "vllm_ascend/_310p/model_runner_310p.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "c3394206bdd00b132580ae439555b4f1a70d5bf6", - "subject": "[0.27.1rc][Bugfix][KV Pool] Fix AscendStore hybrid Mamba cache release (#15763)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/distributed/ascend_store/test_pool_scheduler.py", - "tests/ut/distributed/ascend_store/test_pool_worker.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/ascend_store/pool_scheduler.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/ascend_store/pool_worker.py" - ] - }, - { - "commit": "1918f3527425df019fd3f211d7961316edef985f", - "subject": "[0.27.1rc][Doc][Misc] Update DeepSeek-V4-Flash A3 PD separation DSpark configuration (#15696)", - "topics": [ - "deepseek-v4-dspark", - "communication", - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/tutorials/models/DeepSeek-V4-Flash.md" - ] - }, - { - "commit": "5313164cb62134291b396f94e6862442035f4eda", - "subject": "[v0.27.0][Doc] Update DeepSeek V4 Pro deployment guide (#15682)", - "topics": [ - "deepseek-v4-dspark", - "communication" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/tutorials/models/DeepSeek-V4-Pro.md" - ] - }, - { - "commit": "455117778ae5d4c305fc4d11a9330c7620eb767d", - "subject": "[Backport v0.27.1rc][BugFix][P/D] Fix Mooncake CP transfer group block IDs (#15663)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/kv_offload/test_mooncake_connector.py", - "vllm_ascend/distributed/kv_transfer/kv_p2p/mooncake_connector.py" - ] - }, - { - "commit": "4273a3261e2dae60d7c7869632259faa1740453b", - "subject": "[Backport v0.27.1rc][Doc][Model] Align serving environment variables (#15662)", - "topics": [ - "communication", - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/tutorials/models/Hy4-preview.md", - "docs/source/tutorials/models/Qwen3-Dense.md" - ] - }, - { - "commit": "a20695a4627ce45111c44ae36167a3c93231456f", - "subject": "[v0.27.1rc][BugFix][Scheduler] Preserve Mamba spec window on PD consumers (#15615)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/patch/platform/test_patch_mamba_block_aligned_split.py", - "vllm_ascend/patch/__init__.py", - "vllm_ascend/patch/platform/__init__.py", - "vllm_ascend/patch/platform/patch_mamba_block_aligned_split.py" - ] - }, - { - "commit": "ff02be1e1ffc87d4b8135bca99a93f7d283067c9", - "subject": "[Test][BugFix] Fix inconsistent enable_dsa_cp config in DeepSeek-V3.2-W8A8-EP test (#15581)", - "topics": [ - "communication" - ], - "runtime_patch_match": false, - "paths": [ - "tests/e2e/nightly/multi_node/internal_dp/config/DeepSeek-V3_2-W8A8-EP.yaml" - ] - }, - { - "commit": "9915508a14f561dd00d499a4171460c87b5393b1", - "subject": "[Feature] Support layerwise KV pool for Qwen3.5 hybrid GDN+full_attention models (#12711)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/e2e/pull_request/one_card/test_gdn_layerwise_kv.py", - "tests/ut/distributed/ascend_store/_mock_deps.py", - "tests/ut/distributed/ascend_store/test_ascend_store_connector.py", - "tests/ut/kv_offload/test_ascend_multi_connector.py", - "tests/ut/ops/test_gdn_layerwise_kv.py", - "tests/ut/patch/worker/test_patch_mamba_utils.py", - "vllm_ascend/attention/utils.py", - "vllm_ascend/distributed/kv_transfer/ascend_multi_connector.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/ascend_store/ascend_store_connector.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/ascend_store/pool_scheduler.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/ascend_store/pool_worker.py", - "vllm_ascend/ops/gdn.py", - "vllm_ascend/patch/__init__.py", - "vllm_ascend/patch/worker/patch_mamba_utils.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "730f1337b2341ed166b34f99aeb314efda2876b2", - "subject": "[Test][MRV2] Move SFA PCP graph accuracy test to four cards (#15264)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/scripts/test_config.yaml", - "tests/e2e/pull_request/eight_card/model_runner_v2/test_glm5_2.py", - "tests/e2e/pull_request/four_card/context_parallel/test_accuracy_v2.py" - ] - }, - { - "commit": "343743af94706b6b640478712f5235f8161b0e5e", - "subject": "[CI] update baseline for kimi-k2-thinking.yaml (#15537)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/configs/nightly_config.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/GLM-5.1-W8A8C8-A3_198k_function.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/GLM-5.1-W8A8C8-A3_198k_function_perf.yaml", - "tests/e2e/nightly/single_node/models/configs/Kimi-K2-Thinking.yaml" - ] - }, - { - "commit": "35e7c6d8bf7cd1065604e66f99ec16525e096beb", - "subject": "[Bugfix][MRV2] Fix full-graph param updates for mixed-attention models under PP (#15405)", - "topics": [ - "deepseek-v4-dspark", - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/models/minimax_m3/test_msa_m3.py", - "vllm_ascend/models/minimax_m3/msa_m3.py" - ] - }, - { - "commit": "beedc8b8db84d56c72a2d4d6a3c9de85a741f8e5", - "subject": "[Feature][BugFix][Spec Decode] Support DSpark with LMHead TP (#15495)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/models/deepseek_v4/dspark.py", - "vllm_ascend/spec_decode/llm_base_proposer.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "a563601e01175cbec9448142aebf4ae992842693", - "subject": "[main][Ops][Feature] Use mega_moe_max_tokens for CANN MegaMoe symmetric buffer allocation (#15014)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "docs/source/user_guide/configuration/additional_config.md", - "tests/ut/ops/test_moe_comm_method.py", - "tests/ut/test_ascend_config.py", - "tests/ut/test_ascend_forward_context.py", - "vllm_ascend/ascend_config.py", - "vllm_ascend/ascend_forward_context.py", - "vllm_ascend/ops/fused_moe/moe_comm_method.py" - ] - }, - { - "commit": "45e0634c1ba2ccbc400cc2167009a11b687d74ef", - "subject": "[Performance]Support IndexCache TopK reuse (#14914)", - "topics": [ - "deepseek-v4-dspark", - "communication" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/attention/context_parallel/dsa_cp.py" - ] - }, - { - "commit": "4e6fb74b23d3f66a5dc1b20219dce04aa66e6941", - "subject": "[Bugfix][Spec Decode] Disable AutoBlockify for rejection sampling (#15118)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "tests/e2e/pull_request/four_card/model_runner_v2/test_deepseek_v4.py", - "vllm_ascend/worker/v2/spec_decode/rejection_sampler_utils.py" - ] - }, - { - "commit": "89df31d7ed27658101d84ac0c3968163b2bee29a", - "subject": "[Feature] GLM5.2 rotation v2 Dspark support (#15210)", - "topics": [ - "deepseek-v4-dspark", - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "tests/e2e/coverage_taxonomy.py", - "tests/e2e/pull_request/eight_card/model_runner_v2/test_glm5_2.py", - "tests/ut/quantization/test_modelslim_config.py", - "tests/ut/spec_decode/test_dspark_speculator.py", - "vllm_ascend/models/deepseek_v4/dspark.py", - "vllm_ascend/models/kimi_k3.py", - "vllm_ascend/models/kimi_k3_dspark.py", - "vllm_ascend/models/llama_eagle3.py", - "vllm_ascend/models/qwen3_dspark.py", - "vllm_ascend/utils.py", - "vllm_ascend/worker/v2/spec_decode/dspark/speculator.py" - ] - }, - { - "commit": "390cf0ac89dc3150fe1bf34f165d9692839630f8", - "subject": "[Doc][Misc] Update template (#14880)", - "topics": [ - "deepseek-v4-dspark", - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "docs/hooks/nav_titles.py", - "docs/source/_templates/Model-Deployment-Tutorial-Template.md", - "docs/source/_templates/Model-Deployment-Tutorial-Template.zh.md", - "docs/source/_templates/template-supplement.md", - "docs/source/_templates/template-supplement.zh.md", - "docs/source/locale/zh_CN/LC_MESSAGES/faqs.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/DeepSeek-V3.1.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/DeepSeek-V4-Flash.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/DeepSeek-V4-Pro.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/DeepSeekOCR2.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Gemma4.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Kimi-K2.5.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Kimi-K2.6.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/MiniMax-M2.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/PaddleOCR-VL.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen-VL-Dense.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-235B-A22B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-30B-A3B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-Coder-30B-A3B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-Next.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-Omni-30B-A3B-Thinking.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3.6-35B-A3B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3.8-27B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3_embedding.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/index.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/support_matrix/feature_matrix.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/support_matrix/supported_features.po" - ] - }, - { - "commit": "189d207595e678ef43b62adf4e6ceb7e51567474", - "subject": "[BugFix][Platform] Fix SP switch configuration (#15392)", - "topics": [ - "communication", - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "tests/e2e/nightly/multi_node/internal_dp/config/GLM5_1-W8A8-A2-dual-nodes.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/Qwen3-235B-A22B-A2.yaml", - "vllm_ascend/platform.py" - ] - }, - { - "commit": "30f54b5c341f015bcc124022e62156d3e5ef5919", - "subject": "[BugFix][MoE] Fix MegaMoe prefill buffer sizing (#14439)", - "topics": [ - "deepseek-v4-dspark", - "communication", - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/ops/test_fused_moe.py", - "tests/ut/quantization/methods/test_w4a8.py", - "vllm_ascend/ascend_forward_context.py", - "vllm_ascend/ops/fused_moe/moe_comm_method.py", - "vllm_ascend/ops/fused_moe/routed_experts.py", - "vllm_ascend/quantization/methods/w4a8/w4a8.py", - "vllm_ascend/quantization/methods/w8a8/w8a8_dynamic.py", - "vllm_ascend/utils.py" - ] - }, - { - "commit": "0bf74af47577658782d81d0c18d2821b9abe66cc", - "subject": "[Doc] Update installation links in GLM5.3 tutorial (#15508)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/tutorials/models/GLM5.3.md" - ] - }, - { - "commit": "f69831343a1850a363f4892b1f5d5cfe8e4a051d", - "subject": "[BugFix] Fix DSpark draft attn_sink loading under DSA-CP with TP>1 (#15422)", - "topics": [ - "deepseek-v4-dspark", - "communication" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/models/deepseek_v4/dspark.py" - ] - }, - { - "commit": "8b72fdb97b2b0d8eccaf6ac7ac414a151e09810e", - "subject": "[Feature][MRV2] Support Eagle3 and DSpark with PP (#14822)", - "topics": [ - "deepseek-v4-dspark", - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/scripts/test_config.yaml", - "tests/e2e/pull_request/eight_card/model_runner_v2/test_spec_pp_accuracy.py", - "vllm_ascend/models/deepseek_v4/model.py", - "vllm_ascend/models/minimax_m3/minimax_m3.py", - "vllm_ascend/patch/platform/patch_pp_mtp.py", - "vllm_ascend/patch/platform/patch_use_v2_model_runner.py", - "vllm_ascend/patch/worker/patch_v2/patch_dspark.py", - "vllm_ascend/worker/v2/model_runner.py", - "vllm_ascend/worker/v2/pp_utils.py", - "vllm_ascend/worker/v2/spec_decode/autoregressive/speculator.py", - "vllm_ascend/worker/v2/spec_decode/eagle/speculator.py" - ] - }, - { - "commit": "6e518cdfd035151b1f00a3c2fa6ca05520e60d60", - "subject": "[Doc] Add Triton operator descriptions for l2norm, rope, rope_siso and min_p (#15135)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/ops/triton/docs/l2norm.md", - "vllm_ascend/ops/triton/docs/min_p.md", - "vllm_ascend/ops/triton/docs/rope.md", - "vllm_ascend/ops/triton/docs/rope_siso.md" - ] - }, - { - "commit": "0fcae3ddb7ecee6ea7c42c0df2a90441d220c992", - "subject": "[Doc] Add apply_penalties Triton operator description (#15087)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/ops/triton/docs/apply_penalties.md" - ] - }, - { - "commit": "173665ce9ee304925c822e692ec1342f77670881", - "subject": "[Feature][xlite] Add preliminary support for GLM-5.2 with shared DSA indexer in the xlite backend (#15137)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "requirements-dev.txt", - "vllm_ascend/xlite/utils.py", - "vllm_ascend/xlite/xlite.py", - "vllm_ascend/xlite/xlite_model_runner.py" - ] - }, - { - "commit": "8a4b8a2ebae4e84ad4e01fd433c90c990b011e9e", - "subject": "[CI][Test]Remove NPU from the ut test case. (#11594)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/scripts/test_config.yaml", - "tests/e2e/pull_request/one_card/attention_utils.py", - "tests/e2e/pull_request/one_card/test_attention_v1_precision.py", - "tests/e2e/pull_request/one_card/test_eplb_map_to_physical_npu.py", - "tests/e2e/pull_request/one_card/test_gdn_layerwise_kv.py", - "tests/e2e/pull_request/one_card/test_gumbel_sampling.py", - "tests/e2e/pull_request/one_card/test_mla_precision.py", - "tests/e2e/pull_request/one_card/test_model_runner_v1_with_device.py", - "tests/e2e/pull_request/one_card/test_sfa_v1_precision.py", - "tests/e2e/pull_request/one_card/test_w4a4_flatquant.py", - "tests/e2e/pull_request/one_card/test_w4a4_laos_dynamic.py", - "tests/e2e/pull_request/one_card/test_w8a16.py", - "tests/e2e/pull_request/one_card/test_w8a8_dynamic.py", - "tests/e2e/pull_request/one_card/test_w8a8_static.py", - "tests/e2e/pull_request/two_card/test_activation.py", - "tests/e2e/pull_request/two_card/test_kimi_kda_fused_norm_gate.py", - "tests/ut/attention/a2/__init__.py", - "tests/ut/attention/test_attention_cp.py", - "tests/ut/attention/test_attention_cp_precision.py", - "tests/ut/attention/test_attention_v1.py", - "tests/ut/attention/test_common_cp.py", - "tests/ut/attention/test_mla_cp.py", - "tests/ut/attention/test_mla_cp_precision.py", - "tests/ut/attention/test_mla_v1.py", - "tests/ut/attention/test_sfa_v1.py", - "tests/ut/compilation/a2/__init__.py", - "tests/ut/compilation/test_acl_graph.py", - "tests/ut/conftest.py", - "tests/ut/core/test_profiling_chunk.py", - "tests/ut/core/test_recompute_scheduler_short_request_first.py", - "tests/ut/device_allocator/a2/__init__.py", - "tests/ut/device_allocator/test_find_loaded_library.py", - "tests/ut/eplb/core/a2/__init__.py", - "tests/ut/eplb/core/expert_map.json", - "tests/ut/eplb/core/test_eplb_utils.py", - "tests/ut/helpers/golden_copy_and_expand.py", - "tests/ut/kv_offload/a2/__init__.py", - "tests/ut/kv_offload/test_mooncake_connector.py", - "tests/ut/kv_offload/test_mooncake_hybrid_connector.py", - "tests/ut/kv_offload/test_remote_decode_lifecycle.py", - "tests/ut/kv_offload/test_remote_prefill_lifecycle.py", - "tests/ut/kv_offload/utils.py", - "tests/ut/lora/test_quant_moe.py", - "tests/ut/ops/a2/__init__.py", - "tests/ut/ops/a3_2/__init__.py", - "tests/ut/ops/test_activation.py", - "tests/ut/ops/test_comm_utils.py", - "tests/ut/ops/test_gdn_chunk_meta.py", - "tests/ut/ops/test_moe_mlp.py", - "tests/ut/ops/test_token_dispatcher.py", - "tests/ut/profiler/test_torch_npu_profiler.py", - "tests/ut/quantization/methods/a2/__init__.py", - "tests/ut/quantization/methods/test_w4a16.py", - "tests/ut/quantization/test_modelslim_config.py", - "tests/ut/spec_decode/a2/__init__.py", - "tests/ut/spec_decode/test_eagle_proposer.py", - "tests/ut/test_platform.py", - "tests/ut/test_utils.py", - "tests/ut/worker/a2/__init__.py", - "tests/ut/worker/test_block_table.py", - "tests/ut/worker/test_model_runner_v1.py", - "tests/ut/worker/test_worker_multi_instance.py", - "tests/ut/worker/test_worker_v1.py", - "tests/ut/worker/test_worker_v2.py" - ] - }, - { - "commit": "2e31c89f11867e4b83815873d62f6f9389232c39", - "subject": "[CI] Add Rust tool parser to Docker images (#15156)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "Dockerfile", - "Dockerfile.310p", - "Dockerfile.310p.openEuler", - "Dockerfile.a3", - "Dockerfile.a3.openEuler", - "Dockerfile.a5", - "Dockerfile.a5.openEuler", - "Dockerfile.openEuler" - ] - }, - { - "commit": "cb0d60667fe592f870d6d9e36a12c3767dabe884", - "subject": "[Doc] Update doc for GLM-5.3 (#15375)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "docs/hooks/nav_titles.py", - "docs/source/tutorials/models/GLM5.3.md", - "docs/source/user_guide/support_matrix/supported_models.md", - "mkdocs.yml" - ] - }, - { - "commit": "7388730ffedc12e388223873d18d5e3c0acfd016", - "subject": "Revert \"[BugFix] Chunk wq_b matmul for NPU 65536 dimension limit (#9780)\" (#14459)", - "topics": [ - "communication" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/attention/context_parallel/dsa_cp.py", - "vllm_ascend/quantization/methods/w8a8/w8a8_dynamic.py" - ] - }, - { - "commit": "1f3418a1a7fbede99f78d95f12b4cff1bd58a805", - "subject": "[Performance][Spec Decode] Shard DeepSeek V4 DSpark main projection across TP ranks (#15123)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/models/deepseek_v4/dspark.py" - ] - }, - { - "commit": "d4ebe8a0e0fdf56135507259f90f80a45f736cab", - "subject": "[Refactor][Device][6/N] Migrate model execution to hardware profiles (#15407)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/attention/test_dsa_attn_kv_plan.py", - "tests/ut/device/test_hardware_profile.py", - "tests/ut/worker/a2/test_model_runner_v1.py", - "tests/ut/worker/test_attn_utils_v2.py", - "vllm_ascend/attention/dsa_attn_kv_plan.py", - "vllm_ascend/device/hardware_profile.py", - "vllm_ascend/models/deepseek_v4/compressor.py", - "vllm_ascend/models/deepseek_v4/indexer.py", - "vllm_ascend/models/layer/attention/layer.py", - "vllm_ascend/ops/dsa.py", - "vllm_ascend/worker/model_runner_v1.py", - "vllm_ascend/worker/v2/attn_utils.py" - ] - }, - { - "commit": "583aee2800142494ebc1907d4362222b0bb108df", - "subject": "[misc] _MEGA_MOE_SUPPORTED lazy init (#15448)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/ascend_config.py", - "vllm_ascend/ops/fused_moe/moe_comm_method.py" - ] - }, - { - "commit": "c2806729033d98c53c2a782fefa424791e466c5f", - "subject": "[Feature][Model] Support GLM-5.3 native FP8 checkpoints on Ascend 950 (#15346)", - "topics": [ - "priority-models", - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/attention/a2/test_sfa_v1.py", - "tests/ut/quantization/methods/test_fp8_block.py", - "tests/ut/quantization/test_method_adapters.py", - "tests/ut/spec_decode/test_llm_base_proposer.py", - "vllm_ascend/attention/sfa_v1.py", - "vllm_ascend/distributed/kv_transfer/sparse_kv_offload/sparse_kv_offload_manager.py", - "vllm_ascend/models/deepseek_mtp.py", - "vllm_ascend/quantization/method_adapters.py", - "vllm_ascend/quantization/methods/w8a8/fp8_block.py", - "vllm_ascend/spec_decode/llm_base_proposer.py" - ] - }, - { - "commit": "8db29fde7ec642061230048db696ce74106d2142", - "subject": "[Feature][Model] Use SparseFlashMla for DSV4 A5 BF16 KV (#13716)", - "topics": [ - "deepseek-v4-dspark", - "communication", - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/workflows/scripts/test_config.yaml", - "tests/ut/attention/test_dsa_attn_kv_plan.py", - "tests/ut/attention/test_dsa_v1.py", - "tests/ut/attention/test_dsv4_block_geometry.py", - "tests/ut/attention/test_sparse_flash_mla.py", - "tests/ut/conftest.py", - "tests/ut/models/test_deepseek_v4_compressor.py", - "tests/ut/test_ascend_forward_context.py", - "vllm_ascend/ascend_forward_context.py", - "vllm_ascend/attention/context_parallel/dsa_cp.py", - "vllm_ascend/attention/dsa_attn_kv_plan.py", - "vllm_ascend/attention/dsa_v1.py", - "vllm_ascend/attention/sparse_flash_mla.py", - "vllm_ascend/device/device_op.py", - "vllm_ascend/models/deepseek_v4/compressor.py", - "vllm_ascend/models/deepseek_v4/dspark.py", - "vllm_ascend/models/deepseek_v4/indexer.py", - "vllm_ascend/models/deepseek_v4/model.py", - "vllm_ascend/models/layer/attention/layer.py", - "vllm_ascend/ops/linear.py", - "vllm_ascend/platform.py" - ] - }, - { - "commit": "dac12ac39881cdaf8fc45aacda71197fca85d802", - "subject": "[main][Attention][Feature] Rewrite SFA DCP sparse index remap with Triton (#15197)", - "topics": [ - "priority-models", - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_sparse_index_remap_triton.py", - "vllm_ascend/attention/context_parallel/sfa_cp.py", - "vllm_ascend/ops/triton/docs/sparse_index_remap.md", - "vllm_ascend/ops/triton/sparse_index_remap.py" - ] - }, - { - "commit": "1a69fd913e625f84af4c243d3ab103b95f29618b", - "subject": "[main][bugfix] skip generate_mtp_attention_mask_for_decode with sparse attention to avoid hostbound (#15275)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/worker/dcp_utils.py" - ] - }, - { - "commit": "63b0a64f307e75b406379ca8f15167649c8eb78c", - "subject": "[Spec Decode] [Doc] Support speculative decode KV slidng window for Dspark (#14596)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/developer_guide/Design_Documents/speculative_kv_sliding_window.md", - "docs/source/user_guide/feature_guide/speculative_decoding.md", - "tests/e2e/pull_request/one_card/spec_decode/test_dspark.py", - "tests/e2e/pull_request/one_card/spec_decode/utils.py", - "vllm_ascend/spec_decode/llm_base_proposer.py", - "vllm_ascend/spec_decode/utils.py" - ] - }, - { - "commit": "a8501e3d21d84e1cbf9087ab5a848531b9bdb0ae", - "subject": "[Refactor][Device][5/N] Migrate attention and quantization to hardware profiles (#15376)", - "topics": [ - "deepseek-v4-dspark", - "communication" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/attention/a2/test_attention_v1.py", - "tests/ut/attention/a2/test_mla_v1.py", - "tests/ut/device/test_hardware_profile.py", - "tests/ut/quantization/test_utils.py", - "tests/ut/worker/test_attn_utils_v2.py", - "vllm_ascend/attention/attention_v1.py", - "vllm_ascend/attention/context_parallel/dsa_cp.py", - "vllm_ascend/attention/dsa_v1.py", - "vllm_ascend/attention/mla_v1.py", - "vllm_ascend/attention/sfa_v1.py", - "vllm_ascend/attention/utils.py", - "vllm_ascend/device/hardware_profile.py", - "vllm_ascend/models/deepseek_v4/indexer.py", - "vllm_ascend/quantization/configs/modelslim_config.py", - "vllm_ascend/quantization/methods/kv_cache/kv_c8.py", - "vllm_ascend/quantization/utils.py" - ] - }, - { - "commit": "cde915d2b56edb6483cef7fd77b9bfbeb03cb059", - "subject": "[BugFix][MRV2] Isolate Eagle draft parallel config (#15354)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/worker/v2/spec_decode/eagle/speculator.py" - ] - }, - { - "commit": "8f445db7c99d9620c32981cbb0f7bbd785c83ff4", - "subject": "[CI] Recitify GLM-5.2 and DeepSeek-V4 spec decode E2E (#13960)", - "topics": [ - "deepseek-v4-dspark", - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "tests/e2e/pull_request/eight_card/test_glm5_2.py", - "tests/e2e/pull_request/four_card/spec_decode/test_dspark_deepseekv4.py", - "tests/e2e/pull_request/utils.py", - "vllm_ascend/models/__init__.py" - ] - }, - { - "commit": "0b8cd0aeb01c25a4896ff78caddc834361a9af20", - "subject": "[Doc][Getting Started] Rework Quick Start and Installation guides (#15309)", - "topics": [ - "deepseek-v4-dspark", - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/scripts/detect_po_changes.py", - ".github/workflows/scripts/po_translate.py", - "docs/Makefile", - "docs/README.md", - "docs/hooks/macros_main.py", - "docs/hooks/nav_titles.py", - "docs/overrides/main.html", - "docs/source/_templates/Model-Deployment-Tutorial-Template.md", - "docs/source/_templates/Model-Deployment-Tutorial-Template.zh.md", - "docs/source/assets/hardware-software-stack-overview.png", - "docs/source/community/versioning_policy.md", - "docs/source/conf.py", - "docs/source/developer_guide/performance_and_debug/performance_benchmark.md", - "docs/source/getting_started.md", - "docs/source/getting_started/installation.md", - "docs/source/getting_started/installation/base_environment.inc.md", - "docs/source/getting_started/installation/cann_image/atlas-200i-pro.inc.md", - "docs/source/getting_started/installation/cann_image/atlas-300i-duo.inc.md", - "docs/source/getting_started/installation/cann_image/atlas-950dt.inc.md", - "docs/source/getting_started/installation/cann_image/atlas-a2.inc.md", - "docs/source/getting_started/installation/cann_image/atlas-a3.inc.md", - "docs/source/getting_started/installation/existing_cann_environment.inc.md", - "docs/source/getting_started/installation/install_vllm_ascend.inc.md", - "docs/source/getting_started/installation/prebuilt_image.inc.md", - "docs/source/getting_started/installation/supported_hardware.inc.md", - "docs/source/getting_started/overview.md", - "docs/source/getting_started/quick_start.md", - "docs/source/getting_started/quick_start/ascend_image/atlas-200i-pro.inc.md", - "docs/source/getting_started/quick_start/ascend_image/atlas-300i-duo.inc.md", - "docs/source/getting_started/quick_start/ascend_image/atlas-950dt.inc.md", - "docs/source/getting_started/quick_start/ascend_image/atlas-a2.inc.md", - "docs/source/getting_started/quick_start/ascend_image/atlas-a3.inc.md", - "docs/source/getting_started/quick_start/ascend_image/image_download_mirror.inc.md", - "docs/source/getting_started/quick_start/ascend_image/verify_container.inc.md", - "docs/source/getting_started/quick_start/offline/qwen3-0.6b-310p.inc.md", - "docs/source/getting_started/quick_start/offline/qwen3-0.6b.inc.md", - "docs/source/getting_started/quick_start/online/qwen3-0.6b-310p.inc.md", - "docs/source/getting_started/quick_start/online/qwen3-0.6b.inc.md", - "docs/source/installation.md", - "docs/source/javascripts/tabbed-toc.js", - "docs/source/quick_start.md", - "docs/source/stylesheets/extra.css", - "docs/source/tutorials/models/Cohere-Transcribe.md", - "docs/source/tutorials/models/DeepSeek-R1.md", - "docs/source/tutorials/models/DeepSeek-V3.1.md", - "docs/source/tutorials/models/DeepSeek-V3.2.md", - "docs/source/tutorials/models/DeepSeek-V4-Flash.md", - "docs/source/tutorials/models/DeepSeek-V4-Pro.md", - "docs/source/tutorials/models/DeepSeekOCR2.md", - "docs/source/tutorials/models/GLM4.x.md", - "docs/source/tutorials/models/GLM5.2.md", - "docs/source/tutorials/models/GLM5.md", - "docs/source/tutorials/models/Gemma4.md", - "docs/source/tutorials/models/Hy3-preview.md", - "docs/source/tutorials/models/Hy4-preview.md", - "docs/source/tutorials/models/InternVL3.5.md", - "docs/source/tutorials/models/Kimi-K2-Thinking.md", - "docs/source/tutorials/models/Kimi-K2.5.md", - "docs/source/tutorials/models/Kimi-K2.6.md", - "docs/source/tutorials/models/Kimi-K3.md", - "docs/source/tutorials/models/LLaVA-OneVision-Qwen2-0.5B-OV.md", - "docs/source/tutorials/models/MiniMax-M2.md", - "docs/source/tutorials/models/MiniMax-M3.md", - "docs/source/tutorials/models/Minitron-8B-Base.md", - "docs/source/tutorials/models/Mixtral-8x7B-Instruct-v0.1.md", - "docs/source/tutorials/models/PaddleOCR-VL.md", - "docs/source/tutorials/models/Qwen-VL-Dense.md", - "docs/source/tutorials/models/Qwen2.5-Math-RM-72B.md", - "docs/source/tutorials/models/Qwen3-235B-A22B.md", - "docs/source/tutorials/models/Qwen3-30B-A3B.md", - "docs/source/tutorials/models/Qwen3-ASR-1.7B.md", - "docs/source/tutorials/models/Qwen3-Coder-30B-A3B.md", - "docs/source/tutorials/models/Qwen3-Dense.md", - "docs/source/tutorials/models/Qwen3-Embedding.md", - "docs/source/tutorials/models/Qwen3-Next.md", - "docs/source/tutorials/models/Qwen3-Reranker.md", - "docs/source/tutorials/models/Qwen3-VL-235B-A22B-Instruct.md", - "docs/source/tutorials/models/Qwen3-VL-30B-A3B-Instruct.md", - "docs/source/tutorials/models/Qwen3-VL-Embedding.md", - "docs/source/tutorials/models/Qwen3-VL-Reranker.md", - "docs/source/tutorials/models/Qwen3.5-27B-Qwen3.6-27B.md", - "docs/source/tutorials/models/Qwen3.5-397B-A17B.md", - "docs/source/tutorials/models/Qwen3.5-Dense.md", - "docs/source/tutorials/models/Qwen3.6-35B-A3B.md", - "docs/source/tutorials/models/Qwen3.8-2.4T-A95B.md", - "docs/source/tutorials/models/Qwen3.8-27B.md", - "docs/source/tutorials/models/gpt-oss-120b.md", - "docs/source/user_guide/feature_guide/batch_invariance.md", - "mkdocs.yml", - "tests/e2e/doctests/001-quickstart-test.sh", - "tests/e2e/doctests/002-pip-binary-installation-test.sh", - "tests/ut/_tools/test_generate_zh_docs.py", - "tests/ut/test_dependency_documentation.py", - "tools/generate_zh_docs.py", - "tools/rtd_build.sh" - ] - }, - { - "commit": "186281cecb79d0c94f6970e8571868a978fc4150", - "subject": "[Doc][Misc] Add docs for matmul_bias_persistent, linear_persistent, mean and _rms_norm operators (#15230)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/ops/triton/docs/linear_persistent.md", - "vllm_ascend/ops/triton/docs/matmul_bias_persistent.md", - "vllm_ascend/ops/triton/docs/mean.md", - "vllm_ascend/ops/triton/docs/rms_norm.md" - ] - }, - { - "commit": "813f7374bb2a0fa88bdd50c3d726a37e3614bbd7", - "subject": "[Bugfix][Config][Test] Fix ineffective megamoe rollback from stale _MEGA_MOE_SUPPORTED import (#15303)", - "topics": [ - "communication", - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/ops/test_fused_moe.py", - "tests/ut/test_ascend_config.py", - "vllm_ascend/ascend_config.py", - "vllm_ascend/ascend_forward_context.py", - "vllm_ascend/ops/fused_moe/moe_comm_method.py", - "vllm_ascend/ops/fused_moe/routed_experts.py", - "vllm_ascend/quantization/methods/w4a8/w4a8.py", - "vllm_ascend/quantization/methods/w8a8/w8a8_dynamic.py" - ] - }, - { - "commit": "843dd09a9b315c21789d5d2663a7d15b2a0dca77", - "subject": "[Refactor][Delete][MRV2]Patch upstream mrv2 apply_top_k_top_p with current triton implementation (#14887)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_apply_top_k_top_p_triton.py", - "vllm_ascend/ops/triton/v2/sample/apply_top_k_top_p_triton.py", - "vllm_ascend/ops/triton/v2/sample/docs/apply_top_k_top_p.md", - "vllm_ascend/patch/worker/patch_v2/patch_triton.py", - "vllm_ascend/worker/v2/sample/apply_top_k_top_p.py" - ] - }, - { - "commit": "f8f130809358e2e4451a0b9ed6ec82902f8d50a0", - "subject": "[Test][Misc] Move 2 size cases to double_node (#15141)", - "topics": [ - "deepseek-v4-dspark", - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/configs/weekly_config.yaml", - ".github/workflows/schedule_weekly_test_a3.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/DeepSeek-V4-Pro-w4a8-PD-no-prefix.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/GLM-5.2-W8A8C8-A3-Memcache-PD.yaml" - ] - }, - { - "commit": "4b25eb34793efa9f00201d654eb2f107ecebfbbd", - "subject": "[bugfix] roll back megamoe op (#15267)", - "topics": [ - "communication", - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/ascend_config.py" - ] - }, - { - "commit": "5c59cfba98045f49645284745be035346a4d82d8", - "subject": "[Performance](attention): fuse SFA DCP output all-to-all (#14457)", - "topics": [ - "communication", - "priority-models", - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/workflows/scripts/test_config.yaml", - "tests/e2e/nightly/single_node/ops/multicard_ops_a3/test_sfa_cp_a2a.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_sfa_cp_a2a.py", - "tests/ut/attention/a2/test_sfa_cp_precision.py", - "tests/ut/attention/test_sfa_cp_precision.py", - "vllm_ascend/attention/context_parallel/sfa_cp.py", - "vllm_ascend/ops/triton/docs/sfa_cp_a2a_fused.md", - "vllm_ascend/ops/triton/sfa_cp.py" - ] - }, - { - "commit": "bb6d5b82ecb97da8fa0f8fa1905af4a717c642a7", - "subject": "[Misc] remove compatibility with dispatch_v1/combine_v1 interface (#15157)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/ops/a2/test_token_dispatcher.py", - "vllm_ascend/ops/fused_moe/token_dispatcher.py" - ] - }, - { - "commit": "6588370e7457098bff6f736582667906e65b2326", - "subject": "[Feature][EPLB] Support EPLB for MiniMax-M3 (#14973)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/models/minimax_m3/test_minimax_m3.py", - "tests/ut/models/minimax_m3/test_minimax_m3_eplb.py", - "tests/ut/models/minimax_m3/test_minimax_m3_vit.py", - "vllm_ascend/models/minimax_m3/minimax_m3.py", - "vllm_ascend/models/minimax_m3/minimax_m3_vl.py" - ] - }, - { - "commit": "4b2828cad35acc2814bd9b1ac502d7c50a019350", - "subject": "[Misc] Update Ascendconfig docstring (#15058)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/ascend_config.py" - ] - }, - { - "commit": "e41435157d3a80937bb01f60efb46b28d663f2d5", - "subject": "[BugFix] Release unused unquantized weights for fused MC2 (#15216)", - "topics": [ - "communication", - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/ops/fused_moe/routed_experts.py" - ] - }, - { - "commit": "895ca078b2ce52e4ec40287776233e693aa592d7", - "subject": "[Refactor][Config] Remove MLAPO environment variable (#14829)", - "topics": [ - "deepseek-v4-dspark", - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/tutorials/models/DeepSeek-V3.1.md", - "docs/source/tutorials/models/DeepSeek-V3.2.md", - "docs/source/tutorials/models/GLM5.2.md", - "docs/source/tutorials/models/GLM5.md", - "docs/source/tutorials/models/Kimi-K2.5.md", - "docs/source/tutorials/models/Kimi-K2.6.md", - "docs/source/tutorials/models/Mixtral-8x7B-Instruct-v0.1.md", - "docs/source/user_guide/configuration/additional_config.md", - "tests/e2e/models/configs/Mixtral-8x7B-Instruct-v0.1.yaml", - "tests/e2e/nightly/multi_node/external_dp/config/GLM-5.1-W8A8C8-128k-1k-90-50-PD.yaml", - "tests/e2e/nightly/multi_node/external_dp/config/GLM5_1-W8A8-EP-external.yaml", - "tests/e2e/nightly/multi_node/external_dp/config/Kimi-K2.6-W4A8-64k-1k-TPOT50-PD.yaml", - "tests/e2e/nightly/multi_node/external_dp/scripts/utils.py", - "tests/e2e/nightly/multi_node/internal_dp/config/DeepSeek-V3.1-BF16.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/DeepSeek-V3_2-W8A8-A3-dual-nodes.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/DeepSeek-V3_2-W8A8-EP.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/GLM-5.1-W8A8C8-A3_128k_90_50.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/GLM-5.1-W8A8C8-A3_198k_function.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/GLM5_1-W8A8-A2-dual-nodes.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/GLM5_1-W8A8-A3-dual-nodes.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/GLM5_1-W8A8-EP.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/GLM5_2-W8A8-A3-dual-nodes.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/Kimi-K2_5-W4A8-A2-dual-nodes.yaml", - "tests/e2e/nightly/multi_node/internal_dp/scripts/test_multi_node.py", - "tests/e2e/nightly/single_node/models/configs/DeepSeek-V3.2-W8A8-DCP.yaml", - "tests/e2e/nightly/single_node/models/configs/DeepSeek-V3.2-W8A8.yaml", - "tests/e2e/nightly/single_node/models/configs/GLM-5.1-W8A8-PrefillMC2.yaml", - "tests/e2e/nightly/single_node/models/configs/Kimi-K2.5.yaml", - "tests/e2e/nightly/single_node/models/configs/Kimi-K2.6-w4a8-A3.yaml", - "tests/e2e/nightly/single_node/models/scripts/test_single_node.py", - "tests/e2e/weekly/multi_node/external_dp/config/DeepSeek-V4-Flash-Memcache-PD-acc.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/DeepSeek_V3.1T_MTP1_128K_1K_PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/DeepSeek_V3.1T_MTP1_3_5K_1_5K_PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/DeepSeek_V3.1T_MTP1_PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/DeepSeek_V3.1T_MTP3_PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/DeepSeek_V3.1T_layerwise_PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/DeepSeek_V3.2T_MTP2_PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/DeepSeek_V3.2T_MTP3_PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/GLM-5.1-W8A8C8-3.5k-1.5k-0-20-PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/GLM-5.1-W8A8C8-3.5k-1.5k-0-50-PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/GLM-5.1-W8A8C8-64k-1k-90-50-PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/GLM-5.2-W8A8C8-128k-1k-90-50-PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/GLM-5.2-W8A8C8-3.5k-1.5k-0-20-PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/GLM-5.2-W8A8C8-3.5k-1.5k-0-50-PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/GLM-5.2-W8A8C8-64k-1k-90-50-PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/GLM_5_1_PD_in32k_bs16-0.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/GLM_5_1_PD_in32k_bs20-90.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/GLM_5_1_PD_in64k_bs16-90.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/Kimi-K2.5-W4A8-128k-1k-TPOT50.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/Kimi-K2.5-W4A8-16k-1k-TPOT50.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/Kimi-K2.6-W4A8-128k-1k-TPOT50-PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/Kimi-K2.6-W4A8-3.5k-1.5k-TPOT50-PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/Kimi-K2.7-W4A8-128k-1k-TPOT50-PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/Kimi-K2.7-W4A8-3.5k-1.5k-TPOT50-PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/Kimi-K2.7-W4A8-64k-1k-TPOT50-PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/QWEN3_235B_PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/QWEN3_235B_PD_3_5K_1_5k.yaml", - "tests/e2e/weekly/multi_node/internal_dp/config/DeepSeek-V3_2-W8A8-EP_weekly.yaml", - "tests/e2e/weekly/multi_node/internal_dp/config/GLM-5.1-W8A8C8-A3_3.5k_1.5k_0_20.yaml", - "tests/e2e/weekly/multi_node/internal_dp/config/GLM-5.1-W8A8C8-A3_3.5k_1.5k_0_50.yaml", - "tests/e2e/weekly/multi_node/internal_dp/config/GLM-5.1-W8A8C8-A3_64k_90_50.yaml", - "tests/e2e/weekly/multi_node/internal_dp/config/GLM-5.2-W8A8C8-A3_128k_90_50.yaml", - "tests/e2e/weekly/multi_node/internal_dp/config/GLM-5.2-W8A8C8-A3_198k_function.yaml", - "tests/e2e/weekly/multi_node/internal_dp/config/GLM-5.2-W8A8C8-A3_3.5k_1.5k_0_20.yaml", - "tests/e2e/weekly/multi_node/internal_dp/config/GLM-5.2-W8A8C8-A3_3.5k_1.5k_0_50.yaml", - "tests/e2e/weekly/multi_node/internal_dp/config/GLM-5.2-W8A8C8-A3_64k_90_50.yaml", - "tests/e2e/weekly/multi_node/internal_dp/config/Kimi-K2.6-W4A8-254k-1k-TPOT20-PD.yaml", - "tests/e2e/weekly/multi_node/internal_dp/config/Kimi-K2.7-W4A8-254k-1k-TPOT20-PD.yaml", - "tests/e2e/weekly/single_node/configs/DeepSeek-V3.2-W8A8_A3_weekly.yaml", - "tests/e2e/weekly/single_node/configs/GLM-5_1-W8A8_A3_weekly.yaml", - "tests/e2e/weekly/single_node/configs/Kimi-K2.6-w4a8-A3.yaml", - "tests/e2e/weekly/single_node/configs/Kimi-K2.7-w4a8-A3.yaml", - "tests/ut/test_ascend_config.py", - "vllm_ascend/ascend_config.py", - "vllm_ascend/envs.py" - ] - }, - { - "commit": "73acadef668c4346558ae7061236742256cc8331", - "subject": "[Refactor][Quantization] Reorganize quantization module: group configs and methods into subdirectories (#14030)", - "topics": [ - "deepseek-v4-dspark", - "communication", - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/_310p/quantization/test_modelslim_config_310.py", - "tests/ut/attention/a2/test_sfa_v1_precision.py", - "tests/ut/quantization/methods/a2/test_w4a16.py", - "tests/ut/quantization/methods/a2/test_w4a4_flatquant.py", - "tests/ut/quantization/methods/a2/test_w4a4_laos_dynamic.py", - "tests/ut/quantization/methods/a2/test_w8a16.py", - "tests/ut/quantization/methods/a2/test_w8a8_dynamic.py", - "tests/ut/quantization/methods/a2/test_w8a8_static.py", - "tests/ut/quantization/methods/test_fp8_block.py", - "tests/ut/quantization/methods/test_kv_c8.py", - "tests/ut/quantization/methods/test_moe_logical_experts.py", - "tests/ut/quantization/methods/test_w4a16_mxfp4.py", - "tests/ut/quantization/methods/test_w4a4_mxfp4.py", - "tests/ut/quantization/methods/test_w4a4_mxfp4_flatquant_dynamic.py", - "tests/ut/quantization/methods/test_w4a8.py", - "tests/ut/quantization/methods/test_w4a8_mxfp4.py", - "tests/ut/quantization/methods/test_w8a8_mxfp8.py", - "tests/ut/quantization/methods/test_w8a8_pdmix.py", - "tests/ut/quantization/methods/test_w8a8fp8_dynamic.py", - "tests/ut/quantization/test_compressed_tensors_config.py", - "tests/ut/quantization/test_fp8_config.py", - "tests/ut/quantization/test_modelopt_mxfp8_config.py", - "tests/ut/quantization/test_modelslim_config.py", - "tests/ut/quantization/test_utils.py", - "tests/ut/test_platform.py", - "vllm_ascend/_310p/quantization/modelslim_config.py", - "vllm_ascend/attention/context_parallel/dsa_cp.py", - "vllm_ascend/attention/dsa_v1.py", - "vllm_ascend/attention/mla_v1.py", - "vllm_ascend/models/deepseek_v4/indexer.py", - "vllm_ascend/ops/cv_linear.py", - "vllm_ascend/patch/platform/patch_fused_moe.py", - "vllm_ascend/quantization/__init__.py", - "vllm_ascend/quantization/configs/__init__.py", - "vllm_ascend/quantization/configs/compressed_tensors_config.py", - "vllm_ascend/quantization/configs/fp8_config.py", - "vllm_ascend/quantization/configs/modelopt_mxfp8_config.py", - "vllm_ascend/quantization/configs/modelslim_config.py", - "vllm_ascend/quantization/methods/__init__.py", - "vllm_ascend/quantization/methods/fp8.py", - "vllm_ascend/quantization/methods/kv_cache/__init__.py", - "vllm_ascend/quantization/methods/kv_cache/kv_c8.py", - "vllm_ascend/quantization/methods/w4a4/__init__.py", - "vllm_ascend/quantization/methods/w4a4/w4a4_flatquant.py", - "vllm_ascend/quantization/methods/w4a4/w4a4_laos_dynamic.py", - "vllm_ascend/quantization/methods/w4a4/w4a4_mxfp4.py", - "vllm_ascend/quantization/methods/w4a4/w4a4_mxfp4_flatquant.py", - "vllm_ascend/quantization/methods/w4a8/__init__.py", - "vllm_ascend/quantization/methods/w4a8/w4a8.py", - "vllm_ascend/quantization/methods/w4a8/w4a8_mxfp4.py", - "vllm_ascend/quantization/methods/w8a8/__init__.py", - "vllm_ascend/quantization/methods/w8a8/fp8_block.py", - "vllm_ascend/quantization/methods/w8a8/w8a8_dynamic.py", - "vllm_ascend/quantization/methods/w8a8/w8a8_mxfp8.py", - "vllm_ascend/quantization/methods/w8a8/w8a8_pdmix.py", - "vllm_ascend/quantization/methods/w8a8/w8a8_static.py", - "vllm_ascend/quantization/methods/w8a8/w8a8fp8_dynamic.py", - "vllm_ascend/quantization/methods/wna16/__init__.py", - "vllm_ascend/quantization/methods/wna16/w4a16.py", - "vllm_ascend/quantization/methods/wna16/w4a16_mxfp4.py", - "vllm_ascend/quantization/methods/wna16/w8a16.py", - "vllm_ascend/quantization/utils.py" - ] - }, - { - "commit": "be95ca98d044f44e465d159117fc9e6425a13b9d", - "subject": "[Refactor][Device][3/N] Migrate core runtime selection to hardware profiles (#14076)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/_310p/ops/test_mm_encoder_attention_310.py", - "tests/ut/_310p/quantization/test_w8a8_dynamic_310.py", - "tests/ut/_310p/quantization/test_w8a8_static_310.py", - "tests/ut/device/test_hardware_profile.py", - "tests/ut/device_allocator/test_cpu_binding.py", - "tests/ut/lora/test_punica_npu.py", - "tests/ut/test_ascend_config.py", - "tests/ut/test_platform.py", - "tests/ut/test_utils.py", - "vllm_ascend/ascend_config.py", - "vllm_ascend/cpu_binding.py", - "vllm_ascend/device/__init__.py", - "vllm_ascend/device/device_op.py", - "vllm_ascend/device/hardware_profile.py", - "vllm_ascend/lora/punica_npu.py", - "vllm_ascend/meta_registration.py", - "vllm_ascend/ops/linear.py", - "vllm_ascend/platform.py", - "vllm_ascend/utils.py" - ] - }, - { - "commit": "d37529f4df49e9da7a50123f4f810e4657e57013", - "subject": "[Feature][EPLB] Add asynchronous EPLB support for Model Runner V2 (#14099)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/workflows/scripts/test_config.yaml", - "docs/hooks/nav_titles.py", - "docs/source/developer_guide/Design_Documents/model_runner_v1_eplb.md", - "docs/source/developer_guide/Design_Documents/model_runner_v2_eplb.md", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/Design_Documents/model_runner_v1_eplb.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/Design_Documents/model_runner_v2_eplb.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/configuration/additional_config.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/expert_parallelism_load_balancer.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/support_matrix/supported_features.po", - "docs/source/user_guide/configuration/additional_config.md", - "docs/source/user_guide/feature_guide/expert_parallelism_load_balancer.md", - "docs/source/user_guide/support_matrix/supported_features.md", - "mkdocs.yml", - "tests/e2e/pull_request/four_card/test_qwen3_mrv2_eplb.py", - "tests/ut/distributed/eplb/__init__.py", - "tests/ut/distributed/eplb/test_communicator.py", - "tests/ut/distributed/eplb/test_state.py", - "tests/ut/distributed/test_eplb_communicator.py", - "tests/ut/ops/a2/test_eplb.py", - "tests/ut/ops/test_eplb.py", - "tests/ut/ops/test_fused_moe.py", - "tests/ut/patch/platform/test_patch_eplb.py", - "tests/ut/patch/platform/test_patch_fused_moe.py", - "tests/ut/test_platform.py", - "tests/ut/worker/v2/test_eplb_load_collection_phase.py", - "vllm_ascend/distributed/eplb/__init__.py", - "vllm_ascend/distributed/eplb/communicator.py", - "vllm_ascend/distributed/eplb/state.py", - "vllm_ascend/distributed/eplb_communicator.py", - "vllm_ascend/ops/fused_moe/eplb.py", - "vllm_ascend/ops/fused_moe/routed_experts.py", - "vllm_ascend/ops/triton/eplb.py", - "vllm_ascend/patch/platform/patch_eplb.py", - "vllm_ascend/patch/platform/patch_fused_moe.py", - "vllm_ascend/platform.py", - "vllm_ascend/worker/v2/eplb.py" - ] - }, - { - "commit": "e98b3757c36e9384d5c621edbfbd5df5056de96e", - "subject": "[Doc][Misc] Align Kimi-K3 tutorial with the deployment template (#15186)", - "topics": [ - "deepseek-v4-dspark", - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "docs/hooks/nav_titles.py", - "docs/source/tutorials/models/Kimi-K3.md", - "docs/source/tutorials/models/index.md", - "docs/source/user_guide/support_matrix/supported_models.md", - "mkdocs.yml" - ] - }, - { - "commit": "81057d3b731395fc07cbe99ff377c1bacc81dff2", - "subject": "[Doc][Misc] Add docs for fused_gdn_gating, postprocess_mamba and swiglustep operators (#15080)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/ops/triton/docs/fused_gdn_gating.md", - "vllm_ascend/ops/triton/docs/postprocess_mamba.md", - "vllm_ascend/ops/triton/docs/swiglustep.md" - ] - }, - { - "commit": "9cb118ee31dbb0ff5ed71b075579dc97c1c68e35", - "subject": "[CI][Model] Stabilize MiniMax-M3-W8A8-A3 nightly validation (#15071)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "tests/e2e/nightly/single_node/models/configs/MiniMax-M3-W8A8-A3.yaml" - ] - }, - { - "commit": "0c4d84e7f6cfcd4b26fc39608cb6d289019d7a29", - "subject": "[Refactor][Config] Remove weight NZ environment variable (#14830)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/tutorials/models/DeepSeekOCR2.md", - "docs/source/tutorials/models/GLM5.2.md", - "docs/source/user_guide/configuration/additional_config.md", - "docs/source/user_guide/feature_guide/rl.md", - "docs/source/user_guide/feature_guide/sleep_mode.md", - "examples/rl/rlhf_async_new_apis.py", - "tests/e2e/nightly/single_node/models/configs/DeepSeek-V3.2-W8A8-DCP.yaml", - "tests/e2e/nightly/single_node/models/configs/Kimi-K2.5.yaml", - "tests/e2e/nightly/single_node/models/configs/MiniMax-M2.5-w8a8-QuaRot-A2.yaml", - "tests/e2e/nightly/single_node/models/configs/Qwen3-VL-235B-A22B-Instruct-W8A8.yaml", - "tests/e2e/nightly/single_node/models/configs/Qwen3.5-27B-w8a8-A2.yaml", - "tests/e2e/pull_request/one_card/rlhf/conftest.py", - "tests/e2e/pull_request/one_card/rlhf/state_transitions/test_sleep_wake.py", - "tests/e2e/pull_request/one_card/test_npu_ipc_weight_transfer.py", - "tests/e2e/pull_request/one_card/test_xlite.py", - "tests/e2e/pull_request/two_card/test_hccl_weight_transfer.py", - "tests/e2e/pull_request/two_card/test_xlite.py", - "tests/e2e/weekly/multi_node/external_dp/config/GLM-5.2-W4A8C8-1M-PD-DCP.yaml", - "tests/e2e/weekly/single_node/models/test_qwen3_30b_acc.py", - "tests/ut/test_ascend_config.py", - "tests/ut/worker/a2/test_worker_v1.py", - "vllm_ascend/ascend_config.py", - "vllm_ascend/envs.py", - "vllm_ascend/utils.py" - ] - }, - { - "commit": "cfea50dde154800657197740b3e6db53dbe69086", - "subject": "[CI][Model] Stabilize MiniMax-M3 nightly smoke test (#15028)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "tests/e2e/nightly/single_node/models/configs/MiniMax-M3-BF16-A3.yaml" - ] - }, - { - "commit": "2617fb6abf541145ea7f12d1dd39c757f1180b66", - "subject": "[Feature] Sfa/MRV2 pcp support (#14026)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/scripts/test_config.yaml", - "tests/e2e/coverage_taxonomy.py", - "tests/e2e/pull_request/eight_card/model_runner_v2/test_glm5_2.py", - "tests/ut/attention/a2/test_sfa_v1.py", - "tests/ut/attention/test_sfa_cp.py", - "tests/ut/quantization/test_modelslim_config.py", - "tests/ut/worker/test_attn_utils_v2.py", - "tests/ut/worker/test_model_runner_v2.py", - "tests/ut/worker/test_pcp_manager_v2.py", - "vllm_ascend/attention/context_parallel/sfa_cp.py", - "vllm_ascend/attention/indexer.py", - "vllm_ascend/attention/sfa_v1.py", - "vllm_ascend/worker/v2/aclgraph_utils.py", - "vllm_ascend/worker/v2/model_runner.py", - "vllm_ascend/worker/v2/model_states/default.py", - "vllm_ascend/worker/v2/pcp_manager.py" - ] - }, - { - "commit": "247c9f49a49f7fa8b24c7821a0e73f1dcb887f3f", - "subject": "[CI] offline dspark e2e (#15187)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "tests/e2e/pull_request/four_card/spec_decode/test_dspark_deepseekv4.py" - ] - }, - { - "commit": "d72770f5bc63346858f990837836d291dbe54887", - "subject": "[Doc] Add Triton operator description specification template (#14966)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_fused_qkvzba_split_reshape_cat.py", - "vllm_ascend/ops/triton/docs/fused_qkvzba_split_reshape.md", - "vllm_ascend/ops/triton/docs/triton_description_specification.md" - ] - }, - { - "commit": "0a023b094e9e88ffaca0b1fda02529cef6277f8e", - "subject": "[Feature][Model] Add Kimi K3 support for vLLM 0.27 (#14454)", - "topics": [ - "deepseek-v4-dspark", - "priority-models", - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/workflows/scripts/runner_label.json", - ".github/workflows/scripts/test_config.yaml", - ".gitleaks.toml", - ".pre-commit-config.yaml", - "csrc/attention/chunk_kda_fwd/CMakeLists.txt", - "csrc/attention/chunk_kda_fwd/README.md", - "csrc/attention/chunk_kda_fwd/chunk_kda_fwd_torch_adpt.h", - "csrc/attention/chunk_kda_fwd/docs/api.md", - "csrc/attention/chunk_kda_fwd/docs/design.md", - "csrc/attention/chunk_kda_fwd/op_host/CMakeLists.txt", - "csrc/attention/chunk_kda_fwd/op_host/arch35/chunk_kda_fwd_tiling_impl.h", - "csrc/attention/chunk_kda_fwd/op_host/chunk_kda_fwd_def.cpp", - "csrc/attention/chunk_kda_fwd/op_host/chunk_kda_fwd_tiling.cpp", - "csrc/attention/chunk_kda_fwd/op_host/chunk_kda_fwd_tiling.h", - "csrc/attention/chunk_kda_fwd/op_host/op_api/aclnn_chunk_kda_fwd.cpp", - "csrc/attention/chunk_kda_fwd/op_host/op_api/aclnn_chunk_kda_fwd.h", - "csrc/attention/chunk_kda_fwd/op_host/op_api/chunk_kda_fwd.cpp", - "csrc/attention/chunk_kda_fwd/op_host/op_api/chunk_kda_fwd.h", - "csrc/attention/chunk_kda_fwd/op_kernel/arch35/chunk_kda_fwd_finalize.h", - "csrc/attention/chunk_kda_fwd/op_kernel/arch35/chunk_kda_fwd_fwd_h.h", - "csrc/attention/chunk_kda_fwd/op_kernel/arch35/chunk_kda_fwd_impl.h", - "csrc/attention/chunk_kda_fwd/op_kernel/arch35/chunk_kda_fwd_post_wu.h", - "csrc/attention/chunk_kda_fwd/op_kernel/arch35/chunk_kda_fwd_prepare.h", - "csrc/attention/chunk_kda_fwd/op_kernel/chunk_kda_fwd.cpp", - "csrc/attention/chunk_kda_fwd/op_kernel/chunk_kda_fwd_common.h", - "csrc/attention/chunk_kda_fwd/op_kernel/chunk_kda_fwd_finalize.h", - "csrc/attention/chunk_kda_fwd/op_kernel/chunk_kda_fwd_post_wu.h", - "csrc/attention/chunk_kda_fwd/op_kernel/chunk_kda_fwd_prepare.h", - "csrc/attention/chunk_kda_fwd/op_kernel/chunk_kda_fwd_varlen.h", - "csrc/attention/kda_gate_cumsum/CMakeLists.txt", - "csrc/attention/kda_gate_cumsum/kda_gate_cumsum_torch_adpt.h", - "csrc/attention/kda_gate_cumsum/op_host/CMakeLists.txt", - "csrc/attention/kda_gate_cumsum/op_host/kda_gate_cumsum_def.cpp", - "csrc/attention/kda_gate_cumsum/op_host/kda_gate_cumsum_tiling.cpp", - "csrc/attention/kda_gate_cumsum/op_host/kda_gate_cumsum_tiling.h", - "csrc/attention/kda_gate_cumsum/op_host/op_api/aclnn_kda_gate_cumsum.cpp", - "csrc/attention/kda_gate_cumsum/op_host/op_api/aclnn_kda_gate_cumsum.h", - "csrc/attention/kda_gate_cumsum/op_host/op_api/kda_gate_cumsum.cpp", - "csrc/attention/kda_gate_cumsum/op_host/op_api/kda_gate_cumsum.h", - "csrc/attention/kda_gate_cumsum/op_kernel/kda_gate_cumsum.cpp", - "csrc/attention/kda_gate_cumsum/op_kernel/kda_gate_cumsum_kernel.h", - "csrc/attention/kda_layout_swap12/CMakeLists.txt", - "csrc/attention/kda_layout_swap12/kda_layout_swap12_torch_adpt.h", - "csrc/attention/kda_layout_swap12/op_host/CMakeLists.txt", - "csrc/attention/kda_layout_swap12/op_host/kda_layout_swap12_def.cpp", - "csrc/attention/kda_layout_swap12/op_host/kda_layout_swap12_tiling.cpp", - "csrc/attention/kda_layout_swap12/op_host/kda_layout_swap12_tiling.h", - "csrc/attention/kda_layout_swap12/op_host/op_api/aclnn_kda_layout_swap12.cpp", - "csrc/attention/kda_layout_swap12/op_host/op_api/aclnn_kda_layout_swap12.h", - "csrc/attention/kda_layout_swap12/op_host/op_api/kda_layout_swap12.cpp", - "csrc/attention/kda_layout_swap12/op_host/op_api/kda_layout_swap12.h", - "csrc/attention/kda_layout_swap12/op_kernel/kda_layout_swap12.cpp", - "csrc/attention/kda_torch_adpt_common.h", - "csrc/attention/mla_prolog_v3/CMakeLists.txt", - "csrc/attention/mla_prolog_v3/docs/api.md", - "csrc/attention/mla_prolog_v3/mla_prolog_v3_torch_adpt.h", - "csrc/attention/mla_prolog_v3/op_api/aclnn_mla_prolog_v3_weight_nz.cpp", - "csrc/attention/mla_prolog_v3/op_api/aclnn_mla_prolog_v3_weight_nz.h", - "csrc/attention/mla_prolog_v3/op_host/CMakeLists.txt", - "csrc/attention/mla_prolog_v3/op_host/mla_prolog_infershape.cpp", - "csrc/attention/mla_prolog_v3/op_host/mla_prolog_infershape.h", - "csrc/attention/mla_prolog_v3/op_host/mla_prolog_tiling.cpp", - "csrc/attention/mla_prolog_v3/op_host/mla_prolog_tiling.h", - "csrc/attention/mla_prolog_v3/op_host/mla_prolog_tiling_check.cpp", - "csrc/attention/mla_prolog_v3/op_host/mla_prolog_tiling_check.h", - "csrc/attention/mla_prolog_v3/op_host/mla_prolog_v3_def.cpp", - "csrc/attention/mla_prolog_v3/op_host/mla_prolog_v3_infershape.cpp", - "csrc/attention/mla_prolog_v3/op_host/mla_prolog_v3_infershape.h", - "csrc/attention/mla_prolog_v3/op_host/mla_prolog_v3_tiling.h", - "csrc/attention/mla_prolog_v3/op_host/mla_prolog_v3_tiling_register.cpp", - "csrc/attention/mla_prolog_v3/op_kernel/arch35/kernel_mla_prolog_split_m.h", - "csrc/attention/mla_prolog_v3/op_kernel/arch35/kernel_mla_prolog_split_n.h", - "csrc/attention/mla_prolog_v3/op_kernel/arch35/mla_prolog_comm.h", - "csrc/attention/mla_prolog_v3/op_kernel/arch35/mla_prolog_vector_comm.h", - "csrc/attention/mla_prolog_v3/op_kernel/arch35/service_dequant.h", - "csrc/attention/mla_prolog_v3/op_kernel/arch35/service_dynamic_quant_qn_mul_qr.h", - "csrc/attention/mla_prolog_v3/op_kernel/arch35/service_gather_sin_cos.h", - "csrc/attention/mla_prolog_v3/op_kernel/arch35/service_matmul.h", - "csrc/attention/mla_prolog_v3/op_kernel/arch35/service_rms_norm.h", - "csrc/attention/mla_prolog_v3/op_kernel/arch35/service_rope.h", - "csrc/attention/mla_prolog_v3/op_kernel/arch35/service_rotary_position_embedding.h", - "csrc/attention/mla_prolog_v3/op_kernel/arch35/service_scatter_cache.h", - "csrc/attention/mla_prolog_v3/op_kernel/arch35/vf/vf_comm.h", - "csrc/attention/mla_prolog_v3/op_kernel/arch35/vf/vf_dequant.h", - "csrc/attention/mla_prolog_v3/op_kernel/arch35/vf/vf_dynamic_quant.h", - "csrc/attention/mla_prolog_v3/op_kernel/arch35/vf/vf_mul_qr.h", - "csrc/attention/mla_prolog_v3/op_kernel/arch35/vf/vf_quant_perchannel.h", - "csrc/attention/mla_prolog_v3/op_kernel/arch35/vf/vf_quant_pertensor.h", - "csrc/attention/mla_prolog_v3/op_kernel/arch35/vf/vf_rms_norm.h", - "csrc/attention/mla_prolog_v3/op_kernel/arch35/vf/vf_rope.h", - "csrc/attention/mla_prolog_v3/op_kernel/mla_prolog_template_tiling_key.h", - "csrc/attention/mla_prolog_v3/op_kernel/mla_prolog_tiling_data.h", - "csrc/attention/mla_prolog_v3/op_kernel/mla_prolog_v3.cpp", - "csrc/attention/recurrent_kda/CMakeLists.txt", - "csrc/attention/recurrent_kda/op_host/CMakeLists.txt", - "csrc/attention/recurrent_kda/op_host/op_api/aclnn_recurrent_kda.cpp", - "csrc/attention/recurrent_kda/op_host/op_api/aclnn_recurrent_kda.h", - "csrc/attention/recurrent_kda/op_host/op_api/recurrent_kda.cpp", - "csrc/attention/recurrent_kda/op_host/op_api/recurrent_kda.h", - "csrc/attention/recurrent_kda/op_host/recurrent_kda_def.cpp", - "csrc/attention/recurrent_kda/op_host/recurrent_kda_infershape.cpp", - "csrc/attention/recurrent_kda/op_host/recurrent_kda_tiling.cpp", - "csrc/attention/recurrent_kda/op_host/recurrent_kda_tiling.h", - "csrc/attention/recurrent_kda/op_host/recurrent_kda_tiling_processor.h", - "csrc/attention/recurrent_kda/op_kernel/arch35/recurrent_kda.h", - "csrc/attention/recurrent_kda/op_kernel/recurrent_kda.cpp", - "csrc/attention/recurrent_kda/op_kernel/recurrent_kda.h", - "csrc/attention/recurrent_kda/op_kernel/recurrent_kda_struct.h", - "csrc/attention/recurrent_kda/op_kernel/recurrent_kda_tiling_data.h", - "csrc/attention/recurrent_kda/recurrent_kda_torch_adpt.h", - "csrc/build_aclnn.sh", - "csrc/moe/chunk_gated_delta_rule_fwd_h/op_host/chunk_gated_delta_rule_fwd_h_def.cpp", - "csrc/moe/chunk_gated_delta_rule_fwd_h/op_host/chunk_gated_delta_rule_fwd_h_tiling.cpp", - "csrc/moe/chunk_gated_delta_rule_fwd_h/op_host/chunk_gated_delta_rule_fwd_h_tiling.h", - "csrc/moe/chunk_gated_delta_rule_fwd_h/op_host/chunk_gated_delta_rule_fwd_h_tiling_processor.h", - "csrc/moe/chunk_gated_delta_rule_fwd_h/op_host/op_api/aclnn_chunk_gated_delta_rule_fwd_h.cpp", - "csrc/moe/chunk_gated_delta_rule_fwd_h/op_host/op_api/aclnn_chunk_gated_delta_rule_fwd_h.h", - "csrc/moe/chunk_gated_delta_rule_fwd_h/op_host/op_api/chunk_gated_delta_rule_fwd_h.cpp", - "csrc/moe/chunk_gated_delta_rule_fwd_h/op_host/op_api/chunk_gated_delta_rule_fwd_h.h", - "csrc/moe/chunk_gated_delta_rule_fwd_h/op_kernel/arch20/gemm/block/block_scheduler_gdn_fwd_h.hpp", - "csrc/moe/chunk_gated_delta_rule_fwd_h/op_kernel/arch20/gemm/kernel/gdn_fwd_h_kernel.hpp", - "csrc/moe/chunk_gated_delta_rule_fwd_h/op_kernel/arch22/epilogue/block/block_epilogue_gdn_fwdh_update.hpp", - "csrc/moe/chunk_gated_delta_rule_fwd_h/op_kernel/arch22/epilogue/block/block_epilogue_gdn_fwdh_vnew.hpp", - "csrc/moe/chunk_gated_delta_rule_fwd_h/op_kernel/arch22/epilogue/gdn_fwd_h_epilogue_policies.hpp", - "csrc/moe/chunk_gated_delta_rule_fwd_h/op_kernel/arch22/gemm/block/block_scheduler_gdn_fwd_h.hpp", - "csrc/moe/chunk_gated_delta_rule_fwd_h/op_kernel/arch22/gemm/kernel/gdn_fwd_h_kernel.hpp", - "csrc/moe/chunk_gated_delta_rule_fwd_h/op_kernel/arch35/epilogue/block/block_epilogue_gdn_fwdh_regbase.hpp", - "csrc/moe/chunk_gated_delta_rule_fwd_h/op_kernel/arch35/epilogue/block/block_epilogue_gdn_fwdh_update.hpp", - "csrc/moe/chunk_gated_delta_rule_fwd_h/op_kernel/arch35/epilogue/block/block_epilogue_gdn_fwdh_vnew.hpp", - "csrc/moe/chunk_gated_delta_rule_fwd_h/op_kernel/arch35/epilogue/gdn_fwd_h_epilogue_policies.hpp", - "csrc/moe/chunk_gated_delta_rule_fwd_h/op_kernel/arch35/gemm/block/block_scheduler_gdn_fwd_h.hpp", - "csrc/moe/chunk_gated_delta_rule_fwd_h/op_kernel/arch35/gemm/kernel/gdn_fwd_h_kernel.hpp", - "csrc/moe/chunk_gated_delta_rule_fwd_h/op_kernel/chunk_gated_delta_rule_fwd_h.cpp", - "csrc/moe/chunk_gated_delta_rule_fwd_h/op_kernel/chunk_gated_delta_rule_fwd_h_struct.h", - "csrc/moe/chunk_gated_delta_rule_fwd_h/op_kernel/epilogue/block/block_epilogue_gdn_fwdh_update.hpp", - "csrc/moe/chunk_gated_delta_rule_fwd_h/op_kernel/epilogue/block/block_epilogue_gdn_fwdh_vnew.hpp", - "csrc/moe/chunk_gated_delta_rule_fwd_h/op_kernel/epilogue/gdn_fwd_h_epilogue_policies.hpp", - "csrc/moe/chunk_gated_delta_rule_fwd_h/op_kernel/gemm/block/block_scheduler_gdn_fwd_h.hpp", - "csrc/moe/chunk_gated_delta_rule_fwd_h/op_kernel/gemm/kernel/gdn_fwd_h_kernel.hpp", - "csrc/moe/common/kernel_utils/block/block_mmad_pingpong_tla.hpp", - "csrc/moe/common/kernel_utils/block/block_mmad_pingpong_tla_multi.hpp", - "csrc/moe/common/kernel_utils/block/block_mmad_pingpong_tla_preloadA_l1B.hpp", - "csrc/moe/common/kernel_utils/tile/copy_l0c_to_ub.hpp", - "csrc/moe/common/kernel_utils/vector/regbase.hpp", - "csrc/moe/dequant_situ_quant/CMakeLists.txt", - "csrc/moe/dequant_situ_quant/dequant_situ_quant_torch_adpt.h", - "csrc/moe/dequant_situ_quant/docs/aclnnDequantSituQuant.md", - "csrc/moe/dequant_situ_quant/op_graph/dequant_situ_quant_proto.h", - "csrc/moe/dequant_situ_quant/op_host/CMakeLists.txt", - "csrc/moe/dequant_situ_quant/op_host/dequant_situ_quant_def.cpp", - "csrc/moe/dequant_situ_quant/op_host/dequant_situ_quant_infershape.cpp", - "csrc/moe/dequant_situ_quant/op_host/dequant_situ_quant_tiling.cpp", - "csrc/moe/dequant_situ_quant/op_host/dequant_situ_quant_tiling.h", - "csrc/moe/dequant_situ_quant/op_kernel/dequant_situ_quant.cpp", - "csrc/moe/dequant_situ_quant/op_kernel/dequant_situ_quant.h", - "csrc/moe/situ_mx_quant/CMakeLists.txt", - "csrc/moe/situ_mx_quant/docs/aclnnSituMxQuant.md", - "csrc/moe/situ_mx_quant/op_host/CMakeLists.txt", - "csrc/moe/situ_mx_quant/op_host/arch35/situ_mx_quant_tiling_arch35.cpp", - "csrc/moe/situ_mx_quant/op_host/arch35/situ_mx_quant_tiling_arch35.h", - "csrc/moe/situ_mx_quant/op_host/config/ascend950/situ_mx_quant_binary.json", - "csrc/moe/situ_mx_quant/op_host/situ_mx_quant_def.cpp", - "csrc/moe/situ_mx_quant/op_host/situ_mx_quant_infershape.cpp", - "csrc/moe/situ_mx_quant/op_kernel/arch35/situ_mx_quant_axis_last.h", - "csrc/moe/situ_mx_quant/op_kernel/arch35/situ_mx_quant_common.h", - "csrc/moe/situ_mx_quant/op_kernel/arch35/situ_mx_quant_tiling_data.h", - "csrc/moe/situ_mx_quant/op_kernel/arch35/situ_mx_quant_tiling_key.h", - "csrc/moe/situ_mx_quant/op_kernel/inc/kernel_utils.h", - "csrc/moe/situ_mx_quant/op_kernel/inc/platform.h", - "csrc/moe/situ_mx_quant/op_kernel/situ_mx_quant_apt.cpp", - "csrc/moe/situ_mx_quant/situ_mx_quant_torch_adpt.h", - "csrc/torch_binding.cpp", - "csrc/torch_binding_meta.cpp", - "docs/source/tutorials/models/Kimi-K3.md", - "docs/source/tutorials/models/index.md", - "examples/disaggregated_prefill_v1/load_balance_proxy_layerwise_server_example.py", - "examples/disaggregated_prefill_v1/load_balance_proxy_server_example.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_chunk_gated_delta_rule_fwd_h_aclnn.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_chunk_kda_aclnn.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_dequant_situ_quant.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_kimi_k3_chunk_kda_tail_npu.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_kimi_k3_situ_fusion_cases.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_kimi_kda_ascendc_npu.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_kimi_kda_recurrent_ascendc_npu.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_mla_prolog_v3.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_kimi_k3_fusions.py", - "tests/e2e/pull_request/sixteen_card/test_kimi_k3.py", - "tests/ut/_310p/fused_moe/test_shared_fused_moe_310.py", - "tests/ut/attention/a2/test_mla_v1.py", - "tests/ut/kv_offload/test_mooncake_connector.py", - "tests/ut/model_executor/test_qwen3_dspark.py", - "tests/ut/models/test_kimi_k3_adapter.py", - "tests/ut/ops/a3_2/test_kimi_kda_fused_norm_gate.py", - "tests/ut/ops/test_fused_moe.py", - "tests/ut/ops/test_gdn_attn_builder.py", - "tests/ut/ops/test_kimi_kda.py", - "tests/ut/ops/test_layernorm.py", - "tests/ut/ops/test_moe_mlp.py", - "tests/ut/ops/test_moe_runtime_args.py", - "tests/ut/ops/test_prepare_finalize.py", - "tests/ut/patch/platform/test_patch_eplb.py", - "tests/ut/patch/platform/test_patch_speculative_config_dspark.py", - "tests/ut/patch/platform/test_prefix_cache_cp_patches.py", - "tests/ut/patch/worker/test_patch_mamba_utils.py", - "tests/ut/patch/worker/test_patch_mamba_utils_source.py", - "tests/ut/patch/worker/test_patch_mamba_utils_uniform_groups.py", - "tests/ut/quantization/test_modelslim_config.py", - "tests/ut/quantization/test_utils.py", - "tests/ut/spec_decode/test_dspark_proposer.py", - "tests/ut/spec_decode/test_llm_base_proposer.py", - "tests/ut/spec_decode/test_utils.py", - "tests/ut/test_compressed_prefix_cache.py", - "tests/ut/test_platform.py", - "tests/ut/test_utils.py", - "tests/ut/worker/a2/test_block_table.py", - "tests/ut/worker/a2/test_model_runner_v1.py", - "tests/ut/worker/a2/test_model_runner_v1_with_device.py", - "vllm_ascend/attention/mla_v1.py", - "vllm_ascend/core/kv_cache_interface.py", - "vllm_ascend/distributed/kv_transfer/kv_p2p/mooncake_connector.py", - "vllm_ascend/models/__init__.py", - "vllm_ascend/models/kimi_k3.py", - "vllm_ascend/models/kimi_k3_dspark.py", - "vllm_ascend/models/kimi_k3_mtp.py", - "vllm_ascend/models/llama_eagle3.py", - "vllm_ascend/models/qwen3_dspark.py", - "vllm_ascend/ops/fused_moe/dataclass/moe_mlp.py", - "vllm_ascend/ops/fused_moe/fused_moe.py", - "vllm_ascend/ops/fused_moe/moe_mlp.py", - "vllm_ascend/ops/fused_moe/prepare_finalize.py", - "vllm_ascend/ops/fused_moe/shared_experts.py", - "vllm_ascend/ops/gdn_attn_builder.py", - "vllm_ascend/ops/kimi_kda.py", - "vllm_ascend/ops/layernorm.py", - "vllm_ascend/ops/mla.py", - "vllm_ascend/ops/mm_encoder_attention.py", - "vllm_ascend/ops/triton/kda/kda.py", - "vllm_ascend/ops/triton/kimi_k3/__init__.py", - "vllm_ascend/ops/triton/kimi_k3/attention_residual.md", - "vllm_ascend/ops/triton/kimi_k3/attention_residual.py", - "vllm_ascend/patch/platform/patch_kv_cache_coordinator.py", - "vllm_ascend/patch/platform/patch_kv_cache_utils.py", - "vllm_ascend/patch/platform/patch_speculative_config.py", - "vllm_ascend/patch/worker/patch_mamba_utils.py", - "vllm_ascend/platform.py", - "vllm_ascend/quantization/modelslim_config.py", - "vllm_ascend/quantization/utils.py", - "vllm_ascend/spec_decode/dspark_proposer.py", - "vllm_ascend/spec_decode/llm_base_proposer.py", - "vllm_ascend/utils.py", - "vllm_ascend/worker/block_table.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "32d9a31387ee7f14bd9c01a8d69dab4a8a397741", - "subject": "[feature] add param additional_config.mc2_comm_alg (#15094)", - "topics": [ - "priority-models", - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "docs/source/tutorials/models/GLM5.2.md", - "docs/source/user_guide/configuration/additional_config.md", - "tests/e2e/nightly/single_node/models/configs/DeepSeek-V3.2-W8A8-DCP.yaml", - "tests/ut/ops/a2/test_token_dispatcher.py", - "tests/ut/test_ascend_config.py", - "tests/ut/test_platform.py", - "vllm_ascend/ascend_config.py", - "vllm_ascend/ops/fused_moe/token_dispatcher.py", - "vllm_ascend/platform.py", - "vllm_ascend/utils.py" - ] - }, - { - "commit": "8091873764215e69632a8d50dedddced12fcc5b9", - "subject": "[BugFix][Mamba] Fix PtrOffsetInfo AxisInfo assertion failure (#15032)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_precopy.py", - "tests/ut/patch/worker/test_patch_mamba_utils_source.py", - "vllm_ascend/ops/triton/v2/mamba/__init__.py", - "vllm_ascend/ops/triton/v2/mamba/docs/mamba_precopy.md", - "vllm_ascend/ops/triton/v2/mamba/precopy.py", - "vllm_ascend/patch/worker/patch_mamba_utils.py", - "vllm_ascend/patch/worker/patch_v2/patch_triton.py" - ] - }, - { - "commit": "72059e41b239e2ea469a0fb2d9eb0cb7ee4e3085", - "subject": "[Feature][MRV2] Reapply basic MLA prefill context parallelism (#14878)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/attention/a2/test_mla_v1.py", - "tests/ut/worker/test_attn_utils_v2.py", - "vllm_ascend/attention/mla_v1.py" - ] - }, - { - "commit": "453f409a704e0b3ba2dbf399a1d64ad4d3764ffa", - "subject": "[CI][Model] Stabilize MiniMax-M3 nightly validation (#14930)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "tests/e2e/nightly/single_node/models/configs/MiniMax-M3-BF16-A3.yaml" - ] - }, - { - "commit": "6ff998cae17f58e43d3bdaf5f5f20723158b1cfa", - "subject": "[Doc][Misc] Update Public FAQs links to relative paths (#15084)", - "topics": [ - "deepseek-v4-dspark", - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/tutorials/models/Cohere-Transcribe.md", - "docs/source/tutorials/models/DeepSeek-R1.md", - "docs/source/tutorials/models/DeepSeek-V3.1.md", - "docs/source/tutorials/models/DeepSeek-V3.2.md", - "docs/source/tutorials/models/DeepSeek-V4-Flash.md", - "docs/source/tutorials/models/DeepSeek-V4-Pro.md", - "docs/source/tutorials/models/DeepSeekOCR2.md", - "docs/source/tutorials/models/GLM5.md", - "docs/source/tutorials/models/Gemma4.md", - "docs/source/tutorials/models/InternVL3.5.md", - "docs/source/tutorials/models/Kimi-K2-Thinking.md", - "docs/source/tutorials/models/Kimi-K2.5.md", - "docs/source/tutorials/models/Kimi-K2.6.md", - "docs/source/tutorials/models/MiniMax-M2.md", - "docs/source/tutorials/models/PaddleOCR-VL.md", - "docs/source/tutorials/models/Qwen-VL-Dense.md", - "docs/source/tutorials/models/Qwen3-235B-A22B.md", - "docs/source/tutorials/models/Qwen3-30B-A3B.md", - "docs/source/tutorials/models/Qwen3-ASR-1.7B.md", - "docs/source/tutorials/models/Qwen3-Coder-30B-A3B.md", - "docs/source/tutorials/models/Qwen3-Dense.md", - "docs/source/tutorials/models/Qwen3-Embedding.md", - "docs/source/tutorials/models/Qwen3-Next.md", - "docs/source/tutorials/models/Qwen3-Omni-30B-A3B-Thinking.md", - "docs/source/tutorials/models/Qwen3-Reranker.md", - "docs/source/tutorials/models/Qwen3-VL-235B-A22B-Instruct.md", - "docs/source/tutorials/models/Qwen3-VL-30B-A3B-Instruct.md", - "docs/source/tutorials/models/Qwen3-VL-Embedding.md", - "docs/source/tutorials/models/Qwen3-VL-Reranker.md", - "docs/source/tutorials/models/Qwen3.5-27B-Qwen3.6-27B.md", - "docs/source/tutorials/models/Qwen3.5-397B-A17B.md", - "docs/source/tutorials/models/Qwen3.5-Dense.md", - "docs/source/tutorials/models/Qwen3.6-35B-A3B.md", - "docs/source/tutorials/models/Qwen3.8-2.4T-A95B.md", - "docs/source/tutorials/models/Qwen3.8-27B.md" - ] - }, - { - "commit": "251334c84ca9113622f2dbda70cf17867379ab0b", - "subject": "[Refactor][Config] Remove transpose KV cache environment variable (#14832)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "docs/source/user_guide/configuration/additional_config.md", - "tests/ut/test_ascend_config.py", - "vllm_ascend/ascend_config.py", - "vllm_ascend/distributed/kv_transfer/kv_p2p/mooncake_connector.py", - "vllm_ascend/envs.py" - ] - }, - { - "commit": "e7cf7e49c3134a1c2ec8b5271073cbd4297f59f0", - "subject": "[Doc][Misc] Update Ascend environment for Qwen3 and MiniMax-M2/M3 (#15009)", - "topics": [ - "communication", - "priority-models", - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/tutorials/models/MiniMax-M2.md", - "docs/source/tutorials/models/MiniMax-M3.md", - "docs/source/tutorials/models/Qwen3-235B-A22B.md", - "docs/source/tutorials/models/Qwen3-30B-A3B.md", - "docs/source/tutorials/models/Qwen3-Coder-30B-A3B.md", - "docs/source/tutorials/models/Qwen3-Dense.md" - ] - }, - { - "commit": "b166a6a0ab89a6806a362d5205d31b71fd1fc61e", - "subject": "[Doc][Feature] Add Atlas 800 A2 deployment for Qwen3.8-2.4T-A95B (#14896)", - "topics": [ - "communication" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/tutorials/models/Qwen3.8-2.4T-A95B.md" - ] - }, - { - "commit": "9bef60ad56998e5c5dc8dd3d7d35ef0e75cb37e2", - "subject": "[Refactor][Config] Remove balance scheduling environment variable (#14827)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/developer_guide/Design_Documents/balance_schedule_refactor.md", - "docs/source/developer_guide/Design_Documents/dynamic_chunked_pipeline_parallel.md", - "docs/source/tutorials/models/DeepSeek-R1.md", - "docs/source/tutorials/models/DeepSeek-V3.1.md", - "docs/source/tutorials/models/GLM4.x.md", - "docs/source/tutorials/models/GLM5.2.md", - "docs/source/tutorials/models/GLM5.md", - "docs/source/tutorials/models/Kimi-K2.5.md", - "docs/source/tutorials/models/Kimi-K2.6.md", - "docs/source/tutorials/models/MiniMax-M2.md", - "docs/source/tutorials/models/Mixtral-8x7B-Instruct-v0.1.md", - "docs/source/tutorials/models/Qwen3-VL-235B-A22B-Instruct.md", - "docs/source/user_guide/configuration/additional_config.md", - "docs/source/user_guide/feature_guide/dynamic_chunk_pipeline_parallel.md", - "docs/source/user_guide/feature_guide/short_request_first.md", - "tests/e2e/nightly/multi_node/internal_dp/config/DeepSeek-V3.1-BF16.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/GLM5_1-W8A8-A2-dual-nodes.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/GLM5_1-W8A8-A3-dual-nodes.yaml", - "tests/e2e/nightly/single_node/models/configs/GLM-4.7.yaml", - "tests/e2e/nightly/single_node/models/configs/Kimi-K2.6-w4a8-A3.yaml", - "tests/e2e/nightly/single_node/models/configs/Qwen3-VL-235B-A22B-Instruct-W8A8.yaml", - "tests/e2e/weekly/single_node/configs/GLM-5.yaml", - "tests/e2e/weekly/single_node/configs/GLM-5_1-W8A8_A3_weekly.yaml", - "tests/e2e/weekly/single_node/configs/Kimi-K2.6-w4a8-A3.yaml", - "tests/e2e/weekly/single_node/configs/Kimi-K2.7-w4a8-A3.yaml", - "tests/e2e/weekly/single_node/configs/Qwen3.5-122B-A10B-w4a8-A3-accuracy.yaml", - "tests/e2e/weekly/single_node/configs/Qwen3.5-122B-A10B-w4a8-A3.yaml", - "tests/e2e/weekly/single_node/configs/Qwen3.5-397B-A17B-w8a8-64k-1k-90-50-single.yaml", - "tests/e2e/weekly/single_node/configs/Qwen3.6-35B-A3B-w4a8-A3.yaml", - "tests/e2e/weekly/single_node/configs/Qwen3.6-35B-A3B-w8a8-A3-accuracy.yaml", - "vllm_ascend/ascend_config.py", - "vllm_ascend/envs.py", - "vllm_ascend/patch/__init__.py" - ] - }, - { - "commit": "e8d9c1ac39ef1b09f1502a62eab7bc9d31fccc42", - "subject": "[CI]fix the failed weekly cases (#14972)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/configs/nightly_config.yaml", - ".github/workflows/configs/weekly_config.yaml", - "tests/e2e/nightly/multi_node/external_dp/config/GLM-5.1-W8A8C8-128k-1k-90-50-PD.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/GLM-5.1-W8A8C8-A3_128k_90_50.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/GLM-5.1-W8A8C8-A3_198k_function.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/GLM5_1-W8A8-EP.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/GLM-5.1-W8A8C8-3.5k-1.5k-0-20-PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/GLM-5.1-W8A8C8-3.5k-1.5k-0-50-PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/GLM-5.1-W8A8C8-64k-1k-90-50-PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/GLM-5.2-W8A8C8-128k-1k-90-50-PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/GLM-5.2-W8A8C8-3.5k-1.5k-0-20-PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/GLM-5.2-W8A8C8-3.5k-1.5k-0-50-PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/GLM-5.2-W8A8C8-64k-1k-90-50-PD.yaml", - "tests/e2e/weekly/multi_node/internal_dp/config/GLM-5.1-W8A8C8-A3_3.5k_1.5k_0_20.yaml", - "tests/e2e/weekly/multi_node/internal_dp/config/GLM-5.1-W8A8C8-A3_3.5k_1.5k_0_50.yaml", - "tests/e2e/weekly/multi_node/internal_dp/config/GLM-5.1-W8A8C8-A3_64k_90_50.yaml", - "tests/e2e/weekly/multi_node/internal_dp/config/GLM-5.2-W8A8C8-A3_128k_90_50.yaml", - "tests/e2e/weekly/multi_node/internal_dp/config/GLM-5.2-W8A8C8-A3_198k_function.yaml", - "tests/e2e/weekly/multi_node/internal_dp/config/GLM-5.2-W8A8C8-A3_3.5k_1.5k_0_20.yaml", - "tests/e2e/weekly/multi_node/internal_dp/config/GLM-5.2-W8A8C8-A3_3.5k_1.5k_0_50.yaml", - "tests/e2e/weekly/multi_node/internal_dp/config/GLM-5.2-W8A8C8-A3_64k_90_50.yaml", - "tests/e2e/weekly/single_node/configs/Qwen3.5-27B-w8a8-A3.yaml" - ] - }, - { - "commit": "40f94b79fc41959d3c1d37875020b49ad29d2072", - "subject": "[Feature][MRV2] Support DSA decode ACLGraph (#14762)", - "topics": [ - "deepseek-v4-dspark", - "communication" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/attention/test_dsa_v1.py", - "tests/ut/attention/test_dsv4_block_geometry.py", - "tests/ut/models/test_deepseek_v4_compressor.py", - "vllm_ascend/attention/context_parallel/dsa_cp.py", - "vllm_ascend/attention/dsa_v1.py", - "vllm_ascend/models/deepseek_v4/compressor.py", - "vllm_ascend/worker/model_runner_v1.py", - "vllm_ascend/worker/v2/attn_utils.py", - "vllm_ascend/worker/v2/model_states/default.py" - ] - }, - { - "commit": "128ecf890fc36f29492f7232214e8d63d2e3c28e", - "subject": "[Refactor]unify calls to torch_npu.npu_mla_prolog_v3 (#14676)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/attention/a2/test_mla_v1.py", - "vllm_ascend/attention/mla_v1.py", - "vllm_ascend/device/device_op.py", - "vllm_ascend/ops/fused_moe/routed_experts.py" - ] - }, - { - "commit": "b7fc48b65ee20967e141364553a86ae3d5fd2f91", - "subject": "[CI] Overhaul PR test scheduling with shared 800I/800T NPU pools (#14793)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - ".github/TEST_README.md", - ".github/actionlint.yaml", - ".github/workflows/_selected_tests.yaml", - ".github/workflows/pr_test.yaml", - ".github/workflows/scripts/coverage.py", - ".github/workflows/scripts/run_selected_tests.sh", - ".github/workflows/scripts/runner_label.json", - ".github/workflows/scripts/select_tests.py", - ".github/workflows/scripts/test_config.yaml", - ".github/workflows/scripts/test_select_tests.py", - "tests/e2e/pull_request/four_card/spec_decode/test_mtp_step3p5.py", - "tests/e2e/pull_request/four_card/test_profiling_chunk_performance.py", - "tests/e2e/pull_request/two_card/lora/test_llama32_lora_tp2.py", - "tests/e2e/pull_request/two_card/test_shared_expert_dp.py" - ] - }, - { - "commit": "1d9cf2485de31b219a0770476ca3f7d5b2493793", - "subject": "[CI][BugFix][MRV2]Down the accptance of dspark (#14890)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "tests/e2e/pull_request/four_card/model_runner_v2/test_deepseek_v4.py" - ] - }, - { - "commit": "ce8f64e5617bbeb315e7b736fa8199a36457f5ae", - "subject": "[CI][MRV2] Temporarily skip DeepSeek V4 MTP eager test (#15007)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "tests/e2e/pull_request/four_card/model_runner_v2/test_deepseek_v4.py" - ] - }, - { - "commit": "80c833fb86b105391d164c9e3fc60d0bfecbf85e", - "subject": "[Refactor][Config] Remove fused MC2 environment variable (#14831)", - "topics": [ - "deepseek-v4-dspark", - "communication", - "priority-models", - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "docs/source/tutorials/models/DeepSeek-V4-Pro.md", - "docs/source/tutorials/models/GLM4.x.md", - "docs/source/tutorials/models/GLM5.2.md", - "docs/source/tutorials/models/GLM5.md", - "docs/source/tutorials/models/InternVL3.5.md", - "docs/source/tutorials/models/MiniMax-M2.md", - "docs/source/tutorials/models/Qwen3-VL-235B-A22B-Instruct.md", - "docs/source/tutorials/models/Qwen3-VL-30B-A3B-Instruct.md", - "docs/source/tutorials/models/Qwen3.8-2.4T-A95B.md", - "docs/source/user_guide/configuration/additional_config.md", - "tests/e2e/nightly/multi_node/external_dp/config/DeepSeek-V4-Pro-w4a8-prefix-cache-PD.yaml", - "tests/e2e/nightly/multi_node/external_dp/config/DeepSeek-V4-flash-w8a8-PD-prefix.yaml", - "tests/e2e/nightly/multi_node/external_dp/config/GLM-5.1-W8A8C8-128k-1k-90-50-PD.yaml", - "tests/e2e/nightly/multi_node/external_dp/config/GLM-5.2-W8A8C8-128k-1k-90-50-PD.yaml", - "tests/e2e/nightly/multi_node/external_dp/config/GLM5_1-W8A8-EP-external.yaml", - "tests/e2e/nightly/multi_node/external_dp/config/Kimi-K2.6-W4A8-64k-1k-TPOT50-PD.yaml", - "tests/e2e/nightly/multi_node/external_dp/config/Minimax_m2.7_in128k_1k_prefix90_tpot50.yaml", - "tests/e2e/nightly/multi_node/external_dp/scripts/utils.py", - "tests/e2e/nightly/multi_node/internal_dp/config/GLM-5.1-W8A8C8-A3_128k_90_50.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/GLM-5.1-W8A8C8-A3_198k_function.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/GLM-5.2-W8A8C8-A3_128k_90_50.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/GLM-5.2-W8A8C8-A3_198k_function.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/GLM5_1-W8A8-EP.yaml", - "tests/e2e/nightly/multi_node/internal_dp/scripts/test_multi_node.py", - "tests/e2e/nightly/single_node/models/configs/DeepSeek-V3.2-W8A8-DCP.yaml", - "tests/e2e/nightly/single_node/models/configs/DeepSeek-V4-Flash-W8A8-A3-Breakable.yaml", - "tests/e2e/nightly/single_node/models/configs/DeepSeek-V4-Flash-W8A8-A3.yaml", - "tests/e2e/nightly/single_node/models/configs/GLM-4.7.yaml", - "tests/e2e/nightly/single_node/models/configs/Kimi-K2.6-w4a8-A3.yaml", - "tests/e2e/nightly/single_node/models/configs/MiniMax-M2.5-w8a8-QuaRot-A2.yaml", - "tests/e2e/nightly/single_node/models/configs/Qwen3-VL-235B-A22B-Instruct-W8A8.yaml", - "tests/e2e/nightly/single_node/models/configs/Qwen3.5-122B-A10B-W8A8-A3.yaml", - "tests/e2e/nightly/single_node/models/configs/Qwen3.5-27B-w8a8-A2.yaml", - "tests/e2e/nightly/single_node/models/configs/Qwen3.5-397B-A17B-W8A8-mtp-A3.yaml", - "tests/e2e/nightly/single_node/models/scripts/test_single_node.py", - "tests/e2e/weekly/multi_node/external_dp/config/DeepSeek-V4-Pro-w4a8-PD-no-prefix.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/DeepSeek-V4-Pro-w4a8-prefix-cache-PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/DeepSeek-V4-flash-w8a8-PD-no-prefix.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/DeepSeek-V4-flash-w8a8-PD-prefix-weekly.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/DeepSeek_V3.2T_MTP2_PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/DeepSeek_V3.2T_MTP3_PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/GLM-5.1-W8A8C8-3.5k-1.5k-0-20-PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/GLM-5.1-W8A8C8-3.5k-1.5k-0-50-PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/GLM-5.1-W8A8C8-64k-1k-90-50-PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/GLM-5.2-W8A8C8-3.5k-1.5k-0-20-PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/GLM-5.2-W8A8C8-3.5k-1.5k-0-50-PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/GLM-5.2-W8A8C8-64k-1k-90-50-PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/GLM_5_1_PD_in32k_bs16-0.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/GLM_5_1_PD_in32k_bs20-90.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/GLM_5_1_PD_in64k_bs16-90.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/Kimi-K2.5-W4A8-128k-1k-TPOT50.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/Kimi-K2.6-W4A8-128k-1k-TPOT50-PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/Kimi-K2.6-W4A8-254k-1k-TPOT20-PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/Kimi-K2.6-W4A8-3.5k-1.5k-TPOT50-PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/Kimi-K2.7-W4A8-128k-1k-TPOT50-PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/Kimi-K2.7-W4A8-254k-1k-TPOT20-PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/Kimi-K2.7-W4A8-3.5k-1.5k-TPOT50-PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/Kimi-K2.7-W4A8-64k-1k-TPOT50-PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/MiniMax-PD-in32k-bs4-1.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/Minimax_m2.7_in198_prefix99.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/Minimax_m2.7_in3_5_tpot20.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/Minimax_m2.7_in3_5_tpot50.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/Minimax_m2.7_in64k_1k_prefix90_tpot50.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/Qwen-3.5-397B-A17B-W8A8-PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/Qwen3.5-122B-A10B-w4a8-128k-1k-TPOT50-90-PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/Qwen3.5-122B-A10B-w4a8-16k-1k-TPOT50-0-PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/Qwen3.5-122B-A10B-w4a8-984k-1k-0-PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/Qwen3.5-397B-w4a8_128k-1k-90-50-PD.yaml", - "tests/e2e/weekly/multi_node/internal_dp/config/DeepSeek-V3_2-W8A8-EP_weekly.yaml", - "tests/e2e/weekly/multi_node/internal_dp/config/GLM-5.1-W8A8C8-A3_3.5k_1.5k_0_20.yaml", - "tests/e2e/weekly/multi_node/internal_dp/config/GLM-5.1-W8A8C8-A3_3.5k_1.5k_0_50.yaml", - "tests/e2e/weekly/multi_node/internal_dp/config/GLM-5.1-W8A8C8-A3_64k_90_50.yaml", - "tests/e2e/weekly/multi_node/internal_dp/config/GLM-5.2-W8A8C8-A3_3.5k_1.5k_0_20.yaml", - "tests/e2e/weekly/multi_node/internal_dp/config/GLM-5.2-W8A8C8-A3_3.5k_1.5k_0_50.yaml", - "tests/e2e/weekly/multi_node/internal_dp/config/GLM-5.2-W8A8C8-A3_64k_90_50.yaml", - "tests/e2e/weekly/single_node/configs/Kimi-K2.6-w4a8-A3.yaml", - "tests/e2e/weekly/single_node/configs/Kimi-K2.7-w4a8-A3.yaml", - "tests/e2e/weekly/single_node/configs/Qwen3.5-122B-A10B-W8A8-A2.yaml", - "tests/e2e/weekly/single_node/configs/Qwen3.5-122B-A10B-W8A8-A3.yaml", - "tests/e2e/weekly/single_node/configs/Qwen3.5-122B-A10B-w4a8-A3-accuracy.yaml", - "tests/e2e/weekly/single_node/configs/Qwen3.5-122B-A10B-w4a8-A3.yaml", - "tests/e2e/weekly/single_node/configs/Qwen3.5-397B-A17B-W8A8-mtp-A3.yaml", - "tests/e2e/weekly/single_node/configs/Qwen3.5-397B-A17B-W8A8-mtp-A3_weekly.yaml", - "tests/e2e/weekly/single_node/configs/Qwen3.5-397B-A17B-w8a8-3.5k-1.5k-0-50-single.yaml", - "tests/e2e/weekly/single_node/configs/Qwen3.5-397B-A17B-w8a8-64k-1k-90-50-single.yaml", - "tests/e2e/weekly/single_node/configs/Qwen3.6-35B-A3B-w4a8-A3.yaml", - "tests/ut/lora/test_lora.py", - "tests/ut/test_ascend_config.py", - "vllm_ascend/ascend_config.py", - "vllm_ascend/envs.py", - "vllm_ascend/lora/fused_moe.py", - "vllm_ascend/platform.py" - ] - }, - { - "commit": "59a36f29392eb6ad7bf9068933b997dd33761d46", - "subject": "[Feature][310P] Adapting MRV2 for the 310P (#14761)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/workflows/scripts/test_config.yaml", - "tests/e2e/pull_request/four_card/_310p/test_model_runner_v2_310p.py", - "tests/ut/_310p/test_model_runner_v2_310p.py", - "vllm_ascend/_310p/attention/metadata_builder.py", - "vllm_ascend/_310p/worker/__init__.py", - "vllm_ascend/_310p/worker/v2/__init__.py", - "vllm_ascend/_310p/worker/v2/block_table.py", - "vllm_ascend/_310p/worker/v2/model_runner.py", - "vllm_ascend/_310p/worker/v2/model_state.py", - "vllm_ascend/_310p/worker/v2/sampler.py", - "vllm_ascend/_310p/worker/v2/states.py", - "vllm_ascend/_310p/worker_310p.py", - "vllm_ascend/patch/platform/patch_use_v2_model_runner.py", - "vllm_ascend/patch/worker/patch_v2/patch_block_table.py", - "vllm_ascend/worker/v2/model_states/__init__.py" - ] - }, - { - "commit": "994765d7ab3ac2661865d149c803146927366b26", - "subject": "[Performance] use sort-based top-k/top-p in non-reduce mode for consistent performance (#13842)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/sample/sampler.py" - ] - }, - { - "commit": "ca5603874545031addd771c457a4d8c3e2eeedf1", - "subject": "[BugFix] Add mlapo_keep_prefill_weights option to keep prefill weights on NPU (#14216)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/user_guide/configuration/additional_config.md", - "vllm_ascend/ascend_config.py", - "vllm_ascend/attention/mla_v1.py" - ] - }, - { - "commit": "061a92696e8ca2ab8ba161a68384001d441ffc73", - "subject": "[Feature][Quantization] Serve native block-wise FP8 checkpoints for qwen3.8-27b-fp8 (#14852)", - "topics": [ - "deepseek-v4-dspark", - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "docs/source/user_guide/feature_guide/quantization.md", - "gitleaks", - "tests/ut/quantization/methods/test_fp8_block.py", - "tests/ut/quantization/test_fp8_config.py", - "vllm_ascend/quantization/fp8_config.py", - "vllm_ascend/quantization/method_adapters.py", - "vllm_ascend/quantization/methods/__init__.py", - "vllm_ascend/quantization/methods/fp8_block.py" - ] - }, - { - "commit": "2452b1fd06a348b11ad4e03548049f97c9caeb0d", - "subject": "[Doc][MRV2] Add prefill context parallelism guide (#14571)", - "topics": [ - "communication" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/user_guide/feature_guide/context_parallel.md" - ] - }, - { - "commit": "aeb8e76d35158ceab5fa4d0b8c6670f0928b6b3f", - "subject": "[BugFix][Mamba] Fix staged-write kernel and duplicate is_prefilling argument (#14649)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/patch/worker/test_patch_mamba_utils_source.py", - "vllm_ascend/ops/triton/mamba/precopy.py", - "vllm_ascend/patch/worker/patch_mamba_utils.py", - "vllm_ascend/worker/v2/attn_utils.py" - ] - }, - { - "commit": "b6de3df77d5084c17b4f6f314159e113b43107f2", - "subject": "[BugFix][Device Allocator] Fix CaMem sleep race with in-flight graph replay (#13757)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/device_allocator/camem.py" - ] - }, - { - "commit": "cecf7e3653e65acd7b4d77f9cae35324adda993b", - "subject": "[Doc] Translated Doc files 2026-08-25 (#14873)", - "topics": [ - "deepseek-v4-dspark", - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/locale/zh_CN/LC_MESSAGES/community/contributors.po", - "docs/source/locale/zh_CN/LC_MESSAGES/community/versioning_policy.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/Design_Documents/context_parallel.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/Design_Documents/dynamic_chunked_pipeline_parallel.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/contribution/testing.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/performance_and_debug/performance_benchmark.po", - "docs/source/locale/zh_CN/LC_MESSAGES/faqs.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/features/dynamic_chunked_pipeline_parallel.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Cohere-Transcribe.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/DeepSeek-R1.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/DeepSeek-V3.1.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/DeepSeek-V3.2.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/DeepSeek-V4-Flash.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/DeepSeek-V4-Pro.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/DeepSeekOCR2.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Dots3-Note.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/GLM4.x.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/GLM5.2.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/GLM5.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Gemma4.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Hy3-preview.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/InternVL3.5.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Kimi-K2-Thinking.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Kimi-K2.5.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Kimi-K2.6.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/LLaVA-OneVision-Qwen2-0.5B-OV.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/MiniMax-M2.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/MiniMax-M3.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/PaddleOCR-VL.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen-VL-Dense.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen2.5-Math-RM-72B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-235B-A22B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-30B-A3B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-ASR-1.7B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-Coder-30B-A3B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-Dense.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-Embedding.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-Next.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-Omni-30B-A3B-Thinking.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-Reranker.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-VL-235B-A22B-Instruct.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-VL-30B-A3B-Instruct.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-VL-Embedding.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-VL-Reranker.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3.5-27B-Qwen3.6-27B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3.5-397B-A17B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3.5-Dense.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3.6-35B-A3B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3.8-2.4T-A95B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3.8-27B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/gpt-oss-120b.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/configuration/additional_config.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/context_parallel.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/dynamic_chunk_pipeline_parallel.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/flash_attention.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/kv_cache_cpu_offload.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/kv_pool.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/sequence_parallelism.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/sleep_mode.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/index.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/ms_service_metric.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/release_notes.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/support_matrix/feature_matrix.po" - ] - }, - { - "commit": "6fd6ea161b904ee4b379f6a18bc29d3e076cf25a", - "subject": "[BugFix][Config] Clean up stale weekly and nightly additional configs (#14934)", - "topics": [ - "priority-models", - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/e2e/nightly/multi_node/external_dp/config/GLM-5.1-W8A8C8-128k-1k-90-50-PD.yaml", - "tests/e2e/nightly/multi_node/external_dp/config/GLM-5.2-W8A8C8-128k-1k-90-50-PD.yaml", - "tests/e2e/nightly/multi_node/external_dp/config/Kimi-K2.6-W4A8-64k-1k-TPOT50-PD.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/DeepSeek-R1-W8A8-EPLB.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/GLM-5.1-W8A8C8-A3_128k_90_50.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/GLM-5.1-W8A8C8-A3_198k_function.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/GLM-5.2-W8A8C8-A3_128k_90_50.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/GLM-5.2-W8A8C8-A3_198k_function.yaml", - "tests/e2e/nightly/single_node/models/configs/DeepSeek-R1-0528-W8A8.yaml", - "tests/e2e/nightly/single_node/models/configs/MiniMax-M2.5-w8a8-QuaRot-A2.yaml", - "tests/e2e/nightly/single_node/models/configs/Minimax_m2.7_w8a8_A3.yaml", - "tests/e2e/nightly/single_node/models/configs/Prefix-Cache-DeepSeek-R1-0528-W8A8.yaml", - "tests/e2e/nightly/single_node/models/configs/Prefix-Cache-Qwen3-32B-Int8.yaml", - "tests/e2e/nightly/single_node/models/configs/Qwen3.5-27B-w8a8-A2.yaml", - "tests/e2e/nightly/single_node/models/configs/Qwen3_32b_w8a8_ml_A3.yaml", - "tests/e2e/nightly/single_node/models/scripts/test_single_node.py", - "tests/e2e/weekly/multi_node/external_dp/config/GLM-5.1-W8A8C8-3.5k-1.5k-0-20-PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/GLM-5.1-W8A8C8-3.5k-1.5k-0-50-PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/GLM-5.1-W8A8C8-64k-1k-90-50-PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/GLM-5.2-W8A8C8-3.5k-1.5k-0-20-PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/GLM-5.2-W8A8C8-3.5k-1.5k-0-50-PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/GLM-5.2-W8A8C8-64k-1k-90-50-PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/Kimi-K2.6-W4A8-128k-1k-TPOT50-PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/Kimi-K2.6-W4A8-254k-1k-TPOT20-PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/Kimi-K2.6-W4A8-3.5k-1.5k-TPOT50-PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/Kimi-K2.7-W4A8-128k-1k-TPOT50-PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/Kimi-K2.7-W4A8-254k-1k-TPOT20-PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/Kimi-K2.7-W4A8-3.5k-1.5k-TPOT50-PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/Kimi-K2.7-W4A8-64k-1k-TPOT50-PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/QWEN3_235B_PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/QWEN3_235B_PD_3_5K_1_5k.yaml", - "tests/e2e/weekly/multi_node/internal_dp/config/GLM-5.1-W8A8C8-A3_3.5k_1.5k_0_20.yaml", - "tests/e2e/weekly/multi_node/internal_dp/config/GLM-5.1-W8A8C8-A3_3.5k_1.5k_0_50.yaml", - "tests/e2e/weekly/multi_node/internal_dp/config/GLM-5.1-W8A8C8-A3_64k_90_50.yaml", - "tests/e2e/weekly/multi_node/internal_dp/config/GLM-5.2-W8A8C8-A3_3.5k_1.5k_0_20.yaml", - "tests/e2e/weekly/multi_node/internal_dp/config/GLM-5.2-W8A8C8-A3_3.5k_1.5k_0_50.yaml", - "tests/e2e/weekly/multi_node/internal_dp/config/GLM-5.2-W8A8C8-A3_64k_90_50.yaml", - "tests/e2e/weekly/single_node/configs/Deepseek_R1_W8A8_Reasoning_output_A3.yaml", - "tests/e2e/weekly/single_node/configs/Minimax_M2.7_w8a8_A3.yaml", - "tests/e2e/weekly/single_node/configs/Qwen3-235B-A22B-W8A8_piecewise_fullgraph_A3.yaml", - "tests/e2e/weekly/single_node/configs/Qwen3-32B.yaml", - "tests/e2e/weekly/single_node/configs/Qwen3_32b_W8A8_wl_A3.yaml", - "tests/e2e/weekly/single_node/configs/Qwq_32b_A3.yaml" - ] - }, - { - "commit": "d1523852b416be34cfea126a9134bb3519771834", - "subject": "[Feature] Add fused_sparse_attention_overlap operator for DSA with CPU offloading (#14278)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "csrc/attention/fused_sparse_attention_overlap/CMakeLists.txt", - "csrc/attention/fused_sparse_attention_overlap/README.md", - "csrc/attention/fused_sparse_attention_overlap/fused_sparse_attention_overlap_torch_adpt.h", - "csrc/attention/fused_sparse_attention_overlap/op_host/CMakeLists.txt", - "csrc/attention/fused_sparse_attention_overlap/op_host/fused_sparse_attention_overlap_def.cpp", - "csrc/attention/fused_sparse_attention_overlap/op_host/fused_sparse_attention_overlap_proto.cpp", - "csrc/attention/fused_sparse_attention_overlap/op_host/fused_sparse_attention_overlap_tiling.cpp", - "csrc/attention/fused_sparse_attention_overlap/op_host/fused_sparse_attention_overlap_tiling.h", - "csrc/attention/fused_sparse_attention_overlap/op_kernel/arch22/fused_sparse_attention_overlap_kernel_mla.h", - "csrc/attention/fused_sparse_attention_overlap/op_kernel/arch22/fused_sparse_attention_overlap_service_cube_mla.h", - "csrc/attention/fused_sparse_attention_overlap/op_kernel/arch22/fused_sparse_attention_overlap_service_vector_mla.h", - "csrc/attention/fused_sparse_attention_overlap/op_kernel/arch35/fused_sparse_attention_overlap_common_arch35.h", - "csrc/attention/fused_sparse_attention_overlap/op_kernel/arch35/fused_sparse_attention_overlap_kernel_mla.h", - "csrc/attention/fused_sparse_attention_overlap/op_kernel/arch35/fused_sparse_attention_overlap_kvcache.h", - "csrc/attention/fused_sparse_attention_overlap/op_kernel/arch35/fused_sparse_attention_overlap_service_cube_mla.h", - "csrc/attention/fused_sparse_attention_overlap/op_kernel/arch35/fused_sparse_attention_overlap_service_vector_mla.h", - "csrc/attention/fused_sparse_attention_overlap/op_kernel/arch35/util_regbase.h", - "csrc/attention/fused_sparse_attention_overlap/op_kernel/fused_sparse_attention_overlap.cpp", - "csrc/attention/fused_sparse_attention_overlap/op_kernel/fused_sparse_attention_overlap_common.h", - "csrc/attention/fused_sparse_attention_overlap/op_kernel/fused_sparse_attention_overlap_template_tiling_key.h", - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_fused_sparse_attention_overlap.py" - ] - }, - { - "commit": "5efd9c05019aa62b910c16f7ef34d37d7638d940", - "subject": "[BugFix][310P] Fix Moe aclgraph error caued by routing to aclop on 310p (#14863)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/_310p/fused_moe/test_shared_fused_moe_310.py", - "vllm_ascend/_310p/fused_moe/fused_moe.py" - ] - }, - { - "commit": "fb5533a6869f8b02e61811933569661fc055850c", - "subject": "[BugFix][MiniMax M3] Fix MXFP8 RMSNorm and sparse index bounds (#14704)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "tests/e2e/pull_request/one_card/test_minimax_m3_sparse_attn.py", - "tests/ut/ops/test_layernorm.py", - "vllm_ascend/models/minimax_m3/ops/msa_m3_triton_a5.py", - "vllm_ascend/ops/layernorm.py" - ] - }, - { - "commit": "d7697b7476c598ac4cdc612da6a219cb46cccb7e", - "subject": "[Ops][Bugfix] Fix triton stub on x86 machines (#14546)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/__init__.py" - ] - }, - { - "commit": "12276de8059151c56c876154abc536b92d8681e4", - "subject": "[Feature] Validate IndexShare and IndexCache PP partition (#14088)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/test_platform.py", - "vllm_ascend/platform.py" - ] - }, - { - "commit": "2497666c93b222a98c190f89172b1b1fce3d9daf", - "subject": "[BugFix][Performance] Skip identity d2t mapping for full-vocab drafts (#14738)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/spec_decode/llm_base_proposer.py" - ] - }, - { - "commit": "c073c3b7587e03bbe78a6078ffc20b1343795b21", - "subject": "[BugFix] Restore MoE and DSA control buffers after level-2 wake (#13352)", - "topics": [ - "deepseek-v4-dspark", - "communication", - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/attention/test_attention_utils.py", - "tests/ut/worker/a2/test_worker_v1.py", - "vllm_ascend/attention/context_parallel/dsa_cp.py", - "vllm_ascend/attention/dsa_v1.py", - "vllm_ascend/attention/utils.py", - "vllm_ascend/ops/fused_moe/fused_moe.py", - "vllm_ascend/worker/worker.py" - ] - }, - { - "commit": "529b455750a2918683b335629276f3ddf03a4e99", - "subject": "[Bugfix][SpecDecode] Support probabilistic draft sampling on MRV1 (#14485)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/_310p/spec_decode/test_llm_base_proposer_310.py", - "tests/ut/spec_decode/a2/test_eagle_proposer.py", - "tests/ut/spec_decode/test_dflash2_proposer.py", - "tests/ut/spec_decode/test_dspark_proposer.py", - "vllm_ascend/_310p/spec_decode/llm_base_proposer_310.py", - "vllm_ascend/spec_decode/dflash2_proposer.py", - "vllm_ascend/spec_decode/dspark_proposer.py", - "vllm_ascend/spec_decode/llm_base_proposer.py", - "vllm_ascend/spec_decode/step3p5.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "37e382498c81fdbcfce8529ea1d53570e0d239a5", - "subject": "Revert \"[Feature][MRV2] Support basic MLA prefill context parallelism\" (#14853)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/attention/a2/test_mla_v1.py", - "tests/ut/worker/test_attn_utils_v2.py", - "vllm_ascend/attention/mla_v1.py", - "vllm_ascend/worker/v2/attn_utils.py" - ] - }, - { - "commit": "1a086ce8ab27a01d2b2fdef32797b6d989f4727f", - "subject": "[main][Ops][Misc] Optimize speculative decoding drafting metadata preparation (#14103)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/worker/test_dcp_manager.py", - "vllm_ascend/worker/dcp_utils.py" - ] - }, - { - "commit": "581051a80a3568fd4487484bfb7b13a4a9d2abf5", - "subject": "[Feature][MRV2] Support basic MLA prefill context parallelism (#14008)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/attention/a2/test_mla_v1.py", - "tests/ut/worker/test_attn_utils_v2.py", - "vllm_ascend/attention/mla_v1.py", - "vllm_ascend/worker/v2/attn_utils.py" - ] - }, - { - "commit": "b0a0aa59cfee107fc6d3e20a2580e68a1795ae2b", - "subject": "[1/N][Feature] Enable breakable ACL graph capture/replay on MRV1 (#12744)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/scripts/test_config.yaml", - "tests/e2e/nightly/single_node/models/configs/DeepSeek-V4-Flash-W8A8-A3-Breakable.yaml", - "tests/e2e/pull_request/four_card/test_graph_mode.py", - "vllm_ascend/compilation/breakable_aclgraph.py", - "vllm_ascend/ops/dsa.py", - "vllm_ascend/ops/mla.py", - "vllm_ascend/platform.py", - "vllm_ascend/spec_decode/llm_base_proposer.py", - "vllm_ascend/worker/model_runner_v1.py", - "vllm_ascend/worker/v2/aclgraph_utils.py", - "vllm_ascend/worker/v2/model_runner.py", - "vllm_ascend/worker/v2/utils.py" - ] - }, - { - "commit": "9fdcd6e77204a74ce8c4ea1f4e6dd3c1f78242a9", - "subject": "[CI][E2E] Add MiniMax M3 W8A8 A3 nightly coverage (#14009)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/misc/model_dataset_list.json", - "tests/e2e/nightly/single_node/models/configs/MiniMax-M3-W8A8-A3.yaml", - "tests/e2e/pull_request/eight_card/test_minimax_m3.py", - "tools/aisbench.py" - ] - }, - { - "commit": "22e214ad77fa1cda43960706c605814e7b947f8d", - "subject": "[Doc][Misc] Update Ascend environment for Qwen3 and MiniMax-M2/M3 (#14481)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/tutorials/models/MiniMax-M2.md", - "docs/source/tutorials/models/MiniMax-M3.md", - "docs/source/tutorials/models/Qwen3-235B-A22B.md", - "docs/source/tutorials/models/Qwen3-30B-A3B.md", - "docs/source/tutorials/models/Qwen3-Coder-30B-A3B.md", - "docs/source/tutorials/models/Qwen3-Dense.md" - ] - }, - { - "commit": "649582947376908c69e4759c79fadd62ce898ac5", - "subject": "[Ci] main2main vllm 0821 (#14746)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - ".github/vllm-main-verified.commit", - "tests/e2e/conftest.py", - "tests/ut/core/test_profiling_chunk.py", - "tests/ut/core/test_recompute_scheduler.py", - "tests/ut/patch/platform/test_patch_structured_output.py", - "vllm_ascend/attention/attention_v1.py", - "vllm_ascend/compilation/acl_graph.py", - "vllm_ascend/models/deepseek_mtp.py", - "vllm_ascend/spec_decode/llm_base_proposer.py", - "vllm_ascend/worker/v2/spec_decode/dflash/speculator.py" - ] - }, - { - "commit": "b7de842253ad8da059c31ff7e5e9357974d7929a", - "subject": "[Doc][Misc] Standardize memory units, IP addresses, and reference links in model deployment tutorials (#14797)", - "topics": [ - "deepseek-v4-dspark", - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - ".agents/skills/vllm-ascend-release/references/ref-past-release-notes-highlight.md", - "docs/source/_templates/Model-Deployment-Tutorial-Template.md", - "docs/source/_templates/Model-Deployment-Tutorial-Template.zh.md", - "docs/source/developer_guide/contribution/testing.md", - "docs/source/developer_guide/evaluation/using_ais_bench.md", - "docs/source/developer_guide/performance_and_debug/performance_benchmark.md", - "docs/source/installation.md", - "docs/source/tutorials/features/dynamic_chunked_pipeline_parallel.md", - "docs/source/tutorials/models/Cohere-Transcribe.md", - "docs/source/tutorials/models/DeepSeek-R1.md", - "docs/source/tutorials/models/DeepSeek-V3.1.md", - "docs/source/tutorials/models/DeepSeek-V3.2.md", - "docs/source/tutorials/models/DeepSeek-V4-Flash.md", - "docs/source/tutorials/models/DeepSeek-V4-Pro.md", - "docs/source/tutorials/models/DeepSeekOCR2.md", - "docs/source/tutorials/models/Dots3-Note.md", - "docs/source/tutorials/models/GLM4.x.md", - "docs/source/tutorials/models/GLM5.2.md", - "docs/source/tutorials/models/GLM5.md", - "docs/source/tutorials/models/Gemma4.md", - "docs/source/tutorials/models/Hy3-preview.md", - "docs/source/tutorials/models/InternVL3.5.md", - "docs/source/tutorials/models/Kimi-K2-Thinking.md", - "docs/source/tutorials/models/Kimi-K2.5.md", - "docs/source/tutorials/models/Kimi-K2.6.md", - "docs/source/tutorials/models/LLaVA-OneVision-Qwen2-0.5B-OV.md", - "docs/source/tutorials/models/MiniMax-M2.md", - "docs/source/tutorials/models/MiniMax-M3.md", - "docs/source/tutorials/models/PaddleOCR-VL.md", - "docs/source/tutorials/models/Qwen-VL-Dense.md", - "docs/source/tutorials/models/Qwen2.5-Math-RM-72B.md", - "docs/source/tutorials/models/Qwen3-235B-A22B.md", - "docs/source/tutorials/models/Qwen3-30B-A3B.md", - "docs/source/tutorials/models/Qwen3-ASR-1.7B.md", - "docs/source/tutorials/models/Qwen3-Coder-30B-A3B.md", - "docs/source/tutorials/models/Qwen3-Dense.md", - "docs/source/tutorials/models/Qwen3-Embedding.md", - "docs/source/tutorials/models/Qwen3-Next.md", - "docs/source/tutorials/models/Qwen3-Omni-30B-A3B-Thinking.md", - "docs/source/tutorials/models/Qwen3-Reranker.md", - "docs/source/tutorials/models/Qwen3-VL-235B-A22B-Instruct.md", - "docs/source/tutorials/models/Qwen3-VL-30B-A3B-Instruct.md", - "docs/source/tutorials/models/Qwen3-VL-Embedding.md", - "docs/source/tutorials/models/Qwen3-VL-Reranker.md", - "docs/source/tutorials/models/Qwen3.5-27B-Qwen3.6-27B.md", - "docs/source/tutorials/models/Qwen3.5-397B-A17B.md", - "docs/source/tutorials/models/Qwen3.5-Dense.md", - "docs/source/tutorials/models/Qwen3.6-35B-A3B.md", - "docs/source/tutorials/models/Qwen3.8-2.4T-A95B.md", - "docs/source/tutorials/models/Qwen3.8-27B.md", - "docs/source/tutorials/models/gpt-oss-120b.md", - "docs/source/user_guide/configuration/additional_config.md", - "docs/source/user_guide/feature_guide/kv_pool.md", - "docs/source/user_guide/release_notes.md" - ] - }, - { - "commit": "6958febef951f663fc54bc51403ffb9638d5a1fe", - "subject": "[Feature] add DFlash2 for MRV1 (#14533)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/misc/model_dataset_list.json", - ".github/workflows/scripts/test_config.yaml", - "tests/e2e/pull_request/two_card/spec_decode/test_spec_decode.py", - "tests/ut/spec_decode/test_dflash2_proposer.py", - "vllm_ascend/models/__init__.py", - "vllm_ascend/models/qwen3_dflash2.py", - "vllm_ascend/ops/triton/spec_decode/utils.py", - "vllm_ascend/spec_decode/__init__.py", - "vllm_ascend/spec_decode/dflash2_proposer.py", - "vllm_ascend/spec_decode/llm_base_proposer.py" - ] - }, - { - "commit": "4146cae0d0e567f9f66e2296294cd3eadbf9f1b9", - "subject": "[Feature]minimax-m3: integrate AscendC MSA index score (#14539)", - "topics": [ - "priority-models", - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "csrc/attention/msa_index_score/CMakeLists.txt", - "csrc/attention/msa_index_score/README.md", - "csrc/attention/msa_index_score/docs/aclnnMsaIndexScore.md", - "csrc/attention/msa_index_score/examples/test_aclnn_msa_index_score.cpp", - "csrc/attention/msa_index_score/msa_index_score_torch_adpt.h", - "csrc/attention/msa_index_score/op_host/CMakeLists.txt", - "csrc/attention/msa_index_score/op_host/msa_index_score_def.cpp", - "csrc/attention/msa_index_score/op_host/msa_index_score_infershape.cpp", - "csrc/attention/msa_index_score/op_host/msa_index_score_tiling.cpp", - "csrc/attention/msa_index_score/op_host/msa_index_score_tiling.h", - "csrc/attention/msa_index_score/op_kernel/arch22/msa_block_mmad.h", - "csrc/attention/msa_index_score/op_kernel/arch22/msa_index_score_epilogue.h", - "csrc/attention/msa_index_score/op_kernel/arch22/msa_index_score_kernel.h", - "csrc/attention/msa_index_score/op_kernel/arch22/msa_index_score_task.h", - "csrc/attention/msa_index_score/op_kernel/msa_index_score.cpp", - "csrc/attention/msa_index_score/op_kernel/msa_index_score_common.h", - "csrc/attention/msa_index_score/op_kernel/msa_index_score_debug.h", - "csrc/attention/msa_index_score/tests/CMakeLists.txt", - "csrc/attention/msa_index_score/tests/README.md", - "csrc/attention/msa_index_score/tests/golden/msa_index_score_golden.py", - "csrc/build_aclnn.sh", - "csrc/cmake/custom_build.cmake", - "csrc/cmake/func.cmake", - "csrc/torch_binding.cpp", - "csrc/torch_binding_meta.cpp", - "tests/ut/models/minimax_m3/test_msa_m3.py", - "vllm_ascend/models/minimax_m3/msa_m3.py", - "vllm_ascend/models/minimax_m3/ops/msa_m3_npu.py" - ] - }, - { - "commit": "657df3efae6479822970081636ffb9684f936d2f", - "subject": "[BugFix][SFA] Use DCP-sharded slot mapping in prolog v3 KV write (#14536)", - "topics": [ - "priority-models", - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/attention/sfa_v1.py" - ] - }, - { - "commit": "d0de46c794a301566288d8d70de7589505bcc8f8", - "subject": "[Feature] Support MTP with PP in MRV2 (#14184)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/patch/__init__.py", - "vllm_ascend/patch/worker/patch_v2/patch_spec_pp.py", - "vllm_ascend/worker/v2/model_runner.py" - ] - }, - { - "commit": "9794b38fa7f5938165fbd96d2299e83326c5b077", - "subject": "[CI] Add DeepSeek V4 MTP acceptance rate golden (#14522)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "tests/e2e/pull_request/four_card/model_runner_v2/test_deepseek_v4.py" - ] - }, - { - "commit": "bb4d264f8eeb8eff9073c2619153cbad4fe18b52", - "subject": "[Feature][MRV2] Support MTP with SFA in FullGraph mode (#14310)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "tests/e2e/pull_request/eight_card/model_runner_v2/test_glm5_2.py", - "tests/ut/spec_decode/test_eagle_aclgraph_source_regression.py", - "vllm_ascend/models/deepseek_mtp.py", - "vllm_ascend/patch/__init__.py", - "vllm_ascend/patch/worker/patch_v2/patch_eagle_speculator.py", - "vllm_ascend/worker/v2/README.md", - "vllm_ascend/worker/v2/aclgraph_utils.py", - "vllm_ascend/worker/v2/spec_decode/autoregressive/aclgraph.py", - "vllm_ascend/worker/v2/spec_decode/autoregressive/speculator.py" - ] - }, - { - "commit": "9a0cdb27a656520048675ea24a30b83c716a93e3", - "subject": "[Feature][MRV2] Support DSA prefill context parallelism (#14037)", - "topics": [ - "deepseek-v4-dspark", - "communication" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/attention/test_dsa_v1.py", - "tests/ut/models/test_deepseek_v4_indexer.py", - "tests/ut/worker/test_pcp_manager_v2.py", - "vllm_ascend/attention/context_parallel/dsa_cp.py", - "vllm_ascend/attention/dsa_v1.py", - "vllm_ascend/models/deepseek_v4/indexer.py", - "vllm_ascend/worker/v2/attn_utils.py", - "vllm_ascend/worker/v2/input_batch.py", - "vllm_ascend/worker/v2/model_runner.py", - "vllm_ascend/worker/v2/model_states/default.py", - "vllm_ascend/worker/v2/pcp_manager.py" - ] - }, - { - "commit": "014af4453bbea723bc6d002bf3365f086f512d83", - "subject": "[CI] add dynamic speculative decoding tests for deepseek-v4-flash (#14538)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "tests/e2e/pull_request/four_card/spec_decode/test_dspark_deepseekv4.py", - "vllm_ascend/models/deepseek_v4/dspark.py", - "vllm_ascend/models/qwen3_dspark.py", - "vllm_ascend/spec_decode/dspark_proposer.py", - "vllm_ascend/spec_decode/utils.py" - ] - }, - { - "commit": "9cf7a0a23944b032d512040f4da542777042cc3e", - "subject": "[Refactor][Config] Migrate AscendConfig to @config pydantic dataclass (#13838)", - "topics": [ - "communication", - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/workflows/scripts/test_config.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/DeepSeek-V3.1-BF16.yaml", - "tests/e2e/pull_request/two_card/test_deepseek_multistream_moe.py", - "tests/ut/compilation/test_graph_fusion_pass_manager.py", - "tests/ut/core/test_batch_job_aware_scheduler.py", - "tests/ut/core/test_profiling_chunk.py", - "tests/ut/model_loader/rfork/test_rfork_loader.py", - "tests/ut/quantization/test_utils.py", - "tests/ut/spec_decode/test_llm_base_proposer.py", - "tests/ut/test_ascend_config.py", - "tests/ut/test_platform.py", - "tests/ut/test_utils.py", - "vllm_ascend/ascend_config.py", - "vllm_ascend/compilation/graph_fusion_pass_manager.py", - "vllm_ascend/config_utils.py", - "vllm_ascend/core/dyntra_lb_scheduler.py", - "vllm_ascend/envs.py", - "vllm_ascend/model_loader/rfork/rfork_loader.py", - "vllm_ascend/patch/platform/patch_dyntra_lb_core.py", - "vllm_ascend/platform.py", - "vllm_ascend/quantization/utils.py", - "vllm_ascend/spec_decode/llm_base_proposer.py", - "vllm_ascend/utils.py" - ] - }, - { - "commit": "182b75ea84ff13ece27b57391331ef8935410aea", - "subject": "[Test] add memcache dsv4-flash weekly case (#14760)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/configs/weekly_config.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/DeepSeek-V4-Flash-Memcache-PD-acc.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/Qwen2.5-7B-Instruct-mooncake-PD.yaml" - ] - }, - { - "commit": "a348d4e23c131b8c2d3f4233cd910b340260d7c4", - "subject": "[BugFix][Model] Propagate DeepSeek V4 routed SwiGLU limit (#14398)", - "topics": [ - "deepseek-v4-dspark", - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/model_executor/test_deepseek_v4.py", - "vllm_ascend/models/deepseek_v4/model.py" - ] - }, - { - "commit": "175ef4d49c2072f9233619636f0309f4f88b4e2c", - "subject": "[CI] main2main vllm 0814 cdc4824 (#14131)", - "topics": [ - "deepseek-v4-dspark", - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/vllm-main-verified.commit", - "tests/e2e/pull_request/two_card/lora/test_qwen3moe_lora.py", - "tests/ut/kv_offload/test_native_cpu_offload.py", - "tests/ut/ops/test_routed_experts.py", - "tests/ut/patch/platform/test_patch_structured_output.py", - "tests/ut/patch/worker/test_patch_mamba_utils_source.py", - "tests/ut/patch/worker/test_patch_qwen3_5_mtp.py", - "tests/ut/spec_decode/test_extract_hidden_states_proposer.py", - "vllm_ascend/_310p/fused_moe/grouped_topk_router.py", - "vllm_ascend/__init__.py", - "vllm_ascend/models/qwen3_dspark.py", - "vllm_ascend/ops/fused_moe/fused_moe.py", - "vllm_ascend/ops/triton/mamba/postprocess.py", - "vllm_ascend/patch/__init__.py", - "vllm_ascend/patch/platform/patch_kv_cache_coordinator.py", - "vllm_ascend/patch/platform/patch_vision.py", - "vllm_ascend/patch/worker/patch_distributed.py", - "vllm_ascend/patch/worker/patch_qwen3_5.py", - "vllm_ascend/worker/v2/aclgraph_utils.py", - "vllm_ascend/worker/v2/input_batch.py", - "vllm_ascend/worker/v2/model_runner.py" - ] - }, - { - "commit": "92616f3a64e7218655c2d009aa8ecf855091e54d", - "subject": "[Refactor] remove FlashComm v1 (#13946)", - "topics": [ - "deepseek-v4-dspark", - "communication", - "priority-models", - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/workflows/scripts/run_selected_tests.sh", - ".github/workflows/scripts/test_config.yaml", - "docs/source/_templates/Model-Deployment-Tutorial-Template.md", - "docs/source/_templates/Model-Deployment-Tutorial-Template.zh.md", - "docs/source/developer_guide/Design_Documents/context_parallel.md", - "docs/source/developer_guide/contribution/multi_node_test.md", - "docs/source/faqs.md", - "docs/source/tutorials/features/pd_disaggregation_mooncake_multi_node.md", - "docs/source/tutorials/features/suffix_speculative_decoding.md", - "docs/source/tutorials/models/DeepSeek-V3.1.md", - "docs/source/tutorials/models/DeepSeek-V3.2.md", - "docs/source/tutorials/models/DeepSeek-V4-Flash.md", - "docs/source/tutorials/models/DeepSeek-V4-Pro.md", - "docs/source/tutorials/models/Dots3-Note.md", - "docs/source/tutorials/models/GLM4.x.md", - "docs/source/tutorials/models/GLM5.2.md", - "docs/source/tutorials/models/GLM5.md", - "docs/source/tutorials/models/InternVL3.5.md", - "docs/source/tutorials/models/Kimi-K2.5.md", - "docs/source/tutorials/models/Kimi-K2.6.md", - "docs/source/tutorials/models/MiniMax-M2.md", - "docs/source/tutorials/models/Mixtral-8x7B-Instruct-v0.1.md", - "docs/source/tutorials/models/Qwen3-235B-A22B.md", - "docs/source/tutorials/models/Qwen3-30B-A3B.md", - "docs/source/tutorials/models/Qwen3-Coder-30B-A3B.md", - "docs/source/tutorials/models/Qwen3-Dense.md", - "docs/source/tutorials/models/Qwen3-Omni-30B-A3B-Thinking.md", - "docs/source/tutorials/models/Qwen3-VL-235B-A22B-Instruct.md", - "docs/source/tutorials/models/Qwen3-VL-30B-A3B-Instruct.md", - "docs/source/tutorials/models/Qwen3.5-27B-Qwen3.6-27B.md", - "docs/source/tutorials/models/Qwen3.5-397B-A17B.md", - "docs/source/tutorials/models/Qwen3.6-35B-A3B.md", - "docs/source/tutorials/models/Qwen3.8-2.4T-A95B.md", - "docs/source/user_guide/configuration/additional_config.md", - "docs/source/user_guide/feature_guide/large_scale_ep.md", - "docs/source/user_guide/feature_guide/sequence_parallelism.md", - "docs/source/user_guide/support_matrix/feature_matrix.md", - "tests/e2e/coverage_taxonomy.py", - "tests/e2e/models/configs/Mixtral-8x7B-Instruct-v0.1.yaml", - "tests/e2e/nightly/multi_node/external_dp/config/GLM5_1-W8A8-EP-external.yaml", - "tests/e2e/nightly/multi_node/external_dp/config/Kimi-K2.6-W4A8-64k-1k-TPOT50-PD.yaml", - "tests/e2e/nightly/multi_node/external_dp/config/Minimax_m2.7_in128k_1k_prefix90_tpot50.yaml", - "tests/e2e/nightly/multi_node/external_dp/scripts/utils.py", - "tests/e2e/nightly/multi_node/internal_dp/config/DeepSeek-V3.1-BF16.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/DeepSeek-V3_2-W8A8-A3-dual-nodes.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/DeepSeek-V3_2-W8A8-EP.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/GLM5_1-W8A8-EP.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/Kimi-K2_5-W4A8-A2-dual-nodes.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/Qwen3-235B-disagg-pd.yaml", - "tests/e2e/nightly/multi_node/internal_dp/scripts/test_multi_node.py", - "tests/e2e/nightly/single_node/models/configs/DeepSeek-V3.2-W8A8-DCP.yaml", - "tests/e2e/nightly/single_node/models/configs/DeepSeek-V3.2-W8A8.yaml", - "tests/e2e/nightly/single_node/models/configs/DeepSeek-V4-Flash-W8A8-A3.yaml", - "tests/e2e/nightly/single_node/models/configs/GLM-4.7.yaml", - "tests/e2e/nightly/single_node/models/configs/GLM-5.1-W8A8-PrefillMC2.yaml", - "tests/e2e/nightly/single_node/models/configs/Kimi-K2.5.yaml", - "tests/e2e/nightly/single_node/models/configs/MiniMax-M2.5-w8a8-QuaRot-A2.yaml", - "tests/e2e/nightly/single_node/models/configs/MiniMax-M2.5-w8a8-QuaRot-A3.yaml", - "tests/e2e/nightly/single_node/models/configs/MiniMax-M3-BF16-A3.yaml", - "tests/e2e/nightly/single_node/models/configs/MiniMax-M3-W8A8-A3.yaml", - "tests/e2e/nightly/single_node/models/configs/Qwen3-235B-A22B-W8A8.yaml", - "tests/e2e/nightly/single_node/models/configs/Qwen3-32B-Int8-A2.yaml", - "tests/e2e/nightly/single_node/models/configs/Qwen3-32B-Int8.yaml", - "tests/e2e/nightly/single_node/models/configs/Qwen3-32B-W8A8C8-A3.yaml", - "tests/e2e/nightly/single_node/models/configs/Qwen3-VL-235B-A22B-Instruct-W8A8.yaml", - "tests/e2e/nightly/single_node/models/configs/Qwen3-VL-32B-Instruct-W8A8.yaml", - "tests/e2e/nightly/single_node/models/configs/Qwen3.5-122B-A10B-W8A8-A3.yaml", - "tests/e2e/nightly/single_node/models/scripts/test_single_node.py", - "tests/e2e/pull_request/eight_card/test_minimax_m3.py", - "tests/e2e/pull_request/four_card/context_parallel/test_accuracy.py", - "tests/e2e/pull_request/four_card/spec_decode/test_dspark_deepseekv4.py", - "tests/e2e/pull_request/four_card/test_deepseek_v3_2_w8a8_pruning.py", - "tests/e2e/pull_request/four_card/test_deepseek_v4.py", - "tests/e2e/pull_request/four_card/test_graph_mode.py", - "tests/e2e/pull_request/four_card/test_profiling_chunk_performance.py", - "tests/e2e/pull_request/four_card/test_qwen3_5.py", - "tests/e2e/pull_request/four_card/test_qwen3_next.py", - "tests/e2e/pull_request/four_card/test_sequence_parallel_linear.py", - "tests/e2e/pull_request/one_card/compile/test_graphex_norm_quant_fusion.py", - "tests/e2e/pull_request/one_card/compile/test_norm_quant_fusion.py", - "tests/e2e/pull_request/two_card/lora/test_qwen3moe_lora.py", - "tests/e2e/pull_request/two_card/spec_decode/test_spec_decode.py", - "tests/e2e/pull_request/two_card/test_flashcomm_distributed.py", - "tests/e2e/pull_request/two_card/test_sequence_parallelism_moe.py", - "tests/e2e/pull_request/two_card/test_shared_expert_dp.py", - "tests/e2e/pull_request/two_card/test_sp_pass.py", - "tests/e2e/weekly/multi_node/external_dp/config/DeepSeek-V4-Pro-w4a8-PD-no-prefix.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/DeepSeek-V4-Pro-w4a8-prefix-cache-PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/DeepSeek-V4-flash-w8a8-PD-no-prefix.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/DeepSeek-V4-flash-w8a8-PD-prefix-weekly.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/DeepSeek_V3.1T_layerwise_PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/DeepSeek_V3.2T_MTP2_PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/DeepSeek_V3.2T_MTP3_PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/GLM_5_1_PD_in32k_bs16-0.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/GLM_5_1_PD_in32k_bs20-90.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/GLM_5_1_PD_in64k_bs16-90.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/Kimi-K2.5-W4A8-128k-1k-TPOT50.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/Kimi-K2.5-W4A8-16k-1k-TPOT50.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/Kimi-K2.6-W4A8-128k-1k-TPOT50-PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/Kimi-K2.6-W4A8-254k-1k-TPOT20-PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/Kimi-K2.6-W4A8-3.5k-1.5k-TPOT50-PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/Kimi-K2.7-W4A8-128k-1k-TPOT50-PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/Kimi-K2.7-W4A8-254k-1k-TPOT20-PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/Kimi-K2.7-W4A8-3.5k-1.5k-TPOT50-PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/Kimi-K2.7-W4A8-64k-1k-TPOT50-PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/MiniMax-PD-in32k-bs4-1.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/Minimax_m2.7_in3_5_tpot20.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/Minimax_m2.7_in3_5_tpot50.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/Minimax_m2.7_in64k_1k_prefix90_tpot50.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/QWEN3_235B_PD_3_5K_1_5k.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/Qwen-3.5-397B-A17B-W8A8-PD.yaml", - "tests/e2e/weekly/multi_node/internal_dp/config/DeepSeek-V3_2-W8A8-EP_weekly.yaml", - "tests/e2e/weekly/single_node/configs/DeepSeek-V3.2-W8A8_A3_weekly.yaml", - "tests/e2e/weekly/single_node/configs/Kimi-K2.7-w4a8-A3.yaml", - "tests/e2e/weekly/single_node/configs/MiniMax-M2.5-W8A8-A3.yaml", - "tests/e2e/weekly/single_node/configs/Qwen3-235B-A22B-W8A8_piecewise_fullgraph_A3.yaml", - "tests/e2e/weekly/single_node/configs/Qwen3.5-122B-A10B-W8A8-A2.yaml", - "tests/e2e/weekly/single_node/configs/Qwen3.5-122B-A10B-W8A8-A3.yaml", - "tests/e2e/weekly/single_node/configs/Qwen3.6-35B-A3B-w4a8-A3.yaml", - "tests/e2e/weekly/single_node/configs/Qwen3.6-35B-A3B-w8a8-A3-accuracy.yaml", - "tests/e2e/weekly/single_node/configs/Qwen3_235b_w8a8_A5.yaml", - "tests/e2e/weekly/single_node/configs/Qwq_32b_A3.yaml", - "tests/e2e/weekly/single_node/models/test_qwen3_30b_acc.py", - "tests/ut/_310p/test_model_runner_310p.py", - "tests/ut/attention/a2/test_attention_v1.py", - "tests/ut/attention/a2/test_attention_v1_precision.py", - "tests/ut/attention/a2/test_mla_precision.py", - "tests/ut/attention/a2/test_mla_v1.py", - "tests/ut/attention/test_dsa_cp_o_proj_tp.py", - "tests/ut/attention/test_sfa_o_proj_tp.py", - "tests/ut/ops/test_fused_moe.py", - "tests/ut/ops/test_linear.py", - "tests/ut/ops/test_mla.py", - "tests/ut/ops/test_moe_mlp.py", - "tests/ut/ops/test_prepare_finalize.py", - "tests/ut/ops/test_register_custom_ops.py", - "tests/ut/ops/test_rotary_embedding.py", - "tests/ut/ops/test_vocab_parallel_embedding.py", - "tests/ut/spec_decode/a2/test_eagle_proposer.py", - "tests/ut/spec_decode/test_llm_base_proposer.py", - "tests/ut/spec_decode/test_speculators_vwn_eagle3.py", - "tests/ut/test_ascend_config.py", - "tests/ut/test_ascend_forward_context.py", - "tests/ut/test_platform.py", - "tests/ut/test_utils.py", - "tests/ut/worker/test_model_runner_v2_flashcomm.py", - "vllm_ascend/_310p/model_runner_310p.py", - "vllm_ascend/ascend_config.py", - "vllm_ascend/ascend_forward_context.py", - "vllm_ascend/attention/attention_v1.py", - "vllm_ascend/attention/context_parallel/dsa_cp.py", - "vllm_ascend/attention/context_parallel/sfa_cp.py", - "vllm_ascend/attention/dsa_v1.py", - "vllm_ascend/attention/mla_v1.py", - "vllm_ascend/attention/sfa_kv_offload.py", - "vllm_ascend/attention/sfa_v1.py", - "vllm_ascend/compilation/graph_fusion_pass_manager.py", - "vllm_ascend/compilation/passes/norm_quant_fusion_pass.py", - "vllm_ascend/compilation/passes/sequence_parallelism.py", - "vllm_ascend/compilation/passes/sequence_parallelism_moe.py", - "vllm_ascend/envs.py", - "vllm_ascend/models/deepseek_v4/model.py", - "vllm_ascend/models/minimax_m3/minimax_m3_vl.py", - "vllm_ascend/ops/dsa.py", - "vllm_ascend/ops/fused_moe/fused_moe.py", - "vllm_ascend/ops/fused_moe/prepare_finalize.py", - "vllm_ascend/ops/fused_moe/routed_experts.py", - "vllm_ascend/ops/fused_moe/router/fused_topk_router.py", - "vllm_ascend/ops/fused_moe/shared_experts.py", - "vllm_ascend/ops/layernorm.py", - "vllm_ascend/ops/linear.py", - "vllm_ascend/ops/linear_op.py", - "vllm_ascend/ops/mla.py", - "vllm_ascend/ops/register_custom_ops.py", - "vllm_ascend/ops/rotary_embedding.py", - "vllm_ascend/ops/vocab_parallel_embedding.py", - "vllm_ascend/patch/__init__.py", - "vllm_ascend/patch/worker/patch_qwen3_5.py", - "vllm_ascend/patch/worker/patch_qwen3vl.py", - "vllm_ascend/platform.py", - "vllm_ascend/spec_decode/llm_base_proposer.py", - "vllm_ascend/spec_decode/utils.py", - "vllm_ascend/utils.py", - "vllm_ascend/worker/model_runner_v1.py", - "vllm_ascend/worker/v2/model_runner.py", - "vllm_ascend/worker/v2/sp_utils.py", - "vllm_ascend/worker/worker.py" - ] - }, - { - "commit": "4ce367a7d12db55c3dbe9b670eff52b2e14b3b9a", - "subject": "[BugFix][MRV2] Fix DSpark spec decoding on model runner v2 (#14696)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/attention/dsa_v1.py", - "vllm_ascend/models/deepseek_v4/model.py" - ] - }, - { - "commit": "d8268faea90dd9b999a0a82f6d133751d57b5770", - "subject": "[Test][Feature] Add GLM-5.2 with DCP E2E test configuration (#14719)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/configs/weekly_config.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/GLM-5.2-W4A8C8-1M-PD-DCP.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/Minimax_m2.7_in3_5_tpot20.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/Minimax_m2.7_in3_5_tpot50.yaml" - ] - }, - { - "commit": "7f42a9b449a0816e247e95a0d3914bb9c302f2ac", - "subject": "[Doc] Translated Doc files 2026-08-19 (#14523)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/locale/zh_CN/LC_MESSAGES/community/contributors.po", - "docs/source/locale/zh_CN/LC_MESSAGES/community/governance.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/Design_Documents/layerwise_and_sparse_kv_cache_offloading.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/DeepSeek-V3.1.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/GLM5.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Gemma4.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/InternVL3.5.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/MiniMax-M3.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-Dense.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-VL-235B-A22B-Instruct.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-VL-30B-A3B-Instruct.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3.5-27B-Qwen3.6-27B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3.5-397B-A17B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3.6-35B-A3B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/layerwise_and_sparse_kv_cache_offloading.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/lora.po" - ] - }, - { - "commit": "8548ed23a3b4a19a40d2638b7b86e1f1abc0fa96", - "subject": "[Feature][EPLB] Eplb supports Mega MoE (#13994)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/quantization/methods/w4a8.py", - "vllm_ascend/quantization/methods/w8a8_dynamic.py" - ] - }, - { - "commit": "dade7da62203343a30e20d8249dbfa9aa0635104", - "subject": "[BugFix]Fix dsv4 quant name issue (#14490)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/models/deepseek_v4/mtp.py" - ] - }, - { - "commit": "d61d083f252848735bec9b7fa5ae86e0f3922ffd", - "subject": "[Refactor] Remove MiniMax-M2 linear attn forward_qk patch (#13992)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/patch/__init__.py", - "vllm_ascend/patch/worker/__init__.py", - "vllm_ascend/patch/worker/patch_minimax_m2_linear_attn.py" - ] - }, - { - "commit": "1acbcc36c30717a233c26a032038089533f6e9d8", - "subject": "[Feature][Model] Support MiniMax M3 MXFP8 inference on Ascend 950 (#14412)", - "topics": [ - "priority-models", - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "csrc/attention/k2q_csr/CMakeLists.txt", - "csrc/attention/k2q_csr/README.md", - "csrc/attention/k2q_csr/k2q_csr_common/op_host/arch35/k2q_csr_tiling_arch35.cpp", - "csrc/attention/k2q_csr/k2q_csr_common/op_host/arch35/k2q_csr_tiling_arch35.h", - "csrc/attention/k2q_csr/k2q_csr_common/op_host/k2q_csr_tiling_common.h", - "csrc/attention/k2q_csr/k2q_csr_common/op_kernel/arch35/k2q_csr_simt_common_arch35.h", - "csrc/attention/k2q_csr/k2q_csr_common/op_kernel/arch35/k2q_csr_simt_hist_arch35.h", - "csrc/attention/k2q_csr/k2q_csr_common/op_kernel/arch35/k2q_csr_simt_scatter_arch35.h", - "csrc/attention/k2q_csr/k2q_csr_common/op_kernel/k2q_csr_common.h", - "csrc/attention/k2q_csr/k2q_csr_common/op_kernel/k2q_csr_mc.h", - "csrc/attention/k2q_csr/k2q_csr_common/op_kernel/k2q_csr_prefix.h", - "csrc/attention/k2q_csr/k2q_csr_common/op_kernel/k2q_csr_scratch.h", - "csrc/attention/k2q_csr/k2q_csr_common/op_kernel/k2q_csr_stage_meta.h", - "csrc/attention/k2q_csr/k2q_csr_common/op_kernel/k2q_csr_tiling.h", - "csrc/attention/k2q_csr/k2q_csr_hist/op_host/CMakeLists.txt", - "csrc/attention/k2q_csr/k2q_csr_hist/op_host/k2q_csr_hist_def.cpp", - "csrc/attention/k2q_csr/k2q_csr_hist/op_host/k2q_csr_hist_infershape.cpp", - "csrc/attention/k2q_csr/k2q_csr_hist/op_host/k2q_csr_hist_tiling.cpp", - "csrc/attention/k2q_csr/k2q_csr_hist/op_kernel/common/arch35/k2q_csr_simt_common_arch35.h", - "csrc/attention/k2q_csr/k2q_csr_hist/op_kernel/common/arch35/k2q_csr_simt_hist_arch35.h", - "csrc/attention/k2q_csr/k2q_csr_hist/op_kernel/common/arch35/k2q_csr_simt_scatter_arch35.h", - "csrc/attention/k2q_csr/k2q_csr_hist/op_kernel/common/k2q_csr_common.h", - "csrc/attention/k2q_csr/k2q_csr_hist/op_kernel/common/k2q_csr_mc.h", - "csrc/attention/k2q_csr/k2q_csr_hist/op_kernel/common/k2q_csr_prefix.h", - "csrc/attention/k2q_csr/k2q_csr_hist/op_kernel/common/k2q_csr_scratch.h", - "csrc/attention/k2q_csr/k2q_csr_hist/op_kernel/common/k2q_csr_stage_meta.h", - "csrc/attention/k2q_csr/k2q_csr_hist/op_kernel/common/k2q_csr_tiling.h", - "csrc/attention/k2q_csr/k2q_csr_hist/op_kernel/k2q_csr_hist.cpp", - "csrc/attention/k2q_csr/k2q_csr_hist/op_kernel/k2q_csr_hist_apt.cpp", - "csrc/attention/k2q_csr/k2q_csr_meta/op_host/CMakeLists.txt", - "csrc/attention/k2q_csr/k2q_csr_meta/op_host/k2q_csr_meta_def.cpp", - "csrc/attention/k2q_csr/k2q_csr_meta/op_host/k2q_csr_meta_infershape.cpp", - "csrc/attention/k2q_csr/k2q_csr_meta/op_host/k2q_csr_meta_tiling.cpp", - "csrc/attention/k2q_csr/k2q_csr_meta/op_kernel/common/arch35/k2q_csr_simt_common_arch35.h", - "csrc/attention/k2q_csr/k2q_csr_meta/op_kernel/common/arch35/k2q_csr_simt_hist_arch35.h", - "csrc/attention/k2q_csr/k2q_csr_meta/op_kernel/common/arch35/k2q_csr_simt_scatter_arch35.h", - "csrc/attention/k2q_csr/k2q_csr_meta/op_kernel/common/k2q_csr_common.h", - "csrc/attention/k2q_csr/k2q_csr_meta/op_kernel/common/k2q_csr_mc.h", - "csrc/attention/k2q_csr/k2q_csr_meta/op_kernel/common/k2q_csr_prefix.h", - "csrc/attention/k2q_csr/k2q_csr_meta/op_kernel/common/k2q_csr_scratch.h", - "csrc/attention/k2q_csr/k2q_csr_meta/op_kernel/common/k2q_csr_stage_meta.h", - "csrc/attention/k2q_csr/k2q_csr_meta/op_kernel/common/k2q_csr_tiling.h", - "csrc/attention/k2q_csr/k2q_csr_meta/op_kernel/k2q_csr_meta.cpp", - "csrc/attention/k2q_csr/k2q_csr_meta/op_kernel/k2q_csr_meta_apt.cpp", - "csrc/attention/k2q_csr/k2q_csr_row_prefix/op_host/CMakeLists.txt", - "csrc/attention/k2q_csr/k2q_csr_row_prefix/op_host/k2q_csr_row_prefix_def.cpp", - "csrc/attention/k2q_csr/k2q_csr_row_prefix/op_host/k2q_csr_row_prefix_infershape.cpp", - "csrc/attention/k2q_csr/k2q_csr_row_prefix/op_host/k2q_csr_row_prefix_tiling.cpp", - "csrc/attention/k2q_csr/k2q_csr_row_prefix/op_kernel/common/arch35/k2q_csr_simt_common_arch35.h", - "csrc/attention/k2q_csr/k2q_csr_row_prefix/op_kernel/common/arch35/k2q_csr_simt_hist_arch35.h", - "csrc/attention/k2q_csr/k2q_csr_row_prefix/op_kernel/common/arch35/k2q_csr_simt_scatter_arch35.h", - "csrc/attention/k2q_csr/k2q_csr_row_prefix/op_kernel/common/k2q_csr_common.h", - "csrc/attention/k2q_csr/k2q_csr_row_prefix/op_kernel/common/k2q_csr_mc.h", - "csrc/attention/k2q_csr/k2q_csr_row_prefix/op_kernel/common/k2q_csr_prefix.h", - "csrc/attention/k2q_csr/k2q_csr_row_prefix/op_kernel/common/k2q_csr_scratch.h", - "csrc/attention/k2q_csr/k2q_csr_row_prefix/op_kernel/common/k2q_csr_stage_meta.h", - "csrc/attention/k2q_csr/k2q_csr_row_prefix/op_kernel/common/k2q_csr_tiling.h", - "csrc/attention/k2q_csr/k2q_csr_row_prefix/op_kernel/k2q_csr_row_prefix.cpp", - "csrc/attention/k2q_csr/k2q_csr_row_prefix/op_kernel/k2q_csr_row_prefix_apt.cpp", - "csrc/attention/k2q_csr/k2q_csr_scatter/op_host/CMakeLists.txt", - "csrc/attention/k2q_csr/k2q_csr_scatter/op_host/k2q_csr_scatter_def.cpp", - "csrc/attention/k2q_csr/k2q_csr_scatter/op_host/k2q_csr_scatter_infershape.cpp", - "csrc/attention/k2q_csr/k2q_csr_scatter/op_host/k2q_csr_scatter_tiling.cpp", - "csrc/attention/k2q_csr/k2q_csr_scatter/op_kernel/common/arch35/k2q_csr_simt_common_arch35.h", - "csrc/attention/k2q_csr/k2q_csr_scatter/op_kernel/common/arch35/k2q_csr_simt_hist_arch35.h", - "csrc/attention/k2q_csr/k2q_csr_scatter/op_kernel/common/arch35/k2q_csr_simt_scatter_arch35.h", - "csrc/attention/k2q_csr/k2q_csr_scatter/op_kernel/common/k2q_csr_common.h", - "csrc/attention/k2q_csr/k2q_csr_scatter/op_kernel/common/k2q_csr_mc.h", - "csrc/attention/k2q_csr/k2q_csr_scatter/op_kernel/common/k2q_csr_prefix.h", - "csrc/attention/k2q_csr/k2q_csr_scatter/op_kernel/common/k2q_csr_scratch.h", - "csrc/attention/k2q_csr/k2q_csr_scatter/op_kernel/common/k2q_csr_stage_meta.h", - "csrc/attention/k2q_csr/k2q_csr_scatter/op_kernel/common/k2q_csr_tiling.h", - "csrc/attention/k2q_csr/k2q_csr_scatter/op_kernel/k2q_csr_scatter.cpp", - "csrc/attention/k2q_csr/k2q_csr_scatter/op_kernel/k2q_csr_scatter_apt.cpp", - "csrc/attention/k2q_csr/k2q_csr_tile_prefix/op_host/CMakeLists.txt", - "csrc/attention/k2q_csr/k2q_csr_tile_prefix/op_host/k2q_csr_tile_prefix_def.cpp", - "csrc/attention/k2q_csr/k2q_csr_tile_prefix/op_host/k2q_csr_tile_prefix_infershape.cpp", - "csrc/attention/k2q_csr/k2q_csr_tile_prefix/op_host/k2q_csr_tile_prefix_tiling.cpp", - "csrc/attention/k2q_csr/k2q_csr_tile_prefix/op_kernel/common/arch35/k2q_csr_simt_common_arch35.h", - "csrc/attention/k2q_csr/k2q_csr_tile_prefix/op_kernel/common/arch35/k2q_csr_simt_hist_arch35.h", - "csrc/attention/k2q_csr/k2q_csr_tile_prefix/op_kernel/common/arch35/k2q_csr_simt_scatter_arch35.h", - "csrc/attention/k2q_csr/k2q_csr_tile_prefix/op_kernel/common/k2q_csr_common.h", - "csrc/attention/k2q_csr/k2q_csr_tile_prefix/op_kernel/common/k2q_csr_mc.h", - "csrc/attention/k2q_csr/k2q_csr_tile_prefix/op_kernel/common/k2q_csr_prefix.h", - "csrc/attention/k2q_csr/k2q_csr_tile_prefix/op_kernel/common/k2q_csr_scratch.h", - "csrc/attention/k2q_csr/k2q_csr_tile_prefix/op_kernel/common/k2q_csr_stage_meta.h", - "csrc/attention/k2q_csr/k2q_csr_tile_prefix/op_kernel/common/k2q_csr_tiling.h", - "csrc/attention/k2q_csr/k2q_csr_tile_prefix/op_kernel/k2q_csr_tile_prefix.cpp", - "csrc/attention/k2q_csr/k2q_csr_tile_prefix/op_kernel/k2q_csr_tile_prefix_apt.cpp", - "csrc/attention/k2q_csr/k2q_csr_torch_adpt.h", - "csrc/attention/sparse_attention_score_prefill/CMakeLists.txt", - "csrc/attention/sparse_attention_score_prefill/sparse_attention_score_prefill_torch_adpt.h", - "csrc/build_aclnn.sh", - "csrc/torch_binding.cpp", - "csrc/torch_binding_meta.cpp", - "tests/ut/models/minimax_m3/test_minimax_m3.py", - "tests/ut/models/minimax_m3/test_msa_m3.py", - "tests/ut/ops/test_moe_mlp.py", - "tests/ut/quantization/methods/test_w8a8_mxfp8.py", - "tests/ut/quantization/test_modelopt_mxfp8_config.py", - "tests/ut/quantization/test_utils.py", - "vllm_ascend/device/device_op.py", - "vllm_ascend/models/minimax_m3/minimax_m3.py", - "vllm_ascend/models/minimax_m3/msa_m3.py", - "vllm_ascend/models/minimax_m3/ops/msa_m3_npu.py", - "vllm_ascend/models/minimax_m3/ops/msa_m3_triton_a5.py", - "vllm_ascend/ops/fused_moe/moe_mlp.py", - "vllm_ascend/ops/fused_moe/prepare_finalize.py", - "vllm_ascend/platform.py", - "vllm_ascend/quantization/__init__.py", - "vllm_ascend/quantization/methods/w8a8_mxfp8.py", - "vllm_ascend/quantization/modelopt_mxfp8_config.py", - "vllm_ascend/quantization/utils.py" - ] - }, - { - "commit": "749bb6c82d2aed0d54f7eb898f33ea5a8169ba2e", - "subject": "[Refactor][Feature] Remove compress attention manager for DeepSeek V4 (#13242)", - "topics": [ - "deepseek-v4-dspark", - "communication" - ], - "runtime_patch_match": false, - "paths": [ - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_compressor_metadata.py", - "tests/ut/attention/test_dsa_v1.py", - "tests/ut/attention/test_dsv4_block_geometry.py", - "tests/ut/distributed/ascend_store/test_coordinator.py", - "tests/ut/distributed/ascend_store/test_kv_transfer.py", - "tests/ut/distributed/ascend_store/test_metadata.py", - "tests/ut/kv_offload/test_mooncake_hybrid_connector.py", - "tests/ut/models/test_deepseek_v4_compressor.py", - "tests/ut/patch/platform/test_prefix_cache_cp_patches.py", - "tests/ut/test_compressed_prefix_cache.py", - "tests/ut/worker/a2/test_block_table.py", - "tests/ut/worker/a2/test_model_runner_v1.py", - "tests/ut/worker/test_attn_utils_v2.py", - "vllm_ascend/attention/context_parallel/dsa_cp.py", - "vllm_ascend/attention/dsa_v1.py", - "vllm_ascend/core/kv_cache_interface.py", - "vllm_ascend/core/single_type_kv_cache_manager.py", - "vllm_ascend/distributed/kv_transfer/kv_p2p/mooncake_hybrid_connector.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/ascend_store/coordinator.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/ascend_store/kv_transfer.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/ascend_store/metadata.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/ascend_store/pool_scheduler.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/ascend_store/pool_worker.py", - "vllm_ascend/models/deepseek_v4/compressor.py", - "vllm_ascend/models/deepseek_v4/indexer.py", - "vllm_ascend/models/deepseek_v4/model.py", - "vllm_ascend/models/layer/attention/layer.py", - "vllm_ascend/patch/platform/patch_kv_cache_coordinator.py", - "vllm_ascend/patch/platform/patch_kv_cache_utils.py", - "vllm_ascend/spec_decode/llm_base_proposer.py", - "vllm_ascend/spec_decode/step3p5.py", - "vllm_ascend/worker/block_table.py", - "vllm_ascend/worker/model_runner_v1.py", - "vllm_ascend/worker/v2/attn_utils.py" - ] - }, - { - "commit": "59424918011114079869052d84155682325f032b", - "subject": "[Feature][MRV2] Support DSV4 Dspark (#12968)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "tests/e2e/pull_request/eight_card/model_runner_v2/__init__.py", - "tests/e2e/pull_request/four_card/model_runner_v2/test_deepseek_v4.py", - "tests/ut/attention/test_dsa_v1.py", - "vllm_ascend/attention/dsa_v1.py", - "vllm_ascend/models/deepseek_v4/dspark.py", - "vllm_ascend/patch/__init__.py", - "vllm_ascend/patch/worker/__init__.py", - "vllm_ascend/patch/worker/patch_v2/patch_dspark.py", - "vllm_ascend/worker/v2/attn_utils.py", - "vllm_ascend/worker/v2/spec_decode/autoregressive/speculator.py", - "vllm_ascend/worker/v2/spec_decode/dspark/speculator.py" - ] - }, - { - "commit": "98f2ce96c6afb7c3badf240299b129a783813d76", - "subject": "[v0.23.0][Doc][Misc] Sync v0.23.0 release docs to main (#14379)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/schedule_image_build_and_push.yaml", - ".github/workflows/schedule_release_code_and_wheel.yml", - "README.md", - "README.zh.md", - "docs/source/community/contributors.md", - "docs/source/community/versioning_policy.md", - "docs/source/faqs.md", - "docs/source/user_guide/release_notes.md", - "mkdocs.yml" - ] - }, - { - "commit": "67b1ac5e7a2f0d98606c720d500cdb3f8d675054", - "subject": "[CI] update nightly cases perf baseline (#14553)", - "topics": [ - "deepseek-v4-dspark", - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "tests/e2e/nightly/multi_node/internal_dp/config/DeepSeek-V3.1-BF16.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/DeepSeek-V3_2-W8A8-A3-dual-nodes.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/DeepSeek-V3_2-W8A8-EP.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/GLM5_1-W8A8-A2-dual-nodes.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/GLM5_1-W8A8-A3-dual-nodes.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/GLM5_2-W8A8-A3-dual-nodes.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/Kimi-K2_5-W4A8-A2-dual-nodes.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/Qwen3-235B-A22B-A2.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/Qwen3-235B-A22B.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/Qwen3-235B-disagg-pd.yaml", - "tests/e2e/nightly/single_node/models/configs/DeepSeek-R1-0528-W8A8.yaml", - "tests/e2e/nightly/single_node/models/configs/DeepSeek-V3.2-W8A8.yaml", - "tests/e2e/nightly/single_node/models/configs/DeepSeek-V4-Flash-W8A8-A3.yaml", - "tests/e2e/nightly/single_node/models/configs/GLM-5.1-W8A8-PrefillMC2.yaml", - "tests/e2e/nightly/single_node/models/configs/Kimi-K2-Thinking.yaml", - "tests/e2e/nightly/single_node/models/configs/Kimi-K2.5.yaml", - "tests/e2e/nightly/single_node/models/configs/Prefix-Cache-DeepSeek-R1-0528-W8A8.yaml", - "tests/e2e/nightly/single_node/models/configs/Prefix-Cache-Qwen3-32B-Int8.yaml", - "tests/e2e/nightly/single_node/models/configs/Qwen3-30B-A3B-W8A8.yaml", - "tests/e2e/nightly/single_node/models/configs/Qwen3-30B-QuaRot-eagle3.yaml", - "tests/e2e/nightly/single_node/models/configs/Qwen3-32B-Int8-A2.yaml", - "tests/e2e/nightly/single_node/models/configs/Qwen3-32B-Int8.yaml", - "tests/e2e/nightly/single_node/models/configs/Qwen3-32B-QuaRot-eagle3.yaml", - "tests/e2e/nightly/single_node/models/configs/Qwen3.5-122B-A10B-W8A8-A3.yaml", - "tests/e2e/nightly/single_node/models/configs/Qwen3.5-27B-w8a8-A2.yaml", - "tests/e2e/nightly/single_node/models/configs/Qwen3.5-27B-w8a8-A3.yaml", - "tests/e2e/nightly/single_node/models/configs/Qwen3.5-397B-A17B-W8A8-mtp-A3.yaml", - "tests/e2e/nightly/single_node/models/configs/Qwen3.5-397B-A17B-w4a8-mtp-A2.yaml", - "tests/e2e/weekly/multi_node/internal_dp/config/DeepSeek-V3_2-W8A8-EP_weekly.yaml" - ] - }, - { - "commit": "424e27e1fd2b1c6e0d7fe659b489b87c1223a33c", - "subject": "[MRV2][Feature][Bugfix] Support and fix Top-K/Top-P Triton kernel on Ascend (#13847)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_apply_top_k_top_p_triton.py", - "vllm_ascend/ops/triton/v2/__init__.py", - "vllm_ascend/ops/triton/v2/sample/__init__.py", - "vllm_ascend/ops/triton/v2/sample/apply_top_k_top_p_triton.py", - "vllm_ascend/ops/triton/v2/sample/docs/apply_top_k_top_p.md", - "vllm_ascend/patch/worker/patch_v2/patch_triton.py" - ] - }, - { - "commit": "57e6769562663c2ed05d206fd6eb03b5b8a331cd", - "subject": "[BugFix] allocate MTP hidden buffer only for speculative decoding (#14250)", - "topics": [ - "deepseek-v4-dspark", - "communication" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/models/deepseek_v4/model.py" - ] - }, - { - "commit": "34fce3eacef678dc875ad55db8782beae4d1b460", - "subject": "[Ops][Feature] Enhance batch-invariant Triton ops and add comprehensive precision tests (#14331)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_batch_invariant_ops.py", - "vllm_ascend/ops/triton/batch_invariant/matmul.py", - "vllm_ascend/ops/triton/batch_invariant/softmax.py" - ] - }, - { - "commit": "f61a694c067ac0dbc67cdc876b64a2d313c062a6", - "subject": "[Refactor] Remove MiniMax-M2 fp8 dequant load_weights patch (#13782)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/patch/worker/patch_minimax_m2.py" - ] - }, - { - "commit": "2c870a615782faa455f7d167d8f509d6cb250bc0", - "subject": "[Refactor] Remove MiniMax-M2 MoE fp32 gate patch (#13789)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/patch/worker/patch_minimax_m2.py" - ] - }, - { - "commit": "78878c2968164b3ce14591f31e1ff98f84a4aaeb", - "subject": "[Refactor] Remove MiniMax-M2 linear attn __init__/weight_loader patches (#13800)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/patch/worker/patch_minimax_m2_linear_attn.py" - ] - }, - { - "commit": "27a94764b5ead50ed3e42ab52a257c2173032750", - "subject": "[Bugfix][MRV2] Support probabilistic rejection sampling for spec decode (#13470)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/patch/worker/patch_v2/patch_triton.py", - "vllm_ascend/worker/v2/sample/gumbel.py", - "vllm_ascend/worker/v2/spec_decode/dflash/speculator.py", - "vllm_ascend/worker/v2/spec_decode/dspark/speculator.py", - "vllm_ascend/worker/v2/spec_decode/rejection_sampler_utils.py" - ] - }, - { - "commit": "99131964c7be294a66d25e66ff4c1d1a0ac58444", - "subject": "[Refactor][DSA][4/N] modularize the indexer/compressor of dsa_v1 (#14375)", - "topics": [ - "deepseek-v4-dspark", - "communication", - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/workflows/scripts/test_config.yaml", - "tests/ut/attention/test_dsa_cp.py", - "tests/ut/attention/test_dsa_v1.py", - "tests/ut/models/test_deepseek_v4_compressor.py", - "tests/ut/models/test_deepseek_v4_indexer.py", - "tests/ut/models/test_deepseek_v4_moe.py", - "tests/ut/test_ascend_forward_context.py", - "tests/ut/worker/test_attn_utils_v2.py", - "vllm_ascend/attention/context_parallel/dsa_cp.py", - "vllm_ascend/attention/dsa_v1.py", - "vllm_ascend/models/__init__.py", - "vllm_ascend/models/deepseek_v4/__init__.py", - "vllm_ascend/models/deepseek_v4/compressor.py", - "vllm_ascend/models/deepseek_v4/dspark.py", - "vllm_ascend/models/deepseek_v4/indexer.py", - "vllm_ascend/models/deepseek_v4/model.py", - "vllm_ascend/models/deepseek_v4/mtp.py", - "vllm_ascend/ops/dsa.py", - "vllm_ascend/spec_decode/llm_base_proposer.py" - ] - }, - { - "commit": "e5fa7780b0c0cd439d6aca43268c5eda237fa568", - "subject": "[Doc] Translated Doc files 2026-08-16 (#14365)", - "topics": [ - "deepseek-v4-dspark", - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/locale/zh_CN/LC_MESSAGES/community/slash-commands.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/DeepSeek-V4-Flash.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/DeepSeek-V4-Pro.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Dots3-Note.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/GLM5.2.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3.8-2.4T-A95B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3.8-27B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/configuration/additional_config.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/pipeline_parallel.po" - ] - }, - { - "commit": "6b424bfa9e891d78544c2fd737a5486ddc5f1e35", - "subject": "[Refactor][Distributed] Remove all_to_all patch and switch lm-head TP to all_to_all_single (#12923)", - "topics": [ - "communication", - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/distributed/test_communicator.py", - "tests/ut/ops/test_lmhead_all_to_all.py", - "tests/ut/ops/test_vocab_parallel_embedding.py", - "tests/ut/patch/worker/patch_common/test_patch_distributed.py", - "vllm_ascend/distributed/device_communicators/npu_communicator.py", - "vllm_ascend/ops/vocab_parallel_embedding.py", - "vllm_ascend/patch/__init__.py", - "vllm_ascend/patch/worker/patch_distributed.py", - "vllm_ascend/spec_decode/llm_base_proposer.py", - "vllm_ascend/spec_decode/step3p5.py" - ] - }, - { - "commit": "37e343d819c257c3d1261eb4a41acbda56908207", - "subject": "[CI][Spec Decode] Add case for MoE+DP hang (#14019)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "tests/e2e/pull_request/two_card/spec_decode/test_spec_decode.py" - ] - }, - { - "commit": "035b8d40718dc097e159e3590975d712535e4245", - "subject": "[BugFix][SpecDecode] Fix low MTP acceptance rate for SFA (MTP>1) (#14236)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/attention/a2/test_sfa_v1.py", - "vllm_ascend/attention/sfa_v1.py" - ] - }, - { - "commit": "58d624ad4d89584ac30a3322f4d52945c64780d7", - "subject": "[CI] Re-enable DeepSeek V4 test (#14391)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "tests/e2e/pull_request/four_card/model_runner_v2/test_deepseek_v4.py" - ] - }, - { - "commit": "755ab8603090601c7104f3b6b57bad145f9f0916", - "subject": "[Doc][Misc] Consolidate layerwise and sparse KV cache offloading guidance (#14056)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/assets/layerwise_kv_offload_concept.png", - "docs/source/assets/sparse_kv_offload_concept.png", - "docs/source/developer_guide/Design_Documents/layerwise_and_sparse_kv_cache_offloading.md", - "docs/source/developer_guide/Design_Documents/sfa_remote_d2h_connector.md", - "docs/source/user_guide/feature_guide/layerwise_and_sparse_kv_cache_offloading.md", - "mkdocs.yml" - ] - }, - { - "commit": "858ffad5fe86cc18e894dbd2399872a07b7f7542", - "subject": "[Bugfix] fix bug for mega moe (#13675)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/ops/test_fused_moe.py", - "tests/ut/test_ascend_config.py", - "tests/ut/test_ascend_forward_context.py", - "vllm_ascend/ascend_config.py", - "vllm_ascend/ascend_forward_context.py", - "vllm_ascend/ops/fused_moe/moe_comm_method.py", - "vllm_ascend/ops/fused_moe/moe_utils.py", - "vllm_ascend/ops/fused_moe/routed_experts.py", - "vllm_ascend/quantization/methods/w4a8.py", - "vllm_ascend/quantization/methods/w8a8_dynamic.py" - ] - }, - { - "commit": "0f9e938aba6e81b100588253dc6a58c78715dac9", - "subject": "[Doc][Misc] Update navigation titles (#14290)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "docs/hooks/nav_titles.py", - "docs/source/_templates/Model-Deployment-Tutorial-Template.md", - "docs/source/_templates/Model-Deployment-Tutorial-Template.zh.md", - "docs/source/tutorials/models/DeepSeek-V3.1.md", - "docs/source/tutorials/models/GLM5.md", - "docs/source/tutorials/models/Gemma4.md", - "docs/source/tutorials/models/InternVL3.5.md", - "docs/source/tutorials/models/Qwen3-Dense.md", - "docs/source/tutorials/models/Qwen3.5-27B-Qwen3.6-27B.md", - "docs/source/tutorials/models/Qwen3.5-397B-A17B.md", - "docs/source/tutorials/models/Qwen3.6-35B-A3B.md" - ] - }, - { - "commit": "2e34ccf65795aed0e2514532bc2cbdd3542e4ed6", - "subject": "[BugFix][Model] Fix MiniMax-M3 shared vision tower pruning (#14210)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/tutorials/models/MiniMax-M3.md", - "tests/ut/models/minimax_m3/test_minimax_m3_vit.py", - "vllm_ascend/models/minimax_m3/minimax_m3_vl.py" - ] - }, - { - "commit": "8fcb3bb9e84b85ebe0cc7f029f5fce087c0a992f", - "subject": "[Feature][MRV2] Support MTP with SFA in eager mode (#14105)", - "topics": [ - "communication", - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/scripts/test_config.yaml", - "tests/e2e/pull_request/eight_card/model_runner_v2/__init__.py", - "tests/e2e/pull_request/eight_card/model_runner_v2/test_glm5_2.py", - "vllm_ascend/worker/v2/spec_decode/autoregressive/speculator.py" - ] - }, - { - "commit": "d5e9816065ede613327d93908f87fee9f5c47128", - "subject": "[CI] update glm5.1-w8a8c8 glm5.2-w8a8c8 qwen3.6-35B qwen3.5-122B-w4a8 (#14315)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/configs/nightly_config.yaml", - ".github/workflows/configs/weekly_config.yaml", - ".github/workflows/misc/model_dataset_list.json", - "tests/e2e/nightly/multi_node/external_dp/config/GLM-5.1-W8A8C8-128k-1k-90-50-PD.yaml", - "tests/e2e/nightly/multi_node/external_dp/config/GLM-5.2-W8A8C8-128k-1k-90-50-PD.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/GLM-5.1-W8A8C8-A3_198k_function.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/GLM-5.2-W8A8C8-A3_128k_90_50.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/GLM-5.2-W8A8C8-A3_198k_function.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/GLM-5.1-W8A8C8-3.5k-1.5k-0-20-PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/GLM-5.1-W8A8C8-3.5k-1.5k-0-50-PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/GLM-5.1-W8A8C8-64k-1k-90-50-PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/GLM-5.2-W8A8C8-3.5k-1.5k-0-20-PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/GLM-5.2-W8A8C8-3.5k-1.5k-0-50-PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/GLM-5.2-W8A8C8-64k-1k-90-50-PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/Kimi-K2.7-W4A8-128k-1k-TPOT50-PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/Kimi-K2.7-W4A8-64k-1k-TPOT50-PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/Qwen3.5-122B-A10B-w4a8-128k-1k-TPOT50-90-PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/Qwen3.5-122B-A10B-w4a8-16k-1k-TPOT50-0-PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/Qwen3.5-122B-A10B-w4a8-984k-1k-0-PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/Qwen3.5-397B-w4a8_128k-1k-90-50-PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/Qwen3.5-397B-w4a8_3.5k-1.5k-0-50-PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/Qwen3.5-397B-w4a8_64k-1k-90-50-PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/Qwen3.5-397B-w4a8_986k-function-PD.yaml", - "tests/e2e/weekly/multi_node/internal_dp/config/GLM-5.1-W8A8C8-A3_3.5k_1.5k_0_20.yaml", - "tests/e2e/weekly/multi_node/internal_dp/config/GLM-5.1-W8A8C8-A3_3.5k_1.5k_0_50.yaml", - "tests/e2e/weekly/multi_node/internal_dp/config/GLM-5.1-W8A8C8-A3_64k_90_50.yaml", - "tests/e2e/weekly/multi_node/internal_dp/config/GLM-5.2-W8A8C8-A3_3.5k_1.5k_0_20.yaml", - "tests/e2e/weekly/multi_node/internal_dp/config/GLM-5.2-W8A8C8-A3_3.5k_1.5k_0_50.yaml", - "tests/e2e/weekly/multi_node/internal_dp/config/GLM-5.2-W8A8C8-A3_64k_90_50.yaml", - "tests/e2e/weekly/single_node/configs/Kimi-K2.6-w4a8-A3.yaml", - "tests/e2e/weekly/single_node/configs/Kimi-K2.7-w4a8-A3.yaml", - "tests/e2e/weekly/single_node/configs/Qwen3.5-122B-A10B-w4a8-A3-accuracy.yaml", - "tests/e2e/weekly/single_node/configs/Qwen3.5-122B-A10B-w4a8-A3.yaml", - "tests/e2e/weekly/single_node/configs/Qwen3.5-397B-A17B-w8a8-3.5k-1.5k-0-50-single.yaml", - "tests/e2e/weekly/single_node/configs/Qwen3.6-35B-A3B-w4a8-A3.yaml", - "tests/e2e/weekly/single_node/configs/Qwen3.6-35B-A3B-w8a8-A3-accuracy.yaml" - ] - }, - { - "commit": "3eb8dede168ad9e24e3bb548ed35db7e658dd40e", - "subject": "[Refactor] decouple flashcomm and shared expert dp (#13158)", - "topics": [ - "deepseek-v4-dspark", - "communication", - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "docs/source/user_guide/configuration/additional_config.md", - "tests/e2e/pull_request/four_card/spec_decode/test_dspark_deepseekv4.py", - "tests/e2e/pull_request/two_card/test_shared_expert_dp.py", - "tests/ut/ops/test_fused_moe.py", - "tests/ut/ops/test_linear.py", - "tests/ut/ops/test_moe_comm_method.py", - "tests/ut/ops/test_prepare_finalize.py", - "tests/ut/ops/test_vocab_parallel_embedding.py", - "tests/ut/spec_decode/a2/test_eagle_proposer.py", - "tests/ut/test_ascend_config.py", - "vllm_ascend/ascend_config.py", - "vllm_ascend/ops/fused_moe/fused_moe.py", - "vllm_ascend/ops/fused_moe/moe_comm_method.py", - "vllm_ascend/ops/fused_moe/prepare_finalize.py", - "vllm_ascend/ops/fused_moe/routed_experts.py", - "vllm_ascend/ops/fused_moe/shared_experts.py", - "vllm_ascend/ops/linear_op.py", - "vllm_ascend/spec_decode/llm_base_proposer.py", - "vllm_ascend/utils.py" - ] - }, - { - "commit": "cb1e180dc3734cc8e1fa89e40da8310cad01be10", - "subject": "[Refactor][BugFix][DSA][4/N] fix the precision issue of dsa_v1 under dspark (#14248)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/attention/test_dsa_v1.py", - "vllm_ascend/attention/dsa_v1.py" - ] - }, - { - "commit": "bf7382b5ef406e788b744324d9185ecd0bbf7c6c", - "subject": "[Feature] add Confidence-Head for Deepseek-V4 (#14249)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/models/deepseek_v4_dspark.py" - ] - }, - { - "commit": "463eb4bc018aa06e00aca4b35f7a3782e8e95e46", - "subject": "[Refactor]refactor fp8 config (#13651)", - "topics": [ - "deepseek-v4-dspark", - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/quantization/test_fp8_config.py", - "vllm_ascend/quantization/fp8_config.py", - "vllm_ascend/quantization/methods/fp8.py" - ] - }, - { - "commit": "f91932bc91760aed4499f6a4603b3f8ac67000e1", - "subject": "[CI] invoke dspark deepseek ci (#13930)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/scripts/test_config.yaml" - ] - }, - { - "commit": "9080ea88bc2d753aa4735144be971be356e970ff", - "subject": "[Feature]Minimax M3 upgrade msa decode performance; Remove e_correction_bias downcast (#13987)", - "topics": [ - "priority-models", - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "csrc/attention/sparse_attention_score/README.md", - "csrc/attention/sparse_attention_score/docs/sparse_attention_score_design.md", - "csrc/attention/sparse_attention_score/op_host/CMakeLists.txt", - "csrc/attention/sparse_attention_score/op_host/op_api/aclnn_sparse_attention_score.cpp", - "csrc/attention/sparse_attention_score/op_host/op_api/sparse_attention_score.cpp", - "csrc/attention/sparse_attention_score/op_host/op_api/sparse_attention_score.h", - "csrc/attention/sparse_attention_score/op_host/sparse_attention_score_def.cpp", - "csrc/attention/sparse_attention_score/op_host/sparse_attention_score_infershape.cpp", - "csrc/attention/sparse_attention_score/op_host/sparse_attention_score_tiling.cpp", - "csrc/attention/sparse_attention_score/op_host/sparse_attention_score_tiling.h", - "csrc/attention/sparse_attention_score/op_kernel/arch22/sparse_attention_score_fd_combine_arch22.h", - "csrc/attention/sparse_attention_score/op_kernel/arch22/sparse_attention_score_kernel_arch22.h", - "csrc/attention/sparse_attention_score/op_kernel/arch35/sparse_attention_score_fd_combine_arch35.h", - "csrc/attention/sparse_attention_score/op_kernel/arch35/sparse_attention_score_kernel_arch35.h", - "csrc/attention/sparse_attention_score/op_kernel/arch35/sparse_attention_score_kernel_arch35_full_quant.h", - "csrc/attention/sparse_attention_score/op_kernel/attn_infra/coord.hpp", - "csrc/attention/sparse_attention_score/op_kernel/attn_infra/epilogue/block/block_epilogue_online_softmax_arch35_reg_low_prec.hpp", - "csrc/attention/sparse_attention_score/op_kernel/attn_infra/epilogue/block/block_epilogue_online_softmax_arch35_reg_low_prec_bf16.hpp", - "csrc/attention/sparse_attention_score/op_kernel/attn_infra/epilogue/block/block_epilogue_rescale_o.hpp", - "csrc/attention/sparse_attention_score/op_kernel/attn_infra/epilogue/block/block_epilogue_rescale_o_arch35_reg_high_prec.hpp", - "csrc/attention/sparse_attention_score/op_kernel/attn_infra/matrix_coord.hpp", - "csrc/attention/sparse_attention_score/op_kernel/kernel_common.hpp", - "csrc/attention/sparse_attention_score/op_kernel/sparse_attention_score.cpp", - "csrc/attention/sparse_attention_score/op_kernel/sparse_attention_score_kernel_interface.cpp", - "csrc/attention/sparse_attention_score/op_kernel/sparse_attention_score_tilingkey.h", - "csrc/attention/sparse_attention_score/tests/CMakeLists.txt", - "csrc/attention/sparse_attention_score/tests/FLASH_DECODING_GENERALIZED_TEST.md", - "csrc/attention/sparse_attention_score/tests/run_bf16_fd_minimal.py", - "csrc/attention/sparse_attention_score/tests/test_bf16.py", - "csrc/attention/sparse_attention_score/tests/test_flash_decoding_generalized.py", - "csrc/attention/sparse_attention_score/tests/test_fp16.py", - "csrc/attention/sparse_attention_score/tests/test_sparse_attention_score_fp8.py", - "csrc/attention/sparse_attention_score/tests/ut/CMakeLists.txt", - "csrc/attention/sparse_attention_score/tests/ut/op_host/CMakeLists.txt", - "csrc/attention/sparse_attention_score/tests/ut/op_host/test_sparse_attention_score_fd_tiling.cpp", - "tests/e2e/pull_request/one_card/test_minimax_m3_sparse_attn.py", - "tests/ut/models/minimax_m3/test_msa_m3.py", - "vllm_ascend/models/minimax_m3/msa_m3.py", - "vllm_ascend/models/minimax_m3/ops/msa_m3_npu.py", - "vllm_ascend/ops/fused_moe/routed_experts.py" - ] - }, - { - "commit": "b25f49dd0281726017041eed92bd0edc56961a2e", - "subject": "[Doc] GLM deployment on A3 over RoCE (#14077)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/tutorials/models/GLM5.2.md" - ] - }, - { - "commit": "f0c320e6fa88f00dc059ef8dbc1a99305e88ac13", - "subject": "[Refactor] Clean some useless code and change some parameter type to avoid re-comppilation inside minimax m3 triton (#13532)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/models/minimax_m3/ops/msa_m3_triton.py" - ] - }, - { - "commit": "6ae53224fca8a213c156dab0a3ac3651c01f5510", - "subject": "[Refactor][Ops] Remove unused custom operators (#12896)", - "topics": [ - "deepseek-v4-dspark", - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "csrc/attention/indexer_compress_epilog/CMakeLists.txt", - "csrc/attention/indexer_compress_epilog/op_host/CMakeLists.txt", - "csrc/attention/indexer_compress_epilog/op_host/indexer_compress_epilog_def.cpp", - "csrc/attention/indexer_compress_epilog/op_host/indexer_compress_epilog_proto.cpp", - "csrc/attention/indexer_compress_epilog/op_host/indexer_compress_epilog_tiling.cpp", - "csrc/attention/indexer_compress_epilog/op_host/indexer_compress_epilog_tiling.h", - "csrc/attention/indexer_compress_epilog/op_kernel/indexer_compress_epilog.cpp", - "csrc/attention/indexer_compress_epilog/op_kernel/indexer_compress_epilog_base.h", - "csrc/attention/indexer_compress_epilog/op_kernel/indexer_compress_epilog_multi_row.h", - "csrc/attention/indexer_compress_epilog/op_kernel/indexer_compress_epilog_multi_row_mx_fp8.h", - "csrc/attention/indexer_compress_epilog/op_kernel/indexer_compress_epilog_single_row.h", - "csrc/attention/indexer_compress_epilog/op_kernel/indexer_compress_epilog_single_row_mx_fp8.h", - "csrc/attention/load_index_kv_cache/CMakeLists.txt", - "csrc/attention/load_index_kv_cache/op_host/CMakeLists.txt", - "csrc/attention/load_index_kv_cache/op_host/load_index_kv_cache_def.cpp", - "csrc/attention/load_index_kv_cache/op_host/load_index_kv_cache_proto.cpp", - "csrc/attention/load_index_kv_cache/op_host/load_index_kv_cache_tiling.cpp", - "csrc/attention/load_index_kv_cache/op_host/load_index_kv_cache_tiling.h", - "csrc/attention/load_index_kv_cache/op_kernel/load_index_kv_cache.cpp", - "csrc/attention/load_index_kv_cache/op_kernel/load_index_kv_cache_base.h", - "csrc/attention/load_index_kv_cache/op_kernel/load_index_kv_cache_perf.h", - "csrc/build_aclnn.sh", - "csrc/moe/hc_pre_inv_rms/CMakeLists.txt", - "csrc/moe/hc_pre_inv_rms/op_host/CMakeLists.txt", - "csrc/moe/hc_pre_inv_rms/op_host/hc_pre_inv_rms_def.cpp", - "csrc/moe/hc_pre_inv_rms/op_host/hc_pre_inv_rms_proto.cpp", - "csrc/moe/hc_pre_inv_rms/op_host/hc_pre_inv_rms_tiling.cpp", - "csrc/moe/hc_pre_inv_rms/op_host/hc_pre_inv_rms_tiling.h", - "csrc/moe/hc_pre_inv_rms/op_host/hc_pre_inv_rms_tiling_arch35.h", - "csrc/moe/hc_pre_inv_rms/op_host/hc_pre_inv_rms_tiling_large_d.h", - "csrc/moe/hc_pre_inv_rms/op_kernel/hc_pre_inv_rms.cpp", - "csrc/moe/hc_pre_inv_rms/op_kernel/hc_pre_inv_rms_full_load.h", - "csrc/moe/hc_pre_inv_rms/op_kernel/hc_pre_inv_rms_full_load_large_d.h", - "csrc/moe/hc_pre_inv_rms/op_kernel/hc_pre_inv_rms_full_load_regbase.h", - "csrc/moe/hc_pre_sinkhorn/CMakeLists.txt", - "csrc/moe/hc_pre_sinkhorn/op_host/CMakeLists.txt", - "csrc/moe/hc_pre_sinkhorn/op_host/hc_pre_sinkhorn_def.cpp", - "csrc/moe/hc_pre_sinkhorn/op_host/hc_pre_sinkhorn_proto.cpp", - "csrc/moe/hc_pre_sinkhorn/op_host/hc_pre_sinkhorn_tiling.cpp", - "csrc/moe/hc_pre_sinkhorn/op_host/hc_pre_sinkhorn_tiling.h", - "csrc/moe/hc_pre_sinkhorn/op_kernel/hc_pre_sinkhorn.cpp", - "csrc/moe/hc_pre_sinkhorn/op_kernel/hc_pre_sinkhorn_base.h", - "csrc/moe/hc_pre_sinkhorn/op_kernel/hc_pre_sinkhorn_perf.h", - "csrc/moe/hc_pre_sinkhorn/op_kernel/hc_pre_sinkhorn_regbase_base.h", - "csrc/moe/hc_pre_sinkhorn/op_kernel/hc_pre_sinkhorn_regbase_perf.h", - "csrc/torch_binding.cpp", - "csrc/torch_binding_meta.cpp", - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_npu_hc_pre.py" - ] - }, - { - "commit": "acbd2bb28de6d5104a538806950687523c20407a", - "subject": "[Performance] Optimized Kernel copy_and_expand_dflash_and_dspark_inputs_kernel (#13191)", - "topics": [ - "deepseek-v4-dspark", - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_copy_and_expand_dflash_dspark.py", - "tests/ut/spec_decode/test_dspark_proposer.py", - "vllm_ascend/ops/triton/spec_decode/utils.py", - "vllm_ascend/spec_decode/dflash_proposer.py", - "vllm_ascend/spec_decode/dspark_proposer.py" - ] - }, - { - "commit": "553ec08afa4b47395384fbec40a7f1ed6ce541ee", - "subject": "[Doc] Sync DeepSeek-V4 Flash&Pro tutorial docs from v0.23.0 (#14215)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/tutorials/models/DeepSeek-V4-Flash.md", - "docs/source/tutorials/models/DeepSeek-V4-Pro.md" - ] - }, - { - "commit": "b4b04c5eb15532f87f55072b5d4cd4de77ef1fca", - "subject": "[CI][Misc] main2main vllm v0.27.1 (#14027)", - "topics": [ - "deepseek-v4-dspark", - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/vllm-release-tag.commit", - "Dockerfile", - "Dockerfile.310p", - "Dockerfile.310p.openEuler", - "Dockerfile.a3", - "Dockerfile.a3.openEuler", - "Dockerfile.a5", - "Dockerfile.a5.openEuler", - "Dockerfile.openEuler", - "README.md", - "README.zh.md", - "examples/rl/rlhf_http_npu_ipc.py", - "tests/e2e/conftest.py", - "tests/e2e/pull_request/one_card/test_guided_decoding.py", - "tests/e2e/pull_request/one_card/test_npu_ipc_weight_transfer.py", - "tests/ut/distributed/weight_transfer/test_npu_ipc_engine.py", - "tests/ut/patch/platform/test_deepseek_v4_thinking.py", - "tests/ut/spec_decode/test_dspark_proposer.py", - "vllm_ascend/_310p/ops/vocab_parallel_embedding.py", - "vllm_ascend/distributed/device_communicators/npu_communicator.py", - "vllm_ascend/distributed/kv_transfer/kv_p2p/mooncake_connector.py", - "vllm_ascend/distributed/kv_transfer/kv_p2p/mooncake_hybrid_connector.py", - "vllm_ascend/distributed/weight_transfer/__init__.py", - "vllm_ascend/distributed/weight_transfer/npu_ipc_engine.py", - "vllm_ascend/ops/fused_moe/fused_moe.py", - "vllm_ascend/ops/vocab_parallel_embedding.py", - "vllm_ascend/patch/platform/patch_fused_moe.py", - "vllm_ascend/spec_decode/dspark_proposer.py", - "vllm_ascend/worker/model_runner_v1.py", - "vllm_ascend/worker/npu_input_batch.py", - "vllm_ascend/worker/v2/sample/gumbel.py", - "vllm_ascend/worker/v2/spec_decode/autoregressive/speculator.py", - "vllm_ascend/worker/v2/spec_decode/dflash/speculator.py", - "vllm_ascend/worker/v2/spec_decode/dspark/speculator.py", - "vllm_ascend/worker/v2/spec_decode/eagle/aclgraph.py" - ] - }, - { - "commit": "c04eb05cbbc04f90ca0bb863e5704b923cc2d080", - "subject": "[CI] Temporarily skip DeepSeek V4 test due to PR conflicts (#14189)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "tests/e2e/pull_request/four_card/model_runner_v2/test_deepseek_v4.py" - ] - }, - { - "commit": "26a68272cd4de2f81c920b5d383cfe65bec96f64", - "subject": "[Doc] Translated Doc files 2026-08-13 (#14155)", - "topics": [ - "deepseek-v4-dspark", - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/Design_Documents/sfa_remote_d2h_connector.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/performance_and_debug/msprobe_guide.po", - "docs/source/locale/zh_CN/LC_MESSAGES/installation.po", - "docs/source/locale/zh_CN/LC_MESSAGES/quick_start.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/DeepSeek-V4-Flash.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/GLM5.2.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/configuration/additional_config.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/dyntra_lb.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/index.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/layerwise_and_sparse_kv_cache_offloading.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/layerwise_kv_pool.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/pipeline_parallel.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/support_matrix/supported_features.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/support_matrix/supported_models.po" - ] - }, - { - "commit": "6af9257e449ca139ccd228f0d71ca7d2c09909c9", - "subject": "[Feature] Support dynamic speculative decoding with DFlash and Unified dynamic selection logic (#13819)", - "topics": [ - "deepseek-v4-dspark", - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/scripts/test_config.yaml", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/configuration/additional_config.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/speculative_decoding.po", - "docs/source/user_guide/configuration/additional_config.md", - "docs/source/user_guide/feature_guide/speculative_decoding.md", - "tests/e2e/pull_request/one_card/spec_decode/test_dynamic.py", - "tests/e2e/pull_request/one_card/spec_decode/utils.py", - "vllm_ascend/ascend_config.py", - "vllm_ascend/models/qwen3_dspark.py", - "vllm_ascend/spec_decode/dflash_proposer.py", - "vllm_ascend/spec_decode/dspark_proposer.py", - "vllm_ascend/spec_decode/llm_base_proposer.py", - "vllm_ascend/spec_decode/utils.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "3505560d69e81dc5234a05ccd31860cfced05fb6", - "subject": "[Refactor][DSA][3/N] unified the prefill and decode processes in DSA v1 (#13995)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/attention/test_dsa_v1.py", - "tests/ut/worker/test_attn_utils_v2.py", - "vllm_ascend/attention/dsa_v1.py", - "vllm_ascend/spec_decode/llm_base_proposer.py", - "vllm_ascend/spec_decode/step3p5.py", - "vllm_ascend/worker/model_runner_v1.py", - "vllm_ascend/worker/v2/attn_utils.py" - ] - }, - { - "commit": "11e95c1059d428c53283ccf4073b5578ff1f71e8", - "subject": "[Refactor]: reorganize ascend store kv pool modules (#13354)", - "topics": [ - "communication" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/scripts/test_config.yaml", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/lmcache_ascend_deployment.po", - "docs/source/user_guide/feature_guide/lmcache_ascend_deployment.md", - "tests/ut/conftest.py", - "tests/ut/distributed/ascend_store/_mock_deps.py", - "tests/ut/distributed/ascend_store/test_backend.py", - "tests/ut/distributed/ascend_store/test_coordinator.py", - "tests/ut/distributed/ascend_store/test_kv_transfer.py", - "tests/ut/distributed/ascend_store/test_metadata.py", - "tests/ut/distributed/ascend_store/test_pool_scheduler.py", - "tests/ut/distributed/ascend_store/test_pool_worker.py", - "tests/ut/distributed/mooncake/test_mooncake_kv_transfer.py", - "vllm_ascend/attention/attention_v1.py", - "vllm_ascend/attention/context_parallel/attention_cp.py", - "vllm_ascend/attention/context_parallel/dsa_cp.py", - "vllm_ascend/attention/dsa_v1.py", - "vllm_ascend/attention/mla_v1.py", - "vllm_ascend/attention/sfa_v1.py", - "vllm_ascend/distributed/kv_transfer/__init__.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/ascend_store/ascend_store_connector.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/ascend_store/attention_fence.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/ascend_store/backend/base.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/ascend_store/backend/memcache_backend.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/ascend_store/backend/mooncake_backend.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/ascend_store/backend/yuanrong_backend.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/ascend_store/coordinator.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/ascend_store/kv_transfer.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/ascend_store/metadata.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/ascend_store/pool_scheduler.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/ascend_store/pool_worker.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/lmcache_ascend_connector.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/ucm_connector/__init__.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/ucm_connector/connector.py" - ] - }, - { - "commit": "ffa55704fb441270d29cf311fab97b1d298bf2a9", - "subject": "[Feature] DyntraLB: Dynamic Intra-decoder Data-parallel Load Balancing (#12292)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "docs/source/user_guide/configuration/additional_config.md", - "docs/source/user_guide/feature_guide/dyntra_lb.md", - "docs/source/user_guide/feature_guide/index.md", - "tests/ut/core/test_dyntra_lb_recompute_scheduler.py", - "tests/ut/core/test_dyntra_lb_scheduler.py", - "tests/ut/patch/platform/test_patch_balance_schedule.py", - "tests/ut/patch/platform/test_patch_dyntra_lb_core.py", - "tests/ut/test_ascend_config.py", - "tests/ut/test_platform.py", - "vllm_ascend/ascend_config.py", - "vllm_ascend/core/dyntra_lb_scheduler.py", - "vllm_ascend/core/recompute_scheduler.py", - "vllm_ascend/patch/__init__.py", - "vllm_ascend/patch/platform/__init__.py", - "vllm_ascend/patch/platform/patch_dyntra_lb_core.py", - "vllm_ascend/platform.py" - ] - }, - { - "commit": "954b1d9425fe249c649c1fc824e3c06c6f01a25d", - "subject": "[Test][BugFix] Fix nightly file not found (#14052)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/configs/nightly_config.yaml", - ".github/workflows/schedule_nightly_test_a3.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/GLM-5.1-W8A8C8-A3_128k_90_50.yaml" - ] - }, - { - "commit": "63bd2c418c8914201981f1150fc0e0ae5724896e", - "subject": "[Refactor][Ops] Align routing replay capture with upstream (#13882)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/ops/test_fused_moe.py", - "tests/ut/worker/test_routing_replay.py", - "vllm_ascend/ops/fused_moe/routed_experts.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "e76d8da934d7060a6cef473e43c295dd4e3e258e", - "subject": "[BugFix][SpecDecode] Avoid DSpark crash when causality API is absent (#13935)", - "topics": [ - "deepseek-v4-dspark", - "communication", - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/spec_decode/test_dspark_proposer.py", - "vllm_ascend/spec_decode/dspark_proposer.py" - ] - }, - { - "commit": "de35b40196dbd6c21d075a31183982355149c9b0", - "subject": "[Feature][MRV2] Support MTP with DSA in eager mode (#13874)", - "topics": [ - "deepseek-v4-dspark", - "communication", - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/scripts/test_config.yaml", - "tests/e2e/pull_request/four_card/model_runner_v2/__init__.py", - "tests/e2e/pull_request/four_card/model_runner_v2/test_deepseek_v4.py", - "vllm_ascend/models/deepseek_v4_mtp.py", - "vllm_ascend/worker/v2/spec_decode/autoregressive/speculator.py" - ] - }, - { - "commit": "77c12740036c29d3905be53f41594c3c65e41894", - "subject": "[BugFix] correct MiniMax-M3 index query normalization head dimension (#14001)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/models/minimax_m3/minimax_m3.py" - ] - }, - { - "commit": "054bd698ab85e307c471dbe7d48b979952aa17f3", - "subject": "[Platform][Refactor] Refactor NPUPlatform check_and_update_config method (#13692)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/test_platform.py", - "vllm_ascend/platform.py" - ] - }, - { - "commit": "aa48bd68794c0acf32e6d3de116a028c180520b2", - "subject": "[Refactor]Refactor token dispatcher routing initialization for quantized alltoall MoE (#13641)", - "topics": [ - "deepseek-v4-dspark", - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/ops/fused_moe/token_dispatcher.py" - ] - }, - { - "commit": "a840f0574bdeeae44161cd8116f6568ec3a0b8c9", - "subject": "[Doc] Tune DeepSeek V4 Flash DSpark configurations (#13981)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/tutorials/models/DeepSeek-V4-Flash.md" - ] - }, - { - "commit": "7eee2499afcf8a982cc096eb4e478a63a77da1ab", - "subject": "[Refactor][SFA] Decouple DSA-CP and DCP backends (#13921)", - "topics": [ - "communication", - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/attention/a2/test_sfa_v1.py", - "tests/ut/attention/test_sfa_cp.py", - "tests/ut/attention/test_sfa_o_proj_tp.py", - "tests/ut/spec_decode/test_speculators_vwn_eagle3.py", - "vllm_ascend/attention/context_parallel/sfa_cp.py", - "vllm_ascend/attention/sfa_kv_offload.py", - "vllm_ascend/attention/sfa_v1.py" - ] - }, - { - "commit": "21a4a6832b277b44ca8b665e73ef2270473fc38c", - "subject": "[Misc] Remove patch files related to rotary quantization, and register new mtp/eagle3 model classes (#10896)", - "topics": [ - "deepseek-v4-dspark", - "priority-models", - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/model_executor/test_qwen3_dspark.py", - "vllm_ascend/models/__init__.py", - "vllm_ascend/models/deepseek_mtp.py", - "vllm_ascend/models/deepseek_v4_dspark.py", - "vllm_ascend/models/llama_eagle3.py", - "vllm_ascend/models/qwen3_dspark.py", - "vllm_ascend/patch/__init__.py", - "vllm_ascend/patch/worker/__init__.py", - "vllm_ascend/patch/worker/patch_deepseek_mtp.py", - "vllm_ascend/patch/worker/patch_draft_quarot.py", - "vllm_ascend/utils.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "c125a896a4271d11e46c91e958d40ed2f09fa09d", - "subject": "[Refactor][Device][2/N] Introduce capability-oriented hardware profiles (#13721)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/device/test_device_config.py", - "tests/ut/device/test_hardware_profile.py", - "vllm_ascend/device/__init__.py", - "vllm_ascend/device/hardware_profile.py" - ] - }, - { - "commit": "af337dfa8def7d1275d05c854071c299c5ef7d3c", - "subject": "[Refactor][DSA][2/N] rectify the dsa_cp metadata data (#13918)", - "topics": [ - "communication" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/attention/test_dsa_v1.py", - "vllm_ascend/attention/context_parallel/dsa_cp.py", - "vllm_ascend/attention/dsa_v1.py" - ] - }, - { - "commit": "1aad8745e4a3e02644c76d8a44bcbbd0a4113f59", - "subject": "[Doc] Reapply GLM-5.2 DCP and sfa_c8 guidance (#14017)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/tutorials/models/GLM5.2.md" - ] - }, - { - "commit": "7a25ef83a605f4709a9f67af12a9bc0f5051e705", - "subject": "[Feature][KV Transfer] Add SFA PD RD2H connector for kv cache offload (#12831)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/developer_guide/Design_Documents/sfa_remote_d2h_connector.md", - "docs/source/user_guide/feature_guide/index.md", - "docs/source/user_guide/feature_guide/layerwise_and_sparse_kv_cache_offloading.md", - "docs/source/user_guide/feature_guide/layerwise_kv_pool.md", - "docs/source/user_guide/feature_guide/layerwise_prefill_kv_offload.md", - "docs/source/user_guide/feature_guide/sparse_kv_cache_offload.md", - "examples/disaggregated_prefill_v1/load_balance_proxy_layerwise_server_example.py", - "mkdocs.yml", - "tests/ut/attention/a2/test_sfa_v1.py", - "tests/ut/distributed/ascend_store/_mock_deps.py", - "tests/ut/distributed/ascend_store/test_kv_transfer.py", - "tests/ut/distributed/ascend_store/test_layerwise_cache_layout.py", - "tests/ut/distributed/ascend_store/test_pool_scheduler.py", - "tests/ut/distributed/ascend_store/test_pool_worker.py", - "tests/ut/kv_offload/test_ascend_multi_connector.py", - "tests/ut/kv_offload/test_memfabric_transfer_engine.py", - "tests/ut/kv_offload/test_sfa_pd_rd2h_connector.py", - "tests/ut/worker/a2/test_model_runner_v1.py", - "tests/ut/worker/a2/test_worker_multi_instance.py", - "tests/ut/worker/a2/test_worker_v1.py", - "tools/test_memfabric_pd_read.py", - "vllm_ascend/attention/sfa_v1.py", - "vllm_ascend/attention/utils.py", - "vllm_ascend/distributed/kv_transfer/__init__.py", - "vllm_ascend/distributed/kv_transfer/ascend_multi_connector.py", - "vllm_ascend/distributed/kv_transfer/kv_p2p/mooncake_layerwise_connector.py", - "vllm_ascend/distributed/kv_transfer/kv_p2p/sfa_pd_rd2h/__init__.py", - "vllm_ascend/distributed/kv_transfer/kv_p2p/sfa_pd_rd2h/connector.py", - "vllm_ascend/distributed/kv_transfer/kv_p2p/sfa_pd_rd2h/protocol.py", - "vllm_ascend/distributed/kv_transfer/kv_p2p/sfa_pd_rd2h/read_thread.py", - "vllm_ascend/distributed/kv_transfer/kv_p2p/sfa_pd_rd2h/scheduler.py", - "vllm_ascend/distributed/kv_transfer/kv_p2p/sfa_pd_rd2h/send_thread.py", - "vllm_ascend/distributed/kv_transfer/kv_p2p/sfa_pd_rd2h/worker.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/ascend_store/kv_transfer.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/ascend_store/layerwise_cache_layout.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/ascend_store/pool_scheduler.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/ascend_store/pool_worker.py", - "vllm_ascend/distributed/kv_transfer/utils/memfabric_transfer_engine.py", - "vllm_ascend/worker/worker.py" - ] - }, - { - "commit": "27bc2b451b29e134adb2250b0c12f5f294cfdf03", - "subject": "[Refactor][Ops] Clean up fused MoE dataclass imports (#13938)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_fused_moe.py", - "tests/ut/_310p/fused_moe/test_moe_mlp_310.py", - "tests/ut/lora/test_quant_moe.py", - "tests/ut/ops/a2/test_token_dispatcher.py", - "tests/ut/ops/test_moe_comm_method.py", - "tests/ut/ops/test_moe_mlp.py", - "tests/ut/ops/test_moe_runtime_args.py", - "vllm_ascend/_310p/fused_moe/moe_comm_method.py", - "vllm_ascend/_310p/fused_moe/moe_mlp.py", - "vllm_ascend/lora/quant_moe.py", - "vllm_ascend/ops/fused_moe/dataclass/fused_experts.py", - "vllm_ascend/ops/fused_moe/dataclass/moe_mlp.py", - "vllm_ascend/ops/fused_moe/dataclass/token_dispatcher.py", - "vllm_ascend/ops/fused_moe/moe_comm_method.py" - ] - }, - { - "commit": "7485331ddec0514fe05dafd48678e72011e64c48", - "subject": "[Misc] Upgrade CANN(9.1.0), PTA(2.10.0.post4), and Triton(3.2.2) (#13901)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/_build_csrc_cache.yaml", - ".github/workflows/_e2e_nightly_multi_node.yaml", - ".github/workflows/_e2e_nightly_multi_node_560t.yaml", - ".github/workflows/_e2e_nightly_single_node.yaml", - ".github/workflows/_e2e_nightly_single_node_560t.yaml", - ".github/workflows/_ensure_csrc_cache.yaml", - ".github/workflows/_selected_tests.yaml", - ".github/workflows/_selected_tests_upstream.yaml", - ".github/workflows/dockerfiles/Dockerfile.buildwheel.310p", - ".github/workflows/dockerfiles/Dockerfile.buildwheel.a2", - ".github/workflows/dockerfiles/Dockerfile.buildwheel.a3", - ".github/workflows/pr_test.yaml", - ".github/workflows/schedule_doc_linkcheck.yaml", - ".github/workflows/schedule_e2e_upstream_test.yaml", - ".github/workflows/schedule_main2main.yaml", - ".github/workflows/schedule_nightly_test_a2.yaml", - ".github/workflows/schedule_weekly_test_a2.yaml", - ".github/workflows/scripts/csrc_cache_targets.json", - ".github/workflows/scripts/runner_label.json", - "Dockerfile", - "Dockerfile.310p", - "Dockerfile.310p.openEuler", - "Dockerfile.a3", - "Dockerfile.a3.openEuler", - "Dockerfile.a5", - "Dockerfile.a5.openEuler", - "Dockerfile.openEuler", - "README.md", - "README.zh.md", - "csrc/version.info", - "docs/hooks/macros_main.py", - "docs/source/installation.md", - "docs/source/quick_start.md", - "mkdocs.yml", - "pyproject.toml", - "requirements.txt", - "tests/e2e/nightly/multi_node/scripts/run.sh", - "tests/e2e/pull_request/one_card/rlhf/conftest.py", - "tests/e2e/vllm_interface/vllm_test.cfg", - "tests/ut/ops/a2/test_gdn_layerwise_kv.py", - "vllm_ascend/patch/worker/patch_v2/patch_uva.py", - "vllm_ascend/platform.py" - ] - }, - { - "commit": "e07969ed8ec8cc6542a23714b2d5b9c7a4a83423", - "subject": "[Doc] Translated Doc files 2026-08-10 (#13891)", - "topics": [ - "deepseek-v4-dspark", - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/locale/zh_CN/LC_MESSAGES/community/versioning_policy.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/Design_Documents/model_runner_v2_eplb.po", - "docs/source/locale/zh_CN/LC_MESSAGES/faqs.po", - "docs/source/locale/zh_CN/LC_MESSAGES/installation.po", - "docs/source/locale/zh_CN/LC_MESSAGES/quick_start.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/DeepSeek-R1.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/DeepSeek-V3.1.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/DeepSeek-V3.2.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/DeepSeek-V4-Flash.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/DeepSeek-V4-Pro.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/DeepSeekOCR2.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/GLM4.x.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/GLM5.2.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/GLM5.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Gemma4.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/InternVL3.5.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Kimi-K2-Thinking.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Kimi-K2.6.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/MiniMax-M2.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/MiniMax-M3.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/PaddleOCR-VL.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen-VL-Dense.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-235B-A22B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-30B-A3B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-ASR-1.7B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-Coder-30B-A3B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-Dense.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-Embedding.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-Next.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-Omni-30B-A3B-Thinking.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-Reranker.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-VL-235B-A22B-Instruct.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-VL-30B-A3B-Instruct.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-VL-Embedding.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-VL-Reranker.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3.5-27B-Qwen3.6-27B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3.5-397B-A17B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3.5-Dense.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3.6-35B-A3B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/configuration/additional_config.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/Ai_QoS_introduction_en.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/Fine_grained_TP.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/expert_parallelism_load_balancer.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/index.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/kv_pool.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/layerwise_kv_pool.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/layerwise_prefill_kv_offload.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/sparse_kv_cache_offload.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/speculative_decoding.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/release_notes.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/support_matrix/feature_matrix.po" - ] - }, - { - "commit": "c7972a3dcaad34c3293f1a68223b800d6dc0865a", - "subject": "[CI]Main2Main 0807 (#13477)", - "topics": [ - "deepseek-v4-dspark", - "priority-models", - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/vllm-main-verified.commit", - "tests/e2e/conftest.py", - "tests/e2e/pull_request/one_card/spec_decode/test_extract_hidden_states.py", - "tests/ut/models/minimax_m3/test_minimax_m3.py", - "tests/ut/models/test_deepseek_v4_moe.py", - "tests/ut/patch/platform/test_deepseek_v4_thinking.py", - "tests/ut/spec_decode/test_speculators_vwn_eagle3.py", - "vllm_ascend/models/deepseek_v4.py", - "vllm_ascend/models/layer/attention/layer.py", - "vllm_ascend/models/minimax_m3/minimax_m3.py", - "vllm_ascend/ops/fused_moe/fused_moe.py", - "vllm_ascend/ops/fused_moe/routed_experts.py", - "vllm_ascend/ops/triton/mamba/postprocess.py", - "vllm_ascend/patch/__init__.py", - "vllm_ascend/patch/platform/__init__.py", - "vllm_ascend/patch/platform/patch_fused_moe.py", - "vllm_ascend/patch/platform/patch_vision.py", - "vllm_ascend/patch/worker/patch_fused_moe.py", - "vllm_ascend/quantization/compressed_tensors_config.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "a47fbe2648c55fa757c81df727fd1c0b844025ac", - "subject": "[Feature][MRV2] SFA model runner v2 adaptation (#13069)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/patch/__init__.py", - "vllm_ascend/patch/worker/patch_v2/patch_attn_utils.py", - "vllm_ascend/worker/v2/attn_utils.py" - ] - }, - { - "commit": "b37d397769ca7d4a71e40e64fa112311c8b1dadd", - "subject": "[CI][Test] Update MiniMax-M3 W8A8 model to Eco-Tech/MiniMax-M3-w8a8-0626 (#13681)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/misc/model_dataset_list.json", - "tests/e2e/nightly/single_node/models/configs/MiniMax-M3-W8A8-A3.yaml", - "tests/e2e/pull_request/eight_card/test_minimax_m3.py" - ] - }, - { - "commit": "6fadabbfb5e18c60aa328845b3145d91a8d2b955", - "subject": "[Doc][Feature] Add Ascend 950 Products to supported models matrix (#13920)", - "topics": [ - "deepseek-v4-dspark", - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/user_guide/support_matrix/supported_models.md" - ] - }, - { - "commit": "059df1f41933888e1f0f65ba1fd2f0e728a544f5", - "subject": "[BugFix][SFA] Guard DSA-CP graph padding sequence lengths (#13779)", - "topics": [ - "communication" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/attention/a2/test_sfa_v1.py", - "vllm_ascend/attention/sfa_v1.py" - ] - }, - { - "commit": "5ef662e6bdebe35be5b3aafccfb99e1426da5536", - "subject": "[Test][Misc] Restore weekly use case (#13908)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/configs/nightly_config.yaml", - ".github/workflows/configs/weekly_config.yaml", - "tests/e2e/nightly/multi_node/external_dp/config/DeepSeek-V4-Pro-w4a8-prefix-cache-PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/DeepSeek-V4-Pro-w4a8-prefix-cache-PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/DeepSeek-V4-flash-w8a8-PD-prefix-weekly.yaml", - "tests/e2e/weekly/single_node/configs/Kimi-K2.6-w4a8-A3-weekly.yaml" - ] - }, - { - "commit": "2d1d49143f2c0efc58d5c8e80f4ddb58429ad8bb", - "subject": "[Refactor][Ops] Split fused MoE runtime dataclasses (#13888)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_fused_moe.py", - "tests/ut/_310p/fused_moe/test_moe_mlp_310.py", - "tests/ut/lora/test_quant_moe.py", - "tests/ut/ops/a2/test_token_dispatcher.py", - "tests/ut/ops/test_moe_comm_method.py", - "tests/ut/ops/test_moe_mlp.py", - "tests/ut/ops/test_moe_runtime_args.py", - "vllm_ascend/_310p/fused_moe/fused_moe.py", - "vllm_ascend/_310p/fused_moe/moe_comm_method.py", - "vllm_ascend/_310p/fused_moe/moe_mlp.py", - "vllm_ascend/_310p/fused_moe/token_dispatcher.py", - "vllm_ascend/_310p/quantization/methods/w8a8_dynamic.py", - "vllm_ascend/lora/quant_moe.py", - "vllm_ascend/ops/fused_moe/dataclass/__init__.py", - "vllm_ascend/ops/fused_moe/dataclass/fused_experts.py", - "vllm_ascend/ops/fused_moe/dataclass/moe_mlp.py", - "vllm_ascend/ops/fused_moe/dataclass/moe_quant.py", - "vllm_ascend/ops/fused_moe/dataclass/prepare_finalize.py", - "vllm_ascend/ops/fused_moe/dataclass/router_input.py", - "vllm_ascend/ops/fused_moe/dataclass/token_dispatcher.py", - "vllm_ascend/ops/fused_moe/moe_comm_method.py", - "vllm_ascend/ops/fused_moe/moe_mlp.py", - "vllm_ascend/ops/fused_moe/moe_runtime_args.py", - "vllm_ascend/ops/fused_moe/moe_stage_contracts.py", - "vllm_ascend/ops/fused_moe/moe_utils.py", - "vllm_ascend/ops/fused_moe/prepare_finalize.py", - "vllm_ascend/ops/fused_moe/routed_experts.py", - "vllm_ascend/ops/fused_moe/token_dispatcher.py", - "vllm_ascend/quantization/methods/w4a16.py", - "vllm_ascend/quantization/methods/w4a16_mxfp4.py", - "vllm_ascend/quantization/methods/w4a4_mxfp4.py", - "vllm_ascend/quantization/methods/w4a8.py", - "vllm_ascend/quantization/methods/w4a8_mxfp4.py", - "vllm_ascend/quantization/methods/w8a8_dynamic.py", - "vllm_ascend/quantization/methods/w8a8_mxfp8.py" - ] - }, - { - "commit": "c3e65c6ff7d3a8011077dad3f82ebc6619a918e4", - "subject": "[BugFix][CI] Fix GLM5.2 DSpark CI (#13890)", - "topics": [ - "deepseek-v4-dspark", - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "tests/e2e/pull_request/eight_card/test_glm5_2.py", - "tests/ut/model_executor/test_qwen3_dspark.py", - "tests/ut/spec_decode/test_dspark_proposer.py", - "vllm_ascend/models/qwen3_dspark.py", - "vllm_ascend/spec_decode/dspark_proposer.py" - ] - }, - { - "commit": "03bb92615fb4a45efdb90ca2188c318369d9d7ad", - "subject": "[CI] update nightly cases: GLM5.1-W8A8C8 GLM5.2-W8A8C8 (#13917)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "tests/e2e/weekly/multi_node/external_dp/config/GLM-5.1-W8A8C8-3.5k-1.5k-0-20-PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/GLM-5.1-W8A8C8-3.5k-1.5k-0-50-PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/GLM-5.1-W8A8C8-64k-1k-90-50-PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/GLM-5.2-W4A8C8-128k-1k-90-50-PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/GLM-5.2-W4A8C8-3.5k-1.5k-0-20-PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/GLM-5.2-W4A8C8-3.5k-1.5k-0-50-PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/GLM-5.2-W4A8C8-64k-1k-90-50-PD.yaml" - ] - }, - { - "commit": "bc8be264792f444faad66fd7fa29fce6dc1549f9", - "subject": "[Refactor][Ops] Remove unused fused_recurrent_gated_delta_rule_fwd_kernel, use AscendC op as test reference (#13769)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_fused_recurrent_gated_delta_rule.py", - "vllm_ascend/ops/triton/fla/sigmoid_gating.py", - "vllm_ascend/patch/worker/patch_triton.py" - ] - }, - { - "commit": "829c1a182f053118eb5159ff6e6a762962f992e7", - "subject": "[Refactor][device][1/N] Refactor Ascend hardware configuration and discovery (#13672)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "setup.py", - "tests/ut/device/test_device_config.py", - "tests/ut/patch/worker/test_patch_kimi_k25.py", - "vllm_ascend/device/__init__.py", - "vllm_ascend/device/device_config.py", - "vllm_ascend/device/hardware.py", - "vllm_ascend/utils.py" - ] - }, - { - "commit": "ac19e1e647785be51d22a87f336ba03c02357e18", - "subject": "[BugFix][MRV2] Unify update_stream across main and draft to fix fullgraph deadlock (#13600)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/scripts/test_config.yaml", - "tests/e2e/pull_request/two_card/model_runner_v2/test_spec_decode_data_parallel.py", - "vllm_ascend/spec_decode/llm_base_proposer.py", - "vllm_ascend/worker/model_runner_v1.py", - "vllm_ascend/worker/v2/aclgraph_utils.py", - "vllm_ascend/worker/v2/model_runner.py", - "vllm_ascend/worker/v2/spec_decode/autoregressive/speculator.py", - "vllm_ascend/worker/v2/spec_decode/dflash/aclgraph.py", - "vllm_ascend/worker/v2/spec_decode/dflash/speculator.py", - "vllm_ascend/worker/v2/spec_decode/dspark/speculator.py", - "vllm_ascend/worker/v2/spec_decode/eagle/aclgraph.py" - ] - }, - { - "commit": "30b44103d3762c03aa0c7e1e3234171234c39f09", - "subject": "[Feature][LoRA] 1.Support W8A8 dynamic multi-LoRA inference for Moe models 2.Generral moe quant frame work for Lora inference 3. quanted shared experted lora support (#13365)", - "topics": [ - "deepseek-v4-dspark", - "communication", - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/lora/test_lora.py", - "tests/ut/lora/test_quant_moe.py", - "tests/ut/ops/a2/test_token_dispatcher.py", - "tests/ut/ops/test_fused_moe.py", - "tests/ut/ops/test_moe_mlp.py", - "tests/ut/quantization/methods/a2/test_w8a8_dynamic.py", - "vllm_ascend/lora/fused_moe.py", - "vllm_ascend/lora/punica_npu.py", - "vllm_ascend/lora/quant_moe.py", - "vllm_ascend/ops/fused_moe/fused_moe.py", - "vllm_ascend/ops/fused_moe/moe_mlp.py", - "vllm_ascend/ops/fused_moe/shared_experts.py", - "vllm_ascend/ops/fused_moe/token_dispatcher.py", - "vllm_ascend/quantization/methods/w8a8_dynamic.py" - ] - }, - { - "commit": "2cd683a396e5b8771b49b6e2ff941352458aabd7", - "subject": "[Test] Fix the weekly test case configuration (#13878)", - "topics": [ - "deepseek-v4-dspark", - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/configs/nightly_config.yaml", - "tests/e2e/nightly/multi_node/external_dp/config/DeepSeek-V4-Pro-w4a8-1M-PD.yaml", - "tests/e2e/nightly/multi_node/external_dp/config/DeepSeek-V4-flash-w8a8-PD-prefix.yaml", - "tests/e2e/nightly/multi_node/external_dp/config/Kimi-K2.6-W4A8-64k-1k-TPOT50-PD.yaml", - "tests/e2e/nightly/multi_node/external_dp/config/Minimax_m2.7_in128k_1k_prefix90_tpot50.yaml", - "tests/e2e/nightly/single_node/models/configs/Kimi-K2.6-w4a8-A3.yaml", - "tests/e2e/nightly/single_node/models/configs/Minimax_m2.7_w8a8_A3.yaml", - "tests/e2e/weekly/multi_node/internal_dp/config/GLM-5.1-W8A8C8-A3_64k_128k_90_50.yaml", - "tests/e2e/weekly/single_node/configs/Kimi-K2.6-w4a8-A3.yaml", - "tests/e2e/weekly/single_node/configs/Minimax_m2.7_w8a8_A3.yaml" - ] - }, - { - "commit": "efda0446db94fcfc57682d02dee3276fa8dea31e", - "subject": "[Refactor][quantization]Remove deprecated W4A8Linear/W8A8PDMixMoE schemes and sync W4A8 MoE refactor (#13713)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/workflows/misc/model_dataset_list.json", - "docs/source/developer_guide/Design_Documents/quantization.md", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/Design_Documents/quantization.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-Dense.po", - "docs/source/tutorials/models/Qwen3-Dense.md", - "examples/quantization/llm-compressor/w4a8_dynamic_moe.py", - "examples/quantization/llm-compressor/w8a8_int8.py", - "examples/quantization/llm-compressor/w8a8_int8_dynamic.py", - "examples/quantization/llm-compressor/w8a8_int8_dynamic_moe.py", - "tests/e2e/nightly/single_node/models/configs/Qwen3-30B-A3B-W4A8-llm-compressor.yaml", - "tests/ut/quantization/methods/a2/test_w4a8.py", - "tests/ut/quantization/methods/test_w4a8.py", - "tests/ut/quantization/methods/test_w8a8_pdmix.py", - "vllm_ascend/quantization/methods/__init__.py", - "vllm_ascend/quantization/methods/w4a8.py", - "vllm_ascend/quantization/methods/w8a8_pdmix.py" - ] - }, - { - "commit": "168ecfcba983fa326784c49e091f6a7f74c0b7b6", - "subject": "[Refactor][Ops] Simplify fused MoE runtime contracts (#13759)", - "topics": [ - "deepseek-v4-dspark", - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/_310p/fused_moe/test_shared_fused_moe_310.py", - "tests/ut/ops/test_fused_moe.py", - "tests/ut/ops/test_moe_runtime_args.py", - "vllm_ascend/_310p/fused_moe/fused_moe.py", - "vllm_ascend/_310p/quantization/methods/w8a8_dynamic.py", - "vllm_ascend/models/qwen3_dspark.py", - "vllm_ascend/ops/fused_moe/fused_moe.py", - "vllm_ascend/ops/fused_moe/moe_comm_method.py", - "vllm_ascend/ops/fused_moe/moe_runtime_args.py", - "vllm_ascend/ops/fused_moe/moe_utils.py", - "vllm_ascend/ops/fused_moe/routed_experts.py", - "vllm_ascend/ops/fused_moe/router/router_factory.py", - "vllm_ascend/ops/fused_moe/shared_experts.py", - "vllm_ascend/patch/platform/patch_fused_moe.py", - "vllm_ascend/quantization/methods/w4a16.py", - "vllm_ascend/quantization/methods/w4a16_mxfp4.py", - "vllm_ascend/quantization/methods/w4a4_mxfp4.py", - "vllm_ascend/quantization/methods/w4a8.py", - "vllm_ascend/quantization/methods/w4a8_mxfp4.py", - "vllm_ascend/quantization/methods/w8a8_dynamic.py", - "vllm_ascend/quantization/methods/w8a8_mxfp8.py", - "vllm_ascend/utils.py" - ] - }, - { - "commit": "21eedaf041722d9f5fb9c3fe8857d8c161d511a5", - "subject": "[CI]Skip error glm5.2 dspark test (#13832)", - "topics": [ - "deepseek-v4-dspark", - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "tests/e2e/pull_request/eight_card/test_glm5_2.py" - ] - }, - { - "commit": "d6e8f1ad4bd67cc48a349cd7cf8efab98eecad13", - "subject": "[Misc] Remove unused fused_sigmoid_gating_delta_rule_update (#13725)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_fused_sigmoid_gating_delta_rule.py", - "vllm_ascend/ops/triton/fla/sigmoid_gating.py" - ] - }, - { - "commit": "a6ae7f57601e9bafdc18ec1e3a5035e53d5fad6f", - "subject": "Revert \"[Performance] remove D2H sync in QLIMetadata builder for DSA_CP\" (#13290)", - "topics": [ - "communication" - ], - "runtime_patch_match": false, - "paths": [ - "tests/e2e/pull_request/one_card/model_runner_v2/test_basic.py", - "vllm_ascend/attention/context_parallel/dsa_cp.py" - ] - }, - { - "commit": "a533aed9815e1ba638a1aa5254f0fb11c15df4fd", - "subject": "[Refactor][DSA] Move TP weight switching into quantization methods (#12982)", - "topics": [ - "deepseek-v4-dspark", - "communication" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/attention/test_dsa_cp_o_proj_tp.py", - "tests/ut/attention/test_sfa_o_proj_tp.py", - "tests/ut/distributed/ascend_store/_mock_deps.py", - "tests/ut/quantization/test_tp_weight_switch.py", - "vllm_ascend/attention/context_parallel/dsa_cp.py", - "vllm_ascend/attention/context_parallel/sfa_cp.py", - "vllm_ascend/attention/sfa_v1.py", - "vllm_ascend/ops/linear.py", - "vllm_ascend/quantization/compressed_tensors_config.py", - "vllm_ascend/quantization/methods/__init__.py", - "vllm_ascend/quantization/methods/base.py", - "vllm_ascend/quantization/methods/fp8.py", - "vllm_ascend/quantization/methods/w4a4_mxfp4.py", - "vllm_ascend/quantization/methods/w4a8_mxfp4.py", - "vllm_ascend/quantization/methods/w8a8_dynamic.py", - "vllm_ascend/quantization/methods/w8a8_mxfp8.py", - "vllm_ascend/quantization/methods/w8a8_static.py", - "vllm_ascend/quantization/methods/w8a8fp8_dynamic.py", - "vllm_ascend/quantization/tp_weight_switch.py" - ] - }, - { - "commit": "77d13e2c0bfa9862b711b68811889e36b9d56aca", - "subject": "[Performance] Add fused torch_npu chunk gated delta rule for GDN prefill (#12647)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/ops/gdn.py" - ] - }, - { - "commit": "7e5ad066648b389ed5d04e6fc8814d818451704c", - "subject": "[Test][Misc] Modify weekly E2E test configurations (#13368)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "tests/e2e/weekly/multi_node/external_dp/config/DeepSeek-V4-Pro-w4a8-1M-PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/DeepSeek-V4-Pro-w4a8-prefix-cache-PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/DeepSeek-V4-flash-w8a8-PD-no-prefix.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/DeepSeek-V4-flash-w8a8-PD-prefix.yaml", - "tests/e2e/weekly/single_node/configs/Qwen3.5-122B-A10B-W8A8-A3.yaml" - ] - }, - { - "commit": "c872c387bb18c5ae16c91b0bcf06655007c20d37", - "subject": "[CI]update GLM5.1-w8a8c8 weekly cases (#13370)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/configs/weekly_config.yaml", - ".github/workflows/misc/model_dataset_list.json", - "tests/e2e/weekly/multi_node/external_dp/config/GLM-5.1-W8A8C8-128k-1k-90-50-PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/GLM-5.1-W8A8C8-3.5k-1.5k-0-20-PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/GLM-5.1-W8A8C8-3.5k-1.5k-0-50-PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/GLM-5.1-W8A8C8-64k-1k-90-50-PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/GLM-5.2-W4A8C8-128k-1k-90-50-PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/GLM-5.2-W4A8C8-3.5k-1.5k-0-20-PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/GLM-5.2-W4A8C8-3.5k-1.5k-0-50-PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/GLM-5.2-W4A8C8-64k-1k-90-50-PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/Qwen-3.5-397B-A17B-W8A8-PD.yaml", - "tests/e2e/weekly/multi_node/internal_dp/config/DeepSeek-V3_2-W8A8-EP_weekly.yaml", - "tests/e2e/weekly/multi_node/internal_dp/config/GLM-5.1-W8A8C8-A3_198k_function.yaml", - "tests/e2e/weekly/multi_node/internal_dp/config/GLM-5.1-W8A8C8-A3_3.5k_1.5k_0_50.yaml", - "tests/e2e/weekly/multi_node/internal_dp/config/GLM-5.1-W8A8C8-A3_64k_128k_90_50.yaml", - "tests/e2e/weekly/multi_node/internal_dp/config/GLM-5.2-W4A8C8-A3_198k_function.yaml", - "tests/e2e/weekly/multi_node/internal_dp/config/GLM-5.2-W4A8C8-A3_3.5k_1.5k_0_50.yaml", - "tests/e2e/weekly/multi_node/internal_dp/config/GLM-5.2-W4A8C8-A3_64k_128k_90_50.yaml" - ] - }, - { - "commit": "041f111a17930dc9a096b79d6144567333cba87c", - "subject": "[BugFix] Fix GLM-5.2 DSpark acceptance rate problem (#13530)", - "topics": [ - "deepseek-v4-dspark", - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/labled_download_model_dataset.yaml", - ".github/workflows/misc/model_dataset_list.json", - ".github/workflows/scripts/test_config.yaml", - "tests/e2e/nightly/single_node/models/configs/GLM-5.2-W8A8-DSpark.yaml", - "tests/e2e/pull_request/eight_card/test_glm5_2.py", - "tests/ut/spec_decode/test_dspark_proposer.py", - "vllm_ascend/spec_decode/dspark_proposer.py" - ] - }, - { - "commit": "683b6c009d57714eaf9363f369c7ed4fa0084f5c", - "subject": "[Feature] Dspark in A5 (#13506)", - "topics": [ - "deepseek-v4-dspark", - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/workflows/scripts/test_config.yaml", - "csrc/attention/kv_quant_sparse_attn_sharedkv/op_host/kv_quant_sparse_attn_sharedkv_check.h", - "csrc/attention/kv_quant_sparse_attn_sharedkv/op_host/kv_quant_sparse_attn_sharedkv_check_existance.cpp", - "csrc/attention/kv_quant_sparse_attn_sharedkv/op_host/kv_quant_sparse_attn_sharedkv_check_feature.cpp", - "csrc/attention/kv_quant_sparse_attn_sharedkv/op_host/kv_quant_sparse_attn_sharedkv_tiling.cpp", - "csrc/attention/kv_quant_sparse_attn_sharedkv/op_host/kv_quant_sparse_attn_sharedkv_tiling.h", - "csrc/attention/kv_quant_sparse_attn_sharedkv/op_kernel/arch35/kv_quant_sparse_attn_sharedkv_scfa_block_vector.h", - "csrc/attention/kv_quant_sparse_attn_sharedkv/op_kernel/arch35/kv_quant_sparse_attn_sharedkv_scfa_kernel.h", - "csrc/attention/kv_quant_sparse_attn_sharedkv/op_kernel/arch35/util_regbase.h", - "csrc/attention/kv_quant_sparse_attn_sharedkv/op_kernel/kv_quant_sparse_attn_sharedkv.cpp", - "csrc/attention/kv_quant_sparse_attn_sharedkv_metadata/op_kernel_aicpu/kv_quant_sparse_attn_sharedkv_metadata_aicpu.cpp", - "vllm_ascend/models/deepseek_v4_dspark.py", - "vllm_ascend/ops/fused_moe/shared_experts.py", - "vllm_ascend/quantization/methods/fp8.py" - ] - }, - { - "commit": "1bff043fe717aaafeacf1051313ac9b8256311d0", - "subject": "[Feature][MRV2] Support DSA in ModelRunner V2 (#13155)", - "topics": [ - "deepseek-v4-dspark", - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/_310p/fused_moe/test_shared_fused_moe_310.py", - "tests/ut/models/test_deepseek_v4_moe.py", - "tests/ut/ops/test_fused_moe.py", - "tests/ut/test_ascend_forward_context.py", - "tests/ut/worker/test_attn_utils_v2.py", - "vllm_ascend/ascend_forward_context.py", - "vllm_ascend/attention/dsa_v1.py", - "vllm_ascend/attention/mla_v1.py", - "vllm_ascend/core/kv_cache_interface.py", - "vllm_ascend/models/deepseek_v4.py", - "vllm_ascend/models/deepseek_v4_dspark.py", - "vllm_ascend/models/deepseek_v4_mtp.py", - "vllm_ascend/models/layer/attention/layer.py", - "vllm_ascend/ops/dsa.py", - "vllm_ascend/ops/fused_moe/fused_moe.py", - "vllm_ascend/ops/fused_moe/prepare_finalize.py", - "vllm_ascend/ops/fused_moe/routed_experts.py", - "vllm_ascend/ops/fused_moe/router/fused_topk_router.py", - "vllm_ascend/patch/__init__.py", - "vllm_ascend/patch/worker/__init__.py", - "vllm_ascend/patch/worker/patch_bind_kv_cache.py", - "vllm_ascend/patch/worker/patch_v2/patch_attn_utils.py", - "vllm_ascend/worker/model_runner_v1.py", - "vllm_ascend/worker/v2/attn_utils.py", - "vllm_ascend/worker/v2/block_table.py", - "vllm_ascend/worker/v2/model_runner.py", - "vllm_ascend/worker/v2/model_states/default.py" - ] - }, - { - "commit": "8796762978f045c5e5d0cb281b7f46ddb20ff121", - "subject": "[Feature]Add Confidence head and Dynamic Lengths to Verification Pipeline (#13216)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/scripts/test_config.yaml", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/configuration/additional_config.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/speculative_decoding.po", - "docs/source/user_guide/configuration/additional_config.md", - "docs/source/user_guide/feature_guide/speculative_decoding.md", - "tests/e2e/pull_request/one_card/spec_decode/test_dynamic.py", - "tests/e2e/pull_request/one_card/spec_decode/utils.py", - "tests/ut/model_executor/test_qwen3_dspark.py", - "vllm_ascend/ascend_config.py", - "vllm_ascend/models/qwen3_dspark.py", - "vllm_ascend/spec_decode/dspark_proposer.py", - "vllm_ascend/spec_decode/llm_base_proposer.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "032b3f4a040592f9cd0c3775ff300a23cb8ddbe2", - "subject": "[BugFix][Worker] Fix Level-2 sleep MoE weight reload layout (#13012)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/sleep_mode.po", - "docs/source/user_guide/feature_guide/sleep_mode.md", - "examples/offline_external_launcher.py", - "examples/offline_weight_load.py", - "tests/ut/ops/test_fused_moe.py", - "tests/ut/worker/a2/test_worker_v1.py", - "vllm_ascend/ops/fused_moe/routed_experts.py", - "vllm_ascend/worker/worker.py" - ] - }, - { - "commit": "8ef4255b0d72932f93e7a96c3e6c2d4180a8dbec", - "subject": "[Doc][Misc] Update GLM-5 and GLM-5.2 deployment guides (#13429)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/tutorials/models/GLM5.2.md", - "docs/source/tutorials/models/GLM5.md" - ] - }, - { - "commit": "40348c31de871c3dd0595a1ec98a899bcd5fe3d6", - "subject": "[Refactor][Ops] Remove legacy fused MoE helper modules (#13621)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_fused_moe.py", - "tests/ut/_310p/fused_moe/test_experts_selector_310.py", - "tests/ut/_310p/fused_moe/test_shared_fused_moe_310.py", - "tests/ut/_310p/quantization/test_w8a8_dynamic_moe_310.py", - "tests/ut/ops/a3_2/test_select_experts.py", - "tests/ut/ops/test_comm_utils.py", - "tests/ut/ops/test_fused_moe.py", - "tests/ut/quantization/methods/a2/test_w4a16.py", - "tests/ut/quantization/methods/a2/test_w4a8.py", - "tests/ut/quantization/methods/a2/test_w8a8_dynamic.py", - "tests/ut/quantization/methods/test_moe_logical_experts.py", - "tests/ut/quantization/methods/test_w4a16_mxfp4.py", - "tests/ut/quantization/methods/test_w4a4_mxfp4.py", - "tests/ut/quantization/methods/test_w4a8_mxfp4.py", - "tests/ut/quantization/methods/test_w8a8_mxfp8.py", - "tests/ut/quantization/methods/test_w8a8fp8_dynamic.py", - "vllm_ascend/_310p/fused_moe/experts_selector.py", - "vllm_ascend/_310p/fused_moe/fused_moe.py", - "vllm_ascend/_310p/quantization/methods/w8a8_dynamic.py", - "vllm_ascend/lora/fused_moe.py", - "vllm_ascend/ops/fused_moe/experts_selector.py", - "vllm_ascend/ops/fused_moe/fused_moe.py", - "vllm_ascend/ops/fused_moe/moe_comm_method.py", - "vllm_ascend/ops/fused_moe/moe_utils.py", - "vllm_ascend/ops/fused_moe/routed_experts.py", - "vllm_ascend/ops/fused_moe/token_dispatcher.py", - "vllm_ascend/quantization/methods/base.py", - "vllm_ascend/quantization/methods/w4a16.py", - "vllm_ascend/quantization/methods/w4a16_mxfp4.py", - "vllm_ascend/quantization/methods/w4a4_mxfp4.py", - "vllm_ascend/quantization/methods/w4a8.py", - "vllm_ascend/quantization/methods/w4a8_mxfp4.py", - "vllm_ascend/quantization/methods/w8a8_dynamic.py", - "vllm_ascend/quantization/methods/w8a8_mxfp8.py" - ] - }, - { - "commit": "8be8909d4ecf3f44430f2e54ddda82f6980f6fd0", - "subject": "[Refactor][DSA][1/N] Remove dsa_v1 dead code (#13691)", - "topics": [ - "communication" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/attention/context_parallel/dsa_cp.py", - "vllm_ascend/attention/dsa_v1.py" - ] - }, - { - "commit": "7314822940a5c201b68ebd6e4db7deac3067ccfb", - "subject": "[Doc][Model] Add MiniMax-M3 tutorial (#13476)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "docs/hooks/nav_titles.py", - "docs/source/tutorials/models/MiniMax-M3.md", - "docs/source/user_guide/support_matrix/supported_models.md", - "mkdocs.yml" - ] - }, - { - "commit": "dab1c351787a219f8a28fa2bb0b1a8c67baa9617", - "subject": "[Refactor][DSA] remove useless abstract base class for DSA (#13226)", - "topics": [ - "communication" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/scripts/test_config.yaml", - "vllm_ascend/attention/abstract.py", - "vllm_ascend/attention/context_parallel/dsa_cp.py", - "vllm_ascend/attention/dsa_v1.py", - "vllm_ascend/models/layer/attention/layer.py" - ] - }, - { - "commit": "0703581e8d530d2ae8079a740ec70843883add5f", - "subject": "[Feature][EPLB] Integrate upstream EPLB with Model Runner V2 (#12804)", - "topics": [ - "deepseek-v4-dspark", - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/workflows/scripts/test_config.yaml", - "docs/hooks/nav_titles.py", - "docs/source/developer_guide/Design_Documents/model_runner_v2_eplb.md", - "docs/source/user_guide/configuration/additional_config.md", - "docs/source/user_guide/feature_guide/expert_parallelism_load_balancer.md", - "docs/source/user_guide/support_matrix/feature_matrix.md", - "docs/source/user_guide/support_matrix/supported_features.md", - "mkdocs.yml", - "tests/e2e/conftest.py", - "tests/e2e/pull_request/four_card/test_qwen3_mrv2_eplb.py", - "tests/ut/distributed/test_eplb_communicator.py", - "tests/ut/distributed/test_eplb_state.py", - "tests/ut/models/test_deepseek_v4_moe.py", - "tests/ut/ops/test_eplb.py", - "tests/ut/ops/test_fused_moe.py", - "tests/ut/ops/test_routed_experts.py", - "tests/ut/patch/platform/test_patch_eplb.py", - "tests/ut/patch/platform/test_patch_fused_moe.py", - "tests/ut/quantization/methods/a2/test_w4a8.py", - "tests/ut/quantization/methods/a2/test_w8a8_dynamic.py", - "tests/ut/quantization/methods/test_moe_logical_experts.py", - "tests/ut/quantization/methods/test_w4a4_mxfp4.py", - "tests/ut/quantization/methods/test_w8a8_mxfp8.py", - "tests/ut/quantization/test_method_adapters.py", - "tests/ut/test_ascend_config.py", - "tests/ut/test_logger.py", - "tests/ut/test_platform.py", - "tests/ut/worker/v2/test_eplb_load_collection_phase.py", - "vllm_ascend/ascend_config.py", - "vllm_ascend/distributed/eplb_communicator.py", - "vllm_ascend/distributed/eplb_state.py", - "vllm_ascend/models/deepseek_v4.py", - "vllm_ascend/ops/fused_moe/eplb.py", - "vllm_ascend/ops/fused_moe/routed_experts.py", - "vllm_ascend/patch/__init__.py", - "vllm_ascend/patch/platform/__init__.py", - "vllm_ascend/patch/platform/patch_eplb.py", - "vllm_ascend/patch/platform/patch_fused_moe.py", - "vllm_ascend/platform.py", - "vllm_ascend/quantization/method_adapters.py", - "vllm_ascend/quantization/methods/base.py", - "vllm_ascend/quantization/methods/w4a16.py", - "vllm_ascend/quantization/methods/w4a16_mxfp4.py", - "vllm_ascend/quantization/methods/w4a4_mxfp4.py", - "vllm_ascend/quantization/methods/w4a8.py", - "vllm_ascend/quantization/methods/w4a8_mxfp4.py", - "vllm_ascend/quantization/methods/w8a8_dynamic.py", - "vllm_ascend/quantization/methods/w8a8_mxfp8.py", - "vllm_ascend/worker/v2/eplb.py", - "vllm_ascend/worker/v2/model_runner.py" - ] - }, - { - "commit": "9f3aa1e70ad16648e1c42f746cda28fdf16c3929", - "subject": "[BugFix][MoE] Fix W8A8 scale handling for dynamic EPLB (#13414)", - "topics": [ - "priority-models", - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/ops/test_moe_mlp.py", - "vllm_ascend/ops/fused_moe/moe_mlp.py" - ] - }, - { - "commit": "ec4a9b9ca93afab7acba60eb104765b84bff58e4", - "subject": "[BugFix][Core][Scheduler] Sync recompute cache handling (#13503)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/core/recompute_scheduler.py" - ] - }, - { - "commit": "70f69d9a3e9e92b6b03764dacfa7ff85fc7fbffb", - "subject": "[BugFix][Core] Fix Qwen3-VL MoE residual corruption under FlashComm1 with deepstack (#13538)", - "topics": [ - "communication" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/compilation/passes/sequence_parallelism.py", - "vllm_ascend/ops/register_custom_ops.py" - ] - }, - { - "commit": "bc5a79ede16fc4ded332471ae1fbc6303fb000f4", - "subject": "[BugFix][RFork] Consolidate transfer layout and fallback fixes (#12995)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/model_loader/rfork/test_rfork_loader.py", - "tests/ut/model_loader/rfork/test_transfer_backend.py", - "vllm_ascend/model_loader/rfork/rfork_loader.py", - "vllm_ascend/model_loader/rfork/rfork_worker.py", - "vllm_ascend/model_loader/rfork/transfer_backend.py" - ] - }, - { - "commit": "9a52ca5fc36c1852241822863c50717bee5dc761", - "subject": "[Feature] SD spec decode op (#12368)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/user_guide/feature_guide/speculative_decoding.md", - "examples/offline_DSD_k_tuner/generate_config.py", - "examples/offline_DSD_k_tuner/manager.py", - "tests/e2e/pull_request/one_card/spec_decode/test_dflash.py", - "tests/e2e/pull_request/one_card/spec_decode/test_eagle.py", - "tests/e2e/pull_request/one_card/spec_decode/test_mtp_eagle_correctness.py", - "tests/ut/spec_decode/a2/test_eagle_proposer.py", - "tests/ut/spec_decode/test_dynamic_sd.py", - "vllm_ascend/platform.py", - "vllm_ascend/spec_decode/llm_base_proposer.py", - "vllm_ascend/spec_decode/medusa_proposer.py", - "vllm_ascend/spec_decode/step3p5.py", - "vllm_ascend/spec_decode/suffix_proposer.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "d6d93c9c3eca63717610715f675eff85c742dd8c", - "subject": "[BugFix] Restore synchronous AscendStore save (#13120)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/distributed/ascend_store/test_pool_worker.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/ascend_store/pool_worker.py" - ] - }, - { - "commit": "10d81b8d29224b99039821f015fbe485fc7b7bdc", - "subject": "[Doc] Clarify fine-grained TP interplay with standard tensor_parallel_size (#13488)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/user_guide/feature_guide/Fine_grained_TP.md" - ] - }, - { - "commit": "3829122510c00dfc6b4b94d6f96c947a7590043c", - "subject": "[DOC] Add configuration guidance for DeepSeek-V4-Flash DSpark (#13367)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/tutorials/models/DeepSeek-V4-Flash.md" - ] - }, - { - "commit": "2627f62bf8a48ccdf87e1f9b38d2efb86142c7c8", - "subject": "[Refactor][quantization]remove mxfp_compat compatibility shim and inline dtype references (#13447)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/conftest.py", - "tests/ut/quantization/methods/test_w4a16_mxfp4.py", - "tests/ut/quantization/methods/test_w4a4_mxfp4.py", - "tests/ut/quantization/methods/test_w4a4_mxfp4_flatquant_dynamic.py", - "tests/ut/quantization/methods/test_w4a8_mxfp4.py", - "tests/ut/quantization/methods/test_w8a8_mxfp8.py", - "tests/ut/quantization/test_quant_parser.py", - "vllm_ascend/attention/sfa_v1.py", - "vllm_ascend/compilation/passes/norm_quant_fusion_pass.py", - "vllm_ascend/device/device_op.py", - "vllm_ascend/device/mxfp_compat.py", - "vllm_ascend/ops/fused_moe/moe_mlp.py", - "vllm_ascend/quantization/methods/w4a16_mxfp4.py", - "vllm_ascend/quantization/methods/w4a4_mxfp4.py", - "vllm_ascend/quantization/methods/w4a4_mxfp4_flatquant.py", - "vllm_ascend/quantization/methods/w4a8_mxfp4.py", - "vllm_ascend/quantization/methods/w8a8_mxfp8.py", - "vllm_ascend/quantization/quant_parser.py", - "vllm_ascend/quantization/utils.py" - ] - }, - { - "commit": "bdab23475a7dc72333d0c51a2bbc8ebbecb97b69", - "subject": "[MTP][BugFix] Preserve MoE weight loaders during online updates (#13337)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/ops/test_fused_moe.py", - "tests/ut/worker/a2/test_worker_v1.py", - "vllm_ascend/ops/fused_moe/routed_experts.py", - "vllm_ascend/worker/worker.py" - ] - }, - { - "commit": "37572711d15a1a022ad540ad0f83802f0516ddc0", - "subject": "[Feature][xlite] Support MLA and DSA (`DeepseekV3ForCausalLM`, `DeepseekV32ForCausalLM` and `GlmMoeDsaForCausalLM`) in the xlite adapter (#13378)", - "topics": [ - "priority-models", - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/configuration/additional_config.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/graph_mode.po", - "docs/source/user_guide/configuration/additional_config.md", - "docs/source/user_guide/feature_guide/graph_mode.md", - "vllm_ascend/xlite/xlite.py" - ] - }, - { - "commit": "26f350eadbb83dd57fbe198fbb2c096657322ffe", - "subject": "[Refactor][Ops] Move expert routing into router classes (#13417)", - "topics": [ - "deepseek-v4-dspark", - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/_310p/fused_moe/test_experts_selector_310.py", - "tests/ut/_310p/fused_moe/test_shared_fused_moe_310.py", - "tests/ut/_310p/quantization/test_w8a8_dynamic_moe_310.py", - "tests/ut/ops/test_fused_moe.py", - "tests/ut/quantization/methods/a2/test_w4a16.py", - "tests/ut/quantization/methods/a2/test_w4a8.py", - "tests/ut/quantization/methods/a2/test_w8a8_dynamic.py", - "tests/ut/quantization/methods/test_w4a16_mxfp4.py", - "tests/ut/quantization/methods/test_w4a4_mxfp4.py", - "tests/ut/quantization/methods/test_w4a8_mxfp4.py", - "tests/ut/quantization/methods/test_w8a8_mxfp8.py", - "tests/ut/quantization/methods/test_w8a8fp8_dynamic.py", - "tests/ut/quantization/test_method_adapters.py", - "vllm_ascend/_310p/fused_moe/experts_selector.py", - "vllm_ascend/_310p/fused_moe/fused_moe.py", - "vllm_ascend/_310p/fused_moe/grouped_topk_router.py", - "vllm_ascend/_310p/quantization/methods/w8a8_dynamic.py", - "vllm_ascend/models/deepseek_v4.py", - "vllm_ascend/ops/fused_moe/routed_experts.py", - "vllm_ascend/ops/fused_moe/router/__init__.py", - "vllm_ascend/ops/fused_moe/router/fused_topk_router.py", - "vllm_ascend/ops/fused_moe/router/grouped_topk_router.py", - "vllm_ascend/ops/fused_moe/router/router_factory.py", - "vllm_ascend/patch/platform/patch_fused_moe.py", - "vllm_ascend/quantization/method_adapters.py", - "vllm_ascend/quantization/methods/base.py", - "vllm_ascend/quantization/methods/w4a16.py", - "vllm_ascend/quantization/methods/w4a16_mxfp4.py", - "vllm_ascend/quantization/methods/w4a4_mxfp4.py", - "vllm_ascend/quantization/methods/w4a8.py", - "vllm_ascend/quantization/methods/w4a8_mxfp4.py", - "vllm_ascend/quantization/methods/w8a8_dynamic.py", - "vllm_ascend/quantization/methods/w8a8_mxfp8.py" - ] - }, - { - "commit": "2c47591044037733ce14fbe509803eb213c57fab", - "subject": "[Platform][Refactor] Refactor NPUPlatform for better organization and clarity (#13484)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/core/test_profiling_chunk.py", - "tests/ut/eplb/core/a2/test_eplb_utils.py", - "tests/ut/test_platform.py", - "vllm_ascend/platform.py" - ] - }, - { - "commit": "0b5dd06ec999831315e4822ae244f0d0ce302716", - "subject": "[BugFix] fix fiaV2 contiguous err in GQA (#13456)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/attention/attention_v1.py" - ] - }, - { - "commit": "646684f43ce4bdc737203a8df1149e37ea2ff824", - "subject": "[BugFix][FusedMoE] Restore BF16 quant method initialization (#13412)", - "topics": [ - "priority-models", - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/_310p/fused_moe/test_shared_fused_moe_310.py", - "tests/ut/ops/test_fused_moe.py", - "vllm_ascend/_310p/fused_moe/fused_moe.py", - "vllm_ascend/ops/fused_moe/routed_experts.py" - ] - }, - { - "commit": "b80d4e621a303d1a4077db5e4d80586d2446fb8f", - "subject": "[Doc] Fix link errors and add section anchors (#13485)", - "topics": [ - "deepseek-v4-dspark", - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/_templates/Model-Deployment-Tutorial-Template.md", - "docs/source/quick_start.md", - "docs/source/tutorials/models/DeepSeek-R1.md", - "docs/source/tutorials/models/DeepSeek-V3.1.md", - "docs/source/tutorials/models/DeepSeek-V3.2.md", - "docs/source/tutorials/models/DeepSeek-V4-Flash.md", - "docs/source/tutorials/models/DeepSeek-V4-Pro.md", - "docs/source/tutorials/models/DeepSeekOCR2.md", - "docs/source/tutorials/models/GLM4.x.md", - "docs/source/tutorials/models/GLM5.md", - "docs/source/tutorials/models/Gemma4.md", - "docs/source/tutorials/models/InternVL3.5.md", - "docs/source/tutorials/models/Kimi-K2-Thinking.md", - "docs/source/tutorials/models/Kimi-K2.6.md", - "docs/source/tutorials/models/MiniMax-M2.md", - "docs/source/tutorials/models/PaddleOCR-VL.md", - "docs/source/tutorials/models/Qwen3-235B-A22B.md", - "docs/source/tutorials/models/Qwen3-30B-A3B.md", - "docs/source/tutorials/models/Qwen3-ASR-1.7B.md", - "docs/source/tutorials/models/Qwen3-Coder-30B-A3B.md", - "docs/source/tutorials/models/Qwen3-Dense.md", - "docs/source/tutorials/models/Qwen3-Embedding.md", - "docs/source/tutorials/models/Qwen3-Next.md", - "docs/source/tutorials/models/Qwen3-Omni-30B-A3B-Thinking.md", - "docs/source/tutorials/models/Qwen3-Reranker.md", - "docs/source/tutorials/models/Qwen3-VL-235B-A22B-Instruct.md", - "docs/source/tutorials/models/Qwen3-VL-30B-A3B-Instruct.md", - "docs/source/tutorials/models/Qwen3-VL-Embedding.md", - "docs/source/tutorials/models/Qwen3-VL-Reranker.md", - "docs/source/tutorials/models/Qwen3.5-27B-Qwen3.6-27B.md", - "docs/source/tutorials/models/Qwen3.5-397B-A17B.md", - "docs/source/tutorials/models/Qwen3.5-Dense.md", - "docs/source/tutorials/models/Qwen3.6-35B-A3B.md" - ] - }, - { - "commit": "12df7bef2440c1b393322adf7191012c6137b5a2", - "subject": "[Doc] Add dspark/dflash sections and complete offline/online examples (#13388)", - "topics": [ - "deepseek-v4-dspark", - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/user_guide/feature_guide/speculative_decoding.md" - ] - }, - { - "commit": "0af2fa3a018b40e9d55caa2eb04dca52f3574353", - "subject": "[BugFix] Report prefiller cached tokens in PD proxy\uff0cadded first curl and non-streaming scenarios (#13040)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "examples/disaggregated_prefill_v1/load_balance_proxy_server_example.py" - ] - }, - { - "commit": "61221e9add8c717b304005bd9d48d6215d035be7", - "subject": "[Doc] Translated Doc files 2026-08-04 (#13450)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/GLM5.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Kimi-K2-Thinking.po" - ] - }, - { - "commit": "c8f9064617af1c58beb1875dc7f39968db69a35c", - "subject": "[MRV2][Feature] Add FlashComm1 support to Model Runner V2 (#12992)", - "topics": [ - "communication" - ], - "runtime_patch_match": false, - "paths": [ - "tests/e2e/pull_request/two_card/test_flashcomm_distributed.py", - "tests/ut/worker/test_model_runner_v2_flashcomm.py", - "vllm_ascend/ops/register_custom_ops.py", - "vllm_ascend/worker/v2/model_runner.py", - "vllm_ascend/worker/v2/sp_utils.py" - ] - }, - { - "commit": "bf016a1bc151ec1afcdeee50a9542b123009ce93", - "subject": "[CI] vllm main2main 0731 0351e9a (#13358)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/vllm-main-verified.commit", - "examples/rl/rlhf_http_npu_ipc.py", - "tests/e2e/pull_request/one_card/test_guided_decoding.py", - "tests/e2e/pull_request/one_card/test_npu_ipc_weight_transfer.py", - "tests/ut/distributed/weight_transfer/test_npu_ipc_engine.py", - "tests/ut/patch/platform/test_patch_structured_output.py", - "vllm_ascend/_310p/ops/vocab_parallel_embedding.py", - "vllm_ascend/__init__.py", - "vllm_ascend/distributed/device_communicators/npu_communicator.py", - "vllm_ascend/distributed/kv_transfer/kv_p2p/mooncake_connector.py", - "vllm_ascend/distributed/kv_transfer/kv_p2p/mooncake_hybrid_connector.py", - "vllm_ascend/distributed/weight_transfer/__init__.py", - "vllm_ascend/distributed/weight_transfer/npu_ipc_engine.py", - "vllm_ascend/ops/fused_moe/fused_moe.py", - "vllm_ascend/ops/vocab_parallel_embedding.py", - "vllm_ascend/patch/worker/patch_distributed.py", - "vllm_ascend/worker/npu_input_batch.py", - "vllm_ascend/worker/v2/spec_decode/dflash/speculator.py", - "vllm_ascend/worker/v2/spec_decode/eagle/aclgraph.py" - ] - }, - { - "commit": "6d3bed3d3ceb05a6b5ce568badfee4725f47408d", - "subject": "[CI] Change default main2main model from dsv4-pro to dsv4-flash (#13409)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/schedule_main2main.yaml" - ] - }, - { - "commit": "97f72b814140520e7a20622dc76b2d2fcdca0f7a", - "subject": "[Refactor][FusedMoE] Split shared expert executor (#13328)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/_310p/fused_moe/test_shared_fused_moe_310.py", - "tests/ut/ops/a3_2/test_select_experts.py", - "tests/ut/ops/test_fused_moe.py", - "vllm_ascend/_310p/fused_moe/fused_moe.py", - "vllm_ascend/ops/fused_moe/fused_moe.py", - "vllm_ascend/ops/fused_moe/routed_experts.py", - "vllm_ascend/ops/fused_moe/shared_experts.py" - ] - }, - { - "commit": "4ac4860d4bb4a43213ecae8a4acf832ea17f72a8", - "subject": "[Core][BugFix][Test] Fix sequence parallelism for both MoE and dense model (#12772)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/e2e/pull_request/two_card/test_sequence_parallelism_moe.py", - "vllm_ascend/compilation/passes/sequence_parallelism.py", - "vllm_ascend/compilation/passes/sequence_parallelism_moe.py", - "vllm_ascend/ops/fused_moe/fused_moe.py", - "vllm_ascend/ops/fused_moe/prepare_finalize.py", - "vllm_ascend/ops/register_custom_ops.py" - ] - }, - { - "commit": "fd9298242a923fe85546564cb7d30a2d2fe9d688", - "subject": "[Performance][MiniMax M3] Optimize index score kernel and TP communication (#13142)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/models/minimax_m3/msa_m3.py", - "vllm_ascend/models/minimax_m3/ops/msa_m3_triton.py" - ] - }, - { - "commit": "36104e13b7ba38214220faf87a40deeb45f7c252", - "subject": "[Doc] Translated Doc files 2026-08-01 (#13319)", - "topics": [ - "deepseek-v4-dspark", - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/locale/zh_CN/LC_MESSAGES/community/governance.po", - "docs/source/locale/zh_CN/LC_MESSAGES/community/issue-workflow-guidelines.po", - "docs/source/locale/zh_CN/LC_MESSAGES/community/user_stories/llamafactory.po", - "docs/source/locale/zh_CN/LC_MESSAGES/community/versioning_policy.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/Design_Documents/ACL_Graph.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/Design_Documents/KV_Cache_Pool_Guide.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/Design_Documents/add_custom_aclnn_op.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/Design_Documents/balance_schedule_refactor.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/Design_Documents/context_parallel.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/Design_Documents/npugraph_ex.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/Design_Documents/patch.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/contribution/e2e_ci_test.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/contribution/multi_node_test.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/contribution/testing.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/evaluation/using_ais_bench.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/evaluation/using_lm_eval.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/performance_and_debug/msprobe_guide.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/performance_and_debug/optimization_and_tuning.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/performance_and_debug/performance_benchmark.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/performance_and_debug/service_profiling_guide.po", - "docs/source/locale/zh_CN/LC_MESSAGES/faqs.po", - "docs/source/locale/zh_CN/LC_MESSAGES/index.po", - "docs/source/locale/zh_CN/LC_MESSAGES/installation.po", - "docs/source/locale/zh_CN/LC_MESSAGES/quick_start.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/features/dynamic_chunked_pipeline_parallel.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/features/pd_colocated_mooncake_multi_instance.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/features/pd_disaggregation_mooncake_multi_node.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/features/pd_disaggregation_mooncake_single_node.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/features/ray.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/features/suffix_speculative_decoding.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/DeepSeek-R1.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/DeepSeek-V3.1.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/DeepSeek-V3.2.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/DeepSeek-V4-Flash.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/DeepSeek-V4-Pro.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/DeepSeekOCR2.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/GLM4.x.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/GLM5.2.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/GLM5.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Hunyuan-A13B-Instruct.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Hy3-preview.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/InternVL3.5.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Kimi-K2-Thinking.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Kimi-K2.5.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Kimi-K2.6.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/LLaVA-OneVision-Qwen2-0.5B-OV.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/MiniMax-M2.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Minitron-8B-Base.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Mixtral-8x7B-Instruct-v0.1.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/PaddleOCR-VL.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen-VL-Dense.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen2.5-Math-RM-72B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-235B-A22B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-30B-A3B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-ASR-1.7B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-Dense.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-Embedding.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-Next.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-Omni-30B-A3B-Thinking.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-Reranker.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-VL-235B-A22B-Instruct.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-VL-30B-A3B-Instruct.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-VL-Embedding.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-VL-Reranker.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3.5-27B-Qwen3.6-27B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3.5-397B-A17B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3.5-Dense.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3.6-35B-A3B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/gpt-oss-120b.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/configuration/additional_config.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/deployment_guide/using_mindie_motor.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/deployment_guide/using_volcano_kthena.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/Ai_QoS_introduction_en.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/batch_invariance.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/batch_job_aware_scheduler.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/context_parallel.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/dynamic_chunk_pipeline_parallel.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/epd_disaggregation.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/expert_parallelism_load_balancer.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/flash_attention.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/graph_mode.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/kv_pool.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/large_scale_ep.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/lmcache_ascend_deployment.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/lora.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/netloader.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/quantization.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/sequence_parallelism.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/short_request_first.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/speculative_decoding.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/structured_output.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/support_matrix/feature_matrix.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/support_matrix/supported_models.po" - ] - }, - { - "commit": "c511ca4a97257c5e5c4643fb9d7d293516344709", - "subject": "[Doc][Misc] Add deployment reference notice for GLM-5 (#12958)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/tutorials/models/GLM5.md" - ] - }, - { - "commit": "d8777045ca25e480786bcc7d08565d65e3afafbe", - "subject": "[Feature] Support MLAPO cache with non-contiguous tensors and add quant mode 1 (#12830)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "csrc/mla_preprocess/mla_preprocess_torch_adpt.h", - "csrc/mla_preprocess/op_host/mla_preprocess.h", - "csrc/mla_preprocess/op_host/tiling/mla_preprocess_tiling.h", - "csrc/mla_preprocess/op_kernel/mla_preprocess.h", - "csrc/mla_preprocess/op_kernel/mla_preprocess_kernel.cpp", - "csrc/mla_preprocess/op_kernel/mla_preprocess_mix_bf16.hpp", - "csrc/mla_preprocess/op_kernel/mla_preprocess_mix_bf16_nq.hpp", - "csrc/mla_preprocess/op_kernel/mla_preprocess_mix_bf16_qdown.hpp", - "csrc/mla_preprocess/op_kernel/mla_preprocess_mix_fp16.hpp", - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_mla_preprocess.py" - ] - }, - { - "commit": "5d6ddd906dc13b2e83242cb5433856e0fb875434", - "subject": "[Refactor][Ops] Move Ascend routed experts logic out of MoE runner (#13268)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/_310p/fused_moe/test_shared_fused_moe_310.py", - "tests/ut/conftest.py", - "tests/ut/ops/a3_2/test_select_experts.py", - "tests/ut/ops/test_fused_moe.py", - "vllm_ascend/_310p/fused_moe/fused_moe.py", - "vllm_ascend/ops/fused_moe/fused_moe.py", - "vllm_ascend/ops/fused_moe/routed_experts.py", - "vllm_ascend/patch/platform/patch_fused_moe.py", - "vllm_ascend/quantization/compressed_tensors_config.py", - "vllm_ascend/quantization/modelslim_config.py" - ] - }, - { - "commit": "850b363ac26011f6794e0337f3cfea9c71b5f677", - "subject": "[Test][Misc] Update DeepSeek-V4-Flash-w8a8-PD-no-prefix test configuration (#13229)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/configs/weekly_config.yaml", - ".github/workflows/misc/model_dataset_list.json", - "tests/e2e/weekly/multi_node/external_dp/config/DeepSeek-V4-Pro-w4a8-1M-PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/DeepSeek-V4-Pro-w4a8-PD-no-prefix.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/DeepSeek-V4-Pro-w4a8-prefix-cache-PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/DeepSeek-V4-flash-w8a8-PD-no-prefix.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/DeepSeek-V4-flash-w8a8-PD-prefix.yaml" - ] - }, - { - "commit": "fa3b1bd7e0e604c17fce2c44ce873725bfd176d8", - "subject": "[Feature]dsv4 dsa_cp support dspark (#13249)", - "topics": [ - "deepseek-v4-dspark", - "communication" - ], - "runtime_patch_match": false, - "paths": [ - "tests/e2e/pull_request/four_card/spec_decode/test_dspark_deepseekv4.py", - "vllm_ascend/attention/context_parallel/dsa_cp.py" - ] - }, - { - "commit": "87ba941026b1285321d36f39bc59cb19222dd7af", - "subject": "[1/N][Refactor][Core] remove function find_longest_cache_hit_per_group in patch and support DSV4 decode partial group hits (#12761)", - "topics": [ - "deepseek-v4-dspark", - "communication", - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/core/test_recompute_scheduler.py", - "tests/ut/test_compressed_prefix_cache.py", - "vllm_ascend/core/recompute_scheduler.py", - "vllm_ascend/patch/platform/patch_kv_cache_coordinator.py" - ] - }, - { - "commit": "010e089f31111b8eb639d284ff58c2023ecbe52c", - "subject": "[CI][BugFix] Add MiniMax-M3 nightly coverage and fix SFA indexer KV cache (#12846)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - ".github/TEST_README.md", - ".github/workflows/configs/nightly_config.yaml", - ".github/workflows/misc/model_dataset_list.json", - ".github/workflows/scripts/test_config.yaml", - ".github/workflows/scripts/test_select_tests.py", - "tests/e2e/generate_coverage_html.py", - "tests/e2e/nightly/single_node/models/configs/MiniMax-M3-BF16-A3.yaml", - "tests/e2e/nightly/single_node/models/configs/MiniMax-M3-W8A8-A3.yaml", - "tests/e2e/pull_request/eight_card/__init__.py", - "tests/e2e/pull_request/eight_card/test_minimax_m3.py", - "tests/ut/models/minimax_m3/test_minimax_m3.py", - "tests/ut/models/minimax_m3/test_minimax_m3_vit.py", - "tests/ut/models/minimax_m3/test_msa_m3.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "842c8a9dacbb7f54ed5b909d5edb786ae208dd25", - "subject": "[Feature] Add RoPE switch support to MLA preprocess (#12850)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "csrc/mla_preprocess/mla_preprocess_torch_adpt.h", - "csrc/mla_preprocess/op_host/mla_preprocess.h", - "csrc/mla_preprocess/op_host/tiling/mla_preprocess_tiling.h", - "csrc/mla_preprocess/op_kernel/mla_preprocess_kernel.cpp", - "csrc/mla_preprocess/op_kernel/mla_preprocess_mix_bf16.hpp", - "csrc/mla_preprocess/op_kernel/mla_preprocess_mix_bf16_nq.hpp", - "csrc/mla_preprocess/op_kernel/mla_preprocess_mix_bf16_qdown.hpp", - "csrc/mla_preprocess/op_kernel/mla_preprocess_mix_fp16.hpp", - "csrc/torch_binding.cpp", - "csrc/torch_binding_meta.cpp", - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_mla_preprocess.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_mla_preprocess_nq.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_mla_preprocess_qdown.py" - ] - }, - { - "commit": "9a3cdf26c5e03f39b9dd5bbc215375f3f07e0ac8", - "subject": "[BugFix][NetLoader] Fix processed-layout P2P for INT8_CACHE=no (#12885)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "docs/source/user_guide/feature_guide/netloader.md", - "tests/ut/model_loader/netloader/test_netloader.py", - "tests/ut/model_loader/netloader/test_netloader_elastic.py", - "tests/ut/model_loader/netloader/test_netloader_load.py", - "vllm_ascend/model_loader/netloader/executor/elastic_load.py", - "vllm_ascend/model_loader/netloader/interaction/elastic.py", - "vllm_ascend/model_loader/netloader/load.py", - "vllm_ascend/model_loader/netloader/netloader.py" - ] - }, - { - "commit": "7a337f9a0b429fb3cf7e805c6417a573c552f711", - "subject": "[BugFix][Ops] Remove unsupported SwiGLU group quant arguments (#13149)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/ops/fused_moe/fused_moe.py" - ] - }, - { - "commit": "0918fb62e16862f2ddee5c311f320919d485835b", - "subject": "[Doc][Misc] Fix links, typos, and formatting in Ascend tutorials (#12858)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/tutorials/features/dynamic_chunked_pipeline_parallel.md", - "docs/source/tutorials/features/pd_disaggregation_mooncake_multi_node.md", - "docs/source/tutorials/features/pd_disaggregation_mooncake_single_node.md", - "docs/source/tutorials/features/ray.md", - "docs/source/tutorials/features/suffix_speculative_decoding.md", - "docs/source/tutorials/models/DeepSeek-R1.md", - "docs/source/tutorials/models/DeepSeek-V3.1.md", - "docs/source/tutorials/models/DeepSeek-V3.2.md", - "docs/source/tutorials/models/GLM4.x.md", - "docs/source/tutorials/models/GLM5.2.md", - "docs/source/tutorials/models/GLM5.md", - "docs/source/tutorials/models/Hy3-preview.md", - "docs/source/tutorials/models/InternVL3.5.md", - "docs/source/tutorials/models/Kimi-K2-Thinking.md", - "docs/source/tutorials/models/Kimi-K2.5.md", - "docs/source/tutorials/models/Kimi-K2.6.md", - "docs/source/tutorials/models/MiniMax-M2.md", - "docs/source/tutorials/models/Mixtral-8x7B-Instruct-v0.1.md", - "docs/source/tutorials/models/PaddleOCR-VL.md", - "docs/source/tutorials/models/Qwen-VL-Dense.md", - "docs/source/tutorials/models/Qwen2.5-Math-RM-72B.md", - "docs/source/tutorials/models/Qwen3-235B-A22B.md", - "docs/source/tutorials/models/Qwen3-30B-A3B.md", - "docs/source/tutorials/models/Qwen3-ASR-1.7B.md", - "docs/source/tutorials/models/Qwen3-Coder-30B-A3B.md", - "docs/source/tutorials/models/Qwen3-Dense.md", - "docs/source/tutorials/models/Qwen3-Embedding.md", - "docs/source/tutorials/models/Qwen3-Next.md", - "docs/source/tutorials/models/Qwen3-Reranker.md", - "docs/source/tutorials/models/Qwen3-VL-235B-A22B-Instruct.md", - "docs/source/tutorials/models/Qwen3-VL-30B-A3B-Instruct.md", - "docs/source/tutorials/models/Qwen3-VL-Embedding.md", - "docs/source/tutorials/models/Qwen3-VL-Reranker.md", - "docs/source/tutorials/models/Qwen3.5-27B-Qwen3.6-27B.md", - "docs/source/tutorials/models/Qwen3.5-397B-A17B.md", - "docs/source/tutorials/models/Qwen3.5-Dense.md", - "docs/source/tutorials/models/Qwen3.6-35B-A3B.md", - "docs/source/user_guide/feature_guide/kv_pool.md" - ] - }, - { - "commit": "13ed9f234e536b43efb8aa68b5db2375edb6cadb", - "subject": "[Doc] AIDD Phase II Revision (#13162)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/developer_guide/contribution/testing.md", - "docs/source/tutorials/models/DeepSeek-V3.1.md", - "docs/source/tutorials/models/DeepSeek-V4-Flash.md", - "docs/source/tutorials/models/GLM4.x.md", - "docs/source/tutorials/models/Qwen3-Next.md", - "docs/source/tutorials/models/Qwen3-Omni-30B-A3B-Thinking.md", - "docs/source/tutorials/models/gpt-oss-120b.md", - "docs/source/user_guide/feature_guide/quantization.md" - ] - }, - { - "commit": "c8710f2443b05de3e533d628bb549cca8c573b7a", - "subject": "[Doc][Misc] Standardize hardware specifications, fix typos, and correct links in model tutorials (#13100)", - "topics": [ - "deepseek-v4-dspark", - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/tutorials/models/DeepSeek-R1.md", - "docs/source/tutorials/models/DeepSeek-V3.1.md", - "docs/source/tutorials/models/DeepSeek-V3.2.md", - "docs/source/tutorials/models/DeepSeek-V4-Flash.md", - "docs/source/tutorials/models/DeepSeek-V4-Pro.md", - "docs/source/tutorials/models/DeepSeekOCR2.md", - "docs/source/tutorials/models/GLM4.x.md", - "docs/source/tutorials/models/GLM5.2.md", - "docs/source/tutorials/models/GLM5.md", - "docs/source/tutorials/models/LLaVA-OneVision-Qwen2-0.5B-OV.md", - "docs/source/tutorials/models/MiniMax-M2.md", - "docs/source/tutorials/models/Minitron-8B-Base.md", - "docs/source/tutorials/models/Mixtral-8x7B-Instruct-v0.1.md", - "docs/source/tutorials/models/PaddleOCR-VL.md", - "docs/source/tutorials/models/Qwen-VL-Dense.md", - "docs/source/tutorials/models/Qwen2.5-Math-RM-72B.md", - "docs/source/tutorials/models/Qwen3-235B-A22B.md", - "docs/source/tutorials/models/Qwen3-30B-A3B.md", - "docs/source/tutorials/models/Qwen3-Dense.md", - "docs/source/tutorials/models/Qwen3-Next.md", - "docs/source/tutorials/models/Qwen3-Omni-30B-A3B-Thinking.md", - "docs/source/tutorials/models/Qwen3.5-27B-Qwen3.6-27B.md", - "docs/source/tutorials/models/Qwen3.5-397B-A17B.md", - "docs/source/tutorials/models/gpt-oss-120b.md" - ] - }, - { - "commit": "8b39f17a66d5e9504645df8099203acc9d23d376", - "subject": "[Doc] Remove redundant async-scheduling config. (#13093)", - "topics": [ - "deepseek-v4-dspark", - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/_templates/Model-Deployment-Tutorial-Template.zh.md", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/DeepSeek-V4-Flash.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/DeepSeek-V4-Pro.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-VL-235B-A22B-Instruct.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-VL-30B-A3B-Instruct.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3.5-27B-Qwen3.6-27B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3.6-35B-A3B.po", - "docs/source/tutorials/models/DeepSeek-V4-Flash.md", - "docs/source/tutorials/models/DeepSeek-V4-Pro.md", - "docs/source/tutorials/models/GLM5.2.md", - "docs/source/tutorials/models/InternVL3.5.md", - "docs/source/tutorials/models/Kimi-K2.6.md", - "docs/source/tutorials/models/MiniMax-M2.md", - "docs/source/tutorials/models/Qwen3-235B-A22B.md", - "docs/source/tutorials/models/Qwen3-30B-A3B.md", - "docs/source/tutorials/models/Qwen3-Coder-30B-A3B.md", - "docs/source/tutorials/models/Qwen3-Dense.md", - "docs/source/tutorials/models/Qwen3-Omni-30B-A3B-Thinking.md", - "docs/source/tutorials/models/Qwen3-VL-235B-A22B-Instruct.md", - "docs/source/tutorials/models/Qwen3-VL-30B-A3B-Instruct.md", - "docs/source/tutorials/models/Qwen3.5-27B-Qwen3.6-27B.md", - "docs/source/tutorials/models/Qwen3.6-35B-A3B.md" - ] - }, - { - "commit": "7662ec925ab6ddfd19e1c6401d43a23a127945e4", - "subject": "[CI] main2main vllm v0.26.0 (#12859)", - "topics": [ - "deepseek-v4-dspark", - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/vllm-release-tag.commit", - ".gitleaks.toml", - "Dockerfile", - "Dockerfile.310p", - "Dockerfile.310p.openEuler", - "Dockerfile.a3", - "Dockerfile.a3.openEuler", - "Dockerfile.a5", - "Dockerfile.a5.openEuler", - "Dockerfile.openEuler", - "README.md", - "README.zh.md", - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_fused_recurrent_gated_delta_rule.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_fused_sigmoid_gating_delta_rule.py", - "tests/e2e/pull_request/one_card/model_runner_v2/test_basic.py", - "tests/ut/distributed/ascend_store/test_coordinator.py", - "tests/ut/ops/test_gdn_attn_builder.py", - "tests/ut/ops/test_gdn_layerwise_kv.py", - "tests/ut/patch/test_hunyuan_vl_processor_compat.py", - "tests/ut/test_compressed_prefix_cache.py", - "tests/ut/worker/test_pcp_manager_v2.py", - "vllm_ascend/_310p/ops/fla/idex.py", - "vllm_ascend/_310p/ops/fla/l2norm.py", - "vllm_ascend/__init__.py", - "vllm_ascend/core/recompute_scheduler.py", - "vllm_ascend/core/scheduler_profiling_chunk.py", - "vllm_ascend/core/single_type_kv_cache_manager.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/ascend_store/coordinator.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/ascend_store/pool_scheduler.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/ascend_store/pool_worker.py", - "vllm_ascend/ops/bailing_moe_linear_attn.py", - "vllm_ascend/ops/gdn.py", - "vllm_ascend/ops/triton/fla/chunk.py", - "vllm_ascend/ops/vocab_parallel_embedding.py", - "vllm_ascend/patch/__init__.py", - "vllm_ascend/patch/hunyuan_vl_processor_compat.py", - "vllm_ascend/patch/platform/patch_kv_cache_coordinator.py", - "vllm_ascend/patch/platform/patch_mamba_manager.py", - "vllm_ascend/patch/worker/__init__.py", - "vllm_ascend/patch/worker/patch_idex_310.py", - "vllm_ascend/patch/worker/patch_qwen3_5.py", - "vllm_ascend/patch/worker/patch_triton.py", - "vllm_ascend/worker/block_table.py", - "vllm_ascend/worker/v2/attn_utils.py", - "vllm_ascend/worker/v2/model_runner.py", - "vllm_ascend/worker/v2/sample/gumbel.py", - "vllm_ascend/worker/v2/spec_decode/autoregressive/speculator.py", - "vllm_ascend/worker/v2/spec_decode/dflash/aclgraph.py", - "vllm_ascend/worker/v2/spec_decode/dflash/speculator.py", - "vllm_ascend/worker/v2/spec_decode/dspark/speculator.py", - "vllm_ascend/worker/v2/spec_decode/eagle/aclgraph.py", - "vllm_ascend/worker/v2/spec_decode/mtp/speculator.py" - ] - }, - { - "commit": "b6fe31d114acbf29fb5f1c08ad1d60060a121f7b", - "subject": "[BugFix][main] remove unnecessary lazy_init for memcache backend (#13029)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/distributed/kv_transfer/kv_pool/ascend_store/pool_worker.py" - ] - }, - { - "commit": "21382607d15a835728d9ea493ab101e7209799b5", - "subject": "[BugFix] Remove -Werror from csrc CMakeLists.txt files (#13091)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "csrc/attention/compressor/op_host/CMakeLists.txt", - "csrc/attention/compressor_metadata/op_host/CMakeLists.txt", - "csrc/attention/indexer_compress_epilog/op_host/CMakeLists.txt", - "csrc/attention/indexer_compress_epilog_v2/op_host/CMakeLists.txt", - "csrc/attention/inplace_partial_rotary_mul/op_host/CMakeLists.txt", - "csrc/attention/kv_compress_epilog/op_host/CMakeLists.txt", - "csrc/attention/kv_quant_sparse_attn_sharedkv/op_host/CMakeLists.txt", - "csrc/attention/kv_quant_sparse_flash_attention/op_host/CMakeLists.txt", - "csrc/attention/lightning_indexer/op_host/CMakeLists.txt", - "csrc/attention/lightning_indexer_quant/op_host/CMakeLists.txt", - "csrc/attention/lightning_indexer_vllm/op_host/CMakeLists.txt", - "csrc/attention/load_index_kv_cache/op_host/CMakeLists.txt", - "csrc/attention/ngram_spec_decode/op_host/CMakeLists.txt", - "csrc/attention/recurrent_gated_delta_rule/op_host/CMakeLists.txt", - "csrc/attention/recurrent_gated_delta_rule_v310/op_host/CMakeLists.txt", - "csrc/attention/rms_norm_dynamic_quant/op_host/CMakeLists.txt", - "csrc/attention/sparse_attention_score/op_host/CMakeLists.txt", - "csrc/attention/sparse_attn_sharedkv/op_host/CMakeLists.txt", - "csrc/attention/sparse_flash_attention/op_host/CMakeLists.txt", - "csrc/attention/store_kv_block/op_host/CMakeLists.txt", - "csrc/attention/vllm_quant_lightning_indexer/op_host/CMakeLists.txt", - "csrc/gmm/grouped_matmul_swiglu_quant/op_host/CMakeLists.txt", - "csrc/gmm/grouped_matmul_swiglu_quant_v2/op_host/CMakeLists.txt", - "csrc/gmm/grouped_matmul_swiglu_quant_weight_nz_tensor_list/op_host/CMakeLists.txt", - "csrc/mc2/dispatch_ffn_combine/op_host/CMakeLists.txt", - "csrc/mc2/dispatch_ffn_combine_bf16/op_host/CMakeLists.txt", - "csrc/mc2/dispatch_ffn_combine_w4_a8/op_host/CMakeLists.txt", - "csrc/moe/add_rms_norm_bias/op_host/CMakeLists.txt", - "csrc/moe/causal_conv1d/op_host/CMakeLists.txt", - "csrc/moe/causal_conv1d_v310/op_host/CMakeLists.txt", - "csrc/moe/chunk_fwd_o/op_host/CMakeLists.txt", - "csrc/moe/chunk_gated_delta_rule_fwd_h/op_host/CMakeLists.txt", - "csrc/moe/copy_and_expand_eagle_inputs/op_host/CMakeLists.txt", - "csrc/moe/dequant_swiglu_quant/op_host/CMakeLists.txt", - "csrc/moe/hc_post/op_host/CMakeLists.txt", - "csrc/moe/hc_pre/op_host/CMakeLists.txt", - "csrc/moe/hc_pre_inv_rms/op_host/CMakeLists.txt", - "csrc/moe/hc_pre_sinkhorn/op_host/CMakeLists.txt", - "csrc/moe/moe_gating_top_k/op_host/CMakeLists.txt", - "csrc/moe/moe_gating_top_k_hash/op_host/CMakeLists.txt", - "csrc/moe/moe_grouped_matmul/op_host/CMakeLists.txt", - "csrc/moe/scatter_nd_update_v2/op_host/CMakeLists.txt", - "csrc/moe/swiglu_group_quant/op_host/CMakeLists.txt", - "csrc/moe/transpose_kv_cache_by_block/op_host/CMakeLists.txt" - ] - }, - { - "commit": "3d0f7d6f8d869b8931285bf61b0e1a7a9734bb45", - "subject": "[Feature] add MiniMax M3 VL model support (#12844)", - "topics": [ - "priority-models", - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/models/__init__.py", - "vllm_ascend/models/minimax_m3/__init__.py", - "vllm_ascend/models/minimax_m3/minimax_m3_vl.py" - ] - }, - { - "commit": "9000fa12184d5562ae83310f0a21f4f10f0ab9e2", - "subject": "[CI] modify weekly cases:dsv4,dsv3.2-EP,GLM5.yaml (#13044)", - "topics": [ - "deepseek-v4-dspark", - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "tests/e2e/weekly/multi_node/external_dp/config/DeepSeek-V4-flash-w8a8-PD.yaml", - "tests/e2e/weekly/multi_node/internal_dp/config/DeepSeek-V3_2-W8A8-EP_weekly.yaml", - "tests/e2e/weekly/single_node/configs/GLM-5.yaml", - "tests/e2e/weekly/single_node/configs/GLM-5_1-W8A8_A3_weekly.yaml" - ] - }, - { - "commit": "d6a340182e49998f61b6dc3014f45f4ac100e1ec", - "subject": "[Feature][Model] Add MiniMax M3 model core support (#13009)", - "topics": [ - "communication", - "priority-models", - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/ascend_config.py", - "vllm_ascend/models/__init__.py", - "vllm_ascend/models/minimax_m3/__init__.py", - "vllm_ascend/models/minimax_m3/minimax_m3.py", - "vllm_ascend/models/minimax_m3/msa_m3.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "261a3e9f9ef0276ef36e5a57b3759c071c86a4fb", - "subject": "[Misc] M2m 0724 continuation (#12648)", - "topics": [ - "deepseek-v4-dspark", - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - ".github/vllm-main-verified.commit", - "vllm_ascend/_310p/model_runner_310p.py", - "vllm_ascend/core/recompute_scheduler.py", - "vllm_ascend/worker/v2/attn_utils.py", - "vllm_ascend/worker/v2/spec_decode/autoregressive/speculator.py", - "vllm_ascend/worker/v2/spec_decode/dflash/aclgraph.py", - "vllm_ascend/worker/v2/spec_decode/dflash/speculator.py", - "vllm_ascend/worker/v2/spec_decode/dspark/speculator.py", - "vllm_ascend/worker/v2/spec_decode/eagle/aclgraph.py", - "vllm_ascend/worker/v2/spec_decode/mtp/speculator.py" - ] - }, - { - "commit": "019d3e944e423b1d1cbf0a0770e015d8813e1504", - "subject": "[Doc] Add docs for mega moe (#12926)", - "topics": [ - "communication", - "priority-models", - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/GLM5.2.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/GLM5.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/InternVL3.5.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/configuration/additional_config.po", - "docs/source/tutorials/models/GLM5.2.md", - "docs/source/tutorials/models/GLM5.md", - "docs/source/tutorials/models/InternVL3.5.md", - "vllm_ascend/ascend_forward_context.py", - "vllm_ascend/envs.py", - "vllm_ascend/lora/fused_moe.py", - "vllm_ascend/ops/fused_moe/fused_moe.py" - ] - }, - { - "commit": "7573ea0e6e94e165181423b292da87bfd8d6c10f", - "subject": "[Revert] Revert [feature] reduce cases that required DP padding (#11772) (#13022)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - ".gitleaks.toml", - "tests/ut/ops/a2/test_token_dispatcher.py", - "tests/ut/spec_decode/a2/test_eagle_proposer.py", - "vllm_ascend/ascend_config.py", - "vllm_ascend/ascend_forward_context.py", - "vllm_ascend/ops/fused_moe/token_dispatcher.py", - "vllm_ascend/spec_decode/extract_hidden_states_proposer.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "60cba9393c54a8e43d1e4cdc4517be877f899429", - "subject": "[Feature] Split c8 to li c8 and sfa c8 (#12470)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/configuration/additional_config.po", - "docs/source/user_guide/configuration/additional_config.md", - "tests/e2e/nightly/single_node/models/configs/DeepSeek-V3.2-W8A8-DCP.yaml", - "tests/e2e/pull_request/four_card/context_parallel/test_accuracy.py", - "tests/ut/attention/a2/test_sfa_v1.py", - "tests/ut/test_ascend_config.py", - "tests/ut/worker/a2/test_model_runner_v1.py", - "vllm_ascend/ascend_config.py", - "vllm_ascend/attention/context_parallel/sfa_cp.py", - "vllm_ascend/attention/sfa_v1.py", - "vllm_ascend/core/kv_cache_interface.py", - "vllm_ascend/device/device_op.py", - "vllm_ascend/utils.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "7607b4cb74542a573dd937b621bd97237e7d79f7", - "subject": "[CI] Add full ut protection for dspark (#12791)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/spec_decode/test_dspark_proposer.py" - ] - }, - { - "commit": "81d3450128528be2c343232fcc28220814a15fd6", - "subject": "[BugFix] bug fix for mega_moe (#12936)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/ascend_forward_context.py", - "vllm_ascend/ops/fused_moe/moe_comm_method.py" - ] - }, - { - "commit": "32f577cd3f33c190a2e618c17a7cc089d6769eb0", - "subject": "[BugFix]Fix the precision issue of the npu_dequant_swiglu_quant operator when x.shape=[2,192] (#12123)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "csrc/moe/dequant_swiglu_quant/op_host/dequant_swiglu_quant_tiling.cpp", - "csrc/moe/dequant_swiglu_quant/op_kernel/dequant_swiglu_quant.h", - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_dequant_swiglu_quant.py" - ] - }, - { - "commit": "a8f7be637f30895ebc73f3b29912e557459f7661", - "subject": "[feature] reduce cases that required DP padding (#11772)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - ".gitleaks.toml", - "tests/ut/ops/a2/test_token_dispatcher.py", - "tests/ut/spec_decode/a2/test_eagle_proposer.py", - "vllm_ascend/ascend_config.py", - "vllm_ascend/ascend_forward_context.py", - "vllm_ascend/ops/fused_moe/token_dispatcher.py", - "vllm_ascend/spec_decode/extract_hidden_states_proposer.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "9901d29a3cc1cae854a14fe5db29fabe8c08e905", - "subject": "[Kernel][Feature] Add AscendC and Triton sparse attention support for MiniMax M3 (#12833)", - "topics": [ - "priority-models", - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/workflows/scripts/gitleaks.sh", - ".github/workflows/scripts/test_config.yaml", - "csrc/attention/sparse_attention_score/CMakeLists.txt", - "csrc/attention/sparse_attention_score/op_host/CMakeLists.txt", - "csrc/attention/sparse_attention_score/op_host/op_api/aclnn_sparse_attention_score.cpp", - "csrc/attention/sparse_attention_score/op_host/op_api/aclnn_sparse_attention_score.h", - "csrc/attention/sparse_attention_score/op_host/op_api/sparse_attention_score.cpp", - "csrc/attention/sparse_attention_score/op_host/op_api/sparse_attention_score.h", - "csrc/attention/sparse_attention_score/op_host/sparse_attention_score_def.cpp", - "csrc/attention/sparse_attention_score/op_host/sparse_attention_score_infershape.cpp", - "csrc/attention/sparse_attention_score/op_host/sparse_attention_score_tiling.cpp", - "csrc/attention/sparse_attention_score/op_host/sparse_attention_score_tiling.h", - "csrc/attention/sparse_attention_score/op_kernel/arch22/kernel_utils.hpp", - "csrc/attention/sparse_attention_score/op_kernel/arch22/sparse_attention_score_kernel_arch22.h", - "csrc/attention/sparse_attention_score/op_kernel/arch35/kernel_utils.hpp", - "csrc/attention/sparse_attention_score/op_kernel/arch35/sparse_attention_score_kernel_arch35.h", - "csrc/attention/sparse_attention_score/op_kernel/arch35/sparse_attention_score_kernel_arch35_full_quant.h", - "csrc/attention/sparse_attention_score/op_kernel/attn_infra/arch/arch.hpp", - "csrc/attention/sparse_attention_score/op_kernel/attn_infra/arch/cross_core_sync.hpp", - "csrc/attention/sparse_attention_score/op_kernel/attn_infra/arch/local_tensor_buffer.hpp", - "csrc/attention/sparse_attention_score/op_kernel/attn_infra/arch/resource.hpp", - "csrc/attention/sparse_attention_score/op_kernel/attn_infra/base_defs.hpp", - "csrc/attention/sparse_attention_score/op_kernel/attn_infra/coord.hpp", - "csrc/attention/sparse_attention_score/op_kernel/attn_infra/detail/alignment.hpp", - "csrc/attention/sparse_attention_score/op_kernel/attn_infra/detail/dependent_false.hpp", - "csrc/attention/sparse_attention_score/op_kernel/attn_infra/detail/macros.hpp", - "csrc/attention/sparse_attention_score/op_kernel/attn_infra/detail/tag_to_layout.hpp", - "csrc/attention/sparse_attention_score/op_kernel/attn_infra/epilogue/block/block_epilogue.hpp", - "csrc/attention/sparse_attention_score/op_kernel/attn_infra/epilogue/block/block_epilogue_mask2idx_arch35.hpp", - "csrc/attention/sparse_attention_score/op_kernel/attn_infra/epilogue/block/block_epilogue_online_softmax.hpp", - "csrc/attention/sparse_attention_score/op_kernel/attn_infra/epilogue/block/block_epilogue_online_softmax_arch35_reg_low_prec.hpp", - "csrc/attention/sparse_attention_score/op_kernel/attn_infra/epilogue/block/block_epilogue_online_softmax_arch35_reg_low_prec_bf16.hpp", - "csrc/attention/sparse_attention_score/op_kernel/attn_infra/epilogue/block/block_epilogue_online_softmax_low_prec.hpp", - "csrc/attention/sparse_attention_score/op_kernel/attn_infra/epilogue/block/block_epilogue_rescale_o.hpp", - "csrc/attention/sparse_attention_score/op_kernel/attn_infra/epilogue/block/block_epilogue_rescale_o_arch35_reg_high_prec.hpp", - "csrc/attention/sparse_attention_score/op_kernel/attn_infra/epilogue/block/block_epilogue_rescale_o_low_prec.hpp", - "csrc/attention/sparse_attention_score/op_kernel/attn_infra/epilogue/dispatch_policy.hpp", - "csrc/attention/sparse_attention_score/op_kernel/attn_infra/epilogue/tile_common/copy_gm_to_ub.hpp", - "csrc/attention/sparse_attention_score/op_kernel/attn_infra/epilogue/tile_common/copy_gm_to_ub_tla.hpp", - "csrc/attention/sparse_attention_score/op_kernel/attn_infra/epilogue/tile_common/copy_ub_to_gm.hpp", - "csrc/attention/sparse_attention_score/op_kernel/attn_infra/epilogue/tile_common/copy_ub_to_gm_tla.hpp", - "csrc/attention/sparse_attention_score/op_kernel/attn_infra/epilogue/tile_common/tile_copy.hpp", - "csrc/attention/sparse_attention_score/op_kernel/attn_infra/gemm/block/block_mmad.hpp", - "csrc/attention/sparse_attention_score/op_kernel/attn_infra/gemm/block/block_mmad_pv.hpp", - "csrc/attention/sparse_attention_score/op_kernel/attn_infra/gemm/block/block_mmad_pv_arch35_ABf16_C_to_UB.hpp", - "csrc/attention/sparse_attention_score/op_kernel/attn_infra/gemm/block/block_mmad_qk.hpp", - "csrc/attention/sparse_attention_score/op_kernel/attn_infra/gemm/block/block_mmad_qk_arch35_ABf16_C_to_UB.hpp", - "csrc/attention/sparse_attention_score/op_kernel/attn_infra/gemm/dispatch_policy.hpp", - "csrc/attention/sparse_attention_score/op_kernel/attn_infra/gemm/gemm_type.hpp", - "csrc/attention/sparse_attention_score/op_kernel/attn_infra/gemm/helper.hpp", - "csrc/attention/sparse_attention_score/op_kernel/attn_infra/gemm/tile_common/copy_gm_to_l1_a2.hpp", - "csrc/attention/sparse_attention_score/op_kernel/attn_infra/gemm/tile_common/copy_gm_to_l1_a5.hpp", - "csrc/attention/sparse_attention_score/op_kernel/attn_infra/gemm/tile_common/copy_gm_to_ub.hpp", - "csrc/attention/sparse_attention_score/op_kernel/attn_infra/gemm/tile_common/copy_l0c_to_dst.hpp", - "csrc/attention/sparse_attention_score/op_kernel/attn_infra/gemm/tile_common/copy_l0c_to_gm_a2.hpp", - "csrc/attention/sparse_attention_score/op_kernel/attn_infra/gemm/tile_common/copy_l0c_to_ub_a5.hpp", - "csrc/attention/sparse_attention_score/op_kernel/attn_infra/gemm/tile_common/copy_l1_to_bt.hpp", - "csrc/attention/sparse_attention_score/op_kernel/attn_infra/gemm/tile_common/copy_l1_to_l0a_a2.hpp", - "csrc/attention/sparse_attention_score/op_kernel/attn_infra/gemm/tile_common/copy_l1_to_l0a_a5.hpp", - "csrc/attention/sparse_attention_score/op_kernel/attn_infra/gemm/tile_common/copy_l1_to_l0b_a2.hpp", - "csrc/attention/sparse_attention_score/op_kernel/attn_infra/gemm/tile_common/copy_l1_to_l0b_a5.hpp", - "csrc/attention/sparse_attention_score/op_kernel/attn_infra/gemm/tile_common/copy_ub_to_gm.hpp", - "csrc/attention/sparse_attention_score/op_kernel/attn_infra/gemm/tile_common/tile_copy.hpp", - "csrc/attention/sparse_attention_score/op_kernel/attn_infra/gemm/tile_common/tile_copy_tla.hpp", - "csrc/attention/sparse_attention_score/op_kernel/attn_infra/gemm/tile_common/tile_mmad.hpp", - "csrc/attention/sparse_attention_score/op_kernel/attn_infra/gemm_coord.hpp", - "csrc/attention/sparse_attention_score/op_kernel/attn_infra/layout/layout.hpp", - "csrc/attention/sparse_attention_score/op_kernel/attn_infra/layout/matrix.hpp", - "csrc/attention/sparse_attention_score/op_kernel/attn_infra/layout/vector.hpp", - "csrc/attention/sparse_attention_score/op_kernel/attn_infra/matrix_coord.hpp", - "csrc/attention/sparse_attention_score/op_kernel/attn_infra/status.hpp", - "csrc/attention/sparse_attention_score/op_kernel/kernel_common.hpp", - "csrc/attention/sparse_attention_score/op_kernel/sparse_attention_score.cpp", - "csrc/attention/sparse_attention_score/op_kernel/sparse_attention_score_kernel_interface.cpp", - "csrc/attention/sparse_attention_score/op_kernel/sparse_attention_score_tilingkey.h", - "csrc/attention/sparse_attention_score/op_kernel/tla/int_tuple.hpp", - "csrc/attention/sparse_attention_score/op_kernel/tla/layout.hpp", - "csrc/attention/sparse_attention_score/op_kernel/tla/numeric/integer_sequence.hpp", - "csrc/attention/sparse_attention_score/op_kernel/tla/numeric/integral_constant.hpp", - "csrc/attention/sparse_attention_score/op_kernel/tla/numeric/math.hpp", - "csrc/attention/sparse_attention_score/op_kernel/tla/tensor.hpp", - "csrc/attention/sparse_attention_score/op_kernel/tla/tuple.hpp", - "csrc/attention/sparse_attention_score/op_kernel/tla/type_traits.hpp", - "csrc/attention/sparse_attention_score/sparse_attention_score_torch_adpt.h", - "csrc/attention/sparse_attention_score/tests/test_bf16.py", - "csrc/attention/sparse_attention_score/tests/test_fp16.py", - "csrc/attention/sparse_attention_score/tools/fix_mmad_deq.py", - "csrc/build_aclnn.sh", - "csrc/torch_binding.cpp", - "csrc/torch_binding_meta.cpp", - "tests/e2e/conftest.py", - "tests/e2e/pull_request/one_card/test_minimax_m3_sparse_attn.py", - "tests/ut/conftest.py", - "vllm_ascend/models/minimax_m3/ops/__init__.py", - "vllm_ascend/models/minimax_m3/ops/msa_m3_npu.py", - "vllm_ascend/models/minimax_m3/ops/msa_m3_triton.py" - ] - }, - { - "commit": "db44297862a85727d07ae8617f283c36545adf6c", - "subject": "[Doc] Translated Doc files 2026-07-22 (#12538)", - "topics": [ - "deepseek-v4-dspark", - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/locale/zh_CN/LC_MESSAGES/community/contributors.po", - "docs/source/locale/zh_CN/LC_MESSAGES/community/versioning_policy.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/contribution/nightly_ci_test.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/performance_and_debug/index.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/performance_and_debug/optimization_and_tuning.po", - "docs/source/locale/zh_CN/LC_MESSAGES/faqs.po", - "docs/source/locale/zh_CN/LC_MESSAGES/getting_started.po", - "docs/source/locale/zh_CN/LC_MESSAGES/installation.po", - "docs/source/locale/zh_CN/LC_MESSAGES/quick_start.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/DeepSeek-V4-Flash.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/GLM5.2.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen-VL-Dense.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-30B-A3B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-ASR-1.7B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3.5-27B-Qwen3.6-27B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3.5-Dense.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3.6-35B-A3B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/release_notes.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/support_matrix/supported_models.po" - ] - }, - { - "commit": "ddc85dda76d52fcb0ef4adb2f86fda64efb9a8b3", - "subject": "[Refactor][Misc] Remove unused shared expert DP flags (#12908)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/configuration/additional_config.po", - "docs/source/user_guide/configuration/additional_config.md", - "tests/ut/ops/test_mla.py", - "vllm_ascend/attention/mla_v1.py", - "vllm_ascend/attention/sfa_v1.py", - "vllm_ascend/ops/mla.py" - ] - }, - { - "commit": "d567f6eeb9d9e7fd68fbf8f4d05d71591edfd07b", - "subject": "[Feature] layer and model runner support MiniMax M3 (#12835)", - "topics": [ - "priority-models", - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/ops/test_linear.py", - "vllm_ascend/ops/fused_moe/fused_moe.py", - "vllm_ascend/ops/fused_moe/moe_comm_method.py", - "vllm_ascend/ops/fused_moe/moe_mlp.py", - "vllm_ascend/ops/fused_moe/moe_runtime_args.py", - "vllm_ascend/ops/fused_moe/moe_stage_contracts.py", - "vllm_ascend/ops/linear_op.py", - "vllm_ascend/ops/rotary_embedding.py", - "vllm_ascend/quantization/methods/w8a8_dynamic.py", - "vllm_ascend/quantization/modelslim_config.py" - ] - }, - { - "commit": "39d1cf9b3352f3f5b4571894295e64ca515b8ffb", - "subject": "[feature] Support dspark speculative method and synchronize reorder batch threshold calculation with vLLM (#12662)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/ops/gdn_attn_builder.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "69f72c642e79e2b68d075ca52b9f92e0862fad9e", - "subject": "[Refactor][PCP][1/N] remove PCP from MRV1. (#12592)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "docs/source/developer_guide/Design_Documents/context_parallel.md", - "tests/e2e/coverage.html", - "tests/e2e/coverage_taxonomy.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_chunk_gated_delta_rule.py", - "tests/e2e/pull_request/four_card/context_parallel/test_accuracy.py", - "tests/e2e/pull_request/four_card/test_pipeline_parallel.py", - "tests/e2e/pull_request/two_card/spec_decode/test_spec_decode.py", - "tests/ut/_310p/test_block_table_310p.py", - "tests/ut/_310p/test_model_runner_310p.py", - "tests/ut/attention/a2/test_attention_cp.py", - "tests/ut/attention/a2/test_attention_cp_precision.py", - "tests/ut/attention/a2/test_attention_v1.py", - "tests/ut/attention/a2/test_common_cp.py", - "tests/ut/attention/a2/test_mla_cp.py", - "tests/ut/attention/a2/test_mla_cp_precision.py", - "tests/ut/attention/a2/test_mla_v1.py", - "tests/ut/attention/a2/test_sfa_cp_precision.py", - "tests/ut/attention/a2/test_sfa_v1.py", - "tests/ut/attention/test_attention_utils.py", - "tests/ut/attention/test_sfa_cp.py", - "tests/ut/attention/utils.py", - "tests/ut/compilation/a2/test_acl_graph.py", - "tests/ut/ops/test_gdn_attn_builder.py", - "tests/ut/patch/platform/test_prefix_cache_cp_patches.py", - "tests/ut/spec_decode/a2/test_eagle_proposer.py", - "tests/ut/spec_decode/test_eagle_aclgraph_source_regression.py", - "tests/ut/spec_decode/test_extract_hidden_states_proposer.py", - "tests/ut/spec_decode/test_speculators_vwn_eagle3.py", - "tests/ut/spec_decode/test_step3p5_source_regression.py", - "tests/ut/test_platform.py", - "tests/ut/worker/a2/test_block_table.py", - "tests/ut/worker/a2/test_model_runner_v1.py", - "tests/ut/worker/a2/test_model_runner_v1_with_device.py", - "tests/ut/worker/test_dcp_manager.py", - "vllm_ascend/_310p/block_table.py", - "vllm_ascend/_310p/model_runner_310p.py", - "vllm_ascend/_310p/spec_decode/llm_base_proposer_310.py", - "vllm_ascend/attention/attention_mask.py", - "vllm_ascend/attention/attention_v1.py", - "vllm_ascend/attention/context_parallel/attention_cp.py", - "vllm_ascend/attention/context_parallel/common_cp.py", - "vllm_ascend/attention/context_parallel/mla_cp.py", - "vllm_ascend/attention/context_parallel/sfa_cp.py", - "vllm_ascend/attention/dsa_v1.py", - "vllm_ascend/attention/mla_v1.py", - "vllm_ascend/attention/sfa_v1.py", - "vllm_ascend/attention/utils.py", - "vllm_ascend/compilation/acl_graph.py", - "vllm_ascend/core/kv_cache_interface.py", - "vllm_ascend/core/single_type_kv_cache_manager.py", - "vllm_ascend/patch/__init__.py", - "vllm_ascend/patch/platform/patch_kv_cache_coordinator.py", - "vllm_ascend/patch/platform/patch_kv_cache_utils.py", - "vllm_ascend/platform.py", - "vllm_ascend/spec_decode/llm_base_proposer.py", - "vllm_ascend/spec_decode/step3p5.py", - "vllm_ascend/utils.py", - "vllm_ascend/worker/block_table.py", - "vllm_ascend/worker/dcp_utils.py", - "vllm_ascend/worker/model_runner_v1.py", - "vllm_ascend/worker/pcp_utils.py", - "vllm_ascend/worker/v2/aclgraph_utils.py", - "vllm_ascend/worker/v2/attn_utils.py", - "vllm_ascend/worker/v2/spec_decode/dflash/aclgraph.py", - "vllm_ascend/worker/v2/spec_decode/eagle/aclgraph.py" - ] - }, - { - "commit": "2b1ac612496e8306c6d7d95d175e93fad90057b1", - "subject": "[BugFix][DSA-CP] Fix invalid local sequence lengths for graph padding (#12840)", - "topics": [ - "communication" - ], - "runtime_patch_match": false, - "paths": [ - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_build_local_metadata_triton.py", - "vllm_ascend/attention/context_parallel/dsa_cp.py", - "vllm_ascend/ops/triton/dsa_cp.py" - ] - }, - { - "commit": "9e7281542efadb45608e887ea648ad2968a19484", - "subject": "[Feature] Add Mega moe support for a3 (#11701)", - "topics": [ - "communication", - "priority-models", - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/quantization/methods/a2/test_w4a8.py", - "tests/ut/test_ascend_forward_context.py", - "vllm_ascend/ascend_forward_context.py", - "vllm_ascend/ops/fused_moe/comm_utils.py", - "vllm_ascend/ops/fused_moe/moe_comm_method.py", - "vllm_ascend/ops/fused_moe/token_dispatcher.py", - "vllm_ascend/platform.py", - "vllm_ascend/quantization/methods/w4a8.py", - "vllm_ascend/quantization/methods/w8a8_dynamic.py" - ] - }, - { - "commit": "a7e0575095585f7164d0df34a31b7014e339af03", - "subject": "[Performance] Support splitting mixed ChunkedPrefill into separate decode and prefill attention calls (#11948)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/attention/attention_v1.py" - ] - }, - { - "commit": "875f96e9b6d9a890a290c40c5b2e2da923ad927d", - "subject": "[BugFix][Refactor] Fix `sp` in dspark (#12777)", - "topics": [ - "deepseek-v4-dspark", - "communication", - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/spec_decode/test_llm_base_proposer.py", - "vllm_ascend/spec_decode/llm_base_proposer.py", - "vllm_ascend/spec_decode/utils.py" - ] - }, - { - "commit": "ae2397f2d4958a65e33ec6a91c1e4ad1aa8c684c", - "subject": "[main][Doc][Feature] Add info for eplb (#12816)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/user_guide/feature_guide/expert_parallelism_load_balancer.md" - ] - }, - { - "commit": "ab6093485a7952d3db1a1af421a34daa5de030ae", - "subject": "[Misc][Host Profiling] Add more host profiling (#12774)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/test_profiling_config.py", - "vllm_ascend/core/recompute_scheduler.py", - "vllm_ascend/profiling_config.py" - ] - }, - { - "commit": "03cc154cb33d499f21db1afab63cdc6923797ee1", - "subject": "[BugFix][GLM-5.2][SpecDecode] Fix DSpark W8A8 eager acceptance (#12262)", - "topics": [ - "deepseek-v4-dspark", - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/misc/model_dataset_list.json", - "tests/e2e/nightly/single_node/models/configs/GLM-5.2-W8A8-DSpark.yaml", - "tests/ut/model_executor/test_qwen3_dspark.py", - "tests/ut/spec_decode/test_dspark_proposer.py", - "vllm_ascend/models/__init__.py", - "vllm_ascend/models/qwen3_dspark.py", - "vllm_ascend/spec_decode/dspark_proposer.py", - "vllm_ascend/spec_decode/llm_base_proposer.py" - ] - }, - { - "commit": "84c647f198944ccdd306eb196d35e59c1e2b70c5", - "subject": "[Performance][DSA-CP] Optimize local token metadata computation with fused Triton kernel and caching (#12193)", - "topics": [ - "deepseek-v4-dspark", - "communication", - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_build_local_metadata_triton.py", - "tests/ut/ops/test_rope_proxy.py", - "vllm_ascend/attention/context_parallel/dsa_cp.py", - "vllm_ascend/ops/rope_dsv4.py", - "vllm_ascend/ops/triton/dsa_cp.py" - ] - }, - { - "commit": "9c03b2a46b808ae19c27354e0b0e700694960a8c", - "subject": "[CI] Main2main: upgrade vLLM commit to 0717 (#12502)", - "topics": [ - "deepseek-v4-dspark", - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/vllm-main-verified.commit", - "tests/e2e/conftest.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_fused_recurrent_gated_delta_rule.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_fused_sigmoid_gating_delta_rule.py", - "tests/e2e/pull_request/one_card/model_runner_v2/test_basic.py", - "tests/ut/core/test_recompute_scheduler.py", - "tests/ut/distributed/ascend_store/test_config_data.py", - "tests/ut/distributed/ascend_store/test_coordinator.py", - "tests/ut/distributed/ascend_store/test_pool_worker.py", - "tests/ut/ops/a2/test_gdn_layerwise_kv.py", - "tests/ut/ops/test_gdn_attn_builder.py", - "tests/ut/ops/test_gdn_layerwise_kv.py", - "tests/ut/ops/test_vocab_parallel_embedding.py", - "tests/ut/patch/platform/test_prefix_cache_cp_patches.py", - "tests/ut/patch/worker/test_patch_qwen3_5_mtp.py", - "tests/ut/test_compressed_prefix_cache.py", - "vllm_ascend/_310p/ops/fla/idex.py", - "vllm_ascend/_310p/ops/fla/l2norm.py", - "vllm_ascend/core/recompute_scheduler.py", - "vllm_ascend/core/scheduler_profiling_chunk.py", - "vllm_ascend/core/single_type_kv_cache_manager.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/ascend_store/config_data.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/ascend_store/coordinator.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/ascend_store/pool_scheduler.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/ascend_store/pool_worker.py", - "vllm_ascend/ops/bailing_moe_linear_attn.py", - "vllm_ascend/ops/gdn.py", - "vllm_ascend/ops/triton/fla/chunk.py", - "vllm_ascend/ops/vocab_parallel_embedding.py", - "vllm_ascend/patch/__init__.py", - "vllm_ascend/patch/platform/patch_kv_cache_coordinator.py", - "vllm_ascend/patch/platform/patch_mamba_manager.py", - "vllm_ascend/patch/worker/__init__.py", - "vllm_ascend/patch/worker/patch_idex_310.py", - "vllm_ascend/patch/worker/patch_qwen3_5.py", - "vllm_ascend/patch/worker/patch_triton.py", - "vllm_ascend/patch/worker/patch_v2/patch_eagle_speculator.py", - "vllm_ascend/worker/v2/spec_decode/dflash/speculator.py", - "vllm_ascend/worker/v2/spec_decode/dspark/speculator.py", - "vllm_ascend/worker/v2/spec_decode/eagle/aclgraph.py", - "vllm_ascend/worker/v2/spec_decode/eagle/speculator.py" - ] - }, - { - "commit": "b909a4703de62ff606c9c1870690483a89f95ee6", - "subject": "[Feature][Attention] Support compact KV gather for SFA DCP prefill (#11980)", - "topics": [ - "communication", - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "docs/source/developer_guide/Design_Documents/context_parallel.md", - "tests/e2e/pull_request/four_card/context_parallel/test_accuracy.py", - "tests/ut/attention/test_sfa_cp.py", - "vllm_ascend/attention/context_parallel/sfa_cp.py", - "vllm_ascend/attention/sfa_v1.py", - "vllm_ascend/platform.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "f528cdd561969730504bfe8b5e96b6b7e69d6c4f", - "subject": "[BugFix][Worker] Replace copy_snapshot_to_gpu with synchronize_input_prep for query_start_loc buffers (#12630)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/worker/test_utils.py", - "vllm_ascend/_310p/model_runner_310p.py", - "vllm_ascend/spec_decode/llm_base_proposer.py", - "vllm_ascend/spec_decode/step3p5.py", - "vllm_ascend/worker/model_runner_v1.py", - "vllm_ascend/worker/pcp_utils.py", - "vllm_ascend/worker/utils.py" - ] - }, - { - "commit": "69ca54e89b3f34f1c40dfc2746d306c0a7368cde", - "subject": "[BugFix] Fix SFA DSA-CP prefill o_proj handling (#12683)", - "topics": [ - "communication" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/developer_guide/Design_Documents/context_parallel.md", - "tests/e2e/nightly/multi_node/internal_dp/config/DeepSeek-V3_2-W8A8-EP.yaml", - "tests/ut/test_utils.py", - "vllm_ascend/attention/sfa_v1.py", - "vllm_ascend/utils.py" - ] - }, - { - "commit": "91dc66476d66eba110b79c119d56f0fe22bf80e9", - "subject": "[Bugfix] Fix multi dp in dspark (#12612)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/spec_decode/test_dspark_proposer.py", - "vllm_ascend/spec_decode/dspark_proposer.py", - "vllm_ascend/spec_decode/llm_base_proposer.py" - ] - }, - { - "commit": "3ba88c0b0b1143a0544a571a9fa0569191d3b26f", - "subject": "[BugFix]match routed experts compressed tensors scheme (#12665)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/quantization/test_compressed_tensors_config.py", - "vllm_ascend/quantization/compressed_tensors_config.py" - ] - }, - { - "commit": "89578bc37fce53c5f3d17950c0c1dc5178b9e656", - "subject": "[Bugfix] fix low acc communication problem on W4A16MXFP (#12585)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/ops/fused_moe/token_dispatcher.py" - ] - }, - { - "commit": "4d2f5b4f6e8ba5ed0f1c907443c4e6fd29b53f83", - "subject": "[Feature][SFA] Support C8 SFA with DCP replicated indexer (#11870)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "tests/e2e/pull_request/four_card/context_parallel/test_accuracy.py", - "tests/ut/attention/a2/test_sfa_v1.py", - "tests/ut/attention/test_sfa_cp.py", - "vllm_ascend/attention/context_parallel/sfa_cp.py", - "vllm_ascend/device/device_op.py" - ] - }, - { - "commit": "c88d7a0a2473329377c9bb2d15429a2145d9ac4f", - "subject": "[Performance] remove D2H sync in QLIMetadata builder for DSA_CP (#12536)", - "topics": [ - "communication" - ], - "runtime_patch_match": false, - "paths": [ - "tests/e2e/pull_request/one_card/model_runner_v2/test_basic.py", - "vllm_ascend/attention/context_parallel/dsa_cp.py" - ] - }, - { - "commit": "61eb47b4d868246d1da446b945c7af82f0e3aea4", - "subject": "[Bugfix] fix main2main eplb (#12650)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "docs/source/user_guide/feature_guide/expert_parallelism_load_balancer.md", - "tests/e2e/pull_request/two_card/test_qwen3_30b_a3b.py", - "tests/e2e/pull_request/two_card/test_qwen3_moe_eplb.py", - "tests/ut/eplb/adaptor/test_vllm_adaptor.py", - "tests/ut/eplb/core/a2/test_eplb_utils.py", - "tests/ut/ops/test_fused_moe.py", - "vllm_ascend/eplb/adaptor/vllm_adaptor.py", - "vllm_ascend/eplb/core/eplb_utils.py", - "vllm_ascend/ops/fused_moe/fused_moe.py", - "vllm_ascend/patch/platform/patch_fused_moe.py" - ] - }, - { - "commit": "625cef2f197d1cba04022fe681e5c9fadc3776fc", - "subject": "[Revert] \"[Bugfix] Fix main2main eplb\" (#12619)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "docs/source/user_guide/feature_guide/expert_parallelism_load_balancer.md", - "tests/e2e/pull_request/two_card/test_qwen3_30b_a3b.py", - "tests/e2e/pull_request/two_card/test_qwen3_moe_eplb.py", - "tests/ut/eplb/core/a2/test_eplb_utils.py", - "tests/ut/ops/test_fused_moe.py", - "tests/ut/quantization/methods/test_w4a8_mxfp4.py", - "vllm_ascend/eplb/adaptor/vllm_adaptor.py", - "vllm_ascend/eplb/core/eplb_utils.py", - "vllm_ascend/ops/fused_moe/fused_moe.py", - "vllm_ascend/patch/platform/patch_fused_moe.py" - ] - }, - { - "commit": "b2dad425a4af3898407307e64275235180f2cc74", - "subject": "[Doc]Remove redundant `async_scheduling` config for EAGLE-based speculative decoding (#12528)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/speculative_decoding.po", - "docs/source/user_guide/feature_guide/speculative_decoding.md" - ] - }, - { - "commit": "2dfa766de129c5450e276fc226740b127f0fdd4e", - "subject": "[Bugfix] Fix main2main eplb (#12515)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "docs/source/user_guide/feature_guide/expert_parallelism_load_balancer.md", - "tests/e2e/pull_request/two_card/test_qwen3_30b_a3b.py", - "tests/e2e/pull_request/two_card/test_qwen3_moe_eplb.py", - "tests/ut/eplb/core/a2/test_eplb_utils.py", - "tests/ut/ops/test_fused_moe.py", - "tests/ut/quantization/methods/test_w4a8_mxfp4.py", - "vllm_ascend/eplb/adaptor/vllm_adaptor.py", - "vllm_ascend/eplb/core/eplb_utils.py", - "vllm_ascend/ops/fused_moe/fused_moe.py", - "vllm_ascend/patch/platform/patch_fused_moe.py" - ] - }, - { - "commit": "9327cc399cad4b1ca63305de951bdc644e75d318", - "subject": "[CI] Remove stale Transformers nightly overrides (#12503)", - "topics": [ - "deepseek-v4-dspark", - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "tests/e2e/nightly/multi_node/internal_dp/config/GLM5_1-W8A8-A2-dual-nodes.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/GLM5_1-W8A8-A3-dual-nodes.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/GLM5_2-W8A8-A3-dual-nodes.yaml", - "tests/e2e/nightly/single_node/models/configs/DeepSeek-V4-Flash-W8A8-A3.yaml" - ] - }, - { - "commit": "642026d0c457c9e0e87c4202bf57b6c3299d3af6", - "subject": "[CI]fix: include RoutedExperts in _is_fused_moe_layer for compressed_tensors and fp8 configs (#12495)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/quantization/compressed_tensors_config.py", - "vllm_ascend/quantization/fp8_config.py" - ] - }, - { - "commit": "f3a9ca955c717aa6c72d475776af2c39f9a73cdc", - "subject": "[CI]enable_fused_mc2 must be 0 or 1 (#12473)", - "topics": [ - "communication", - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/e2e/nightly/multi_node/internal_dp/config/Qwen3-235B-disagg-pd.yaml" - ] - }, - { - "commit": "ce229dc2dd3f8e5932d3749e20c6cbb9914e5968", - "subject": "[Feature] Support AlltoAll + EP + LoRA in non\u2011quantized mode (#12451)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/workflows/scripts/test_config.yaml", - "tests/e2e/pull_request/two_card/lora/test_qwen3moe_lora.py", - "tests/e2e/pull_request/two_card/lora/test_qwen3moe_lora_tp.py", - "vllm_ascend/ascend_forward_context.py", - "vllm_ascend/lora/fused_moe.py", - "vllm_ascend/ops/fused_moe/fused_moe.py", - "vllm_ascend/ops/fused_moe/moe_comm_method.py", - "vllm_ascend/ops/fused_moe/moe_mlp.py", - "vllm_ascend/ops/fused_moe/prepare_finalize.py", - "vllm_ascend/ops/fused_moe/token_dispatcher.py" - ] - }, - { - "commit": "44fc51ffb18dd05ca53c6509eae0058ba3c39333", - "subject": "[Doc][Misc] Remove unused external DP (generic_dp) proxy and docs (#12220)", - "topics": [ - "priority-models", - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "docs/hooks/nav_titles.py", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/external_dp.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/index.po", - "docs/source/user_guide/feature_guide/external_dp.md", - "docs/source/user_guide/feature_guide/index.md", - "examples/external_online_dp/README.md", - "examples/external_online_dp/dp_load_balance_proxy_server.py", - "mkdocs.yml", - "tests/e2e/nightly/multi_node/external_dp/config/template.md", - "tests/e2e/nightly/multi_node/external_dp/scripts/external_dp_config.py", - "tests/e2e/nightly/multi_node/external_dp/scripts/runtime.py" - ] - }, - { - "commit": "2184460b0d2e41c995f754f2aa7af5a81fadec27", - "subject": "[Doc] add deepseek-v4-flash-dspark cookbook (#12491)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/tutorials/models/DeepSeek-V4-Flash.md" - ] - }, - { - "commit": "e66d74d3519fdd63b6fe4a3b7e159e3139fc3a03", - "subject": "[Attention][Misc] Refactor MLAPO weight processing and preprocessing logic for Ascend SFA (#11529)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/attention/a2/test_sfa_v1.py", - "vllm_ascend/attention/sfa_v1.py", - "vllm_ascend/device/device_op.py" - ] - }, - { - "commit": "57fbf9e7a0cc3f7b227fcb13fd122d4d4222e83d", - "subject": "[CI] fix nighlty test (#12512)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/_310p/fused_moe/test_shared_fused_moe_310.py", - "tests/ut/ops/test_fused_moe.py", - "vllm_ascend/_310p/fused_moe/fused_moe.py", - "vllm_ascend/ops/fused_moe/fused_moe.py" - ] - }, - { - "commit": "070825300ea5322acd44ce5d5974cb1b19b1a96b", - "subject": "[BugFix] Fix FIA decode mask handling on main (#11266)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/attention/attention_v1.py" - ] - }, - { - "commit": "138da102b340025ce315a4d65054f9a0c7e18f7b", - "subject": "[Doc][Misc] Update deployment tutorial templates (#12474)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/_templates/Model-Deployment-Tutorial-Template.md", - "docs/source/_templates/Model-Deployment-Tutorial-Template.zh.md", - "docs/source/tutorials/models/DeepSeek-R1.md", - "docs/source/tutorials/models/DeepSeek-V3.1.md", - "docs/source/tutorials/models/GLM5.md", - "docs/source/tutorials/models/Kimi-K2.5.md", - "docs/source/tutorials/models/Kimi-K2.6.md", - "docs/source/user_guide/feature_guide/recompute_cpu_offload.md" - ] - }, - { - "commit": "06b06369d1333a7c935fd1346d72965bd0e03ae8", - "subject": "[Doc]Update GLM5.2.md (#12441)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/tutorials/models/GLM5.2.md" - ] - }, - { - "commit": "e60a39213d15f79bec15c694f63c3255d4b70e55", - "subject": "[CI] Adjust the acception baseline for deepseek dspark (#12456)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "tests/e2e/pull_request/four_card/spec_decode/test_dspark_deepseekv4.py" - ] - }, - { - "commit": "8fe122d95fc8cc94f32525f9ed36c2b5e3fbfdcc", - "subject": "[Feature][Refactor] DSv4 DSpark (#11431)", - "topics": [ - "deepseek-v4-dspark", - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/workflows/misc/model_dataset_list.json", - ".github/workflows/scripts/test_config.yaml", - "csrc/attention/sparse_attn_sharedkv/op_host/sparse_attn_sharedkv_tiling.cpp", - "csrc/attention/sparse_attn_sharedkv/op_host/sparse_attn_sharedkv_tiling.h", - "csrc/attention/sparse_attn_sharedkv/op_kernel/arch32/sparse_attn_sharedkv_scfa_kernel.h", - "csrc/attention/sparse_attn_sharedkv/op_kernel/arch32/sparse_attn_sharedkv_swa_block_cube.h", - "csrc/attention/sparse_attn_sharedkv/op_kernel/arch32/sparse_attn_sharedkv_swa_kernel.h", - "csrc/attention/sparse_attn_sharedkv/op_kernel/sparse_attn_sharedkv.cpp", - "csrc/attention/sparse_attn_sharedkv/op_kernel/sparse_attn_sharedkv_common.h", - "csrc/attention/sparse_attn_sharedkv_metadata/op_kernel_aicpu/sparse_attn_sharedkv_metadata_aicpu.cpp", - "tests/e2e/pull_request/four_card/spec_decode/test_dspark_deepseekv4.py", - "tests/e2e/pull_request/one_card/spec_decode/utils.py", - "tests/ut/spec_decode/a2/test_eagle_proposer.py", - "typos.toml", - "vllm_ascend/attention/dsa_v1.py", - "vllm_ascend/models/__init__.py", - "vllm_ascend/models/deepseek_v4.py", - "vllm_ascend/models/deepseek_v4_dspark.py", - "vllm_ascend/ops/mhc.py", - "vllm_ascend/patch/__init__.py", - "vllm_ascend/patch/platform/patch_speculative_config.py", - "vllm_ascend/patch/worker/__init__.py", - "vllm_ascend/patch/worker/patch_qwen3_dspark.py", - "vllm_ascend/spec_decode/__init__.py", - "vllm_ascend/spec_decode/dflash_proposer.py", - "vllm_ascend/spec_decode/dspark_proposer.py", - "vllm_ascend/spec_decode/llm_base_proposer.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "76ab5e28b965dd0c179bb85cf715b43152006fea", - "subject": "[Revert][Refactor][Attention] Restore paged attention fallback (reverts #11899) (#12228)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "docs/source/faqs.md", - "docs/source/locale/zh_CN/LC_MESSAGES/faqs.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/configuration/additional_config.po", - "docs/source/tutorials/features/suffix_speculative_decoding.md", - "docs/source/tutorials/models/Qwen3-Dense.md", - "tests/e2e/pull_request/two_card/test_qwen3_performance.py", - "tests/e2e/weekly/single_node/configs/Qwen3-32B.yaml", - "tests/ut/_310p/attention/test_attention_v1_310.py", - "tests/ut/attention/a2/test_attention_v1.py", - "vllm_ascend/ascend_config.py", - "vllm_ascend/attention/attention_v1.py", - "vllm_ascend/attention/utils.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "fd69c96a5aad41108523b7e8bdcc167bd2093f0a", - "subject": "[Feature][MRV2] Support FullGraph for DSpark (#12017)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "tests/e2e/pull_request/one_card/model_runner_v2/test_basic.py", - "vllm_ascend/worker/v2/aclgraph_utils.py", - "vllm_ascend/worker/v2/spec_decode/dspark/speculator.py" - ] - }, - { - "commit": "fe7bfc4746421a5f9ec99826c5e108a7c8edd916", - "subject": "[Misc] Upgrade vllm to 0.25.0 (#12233)", - "topics": [ - "deepseek-v4-dspark", - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/vllm-release-tag.commit", - "Dockerfile", - "Dockerfile.310p", - "Dockerfile.310p.openEuler", - "Dockerfile.a3", - "Dockerfile.a3.openEuler", - "Dockerfile.a5", - "Dockerfile.a5.openEuler", - "Dockerfile.openEuler", - "README.md", - "README.zh.md", - "pyproject.toml", - "requirements.txt", - "tests/e2e/pull_request/four_card/spec_decode/test_mtp_qwen3_next.py", - "tests/e2e/pull_request/four_card/spec_decode/test_mtp_step3p5.py", - "tests/e2e/pull_request/one_card/model_runner_v2/test_basic.py", - "tests/e2e/pull_request/one_card/model_runner_v2/test_uva.py", - "tests/e2e/pull_request/one_card/spec_decode/test_dflash.py", - "tests/e2e/pull_request/one_card/spec_decode/test_draft_parallel.py", - "tests/e2e/pull_request/one_card/spec_decode/test_dspark.py", - "tests/e2e/pull_request/one_card/spec_decode/test_eagle.py", - "tests/e2e/pull_request/one_card/spec_decode/test_ngram_npu.py", - "tests/e2e/pull_request/one_card/test_guided_decoding.py", - "tests/e2e/pull_request/two_card/model_runner_v2/test_data_parallel.py", - "tests/e2e/pull_request/two_card/spec_decode/test_spec_decode.py", - "tests/e2e/pull_request/two_card/test_gpt_oss_distributed.py", - "tests/e2e/pull_request/two_card/test_qwen3_30b_a3b.py", - "tests/ut/_310p/fused_moe/test_shared_fused_moe_310.py", - "tests/ut/distributed/weight_transfer/test_npu_ipc_engine.py", - "tests/ut/ops/test_fused_moe.py", - "tests/ut/patch/platform/test_patch_deepseek_v4_tool_call_parser.py", - "tests/ut/patch/platform/test_patch_parser_reasoning_usage.py", - "tests/ut/patch/test_hunyuan_vl_processor_compat.py", - "vllm_ascend/_310p/fused_moe/fused_moe.py", - "vllm_ascend/core/single_type_kv_cache_manager.py", - "vllm_ascend/distributed/weight_transfer/hccl_engine.py", - "vllm_ascend/distributed/weight_transfer/npu_ipc_engine.py", - "vllm_ascend/ops/fused_moe/fused_moe.py", - "vllm_ascend/ops/gdn.py", - "vllm_ascend/patch/__init__.py", - "vllm_ascend/patch/hunyuan_vl_processor_compat.py", - "vllm_ascend/patch/platform/__init__.py", - "vllm_ascend/patch/platform/patch_deepseek_v4_tool_call_parser.py", - "vllm_ascend/patch/platform/patch_kv_cache_coordinator.py", - "vllm_ascend/patch/platform/patch_parser_reasoning_usage.py", - "vllm_ascend/patch/worker/__init__.py", - "vllm_ascend/patch/worker/patch_deepseek_v2.py", - "vllm_ascend/patch/worker/patch_qwen3_5.py", - "vllm_ascend/patch/worker/patch_qwen3_dflash.py", - "vllm_ascend/spec_decode/llm_base_proposer.py", - "vllm_ascend/worker/block_table.py", - "vllm_ascend/worker/model_runner_v1.py", - "vllm_ascend/worker/v2/spec_decode/dflash/aclgraph.py", - "vllm_ascend/worker/worker.py" - ] - }, - { - "commit": "093ce8cb78fa3d558482098110704d53eed16f88", - "subject": "[Feature] Support NZ static buffers for prefetch offload (#11945)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/scripts/test_config.yaml", - "tests/e2e/pull_request/one_card/test_cpu_weight_offload.py", - "tests/ut/model_executor/test_offloader.py", - "vllm_ascend/model_executor/offloader/__init__.py", - "vllm_ascend/model_executor/offloader/base.py", - "vllm_ascend/model_executor/offloader/prefetch.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "b2dc738906bb191ef8e1ceace1f81de3a4336bf9", - "subject": "[Refactor][Patch] Remove obsolete parser patches (#11453)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/patch/platform/test_patch_glm47_tool_call_parser.py", - "tests/ut/patch/platform/test_patch_glm_tool_call_streaming.py", - "tests/ut/patch/platform/test_patch_tool_choice_none_content.py", - "vllm_ascend/patch/__init__.py", - "vllm_ascend/patch/platform/__init__.py", - "vllm_ascend/patch/platform/patch_glm47_tool_call_parser.py", - "vllm_ascend/patch/platform/patch_glm_tool_call_streaming.py", - "vllm_ascend/patch/platform/patch_tool_choice_none_content.py" - ] - }, - { - "commit": "8ad564eee8c1286360cd4284825a16df8977405d", - "subject": "[Doc] Update DeepSeek v4 support status and rename CANN install guide (#12358)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/installation.md", - "docs/source/locale/zh_CN/LC_MESSAGES/installation.po", - "docs/source/user_guide/support_matrix/supported_models.md" - ] - }, - { - "commit": "e1bd15041af9d9e1a274ae381da0bc0dd8b50e4d", - "subject": "[Refactor]refact hamming ops and call\uff0cremove sparse action (#12049)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/workflows/scripts/test_config.yaml", - "csrc/attention/reshape_and_cache_bnsd/CMakeLists.txt", - "csrc/attention/reshape_and_cache_bnsd/op_host/CMakeLists.txt", - "csrc/attention/reshape_and_cache_bnsd/op_host/reshape_and_cache_bnsd.cpp", - "csrc/attention/reshape_and_cache_bnsd/op_host/reshape_and_cache_bnsd_proto.cpp", - "csrc/attention/reshape_and_cache_bnsd/op_host/reshape_and_cache_bnsd_tiling.cpp", - "csrc/attention/reshape_and_cache_bnsd/op_host/reshape_and_cache_bnsd_tiling.h", - "csrc/attention/reshape_and_cache_bnsd/op_kernel/kernel_utils.h", - "csrc/attention/reshape_and_cache_bnsd/op_kernel/reshape_and_cache_bnsd.cpp", - "csrc/build_aclnn.sh", - "csrc/moe/hamming_dist_top_k/CMakeLists.txt", - "csrc/moe/hamming_dist_top_k/op_host/CMakeLists.txt", - "csrc/moe/hamming_dist_top_k/op_host/hamming_dist_top_k.cpp", - "csrc/moe/hamming_dist_top_k/op_host/hamming_dist_top_k.h", - "csrc/moe/hamming_dist_top_k/op_host/hamming_dist_top_k_def.cpp", - "csrc/moe/hamming_dist_top_k/op_host/hamming_dist_top_k_proto.cpp", - "csrc/moe/hamming_dist_top_k/op_host/hamming_dist_top_k_split.cpp", - "csrc/moe/hamming_dist_top_k/op_host/hamming_dist_top_k_split.h", - "csrc/moe/hamming_dist_top_k/op_host/hamming_dist_top_k_tiling.cpp", - "csrc/moe/hamming_dist_top_k/op_host/hamming_dist_top_k_tiling.h", - "csrc/moe/hamming_dist_top_k/op_host/op_host_util.h", - "csrc/moe/hamming_dist_top_k/op_kernel/hamming_dist_top_k.cpp", - "csrc/moe/hamming_dist_top_k/op_kernel/hamming_dist_top_k_base.h", - "csrc/moe/hamming_dist_top_k/op_kernel/hamming_dist_top_k_parallel.h", - "csrc/moe/hamming_dist_top_k/op_kernel/hamming_dist_top_k_split_s.h", - "csrc/torch_binding.cpp", - "csrc/torch_binding_meta.cpp", - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_hamming_dist_top_k.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_reshape_and_cache_bnsd.py", - "tests/ut/attention/a2/test_attention_v1.py", - "tests/ut/worker/a2/test_kvcomp_utils.py", - "vllm_ascend/ascend_config.py", - "vllm_ascend/attention/attention_v1.py", - "vllm_ascend/attention/kvcomp_attn/__init__.py", - "vllm_ascend/attention/kvcomp_attn/attention_utils.py", - "vllm_ascend/attention/utils.py", - "vllm_ascend/worker/kvcomp_utils.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "c794923e62b4050b8e812ec8f0d0b635088e064d", - "subject": "[Refactor] replace fused_gdn_gating from ascendc to triton (#12035)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "csrc/CMakeLists.txt", - "csrc/attention/fused_gdn_gating/fused_gdn_gating_torch_adpt.h", - "csrc/attention/fused_gdn_gating/op_host/CMakeLists.txt", - "csrc/attention/fused_gdn_gating/op_host/fused_gdn_gating_def.cpp", - "csrc/attention/fused_gdn_gating/op_host/fused_gdn_gating_infershape.cpp", - "csrc/attention/fused_gdn_gating/op_host/fused_gdn_gating_tiling.cpp", - "csrc/attention/fused_gdn_gating/op_host/fused_gdn_gating_tiling.h", - "csrc/attention/fused_gdn_gating/op_host/fused_gdn_gating_tiling_utils.h", - "csrc/attention/fused_gdn_gating/op_host/op_api/aclnn_fused_gdn_gating.cpp", - "csrc/attention/fused_gdn_gating/op_host/op_api/aclnn_fused_gdn_gating.h", - "csrc/attention/fused_gdn_gating/op_host/op_api/fused_gdn_gating.cpp", - "csrc/attention/fused_gdn_gating/op_host/op_api/fused_gdn_gating.h", - "csrc/attention/fused_gdn_gating/op_kernel/fused_gdn_gating.cpp", - "csrc/attention/fused_gdn_gating/op_kernel/fused_gdn_gating.h", - "csrc/attention/fused_gdn_gating/op_kernel/fused_gdn_gating_tiling_data.h", - "csrc/build_aclnn.sh", - "csrc/torch_binding.cpp", - "csrc/torch_binding_meta.cpp", - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_fused_gdn_gating.py", - "vllm_ascend/device/device_op.py" - ] - }, - { - "commit": "d823ceb5f7a155479e6307b36b2acf148d027477", - "subject": "[Doc] Translated Doc files 2026-07-18 (#12335)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/Design_Documents/balance_schedule_refactor.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/Design_Documents/quantization.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/contribution/testing.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/features/pd_disaggregation_mooncake_multi_node.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/GLM5.2.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen-VL-Dense.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-Dense.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-VL-235B-A22B-Instruct.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-VL-30B-A3B-Instruct.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3.6-35B-A3B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/configuration/additional_config.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/recompute_cpu_offload.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/short_request_first.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/speculative_decoding.po" - ] - }, - { - "commit": "5aafc0bce7dd2d92a4187a6613b426351c6c04d3", - "subject": "[Doc][Misc] Fix documentation navigation and formatting issues (#12334)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/developer_guide/performance_and_debug/index.md", - "docs/source/developer_guide/performance_and_debug/optimization_and_tuning.md", - "docs/source/developer_guide/performance_and_debug/service_profiling_guide.md", - "docs/source/user_guide/support_matrix/supported_models.md" - ] - }, - { - "commit": "f9e183086944ef1c0064a1c63344ae4ea3ff42a2", - "subject": "[Doc][Misc] Standardize support matrix terminology and update documentation anchors (#12314)", - "topics": [ - "deepseek-v4-dspark", - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "docs/hooks/nav_titles.py", - "docs/source/_templates/Model-Deployment-Tutorial-Template.zh.md", - "docs/source/developer_guide/Design_Documents/balance_schedule_refactor.md", - "docs/source/developer_guide/contribution/multi_node_test.md", - "docs/source/installation.md", - "docs/source/llms.txt", - "docs/source/locale/zh_CN/LC_MESSAGES/installation.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/features/dynamic_chunked_pipeline_parallel.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/DeepSeek-R1.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/DeepSeek-V3.1.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/DeepSeek-V4-Flash.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/DeepSeek-V4-Pro.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/DeepSeekOCR2.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/GLM5.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Gemma4.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/InternVL3.5.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Kimi-K2-Thinking.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Kimi-K2.5.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Kimi-K2.6.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/MiniMax-M2.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/PaddleOCR-VL.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen-VL-Dense.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen2.5-Math-RM-72B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-235B-A22B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-30B-A3B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-ASR-1.7B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-Coder-30B-A3B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-Dense.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-Next.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-Omni-30B-A3B-Thinking.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-VL-235B-A22B-Instruct.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-VL-30B-A3B-Instruct.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-VL-Embedding.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-VL-Reranker.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3.5-27B-Qwen3.6-27B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3_embedding.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3_reranker.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/index.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/index.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/release_notes.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/support_matrix/feature_matrix.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/support_matrix/index.po", - "docs/source/tutorials/models/Hunyuan-A13B-Instruct.md", - "docs/source/tutorials/models/Kimi-K2.5.md", - "docs/source/tutorials/models/PaddleOCR-VL.md", - "docs/source/tutorials/models/Qwen-VL-Dense.md", - "docs/source/user_guide/support_matrix/feature_matrix.md", - "docs/source/user_guide/support_matrix/index.md", - "docs/source/user_guide/support_matrix/supported_models.md", - "mkdocs.yml" - ] - }, - { - "commit": "34fa3b2165ab9415aadfaa9d8b7e658c14d41b6b", - "subject": "[Doc][Misc] Disable flashcomm1 in Qwen3-Omni-30B-A3B-Thinking tutorial (#12310)", - "topics": [ - "communication" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/tutorials/models/Qwen3-Omni-30B-A3B-Thinking.md" - ] - }, - { - "commit": "2ee50f3847e4873e61d280911b2a24d85f5278d6", - "subject": "[Refactor] Remove matmulallreduce and pass optimization techniques (#12119)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/workflows/scripts/test_config.yaml", - "csrc/build_aclnn.sh", - "csrc/mc2/matmul_allreduce_add_rmsnorm/CMakeLists.txt", - "csrc/mc2/matmul_allreduce_add_rmsnorm/matmul_allreduce_add_rmsnorm_torch_adpt.h", - "csrc/mc2/matmul_allreduce_add_rmsnorm/op_host/CMakeLists.txt", - "csrc/mc2/matmul_allreduce_add_rmsnorm/op_host/matmul_allreduce_add_rmsnorm_def.cpp", - "csrc/mc2/matmul_allreduce_add_rmsnorm/op_host/matmul_allreduce_add_rmsnorm_proto.cpp", - "csrc/mc2/matmul_allreduce_add_rmsnorm/op_host/matmul_allreduce_add_rmsnorm_tiling.cpp", - "csrc/mc2/matmul_allreduce_add_rmsnorm/op_host/matmul_allreduce_add_rmsnorm_workspace.h", - "csrc/mc2/matmul_allreduce_add_rmsnorm/op_host/op_api/aclnn_matmul_allreduce_add_rmsnorm.cpp", - "csrc/mc2/matmul_allreduce_add_rmsnorm/op_host/op_api/aclnn_matmul_allreduce_add_rmsnorm.h", - "csrc/mc2/matmul_allreduce_add_rmsnorm/op_kernel/matmul_allreduce_add_rmsnorm.cpp", - "csrc/mc2/matmul_allreduce_add_rmsnorm/op_kernel/matmul_allreduce_add_rmsnorm_aic_kernel.h", - "csrc/mc2/matmul_allreduce_add_rmsnorm/op_kernel/matmul_allreduce_add_rmsnorm_aiv_kernel.h", - "csrc/mc2/matmul_allreduce_add_rmsnorm/op_kernel/matmul_allreduce_add_rmsnorm_tiling.h", - "csrc/mc2/matmul_allreduce_add_rmsnorm/op_kernel/matmul_allreduce_add_rmsnorm_utils.h", - "csrc/moe/moe_grouped_matmul/op_host/moe_grouped_matmul_cpu.cpp", - "csrc/torch_binding.cpp", - "csrc/torch_binding_meta.cpp", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/configuration/additional_config.po", - "docs/source/user_guide/configuration/additional_config.md", - "tests/e2e/nightly/multi_node/external_dp/scripts/utils.py", - "tests/e2e/nightly/multi_node/internal_dp/scripts/test_multi_node.py", - "tests/e2e/nightly/single_node/models/scripts/test_single_node.py", - "tests/e2e/nightly/single_node/ops/multicard_ops_a2/test_matmul_allreduce_add_rmsnorm.py", - "tests/e2e/pull_request/two_card/test_external_launcher.py", - "tests/ut/ops/test_linear.py", - "tests/ut/spec_decode/a2/test_eagle_proposer.py", - "tests/ut/spec_decode/test_speculators_vwn_eagle3.py", - "tests/ut/test_ascend_config.py", - "tests/ut/test_batch_invariant.py", - "vllm_ascend/ascend_config.py", - "vllm_ascend/batch_invariant.py", - "vllm_ascend/compilation/graph_fusion_pass_manager.py", - "vllm_ascend/compilation/passes/allreduce_rmsnorm_fusion_pass.py", - "vllm_ascend/envs.py", - "vllm_ascend/ops/linear_op.py", - "vllm_ascend/utils.py" - ] - }, - { - "commit": "8abdd1eb322ae43931709d379784da7a94ba91bc", - "subject": "[Doc] Fix glm5.2 1M PD script (#12289)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/tutorials/models/GLM5.2.md" - ] - }, - { - "commit": "2044419f8f3dec6288eb1b1b1f54245d4c2b0904", - "subject": "[BugFix] Disable multistream_overlap_shared_expert when enable_fused_mc2 is enabled (#12237)", - "topics": [ - "communication", - "priority-models", - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "docs/source/tutorials/models/GLM5.2.md", - "docs/source/tutorials/models/GLM5.md", - "tests/e2e/nightly/single_node/models/configs/Qwen3.5-27B-w8a8-A2.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/GLM_5_1_PD_in32k_bs16-0.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/GLM_5_1_PD_in32k_bs20-90.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/GLM_5_1_PD_in64k_bs16-90.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/Kimi-K2.5-W4A8-128k-1k-TPOT50.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/MiniMax-PD-in32k-bs4-1.yaml", - "vllm_ascend/ascend_config.py" - ] - }, - { - "commit": "6582e57267f7eaed407dac091a708d8672fe485e", - "subject": "[Doc][Misc] Update num_speculative_tokens to 1 in model tutorials (#12144)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/tutorials/models/DeepSeek-V3.2.md", - "docs/source/tutorials/models/GLM4.x.md", - "docs/source/tutorials/models/GLM5.2.md", - "docs/source/tutorials/models/GLM5.md", - "docs/source/tutorials/models/Kimi-K2.5.md" - ] - }, - { - "commit": "2c7fd4f2db764e035ce7af2bbc415f5aac01dda6", - "subject": "[Doc](GLM5.2, DCP): update GLM5.2 docs with 1M Context Configuration (#12121)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/tutorials/models/GLM5.2.md" - ] - }, - { - "commit": "c3fff895f78bc155504ae64357361a351e9186fb", - "subject": "[Refactor] Remove dispatch_gmm_combine_decode ops (#12031)", - "topics": [ - "communication", - "priority-models", - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "csrc/build_aclnn.sh", - "csrc/mc2/dispatch_gmm_combine_decode/CMakeLists.txt", - "csrc/mc2/dispatch_gmm_combine_decode/dispatch_gmm_combine_decode_torch_adpt.h", - "csrc/mc2/dispatch_gmm_combine_decode/op_host/CMakeLists.txt", - "csrc/mc2/dispatch_gmm_combine_decode/op_host/dispatch_gmm_combine_decode_def.cpp", - "csrc/mc2/dispatch_gmm_combine_decode/op_host/dispatch_gmm_combine_decode_proto.cpp", - "csrc/mc2/dispatch_gmm_combine_decode/op_host/dispatch_gmm_combine_decode_tiling.cpp", - "csrc/mc2/dispatch_gmm_combine_decode/op_host/op_api/aclnn_dispatch_gmm_combine_decode.cpp", - "csrc/mc2/dispatch_gmm_combine_decode/op_host/op_api/aclnn_dispatch_gmm_combine_decode.h", - "csrc/mc2/dispatch_gmm_combine_decode/op_kernel/dispatch_gmm_combine_decode.cpp", - "csrc/mc2/dispatch_gmm_combine_decode/op_kernel/dispatch_gmm_combine_decode.h", - "csrc/mc2/dispatch_gmm_combine_decode/op_kernel/dispatch_gmm_combine_decode/epilogue/block/block_epilogue.h", - "csrc/mc2/dispatch_gmm_combine_decode/op_kernel/dispatch_gmm_combine_decode/epilogue/block/block_epilogue_bf16_fp16.hpp", - "csrc/mc2/dispatch_gmm_combine_decode/op_kernel/dispatch_gmm_combine_decode/epilogue/block/block_epilogue_per_token_dequant.hpp", - "csrc/mc2/dispatch_gmm_combine_decode/op_kernel/dispatch_gmm_combine_decode/epilogue/block/block_epilogue_per_token_dequant_swiglu.h", - "csrc/mc2/dispatch_gmm_combine_decode/op_kernel/dispatch_gmm_combine_decode/epilogue/block/block_epilogue_swiglu_bf16_fp16.h", - "csrc/mc2/dispatch_gmm_combine_decode/op_kernel/dispatch_gmm_combine_decode/epilogue/dispatch_policy.h", - "csrc/mc2/dispatch_gmm_combine_decode/op_kernel/dispatch_gmm_combine_decode/epilogue/tile/tile_stride_binary.h", - "csrc/mc2/dispatch_gmm_combine_decode/op_kernel/dispatch_gmm_combine_decode/epilogue/tile/tile_stride_muls.h", - "csrc/mc2/dispatch_gmm_combine_decode/op_kernel/dispatch_gmm_combine_decode/gemm/block/block_mmad.h", - "csrc/mc2/dispatch_gmm_combine_decode/op_kernel/dispatch_gmm_combine_decode/gemm/block/block_mmad_preload_async_with_callback_resident_a.h", - "csrc/mc2/dispatch_gmm_combine_decode/op_kernel/dispatch_gmm_combine_decode/gemm/dispatch_policy.h", - "csrc/mc2/dispatch_gmm_combine_decode/op_kernel/dispatch_gmm_combine_decode/gemm/kernel/grouped_matmul_slice_m_multistage_workspace_bf16_fp16.h", - "csrc/mc2/dispatch_gmm_combine_decode/op_kernel/dispatch_gmm_combine_decode/gemm/kernel/grouped_matmul_slice_m_per_token_dequant_multistage_workspace.h", - "csrc/mc2/dispatch_gmm_combine_decode/op_kernel/dispatch_gmm_combine_decode/gemm/kernel/grouped_matmul_slice_m_per_token_dequant_swiglu_quant_multistage_workspace.h", - "csrc/mc2/dispatch_gmm_combine_decode/op_kernel/dispatch_gmm_combine_decode/gemm/kernel/grouped_matmul_slice_m_swiglu_multistage_workspace_bf16_fp16.h", - "csrc/mc2/dispatch_gmm_combine_decode/op_kernel/dispatch_gmm_combine_decode/raw_distributed/cam_moe_distribute_combine.h", - "csrc/mc2/dispatch_gmm_combine_decode/op_kernel/dispatch_gmm_combine_decode/raw_distributed/cam_moe_distribute_dispatch.h", - "csrc/mc2/dispatch_gmm_combine_decode/op_kernel/dispatch_gmm_combine_decode_base.h", - "csrc/mc2/dispatch_gmm_combine_decode/op_kernel/dispatch_gmm_combine_decode_bf16_fp16.h", - "csrc/mc2/dispatch_gmm_combine_decode/op_kernel/dispatch_gmm_combine_decode_tiling.h", - "csrc/mc2/dispatch_gmm_combine_decode/op_kernel/moe_distribute_base.h", - "csrc/torch_binding.cpp", - "csrc/torch_binding_meta.cpp", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/GLM5.2.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/GLM5.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/InternVL3.5.po", - "docs/source/tutorials/models/GLM5.2.md", - "docs/source/tutorials/models/GLM5.md", - "docs/source/tutorials/models/InternVL3.5.md", - "docs/source/tutorials/models/Qwen3-235B-A22B.md", - "tests/e2e/nightly/single_node/ops/multicard_ops_a3/test_dispatch_gmm_combine_decode.py", - "tests/ut/test_ascend_config.py", - "tests/ut/test_ascend_forward_context.py", - "vllm_ascend/ascend_config.py", - "vllm_ascend/ascend_forward_context.py", - "vllm_ascend/envs.py", - "vllm_ascend/ops/fused_moe/moe_comm_method.py", - "vllm_ascend/utils.py" - ] - }, - { - "commit": "c2a8d0c4676120888c8fc7ea72864a8f592c2f2f", - "subject": "[Cherry-pick][main][Feature][KVCache] Add multi-group layerwise KV cache support for DSV4 with MemCache backend (#12147)", - "topics": [ - "deepseek-v4-dspark", - "communication" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/attention/context_parallel/dsa_cp.py", - "vllm_ascend/attention/dsa_v1.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/ascend_store/config_data.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/ascend_store/kv_transfer.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/ascend_store/pool_scheduler.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/ascend_store/pool_worker.py" - ] - }, - { - "commit": "4edcfe270be83e5c28ec48daa222ba185c8d122f", - "subject": "[CI][Misc] Update MiniMax-M2.5 test config: set environment variables and parameters, and adjust baselines (#11972)", - "topics": [ - "communication", - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/e2e/weekly/single_node/configs/MiniMax-M2.5-w8a8-QuaRot-A3.yaml" - ] - }, - { - "commit": "e77502ef715f612f2cc6f3c833174431f26cb765", - "subject": "[Refactor] Remove weight prefetch config (#11949)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/workflows/scripts/test_config.yaml", - "docs/hooks/nav_titles.py", - "docs/source/_templates/Model-Deployment-Tutorial-Template.md", - "docs/source/_templates/Model-Deployment-Tutorial-Template.zh.md", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/configuration/additional_config.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/weight_prefetch.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/support_matrix/supported_models.po", - "docs/source/user_guide/configuration/additional_config.md", - "docs/source/user_guide/feature_guide/weight_prefetch.md", - "docs/source/user_guide/support_matrix/supported_models.md", - "mkdocs.yml", - "tests/e2e/nightly/multi_node/internal_dp/scripts/test_multi_node.py", - "tests/e2e/nightly/single_node/models/configs/Qwen3-32B-Int8-A2.yaml", - "tests/e2e/nightly/single_node/models/configs/Qwen3-32B-Int8.yaml", - "tests/e2e/nightly/single_node/models/configs/Qwen3-VL-32B-Instruct-W8A8.yaml", - "tests/e2e/nightly/single_node/models/configs/Qwen3.5-27B-w8a8-A2.yaml", - "tests/e2e/nightly/single_node/models/scripts/test_single_node.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_fused_moe.py", - "tests/e2e/pull_request/two_card/test_flashcomm_distributed.py", - "tests/ut/attention/a2/test_mla_cp.py", - "tests/ut/attention/a2/test_mla_precision.py", - "tests/ut/attention/a2/test_mla_v1.py", - "tests/ut/ops/a2/test_weight_prefetch.py", - "tests/ut/ops/a3_2/test_activation.py", - "tests/ut/ops/a3_2/test_select_experts.py", - "tests/ut/ops/test_linear.py", - "tests/ut/ops/test_moe_mlp.py", - "tests/ut/quantization/methods/a2/test_w8a8_static.py", - "tests/ut/spec_decode/test_speculators_vwn_eagle3.py", - "vllm_ascend/ascend_config.py", - "vllm_ascend/attention/mla_v1.py", - "vllm_ascend/attention/sfa_v1.py", - "vllm_ascend/ops/activation.py", - "vllm_ascend/ops/fused_moe/experts_selector.py", - "vllm_ascend/ops/fused_moe/moe_mlp.py", - "vllm_ascend/ops/layernorm.py", - "vllm_ascend/ops/linear_op.py", - "vllm_ascend/ops/register_custom_ops.py", - "vllm_ascend/ops/weight_prefetch.py", - "vllm_ascend/quantization/methods/w8a8_static.py", - "vllm_ascend/utils.py", - "vllm_ascend/worker/model_runner_v1.py", - "vllm_ascend/worker/v2/model_runner.py" - ] - }, - { - "commit": "9f692f3db67160525ac93cd4792e477db1530c61", - "subject": "[KV Cache][Feature] Support Layerwise KV Pooling with Memcache Backendche (#11444)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/workflows/scripts/test_config.yaml", - "docs/source/user_guide/feature_guide/index.md", - "docs/source/user_guide/feature_guide/layerwise_kv_pool.md", - "tests/ut/attention/a2/test_mla_v1.py", - "tests/ut/conftest.py", - "tests/ut/distributed/ascend_store/_mock_deps.py", - "tests/ut/distributed/ascend_store/test_ascend_store_connector.py", - "tests/ut/distributed/ascend_store/test_config_data.py", - "tests/ut/distributed/ascend_store/test_kv_transfer.py", - "tests/ut/distributed/ascend_store/test_pool_scheduler.py", - "tests/ut/distributed/ascend_store/test_pool_worker.py", - "tests/ut/distributed/mooncake/test_mooncake_kv_transfer.py", - "vllm_ascend/attention/attention_v1.py", - "vllm_ascend/attention/context_parallel/attention_cp.py", - "vllm_ascend/attention/context_parallel/mla_cp.py", - "vllm_ascend/attention/mla_v1.py", - "vllm_ascend/attention/sfa_v1.py", - "vllm_ascend/attention/utils.py", - "vllm_ascend/distributed/kv_transfer/kv_p2p/mooncake_layerwise_connector.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/ascend_store/ascend_store_connector.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/ascend_store/backend/__init__.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/ascend_store/backend/backend.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/ascend_store/backend/memcache_backend.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/ascend_store/backend/mooncake_backend.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/ascend_store/config_data.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/ascend_store/kv_transfer.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/ascend_store/pool_scheduler.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/ascend_store/pool_worker.py", - "vllm_ascend/memcache_comm_fence.py" - ] - }, - { - "commit": "15818534ecff88c3e04b4bfa234d0067bf17f3b3", - "subject": "[CI] vllm drop v0.23.0 and upgrade v0.24.0 (#12019)", - "topics": [ - "deepseek-v4-dspark", - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/vllm-release-tag.commit", - "Dockerfile", - "Dockerfile.310p", - "Dockerfile.310p.openEuler", - "Dockerfile.a3", - "Dockerfile.a3.openEuler", - "Dockerfile.a5", - "Dockerfile.a5.openEuler", - "Dockerfile.openEuler", - "README.md", - "README.zh.md", - "docs/source/community/slash-commands.md", - "docs/source/developer_guide/Design_Documents/balance_schedule_refactor.md", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/Design_Documents/balance_schedule_refactor.po", - "examples/offline_data_parallel.py", - "tests/e2e/conftest.py", - "tests/e2e/pull_request/one_card/model_runner_v2/test_basic.py", - "tests/e2e/pull_request/one_card/model_runner_v2/test_uva.py", - "tests/e2e/pull_request/one_card/spec_decode/test_dspark.py", - "tests/e2e/pull_request/one_card/test_guided_decoding.py", - "tests/e2e/pull_request/two_card/test_gpt_oss_distributed.py", - "tests/e2e/pull_request/two_card/test_qwen3_30b_a3b.py", - "tests/e2e/pull_request/two_card/test_qwen3_moe_eplb.py", - "tests/ut/_310p/fused_moe/test_shared_fused_moe_310.py", - "tests/ut/_310p/quantization/test_modelslim_config_310.py", - "tests/ut/distributed/weight_transfer/test_npu_ipc_engine.py", - "tests/ut/eplb/core/a2/test_eplb_utils.py", - "tests/ut/ops/test_fused_moe.py", - "tests/ut/ops/test_gdn_attn_builder.py", - "tests/ut/patch/platform/test_patch_balance_schedule.py", - "tests/ut/patch/platform/test_patch_deepseek_v4_tool_call_parser.py", - "tests/ut/patch/test_hunyuan_vl_processor_compat.py", - "tests/ut/quantization/test_compressed_tensors_config.py", - "tests/ut/quantization/test_modelslim_config.py", - "tests/ut/spec_decode/a2/test_eagle_proposer.py", - "tests/ut/spec_decode/test_extract_hidden_states_proposer.py", - "vllm_ascend/_310p/fused_moe/fused_moe.py", - "vllm_ascend/_310p/quantization/modelslim_config.py", - "vllm_ascend/_310p/worker_310p.py", - "vllm_ascend/core/single_type_kv_cache_manager.py", - "vllm_ascend/distributed/weight_transfer/hccl_engine.py", - "vllm_ascend/distributed/weight_transfer/npu_ipc_engine.py", - "vllm_ascend/models/deepseek_v4.py", - "vllm_ascend/models/deepseek_v4_mtp.py", - "vllm_ascend/ops/fused_moe/fused_moe.py", - "vllm_ascend/ops/fused_moe/fused_moe_0_23_0.py", - "vllm_ascend/ops/fused_moe/moe_runtime_args.py", - "vllm_ascend/ops/gdn.py", - "vllm_ascend/patch/hunyuan_vl_processor_compat.py", - "vllm_ascend/patch/platform/__init__.py", - "vllm_ascend/patch/platform/patch_balance_schedule.py", - "vllm_ascend/patch/platform/patch_deepseek_v4_tool_call_parser.py", - "vllm_ascend/patch/platform/patch_dp_device_ids.py", - "vllm_ascend/patch/platform/patch_fused_moe.py", - "vllm_ascend/patch/platform/patch_kv_cache_coordinator.py", - "vllm_ascend/patch/platform/patch_kv_cache_utils.py", - "vllm_ascend/patch/platform/patch_profiling_chunk.py", - "vllm_ascend/patch/platform/patch_torch_accelerator.py", - "vllm_ascend/patch/worker/__init__.py", - "vllm_ascend/patch/worker/patch_deepseek_v2.py", - "vllm_ascend/patch/worker/patch_qwen3_5.py", - "vllm_ascend/patch/worker/patch_qwen3_dflash.py", - "vllm_ascend/quantization/compressed_tensors_config.py", - "vllm_ascend/quantization/fp8_config.py", - "vllm_ascend/quantization/modelslim_config.py", - "vllm_ascend/spec_decode/llm_base_proposer.py", - "vllm_ascend/spec_decode/ngram_proposer.py", - "vllm_ascend/spec_decode/suffix_proposer.py", - "vllm_ascend/utils.py", - "vllm_ascend/worker/block_table.py", - "vllm_ascend/worker/encoder_acl_graph.py", - "vllm_ascend/worker/model_runner_v1.py", - "vllm_ascend/worker/worker.py" - ] - }, - { - "commit": "58c315652ae0450698c1e499638e2ee7094857d1", - "subject": "[Bugfix][310P] Adapt unified spec decoding method and fix spec decoding accuracy issue. (#11918)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/_310p/attention/test_attention_v1_310.py", - "tests/ut/_310p/ops/test_rotary_embedding_310.py", - "tests/ut/_310p/spec_decode/test_llm_base_proposer_310.py", - "vllm_ascend/_310p/attention/metadata_builder.py", - "vllm_ascend/_310p/model_runner_310p.py", - "vllm_ascend/_310p/ops/rotary_embedding.py", - "vllm_ascend/_310p/spec_decode/llm_base_proposer_310.py", - "vllm_ascend/patch/worker/patch_idex_310.py" - ] - }, - { - "commit": "77baade98ded755b3e5ea30091bcf40a30ae0335", - "subject": "[Refactor] Remove multistream overlap gate (#11956)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/CODEOWNERS", - ".github/workflows/scripts/test_config.yaml", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/configuration/additional_config.po", - "docs/source/user_guide/configuration/additional_config.md", - "tests/ut/ops/test_fused_moe.py", - "tests/ut/ops/test_prepare_finalize.py", - "tests/ut/quantization/conftest_quantization.py", - "tests/ut/quantization/methods/a2/test_w8a8_dynamic.py", - "tests/ut/quantization/methods/test_w8a8fp8_dynamic.py", - "tests/ut/test_flash_common3_context.py", - "vllm_ascend/_310p/fused_moe/fused_moe.py", - "vllm_ascend/ascend_config.py", - "vllm_ascend/distributed/parallel_state.py", - "vllm_ascend/distributed/utils.py", - "vllm_ascend/flash_common3_context.py", - "vllm_ascend/lora/fused_moe.py", - "vllm_ascend/ops/fused_moe/fused_moe.py", - "vllm_ascend/ops/fused_moe/fused_moe_0_23_0.py", - "vllm_ascend/ops/fused_moe/prepare_finalize.py", - "vllm_ascend/quantization/methods/w8a8_dynamic.py", - "vllm_ascend/quantization/methods/w8a8_mxfp8.py" - ] - }, - { - "commit": "6e784075dcc36b603296f03a50cdc005cffe5c61", - "subject": "[BugFix][MRV2] Fix quant and DP full graph mode for mrv2 (#11727)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/scripts/test_config.yaml", - "tests/e2e/pull_request/two_card/model_runner_v2/test_data_parallel.py", - "tests/ut/worker/a2/test_worker_v1.py", - "vllm_ascend/quantization/method_adapters.py", - "vllm_ascend/worker/v2/aclgraph_utils.py", - "vllm_ascend/worker/v2/spec_decode/eagle/aclgraph.py", - "vllm_ascend/worker/worker.py" - ] - }, - { - "commit": "b5630f15877c7df0ab92ad40d71ffcbd5aa02f2b", - "subject": "[BugFix]Fix fused_infer_attention ops contiguous err in GQA (#12025)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/attention/context_parallel/attention_cp.py", - "vllm_ascend/device/device_op.py" - ] - }, - { - "commit": "896012151dbaaa444e940f19106fde877d616dc9", - "subject": "[Refactor] remove layer sharding (#11953)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - ".agents/skills/vllm-ascend-release/references/ref-past-release-notes-highlight.md", - ".github/workflows/scripts/test_config.yaml", - "docs/hooks/nav_titles.py", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/GLM5.2.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/GLM5.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/InternVL3.5.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/configuration/additional_config.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/layer_sharding.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/rfork.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/release_notes.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/support_matrix/feature_matrix.po", - "docs/source/tutorials/models/DeepSeek-V3.2.md", - "docs/source/tutorials/models/GLM5.2.md", - "docs/source/tutorials/models/GLM5.md", - "docs/source/tutorials/models/InternVL3.5.md", - "docs/source/user_guide/configuration/additional_config.md", - "docs/source/user_guide/feature_guide/images/layer_sharding.png", - "docs/source/user_guide/feature_guide/layer_sharding.md", - "docs/source/user_guide/feature_guide/rfork.md", - "docs/source/user_guide/release_notes.md", - "docs/source/user_guide/support_matrix/feature_matrix.md", - "mkdocs.yml", - "tests/e2e/coverage.html", - "tests/e2e/nightly/multi_node/internal_dp/config/DeepSeek-V3_2-W8A8-EP.yaml", - "tests/e2e/pull_request/two_card/test_flashcomm_distributed.py", - "tests/e2e/weekly/multi_node/external_dp/config/DeepSeek_V3.2T_MTP2_PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/DeepSeek_V3.2T_MTP3_PD.yaml", - "tests/e2e/weekly/multi_node/internal_dp/config/DeepSeek-V3_2-W8A8-EP_weekly.yaml", - "tests/ut/attention/a2/test_mla_v1.py", - "tests/ut/attention/a2/test_sfa_v1.py", - "tests/ut/attention/test_sfa_cp.py", - "tests/ut/model_loader/rfork/test_rfork_loader.py", - "tests/ut/ops/test_flashcomm2_oshard_manager.py", - "tests/ut/ops/test_layer_shard_linear.py", - "tests/ut/ops/test_linear.py", - "tests/ut/quantization/test_method_adapters.py", - "tests/ut/test_ascend_config.py", - "tests/ut/test_platform.py", - "tests/ut/test_utils.py", - "vllm_ascend/ascend_config.py", - "vllm_ascend/attention/attention_v1.py", - "vllm_ascend/attention/mla_v1.py", - "vllm_ascend/attention/sfa_v1.py", - "vllm_ascend/distributed/parallel_state.py", - "vllm_ascend/model_loader/rfork/rfork_loader.py", - "vllm_ascend/ops/flashcomm2_oshard_manager.py", - "vllm_ascend/ops/layer_shard_linear.py", - "vllm_ascend/ops/linear_op.py", - "vllm_ascend/platform.py", - "vllm_ascend/quantization/method_adapters.py", - "vllm_ascend/utils.py" - ] - }, - { - "commit": "d4d32d725d7614bedf06b09d848e60c24e0c66c2", - "subject": "[BugFix]Fix fused_infer_attention ops contiguous err in mla_cp (#11986)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/attention/context_parallel/mla_cp.py" - ] - }, - { - "commit": "6e52ce3e79f78a788dc154761880e94639e46267", - "subject": "[BugFix] Fix AllGather MoE finalize order with DP and PCP (#11921)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/ops/fused_moe/prepare_finalize.py" - ] - }, - { - "commit": "1a1392bf0adc0630bc7f4b3f2a385ed46b272c5a", - "subject": "[BugFix][Attention] Fix stale KV lengths in hamming sparse attention under ACL graph replay (#10895)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/attention/a2/test_attention_v1.py", - "vllm_ascend/attention/attention_v1.py", - "vllm_ascend/attention/kvcomp_attn/attention_utils.py" - ] - }, - { - "commit": "ab5dcdb1c38bd8f7cd2e3f5c862dd64d7eba6b4e", - "subject": "[Ascend950][Bugfix]Fix allgatherEP MXFPW4A4 without sp (#11924)", - "topics": [ - "communication", - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/ops/a2/test_token_dispatcher.py", - "vllm_ascend/ops/fused_moe/token_dispatcher.py" - ] - }, - { - "commit": "f6b33f49cd732733769374c19b68d26a5c210ec0", - "subject": "[Refactor][Attention] Remove paged attention (#11899)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "csrc/attention/kv_quant_sparse_flash_attention/op_host/kv_quant_sparse_flash_attention_tiling.cpp", - "docs/source/faqs.md", - "docs/source/locale/zh_CN/LC_MESSAGES/faqs.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/configuration/additional_config.po", - "docs/source/tutorials/features/suffix_speculative_decoding.md", - "docs/source/tutorials/models/Qwen3-Dense.md", - "docs/source/user_guide/configuration/additional_config.md", - "tests/e2e/pull_request/two_card/test_qwen3_performance.py", - "tests/e2e/weekly/single_node/configs/Qwen3-32B.yaml", - "tests/ut/_310p/attention/test_attention_v1_310.py", - "tests/ut/attention/a2/test_attention_v1.py", - "vllm_ascend/ascend_config.py", - "vllm_ascend/attention/attention_v1.py", - "vllm_ascend/attention/utils.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "5083d8844310831258f085ea6dfcac4a2f76ef58", - "subject": "[CI]main2main 0710 (#11709)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - ".github/vllm-main-verified.commit", - "pyproject.toml", - "requirements.txt", - "tests/e2e/pull_request/one_card/spec_decode/test_extract_hidden_states.py", - "tests/ut/patch/platform/test_patch_deepseek_v4_tool_call_parser.py", - "tests/ut/patch/test_hunyuan_vl_processor_compat.py", - "vllm_ascend/__init__.py", - "vllm_ascend/core/single_type_kv_cache_manager.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/cpu_offload/cpu_kv_cache_manager.py", - "vllm_ascend/patch/__init__.py", - "vllm_ascend/patch/hunyuan_vl_processor_compat.py", - "vllm_ascend/patch/platform/__init__.py", - "vllm_ascend/patch/platform/patch_deepseek_v4_tool_call_parser.py", - "vllm_ascend/patch/platform/patch_kv_cache_coordinator.py", - "vllm_ascend/worker/block_table.py", - "vllm_ascend/worker/model_runner_v1.py", - "vllm_ascend/worker/v2/attn_utils.py", - "vllm_ascend/worker/v2/spec_decode/dflash/speculator.py", - "vllm_ascend/worker/v2/spec_decode/rejection_sampler_utils.py" - ] - }, - { - "commit": "a0d5f932dba9ca43251ab884b357c3c4a27b0fd3", - "subject": "[Doc][Misc] Update vLLM serve configuration for GLM models (#11640)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/tutorials/models/GLM4.x.md", - "docs/source/tutorials/models/GLM5.2.md", - "docs/source/tutorials/models/GLM5.md" - ] - }, - { - "commit": "038a844fbcb84e7aa0d8ef1b06225494c5a77a78", - "subject": "[Doc][Misc] Remove invalid fuse_muls_add from additional-config (#11911)", - "topics": [ - "priority-models", - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/tutorials/models/GLM5.2.md", - "docs/source/tutorials/models/GLM5.md", - "tests/e2e/nightly/multi_node/external_dp/config/GLM5_1-W8A8-EP-external.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/GLM5_1-W8A8-A2-dual-nodes.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/GLM5_1-W8A8-A3-dual-nodes.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/GLM5_1-W8A8-EP.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/GLM5_2-W8A8-A3-dual-nodes.yaml", - "tests/e2e/nightly/single_node/models/configs/DeepSeek-V3.2-W8A8-DCP.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/GLM_5_1_PD_in32k_bs16-0.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/GLM_5_1_PD_in32k_bs20-90.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/GLM_5_1_PD_in64k_bs16-90.yaml", - "tests/e2e/weekly/single_node/configs/GLM-5.yaml", - "tests/e2e/weekly/single_node/configs/GLM-5_1-W8A8_A3_weekly.yaml" - ] - }, - { - "commit": "15474a661e4cfd28a135e1f87a193911829d0c50", - "subject": "[Feature] Support Step3.5/3.7 Flash for Ascend950 (#10556)", - "topics": [ - "communication", - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/ops/test_moe_mlp.py", - "vllm_ascend/ops/fused_moe/moe_mlp.py", - "vllm_ascend/patch/__init__.py", - "vllm_ascend/patch/worker/__init__.py", - "vllm_ascend/patch/worker/patch_step3p5.py", - "vllm_ascend/utils.py" - ] - }, - { - "commit": "41ff81e1a7a92e6e3f546198d80702d78b7a50b7", - "subject": "[Feature] Add qwen/glm dspark for mrv1 (#11765)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/scripts/test_config.yaml", - "tests/e2e/pull_request/one_card/spec_decode/test_dspark.py", - "tests/e2e/pull_request/one_card/spec_decode/utils.py", - "vllm_ascend/ops/triton/spec_decode/utils.py", - "vllm_ascend/patch/__init__.py", - "vllm_ascend/patch/worker/__init__.py", - "vllm_ascend/patch/worker/patch_qwen3_dspark.py", - "vllm_ascend/spec_decode/__init__.py", - "vllm_ascend/spec_decode/dflash_proposer.py", - "vllm_ascend/spec_decode/dspark_proposer.py", - "vllm_ascend/spec_decode/llm_base_proposer.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "3f3493ed9392570421fa320294fdb18758cd8334", - "subject": "[Misc] Fix issues related to graph fusion (#11776)", - "topics": [ - "priority-models", - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "docs/source/tutorials/models/GLM5.2.md", - "docs/source/tutorials/models/Qwen3-Dense.md", - "docs/source/user_guide/configuration/additional_config.md", - "tests/e2e/nightly/multi_node/internal_dp/config/GLM5_1-W8A8-A2-dual-nodes.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/GLM5_1-W8A8-A3-dual-nodes.yaml", - "tests/e2e/weekly/single_node/configs/GLM-5_1-W8A8_A3_weekly.yaml", - "vllm_ascend/compilation/passes/norm_quant_fusion_pass.py" - ] - }, - { - "commit": "3e34542d65d11ba62b96adad5bd765b3388e22ec", - "subject": "[Test] Add model and coverage marker for e2e test (#11838)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/scripts/coverage.py", - "pyproject.toml", - "tests/e2e/conftest.py", - "tests/e2e/coverage.html", - "tests/e2e/coverage.md", - "tests/e2e/coverage_taxonomy.py", - "tests/e2e/generate_coverage_html.py", - "tests/e2e/generate_coverage_md.py", - "tests/e2e/pull_request/four_card/test_deepseek_v4.py", - "tests/e2e/pull_request/one_card/lora/test_qwen3_multi_loras.py", - "tests/e2e/pull_request/one_card/test_batch_invariant.py", - "tests/e2e/pull_request/one_card/test_qwen3_0_6b.py", - "tests/e2e/pull_request/two_card/test_qwen3_30b_a3b.py" - ] - }, - { - "commit": "f932d3404f0674aafc25522956e31b84de7709c4", - "subject": "[BugFix]Fix hc_pre ops issue (#11825)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "csrc/moe/hc_pre/op_kernel/hc_pre_m_k_split_core_arch35.h" - ] - }, - { - "commit": "587ac6aa8920881a4276ee6cea30c33392424e9e", - "subject": "[Feature][Refactor] Support SFA C8 on A3 with unified packed KV cache layout (#11228)", - "topics": [ - "priority-models", - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/e2e/pull_request/four_card/context_parallel/test_accuracy.py", - "tests/ut/attention/a2/test_sfa_v1.py", - "tests/ut/worker/a2/test_model_runner_v1.py", - "vllm_ascend/attention/sfa_v1.py", - "vllm_ascend/attention/utils.py", - "vllm_ascend/core/kv_cache_interface.py", - "vllm_ascend/device/device_op.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "2f8934ae30228d1b6874a7d41bbf9d23c0e77544", - "subject": "[BugFix] Fix Qwen3.x KV cache binding for multiple layers (#11470)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/patch/worker/patch_qwen3_next_mtp.py" - ] - }, - { - "commit": "dbcbf025a15ffbe33ca3a481924d6799a212efdc", - "subject": "[BugFix]Fix fused_infer_attention ops contiguous err (#11790)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/attention/mla_v1.py" - ] - }, - { - "commit": "accfe253e3b4ac39b80c94e6ddb4b192cb13eaa9", - "subject": "[CI] add Deepseekv4-flash-w8a8-PD.yaml and modify Kimi-k2.5-w4a8-16k-1k-TPOT50.yaml (#11734)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/misc/model_dataset_list.json", - "tests/e2e/weekly/multi_node/external_dp/config/DeepSeek-V4-flash-w8a8-PD.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/Kimi-K2.5-W4A8-16k-1k-TPOT50.yaml" - ] - }, - { - "commit": "d183f00db71c0d303f5d0d035c1d3b35425d9d1f", - "subject": "[BugFix] Guard mixed structured output backends (#11589)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "tests/e2e/pull_request/one_card/test_guided_decoding.py", - "tests/ut/patch/platform/test_patch_structured_output.py", - "vllm_ascend/patch/__init__.py", - "vllm_ascend/patch/platform/__init__.py", - "vllm_ascend/patch/platform/patch_structured_output.py" - ] - }, - { - "commit": "7d953ad85b0f807e0ffaae9a7e30cfd861f821c1", - "subject": "[Misc] Standardize the naming convention for QuantType enum. (#11694)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/eplb/adaptor/test_vllm_adaptor.py", - "tests/ut/ops/a2/test_token_dispatcher.py", - "tests/ut/ops/test_moe_mlp.py", - "tests/ut/ops/test_moe_runtime_args.py", - "tests/ut/quantization/methods/test_w8a8fp8_dynamic.py", - "vllm_ascend/device/device_op.py", - "vllm_ascend/eplb/adaptor/vllm_adaptor.py", - "vllm_ascend/ops/fused_moe/moe_mlp.py", - "vllm_ascend/ops/fused_moe/moe_runtime_args.py", - "vllm_ascend/ops/fused_moe/moe_stage_params.py", - "vllm_ascend/ops/fused_moe/prepare_finalize.py", - "vllm_ascend/ops/fused_moe/token_dispatcher.py", - "vllm_ascend/quantization/methods/w4a16_mxfp4.py", - "vllm_ascend/quantization/methods/w4a4_mxfp4.py", - "vllm_ascend/quantization/methods/w8a8_mxfp8.py", - "vllm_ascend/quantization/methods/w8a8fp8_dynamic.py", - "vllm_ascend/quantization/quant_type.py" - ] - }, - { - "commit": "83b987712b31917fb1aabc5a905d8925324b3add", - "subject": "[BugFix] Skip FlashComm1/2 For ViT (#10941)", - "topics": [ - "communication" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/ops/linear_op.py" - ] - }, - { - "commit": "abfb0e59f0b8ef7e84b5e903e8a28adaf7394684", - "subject": "[Feature] Add triton swiglustep kernel for SwigluStep activation (#11467)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_swiglustep.py", - "tests/ut/ops/a3_2/test_activation.py", - "vllm_ascend/ops/activation.py", - "vllm_ascend/ops/triton/activation/swiglustep.py" - ] - }, - { - "commit": "6cd692a8038cc0e8e8c2cd285244b46893f0c715", - "subject": "[Feature][SFA] Support DCP with replicate-indexer for SFA (#11443)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/workflows/configs/nightly_config.yaml", - "csrc/attention/lightning_indexer/op_host/lightning_indexer_tiling.cpp", - "csrc/attention/sparse_flash_attention/op_host/sparse_flash_attention_tiling.cpp", - "csrc/attention/sparse_flash_attention/sparse_flash_attention_torch_adpt.h", - "csrc/torch_binding_meta.cpp", - "tests/e2e/nightly/single_node/models/configs/DeepSeek-V3.2-W8A8-DCP.yaml", - "tests/e2e/pull_request/four_card/context_parallel/test_accuracy.py", - "tests/ut/attention/a2/test_sfa_v1.py", - "tests/ut/attention/test_sfa_cp.py", - "tests/ut/worker/a2/test_model_runner_v1.py", - "vllm_ascend/ascend_config.py", - "vllm_ascend/attention/context_parallel/sfa_cp.py", - "vllm_ascend/attention/sfa_v1.py", - "vllm_ascend/core/kv_cache_interface.py", - "vllm_ascend/platform.py", - "vllm_ascend/spec_decode/llm_base_proposer.py", - "vllm_ascend/utils.py", - "vllm_ascend/worker/model_runner_v1.py", - "vllm_ascend/worker/pcp_utils.py" - ] - }, - { - "commit": "4ad76e0e92118707d3586459f4b16ac82bb34e6d", - "subject": "[BugFix][Prefix Caching][DSv4] Align SlidingWindowManager scheduler block size with LCM block size (#11383)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/patch/platform/test_prefix_cache_cp_patches.py", - "vllm_ascend/patch/platform/patch_kv_cache_coordinator.py" - ] - }, - { - "commit": "254121657e7640d183d39b5d724a24fb279c4001", - "subject": "[feature] Support Gemma4 ModelSlim quantization (#11575)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/quantization/test_modelslim_config.py", - "vllm_ascend/quantization/modelslim_config.py" - ] - }, - { - "commit": "cc348a087aaee1e5bccf0c3260f57336984f30c8", - "subject": "[Feature] Adapt FlashComm1 SP for Step3.5 (#9597)", - "topics": [ - "communication" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/ops/test_linear.py", - "vllm_ascend/ops/linear_op.py" - ] - }, - { - "commit": "eb0ce34e98ea66d83482d1573283fb98afa93b83", - "subject": "[BugFix][FusedMoE] Use model activation instead of SwiGLU in quantized MoE MLP (in-branch variant) (#11732)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/ops/test_moe_mlp.py", - "vllm_ascend/ops/fused_moe/moe_mlp.py" - ] - }, - { - "commit": "8d808fdde1f704402a95662a3357bbecfe053fa6", - "subject": "[Ascend950][Bugfix]Fix allgatherEP MXFPW4A8 quantization (#11663)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/quantization/methods/w4a8_mxfp4.py" - ] - }, - { - "commit": "7fe0efe90df2c07449cd09d8e7896b9260a49bc7", - "subject": "[Feature][Ops] Add kv quant sparse flash attention op (#11626)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "csrc/attention/kv_quant_sparse_flash_attention/CMakeLists.txt", - "csrc/attention/kv_quant_sparse_flash_attention/README.md", - "csrc/attention/kv_quant_sparse_flash_attention/kv_quant_sparse_flash_attention_torch_adpt.h", - "csrc/attention/kv_quant_sparse_flash_attention/op_host/CMakeLists.txt", - "csrc/attention/kv_quant_sparse_flash_attention/op_host/kv_quant_sparse_flash_attention_def.cpp", - "csrc/attention/kv_quant_sparse_flash_attention/op_host/kv_quant_sparse_flash_attention_infershape.cpp", - "csrc/attention/kv_quant_sparse_flash_attention/op_host/kv_quant_sparse_flash_attention_tiling.cpp", - "csrc/attention/kv_quant_sparse_flash_attention/op_host/kv_quant_sparse_flash_attention_tiling.h", - "csrc/attention/kv_quant_sparse_flash_attention/op_kernel/arch35/kv_quant_sparse_flash_attention_common_arch35.h", - "csrc/attention/kv_quant_sparse_flash_attention/op_kernel/arch35/kv_quant_sparse_flash_attention_kernel_mla.h", - "csrc/attention/kv_quant_sparse_flash_attention/op_kernel/arch35/kv_quant_sparse_flash_attention_kvcache.h", - "csrc/attention/kv_quant_sparse_flash_attention/op_kernel/arch35/kv_quant_sparse_flash_attention_service_cube_mla.h", - "csrc/attention/kv_quant_sparse_flash_attention/op_kernel/arch35/kv_quant_sparse_flash_attention_service_vector_mla.h", - "csrc/attention/kv_quant_sparse_flash_attention/op_kernel/kv_quant_sparse_flash_attention.cpp", - "csrc/attention/kv_quant_sparse_flash_attention/op_kernel/kv_quant_sparse_flash_attention_common.h", - "csrc/attention/kv_quant_sparse_flash_attention/op_kernel/kv_quant_sparse_flash_attention_kernel_mla.h", - "csrc/attention/kv_quant_sparse_flash_attention/op_kernel/kv_quant_sparse_flash_attention_service_cube_mla.h", - "csrc/attention/kv_quant_sparse_flash_attention/op_kernel/kv_quant_sparse_flash_attention_service_vector_mla.h", - "csrc/attention/kv_quant_sparse_flash_attention/op_kernel/kv_quant_sparse_flash_attention_template_tiling_key.h", - "csrc/build_aclnn.sh", - "csrc/torch_binding.cpp", - "csrc/torch_binding_meta.cpp", - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_kv_quant_sparse_flash_attention.py" - ] - }, - { - "commit": "53f60bfebd52edd9aff0c6b535d59d218b7d8976", - "subject": "[Feature] Add mxfp quant for token dispatch to fit low acc communication with MXFP4 fixed (#11614)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/ops/fused_moe/moe_stage_params.py", - "vllm_ascend/ops/fused_moe/token_dispatcher.py" - ] - }, - { - "commit": "1a13272660ecb5eece7b1749a782f59ec1d9117d", - "subject": "[Feature][MRV2] Add qwen dspark support for mrv2 (#11602)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/misc/model_dataset_list.json", - "tests/e2e/pull_request/one_card/model_runner_v2/test_basic.py", - "tests/e2e/pull_request/one_card/model_runner_v2/utils.py", - "vllm_ascend/patch/worker/patch_qwen3_dflash.py", - "vllm_ascend/patch/worker/patch_v2/patch_triton.py", - "vllm_ascend/worker/v2/spec_decode/__init__.py", - "vllm_ascend/worker/v2/spec_decode/dflash/speculator.py", - "vllm_ascend/worker/v2/spec_decode/dspark/__init__.py", - "vllm_ascend/worker/v2/spec_decode/dspark/speculator.py" - ] - }, - { - "commit": "44613d173ccc63648d4954ca5ba416401b647ee7", - "subject": "[CI] Fix po translate bug (#11688)", - "topics": [ - "deepseek-v4-dspark", - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/pr_test.yaml", - ".github/workflows/scripts/detect_po_changes.py", - ".github/workflows/scripts/po_translate.py", - "docs/source/locale/zh_CN/LC_MESSAGES/community/contributors.po", - "docs/source/locale/zh_CN/LC_MESSAGES/community/issue-workflow-guidelines.po", - "docs/source/locale/zh_CN/LC_MESSAGES/community/slash-commands.po", - "docs/source/locale/zh_CN/LC_MESSAGES/community/versioning_policy.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/Design_Documents/cpu_binding.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/Design_Documents/disaggregated_prefill.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/Design_Documents/dynamic_chunked_pipeline_parallel.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/Design_Documents/npugraph_ex.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/Design_Documents/quantization.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/contribution/doc_writing.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/contribution/e2e_ci_test.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/contribution/nightly_ci_test.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/evaluation/index.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/performance_and_debug/index.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/performance_and_debug/msprobe_guide.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/performance_and_debug/performance_benchmark.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/performance_and_debug/service_profiling_guide.po", - "docs/source/locale/zh_CN/LC_MESSAGES/faqs.po", - "docs/source/locale/zh_CN/LC_MESSAGES/installation.po", - "docs/source/locale/zh_CN/LC_MESSAGES/quick_start.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/features/dynamic_chunked_pipeline_parallel.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/features/long_sequence_context_parallel_multi_node.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/features/long_sequence_context_parallel_single_node.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/features/pd_colocated_mooncake_multi_instance.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/features/pd_disaggregation_mooncake_multi_node.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/features/pd_disaggregation_mooncake_single_node.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/features/suffix_speculative_decoding.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/hardwares/310p.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/DeepSeek-R1.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/DeepSeek-V3.1.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/DeepSeek-V4-Flash.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/DeepSeek-V4-Pro.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/DeepSeekOCR2.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/GLM4.x.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/GLM5.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Hunyuan-A13B-Instruct.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Hy3-preview.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Kimi-K2-Thinking.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Kimi-K2.5.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Kimi-K2.6.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/LLaVA-OneVision-Qwen2-0.5B-OV.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/MiniMax-M2.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Minitron-8B-Base.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/PaddleOCR-VL.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen-VL-Dense.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-235B-A22B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-30B-A3B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-Coder-30B-A3B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-Dense.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-Next.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-Omni-30B-A3B-Thinking.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-VL-235B-A22B-Instruct.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-VL-30B-A3B-Instruct.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3.5-27B-Qwen3.6-27B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3.5-397B-A17B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3.6-35B-A3B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/gpt-oss-120b.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/configuration/additional_config.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/deployment_guide/index.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/Ai_QoS_introduction_en.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/Fine_grained_TP.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/context_parallel.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/cpu_binding.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/dynamic_chunk_pipeline_parallel.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/expert_parallelism_load_balancer.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/flash_attention.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/graph_mode.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/kv_pool.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/large_scale_ep.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/lora.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/netloader.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/rfork.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/sequence_parallelism.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/speculative_decoding.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/ucm_deployment.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/release_notes.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/support_matrix/feature_matrix.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/support_matrix/supported_features.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/support_matrix/supported_models.po" - ] - }, - { - "commit": "8cecff7548c75d549be584294059cf87297776ac", - "subject": "[Feature] Support Gemma4 graph execution on A2/A3 (#11536)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/attention/a2/test_attention_v1.py", - "vllm_ascend/attention/attention_v1.py", - "vllm_ascend/attention/utils.py", - "vllm_ascend/ops/triton/rope.py" - ] - }, - { - "commit": "1eb84a227715bbba4830bc3f1126cbc1b61fa9ae", - "subject": "[Doc] Fix CN doc problem (#11664)", - "topics": [ - "deepseek-v4-dspark", - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/scripts/detect_po_changes.py", - "docs/source/locale/zh_CN/LC_MESSAGES/community/slash-commands.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/Design_Documents/ModelRunner_prepare_inputs.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/Design_Documents/disaggregated_prefill.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/contribution/doc_writing.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/contribution/multi_node_test.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/contribution/testing.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/evaluation/using_evalscope.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/evaluation/using_lm_eval.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/performance_and_debug/msprobe_guide.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/performance_and_debug/optimization_and_tuning.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/performance_and_debug/service_profiling_guide.po", - "docs/source/locale/zh_CN/LC_MESSAGES/installation.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/features/dynamic_chunked_pipeline_parallel.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/features/long_sequence_context_parallel_multi_node.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/features/long_sequence_context_parallel_single_node.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/features/pd_disaggregation_mooncake_multi_node.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/hardwares/310p.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/DeepSeek-R1.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/DeepSeek-V3.1.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/DeepSeek-V3.2.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/DeepSeek-V4-Flash.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/DeepSeek-V4-Pro.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/DeepSeekOCR2.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/GLM4.x.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/GLM5.2.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/GLM5.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Hunyuan-A13B-Instruct.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Hy3-preview.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/InternVL3.5.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Kimi-K2-Thinking.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Kimi-K2.5.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Kimi-K2.6.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/LLaVA-OneVision-Qwen2-0.5B-OV.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/MiniMax-M2.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Mixtral-8x7B-Instruct-v0.1.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/PaddleOCR-VL.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen-VL-Dense.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-235B-A22B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-30B-A3B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-Coder-30B-A3B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-Dense.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-Next.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-Omni-30B-A3B-Thinking.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-VL-235B-A22B-Instruct.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-VL-30B-A3B-Instruct.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-VL-Reranker.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3.5-27B-Qwen3.6-27B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3_reranker.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/gpt-oss-120b.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/deployment_guide/using_volcano_kthena.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/Fine_grained_TP.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/cpu_binding.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/dynamic_chunk_pipeline_parallel.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/epd_disaggregation.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/graph_mode.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/kv_pool.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/large_scale_ep.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/layer_sharding.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/lora.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/netloader.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/quantization.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/rfork.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/speculative_decoding.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/ucm_deployment.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/release_notes.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/support_matrix/supported_models.po", - "docs/source/tutorials/models/Qwen3.5-27B-Qwen3.6-27B.md", - "tools/generate_zh_docs.py" - ] - }, - { - "commit": "9872bb996f0d7d459ff79d6503f582931b817425", - "subject": "[BugFix] Avoid allocating KV cache for skipped indexers (#11065)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/worker/a2/test_model_runner_v1.py", - "vllm_ascend/core/kv_cache_interface.py", - "vllm_ascend/utils.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "ceb8e180d6c4ba50c3dec5cd51ee9b8724d12e7e", - "subject": "[BugFix][EPLB] Split unquant fused MC2 weights (#11282)", - "topics": [ - "communication", - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/eplb/adaptor/test_vllm_adaptor.py", - "tests/ut/ops/test_fused_moe.py", - "tests/ut/ops/test_moe_mlp.py", - "vllm_ascend/eplb/adaptor/vllm_adaptor.py", - "vllm_ascend/ops/fused_moe/fused_moe.py" - ] - }, - { - "commit": "176bbad6dba7f66058be5aa002b54887ca29a871", - "subject": "[Feature][LoRA][MoE] Support LoRA with unquantified MoE models (#10977)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/workflows/scripts/test_config.yaml", - "docs/source/user_guide/feature_guide/lora.md", - "tests/e2e/conftest.py", - "tests/e2e/pull_request/one_card/lora/test_olmoe_lora.py", - "tests/e2e/pull_request/two_card/lora/test_qwen3moe_lora_tp.py", - "vllm_ascend/lora/fused_moe.py", - "vllm_ascend/lora/punica_npu.py", - "vllm_ascend/lora/utils.py", - "vllm_ascend/ops/fused_moe/fused_moe.py", - "vllm_ascend/ops/fused_moe/fused_moe_0_23_0.py", - "vllm_ascend/ops/fused_moe/moe_mlp.py", - "vllm_ascend/ops/fused_moe/moe_runtime_args.py", - "vllm_ascend/ops/fused_moe/moe_stage_contracts.py" - ] - }, - { - "commit": "9e0c6ba4af2e1cc892ba43bf5cdc3e4fca3046aa", - "subject": "[Doc] Translated Doc files 2026-07-08 (#11655)", - "topics": [ - "deepseek-v4-dspark", - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/locale/zh_CN/LC_MESSAGES/community/slash-commands.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/Design_Documents/ModelRunner_prepare_inputs.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/Design_Documents/cpu_binding.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/Design_Documents/quantization.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/contribution/doc_writing.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/contribution/testing.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/evaluation/using_lm_eval.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/performance_and_debug/optimization_and_tuning.po", - "docs/source/locale/zh_CN/LC_MESSAGES/faqs.po", - "docs/source/locale/zh_CN/LC_MESSAGES/installation.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/features/dynamic_chunked_pipeline_parallel.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/features/long_sequence_context_parallel_multi_node.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/features/long_sequence_context_parallel_single_node.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/features/pd_disaggregation_mooncake_multi_node.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/hardwares/310p.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/DeepSeek-R1.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/DeepSeek-V3.1.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/DeepSeek-V3.2.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/DeepSeek-V4-Flash.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/DeepSeek-V4-Pro.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/DeepSeekOCR2.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/GLM4.x.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/GLM5.2.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/GLM5.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Hunyuan-A13B-Instruct.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Hy3-preview.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/InternVL3.5.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Kimi-K2-Thinking.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Kimi-K2.5.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Kimi-K2.6.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/LLaVA-OneVision-Qwen2-0.5B-OV.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Mixtral-8x7B-Instruct-v0.1.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/PaddleOCR-VL.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-235B-A22B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-30B-A3B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-Coder-30B-A3B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-Dense.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-Next.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-Omni-30B-A3B-Thinking.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-VL-235B-A22B-Instruct.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-VL-Reranker.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3.5-27B-Qwen3.6-27B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3_reranker.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/deployment_guide/using_volcano_kthena.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/cpu_binding.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/dynamic_chunk_pipeline_parallel.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/flash_attention.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/graph_mode.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/kv_pool.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/quantization.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/ucm_deployment.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/support_matrix/supported_models.po" - ] - }, - { - "commit": "59e6f47fbe80e946ea2a247b4b98184af11f4c22", - "subject": "[DOC] Fix wrong CN translate (#11652)", - "topics": [ - "deepseek-v4-dspark", - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/scripts/detect_po_changes.py", - ".github/workflows/scripts/po_translate.py", - "docs/source/locale/zh_CN/LC_MESSAGES/community/slash-commands.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/Design_Documents/ModelRunner_prepare_inputs.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/Design_Documents/cpu_binding.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/Design_Documents/quantization.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/contribution/doc_writing.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/contribution/testing.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/evaluation/index.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/evaluation/using_evalscope.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/evaluation/using_lm_eval.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/performance_and_debug/index.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/performance_and_debug/msprobe_guide.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/performance_and_debug/optimization_and_tuning.po", - "docs/source/locale/zh_CN/LC_MESSAGES/faqs.po", - "docs/source/locale/zh_CN/LC_MESSAGES/installation.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/features/dynamic_chunked_pipeline_parallel.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/features/long_sequence_context_parallel_multi_node.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/features/long_sequence_context_parallel_single_node.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/features/pd_disaggregation_mooncake_multi_node.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/hardwares/310p.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/DeepSeek-R1.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/DeepSeek-V3.1.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/DeepSeek-V3.2.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/DeepSeek-V4-Flash.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/DeepSeek-V4-Pro.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/DeepSeekOCR2.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/GLM4.x.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/GLM5.2.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/GLM5.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Hunyuan-A13B-Instruct.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Hy3-preview.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/InternVL3.5.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Kimi-K2-Thinking.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Kimi-K2.5.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Kimi-K2.6.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/LLaVA-OneVision-Qwen2-0.5B-OV.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/MiniMax-M2.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Mixtral-8x7B-Instruct-v0.1.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/PaddleOCR-VL.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen-VL-Dense.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-235B-A22B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-30B-A3B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-Coder-30B-A3B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-Dense.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-Next.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-Omni-30B-A3B-Thinking.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-VL-235B-A22B-Instruct.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-VL-30B-A3B-Instruct.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-VL-Reranker.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3.5-27B-Qwen3.6-27B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3_reranker.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/gpt-oss-120b.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/deployment_guide/index.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/deployment_guide/using_volcano_kthena.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/cpu_binding.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/dynamic_chunk_pipeline_parallel.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/flash_attention.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/graph_mode.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/kv_pool.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/large_scale_ep.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/layer_sharding.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/quantization.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/speculative_decoding.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/ucm_deployment.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/release_notes.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/support_matrix/supported_models.po" - ] - }, - { - "commit": "32b086054694fcae13123ecc76a65825746e3565", - "subject": "[Doc] Translated Doc files 2026-07-08 (#11648)", - "topics": [ - "deepseek-v4-dspark", - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/locale/zh_CN/LC_MESSAGES/community/contributors.po", - "docs/source/locale/zh_CN/LC_MESSAGES/community/governance.po", - "docs/source/locale/zh_CN/LC_MESSAGES/community/index.po", - "docs/source/locale/zh_CN/LC_MESSAGES/community/issue-workflow-guidelines.po", - "docs/source/locale/zh_CN/LC_MESSAGES/community/slash-commands.po", - "docs/source/locale/zh_CN/LC_MESSAGES/community/user_stories/index.po", - "docs/source/locale/zh_CN/LC_MESSAGES/community/user_stories/llamafactory.po", - "docs/source/locale/zh_CN/LC_MESSAGES/community/versioning_policy.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/Design_Documents/ACL_Graph.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/Design_Documents/KV_Cache_Pool_Guide.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/Design_Documents/ModelRunner_prepare_inputs.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/Design_Documents/add_custom_aclnn_op.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/Design_Documents/context_parallel.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/Design_Documents/disaggregated_prefill.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/Design_Documents/dynamic_chunked_pipeline_parallel.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/Design_Documents/eplb_swift_balancer.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/Design_Documents/index.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/Design_Documents/npugraph_ex.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/Design_Documents/patch.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/contribution/doc_writing.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/contribution/e2e_ci_test.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/contribution/index.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/contribution/multi_node_test.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/contribution/nightly_ci_test.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/contribution/testing.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/evaluation/using_ais_bench.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/evaluation/using_evalscope.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/evaluation/using_lm_eval.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/evaluation/using_opencompass.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/index.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/performance_and_debug/optimization_and_tuning.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/performance_and_debug/performance_benchmark.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/performance_and_debug/service_profiling_guide.po", - "docs/source/locale/zh_CN/LC_MESSAGES/getting_started.po", - "docs/source/locale/zh_CN/LC_MESSAGES/index.po", - "docs/source/locale/zh_CN/LC_MESSAGES/installation.po", - "docs/source/locale/zh_CN/LC_MESSAGES/quick_start.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/features/dynamic_chunked_pipeline_parallel.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/features/index.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/features/long_sequence_context_parallel_multi_node.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/features/long_sequence_context_parallel_single_node.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/features/pd_colocated_mooncake_multi_instance.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/features/pd_disaggregation_mooncake_multi_node.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/features/pd_disaggregation_mooncake_single_node.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/features/ray.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/features/suffix_speculative_decoding.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/hardwares/310p.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/hardwares/index.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/DeepSeek-R1.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/DeepSeek-V3.1.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/DeepSeek-V3.2.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/DeepSeek-V4-Flash.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/DeepSeek-V4-Pro.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/DeepSeekOCR2.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/GLM4.x.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/GLM5.2.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/GLM5.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Hunyuan-A13B-Instruct.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Hy3-preview.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/InternVL3.5.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Kimi-K2.5.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Kimi-K2.6.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/LLaVA-OneVision-Qwen2-0.5B-OV.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Minitron-8B-Base.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Mixtral-8x7B-Instruct-v0.1.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/PaddleOCR-VL.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen-VL-Dense.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen2.5-Math-RM-72B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-235B-A22B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-30B-A3B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-ASR-1.7B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-Coder-30B-A3B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-Dense.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-Next.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-Omni-30B-A3B-Thinking.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-VL-235B-A22B-Instruct.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-VL-Embedding.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-VL-Reranker.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3.5-27B-Qwen3.6-27B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3.5-397B-A17B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3.6-35B-A3B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3_embedding.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3_reranker.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/gpt-oss-120b.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/index.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/configuration/additional_config.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/configuration/env_vars.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/configuration/index.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/deployment_guide/using_mindie_motor.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/deployment_guide/using_volcano_kthena.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/Ai_QoS_introduction_en.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/Fine_grained_TP.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/batch_invariance.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/context_parallel.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/cpu_binding.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/dynamic_batch.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/epd_disaggregation.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/expert_parallelism_load_balancer.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/external_dp.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/graph_mode.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/index.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/kv_cache_cpu_offload.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/kv_pool.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/large_scale_ep.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/layer_sharding.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/lmcache_ascend_deployment.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/lora.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/netloader.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/quantization.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/rfork.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/sequence_parallelism.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/sleep_mode.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/speculative_decoding.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/structured_output.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/ucm_deployment.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/weight_prefetch.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/index.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/support_matrix/feature_matrix.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/support_matrix/index.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/support_matrix/supported_features.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/support_matrix/supported_models.po" - ] - }, - { - "commit": "a8cfc1918b1b0d05961f51f0431213b277f5e8f9", - "subject": "[CI] update weekly cases: Deepseekv3.2-w8a8-A3.yaml GLM5.1-W8A8-a3-weekly.yaml Qwen3.5-397B-A17B-w8a8-mtp-A3-weekly.yaml (#11518)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "tests/e2e/weekly/multi_node/external_dp/config/GLM_5_1_PD_in32k_bs16-0.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/Kimi-K2.5-W4A8-16k-1k-TPOT50.yaml", - "tests/e2e/weekly/single_node/configs/DeepSeek-V3.2-W8A8_A3_weekly.yaml", - "tests/e2e/weekly/single_node/configs/GLM-5_1-W8A8_A3_weekly.yaml", - "tests/e2e/weekly/single_node/configs/Qwen3.5-397B-A17B-W8A8-mtp-A3_weekly.yaml" - ] - }, - { - "commit": "41b3a236481048318ddaec36dd56f4b8d4eba7b9", - "subject": "[Doc] Fix doc link url bug (#11606)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/scripts/detect_po_changes.py", - ".github/workflows/scripts/po_translate.py", - "docs/source/tutorials/models/DeepSeek-R1.md", - "docs/source/tutorials/models/DeepSeek-V3.1.md", - "docs/source/tutorials/models/DeepSeek-V4-Flash.md", - "docs/source/tutorials/models/DeepSeek-V4-Pro.md", - "docs/source/tutorials/models/DeepSeekOCR2.md", - "docs/source/tutorials/models/Kimi-K2-Thinking.md", - "docs/source/tutorials/models/Kimi-K2.5.md", - "docs/source/tutorials/models/Kimi-K2.6.md", - "docs/source/tutorials/models/MiniMax-M2.md", - "docs/source/tutorials/models/PaddleOCR-VL.md", - "docs/source/tutorials/models/Qwen-VL-Dense.md", - "docs/source/tutorials/models/Qwen3-235B-A22B.md", - "docs/source/tutorials/models/Qwen3-30B-A3B.md", - "docs/source/tutorials/models/Qwen3-Coder-30B-A3B.md", - "docs/source/tutorials/models/Qwen3-Dense.md", - "docs/source/tutorials/models/Qwen3-Next.md", - "docs/source/tutorials/models/Qwen3-Omni-30B-A3B-Thinking.md", - "docs/source/tutorials/models/Qwen3.5-27B-Qwen3.6-27B.md", - "mkdocs.yml" - ] - }, - { - "commit": "0d8b7ae678b511c5b5f16a046472fffac8796a51", - "subject": "[main][CI] update nightly VLLM_ENGINE_READY_TIMEOUT_S (#11530)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "tests/e2e/nightly/multi_node/external_dp/config/GLM5_1-W8A8-EP-external.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/DeepSeek-R1-W8A8-EPLB.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/DeepSeek-R1-W8A8-longseq.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/DeepSeek-V3_2-W8A8-A3-dual-nodes.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/DeepSeek-V3_2-W8A8-EP.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/GLM5_1-W8A8-EP.yaml", - "tests/e2e/nightly/single_node/models/configs/DeepSeek-V3.2-W8A8.yaml" - ] - }, - { - "commit": "e0d0502cc3d1fb521a9e77941e6c9b9f89dfb2a7", - "subject": "[BugFix] Revert all2all quantization (#11580)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/ops/fused_moe/moe_stage_params.py", - "vllm_ascend/ops/fused_moe/token_dispatcher.py" - ] - }, - { - "commit": "cfc277104c35938022eabf2582dbbf3de510f6b0", - "subject": "[Doc][Misc] Update DeepSeek, GLM, and Kimi model deployment guides (#11534)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/tutorials/models/DeepSeek-R1.md", - "docs/source/tutorials/models/DeepSeek-V3.1.md", - "docs/source/tutorials/models/GLM5.2.md", - "docs/source/tutorials/models/GLM5.md", - "docs/source/tutorials/models/Kimi-K2.5.md", - "docs/source/tutorials/models/Kimi-K2.6.md" - ] - }, - { - "commit": "18aabe4d362a8aa6ea86defc3626878ee7b568d7", - "subject": "[Doc] Update cn doc content (#11549)", - "topics": [ - "deepseek-v4-dspark", - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/schedule_doc_translate.yaml", - ".github/workflows/scripts/detect_po_changes.py", - ".github/workflows/scripts/po_translate.py", - "docs/hooks/macros_main.py", - "docs/hooks/nav_titles.py", - "docs/source/conf.py", - "docs/source/locale/zh_CN/LC_MESSAGES/community/contributors.po", - "docs/source/locale/zh_CN/LC_MESSAGES/community/governance.po", - "docs/source/locale/zh_CN/LC_MESSAGES/community/index.po", - "docs/source/locale/zh_CN/LC_MESSAGES/community/issue-workflow-guidelines.po", - "docs/source/locale/zh_CN/LC_MESSAGES/community/slash-commands.po", - "docs/source/locale/zh_CN/LC_MESSAGES/community/user_stories/index.po", - "docs/source/locale/zh_CN/LC_MESSAGES/community/user_stories/llamafactory.po", - "docs/source/locale/zh_CN/LC_MESSAGES/community/versioning_policy.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/Design_Documents/ACL_Graph.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/Design_Documents/KV_Cache_Pool_Guide.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/Design_Documents/ModelRunner_prepare_inputs.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/Design_Documents/add_custom_aclnn_op.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/Design_Documents/context_parallel.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/Design_Documents/cpu_binding.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/Design_Documents/disaggregated_prefill.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/Design_Documents/dynamic_chunked_pipeline_parallel.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/Design_Documents/eplb_swift_balancer.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/Design_Documents/index.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/Design_Documents/npugraph_ex.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/Design_Documents/patch.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/Design_Documents/quantization.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/contribution/doc_writing.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/contribution/e2e_ci_test.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/contribution/index.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/contribution/multi_node_test.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/contribution/nightly_ci_test.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/contribution/testing.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/evaluation/index.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/evaluation/using_ais_bench.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/evaluation/using_evalscope.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/evaluation/using_lm_eval.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/evaluation/using_opencompass.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/index.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/performance_and_debug/index.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/performance_and_debug/msprobe_guide.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/performance_and_debug/optimization_and_tuning.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/performance_and_debug/performance_benchmark.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/performance_and_debug/service_profiling_guide.po", - "docs/source/locale/zh_CN/LC_MESSAGES/faqs.po", - "docs/source/locale/zh_CN/LC_MESSAGES/getting_started.po", - "docs/source/locale/zh_CN/LC_MESSAGES/index.po", - "docs/source/locale/zh_CN/LC_MESSAGES/installation.po", - "docs/source/locale/zh_CN/LC_MESSAGES/quick_start.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/features/dynamic_chunked_pipeline_parallel.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/features/index.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/features/long_sequence_context_parallel_multi_node.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/features/long_sequence_context_parallel_single_node.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/features/pd_colocated_mooncake_multi_instance.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/features/pd_disaggregation_mooncake_multi_node.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/features/pd_disaggregation_mooncake_single_node.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/features/ray.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/features/suffix_speculative_decoding.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/hardwares/310p.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/hardwares/index.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/DeepSeek-R1.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/DeepSeek-V3.1.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/DeepSeek-V3.2.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/DeepSeek-V4-Flash.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/DeepSeek-V4-Pro.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/DeepSeekOCR2.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/GLM4.x.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/GLM5.2.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/GLM5.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Hunyuan-A13B-Instruct.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Hy3-preview.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/InternVL3.5.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Kimi-K2-Thinking.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Kimi-K2.5.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Kimi-K2.6.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/LLaVA-OneVision-Qwen2-0.5B-OV.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/MiniMax-M2.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Minitron-8B-Base.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Mixtral-8x7B-Instruct-v0.1.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/PaddleOCR-VL.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen-VL-Dense.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen2.5-Math-RM-72B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-235B-A22B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-30B-A3B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-ASR-1.7B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-Coder-30B-A3B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-Dense.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-Next.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-Omni-30B-A3B-Thinking.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-VL-235B-A22B-Instruct.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-VL-30B-A3B-Instruct.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-VL-Embedding.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-VL-Reranker.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3.5-27B-Qwen3.6-27B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3.5-397B-A17B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3.6-35B-A3B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3_embedding.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3_reranker.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/gpt-oss-120b.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/index.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/configuration/additional_config.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/configuration/env_vars.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/configuration/index.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/deployment_guide/index.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/deployment_guide/using_mindie_motor.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/deployment_guide/using_volcano_kthena.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/Ai_QoS_introduction_en.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/Fine_grained_TP.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/batch_invariance.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/context_parallel.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/cpu_binding.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/dynamic_batch.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/dynamic_chunk_pipeline_parallel.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/epd_disaggregation.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/expert_parallelism_load_balancer.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/external_dp.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/flash_attention.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/graph_mode.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/index.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/kv_cache_cpu_offload.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/kv_pool.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/large_scale_ep.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/layer_sharding.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/lmcache_ascend_deployment.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/lora.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/netloader.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/quantization.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/rfork.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/sequence_parallelism.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/sleep_mode.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/speculative_decoding.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/structured_output.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/ucm_deployment.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/weight_prefetch.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/index.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/release_notes.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/support_matrix/feature_matrix.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/support_matrix/index.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/support_matrix/supported_features.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/support_matrix/supported_models.po", - "mkdocs.yml", - "tools/generate_zh_docs.py", - "tools/sphinx-lint.sh", - "typos.toml" - ] - }, - { - "commit": "d48e23d39e29f1922c67c1681765da3cb6687ccf", - "subject": "[Doc] Fix DeepSeek-V4 documentation for Docker deployment (#11554)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/tutorials/models/DeepSeek-V3.1.md", - "docs/source/tutorials/models/DeepSeek-V4-Flash.md", - "docs/source/tutorials/models/DeepSeek-V4-Pro.md", - "docs/source/tutorials/models/DeepSeekOCR2.md", - "docs/source/tutorials/models/Kimi-K2-Thinking.md", - "docs/source/tutorials/models/Kimi-K2.5.md", - "docs/source/tutorials/models/Kimi-K2.6.md", - "docs/source/tutorials/models/MiniMax-M2.md", - "docs/source/tutorials/models/PaddleOCR-VL.md", - "docs/source/tutorials/models/Qwen-VL-Dense.md", - "docs/source/tutorials/models/Qwen3-235B-A22B.md", - "docs/source/tutorials/models/Qwen3-30B-A3B.md", - "docs/source/tutorials/models/Qwen3-Coder-30B-A3B.md", - "docs/source/tutorials/models/Qwen3-Dense.md", - "docs/source/tutorials/models/Qwen3-Next.md", - "docs/source/tutorials/models/Qwen3-Omni-30B-A3B-Thinking.md", - "docs/source/tutorials/models/Qwen3.5-27B-Qwen3.6-27B.md" - ] - }, - { - "commit": "9e7fad123ac0d79527096515d2f7d239262a50d8", - "subject": "[Doc] The usability of the DeepSeek-V4 documentation is improved (#11525)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/tutorials/models/DeepSeek-V4-Flash.md", - "docs/source/tutorials/models/DeepSeek-V4-Pro.md" - ] - }, - { - "commit": "fbbf5167ada3f3b59909bcf947e69036fc4af425", - "subject": "[Doc] migrate doc from sphinx to mkdocs (#11252)", - "topics": [ - "deepseek-v4-dspark", - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - ".agents/skills/vllm-ascend-release/SKILL.md", - ".agents/skills/vllm-ascend-release/references/version-files.yaml", - ".agents/skills/vllm-ascend-release/templates/release-checklist-template.md", - ".github/ISSUE_TEMPLATE/900-release-checklist.yml", - ".github/workflows/labeled_doctest.yaml", - ".github/workflows/schedule_doc_linkcheck.yaml", - ".github/workflows/scripts/check_md_links.py", - ".gitignore", - ".readthedocs.yaml", - "docs/Makefile", - "docs/README.md", - "docs/hooks/model_code.py", - "docs/hooks/nav_titles.py", - "docs/overrides/main.html", - "docs/requirements-docs.txt", - "docs/source/community/index.md", - "docs/source/community/user_stories/index.md", - "docs/source/community/versioning_policy.md", - "docs/source/conf.py", - "docs/source/developer_guide/Design_Documents/index.md", - "docs/source/developer_guide/Design_Documents/npugraph_ex.md", - "docs/source/developer_guide/contribution/doc_writing.md", - "docs/source/developer_guide/contribution/e2e_ci_test.md", - "docs/source/developer_guide/contribution/index.md", - "docs/source/developer_guide/contribution/multi_node_test.md", - "docs/source/developer_guide/contribution/nightly_ci_test.md", - "docs/source/developer_guide/contribution/testing.md", - "docs/source/developer_guide/evaluation/index.md", - "docs/source/developer_guide/evaluation/using_ais_bench.md", - "docs/source/developer_guide/evaluation/using_evalscope.md", - "docs/source/developer_guide/evaluation/using_lm_eval.md", - "docs/source/developer_guide/evaluation/using_opencompass.md", - "docs/source/developer_guide/index.md", - "docs/source/developer_guide/performance_and_debug/index.md", - "docs/source/developer_guide/performance_and_debug/optimization_and_tuning.md", - "docs/source/developer_guide/performance_and_debug/performance_benchmark.md", - "docs/source/faqs.md", - "docs/source/getting_started.md", - "docs/source/index.md", - "docs/source/installation.md", - "docs/source/locale/zh_CN/LC_MESSAGES/community/versioning_policy.po", - "docs/source/locale/zh_CN/LC_MESSAGES/index.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Kimi-K2-Thinking.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-30B-A3B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-Coder-30B-A3B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3.5-27B-Qwen3.6-27B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/layerwise_kv_pool.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/sequence_parallelism.po", - "docs/source/quick_start.md", - "docs/source/stylesheets/extra.css", - "docs/source/tutorials/features/dynamic_chunked_pipeline_parallel.md", - "docs/source/tutorials/features/index.md", - "docs/source/tutorials/features/long_sequence_context_parallel_multi_node.md", - "docs/source/tutorials/features/long_sequence_context_parallel_single_node.md", - "docs/source/tutorials/features/pd_colocated_mooncake_multi_instance.md", - "docs/source/tutorials/features/pd_disaggregation_mooncake_multi_node.md", - "docs/source/tutorials/features/pd_disaggregation_mooncake_single_node.md", - "docs/source/tutorials/features/ray.md", - "docs/source/tutorials/features/suffix_speculative_decoding.md", - "docs/source/tutorials/hardwares/310p.md", - "docs/source/tutorials/hardwares/index.md", - "docs/source/tutorials/models/DeepSeek-R1.md", - "docs/source/tutorials/models/DeepSeek-V3.1.md", - "docs/source/tutorials/models/DeepSeek-V3.2.md", - "docs/source/tutorials/models/DeepSeek-V4-Flash.md", - "docs/source/tutorials/models/DeepSeek-V4-Pro.md", - "docs/source/tutorials/models/DeepSeekOCR2.md", - "docs/source/tutorials/models/GLM4.x.md", - "docs/source/tutorials/models/GLM5.2.md", - "docs/source/tutorials/models/GLM5.md", - "docs/source/tutorials/models/Hunyuan-A13B-Instruct.md", - "docs/source/tutorials/models/Hy3-preview.md", - "docs/source/tutorials/models/InternVL3.5.md", - "docs/source/tutorials/models/Kimi-K2-Thinking.md", - "docs/source/tutorials/models/Kimi-K2.5.md", - "docs/source/tutorials/models/Kimi-K2.6.md", - "docs/source/tutorials/models/LLaVA-OneVision-Qwen2-0.5B-OV.md", - "docs/source/tutorials/models/MiniMax-M2.md", - "docs/source/tutorials/models/Minitron-8B-Base.md", - "docs/source/tutorials/models/Mixtral-8x7B-Instruct-v0.1.md", - "docs/source/tutorials/models/PaddleOCR-VL.md", - "docs/source/tutorials/models/Qwen-VL-Dense.md", - "docs/source/tutorials/models/Qwen2.5-Math-RM-72B.md", - "docs/source/tutorials/models/Qwen3-235B-A22B.md", - "docs/source/tutorials/models/Qwen3-30B-A3B.md", - "docs/source/tutorials/models/Qwen3-ASR-1.7B.md", - "docs/source/tutorials/models/Qwen3-Coder-30B-A3B.md", - "docs/source/tutorials/models/Qwen3-Dense.md", - "docs/source/tutorials/models/Qwen3-Next.md", - "docs/source/tutorials/models/Qwen3-Omni-30B-A3B-Thinking.md", - "docs/source/tutorials/models/Qwen3-VL-235B-A22B-Instruct.md", - "docs/source/tutorials/models/Qwen3-VL-30B-A3B-Instruct.md", - "docs/source/tutorials/models/Qwen3-VL-Embedding.md", - "docs/source/tutorials/models/Qwen3.5-27B-Qwen3.6-27B.md", - "docs/source/tutorials/models/Qwen3.5-397B-A17B.md", - "docs/source/tutorials/models/Qwen3.6-35B-A3B.md", - "docs/source/tutorials/models/Qwen3_embedding.md", - "docs/source/tutorials/models/gpt-oss-120b.md", - "docs/source/tutorials/models/index.md", - "docs/source/user_guide/configuration/env_vars.md", - "docs/source/user_guide/configuration/index.md", - "docs/source/user_guide/deployment_guide/index.md", - "docs/source/user_guide/feature_guide/batch_invariance.md", - "docs/source/user_guide/feature_guide/dynamic_chunk_pipeline_parallel.md", - "docs/source/user_guide/feature_guide/flash_attention.md", - "docs/source/user_guide/feature_guide/graph_mode.md", - "docs/source/user_guide/feature_guide/index.md", - "docs/source/user_guide/feature_guide/large_scale_ep.md", - "docs/source/user_guide/feature_guide/rfork.md", - "docs/source/user_guide/feature_guide/sleep_mode.md", - "docs/source/user_guide/feature_guide/speculative_decoding.md", - "docs/source/user_guide/image-1.png", - "docs/source/user_guide/image.png", - "docs/source/user_guide/index.md", - "docs/source/user_guide/support_matrix/index.md", - "main.py", - "mkdocs.yml", - "tools/check_forbidden_imports.py", - "tools/generate_zh_docs.py", - "tools/rtd_build.sh", - "tools/set_release_flag.py" - ] - }, - { - "commit": "8116bcdc4528759e832a5dbd4d4cc7cee1786ca3", - "subject": "[BugFix][KV Pool] Fix MooncakeBackend lazy init failure on cross-thread store setup (#11459)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/distributed/kv_transfer/kv_pool/ascend_store/backend/mooncake_backend.py" - ] - }, - { - "commit": "f395a0b5cea6c5adaa72e9c93de37cfd754bc89d", - "subject": "[BugFix][P/D] Restrict recompute scheduler to decode nodes (#11265)", - "topics": [ - "deepseek-v4-dspark", - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/DeepSeek-V3.1.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Kimi-K2.5.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Kimi-K2.6.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/configuration/additional_config.po", - "docs/source/tutorials/features/long_sequence_context_parallel_multi_node.md", - "docs/source/tutorials/features/pd_disaggregation_mooncake_multi_node.md", - "docs/source/tutorials/models/DeepSeek-V3.1.md", - "docs/source/tutorials/models/DeepSeek-V4-Flash.md", - "docs/source/tutorials/models/DeepSeek-V4-Pro.md", - "docs/source/tutorials/models/GLM4.x.md", - "docs/source/tutorials/models/GLM5.2.md", - "docs/source/tutorials/models/GLM5.md", - "docs/source/tutorials/models/Kimi-K2.5.md", - "docs/source/tutorials/models/Kimi-K2.6.md", - "docs/source/tutorials/models/Qwen3.5-27B-Qwen3.6-27B.md", - "docs/source/tutorials/models/Qwen3.5-397B-A17B.md", - "docs/source/user_guide/configuration/additional_config.md", - "tests/e2e/nightly/multi_node/external_dp/config/GLM5_1-W8A8-EP-external.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/GLM5_1-W8A8-EP.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/GLM_5_1_PD_in32k_bs16-0.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/GLM_5_1_PD_in32k_bs20-90.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/GLM_5_1_PD_in64k_bs16-90.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/MiniMax-PD-in32k-bs4-1.yaml", - "tests/ut/test_platform.py", - "vllm_ascend/ascend_config.py", - "vllm_ascend/platform.py" - ] - }, - { - "commit": "cd76505e8afa6dc428947b81f734b1525ffc6b2e", - "subject": "[CI] main2main 0703 commit: 1f486d9 (#10454)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/vllm-main-verified.commit", - "tests/ut/distributed/weight_transfer/test_npu_ipc_engine.py", - "vllm_ascend/_310p/fused_moe/fused_moe.py", - "vllm_ascend/distributed/device_communicators/npu_communicator.py", - "vllm_ascend/distributed/weight_transfer/hccl_engine.py", - "vllm_ascend/distributed/weight_transfer/npu_ipc_engine.py", - "vllm_ascend/ops/fused_moe/fused_moe.py", - "vllm_ascend/patch/platform/patch_torch_accelerator.py", - "vllm_ascend/patch/worker/patch_deepseek_v2.py", - "vllm_ascend/patch/worker/patch_qwen3_dflash.py", - "vllm_ascend/worker/v2/model_runner.py", - "vllm_ascend/worker/v2/spec_decode/rejection_sampler_utils.py", - "vllm_ascend/worker/worker.py" - ] - }, - { - "commit": "89197e4a19e0f299585f1c71d24630b914fa0668", - "subject": "[Doc][Misc] Update and standardize model deployment tutorials for DeepSeek, GLM, and Kimi (#11335)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/tutorials/models/DeepSeek-R1.md", - "docs/source/tutorials/models/DeepSeek-V3.1.md", - "docs/source/tutorials/models/GLM5.2.md", - "docs/source/tutorials/models/GLM5.md", - "docs/source/tutorials/models/Kimi-K2.5.md", - "docs/source/tutorials/models/Kimi-K2.6.md" - ] - }, - { - "commit": "8f12262c0a682231140f984a82581cc5efda1fdf", - "subject": "[Feature][Performance] Support o-proj TP weights async allgather in DSA-CP (#10694)", - "topics": [ - "deepseek-v4-dspark", - "communication" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/attention/context_parallel/dsa_cp.py", - "vllm_ascend/patch/__init__.py", - "vllm_ascend/patch/worker/__init__.py", - "vllm_ascend/patch/worker/patch_process_weights_after_loading.py" - ] - }, - { - "commit": "dfccd03e69d0d49167fc9a50805c229bb24c58f7", - "subject": "[Ascend950][Feature]Add allgatherEP MXFP4 quantization (#11287)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/ops/a2/test_token_dispatcher.py", - "vllm_ascend/device/device_op.py", - "vllm_ascend/ops/fused_moe/moe_comm_method.py", - "vllm_ascend/ops/fused_moe/prepare_finalize.py", - "vllm_ascend/ops/fused_moe/token_dispatcher.py", - "vllm_ascend/quantization/methods/w4a4_mxfp4.py" - ] - }, - { - "commit": "56a1f19afe3c4d10b2e636477e7e45554f09c8e6", - "subject": "[Refactor][Test] Reapply MC2 capacity and MoE comm selection (#11429)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/attention/a2/test_attention_v1.py", - "tests/ut/attention/a2/test_mla_v1.py", - "tests/ut/ops/a2/test_token_dispatcher.py", - "tests/ut/test_ascend_forward_context.py", - "vllm_ascend/ascend_forward_context.py", - "vllm_ascend/ops/fused_moe/token_dispatcher.py", - "vllm_ascend/utils.py" - ] - }, - { - "commit": "88f7fe4106f05d9a4ef16b3fec67cb18ea8b8cba", - "subject": "[Refactor][ops]Replace hardcoded max_output_size with configurable mega_moe_max_tokens (#11258)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "docs/source/user_guide/configuration/additional_config.md", - "vllm_ascend/ascend_config.py", - "vllm_ascend/ops/fused_moe/moe_comm_method.py" - ] - }, - { - "commit": "ae7203aaeeaf5b843b559c327cce1a82ed1fd28d", - "subject": "[Feature] Support Gemma4 large-head attention on A2 (#11091)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/attention/a2/test_attention_v1.py", - "vllm_ascend/attention/attention_v1.py", - "vllm_ascend/attention/utils.py", - "vllm_ascend/device/device_op.py", - "vllm_ascend/device/utils.py" - ] - }, - { - "commit": "05e095a202bdcfef4da61168eae34bfd3b99da13", - "subject": "[CI] main2mian v0.24.0 (#11227)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - ".github/vllm-main-verified.commit", - "examples/offline_data_parallel.py", - "tests/e2e/conftest.py", - "tests/ut/core/test_profiling_chunk.py", - "tests/ut/ops/test_gdn_attn_builder.py", - "tests/ut/quantization/methods/test_w4a16_mxfp4.py", - "tests/ut/spec_decode/test_extract_hidden_states_proposer.py", - "tests/ut/worker/a2/test_worker_v1.py", - "vllm_ascend/_310p/worker_310p.py", - "vllm_ascend/__init__.py", - "vllm_ascend/patch/platform/__init__.py", - "vllm_ascend/patch/platform/patch_dp_device_ids.py", - "vllm_ascend/patch/platform/patch_kv_cache_utils.py", - "vllm_ascend/patch/worker/patch_mamba_utils.py", - "vllm_ascend/simple_kv_offload/copy_backend.py", - "vllm_ascend/worker/model_runner_v1.py", - "vllm_ascend/worker/utils.py", - "vllm_ascend/worker/v2/attn_utils.py", - "vllm_ascend/worker/v2/spec_decode/rejection_sampler_utils.py", - "vllm_ascend/worker/worker.py" - ] - }, - { - "commit": "951180e5feb9e273a5dd5e34266ef231a8381f70", - "subject": "[BugFix] Fix AscendStore eagle store mask propagation (#11370)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/distributed/ascend_store/_mock_deps.py", - "tests/ut/distributed/ascend_store/test_coordinator.py", - "tests/ut/distributed/ascend_store/test_pool_worker.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/ascend_store/coordinator.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/ascend_store/pool_worker.py" - ] - }, - { - "commit": "0dc58f8e814f33eee64f4c652bb653ad05cfac73", - "subject": "[BugFix][Ops][310p]:fix the accuracy issue caused by MoEGatingTopkSoftmax (#11391)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/_310p/fused_moe/test_experts_selector_310.py", - "vllm_ascend/_310p/fused_moe/experts_selector.py" - ] - }, - { - "commit": "059611dc1e625feaf9f88f909d6ba1851adb8f56", - "subject": "[CI] Fix run ruff check error caused by w4a16mxfp4 (#11432)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/ops/fused_moe/moe_runtime_args.py", - "vllm_ascend/quantization/methods/w4a16_mxfp4.py" - ] - }, - { - "commit": "434afac207679d6447095d819abd6528713480c7", - "subject": "[BugFix] Avoid 310P Mamba align postprocess hang for MTP (#11353)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/_310p/test_mamba_align_fallback_310p_source.py", - "vllm_ascend/patch/worker/patch_mamba_utils.py" - ] - }, - { - "commit": "f81dd51845185d18ea26cfbef2042f9b3bc0de2e", - "subject": "[Feature][Quant] w4a16-mxfp4 for vllm-ascend (#11014)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/quantization/methods/test_w4a16_mxfp4.py", - "vllm_ascend/device/device_op.py", - "vllm_ascend/ops/fused_moe/moe_mlp.py", - "vllm_ascend/ops/fused_moe/moe_runtime_args.py", - "vllm_ascend/ops/fused_moe/moe_stage_params.py", - "vllm_ascend/quantization/methods/__init__.py", - "vllm_ascend/quantization/methods/w4a16_mxfp4.py", - "vllm_ascend/quantization/quant_parser.py", - "vllm_ascend/quantization/quant_type.py" - ] - }, - { - "commit": "8c28b0eb1383c12635c78f2f00bcf0cd6f7a5417", - "subject": "[CI] upgrade vllm to 0619 (#10935)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - ".github/vllm-main-verified.commit", - "tests/e2e/pull_request/one_card/test_guided_decoding.py", - "tests/ut/patch/platform/test_patch_glm47_tool_call_parser.py", - "tests/ut/patch/platform/test_patch_minimax_m2_tool_call_parser.py", - "tests/ut/patch/platform/test_patch_minimax_usage_accounting.py", - "vllm_ascend/core/scheduler_profiling_chunk.py", - "vllm_ascend/ops/fused_moe/fused_moe.py", - "vllm_ascend/patch/__init__.py", - "vllm_ascend/patch/platform/__init__.py", - "vllm_ascend/patch/platform/patch_balance_schedule.py", - "vllm_ascend/patch/platform/patch_profiling_chunk.py", - "vllm_ascend/patch/platform/patch_use_v2_model_runner.py", - "vllm_ascend/patch/worker/patch_v2/patch_use_v2_model_runner.py", - "vllm_ascend/spec_decode/ngram_proposer.py", - "vllm_ascend/spec_decode/suffix_proposer.py", - "vllm_ascend/worker/encoder_acl_graph.py", - "vllm_ascend/worker/model_runner_v1.py", - "vllm_ascend/worker/v2/aclgraph_utils.py", - "vllm_ascend/worker/v2/model_runner.py", - "vllm_ascend/worker/v2/spec_decode/eagle/aclgraph.py" - ] - }, - { - "commit": "29894920f062ea0d164157a917fb70cda913f892", - "subject": "Revert \"[Refactor][Test] Simplify MC2 capacity and MoE comm selection\" (#11415)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/workflows/_selected_tests.yaml", - "csrc/build.sh", - "csrc/cmake/scripts/custom/install.sh", - "csrc/cmake/scripts/custom/upgrade.sh", - "csrc/scripts/package/ops_transformer/scripts/install.sh", - "csrc/scripts/util/merge_aicpu_info_json.sh", - "tests/ut/attention/a2/test_attention_v1.py", - "tests/ut/attention/a2/test_mla_v1.py", - "tests/ut/ops/a2/test_token_dispatcher.py", - "tests/ut/spec_decode/a2/test_eagle_proposer.py", - "tests/ut/test_ascend_forward_context.py", - "tools/shellcheck.sh", - "vllm_ascend/ascend_forward_context.py", - "vllm_ascend/ops/fused_moe/token_dispatcher.py", - "vllm_ascend/utils.py" - ] - }, - { - "commit": "7d476afdfad798f74283961fcaf1bbbf170aa02e", - "subject": "[Doc] Update Chinese translations (#11340)", - "topics": [ - "deepseek-v4-dspark", - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/scripts/po_translate.py", - "docs/source/_templates/Model-Deployment-Tutorial-Template.md", - "docs/source/_templates/Model-Deployment-Tutorial-Template.zh.md", - "docs/source/locale/zh_CN/LC_MESSAGES/community/contributors.po", - "docs/source/locale/zh_CN/LC_MESSAGES/community/governance.po", - "docs/source/locale/zh_CN/LC_MESSAGES/community/slash-commands.po", - "docs/source/locale/zh_CN/LC_MESSAGES/community/versioning_policy.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/Design_Documents/KV_Cache_Pool_Guide.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/Design_Documents/ModelRunner_prepare_inputs.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/Design_Documents/context_parallel.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/Design_Documents/cpu_binding.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/Design_Documents/disaggregated_prefill.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/Design_Documents/eplb_swift_balancer.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/contribution/nightly_ci_test.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/contribution/testing.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/performance_and_debug/optimization_and_tuning.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/performance_and_debug/performance_benchmark.po", - "docs/source/locale/zh_CN/LC_MESSAGES/faqs.po", - "docs/source/locale/zh_CN/LC_MESSAGES/installation.po", - "docs/source/locale/zh_CN/LC_MESSAGES/quick_start.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/features/dynamic_chunked_pipeline_parallel.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/features/long_sequence_context_parallel_multi_node.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/features/long_sequence_context_parallel_single_node.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/features/pd_disaggregation_mooncake_multi_node.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/hardwares/310p.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/DeepSeek-V4-Flash.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/DeepSeek-V4-Pro.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/DeepSeekOCR2.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/GLM4.x.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/GLM5.2.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/GLM5.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/InternVL3.5.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Kimi-K2-Thinking.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Kimi-K2.5.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Kimi-K2.6.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/MiniMax-M2.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/PaddleOCR-VL.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen-VL-Dense.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-235B-A22B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-30B-A3B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-ASR-1.7B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-Coder-30B-A3B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-Dense.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-Next.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-Omni-30B-A3B-Thinking.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-VL-30B-A3B-Instruct.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3.5-27B-Qwen3.6-27B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3.5-397B-A17B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3.6-35B-A3B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3_embedding.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/configuration/additional_config.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/configuration/env_vars.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/deployment_guide/using_mindie_motor.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/dynamic_batch.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/dynamic_chunk_pipeline_parallel.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/expert_parallelism_load_balancer.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/flash_attention.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/kv_pool.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/layer_sharding.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/rfork.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/sequence_parallelism.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/sleep_mode.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/weight_prefetch.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/release_notes.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/support_matrix/feature_matrix.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/support_matrix/supported_models.po", - "docs/source/user_guide/feature_guide/sequence_parallelism.md" - ] - }, - { - "commit": "aabec5efb082c960ca54ac9006e1780f0057378e", - "subject": "[Refactor][Test] Simplify MC2 capacity and MoE comm selection (#11155)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/attention/a2/test_attention_v1.py", - "tests/ut/attention/a2/test_mla_v1.py", - "tests/ut/ops/a2/test_token_dispatcher.py", - "tests/ut/test_ascend_forward_context.py", - "vllm_ascend/ascend_forward_context.py", - "vllm_ascend/ops/fused_moe/token_dispatcher.py", - "vllm_ascend/utils.py" - ] - }, - { - "commit": "fbc2dc570a0c7543af2942a6a8451ac17e3c4582", - "subject": "[BugFix] Add the verification for GLM spec (#11327)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "tests/e2e/pull_request/four_card/_310p/test_moe_model_310p.py", - "tests/e2e/pull_request/one_card/_310p/test_classification_310p.py", - "vllm_ascend/spec_decode/llm_base_proposer.py" - ] - }, - { - "commit": "bb2666df29c4cddab74fd340103f18bd5bf2bb9f", - "subject": "[Feature] Add rmsnorm dynamic mx quant fusion pass (#10730)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/compilation/passes/norm_quant_fusion_pass.py", - "vllm_ascend/device/mxfp_compat.py" - ] - }, - { - "commit": "5da6e5360f10cb061412d6b91aab575fdda03869", - "subject": "[main][Feature] enable prefill with mc2 with additional_config (#11022)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/configs/nightly_config.yaml", - "docs/source/user_guide/configuration/additional_config.md", - "tests/e2e/nightly/single_node/models/configs/GLM-5.1-W8A8-PrefillMC2.yaml", - "vllm_ascend/ascend_config.py", - "vllm_ascend/ascend_forward_context.py" - ] - }, - { - "commit": "929e461de828de233c2047def41a26a02c768084", - "subject": "[Bugfix] Fix GLM-5.1 IndexCache weight loading (#11363)", - "topics": [ - "deepseek-v4-dspark", - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/patch/worker/test_patch_deepseek_v2.py", - "vllm_ascend/patch/__init__.py", - "vllm_ascend/patch/worker/patch_deepseek_v2.py" - ] - }, - { - "commit": "e6960fa41ab50cdec36b17b2f239895f9418e627", - "subject": "[Feature][KV Pool] Introduce AscendStoreCoordinator for hybrid cache-hit and mask coordination (#10393)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/distributed/ascend_store/_mock_deps.py", - "tests/ut/distributed/ascend_store/test_ascend_store_connector.py", - "tests/ut/distributed/ascend_store/test_config_data.py", - "tests/ut/distributed/ascend_store/test_coordinator.py", - "tests/ut/distributed/ascend_store/test_kv_transfer.py", - "tests/ut/distributed/ascend_store/test_pool_worker.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/ascend_store/config_data.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/ascend_store/coordinator.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/ascend_store/kv_transfer.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/ascend_store/pool_scheduler.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/ascend_store/pool_worker.py" - ] - }, - { - "commit": "4db5765b4257ee610f1a46c5b60689474d1af4b3", - "subject": "[Feature]Adapt GLM-5.2 (#11264)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/patch/worker/patch_deepseek_mtp.py", - "vllm_ascend/spec_decode/llm_base_proposer.py" - ] - }, - { - "commit": "d0446f16d72ad5467d74585e9ddcd3cbb45c8793", - "subject": "[BugFix][ModelLoader] Fix RFork seed isolation across parallel ranks (#10942)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/user_guide/feature_guide/rfork.md", - "tests/ut/model_loader/rfork/test_rfork_loader.py", - "vllm_ascend/model_loader/rfork/rfork_loader.py", - "vllm_ascend/model_loader/rfork/rfork_worker.py", - "vllm_ascend/model_loader/rfork/seed_protocol.py" - ] - }, - { - "commit": "cfd633d063d5d1d78484316f762301493c3d6a67", - "subject": "[Feature] Add mxfp quant for token dispatch to fit low acc communication (#10749)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/ops/fused_moe/moe_stage_params.py", - "vllm_ascend/ops/fused_moe/token_dispatcher.py" - ] - }, - { - "commit": "7cd9e128cedb3851f7303c9930088d2b861c0aa0", - "subject": "[Feature] Support Step3P7/Step3P5 with Step3P5 MTP on Ascend (#10697)", - "topics": [ - "communication", - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/workflows/scripts/test_config.yaml", - "tests/ut/ops/a3_2/test_activation.py", - "tests/ut/ops/test_moe_mlp.py", - "tests/ut/quantization/test_modelslim_config.py", - "tests/ut/spec_decode/test_step3p5_source_regression.py", - "vllm_ascend/attention/attention_v1.py", - "vllm_ascend/ops/activation.py", - "vllm_ascend/ops/fused_moe/moe_mlp.py", - "vllm_ascend/ops/weight_prefetch.py", - "vllm_ascend/patch/platform/patch_speculative_config.py", - "vllm_ascend/quantization/modelslim_config.py", - "vllm_ascend/spec_decode/__init__.py", - "vllm_ascend/spec_decode/llm_base_proposer.py", - "vllm_ascend/spec_decode/step3p5.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "1930088f960aba65eeaae82e9617d090283edc1f", - "subject": "[Feature] Support dsv4 mtp graph (#11062)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "tests/e2e/pull_request/four_card/test_deepseek_v4.py", - "tests/ut/spec_decode/a2/test_eagle_proposer.py", - "vllm_ascend/_310p/model_runner_310p.py", - "vllm_ascend/attention/dsa_v1.py", - "vllm_ascend/ops/rope_dsv4.py", - "vllm_ascend/spec_decode/llm_base_proposer.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "cadefd2cb87a8b0dc6d4e80abc2d770121a92f42", - "subject": "[Performance] DSV4 prefix cache hit rate optimize (#11107)", - "topics": [ - "deepseek-v4-dspark", - "communication" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/patch/platform/test_prefix_cache_cp_patches.py", - "vllm_ascend/patch/platform/patch_kv_cache_coordinator.py" - ] - }, - { - "commit": "fd2aa3f59bed3a8c846c37b73dad7a6a0d10e466", - "subject": "[Performance][PD] Parallelize KV cache receive via ThreadPoolExecutor (#10548)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/kv_offload/test_mooncake_connector.py", - "tests/ut/kv_offload/test_mooncake_hybrid_connector.py", - "vllm_ascend/distributed/kv_transfer/kv_p2p/mooncake_connector.py", - "vllm_ascend/distributed/kv_transfer/kv_p2p/mooncake_hybrid_connector.py" - ] - }, - { - "commit": "801a6b41d29d23399bd8c9ebc2ea8883e2beefae", - "subject": "[Misc] upgrade vLLM to a30addc7548a9a8b9b3323a7bc3eb7d7c4895d1c (#10459)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/vllm-main-verified.commit", - "requirements-dev.txt", - "tests/e2e/pull_request/one_card/spec_decode/test_extract_hidden_states.py", - "tests/ut/patch/platform/test_patch_tool_choice_none_content.py", - "vllm_ascend/ops/fused_moe/fused_moe.py", - "vllm_ascend/worker/v2/attn_utils.py", - "vllm_ascend/worker/v2/model_runner.py" - ] - }, - { - "commit": "52da9d92fd0f68a63174736fa90799cf8e73fb75", - "subject": "[CI] Modify the threshold of the gsm8k lite dataset to 3 questions (#11247)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "tests/e2e/nightly/multi_node/internal_dp/config/DeepSeek-V3.1-BF16.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/DeepSeek-V3_2-W8A8-A3-dual-nodes.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/DeepSeek-V3_2-W8A8-EP.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/Qwen3-235B-A22B-A2.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/Qwen3-235B-A22B-Mooncake-Layerwise.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/Qwen3-235B-disagg-pd.yaml", - "tests/e2e/nightly/single_node/models/configs/DeepSeek-R1-0528-W8A8.yaml", - "tests/e2e/nightly/single_node/models/configs/GLM-4.7.yaml", - "tests/e2e/nightly/single_node/models/configs/Hy3-preview.yaml", - "tests/e2e/nightly/single_node/models/configs/Kimi-K2-Thinking.yaml", - "tests/e2e/nightly/single_node/models/configs/MTPX-DeepSeek-R1-0528-W8A8.yaml", - "tests/e2e/nightly/single_node/models/configs/Qwen3-235B-A22B-W8A8.yaml", - "tests/e2e/nightly/single_node/models/configs/Qwen3-30B-A3B-W4A8-llm-compressor.yaml", - "tests/e2e/nightly/single_node/models/configs/Qwen3-32B-Int8.yaml", - "tests/e2e/weekly/multi_node/internal_dp/config/DeepSeek-V3.yaml", - "tests/e2e/weekly/multi_node/internal_dp/config/GLM-4.7-W8A8C8-Mooncake-Layerwise.yaml", - "tests/e2e/weekly/single_node/configs/GLM-5.yaml", - "tests/e2e/weekly/single_node/configs/MiniMax-M2.5-W8A8-A3.yaml" - ] - }, - { - "commit": "55b8cd0d8fcec123b02f90a9dc50d7f6061e650c", - "subject": "[BugFix][Model] Remove redundant SFA o_proj TP weights (#11068)", - "topics": [ - "communication", - "priority-models", - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "docs/source/developer_guide/Design_Documents/context_parallel.md", - "tests/ut/attention/test_sfa_o_proj_tp.py", - "tests/ut/quantization/methods/test_w4a4_mxfp4.py", - "tests/ut/quantization/methods/test_w8a8_mxfp8.py", - "vllm_ascend/attention/sfa_v1.py", - "vllm_ascend/quantization/methods/w4a4_mxfp4.py", - "vllm_ascend/quantization/methods/w8a8_mxfp8.py" - ] - }, - { - "commit": "5e19a5a8f2c7ab9cbfb8ba5ca674f4adc8cfb51b", - "subject": "[Refactor][EPLB] Support mixed expert weight layouts (#11120)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/workflows/scripts/run_selected_tests.sh", - "tests/ut/eplb/adaptor/test_vllm_adaptor.py", - "tests/ut/eplb/core/test_eplb_device_transfer_loader.py", - "vllm_ascend/eplb/adaptor/vllm_adaptor.py", - "vllm_ascend/eplb/core/eplb_device_transfer_loader.py", - "vllm_ascend/spec_decode/llm_base_proposer.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "a99f254930f0b6171716edc614e8e136768b2d44", - "subject": "[BugFix]Fix moe allgather NaN issue (#10579)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/device/device_op.py", - "vllm_ascend/ops/fused_moe/token_dispatcher.py" - ] - }, - { - "commit": "cbfe0e14716c92c963af3903bc323c86c08f40e7", - "subject": "[Performance][Attention] Build DSA compressor metadata on device (#10741)", - "topics": [ - "deepseek-v4-dspark", - "communication" - ], - "runtime_patch_match": false, - "paths": [ - "csrc/attention/compressor_metadata/CMakeLists.txt", - "csrc/attention/compressor_metadata/op_host/CMakeLists.txt", - "csrc/attention/compressor_metadata/op_host/compressor_metadata_def.cpp", - "csrc/attention/compressor_metadata/op_host/compressor_metadata_tiling.cpp", - "csrc/attention/compressor_metadata/op_host/compressor_metadata_tiling.h", - "csrc/attention/compressor_metadata/op_kernel/compressor_metadata.cpp", - "csrc/attention/compressor_metadata/op_kernel/compressor_metadata.h", - "csrc/build_aclnn.sh", - "csrc/torch_binding.cpp", - "csrc/torch_binding_meta.cpp", - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_compressor_metadata.py", - "vllm_ascend/_310p/model_runner_310p.py", - "vllm_ascend/attention/context_parallel/dsa_cp.py", - "vllm_ascend/attention/dsa_v1.py", - "vllm_ascend/device/device_op.py", - "vllm_ascend/ops/rope_dsv4.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "2f69c377b17f62793919a54aebc10cf7c55e6687", - "subject": "[BugFix][Model] fix act fn in dsv4 (#11184)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/models/deepseek_v4.py" - ] - }, - { - "commit": "354b5ae94e326acee8895290d424aba5a4efcde7", - "subject": "[Feature] [Ops] Add swiglu_limit support in unquant_apply_mlp (#9897)", - "topics": [ - "deepseek-v4-dspark", - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/ops/fused_moe/moe_mlp.py" - ] - }, - { - "commit": "cc3c8897028b629506ebf59e37bed62e89d4413d", - "subject": "[CI] Add GLM-5.2 model to model dataset list (#11046)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/misc/model_dataset_list.json", - "tests/e2e/nightly/multi_node/internal_dp/config/GLM5_2-W8A8-A3-dual-nodes.yaml" - ] - }, - { - "commit": "1c57c6a0a95dd99b06d7d75d3dd014f50f6cdb63", - "subject": "[Misc] main2main 0608(refactor fused moe) (#11081)", - "topics": [ - "deepseek-v4-dspark", - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/vllm-main-verified.commit", - "tests/e2e/pull_request/two_card/test_external_launcher.py", - "tests/e2e/pull_request/two_card/test_qwen3_30b_a3b.py", - "tests/e2e/pull_request/two_card/test_qwen3_moe_eplb.py", - "tests/ut/_310p/quantization/test_modelslim_config_310.py", - "tests/ut/eplb/core/a2/test_eplb_utils.py", - "tests/ut/ops/test_fused_moe.py", - "tests/ut/quantization/test_compressed_tensors_config.py", - "tests/ut/quantization/test_modelslim_config.py", - "vllm_ascend/_310p/fused_moe/fused_moe.py", - "vllm_ascend/_310p/quantization/modelslim_config.py", - "vllm_ascend/models/deepseek_v4.py", - "vllm_ascend/models/deepseek_v4_mtp.py", - "vllm_ascend/ops/fused_moe/fused_moe.py", - "vllm_ascend/ops/fused_moe/fused_moe_0_23_0.py", - "vllm_ascend/ops/fused_moe/moe_runtime_args.py", - "vllm_ascend/patch/platform/__init__.py", - "vllm_ascend/patch/platform/patch_fused_moe.py", - "vllm_ascend/patch/worker/__init__.py", - "vllm_ascend/patch/worker/patch_fused_moe.py", - "vllm_ascend/quantization/compressed_tensors_config.py", - "vllm_ascend/quantization/fp8_config.py", - "vllm_ascend/quantization/modelslim_config.py", - "vllm_ascend/utils.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "798f5596f1f8b711929c072eb0758c85a30b2436", - "subject": "[Doc][Misc] Refresh PaddleOCR-VL deployment tutorial documentation (#10840)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/tutorials/models/PaddleOCR-VL.md" - ] - }, - { - "commit": "ef45723a5284d2d217a3a36d11e536f644caa938", - "subject": "[Doc]add Recommended Configurations to GLM5.md (#11010)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/tutorials/models/GLM5.md" - ] - }, - { - "commit": "234a6bb7e436fd53e61f04f186d725de1b6fb233", - "subject": "[Ops][Misc] Remove unused load of b_g in fused_recurrent_gated_delta_rule_fwd_kernel (#10929)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/ops/triton/fla/sigmoid_gating.py" - ] - }, - { - "commit": "62f364f3dd9a62fc4790ac7b44f48fe6f3b3337b", - "subject": "[Feature] [BugFix] Support fine-grained Tensor Parallelism (OTP and Embedding TP) in DSA (#10586)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/ascend_config.py", - "vllm_ascend/attention/dsa_v1.py", - "vllm_ascend/ops/dsa.py", - "vllm_ascend/ops/linear_op.py", - "vllm_ascend/ops/vocab_parallel_embedding.py", - "vllm_ascend/utils.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "baf4a9d0bd1e6432ed799b0d1ee16c7ebe73a5a0", - "subject": "[BugFix][Model] Sync HcPre fused path and use v2 (#10980)", - "topics": [ - "deepseek-v4-dspark", - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "csrc/moe/hc_pre/op_host/hc_pre_tiling_arch35.h", - "csrc/moe/hc_pre/op_kernel/hc_pre_base_arch35.h", - "csrc/moe/hc_pre/op_kernel/hc_pre_cube_compute.h", - "csrc/moe/hc_pre/op_kernel/hc_pre_m_k_split_core_arch35.h", - "csrc/moe/hc_pre/op_kernel/hc_pre_m_split_core_arch35.h", - "csrc/torch_binding.cpp", - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_npu_hc_pre.py", - "vllm_ascend/models/deepseek_v4.py" - ] - }, - { - "commit": "0e982ed5479470c4f55a6eac0dc96affbf704116", - "subject": "[CI] Add GLM5.1 ;Deepseekv3.2;Qwen3.5-397B PD cases (#10975)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/schedule_weekly_test_a3.yaml", - "tests/e2e/nightly/single_node/models/configs/DeepSeek-V3.2-W8A8.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/GLM_5_1_PD_in32k_bs16-0.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/GLM_5_1_PD_in32k_bs20-90.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/GLM_5_1_PD_in64k_bs16-90.yaml", - "tests/e2e/weekly/multi_node/external_dp/config/Qwen-3.5-397B-A17B-W8A8-PD.yaml" - ] - }, - { - "commit": "c3595e4ca3ba8fea494c7fdc0f59d041c0054148", - "subject": "[CI][Misc] Modify accuracy test thresholds for multiple models (#11041)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "tests/e2e/nightly/multi_node/internal_dp/config/DeepSeek-V3_2-W8A8-A3-dual-nodes.yaml", - "tests/e2e/nightly/single_node/models/configs/DeepSeek-V4-Flash-W8A8-A3.yaml", - "tests/e2e/nightly/single_node/models/configs/MTPX-DeepSeek-R1-0528-W8A8.yaml", - "tests/e2e/nightly/single_node/models/configs/Qwen3-32B-Int8-A2.yaml", - "tests/e2e/weekly/single_node/configs/Kimi-K2.5.yaml", - "tests/e2e/weekly/single_node/configs/Qwen3.5-122B-A10B-W8A8-A3.yaml" - ] - }, - { - "commit": "c5ae281d9bfc60390e93cdbf3a9784c2f8096249", - "subject": "[Feature] Adapt GLM-5.2 (#10441)", - "topics": [ - "communication", - "priority-models", - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/workflows/schedule_nightly_test_a3.yaml", - "docs/source/developer_guide/contribution/nightly_ci_test.md", - "tests/e2e/nightly/multi_node/internal_dp/config/GLM5_2-W8A8-A3-dual-nodes.yaml", - "vllm_ascend/attention/sfa_v1.py", - "vllm_ascend/patch/__init__.py", - "vllm_ascend/patch/worker/__init__.py", - "vllm_ascend/patch/worker/patch_deepseek_v2.py" - ] - }, - { - "commit": "463a6a1d065d246918e4cb4319e07ae63a2d2c9e", - "subject": "[Feature]Support IndexCache top-k index reuse for GLM5.2 (#10510)", - "topics": [ - "deepseek-v4-dspark", - "priority-models", - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/spec_decode/a2/test_eagle_proposer.py", - "vllm_ascend/spec_decode/llm_base_proposer.py" - ] - }, - { - "commit": "42c77b3ab50476800e48a67d93eb7404dd8649e7", - "subject": "[Doc] Fix documentation typos and resolve merge conflicts (#11020)", - "topics": [ - "deepseek-v4-dspark", - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/_templates/Model-Deployment-Tutorial-Template.md", - "docs/source/community/governance.md", - "docs/source/developer_guide/Design_Documents/ModelRunner_prepare_inputs.md", - "docs/source/developer_guide/Design_Documents/disaggregated_prefill.md", - "docs/source/developer_guide/Design_Documents/eplb_swift_balancer.md", - "docs/source/developer_guide/contribution/testing.md", - "docs/source/developer_guide/evaluation/using_lm_eval.md", - "docs/source/developer_guide/performance_and_debug/performance_benchmark.md", - "docs/source/installation.md", - "docs/source/quick_start.md", - "docs/source/tutorials/features/long_sequence_context_parallel_single_node.md", - "docs/source/tutorials/features/pd_disaggregation_mooncake_multi_node.md", - "docs/source/tutorials/models/DeepSeek-V4-Flash.md", - "docs/source/tutorials/models/DeepSeekOCR2.md", - "docs/source/tutorials/models/GLM4.x.md", - "docs/source/tutorials/models/GLM5.2.md", - "docs/source/tutorials/models/GLM5.md", - "docs/source/tutorials/models/Kimi-K2.5.md", - "docs/source/tutorials/models/Kimi-K2.6.md", - "docs/source/tutorials/models/MiniMax-M2.5.md", - "docs/source/tutorials/models/PaddleOCR-VL.md", - "docs/source/tutorials/models/Qwen-VL-Dense.md", - "docs/source/tutorials/models/Qwen3-235B-A22B.md", - "docs/source/tutorials/models/Qwen3-VL-30B-A3B-Instruct.md", - "docs/source/tutorials/models/Qwen3.5-397B-A17B.md", - "docs/source/tutorials/models/Qwen3_embedding.md", - "docs/source/user_guide/configuration/additional_config.md", - "docs/source/user_guide/feature_guide/flash_attention.md", - "docs/source/user_guide/feature_guide/kv_pool.md", - "docs/source/user_guide/feature_guide/layer_sharding.md", - "docs/source/user_guide/feature_guide/rfork.md", - "docs/source/user_guide/feature_guide/weight_prefetch.md", - "docs/source/user_guide/release_notes.md", - "docs/source/user_guide/support_matrix/feature_matrix.md" - ] - }, - { - "commit": "8cc7bcd4c35ab7a4fbd50b36c362b136df7c0d95", - "subject": "[Feature] A5 Support Gemma4 (#10643)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/attention/a2/test_attention_cp.py", - "tests/ut/attention/a2/test_attention_v1.py", - "tests/ut/ops/a3_2/test_select_experts.py", - "vllm_ascend/_310p/fused_moe/experts_selector.py", - "vllm_ascend/ascend_forward_context.py", - "vllm_ascend/attention/attention_v1.py", - "vllm_ascend/attention/utils.py", - "vllm_ascend/ops/fused_moe/experts_selector.py", - "vllm_ascend/ops/fused_moe/moe_mlp.py" - ] - }, - { - "commit": "d901dd421d1d030343af6fedc9723f9a37d9f72c", - "subject": "[Refactor][Patch] Trim upstreamed MiniMax-M2 hooks (#10906)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/patch/__init__.py", - "vllm_ascend/patch/platform/patch_minimax_m2_config.py", - "vllm_ascend/patch/worker/patch_minimax_m2.py" - ] - }, - { - "commit": "8fd6a0c2c39d5931e62edc2f342c9dad94d3701d", - "subject": "[Doc] Increased the restrictions on the transformer version (#10992)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/tutorials/models/GLM5.md" - ] - }, - { - "commit": "17f14dd4c194375b619fe1b348a4f0d71ef0e33b", - "subject": "[Doc] fix GLM5.2 doc issue (#10826)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/tutorials/models/GLM5.2.md" - ] - }, - { - "commit": "f9ce51c22ee0d7983e81d0959eacc01146e1c0d2", - "subject": "[Doc] Standardize DeepSeek-V4-Flash and DeepSeek-V4-Pro deployment tutorial structure (#10693)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/tutorials/models/DeepSeek-V4-Flash.md", - "docs/source/tutorials/models/DeepSeek-V4-Pro.md" - ] - }, - { - "commit": "db4cc43041407553675ff4a35d1b5758129fc73e", - "subject": "[BugFix] avoid experimental QLI overriding native QLI (#10900)", - "topics": [ - "communication" - ], - "runtime_patch_match": false, - "paths": [ - "csrc/attention/vllm_quant_lightning_indexer/CMakeLists.txt", - "csrc/attention/vllm_quant_lightning_indexer/README.md", - "csrc/attention/vllm_quant_lightning_indexer/op_host/CMakeLists.txt", - "csrc/attention/vllm_quant_lightning_indexer/op_host/vllm_quant_lightning_indexer_def.cpp", - "csrc/attention/vllm_quant_lightning_indexer/op_host/vllm_quant_lightning_indexer_infershape.cpp", - "csrc/attention/vllm_quant_lightning_indexer/op_host/vllm_quant_lightning_indexer_tiling.cpp", - "csrc/attention/vllm_quant_lightning_indexer/op_host/vllm_quant_lightning_indexer_tiling.h", - "csrc/attention/vllm_quant_lightning_indexer/op_kernel/arch32/quant_lightning_indexer_common.h", - "csrc/attention/vllm_quant_lightning_indexer/op_kernel/arch32/quant_lightning_indexer_kernel.h", - "csrc/attention/vllm_quant_lightning_indexer/op_kernel/arch32/quant_lightning_indexer_service_cube.h", - "csrc/attention/vllm_quant_lightning_indexer/op_kernel/arch32/quant_lightning_indexer_service_vector.h", - "csrc/attention/vllm_quant_lightning_indexer/op_kernel/arch32/quant_lightning_indexer_vector.h", - "csrc/attention/vllm_quant_lightning_indexer/op_kernel/arch35/quant_lightning_indexer_common.h", - "csrc/attention/vllm_quant_lightning_indexer/op_kernel/arch35/quant_lightning_indexer_kernel.h", - "csrc/attention/vllm_quant_lightning_indexer/op_kernel/arch35/quant_lightning_indexer_service_cube.h", - "csrc/attention/vllm_quant_lightning_indexer/op_kernel/arch35/quant_lightning_indexer_service_vector.h", - "csrc/attention/vllm_quant_lightning_indexer/op_kernel/arch35/vf/quant_lightning_indexer_topk.h", - "csrc/attention/vllm_quant_lightning_indexer/op_kernel/arch35/vf/quant_lightning_indexer_vector1.h", - "csrc/attention/vllm_quant_lightning_indexer/op_kernel/arch35/vf/vf_topk.h", - "csrc/attention/vllm_quant_lightning_indexer/op_kernel/arch35/vf/vf_topk_16_gather.h", - "csrc/attention/vllm_quant_lightning_indexer/op_kernel/vllm_quant_lightning_indexer.cpp", - "csrc/attention/vllm_quant_lightning_indexer/op_kernel/vllm_quant_lightning_indexer_metadata.h", - "csrc/attention/vllm_quant_lightning_indexer/op_kernel/vllm_quant_lightning_indexer_template_tiling_key.h", - "csrc/attention/vllm_quant_lightning_indexer_metadata/CMakeLists.txt", - "csrc/attention/vllm_quant_lightning_indexer_metadata/README.md", - "csrc/attention/vllm_quant_lightning_indexer_metadata/op_api/aclnn_vllm_quant_lightning_indexer_metadata.cpp", - "csrc/attention/vllm_quant_lightning_indexer_metadata/op_api/aclnn_vllm_quant_lightning_indexer_metadata.h", - "csrc/attention/vllm_quant_lightning_indexer_metadata/op_api/l0_vllm_quant_lightning_indexer_metadata.cpp", - "csrc/attention/vllm_quant_lightning_indexer_metadata/op_api/l0_vllm_quant_lightning_indexer_metadata.h", - "csrc/attention/vllm_quant_lightning_indexer_metadata/op_graph/vllm_quant_lightning_indexer_metadata_proto.h", - "csrc/attention/vllm_quant_lightning_indexer_metadata/op_host/vllm_quant_lightning_indexer_metadata_infershape.cpp", - "csrc/attention/vllm_quant_lightning_indexer_metadata/op_kernel_aicpu/vllm_quant_lightning_indexer_metadata_aicpu.cpp", - "csrc/attention/vllm_quant_lightning_indexer_metadata/op_kernel_aicpu/vllm_quant_lightning_indexer_metadata_aicpu.h", - "csrc/attention/vllm_quant_lightning_indexer_metadata/op_kernel_aicpu/vllm_quant_lightning_indexer_metadata_aicpu.json", - "csrc/build_aclnn.sh", - "csrc/torch_binding.cpp", - "csrc/torch_binding_meta.cpp", - "vllm_ascend/attention/context_parallel/dsa_cp.py", - "vllm_ascend/attention/dsa_v1.py" - ] - }, - { - "commit": "c3ebfc44de79d0eafab025e7a394f4fbaeadd816", - "subject": "[Feature] fused mc2 w4a8 workspace memory reuse (#9927)", - "topics": [ - "communication", - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "csrc/mc2/dispatch_ffn_combine_w4_a8/op_host/dispatch_ffn_combine_w4_a8_tiling.cpp", - "csrc/mc2/dispatch_ffn_combine_w4_a8/op_kernel/dispatch_ffn_combine_w4_a8_kernel.hpp", - "csrc/mc2/dispatch_ffn_combine_w4_a8/op_kernel/utils/block_epilogue_w4a8post_pertoken_swiglu.hpp" - ] - }, - { - "commit": "419755b4fbb07d36baa947d78cd94cb31931451d", - "subject": "[BugFix] Pad seq_lens/block_table to fix FIA error 561002 with flashcomm1 (#10769)", - "topics": [ - "communication", - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/attention/attention_v1.py" - ] - }, - { - "commit": "e2f1028f6da43130adfda5ce655c4a290659470b", - "subject": "[BugFix] Pad actual_seq_lengths with 0 for EOD scenario in npu_fusion_attention (#10956)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/attention/attention_v1.py" - ] - }, - { - "commit": "aafebda8a4dda6f77548b9af2418de82f0577c23", - "subject": "[BugFix]mega moe w4a8 assert bias fix (#10846)", - "topics": [ - "communication", - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/ops/fused_moe/moe_comm_method.py" - ] - }, - { - "commit": "f4500fb764aebe258c5e1170c6a16761a162e1cf", - "subject": "[Bugfix] Fix triton error when activing enable-prefix-caching in decode node (#10888)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_postprocess_mamba.py", - "vllm_ascend/ops/triton/mamba/postprocess.py", - "vllm_ascend/patch/worker/patch_mamba_utils.py" - ] - }, - { - "commit": "36d661fa4ad05402f081db029e533b6d244ad0ea", - "subject": "[Feature][KVCache] Use KVCacheSpecRegistry mechanism to custom KVCacheSpec (#9522)", - "topics": [ - "deepseek-v4-dspark", - "communication" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/attention/context_parallel/dsa_cp.py", - "vllm_ascend/attention/context_parallel/mla_cp.py", - "vllm_ascend/attention/dsa_v1.py", - "vllm_ascend/attention/mla_v1.py", - "vllm_ascend/core/kv_cache_interface.py", - "vllm_ascend/core/single_type_kv_cache_manager.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/cpu_offload/metadata.py", - "vllm_ascend/models/deepseek_v4.py", - "vllm_ascend/models/layer/attention/layer.py", - "vllm_ascend/patch/__init__.py", - "vllm_ascend/patch/platform/__init__.py", - "vllm_ascend/patch/platform/patch_kv_cache_coordinator.py", - "vllm_ascend/patch/worker/patch_v2/patch_attn_utils.py", - "vllm_ascend/platform.py", - "vllm_ascend/spec_decode/llm_base_proposer.py", - "vllm_ascend/utils.py", - "vllm_ascend/worker/model_runner_v1.py", - "vllm_ascend/worker/v2/attn_utils.py" - ] - }, - { - "commit": "55c27ec8b97fb67b8b67802bd53b8db3760e9468", - "subject": "[Feature] Migrate KDA Triton Op to Ascend NPU (#9035)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_chunk_kda_npu.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_fused_recurrent_kda_npu.py", - "vllm_ascend/ops/triton/kda/__init__.py", - "vllm_ascend/ops/triton/kda/chunk_delta_h.py", - "vllm_ascend/ops/triton/kda/cumsum.py", - "vllm_ascend/ops/triton/kda/fused_recurrent_kda.py", - "vllm_ascend/ops/triton/kda/kda.py", - "vllm_ascend/ops/triton/kda/l2norm.py", - "vllm_ascend/ops/triton/kda/solve_tril.py", - "vllm_ascend/ops/triton/kda/utils.py" - ] - }, - { - "commit": "0233ddf339527947a90baf9facd10cd6888aa04d", - "subject": "[Performance][310P] Replace l2norm with RMSNorm operator on 310P (#10408)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/_310p/ops/test_chunk_gated_delta_rule_310.py", - "vllm_ascend/_310p/ops/fla/__init__.py", - "vllm_ascend/_310p/ops/fla/chunk_gated_delta_rule.py", - "vllm_ascend/_310p/ops/fla/fused_recurrent_gated_delta_rule.py", - "vllm_ascend/_310p/ops/fla/gdn_310.py", - "vllm_ascend/_310p/ops/fla/l2norm.py" - ] - }, - { - "commit": "4fbf24f3eeb0fdf1e97fbc6d8287813d8c193dfa", - "subject": "[BugFix] fix w4a8 weight loading of DSACP in sfa_v1.py (#10523)", - "topics": [ - "communication" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/attention/sfa_v1.py" - ] - }, - { - "commit": "f302cf759f639b90c343a0cde9c97b66fb7a89bd", - "subject": "[CI] Modify accuracy threshold for aime2025 dataset to 10 (#10881)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "tests/e2e/nightly/multi_node/internal_dp/config/DeepSeek-V3_2-W8A8-EP.yaml", - "tests/e2e/nightly/single_node/models/configs/Kimi-K2.5.yaml", - "tests/e2e/nightly/single_node/models/configs/MiniMax-M2.5-w8a8-QuaRot-A3.yaml", - "tests/e2e/nightly/single_node/models/configs/Qwen3.5-122B-A10B-W8A8-A3.yaml" - ] - }, - { - "commit": "ed8e8b77c00362ed55d40107e41cd7f224006b81", - "subject": "[BugFix]Resolve the bubble in the PP pipeline of PP+MTP and fix the verification of PP in the pd separation scenario. (#10855)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/distributed/kv_transfer/kv_p2p/mooncake_connector.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "8f8c56f9eba431d31bdd6c1bf86f40111d880b4d", - "subject": "[BugFix][SpecDecode] Fix low MTP acceptance rate for SFA+DSA_CP (MTP>1) (#10878)", - "topics": [ - "communication" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/attention/context_parallel/dsa_cp.py", - "vllm_ascend/attention/dsa_v1.py", - "vllm_ascend/attention/sfa_v1.py", - "vllm_ascend/spec_decode/llm_base_proposer.py" - ] - }, - { - "commit": "c09379425bd71ff6aba165f3b0c00894e9c9458a", - "subject": "[BugFix] add logic when KVCacheSpec group has compress ratio > 1 (#10865)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/worker/block_table.py" - ] - }, - { - "commit": "0d1d5bf01cbffdbaec7d2ba481b511c3714ac3b5", - "subject": "[BugFix] isolate KV cache from sleep cleanup with dedicated memory tag for hadamard matrix (#10775)", - "topics": [ - "communication" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/attention/context_parallel/dsa_cp.py", - "vllm_ascend/attention/dsa_v1.py", - "vllm_ascend/device_allocator/camem.py" - ] - }, - { - "commit": "5bd4e3950c2c1df9e8d4eabf6accaf6f071eef74", - "subject": "[BugFix](pcp): align decode classification with vLLM reorder logic (#10580)", - "topics": [ - "communication", - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/e2e/pull_request/four_card/context_parallel/test_accuracy.py", - "tests/ut/ops/test_gdn_attn_builder.py", - "tests/ut/worker/test_pcp_manager.py", - "vllm_ascend/_310p/model_runner_310p.py", - "vllm_ascend/attention/context_parallel/attention_cp.py", - "vllm_ascend/attention/context_parallel/dsa_cp.py", - "vllm_ascend/attention/context_parallel/sfa_cp.py", - "vllm_ascend/attention/mla_v1.py", - "vllm_ascend/attention/utils.py", - "vllm_ascend/ops/gdn_attn_builder.py", - "vllm_ascend/spec_decode/dflash_proposer.py", - "vllm_ascend/spec_decode/llm_base_proposer.py", - "vllm_ascend/worker/model_runner_v1.py", - "vllm_ascend/worker/pcp_utils.py" - ] - }, - { - "commit": "9076e997aba319093f0fbbf06cd5c4eec46132bf", - "subject": "Revert \"[BugFix][SpecDecode] Fix low MTP acceptance rate for SFA+DSA_CP (MTP>1) (#10825)\" (#10868)", - "topics": [ - "communication" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/attention/sfa_v1.py", - "vllm_ascend/spec_decode/llm_base_proposer.py" - ] - }, - { - "commit": "21e8dcf64f6f8795d9323a97a6c02431cd84d7db", - "subject": "[BugFix][SpecDecode] Fix low MTP acceptance rate for SFA+DSA_CP (MTP>1) (#10825)", - "topics": [ - "communication" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/attention/sfa_v1.py", - "vllm_ascend/spec_decode/llm_base_proposer.py" - ] - }, - { - "commit": "620555f18b6099419ecb11a5e10922e574684a20", - "subject": "[BugFix]: multiple config reading issues for env var and additional_config (#10786)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/ascend_forward_context.py", - "vllm_ascend/distributed/kv_transfer/kv_p2p/mooncake_hybrid_connector.py", - "vllm_ascend/patch/platform/patch_balance_schedule.py" - ] - }, - { - "commit": "07aedaa1ecae2a76ae838d8342e38d542bee0cff", - "subject": "[Bugfix][Feature] Adapt DSA-CP for sparse C8 MXFP8 dynamic quant in SFA (#10060)", - "topics": [ - "communication", - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/attention/sfa_v1.py" - ] - }, - { - "commit": "bdc152322e86002a5b887d2f993b5bda429af453", - "subject": "[Ops][Feature] Add Ascend 310P support for chunk_gated_delta_rule_fwd_h and chunk_fwd_o (#9458)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/_e2e_nightly_single_node.yaml", - ".github/workflows/_e2e_nightly_single_node_models.yaml", - ".github/workflows/_schedule_image_build.yaml", - ".github/workflows/_selected_tests.yaml", - ".github/workflows/push_build_csrc_cache.yaml", - ".github/workflows/schedule_release_code_and_wheel.yml", - "csrc/build_aclnn.sh", - "csrc/moe/chunk_fwd_o/op_host/chunk_fwd_o_def.cpp", - "csrc/moe/chunk_fwd_o/op_kernel/arch20/compat_310p.h", - "csrc/moe/chunk_fwd_o/op_kernel/arch20/epilogue/block/block_epilogue_gdn_fwdo_output.hpp", - "csrc/moe/chunk_fwd_o/op_kernel/arch20/epilogue/block/block_epilogue_gdn_fwdo_qkmask.hpp", - "csrc/moe/chunk_fwd_o/op_kernel/arch20/epilogue/gdn_fwd_o_epilogue_policies.hpp", - "csrc/moe/chunk_fwd_o/op_kernel/arch20/gemm/block/block_scheduler_gdn_fwd_o.hpp", - "csrc/moe/chunk_fwd_o/op_kernel/arch20/gemm/kernel/gdn_fwd_o_kernel.hpp", - "csrc/moe/chunk_fwd_o/op_kernel/arch22/epilogue/block/block_epilogue_gdn_fwdo_output.hpp", - "csrc/moe/chunk_fwd_o/op_kernel/arch22/epilogue/block/block_epilogue_gdn_fwdo_qkmask.hpp", - "csrc/moe/chunk_fwd_o/op_kernel/arch22/epilogue/gdn_fwd_o_epilogue_policies.hpp", - "csrc/moe/chunk_fwd_o/op_kernel/arch22/gemm/block/block_scheduler_gdn_fwd_o.hpp", - "csrc/moe/chunk_fwd_o/op_kernel/arch22/gemm/kernel/gdn_fwd_o_kernel.hpp", - "csrc/moe/chunk_fwd_o/op_kernel/chunk_fwd_o.cpp", - "csrc/moe/chunk_gated_delta_rule_fwd_h/op_host/chunk_gated_delta_rule_fwd_h_def.cpp", - "csrc/moe/chunk_gated_delta_rule_fwd_h/op_kernel/arch20/compat_310p.h", - "csrc/moe/chunk_gated_delta_rule_fwd_h/op_kernel/arch20/epilogue/block/block_epilogue_gdn_fwdh_update.hpp", - "csrc/moe/chunk_gated_delta_rule_fwd_h/op_kernel/arch20/epilogue/block/block_epilogue_gdn_fwdh_vnew.hpp", - "csrc/moe/chunk_gated_delta_rule_fwd_h/op_kernel/arch20/epilogue/gdn_fwd_h_epilogue_policies.hpp", - "csrc/moe/chunk_gated_delta_rule_fwd_h/op_kernel/arch20/gemm/block/block_scheduler_gdn_fwd_h.hpp", - "csrc/moe/chunk_gated_delta_rule_fwd_h/op_kernel/arch20/gemm/kernel/gdn_fwd_h_kernel.hpp", - "csrc/moe/chunk_gated_delta_rule_fwd_h/op_kernel/arch22/epilogue/block/block_epilogue_gdn_fwdh_update.hpp", - "csrc/moe/chunk_gated_delta_rule_fwd_h/op_kernel/arch22/epilogue/block/block_epilogue_gdn_fwdh_vnew.hpp", - "csrc/moe/chunk_gated_delta_rule_fwd_h/op_kernel/arch22/epilogue/gdn_fwd_h_epilogue_policies.hpp", - "csrc/moe/chunk_gated_delta_rule_fwd_h/op_kernel/arch22/gemm/block/block_scheduler_gdn_fwd_h.hpp", - "csrc/moe/chunk_gated_delta_rule_fwd_h/op_kernel/arch22/gemm/kernel/gdn_fwd_h_kernel.hpp", - "csrc/moe/chunk_gated_delta_rule_fwd_h/op_kernel/chunk_gated_delta_rule_fwd_h.cpp", - "csrc/moe/common/kernel_utils/block/block_mmad_pingpong_tla_multi.hpp", - "csrc/torch_binding.cpp", - "csrc/torch_binding_meta.cpp", - "tests/e2e/nightly/310p/single_node/ops/singlecard_ops/test_chunk_fwd_o_310.py", - "tests/e2e/nightly/310p/single_node/ops/singlecard_ops/test_chunk_gated_delta_rule_fwd_h_aclnn.py", - "tests/e2e/nightly/multi_node/scripts/run.sh", - "tests/ut/attention/a2/test_mla_cp_precision.py", - "tests/ut/attention/a2/test_sfa_cp_precision.py", - "tests/ut/ops/test_fused_moe.py", - "tests/ut/worker/a2/test_kvcomp_utils.py", - "vllm_ascend/_310p/ops/fla/__init__.py", - "vllm_ascend/_310p/ops/fla/chunk_gated_delta_rule.py", - "vllm_ascend/_310p/ops/fla/gdn_310.py" - ] - }, - { - "commit": "16e9156e9607e4f99ab52723d5048312df27e12a", - "subject": "[BugFix] Fix Index Cache when DSA_CP is Enable (#10827)", - "topics": [ - "deepseek-v4-dspark", - "communication" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/attention/sfa_v1.py" - ] - }, - { - "commit": "4467a23d084ee8c497e663d9caf85c5329851a78", - "subject": "[BugFix]Combine the features of PP and EPLB. (#10486)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/eplb/adaptor/test_vllm_adaptor.py", - "tests/ut/eplb/test_eplb_updator.py", - "vllm_ascend/eplb/adaptor/vllm_adaptor.py", - "vllm_ascend/eplb/core/eplb_device_transfer_loader.py", - "vllm_ascend/eplb/core/eplb_worker.py", - "vllm_ascend/eplb/eplb_updator.py", - "vllm_ascend/eplb/utils.py", - "vllm_ascend/ops/fused_moe/fused_moe.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "9788211ce67a18aa2f76bc3bcb176e378edc13a8", - "subject": "[Feature] 310P image support RC mode (#10550)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "Dockerfile.310p", - "Dockerfile.310p.openEuler" - ] - }, - { - "commit": "5f195235a793774b5b6635158513c0e8f2e2c63f", - "subject": "[Feature][EPLB] Optimize log2phy mapping for TP (#9239)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/eplb/core/a2/test_eplb_utils.py", - "vllm_ascend/eplb/core/eplb_utils.py", - "vllm_ascend/eplb/core/eplb_worker.py", - "vllm_ascend/ops/fused_moe/fused_moe.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "6329838c9fa47becfc05aa65ada8a03208cf037c", - "subject": "[Feature][P/D][Core][KVOffload] Support recompute KVCache offloading on P/D disaggregation decoder nodes (#10742)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/core/test_recompute_scheduler.py", - "tests/ut/kv_offload/test_recompute_cpu_offload.py", - "vllm_ascend/core/recompute_scheduler.py", - "vllm_ascend/distributed/kv_transfer/__init__.py", - "vllm_ascend/distributed/kv_transfer/ascend_multi_connector.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/recompute_cpu_offload/__init__.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/recompute_cpu_offload/manager.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/recompute_cpu_offload/metadata.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/recompute_cpu_offload/recompute_cpu_offload_connector.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/recompute_cpu_offload/worker.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "682cc2b938446b73a67e9ddfcc5ca2203a3f8088", - "subject": "[Misc] Standardize logging and prefixing across ascend modules (#10754)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/ascend_forward_context.py", - "vllm_ascend/batch_invariant.py", - "vllm_ascend/core/profiling_chunk_predictor.py", - "vllm_ascend/core/recompute_scheduler.py", - "vllm_ascend/core/scheduler_dynamic_batch.py", - "vllm_ascend/core/scheduler_profiling_chunk.py", - "vllm_ascend/device_allocator/camem.py", - "vllm_ascend/ops/flashcomm2_oshard_manager.py", - "vllm_ascend/ops/layer_shard_linear.py", - "vllm_ascend/ops/linear_op.py", - "vllm_ascend/worker/kvcomp_utils.py", - "vllm_ascend/worker/model_runner_v1.py", - "vllm_ascend/worker/pcp_utils.py", - "vllm_ascend/worker/v2/aclgraph_utils.py", - "vllm_ascend/worker/v2/input_batch.py", - "vllm_ascend/worker/v2/model_runner.py", - "vllm_ascend/worker/v2/spec_decode/eagle/aclgraph.py", - "vllm_ascend/worker/v2/spec_decode/eagle/speculator.py", - "vllm_ascend/worker/v2/utils.py", - "vllm_ascend/worker/worker.py" - ] - }, - { - "commit": "2539c015aa9754951e99c8478424b96bb329b8c5", - "subject": "[Feature] Add swiglustep_and_mul activation for SwigluStep models\uff08Step3.7 Flash\uff09 (#10491)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/ops/a3_2/test_activation.py", - "vllm_ascend/ops/activation.py", - "vllm_ascend/ops/fused_moe/moe_mlp.py" - ] - }, - { - "commit": "4356efc3d77761fa4e8724b134b445d6835bb62a", - "subject": "[CI] Modify the accuracy threshold of glm5 in aime2025 (#10789)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "tests/e2e/nightly/multi_node/internal_dp/config/GLM5_1-W8A8-A3-dual-nodes.yaml" - ] - }, - { - "commit": "24249a3032f461543308b1bb9eb1c2c9572d70f9", - "subject": "[Bugfix][RFork] Support quantized and draft transfers (#10128)", - "topics": [ - "deepseek-v4-dspark", - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/user_guide/feature_guide/layer_sharding.md", - "docs/source/user_guide/feature_guide/rfork.md", - "tests/ut/model_loader/rfork/test_rfork_loader.py", - "tests/ut/model_loader/rfork/test_transfer_backend.py", - "vllm_ascend/model_loader/rfork/rfork_loader.py", - "vllm_ascend/model_loader/rfork/rfork_worker.py", - "vllm_ascend/model_loader/rfork/seed_protocol.py", - "vllm_ascend/model_loader/rfork/seed_server.py", - "vllm_ascend/model_loader/rfork/transfer_backend.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "5dc65ddfb0aa66cb7f61e8d24c7720e519cee9c1", - "subject": "[Doc] Translated Doc files 2026-06-22 (#10764)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/locale/zh_CN/LC_MESSAGES/community/versioning_policy.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/Design_Documents/context_parallel.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/Design_Documents/cpu_binding.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/Design_Documents/eplb_swift_balancer.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/performance_and_debug/optimization_and_tuning.po", - "docs/source/locale/zh_CN/LC_MESSAGES/faqs.po", - "docs/source/locale/zh_CN/LC_MESSAGES/installation.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/features/dynamic_chunked_pipeline_parallel.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/DeepSeek-V3.2.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/GLM4.x.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/GLM5.2.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/GLM5.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Kimi-K2.6.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/configuration/additional_config.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/Ai_QoS_introduction_en.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/batch_invariance.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/cpu_binding.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/epd_disaggregation.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/eplb_swift_balancer.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/graph_mode.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/kv_pool.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/layerwise_kv_pool.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/quantization.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/sleep_mode.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/speculative_decoding.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/release_notes.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/support_matrix/supported_models.po" - ] - }, - { - "commit": "4f87db4e0499717ebd36e1925723a18ebf689a99", - "subject": "[Feature] Selectively collect eplb heat for prefill/decode requests. (#10647)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/user_guide/configuration/additional_config.md", - "tests/ut/ops/test_fused_moe.py", - "vllm_ascend/ascend_config.py", - "vllm_ascend/ascend_forward_context.py", - "vllm_ascend/eplb/eplb_updator.py", - "vllm_ascend/ops/fused_moe/fused_moe.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "92eeab2fe9a73394d2118de3c8459d69cc9d8e62", - "subject": "[Feature] vllm-ascend sleep mode optimization for residual memory usage (#9585)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "docs/source/user_guide/configuration/additional_config.md", - "docs/source/user_guide/feature_guide/sleep_mode.md", - "tests/ut/compilation/a2/test_acl_graph.py", - "tests/ut/device_allocator/test_sleep_mem_optimized.py", - "tests/ut/patch/worker/patch_common/test_patch_distributed.py", - "tests/ut/worker/a2/test_worker_v1.py", - "vllm_ascend/ascend_config.py", - "vllm_ascend/compilation/acl_graph.py", - "vllm_ascend/device_allocator/sleep_mem_optimized.py", - "vllm_ascend/ops/fused_moe/token_dispatcher.py", - "vllm_ascend/patch/worker/patch_distributed.py", - "vllm_ascend/worker/worker.py" - ] - }, - { - "commit": "1eaaf34e6c8b5439b4964e8c083d6e5507d3c009", - "subject": "[BugFix]Fix multi-DP deadlock for extract_hidden_states proposer on MoE targets (#9689)", - "topics": [ - "communication" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/spec_decode/test_extract_hidden_states_proposer.py", - "tests/ut/test_utils.py", - "vllm_ascend/spec_decode/extract_hidden_states_proposer.py", - "vllm_ascend/utils.py" - ] - }, - { - "commit": "9d1c855e900f251e1672bdce0ba978260cafac77", - "subject": "[Feature]support qwen3.5+pcp+flashcomm1 (#10596)", - "topics": [ - "communication", - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/attention/context_parallel/attention_cp.py", - "vllm_ascend/ops/fused_moe/prepare_finalize.py", - "vllm_ascend/ops/gdn_attn_builder.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "8638a6950e4b520d0ad29dcff15837266799e517", - "subject": "[Doc] Update KV Pool document regarding SSD, troubleshooting and dependencies (#10617)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/user_guide/feature_guide/kv_pool.md" - ] - }, - { - "commit": "9c974424d6f3e7d25c654cd50ce708fa22bb25b9", - "subject": "[DOC]fix glm5.2 doc (#10618)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/tutorials/models/GLM5.2.md" - ] - }, - { - "commit": "fe27bf428eba925ea738b07847012def75ded60b", - "subject": "[Doc][Misc] Improve readability and fix typos in documentation (#10588)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/tutorials/models/GLM5.md" - ] - }, - { - "commit": "2d81d770bf509075ed7891c2ca445918e9805805", - "subject": "[BugFix][dsv4-flash] add dsa-cp support for a5 (#10026)", - "topics": [ - "deepseek-v4-dspark", - "communication" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/attention/context_parallel/dsa_cp.py" - ] - }, - { - "commit": "a4440739a78c1820a922c1de795431a483eed366", - "subject": "[Refactor][Patch] Remove redundant DeepSeek V4 thinking patch (#10612)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/patch/platform/test_deepseek_v4_thinking.py", - "vllm_ascend/patch/__init__.py", - "vllm_ascend/patch/platform/__init__.py", - "vllm_ascend/patch/platform/patch_deepseek_v4_thinking.py" - ] - }, - { - "commit": "641678fe755ec729052483213159446de4dd5ab0", - "subject": "[Doc]fix GLM5.2-w8v8 to GLM-5.2-w8v8 (#10625)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/tutorials/models/GLM5.2.md" - ] - }, - { - "commit": "80e9cea7adff3b1994d5f74d26227185280133a2", - "subject": "[Feature] ViT encoder ACL graph (#10549)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/workflows/scripts/test_config.yaml", - "tests/e2e/coverage.md", - "tests/e2e/pull_request/two_card/test_qwen3_6_27b_fia.py", - "vllm_ascend/ops/mm_encoder_attention.py", - "vllm_ascend/worker/encoder_acl_graph.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "c8b905b2c93157a159cd1b37535cc0e65fd8b45f", - "subject": "[Ops][Misc] Modify triton ops logs (#10482)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/ops/triton/activation/swiglu_quant.py", - "vllm_ascend/ops/triton/fla/chunk.py", - "vllm_ascend/ops/triton/fla/cumsum.py", - "vllm_ascend/ops/triton/fla/l2norm.py", - "vllm_ascend/ops/triton/fla/layernorm_guard.py", - "vllm_ascend/ops/triton/fla/utils.py", - "vllm_ascend/ops/triton/gdn_chunk_meta.py", - "vllm_ascend/ops/triton/layernorm_gated.py", - "vllm_ascend/ops/triton/mamba/causal_conv1d.py", - "vllm_ascend/ops/triton/rms_norm.py" - ] - }, - { - "commit": "d8e46e7fa8c88c1530fa8962889fb6e6c6accb5a", - "subject": "[Doc] Add GLM-5.2 multi-node deployment guides for Atlas 800 A2 (#10582)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/tutorials/models/GLM5.2.md" - ] - }, - { - "commit": "ea6e8d2200fd5fcdd0dcf2e374c4bae60170031a", - "subject": "[Feature] Support extract_hidden_states for hybrid models on Ascend NPU (#9763)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/e2e/pull_request/one_card/spec_decode/test_extract_hidden_states.py", - "vllm_ascend/patch/platform/patch_mamba_config.py", - "vllm_ascend/utils.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "11803e307067b36772f96b67997cc1fc2325c998", - "subject": "\"[Ascend950] [BugFix]Fix the issue where MOE model output is garbled when communicating with allgether while SP is disabled.\u201c (#10557)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/ops/fused_moe/fused_moe.py" - ] - }, - { - "commit": "789983f4c05c08e6fc4c09303e3b758020fce43b", - "subject": "[Refactor][Op] Align fused_gdn_gating API and softplus semantics (#10553)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "csrc/attention/fused_gdn_gating/fused_gdn_gating_torch_adpt.h", - "csrc/attention/fused_gdn_gating/op_host/fused_gdn_gating_def.cpp", - "csrc/attention/fused_gdn_gating/op_host/fused_gdn_gating_tiling.cpp", - "csrc/attention/fused_gdn_gating/op_host/fused_gdn_gating_tiling_utils.h", - "csrc/attention/fused_gdn_gating/op_host/op_api/aclnn_fused_gdn_gating.cpp", - "csrc/attention/fused_gdn_gating/op_host/op_api/aclnn_fused_gdn_gating.h", - "csrc/attention/fused_gdn_gating/op_host/op_api/fused_gdn_gating.cpp", - "csrc/attention/fused_gdn_gating/op_host/op_api/fused_gdn_gating.h", - "csrc/attention/fused_gdn_gating/op_kernel/fused_gdn_gating.cpp", - "csrc/attention/fused_gdn_gating/op_kernel/fused_gdn_gating.h", - "csrc/attention/fused_gdn_gating/op_kernel/fused_gdn_gating_tiling_data.h", - "csrc/torch_binding.cpp", - "csrc/torch_binding_meta.cpp", - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_fused_gdn_gating.py", - "vllm_ascend/device/device_op.py" - ] - }, - { - "commit": "3ac10c803a33aaced122e0c74e16425723af5251", - "subject": "[doc]add GLM5.2 documents (#10544)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/tutorials/models/GLM5.2.md", - "docs/source/tutorials/models/index.md", - "docs/source/user_guide/support_matrix/supported_models.md" - ] - }, - { - "commit": "2b3910fc781c750456490ba714e1a741078f32e5", - "subject": "[Attention] [BugFix]Enable multistream_dsv4_dsa_overlap and remove redundant code (#10518)", - "topics": [ - "deepseek-v4-dspark", - "communication" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/user_guide/configuration/additional_config.md", - "tests/e2e/nightly/single_node/models/configs/DeepSeek-V4-Flash-W8A8-A3.yaml", - "tests/e2e/pull_request/four_card/long_sequence/test_basic.py", - "vllm_ascend/ascend_config.py", - "vllm_ascend/attention/context_parallel/dsa_cp.py", - "vllm_ascend/attention/dsa_v1.py", - "vllm_ascend/ops/dsa.py" - ] - }, - { - "commit": "344931832c3f7fb14ce7fe3f8d5c9d8a7b61e2fa", - "subject": "[Performance] Optimize split_qkv_tp_rmsnorm_rope with grid stride andh loading (#9830)", - "topics": [ - "communication" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/ops/triton/linearnorm/split_qkv_tp_rmsnorm_rope.py" - ] - }, - { - "commit": "13fd5983f7f43ba97e56c5c016448851bc96e602", - "subject": "[Feature] Mooncake connectot support dsv4 (#10342)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/kv_offload/test_mooncake_connector.py", - "tests/ut/kv_offload/test_mooncake_layerwise_connector.py", - "vllm_ascend/core/recompute_scheduler.py", - "vllm_ascend/distributed/kv_transfer/kv_p2p/mooncake_connector.py", - "vllm_ascend/distributed/kv_transfer/kv_p2p/mooncake_layerwise_connector.py" - ] - }, - { - "commit": "e88c291e4e87b8aa9ed592ed21aeca533552990f", - "subject": "[Feature] SSD support multiple DP in each machine (#10477)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/distributed/test_parallel_state.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/ascend_store/backend/mooncake_backend.py", - "vllm_ascend/distributed/parallel_state.py" - ] - }, - { - "commit": "bb5edc329eefc72bfee067b5d40fdc4721d9b379", - "subject": "[CI] Added the nightly test case DeepSeek-V4-Flash-W8A8-A3 and fixed the failed weekly test case. (#10543)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/schedule_nightly_test_a3.yaml", - "tests/e2e/nightly/single_node/models/configs/DeepSeek-V4-Flash-W8A8-A3.yaml", - "tests/e2e/weekly/single_node/configs/Kimi-K2.5.yaml" - ] - }, - { - "commit": "fad646478dff693840cd2e3d67098cc8b51142a6", - "subject": "[Feature] Add KV consumer partial-group caching for hybrid Mamba models (#10009)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/core/recompute_scheduler.py", - "vllm_ascend/distributed/kv_transfer/kv_p2p/mooncake_connector.py", - "vllm_ascend/distributed/kv_transfer/kv_p2p/mooncake_layerwise_connector.py", - "vllm_ascend/patch/__init__.py", - "vllm_ascend/patch/platform/patch_kv_cache_coordinator.py" - ] - }, - { - "commit": "5e45ac996100ae48dec8b94795ec5ce52f7feb8a", - "subject": "[BugFix] fix qwen3.5 accuracy bug when sett cp-kv-cache-interleave-si\u2026 (#10442)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/attention/context_parallel/attention_cp.py", - "vllm_ascend/ops/fused_moe/prepare_finalize.py", - "vllm_ascend/worker/block_table.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "43c7c9a48344651081f6736eb7bd151c05b58f55", - "subject": "[Misc] Main2main upgrade vllm to v0.22.1 (#10476)", - "topics": [ - "deepseek-v4-dspark", - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/vllm-release-tag.commit", - ".github/workflows/schedule_image_build_and_push.yaml", - "Dockerfile", - "Dockerfile.310p", - "Dockerfile.310p.openEuler", - "Dockerfile.a3", - "Dockerfile.a3.openEuler", - "Dockerfile.a5", - "Dockerfile.a5.openEuler", - "Dockerfile.openEuler", - "README.md", - "README.zh.md", - "docs/source/community/slash-commands.md", - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_fused_qkvzba_split_reshape_cat.py", - "tests/e2e/pull_request/one_card/model_runner_v2/test_basic.py", - "tests/e2e/pull_request/one_card/spec_decode/utils.py", - "tests/e2e/pull_request/one_card/test_guided_decoding.py", - "tests/ut/patch/platform/test_patch_glm47_tool_call_parser.py", - "tests/ut/patch/platform/test_patch_tool_choice_none_content.py", - "tests/ut/patch/platform/test_prefix_cache_cp_patches.py", - "tests/ut/spec_decode/test_extract_hidden_states_proposer.py", - "tests/ut/test_compressed_prefix_cache.py", - "vllm_ascend/_310p/ops/fla/gdn_310.py", - "vllm_ascend/core/recompute_scheduler.py", - "vllm_ascend/core/single_type_kv_cache_manager.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/cpu_offload/cpu_kv_cache_manager.py", - "vllm_ascend/distributed/utils.py", - "vllm_ascend/eplb/core/eplb_utils.py", - "vllm_ascend/models/deepseek_v4.py", - "vllm_ascend/ops/bailing_moe_linear_attn.py", - "vllm_ascend/ops/fused_moe/fused_moe.py", - "vllm_ascend/ops/gdn.py", - "vllm_ascend/patch/platform/patch_kv_cache_coordinator.py", - "vllm_ascend/patch/platform/patch_mamba_manager.py", - "vllm_ascend/patch/platform/patch_tool_choice_none_content.py", - "vllm_ascend/patch/worker/__init__.py", - "vllm_ascend/patch/worker/patch_idex_310.py", - "vllm_ascend/patch/worker/patch_minimax_m2.py", - "vllm_ascend/patch/worker/patch_minimax_m2_linear_attn.py", - "vllm_ascend/patch/worker/patch_qwen3_5.py", - "vllm_ascend/spec_decode/llm_base_proposer.py", - "vllm_ascend/worker/model_runner_v1.py", - "vllm_ascend/worker/v2/block_table.py", - "vllm_ascend/worker/v2/model_runner.py", - "vllm_ascend/worker/v2/spec_decode/eagle/aclgraph.py", - "vllm_ascend/worker/v2/spec_decode/eagle/speculator.py", - "vllm_ascend/worker/v2/states.py", - "vllm_ascend/worker/worker.py" - ] - }, - { - "commit": "e9ad553c0af677aa4f8a40971822a57b4b007714", - "subject": "[Doc] Optimize documentation for context parallel, EPLB, and speculative decoding; remove Multi-Token Prediction guide (#10332)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/developer_guide/Design_Documents/context_parallel.md", - "docs/source/developer_guide/Design_Documents/eplb_swift_balancer.md", - "docs/source/tutorials/models/GLM5.md", - "docs/source/user_guide/configuration/additional_config.md", - "docs/source/user_guide/feature_guide/Multi_Token_Prediction.md", - "docs/source/user_guide/feature_guide/eplb_swift_balancer.md", - "docs/source/user_guide/feature_guide/index.md", - "docs/source/user_guide/feature_guide/speculative_decoding.md" - ] - }, - { - "commit": "1078d828e0a854819d80320bdc04460cd51f9a96", - "subject": "[Quantization][BugFix] dsv4 quant_model_substr_mapping key fix (#10458)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/quantization/modelslim_config.py" - ] - }, - { - "commit": "849a92e60b1e7b27036d90e0e5f5002c880fb845", - "subject": "[Bugfix][DSV4] Enhance QLI workspace size calculation for ASCEND 950 (#10041)", - "topics": [ - "deepseek-v4-dspark", - "communication" - ], - "runtime_patch_match": false, - "paths": [ - "csrc/attention/quant_lightning_indexer/op_host/quant_lightning_indexer_tiling.cpp", - "csrc/attention/quant_lightning_indexer/op_kernel/arch35/quant_lightning_indexer_kernel.h" - ] - }, - { - "commit": "8afdf356f6a2496bedfc538253366ef1a8c0d9aa", - "subject": "[Ascend950][Feature] Add support for W8A8FP8 dynamic quantization on Ascend950 (#10236)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/quantization/methods/test_w8a8fp8_dynamic.py", - "vllm_ascend/device/device_op.py", - "vllm_ascend/ops/fused_moe/moe_mlp.py", - "vllm_ascend/ops/fused_moe/moe_runtime_args.py", - "vllm_ascend/ops/fused_moe/moe_stage_params.py", - "vllm_ascend/ops/fused_moe/token_dispatcher.py", - "vllm_ascend/quantization/compressed_tensors_config.py", - "vllm_ascend/quantization/methods/__init__.py", - "vllm_ascend/quantization/methods/w8a8_dynamic.py", - "vllm_ascend/quantization/methods/w8a8fp8_dynamic.py", - "vllm_ascend/quantization/quant_type.py" - ] - }, - { - "commit": "72797cfa159e76a00493eda27c21fa4bb9c8fe9c", - "subject": "[Misc] Main2Main 0605 (#10250)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - ".github/vllm-main-verified.commit", - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_fused_qkvzba_split_reshape_cat.py", - "tests/e2e/pull_request/four_card/test_data_parallel_tp2.py", - "tests/e2e/pull_request/four_card/test_qwen3_next.py", - "tests/e2e/pull_request/one_card/test_guided_decoding.py", - "tests/ut/patch/platform/test_patch_glm47_tool_call_parser.py", - "tests/ut/patch/platform/test_patch_tool_choice_none_content.py", - "tests/ut/patch/platform/test_prefix_cache_cp_patches.py", - "tests/ut/test_compressed_prefix_cache.py", - "vllm_ascend/_310p/kv_block_zeroer.py", - "vllm_ascend/attention/context_parallel/attention_cp.py", - "vllm_ascend/attention/context_parallel/common_cp.py", - "vllm_ascend/attention/context_parallel/mla_cp.py", - "vllm_ascend/core/recompute_scheduler.py", - "vllm_ascend/core/single_type_kv_cache_manager.py", - "vllm_ascend/distributed/kv_transfer/kv_p2p/mooncake_connector.py", - "vllm_ascend/distributed/kv_transfer/kv_p2p/mooncake_layerwise_connector.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/ascend_store/pool_worker.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/cpu_offload/cpu_kv_cache_manager.py", - "vllm_ascend/distributed/utils.py", - "vllm_ascend/ops/bailing_moe_linear_attn.py", - "vllm_ascend/patch/platform/patch_kv_cache_coordinator.py", - "vllm_ascend/patch/platform/patch_mamba_manager.py", - "vllm_ascend/patch/platform/patch_tool_choice_none_content.py", - "vllm_ascend/patch/worker/patch_minimax_m2.py", - "vllm_ascend/patch/worker/patch_minimax_m2_linear_attn.py", - "vllm_ascend/platform.py", - "vllm_ascend/spec_decode/llm_base_proposer.py", - "vllm_ascend/worker/utils.py", - "vllm_ascend/worker/v2/model_runner.py", - "vllm_ascend/worker/v2/spec_decode/eagle/aclgraph.py", - "vllm_ascend/worker/v2/spec_decode/eagle/speculator.py", - "vllm_ascend/worker/v2/states.py", - "vllm_ascend/worker/worker.py" - ] - }, - { - "commit": "1a44dc5f2610e87a9b0f6ddc1182c2834a3af577", - "subject": "[BugFix][SpecDecode] Sanitize MTP placeholders before model forward (#10307)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/sample/test_rejection_sampler.py", - "tests/ut/worker/a2/test_model_runner_v1.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "49a1fed17ee148aa11c7a70f59ffc068dafa387b", - "subject": "[BugFix][Worker] Skip DSV4 pre-KV graph memory profiling (#10369)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/worker/a2/test_worker_multi_instance.py", - "vllm_ascend/worker/worker.py" - ] - }, - { - "commit": "d93ae1df058f3fbece683256056a5b5795d80630", - "subject": "[Misc][Ops]Revert \"add trion ops log (#10069)\" (#10414)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/ops/triton/activation/swiglu_quant.py", - "vllm_ascend/ops/triton/bincount.py", - "vllm_ascend/ops/triton/fla/chunk.py", - "vllm_ascend/ops/triton/fla/fused_qkvzba_split_reshape.py", - "vllm_ascend/ops/triton/fused_gdn_gating.py", - "vllm_ascend/ops/triton/layernorm_gated.py", - "vllm_ascend/ops/triton/linearnorm/split_qkv_rmsnorm_mrope.py", - "vllm_ascend/ops/triton/linearnorm/split_qkv_rmsnorm_rope.py", - "vllm_ascend/ops/triton/linearnorm/split_qkv_tp_rmsnorm_rope.py", - "vllm_ascend/ops/triton/mamba/causal_conv1d.py", - "vllm_ascend/ops/triton/mamba/lightning_attn.py", - "vllm_ascend/ops/triton/mul_add.py", - "vllm_ascend/ops/triton/muls_add.py", - "vllm_ascend/ops/triton/penalty.py", - "vllm_ascend/ops/triton/rms_norm.py", - "vllm_ascend/ops/triton/rope.py", - "vllm_ascend/ops/triton/triton_utils.py" - ] - }, - { - "commit": "bbeba20ce768023d4a78c8bf224ac5d3d1b1256d", - "subject": "[Feature][DSv4] Support compressor block size [32,64,128] to improve automatic prefix cache hit rate (#10354)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/attention/a2/test_sfa_v1.py", - "vllm_ascend/attention/dsa_v1.py", - "vllm_ascend/models/deepseek_v4.py", - "vllm_ascend/models/layer/attention/layer.py", - "vllm_ascend/utils.py" - ] - }, - { - "commit": "b05d46f040b01daa3a30caece698439799e9fe0d", - "subject": "[CI][Test] Add MTP speculative decoding test for DeepSeek V4 (#10392)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "tests/e2e/pull_request/four_card/test_deepseek_v4.py" - ] - }, - { - "commit": "971d50b3f982fd2ba7d9ffdee35e985fc54cd808", - "subject": "[BugFix][Attention] Fix DSA v1 W8A8 dynamic conflict (#9476)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/attention/dsa_v1.py" - ] - }, - { - "commit": "eef0b36b3cac8f6ae5a23ac56036ec678ad6ed79", - "subject": "[BugFix][PrefixCache] Fix DeepSeek-V4 compressed prefix lookup (#10297)", - "topics": [ - "deepseek-v4-dspark", - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/scripts/test_config.yaml", - "tests/ut/test_compressed_prefix_cache.py", - "vllm_ascend/core/single_type_kv_cache_manager.py", - "vllm_ascend/patch/platform/patch_kv_cache_coordinator.py" - ] - }, - { - "commit": "cebff41ef28ecf08a0ae9ad7542b717c370edd79", - "subject": "[Refactor][Model] Move DeepSeek V4 cache hooks into model (#10327)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/models/deepseek_v4.py", - "vllm_ascend/ops/dsa.py", - "vllm_ascend/patch/worker/__init__.py", - "vllm_ascend/patch/worker/patch_deepseek_compressor.py" - ] - }, - { - "commit": "3dff82b574771711f27aa9a37418beadccf8aa0c", - "subject": "[BugFix][Misc] Remove DeepSeek V4 patch env (#10333)", - "topics": [ - "deepseek-v4-dspark", - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/tutorials/models/DeepSeek-V4-Flash.md", - "docs/source/tutorials/models/DeepSeek-V4-Pro.md", - "tests/e2e/coverage.md", - "tests/e2e/generate_coverage_md.py", - "tests/e2e/nightly/single_node/models/configs/DeepSeek-V4-Flash-W8A8-A3.yaml", - "tests/e2e/pull_request/four_card/long_sequence/test_basic.py", - "tests/e2e/pull_request/four_card/test_deepseek_v4.py", - "tests/ut/patch/platform/test_prefix_cache_cp_patches.py", - "vllm_ascend/envs.py", - "vllm_ascend/patch/platform/__init__.py", - "vllm_ascend/patch/platform/patch_kv_cache_coordinator.py" - ] - }, - { - "commit": "22b9098719daa134b10340b0cf26fe6a4798a062", - "subject": "[BugFix][CI] Fix GLM5.1-W8A8 MTP load weight error when vLLM is updated to v0.21.0 (#10317)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/patch/__init__.py", - "vllm_ascend/patch/worker/patch_deepseek_mtp.py" - ] - }, - { - "commit": "969baeda89c88666d123e1b55f99235b07051a69", - "subject": "[Feature][310p]Qwen3.5 Support MTP and graphmode (#10309)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/e2e/pull_request/one_card/_310p/test_spec_decode_mtp_310p.py", - "tests/ut/_310p/attention/test_attention_v1_310.py", - "tests/ut/_310p/sample/test_sampler_310.py", - "tests/ut/_310p/test_kv_block_zeroer_310p.py", - "vllm_ascend/_310p/attention/attention_v1.py", - "vllm_ascend/_310p/attention/metadata_builder.py", - "vllm_ascend/_310p/kv_block_zeroer.py", - "vllm_ascend/_310p/model_runner_310p.py", - "vllm_ascend/_310p/ops/fla/gdn_310.py", - "vllm_ascend/_310p/sample/rejection_sampler.py", - "vllm_ascend/_310p/sample/sampler.py", - "vllm_ascend/_310p/spec_decode/__init__.py", - "vllm_ascend/_310p/spec_decode/llm_base_proposer_310.py", - "vllm_ascend/compilation/graph_fusion_pass_manager.py", - "vllm_ascend/patch/worker/patch_idex_310.py" - ] - }, - { - "commit": "6ce137f8765e6b85ea59b40eb14acc8ee8fd1e17", - "subject": "[Bugfix] use endswith for .scale parameter matching in MTP model (#10316)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/models/deepseek_v4_mtp.py" - ] - }, - { - "commit": "0b5223c5cb08aa6b4f2a1ec4a12568ac9ff6bb3e", - "subject": "[BugFix] Preserve FP32 MM encoder attention support (#10200)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/device/test_device_op.py", - "vllm_ascend/device/device_op.py" - ] - }, - { - "commit": "5f76de9682bdab9292a48d4f51793f6d40883456", - "subject": "[Misc][Ops] Fix a bug inside a triton operator fused_sigmoid_gating_delta_rule_update_kernel (#10016)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/ops/triton/fla/sigmoid_gating.py" - ] - }, - { - "commit": "976d5766af9d04f958a5e47affd78255ee8ca585", - "subject": "[BugFix][DeepSeek] Preserve no-group dequant SwiGLU clamp (#10258)", - "topics": [ - "deepseek-v4-dspark", - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "csrc/moe/dequant_swiglu_quant/op_host/dequant_swiglu_quant_tiling.cpp", - "csrc/torch_binding.cpp" - ] - }, - { - "commit": "5d652de7a8b58e0462ce34964c5173f20fb81f8f", - "subject": "[BugFix]fix A3 DSV4 precision (#10221)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/attention/dsa_v1.py", - "vllm_ascend/device/device_op.py" - ] - }, - { - "commit": "fcf5a1d549896070d9baf2e58240503a4ca24fd3", - "subject": "[BugFix] [Kimi] Fix dense draft model (Kimi-MLA) misidentified as MoE (#10266)", - "topics": [ - "communication" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/utils.py" - ] - }, - { - "commit": "871759a2319f4c55e40880f6657ba5fd38866f58", - "subject": "[Performance](dsa_cp): recompute local_seq_lens for cpu to remove D2H (#9892)", - "topics": [ - "communication" - ], - "runtime_patch_match": false, - "paths": [ - "tests/e2e/pull_request/four_card/long_sequence/test_basic.py", - "vllm_ascend/attention/context_parallel/dsa_cp.py" - ] - }, - { - "commit": "97c7f0dadf595bb766236a447735a2448dd40874", - "subject": "[Misc][Ops]add trion ops log (#10069)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/ops/triton/activation/swiglu_quant.py", - "vllm_ascend/ops/triton/bincount.py", - "vllm_ascend/ops/triton/fla/chunk.py", - "vllm_ascend/ops/triton/fla/fused_qkvzba_split_reshape.py", - "vllm_ascend/ops/triton/fused_gdn_gating.py", - "vllm_ascend/ops/triton/layernorm_gated.py", - "vllm_ascend/ops/triton/linearnorm/split_qkv_rmsnorm_mrope.py", - "vllm_ascend/ops/triton/linearnorm/split_qkv_rmsnorm_rope.py", - "vllm_ascend/ops/triton/linearnorm/split_qkv_tp_rmsnorm_rope.py", - "vllm_ascend/ops/triton/mamba/causal_conv1d.py", - "vllm_ascend/ops/triton/mamba/lightning_attn.py", - "vllm_ascend/ops/triton/mul_add.py", - "vllm_ascend/ops/triton/muls_add.py", - "vllm_ascend/ops/triton/penalty.py", - "vllm_ascend/ops/triton/rms_norm.py", - "vllm_ascend/ops/triton/rope.py", - "vllm_ascend/ops/triton/triton_utils.py" - ] - }, - { - "commit": "9914ffa27058165634dd9f46d37dc75debe8ce10", - "subject": "[Misc] Enhance error messages and logging for better clarity and traceability (#10201)", - "topics": [ - "communication", - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/attention/context_parallel/dsa_cp.py", - "vllm_ascend/eplb/core/eplb_device_transfer_loader.py", - "vllm_ascend/eplb/core/eplb_utils.py", - "vllm_ascend/eplb/core/eplb_worker.py", - "vllm_ascend/eplb/core/policy/policy_factory.py", - "vllm_ascend/eplb/core/policy/policy_flashlb.py", - "vllm_ascend/eplb/eplb_updator.py", - "vllm_ascend/ops/fused_moe/fused_moe.py", - "vllm_ascend/ops/fused_moe/moe_comm_method.py", - "vllm_ascend/sample/rejection_sampler.py", - "vllm_ascend/sample/sampler.py", - "vllm_ascend/spec_decode/llm_base_proposer.py" - ] - }, - { - "commit": "7cb4cd119c7772b0286feffa6de41527f4c864c5", - "subject": "[BugFix] Trim SWA transfer blocks before clipping (#10254)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/kv_offload/test_mooncake_hybrid_connector.py", - "vllm_ascend/distributed/kv_transfer/kv_p2p/mooncake_hybrid_connector.py" - ] - }, - { - "commit": "127e560eb5a95ed11b23a98188f728657a56412b", - "subject": "[CI] Modify the aiscench version and take some unstable test cases offline. (#10219)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/_e2e_nightly_single_node.yaml", - ".github/workflows/dockerfiles/Dockerfile.nightly.a2", - ".github/workflows/dockerfiles/Dockerfile.nightly.a3", - ".github/workflows/schedule_nightly_test_a2.yaml", - ".github/workflows/schedule_nightly_test_a3.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/DeepSeek-V3_2-W8A8-EP.yaml", - "tests/e2e/nightly/single_node/models/configs/DeepSeek-V3.2-W8A8.yaml" - ] - }, - { - "commit": "77ff28652b329796450f5448c7ab51b054697535", - "subject": "[Feature][SpecDecode] fix Magic MTP and add Entropy Verify (#9772)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/user_guide/configuration/additional_config.md", - "docs/source/user_guide/feature_guide/speculative_decoding.md", - "tests/ut/sample/test_rejection_sampler.py", - "vllm_ascend/ascend_config.py", - "vllm_ascend/ops/triton/reject_sample.py", - "vllm_ascend/sample/rejection_sampler.py" - ] - }, - { - "commit": "fc0b9e3540bc85dfe355ebb244c80965ab39a2b2", - "subject": "[Feature][310p] support compressed mask for 310P flashattention and pagedattention-split-fuse (#10181)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/_310p/attention/test_attention_mask_310.py", - "tests/ut/_310p/attention/test_attention_v1_310.py", - "vllm_ascend/_310p/attention/attention_mask.py", - "vllm_ascend/_310p/attention/attention_v1.py" - ] - }, - { - "commit": "c0d1bc215db8808979be446e0c74621e4b608e26", - "subject": "[BugFix]fix w4a8 mxfp quantization in shared experts (#10153)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/ops/fused_moe/fused_moe.py", - "vllm_ascend/quantization/methods/w4a8_mxfp4.py", - "vllm_ascend/quantization/methods/w8a8_mxfp8.py" - ] - }, - { - "commit": "2b4a9daa66e78e156d72b2c7eb36f82ccef3735a", - "subject": "[Misc] Main2main Upgrade vLLM to v0.21.0 (#9835)", - "topics": [ - "deepseek-v4-dspark", - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/workflows/pr_test.yaml", - ".github/workflows/schedule_vllm_e2e_test.yaml", - "Dockerfile", - "Dockerfile.310p", - "Dockerfile.310p.openEuler", - "Dockerfile.a3", - "Dockerfile.a3.openEuler", - "Dockerfile.a5", - "Dockerfile.a5.openEuler", - "Dockerfile.openEuler", - "README.md", - "README.zh.md", - "docs/source/conf.py", - "tests/e2e/pull_request/four_card/long_sequence/test_basic.py", - "tests/e2e/pull_request/one_card/model_runner_v2/test_basic.py", - "tests/e2e/pull_request/one_card/pooling/test_classification.py", - "tests/e2e/pull_request/one_card/spec_decode/utils.py", - "tests/e2e/pull_request/two_card/test_moe_routing_replay.py", - "tests/ut/spec_decode/test_extract_hidden_states_proposer.py", - "tests/ut/test_platform.py", - "vllm_ascend/_310p/ops/fla/gdn_310.py", - "vllm_ascend/core/scheduler_profiling_chunk.py", - "vllm_ascend/core/single_type_kv_cache_manager.py", - "vllm_ascend/eplb/core/eplb_utils.py", - "vllm_ascend/models/deepseek_v4.py", - "vllm_ascend/ops/fused_moe/fused_moe.py", - "vllm_ascend/ops/gdn.py", - "vllm_ascend/patch/__init__.py", - "vllm_ascend/patch/platform/patch_mla_prefill_backend.py", - "vllm_ascend/patch/worker/__init__.py", - "vllm_ascend/patch/worker/patch_deepseek_compressor.py", - "vllm_ascend/patch/worker/patch_deepseek_mtp.py", - "vllm_ascend/patch/worker/patch_idex_310.py", - "vllm_ascend/patch/worker/patch_qwen3_5.py", - "vllm_ascend/patch/worker/patch_triton.py", - "vllm_ascend/patch/worker/patch_v2/patch_attn_utils.py", - "vllm_ascend/worker/model_runner_v1.py", - "vllm_ascend/worker/v2/block_table.py", - "vllm_ascend/worker/v2/spec_decode/eagle/speculator.py", - "vllm_ascend/worker/worker.py" - ] - }, - { - "commit": "d8c7fe580cb07c636406a93726aaba80a0c08727", - "subject": "[Refactor] improve log quality for better debugging (#10086)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/distributed/kv_transfer/test_kv_transfer_failures.py", - "tests/ut/test_platform.py", - "vllm_ascend/ascend_config.py", - "vllm_ascend/attention/mla_v1.py", - "vllm_ascend/attention/sfa_v1.py", - "vllm_ascend/cpu_binding.py", - "vllm_ascend/distributed/device_communicators/pyhccl_wrapper.py", - "vllm_ascend/distributed/kv_transfer/kv_p2p/mooncake_connector.py", - "vllm_ascend/distributed/kv_transfer/kv_p2p/mooncake_hybrid_connector.py", - "vllm_ascend/distributed/kv_transfer/kv_p2p/mooncake_layerwise_connector.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/ascend_store/backend/memcache_backend.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/ascend_store/backend/mooncake_backend.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/ascend_store/backend/yuanrong_backend.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/ascend_store/kv_transfer.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/ascend_store/pool_worker.py", - "vllm_ascend/platform.py", - "vllm_ascend/utils.py" - ] - }, - { - "commit": "24f42a645f6edf00aeb53051ea195c3347cbf11c", - "subject": "[Performance]irregular mask build for pcp/dcp+spec decode (#9678)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/attention/a2/test_attention_cp.py", - "tests/ut/attention/a2/test_attention_cp_precision.py", - "tests/ut/attention/a2/test_mla_cp.py", - "tests/ut/attention/a2/test_mla_cp_precision.py", - "tests/ut/compilation/a2/test_acl_graph.py", - "vllm_ascend/attention/attention_v1.py", - "vllm_ascend/attention/context_parallel/attention_cp.py", - "vllm_ascend/attention/context_parallel/common_cp.py", - "vllm_ascend/attention/context_parallel/mla_cp.py", - "vllm_ascend/attention/mla_v1.py", - "vllm_ascend/attention/utils.py", - "vllm_ascend/spec_decode/llm_base_proposer.py", - "vllm_ascend/worker/model_runner_v1.py", - "vllm_ascend/worker/pcp_utils.py" - ] - }, - { - "commit": "f5c97749fa36f80648a56198e60800f3a0dac3ff", - "subject": "[PrefixCache][BugFix] Fix Deepseek-V4 compress attn groups prefix caching hit (#9903)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/core/single_type_kv_cache_manager.py", - "vllm_ascend/patch/platform/patch_kv_cache_coordinator.py" - ] - }, - { - "commit": "a87f2b7535556917858d8cad2585832704155397", - "subject": "[Doc] Translated Doc files 2026-06-07 (#10139)", - "topics": [ - "deepseek-v4-dspark", - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/locale/zh_CN/LC_MESSAGES/community/slash-commands.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/contribution/e2e_ci_test.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/contribution/testing.po", - "docs/source/locale/zh_CN/LC_MESSAGES/installation.po", - "docs/source/locale/zh_CN/LC_MESSAGES/quick_start.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/DeepSeek-V4-Flash.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/DeepSeek-V4-Pro.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/GLM5.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/InternVL3.5.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/MiniMax-M2.5.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/batch_invariance.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/dynamic_chunk_pipeline_parallel.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/epd_disaggregation.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/external_dp.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/kv_pool.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/large_scale_ep.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/netloader.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/ucm_deployment.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/weight_prefetch.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/release_notes.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/support_matrix/supported_models.po" - ] - }, - { - "commit": "22997bb62622c6868c314d6e6a626d5751d65419", - "subject": "[Attention][Feature] Support Sparse Flash Attention on Ascend A5 devices (#9825)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/attention/a2/test_sfa_v1.py", - "vllm_ascend/ascend_config.py", - "vllm_ascend/attention/sfa_v1.py", - "vllm_ascend/device/device_op.py", - "vllm_ascend/patch/platform/patch_kv_cache_interface.py", - "vllm_ascend/quantization/methods/w8a8_mxfp8.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "f7bdcf4af66fb012a8265a5c60415722e52d10fc", - "subject": "[Test] Fix e2e test error (#10133)", - "topics": [ - "communication" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/_selected_tests.yaml", - "tests/e2e/pull_request/four_card/test_profiling_chunk_performance.py" - ] - }, - { - "commit": "a1f7d1f372e6b3a9a8aef7405ab4031ac5ce357f", - "subject": "[Ascend950] [BugFix] Fix split_qkv_rmsnorm_rope Triton kernel accuracy issue on A5 (#9849)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/compilation/passes/qknorm_rope_fusion_pass.py", - "vllm_ascend/device/device_op.py", - "vllm_ascend/ops/__init__.py", - "vllm_ascend/ops/triton/linearnorm/split_qkv_rmsnorm_rope_simt.py" - ] - }, - { - "commit": "8991ca5f44af407336892834d1c641f4ee95c3e8", - "subject": "[Misc] Rehash AscendStore grouped keys for DSV4/compress layouts (#9789)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/distributed/ascend_store/_mock_deps.py", - "tests/ut/distributed/ascend_store/test_config_data.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/ascend_store/config_data.py" - ] - }, - { - "commit": "d9c0a7fb3052131f008ce50f903de5038c8e7911", - "subject": "[BugFix][Ascend950] fix Qwen3_5 error since fused_gdn_gating is unavailable on A5 (#10083)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/device/device_op.py", - "vllm_ascend/ops/gdn.py" - ] - }, - { - "commit": "b0f260b45ae664a2479ffa3103959147cbe25be0", - "subject": "[BugFix][DOC] Del engine_id and connector_path (#9916)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/developer_guide/Design_Documents/disaggregated_prefill.md", - "docs/source/developer_guide/contribution/multi_node_test.md", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/Design_Documents/disaggregated_prefill.po", - "docs/source/tutorials/features/long_sequence_context_parallel_multi_node.md", - "docs/source/tutorials/features/pd_disaggregation_mooncake_multi_node.md", - "docs/source/tutorials/features/pd_disaggregation_mooncake_single_node.md", - "docs/source/tutorials/models/DeepSeek-V3.1.md", - "docs/source/tutorials/models/DeepSeek-V3.2.md", - "docs/source/tutorials/models/GLM4.x.md", - "docs/source/tutorials/models/GLM5.md", - "docs/source/tutorials/models/Kimi-K2.5.md", - "docs/source/tutorials/models/Qwen3-235B-A22B.md", - "docs/source/tutorials/models/Qwen3.5-397B-A17B.md", - "docs/source/user_guide/deployment_guide/using_volcano_kthena.md", - "docs/source/user_guide/feature_guide/kv_pool.md", - "docs/source/user_guide/feature_guide/large_scale_ep.md", - "docs/source/user_guide/feature_guide/ucm_deployment.md", - "examples/disaggregated_prefill_v1/mooncake_connector_deployment_guide.md", - "examples/epd_disaggregated/epd_disaggregated_guide.md", - "tests/e2e/nightly/multi_node/external_dp/config/GLM5_1-W8A8-EP-external.yaml", - "tests/e2e/nightly/multi_node/external_dp/config/template.md", - "tests/e2e/nightly/multi_node/internal_dp/config/DeepSeek-R1-W8A8-EPLB.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/DeepSeek-R1-W8A8-longseq.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/DeepSeek-V3_2-W8A8-EP.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/GLM5_1-W8A8-EP.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/Qwen3-235B-A22B-Mooncake-Layerwise.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/Qwen3-235B-W8A8-EPLB.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/Qwen3-235B-W8A8-longseq.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/Qwen3-235B-W8A8.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/Qwen3-235B-disagg-pd.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/Qwen3-VL-235B-disagg-pd.yaml", - "tests/e2e/pull_request/four_card/test_deepseek_v3_2_w8a8_pruning.py", - "tests/e2e/weekly/multi_node/internal_dp/config/DeepSeek-V3.yaml", - "tests/e2e/weekly/multi_node/internal_dp/config/DeepSeek-V3_2-W8A8-EP_weekly.yaml", - "tests/e2e/weekly/multi_node/internal_dp/config/GLM-4.7-W8A8C8-Mooncake-Layerwise.yaml" - ] - }, - { - "commit": "4b4b5e6166ffedb9f6c6846459f5b234b162fc9e", - "subject": "[CI] delete DeepSeek-R1-W8A8.yaml update DeepSeek-R1-0528-W8A8-EPLB.yaml (#10055)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "tests/e2e/nightly/multi_node/internal_dp/config/DeepSeek-R1-W8A8.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/GLM5_1-W8A8-A3-dual-nodes.yaml", - "tests/e2e/nightly/single_node/models/configs/DeepSeek-R1-0528-W8A8.yaml" - ] - }, - { - "commit": "6230b36103676268fc8b28ec397eb0af41fe3c7c", - "subject": "[Doc][Misc] Update release notes known issues (#10039)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/user_guide/release_notes.md" - ] - }, - { - "commit": "028e538c84ae0d2e33f246e0d0778cbc4791582e", - "subject": "[BugFix] fix dsv4 piecewise scenario (#10003)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/platform.py" - ] - }, - { - "commit": "16747f8b0225d9093bcd528f4ba9c7f2a92f5516", - "subject": "[Test][CI] final part for selected test (#10017)", - "topics": [ - "deepseek-v4-dspark", - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - ".github/TEST_README.md", - ".github/workflows/_parse_trigger.yaml", - ".github/workflows/_selected_tests.yaml", - ".github/workflows/pr_test.yaml", - ".github/workflows/pr_test_full.yaml", - ".github/workflows/scripts/run_selected_tests.sh", - ".github/workflows/scripts/select_tests.py", - ".github/workflows/scripts/test_config.yaml", - ".github/workflows/scripts/test_select_tests.py", - "AGENTS.md", - "docs/source/developer_guide/contribution/e2e_ci_test.md", - "docs/source/developer_guide/contribution/testing.md", - "tests/e2e/pull_request/coverage.md", - "tests/e2e/pull_request/four_card/_310p/__init__.py", - "tests/e2e/pull_request/four_card/_310p/test_dense_model_310p.py", - "tests/e2e/pull_request/four_card/_310p/test_moe_model_310p.py", - "tests/e2e/pull_request/four_card/_310p/test_vl_model_310p.py", - "tests/e2e/pull_request/four_card/__init__.py", - "tests/e2e/pull_request/four_card/long_sequence/__init__.py", - "tests/e2e/pull_request/four_card/long_sequence/test_accuracy.py", - "tests/e2e/pull_request/four_card/long_sequence/test_basic.py", - "tests/e2e/pull_request/four_card/long_sequence/test_chunked_prefill_cp.py", - "tests/e2e/pull_request/four_card/long_sequence/test_mtp.py", - "tests/e2e/pull_request/four_card/long_sequence/test_prefix_caching_cp.py", - "tests/e2e/pull_request/four_card/spec_decode/__init__.py", - "tests/e2e/pull_request/four_card/spec_decode/test_mtp_qwen3_next.py", - "tests/e2e/pull_request/four_card/test_data_parallel_tp2.py", - "tests/e2e/pull_request/four_card/test_deepseek_v3_2_w8a8_pruning.py", - "tests/e2e/pull_request/four_card/test_deepseek_v4.py", - "tests/e2e/pull_request/four_card/test_pipeline_parallel.py", - "tests/e2e/pull_request/four_card/test_profiling_chunk_performance.py", - "tests/e2e/pull_request/four_card/test_qwen3_5.py", - "tests/e2e/pull_request/four_card/test_qwen3_next.py", - "tests/e2e/pull_request/full/two_cards/test_qwen3_moe.py", - "tests/e2e/pull_request/light/two_card/__init__.py", - "tests/e2e/pull_request/light/two_card/lora/__init__.py", - "tests/e2e/pull_request/light/two_card/spec_decode/__init__.py", - "tests/e2e/pull_request/one_card/_310p/__init__.py", - "tests/e2e/pull_request/one_card/_310p/test_classification_310p.py", - "tests/e2e/pull_request/one_card/_310p/test_dense_model_310p.py", - "tests/e2e/pull_request/one_card/_310p/test_embedding_310p.py", - "tests/e2e/pull_request/one_card/_310p/test_scoring_310p.py", - "tests/e2e/pull_request/one_card/_310p/test_vl_model_310p.py", - "tests/e2e/pull_request/one_card/__init__.py", - "tests/e2e/pull_request/one_card/aclgraph/__init__.py", - "tests/e2e/pull_request/one_card/aclgraph/test_aclgraph_accuracy.py", - "tests/e2e/pull_request/one_card/aclgraph/test_aclgraph_batch_invariant.py", - "tests/e2e/pull_request/one_card/aclgraph/test_aclgraph_mem.py", - "tests/e2e/pull_request/one_card/compile/__init__.py", - "tests/e2e/pull_request/one_card/compile/backend.py", - "tests/e2e/pull_request/one_card/compile/test_graphex_norm_quant_fusion.py", - "tests/e2e/pull_request/one_card/compile/test_graphex_qknorm_rope_fusion.py", - "tests/e2e/pull_request/one_card/compile/test_norm_quant_fusion.py", - "tests/e2e/pull_request/one_card/lora/__init__.py", - "tests/e2e/pull_request/one_card/lora/test_ilama_lora.py", - "tests/e2e/pull_request/one_card/lora/test_llama32_lora.py", - "tests/e2e/pull_request/one_card/lora/test_lora_with_spec_decode.py", - "tests/e2e/pull_request/one_card/lora/test_qwen35_densemodel_lora.py", - "tests/e2e/pull_request/one_card/lora/test_qwen3_multi_loras.py", - "tests/e2e/pull_request/one_card/lora/test_qwen3_reranker_lora.py", - "tests/e2e/pull_request/one_card/model_runner_v2/__init__.py", - "tests/e2e/pull_request/one_card/model_runner_v2/test_basic.py", - "tests/e2e/pull_request/one_card/pooling/__init__.py", - "tests/e2e/pull_request/one_card/pooling/template/qwen3_reranker.jinja", - "tests/e2e/pull_request/one_card/pooling/test_classification.py", - "tests/e2e/pull_request/one_card/pooling/test_embedding.py", - "tests/e2e/pull_request/one_card/pooling/test_scoring.py", - "tests/e2e/pull_request/one_card/spec_decode/__init__.py", - "tests/e2e/pull_request/one_card/spec_decode/conftest.py", - "tests/e2e/pull_request/one_card/spec_decode/test_dflash.py", - "tests/e2e/pull_request/one_card/spec_decode/test_draft_parallel.py", - "tests/e2e/pull_request/one_card/spec_decode/test_eagle.py", - "tests/e2e/pull_request/one_card/spec_decode/test_extract_hidden_states.py", - "tests/e2e/pull_request/one_card/spec_decode/test_mtp_eagle_correctness.py", - "tests/e2e/pull_request/one_card/spec_decode/test_ngram.py", - "tests/e2e/pull_request/one_card/spec_decode/test_ngram_npu.py", - "tests/e2e/pull_request/one_card/spec_decode/test_suffix.py", - "tests/e2e/pull_request/one_card/spec_decode/utils.py", - "tests/e2e/pull_request/one_card/test_attention_fa3.py", - "tests/e2e/pull_request/one_card/test_batch_invariant.py", - "tests/e2e/pull_request/one_card/test_camem.py", - "tests/e2e/pull_request/one_card/test_completion_with_prompt_embeds.py", - "tests/e2e/pull_request/one_card/test_cpu_offloading.py", - "tests/e2e/pull_request/one_card/test_guided_decoding.py", - "tests/e2e/pull_request/one_card/test_minicpm.py", - "tests/e2e/pull_request/one_card/test_multi_instance.py", - "tests/e2e/pull_request/one_card/test_multistream_overlap_shared_expert.py", - "tests/e2e/pull_request/one_card/test_qwen3_0_6b.py", - "tests/e2e/pull_request/one_card/test_qwen3_5_0_8b.py", - "tests/e2e/pull_request/one_card/test_qwen3_8b_w8a8.py", - "tests/e2e/pull_request/one_card/test_qwen3_embedding_0_6b.py", - "tests/e2e/pull_request/one_card/test_sampler.py", - "tests/e2e/pull_request/one_card/test_vlm.py", - "tests/e2e/pull_request/one_card/test_xlite.py", - "tests/e2e/pull_request/two_card/__init__.py", - "tests/e2e/pull_request/two_card/aclgraph/test_aclgraph_capture_replay.py", - "tests/e2e/pull_request/two_card/aclgraph/test_full_graph_mode.py", - "tests/e2e/pull_request/two_card/aclgraph/test_single_request_aclgraph.py", - "tests/e2e/pull_request/two_card/lora/__init__.py", - "tests/e2e/pull_request/two_card/lora/test_ilama_lora_tp2.py", - "tests/e2e/pull_request/two_card/lora/test_llama32_lora_tp2.py", - "tests/e2e/pull_request/two_card/spec_decode/__init__.py", - "tests/e2e/pull_request/two_card/spec_decode/test_spec_decode.py", - "tests/e2e/pull_request/two_card/test_data_parallel.py", - "tests/e2e/pull_request/two_card/test_deepseek_multistream_moe.py", - "tests/e2e/pull_request/two_card/test_disaggregated_encoder.py", - "tests/e2e/pull_request/two_card/test_external_launcher.py", - "tests/e2e/pull_request/two_card/test_flashcomm_distributed.py", - "tests/e2e/pull_request/two_card/test_gpt_oss_distributed.py", - "tests/e2e/pull_request/two_card/test_moe_routing_replay.py", - "tests/e2e/pull_request/two_card/test_offline_weight_load.py", - "tests/e2e/pull_request/two_card/test_prefix_caching.py", - "tests/e2e/pull_request/two_card/test_qwen3_30b_a3b.py", - "tests/e2e/pull_request/two_card/test_qwen3_moe_eplb.py", - "tests/e2e/pull_request/two_card/test_qwen3_performance.py", - "tests/e2e/pull_request/two_card/test_qwen3_vl_30b_a3b_instruct.py", - "tests/e2e/pull_request/two_card/test_sequence_parallelism_moe.py", - "tests/e2e/pull_request/two_card/test_shared_expert_dp.py", - "tests/e2e/pull_request/two_card/test_sp_pass.py" - ] - }, - { - "commit": "b67d0b4e59e6e03e5a051fae75033708beab05a8", - "subject": "[Doc]Update GLM-5 documentation and support matrix (#9889)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/tutorials/models/GLM5.md", - "docs/source/user_guide/support_matrix/supported_models.md" - ] - }, - { - "commit": "3dc73a69a92bcfcc9e3e3ee8e61853f359b90026", - "subject": "[Doc][Model] Add DeepSeek-V4 Flash and Pro documentation (#9995)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/tutorials/models/DeepSeek-V4-Flash.md", - "docs/source/tutorials/models/DeepSeek-V4-Pro.md", - "docs/source/tutorials/models/index.md", - "docs/source/user_guide/support_matrix/supported_models.md" - ] - }, - { - "commit": "3ca55109d305a2704b0d04b760ca3a0fc1e1ee6d", - "subject": "[Doc] Translated Doc files 2026-06-04 (#9989)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/locale/zh_CN/LC_MESSAGES/community/user_stories/index.po", - "docs/source/locale/zh_CN/LC_MESSAGES/community/user_stories/llamafactory.po", - "docs/source/locale/zh_CN/LC_MESSAGES/community/versioning_policy.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/Design_Documents/ACL_Graph.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/contribution/doc_writing.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/contribution/e2e_ci_test.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/contribution/multi_node_test.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/contribution/testing.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/performance_and_debug/optimization_and_tuning.po", - "docs/source/locale/zh_CN/LC_MESSAGES/faqs.po", - "docs/source/locale/zh_CN/LC_MESSAGES/installation.po", - "docs/source/locale/zh_CN/LC_MESSAGES/quick_start.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/features/long_sequence_context_parallel_multi_node.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/features/pd_colocated_mooncake_multi_instance.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/features/pd_disaggregation_mooncake_single_node.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/DeepSeek-V3.1.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/DeepSeek-V3.2.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/DeepSeekOCR2.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/GLM4.x.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/GLM5.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Hy3-preview.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Kimi-K2.5.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-Dense.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/gpt-oss-120b.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/configuration/additional_config.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/configuration/env_vars.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/eplb_swift_balancer.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/graph_mode.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/kv_pool.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/release_notes.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/support_matrix/feature_matrix.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/support_matrix/supported_models.po" - ] - }, - { - "commit": "33e92c4a8d294c5ea125783ffb4425b5a2ca2e30", - "subject": "[Test] Add more test to selected way (#9967)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/_pre_commit.yml", - ".github/workflows/_selected_tests.yaml", - ".github/workflows/pr_test_light.yaml", - ".github/workflows/scripts/run_selected_tests.sh", - ".github/workflows/scripts/test_config.yaml", - "tests/e2e/pull_request/full/one_card/compile/test_npu_storage_metadata.py", - "tests/e2e/pull_request/full/one_card/test_auto_fit_max_mode_len.py", - "tests/e2e/pull_request/full/two_cards/test_sequence_parallelism_moe.py", - "tests/e2e/pull_request/light/coverage.md", - "tests/e2e/pull_request/light/one_card/aclgraph/__init__.py", - "tests/e2e/pull_request/light/one_card/aclgraph/test_aclgraph_accuracy.py", - "tests/e2e/pull_request/light/one_card/aclgraph/test_aclgraph_batch_invariant.py", - "tests/e2e/pull_request/light/one_card/aclgraph/test_aclgraph_mem.py", - "tests/e2e/pull_request/light/one_card/compile/__init__.py", - "tests/e2e/pull_request/light/one_card/compile/backend.py", - "tests/e2e/pull_request/light/one_card/compile/test_graphex_norm_quant_fusion.py", - "tests/e2e/pull_request/light/one_card/compile/test_graphex_qknorm_rope_fusion.py", - "tests/e2e/pull_request/light/one_card/compile/test_norm_quant_fusion.py", - "tests/e2e/pull_request/light/two_card/test_sp_pass.py" - ] - }, - { - "commit": "2b8739c194ecbc87650d889457d41e1692e0f970", - "subject": "[Feature][Ops] Add LightningIndexer and SparseFlashAttention ACLNN ops (#9491)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "csrc/attention/common/op_kernel/CopyInL1.h", - "csrc/attention/common/op_kernel/FixpipeOut.h", - "csrc/attention/common/op_kernel/arch35/vf/vf_basic_block_aligned128_no_update_sfa.h", - "csrc/attention/common/op_kernel/arch35/vf/vf_basic_block_aligned128_update_sfa.h", - "csrc/attention/common/op_kernel/arch35/vf/vf_basic_block_unaligned128_no_update_sfa.h", - "csrc/attention/common/op_kernel/arch35/vf/vf_basic_block_unaligned128_update_sfa.h", - "csrc/attention/common/op_kernel/arch35/vf/vf_basic_block_unaligned64_no_update_sfa.h", - "csrc/attention/common/op_kernel/arch35/vf/vf_basic_block_unaligned64_update_sfa.h", - "csrc/attention/common/op_kernel/arch35/vf/vf_basic_block_utils.h", - "csrc/attention/common/op_kernel/arch35/vf/vf_flashupdate_new.h", - "csrc/attention/common/op_kernel/arch35/vf/vf_mul_sel_softmaxflashv2_cast_nz_sfa.h", - "csrc/attention/common/op_kernel/buffer.h", - "csrc/attention/common/op_kernel/buffer_manager.h", - "csrc/attention/common/op_kernel/buffer_mix_core.h", - "csrc/attention/common/op_kernel/buffers_policy.h", - "csrc/attention/common/op_kernel/matmul.h", - "csrc/attention/common/op_kernel/memcopy/fa_gm_tensor.h", - "csrc/attention/common/op_kernel/memcopy/fa_l1_tensor.h", - "csrc/attention/common/op_kernel/memcopy/gm_coord.h", - "csrc/attention/common/op_kernel/memcopy/gm_layout.h", - "csrc/attention/common/op_kernel/memcopy/offset_calculator_v2.h", - "csrc/attention/common/op_kernel/memcopy/parser.h", - "csrc/attention/common/op_kernel/offset_calculator.h", - "csrc/attention/lightning_indexer/CMakeLists.txt", - "csrc/attention/lightning_indexer/README.md", - "csrc/attention/lightning_indexer/docs/aclnnLightningIndexer.md", - "csrc/attention/lightning_indexer/lightning_indexer_torch_adpt.h", - "csrc/attention/lightning_indexer/op_host/CMakeLists.txt", - "csrc/attention/lightning_indexer/op_host/lightning_indexer_def.cpp", - "csrc/attention/lightning_indexer/op_host/lightning_indexer_infershape.cpp", - "csrc/attention/lightning_indexer/op_host/lightning_indexer_tiling.cpp", - "csrc/attention/lightning_indexer/op_host/lightning_indexer_tiling.h", - "csrc/attention/lightning_indexer/op_host/op_api/aclnn_lightning_indexer.cpp", - "csrc/attention/lightning_indexer/op_host/op_api/aclnn_lightning_indexer.h", - "csrc/attention/lightning_indexer/op_kernel/arch22/lightning_indexer_kernel.h", - "csrc/attention/lightning_indexer/op_kernel/arch22/lightning_indexer_service_cube.h", - "csrc/attention/lightning_indexer/op_kernel/arch22/lightning_indexer_service_vector.h", - "csrc/attention/lightning_indexer/op_kernel/arch22/lightning_indexer_vector.h", - "csrc/attention/lightning_indexer/op_kernel/arch35/lightning_indexer_kernel.h", - "csrc/attention/lightning_indexer/op_kernel/arch35/lightning_indexer_service_cube.h", - "csrc/attention/lightning_indexer/op_kernel/arch35/lightning_indexer_service_vector.h", - "csrc/attention/lightning_indexer/op_kernel/arch35/vf/lightning_indexer_topk.h", - "csrc/attention/lightning_indexer/op_kernel/arch35/vf/lightning_indexer_vector1.h", - "csrc/attention/lightning_indexer/op_kernel/arch35/vf/vf_topk.h", - "csrc/attention/lightning_indexer/op_kernel/arch35/vf/vf_topk_16_gather.h", - "csrc/attention/lightning_indexer/op_kernel/lightning_indexer.cpp", - "csrc/attention/lightning_indexer/op_kernel/lightning_indexer_common.h", - "csrc/attention/lightning_indexer/op_kernel/lightning_indexer_template_tiling_key.h", - "csrc/attention/sparse_flash_attention/CMakeLists.txt", - "csrc/attention/sparse_flash_attention/README.md", - "csrc/attention/sparse_flash_attention/docs/aclnnSparseFlashAttention.md", - "csrc/attention/sparse_flash_attention/examples/test_aclnn_sparse_flash_attention.cpp", - "csrc/attention/sparse_flash_attention/op_host/CMakeLists.txt", - "csrc/attention/sparse_flash_attention/op_host/op_api/aclnn_sparse_flash_attention.cpp", - "csrc/attention/sparse_flash_attention/op_host/op_api/aclnn_sparse_flash_attention.h", - "csrc/attention/sparse_flash_attention/op_host/sparse_flash_attention_def.cpp", - "csrc/attention/sparse_flash_attention/op_host/sparse_flash_attention_infershape.cpp", - "csrc/attention/sparse_flash_attention/op_host/sparse_flash_attention_proto.cpp", - "csrc/attention/sparse_flash_attention/op_host/sparse_flash_attention_tiling.cpp", - "csrc/attention/sparse_flash_attention/op_host/sparse_flash_attention_tiling.h", - "csrc/attention/sparse_flash_attention/op_kernel/arch22/sparse_flash_attention_kernel_mla.h", - "csrc/attention/sparse_flash_attention/op_kernel/arch22/sparse_flash_attention_service_cube_mla.h", - "csrc/attention/sparse_flash_attention/op_kernel/arch22/sparse_flash_attention_service_vector_mla.h", - "csrc/attention/sparse_flash_attention/op_kernel/arch35/sparse_flash_attention_common_arch35.h", - "csrc/attention/sparse_flash_attention/op_kernel/arch35/sparse_flash_attention_kernel_mla.h", - "csrc/attention/sparse_flash_attention/op_kernel/arch35/sparse_flash_attention_kvcache.h", - "csrc/attention/sparse_flash_attention/op_kernel/arch35/sparse_flash_attention_service_cube_mla.h", - "csrc/attention/sparse_flash_attention/op_kernel/arch35/sparse_flash_attention_service_vector_mla.h", - "csrc/attention/sparse_flash_attention/op_kernel/arch35/util_regbase.h", - "csrc/attention/sparse_flash_attention/op_kernel/sparse_flash_attention.cpp", - "csrc/attention/sparse_flash_attention/op_kernel/sparse_flash_attention_common.h", - "csrc/attention/sparse_flash_attention/op_kernel/sparse_flash_attention_template_tiling_key.h", - "csrc/attention/sparse_flash_attention/sparse_flash_attention_torch_adpt.h", - "csrc/build_aclnn.sh", - "csrc/torch_binding.cpp", - "csrc/torch_binding_meta.cpp", - "tests/ut/attention/test_sfa_cp.py", - "vllm_ascend/attention/context_parallel/sfa_cp.py", - "vllm_ascend/attention/sfa_v1.py" - ] - }, - { - "commit": "66a101c41c1ed9e711ba1264c8bfc779d45fb23b", - "subject": "[Test] Move more test to selected way (#9955)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/_selected_tests.yaml", - ".github/workflows/scripts/test_config.yaml", - "docs/source/community/versioning_policy.md", - "tests/e2e/pull_request/full/one_card/test_async_scheduling.py", - "tests/e2e/pull_request/full/one_card/test_eager_mode_acc.py", - "tests/e2e/pull_request/full/two_cards/test_qwen3_moe.py", - "tests/e2e/pull_request/light/coverage.md", - "tests/e2e/pull_request/light/four_card/_310p/__init__.py", - "tests/e2e/pull_request/light/four_card/_310p/test_dense_model_310p.py", - "tests/e2e/pull_request/light/four_card/_310p/test_moe_model_310p.py", - "tests/e2e/pull_request/light/four_card/_310p/test_vl_model_310p.py", - "tests/e2e/pull_request/light/four_card/long_sequence/__init__.py", - "tests/e2e/pull_request/light/four_card/long_sequence/test_accuracy.py", - "tests/e2e/pull_request/light/four_card/long_sequence/test_basic.py", - "tests/e2e/pull_request/light/four_card/long_sequence/test_chunked_prefill_cp.py", - "tests/e2e/pull_request/light/four_card/long_sequence/test_mtp.py", - "tests/e2e/pull_request/light/four_card/long_sequence/test_prefix_caching_cp.py", - "tests/e2e/pull_request/light/four_card/test_deepseek_v4.py", - "tests/e2e/pull_request/light/one_card/_310p/__init__.py", - "tests/e2e/pull_request/light/one_card/_310p/test_classification_310p.py", - "tests/e2e/pull_request/light/one_card/_310p/test_dense_model_310p.py", - "tests/e2e/pull_request/light/one_card/_310p/test_embedding_310p.py", - "tests/e2e/pull_request/light/one_card/_310p/test_scoring_310p.py", - "tests/e2e/pull_request/light/one_card/_310p/test_vl_model_310p.py", - "tests/e2e/pull_request/light/one_card/lora/__init__.py", - "tests/e2e/pull_request/light/one_card/test_attention_fa3.py", - "tests/e2e/pull_request/light/one_card/test_camem.py", - "tests/e2e/pull_request/light/one_card/test_cpu_offloading.py", - "tests/e2e/pull_request/light/one_card/test_xlite.py", - "tests/e2e/pull_request/light/two_card/lora/__init__.py", - "tests/e2e/pull_request/light/two_card/test_qwen3_moe_eplb.py" - ] - }, - { - "commit": "fcb30bebed4bc493238f2c432844b1f1be16ae6d", - "subject": "[Feature] DeepSeek-V4 for Ascend950 (#9757)", - "topics": [ - "deepseek-v4-dspark", - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/attention/dsa_v1.py", - "vllm_ascend/device/device_op.py", - "vllm_ascend/models/deepseek_v4.py", - "vllm_ascend/models/deepseek_v4_mtp.py", - "vllm_ascend/models/layer/attention/layer.py", - "vllm_ascend/ops/activation.py", - "vllm_ascend/ops/dsa.py", - "vllm_ascend/ops/fused_moe/fused_moe.py", - "vllm_ascend/ops/fused_moe/moe_comm_method.py", - "vllm_ascend/ops/fused_moe/moe_mlp.py", - "vllm_ascend/ops/fused_moe/moe_runtime_args.py", - "vllm_ascend/ops/fused_moe/moe_stage_contracts.py", - "vllm_ascend/ops/linear.py", - "vllm_ascend/ops/rope_dsv4.py", - "vllm_ascend/patch/platform/patch_kv_cache_interface.py", - "vllm_ascend/patch/worker/patch_deepseek_compressor.py", - "vllm_ascend/platform.py", - "vllm_ascend/quantization/__init__.py", - "vllm_ascend/quantization/fp8_config.py", - "vllm_ascend/quantization/method_adapters.py", - "vllm_ascend/quantization/methods/__init__.py", - "vllm_ascend/quantization/methods/fp8.py", - "vllm_ascend/quantization/methods/w4a8.py", - "vllm_ascend/quantization/methods/w4a8_mxfp4.py", - "vllm_ascend/quantization/utils.py", - "vllm_ascend/utils.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "ed3a992169f2484168781c73289b629018847f6c", - "subject": "[Feature] dsv4 supports piecewise graph mode. (#9935)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/platform.py" - ] - }, - { - "commit": "72365fbd29a6e41dadf601016be4bf2972fee348", - "subject": "[Doc](dsa_cp): Add \"enable_dsa_cp\" config for DSv3.2 and GLM5. (#9910)", - "topics": [ - "communication", - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/tutorials/models/DeepSeek-V3.2.md", - "docs/source/tutorials/models/GLM5.md" - ] - }, - { - "commit": "6548b01999044b4e7464be0c556a72af213e2aa9", - "subject": "[BugFix][CI] fix FLASHCOMM1 env leak in CI test (#9924)", - "topics": [ - "deepseek-v4-dspark", - "communication", - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/_selected_tests.yaml", - "tests/e2e/pull_request/full/four_cards/test_profiling_chunk_performance.py", - "tests/e2e/pull_request/full/two_cards/test_shared_expert_dp.py", - "vllm_ascend/models/deepseek_v4.py", - "vllm_ascend/ops/fused_moe/fused_moe.py" - ] - }, - { - "commit": "4335bd2561bdd597475da109ba1fb4d2aa929278", - "subject": "[Ops][Feature] Add fused_gdn_gating AscendC custom operator (#9601)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "csrc/CMakeLists.txt", - "csrc/attention/fused_gdn_gating/fused_gdn_gating_torch_adpt.h", - "csrc/attention/fused_gdn_gating/op_host/CMakeLists.txt", - "csrc/attention/fused_gdn_gating/op_host/fused_gdn_gating_def.cpp", - "csrc/attention/fused_gdn_gating/op_host/fused_gdn_gating_infershape.cpp", - "csrc/attention/fused_gdn_gating/op_host/fused_gdn_gating_tiling.cpp", - "csrc/attention/fused_gdn_gating/op_host/fused_gdn_gating_tiling.h", - "csrc/attention/fused_gdn_gating/op_host/fused_gdn_gating_tiling_utils.h", - "csrc/attention/fused_gdn_gating/op_host/op_api/aclnn_fused_gdn_gating.cpp", - "csrc/attention/fused_gdn_gating/op_host/op_api/aclnn_fused_gdn_gating.h", - "csrc/attention/fused_gdn_gating/op_host/op_api/fused_gdn_gating.cpp", - "csrc/attention/fused_gdn_gating/op_host/op_api/fused_gdn_gating.h", - "csrc/attention/fused_gdn_gating/op_kernel/fused_gdn_gating.cpp", - "csrc/attention/fused_gdn_gating/op_kernel/fused_gdn_gating.h", - "csrc/attention/fused_gdn_gating/op_kernel/fused_gdn_gating_tiling_data.h", - "csrc/build_aclnn.sh", - "csrc/torch_binding.cpp", - "csrc/torch_binding_meta.cpp", - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_fused_gdn_gating.py", - "vllm_ascend/ops/gdn.py" - ] - }, - { - "commit": "de2c7f84cd538c6d085c8a6df8755087f88a421d", - "subject": "[BugFix] Optimize router experts in eager mode and fix communication handling (#9728)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/workflows/scripts/config.yaml", - "tests/e2e/pull_request/full/two_cards/test_moe_routing_replay.py", - "tests/ut/patch/worker/test_patch_routed_experts_capture.py", - "vllm_ascend/patch/__init__.py", - "vllm_ascend/patch/worker/__init__.py", - "vllm_ascend/patch/worker/patch_routed_experts_capture.py" - ] - }, - { - "commit": "bb4d0776eee8fc45c3484a45c971a7049f1a2bbf", - "subject": "[CI] Refactor test folder (#9807)", - "topics": [ - "deepseek-v4-dspark", - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/README.md", - ".github/workflows/README.md", - ".github/workflows/_e2e_test.yaml", - ".github/workflows/_parse_trigger.yaml", - ".github/workflows/_pre_commit.yml", - ".github/workflows/_selected_tests.yaml", - ".github/workflows/dispatch_main2main_bisect.yaml", - ".github/workflows/pr_test_full.yaml", - ".github/workflows/pr_test_light.yaml", - ".github/workflows/scripts/config.yaml", - ".github/workflows/scripts/coverage.py", - ".github/workflows/scripts/determine_smart_e2e_scope.py", - ".github/workflows/scripts/select_tests.py", - ".github/workflows/scripts/smart_ut_README.md", - ".github/workflows/scripts/test_config.yaml", - ".github/workflows/scripts/ut_blacklist.yaml", - ".github/workflows/scripts/ut_config.yaml", - ".gitignore", - ".vllm_ascend/msprobe/msprobe_dump_config.json", - "AGENTS.md", - "docs/source/developer_guide/contribution/e2e_ci_test.md", - "docs/source/developer_guide/contribution/testing.md", - "tests/e2e/310p/data/qwen.png", - "tests/e2e/310p/test_utils.py", - "tests/e2e/pull_request/__init__.py", - "tests/e2e/pull_request/full/four_cards/__init__.py", - "tests/e2e/pull_request/full/four_cards/long_sequence/test_accuracy.py", - "tests/e2e/pull_request/full/four_cards/long_sequence/test_basic.py", - "tests/e2e/pull_request/full/four_cards/long_sequence/test_chunked_prefill_cp.py", - "tests/e2e/pull_request/full/four_cards/long_sequence/test_mtp.py", - "tests/e2e/pull_request/full/four_cards/long_sequence/test_prefix_caching_cp.py", - "tests/e2e/pull_request/full/four_cards/spec_decode/test_mtp_qwen3_next.py", - "tests/e2e/pull_request/full/four_cards/test_data_parallel_tp2.py", - "tests/e2e/pull_request/full/four_cards/test_deepseek_v4.py", - "tests/e2e/pull_request/full/four_cards/test_dense_model_310p.py", - "tests/e2e/pull_request/full/four_cards/test_kimi_k2.py", - "tests/e2e/pull_request/full/four_cards/test_moe_model_310p.py", - "tests/e2e/pull_request/full/four_cards/test_pipeline_parallel.py", - "tests/e2e/pull_request/full/four_cards/test_profiling_chunk_performance.py", - "tests/e2e/pull_request/full/four_cards/test_qwen3_5.py", - "tests/e2e/pull_request/full/four_cards/test_qwen3_next.py", - "tests/e2e/pull_request/full/four_cards/test_vl_model_310p.py", - "tests/e2e/pull_request/full/one_card/__init__.py", - "tests/e2e/pull_request/full/one_card/compile/__init__.py", - "tests/e2e/pull_request/full/one_card/compile/backend.py", - "tests/e2e/pull_request/full/one_card/compile/test_graphex_norm_quant_fusion.py", - "tests/e2e/pull_request/full/one_card/compile/test_graphex_qknorm_rope_fusion.py", - "tests/e2e/pull_request/full/one_card/compile/test_norm_quant_fusion.py", - "tests/e2e/pull_request/full/one_card/compile/test_npu_storage_metadata.py", - "tests/e2e/pull_request/full/one_card/model_runner_v2/__init__.py", - "tests/e2e/pull_request/full/one_card/model_runner_v2/test_basic.py", - "tests/e2e/pull_request/full/one_card/pooling/__init__.py", - "tests/e2e/pull_request/full/one_card/pooling/template/qwen3_reranker.jinja", - "tests/e2e/pull_request/full/one_card/pooling/test_classification.py", - "tests/e2e/pull_request/full/one_card/pooling/test_embedding.py", - "tests/e2e/pull_request/full/one_card/pooling/test_qwen3_reranker_lora.py", - "tests/e2e/pull_request/full/one_card/pooling/test_scoring.py", - "tests/e2e/pull_request/full/one_card/pooling_310p/test_classification_310p.py", - "tests/e2e/pull_request/full/one_card/pooling_310p/test_embedding_310p.py", - "tests/e2e/pull_request/full/one_card/pooling_310p/test_scoring_310p.py", - "tests/e2e/pull_request/full/one_card/spec_decode/__init__.py", - "tests/e2e/pull_request/full/one_card/spec_decode/test_extract_hidden_states.py", - "tests/e2e/pull_request/full/one_card/spec_decode/test_mtp_eagle_correctness.py", - "tests/e2e/pull_request/full/one_card/spec_decode/test_v1_spec_decode.py", - "tests/e2e/pull_request/full/one_card/test_aclgraph_accuracy.py", - "tests/e2e/pull_request/full/one_card/test_aclgraph_batch_invariant.py", - "tests/e2e/pull_request/full/one_card/test_aclgraph_mem.py", - "tests/e2e/pull_request/full/one_card/test_async_scheduling.py", - "tests/e2e/pull_request/full/one_card/test_attention_fa3.py", - "tests/e2e/pull_request/full/one_card/test_auto_fit_max_mode_len.py", - "tests/e2e/pull_request/full/one_card/test_batch_invariant.py", - "tests/e2e/pull_request/full/one_card/test_camem.py", - "tests/e2e/pull_request/full/one_card/test_completion_with_prompt_embeds.py", - "tests/e2e/pull_request/full/one_card/test_cpu_offloading.py", - "tests/e2e/pull_request/full/one_card/test_dense_model_310p.py", - "tests/e2e/pull_request/full/one_card/test_eager_mode_acc.py", - "tests/e2e/pull_request/full/one_card/test_guided_decoding.py", - "tests/e2e/pull_request/full/one_card/test_ilama_lora.py", - "tests/e2e/pull_request/full/one_card/test_llama32_lora.py", - "tests/e2e/pull_request/full/one_card/test_models.py", - "tests/e2e/pull_request/full/one_card/test_multi_instance.py", - "tests/e2e/pull_request/full/one_card/test_multistream_overlap_shared_expert.py", - "tests/e2e/pull_request/full/one_card/test_qwen35_densemodel_lora.py", - "tests/e2e/pull_request/full/one_card/test_qwen3_multi_loras.py", - "tests/e2e/pull_request/full/one_card/test_sampler.py", - "tests/e2e/pull_request/full/one_card/test_vl_model_310p.py", - "tests/e2e/pull_request/full/one_card/test_vlm.py", - "tests/e2e/pull_request/full/one_card/test_xlite.py", - "tests/e2e/pull_request/full/two_cards/__init__.py", - "tests/e2e/pull_request/full/two_cards/spec_decode/test_spec_decode.py", - "tests/e2e/pull_request/full/two_cards/test_aclgraph_capture_replay.py", - "tests/e2e/pull_request/full/two_cards/test_data_parallel.py", - "tests/e2e/pull_request/full/two_cards/test_disaggregated_encoder.py", - "tests/e2e/pull_request/full/two_cards/test_external_launcher.py", - "tests/e2e/pull_request/full/two_cards/test_full_graph_mode.py", - "tests/e2e/pull_request/full/two_cards/test_ilama_lora_tp2.py", - "tests/e2e/pull_request/full/two_cards/test_llama32_lora_tp2.py", - "tests/e2e/pull_request/full/two_cards/test_offline_inference_distributed.py", - "tests/e2e/pull_request/full/two_cards/test_offline_weight_load.py", - "tests/e2e/pull_request/full/two_cards/test_prefix_caching.py", - "tests/e2e/pull_request/full/two_cards/test_qwen3_moe.py", - "tests/e2e/pull_request/full/two_cards/test_qwen3_moe_routing_replay.py", - "tests/e2e/pull_request/full/two_cards/test_qwen3_performance.py", - "tests/e2e/pull_request/full/two_cards/test_sequence_parallelism_moe.py", - "tests/e2e/pull_request/full/two_cards/test_shared_expert_dp.py", - "tests/e2e/pull_request/full/two_cards/test_single_request_aclgraph.py", - "tests/e2e/pull_request/full/two_cards/test_sp_pass.py", - "tests/e2e/pull_request/light/coverage.md", - "tests/e2e/pull_request/light/four_card/__init__.py", - "tests/e2e/pull_request/light/four_card/test_deepseek_v3_2_w8a8_pruning.py", - "tests/e2e/pull_request/light/one_card/__init__.py", - "tests/e2e/pull_request/light/one_card/test_qwen3_0_6b.py", - "tests/e2e/pull_request/light/one_card/test_qwen3_5_0_8b.py", - "tests/e2e/pull_request/light/one_card/test_qwen3_8b_w8a8.py", - "tests/e2e/pull_request/light/one_card/test_qwen3_embedding_0_6b.py", - "tests/e2e/pull_request/light/two_card/__init__.py", - "tests/e2e/pull_request/light/two_card/test_qwen3_30b_a3b.py", - "tests/e2e/pull_request/light/two_card/test_qwen3_vl_30b_a3b_instruct.py", - "tests/e2e/pull_request/utils.py", - "tests/e2e/pull_request/utils_310p.py", - "tests/ut/_310p/__init__.py", - "tests/ut/_310p/attention/__init__.py", - "tests/ut/_310p/attention/test_attention_v1_310.py", - "tests/ut/_310p/fused_moe/__init__.py", - "tests/ut/_310p/fused_moe/test_moe_mlp_310.py", - "tests/ut/_310p/ops/__init__.py", - "tests/ut/_310p/quantization/__init__.py", - "tests/ut/_310p/quantization/test_w8a8_dynamic_310.py", - "tests/ut/_310p/quantization/test_w8a8_static_310.py", - "tests/ut/_310p/sample/__init__.py", - "tests/ut/_fake_weight/__init__.py", - "tests/ut/_fake_weight/config.json", - "tests/ut/_tools/__init__.py", - "tests/ut/_tools/test_ai_qos_tool.py", - "tests/ut/_tools/test_docs_codegen.py", - "tests/ut/attention/__init__.py", - "tests/ut/attention/a2/__init__.py", - "tests/ut/attention/a2/test_attention_cp.py", - "tests/ut/attention/a2/test_attention_cp_precision.py", - "tests/ut/attention/a2/test_attention_v1.py", - "tests/ut/attention/a2/test_attention_v1_precision.py", - "tests/ut/attention/a2/test_common_cp.py", - "tests/ut/attention/a2/test_mla_cp.py", - "tests/ut/attention/a2/test_mla_cp_precision.py", - "tests/ut/attention/a2/test_mla_precision.py", - "tests/ut/attention/a2/test_mla_v1.py", - "tests/ut/attention/a2/test_sfa_cp_precision.py", - "tests/ut/attention/a2/test_sfa_v1.py", - "tests/ut/attention/a2/test_sfa_v1_precision.py", - "tests/ut/compilation/__init__.py", - "tests/ut/compilation/a2/__init__.py", - "tests/ut/compilation/a2/test_acl_graph.py", - "tests/ut/conftest.py", - "tests/ut/core/__init__.py", - "tests/ut/core/test_scheduler_dynamic_batch.py", - "tests/ut/device/__init__.py", - "tests/ut/device/test_device_op.py", - "tests/ut/device_allocator/__init__.py", - "tests/ut/device_allocator/a2/__init__.py", - "tests/ut/device_allocator/a2/test_find_loaded_library.py", - "tests/ut/device_allocator/test_camem.py", - "tests/ut/distributed/__init__.py", - "tests/ut/distributed/ascend_store/__init__.py", - "tests/ut/distributed/device_communicators/__init__.py", - "tests/ut/distributed/device_communicators/test_pyhccl.py", - "tests/ut/distributed/mooncake/__init__.py", - "tests/ut/dummy/test_dummy.py", - "tests/ut/eplb/__init__.py", - "tests/ut/eplb/adaptor/__init__.py", - "tests/ut/eplb/core/__init__.py", - "tests/ut/eplb/core/a2/__init__.py", - "tests/ut/eplb/core/a2/expert_map.json", - "tests/ut/eplb/core/a2/test_eplb_utils.py", - "tests/ut/eplb/core/policy/__init__.py", - "tests/ut/kv_offload/__init__.py", - "tests/ut/kv_offload/a2/__init__.py", - "tests/ut/kv_offload/a2/test_remote_decode_lifecycle.py", - "tests/ut/kv_offload/a2/test_remote_prefill_lifecycle.py", - "tests/ut/kv_offload/test_mooncake_connector.py", - "tests/ut/kv_offload/test_mooncake_layerwise_connector.py", - "tests/ut/kv_offload/utils.py", - "tests/ut/lora/__init__.py", - "tests/ut/lora/test_lora.py", - "tests/ut/model_loader/__init__.py", - "tests/ut/model_loader/netloader/__init__.py", - "tests/ut/ops/__init__.py", - "tests/ut/ops/a2/__init__.py", - "tests/ut/ops/a2/test_gdn_chunk_meta.py", - "tests/ut/ops/a2/test_token_dispatcher.py", - "tests/ut/ops/a2/test_weight_prefetch.py", - "tests/ut/ops/a3_2/__init__.py", - "tests/ut/ops/a3_2/test_activation.py", - "tests/ut/ops/a3_2/test_select_experts.py", - "tests/ut/ops/test_moe_comm_method.py", - "tests/ut/ops/test_vocab_parallel_embedding.py", - "tests/ut/patch/__init__.py", - "tests/ut/patch/platform/__init__.py", - "tests/ut/patch/worker/__init__.py", - "tests/ut/patch/worker/patch_common/__init__.py", - "tests/ut/profiler/__init__.py", - "tests/ut/profiler/test_torch_npu_profiler.py", - "tests/ut/quantization/__init__.py", - "tests/ut/quantization/methods/__init__.py", - "tests/ut/quantization/methods/a2/__init__.py", - "tests/ut/quantization/methods/a2/test_w4a16.py", - "tests/ut/quantization/methods/a2/test_w4a4_flatquant.py", - "tests/ut/quantization/methods/a2/test_w4a4_laos_dynamic.py", - "tests/ut/quantization/methods/a2/test_w4a8.py", - "tests/ut/quantization/methods/a2/test_w8a16.py", - "tests/ut/quantization/methods/a2/test_w8a8_dynamic.py", - "tests/ut/quantization/methods/a2/test_w8a8_static.py", - "tests/ut/sample/__init__.py", - "tests/ut/spec_decode/__init__.py", - "tests/ut/spec_decode/a2/__init__.py", - "tests/ut/spec_decode/a2/test_eagle_proposer.py", - "tests/ut/test_ascend_config.py", - "tests/ut/test_ascend_forward_context.py", - "tests/ut/test_batch_invariant.py", - "tests/ut/test_cpu_binding.py", - "tests/ut/test_flash_common3_context.py", - "tests/ut/test_meta_registration.py", - "tests/ut/test_platform.py", - "tests/ut/test_profiling_config.py", - "tests/ut/test_utils.py", - "tests/ut/worker/__init__.py", - "tests/ut/worker/a2/__init__.py", - "tests/ut/worker/a2/test_block_table.py", - "tests/ut/worker/a2/test_kvcomp_utils.py", - "tests/ut/worker/a2/test_model_runner_v1.py", - "tests/ut/worker/a2/test_model_runner_v1_with_device.py", - "tests/ut/worker/a2/test_worker_multi_instance.py", - "tests/ut/worker/a2/test_worker_v1.py", - "tests/ut/xlite/__init__.py", - "tests/ut/xlite/test_xlite.py", - "tools/bisect_helper.py", - "vllm_ascend/core/scheduler_dynamic_batch.py", - "vllm_ascend/patch/worker/__init__.py", - "vllm_ascend/platform.py", - "vllm_ascend/utils.py" - ] - }, - { - "commit": "ddd9784eaeb9d27545d6e20439502d5dd639b315", - "subject": "[Misc](dsa_cp): introduce additional_config for dsa_cp (#9697)", - "topics": [ - "communication" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/user_guide/configuration/additional_config.md", - "tests/e2e/multicard/4-cards/long_sequence/test_basic.py", - "vllm_ascend/attention/sfa_v1.py", - "vllm_ascend/utils.py" - ] - }, - { - "commit": "7815e9c1bd54cbc18f739a792156d73542301cb5", - "subject": "[CI] update GLM5.1-w8a8-A3-dual-nodes.yaml and GLM5.1-w8a8-A3_weekly.yaml (#9566)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/misc/model_dataset_list.json", - ".github/workflows/schedule_nightly_test_a3.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/DeepSeek-V3_2-W8A8-EP.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/GLM5_1-W8A8-A3-dual-nodes.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/GLM5_1-W8A8-EP.yaml", - "tests/e2e/nightly/single_node/models/configs/DeepSeek-V3.2-W8A8.yaml", - "tests/e2e/weekly/multi_node/internal_dp/config/GLM5_1-W8A8-A3-dual-nodes.yaml", - "tests/e2e/weekly/single_node/configs/DeepSeek-V3.2-W8A8.yaml", - "tests/e2e/weekly/single_node/configs/GLM-5_1-W8A8_A3_weekly.yaml" - ] - }, - { - "commit": "ab32a0cfbf37a751e44a091cdcbf87bbdcea1864", - "subject": "[Feature][Ops] Support A2/A3 and A5 compressor paths (#9350)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "csrc/attention/compressor/op_host/CMakeLists.txt", - "csrc/attention/compressor/op_host/arch32/compressor_tiling.cpp", - "csrc/attention/compressor/op_host/arch32/compressor_tiling.h", - "csrc/attention/compressor/op_host/arch35/compressor_tiling.cpp", - "csrc/attention/compressor/op_host/arch35/compressor_tiling.h", - "csrc/attention/compressor/op_host/compressor_def.cpp", - "csrc/attention/compressor/op_host/compressor_proto.cpp", - "csrc/attention/compressor/op_kernel/arch32/compressor_block_cube_perf.h", - "csrc/attention/compressor/op_kernel/arch32/compressor_block_vec_perf.h", - "csrc/attention/compressor/op_kernel/arch32/compressor_comm.h", - "csrc/attention/compressor/op_kernel/arch32/compressor_kernel.h", - "csrc/attention/compressor/op_kernel/arch32/compressor_kernel_perf.h", - "csrc/attention/compressor/op_kernel/arch32/compressor_template_tiling_key.h", - "csrc/attention/compressor/op_kernel/arch32/compressor_tiling_data.h", - "csrc/attention/compressor/op_kernel/arch32/compressor_tools.h", - "csrc/attention/compressor/op_kernel/arch32/compressor_vector_comm.h", - "csrc/attention/compressor/op_kernel/arch32/rms_norm.h", - "csrc/attention/compressor/op_kernel/arch32/rope.h", - "csrc/attention/compressor/op_kernel/arch32/soft_max.h", - "csrc/attention/compressor/op_kernel/arch35/compressor_block_cube.h", - "csrc/attention/compressor/op_kernel/arch35/compressor_block_cube_full_load.h", - "csrc/attention/compressor/op_kernel/arch35/compressor_block_vec.h", - "csrc/attention/compressor/op_kernel/arch35/compressor_block_vec_full_load.h", - "csrc/attention/compressor/op_kernel/arch35/compressor_comm.h", - "csrc/attention/compressor/op_kernel/arch35/compressor_kernel.h", - "csrc/attention/compressor/op_kernel/arch35/compressor_kernel_full_load.h", - "csrc/attention/compressor/op_kernel/arch35/compressor_template_tiling_key.h", - "csrc/attention/compressor/op_kernel/arch35/compressor_tiling_data.h", - "csrc/attention/compressor/op_kernel/arch35/compressor_tools.h", - "csrc/attention/compressor/op_kernel/arch35/vf/vf_add.h", - "csrc/attention/compressor/op_kernel/arch35/vf/vf_mul.h", - "csrc/attention/compressor/op_kernel/arch35/vf/vf_rms_norm.h", - "csrc/attention/compressor/op_kernel/arch35/vf/vf_rope.h", - "csrc/attention/compressor/op_kernel/arch35/vf/vf_softmax.h", - "csrc/attention/compressor/op_kernel/compressor.cpp" - ] - }, - { - "commit": "0b527eade3f308dfcceeaad99c286f6fe95cd36c", - "subject": "[BugFix] Chunk wq_b matmul for NPU 65536 dimension limit (#9780)", - "topics": [ - "communication" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/attention/context_parallel/dsa_cp.py", - "vllm_ascend/quantization/methods/w8a8_dynamic.py" - ] - }, - { - "commit": "38f0236301174c5908f5f9a8d4e4b63876bbb62b", - "subject": "[BugFix] Resolve multiple DeepSeek V4 PP issues (#9473)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/models/deepseek_v4.py" - ] - }, - { - "commit": "103308d07aed1a92aa6fffafe7d0f54ddfc1096b", - "subject": "[CI] repair nightly custom op ci (#9827)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_npu_moe_gating_top_k.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_chunk_gated_delta_rule.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_rejection_sample.py" - ] - }, - { - "commit": "7bb84bf7b835bf7d3201be5423c7aa5d897a90da", - "subject": "[BugFix] Fix DSA compressed idle dummy graph OOB (#9818)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "6ed836a17d3eba15fffe7a75549445da9c508de5", - "subject": "[Feature][Attention] Support Model GLM4.7-Flash (#9560)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/attention/test_mla_cp_precision.py", - "tests/ut/attention/test_mla_v1.py", - "vllm_ascend/attention/mla_v1.py" - ] - }, - { - "commit": "5d5e5da15c88d84c3fe1d4c65a9a5865ae94ceda", - "subject": "[CI]enable dsv4-test (#9729)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/scripts/config.yaml", - "vllm_ascend/core/single_type_kv_cache_manager.py" - ] - }, - { - "commit": "8f8472ec98d1b8e0fbbd35a8d609c9d1d49585d4", - "subject": "[Ascend950][quant][Feature] Add W4A8 MXFP4 quantization support for Ascend950 (#8265)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/ops/test_moe_runtime_args.py", - "vllm_ascend/device/device_op.py", - "vllm_ascend/ops/fused_moe/moe_mlp.py", - "vllm_ascend/ops/fused_moe/moe_runtime_args.py", - "vllm_ascend/ops/fused_moe/moe_stage_params.py", - "vllm_ascend/ops/fused_moe/prepare_finalize.py", - "vllm_ascend/quantization/methods/__init__.py", - "vllm_ascend/quantization/methods/w4a8_mxfp4.py", - "vllm_ascend/quantization/quant_type.py" - ] - }, - { - "commit": "5d5a7701e9a12de2459654c7a4932940de0a4488", - "subject": "[BugFix] Omit empty tool_calls from OpenAI chat responses (#9791)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/patch/platform/test_patch_tool_choice_none_content.py", - "vllm_ascend/patch/platform/patch_tool_choice_none_content.py" - ] - }, - { - "commit": "dde19f7b06ed24d9e3cc9fed45595408424364a4", - "subject": "[Misc] main2main 0529 (#9726)", - "topics": [ - "deepseek-v4-dspark", - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/workflows/_e2e_test.yaml", - ".github/workflows/_parse_trigger.yaml", - ".github/workflows/dockerfiles/Dockerfile.lint", - ".github/workflows/pr_test_full.yaml", - ".github/workflows/pr_test_light.yaml", - "docs/source/conf.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_fused_qkvzba_split_reshape_cat.py", - "tests/e2e/singlecard/spec_decode/test_extract_hidden_states.py", - "tests/ut/distributed/ascend_store/test_backend.py", - "tests/ut/patch/worker/patch_common/test_patch_gdn_attn.py", - "tests/ut/spec_decode/test_extract_hidden_states_proposer.py", - "vllm_ascend/_310p/ops/fla/gdn_310.py", - "vllm_ascend/compilation/graph_fusion_pass_manager.py", - "vllm_ascend/ops/gdn.py", - "vllm_ascend/ops/triton/fla/chunk.py", - "vllm_ascend/ops/triton/mamba/causal_conv1d.py", - "vllm_ascend/patch/__init__.py", - "vllm_ascend/patch/worker/patch_deepseek_compressor.py", - "vllm_ascend/patch/worker/patch_idex_310.py", - "vllm_ascend/patch/worker/patch_minimax_m2_linear_attn.py", - "vllm_ascend/patch/worker/patch_qwen3_5.py", - "vllm_ascend/patch/worker/patch_triton.py", - "vllm_ascend/patch/worker/patch_v2/patch_attn_utils.py", - "vllm_ascend/worker/model_runner_v1.py", - "vllm_ascend/worker/v2/aclgraph_utils.py", - "vllm_ascend/worker/v2/attn_utils.py", - "vllm_ascend/worker/v2/sample/penalties.py", - "vllm_ascend/worker/v2/spec_decode/eagle/speculator.py" - ] - }, - { - "commit": "f6668c979d9a4803259a2f97e98fa1459bb7d7b6", - "subject": "[Ops][Misc] Remove VLLM_ASCEND_ENABLE_CONTEXT_PARALLEL (#9668)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/tutorials/features/long_sequence_context_parallel_multi_node.md", - "examples/offline_inference_npu_long_seq.py", - "tests/e2e/nightly/multi_node/internal_dp/scripts/test_multi_node.py", - "tests/e2e/nightly/single_node/models/scripts/test_single_node.py", - "tests/ut/ops/test_moe_comm_method.py", - "tests/ut/ops/test_prepare_finalize.py", - "tests/ut/test_ascend_config.py", - "vllm_ascend/ascend_config.py", - "vllm_ascend/envs.py", - "vllm_ascend/ops/fused_moe/prepare_finalize.py", - "vllm_ascend/utils.py" - ] - }, - { - "commit": "f44a1861aa867267b9d4e6f518e93314291fd97b", - "subject": "[Ascend950][Feature]Support MXFP8 FlashcommV3 on 950 (#9671)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/quantization/methods/w8a8_mxfp8.py" - ] - }, - { - "commit": "fd9c3cbf943e399249dfd737bd53dc70ddd96672", - "subject": "[Feature] [P/D] support hybrid attention for mooncake connector (#8850)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/kv_connector/test_mooncake_connector.py", - "vllm_ascend/distributed/kv_transfer/kv_p2p/mooncake_connector.py" - ] - }, - { - "commit": "2ce60baa18f6639a321eacb0920288743666af73", - "subject": "[Performance][310p] optimize moe routing path (#9105)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/e2e/310p/multicard/test_moe_model_multicard.py", - "tests/ut/_310p/fused_moe/test_experts_selector_310.py", - "vllm_ascend/_310p/fused_moe/experts_selector.py", - "vllm_ascend/_310p/fused_moe/fused_moe.py", - "vllm_ascend/_310p/fused_moe/token_dispatcher.py", - "vllm_ascend/_310p/quantization/methods/w8a8_dynamic.py" - ] - }, - { - "commit": "6a7d0ce1b5a70125e2349562429f525d4affb8f0", - "subject": "[BugFix] Lazy initialize KV store on put (#9771)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/distributed/kv_transfer/kv_pool/ascend_store/backend/memcache_backend.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/ascend_store/backend/mooncake_backend.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/ascend_store/pool_worker.py" - ] - }, - { - "commit": "77e1e1fc6430b166abd4715271bf55f817b38630", - "subject": "[BugFix][A2/A3]Preserve fp32 RoPE sin/cos for DSA compressor and rota\u2026 (#9739)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "csrc/attention/compressor/op_host/compressor_def.cpp", - "csrc/attention/compressor/op_host/compressor_tiling.cpp", - "csrc/attention/compressor/op_host/compressor_tiling.h", - "csrc/attention/compressor/op_kernel/arch32/compressor_block_vec_perf.h", - "csrc/attention/compressor/op_kernel/compressor.cpp", - "csrc/attention/compressor/op_kernel/compressor_comm.h", - "csrc/attention/compressor/op_kernel/compressor_template_tiling_key.h", - "csrc/attention/inplace_partial_rotary_mul/op_host/inplace_partial_rotary_mul_a3_tiling.cpp", - "csrc/attention/inplace_partial_rotary_mul/op_host/inplace_partial_rotary_mul_def.cpp", - "csrc/attention/inplace_partial_rotary_mul/op_host/inplace_partial_rotary_mul_proto.cpp", - "csrc/attention/inplace_partial_rotary_mul/op_host/inplace_partial_rotary_mul_tiling.cpp", - "csrc/attention/inplace_partial_rotary_mul/op_kernel/inplace_partial_rotary_mul.cpp", - "csrc/attention/inplace_partial_rotary_mul/op_kernel/inplace_partial_rotary_mul.h", - "vllm_ascend/ops/rope_dsv4.py" - ] - }, - { - "commit": "533cbe349c4c2dafd880103292aa52edcaf0aa0e", - "subject": "[BugFix][KV pool] DSv4 fix lookup/load mismatch (#9745)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/distributed/kv_transfer/kv_pool/ascend_store/config_data.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/ascend_store/pool_worker.py" - ] - }, - { - "commit": "f2d623dce9d1d69759e8ee9c51721d2a7da983e8", - "subject": "[Feature]Optimize workspace usage by workspace reuse for dispatch_ffn_combine operator (#9528)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "csrc/mc2/dispatch_ffn_combine/op_host/dispatch_ffn_combine_tiling.cpp", - "csrc/mc2/dispatch_ffn_combine/op_kernel/dispatch_ffn_combine_kernel.hpp", - "csrc/mc2/dispatch_ffn_combine/op_kernel/utils/block_epilogue_pertoken_swiglu.hpp", - "vllm_ascend/ops/fused_moe/moe_comm_method.py" - ] - }, - { - "commit": "a6684bd383dda4f2dfddd026483b8fcd6b488fe1", - "subject": "[Doc][Test]Add testable docs codegen framework (#8368)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/scripts/ut_config.yaml", - ".pre-commit-config.yaml", - "docs/source/conf.py", - "docs/source/developer_guide/contribution/doc_writing.md", - "docs/source/developer_guide/contribution/index.md", - "docs/source/developer_guide/contribution/multi_node_test.md", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/contribution/doc_writing.po", - "docs/source/tutorials/models/Kimi-K2-Thinking.md", - "docs/source/tutorials/models/Minitron-8B-Base.md", - "docs/source/tutorials/models/Mixtral-8x7B-Instruct-v0.1.md", - "docs/source/tutorials/models/Qwen3-ASR-1.7B.md", - "pyproject.toml", - "tests/e2e/models/configs/Hunyuan-A13B-Instruct.yaml", - "tests/e2e/models/configs/Minitron-8B-Base.yaml", - "tests/e2e/models/configs/Mixtral-8x7B-Instruct-v0.1.yaml", - "tests/e2e/nightly/multi_node/external_dp/config/GLM5_1-W8A8-EP-external.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/DeepSeek-R1-W8A8-EPLB.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/DeepSeek-R1-W8A8-longseq.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/DeepSeek-R1-W8A8.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/DeepSeek-V3.1-BF16.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/DeepSeek-V3_2-W8A8-A3-dual-nodes.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/DeepSeek-V3_2-W8A8-EP.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/GLM5_1-W8A8-A2-dual-nodes.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/GLM5_1-W8A8-A3-dual-nodes.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/GLM5_1-W8A8-EP.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/Kimi-K2_5-W4A8-A2-dual-nodes.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/Qwen3-235B-A22B-A2.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/Qwen3-235B-A22B-Mooncake-Layerwise.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/Qwen3-235B-A22B.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/Qwen3-235B-W8A8-EPLB.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/Qwen3-235B-W8A8-longseq.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/Qwen3-235B-W8A8.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/Qwen3-235B-disagg-pd.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/Qwen3-VL-235B-disagg-pd.yaml", - "tests/e2e/nightly/multi_node/internal_dp/scripts/multi_node_config.py", - "tests/e2e/weekly/multi_node/internal_dp/config/DeepSeek-V3.yaml", - "tests/e2e/weekly/multi_node/internal_dp/config/DeepSeek-V3_2-W8A8-EP_weekly.yaml", - "tests/e2e/weekly/multi_node/internal_dp/config/GLM-4.7-W8A8C8-Mooncake-Layerwise.yaml", - "tests/e2e/weekly/multi_node/internal_dp/config/GLM5_1-W8A8-A3-dual-nodes.yaml", - "tests/ut/tools/test_docs_codegen.py", - "tools/check_docs_yaml_sync.py", - "tools/docs_codegen/__init__.py", - "tools/docs_codegen/cli.py", - "tools/docs_codegen/converters.py", - "tools/docs_codegen/errors.py", - "tools/docs_codegen/generator.py", - "tools/docs_codegen/scanner.py", - "tools/docs_codegen/sphinx_extension.py", - "tools/docs_codegen/utils.py", - "tools/docs_codegen/yaml_loader.py" - ] - }, - { - "commit": "42f2322e2a06eaf4b84f1f371b46930eb52119fb", - "subject": " [BugFix][Model] Fix moe_gating_topk_hash UB out of bounds in dummy load for DeepSeek V4 (#9423)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/models/deepseek_v4.py" - ] - }, - { - "commit": "964370c4c564fc0663190fd88f444befbb59d396", - "subject": "[CI] repari nightly custom op (#9670)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_recurrent_gated_delta_rule.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_apply_penalties_triton.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_causal_conv1d.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_fused_recurrent_gated_delta_rule.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_prepare_inputs_padded.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_rejection_sample.py" - ] - }, - { - "commit": "79d1c6b22efb33e67f2e9421ad043976dd1d6f36", - "subject": "[Feature] Fix the precision issue of dsV4 caused by dequant_swiglu_quant (#9600)", - "topics": [ - "deepseek-v4-dspark", - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "csrc/moe/dequant_swiglu_quant/op_host/dequant_swiglu_quant_def.cpp", - "csrc/moe/dequant_swiglu_quant/op_kernel/dequant_swiglu_quant.h", - "csrc/torch_binding.cpp", - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_dequant_swiglu_quant.py" - ] - }, - { - "commit": "b2af836c5579e798bf4df2b2b3785d2848300517", - "subject": "[BugFix] Update DeepSeek V4 tool-call and thinking patches (#9733)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/patch/platform/test_patch_deepseek_v4_thinking.py", - "tests/ut/patch/platform/test_patch_deepseek_v4_tool_call_parser.py", - "vllm_ascend/patch/__init__.py", - "vllm_ascend/patch/platform/__init__.py", - "vllm_ascend/patch/platform/patch_deepseek_v4_thinking.py", - "vllm_ascend/patch/platform/patch_deepseek_v4_tool_call_parser.py" - ] - }, - { - "commit": "a5a04336c3ee095a8e6c32adb4911acf1636696b", - "subject": "[Feature] Add A5 MLA C8 feature (#9279)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/quantization/test_utils.py", - "vllm_ascend/attention/mla_v1.py", - "vllm_ascend/device/device_op.py", - "vllm_ascend/distributed/kv_transfer/kv_p2p/mooncake_connector.py", - "vllm_ascend/quantization/methods/kv_c8.py", - "vllm_ascend/quantization/modelslim_config.py", - "vllm_ascend/quantization/utils.py", - "vllm_ascend/worker/model_runner_v1.py", - "vllm_ascend/worker/v2/attn_utils.py" - ] - }, - { - "commit": "d3e1c3346511318b258db61d0ff5d50848017314", - "subject": "[CI] add nightly case: DeepSeek-V4-Flash-w8a8-mtp, and modify the old test cases (#9653)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/schedule_nightly_test_a3.yaml", - "tests/e2e/nightly/single_node/models/configs/DeepSeek-V4-Flash-W8A8-A3.yaml", - "tests/e2e/nightly/single_node/models/configs/Kimi-K2.5.yaml", - "tools/aisbench.py" - ] - }, - { - "commit": "78aa7ae36257ecfd02f47c7b341e535a97675076", - "subject": "[BugFix] Keep the router gate in FP32 on NPU for DeepSeek-like models (#9617)", - "topics": [ - "priority-models", - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/e2e/multicard/4-cards/test_pipeline_parallel.py", - "tests/ut/ops/test_gate_linear.py", - "vllm_ascend/ops/fused_moe/fused_moe.py", - "vllm_ascend/ops/fused_moe/gate_linear.py", - "vllm_ascend/utils.py" - ] - }, - { - "commit": "2370040f3d401890342b99ae6b157e5aa142c425", - "subject": "[Feature](dsa_cp, MTP) Add MTP support for DSA_CP (#9714)", - "topics": [ - "communication", - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/attention/context_parallel/dsa_cp.py" - ] - }, - { - "commit": "b5ef071fd8579f241451d5336c244170f433356f", - "subject": "[Performance][BugFix]Move _copy_valid_sampled_token_count after draft_forward (#9656)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "2a77209accf9a49845716cf79d4227bb9237ef32", - "subject": "[BugFix][MoE] Remove redundant renormalization operations (#9692)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/ops/fused_moe/experts_selector.py" - ] - }, - { - "commit": "fac8784c2572b14b1134f04d9818926b4a297f3a", - "subject": "[Feature](dsa, cp): Add DSA context parallel for deepseekv4. (#9531)", - "topics": [ - "deepseek-v4-dspark", - "communication", - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/e2e/multicard/4-cards/long_sequence/test_basic.py", - "vllm_ascend/attention/context_parallel/dsa_cp.py", - "vllm_ascend/attention/dsa_v1.py", - "vllm_ascend/models/deepseek_v4.py", - "vllm_ascend/models/deepseek_v4_mtp.py", - "vllm_ascend/ops/dsa.py", - "vllm_ascend/ops/linear_op.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "13e74b56ad5dd429946271ba4bcfc8135fb41034", - "subject": "[BugFix][Ops] Fix wo_a weight reload in DeepSeek V4 RL scenario (#9612)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/ops/linear.py" - ] - }, - { - "commit": "158635cda8fb8de6d008038665b1730e46ca9052", - "subject": "[Misc] Enhance w4a16.py and w4a8.py with docstirng and detailed log (#9525)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/quantization/methods/test_w4a16.py", - "tests/ut/quantization/methods/test_w4a8.py", - "vllm_ascend/quantization/methods/w4a16.py", - "vllm_ascend/quantization/methods/w4a8.py" - ] - }, - { - "commit": "360d47ceb5ae1c9bfe4755b3a576c83d09428806", - "subject": "[Misc]main2main 0522 (#9399)", - "topics": [ - "deepseek-v4-dspark", - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/dockerfiles/Dockerfile.lint", - ".github/workflows/pr_test_full.yaml", - ".github/workflows/pr_test_light.yaml", - "docs/source/conf.py", - "tests/e2e/310p/multicard/test_vl_model_multicard.py", - "vllm_ascend/models/deepseek_v4.py", - "vllm_ascend/ops/gdn.py", - "vllm_ascend/patch/__init__.py", - "vllm_ascend/patch/platform/__init__.py", - "vllm_ascend/patch/platform/patch_camem_allocator.py", - "vllm_ascend/patch/worker/patch_deepseek_compressor.py", - "vllm_ascend/sample/sampler.py", - "vllm_ascend/worker/model_runner_v1.py", - "vllm_ascend/worker/v2/attn_utils.py", - "vllm_ascend/worker/v2/block_table.py", - "vllm_ascend/worker/v2/sample/gumbel.py", - "vllm_ascend/worker/v2/spec_decode/rejection_sampler_utils.py" - ] - }, - { - "commit": "b5d3edee6e83567193664d85c2fcb61c930e6207", - "subject": "[CI][Nightly] Add the external_dp test framework (#9449)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/_e2e_nightly_multi_node.yaml", - ".github/workflows/schedule_nightly_test_a3.yaml", - ".github/workflows/schedule_weekly_test_a3.yaml", - "docs/source/developer_guide/contribution/multi_node_test.md", - "tests/e2e/conftest.py", - "tests/e2e/nightly/multi_node/external_dp/__init__.py", - "tests/e2e/nightly/multi_node/external_dp/config/GLM5_1-W8A8-EP-external.yaml", - "tests/e2e/nightly/multi_node/external_dp/config/template.md", - "tests/e2e/nightly/multi_node/external_dp/scripts/__init__.py", - "tests/e2e/nightly/multi_node/external_dp/scripts/external_dp_config.py", - "tests/e2e/nightly/multi_node/external_dp/scripts/runtime.py", - "tests/e2e/nightly/multi_node/external_dp/scripts/test_external_dp.py", - "tests/e2e/nightly/multi_node/external_dp/scripts/utils.py", - "tests/e2e/nightly/multi_node/internal_dp/config/DeepSeek-R1-W8A8-EPLB.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/DeepSeek-R1-W8A8-longseq.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/DeepSeek-R1-W8A8.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/DeepSeek-V3.1-BF16.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/DeepSeek-V3_2-W8A8-A3-dual-nodes.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/DeepSeek-V3_2-W8A8-EP.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/GLM5_1-W8A8-A2-dual-nodes.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/GLM5_1-W8A8-A3-dual-nodes.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/GLM5_1-W8A8-EP.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/Kimi-K2_5-W4A8-A2-dual-nodes.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/Qwen3-235B-A22B-A2.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/Qwen3-235B-A22B-Mooncake-Layerwise.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/Qwen3-235B-A22B.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/Qwen3-235B-W8A8-EPLB.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/Qwen3-235B-W8A8-longseq.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/Qwen3-235B-W8A8.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/Qwen3-235B-disagg-pd.yaml", - "tests/e2e/nightly/multi_node/internal_dp/config/Qwen3-VL-235B-disagg-pd.yaml", - "tests/e2e/nightly/multi_node/internal_dp/scripts/__init__.py", - "tests/e2e/nightly/multi_node/internal_dp/scripts/multi_node_config.py", - "tests/e2e/nightly/multi_node/internal_dp/scripts/test_multi_node.py", - "tests/e2e/nightly/multi_node/internal_dp/scripts/utils.py", - "tests/e2e/nightly/multi_node/scripts/__init__.py", - "tests/e2e/nightly/multi_node/scripts/benchmark_results.py", - "tests/e2e/nightly/multi_node/scripts/lws.yaml.jinja2", - "tests/e2e/nightly/multi_node/scripts/run.sh", - "tests/e2e/nightly/multi_node/scripts/utils.py", - "tests/e2e/weekly/multi_node/internal_dp/config/DeepSeek-V3.yaml", - "tests/e2e/weekly/multi_node/internal_dp/config/DeepSeek-V3_2-W8A8-EP_weekly.yaml", - "tests/e2e/weekly/multi_node/internal_dp/config/GLM-4.7-W8A8C8-Mooncake-Layerwise.yaml", - "tests/e2e/weekly/multi_node/internal_dp/config/GLM5_1-W8A8-A3-dual-nodes.yaml" - ] - }, - { - "commit": "11337339cc3de74042da731e249f2a0cd10ced2f", - "subject": "[Misc] Improve logging across model_loader, quantization, and patch modules (#9537)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/quantization/test_utils.py", - "vllm_ascend/_310p/quantization/modelslim_config.py", - "vllm_ascend/model_loader/netloader/interaction/elastic.py", - "vllm_ascend/model_loader/rfork/rfork_loader.py", - "vllm_ascend/model_loader/rfork/rfork_worker.py", - "vllm_ascend/model_loader/rfork/seed_protocol.py", - "vllm_ascend/model_loader/rfork/seed_server.py", - "vllm_ascend/model_loader/rfork/transfer_backend.py", - "vllm_ascend/patch/platform/patch_minimax_m2_config.py", - "vllm_ascend/patch/worker/patch_gqa_c8.py", - "vllm_ascend/patch/worker/patch_minimax_m2_linear_attn.py", - "vllm_ascend/quantization/methods/w4a4_flatquant.py", - "vllm_ascend/quantization/modelslim_config.py", - "vllm_ascend/quantization/utils.py" - ] - }, - { - "commit": "737266135913650ca27e6a9726492e99764d225d", - "subject": "[Refactor] migrate compilation backend from torchair to npugraph_ex (#9201)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "csrc/attention/sparse_attn_sharedkv/README.md", - "docs/source/developer_guide/Design_Documents/npugraph_ex.md", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/Design_Documents/npugraph_ex.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/graph_mode.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/npugraph_ex.po", - "docs/source/user_guide/feature_guide/graph_mode.md", - "mypy.ini", - "tests/e2e/nightly/single_node/ops/multicard_ops_a2/test_matmul_allreduce_add_rmsnorm.py", - "tests/e2e/nightly/single_node/ops/multicard_ops_a3/test_dispatch_gmm_combine_decode.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_hamming_dist_top_k.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_reshape_and_cache_bnsd.py", - "tests/e2e/singlecard/compile/test_graphex_norm_quant_fusion.py", - "tests/e2e/singlecard/compile/test_graphex_qknorm_rope_fusion.py", - "vllm_ascend/compilation/compiler_interface.py", - "vllm_ascend/compilation/passes/base_pattern.py", - "vllm_ascend/patch/__init__.py", - "vllm_ascend/patch/worker/patch_npugraph_ex_triton.py" - ] - }, - { - "commit": "40992df62cb70e25e379e7269125b87d9589cb8e", - "subject": "[Doc]: remove --async-scheduling from configs and docs (#9567)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/_templates/Model-Deployment-Tutorial-Template.md", - "docs/source/tutorials/features/long_sequence_context_parallel_single_node.md", - "docs/source/tutorials/models/DeepSeek-R1.md", - "docs/source/tutorials/models/DeepSeek-V3.1.md", - "docs/source/tutorials/models/DeepSeek-V3.2.md", - "docs/source/tutorials/models/DeepSeekOCR2.md", - "docs/source/tutorials/models/GLM4.x.md", - "docs/source/tutorials/models/GLM5.md", - "docs/source/tutorials/models/Hunyuan-A13B-Instruct.md", - "docs/source/tutorials/models/Kimi-K2.5.md", - "docs/source/tutorials/models/MiniMax-M2.5.md", - "docs/source/tutorials/models/Qwen-VL-Dense.md", - "docs/source/tutorials/models/Qwen3-235B-A22B.md", - "docs/source/tutorials/models/Qwen3-Dense.md", - "docs/source/tutorials/models/Qwen3-VL-235B-A22B-Instruct.md", - "docs/source/tutorials/models/Qwen3.5-27B.md", - "docs/source/tutorials/models/Qwen3.5-397B-A17B.md", - "docs/source/tutorials/models/gpt-oss-120b.md", - "docs/source/user_guide/release_notes.md", - "tests/e2e/multicard/2-cards/test_qwen3_performance.py", - "tests/e2e/nightly/multi_node/config/DeepSeek-V3.1-BF16.yaml", - "tests/e2e/nightly/multi_node/config/DeepSeek-V3_2-W8A8-EP.yaml", - "tests/e2e/nightly/multi_node/config/GLM5_1-W8A8-A2-dual-nodes.yaml", - "tests/e2e/nightly/multi_node/config/GLM5_1-W8A8-A3-dual-nodes.yaml", - "tests/e2e/nightly/multi_node/config/GLM5_1-W8A8-EP.yaml", - "tests/e2e/nightly/multi_node/config/Kimi-K2_5-W4A8-A2-dual-nodes.yaml", - "tests/e2e/nightly/multi_node/config/Qwen3-235B-A22B-A2.yaml", - "tests/e2e/nightly/multi_node/config/Qwen3-235B-disagg-pd.yaml", - "tests/e2e/nightly/multi_node/scripts/test_multi_node.py", - "tests/e2e/nightly/single_node/models/configs/DeepSeek-V3.2-W8A8.yaml", - "tests/e2e/nightly/single_node/models/configs/GLM-4.7.yaml", - "tests/e2e/nightly/single_node/models/configs/Kimi-K2.5.yaml", - "tests/e2e/nightly/single_node/models/configs/MiniMax-M2.5-w8a8-QuaRot-A2.yaml", - "tests/e2e/nightly/single_node/models/configs/MiniMax-M2.5-w8a8-QuaRot-A3.yaml", - "tests/e2e/nightly/single_node/models/configs/Qwen3-235B-A22B-W8A8.yaml", - "tests/e2e/nightly/single_node/models/configs/Qwen3-30B-A3B-W4A8-llm-compressor.yaml", - "tests/e2e/nightly/single_node/models/configs/Qwen3-30B-A3B-W8A8.yaml", - "tests/e2e/nightly/single_node/models/configs/Qwen3-32B-Int8-A2.yaml", - "tests/e2e/nightly/single_node/models/configs/Qwen3-32B-Int8.yaml", - "tests/e2e/nightly/single_node/models/configs/Qwen3-VL-235B-A22B-Instruct-W8A8.yaml", - "tests/e2e/nightly/single_node/models/configs/Qwen3-VL-32B-Instruct-W8A8.yaml", - "tests/e2e/nightly/single_node/models/configs/Qwen3.5-122B-A10B-W8A8-A3.yaml", - "tests/e2e/nightly/single_node/models/configs/Qwen3.5-27B-w8a8-A2.yaml", - "tests/e2e/nightly/single_node/models/configs/Qwen3.5-27B-w8a8-A3.yaml", - "tests/e2e/nightly/single_node/models/configs/Qwen3.5-397B-A17B-W8A8-mtp-A3.yaml", - "tests/e2e/nightly/single_node/models/configs/Qwen3.5-397B-A17B-w4a8-mtp-A2.yaml", - "tests/e2e/nightly/single_node/models/scripts/test_single_node.py", - "tests/e2e/weekly/multi_node/configs/DeepSeek-V3_2-W8A8-EP_weekly.yaml", - "tests/e2e/weekly/multi_node/configs/GLM5_1-W8A8-A3-dual-nodes.yaml", - "tests/e2e/weekly/single_node/configs/DeepSeek-V3.2-W8A8.yaml", - "tests/e2e/weekly/single_node/configs/DeepSeek-V3.2-W8A8_A3_weekly.yaml", - "tests/e2e/weekly/single_node/configs/GLM-5.yaml", - "tests/e2e/weekly/single_node/configs/GLM-5_1-W8A8_A3_weekly.yaml", - "tests/e2e/weekly/single_node/configs/Kimi-K2.5-32k-512.yaml", - "tests/e2e/weekly/single_node/configs/Kimi-K2.5.yaml", - "tests/e2e/weekly/single_node/configs/MiniMax-M2.5-W8A8-A3.yaml", - "tests/e2e/weekly/single_node/configs/MiniMax-M2.5-w8a8-QuaRot-A3.yaml", - "tests/e2e/weekly/single_node/configs/Qwen3.5-122B-A10B-W8A8-A2.yaml", - "tests/e2e/weekly/single_node/configs/Qwen3.5-122B-A10B-W8A8-A3.yaml", - "tests/e2e/weekly/single_node/configs/Qwen3.5-27B-w8a8-A3.yaml", - "tests/e2e/weekly/single_node/configs/Qwen3.5-397B-A17B-W8A8-mtp-A3.yaml", - "tests/e2e/weekly/single_node/configs/Qwen3.5-397B-A17B-W8A8-mtp-A3_weekly.yaml", - "tests/e2e/weekly/single_node/models/test_qwen3_30b_acc.py" - ] - }, - { - "commit": "4dcce5dec8b11727241d99aa3c36b0118442895d", - "subject": "[BugFix] Fix dsv4 mtp acceptance (#9605)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/attention/dsa_v1.py" - ] - }, - { - "commit": "2e94f5417419500db0318c1f62f92a045a6517c1", - "subject": "[BugFix] Cap DSv4 DSA admission to startup pool-sizing bound (#9548)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/core/single_type_kv_cache_manager.py", - "vllm_ascend/patch/platform/patch_kv_cache_coordinator.py" - ] - }, - { - "commit": "b86670f6a3756d8891a8f3df194fb1b72fa7a433", - "subject": "[Ops][BugFix] Disable SwiGLU clamp by default (#9438)", - "topics": [ - "deepseek-v4-dspark", - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "csrc/gmm/grouped_matmul_swiglu_quant/op_host/grouped_matmul_swiglu_quant_def.cpp", - "csrc/gmm/grouped_matmul_swiglu_quant/op_host/grouped_matmul_swiglu_quant_tiling.cpp", - "csrc/gmm/grouped_matmul_swiglu_quant/op_kernel/grouped_matmul_swiglu_quant.h", - "csrc/gmm/grouped_matmul_swiglu_quant/op_kernel/grouped_matmul_swiglu_quant_split_ws.h", - "csrc/gmm/grouped_matmul_swiglu_quant_v2/op_host/grouped_matmul_swiglu_quant_v2_base_tiling.cpp", - "csrc/gmm/grouped_matmul_swiglu_quant_v2/op_host/grouped_matmul_swiglu_quant_v2_base_tiling.h", - "csrc/gmm/grouped_matmul_swiglu_quant_v2/op_host/grouped_matmul_swiglu_quant_v2_def.cpp", - "csrc/gmm/grouped_matmul_swiglu_quant_v2/op_host/grouped_matmul_swiglu_quant_v2_fusion_tiling.cpp", - "csrc/gmm/grouped_matmul_swiglu_quant_v2/op_kernel/grouped_matmul_swiglu_quant_spilit_fusion.h", - "csrc/gmm/grouped_matmul_swiglu_quant_v2/op_kernel/grouped_matmul_swiglu_quant_v2_a4w4_post.h", - "csrc/gmm/grouped_matmul_swiglu_quant_v2/op_kernel/grouped_matmul_swiglu_quant_v2_a8w4_msd_post.h", - "csrc/gmm/grouped_matmul_swiglu_quant_weight_nz_tensor_list/op_host/grouped_matmul_swiglu_quant_weight_nz_tensor_list_def.cpp", - "csrc/gmm/grouped_matmul_swiglu_quant_weight_nz_tensor_list/op_host/grouped_matmul_swiglu_quant_weight_nz_tensor_list_tiling.cpp", - "csrc/moe/dequant_swiglu_quant/op_host/dequant_swiglu_quant_def.cpp", - "csrc/moe/dequant_swiglu_quant/op_host/dequant_swiglu_quant_proto.h", - "csrc/moe/dequant_swiglu_quant/op_host/dequant_swiglu_quant_tiling.cpp", - "csrc/moe/dequant_swiglu_quant/op_host/dequant_swiglu_quant_tiling_arch35.cpp", - "csrc/moe/dequant_swiglu_quant/op_kernel/dequant_swiglu_quant.h", - "csrc/torch_binding.cpp", - "tests/e2e/singlecard/model_runner_v2/test_basic.py", - "vllm_ascend/ops/fused_moe/fused_moe.py" - ] - }, - { - "commit": "816abbe6c9d4f753a2e32916e42ae4d78615ae30", - "subject": "[EPLB][CI] CI of dynamic eplb bugfix (#9150)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/user_guide/feature_guide/eplb_swift_balancer.md", - "tests/e2e/multicard/2-cards/test_qwen3_moe.py", - "vllm_ascend/ops/fused_moe/fused_moe.py" - ] - }, - { - "commit": "880141b4811a09485eaf2accd2b98e3ceb56d5ec", - "subject": "[Attention][Feature] Implement compute-communication overlap for pure-prefill batches (#9504)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/ascend_config.py", - "vllm_ascend/attention/dsa_v1.py" - ] - }, - { - "commit": "02e58a6ca926bb43127ffd04cd2dc474de30ea84", - "subject": "[BugFix] Fixed the issue where cv_indexer_qkv_prepare's multistream parallel capability was not enabled when multistream-dsv4-dsa-overlap=true (#9530)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/attention/dsa_v1.py", - "vllm_ascend/ops/dsa.py" - ] - }, - { - "commit": "f650855a54e3c631a643b901d6a0e6e025e853fd", - "subject": "[CI]repair custom ops ci (#9465)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/e2e/nightly/single_node/ops/conftest.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_add_rms_norm_bias.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_causal_conv1d_310.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_fused_moe.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_recurrent_gated_delta_rule_310.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_bad_words.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_chunk_gated_delta_rule.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_compute_token_logprobs.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_compute_topk_logprobs.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_fused_qkvzba_split_reshape_cat.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_fused_recurrent_gated_delta_rule.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_fused_sigmoid_gating_delta_rule.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_prepare_inputs_padded.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_rejection_sample.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_rope.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_split_qkv_rmsnorm_mrope.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_split_qkv_rmsnorm_rope.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_split_qkv_tp_rmsnorm_rope.py", - "vllm_ascend/worker/v2/sample/logprob.py" - ] - }, - { - "commit": "4b1d43a6264fc9bbbbb06786922300326ac16216", - "subject": "[Performance][ops][310p] Support Qwen3.5-Dense ACLGraph with fused operators (#9104)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "csrc/moe/causal_conv1d_v310/causal_conv1d_310_torch_adpt.h", - "csrc/moe/causal_conv1d_v310/op_host/causal_conv1d_v310_def.cpp", - "csrc/moe/causal_conv1d_v310/op_host/causal_conv1d_v310_tiling.cpp", - "csrc/moe/causal_conv1d_v310/op_host/causal_conv1d_v310_tiling.h", - "csrc/moe/causal_conv1d_v310/op_kernel/causal_conv1d_v310.h", - "csrc/torch_binding.cpp", - "csrc/torch_binding_meta.cpp", - "tests/e2e/310p/singlecard/test_dense_model_singlecard.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_causal_conv1d_310.py", - "vllm_ascend/_310p/ops/fla/gdn_310.py" - ] - }, - { - "commit": "45dc87927fee55371b87bc85fbad9c6479ea2a89", - "subject": "[BugFix]Use Compress ratio to avoid obtaining illegal attributes (#9510)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/distributed/kv_transfer/kv_p2p/mooncake_hybrid_connector.py" - ] - }, - { - "commit": "711a09f73305847f22f9ad062c5e317837f4a26f", - "subject": "[Feature] IndexCache: reuse topk_indices across decode steps for DSA models (DeepSeek V4) (#9390)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "tests/e2e/multicard/4-cards/test_deepseek_v4.py", - "vllm_ascend/attention/dsa_v1.py", - "vllm_ascend/models/deepseek_v4.py", - "vllm_ascend/ops/dsa.py" - ] - }, - { - "commit": "68ad83a9a6ae3cafa92a781e01205ca9049d07ac", - "subject": "[Performance] Optimize shared expert overlap timing in FusedMoE (#9413)", - "topics": [ - "deepseek-v4-dspark", - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/ops/test_fused_moe.py", - "tests/ut/ops/test_linear.py", - "vllm_ascend/ascend_forward_context.py", - "vllm_ascend/models/deepseek_v4.py", - "vllm_ascend/ops/fused_moe/fused_moe.py", - "vllm_ascend/ops/linear.py" - ] - }, - { - "commit": "261ea0be2cc12a6dd50b3ca0239904ac2d89cae4", - "subject": "[BugFix]Fix Deepseek-V4 P/D disaggregation kv_cache_tensor.shared_by may be empty (#9500)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/distributed/kv_transfer/kv_p2p/mooncake_hybrid_connector.py" - ] - }, - { - "commit": "56625a48cc5fae2d2e73c3128950b61bfb080df2", - "subject": "[Ascend950][Feature]enable mc2 dispatch MXFP4 quantization in A5 (#9365)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/ops/test_token_dispatcher.py", - "vllm_ascend/ops/fused_moe/moe_stage_params.py", - "vllm_ascend/ops/fused_moe/token_dispatcher.py" - ] - }, - { - "commit": "c78b5367f6243a6ba71bb76c5be2e0a0e35f6516", - "subject": "[CI] Move nightly cases to weekly and remove some nightly cases (#9461)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/schedule_nightly_test_a2.yaml", - ".github/workflows/schedule_nightly_test_a3.yaml", - ".github/workflows/schedule_weekly_test_a3.yaml", - "tests/e2e/nightly/multi_node/config/DeepSeek-R1-W8A8-A2.yaml", - "tests/e2e/nightly/multi_node/config/DeepSeek-V3_2-W8A8-EP-aime2025.yaml", - "tests/e2e/nightly/multi_node/config/Kimi-K2-Instruct-W8A8.yaml", - "tests/e2e/nightly/single_node/models/configs/DeepSeek-R1-W8A8-HBM.yaml", - "tests/e2e/nightly/single_node/models/configs/GLM-4.5.yaml", - "tests/e2e/nightly/single_node/models/configs/MiniMax-M2.5-A3.yaml", - "tests/e2e/nightly/single_node/models/configs/Qwen3-Next-80B-A3B-Instruct-A2.yaml", - "tests/e2e/nightly/single_node/models/configs/Qwen3-Next-80B-A3B-Instruct-W8A8.yaml", - "tests/e2e/nightly/single_node/models/configs/Qwen3-Next-80B-A3B-Instruct.yaml", - "tests/e2e/weekly/multi_node/configs/DeepSeek-V3.yaml", - "tests/e2e/weekly/multi_node/configs/GLM-4.7-W8A8C8-Mooncake-Layerwise.yaml", - "tests/e2e/weekly/single_node/configs/DeepSeek-V3.2-W8A8_A3_weekly.yaml", - "tests/e2e/weekly/single_node/configs/GLM-5.yaml", - "tests/e2e/weekly/single_node/configs/MiniMax-M2.5-W8A8-A3.yaml", - "tests/e2e/weekly/single_node/configs/Qwen2.5-VL-7B-Instruct-EPD.yaml", - "tests/e2e/weekly/single_node/configs/Qwen3-32B.yaml", - "tests/e2e/weekly/single_node/configs/Qwen3.5-122B-A10B-W8A8-A2.yaml" - ] - }, - { - "commit": "a777779cf56fab9589e224f9bf2b802d6c9442db", - "subject": "[BugFix] Fix precision anomaly of DeepSeek-V4 on A2 when FlashComm is enabled (#9488)", - "topics": [ - "deepseek-v4-dspark", - "communication", - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/ops/test_fused_moe.py", - "vllm_ascend/ops/fused_moe/fused_moe.py" - ] - }, - { - "commit": "3ddac57e0cbcd8d010c3747d0ee4d61b67f59566", - "subject": "[BugFix] Fix serial inference anomaly of DeepSeek-V4 (#9487)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/attention/dsa_v1.py" - ] - }, - { - "commit": "2331144f07d590e29a2fab5d8e9b2d323b052107", - "subject": " [Feature] DeepSeekV4 DSA indexer_select_qli cv multistream overlap (#9441)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/attention/dsa_v1.py", - "vllm_ascend/ops/dsa.py" - ] - }, - { - "commit": "7e817f291bc23935d66440b3576a66c3933ef921", - "subject": "[BugFix]Fix Deepseek-V4 async scheduling with MTP (#9456)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "c1be5934e36d0afd448bca5d2eef5a5ccd8de981", - "subject": "[BugFix][Model] Fix DeepSeek V4 hc_pre op and add 4-card E2E test (#9452)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/misc/model_dataset_list.json", - "csrc/build_aclnn.sh", - "tests/e2e/multicard/4-cards/test_deepseek_v4.py", - "vllm_ascend/models/deepseek_v4.py" - ] - }, - { - "commit": "20e338bd7c1267799dcf611acee025f37cdc9535", - "subject": "[Feature] Implement multistream parallelism overlap feature in DSA compressor process for DeepSeek-V4 (#9450)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/attention/dsa_v1.py", - "vllm_ascend/ops/dsa.py" - ] - }, - { - "commit": "8da8e0e6dbbd9c37d7bc10ce4a2a26d1745ba0d0", - "subject": "[Feature]add fiav2 into acl_graph (#9165)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "docs/source/tutorials/models/gpt-oss-120b.md", - "docs/source/tutorials/models/index.md", - "pyproject.toml", - "tests/ut/attention/test_attention_v1.py", - "vllm_ascend/ascend_forward_context.py", - "vllm_ascend/attention/attention_v1.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "845008118564c618e2584f47bff3bf5ce109e538", - "subject": "[BugFix][Ascend950]Fixup GMM_swiglu_quant and reshape_and_cache on 950 (#9432)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/device/device_op.py", - "vllm_ascend/ops/fused_moe/moe_mlp.py" - ] - }, - { - "commit": "7726528ec34cec8a72931e65215096e3792cbf4a", - "subject": "[Attention][Feature] Add mla prolog cv parallel for dsv4 (#9433)", - "topics": [ - "deepseek-v4-dspark", - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/ascend_config.py", - "vllm_ascend/attention/dsa_v1.py", - "vllm_ascend/ops/cv_linear.py", - "vllm_ascend/ops/dsa.py" - ] - }, - { - "commit": "99796b64b4487571904e61c1222f6339e4413978", - "subject": "[Feature] w4a8 dispatch_ffn_combine: support x_active_mask and fix bias as TensorList (#9403)", - "topics": [ - "communication", - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "csrc/mc2/dispatch_ffn_combine/dispatch_ffn_combine_torch_adpt.h", - "csrc/mc2/dispatch_ffn_combine/op_kernel/dispatch_ffn_combine_kernel.hpp", - "csrc/mc2/dispatch_ffn_combine/op_kernel/utils/block_epilogue_pertoken_swiglu.hpp", - "csrc/mc2/dispatch_ffn_combine_w4_a8/op_host/dispatch_ffn_combine_w4_a8_def.cpp", - "csrc/mc2/dispatch_ffn_combine_w4_a8/op_host/dispatch_ffn_combine_w4_a8_tiling.cpp", - "csrc/mc2/dispatch_ffn_combine_w4_a8/op_host/op_api/aclnn_dispatch_ffn_combine_w4_a8.cpp", - "csrc/mc2/dispatch_ffn_combine_w4_a8/op_host/op_api/aclnn_dispatch_ffn_combine_w4_a8.h", - "csrc/mc2/dispatch_ffn_combine_w4_a8/op_kernel/dispatch_ffn_combine_w4_a8.cpp", - "csrc/mc2/dispatch_ffn_combine_w4_a8/op_kernel/dispatch_ffn_combine_w4_a8.h", - "csrc/mc2/dispatch_ffn_combine_w4_a8/op_kernel/dispatch_ffn_combine_w4_a8_kernel.hpp", - "csrc/mc2/dispatch_ffn_combine_w4_a8/op_kernel/utils/block_epilogue_w4a8post_pertoken_swiglu.hpp", - "csrc/mc2/dispatch_ffn_combine_w4_a8/op_kernel/utils/block_epilogue_w4a8post_pertoken_v2.hpp", - "csrc/torch_binding.cpp", - "csrc/torch_binding_meta.cpp", - "tests/e2e/nightly/single_node/ops/multicard_ops_a3/test_dispatch_ffn_combine_w4a8.py" - ] - }, - { - "commit": "e1db5050aadb8da3fefa702c72a727b9222481bd", - "subject": "[BugFix][Platform] Remove redundant GLM tool parser patch (#9410)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/patch/platform/test_patch_glm_tool_call_parser.py", - "vllm_ascend/patch/__init__.py", - "vllm_ascend/patch/platform/__init__.py", - "vllm_ascend/patch/platform/patch_glm_tool_call_parser.py" - ] - }, - { - "commit": "743de37372c82ecde1e82f2ecd9ec547770c5e6d", - "subject": "[BugFix][Model] Fix NaN hidden states and low MTP acceptance rate when FlashComm1 is enabled (#9436)", - "topics": [ - "deepseek-v4-dspark", - "communication" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/models/deepseek_v4.py" - ] - }, - { - "commit": "771239e819a92bae41e0d95f101045691b77b69a", - "subject": "[Feature] support ffncombine swiglulimit (#9429)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "csrc/mc2/dispatch_ffn_combine/dispatch_ffn_combine_torch_adpt.h", - "csrc/mc2/dispatch_ffn_combine/op_host/dispatch_ffn_combine_def.cpp", - "csrc/mc2/dispatch_ffn_combine/op_host/dispatch_ffn_combine_tiling.cpp", - "csrc/mc2/dispatch_ffn_combine/op_host/op_api/aclnn_dispatch_ffn_combine.cpp", - "csrc/mc2/dispatch_ffn_combine/op_host/op_api/aclnn_dispatch_ffn_combine.h", - "csrc/mc2/dispatch_ffn_combine/op_kernel/dispatch_ffn_combine.h", - "csrc/mc2/dispatch_ffn_combine/op_kernel/dispatch_ffn_combine_kernel.hpp", - "csrc/mc2/dispatch_ffn_combine/op_kernel/dispatch_ffn_combine_tiling.h", - "csrc/mc2/dispatch_ffn_combine/op_kernel/utils/block_epilogue_pertoken_swiglu.hpp", - "csrc/torch_binding.cpp", - "csrc/torch_binding_meta.cpp", - "vllm_ascend/ops/fused_moe/moe_comm_method.py" - ] - }, - { - "commit": "10683d3d7ea5d662be21307f707726917b9fd575", - "subject": "[Feature][Model] Enable W4A8 per-channel GMM Swiglu quant (#9394)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/ops/test_moe_mlp.py", - "tests/ut/ops/test_token_dispatcher.py", - "tests/ut/quantization/methods/test_w4a8.py", - "vllm_ascend/ops/fused_moe/moe_mlp.py", - "vllm_ascend/ops/fused_moe/moe_runtime_args.py", - "vllm_ascend/ops/fused_moe/moe_stage_params.py", - "vllm_ascend/ops/fused_moe/token_dispatcher.py", - "vllm_ascend/quantization/methods/w4a8.py" - ] - }, - { - "commit": "68a4db5554475d8e413f13d84016b86f5d2c18b7", - "subject": "[BugFix][A2]Avoid duplicate MoE all-reduce for DeepSeek V4 tensor outputs (#9389)", - "topics": [ - "deepseek-v4-dspark", - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/models/deepseek_v4.py" - ] - }, - { - "commit": "365c6e5e54218d4edbab6fc95aea8bc01bd5388e", - "subject": "[BugFix][CI][310p] Fix CI error for 310p caused by DSV4 (#9402)", - "topics": [ - "deepseek-v4-dspark", - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/_310p/fused_moe/test_moe_mlp_310.py", - "tests/ut/_310p/test_block_table_310p.py", - "vllm_ascend/_310p/block_table.py", - "vllm_ascend/_310p/fused_moe/moe_comm_method.py", - "vllm_ascend/_310p/model_runner_310p.py", - "vllm_ascend/_310p/npu_input_batch.py" - ] - }, - { - "commit": "0e96eb5879ea3865aae0755b998b5c5701bbfeac", - "subject": "[CI] update weekly cases: Deepseekv3.2-w8a8 ; Glm-5.1-w8a8 ; Qwen3.5-397B-w8a8 (#9357)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "tests/e2e/weekly/multi_node/configs/DeepSeek-V3_2-W8A8-EP_weekly.yaml", - "tests/e2e/weekly/single_node/configs/DeepSeek-V3.2-W8A8_A3_weekly.yaml", - "tests/e2e/weekly/single_node/configs/GLM-5_1-W8A8_A3_weekly.yaml", - "tests/e2e/weekly/single_node/configs/Qwen3.5-397B-A17B-W8A8-mtp-A3_weekly.yaml" - ] - }, - { - "commit": "a8f7beafe7d4bca2cbf669d0ba709536084a12b7", - "subject": "[CI] add weekly case (#9380)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/misc/model_dataset_list.json", - ".github/workflows/schedule_nightly_test_a3.yaml", - ".github/workflows/schedule_weekly_test_a3.yaml", - "tests/e2e/nightly/multi_node/config/GLM5_1-W8A8-A3-dual-nodes.yaml", - "tests/e2e/nightly/single_node/models/configs/DeepSeek-V3.2-W8A8.yaml", - "tests/e2e/nightly/single_node/models/configs/Kimi-K2.5.yaml", - "tests/e2e/nightly/single_node/models/configs/MiniMax-M2.5-w8a8-QuaRot-A3.yaml", - "tests/e2e/nightly/single_node/models/configs/Qwen3.5-122B-A10B-W8A8-A3.yaml", - "tests/e2e/nightly/single_node/models/configs/Qwen3.5-27B-w8a8-A3.yaml", - "tests/e2e/nightly/single_node/models/configs/Qwen3.5-397B-A17B-W8A8-mtp-A3.yaml", - "tests/e2e/weekly/single_node/configs/Kimi-K2.5-32k-512.yaml", - "tests/e2e/weekly/single_node/configs/Kimi-K2.5.yaml", - "tests/e2e/weekly/single_node/configs/Qwen3.5-122B-A10B-W8A8-A3.yaml" - ] - }, - { - "commit": "fcc2e56374026b29bed8e55162057eee9901f1af", - "subject": "[Ascend950][Feature][quant] enable mc2 combine mxfp8 quant in A5 (#9328)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/ops/test_token_dispatcher.py", - "vllm_ascend/ops/fused_moe/moe_stage_contracts.py", - "vllm_ascend/ops/fused_moe/token_dispatcher.py" - ] - }, - { - "commit": "d31c223a8b74a0367ce4946c9053b6abb15d1e32", - "subject": "[Feature][Model] Switch DeepSeekV4 hc_pre to fused op (#9396)", - "topics": [ - "deepseek-v4-dspark", - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "csrc/torch_binding.cpp", - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_npu_hc_pre.py", - "vllm_ascend/models/deepseek_v4.py" - ] - }, - { - "commit": "5921661464b16a59a9a0f2b0e85f42401941fd29", - "subject": "[Refactor][SpecDecode] Move AscendSpecDecodeBaseProposer to llm_base_proposer.py (#9251)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/spec_decode/test_eagle_proposer.py", - "vllm_ascend/spec_decode/draft_proposer.py", - "vllm_ascend/spec_decode/eagle_proposer.py", - "vllm_ascend/spec_decode/llm_base_proposer.py" - ] - }, - { - "commit": "958daf83205ecad0afec3784be0b2a8b38725cd8", - "subject": "[Refactor]: migrate env vars to AscendConfig (#9064)", - "topics": [ - "communication", - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/workflows/_e2e_test.yaml", - "tests/ut/attention/test_sfa_v1.py", - "tests/ut/batch_invariant/test_batch_invariant.py", - "tests/ut/distributed/test_parallel_state.py", - "tests/ut/eplb/adaptor/test_vllm_adaptor.py", - "tests/ut/ops/test_fused_moe.py", - "tests/ut/ops/test_linear.py", - "tests/ut/ops/test_prepare_finalize.py", - "tests/ut/profiler/test_torch_npu_profiler.py", - "tests/ut/quantization/methods/test_w8a16.py", - "tests/ut/quantization/methods/test_w8a8_dynamic.py", - "tests/ut/quantization/methods/test_w8a8_static.py", - "tests/ut/spec_decode/test_eagle_proposer.py", - "tests/ut/test_ascend_config.py", - "tests/ut/test_platform.py", - "tests/ut/test_utils.py", - "tests/ut/worker/test_model_runner_v1_with_device.py", - "tests/ut/worker/test_worker_v1.py", - "vllm_ascend/ascend_config.py", - "vllm_ascend/ascend_forward_context.py", - "vllm_ascend/attention/sfa_v1.py", - "vllm_ascend/attention/utils.py", - "vllm_ascend/batch_invariant.py", - "vllm_ascend/distributed/kv_transfer/kv_p2p/mooncake_connector.py", - "vllm_ascend/envs.py", - "vllm_ascend/eplb/adaptor/vllm_adaptor.py", - "vllm_ascend/ops/fused_moe/fused_moe.py", - "vllm_ascend/ops/fused_moe/moe_comm_method.py", - "vllm_ascend/patch/__init__.py", - "vllm_ascend/patch/platform/__init__.py", - "vllm_ascend/patch/platform/patch_balance_schedule.py", - "vllm_ascend/platform.py", - "vllm_ascend/profiler/torch_npu_profiler.py", - "vllm_ascend/quantization/methods/w8a8_dynamic.py", - "vllm_ascend/utils.py", - "vllm_ascend/worker/worker.py", - "vllm_ascend/xlite/xlite.py" - ] - }, - { - "commit": "eb2bdc2e73297f2d55af1fdce681d2f1d072c23c", - "subject": "[Feature][main][new] The DeepSeek V4 model supports kv pool adaptation. (#9385)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/distributed/kv_transfer/kv_pool/ascend_store/ascend_store_connector.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/ascend_store/backend/mooncake_backend.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/ascend_store/config_data.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/ascend_store/kv_transfer.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/ascend_store/pool_scheduler.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/ascend_store/pool_worker.py" - ] - }, - { - "commit": "19278374b9b1173ce7df83d32479d4820a3b4a88", - "subject": "[BugFix][KVCache] Separate Deepseek-V4 MTP layer KVCache sharding (#9367)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/patch/platform/patch_kv_cache_utils.py" - ] - }, - { - "commit": "420a8ba94d77da4a022e09d61bb19700313aec77", - "subject": "[Ascend950][BugFix] fix npu_add_rms_norm_dynamic_quant error in A5 (#9334)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/compilation/passes/norm_quant_fusion_pass.py" - ] - }, - { - "commit": "7bce23cc0572a7793636ae215015a528e4503f1e", - "subject": "[Feature] Support DeepseekV4 (#9270)", - "topics": [ - "deepseek-v4-dspark", - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "setup.py", - "tests/ut/core/test_profiling_chunk.py", - "tests/ut/eplb/core/test_eplb_utils.py", - "tests/ut/ops/test_fused_moe.py", - "tests/ut/ops/test_moe_comm_method.py", - "tests/ut/patch/platform/test_patch_deepseek_v4_tool_call_parser.py", - "tests/ut/quantization/methods/test_w4a16.py", - "tests/ut/quantization/methods/test_w4a8.py", - "tests/ut/quantization/methods/test_w8a8_dynamic.py", - "tests/ut/quantization/methods/test_w8a8_mxfp8.py", - "tests/ut/worker/test_dsv4_compressed_positions.py", - "tests/ut/worker/test_model_runner_v1.py", - "vllm_ascend/_310p/quantization/methods/w8a8_dynamic.py", - "vllm_ascend/_310p/quantization/modelslim_config.py", - "vllm_ascend/__init__.py", - "vllm_ascend/ascend_config.py", - "vllm_ascend/ascend_forward_context.py", - "vllm_ascend/attention/abstract.py", - "vllm_ascend/attention/dsa_v1.py", - "vllm_ascend/attention/utils.py", - "vllm_ascend/core/single_type_kv_cache_manager.py", - "vllm_ascend/distributed/kv_transfer/__init__.py", - "vllm_ascend/distributed/kv_transfer/kv_p2p/mooncake_hybrid_connector.py", - "vllm_ascend/distributed/utils.py", - "vllm_ascend/envs.py", - "vllm_ascend/models/__init__.py", - "vllm_ascend/models/deepseek_v4.py", - "vllm_ascend/models/deepseek_v4_mtp.py", - "vllm_ascend/models/layer/attention/__init__.py", - "vllm_ascend/models/layer/attention/layer.py", - "vllm_ascend/ops/dsa.py", - "vllm_ascend/ops/fused_moe/experts_selector.py", - "vllm_ascend/ops/fused_moe/fused_moe.py", - "vllm_ascend/ops/fused_moe/moe_comm_method.py", - "vllm_ascend/ops/fused_moe/moe_mlp.py", - "vllm_ascend/ops/fused_moe/moe_runtime_args.py", - "vllm_ascend/ops/fused_moe/moe_stage_contracts.py", - "vllm_ascend/ops/fused_moe/prepare_finalize.py", - "vllm_ascend/ops/linear.py", - "vllm_ascend/ops/linear_op.py", - "vllm_ascend/ops/mhc.py", - "vllm_ascend/ops/rope_dsv4.py", - "vllm_ascend/ops/triton/mul_add.py", - "vllm_ascend/ops/triton/rms_norm.py", - "vllm_ascend/patch/__init__.py", - "vllm_ascend/patch/platform/__init__.py", - "vllm_ascend/patch/platform/patch_deepseek_v4_tool_call_parser.py", - "vllm_ascend/patch/platform/patch_kv_cache_coordinator.py", - "vllm_ascend/patch/platform/patch_kv_cache_interface.py", - "vllm_ascend/patch/platform/patch_kv_cache_utils.py", - "vllm_ascend/patch/platform/patch_speculative_config.py", - "vllm_ascend/patch/worker/__init__.py", - "vllm_ascend/patch/worker/patch_deepseek_compressor.py", - "vllm_ascend/platform.py", - "vllm_ascend/quantization/compressed_tensors_config.py", - "vllm_ascend/quantization/method_adapters.py", - "vllm_ascend/quantization/methods/base.py", - "vllm_ascend/quantization/methods/w4a16.py", - "vllm_ascend/quantization/methods/w4a4_mxfp4.py", - "vllm_ascend/quantization/methods/w4a8.py", - "vllm_ascend/quantization/methods/w8a8_dynamic.py", - "vllm_ascend/quantization/methods/w8a8_mxfp8.py", - "vllm_ascend/quantization/modelslim_config.py", - "vllm_ascend/spec_decode/eagle_proposer.py", - "vllm_ascend/utils.py", - "vllm_ascend/worker/block_table.py", - "vllm_ascend/worker/model_runner_v1.py", - "vllm_ascend/worker/npu_input_batch.py" - ] - }, - { - "commit": "d4b36710fef6cf14544b911d5af656378def55b9", - "subject": "[Ops][BugFix] Register DeepSeek V4 custom ops for 910B (#9339)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - "csrc/build_aclnn.sh" - ] - }, - { - "commit": "55edd9f6635b67bfc8d395b4f0220418f8acee67", - "subject": "[Feature][Ops] Add A5 custom operator build support (#9271)", - "topics": [ - "deepseek-v4-dspark", - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "CMakeLists.txt", - "csrc/attention/indexer_compress_epilog/CMakeLists.txt", - "csrc/attention/indexer_compress_epilog/op_host/CMakeLists.txt", - "csrc/attention/indexer_compress_epilog/op_host/indexer_compress_epilog_def.cpp", - "csrc/attention/indexer_compress_epilog/op_host/indexer_compress_epilog_proto.cpp", - "csrc/attention/indexer_compress_epilog/op_host/indexer_compress_epilog_tiling.cpp", - "csrc/attention/indexer_compress_epilog/op_host/indexer_compress_epilog_tiling.h", - "csrc/attention/indexer_compress_epilog/op_kernel/indexer_compress_epilog.cpp", - "csrc/attention/indexer_compress_epilog/op_kernel/indexer_compress_epilog_base.h", - "csrc/attention/indexer_compress_epilog/op_kernel/indexer_compress_epilog_multi_row.h", - "csrc/attention/indexer_compress_epilog/op_kernel/indexer_compress_epilog_multi_row_mx_fp8.h", - "csrc/attention/indexer_compress_epilog/op_kernel/indexer_compress_epilog_single_row.h", - "csrc/attention/indexer_compress_epilog/op_kernel/indexer_compress_epilog_single_row_mx_fp8.h", - "csrc/attention/indexer_compress_epilog_v2/CMakeLists.txt", - "csrc/attention/indexer_compress_epilog_v2/op_host/CMakeLists.txt", - "csrc/attention/indexer_compress_epilog_v2/op_host/indexer_compress_epilog_v2_def.cpp", - "csrc/attention/indexer_compress_epilog_v2/op_host/indexer_compress_epilog_v2_proto.cpp", - "csrc/attention/indexer_compress_epilog_v2/op_host/indexer_compress_epilog_v2_tiling.cpp", - "csrc/attention/indexer_compress_epilog_v2/op_host/indexer_compress_epilog_v2_tiling.h", - "csrc/attention/indexer_compress_epilog_v2/op_kernel/indexer_compress_epilog_v2.cpp", - "csrc/attention/indexer_compress_epilog_v2/op_kernel/indexer_compress_epilog_v2_base.h", - "csrc/attention/indexer_compress_epilog_v2/op_kernel/indexer_compress_epilog_v2_multi_row.h", - "csrc/attention/indexer_compress_epilog_v2/op_kernel/indexer_compress_epilog_v2_single_row.h", - "csrc/attention/kv_compress_epilog/CMakeLists.txt", - "csrc/attention/kv_compress_epilog/op_host/CMakeLists.txt", - "csrc/attention/kv_compress_epilog/op_host/kv_compress_epilog_def.cpp", - "csrc/attention/kv_compress_epilog/op_host/kv_compress_epilog_tiling_arch35.cpp", - "csrc/attention/kv_compress_epilog/op_host/kv_compress_epilog_tiling_arch35.h", - "csrc/attention/kv_compress_epilog/op_kernel/kv_compress_epilog.cpp", - "csrc/attention/kv_compress_epilog/op_kernel/kv_compress_epilog.h", - "csrc/attention/kv_compress_epilog/op_kernel/kv_compress_epilog_common.h", - "csrc/attention/kv_quant_sparse_attn_sharedkv/CMakeLists.txt", - "csrc/attention/kv_quant_sparse_attn_sharedkv/README.md", - "csrc/attention/kv_quant_sparse_attn_sharedkv/op_host/CMakeLists.txt", - "csrc/attention/kv_quant_sparse_attn_sharedkv/op_host/kv_quant_sparse_attn_sharedkv_check.cpp", - "csrc/attention/kv_quant_sparse_attn_sharedkv/op_host/kv_quant_sparse_attn_sharedkv_check.h", - "csrc/attention/kv_quant_sparse_attn_sharedkv/op_host/kv_quant_sparse_attn_sharedkv_check_consistancy.cpp", - "csrc/attention/kv_quant_sparse_attn_sharedkv/op_host/kv_quant_sparse_attn_sharedkv_check_existance.cpp", - "csrc/attention/kv_quant_sparse_attn_sharedkv/op_host/kv_quant_sparse_attn_sharedkv_check_feature.cpp", - "csrc/attention/kv_quant_sparse_attn_sharedkv/op_host/kv_quant_sparse_attn_sharedkv_check_single_para.cpp", - "csrc/attention/kv_quant_sparse_attn_sharedkv/op_host/kv_quant_sparse_attn_sharedkv_def.cpp", - "csrc/attention/kv_quant_sparse_attn_sharedkv/op_host/kv_quant_sparse_attn_sharedkv_proto.cpp", - "csrc/attention/kv_quant_sparse_attn_sharedkv/op_host/kv_quant_sparse_attn_sharedkv_tiling.cpp", - "csrc/attention/kv_quant_sparse_attn_sharedkv/op_host/kv_quant_sparse_attn_sharedkv_tiling.h", - "csrc/attention/kv_quant_sparse_attn_sharedkv/op_kernel/arch35/common/CopyInL1.h", - "csrc/attention/kv_quant_sparse_attn_sharedkv/op_kernel/arch35/common/FixpipeOut.h", - "csrc/attention/kv_quant_sparse_attn_sharedkv/op_kernel/arch35/common/buffer.h", - "csrc/attention/kv_quant_sparse_attn_sharedkv/op_kernel/arch35/common/buffer_manager.h", - "csrc/attention/kv_quant_sparse_attn_sharedkv/op_kernel/arch35/common/buffers_policy.h", - "csrc/attention/kv_quant_sparse_attn_sharedkv/op_kernel/arch35/common/matmul.h", - "csrc/attention/kv_quant_sparse_attn_sharedkv/op_kernel/arch35/common/offset_calculator.h", - "csrc/attention/kv_quant_sparse_attn_sharedkv/op_kernel/arch35/kv_quant_sparse_attn_sharedkv_common_arch35.h", - "csrc/attention/kv_quant_sparse_attn_sharedkv/op_kernel/arch35/kv_quant_sparse_attn_sharedkv_kvcache.h", - "csrc/attention/kv_quant_sparse_attn_sharedkv/op_kernel/arch35/kv_quant_sparse_attn_sharedkv_scfa_block_cube.h", - "csrc/attention/kv_quant_sparse_attn_sharedkv/op_kernel/arch35/kv_quant_sparse_attn_sharedkv_scfa_block_vector.h", - "csrc/attention/kv_quant_sparse_attn_sharedkv/op_kernel/arch35/kv_quant_sparse_attn_sharedkv_scfa_kernel.h", - "csrc/attention/kv_quant_sparse_attn_sharedkv/op_kernel/arch35/util_regbase.h", - "csrc/attention/kv_quant_sparse_attn_sharedkv/op_kernel/arch35/vf/vf_basic_block_aligned128_no_update_scfa.h", - "csrc/attention/kv_quant_sparse_attn_sharedkv/op_kernel/arch35/vf/vf_basic_block_aligned128_update_scfa.h", - "csrc/attention/kv_quant_sparse_attn_sharedkv/op_kernel/arch35/vf/vf_basic_block_unaligned128_no_update_scfa.h", - "csrc/attention/kv_quant_sparse_attn_sharedkv/op_kernel/arch35/vf/vf_basic_block_unaligned128_update_scfa.h", - "csrc/attention/kv_quant_sparse_attn_sharedkv/op_kernel/arch35/vf/vf_basic_block_unaligned64_no_update_scfa.h", - "csrc/attention/kv_quant_sparse_attn_sharedkv/op_kernel/arch35/vf/vf_basic_block_unaligned64_update_scfa.h", - "csrc/attention/kv_quant_sparse_attn_sharedkv/op_kernel/arch35/vf/vf_basic_block_utils.h", - "csrc/attention/kv_quant_sparse_attn_sharedkv/op_kernel/arch35/vf/vf_flashupdate_new_scfa.h", - "csrc/attention/kv_quant_sparse_attn_sharedkv/op_kernel/arch35/vf/vf_mul_sel_softmaxflashv2_cast_nz_scfa.h", - "csrc/attention/kv_quant_sparse_attn_sharedkv/op_kernel/kv_quant_sparse_attn_sharedkv.cpp", - "csrc/attention/kv_quant_sparse_attn_sharedkv/op_kernel/kv_quant_sparse_attn_sharedkv_common.h", - "csrc/attention/kv_quant_sparse_attn_sharedkv/op_kernel/kv_quant_sparse_attn_sharedkv_metadata.h", - "csrc/attention/kv_quant_sparse_attn_sharedkv/op_kernel/kv_quant_sparse_attn_sharedkv_template_tiling_key.h", - "csrc/attention/kv_quant_sparse_attn_sharedkv_metadata/CMakeLists.txt", - "csrc/attention/kv_quant_sparse_attn_sharedkv_metadata/README.md", - "csrc/attention/kv_quant_sparse_attn_sharedkv_metadata/op_graph/kv_quant_sparse_attn_sharedkv_metadata_proto.h", - "csrc/attention/kv_quant_sparse_attn_sharedkv_metadata/op_host/CMakeLists.txt", - "csrc/attention/kv_quant_sparse_attn_sharedkv_metadata/op_host/kv_quant_sparse_attn_sharedkv_metadata_infershape.cpp", - "csrc/attention/kv_quant_sparse_attn_sharedkv_metadata/op_host/op_api/aclnn_kv_quant_sparse_attn_sharedkv_metadata.cpp", - "csrc/attention/kv_quant_sparse_attn_sharedkv_metadata/op_host/op_api/aclnn_kv_quant_sparse_attn_sharedkv_metadata.h", - "csrc/attention/kv_quant_sparse_attn_sharedkv_metadata/op_host/op_api/l0_kv_quant_sparse_attn_sharedkv_metadata.cpp", - "csrc/attention/kv_quant_sparse_attn_sharedkv_metadata/op_host/op_api/l0_kv_quant_sparse_attn_sharedkv_metadata.h", - "csrc/attention/kv_quant_sparse_attn_sharedkv_metadata/op_kernel_aicpu/CMakeLists.txt", - "csrc/attention/kv_quant_sparse_attn_sharedkv_metadata/op_kernel_aicpu/kv_quant_sparse_attn_sharedkv_metadata_aicpu.cpp", - "csrc/attention/kv_quant_sparse_attn_sharedkv_metadata/op_kernel_aicpu/kv_quant_sparse_attn_sharedkv_metadata_aicpu.h", - "csrc/attention/kv_quant_sparse_attn_sharedkv_metadata/op_kernel_aicpu/kv_quant_sparse_attn_sharedkv_metadata_aicpu.json", - "csrc/attention/load_index_kv_cache/CMakeLists.txt", - "csrc/attention/load_index_kv_cache/op_host/CMakeLists.txt", - "csrc/attention/load_index_kv_cache/op_host/load_index_kv_cache_def.cpp", - "csrc/attention/load_index_kv_cache/op_host/load_index_kv_cache_proto.cpp", - "csrc/attention/load_index_kv_cache/op_host/load_index_kv_cache_tiling.cpp", - "csrc/attention/load_index_kv_cache/op_host/load_index_kv_cache_tiling.h", - "csrc/attention/load_index_kv_cache/op_kernel/load_index_kv_cache.cpp", - "csrc/attention/load_index_kv_cache/op_kernel/load_index_kv_cache_base.h", - "csrc/attention/load_index_kv_cache/op_kernel/load_index_kv_cache_perf.h", - "csrc/attention/quant_lightning_indexer/op_host/quant_lightning_indexer_def.cpp", - "csrc/attention/quant_lightning_indexer/op_kernel/arch35/quant_lightning_indexer_common.h", - "csrc/attention/quant_lightning_indexer/op_kernel/arch35/quant_lightning_indexer_kernel.h", - "csrc/attention/quant_lightning_indexer/op_kernel/arch35/quant_lightning_indexer_service_cube.h", - "csrc/attention/quant_lightning_indexer/op_kernel/arch35/quant_lightning_indexer_service_vector.h", - "csrc/build_aclnn.sh", - "csrc/cmake/scripts/util/const_var.py", - "csrc/moe/hc_pre/CMakeLists.txt", - "csrc/moe/hc_pre/op_host/CMakeLists.txt", - "csrc/moe/hc_pre/op_host/hc_pre_def.cpp", - "csrc/moe/hc_pre/op_host/hc_pre_proto.cpp", - "csrc/moe/hc_pre/op_host/hc_pre_tiling.cpp", - "csrc/moe/hc_pre/op_host/hc_pre_tiling.h", - "csrc/moe/hc_pre/op_host/hc_pre_tiling_arch35.h", - "csrc/moe/hc_pre/op_kernel/hc_pre.cpp", - "csrc/moe/hc_pre/op_kernel/hc_pre_base.h", - "csrc/moe/hc_pre/op_kernel/hc_pre_base_arch35.h", - "csrc/moe/hc_pre/op_kernel/hc_pre_cube_compute.h", - "csrc/moe/hc_pre/op_kernel/hc_pre_cube_compute_arch35.h", - "csrc/moe/hc_pre/op_kernel/hc_pre_m_k_split_core.h", - "csrc/moe/hc_pre/op_kernel/hc_pre_m_k_split_core_arch35.h", - "csrc/moe/hc_pre/op_kernel/hc_pre_m_split_core_arch35.h", - "csrc/moe/swiglu_group_quant/CMakeLists.txt", - "csrc/moe/swiglu_group_quant/op_host/CMakeLists.txt", - "csrc/moe/swiglu_group_quant/op_host/swiglu_group_quant_def.cpp", - "csrc/moe/swiglu_group_quant/op_host/swiglu_group_quant_proto.cpp", - "csrc/moe/swiglu_group_quant/op_host/swiglu_group_quant_tiling.cpp", - "csrc/moe/swiglu_group_quant/op_host/swiglu_group_quant_tiling.h", - "csrc/moe/swiglu_group_quant/op_kernel/swiglu_fp8_quant_per_token.h", - "csrc/moe/swiglu_group_quant/op_kernel/swiglu_group_quant.cpp", - "csrc/moe/swiglu_group_quant/op_kernel/swiglu_group_quant_base.h", - "csrc/moe/swiglu_group_quant/op_kernel/swiglu_group_quant_perf.h", - "csrc/moe/swiglu_group_quant/op_kernel/swiglu_mx_quant_perf.h", - "csrc/torch_binding.cpp", - "csrc/torch_binding_meta.cpp" - ] - }, - { - "commit": "5e7358e9de7f3549a9be5d51c3b67268b55481b0", - "subject": "[CI]Style: Convert `test/` to ruff format(Batch #3) (#6744)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "pyproject.toml", - "tests/e2e/nightly/single_node/models/scripts/single_node_config.py", - "tests/e2e/nightly/single_node/models/scripts/test_single_node.py", - "tests/e2e/nightly/single_node/ops/conftest.py", - "tests/e2e/nightly/single_node/ops/multicard_ops_a2/test_matmul_allreduce_add_rmsnorm.py", - "tests/e2e/nightly/single_node/ops/multicard_ops_a3/test_dispatch_ffn_combine.py", - "tests/e2e/nightly/single_node/ops/multicard_ops_a3/test_dispatch_ffn_combine_bf16.py", - "tests/e2e/nightly/single_node/ops/multicard_ops_a3/test_dispatch_ffn_combine_w4a8.py", - "tests/e2e/nightly/single_node/ops/multicard_ops_a3/test_dispatch_gmm_combine_decode.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_add_rms_norm_bias.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_apply_top_k_top_p_custom.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_batch_matmul_transpose.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_bgmv_expand.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_bgmv_shrink.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_causal_conv1d_310.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_copy_and_expand_eagle_inputs.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_fused_moe.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_gating_top_k_softmax.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_gmm_swiglu_quant_weight_nz_tensor_list.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_grouped_matmul_swiglu_quant.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_hamming_dist_top_k.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_mla_preprocess.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_mla_preprocess_nq.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_mla_preprocess_qdown.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_moe_init_routing_custom.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_ngram_spec_decode.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_npu_moe_gating_top_k.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_recurrent_gated_delta_rule.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_reshape_and_cache_bnsd.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_transpose_kv_cache_by_block.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_vocabparallelembedding.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_apply_penalties_triton.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_bad_words.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_batch_memcpy.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_bincount.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_causal_conv1d.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_compute_slot_mapping.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_compute_token_logprobs.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_compute_topk_logprobs.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_fused_qkvzba_split_reshape_cat.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_fused_sigmoid_gating_delta_rule.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_gdn_chunk_meta.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_l2norm.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_log_softmax.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_min_p.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_mrope.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_muls_add.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_penality.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_post_update.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_prepare_inputs_padded.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_rejection_sample.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_rope.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_split_qkv_rmsnorm_mrope.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_split_qkv_rmsnorm_rope.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_split_qkv_tp_rmsnorm_rope.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_temperature.py" - ] - }, - { - "commit": "9024b5cf83156c6ef181cc59628bfb47d6dde7ec", - "subject": "[EPLB][Feature] EPLB Support w4a8 (#6263)", - "topics": [ - "deepseek-v4-dspark", - "communication", - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "docs/source/user_guide/feature_guide/eplb_swift_balancer.md", - "vllm_ascend/ascend_forward_context.py", - "vllm_ascend/eplb/adaptor/vllm_adaptor.py", - "vllm_ascend/ops/fused_moe/moe_mlp.py", - "vllm_ascend/quantization/methods/w4a8.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "b4f01009860b0b00e243e1eb95d0107b332c1bad", - "subject": "[Ops][Feature] Add DeepSeek V4 custom operators (#9228)", - "topics": [ - "deepseek-v4-dspark", - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "csrc/aclnn_torch_adapter/op_api_common.h", - "csrc/attention/compressor/CMakeLists.txt", - "csrc/attention/compressor/README.md", - "csrc/attention/compressor/op_host/CMakeLists.txt", - "csrc/attention/compressor/op_host/compressor_def.cpp", - "csrc/attention/compressor/op_host/compressor_proto.cpp", - "csrc/attention/compressor/op_host/compressor_tiling.cpp", - "csrc/attention/compressor/op_host/compressor_tiling.h", - "csrc/attention/compressor/op_kernel/arch32/compressor_block_cube_perf.h", - "csrc/attention/compressor/op_kernel/arch32/compressor_block_vec_perf.h", - "csrc/attention/compressor/op_kernel/arch32/compressor_vector_comm.h", - "csrc/attention/compressor/op_kernel/arch32/rms_norm.h", - "csrc/attention/compressor/op_kernel/arch32/rope.h", - "csrc/attention/compressor/op_kernel/arch32/soft_max.h", - "csrc/attention/compressor/op_kernel/arch35/compressor_block_cube_perf.h", - "csrc/attention/compressor/op_kernel/arch35/compressor_block_vec_perf.h", - "csrc/attention/compressor/op_kernel/arch35/vf/vf_add.h", - "csrc/attention/compressor/op_kernel/arch35/vf/vf_mul.h", - "csrc/attention/compressor/op_kernel/arch35/vf/vf_rms_norm.h", - "csrc/attention/compressor/op_kernel/arch35/vf/vf_rope.h", - "csrc/attention/compressor/op_kernel/arch35/vf/vf_softmax.h", - "csrc/attention/compressor/op_kernel/compressor.cpp", - "csrc/attention/compressor/op_kernel/compressor_comm.h", - "csrc/attention/compressor/op_kernel/compressor_kernel.h", - "csrc/attention/compressor/op_kernel/compressor_kernel_perf.h", - "csrc/attention/compressor/op_kernel/compressor_template_tiling_key.h", - "csrc/attention/compressor/op_kernel/compressor_tiling_data.h", - "csrc/attention/compressor/op_kernel/compressor_tools.h", - "csrc/attention/inplace_partial_rotary_mul/CMakeLists.txt", - "csrc/attention/inplace_partial_rotary_mul/op_host/CMakeLists.txt", - "csrc/attention/inplace_partial_rotary_mul/op_host/inplace_partial_rotary_mul_a3_tiling.cpp", - "csrc/attention/inplace_partial_rotary_mul/op_host/inplace_partial_rotary_mul_def.cpp", - "csrc/attention/inplace_partial_rotary_mul/op_host/inplace_partial_rotary_mul_proto.cpp", - "csrc/attention/inplace_partial_rotary_mul/op_host/inplace_partial_rotary_mul_tiling.cpp", - "csrc/attention/inplace_partial_rotary_mul/op_host/inplace_partial_rotary_mul_tiling.h", - "csrc/attention/inplace_partial_rotary_mul/op_host/rope_regbase_tiling_a_and_b.cpp", - "csrc/attention/inplace_partial_rotary_mul/op_host/rope_regbase_tiling_ab.cpp", - "csrc/attention/inplace_partial_rotary_mul/op_host/rope_regbase_tiling_aba_and_ba.cpp", - "csrc/attention/inplace_partial_rotary_mul/op_host/rope_regbase_tiling_bab.cpp", - "csrc/attention/inplace_partial_rotary_mul/op_host/rope_regbase_tiling_base.cpp", - "csrc/attention/inplace_partial_rotary_mul/op_kernel/apply_rotary_pos_emb_common.h", - "csrc/attention/inplace_partial_rotary_mul/op_kernel/inplace_partial_rotary_mul.cpp", - "csrc/attention/inplace_partial_rotary_mul/op_kernel/inplace_partial_rotary_mul.h", - "csrc/attention/inplace_partial_rotary_mul/op_kernel/inplace_partial_rotary_mul_common.h", - "csrc/attention/inplace_partial_rotary_mul/op_kernel/rotary_position_embedding_reg_a_and_b.h", - "csrc/attention/inplace_partial_rotary_mul/op_kernel/rotary_position_embedding_reg_ab.h", - "csrc/attention/inplace_partial_rotary_mul/op_kernel/rotary_position_embedding_reg_aba_and_ba.h", - "csrc/attention/inplace_partial_rotary_mul/op_kernel/rotary_position_embedding_reg_bab.h", - "csrc/attention/inplace_partial_rotary_mul/op_kernel/rotate_half.h", - "csrc/attention/inplace_partial_rotary_mul/op_kernel/rotate_half_base.h", - "csrc/attention/inplace_partial_rotary_mul/op_kernel/rotate_half_bf16.h", - "csrc/attention/inplace_partial_rotary_mul/op_kernel/rotate_interleaved_common.h", - "csrc/attention/inplace_partial_rotary_mul/op_kernel/rotate_interleaved_split_bs.h", - "csrc/attention/inplace_partial_rotary_mul/op_kernel/rotate_interleaved_split_bs_pad.h", - "csrc/attention/inplace_partial_rotary_mul/op_kernel/rotate_interleaved_split_bsn.h", - "csrc/attention/inplace_partial_rotary_mul/op_kernel/rotate_interleaved_split_bsn_pad.h", - "csrc/attention/inplace_partial_rotary_mul/op_kernel/rotate_interleaved_split_s.h", - "csrc/attention/inplace_partial_rotary_mul/op_kernel/rotate_interleaved_split_s_pad.h", - "csrc/attention/quant_lightning_indexer/CMakeLists.txt", - "csrc/attention/quant_lightning_indexer/README.md", - "csrc/attention/quant_lightning_indexer/op_host/CMakeLists.txt", - "csrc/attention/quant_lightning_indexer/op_host/quant_lightning_indexer_def.cpp", - "csrc/attention/quant_lightning_indexer/op_host/quant_lightning_indexer_infershape.cpp", - "csrc/attention/quant_lightning_indexer/op_host/quant_lightning_indexer_tiling.cpp", - "csrc/attention/quant_lightning_indexer/op_host/quant_lightning_indexer_tiling.h", - "csrc/attention/quant_lightning_indexer/op_kernel/arch32/quant_lightning_indexer_common.h", - "csrc/attention/quant_lightning_indexer/op_kernel/arch32/quant_lightning_indexer_kernel.h", - "csrc/attention/quant_lightning_indexer/op_kernel/arch32/quant_lightning_indexer_service_cube.h", - "csrc/attention/quant_lightning_indexer/op_kernel/arch32/quant_lightning_indexer_service_vector.h", - "csrc/attention/quant_lightning_indexer/op_kernel/arch32/quant_lightning_indexer_vector.h", - "csrc/attention/quant_lightning_indexer/op_kernel/arch35/quant_lightning_indexer_common.h", - "csrc/attention/quant_lightning_indexer/op_kernel/arch35/quant_lightning_indexer_kernel.h", - "csrc/attention/quant_lightning_indexer/op_kernel/arch35/quant_lightning_indexer_service_cube.h", - "csrc/attention/quant_lightning_indexer/op_kernel/arch35/quant_lightning_indexer_service_vector.h", - "csrc/attention/quant_lightning_indexer/op_kernel/arch35/vf/quant_lightning_indexer_topk.h", - "csrc/attention/quant_lightning_indexer/op_kernel/arch35/vf/quant_lightning_indexer_vector1.h", - "csrc/attention/quant_lightning_indexer/op_kernel/arch35/vf/vf_topk.h", - "csrc/attention/quant_lightning_indexer/op_kernel/arch35/vf/vf_topk_16_gather.h", - "csrc/attention/quant_lightning_indexer/op_kernel/quant_lightning_indexer.cpp", - "csrc/attention/quant_lightning_indexer/op_kernel/quant_lightning_indexer_metadata.h", - "csrc/attention/quant_lightning_indexer/op_kernel/quant_lightning_indexer_template_tiling_key.h", - "csrc/attention/quant_lightning_indexer_metadata/CMakeLists.txt", - "csrc/attention/quant_lightning_indexer_metadata/README.md", - "csrc/attention/quant_lightning_indexer_metadata/op_api/aclnn_quant_lightning_indexer_metadata.cpp", - "csrc/attention/quant_lightning_indexer_metadata/op_api/aclnn_quant_lightning_indexer_metadata.h", - "csrc/attention/quant_lightning_indexer_metadata/op_api/l0_quant_lightning_indexer_metadata.cpp", - "csrc/attention/quant_lightning_indexer_metadata/op_api/l0_quant_lightning_indexer_metadata.h", - "csrc/attention/quant_lightning_indexer_metadata/op_graph/quant_lightning_indexer_metadata_proto.h", - "csrc/attention/quant_lightning_indexer_metadata/op_host/quant_lightning_indexer_metadata_infershape.cpp", - "csrc/attention/quant_lightning_indexer_metadata/op_kernel_aicpu/quant_lightning_indexer_metadata_aicpu.cpp", - "csrc/attention/quant_lightning_indexer_metadata/op_kernel_aicpu/quant_lightning_indexer_metadata_aicpu.h", - "csrc/attention/quant_lightning_indexer_metadata/op_kernel_aicpu/quant_lightning_indexer_metadata_aicpu.json", - "csrc/attention/rms_norm_dynamic_quant/CMakeLists.txt", - "csrc/attention/rms_norm_dynamic_quant/op_host/CMakeLists.txt", - "csrc/attention/rms_norm_dynamic_quant/op_host/rms_norm_dynamic_quant_def.cpp", - "csrc/attention/rms_norm_dynamic_quant/op_host/rms_norm_dynamic_quant_proto.cpp", - "csrc/attention/rms_norm_dynamic_quant/op_host/rms_norm_dynamic_quant_tiling.cpp", - "csrc/attention/rms_norm_dynamic_quant/op_host/rms_norm_dynamic_quant_tiling.h", - "csrc/attention/rms_norm_dynamic_quant/op_kernel/reduce_common.h", - "csrc/attention/rms_norm_dynamic_quant/op_kernel/rms_norm_dynamic_quant.cpp", - "csrc/attention/rms_norm_dynamic_quant/op_kernel/rms_norm_dynamic_quant_base.h", - "csrc/attention/rms_norm_dynamic_quant/op_kernel/rms_norm_dynamic_quant_cut_d_kernel.h", - "csrc/attention/rms_norm_dynamic_quant/op_kernel/rms_norm_dynamic_quant_helper.h", - "csrc/attention/rms_norm_dynamic_quant/op_kernel/rms_norm_dynamic_quant_normal_kernel.h", - "csrc/attention/rms_norm_dynamic_quant/op_kernel/rms_norm_dynamic_quant_single_row_kernel.h", - "csrc/attention/sparse_attn_sharedkv/CMakeLists.txt", - "csrc/attention/sparse_attn_sharedkv/README.md", - "csrc/attention/sparse_attn_sharedkv/op_host/CMakeLists.txt", - "csrc/attention/sparse_attn_sharedkv/op_host/sparse_attn_sharedkv_def.cpp", - "csrc/attention/sparse_attn_sharedkv/op_host/sparse_attn_sharedkv_proto.cpp", - "csrc/attention/sparse_attn_sharedkv/op_host/sparse_attn_sharedkv_tiling.cpp", - "csrc/attention/sparse_attn_sharedkv/op_host/sparse_attn_sharedkv_tiling.h", - "csrc/attention/sparse_attn_sharedkv/op_kernel/arch32/sparse_attn_sharedkv_scfa_block_cube.h", - "csrc/attention/sparse_attn_sharedkv/op_kernel/arch32/sparse_attn_sharedkv_scfa_block_vector.h", - "csrc/attention/sparse_attn_sharedkv/op_kernel/arch32/sparse_attn_sharedkv_scfa_kernel.h", - "csrc/attention/sparse_attn_sharedkv/op_kernel/arch32/sparse_attn_sharedkv_swa_block_cube.h", - "csrc/attention/sparse_attn_sharedkv/op_kernel/arch32/sparse_attn_sharedkv_swa_block_vector.h", - "csrc/attention/sparse_attn_sharedkv/op_kernel/arch32/sparse_attn_sharedkv_swa_kernel.h", - "csrc/attention/sparse_attn_sharedkv/op_kernel/sparse_attn_sharedkv.cpp", - "csrc/attention/sparse_attn_sharedkv/op_kernel/sparse_attn_sharedkv_common.h", - "csrc/attention/sparse_attn_sharedkv/op_kernel/sparse_attn_sharedkv_metadata.h", - "csrc/attention/sparse_attn_sharedkv/op_kernel/sparse_attn_sharedkv_template_tiling_key.h", - "csrc/attention/sparse_attn_sharedkv_metadata/CMakeLists.txt", - "csrc/attention/sparse_attn_sharedkv_metadata/README.md", - "csrc/attention/sparse_attn_sharedkv_metadata/op_api/aclnn_sparse_attn_sharedkv_metadata.cpp", - "csrc/attention/sparse_attn_sharedkv_metadata/op_api/aclnn_sparse_attn_sharedkv_metadata.h", - "csrc/attention/sparse_attn_sharedkv_metadata/op_api/l0_sparse_attn_sharedkv_metadata.cpp", - "csrc/attention/sparse_attn_sharedkv_metadata/op_api/l0_sparse_attn_sharedkv_metadata.h", - "csrc/attention/sparse_attn_sharedkv_metadata/op_graph/sparse_attn_sharedkv_metadata_proto.h", - "csrc/attention/sparse_attn_sharedkv_metadata/op_host/sparse_attn_sharedkv_metadata_infershape.cpp", - "csrc/attention/sparse_attn_sharedkv_metadata/op_kernel_aicpu/sparse_attn_sharedkv_metadata_aicpu.cpp", - "csrc/attention/sparse_attn_sharedkv_metadata/op_kernel_aicpu/sparse_attn_sharedkv_metadata_aicpu.h", - "csrc/attention/sparse_attn_sharedkv_metadata/op_kernel_aicpu/sparse_attn_sharedkv_metadata_aicpu.json", - "csrc/build_aclnn.sh", - "csrc/cmake/custom_build.cmake", - "csrc/cmake/modules/Findaicpu.cmake", - "csrc/cmake/opbuild.cmake", - "csrc/gmm/grouped_matmul_swiglu_quant/CMakeLists.txt", - "csrc/gmm/grouped_matmul_swiglu_quant/README.md", - "csrc/gmm/grouped_matmul_swiglu_quant/op_host/CMakeLists.txt", - "csrc/gmm/grouped_matmul_swiglu_quant/op_host/grouped_matmul_swiglu_quant_def.cpp", - "csrc/gmm/grouped_matmul_swiglu_quant/op_host/grouped_matmul_swiglu_quant_infershape.cpp", - "csrc/gmm/grouped_matmul_swiglu_quant/op_host/grouped_matmul_swiglu_quant_tiling.cpp", - "csrc/gmm/grouped_matmul_swiglu_quant/op_host/grouped_matmul_swiglu_quant_tiling.h", - "csrc/gmm/grouped_matmul_swiglu_quant/op_host/op_api/aclnn_grouped_matmul_swiglu_quant.cpp", - "csrc/gmm/grouped_matmul_swiglu_quant/op_host/op_api/aclnn_grouped_matmul_swiglu_quant.h", - "csrc/gmm/grouped_matmul_swiglu_quant/op_host/op_api/aclnn_grouped_matmul_swiglu_quant_weight_nz.h", - "csrc/gmm/grouped_matmul_swiglu_quant/op_host/op_api/grouped_matmul_swiglu_quant.cpp", - "csrc/gmm/grouped_matmul_swiglu_quant/op_host/op_api/grouped_matmul_swiglu_quant.h", - "csrc/gmm/grouped_matmul_swiglu_quant/op_kernel/grouped_matmul_swiglu_pipeline.h", - "csrc/gmm/grouped_matmul_swiglu_quant/op_kernel/grouped_matmul_swiglu_quant.cpp", - "csrc/gmm/grouped_matmul_swiglu_quant/op_kernel/grouped_matmul_swiglu_quant.h", - "csrc/gmm/grouped_matmul_swiglu_quant/op_kernel/grouped_matmul_swiglu_quant_a8w4_msd_mid.h", - "csrc/gmm/grouped_matmul_swiglu_quant/op_kernel/grouped_matmul_swiglu_quant_a8w4_msd_post.h", - "csrc/gmm/grouped_matmul_swiglu_quant/op_kernel/grouped_matmul_swiglu_quant_a8w4_msd_pre.h", - "csrc/gmm/grouped_matmul_swiglu_quant/op_kernel/grouped_matmul_swiglu_quant_split_ws.h", - "csrc/gmm/grouped_matmul_swiglu_quant/op_kernel/grouped_matmul_swiglu_quant_utils.h", - "csrc/gmm/grouped_matmul_swiglu_quant_v2/CMakeLists.txt", - "csrc/gmm/grouped_matmul_swiglu_quant_v2/grouped_matmul_swiglu_quant_v2_torch_adpt.h", - "csrc/gmm/grouped_matmul_swiglu_quant_v2/op_host/CMakeLists.txt", - "csrc/gmm/grouped_matmul_swiglu_quant_v2/op_host/grouped_matmul_swiglu_quant_v2_base_tiling.cpp", - "csrc/gmm/grouped_matmul_swiglu_quant_v2/op_host/grouped_matmul_swiglu_quant_v2_base_tiling.h", - "csrc/gmm/grouped_matmul_swiglu_quant_v2/op_host/grouped_matmul_swiglu_quant_v2_def.cpp", - "csrc/gmm/grouped_matmul_swiglu_quant_v2/op_host/grouped_matmul_swiglu_quant_v2_fusion_tiling.cpp", - "csrc/gmm/grouped_matmul_swiglu_quant_v2/op_host/grouped_matmul_swiglu_quant_v2_fusion_tiling.h", - "csrc/gmm/grouped_matmul_swiglu_quant_v2/op_host/grouped_matmul_swiglu_quant_v2_host_utils.h", - "csrc/gmm/grouped_matmul_swiglu_quant_v2/op_host/grouped_matmul_swiglu_quant_v2_infershape.cpp", - "csrc/gmm/grouped_matmul_swiglu_quant_v2/op_host/grouped_matmul_swiglu_quant_v2_tiling.cpp", - "csrc/gmm/grouped_matmul_swiglu_quant_v2/op_host/grouped_matmul_swiglu_quant_v2_tiling.h", - "csrc/gmm/grouped_matmul_swiglu_quant_v2/op_host/op_api/aclnn_grouped_matmul_swiglu_quant_v2.cpp", - "csrc/gmm/grouped_matmul_swiglu_quant_v2/op_host/op_api/aclnn_grouped_matmul_swiglu_quant_v2.h", - "csrc/gmm/grouped_matmul_swiglu_quant_v2/op_host/op_api/aclnn_grouped_matmul_swiglu_quant_weight_nz_v2.h", - "csrc/gmm/grouped_matmul_swiglu_quant_v2/op_host/op_api/gmm_dsq_base.h", - "csrc/gmm/grouped_matmul_swiglu_quant_v2/op_host/op_api/grouped_matmul_swiglu_quant_utils.h", - "csrc/gmm/grouped_matmul_swiglu_quant_v2/op_host/op_api/grouped_matmul_swiglu_quant_v2.cpp", - "csrc/gmm/grouped_matmul_swiglu_quant_v2/op_host/op_api/grouped_matmul_swiglu_quant_v2.h", - "csrc/gmm/grouped_matmul_swiglu_quant_v2/op_host/op_api/grouped_matmul_swiglu_quant_v2_utils.h", - "csrc/gmm/grouped_matmul_swiglu_quant_v2/op_kernel/arch35/grouped_matmul_swiglu_quant_v2_mxquant.h", - "csrc/gmm/grouped_matmul_swiglu_quant_v2/op_kernel/arch35/grouped_matmul_swiglu_quant_v2_pertoken_quant.h", - "csrc/gmm/grouped_matmul_swiglu_quant_v2/op_kernel/arch35/grouped_matmul_swiglu_quant_v2_tiling_key.h", - "csrc/gmm/grouped_matmul_swiglu_quant_v2/op_kernel/grouped_matmul_swiglu_quant_spilit_fusion.h", - "csrc/gmm/grouped_matmul_swiglu_quant_v2/op_kernel/grouped_matmul_swiglu_quant_v2.cpp", - "csrc/gmm/grouped_matmul_swiglu_quant_v2/op_kernel/grouped_matmul_swiglu_quant_v2_a4w4_mid.h", - "csrc/gmm/grouped_matmul_swiglu_quant_v2/op_kernel/grouped_matmul_swiglu_quant_v2_a4w4_pipeline.h", - "csrc/gmm/grouped_matmul_swiglu_quant_v2/op_kernel/grouped_matmul_swiglu_quant_v2_a4w4_post.h", - "csrc/gmm/grouped_matmul_swiglu_quant_v2/op_kernel/grouped_matmul_swiglu_quant_v2_a8w4_msd_mid.h", - "csrc/gmm/grouped_matmul_swiglu_quant_v2/op_kernel/grouped_matmul_swiglu_quant_v2_a8w4_msd_pipeline.h", - "csrc/gmm/grouped_matmul_swiglu_quant_v2/op_kernel/grouped_matmul_swiglu_quant_v2_a8w4_msd_post.h", - "csrc/gmm/grouped_matmul_swiglu_quant_v2/op_kernel/grouped_matmul_swiglu_quant_v2_a8w4_msd_pre.h", - "csrc/gmm/grouped_matmul_swiglu_quant_v2/op_kernel/grouped_matmul_swiglu_quant_v2_apt.cpp", - "csrc/gmm/grouped_matmul_swiglu_quant_v2/op_kernel/grouped_matmul_swiglu_quant_v2_utils.h", - "csrc/gmm/grouped_matmul_swiglu_quant_weight_nz_tensor_list/grouped_matmul_swiglu_quant_torch_adpt.h", - "csrc/gmm/grouped_matmul_swiglu_quant_weight_nz_tensor_list/op_host/grouped_matmul_swiglu_quant_weight_nz_tensor_list.h", - "csrc/gmm/grouped_matmul_swiglu_quant_weight_nz_tensor_list/op_host/grouped_matmul_swiglu_quant_weight_nz_tensor_list_def.cpp", - "csrc/gmm/grouped_matmul_swiglu_quant_weight_nz_tensor_list/op_host/grouped_matmul_swiglu_quant_weight_nz_tensor_list_tiling.cpp", - "csrc/gmm/grouped_matmul_swiglu_quant_weight_nz_tensor_list/op_host/grouped_matmul_swiglu_quant_weight_nz_tensor_list_tiling.h", - "csrc/gmm/grouped_matmul_swiglu_quant_weight_nz_tensor_list/op_host/op_api/aclnn_grouped_matmul_swiglu_quant_weight_nz_tensor_list.cpp", - "csrc/gmm/grouped_matmul_swiglu_quant_weight_nz_tensor_list/op_host/op_api/aclnn_grouped_matmul_swiglu_quant_weight_nz_tensor_list.h", - "csrc/gmm/grouped_matmul_swiglu_quant_weight_nz_tensor_list/op_host/op_api/grouped_matmul_swiglu_quant_weight_nz_tensor_list.cpp", - "csrc/gmm/grouped_matmul_swiglu_quant_weight_nz_tensor_list/op_kernel/grouped_matmul_swiglu_quant_weight_nz_tensor_list.cpp", - "csrc/gmm/grouped_matmul_swiglu_quant_weight_nz_tensor_list/op_kernel/grouped_matmul_swiglu_quant_weight_nz_tensor_list.h", - "csrc/gmm/grouped_matmul_swiglu_quant_weight_nz_tensor_list/op_kernel/grouped_matmul_swiglu_quant_weight_nz_tensor_list_split_ws.h", - "csrc/moe/dequant_swiglu_quant/CMakeLists.txt", - "csrc/moe/dequant_swiglu_quant/op_host/CMakeLists.txt", - "csrc/moe/dequant_swiglu_quant/op_host/dequant_swiglu_quant_def.cpp", - "csrc/moe/dequant_swiglu_quant/op_host/dequant_swiglu_quant_infershape.cpp", - "csrc/moe/dequant_swiglu_quant/op_host/dequant_swiglu_quant_proto.h", - "csrc/moe/dequant_swiglu_quant/op_host/dequant_swiglu_quant_tiling.cpp", - "csrc/moe/dequant_swiglu_quant/op_host/dequant_swiglu_quant_tiling.h", - "csrc/moe/dequant_swiglu_quant/op_host/dequant_swiglu_quant_tiling_arch35.cpp", - "csrc/moe/dequant_swiglu_quant/op_host/dequant_swiglu_quant_tiling_base.cpp", - "csrc/moe/dequant_swiglu_quant/op_host/swi_glu_grad_regbase_tiling.h", - "csrc/moe/dequant_swiglu_quant/op_host/swi_glu_grad_tiling_regbase.h", - "csrc/moe/dequant_swiglu_quant/op_host/swi_glu_tiling.h", - "csrc/moe/dequant_swiglu_quant/op_kernel/dequant_swiglu_quant.cpp", - "csrc/moe/dequant_swiglu_quant/op_kernel/dequant_swiglu_quant.h", - "csrc/moe/dequant_swiglu_quant/op_kernel/dequant_swiglu_quant_apt.cpp", - "csrc/moe/dequant_swiglu_quant/op_kernel/dequant_swiglu_quant_cut_group.h", - "csrc/moe/dequant_swiglu_quant/op_kernel/dequant_swiglu_quant_dynamic_base.hpp", - "csrc/moe/dequant_swiglu_quant/op_kernel/dequant_swiglu_quant_dynamic_bf16.hpp", - "csrc/moe/dequant_swiglu_quant/op_kernel/dequant_swiglu_quant_dynamic_bias_float.hpp", - "csrc/moe/dequant_swiglu_quant/op_kernel/dequant_swiglu_quant_dynamic_bias_int32.hpp", - "csrc/moe/dequant_swiglu_quant/op_kernel/dequant_swiglu_quant_dynamic_performance.hpp", - "csrc/moe/dequant_swiglu_quant/op_kernel/dequant_swiglu_quant_static_base.hpp", - "csrc/moe/dequant_swiglu_quant/op_kernel/dequant_swiglu_quant_static_bf16.hpp", - "csrc/moe/dequant_swiglu_quant/op_kernel/dequant_swiglu_quant_static_bias_float.hpp", - "csrc/moe/dequant_swiglu_quant/op_kernel/dequant_swiglu_quant_static_bias_int32.hpp", - "csrc/moe/dequant_swiglu_quant/tiling_base/error_log.h", - "csrc/moe/dequant_swiglu_quant/tiling_base/static_register_symbol.h", - "csrc/moe/dequant_swiglu_quant/tiling_base/tiling_base.h", - "csrc/moe/dequant_swiglu_quant/tiling_base/tiling_key.h", - "csrc/moe/dequant_swiglu_quant/tiling_base/tiling_templates_registry.h", - "csrc/moe/dequant_swiglu_quant/tiling_base/tiling_util.h", - "csrc/moe/hc_post/CMakeLists.txt", - "csrc/moe/hc_post/op_host/CMakeLists.txt", - "csrc/moe/hc_post/op_host/hc_post_def.cpp", - "csrc/moe/hc_post/op_host/hc_post_proto.cpp", - "csrc/moe/hc_post/op_host/hc_post_tiling.cpp", - "csrc/moe/hc_post/op_host/hc_post_tiling.h", - "csrc/moe/hc_post/op_host/hc_post_tiling_arch35.h", - "csrc/moe/hc_post/op_kernel/hc_post.cpp", - "csrc/moe/hc_post/op_kernel/hc_post_bfloat16.h", - "csrc/moe/hc_post/op_kernel/hc_post_d_split.h", - "csrc/moe/hc_post/op_kernel/hc_post_float32.h", - "csrc/moe/hc_pre_inv_rms/CMakeLists.txt", - "csrc/moe/hc_pre_inv_rms/op_host/CMakeLists.txt", - "csrc/moe/hc_pre_inv_rms/op_host/hc_pre_inv_rms_def.cpp", - "csrc/moe/hc_pre_inv_rms/op_host/hc_pre_inv_rms_proto.cpp", - "csrc/moe/hc_pre_inv_rms/op_host/hc_pre_inv_rms_tiling.cpp", - "csrc/moe/hc_pre_inv_rms/op_host/hc_pre_inv_rms_tiling.h", - "csrc/moe/hc_pre_inv_rms/op_host/hc_pre_inv_rms_tiling_arch35.h", - "csrc/moe/hc_pre_inv_rms/op_host/hc_pre_inv_rms_tiling_large_d.h", - "csrc/moe/hc_pre_inv_rms/op_kernel/hc_pre_inv_rms.cpp", - "csrc/moe/hc_pre_inv_rms/op_kernel/hc_pre_inv_rms_full_load.h", - "csrc/moe/hc_pre_inv_rms/op_kernel/hc_pre_inv_rms_full_load_large_d.h", - "csrc/moe/hc_pre_inv_rms/op_kernel/hc_pre_inv_rms_full_load_regbase.h", - "csrc/moe/hc_pre_sinkhorn/CMakeLists.txt", - "csrc/moe/hc_pre_sinkhorn/op_host/CMakeLists.txt", - "csrc/moe/hc_pre_sinkhorn/op_host/hc_pre_sinkhorn_def.cpp", - "csrc/moe/hc_pre_sinkhorn/op_host/hc_pre_sinkhorn_proto.cpp", - "csrc/moe/hc_pre_sinkhorn/op_host/hc_pre_sinkhorn_tiling.cpp", - "csrc/moe/hc_pre_sinkhorn/op_host/hc_pre_sinkhorn_tiling.h", - "csrc/moe/hc_pre_sinkhorn/op_kernel/hc_pre_sinkhorn.cpp", - "csrc/moe/hc_pre_sinkhorn/op_kernel/hc_pre_sinkhorn_base.h", - "csrc/moe/hc_pre_sinkhorn/op_kernel/hc_pre_sinkhorn_perf.h", - "csrc/moe/hc_pre_sinkhorn/op_kernel/hc_pre_sinkhorn_regbase_base.h", - "csrc/moe/hc_pre_sinkhorn/op_kernel/hc_pre_sinkhorn_regbase_perf.h", - "csrc/moe/moe_gating_top_k_hash/CMakeLists.txt", - "csrc/moe/moe_gating_top_k_hash/op_host/CMakeLists.txt", - "csrc/moe/moe_gating_top_k_hash/op_host/moe_gating_top_k_hash_def.cpp", - "csrc/moe/moe_gating_top_k_hash/op_host/moe_gating_top_k_hash_proto.cpp", - "csrc/moe/moe_gating_top_k_hash/op_host/moe_gating_top_k_hash_tiling.cpp", - "csrc/moe/moe_gating_top_k_hash/op_host/moe_gating_top_k_hash_tiling.h", - "csrc/moe/moe_gating_top_k_hash/op_host/moe_gating_top_k_hash_tiling_arch35.h", - "csrc/moe/moe_gating_top_k_hash/op_kernel/arch35/common.h", - "csrc/moe/moe_gating_top_k_hash/op_kernel/arch35/moe_gating_top_k_hash_regbase.h", - "csrc/moe/moe_gating_top_k_hash/op_kernel/common.h", - "csrc/moe/moe_gating_top_k_hash/op_kernel/common_regbase.h", - "csrc/moe/moe_gating_top_k_hash/op_kernel/moe_gating_top_k_hash.cpp", - "csrc/moe/moe_gating_top_k_hash/op_kernel/moe_gating_top_k_hash_e_k_fullload.h", - "csrc/moe/moe_gating_top_k_hash/op_kernel/moe_gating_top_k_hash_generalized.h", - "csrc/moe/moe_gating_top_k_hash/op_kernel/moe_gating_top_k_hash_regbase.h", - "csrc/moe/moe_gating_top_k_hash/op_kernel/moe_gating_top_k_hash_without_group.h", - "csrc/moe/scatter_nd_update_v2/CMakeLists.txt", - "csrc/moe/scatter_nd_update_v2/op_host/CMakeLists.txt", - "csrc/moe/scatter_nd_update_v2/op_host/config/ascend910_93/scatter_nd_update_v2_binary.json", - "csrc/moe/scatter_nd_update_v2/op_host/config/ascend910_93/scatter_nd_update_v2_simplified_key.ini", - "csrc/moe/scatter_nd_update_v2/op_host/config/ascend910b/scatter_nd_update_v2_binary.json", - "csrc/moe/scatter_nd_update_v2/op_host/config/ascend910b/scatter_nd_update_v2_simplified_key.ini", - "csrc/moe/scatter_nd_update_v2/op_host/op_api/aclnn_scatter_nd_update_v2.cpp", - "csrc/moe/scatter_nd_update_v2/op_host/op_api/aclnn_scatter_nd_update_v2.h", - "csrc/moe/scatter_nd_update_v2/op_host/op_api/scatter_nd_update_v2.cpp", - "csrc/moe/scatter_nd_update_v2/op_host/op_api/scatter_nd_update_v2.h", - "csrc/moe/scatter_nd_update_v2/op_host/scatter_nd_update_infershape.cpp", - "csrc/moe/scatter_nd_update_v2/op_host/scatter_nd_update_v2_def.cpp", - "csrc/moe/scatter_nd_update_v2/op_host/scatter_nd_update_v2_tiling.cpp", - "csrc/moe/scatter_nd_update_v2/op_host/scatter_nd_update_v2_tiling.h", - "csrc/moe/scatter_nd_update_v2/op_kernel/scatter_nd_update_common.h", - "csrc/moe/scatter_nd_update_v2/op_kernel/scatter_nd_update_large_index.h", - "csrc/moe/scatter_nd_update_v2/op_kernel/scatter_nd_update_linear_index.h", - "csrc/moe/scatter_nd_update_v2/op_kernel/scatter_nd_update_no_sort.h", - "csrc/moe/scatter_nd_update_v2/op_kernel/scatter_nd_update_v2.cpp", - "csrc/moe/scatter_nd_update_v2/op_kernel/scatter_nd_update_v2.h", - "csrc/torch_binding.cpp", - "csrc/torch_binding_meta.cpp", - "vllm_ascend/device/device_op.py", - "vllm_ascend/ops/fused_moe/moe_mlp.py" - ] - }, - { - "commit": "86f2822dec22b6384c0af06729352b30024b1fab", - "subject": "[CI] Move part of nightly test to weekly (#9210)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/_e2e_nightly_multi_node.yaml", - ".github/workflows/_e2e_nightly_single_node.yaml", - ".github/workflows/schedule_nightly_test_a3.yaml", - ".github/workflows/schedule_weekly_test_a3.yaml", - "tests/e2e/nightly/multi_node/scripts/lws.yaml.jinja2", - "tests/e2e/nightly/multi_node/scripts/utils.py", - "tests/e2e/nightly/single_node/models/scripts/single_node_config.py", - "tests/e2e/weekly/multi_node/configs/GLM5_1-W8A8-A3-dual-nodes.yaml", - "tests/e2e/weekly/single_node/configs/DeepSeek-V3.2-W8A8.yaml", - "tests/e2e/weekly/single_node/configs/Kimi-K2.5.yaml", - "tests/e2e/weekly/single_node/configs/MiniMax-M2.5-w8a8-QuaRot-A3.yaml", - "tests/e2e/weekly/single_node/configs/Qwen3.5-122B-A10B-W8A8-A3.yaml", - "tests/e2e/weekly/single_node/configs/Qwen3.5-27B-w8a8-A3.yaml", - "tests/e2e/weekly/single_node/configs/Qwen3.5-397B-A17B-W8A8-mtp-A3.yaml" - ] - }, - { - "commit": "0158892a6c475690f733ac2acb6056cc0ca7e07c", - "subject": "[Test] Spplement UT for ops - fused moe (#7768)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/ops/test_fused_moe.py", - "tests/ut/ops/test_select_experts.py" - ] - }, - { - "commit": "70f41127cc89caf46574e0663303520e92be34ac", - "subject": "[CI]Main2main 0515 (#9176)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/workflows/dockerfiles/Dockerfile.lint", - ".github/workflows/pr_test_full.yaml", - ".github/workflows/pr_test_light.yaml", - "docs/source/conf.py", - "tests/e2e/multicard/2-cards/test_qwen3_moe_routing_replay.py", - "tests/ut/core/test_profiling_chunk.py", - "vllm_ascend/ops/fused_moe/fused_moe.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "215028c0bd9411224227ac87d420402b5f4d463b", - "subject": "[Ascend950][Feature][quant] Add allgatherEP quantization support for Ascend950 (#8425)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/ascend_forward_context.py", - "vllm_ascend/ops/fused_moe/moe_comm_method.py", - "vllm_ascend/ops/fused_moe/prepare_finalize.py", - "vllm_ascend/ops/fused_moe/token_dispatcher.py", - "vllm_ascend/quantization/methods/w8a8_mxfp8.py" - ] - }, - { - "commit": "4cb6f8ecf354360cc21e4c934791d7d58edab0b4", - "subject": "[Ops][BugFix] : NPU MoE layers support TP only. (#8532)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/ops/fused_moe/fused_moe.py", - "vllm_ascend/ops/fused_moe/moe_comm_method.py" - ] - }, - { - "commit": "ac357ed4bbc91c6ad16ba028aa0a76066940d0f4", - "subject": "[BugFix] Fix sliding window attention computation and improve throughput (#8508)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/attention/test_attention_v1.py", - "vllm_ascend/attention/attention_v1.py" - ] - }, - { - "commit": "26437c1fe961c612b8e162b882afadd6c45f99fa", - "subject": "[CI] Main2main 0514 (#9155)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/workflows/_e2e_test.yaml", - ".github/workflows/dockerfiles/Dockerfile.lint", - ".github/workflows/pr_test_full.yaml", - ".github/workflows/pr_test_light.yaml", - ".github/workflows/schedule_lint_image_build.yaml", - ".github/workflows/schedule_update_estimated_time.yaml", - ".github/workflows/schedule_vllm_e2e_test.yaml", - "Dockerfile", - "Dockerfile.310p", - "Dockerfile.310p.openEuler", - "Dockerfile.a3", - "Dockerfile.a3.openEuler", - "Dockerfile.openEuler", - "docs/source/conf.py", - "tests/e2e/multicard/2-cards/test_qwen3_moe.py", - "tests/e2e/multicard/2-cards/test_qwen3_moe_routing_replay.py", - "tests/e2e/singlecard/model_runner_v2/test_basic.py", - "tests/ut/spec_decode/test_eagle_proposer.py", - "vllm_ascend/core/scheduler_profiling_chunk.py", - "vllm_ascend/eplb/core/eplb_utils.py", - "vllm_ascend/ops/fused_moe/fused_moe.py", - "vllm_ascend/ops/gdn.py", - "vllm_ascend/patch/platform/patch_balance_schedule.py", - "vllm_ascend/patch/platform/patch_mla_prefill_backend.py", - "vllm_ascend/patch/worker/__init__.py", - "vllm_ascend/patch/worker/patch_deepseek_mtp.py", - "vllm_ascend/patch/worker/patch_v2/patch_triton.py", - "vllm_ascend/worker/model_runner_v1.py", - "vllm_ascend/worker/v2/attn_utils.py", - "vllm_ascend/worker/v2/block_table.py", - "vllm_ascend/worker/v2/model_runner.py", - "vllm_ascend/worker/v2/model_states/default.py", - "vllm_ascend/worker/v2/sample/logprob.py", - "vllm_ascend/worker/v2/spec_decode/rejection_sampler_utils.py", - "vllm_ascend/worker/worker.py" - ] - }, - { - "commit": "3f73174317c0a59431913fdd6519af49ac17f856", - "subject": "[Misc] Upgrade torch-npu to 2.10.0 (#9128)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/_optional_smart_e2e.yaml", - "CMakeLists.txt", - "README.md", - "README.zh.md", - "docs/source/conf.py", - "docs/source/installation.md", - "pyproject.toml", - "requirements-dev.txt", - "requirements.txt", - "tests/e2e/singlecard/compile/test_graphex_qknorm_rope_fusion.py", - "vllm_ascend/lora/punica_npu.py", - "vllm_ascend/quantization/methods/w4a4_laos_dynamic.py" - ] - }, - { - "commit": "1185f7c6695bbb144ce76b8ea29afc9dfbe45dd4", - "subject": "[Feature] Support LoRA with Qwen3.5 dense model. (#9023)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/workflows/scripts/config.yaml", - "tests/e2e/conftest.py", - "tests/e2e/singlecard/test_qwen35_densemodel_lora.py", - "vllm_ascend/lora/utils.py", - "vllm_ascend/ops/gdn.py" - ] - }, - { - "commit": "34235b8881ad1b47d4b7bc0fc0e1afd71b170ba7", - "subject": "[Feature] Update custom op build framework (#8146)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".gitignore", - "CMakeLists.txt", - "csrc/CMakeLists.txt", - "csrc/add_rms_norm_bias/op_host/CMakeLists.txt", - "csrc/apply_top_k_top_p_custom/op_host/CMakeLists.txt", - "csrc/attention/CMakeLists.txt", - "csrc/attention/lightning_indexer_quant/CMakeLists.txt", - "csrc/attention/lightning_indexer_quant/lightning_indexer_quant_torch_adpt.h", - "csrc/attention/lightning_indexer_quant/op_host/CMakeLists.txt", - "csrc/attention/lightning_indexer_quant/op_host/lightning_indexer_quant_def.cpp", - "csrc/attention/lightning_indexer_quant/op_host/lightning_indexer_quant_proto.cpp", - "csrc/attention/lightning_indexer_quant/op_host/lightning_indexer_quant_tiling.cpp", - "csrc/attention/lightning_indexer_quant/op_host/lightning_indexer_quant_tiling.h", - "csrc/attention/lightning_indexer_quant/op_kernel/lightning_indexer_quant.cpp", - "csrc/attention/lightning_indexer_quant/op_kernel/lightning_indexer_quant_common.h", - "csrc/attention/lightning_indexer_quant/op_kernel/lightning_indexer_quant_kernel.h", - "csrc/attention/lightning_indexer_quant/op_kernel/lightning_indexer_quant_service_cube.h", - "csrc/attention/lightning_indexer_quant/op_kernel/lightning_indexer_quant_service_vector.h", - "csrc/attention/lightning_indexer_quant/op_kernel/lightning_indexer_quant_template_tiling_key.h", - "csrc/attention/lightning_indexer_quant/op_kernel/lightning_indexer_quant_vector.h", - "csrc/attention/lightning_indexer_vllm/CMakeLists.txt", - "csrc/attention/lightning_indexer_vllm/lightning_indexer_vllm_torch_adpt.h", - "csrc/attention/lightning_indexer_vllm/op_host/CMakeLists.txt", - "csrc/attention/lightning_indexer_vllm/op_host/lightning_indexer_vllm_def.cpp", - "csrc/attention/lightning_indexer_vllm/op_host/lightning_indexer_vllm_proto.cpp", - "csrc/attention/lightning_indexer_vllm/op_host/lightning_indexer_vllm_tiling.cpp", - "csrc/attention/lightning_indexer_vllm/op_host/lightning_indexer_vllm_tiling.h", - "csrc/attention/lightning_indexer_vllm/op_kernel/lightning_indexer_common.h", - "csrc/attention/lightning_indexer_vllm/op_kernel/lightning_indexer_kernel.h", - "csrc/attention/lightning_indexer_vllm/op_kernel/lightning_indexer_service_cube.h", - "csrc/attention/lightning_indexer_vllm/op_kernel/lightning_indexer_service_vector.h", - "csrc/attention/lightning_indexer_vllm/op_kernel/lightning_indexer_template_tiling_key.h", - "csrc/attention/lightning_indexer_vllm/op_kernel/lightning_indexer_vector.h", - "csrc/attention/lightning_indexer_vllm/op_kernel/lightning_indexer_vllm.cpp", - "csrc/attention/recurrent_gated_delta_rule/CMakeLists.txt", - "csrc/attention/recurrent_gated_delta_rule/op_host/CMakeLists.txt", - "csrc/attention/recurrent_gated_delta_rule/op_host/error_log.h", - "csrc/attention/recurrent_gated_delta_rule/op_host/math_util.h", - "csrc/attention/recurrent_gated_delta_rule/op_host/op_api/aclnn_recurrent_gated_delta_rule.cpp", - "csrc/attention/recurrent_gated_delta_rule/op_host/op_api/aclnn_recurrent_gated_delta_rule.h", - "csrc/attention/recurrent_gated_delta_rule/op_host/op_api/recurrent_gated_delta_rule.cpp", - "csrc/attention/recurrent_gated_delta_rule/op_host/recurrent_gated_delta_rule.h", - "csrc/attention/recurrent_gated_delta_rule/op_host/recurrent_gated_delta_rule_def.cpp", - "csrc/attention/recurrent_gated_delta_rule/op_host/recurrent_gated_delta_rule_infershape.cpp", - "csrc/attention/recurrent_gated_delta_rule/op_host/recurrent_gated_delta_rule_tiling.cpp", - "csrc/attention/recurrent_gated_delta_rule/op_host/recurrent_gated_delta_rule_tiling.h", - "csrc/attention/recurrent_gated_delta_rule/op_host/tiling_base.h", - "csrc/attention/recurrent_gated_delta_rule/op_host/tiling_templates_registry.h", - "csrc/attention/recurrent_gated_delta_rule/op_kernel/recurrent_gated_delta_rule.cpp", - "csrc/attention/recurrent_gated_delta_rule/op_kernel/recurrent_gated_delta_rule.h", - "csrc/attention/recurrent_gated_delta_rule/op_kernel/recurrent_gated_delta_rule_tiling_data.h", - "csrc/attention/recurrent_gated_delta_rule_v310/CMakeLists.txt", - "csrc/attention/recurrent_gated_delta_rule_v310/op_host/CMakeLists.txt", - "csrc/attention/recurrent_gated_delta_rule_v310/op_host/math_util.h", - "csrc/attention/recurrent_gated_delta_rule_v310/op_host/op_api/aclnn_recurrent_gated_delta_rule_v310.cpp", - "csrc/attention/recurrent_gated_delta_rule_v310/op_host/op_api/aclnn_recurrent_gated_delta_rule_v310.h", - "csrc/attention/recurrent_gated_delta_rule_v310/op_host/op_api/recurrent_gated_delta_rule_v310.cpp", - "csrc/attention/recurrent_gated_delta_rule_v310/op_host/recurrent_gated_delta_rule_v310.h", - "csrc/attention/recurrent_gated_delta_rule_v310/op_host/recurrent_gated_delta_rule_v310_def.cpp", - "csrc/attention/recurrent_gated_delta_rule_v310/op_host/recurrent_gated_delta_rule_v310_infershape.cpp", - "csrc/attention/recurrent_gated_delta_rule_v310/op_host/recurrent_gated_delta_rule_v310_tiling.cpp", - "csrc/attention/recurrent_gated_delta_rule_v310/op_host/recurrent_gated_delta_rule_v310_tiling.h", - "csrc/attention/recurrent_gated_delta_rule_v310/op_kernel/recurrent_gated_delta_rule_v310.cpp", - "csrc/attention/recurrent_gated_delta_rule_v310/op_kernel/recurrent_gated_delta_rule_v310.h", - "csrc/attention/recurrent_gated_delta_rule_v310/op_kernel/recurrent_gated_delta_rule_v310_tiling_data.h", - "csrc/attention/recurrent_gated_delta_rule_v310/recurrent_gated_delta_rule_310_torch_adpt.h", - "csrc/attention/recurrent_gated_delta_rule_v310/tiling_base/error_log.h", - "csrc/attention/recurrent_gated_delta_rule_v310/tiling_base/tiling_base.h", - "csrc/attention/recurrent_gated_delta_rule_v310/tiling_base/tiling_templates_registry.h", - "csrc/attention/recurrent_gated_delta_rule_v310/tiling_base/tiling_util.h", - "csrc/attention/reshape_and_cache_bnsd/CMakeLists.txt", - "csrc/attention/reshape_and_cache_bnsd/op_host/CMakeLists.txt", - "csrc/attention/reshape_and_cache_bnsd/op_host/reshape_and_cache_bnsd.cpp", - "csrc/attention/reshape_and_cache_bnsd/op_host/reshape_and_cache_bnsd_proto.cpp", - "csrc/attention/reshape_and_cache_bnsd/op_host/reshape_and_cache_bnsd_tiling.cpp", - "csrc/attention/reshape_and_cache_bnsd/op_host/reshape_and_cache_bnsd_tiling.h", - "csrc/attention/reshape_and_cache_bnsd/op_kernel/kernel_utils.h", - "csrc/attention/reshape_and_cache_bnsd/op_kernel/reshape_and_cache_bnsd.cpp", - "csrc/attention/sparse_flash_attention/CMakeLists.txt", - "csrc/attention/sparse_flash_attention/op_host/CMakeLists.txt", - "csrc/attention/sparse_flash_attention/op_host/sparse_flash_attention_def.cpp", - "csrc/attention/sparse_flash_attention/op_host/sparse_flash_attention_proto.cpp", - "csrc/attention/sparse_flash_attention/op_host/sparse_flash_attention_tiling.cpp", - "csrc/attention/sparse_flash_attention/op_host/sparse_flash_attention_tiling.h", - "csrc/attention/sparse_flash_attention/op_kernel/sparse_flash_attention.cpp", - "csrc/attention/sparse_flash_attention/op_kernel/sparse_flash_attention_common.h", - "csrc/attention/sparse_flash_attention/op_kernel/sparse_flash_attention_kernel_mla.h", - "csrc/attention/sparse_flash_attention/op_kernel/sparse_flash_attention_service_cube_mla.h", - "csrc/attention/sparse_flash_attention/op_kernel/sparse_flash_attention_service_vector_mla.h", - "csrc/attention/sparse_flash_attention/op_kernel/sparse_flash_attention_template_tiling_key.h", - "csrc/attention/sparse_flash_attention/sparse_flash_attention_torch_adpt.h", - "csrc/build.sh", - "csrc/build_aclnn.sh", - "csrc/causal_conv1d/op_host/CMakeLists.txt", - "csrc/causal_conv1d_v310/op_host/CMakeLists.txt", - "csrc/cmake/Third_Party_Open_Source_Software_List.yaml", - "csrc/cmake/aclnn_ops_transformer.h.in", - "csrc/cmake/build_empty_package.cmake", - "csrc/cmake/config.cmake", - "csrc/cmake/config_utest.cmake", - "csrc/cmake/custom_build.cmake", - "csrc/cmake/dependencies.cmake", - "csrc/cmake/func.cmake", - "csrc/cmake/func_examples.cmake", - "csrc/cmake/func_utest.cmake", - "csrc/cmake/gen_ops_info.cmake", - "csrc/cmake/intf.cmake", - "csrc/cmake/intf_pub.cmake", - "csrc/cmake/intf_pub_examples.cmake", - "csrc/cmake/intf_pub_linux.cmake", - "csrc/cmake/intf_pub_llt_gccnative.cmake", - "csrc/cmake/intf_pub_utest.cmake", - "csrc/cmake/makeself_built_in.cmake", - "csrc/cmake/makeself_custom.cmake", - "csrc/cmake/modules/FindOPBASE.cmake", - "csrc/cmake/modules/FindPython.cmake", - "csrc/cmake/modules/Findaicpu.cmake", - "csrc/cmake/modules/Findalog.cmake", - "csrc/cmake/modules/Finddlog.cmake", - "csrc/cmake/modules/Findjson.cmake", - "csrc/cmake/modules/Findmetadef.cmake", - "csrc/cmake/modules/Findnnopbase.cmake", - "csrc/cmake/modules/Findoptiling.cmake", - "csrc/cmake/modules/Findplatform.cmake", - "csrc/cmake/modules/Findruntime.cmake", - "csrc/cmake/modules/Findsecurec.cmake", - "csrc/cmake/modules/Findtilingapi.cmake", - "csrc/cmake/modules/Findunified_dlog.cmake", - "csrc/cmake/obj_func.cmake", - "csrc/cmake/opbuild.cmake", - "csrc/cmake/package.cmake", - "csrc/cmake/rty_obj_func.cmake", - "csrc/cmake/runtimeKB.cmake", - "csrc/cmake/scripts/check_version_compatible.py", - "csrc/cmake/scripts/convert_yaml.py", - "csrc/cmake/scripts/custom/help.info", - "csrc/cmake/scripts/custom/install.sh", - "csrc/cmake/scripts/custom/upgrade.sh", - "csrc/cmake/scripts/examples/get_opapi_abs_path.py", - "csrc/cmake/scripts/examples/get_soc_info.py", - "csrc/cmake/scripts/fix_format.sh", - "csrc/cmake/scripts/parse_changed_files.py", - "csrc/cmake/scripts/prepare.sh", - "csrc/cmake/scripts/utest/gen_coverage.py", - "csrc/cmake/scripts/utest/gen_tiling_data_stub.py", - "csrc/cmake/scripts/util/ascendc_bin_param_build.py", - "csrc/cmake/scripts/util/ascendc_gen_options.py", - "csrc/cmake/scripts/util/ascendc_impl_build.py", - "csrc/cmake/scripts/util/ascendc_ops_config.py", - "csrc/cmake/scripts/util/const_var.py", - "csrc/cmake/scripts/util/gen_version_info.sh", - "csrc/cmake/scripts/util/opdesc_parser.py", - "csrc/cmake/scripts/util/parse_ini_to_json.py", - "csrc/cmake/static.cmake", - "csrc/cmake/symbol.cmake", - "csrc/cmake/third_party/.gitkeep", - "csrc/cmake/third_party/abseil-cpp.cmake", - "csrc/cmake/third_party/ascend_protobuf.cmake", - "csrc/cmake/third_party/build/modules/patch/protobuf-hide_absl_symbols.patch", - "csrc/cmake/third_party/build/modules/patch/protobuf_25.1_change_version.patch", - "csrc/cmake/third_party/gtest.cmake", - "csrc/cmake/third_party/json.cmake", - "csrc/cmake/third_party/makeself-fetch.cmake", - "csrc/cmake/third_party/protobuf.cmake", - "csrc/cmake/third_party/secure_c.cmake", - "csrc/cmake/ut.cmake", - "csrc/cmake/variables.cmake", - "csrc/common/CMakeLists.txt", - "csrc/common/aicpu/cpu_context_util.h", - "csrc/common/include/common/op_api_def.h", - "csrc/common/include/common/tensor_util.cpp", - "csrc/common/include/common/tensor_util.h", - "csrc/common/include/err/ops_err.h", - "csrc/common/include/external/aclnn_kernels/cast.h", - "csrc/common/include/external/aclnn_kernels/common/op_error_check.h", - "csrc/common/include/external/aclnn_kernels/contiguous.h", - "csrc/common/include/external/aclnn_kernels/pad.h", - "csrc/common/include/external/aclnn_kernels/reshape.h", - "csrc/common/include/external/aclnn_kernels/slice.h", - "csrc/common/include/external/aclnn_kernels/transdata.h", - "csrc/common/include/external/aclnn_kernels/transpose.h", - "csrc/common/include/external/aclnn_util.h", - "csrc/common/include/fallback/fallback.h", - "csrc/common/include/fallback/fallback_2stages.h", - "csrc/common/include/fallback/fallback_comm.h", - "csrc/common/include/fallback/fallback_comm_2stages.h", - "csrc/common/include/framework/onnx_common.h", - "csrc/common/include/kernel/common.h", - "csrc/common/include/kernel/common_func.h", - "csrc/common/include/kernel/dropmask.h", - "csrc/common/include/kernel/gm_to_l1_iterator.h", - "csrc/common/include/kernel/gm_to_ub_iterator.h", - "csrc/common/include/kernel/hardware.h", - "csrc/common/include/kernel/iterator.h", - "csrc/common/include/kernel/l0c_to_gm_iterator.h", - "csrc/common/include/kernel/l0c_to_l1_iterator.h", - "csrc/common/include/kernel/l0c_to_ub_iterator.h", - "csrc/common/include/kernel/l1_to_bt_iterator.h", - "csrc/common/include/kernel/l1_to_fb_iterator.h", - "csrc/common/include/kernel/l1_to_l0_iterator.h", - "csrc/common/include/kernel/l1_to_ub_iterator.h", - "csrc/common/include/kernel/layout.h", - "csrc/common/include/kernel/mem.h", - "csrc/common/include/kernel/mma.h", - "csrc/common/include/kernel/pse.h", - "csrc/common/include/kernel/simd.h", - "csrc/common/include/kernel/util.h", - "csrc/common/include/op_graph/op_transformer_proto_extend.h", - "csrc/common/include/static/op_resource.h", - "csrc/common/include/static/static_space.h", - "csrc/common/include/tiling_base/data_copy_transpose_tiling.h", - "csrc/common/include/tiling_base/data_copy_transpose_tiling_def.h", - "csrc/common/include/tiling_base/tiling_base.h", - "csrc/common/include/tiling_base/tiling_key.h", - "csrc/common/include/tiling_base/tiling_templates_registry.h", - "csrc/common/include/tiling_base/tiling_type.h", - "csrc/common/include/tiling_base/tiling_util.h", - "csrc/common/include/tiling_sink/device_op_impl_registry_impl.h", - "csrc/common/include/tiling_sink/tiling_aicpu_task.h", - "csrc/common/src/fallback_comm.cpp", - "csrc/common/src/fallback_comm_2stages.cpp", - "csrc/common/src/framework/CMakeLists.txt", - "csrc/common/src/framework/embedding_bag_onnx_plugin.cpp", - "csrc/common/src/framework/fillwindowcache_onnx_plugin.cpp", - "csrc/common/src/framework/multi_head_attention_onnx_plugin.cpp", - "csrc/common/src/framework/npu_fused_attention_score_fwd_onnx_plugin.cpp", - "csrc/common/src/framework/npu_fused_attention_score_onnx_plugin.cpp", - "csrc/common/src/framework/npu_masked_softmax_with_relposbias_onnx_plugin.cpp", - "csrc/common/src/framework/npu_scaled_masked_softmax_onnx_plugin.cpp", - "csrc/common/src/framework/tfidf_vectorizer_onnx_plugin.cpp", - "csrc/common/src/tiling_base/tiling_util.cpp", - "csrc/common/src/tiling_sink/CMakeLists.txt", - "csrc/common/src/tiling_sink/tiling_sink_registry.cpp", - "csrc/common/stub/CMakeLists.txt", - "csrc/common/stub/inc/framework/stub_ops.h", - "csrc/common/stub/op_api/CMakeLists.txt", - "csrc/common/stub/op_api/aclnn_kernels/cast.h", - "csrc/common/stub/op_api/aclnn_kernels/common/op_error_check.h", - "csrc/common/stub/op_api/aclnn_kernels/contiguous.h", - "csrc/common/stub/op_api/aclnn_kernels/pad.h", - "csrc/common/stub/op_api/aclnn_kernels/reshape.h", - "csrc/common/stub/op_api/aclnn_kernels/slice.h", - "csrc/common/stub/op_api/aclnn_kernels/transdata.h", - "csrc/common/stub/op_api/aclnn_kernels/transpose.h", - "csrc/common/stub/op_api/level0/add.h", - "csrc/common/stub/op_api/level0/arange.h", - "csrc/common/stub/op_api/level0/axpy.h", - "csrc/common/stub/op_api/level0/batch_norm_backward.h", - "csrc/common/stub/op_api/level0/broadcast_to.h", - "csrc/common/stub/op_api/level0/dilation.h", - "csrc/common/stub/op_api/level0/div.h", - "csrc/common/stub/op_api/level0/dot.h", - "csrc/common/stub/op_api/level0/expand.h", - "csrc/common/stub/op_api/level0/fault_injection.h", - "csrc/common/stub/op_api/level0/fill.h", - "csrc/common/stub/op_api/level0/gather_elements.h", - "csrc/common/stub/op_api/level0/gather_v2.h", - "csrc/common/stub/op_api/level0/gather_v3.h", - "csrc/common/stub/op_api/level0/inplace_index_add.h", - "csrc/common/stub/op_api/level0/masked_scatter.h", - "csrc/common/stub/op_api/level0/matmul_v2tov3.h", - "csrc/common/stub/op_api/level0/maximum.h", - "csrc/common/stub/op_api/level0/minimum.h", - "csrc/common/stub/op_api/level0/mul.h", - "csrc/common/stub/op_api/level0/muls.h", - "csrc/common/stub/op_api/level0/ones_like.h", - "csrc/common/stub/op_api/level0/padv3.h", - "csrc/common/stub/op_api/level0/reduce_mean.h", - "csrc/common/stub/op_api/level0/reduce_sum_op.h", - "csrc/common/stub/op_api/level0/shape_op.h", - "csrc/common/stub/op_api/level0/sort.h", - "csrc/common/stub/op_api/level0/squeeze.h", - "csrc/common/stub/op_api/level0/sub.h", - "csrc/common/stub/op_api/level0/tensor_move.h", - "csrc/common/stub/op_api/level0/unsqueeze.h", - "csrc/common/stub/op_api/level0/zero_op.h", - "csrc/common/stub/op_api/opapi_stub.cpp", - "csrc/common/stub/op_tiling/CMakeLists.txt", - "csrc/common/stub/op_tiling/op_cache_def_tiling.h", - "csrc/common/stub/op_tiling/op_cache_tiling.cpp", - "csrc/common/stub/op_tiling/op_cache_tiling.h", - "csrc/common/stub/op_tiling/register/tuning_bank_key_registry.h", - "csrc/common/stub/op_tiling/register/tuning_tiling_reflection_utils.h", - "csrc/common/stub/op_tiling/register/tuning_tiling_registry.h", - "csrc/common/stub/op_tiling/runtime_kb_api.cpp", - "csrc/common/stub/op_tiling/runtime_kb_api.h", - "csrc/common/stub/op_tiling/tbe_tiling_api.cpp", - "csrc/common/stub/op_tiling/tbe_tiling_api.h", - "csrc/copy_and_expand_eagle_inputs/op_host/CMakeLists.txt", - "csrc/dispatch_ffn_combine/op_host/CMakeLists.txt", - "csrc/dispatch_ffn_combine_bf16/op_host/CMakeLists.txt", - "csrc/dispatch_ffn_combine_w4_a8/op_host/CMakeLists.txt", - "csrc/dispatch_gmm_combine_decode/op_host/CMakeLists.txt", - "csrc/dispatch_layout/op_host/CMakeLists.txt", - "csrc/gmm/CMakeLists.txt", - "csrc/gmm/grouped_matmul_swiglu_quant_weight_nz_tensor_list/CMakeLists.txt", - "csrc/gmm/grouped_matmul_swiglu_quant_weight_nz_tensor_list/grouped_matmul_swiglu_quant_torch_adpt.h", - "csrc/gmm/grouped_matmul_swiglu_quant_weight_nz_tensor_list/op_host/CMakeLists.txt", - "csrc/gmm/grouped_matmul_swiglu_quant_weight_nz_tensor_list/op_host/grouped_matmul_swiglu_quant_weight_nz_tensor_list.h", - "csrc/gmm/grouped_matmul_swiglu_quant_weight_nz_tensor_list/op_host/grouped_matmul_swiglu_quant_weight_nz_tensor_list_def.cpp", - "csrc/gmm/grouped_matmul_swiglu_quant_weight_nz_tensor_list/op_host/grouped_matmul_swiglu_quant_weight_nz_tensor_list_infershape.cpp", - "csrc/gmm/grouped_matmul_swiglu_quant_weight_nz_tensor_list/op_host/grouped_matmul_swiglu_quant_weight_nz_tensor_list_proto.cpp", - "csrc/gmm/grouped_matmul_swiglu_quant_weight_nz_tensor_list/op_host/grouped_matmul_swiglu_quant_weight_nz_tensor_list_tiling.cpp", - "csrc/gmm/grouped_matmul_swiglu_quant_weight_nz_tensor_list/op_host/grouped_matmul_swiglu_quant_weight_nz_tensor_list_tiling.h", - "csrc/gmm/grouped_matmul_swiglu_quant_weight_nz_tensor_list/op_host/op_api/aclnn_grouped_matmul_swiglu_quant_weight_nz_tensor_list.cpp", - "csrc/gmm/grouped_matmul_swiglu_quant_weight_nz_tensor_list/op_host/op_api/aclnn_grouped_matmul_swiglu_quant_weight_nz_tensor_list.h", - "csrc/gmm/grouped_matmul_swiglu_quant_weight_nz_tensor_list/op_host/op_api/grouped_matmul_swiglu_quant_weight_nz_tensor_list.cpp", - "csrc/gmm/grouped_matmul_swiglu_quant_weight_nz_tensor_list/op_kernel/grouped_matmul_swiglu_quant_weight_nz_tensor_list.cpp", - "csrc/gmm/grouped_matmul_swiglu_quant_weight_nz_tensor_list/op_kernel/grouped_matmul_swiglu_quant_weight_nz_tensor_list.h", - "csrc/gmm/grouped_matmul_swiglu_quant_weight_nz_tensor_list/op_kernel/grouped_matmul_swiglu_quant_weight_nz_tensor_list_split_ws.h", - "csrc/gmm/grouped_matmul_swiglu_quant_weight_nz_tensor_list/op_kernel/grouped_matmul_swiglu_quant_weight_nz_tensor_list_utils.h", - "csrc/grouped_matmul_swiglu_quant_weight_nz_tensor_list/op_host/CMakeLists.txt", - "csrc/hamming_dist_top_k/op_host/CMakeLists.txt", - "csrc/lightning_indexer_quant/op_host/CMakeLists.txt", - "csrc/lightning_indexer_vllm/op_host/CMakeLists.txt", - "csrc/matmul_allreduce_add_rmsnorm/op_host/CMakeLists.txt", - "csrc/mc2/CMakeLists.txt", - "csrc/mc2/dispatch_ffn_combine/CMakeLists.txt", - "csrc/mc2/dispatch_ffn_combine/dispatch_ffn_combine_torch_adpt.h", - "csrc/mc2/dispatch_ffn_combine/op_host/CMakeLists.txt", - "csrc/mc2/dispatch_ffn_combine/op_host/dispatch_ffn_combine_def.cpp", - "csrc/mc2/dispatch_ffn_combine/op_host/dispatch_ffn_combine_proto.cpp", - "csrc/mc2/dispatch_ffn_combine/op_host/dispatch_ffn_combine_tiling.cpp", - "csrc/mc2/dispatch_ffn_combine/op_host/error_log.h", - "csrc/mc2/dispatch_ffn_combine/op_host/hcom_topo_info.h", - "csrc/mc2/dispatch_ffn_combine/op_host/op_api/aclnn_dispatch_ffn_combine.cpp", - "csrc/mc2/dispatch_ffn_combine/op_host/op_api/aclnn_dispatch_ffn_combine.h", - "csrc/mc2/dispatch_ffn_combine/op_host/tiling_args.h", - "csrc/mc2/dispatch_ffn_combine/op_kernel/dispatch_ffn_combine.cpp", - "csrc/mc2/dispatch_ffn_combine/op_kernel/dispatch_ffn_combine.h", - "csrc/mc2/dispatch_ffn_combine/op_kernel/dispatch_ffn_combine_kernel.hpp", - "csrc/mc2/dispatch_ffn_combine/op_kernel/dispatch_ffn_combine_tiling.h", - "csrc/mc2/dispatch_ffn_combine/op_kernel/moe_init_routing_quant_v2/moe_init_routing_quant_v2.cpp", - "csrc/mc2/dispatch_ffn_combine/op_kernel/moe_init_routing_quant_v2/moe_init_routing_quant_v2_tiling.h", - "csrc/mc2/dispatch_ffn_combine/op_kernel/moe_init_routing_quant_v2/moe_init_routing_v2_tiling.h", - "csrc/mc2/dispatch_ffn_combine/op_kernel/moe_init_routing_quant_v2/moe_v2_common.h", - "csrc/mc2/dispatch_ffn_combine/op_kernel/moe_init_routing_quant_v2/moe_v2_expert_token_out.h", - "csrc/mc2/dispatch_ffn_combine/op_kernel/moe_init_routing_quant_v2/moe_v2_fullload_dynamic_quant.h", - "csrc/mc2/dispatch_ffn_combine/op_kernel/moe_init_routing_quant_v2/moe_v2_fullload_quant.h", - "csrc/mc2/dispatch_ffn_combine/op_kernel/moe_init_routing_quant_v2/moe_v2_fullload_quant_base.h", - "csrc/mc2/dispatch_ffn_combine/op_kernel/moe_init_routing_quant_v2/moe_v2_gather_dynamic_quant.h", - "csrc/mc2/dispatch_ffn_combine/op_kernel/moe_init_routing_quant_v2/moe_v2_gather_out.h", - "csrc/mc2/dispatch_ffn_combine/op_kernel/moe_init_routing_quant_v2/moe_v2_gather_quant.h", - "csrc/mc2/dispatch_ffn_combine/op_kernel/moe_init_routing_quant_v2/moe_v2_init_routing_fullload.h", - "csrc/mc2/dispatch_ffn_combine/op_kernel/moe_init_routing_quant_v2/moe_v2_mrgsort.h", - "csrc/mc2/dispatch_ffn_combine/op_kernel/moe_init_routing_quant_v2/moe_v2_mrgsort_out.h", - "csrc/mc2/dispatch_ffn_combine/op_kernel/moe_init_routing_quant_v2/moe_v2_sort_base.h", - "csrc/mc2/dispatch_ffn_combine/op_kernel/moe_init_routing_quant_v2/moe_v2_sort_multi_core.h", - "csrc/mc2/dispatch_ffn_combine/op_kernel/moe_init_routing_quant_v2/moe_v2_sort_one_core.h", - "csrc/mc2/dispatch_ffn_combine/op_kernel/moe_init_routing_quant_v2/moe_v2_src_to_dst_and_gather.h", - "csrc/mc2/dispatch_ffn_combine/op_kernel/moe_init_routing_quant_v2/moe_v2_src_to_dst_op.h", - "csrc/mc2/dispatch_ffn_combine/op_kernel/moe_init_routing_quant_v2/moe_v2_src_to_dst_with_capacity.h", - "csrc/mc2/dispatch_ffn_combine/op_kernel/moe_init_routing_quant_v2/tiling_base.h", - "csrc/mc2/dispatch_ffn_combine/op_kernel/unpermute/moe_token_unpermute.h", - "csrc/mc2/dispatch_ffn_combine/op_kernel/unpermute/moe_token_unpermute_tiling.h", - "csrc/mc2/dispatch_ffn_combine/op_kernel/utils/block_epilogue_pertoken_row.hpp", - "csrc/mc2/dispatch_ffn_combine/op_kernel/utils/block_epilogue_pertoken_swiglu.hpp", - "csrc/mc2/dispatch_ffn_combine/op_kernel/utils/block_epilogue_pertoken_v2.hpp", - "csrc/mc2/dispatch_ffn_combine/op_kernel/utils/block_mmad_preload_async_fixpipe_quant.hpp", - "csrc/mc2/dispatch_ffn_combine/op_kernel/utils/const_args.hpp", - "csrc/mc2/dispatch_ffn_combine/op_kernel/utils/copy_gm_to_l1_custom.hpp", - "csrc/mc2/dispatch_ffn_combine/op_kernel/utils/copy_l0c_to_gm_custom.hpp", - "csrc/mc2/dispatch_ffn_combine/op_kernel/utils/dispatch_policy_custom.hpp", - "csrc/mc2/dispatch_ffn_combine/op_kernel/utils/get_tensor_addr.hpp", - "csrc/mc2/dispatch_ffn_combine/op_kernel/utils/hccl_shmem.hpp", - "csrc/mc2/dispatch_ffn_combine/op_kernel/utils/layout3d.hpp", - "csrc/mc2/dispatch_ffn_combine/op_kernel/utils/moe_distribute_base.h", - "csrc/mc2/dispatch_ffn_combine/op_kernel/utils/select_helper.hpp", - "csrc/mc2/dispatch_ffn_combine_bf16/CMakeLists.txt", - "csrc/mc2/dispatch_ffn_combine_bf16/op_host/CMakeLists.txt", - "csrc/mc2/dispatch_ffn_combine_bf16/op_host/dispatch_ffn_combine_bf16_def.cpp", - "csrc/mc2/dispatch_ffn_combine_bf16/op_host/dispatch_ffn_combine_bf16_proto.cpp", - "csrc/mc2/dispatch_ffn_combine_bf16/op_host/dispatch_ffn_combine_bf16_tiling.cpp", - "csrc/mc2/dispatch_ffn_combine_bf16/op_host/error_log.h", - "csrc/mc2/dispatch_ffn_combine_bf16/op_host/hcom_topo_info.h", - "csrc/mc2/dispatch_ffn_combine_bf16/op_host/op_api/aclnn_dispatch_ffn_combine_bf16.cpp", - "csrc/mc2/dispatch_ffn_combine_bf16/op_host/op_api/aclnn_dispatch_ffn_combine_bf16.h", - "csrc/mc2/dispatch_ffn_combine_bf16/op_host/tiling_args.h", - "csrc/mc2/dispatch_ffn_combine_bf16/op_kernel/dispatch_ffn_combine_bf16.cpp", - "csrc/mc2/dispatch_ffn_combine_bf16/op_kernel/dispatch_ffn_combine_bf16.h", - "csrc/mc2/dispatch_ffn_combine_bf16/op_kernel/dispatch_ffn_combine_bf16_kernel.hpp", - "csrc/mc2/dispatch_ffn_combine_bf16/op_kernel/dispatch_ffn_combine_bf16_tiling.h", - "csrc/mc2/dispatch_ffn_combine_bf16/op_kernel/moe_init_routing_v2/moe_init_routing_v2.cpp", - "csrc/mc2/dispatch_ffn_combine_bf16/op_kernel/moe_init_routing_v2/moe_init_routing_v2_tiling.h", - "csrc/mc2/dispatch_ffn_combine_bf16/op_kernel/moe_init_routing_v2/moe_v2_common.h", - "csrc/mc2/dispatch_ffn_combine_bf16/op_kernel/moe_init_routing_v2/moe_v2_expert_token_out.h", - "csrc/mc2/dispatch_ffn_combine_bf16/op_kernel/moe_init_routing_v2/moe_v2_gather_out.h", - "csrc/mc2/dispatch_ffn_combine_bf16/op_kernel/moe_init_routing_v2/moe_v2_init_routing_fullload.h", - "csrc/mc2/dispatch_ffn_combine_bf16/op_kernel/moe_init_routing_v2/moe_v2_mrgsort.h", - "csrc/mc2/dispatch_ffn_combine_bf16/op_kernel/moe_init_routing_v2/moe_v2_mrgsort_out.h", - "csrc/mc2/dispatch_ffn_combine_bf16/op_kernel/moe_init_routing_v2/moe_v2_sort_base.h", - "csrc/mc2/dispatch_ffn_combine_bf16/op_kernel/moe_init_routing_v2/moe_v2_sort_multi_core.h", - "csrc/mc2/dispatch_ffn_combine_bf16/op_kernel/moe_init_routing_v2/moe_v2_sort_one_core.h", - "csrc/mc2/dispatch_ffn_combine_bf16/op_kernel/moe_init_routing_v2/moe_v2_src_to_dst_op.h", - "csrc/mc2/dispatch_ffn_combine_bf16/op_kernel/moe_init_routing_v2/moe_v2_src_to_dst_op_simt.h", - "csrc/mc2/dispatch_ffn_combine_bf16/op_kernel/moe_init_routing_v2/moe_v2_src_to_dst_with_capacity.h", - "csrc/mc2/dispatch_ffn_combine_bf16/op_kernel/moe_init_routing_v2/tiling_base.h", - "csrc/mc2/dispatch_ffn_combine_bf16/op_kernel/unpermute/moe_token_unpermute.h", - "csrc/mc2/dispatch_ffn_combine_bf16/op_kernel/unpermute/moe_token_unpermute_tiling.h", - "csrc/mc2/dispatch_ffn_combine_bf16/op_kernel/utils/block_epilogue_pertoken_row.hpp", - "csrc/mc2/dispatch_ffn_combine_bf16/op_kernel/utils/block_epilogue_pertoken_swiglu.hpp", - "csrc/mc2/dispatch_ffn_combine_bf16/op_kernel/utils/block_epilogue_pertoken_v2.hpp", - "csrc/mc2/dispatch_ffn_combine_bf16/op_kernel/utils/block_mmad_preload_async_fixpipe_quant.hpp", - "csrc/mc2/dispatch_ffn_combine_bf16/op_kernel/utils/const_args.hpp", - "csrc/mc2/dispatch_ffn_combine_bf16/op_kernel/utils/copy_gm_to_l1_custom.hpp", - "csrc/mc2/dispatch_ffn_combine_bf16/op_kernel/utils/copy_l0c_to_gm_custom.hpp", - "csrc/mc2/dispatch_ffn_combine_bf16/op_kernel/utils/dispatch_policy_custom.hpp", - "csrc/mc2/dispatch_ffn_combine_bf16/op_kernel/utils/get_tensor_addr.hpp", - "csrc/mc2/dispatch_ffn_combine_bf16/op_kernel/utils/hccl_shmem.hpp", - "csrc/mc2/dispatch_ffn_combine_bf16/op_kernel/utils/layout3d.hpp", - "csrc/mc2/dispatch_ffn_combine_bf16/op_kernel/utils/moe_distribute_base.h", - "csrc/mc2/dispatch_ffn_combine_bf16/op_kernel/utils/select_helper.hpp", - "csrc/mc2/dispatch_ffn_combine_w4_a8/CMakeLists.txt", - "csrc/mc2/dispatch_ffn_combine_w4_a8/op_host/CMakeLists.txt", - "csrc/mc2/dispatch_ffn_combine_w4_a8/op_host/dispatch_ffn_combine_w4_a8_def.cpp", - "csrc/mc2/dispatch_ffn_combine_w4_a8/op_host/dispatch_ffn_combine_w4_a8_proto.cpp", - "csrc/mc2/dispatch_ffn_combine_w4_a8/op_host/dispatch_ffn_combine_w4_a8_tiling.cpp", - "csrc/mc2/dispatch_ffn_combine_w4_a8/op_host/error_log.h", - "csrc/mc2/dispatch_ffn_combine_w4_a8/op_host/hcom_topo_info.h", - "csrc/mc2/dispatch_ffn_combine_w4_a8/op_host/op_api/aclnn_dispatch_ffn_combine_w4_a8.cpp", - "csrc/mc2/dispatch_ffn_combine_w4_a8/op_host/op_api/aclnn_dispatch_ffn_combine_w4_a8.h", - "csrc/mc2/dispatch_ffn_combine_w4_a8/op_host/tiling_args.h", - "csrc/mc2/dispatch_ffn_combine_w4_a8/op_kernel/dispatch_ffn_combine_w4_a8.cpp", - "csrc/mc2/dispatch_ffn_combine_w4_a8/op_kernel/dispatch_ffn_combine_w4_a8.h", - "csrc/mc2/dispatch_ffn_combine_w4_a8/op_kernel/dispatch_ffn_combine_w4_a8_kernel.hpp", - "csrc/mc2/dispatch_ffn_combine_w4_a8/op_kernel/dispatch_ffn_combine_w4_a8_tiling.h", - "csrc/mc2/dispatch_ffn_combine_w4_a8/op_kernel/moe_init_routing_quant_v2/moe_init_routing_quant_v2.cpp", - "csrc/mc2/dispatch_ffn_combine_w4_a8/op_kernel/moe_init_routing_quant_v2/moe_init_routing_quant_v2_tiling.h", - "csrc/mc2/dispatch_ffn_combine_w4_a8/op_kernel/moe_init_routing_quant_v2/moe_init_routing_v2_tiling.h", - "csrc/mc2/dispatch_ffn_combine_w4_a8/op_kernel/moe_init_routing_quant_v2/moe_v2_common.h", - "csrc/mc2/dispatch_ffn_combine_w4_a8/op_kernel/moe_init_routing_quant_v2/moe_v2_expert_token_out.h", - "csrc/mc2/dispatch_ffn_combine_w4_a8/op_kernel/moe_init_routing_quant_v2/moe_v2_fullload_dynamic_quant.h", - "csrc/mc2/dispatch_ffn_combine_w4_a8/op_kernel/moe_init_routing_quant_v2/moe_v2_fullload_quant.h", - "csrc/mc2/dispatch_ffn_combine_w4_a8/op_kernel/moe_init_routing_quant_v2/moe_v2_fullload_quant_base.h", - "csrc/mc2/dispatch_ffn_combine_w4_a8/op_kernel/moe_init_routing_quant_v2/moe_v2_gather_dynamic_quant.h", - "csrc/mc2/dispatch_ffn_combine_w4_a8/op_kernel/moe_init_routing_quant_v2/moe_v2_gather_out.h", - "csrc/mc2/dispatch_ffn_combine_w4_a8/op_kernel/moe_init_routing_quant_v2/moe_v2_gather_quant.h", - "csrc/mc2/dispatch_ffn_combine_w4_a8/op_kernel/moe_init_routing_quant_v2/moe_v2_init_routing_fullload.h", - "csrc/mc2/dispatch_ffn_combine_w4_a8/op_kernel/moe_init_routing_quant_v2/moe_v2_mrgsort.h", - "csrc/mc2/dispatch_ffn_combine_w4_a8/op_kernel/moe_init_routing_quant_v2/moe_v2_mrgsort_out.h", - "csrc/mc2/dispatch_ffn_combine_w4_a8/op_kernel/moe_init_routing_quant_v2/moe_v2_sort_base.h", - "csrc/mc2/dispatch_ffn_combine_w4_a8/op_kernel/moe_init_routing_quant_v2/moe_v2_sort_multi_core.h", - "csrc/mc2/dispatch_ffn_combine_w4_a8/op_kernel/moe_init_routing_quant_v2/moe_v2_sort_one_core.h", - "csrc/mc2/dispatch_ffn_combine_w4_a8/op_kernel/moe_init_routing_quant_v2/moe_v2_src_to_dst_and_gather.h", - "csrc/mc2/dispatch_ffn_combine_w4_a8/op_kernel/moe_init_routing_quant_v2/moe_v2_src_to_dst_op.h", - "csrc/mc2/dispatch_ffn_combine_w4_a8/op_kernel/moe_init_routing_quant_v2/moe_v2_src_to_dst_with_capacity.h", - "csrc/mc2/dispatch_ffn_combine_w4_a8/op_kernel/moe_init_routing_quant_v2/tiling_base.h", - "csrc/mc2/dispatch_ffn_combine_w4_a8/op_kernel/unpermute/moe_token_unpermute.h", - "csrc/mc2/dispatch_ffn_combine_w4_a8/op_kernel/unpermute/moe_token_unpermute_tiling.h", - "csrc/mc2/dispatch_ffn_combine_w4_a8/op_kernel/utils/block_epilogue_w4a8post_pertoken_swiglu.hpp", - "csrc/mc2/dispatch_ffn_combine_w4_a8/op_kernel/utils/block_epilogue_w4a8post_pertoken_v2.hpp", - "csrc/mc2/dispatch_ffn_combine_w4_a8/op_kernel/utils/block_mmad_w4a4.hpp", - "csrc/mc2/dispatch_ffn_combine_w4_a8/op_kernel/utils/const_args.hpp", - "csrc/mc2/dispatch_ffn_combine_w4_a8/op_kernel/utils/copy_gm_to_l1_custom.hpp", - "csrc/mc2/dispatch_ffn_combine_w4_a8/op_kernel/utils/dispatch_policy_custom.hpp", - "csrc/mc2/dispatch_ffn_combine_w4_a8/op_kernel/utils/get_tensor_addr.hpp", - "csrc/mc2/dispatch_ffn_combine_w4_a8/op_kernel/utils/hccl_shmem.hpp", - "csrc/mc2/dispatch_ffn_combine_w4_a8/op_kernel/utils/layout3d.hpp", - "csrc/mc2/dispatch_ffn_combine_w4_a8/op_kernel/utils/moe_distribute_base.h", - "csrc/mc2/dispatch_ffn_combine_w4_a8/op_kernel/utils/select_helper.hpp", - "csrc/mc2/dispatch_gmm_combine_decode/CMakeLists.txt", - "csrc/mc2/dispatch_gmm_combine_decode/dispatch_gmm_combine_decode_torch_adpt.h", - "csrc/mc2/dispatch_gmm_combine_decode/op_host/CMakeLists.txt", - "csrc/mc2/dispatch_gmm_combine_decode/op_host/dispatch_gmm_combine_decode_def.cpp", - "csrc/mc2/dispatch_gmm_combine_decode/op_host/dispatch_gmm_combine_decode_proto.cpp", - "csrc/mc2/dispatch_gmm_combine_decode/op_host/dispatch_gmm_combine_decode_tiling.cpp", - "csrc/mc2/dispatch_gmm_combine_decode/op_host/op_api/aclnn_dispatch_gmm_combine_decode.cpp", - "csrc/mc2/dispatch_gmm_combine_decode/op_host/op_api/aclnn_dispatch_gmm_combine_decode.h", - "csrc/mc2/dispatch_gmm_combine_decode/op_kernel/dispatch_gmm_combine_decode.cpp", - "csrc/mc2/dispatch_gmm_combine_decode/op_kernel/dispatch_gmm_combine_decode.h", - "csrc/mc2/dispatch_gmm_combine_decode/op_kernel/dispatch_gmm_combine_decode/epilogue/block/block_epilogue.h", - "csrc/mc2/dispatch_gmm_combine_decode/op_kernel/dispatch_gmm_combine_decode/epilogue/block/block_epilogue_bf16_fp16.hpp", - "csrc/mc2/dispatch_gmm_combine_decode/op_kernel/dispatch_gmm_combine_decode/epilogue/block/block_epilogue_per_token_dequant.hpp", - "csrc/mc2/dispatch_gmm_combine_decode/op_kernel/dispatch_gmm_combine_decode/epilogue/block/block_epilogue_per_token_dequant_swiglu.h", - "csrc/mc2/dispatch_gmm_combine_decode/op_kernel/dispatch_gmm_combine_decode/epilogue/block/block_epilogue_swiglu_bf16_fp16.h", - "csrc/mc2/dispatch_gmm_combine_decode/op_kernel/dispatch_gmm_combine_decode/epilogue/dispatch_policy.h", - "csrc/mc2/dispatch_gmm_combine_decode/op_kernel/dispatch_gmm_combine_decode/epilogue/tile/tile_stride_binary.h", - "csrc/mc2/dispatch_gmm_combine_decode/op_kernel/dispatch_gmm_combine_decode/epilogue/tile/tile_stride_muls.h", - "csrc/mc2/dispatch_gmm_combine_decode/op_kernel/dispatch_gmm_combine_decode/gemm/block/block_mmad.h", - "csrc/mc2/dispatch_gmm_combine_decode/op_kernel/dispatch_gmm_combine_decode/gemm/block/block_mmad_preload_async_with_callback_resident_a.h", - "csrc/mc2/dispatch_gmm_combine_decode/op_kernel/dispatch_gmm_combine_decode/gemm/dispatch_policy.h", - "csrc/mc2/dispatch_gmm_combine_decode/op_kernel/dispatch_gmm_combine_decode/gemm/kernel/grouped_matmul_slice_m_multistage_workspace_bf16_fp16.h", - "csrc/mc2/dispatch_gmm_combine_decode/op_kernel/dispatch_gmm_combine_decode/gemm/kernel/grouped_matmul_slice_m_per_token_dequant_multistage_workspace.h", - "csrc/mc2/dispatch_gmm_combine_decode/op_kernel/dispatch_gmm_combine_decode/gemm/kernel/grouped_matmul_slice_m_per_token_dequant_swiglu_quant_multistage_workspace.h", - "csrc/mc2/dispatch_gmm_combine_decode/op_kernel/dispatch_gmm_combine_decode/gemm/kernel/grouped_matmul_slice_m_swiglu_multistage_workspace_bf16_fp16.h", - "csrc/mc2/dispatch_gmm_combine_decode/op_kernel/dispatch_gmm_combine_decode/raw_distributed/cam_moe_distribute_combine.h", - "csrc/mc2/dispatch_gmm_combine_decode/op_kernel/dispatch_gmm_combine_decode/raw_distributed/cam_moe_distribute_dispatch.h", - "csrc/mc2/dispatch_gmm_combine_decode/op_kernel/dispatch_gmm_combine_decode_base.h", - "csrc/mc2/dispatch_gmm_combine_decode/op_kernel/dispatch_gmm_combine_decode_bf16_fp16.h", - "csrc/mc2/dispatch_gmm_combine_decode/op_kernel/dispatch_gmm_combine_decode_tiling.h", - "csrc/mc2/dispatch_gmm_combine_decode/op_kernel/moe_distribute_base.h", - "csrc/mc2/dispatch_layout/CMakeLists.txt", - "csrc/mc2/dispatch_layout/dispatch_layout_torch_adpt.h", - "csrc/mc2/dispatch_layout/op_host/CMakeLists.txt", - "csrc/mc2/dispatch_layout/op_host/dispatch_layout.cpp", - "csrc/mc2/dispatch_layout/op_host/dispatch_layout_tiling.cpp", - "csrc/mc2/dispatch_layout/op_host/op_api/aclnn_dispatch_layout.cpp", - "csrc/mc2/dispatch_layout/op_host/op_api/aclnn_dispatch_layout.h", - "csrc/mc2/dispatch_layout/op_kernel/dispatch_layout.cpp", - "csrc/mc2/dispatch_layout/op_kernel/dispatch_layout.h", - "csrc/mc2/dispatch_layout/op_kernel/dispatch_layout_tiling.h", - "csrc/mc2/dispatch_layout/op_kernel/kernel/comm_args.h", - "csrc/mc2/dispatch_layout/op_kernel/kernel/data_copy.h", - "csrc/mc2/dispatch_layout/op_kernel/kernel/dropmask.h", - "csrc/mc2/dispatch_layout/op_kernel/kernel/moe_distribute_base.h", - "csrc/mc2/dispatch_layout/op_kernel/kernel/pse.h", - "csrc/mc2/dispatch_layout/op_kernel/kernel/sync_collectives.h", - "csrc/mc2/dispatch_layout/op_kernel/kernel/util.h", - "csrc/mc2/matmul_allreduce_add_rmsnorm/CMakeLists.txt", - "csrc/mc2/matmul_allreduce_add_rmsnorm/matmul_allreduce_add_rmsnorm_torch_adpt.h", - "csrc/mc2/matmul_allreduce_add_rmsnorm/op_host/CMakeLists.txt", - "csrc/mc2/matmul_allreduce_add_rmsnorm/op_host/matmul_allreduce_add_rmsnorm_def.cpp", - "csrc/mc2/matmul_allreduce_add_rmsnorm/op_host/matmul_allreduce_add_rmsnorm_proto.cpp", - "csrc/mc2/matmul_allreduce_add_rmsnorm/op_host/matmul_allreduce_add_rmsnorm_tiling.cpp", - "csrc/mc2/matmul_allreduce_add_rmsnorm/op_host/matmul_allreduce_add_rmsnorm_workspace.h", - "csrc/mc2/matmul_allreduce_add_rmsnorm/op_host/op_api/aclnn_matmul_allreduce_add_rmsnorm.cpp", - "csrc/mc2/matmul_allreduce_add_rmsnorm/op_host/op_api/aclnn_matmul_allreduce_add_rmsnorm.h", - "csrc/mc2/matmul_allreduce_add_rmsnorm/op_kernel/matmul_allreduce_add_rmsnorm.cpp", - "csrc/mc2/matmul_allreduce_add_rmsnorm/op_kernel/matmul_allreduce_add_rmsnorm_aic_kernel.h", - "csrc/mc2/matmul_allreduce_add_rmsnorm/op_kernel/matmul_allreduce_add_rmsnorm_aiv_kernel.h", - "csrc/mc2/matmul_allreduce_add_rmsnorm/op_kernel/matmul_allreduce_add_rmsnorm_tiling.h", - "csrc/mc2/matmul_allreduce_add_rmsnorm/op_kernel/matmul_allreduce_add_rmsnorm_utils.h", - "csrc/mc2/moe_combine_normal/CMakeLists.txt", - "csrc/mc2/moe_combine_normal/moe_combine_normal_torch_adpt.h", - "csrc/mc2/moe_combine_normal/op_host/CMakeLists.txt", - "csrc/mc2/moe_combine_normal/op_host/moe_combine_normal.cpp", - "csrc/mc2/moe_combine_normal/op_host/moe_combine_normal_tiling.cpp", - "csrc/mc2/moe_combine_normal/op_host/op_api/aclnn_moe_combine_normal.cpp", - "csrc/mc2/moe_combine_normal/op_host/op_api/aclnn_moe_combine_normal.h", - "csrc/mc2/moe_combine_normal/op_kernel/moe_combine_normal.cpp", - "csrc/mc2/moe_combine_normal/op_kernel/moe_combine_normal.h", - "csrc/mc2/moe_combine_normal/op_kernel/moe_combine_normal_tiling.h", - "csrc/mc2/moe_combine_normal/op_kernel/utils/moe_distribute_base.h", - "csrc/mc2/moe_dispatch_normal/CMakeLists.txt", - "csrc/mc2/moe_dispatch_normal/op_host/CMakeLists.txt", - "csrc/mc2/moe_dispatch_normal/op_host/moe_dispatch_normal.cpp", - "csrc/mc2/moe_dispatch_normal/op_host/moe_dispatch_normal_tiling.cpp", - "csrc/mc2/moe_dispatch_normal/op_host/op_api/aclnn_moe_dispatch_normal.cpp", - "csrc/mc2/moe_dispatch_normal/op_host/op_api/aclnn_moe_dispatch_normal.h", - "csrc/mc2/moe_dispatch_normal/op_kernel/moe_dispatch_normal.cpp", - "csrc/mc2/moe_dispatch_normal/op_kernel/moe_dispatch_normal.h", - "csrc/mc2/moe_dispatch_normal/op_kernel/moe_dispatch_normal_tiling.h", - "csrc/mc2/moe_dispatch_normal/op_kernel/utils/moe_distribute_base.h", - "csrc/mc2/notify_dispatch/CMakeLists.txt", - "csrc/mc2/notify_dispatch/op_host/CMakeLists.txt", - "csrc/mc2/notify_dispatch/op_host/notify_dispatch.cpp", - "csrc/mc2/notify_dispatch/op_host/notify_dispatch_tiling.cpp", - "csrc/mc2/notify_dispatch/op_host/op_api/aclnn_notify_dispatch.cpp", - "csrc/mc2/notify_dispatch/op_host/op_api/aclnn_notify_dispatch.h", - "csrc/mc2/notify_dispatch/op_kernel/kernel/comm_args.h", - "csrc/mc2/notify_dispatch/op_kernel/kernel/data_copy.h", - "csrc/mc2/notify_dispatch/op_kernel/kernel/dropmask.h", - "csrc/mc2/notify_dispatch/op_kernel/kernel/moe_distribute_base.h", - "csrc/mc2/notify_dispatch/op_kernel/kernel/pse.h", - "csrc/mc2/notify_dispatch/op_kernel/kernel/sync_collectives.h", - "csrc/mc2/notify_dispatch/op_kernel/kernel/util.h", - "csrc/mc2/notify_dispatch/op_kernel/notify_dispatch.cpp", - "csrc/mc2/notify_dispatch/op_kernel/notify_dispatch.h", - "csrc/mc2/notify_dispatch/op_kernel/notify_dispatch_tiling.h", - "csrc/moe/CMakeLists.txt", - "csrc/moe/add_rms_norm_bias/CMakeLists.txt", - "csrc/moe/add_rms_norm_bias/add_rms_norm_bias_torch_adpt.h", - "csrc/moe/add_rms_norm_bias/op_host/CMakeLists.txt", - "csrc/moe/add_rms_norm_bias/op_host/add_rms_norm_bias_def.cpp", - "csrc/moe/add_rms_norm_bias/op_host/add_rms_norm_bias_infershape.cpp", - "csrc/moe/add_rms_norm_bias/op_host/add_rms_norm_bias_tiling.cpp", - "csrc/moe/add_rms_norm_bias/op_host/add_rms_norm_bias_tiling.h", - "csrc/moe/add_rms_norm_bias/op_host/error_log.h", - "csrc/moe/add_rms_norm_bias/op_kernel/add_rms_norm_bias.cpp", - "csrc/moe/add_rms_norm_bias/op_kernel/add_rms_norm_bias.h", - "csrc/moe/add_rms_norm_bias/op_kernel/add_rms_norm_bias_merge_n.h", - "csrc/moe/add_rms_norm_bias/op_kernel/add_rms_norm_bias_multi_n.h", - "csrc/moe/add_rms_norm_bias/op_kernel/add_rms_norm_bias_single_n.h", - "csrc/moe/add_rms_norm_bias/op_kernel/add_rms_norm_bias_split_d.h", - "csrc/moe/add_rms_norm_bias/op_kernel/reduce_common.h", - "csrc/moe/add_rms_norm_bias/op_kernel/rms_norm_base.h", - "csrc/moe/apply_top_k_top_p_custom/CMakeLists.txt", - "csrc/moe/apply_top_k_top_p_custom/apply_top_k_top_p_custom_torch_adpt.h", - "csrc/moe/apply_top_k_top_p_custom/op_host/CMakeLists.txt", - "csrc/moe/apply_top_k_top_p_custom/op_host/apply_top_k_top_p_custom.h", - "csrc/moe/apply_top_k_top_p_custom/op_host/apply_top_k_top_p_custom_def.cpp", - "csrc/moe/apply_top_k_top_p_custom/op_host/apply_top_k_top_p_custom_tiling.cpp", - "csrc/moe/apply_top_k_top_p_custom/op_host/apply_top_k_top_p_custom_tiling.h", - "csrc/moe/apply_top_k_top_p_custom/op_host/error_log.h", - "csrc/moe/apply_top_k_top_p_custom/op_host/op_api/aclnn_apply_top_k_top_p_custom.cpp", - "csrc/moe/apply_top_k_top_p_custom/op_host/op_api/aclnn_apply_top_k_top_p_custom.h", - "csrc/moe/apply_top_k_top_p_custom/op_host/op_api/apply_top_k_top_p_custom.cpp", - "csrc/moe/apply_top_k_top_p_custom/op_host/sort.h", - "csrc/moe/apply_top_k_top_p_custom/op_kernel/apply_top_k_top_p_custom.cpp", - "csrc/moe/apply_top_k_top_p_custom/op_kernel/apply_top_k_top_p_custom.h", - "csrc/moe/apply_top_k_top_p_custom/op_kernel/apply_top_p_custom.h", - "csrc/moe/causal_conv1d/CMakeLists.txt", - "csrc/moe/causal_conv1d/op_host/CMakeLists.txt", - "csrc/moe/causal_conv1d/op_host/causal_conv1d_def.cpp", - "csrc/moe/causal_conv1d/op_host/causal_conv1d_infershape.cpp", - "csrc/moe/causal_conv1d/op_host/causal_conv1d_tiling.cpp", - "csrc/moe/causal_conv1d/op_host/causal_conv1d_tiling_planner.h", - "csrc/moe/causal_conv1d/op_host/causal_conv1d_tiling_utils.h", - "csrc/moe/causal_conv1d/op_host/causal_conv1d_tiling_validation.h", - "csrc/moe/causal_conv1d/op_host/error_log.h", - "csrc/moe/causal_conv1d/op_host/math_util.h", - "csrc/moe/causal_conv1d/op_host/tiling_util.cpp", - "csrc/moe/causal_conv1d/op_kernel/causal_conv1d.cpp", - "csrc/moe/causal_conv1d/op_kernel/causal_conv1d.h", - "csrc/moe/causal_conv1d/op_kernel/causal_conv1d_common.h", - "csrc/moe/causal_conv1d/op_kernel/causal_conv1d_fn.h", - "csrc/moe/causal_conv1d/op_kernel/causal_conv1d_fn_tasks.h", - "csrc/moe/causal_conv1d/op_kernel/causal_conv1d_tiling_data.h", - "csrc/moe/causal_conv1d/op_kernel/causal_conv1d_tiling_key.h", - "csrc/moe/causal_conv1d/op_kernel/causal_conv1d_update.h", - "csrc/moe/causal_conv1d/tiling_base/data_copy_transpose_tiling.h", - "csrc/moe/causal_conv1d/tiling_base/data_copy_transpose_tiling_def.h", - "csrc/moe/causal_conv1d/tiling_base/error_log.h", - "csrc/moe/causal_conv1d/tiling_base/tiling_base.h", - "csrc/moe/causal_conv1d/tiling_base/tiling_key.h", - "csrc/moe/causal_conv1d/tiling_base/tiling_templates_registry.h", - "csrc/moe/causal_conv1d/tiling_base/tiling_type.h", - "csrc/moe/causal_conv1d/tiling_base/tiling_util.h", - "csrc/moe/causal_conv1d_v310/CMakeLists.txt", - "csrc/moe/causal_conv1d_v310/causal_conv1d_310_torch_adpt.h", - "csrc/moe/causal_conv1d_v310/op_host/CMakeLists.txt", - "csrc/moe/causal_conv1d_v310/op_host/causal_conv1d_v310_def.cpp", - "csrc/moe/causal_conv1d_v310/op_host/causal_conv1d_v310_infershape.cpp", - "csrc/moe/causal_conv1d_v310/op_host/causal_conv1d_v310_tiling.cpp", - "csrc/moe/causal_conv1d_v310/op_host/causal_conv1d_v310_tiling.h", - "csrc/moe/causal_conv1d_v310/op_host/math_util.h", - "csrc/moe/causal_conv1d_v310/op_kernel/causal_conv1d_v310.cpp", - "csrc/moe/causal_conv1d_v310/op_kernel/causal_conv1d_v310.h", - "csrc/moe/causal_conv1d_v310/op_kernel/causal_conv1d_v310_common.h", - "csrc/moe/causal_conv1d_v310/op_kernel/causal_conv1d_v310_tiling_data.h", - "csrc/moe/causal_conv1d_v310/op_kernel/causal_conv1d_v310_tiling_key.h", - "csrc/moe/causal_conv1d_v310/tiling_base/error_log.h", - "csrc/moe/causal_conv1d_v310/tiling_base/tiling_base.h", - "csrc/moe/causal_conv1d_v310/tiling_base/tiling_templates_registry.h", - "csrc/moe/causal_conv1d_v310/tiling_base/tiling_util.h", - "csrc/moe/copy_and_expand_eagle_inputs/CMakeLists.txt", - "csrc/moe/copy_and_expand_eagle_inputs/op_host/CMakeLists.txt", - "csrc/moe/copy_and_expand_eagle_inputs/op_host/copy_and_expand_eagle_inputs_def.cpp", - "csrc/moe/copy_and_expand_eagle_inputs/op_host/copy_and_expand_eagle_inputs_infershape.cpp", - "csrc/moe/copy_and_expand_eagle_inputs/op_host/copy_and_expand_eagle_inputs_tiling.cpp", - "csrc/moe/copy_and_expand_eagle_inputs/op_host/copy_and_expand_eagle_inputs_tiling.h", - "csrc/moe/copy_and_expand_eagle_inputs/op_host/error_log.h", - "csrc/moe/copy_and_expand_eagle_inputs/op_kernel/copy_and_expand_eagle_inputs.cpp", - "csrc/moe/hamming_dist_top_k/CMakeLists.txt", - "csrc/moe/hamming_dist_top_k/op_host/CMakeLists.txt", - "csrc/moe/hamming_dist_top_k/op_host/hamming_dist_top_k.cpp", - "csrc/moe/hamming_dist_top_k/op_host/hamming_dist_top_k.h", - "csrc/moe/hamming_dist_top_k/op_host/hamming_dist_top_k_def.cpp", - "csrc/moe/hamming_dist_top_k/op_host/hamming_dist_top_k_proto.cpp", - "csrc/moe/hamming_dist_top_k/op_host/hamming_dist_top_k_split.cpp", - "csrc/moe/hamming_dist_top_k/op_host/hamming_dist_top_k_split.h", - "csrc/moe/hamming_dist_top_k/op_host/hamming_dist_top_k_tiling.cpp", - "csrc/moe/hamming_dist_top_k/op_host/hamming_dist_top_k_tiling.h", - "csrc/moe/hamming_dist_top_k/op_host/op_host_util.h", - "csrc/moe/hamming_dist_top_k/op_kernel/hamming_dist_top_k.cpp", - "csrc/moe/hamming_dist_top_k/op_kernel/hamming_dist_top_k_base.h", - "csrc/moe/hamming_dist_top_k/op_kernel/hamming_dist_top_k_parallel.h", - "csrc/moe/hamming_dist_top_k/op_kernel/hamming_dist_top_k_split_s.h", - "csrc/moe/moe_gating_top_k/CMakeLists.txt", - "csrc/moe/moe_gating_top_k/moe_gating_top_k_torch_adpt.h", - "csrc/moe/moe_gating_top_k/op_host/CMakeLists.txt", - "csrc/moe/moe_gating_top_k/op_host/error_log.h", - "csrc/moe/moe_gating_top_k/op_host/math_util.h", - "csrc/moe/moe_gating_top_k/op_host/moe_gating_top_k_def.cpp", - "csrc/moe/moe_gating_top_k/op_host/moe_gating_top_k_infershape.cpp", - "csrc/moe/moe_gating_top_k/op_host/moe_gating_top_k_proto.cpp", - "csrc/moe/moe_gating_top_k/op_host/moe_gating_top_k_proto.h", - "csrc/moe/moe_gating_top_k/op_host/moe_gating_top_k_tiling.cpp", - "csrc/moe/moe_gating_top_k/op_host/moe_gating_top_k_tiling.h", - "csrc/moe/moe_gating_top_k/op_host/moe_gating_top_k_tiling_arch35.cpp", - "csrc/moe/moe_gating_top_k/op_host/moe_gating_top_k_tiling_base.cpp", - "csrc/moe/moe_gating_top_k/op_kernel/common.h", - "csrc/moe/moe_gating_top_k/op_kernel/error_log.h", - "csrc/moe/moe_gating_top_k/op_kernel/moe_gating_top_k.cpp", - "csrc/moe/moe_gating_top_k/op_kernel/moe_gating_top_k_apt.cpp", - "csrc/moe/moe_gating_top_k/op_kernel/moe_gating_top_k_e_k_fullload.h", - "csrc/moe/moe_gating_top_k/op_kernel/moe_gating_top_k_generalized.h", - "csrc/moe/moe_gating_top_k/op_kernel/moe_gating_top_k_without_group.h", - "csrc/moe/moe_gating_top_k/tiling_base/data_copy_transpose_tiling.h", - "csrc/moe/moe_gating_top_k/tiling_base/data_copy_transpose_tiling_def.h", - "csrc/moe/moe_gating_top_k/tiling_base/error_log.h", - "csrc/moe/moe_gating_top_k/tiling_base/tiling_base.h", - "csrc/moe/moe_gating_top_k/tiling_base/tiling_key.h", - "csrc/moe/moe_gating_top_k/tiling_base/tiling_templates_registry.h", - "csrc/moe/moe_gating_top_k/tiling_base/tiling_type.h", - "csrc/moe/moe_gating_top_k/tiling_base/tiling_util.h", - "csrc/moe/moe_grouped_matmul/CMakeLists.txt", - "csrc/moe/moe_grouped_matmul/op_host/CMakeLists.txt", - "csrc/moe/moe_grouped_matmul/op_host/moe_grouped_matmul_cpu.cpp", - "csrc/moe/moe_grouped_matmul/op_host/moe_grouped_matmul_def.cpp", - "csrc/moe/moe_grouped_matmul/op_host/moe_grouped_matmul_infershape.cpp", - "csrc/moe/moe_grouped_matmul/op_host/moe_grouped_matmul_tiling.h", - "csrc/moe/moe_grouped_matmul/op_host/op_api/aclnn_moe_grouped_matmul.cpp", - "csrc/moe/moe_grouped_matmul/op_host/op_api/aclnn_moe_grouped_matmul.h", - "csrc/moe/moe_grouped_matmul/op_host/op_api/aclnn_moe_grouped_matmul_weight_nz.h", - "csrc/moe/moe_grouped_matmul/op_host/op_api/moe_grouped_matmul_l0.cpp", - "csrc/moe/moe_grouped_matmul/op_host/op_api/moe_grouped_matmul_l0.h", - "csrc/moe/moe_grouped_matmul/op_kernel/moe_grouped_matmul.cpp", - "csrc/moe/moe_grouped_matmul/op_kernel/moe_grouped_matmul.h", - "csrc/moe/moe_grouped_matmul/op_kernel/moe_grouped_matmul_tiling_data.h", - "csrc/moe/moe_init_routing_custom/CMakeLists.txt", - "csrc/moe/moe_init_routing_custom/moe_init_routing_custom_torch_adpt.h", - "csrc/moe/moe_init_routing_custom/op_host/CMakeLists.txt", - "csrc/moe/moe_init_routing_custom/op_host/aclnn_moe_init_routing_custom.cpp", - "csrc/moe/moe_init_routing_custom/op_host/aclnn_moe_init_routing_custom.h", - "csrc/moe/moe_init_routing_custom/op_host/moe_init_routing_custom.cpp", - "csrc/moe/moe_init_routing_custom/op_host/moe_init_routing_custom.h", - "csrc/moe/moe_init_routing_custom/op_host/moe_init_routing_custom_def.cpp", - "csrc/moe/moe_init_routing_custom/op_host/moe_init_routing_custom_infershape.cpp", - "csrc/moe/moe_init_routing_custom/op_host/moe_init_routing_custom_tiling.cpp", - "csrc/moe/moe_init_routing_custom/op_host/moe_init_routing_custom_tiling.h", - "csrc/moe/moe_init_routing_custom/op_host/moe_init_routing_custom_tiling_base.cpp", - "csrc/moe/moe_init_routing_custom/op_kernel/moe_custom_common.h", - "csrc/moe/moe_init_routing_custom/op_kernel/moe_custom_expert_tokens_count.h", - "csrc/moe/moe_init_routing_custom/op_kernel/moe_custom_full_load.h", - "csrc/moe/moe_init_routing_custom/op_kernel/moe_custom_full_load_base.h", - "csrc/moe/moe_init_routing_custom/op_kernel/moe_custom_full_load_dynamic_quant.h", - "csrc/moe/moe_init_routing_custom/op_kernel/moe_custom_full_load_static_quant.h", - "csrc/moe/moe_init_routing_custom/op_kernel/moe_custom_full_load_unquantized.h", - "csrc/moe/moe_init_routing_custom/op_kernel/moe_custom_gather_droppad_static_quant.h", - "csrc/moe/moe_init_routing_custom/op_kernel/moe_custom_gather_dynamic_quant.h", - "csrc/moe/moe_init_routing_custom/op_kernel/moe_custom_gather_out.h", - "csrc/moe/moe_init_routing_custom/op_kernel/moe_custom_gather_out_droppad.h", - "csrc/moe/moe_init_routing_custom/op_kernel/moe_custom_gather_sort_multi_core.h", - "csrc/moe/moe_init_routing_custom/op_kernel/moe_custom_gather_static_quant.h", - "csrc/moe/moe_init_routing_custom/op_kernel/moe_custom_mrgsort.h", - "csrc/moe/moe_init_routing_custom/op_kernel/moe_custom_mrgsort_out.h", - "csrc/moe/moe_init_routing_custom/op_kernel/moe_custom_mrgsort_out_performance.h", - "csrc/moe/moe_init_routing_custom/op_kernel/moe_custom_mrgsort_performance.h", - "csrc/moe/moe_init_routing_custom/op_kernel/moe_custom_row_idx_gather.h", - "csrc/moe/moe_init_routing_custom/op_kernel/moe_custom_row_idx_gather_droppad.h", - "csrc/moe/moe_init_routing_custom/op_kernel/moe_custom_row_idx_gather_droppad_dynamic.h", - "csrc/moe/moe_init_routing_custom/op_kernel/moe_custom_sort_actual_expert.h", - "csrc/moe/moe_init_routing_custom/op_kernel/moe_custom_sort_base.h", - "csrc/moe/moe_init_routing_custom/op_kernel/moe_custom_sort_multi_core.h", - "csrc/moe/moe_init_routing_custom/op_kernel/moe_custom_sort_multi_core_performance.h", - "csrc/moe/moe_init_routing_custom/op_kernel/moe_custom_sort_one_core.h", - "csrc/moe/moe_init_routing_custom/op_kernel/moe_init_routing_custom.cpp", - "csrc/moe/transpose_kv_cache_by_block/CMakeLists.txt", - "csrc/moe/transpose_kv_cache_by_block/op_host/CMakeLists.txt", - "csrc/moe/transpose_kv_cache_by_block/op_host/transpose_kv_cache_by_block_def.cpp", - "csrc/moe/transpose_kv_cache_by_block/op_host/transpose_kv_cache_by_block_proto.cpp", - "csrc/moe/transpose_kv_cache_by_block/op_host/transpose_kv_cache_by_block_tiling.cpp", - "csrc/moe/transpose_kv_cache_by_block/op_host/transpose_kv_cache_by_block_tiling.h", - "csrc/moe/transpose_kv_cache_by_block/op_kernel/common.h", - "csrc/moe/transpose_kv_cache_by_block/op_kernel/full_load.h", - "csrc/moe/transpose_kv_cache_by_block/op_kernel/general.h", - "csrc/moe/transpose_kv_cache_by_block/op_kernel/transpose_kv_cache_by_block.cpp", - "csrc/moe_combine_normal/op_host/CMakeLists.txt", - "csrc/moe_dispatch_normal/op_host/CMakeLists.txt", - "csrc/moe_gating_top_k/op_host/CMakeLists.txt", - "csrc/moe_grouped_matmul/op_host/CMakeLists.txt", - "csrc/moe_init_routing_custom/op_host/CMakeLists.txt", - "csrc/notify_dispatch/op_host/CMakeLists.txt", - "csrc/recurrent_gated_delta_rule/op_host/CMakeLists.txt", - "csrc/recurrent_gated_delta_rule_v310/op_host/CMakeLists.txt", - "csrc/reshape_and_cache_bnsd/op_host/CMakeLists.txt", - "csrc/scripts/opgen/opgen_standalone.py", - "csrc/scripts/opgen/template/CMakeLists.txt", - "csrc/scripts/opgen/template/add/CMakeLists.txt", - "csrc/scripts/opgen/template/add/examples/test_aclnn_add_example.cpp", - "csrc/scripts/opgen/template/add/op_host/CMakeLists.txt", - "csrc/scripts/opgen/template/add/op_host/add_example_def.cpp", - "csrc/scripts/opgen/template/add/op_host/add_example_infershape.cpp", - "csrc/scripts/opgen/template/add/op_host/add_example_tiling.cpp", - "csrc/scripts/opgen/template/add/op_kernel/add_example.cpp", - "csrc/scripts/opgen/template/add/op_kernel/add_example.h", - "csrc/scripts/opgen/template/add/op_kernel/add_example_tiling_data.h", - "csrc/scripts/opgen/template/add/op_kernel/add_example_tiling_key.h", - "csrc/scripts/opgen/template/add/tests/ut/.gitkeep", - "csrc/scripts/package/common/__init__.py", - "csrc/scripts/package/common/cfg/path.cfg", - "csrc/scripts/package/common/py/__init__.py", - "csrc/scripts/package/common/py/filelist.py", - "csrc/scripts/package/common/py/merge_binary_info_config.py", - "csrc/scripts/package/common/py/packer.py", - "csrc/scripts/package/common/py/pkg_parser.py", - "csrc/scripts/package/common/py/utils/comm_log.py", - "csrc/scripts/package/common/py/utils/funcbase.py", - "csrc/scripts/package/common/py/utils/pkg_utils.py", - "csrc/scripts/package/common/py/version_info.py", - "csrc/scripts/package/common/sh/check_version_required.awk", - "csrc/scripts/package/common/sh/cleanup.sh", - "csrc/scripts/package/common/sh/common_func.inc", - "csrc/scripts/package/common/sh/common_func_v2.inc", - "csrc/scripts/package/common/sh/common_installer.inc", - "csrc/scripts/package/common/sh/common_interface.csh", - "csrc/scripts/package/common/sh/common_interface.fish", - "csrc/scripts/package/common/sh/common_interface.sh", - "csrc/scripts/package/common/sh/install_common_parser.sh", - "csrc/scripts/package/common/sh/multi_version.inc", - "csrc/scripts/package/common/sh/script_operator.inc", - "csrc/scripts/package/common/sh/version_cfg.inc", - "csrc/scripts/package/common/sh/version_compatiable.inc", - "csrc/scripts/package/latest_manager/scripts/filelist.csv", - "csrc/scripts/package/latest_manager/scripts/install.sh", - "csrc/scripts/package/latest_manager/scripts/manager.sh", - "csrc/scripts/package/latest_manager/scripts/manager_func.sh", - "csrc/scripts/package/latest_manager/scripts/uninstall.sh", - "csrc/scripts/package/latest_manager/scripts/version.info", - "csrc/scripts/package/module/ascend/EngineeringCommon.xml", - "csrc/scripts/package/module/ascend/EngineeringFiles.xml", - "csrc/scripts/package/module/ascend/OpsTransformer.xml", - "csrc/scripts/package/module/ascend/OpsTransformerInc.xml", - "csrc/scripts/package/ops_transformer/ops_transformer.xml", - "csrc/scripts/package/ops_transformer/scripts/cleanup.sh", - "csrc/scripts/package/ops_transformer/scripts/empty_package_scripts/cleanup.sh", - "csrc/scripts/package/ops_transformer/scripts/empty_package_scripts/install.sh", - "csrc/scripts/package/ops_transformer/scripts/help.info", - "csrc/scripts/package/ops_transformer/scripts/install.sh", - "csrc/scripts/package/ops_transformer/scripts/opp_common.sh", - "csrc/scripts/package/ops_transformer/scripts/opp_custom_install.sh", - "csrc/scripts/package/ops_transformer/scripts/opp_custom_uninstall.sh", - "csrc/scripts/package/ops_transformer/scripts/opp_install.sh", - "csrc/scripts/package/ops_transformer/scripts/opp_uninstall.sh", - "csrc/scripts/package/ops_transformer/scripts/uninstall.sh", - "csrc/scripts/package/ops_transformer/scripts/ver_check.sh", - "csrc/scripts/package/package.py", - "csrc/scripts/util/__init__.py", - "csrc/scripts/util/build_opp_kernel_static.py", - "csrc/scripts/util/const_var.py", - "csrc/scripts/util/insert_op_info.py", - "csrc/scripts/util/merge_aicpu_info_json.sh", - "csrc/scripts/util/merge_proto.py", - "csrc/scripts/util/modify_gen_aclnn.py", - "csrc/sparse_flash_attention/op_host/CMakeLists.txt", - "csrc/torch_binding.cpp", - "csrc/transpose_kv_cache_by_block/op_host/CMakeLists.txt", - "csrc/utils/inc/kernel/moe_distribute_base.h", - "csrc/version.info", - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_bgmv_expand.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_bgmv_shrink.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_moe_init_routing_custom.py", - "vllm_ascend/platform.py", - "vllm_ascend/utils.py" - ] - }, - { - "commit": "56101406f56ad39afa5c29b5347c92a0534b0c8e", - "subject": "[CI][310P] Fixing the error reported when uninstalling Triton and fix moe bug on 310 (#9022)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/workflows/_e2e_test.yaml", - "vllm_ascend/_310p/fused_moe/fused_moe.py", - "vllm_ascend/_310p/ops/fla/idex.py", - "vllm_ascend/patch/worker/__init__.py", - "vllm_ascend/patch/worker/patch_idex_310.py" - ] - }, - { - "commit": "8486a744f4e4653d0e4f0fc48684517ffafdc25b", - "subject": "[BugFix] Fix quantization accuracy bug (#9036)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/_310p/fused_moe/experts_selector.py", - "vllm_ascend/_310p/quantization/methods/w8a8_dynamic.py", - "vllm_ascend/ops/fused_moe/experts_selector.py", - "vllm_ascend/ops/fused_moe/fused_moe.py", - "vllm_ascend/quantization/methods/w4a16.py", - "vllm_ascend/quantization/methods/w4a4_mxfp4.py", - "vllm_ascend/quantization/methods/w4a8.py", - "vllm_ascend/quantization/methods/w8a8_mxfp8.py" - ] - }, - { - "commit": "3636a3dc9e7abe08e9eb27ef56d0c6ef89d71ab3", - "subject": "[Misc][Main2Main] Upgrade vLLM to v0.20.1 and 0506 (#8983)", - "topics": [ - "deepseek-v4-dspark" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/dockerfiles/Dockerfile.lint", - ".github/workflows/pr_test_full.yaml", - ".github/workflows/pr_test_light.yaml", - ".github/workflows/schedule_update_estimated_time.yaml", - ".github/workflows/schedule_vllm_e2e_test.yaml", - ".github/workflows/scripts/config.yaml", - "Dockerfile", - "Dockerfile.310p", - "Dockerfile.310p.openEuler", - "Dockerfile.a3", - "Dockerfile.a3.openEuler", - "Dockerfile.openEuler", - "docs/source/conf.py", - "mypy.ini", - "tests/e2e/multicard/2-cards/test_qwen3_moe.py", - "tests/e2e/singlecard/model_runner_v2/test_basic.py", - "tests/ut/worker/test_model_runner_v2.py", - "vllm_ascend/core/scheduler_profiling_chunk.py", - "vllm_ascend/patch/platform/__init__.py", - "vllm_ascend/patch/platform/patch_mla_prefill_backend.py", - "vllm_ascend/patch/worker/patch_v2/patch_triton.py", - "vllm_ascend/worker/npu_input_batch.py", - "vllm_ascend/worker/v2/aclgraph_utils.py", - "vllm_ascend/worker/v2/sample/logprob.py", - "vllm_ascend/worker/v2/spec_decode/__init__.py", - "vllm_ascend/worker/v2/spec_decode/eagle/aclgraph.py", - "vllm_ascend/worker/v2/spec_decode/eagle/speculator.py", - "vllm_ascend/worker/v2/spec_decode/probabilistic_rejection_sampler_utils.py", - "vllm_ascend/worker/worker.py" - ] - }, - { - "commit": "bd25f2e8fb6db1c63b77ca1a520baf52aef662d6", - "subject": "[Community] Nominate new maintainer @linfeng-yuan (#8996)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/community/contributors.md" - ] - }, - { - "commit": "7fd2cede36c6d661e0f06ddfb35502659639660e", - "subject": "[Misc][Main2Main] Upgrade vLLM to 0427 (#8899)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/workflows/pr_test_full.yaml", - ".github/workflows/pr_test_light.yaml", - ".gitignore", - "csrc/third_party/catlass", - "docs/source/conf.py", - "tests/ut/_310p/fused_moe/test_shared_fused_moe_310.py", - "tests/ut/ops/test_fused_moe.py", - "tests/ut/spec_decode/test_eagle_proposer.py", - "vllm_ascend/_310p/fused_moe/fused_moe.py", - "vllm_ascend/__init__.py", - "vllm_ascend/attention/utils.py", - "vllm_ascend/core/scheduler_profiling_chunk.py", - "vllm_ascend/lora/utils.py", - "vllm_ascend/ops/fused_moe/fused_moe.py", - "vllm_ascend/ops/layernorm.py", - "vllm_ascend/patch/__init__.py", - "vllm_ascend/patch/platform/__init__.py", - "vllm_ascend/patch/platform/patch_kv_cache_utils.py", - "vllm_ascend/sample/rejection_sampler.py", - "vllm_ascend/spec_decode/dflash_proposer.py", - "vllm_ascend/spec_decode/eagle_proposer.py", - "vllm_ascend/utils.py", - "vllm_ascend/worker/model_runner_v1.py", - "vllm_ascend/worker/v2/attn_utils.py", - "vllm_ascend/worker/v2/model_runner.py", - "vllm_ascend/worker/worker.py" - ] - }, - { - "commit": "ba074eb4bf095264b2d1280447281633ab66e112", - "subject": "[Feature] enable indexCache on npu (#8398)", - "topics": [ - "deepseek-v4-dspark", - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/attention/sfa_v1.py", - "vllm_ascend/ops/mla.py" - ] - }, - { - "commit": "66cc615237aa0eceaca70fd0f492d83bf506c926", - "subject": "[Feature][Doc] Add AI QoS module, tuning tool, and user guide (#8706)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - ".pre-commit-config.yaml", - "docs/source/user_guide/feature_guide/Ai_QoS_introduction_en.md", - "docs/source/user_guide/feature_guide/images/ai_qos1.png", - "docs/source/user_guide/feature_guide/images/ai_qos2.png", - "docs/source/user_guide/feature_guide/index.md", - "tests/ut/test_ai_qos_tool.py", - "tools/ai_qos.py", - "tools/ai_qos/CMakeLists.txt", - "tools/ai_qos/ai_qos.cpp", - "typos.toml" - ] - }, - { - "commit": "ac53f923ba91077c0b29cf11db01f0ecdef3677e", - "subject": "[Feature] update dataWithScale and combine in dispatch_ffn_kernel (#8902)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "csrc/dispatch_ffn_combine/op_kernel/dispatch_ffn_combine.h", - "csrc/dispatch_ffn_combine/op_kernel/dispatch_ffn_combine_kernel.hpp", - "csrc/dispatch_ffn_combine/op_kernel/moe_init_routing_quant_v2/moe_v2_fullload_dynamic_quant.h", - "csrc/dispatch_ffn_combine/op_kernel/moe_init_routing_quant_v2/moe_v2_gather_dynamic_quant.h", - "csrc/dispatch_ffn_combine/op_kernel/utils/block_epilogue_pertoken_v2.hpp" - ] - }, - { - "commit": "f62ddb51765f6f865a222699ad27b608a17f6787", - "subject": "[CI] Add nightly case:GLM-5_1-W8A8;GLM-5.1-W8A8;Qwen3.5-397B-w8a8;Qwen3.5-397B-w4a8;deepseekv3.2 (#8680)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/misc/model_dataset_list.json", - ".github/workflows/schedule_nightly_test_a2.yaml", - ".github/workflows/schedule_nightly_test_a3.yaml", - "tests/e2e/nightly/multi_node/config/GLM5_1-W8A8-A2-dual-nodes.yaml", - "tests/e2e/nightly/multi_node/config/GLM5_1-W8A8-A3-dual-nodes.yaml", - "tests/e2e/nightly/multi_node/config/GLM5_1-W8A8-EP.yaml", - "tests/e2e/nightly/multi_node/scripts/multi_node_config.py", - "tests/e2e/nightly/multi_node/scripts/test_multi_node.py", - "tests/e2e/nightly/single_node/models/configs/DeepSeek-V3.2-W8A8.yaml", - "tests/e2e/nightly/single_node/models/configs/Qwen3.5-397B-A17B-W8A8-mtp-A3.yaml", - "tests/e2e/nightly/single_node/models/configs/Qwen3.5-397B-A17B-w4a8-mtp-A2.yaml" - ] - }, - { - "commit": "46b77d5bf1801d6b2cfe4f945fae6c42877971bf", - "subject": "[Misc] Drop useless patch (#8889)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/patch/worker/patch_common/test_patch_routed_experts_capturer.py", - "vllm_ascend/patch/__init__.py", - "vllm_ascend/patch/worker/__init__.py", - "vllm_ascend/patch/worker/patch_multimodal_merge.py", - "vllm_ascend/patch/worker/patch_routed_experts_capturer.py" - ] - }, - { - "commit": "e7336feb2cadda2941807f88e6b7913d4198abf7", - "subject": "[Doc] Translated Doc files 2026-05-06 (#8881)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/locale/zh_CN/LC_MESSAGES/community/contributors.po", - "docs/source/locale/zh_CN/LC_MESSAGES/community/user_stories/index.po", - "docs/source/locale/zh_CN/LC_MESSAGES/community/user_stories/llamafactory.po", - "docs/source/locale/zh_CN/LC_MESSAGES/community/versioning_policy.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/Design_Documents/ACL_Graph.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/Design_Documents/add_custom_aclnn_op.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/Design_Documents/cpu_binding.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/Design_Documents/dynamic_chunked_pipeline_parallel.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/contribution/testing.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/evaluation/using_ais_bench.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/performance_and_debug/optimization_and_tuning.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/performance_and_debug/performance_benchmark.po", - "docs/source/locale/zh_CN/LC_MESSAGES/faqs.po", - "docs/source/locale/zh_CN/LC_MESSAGES/quick_start.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/features/dynamic_chunked_pipeline_parallel.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/DeepSeekOCR2.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/GLM4.x.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/GLM5.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Kimi-K2.5.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen-VL-Dense.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3.5-27B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3.5-397B-A17B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/configuration/additional_config.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/cpu_binding.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/dynamic_chunk_pipeline_parallel.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/epd_disaggregation.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/kv_pool.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/lora.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/quantization.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/ucm_deployment.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/release_notes.po" - ] - }, - { - "commit": "c7749799e30af5f1c86b535c74be9dac695093bb", - "subject": "[BugFix] Fix Ascend MoE routing expert count with EPLB (#8864)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/quantization/methods/test_moe_logical_experts.py", - "vllm_ascend/ops/fused_moe/fused_moe.py", - "vllm_ascend/quantization/methods/base.py", - "vllm_ascend/quantization/methods/w4a16.py", - "vllm_ascend/quantization/methods/w4a4_mxfp4.py", - "vllm_ascend/quantization/methods/w4a8.py", - "vllm_ascend/quantization/methods/w8a8_dynamic.py", - "vllm_ascend/quantization/methods/w8a8_mxfp8.py" - ] - }, - { - "commit": "37e36a82f844505ea2bf255304c7fd43b4d1d0c7", - "subject": "[Misc] Drop vlllm 0.19.1 support (#8855)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/workflows/pr_test_full.yaml", - ".github/workflows/pr_test_light.yaml", - ".github/workflows/schedule_update_estimated_time.yaml", - ".github/workflows/schedule_vllm_e2e_test.yaml", - "Dockerfile", - "Dockerfile.310p", - "Dockerfile.310p.openEuler", - "Dockerfile.a3", - "Dockerfile.a3.openEuler", - "Dockerfile.openEuler", - "docs/source/community/versioning_policy.md", - "tests/e2e/singlecard/model_runner_v2/test_basic.py", - "tests/e2e/singlecard/spec_decode/test_v1_spec_decode.py", - "tests/ut/ops/test_mla.py", - "vllm_ascend/_310p/fused_moe/fused_moe.py", - "vllm_ascend/ascend_forward_context.py", - "vllm_ascend/attention/context_parallel/sfa_cp.py", - "vllm_ascend/attention/sfa_v1.py", - "vllm_ascend/core/recompute_scheduler.py", - "vllm_ascend/core/scheduler_dynamic_batch.py", - "vllm_ascend/core/scheduler_profiling_chunk.py", - "vllm_ascend/ops/fused_moe/fused_moe.py", - "vllm_ascend/ops/mla.py", - "vllm_ascend/patch/platform/patch_balance_schedule.py", - "vllm_ascend/patch/worker/__init__.py", - "vllm_ascend/patch/worker/patch_qwen3vl.py", - "vllm_ascend/platform.py", - "vllm_ascend/spec_decode/__init__.py", - "vllm_ascend/spec_decode/eagle_proposer.py", - "vllm_ascend/worker/worker.py" - ] - }, - { - "commit": "7b9438d524ea00411c158318890b0e3f89358009", - "subject": "[Doc] Add GLM-5.1 to models tutorials (#8054)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/tutorials/models/GLM5.md" - ] - }, - { - "commit": "34fc65127e45e71cb18f4ad3506673920ee74665", - "subject": "[Performance][MLA][PCP] Optimize MLA PCP prefill attention (#8787)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/attention/test_mla_cp.py", - "tests/ut/worker/test_pcp_manager.py", - "vllm_ascend/attention/context_parallel/common_cp.py", - "vllm_ascend/attention/context_parallel/mla_cp.py", - "vllm_ascend/attention/utils.py", - "vllm_ascend/worker/pcp_utils.py" - ] - }, - { - "commit": "89b739a31e9b92492c9cebd566a08f872609c39b", - "subject": "[Ops][Feature] Support Bailing quantization (#8709)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/quantization/modelslim_config.py" - ] - }, - { - "commit": "3ffda749d1afcc63afff88db5524e04673d87586", - "subject": "[BugFix]Correct MLAPO (#8647)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/attention/test_mla_v1.py", - "vllm_ascend/attention/mla_v1.py" - ] - }, - { - "commit": "b62ed9dd2064d96d0a15f33d0fdaffb5663340ae", - "subject": "[EPLB][BugFix] The construction of topk_ids uses the number of logical experts (#8501)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/ops/test_fused_moe.py", - "tests/ut/quantization/methods/test_w4a16.py", - "tests/ut/quantization/methods/test_w4a4_mxfp4.py", - "tests/ut/quantization/methods/test_w4a8.py", - "tests/ut/quantization/methods/test_w8a8_dynamic.py", - "tests/ut/quantization/methods/test_w8a8_mxfp8.py", - "vllm_ascend/_310p/fused_moe/experts_selector.py", - "vllm_ascend/_310p/fused_moe/fused_moe.py", - "vllm_ascend/_310p/quantization/methods/w8a8_dynamic.py", - "vllm_ascend/ops/fused_moe/experts_selector.py", - "vllm_ascend/ops/fused_moe/fused_moe.py", - "vllm_ascend/quantization/method_adapters.py", - "vllm_ascend/quantization/methods/base.py", - "vllm_ascend/quantization/methods/w4a16.py", - "vllm_ascend/quantization/methods/w4a4_mxfp4.py", - "vllm_ascend/quantization/methods/w4a8.py", - "vllm_ascend/quantization/methods/w8a8_dynamic.py", - "vllm_ascend/quantization/methods/w8a8_mxfp8.py" - ] - }, - { - "commit": "c9aff2b095e0e3b9295bc4cadc497e5d8f447840", - "subject": "[Performance] In scenarios A2 and A3, replace npu_fusion_attention with the _npu_flash_attention_unpad operator. (#8671)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/device/device_op.py", - "vllm_ascend/ops/mm_encoder_attention.py" - ] - }, - { - "commit": "eb3548add4b8011c7ccd94aafb4dd3aa967e0e02", - "subject": "[Misc][main2main] Adapt vllm-ascend to vLLM 0420 (#8428)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/workflows/dockerfiles/Dockerfile.lint", - ".github/workflows/pr_test_full.yaml", - ".github/workflows/pr_test_light.yaml", - "docs/source/conf.py", - "tests/ut/ops/test_mla.py", - "vllm_ascend/ascend_forward_context.py", - "vllm_ascend/attention/context_parallel/sfa_cp.py", - "vllm_ascend/attention/sfa_v1.py", - "vllm_ascend/ops/mla.py", - "vllm_ascend/platform.py", - "vllm_ascend/worker/worker.py" - ] - }, - { - "commit": "a5622f56d906cf90ba54f604a2bc767de77d971a", - "subject": "[Feature]Add xmask feature for dispatch_ffn_combine operator (only for w8a8 branch) (#8560)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "csrc/dispatch_ffn_combine/dispatch_ffn_combine_torch_adpt.h", - "csrc/dispatch_ffn_combine/op_host/aclnn_dispatch_ffn_combine.cpp", - "csrc/dispatch_ffn_combine/op_host/aclnn_dispatch_ffn_combine.h", - "csrc/dispatch_ffn_combine/op_host/dispatch_ffn_combine_def.cpp", - "csrc/dispatch_ffn_combine/op_host/dispatch_ffn_combine_tiling.cpp", - "csrc/dispatch_ffn_combine/op_kernel/dispatch_ffn_combine.cpp", - "csrc/dispatch_ffn_combine/op_kernel/dispatch_ffn_combine.h", - "csrc/dispatch_ffn_combine/op_kernel/dispatch_ffn_combine_kernel.hpp", - "csrc/dispatch_ffn_combine/op_kernel/utils/block_epilogue_pertoken_v2.hpp", - "csrc/torch_binding.cpp", - "csrc/torch_binding_meta.cpp", - "tests/e2e/nightly/single_node/ops/multicard_ops_a3/test_dispatch_ffn_combine.py", - "vllm_ascend/ascend_forward_context.py", - "vllm_ascend/ops/fused_moe/moe_comm_method.py" - ] - }, - { - "commit": "cb18e84e65d3893aa21ab680670a08d4d3e42be4", - "subject": "[Misc]update vllm to v0.19.1 (#8448)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/pr_test_full.yaml", - ".github/workflows/pr_test_light.yaml", - ".github/workflows/schedule_update_estimated_time.yaml", - ".github/workflows/schedule_vllm_e2e_test.yaml", - ".github/workflows/scripts/config.yaml", - "Dockerfile", - "Dockerfile.310p", - "Dockerfile.310p.openEuler", - "Dockerfile.a3", - "Dockerfile.a3.openEuler", - "Dockerfile.openEuler", - "docs/source/conf.py", - "examples/prompt_embed_inference.py", - "examples/prompt_embedding_inference.py", - "pyproject.toml", - "requirements-dev.txt", - "requirements.txt", - "tests/e2e/multicard/2-cards/test_expert_parallel.py", - "tests/e2e/multicard/2-cards/test_offline_inference_distributed.py", - "tests/e2e/multicard/4-cards/long_sequence/test_accuracy.py", - "tests/e2e/multicard/4-cards/test_pipeline_parallel.py", - "tests/e2e/nightly/multi_node/scripts/run.sh", - "tests/e2e/singlecard/model_runner_v2/test_basic.py", - "tests/e2e/singlecard/spec_decode/test_v1_spec_decode.py", - "tests/e2e/singlecard/test_completion_with_prompt_embeds.py", - "vllm_ascend/_310p/fused_moe/fused_moe.py", - "vllm_ascend/core/recompute_scheduler.py", - "vllm_ascend/core/scheduler_dynamic_batch.py", - "vllm_ascend/core/scheduler_profiling_chunk.py", - "vllm_ascend/ops/fused_moe/fused_moe.py", - "vllm_ascend/patch/platform/patch_balance_schedule.py", - "vllm_ascend/patch/worker/__init__.py", - "vllm_ascend/patch/worker/patch_qwen3vl.py", - "vllm_ascend/spec_decode/__init__.py", - "vllm_ascend/spec_decode/eagle_proposer.py" - ] - }, - { - "commit": "a098a07ab39e6cf8de0d4267e25774ec4536be97", - "subject": "[Refactor][Misc] Use lazy formatting for log (#8756)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "examples/disaggregated_prefill_v1/load_balance_proxy_layerwise_server_example.py", - "examples/disaggregated_prefill_v1/load_balance_proxy_server_example.py", - "examples/epd_disaggregated/epd_load_balance_proxy_layerwise_server_example.py", - "examples/external_online_dp/dp_load_balance_proxy_server.py", - "pyproject.toml", - "setup.py", - "tests/e2e/conftest.py", - "tools/vllm_bench.py", - "vllm_ascend/ascend_config.py", - "vllm_ascend/compilation/passes/sequence_parallelism.py", - "vllm_ascend/compilation/passes/sequence_parallelism_moe.py", - "vllm_ascend/compilation/passes/utils/npugraph_ex_utils_check.py", - "vllm_ascend/core/scheduler_dynamic_batch.py", - "vllm_ascend/cpu_binding.py", - "vllm_ascend/distributed/kv_transfer/kv_p2p/mooncake_connector.py", - "vllm_ascend/distributed/kv_transfer/kv_p2p/mooncake_layerwise_connector.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/ascend_store/backend/memcache_backend.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/ascend_store/backend/mooncake_backend.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/ascend_store/config_data.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/ascend_store/kv_transfer.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/ascend_store/pool_scheduler.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/ascend_store/pool_worker.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/cpu_offload/cpu_kv_cache_manager.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/cpu_offload/cpu_offload_connector.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/cpu_offload/metadata.py", - "vllm_ascend/distributed/kv_transfer/utils/utils.py", - "vllm_ascend/eplb/adaptor/vllm_adaptor.py", - "vllm_ascend/eplb/core/eplb_worker.py", - "vllm_ascend/eplb/eplb_updator.py", - "vllm_ascend/model_loader/netloader/executor/elastic_load.py", - "vllm_ascend/model_loader/netloader/interaction/elastic.py", - "vllm_ascend/model_loader/netloader/load.py", - "vllm_ascend/model_loader/netloader/netloader.py", - "vllm_ascend/model_loader/netloader/utils.py", - "vllm_ascend/model_loader/rfork/rfork_loader.py", - "vllm_ascend/ops/fused_moe/fused_moe.py", - "vllm_ascend/patch/worker/patch_draft_quarot.py", - "vllm_ascend/patch/worker/patch_weight_utils.py", - "vllm_ascend/platform.py", - "vllm_ascend/profiling_config.py", - "vllm_ascend/quantization/utils.py", - "vllm_ascend/utils.py", - "vllm_ascend/worker/worker.py", - "vllm_ascend/xlite/xlite.py" - ] - }, - { - "commit": "189ab05f100034cdbe65b49e25f9c6a50ec0376e", - "subject": "[CI]Add timeout check for custom op CI and optimize test parameters (#8714)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/_e2e_nightly_single_node.yaml", - "tests/e2e/nightly/single_node/ops/conftest.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_copy_and_expand_eagle_inputs.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_moe_init_routing_custom.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_causal_conv1d.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_fused_qkvzba_split_reshape_cat.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_penality.py" - ] - }, - { - "commit": "e15d766abaf1de664721247ac9a77a0cb5fb0b4d", - "subject": "[Attention][BugFix] Fix the atten_mask in the npu_fused_infer_attention_score_v2 (#8387)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/_310p/attention/test_attention_mask_310.py", - "vllm_ascend/_310p/attention/attention_mask.py", - "vllm_ascend/attention/attention_mask.py", - "vllm_ascend/attention/attention_v1.py" - ] - }, - { - "commit": "dedb0d36403dac9afc8827dd3729783263cc36a5", - "subject": "[CI][Docs] Add scheduled Sphinx link check for docs (#8273)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/schedule_doc_linkcheck.yaml", - "docs/source/community/contributors.md", - "docs/source/community/user_stories/index.md", - "docs/source/community/user_stories/llamafactory.md", - "docs/source/conf.py", - "docs/source/developer_guide/Design_Documents/ACL_Graph.md", - "docs/source/developer_guide/contribution/testing.md", - "docs/source/developer_guide/performance_and_debug/performance_benchmark.md", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/GLM4.x.po", - "docs/source/tutorials/models/GLM4.x.md", - "docs/source/tutorials/models/GLM5.md", - "docs/source/tutorials/models/Kimi-K2.5.md", - "docs/source/tutorials/models/Qwen3.5-27B.md", - "docs/source/tutorials/models/Qwen3.5-397B-A17B.md", - "docs/source/user_guide/feature_guide/epd_disaggregation.md", - "docs/source/user_guide/feature_guide/kv_pool.md", - "docs/source/user_guide/feature_guide/lora.md", - "docs/source/user_guide/feature_guide/quantization.md", - "docs/source/user_guide/release_notes.md", - "docs/source/user_guide/support_matrix/supported_features.md", - "tools/collect_user_first_contribution.sh" - ] - }, - { - "commit": "7d90f7095c8891b4c58d68bef2b60e95128fbfc0", - "subject": "[Ops][BugFix] Add None guard for all_moe_layers in fused_moe (#8620)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/ops/fused_moe/fused_moe.py" - ] - }, - { - "commit": "7899cae51d8e168e5a264f82db8bfdfacaae09cf", - "subject": "[Ops][Misc] Refactor GDN recurrent attention logic (#8581)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/e2e/multicard/4-cards/long_sequence/test_basic.py", - "vllm_ascend/ops/gdn.py" - ] - }, - { - "commit": "13b56fbab25c306edf865ff379af640eab36e2a6", - "subject": "[BugFix]Backport validate pd mode feature gates no fused mc2 (#8582)", - "topics": [ - "communication", - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/test_platform.py", - "vllm_ascend/envs.py", - "vllm_ascend/platform.py" - ] - }, - { - "commit": "8bf4e3bdb754c50777778707e640d53c5d0b0785", - "subject": "[Doc][Misc] Modifying Configuration Parameters (#8617)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/tutorials/models/DeepSeek-V3.2.md", - "docs/source/tutorials/models/GLM4.x.md", - "docs/source/tutorials/models/GLM5.md", - "docs/source/tutorials/models/Qwen3.5-397B-A17B.md", - "tests/e2e/nightly/multi_node/config/DeepSeek-R1-W8A8-longseq.yaml", - "tests/e2e/nightly/multi_node/config/DeepSeek-V3_2-W8A8-EP-aime2025.yaml", - "tests/e2e/nightly/multi_node/config/DeepSeek-V3_2-W8A8-EP.yaml", - "vllm_ascend/distributed/kv_transfer/kv_p2p/mooncake_connector.py" - ] - }, - { - "commit": "f53051be7e5a11e737425f360728a86fdf6ff0df", - "subject": "[Feature] Supports GLM4.7 C8 scenarios (#8174)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/quantization/test_kv_c8.py", - "tests/ut/quantization/test_modelslim_config.py", - "vllm_ascend/attention/attention_v1.py", - "vllm_ascend/distributed/kv_transfer/kv_p2p/mooncake_layerwise_connector.py", - "vllm_ascend/patch/__init__.py", - "vllm_ascend/patch/worker/__init__.py", - "vllm_ascend/patch/worker/patch_gqa_c8.py", - "vllm_ascend/quantization/methods/kv_c8.py", - "vllm_ascend/quantization/modelslim_config.py" - ] - }, - { - "commit": "c089b6302693ea983b8994c951d4b332ac52b0d7", - "subject": "[Doc] Update GLM4.7 documentation (#8450)", - "topics": [ - "communication" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/tutorials/models/GLM4.x.md" - ] - }, - { - "commit": "21bbd7e1a2adc292811fbcab99ea1682f577e135", - "subject": "[Feature] Add recurrent ascendc ops (#8055)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "csrc/build_aclnn.sh", - "csrc/recurrent_gated_delta_rule/op_host/CMakeLists.txt", - "csrc/recurrent_gated_delta_rule/op_host/aclnn_recurrent_gated_delta_rule.cpp", - "csrc/recurrent_gated_delta_rule/op_host/aclnn_recurrent_gated_delta_rule.h", - "csrc/recurrent_gated_delta_rule/op_host/error_log.h", - "csrc/recurrent_gated_delta_rule/op_host/math_util.h", - "csrc/recurrent_gated_delta_rule/op_host/recurrent_gated_delta_rule.cpp", - "csrc/recurrent_gated_delta_rule/op_host/recurrent_gated_delta_rule.h", - "csrc/recurrent_gated_delta_rule/op_host/recurrent_gated_delta_rule_def.cpp", - "csrc/recurrent_gated_delta_rule/op_host/recurrent_gated_delta_rule_infershape.cpp", - "csrc/recurrent_gated_delta_rule/op_host/recurrent_gated_delta_rule_tiling.cpp", - "csrc/recurrent_gated_delta_rule/op_host/recurrent_gated_delta_rule_tiling.h", - "csrc/recurrent_gated_delta_rule/op_host/tiling_base.h", - "csrc/recurrent_gated_delta_rule/op_host/tiling_templates_registry.h", - "csrc/recurrent_gated_delta_rule/op_kernel/recurrent_gated_delta_rule.cpp", - "csrc/recurrent_gated_delta_rule/op_kernel/recurrent_gated_delta_rule.h", - "csrc/recurrent_gated_delta_rule/op_kernel/recurrent_gated_delta_rule_tiling_data.h", - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_recurrent_gated_delta_rule.py", - "vllm_ascend/ops/gdn.py" - ] - }, - { - "commit": "2167c066d4c0603edefbce2ea02bb9c129d31a98", - "subject": "[Refactor] Replace unquantized_gemm monkey-patch with proper custom op dispatch (#8483)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/ops/test_linear.py", - "vllm_ascend/compilation/passes/allreduce_rmsnorm_fusion_pass.py", - "vllm_ascend/ops/linear.py", - "vllm_ascend/patch/__init__.py", - "vllm_ascend/patch/worker/__init__.py", - "vllm_ascend/patch/worker/patch_unquantized_gemm.py" - ] - }, - { - "commit": "a59a13cd0db6f36ac3b53097a6a4b6ad7adbd168", - "subject": "[Feature][A5] Support MXFP8 MLAPO (#7797)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/attention/mla_v1.py", - "vllm_ascend/attention/utils.py", - "vllm_ascend/device/device_op.py" - ] - }, - { - "commit": "29ab3eb972ca03d35bb811a53ae37b8509fef2a9", - "subject": "[Doc] Translated Doc files 2026-04-21 (#8485)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/locale/zh_CN/LC_MESSAGES/community/contributors.po", - "docs/source/locale/zh_CN/LC_MESSAGES/community/versioning_policy.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/Design_Documents/ACL_Graph.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/Design_Documents/KV_Cache_Pool_Guide.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/Design_Documents/ModelRunner_prepare_inputs.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/Design_Documents/cpu_binding.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/Design_Documents/disaggregated_prefill.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/Design_Documents/eplb_swift_balancer.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/Design_Documents/npugraph_ex.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/Design_Documents/patch.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/Design_Documents/quantization.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/contribution/e2e_ci_test.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/contribution/index.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/contribution/multi_node_test.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/evaluation/using_ais_bench.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/evaluation/using_evalscope.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/evaluation/using_lm_eval.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/evaluation/using_opencompass.po", - "docs/source/locale/zh_CN/LC_MESSAGES/faqs.po", - "docs/source/locale/zh_CN/LC_MESSAGES/installation.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/features/long_sequence_context_parallel_multi_node.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/features/long_sequence_context_parallel_single_node.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/features/pd_colocated_mooncake_multi_instance.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/features/pd_disaggregation_mooncake_multi_node.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/features/pd_disaggregation_mooncake_single_node.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/DeepSeek-V3.1.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/DeepSeekOCR2.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/GLM4.x.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/GLM5.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Kimi-K2.5.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/MiniMax-M2.5.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/PaddleOCR-VL.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen-VL-Dense.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen2.5-Omni.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-Next.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-Omni-30B-A3B-Thinking.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3.5-27B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3.5-397B-A17B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3_embedding.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/configuration/additional_config.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/Fine_grained_TP.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/batch_invariance.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/context_parallel.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/cpu_binding.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/dynamic_batch.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/epd_disaggregation.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/eplb_swift_balancer.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/external_dp.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/kv_pool.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/large_scale_ep.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/layer_sharding.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/netloader.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/npugraph_ex.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/sleep_mode.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/ucm_deployment.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/weight_prefetch.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/release_notes.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/support_matrix/supported_models.po" - ] - }, - { - "commit": "84bddfdccc9224dbb5af6bbd14db299ebc7d0b4d", - "subject": "[Feature] Hamming-based sparse attention operators (#8346)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".pre-commit-config.yaml", - "csrc/build_aclnn.sh", - "csrc/hamming_dist_top_k/op_host/CMakeLists.txt", - "csrc/hamming_dist_top_k/op_host/hamming_dist_top_k.cpp", - "csrc/hamming_dist_top_k/op_host/hamming_dist_top_k.h", - "csrc/hamming_dist_top_k/op_host/hamming_dist_top_k_def.cpp", - "csrc/hamming_dist_top_k/op_host/hamming_dist_top_k_proto.cpp", - "csrc/hamming_dist_top_k/op_host/hamming_dist_top_k_split.cpp", - "csrc/hamming_dist_top_k/op_host/hamming_dist_top_k_split.h", - "csrc/hamming_dist_top_k/op_host/hamming_dist_top_k_tiling.cpp", - "csrc/hamming_dist_top_k/op_host/hamming_dist_top_k_tiling.h", - "csrc/hamming_dist_top_k/op_host/op_host_util.h", - "csrc/hamming_dist_top_k/op_kernel/hamming_dist_top_k.cpp", - "csrc/hamming_dist_top_k/op_kernel/hamming_dist_top_k_base.h", - "csrc/hamming_dist_top_k/op_kernel/hamming_dist_top_k_parallel.h", - "csrc/hamming_dist_top_k/op_kernel/hamming_dist_top_k_split_s.h", - "csrc/reshape_and_cache_bnsd/op_host/CMakeLists.txt", - "csrc/reshape_and_cache_bnsd/op_host/reshape_and_cache_bnsd.cpp", - "csrc/reshape_and_cache_bnsd/op_host/reshape_and_cache_bnsd_proto.cpp", - "csrc/reshape_and_cache_bnsd/op_host/reshape_and_cache_bnsd_tiling.cpp", - "csrc/reshape_and_cache_bnsd/op_host/reshape_and_cache_bnsd_tiling.h", - "csrc/reshape_and_cache_bnsd/op_kernel/kernel_utils.h", - "csrc/reshape_and_cache_bnsd/op_kernel/reshape_and_cache_bnsd.cpp", - "csrc/torch_binding.cpp", - "csrc/torch_binding_meta.cpp", - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_hamming_dist_top_k.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_reshape_and_cache_bnsd.py" - ] - }, - { - "commit": "840b8f05d425ac7d01487bb6362ddf11855a62ea", - "subject": "[Misc] Remove useless patch (#8446)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/patch/__init__.py", - "vllm_ascend/patch/platform/__init__.py", - "vllm_ascend/patch/platform/patch_fusion_matcher_compat_ops.py" - ] - }, - { - "commit": "66eea45ade9aa2294da0a9619ee1145130a92c83", - "subject": "[CI]Style: Convert `test/` to ruff format(Batch #8) (#7438)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "pyproject.toml", - "tests/ut/ops/test_activation.py", - "tests/ut/ops/test_comm_utils.py", - "tests/ut/ops/test_fused_moe.py", - "tests/ut/ops/test_gdn_chunk_meta.py", - "tests/ut/ops/test_linear.py", - "tests/ut/ops/test_mla.py", - "tests/ut/ops/test_moe_comm_method.py", - "tests/ut/ops/test_moe_runtime_args.py", - "tests/ut/ops/test_prepare_finalize.py", - "tests/ut/ops/test_rotary_embedding.py", - "tests/ut/ops/test_token_dispatcher.py", - "tests/ut/ops/test_vocab_parallel_embedding.py", - "tests/ut/patch/worker/patch_common/test_hccl_pg_registry.py", - "tests/ut/patch/worker/patch_common/test_patch_distributed.py", - "tests/ut/patch/worker/patch_common/test_patch_gdn_attn.py", - "tests/ut/patch/worker/patch_common/test_patch_routed_experts_capturer.py", - "tests/ut/quantization/test_kv_c8.py", - "tests/ut/quantization/test_modelslim_config.py", - "tests/ut/quantization/test_quant_utils.py", - "tests/ut/quantization/test_w4a16.py", - "tests/ut/quantization/test_w4a4_flatquant_dynamic.py", - "tests/ut/quantization/test_w4a8_dynamic.py", - "tests/ut/quantization/test_w8a16.py", - "tests/ut/quantization/test_w8a8.py", - "tests/ut/quantization/test_w8a8_dynamic.py", - "tests/ut/sample/test_rejection_sampler.py", - "tests/ut/sample/test_sampler.py", - "tests/ut/spec_decode/test_eagle_proposer.py", - "tests/ut/test_ascend_config.py", - "tests/ut/test_envs.py", - "tests/ut/test_platform.py", - "tests/ut/test_utils.py", - "tests/ut/worker/test_model_runner_v1.py", - "tests/ut/worker/test_pcp_manager.py", - "tests/ut/worker/test_worker_multi_instance.py", - "tests/ut/worker/test_worker_v1.py" - ] - }, - { - "commit": "756029b50dcce61d82d08291ae8dc5f9f88b2cba", - "subject": "[BugFix] handle enum-based MoE activation in fuse_moe/moe_mlp unquant_apply_mlp (#8465)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/ops/fused_moe/moe_mlp.py" - ] - }, - { - "commit": "700d4829166dbbd6e2aedc049850d57dba8fd173", - "subject": "[BugFix] Gate recompute/balance/fused_mc2 by PD mode (#8373)", - "topics": [ - "communication", - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "docs/source/user_guide/configuration/additional_config.md", - "tests/ut/test_platform.py", - "vllm_ascend/ascend_config.py", - "vllm_ascend/envs.py", - "vllm_ascend/platform.py" - ] - }, - { - "commit": "7324578a7a327ba3deba340612b97493ac9b4c57", - "subject": "[BugFix]w8a8 dispatch ffn combine bias param adapt (#8342)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "csrc/dispatch_ffn_combine/dispatch_ffn_combine_torch_adpt.h", - "csrc/torch_binding.cpp", - "csrc/torch_binding_meta.cpp" - ] - }, - { - "commit": "b7f92f37007b1eba5889618d4549c39d2e4b6ab0", - "subject": "[Ops][BugFix] add null check for weight prefetch (#8276)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/ops/fused_moe/experts_selector.py" - ] - }, - { - "commit": "5b3a347a0217b10b535ef6a6915fe44613e4e5c5", - "subject": "[BugFix][DSv32] Fix DSA-CP PD role gating for deepseek v3.2 (#8290)", - "topics": [ - "communication" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/test_utils.py", - "vllm_ascend/utils.py" - ] - }, - { - "commit": "a83759948ddadf2b2e9915cb34ce05edcf06ddab", - "subject": "[BugFix] Fix quant_bias missing in w8a8_static when flashcomm1 is enabled for GLM-5 (#8220)", - "topics": [ - "communication", - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/quantization/method_adapters.py" - ] - }, - { - "commit": "36b1e0406e1eecfb54d1b87504019eec4142b2e2", - "subject": "[Feature] add DFlash Support (#8118)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/attention/attention_v1.py", - "vllm_ascend/ops/triton/spec_decode/utils.py", - "vllm_ascend/patch/__init__.py", - "vllm_ascend/patch/worker/__init__.py", - "vllm_ascend/patch/worker/patch_qwen3_dflash.py", - "vllm_ascend/spec_decode/__init__.py", - "vllm_ascend/spec_decode/dflash_proposer.py", - "vllm_ascend/spec_decode/eagle_proposer.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "b414cff937ff1c8f18a0f08fdcbd063fbd96bf79", - "subject": "[Doc][Misc] Improve readability and fix typos in documentation (#8266)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "AGENTS.md", - "CLAUDE.md", - "README.md", - "README.zh.md", - "benchmarks/README.md", - "docs/source/community/versioning_policy.md", - "docs/source/developer_guide/Design_Documents/ACL_Graph.md", - "docs/source/developer_guide/Design_Documents/KV_Cache_Pool_Guide.md", - "docs/source/developer_guide/Design_Documents/disaggregated_prefill.md", - "docs/source/developer_guide/Design_Documents/eplb_swift_balancer.md", - "docs/source/developer_guide/Design_Documents/npugraph_ex.md", - "docs/source/developer_guide/Design_Documents/patch.md", - "docs/source/developer_guide/Design_Documents/quantization.md", - "docs/source/developer_guide/contribution/e2e_ci_test.md", - "docs/source/developer_guide/contribution/index.md", - "docs/source/developer_guide/contribution/multi_node_test.md", - "docs/source/developer_guide/contribution/nightly_ci_test.md", - "docs/source/developer_guide/evaluation/using_ais_bench.md", - "docs/source/developer_guide/evaluation/using_evalscope.md", - "docs/source/developer_guide/evaluation/using_lm_eval.md", - "docs/source/developer_guide/evaluation/using_opencompass.md", - "docs/source/developer_guide/performance_and_debug/service_profiling_guide.md", - "docs/source/faqs.md", - "docs/source/installation.md", - "docs/source/tutorials/features/long_sequence_context_parallel_multi_node.md", - "docs/source/tutorials/features/pd_disaggregation_mooncake_multi_node.md", - "docs/source/tutorials/features/pd_disaggregation_mooncake_single_node.md", - "docs/source/tutorials/models/DeepSeek-V3.1.md", - "docs/source/tutorials/models/DeepSeekOCR2.md", - "docs/source/tutorials/models/GLM4.x.md", - "docs/source/tutorials/models/GLM5.md", - "docs/source/tutorials/models/MiniMax-M2.5.md", - "docs/source/tutorials/models/Qwen3-Next.md", - "docs/source/tutorials/models/Qwen3-Omni-30B-A3B-Thinking.md", - "docs/source/tutorials/models/Qwen3-VL-Reranker.md", - "docs/source/tutorials/models/Qwen3.5-397B-A17B.md", - "docs/source/tutorials/models/Qwen3_embedding.md", - "docs/source/tutorials/models/Qwen3_reranker.md", - "docs/source/user_guide/configuration/additional_config.md", - "docs/source/user_guide/feature_guide/batch_invariance.md", - "docs/source/user_guide/feature_guide/cpu_binding.md", - "docs/source/user_guide/feature_guide/dynamic_batch.md", - "docs/source/user_guide/feature_guide/eplb_swift_balancer.md", - "docs/source/user_guide/feature_guide/kv_pool.md", - "docs/source/user_guide/feature_guide/layer_sharding.md", - "docs/source/user_guide/feature_guide/npugraph_ex.md", - "docs/source/user_guide/feature_guide/sleep_mode.md", - "docs/source/user_guide/feature_guide/weight_prefetch.md", - "docs/source/user_guide/support_matrix/supported_models.md", - "examples/disaggregated_prefill_v1/mooncake_connector_deployment_guide.md", - "examples/epd_disaggregated/epd_disaggregated_guide.md", - "examples/external_online_dp/README.md", - "tests/e2e/models/report_template.md" - ] - }, - { - "commit": "73aac4818165dac8b2bdd49bb5b3b121e80ea895", - "subject": "[Test][Misc] Update CI for GLM-5 configuration (#8226)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "tests/e2e/nightly/single_node/models/configs/GLM-5.yaml" - ] - }, - { - "commit": "9e87895a8b0d8052b301309395a1afc083faf9a3", - "subject": "[Feature][Kernel add] Fuse W4A8 dispatch + FFN + combine into a single fused kernel (#7779)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".gitmodules", - "csrc/build_aclnn.sh", - "csrc/dispatch_ffn_combine/dispatch_ffn_combine_torch_adpt.h", - "csrc/dispatch_ffn_combine/op_host/CMakeLists.txt", - "csrc/dispatch_ffn_combine/op_host/dispatch_ffn_combine_proto.cpp", - "csrc/dispatch_ffn_combine/op_kernel/dispatch_ffn_combine_kernel.hpp", - "csrc/dispatch_ffn_combine/op_kernel/moe_init_routing_quant_v2/moe_init_routing_quant_v2_tiling.h", - "csrc/dispatch_ffn_combine/op_kernel/moe_init_routing_quant_v2/moe_init_routing_v2_tiling.h", - "csrc/dispatch_ffn_combine/op_kernel/moe_init_routing_quant_v2/tiling_base.h", - "csrc/dispatch_ffn_combine/op_kernel/utils/block_mmad_preload_async_fixpipe_quant.hpp", - "csrc/dispatch_ffn_combine/op_kernel/utils/copy_l0c_to_gm_custom.hpp", - "csrc/dispatch_ffn_combine_bf16/op_host/CMakeLists.txt", - "csrc/dispatch_ffn_combine_bf16/op_host/dispatch_ffn_combine_bf16_proto.cpp", - "csrc/dispatch_ffn_combine_bf16/op_kernel/dispatch_ffn_combine_bf16.cpp", - "csrc/dispatch_ffn_combine_bf16/op_kernel/dispatch_ffn_combine_bf16_kernel.hpp", - "csrc/dispatch_ffn_combine_bf16/op_kernel/utils/block_mmad_preload_async_fixpipe_quant.hpp", - "csrc/dispatch_ffn_combine_bf16/op_kernel/utils/copy_l0c_to_gm_custom.hpp", - "csrc/dispatch_ffn_combine_w4_a8/op_host/CMakeLists.txt", - "csrc/dispatch_ffn_combine_w4_a8/op_host/aclnn_dispatch_ffn_combine_w4_a8.cpp", - "csrc/dispatch_ffn_combine_w4_a8/op_host/aclnn_dispatch_ffn_combine_w4_a8.h", - "csrc/dispatch_ffn_combine_w4_a8/op_host/dispatch_ffn_combine_w4_a8_def.cpp", - "csrc/dispatch_ffn_combine_w4_a8/op_host/dispatch_ffn_combine_w4_a8_proto.cpp", - "csrc/dispatch_ffn_combine_w4_a8/op_host/dispatch_ffn_combine_w4_a8_tiling.cpp", - "csrc/dispatch_ffn_combine_w4_a8/op_host/error_log.h", - "csrc/dispatch_ffn_combine_w4_a8/op_host/hcom_topo_info.h", - "csrc/dispatch_ffn_combine_w4_a8/op_host/tiling_args.h", - "csrc/dispatch_ffn_combine_w4_a8/op_kernel/dispatch_ffn_combine_w4_a8.cpp", - "csrc/dispatch_ffn_combine_w4_a8/op_kernel/dispatch_ffn_combine_w4_a8.h", - "csrc/dispatch_ffn_combine_w4_a8/op_kernel/dispatch_ffn_combine_w4_a8_kernel.hpp", - "csrc/dispatch_ffn_combine_w4_a8/op_kernel/dispatch_ffn_combine_w4_a8_tiling.h", - "csrc/dispatch_ffn_combine_w4_a8/op_kernel/moe_init_routing_quant_v2/moe_init_routing_quant_v2.cpp", - "csrc/dispatch_ffn_combine_w4_a8/op_kernel/moe_init_routing_quant_v2/moe_init_routing_quant_v2_tiling.h", - "csrc/dispatch_ffn_combine_w4_a8/op_kernel/moe_init_routing_quant_v2/moe_init_routing_v2_tiling.h", - "csrc/dispatch_ffn_combine_w4_a8/op_kernel/moe_init_routing_quant_v2/moe_v2_common.h", - "csrc/dispatch_ffn_combine_w4_a8/op_kernel/moe_init_routing_quant_v2/moe_v2_expert_token_out.h", - "csrc/dispatch_ffn_combine_w4_a8/op_kernel/moe_init_routing_quant_v2/moe_v2_fullload_dynamic_quant.h", - "csrc/dispatch_ffn_combine_w4_a8/op_kernel/moe_init_routing_quant_v2/moe_v2_fullload_quant.h", - "csrc/dispatch_ffn_combine_w4_a8/op_kernel/moe_init_routing_quant_v2/moe_v2_fullload_quant_base.h", - "csrc/dispatch_ffn_combine_w4_a8/op_kernel/moe_init_routing_quant_v2/moe_v2_gather_dynamic_quant.h", - "csrc/dispatch_ffn_combine_w4_a8/op_kernel/moe_init_routing_quant_v2/moe_v2_gather_out.h", - "csrc/dispatch_ffn_combine_w4_a8/op_kernel/moe_init_routing_quant_v2/moe_v2_gather_quant.h", - "csrc/dispatch_ffn_combine_w4_a8/op_kernel/moe_init_routing_quant_v2/moe_v2_init_routing_fullload.h", - "csrc/dispatch_ffn_combine_w4_a8/op_kernel/moe_init_routing_quant_v2/moe_v2_mrgsort.h", - "csrc/dispatch_ffn_combine_w4_a8/op_kernel/moe_init_routing_quant_v2/moe_v2_mrgsort_out.h", - "csrc/dispatch_ffn_combine_w4_a8/op_kernel/moe_init_routing_quant_v2/moe_v2_sort_base.h", - "csrc/dispatch_ffn_combine_w4_a8/op_kernel/moe_init_routing_quant_v2/moe_v2_sort_multi_core.h", - "csrc/dispatch_ffn_combine_w4_a8/op_kernel/moe_init_routing_quant_v2/moe_v2_sort_one_core.h", - "csrc/dispatch_ffn_combine_w4_a8/op_kernel/moe_init_routing_quant_v2/moe_v2_src_to_dst_and_gather.h", - "csrc/dispatch_ffn_combine_w4_a8/op_kernel/moe_init_routing_quant_v2/moe_v2_src_to_dst_op.h", - "csrc/dispatch_ffn_combine_w4_a8/op_kernel/moe_init_routing_quant_v2/moe_v2_src_to_dst_with_capacity.h", - "csrc/dispatch_ffn_combine_w4_a8/op_kernel/moe_init_routing_quant_v2/tiling_base.h", - "csrc/dispatch_ffn_combine_w4_a8/op_kernel/unpermute/moe_token_unpermute.h", - "csrc/dispatch_ffn_combine_w4_a8/op_kernel/unpermute/moe_token_unpermute_tiling.h", - "csrc/dispatch_ffn_combine_w4_a8/op_kernel/utils/block_epilogue_w4a8post_pertoken_swiglu.hpp", - "csrc/dispatch_ffn_combine_w4_a8/op_kernel/utils/block_epilogue_w4a8post_pertoken_v2.hpp", - "csrc/dispatch_ffn_combine_w4_a8/op_kernel/utils/block_mmad_w4a4.hpp", - "csrc/dispatch_ffn_combine_w4_a8/op_kernel/utils/const_args.hpp", - "csrc/dispatch_ffn_combine_w4_a8/op_kernel/utils/copy_gm_to_l1_custom.hpp", - "csrc/dispatch_ffn_combine_w4_a8/op_kernel/utils/dispatch_policy_custom.hpp", - "csrc/dispatch_ffn_combine_w4_a8/op_kernel/utils/get_tensor_addr.hpp", - "csrc/dispatch_ffn_combine_w4_a8/op_kernel/utils/hccl_shmem.hpp", - "csrc/dispatch_ffn_combine_w4_a8/op_kernel/utils/layout3d.hpp", - "csrc/dispatch_ffn_combine_w4_a8/op_kernel/utils/moe_distribute_base.h", - "csrc/dispatch_ffn_combine_w4_a8/op_kernel/utils/select_helper.hpp", - "csrc/dispatch_gmm_combine_decode/op_host/CMakeLists.txt", - "csrc/matmul_allreduce_add_rmsnorm/op_kernel/matmul_allreduce_add_rmsnorm_aic_kernel.h", - "csrc/third_party/catlass", - "csrc/torch_binding.cpp", - "csrc/torch_binding_meta.cpp", - "tests/e2e/nightly/single_node/ops/multicard_ops_a3/test_dispatch_ffn_combine.py", - "tests/e2e/nightly/single_node/ops/multicard_ops_a3/test_dispatch_ffn_combine_bf16.py", - "tests/e2e/nightly/single_node/ops/multicard_ops_a3/test_dispatch_ffn_combine_w4a8.py", - "vllm_ascend/ops/fused_moe/moe_comm_method.py" - ] - }, - { - "commit": "2eabbfd087c30d2c375138c4731e3e3fd171326c", - "subject": "[MRv2][Feature] Add initial MoE models support for MRv2 (#7922)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/e2e/multicard/2-cards/test_qwen3_moe.py", - "tests/ut/test_platform.py", - "tests/ut/worker/test_model_runner_v2.py", - "vllm_ascend/ascend_forward_context.py", - "vllm_ascend/platform.py", - "vllm_ascend/worker/v2/model_runner.py" - ] - }, - { - "commit": "cc862a15d95254816d4fdd083609522b2c07d8a9", - "subject": "[BugFix][310p] Fixed the patch_gdn_attn failure caused by Triton (#8144)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/pr_test_light.yaml", - "vllm_ascend/_310p/fused_moe/fused_moe.py", - "vllm_ascend/patch/worker/__init__.py" - ] - }, - { - "commit": "3736ad849c08a076cae8f2b0c54d2e98096d8801", - "subject": "[Misc] Upgrade vllm version to 0408 (#8060)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/workflows/dockerfiles/Dockerfile.lint", - ".github/workflows/pr_test_full.yaml", - ".github/workflows/pr_test_light.yaml", - "Dockerfile", - "Dockerfile.310p", - "Dockerfile.310p.openEuler", - "Dockerfile.a3", - "Dockerfile.a3.openEuler", - "Dockerfile.openEuler", - "tests/ut/patch/worker/patch_common/test_patch_gdn_attn.py", - "vllm_ascend/_310p/fused_moe/fused_moe.py", - "vllm_ascend/ops/fused_moe/fused_moe.py", - "vllm_ascend/patch/worker/patch_qwen3vl.py", - "vllm_ascend/worker/npu_input_batch.py", - "vllm_ascend/worker/v2/README.md" - ] - }, - { - "commit": "28e29387feeeb5dc17bd6d61f23002243e301c4d", - "subject": "[Doc] Document Translation (#8123)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/locale/zh_CN/LC_MESSAGES/community/contributors.po", - "docs/source/locale/zh_CN/LC_MESSAGES/community/governance.po", - "docs/source/locale/zh_CN/LC_MESSAGES/community/user_stories/index.po", - "docs/source/locale/zh_CN/LC_MESSAGES/community/user_stories/llamafactory.po", - "docs/source/locale/zh_CN/LC_MESSAGES/community/versioning_policy.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/Design_Documents/ACL_Graph.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/Design_Documents/KV_Cache_Pool_Guide.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/Design_Documents/ModelRunner_prepare_inputs.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/Design_Documents/add_custom_aclnn_op.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/Design_Documents/context_parallel.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/Design_Documents/cpu_binding.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/Design_Documents/disaggregated_prefill.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/Design_Documents/eplb_swift_balancer.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/Design_Documents/index.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/Design_Documents/npugraph_ex.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/Design_Documents/patch.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/Design_Documents/quantization.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/contribution/doc_writing.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/contribution/e2e_ci_test.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/contribution/index.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/contribution/multi_node_test.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/contribution/nightly_ci_test.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/contribution/testing.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/evaluation/accuracy_report/index.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/evaluation/index.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/evaluation/using_ais_bench.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/evaluation/using_evalscope.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/evaluation/using_lm_eval.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/evaluation/using_opencompass.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/modeling/adding_a_new_model.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/modeling/adding_a_new_multimodal_model.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/modeling/index.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/performance_and_debug/index.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/performance_and_debug/msprobe_guide.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/performance_and_debug/optimization_and_tuning.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/performance_and_debug/performance_benchmark.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/performance_and_debug/profile_execute_duration.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/performance_and_debug/service_profiling_guide.po", - "docs/source/locale/zh_CN/LC_MESSAGES/faqs.po", - "docs/source/locale/zh_CN/LC_MESSAGES/index.po", - "docs/source/locale/zh_CN/LC_MESSAGES/installation.po", - "docs/source/locale/zh_CN/LC_MESSAGES/quick_start.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/features/index.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/features/long_sequence_context_parallel_multi_node.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/features/long_sequence_context_parallel_single_node.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/features/pd_colocated_mooncake_multi_instance.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/features/pd_disaggregation_mooncake_multi_node.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/features/pd_disaggregation_mooncake_single_node.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/features/ray.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/features/suffix_speculative_decoding.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/hardwares/310p.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/hardwares/index.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/DeepSeek-R1.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/DeepSeek-V3.1.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/DeepSeek-V3.2.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/DeepSeekOCR2.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/GLM4.x.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/GLM5.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Kimi-K2-Thinking.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Kimi-K2.5.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/MiniMax-M2.5.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/PaddleOCR-VL.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen-VL-Dense.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen2.5-7B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen2.5-Omni.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-235B-A22B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-30B-A3B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-32B-W4A4.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-8B-W4A8.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-Coder-30B-A3B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-Dense.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-Next.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-Omni-30B-A3B-Thinking.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-VL-235B-A22B-Instruct.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-VL-30B-A3B-Instruct.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-VL-Embedding.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3-VL-Reranker.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3.5-27B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3.5-397B-A17B.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3_embedding.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/Qwen3_reranker.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/models/index.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/multi_node.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/multi_npu.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/multi_npu_moge.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/multi_npu_quantization.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/multi_npu_qwen3_moe.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/single_node_300i.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/single_npu.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/single_npu_audio.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/single_npu_multimodal.po", - "docs/source/locale/zh_CN/LC_MESSAGES/tutorials/single_npu_qwen3_embedding.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/configuration/additional_config.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/configuration/env_vars.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/configuration/index.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/deployment_guide/index.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/deployment_guide/using_volcano_kthena.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/Fine_grained_TP.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/Multi_Token_Prediction.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/batch_invariance.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/context_parallel.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/cpu_binding.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/dynamic_batch.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/epd_disaggregation.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/eplb_swift_balancer.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/external_dp.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/graph_mode.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/index.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/kv_pool.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/large_scale_ep.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/layer_sharding.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/lmcache_ascend_deployment.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/lora.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/netloader.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/npugraph_ex.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/quantization.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/rfork.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/sequence_parallelism.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/sleep_mode.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/speculative_decoding.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/structured_output.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/ucm_deployment.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/feature_guide/weight_prefetch.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/release_notes.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/support_matrix/index.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/support_matrix/supported_features.po", - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/support_matrix/supported_models.po" - ] - }, - { - "commit": "1fc91ad27fe89e04631dd5e5fce00e631aa627bc", - "subject": "[Refactor] Replace npu_ring_mla with FIA in mla_cp prefill (#6996)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/e2e/multicard/4-cards/long_sequence/test_accuracy.py", - "tests/ut/attention/test_attention_cp.py", - "tests/ut/attention/test_common_cp.py", - "tests/ut/attention/test_mla_cp.py", - "vllm_ascend/attention/context_parallel/attention_cp.py", - "vllm_ascend/attention/context_parallel/common_cp.py", - "vllm_ascend/attention/context_parallel/mla_cp.py", - "vllm_ascend/attention/utils.py", - "vllm_ascend/worker/pcp_utils.py" - ] - }, - { - "commit": "ffe196262ea28e572d9e89ef705ad325d5a447f6", - "subject": "[BugFix] Restore global_bs=0 and mc2_mask for uniform-token dispatching and support inter-node roce hierarchical MC2 communication (#8039)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/ops/fused_moe/moe_stage_contracts.py", - "vllm_ascend/ops/fused_moe/token_dispatcher.py", - "vllm_ascend/utils.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "8c9d10267a575e4915401ba1eb723328edb3953e", - "subject": "[Performance]zero bubble async scheduling and spec decoding (#7640)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/workflows/_unit_test.yaml", - "tests/ut/worker/test_block_table.py", - "vllm_ascend/attention/attention_v1.py", - "vllm_ascend/attention/mla_v1.py", - "vllm_ascend/attention/sfa_v1.py", - "vllm_ascend/attention/utils.py", - "vllm_ascend/spec_decode/eagle_proposer.py", - "vllm_ascend/spec_decode/utils.py", - "vllm_ascend/worker/block_table.py", - "vllm_ascend/worker/model_runner_v1.py", - "vllm_ascend/worker/npu_input_batch.py" - ] - }, - { - "commit": "7a7a569d60e25ce7a3e15274d336268dadc87859", - "subject": "[Doc][Misc] Comprehensive documentation cleanup and grammatical fixes (#8056)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/community/governance.md", - "docs/source/community/user_stories/llamafactory.md", - "docs/source/community/versioning_policy.md", - "docs/source/developer_guide/Design_Documents/ACL_Graph.md", - "docs/source/developer_guide/Design_Documents/ModelRunner_prepare_inputs.md", - "docs/source/developer_guide/Design_Documents/add_custom_aclnn_op.md", - "docs/source/developer_guide/Design_Documents/patch.md", - "docs/source/developer_guide/Design_Documents/quantization.md", - "docs/source/developer_guide/contribution/index.md", - "docs/source/developer_guide/contribution/multi_node_test.md", - "docs/source/developer_guide/contribution/nightly_ci_test.md", - "docs/source/developer_guide/contribution/testing.md", - "docs/source/developer_guide/evaluation/using_evalscope.md", - "docs/source/developer_guide/evaluation/using_opencompass.md", - "docs/source/developer_guide/performance_and_debug/performance_benchmark.md", - "docs/source/tutorials/features/long_sequence_context_parallel_multi_node.md", - "docs/source/tutorials/features/ray.md", - "docs/source/tutorials/features/suffix_speculative_decoding.md", - "docs/source/tutorials/hardwares/310p.md", - "docs/source/tutorials/models/DeepSeek-R1.md", - "docs/source/tutorials/models/DeepSeek-V3.1.md", - "docs/source/tutorials/models/GLM4.x.md", - "docs/source/tutorials/models/GLM5.md", - "docs/source/tutorials/models/Kimi-K2-Thinking.md", - "docs/source/tutorials/models/Kimi-K2.5.md", - "docs/source/tutorials/models/Qwen-VL-Dense.md", - "docs/source/tutorials/models/Qwen2.5-7B.md", - "docs/source/tutorials/models/Qwen2.5-Omni.md", - "docs/source/tutorials/models/Qwen3-235B-A22B.md", - "docs/source/tutorials/models/Qwen3-Dense.md", - "docs/source/tutorials/models/Qwen3-Omni-30B-A3B-Thinking.md", - "docs/source/tutorials/models/Qwen3-VL-235B-A22B-Instruct.md", - "docs/source/tutorials/models/Qwen3.5-27B.md", - "docs/source/user_guide/feature_guide/Fine_grained_TP.md", - "docs/source/user_guide/feature_guide/batch_invariance.md", - "docs/source/user_guide/feature_guide/dynamic_batch.md", - "docs/source/user_guide/feature_guide/epd_disaggregation.md", - "docs/source/user_guide/feature_guide/eplb_swift_balancer.md", - "docs/source/user_guide/feature_guide/sequence_parallelism.md", - "docs/source/user_guide/feature_guide/weight_prefetch.md", - "docs/source/user_guide/release_notes.md", - "docs/source/user_guide/support_matrix/supported_features.md" - ] - }, - { - "commit": "4140e84ef3e14967cc67f0347a0ce1693b09dfc7", - "subject": "[CI] Main2main upgrade vllm to 0330 (#7962)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/workflows/bot_pr_create.yaml", - ".github/workflows/dockerfiles/Dockerfile.lint", - ".github/workflows/pr_test_full.yaml", - ".github/workflows/pr_test_light.yaml", - ".github/workflows/schedule_codecov_refresh.yaml", - "Dockerfile", - "Dockerfile.310p", - "Dockerfile.310p.openEuler", - "Dockerfile.a3", - "Dockerfile.a3.openEuler", - "Dockerfile.openEuler", - "tests/e2e/singlecard/test_cpu_offloading.py", - "vllm_ascend/ops/gdn.py", - "vllm_ascend/patch/worker/__init__.py", - "vllm_ascend/patch/worker/patch_qwen3_5.py", - "vllm_ascend/patch/worker/patch_qwen3_next.py", - "vllm_ascend/quantization/modelslim_config.py", - "vllm_ascend/spec_decode/draft_proposer.py", - "vllm_ascend/utils.py", - "vllm_ascend/worker/block_table.py" - ] - }, - { - "commit": "dbf9f7a3a910722e4ec0180a2fc2fbc578b09bc4", - "subject": "[BugFix][310p] Align 310P GDN state semantics with vLLM (#7902)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_chunk_gated_delta_rule.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_fused_recurrent_gated_delta_rule.py", - "tests/ut/_310p/ops/test_chunk_gated_delta_rule_310.py", - "vllm_ascend/_310p/ops/fla/chunk_gated_delta_rule.py", - "vllm_ascend/_310p/ops/fla/fused_recurrent_gated_delta_rule.py" - ] - }, - { - "commit": "0d768aa9434e947fc6e15a5721c90877be325c76", - "subject": "[Ascend950][quant][Feature] Add W4A4 MXFP4 quantization support for Ascend950 (#7877)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/ops/test_moe_mlp.py", - "tests/ut/ops/test_moe_runtime_args.py", - "tests/ut/ops/test_token_dispatcher.py", - "vllm_ascend/device/device_op.py", - "vllm_ascend/device/mxfp_compat.py", - "vllm_ascend/ops/fused_moe/moe_mlp.py", - "vllm_ascend/ops/fused_moe/moe_runtime_args.py", - "vllm_ascend/ops/fused_moe/moe_stage_params.py", - "vllm_ascend/ops/fused_moe/prepare_finalize.py", - "vllm_ascend/ops/fused_moe/token_dispatcher.py", - "vllm_ascend/quantization/methods/__init__.py", - "vllm_ascend/quantization/methods/w4a4_mxfp4.py", - "vllm_ascend/quantization/quant_type.py" - ] - }, - { - "commit": "d39031b6cf48a84786126f6c1690b87e1db0c55e", - "subject": "[Doc][Misc] Add GLM5 to supported model list and update deployment document for GLM5 (#7958)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/tutorials/models/GLM5.md", - "docs/source/user_guide/support_matrix/supported_models.md" - ] - }, - { - "commit": "1729a6e63ff2fc3d5257cb6408b08bdc52cf12b9", - "subject": "[Feature] support qkv_rmsnorm_mrope for qwen3vl (#7762)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/patch/__init__.py", - "vllm_ascend/patch/worker/__init__.py", - "vllm_ascend/patch/worker/patch_qwen3vl.py" - ] - }, - { - "commit": "37e672a1bc63565441b782b96e0b94782d4de527", - "subject": "[Doc][Misc] Add Parameter Description, best practices and FAQs in GLM5.md (#7850)", - "topics": [ - "communication", - "priority-models", - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/tutorials/models/GLM5.md" - ] - }, - { - "commit": "8e5b20b859a536ff07a6f824dd53bef8a58d74cb", - "subject": "[BugFix] fix the weightsmapper bug of qwen3-vl (#7869)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/quantization/modelslim_config.py" - ] - }, - { - "commit": "f3d25db259b55bc277aa6ef9dd4c856b67b7f870", - "subject": "[EPLB][BugFix] Fix moe_load precision in allgather (#7887)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/eplb/test_eplb_updator.py", - "vllm_ascend/eplb/eplb_updator.py", - "vllm_ascend/ops/fused_moe/fused_moe.py" - ] - }, - { - "commit": "f72a1277933b40ccf86066200a50b80864ba1bb5", - "subject": "[Doc] link updates and formatting fixes (#7734)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/community/user_stories/index.md", - "docs/source/developer_guide/Design_Documents/ACL_Graph.md", - "docs/source/developer_guide/Design_Documents/KV_Cache_Pool_Guide.md", - "docs/source/developer_guide/Design_Documents/ModelRunner_prepare_inputs.md", - "docs/source/developer_guide/Design_Documents/add_custom_aclnn_op.md", - "docs/source/developer_guide/Design_Documents/context_parallel.md", - "docs/source/developer_guide/Design_Documents/cpu_binding.md", - "docs/source/developer_guide/Design_Documents/disaggregated_prefill.md", - "docs/source/developer_guide/Design_Documents/eplb_swift_balancer.md", - "docs/source/developer_guide/Design_Documents/index.md", - "docs/source/developer_guide/Design_Documents/npugraph_ex.md", - "docs/source/developer_guide/Design_Documents/patch.md", - "docs/source/developer_guide/Design_Documents/quantization.md", - "docs/source/developer_guide/contribution/index.md", - "docs/source/developer_guide/contribution/multi_node_test.md", - "docs/source/developer_guide/contribution/testing.md", - "docs/source/developer_guide/evaluation/using_ais_bench.md", - "docs/source/developer_guide/evaluation/using_lm_eval.md", - "docs/source/developer_guide/performance_and_debug/optimization_and_tuning.md", - "docs/source/faqs.md", - "docs/source/index.md", - "docs/source/installation.md", - "docs/source/llms.txt", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/Design_Documents/index.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/Design_Documents/patch.po", - "docs/source/tutorials/features/long_sequence_context_parallel_multi_node.md", - "docs/source/tutorials/features/long_sequence_context_parallel_single_node.md", - "docs/source/tutorials/features/pd_disaggregation_mooncake_multi_node.md", - "docs/source/tutorials/features/pd_disaggregation_mooncake_single_node.md", - "docs/source/tutorials/models/DeepSeek-R1.md", - "docs/source/tutorials/models/DeepSeek-V3.1.md", - "docs/source/tutorials/models/DeepSeek-V3.2.md", - "docs/source/tutorials/models/GLM4.x.md", - "docs/source/tutorials/models/GLM5.md", - "docs/source/tutorials/models/Kimi-K2.5.md", - "docs/source/tutorials/models/PaddleOCR-VL.md", - "docs/source/tutorials/models/Qwen-VL-Dense.md", - "docs/source/tutorials/models/Qwen2.5-7B.md", - "docs/source/tutorials/models/Qwen2.5-Omni.md", - "docs/source/tutorials/models/Qwen3-235B-A22B.md", - "docs/source/tutorials/models/Qwen3-Dense.md", - "docs/source/tutorials/models/Qwen3-Next.md", - "docs/source/tutorials/models/Qwen3-Omni-30B-A3B-Thinking.md", - "docs/source/tutorials/models/Qwen3-VL-235B-A22B-Instruct.md", - "docs/source/tutorials/models/Qwen3-VL-30B-A3B-Instruct.md", - "docs/source/tutorials/models/Qwen3-VL-Reranker.md", - "docs/source/user_guide/feature_guide/context_parallel.md", - "docs/source/user_guide/feature_guide/dynamic_batch.md", - "docs/source/user_guide/feature_guide/epd_disaggregation.md", - "docs/source/user_guide/feature_guide/graph_mode.md", - "docs/source/user_guide/feature_guide/large_scale_ep.md", - "docs/source/user_guide/feature_guide/lmcache_ascend_deployment.md", - "docs/source/user_guide/feature_guide/npugraph_ex.md", - "docs/source/user_guide/feature_guide/speculative_decoding.md", - "docs/source/user_guide/feature_guide/weight_prefetch.md", - "docs/source/user_guide/release_notes.md" - ] - }, - { - "commit": "27924d03e39ed26510a6b0d69f817ca43d5fcfd6", - "subject": "[Feature][310P] support shared experts path in fused MoE for qwen3.5 (#7674)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/e2e/310p/multicard/test_moe_model_multicard.py", - "tests/e2e/310p/singlecard/test_dense_model_singlecard.py", - "tests/ut/_310p/fused_moe/test_shared_fused_moe_310.py", - "vllm_ascend/_310p/fused_moe/fused_moe.py", - "vllm_ascend/patch/worker/patch_qwen3_5_310.py" - ] - }, - { - "commit": "7465131074cddba670efa02cc484eba7d5c9bda7", - "subject": "[Doc][Misc] Update service profiling guide and internal profiling configurations (#7760)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/developer_guide/performance_and_debug/service_profiling_guide.md", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/performance_and_debug/service_profiling_guide.po", - "requirements-dev.txt", - "vllm_ascend/profiling_config.py" - ] - }, - { - "commit": "11e0583474434de15a8968b7fe76a8d5da40f6e7", - "subject": "[Bugfix]support FlashComm1&DCP for Qwen (#7673)", - "topics": [ - "communication" - ], - "runtime_patch_match": false, - "paths": [ - "tests/e2e/multicard/4-cards/long_sequence/test_basic.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "4359ea39c00596638928595f151cf284424c64bc", - "subject": "[v0.18.0][bugfix][eplb] remove unnecessary weight_scale wrap behavior (#7733)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [] - }, - { - "commit": "bd004043da559bf8b37ac66fce187336d76ba4f5", - "subject": "[BugFix][APC] Fix prefix caching support for embedding models (#7452)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/e2e/singlecard/pooling/test_embedding.py", - "vllm_ascend/attention/attention_v1.py" - ] - }, - { - "commit": "2f5c6c74b254928f3e51c776cddfebd73947efa8", - "subject": "Feature: Support Unified Placement for Shared & Router Experts (#7188)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/ascend_config.py", - "vllm_ascend/eplb/core/eplb_utils.py", - "vllm_ascend/ops/fused_moe/experts_selector.py", - "vllm_ascend/ops/fused_moe/fused_moe.py", - "vllm_ascend/quantization/methods/w8a8_dynamic.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "c53c3f58180dcd50fd485926e81532151bb19035", - "subject": "[310p][feat] Added a patch file for 310P to adapt to Qwen3.5 (#7533)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/patch/worker/__init__.py", - "vllm_ascend/patch/worker/patch_qwen3_5_310.py" - ] - }, - { - "commit": "fe7097f3c663af83e39c19358c184913b10c56a3", - "subject": "[kernel] Recompilation optimization triggered by triton function para\u2026 (#7481)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/ops/triton/fla/chunk_delta_h.py", - "vllm_ascend/ops/triton/fla/cumsum.py", - "vllm_ascend/ops/triton/fused_gdn_gating.py", - "vllm_ascend/ops/triton/reject_sample.py", - "vllm_ascend/ops/triton/spec_decode/utils.py" - ] - }, - { - "commit": "28adb8cc849f3fe6c6854d5aee507f2bb6392a03", - "subject": "[feat] support dispatch_v2/combine_v2 hierarchy communication (#7583)", - "topics": [ - "communication", - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "docs/source/user_guide/configuration/additional_config.md", - "vllm_ascend/ascend_config.py", - "vllm_ascend/ops/fused_moe/token_dispatcher.py", - "vllm_ascend/platform.py", - "vllm_ascend/utils.py" - ] - }, - { - "commit": "2db12ed935da1b31bcc7865c2d08c841651b2d38", - "subject": "fix uncompatible between fc1 and non-sp-padding (#7614)", - "topics": [ - "communication" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "2965280bf8f5bb6239f60699795a287b7c142feb", - "subject": "feat(quant): add C8 INT8 KV cache support for GQA attention models (#7474)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/quantization/test_kv_c8.py", - "tests/ut/quantization/test_modelslim_config.py", - "vllm_ascend/attention/attention_v1.py", - "vllm_ascend/patch/__init__.py", - "vllm_ascend/patch/worker/__init__.py", - "vllm_ascend/patch/worker/patch_qwen3_c8.py", - "vllm_ascend/quantization/methods/kv_c8.py", - "vllm_ascend/quantization/modelslim_config.py" - ] - }, - { - "commit": "d452d04656eabc0bf719e1d07093973274eebd6e", - "subject": "[A5][bugfix] Fix fused MoE A5 MXFP8 scale normalization, load-balance routing and gating_topk ops (#7573)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/device/device_op.py", - "vllm_ascend/ops/fused_moe/experts_selector.py", - "vllm_ascend/ops/fused_moe/moe_mlp.py", - "vllm_ascend/quantization/methods/w8a8_mxfp8.py" - ] - }, - { - "commit": "3f4087a8f06b43457bf5980647cb929850c7e1ea", - "subject": "[310P]fused recurrent gated delta rule pytorch core and ut (#7398)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_fused_recurrent_gated_delta_rule.py", - "vllm_ascend/_310p/ops/fla/__init__.py", - "vllm_ascend/_310p/ops/fla/fused_recurrent_gated_delta_rule.py" - ] - }, - { - "commit": "5d12446573020d569e110fd62bb060417f21b136", - "subject": "[Feat][SP] Suport SP for VL MoE models (#7044)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/workflows/scripts/config.yaml", - "docs/source/assets/sp_moe.png", - "docs/source/user_guide/configuration/additional_config.md", - "docs/source/user_guide/feature_guide/sequence_parallelism.md", - "tests/e2e/multicard/2-cards/test_sequence_parallelism_moe.py", - "tests/e2e/multicard/2-cards/test_sp_pass.py", - "tests/e2e/singlecard/compile/backend.py", - "tests/ut/ops/test_prepare_finalize.py", - "vllm_ascend/ascend_config.py", - "vllm_ascend/compilation/graph_fusion_pass_manager.py", - "vllm_ascend/compilation/passes/allgather_chunk_noop_pass.py", - "vllm_ascend/compilation/passes/noop_elimination.py", - "vllm_ascend/compilation/passes/sequence_parallelism.py", - "vllm_ascend/compilation/passes/sequence_parallelism_moe.py", - "vllm_ascend/ops/fused_moe/prepare_finalize.py", - "vllm_ascend/ops/register_custom_ops.py", - "vllm_ascend/patch/__init__.py", - "vllm_ascend/patch/worker/patch_distributed.py", - "vllm_ascend/platform.py", - "vllm_ascend/utils.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "67aad1fce82265cd62a397e6740b4805e9910392", - "subject": "[BugFix][P/D] fix padding error on FullGraph mode && fix layerwise connector mamba accuracy (#7506)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/distributed/kv_transfer/kv_p2p/mooncake_layerwise_connector.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "475b4b0cead3d2126ad3e0063af662533262cc5c", - "subject": "Revert \"GMM custom operator optimization in small batch scenarios (vllm-project#7100)\" (#7557)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "csrc/torch_binding.cpp", - "vllm_ascend/device/device_op.py", - "vllm_ascend/ops/fused_moe/moe_mlp.py" - ] - }, - { - "commit": "83bd77c9833110bca8b7e37e63e92de480200059", - "subject": "[310p]: add rmsnorm gated fallback and unit test (#7424)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/_310p/ops/test_layernorm_310.py", - "vllm_ascend/_310p/ops/layernorm.py", - "vllm_ascend/utils.py" - ] - }, - { - "commit": "e942b62d742ebc5bf128e85bc086d728df8d4935", - "subject": "[features]support split qkv rmsnorm rmope for qwen3.5 (#7368)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/patch/worker/patch_qwen3_5.py" - ] - }, - { - "commit": "fcba91a39218345b439122dea79d8fe4432d24c8", - "subject": "Main2main Upgrade vllm commit to 0320 17:00 (#7510)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/bot_pr_create.yaml", - ".github/workflows/dockerfiles/Dockerfile.lint", - ".github/workflows/pr_test_full.yaml", - ".github/workflows/pr_test_light.yaml", - ".github/workflows/schedule_codecov_refresh.yaml", - "docs/source/community/versioning_policy.md", - "vllm_ascend/ascend_config.py", - "vllm_ascend/ops/fused_moe/fused_moe.py" - ] - }, - { - "commit": "13397e9cb75721f7365cfc4c22738320aac21d54", - "subject": "[310p] Add a PyTorch implementation of the GDN gating operator on 310P (#7430)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_fused_gdn_gating.py", - "vllm_ascend/_310p/ops/fla/__init__.py", - "vllm_ascend/_310p/ops/fla/fused_gdn_gating.py" - ] - }, - { - "commit": "fb283b5820effe930d7f60952aca48177d710e94", - "subject": "[CI] Add nightly CI test cases for the GLM-5 (#7429)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/misc/model_list.json", - ".github/workflows/schedule_nightly_test_a3.yaml", - "tests/e2e/nightly/single_node/models/configs/GLM-5.yaml", - "tests/e2e/nightly/single_node/models/scripts/single_node_config.py", - "tests/e2e/nightly/single_node/models/scripts/test_single_node.py" - ] - }, - { - "commit": "9e2965bae2403d1b9a18fdcd03d94704e1f9f01c", - "subject": "[Feature] Support Flash Comm V1 for VL models (with MLA) (#7390)", - "topics": [ - "communication" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/ops/layernorm.py", - "vllm_ascend/ops/mla.py", - "vllm_ascend/platform.py", - "vllm_ascend/utils.py" - ] - }, - { - "commit": "88d03a783fb28de1498d4c23d94e7fc173855963", - "subject": "[refactor] replace scattered business kwargs with typed request objects and explicit stage boundaries (#7024)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/e2e/multicard/2-cards/test_offline_inference_distributed.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_fused_moe.py", - "tests/ut/_310p/fused_moe/test_moe_mlp_310.py", - "tests/ut/ops/test_activation.py", - "tests/ut/ops/test_fused_moe.py", - "tests/ut/ops/test_moe_comm_method.py", - "tests/ut/ops/test_moe_mlp.py", - "tests/ut/ops/test_moe_runtime_args.py", - "tests/ut/ops/test_prepare_finalize.py", - "tests/ut/ops/test_token_dispatcher.py", - "tests/ut/quantization/test_w4a16.py", - "tests/ut/quantization/test_w8a8_dynamic.py", - "vllm_ascend/_310p/fused_moe/fused_moe.py", - "vllm_ascend/_310p/fused_moe/moe_comm_method.py", - "vllm_ascend/_310p/fused_moe/moe_mlp.py", - "vllm_ascend/_310p/fused_moe/token_dispatcher.py", - "vllm_ascend/_310p/quantization/methods/w8a8_dynamic.py", - "vllm_ascend/ops/fused_moe/fused_moe.py", - "vllm_ascend/ops/fused_moe/moe_comm_method.py", - "vllm_ascend/ops/fused_moe/moe_mlp.py", - "vllm_ascend/ops/fused_moe/moe_runtime_args.py", - "vllm_ascend/ops/fused_moe/moe_stage_contracts.py", - "vllm_ascend/ops/fused_moe/moe_stage_params.py", - "vllm_ascend/ops/fused_moe/prepare_finalize.py", - "vllm_ascend/ops/fused_moe/token_dispatcher.py", - "vllm_ascend/quantization/__init__.py", - "vllm_ascend/quantization/method_adapters.py", - "vllm_ascend/quantization/methods/base.py", - "vllm_ascend/quantization/methods/w4a16.py", - "vllm_ascend/quantization/methods/w4a8.py", - "vllm_ascend/quantization/methods/w8a8_dynamic.py", - "vllm_ascend/quantization/methods/w8a8_mxfp8.py", - "vllm_ascend/quantization/quant_type.py" - ] - }, - { - "commit": "c860535246cc751b6be7d1da2092e4380013598c", - "subject": "\u3010A5\u3011\u3010Qwen VL\u3011Qwen VL adapt for A5 (#7046)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/ops/mm_encoder_attention.py", - "vllm_ascend/ops/rotary_embedding.py" - ] - }, - { - "commit": "a1f321a5564367bdf9c334ad3a177b3b5829d902", - "subject": "[Doc]Refresh model tutorial examples and serving commands (#7426)", - "topics": [ - "priority-models", - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/tutorials/models/DeepSeek-V3.1.md", - "docs/source/tutorials/models/DeepSeek-V3.2.md", - "docs/source/tutorials/models/GLM5.md", - "docs/source/tutorials/models/Kimi-K2-Thinking.md", - "docs/source/tutorials/models/MiniMax-M2.5.md", - "docs/source/tutorials/models/PaddleOCR-VL.md", - "docs/source/tutorials/models/Qwen-VL-Dense.md", - "docs/source/tutorials/models/Qwen2.5-7B.md", - "docs/source/tutorials/models/Qwen2.5-Omni.md", - "docs/source/tutorials/models/Qwen3-235B-A22B.md", - "docs/source/tutorials/models/Qwen3-Coder-30B-A3B.md", - "docs/source/tutorials/models/Qwen3-Dense.md", - "docs/source/tutorials/models/Qwen3-Next.md", - "docs/source/tutorials/models/Qwen3-Omni-30B-A3B-Thinking.md", - "docs/source/tutorials/models/Qwen3-VL-Embedding.md", - "docs/source/tutorials/models/Qwen3.5-27B.md", - "docs/source/tutorials/models/Qwen3.5-397B-A17B.md", - "docs/source/tutorials/models/Qwen3_embedding.md", - "docs/source/tutorials/models/Qwen3_reranker.md" - ] - }, - { - "commit": "9d1452c74d174197ddda6c8ae0d58b7fc02dd7a7", - "subject": "[OPS]add split_qkv_tp_rmsnorm_rope ops (#7376)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_split_qkv_tp_rmsnorm_rope.py", - "vllm_ascend/ops/__init__.py", - "vllm_ascend/ops/triton/linearnorm/split_qkv_tp_rmsnorm_rope.py", - "vllm_ascend/patch/worker/patch_minimax_m2.py" - ] - }, - { - "commit": "05afc7f8c3822997f643535ff007096f122ac0cb", - "subject": "[CI]repair for ci custom ops (#7461)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_add_rms_norm_bias.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_apply_top_k_top_p_custom.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_batch_matmul_transpose.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_gating_top_k_softmax.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_moe_init_routing_custom.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_npu_moe_gating_top_k.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_transpose_kv_cache_by_block.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_causal_conv1d.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_chunk_gated_delta_rule.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_fused_qkvzba_split_reshape_cat.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_fused_sigmoid_gating_delta_rule.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_l2norm.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_penality.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_prepare_inputs_padded.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_rejection_sample.py" - ] - }, - { - "commit": "42bcad7e9b68cf92d9b98a9072eff10edc8d6f33", - "subject": "GMM custom operator optimization in small batch scenarios (#7100)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "csrc/torch_binding.cpp", - "vllm_ascend/device/device_op.py", - "vllm_ascend/ops/fused_moe/moe_mlp.py" - ] - }, - { - "commit": "adc57c5951f0500082d421d5b22cfcb713e83836", - "subject": "[release] Add GLM5 known issue for 2-node PD mixed deployment (#7436)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/user_guide/release_notes.md" - ] - }, - { - "commit": "58725b8b247768c88c1d3dc1ed644c273a770656", - "subject": "[doc] add Prefill-Decode Disaggregation doc for GLM5.md (#7300)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/tutorials/models/GLM5.md", - "docs/source/user_guide/release_notes.md", - "tests/e2e/nightly/single_node/models/configs/GLM-5.yaml" - ] - }, - { - "commit": "a370dfa9623e648439b724569931988a852e462e", - "subject": "[bugfix]Enable dispatch_ffn_combine feature for qwen3.5 (#7066)", - "topics": [ - "communication", - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/ascend_forward_context.py", - "vllm_ascend/ops/fused_moe/fused_moe.py" - ] - }, - { - "commit": "4e62a2ae15ccc3b1344a31e8716f4a27da097858", - "subject": "[Bugfix] fix TransposeKvCacheByBlock op error report in plog (#7235)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "csrc/transpose_kv_cache_by_block/op_host/transpose_kv_cache_by_block_def.cpp" - ] - }, - { - "commit": "3f39ac9c8d12e576487bf435af146e3244187507", - "subject": "[Feature]Supports DSv3.1 PD separation and C8 quantization (#7222)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/attention/test_mla_v1.py", - "tests/ut/quantization/test_kv_c8.py", - "tests/ut/quantization/test_modelslim_config.py", - "vllm_ascend/attention/context_parallel/mla_cp.py", - "vllm_ascend/attention/mla_v1.py", - "vllm_ascend/distributed/kv_transfer/kv_p2p/mooncake_layerwise_connector.py", - "vllm_ascend/ops/mla.py", - "vllm_ascend/patch/__init__.py", - "vllm_ascend/patch/worker/__init__.py", - "vllm_ascend/patch/worker/patch_weight_utils.py", - "vllm_ascend/quantization/methods/__init__.py", - "vllm_ascend/quantization/methods/kv_c8.py", - "vllm_ascend/quantization/modelslim_config.py", - "vllm_ascend/utils.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "4d443b922811e1094fba660a8faed0d493541571", - "subject": "[bugfix] restore pr-7029 and fix patch error (#7294)", - "topics": [ - "priority-models", - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/workflows/scripts/config.yaml", - "csrc/build_aclnn.sh", - "csrc/lightning_indexer_quant/lightning_indexer_quant_torch_adpt.h", - "csrc/lightning_indexer_quant/op_host/CMakeLists.txt", - "csrc/lightning_indexer_quant/op_host/lightning_indexer_quant_def.cpp", - "csrc/lightning_indexer_quant/op_host/lightning_indexer_quant_proto.cpp", - "csrc/lightning_indexer_quant/op_host/lightning_indexer_quant_tiling.cpp", - "csrc/lightning_indexer_quant/op_host/lightning_indexer_quant_tiling.h", - "csrc/lightning_indexer_quant/op_kernel/lightning_indexer_quant.cpp", - "csrc/lightning_indexer_quant/op_kernel/lightning_indexer_quant_common.h", - "csrc/lightning_indexer_quant/op_kernel/lightning_indexer_quant_kernel.h", - "csrc/lightning_indexer_quant/op_kernel/lightning_indexer_quant_service_cube.h", - "csrc/lightning_indexer_quant/op_kernel/lightning_indexer_quant_service_vector.h", - "csrc/lightning_indexer_quant/op_kernel/lightning_indexer_quant_template_tiling_key.h", - "csrc/lightning_indexer_quant/op_kernel/lightning_indexer_quant_vector.h", - "csrc/torch_binding.cpp", - "csrc/torch_binding_meta.cpp", - "tests/e2e/multicard/2-cards/test_offline_inference_distributed.py", - "vllm_ascend/ascend_config.py", - "vllm_ascend/attention/sfa_v1.py", - "vllm_ascend/core/recompute_scheduler.py", - "vllm_ascend/patch/__init__.py", - "vllm_ascend/patch/platform/__init__.py", - "vllm_ascend/patch/platform/patch_kv_cache_interface.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "71c21f76f5d7d0674185340cdff86772c78b8508", - "subject": "[Refactor] Replace npu_ring_mla with FIA in MLA prefill (#5704)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/e2e/singlecard/test_aclgraph_accuracy.py", - "tests/ut/attention/test_mla_cp.py", - "tests/ut/attention/test_mla_v1.py", - "vllm_ascend/attention/context_parallel/common_cp.py", - "vllm_ascend/attention/context_parallel/mla_cp.py", - "vllm_ascend/attention/mla_v1.py" - ] - }, - { - "commit": "0c299f79b92af96b36518b6497d4817a84aeb49e", - "subject": "Revert \"[Perf][1/N] w8a8c8 support in dsv3.2/glm5 (#7029)\" (#7288)", - "topics": [ - "priority-models", - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/workflows/scripts/config.yaml", - "csrc/build_aclnn.sh", - "csrc/lightning_indexer_quant/lightning_indexer_quant_torch_adpt.h", - "csrc/lightning_indexer_quant/op_host/CMakeLists.txt", - "csrc/lightning_indexer_quant/op_host/lightning_indexer_quant_def.cpp", - "csrc/lightning_indexer_quant/op_host/lightning_indexer_quant_proto.cpp", - "csrc/lightning_indexer_quant/op_host/lightning_indexer_quant_tiling.cpp", - "csrc/lightning_indexer_quant/op_host/lightning_indexer_quant_tiling.h", - "csrc/lightning_indexer_quant/op_kernel/lightning_indexer_quant.cpp", - "csrc/lightning_indexer_quant/op_kernel/lightning_indexer_quant_common.h", - "csrc/lightning_indexer_quant/op_kernel/lightning_indexer_quant_kernel.h", - "csrc/lightning_indexer_quant/op_kernel/lightning_indexer_quant_service_cube.h", - "csrc/lightning_indexer_quant/op_kernel/lightning_indexer_quant_service_vector.h", - "csrc/lightning_indexer_quant/op_kernel/lightning_indexer_quant_template_tiling_key.h", - "csrc/lightning_indexer_quant/op_kernel/lightning_indexer_quant_vector.h", - "csrc/torch_binding.cpp", - "csrc/torch_binding_meta.cpp", - "tests/e2e/multicard/2-cards/test_offline_inference_distributed.py", - "vllm_ascend/ascend_config.py", - "vllm_ascend/attention/sfa_v1.py", - "vllm_ascend/patch/__init__.py", - "vllm_ascend/patch/platform/__init__.py", - "vllm_ascend/patch/platform/patch_kv_cache_interface.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "986cd453972b320206fd8f4ce92ab8092f31a77f", - "subject": "[Version] Drop 0.16.0 support (#7153)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/workflows/_e2e_nightly_multi_node.yaml", - ".github/workflows/_e2e_nightly_single_node.yaml", - ".github/workflows/pr_test_full.yaml", - ".github/workflows/pr_test_light.yaml", - ".github/workflows/schedule_nightly_test_a2.yaml", - ".github/workflows/schedule_test_benchmarks.yaml", - "Dockerfile", - "Dockerfile.310p", - "Dockerfile.310p.openEuler", - "Dockerfile.a3", - "Dockerfile.a3.openEuler", - "Dockerfile.openEuler", - "docs/source/conf.py", - "tests/ut/spec_decode/test_eagle_proposer.py", - "vllm_ascend/_310p/fused_moe/fused_moe.py", - "vllm_ascend/ops/fused_moe/fused_moe.py", - "vllm_ascend/patch/worker/__init__.py", - "vllm_ascend/patch/worker/patch_v2_eagle.py", - "vllm_ascend/spec_decode/eagle_proposer.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "7ed9e9de69160daa66ab61400358da0ecf85107a", - "subject": "[Perf][1/N] w8a8c8 support in dsv3.2/glm5 (#7029)", - "topics": [ - "priority-models", - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/workflows/scripts/config.yaml", - "csrc/build_aclnn.sh", - "csrc/lightning_indexer_quant/lightning_indexer_quant_torch_adpt.h", - "csrc/lightning_indexer_quant/op_host/CMakeLists.txt", - "csrc/lightning_indexer_quant/op_host/lightning_indexer_quant_def.cpp", - "csrc/lightning_indexer_quant/op_host/lightning_indexer_quant_proto.cpp", - "csrc/lightning_indexer_quant/op_host/lightning_indexer_quant_tiling.cpp", - "csrc/lightning_indexer_quant/op_host/lightning_indexer_quant_tiling.h", - "csrc/lightning_indexer_quant/op_kernel/lightning_indexer_quant.cpp", - "csrc/lightning_indexer_quant/op_kernel/lightning_indexer_quant_common.h", - "csrc/lightning_indexer_quant/op_kernel/lightning_indexer_quant_kernel.h", - "csrc/lightning_indexer_quant/op_kernel/lightning_indexer_quant_service_cube.h", - "csrc/lightning_indexer_quant/op_kernel/lightning_indexer_quant_service_vector.h", - "csrc/lightning_indexer_quant/op_kernel/lightning_indexer_quant_template_tiling_key.h", - "csrc/lightning_indexer_quant/op_kernel/lightning_indexer_quant_vector.h", - "csrc/torch_binding.cpp", - "csrc/torch_binding_meta.cpp", - "tests/e2e/multicard/2-cards/test_offline_inference_distributed.py", - "vllm_ascend/ascend_config.py", - "vllm_ascend/attention/sfa_v1.py", - "vllm_ascend/patch/__init__.py", - "vllm_ascend/patch/platform/__init__.py", - "vllm_ascend/patch/platform/patch_kv_cache_interface.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "c980e68d407c4767b553e1d4d35deaaf9ac66261", - "subject": "[Feature] support aclgraph for model runner v2 (#7110)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_fused_moe.py", - "tests/e2e/singlecard/model_runner_v2/test_basic.py", - "tests/ut/_310p/attention/test_attention_v1_310.py", - "tests/ut/attention/test_attention_cp.py", - "tests/ut/attention/test_attention_v1.py", - "tests/ut/attention/test_mla_cp.py", - "tests/ut/attention/test_mla_v1.py", - "tests/ut/compilation/test_acl_graph.py", - "tests/ut/ops/test_fused_moe.py", - "tests/ut/ops/test_mla.py", - "tests/ut/ops/test_moe_comm_method.py", - "tests/ut/ops/test_prepare_finalize.py", - "tests/ut/spec_decode/test_eagle_proposer.py", - "vllm_ascend/_310p/fused_moe/fused_moe.py", - "vllm_ascend/_310p/fused_moe/moe_comm_method.py", - "vllm_ascend/_310p/quantization/methods/w8a8_dynamic.py", - "vllm_ascend/ascend_forward_context.py", - "vllm_ascend/attention/attention_v1.py", - "vllm_ascend/attention/context_parallel/attention_cp.py", - "vllm_ascend/attention/context_parallel/mla_cp.py", - "vllm_ascend/attention/mla_v1.py", - "vllm_ascend/attention/sfa_v1.py", - "vllm_ascend/compilation/acl_graph.py", - "vllm_ascend/distributed/utils.py", - "vllm_ascend/ops/fused_moe/fused_moe.py", - "vllm_ascend/ops/fused_moe/moe_comm_method.py", - "vllm_ascend/ops/fused_moe/moe_mlp.py", - "vllm_ascend/ops/fused_moe/prepare_finalize.py", - "vllm_ascend/ops/linear_op.py", - "vllm_ascend/ops/mla.py", - "vllm_ascend/ops/register_custom_ops.py", - "vllm_ascend/ops/rotary_embedding.py", - "vllm_ascend/ops/weight_prefetch.py", - "vllm_ascend/patch/worker/patch_v2_eagle.py", - "vllm_ascend/platform.py", - "vllm_ascend/quantization/methods/w4a16.py", - "vllm_ascend/quantization/methods/w4a8.py", - "vllm_ascend/quantization/methods/w8a8_dynamic.py", - "vllm_ascend/quantization/methods/w8a8_mxfp8.py", - "vllm_ascend/spec_decode/eagle_proposer.py", - "vllm_ascend/worker/v2/README.md", - "vllm_ascend/worker/v2/aclgraph_utils.py", - "vllm_ascend/worker/v2/attn_utils.py", - "vllm_ascend/worker/v2/block_table.py", - "vllm_ascend/worker/v2/input_batch.py", - "vllm_ascend/worker/v2/model_runner.py", - "vllm_ascend/worker/v2/model_states/__init__.py", - "vllm_ascend/worker/v2/model_states/default.py", - "vllm_ascend/worker/v2/sample/sampler.py", - "vllm_ascend/worker/v2/spec_decode/eagle.py", - "vllm_ascend/worker/v2/states.py", - "vllm_ascend/worker/v2/utils.py" - ] - }, - { - "commit": "5fe7942bbde8459e8f636167984c57a4cfa33823", - "subject": "[CI] add action for issue labeler on issue open/edit (#7208)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - ".github/issue-labeler.yml", - ".github/workflows/bot_issue_manage.yaml" - ] - }, - { - "commit": "0c659e91ed458223f63abfaf26e8ad9b3908e4f8", - "subject": "[MTP][Bugfix] Fix GLM5-W8A8 precision issues caused by rotary quant MTP weights (#7139)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/patch/__init__.py", - "vllm_ascend/patch/worker/__init__.py", - "vllm_ascend/patch/worker/patch_deepseek_mtp.py" - ] - }, - { - "commit": "e5343d6eb32f863f72dbb0e503c3fbaefca7c924", - "subject": "[310P][Bugfix]: fix ngram graph replay accuracy error (#7134)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/_310p/model_runner_310p.py" - ] - }, - { - "commit": "132f3c5d0ae9a99a304e632dae77db1c3094eb06", - "subject": "Support per-step heat collection and enhance FlashLB for multi-stage load balancing (#6477)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/eplb/core/eplb_worker.py", - "vllm_ascend/eplb/core/policy/policy_flashlb.py", - "vllm_ascend/eplb/eplb_updator.py", - "vllm_ascend/ops/fused_moe/fused_moe.py" - ] - }, - { - "commit": "54668e73c55df7fe20faef1c80cd5f9281663f41", - "subject": "[Model] Support Minimax-m2.5 on NPU (#7105)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/patch/__init__.py", - "vllm_ascend/patch/platform/__init__.py", - "vllm_ascend/patch/platform/patch_minimax_m2_config.py", - "vllm_ascend/patch/worker/__init__.py", - "vllm_ascend/patch/worker/patch_minimax_m2.py", - "vllm_ascend/patch/worker/patch_minimax_m2_linear_attn.py" - ] - }, - { - "commit": "0f289fa2a8a381a644a5f329af61715a73ec2aec", - "subject": "Add patch_qwen3_5 for triton ops fused_recurrent_gated_delta_rule (#7109)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/patch/__init__.py", - "vllm_ascend/patch/worker/__init__.py", - "vllm_ascend/patch/worker/patch_qwen3_5.py", - "vllm_ascend/spec_decode/eagle_proposer.py" - ] - }, - { - "commit": "9216e1b0505c7e290d8c02cc64cb8817bfdd49f5", - "subject": "[fix] Add support for Qwen3.5 Dense and MoE on Ascend (#6933)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/ops/triton/fused_gdn_gating.py", - "vllm_ascend/quantization/modelslim_config.py" - ] - }, - { - "commit": "542258ac9d9229aab4e8822de42443245a93f001", - "subject": "[feat] parameterize hardcoded MLA dimensions to support GLM5-W8A8 (#6902)", - "topics": [ - "priority-models", - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "csrc/mla_preprocess/mla_preprocess_torch_adpt.h", - "csrc/mla_preprocess/op_host/mla_preprocess.h", - "csrc/mla_preprocess/op_host/tiling/mla_preprocess_tiling.h", - "csrc/mla_preprocess/op_kernel/mla_preprocess.h", - "csrc/mla_preprocess/op_kernel/mla_preprocess_kernel.cpp", - "csrc/mla_preprocess/op_kernel/mla_preprocess_mix_bf16.hpp", - "csrc/mla_preprocess/op_kernel/mla_preprocess_mix_bf16_nq.hpp", - "csrc/mla_preprocess/op_kernel/mla_preprocess_mix_bf16_qdown.hpp", - "csrc/mla_preprocess/op_kernel/mla_preprocess_mix_fp16.hpp" - ] - }, - { - "commit": "a76a509fae4d047ec3cc64b2eaf90e554abd188a", - "subject": "[MOE][Bugfix] Cancel H2D for expert_map (#7000)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/eplb/core/eplb_utils.py", - "vllm_ascend/ops/fused_moe/fused_moe.py" - ] - }, - { - "commit": "65eae6de7bb309636a9c294d46408f092762b3ca", - "subject": "Add Ascend Ops recurrent_gated_delta_rule (#6725)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/e2e/nightly/single_node/models/configs/Qwen3-Next-80B-A3B-Instruct.yaml", - "vllm_ascend/patch/worker/patch_qwen3_next.py" - ] - }, - { - "commit": "23bf5d4d48e6ec09e2b4f726279591a1b42f033b", - "subject": "[EPLB][bugfix] Bugfix for fused mc2 (#6794)", - "topics": [ - "communication", - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/eplb/adaptor/test_vllm_adaptor.py", - "vllm_ascend/ascend_forward_context.py", - "vllm_ascend/eplb/adaptor/vllm_adaptor.py", - "vllm_ascend/ops/fused_moe/moe_comm_method.py", - "vllm_ascend/quantization/methods/w8a8_dynamic.py" - ] - }, - { - "commit": "1c0ecf806ad871b05875c835fe3ae22d6e60ead7", - "subject": "[bugfix] fix pass bug: pass really rope dim for npu_rotary_embedding (#6880)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/compilation/passes/qknorm_rope_fusion_pass.py", - "vllm_ascend/utils.py" - ] - }, - { - "commit": "a51d6366b9a68de3244e9b8435621d40a95b3a68", - "subject": "[Bugfix] Qwen3Next support FlashComm1 (#6830)", - "topics": [ - "communication" - ], - "runtime_patch_match": false, - "paths": [ - "tests/e2e/multicard/4-cards/test_qwen3_next.py", - "vllm_ascend/ops/linear_op.py", - "vllm_ascend/patch/worker/patch_qwen3_next.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "a2696006d1756ba827dbf0bcf3386ecc1d3b01e9", - "subject": "[Refactor][EAGLE] 8/N delete mtp_proposer (re-pull) (#7033)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/spec_decode/test_mtp_proposer.py", - "vllm_ascend/attention/attention_v1.py", - "vllm_ascend/spec_decode/__init__.py", - "vllm_ascend/spec_decode/eagle_proposer.py", - "vllm_ascend/spec_decode/mtp_proposer.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "8c2c82f3e1ddac019de9619e9dde340c9ed863ef", - "subject": "[Bugfix] Fix the moe_forward error when setting enable_static_kernel \u2026 (#6964)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/ops/fused_moe/fused_moe.py" - ] - }, - { - "commit": "18b52afe2bb2e3ac80e988a2788ac1e3bbdb898a", - "subject": "[Ops][Misc] Optimize split_qkv_rmsnorm_rope op (#6827)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_split_qkv_rmsnorm_rope.py", - "tests/e2e/singlecard/compile/test_graphex_qknorm_rope_fusion.py", - "tests/e2e/singlecard/test_aclgraph_accuracy.py", - "vllm_ascend/ops/triton/linearnorm/split_qkv_rmsnorm_rope.py" - ] - }, - { - "commit": "a60e179c7f5ca5ab878bf51f177cbc9e24a6fb6b", - "subject": "[Refactor][EAGLE] 8/N delete mtp_proposer (#7016)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/spec_decode/test_mtp_proposer.py", - "vllm_ascend/attention/attention_v1.py", - "vllm_ascend/spec_decode/__init__.py", - "vllm_ascend/spec_decode/eagle_proposer.py", - "vllm_ascend/spec_decode/mtp_proposer.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "bd571cf6d6727fd8d1ee8221e0b301061cbd57d5", - "subject": "[Main2Main] Upgrade vLLM to 0303 (#6944)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/workflows/bot_pr_create.yaml", - ".github/workflows/dockerfiles/Dockerfile.lint", - ".github/workflows/pr_test_full.yaml", - ".github/workflows/pr_test_light.yaml", - ".github/workflows/schedule_codecov_refresh.yaml", - "docs/source/community/versioning_policy.md", - "tests/ut/_310p/quantization/test_modelslim_config_310.py", - "tests/ut/distributed/test_communicator.py", - "vllm_ascend/core/recompute_scheduler.py", - "vllm_ascend/ops/mm_encoder_attention.py", - "vllm_ascend/patch/__init__.py", - "vllm_ascend/patch/platform/__init__.py", - "vllm_ascend/patch/platform/patch_fusion_matcher_compat_ops.py", - "vllm_ascend/worker/model_runner_v1.py", - "vllm_ascend/worker/worker.py" - ] - }, - { - "commit": "640ecd1b772b1c3dcdc57336b762cc02d011eba8", - "subject": "[BugFix] Fix muls_add fusion not working for GLM5 models (#6928)", - "topics": [ - "priority-models", - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/compilation/passes/muls_add_pass.py" - ] - }, - { - "commit": "43c8da3574c96b9aaeaf4ef360c9b4aaf6a3e305", - "subject": "[Feat]fused_qkvzba_split_reshape supports token number greater than 65536 (#6740)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/ops/triton/fla/fused_qkvzba_split_reshape.py", - "vllm_ascend/patch/worker/patch_qwen3_next.py" - ] - }, - { - "commit": "2bd9c357886f04eca1ce5f9a00fe09aa77570479", - "subject": "[perf][refactor] Refactor and optimize sfa_v1.py for dsv3.2/glm5 (#6874)", - "topics": [ - "priority-models", - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/attention/context_parallel/sfa_cp.py", - "vllm_ascend/attention/sfa_v1.py", - "vllm_ascend/ops/triton/rope.py", - "vllm_ascend/utils.py" - ] - }, - { - "commit": "77e009d9fc0aaf814cf9a7557d6d8f6f62384185", - "subject": "[Feature] Add docs of batch invariance and make some extra operators patch (#6910)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "docs/source/user_guide/feature_guide/batch_invariance.md", - "docs/source/user_guide/feature_guide/index.md", - "tests/ut/batch_invariant/test_batch_invariant.py", - "vllm_ascend/ascend_config.py", - "vllm_ascend/batch_invariant.py", - "vllm_ascend/sample/sampler.py", - "vllm_ascend/utils.py" - ] - }, - { - "commit": "700423156f38dc40083721db382716c05a23de1a", - "subject": "[Triton] Centralize Ascend extension op dispatch in triton_utils (#6937)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/ops/triton/activation/swiglu_quant.py", - "vllm_ascend/ops/triton/fla/solve_tril.py", - "vllm_ascend/ops/triton/linearnorm/split_qkv_rmsnorm_rope.py", - "vllm_ascend/ops/triton/reject_sample.py", - "vllm_ascend/ops/triton/triton_utils.py" - ] - }, - { - "commit": "68d8d20ca249bd9b5c5ca510c08591192aa5c6b6", - "subject": "[misc] move mxfp_compat into device to decouple from quantization init chain (#6918)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/_310p/model_runner_310p.py", - "vllm_ascend/device/device_op.py", - "vllm_ascend/device/mxfp_compat.py", - "vllm_ascend/ops/fused_moe/moe_mlp.py", - "vllm_ascend/quantization/methods/w8a8_mxfp8.py", - "vllm_ascend/quantization/quant_parser.py" - ] - }, - { - "commit": "632801b0adb507024fe4e85790a6eef00661deda", - "subject": "[CI][310P] Add 310p tracked files in CI light. (#6923)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/pr_test_light.yaml" - ] - }, - { - "commit": "16c879cdf75382ef0dace18cc38d3d9a42fadad2", - "subject": "[Triton][Config] Add muls_add triton kernel and refactor AscendCompilationConfig (#5518)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "docs/source/user_guide/configuration/additional_config.md", - "docs/source/user_guide/feature_guide/npugraph_ex.md", - "tests/e2e/multicard/2-cards/test_sp_pass.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_muls_add.py", - "tests/e2e/singlecard/test_aclgraph_accuracy.py", - "tests/ut/test_ascend_config.py", - "vllm_ascend/ascend_config.py", - "vllm_ascend/compilation/compiler_interface.py", - "vllm_ascend/compilation/graph_fusion_pass_manager.py", - "vllm_ascend/compilation/passes/muls_add_pass.py", - "vllm_ascend/ops/register_custom_ops.py", - "vllm_ascend/ops/triton/muls_add.py", - "vllm_ascend/platform.py", - "vllm_ascend/worker/worker.py" - ] - }, - { - "commit": "3c66a970f29a4a976bbb4836e333bc58995b4a9d", - "subject": "add mxfp8 moe quantization (#6670)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/ascend_forward_context.py", - "vllm_ascend/device/device_op.py", - "vllm_ascend/ops/fused_moe/moe_comm_method.py", - "vllm_ascend/ops/fused_moe/moe_mlp.py", - "vllm_ascend/ops/fused_moe/prepare_finalize.py", - "vllm_ascend/ops/fused_moe/token_dispatcher.py", - "vllm_ascend/quantization/methods/base.py", - "vllm_ascend/quantization/methods/w8a8_mxfp8.py", - "vllm_ascend/quantization/mxfp_compat.py", - "vllm_ascend/quantization/quant_parser.py" - ] - }, - { - "commit": "5e24b26a543ae7ab92f7304c127ef27d43420369", - "subject": "[Bugfix] rename enable_flash_comm_v1 back to enable_sp (#6883)", - "topics": [ - "communication" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/ascend_forward_context.py", - "vllm_ascend/ops/linear.py", - "vllm_ascend/ops/linear_op.py", - "vllm_ascend/platform.py", - "vllm_ascend/utils.py", - "vllm_ascend/worker/model_runner_v1.py", - "vllm_ascend/worker/worker.py" - ] - }, - { - "commit": "9d09488b4a5c64ca52987da6f1c0d159e7fe9dae", - "subject": "[Feat] support basic pcp&dcp for qwen3next (#6091)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/e2e/multicard/4-cards/long_sequence/test_basic.py", - "tests/ut/attention/test_attention_cp.py", - "vllm_ascend/ascend_forward_context.py", - "vllm_ascend/attention/attention_v1.py", - "vllm_ascend/attention/context_parallel/attention_cp.py", - "vllm_ascend/attention/context_parallel/common_cp.py", - "vllm_ascend/attention/utils.py", - "vllm_ascend/ops/fused_moe/prepare_finalize.py", - "vllm_ascend/ops/triton/fla/chunk.py", - "vllm_ascend/ops/triton/fla/chunk_delta_h.py", - "vllm_ascend/ops/triton/fla/chunk_delta_hupdate.py", - "vllm_ascend/ops/triton/fla/chunk_o_update.py", - "vllm_ascend/ops/triton/fla/utils.py", - "vllm_ascend/ops/triton/mamba/causal_conv1d.py", - "vllm_ascend/worker/model_runner_v1.py", - "vllm_ascend/worker/pcp_utils.py" - ] - }, - { - "commit": "3d563292f3a54a025bbff6ff19c998b5555f5417", - "subject": "clean 0.15.0 support (#6852)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/community/versioning_policy.md", - "docs/source/user_guide/release_notes.md", - "tests/ut/attention/test_sfa_v1.py", - "tests/ut/quantization/test_modelslim_config.py", - "vllm_ascend/_310p/fused_moe/fused_moe.py", - "vllm_ascend/compilation/passes/sequence_parallelism.py", - "vllm_ascend/ops/mm_encoder_attention.py", - "vllm_ascend/sample/rejection_sampler.py" - ] - }, - { - "commit": "e4458b2d2bc292b82af2387ec25442e942323ee0", - "subject": "[Main2Main] Upgrade vLLM to 0226 (#6813)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/_e2e_nightly_multi_node.yaml", - ".github/workflows/bot_pr_create.yaml", - ".github/workflows/dockerfiles/Dockerfile.lint", - ".github/workflows/pr_test_full.yaml", - ".github/workflows/pr_test_light.yaml", - ".github/workflows/schedule_codecov_refresh.yaml", - ".github/workflows/schedule_nightly_test_a2.yaml", - ".github/workflows/schedule_test_benchmarks.yaml", - "Dockerfile", - "Dockerfile.310p", - "Dockerfile.310p.openEuler", - "Dockerfile.a3", - "Dockerfile.a3.openEuler", - "Dockerfile.openEuler", - "docs/source/community/versioning_policy.md", - "docs/source/conf.py", - "tests/e2e/singlecard/compile/backend.py", - "tests/e2e/singlecard/compile/test_norm_quant_fusion.py", - "tests/ut/eplb/core/test_eplb_utils.py", - "tests/ut/quantization/test_modelslim_config.py", - "vllm_ascend/ascend_forward_context.py", - "vllm_ascend/compilation/graph_fusion_pass_manager.py", - "vllm_ascend/compilation/passes/allreduce_rmsnorm_fusion_pass.py", - "vllm_ascend/compilation/passes/norm_quant_fusion_pass.py", - "vllm_ascend/compilation/passes/qknorm_rope_fusion_pass.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/cpu_offload/cpu_offload_connector.py", - "vllm_ascend/kv_offload/cpu_npu.py", - "vllm_ascend/ops/fused_moe/fused_moe.py", - "vllm_ascend/ops/mla.py", - "vllm_ascend/patch/platform/__init__.py", - "vllm_ascend/patch/worker/patch_qwen3_next_mtp.py", - "vllm_ascend/patch/worker/patch_v2_eagle.py", - "vllm_ascend/platform.py", - "vllm_ascend/quantization/method_adapters.py", - "vllm_ascend/quantization/modelslim_config.py", - "vllm_ascend/spec_decode/eagle_proposer.py", - "vllm_ascend/spec_decode/mtp_proposer.py", - "vllm_ascend/utils.py", - "vllm_ascend/worker/model_runner_v1.py", - "vllm_ascend/worker/worker.py" - ] - }, - { - "commit": "80316c5824417627cda88321293fba1146fe0dc8", - "subject": "[DOC] enable both flashcomm1 and cudagraph (#6807)", - "topics": [ - "communication" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/tutorials/models/DeepSeek-V3.2.md" - ] - }, - { - "commit": "a95c0b8b82ff2e715c0bce903c763a08f13d59ba", - "subject": "[Doc] fix the nit in docs (#6826)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/community/contributors.md", - "docs/source/community/versioning_policy.md", - "docs/source/developer_guide/contribution/index.md", - "docs/source/developer_guide/contribution/multi_node_test.md", - "docs/source/developer_guide/contribution/testing.md", - "docs/source/developer_guide/evaluation/using_ais_bench.md", - "docs/source/developer_guide/evaluation/using_evalscope.md", - "docs/source/developer_guide/feature_guide/ACL_Graph.md", - "docs/source/developer_guide/feature_guide/KV_Cache_Pool_Guide.md", - "docs/source/developer_guide/feature_guide/ModelRunner_prepare_inputs.md", - "docs/source/developer_guide/feature_guide/context_parallel.md", - "docs/source/developer_guide/feature_guide/disaggregated_prefill.md", - "docs/source/developer_guide/feature_guide/eplb_swift_balancer.md", - "docs/source/developer_guide/feature_guide/index.md", - "docs/source/developer_guide/feature_guide/npugraph_ex.md", - "docs/source/developer_guide/feature_guide/patch.md", - "docs/source/developer_guide/performance_and_debug/optimization_and_tuning.md", - "docs/source/faqs.md", - "docs/source/installation.md", - "docs/source/quick_start.md", - "docs/source/tutorials/models/GLM5.md", - "docs/source/tutorials/models/Kimi-K2-Thinking.md", - "docs/source/tutorials/models/PaddleOCR-VL.md", - "docs/source/tutorials/models/Qwen2.5-7B.md", - "docs/source/tutorials/models/Qwen3-30B-A3B.md", - "docs/source/user_guide/feature_guide/eplb_swift_balancer.md", - "docs/source/user_guide/feature_guide/external_dp.md", - "docs/source/user_guide/feature_guide/kv_pool.md", - "docs/source/user_guide/feature_guide/large_scale_ep.md", - "docs/source/user_guide/feature_guide/lora.md" - ] - }, - { - "commit": "5def28dcd3f6330e583671f0880b3452151ef10a", - "subject": "[Feat]support sequence parallelism by pass for VL models (#5632)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/workflows/scripts/config.yaml", - "docs/source/user_guide/configuration/additional_config.md", - "docs/source/user_guide/feature_guide/index.md", - "docs/source/user_guide/feature_guide/sequence_parallelism.md", - "tests/e2e/multicard/2-cards/test_sp_pass.py", - "tests/ut/ops/test_mla.py", - "tests/ut/spec_decode/test_eagle_proposer.py", - "vllm_ascend/ascend_config.py", - "vllm_ascend/ascend_forward_context.py", - "vllm_ascend/compilation/compiler_interface.py", - "vllm_ascend/compilation/graph_fusion_pass_manager.py", - "vllm_ascend/compilation/passes/sequence_parallelism.py", - "vllm_ascend/ops/fused_moe/fused_moe.py", - "vllm_ascend/ops/linear.py", - "vllm_ascend/ops/linear_op.py", - "vllm_ascend/ops/mla.py", - "vllm_ascend/ops/register_custom_ops.py", - "vllm_ascend/platform.py", - "vllm_ascend/spec_decode/eagle_proposer.py", - "vllm_ascend/utils.py", - "vllm_ascend/worker/model_runner_v1.py", - "vllm_ascend/worker/worker.py" - ] - }, - { - "commit": "532f7a82f2700e1d7e83c233e70f61793bc2beeb", - "subject": "[Patch][Misc] Cleanup and update patches (#6802)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/patch/__init__.py", - "vllm_ascend/patch/worker/__init__.py", - "vllm_ascend/patch/worker/patch_deepseek.py" - ] - }, - { - "commit": "e76b69b9ef042f6e4ccc060bc9df76ca0a7bbba9", - "subject": "[BugFix] [310p] Fix attention accuracy issue (#6803)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/e2e/310p/multicard/test_dense_model_multicard.py", - "tests/e2e/310p/multicard/test_moe_model_multicard.py", - "tests/e2e/310p/singlecard/test_dense_model_singlecard.py", - "tests/ut/_310p/attention/test_attention_mask_310.py", - "vllm_ascend/_310p/attention/attention_mask.py", - "vllm_ascend/_310p/attention/attention_v1.py", - "vllm_ascend/_310p/attention/metadata_builder.py", - "vllm_ascend/_310p/fused_moe/fused_moe.py" - ] - }, - { - "commit": "2870f7c8ad20754f5cc09cc5ea25044ffb3c6515", - "subject": "[Feat] Support routing replay (#6696)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/workflows/scripts/config.yaml", - "tests/e2e/multicard/2-cards/test_qwen3_moe_routing_replay.py", - "tests/ut/patch/worker/patch_common/test_patch_routed_experts_capturer.py", - "vllm_ascend/ops/fused_moe/fused_moe.py", - "vllm_ascend/patch/worker/__init__.py", - "vllm_ascend/patch/worker/patch_routed_experts_capturer.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "29e3cdde20944457df4ab45550be1630794954fd", - "subject": "[Doc][Skill] Introduce AI-assisted model-adaptation workflow for vllm-ascend (#6731)", - "topics": [ - "communication" - ], - "runtime_patch_match": false, - "paths": [ - ".agents/skills/vllm-ascend-model-adapter/README.md", - ".agents/skills/vllm-ascend-model-adapter/SKILL.md", - ".agents/skills/vllm-ascend-model-adapter/references/deliverables.md", - ".agents/skills/vllm-ascend-model-adapter/references/fp8-on-npu-lessons.md", - ".agents/skills/vllm-ascend-model-adapter/references/multimodal-ep-aclgraph-lessons.md", - ".agents/skills/vllm-ascend-model-adapter/references/troubleshooting.md", - ".agents/skills/vllm-ascend-model-adapter/references/workflow-checklist.md" - ] - }, - { - "commit": "2260af405f7c9a37e51d96cfbe984e1b96fc601d", - "subject": "[DOC] add request forwarding (#6780)", - "topics": [ - "communication" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/tutorials/models/DeepSeek-V3.2.md" - ] - }, - { - "commit": "ee59429015e34b0b628b12375583860dfc8b8054", - "subject": "upgrade main to 0212 (#6712)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/workflows/bot_pr_create.yaml", - ".github/workflows/dockerfiles/Dockerfile.lint", - ".github/workflows/pr_test_full.yaml", - ".github/workflows/pr_test_light.yaml", - ".github/workflows/schedule_codecov_refresh.yaml", - "docs/source/community/versioning_policy.md", - "requirements.txt", - "tests/ut/eplb/core/test_eplb_utils.py", - "vllm_ascend/_310p/model_runner_310p.py", - "vllm_ascend/ops/fused_moe/fused_moe.py", - "vllm_ascend/utils.py" - ] - }, - { - "commit": "62ea664aa757a7c40c203d42a027ff8e4ae76480", - "subject": "[Lint]Style: Convert `test/` to ruff format(Batch #5) (#6747)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "pyproject.toml", - "tests/e2e/singlecard/compile/backend.py", - "tests/e2e/singlecard/compile/test_graphex_norm_quant_fusion.py", - "tests/e2e/singlecard/compile/test_graphex_qknorm_rope_fusion.py", - "tests/e2e/singlecard/compile/test_norm_quant_fusion.py", - "tests/e2e/singlecard/model_runner_v2/test_basic.py", - "tests/e2e/singlecard/test_aclgraph_accuracy.py", - "tests/e2e/singlecard/test_aclgraph_batch_invariant.py", - "tests/e2e/singlecard/test_aclgraph_mem.py", - "tests/e2e/singlecard/test_async_scheduling.py", - "tests/e2e/singlecard/test_auto_fit_max_mode_len.py", - "tests/e2e/singlecard/test_batch_invariant.py", - "tests/e2e/singlecard/test_camem.py", - "tests/e2e/singlecard/test_completion_with_prompt_embeds.py", - "tests/e2e/singlecard/test_cpu_offloading.py", - "tests/e2e/singlecard/test_guided_decoding.py", - "tests/e2e/singlecard/test_ilama_lora.py", - "tests/e2e/singlecard/test_llama32_lora.py", - "tests/e2e/singlecard/test_models.py", - "tests/e2e/singlecard/test_multistream_overlap_shared_expert.py", - "tests/e2e/singlecard/test_quantization.py", - "tests/e2e/singlecard/test_qwen3_multi_loras.py", - "tests/e2e/singlecard/test_sampler.py", - "tests/e2e/singlecard/test_vlm.py", - "tests/e2e/singlecard/test_xlite.py", - "tests/e2e/singlecard/utils.py" - ] - }, - { - "commit": "f41eeeb11e1799ce8a66dfae8893151cb393171d", - "subject": "Refactor the ops PyTorch adapter\uff0ccleanup for csrc/torch_binding.cpp (#6732)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "csrc/add_rms_norm_bias/add_rms_norm_bias_torch_adpt.h", - "csrc/apply_top_k_top_p_custom/apply_top_k_top_p_custom_torch_adpt.h", - "csrc/batch_matmul_transpose/batch_matmul_transpose_torch_adpt.h", - "csrc/dispatch_ffn_combine/dispatch_ffn_combine_torch_adpt.h", - "csrc/dispatch_gmm_combine_decode/dispatch_gmm_combine_decode_torch_adpt.h", - "csrc/dispatch_layout/dispatch_layout_torch_adpt.h", - "csrc/grouped_matmul_swiglu_quant_weight_nz_tensor_list/grouped_matmul_swiglu_quant_torch_adpt.h", - "csrc/lightning_indexer_vllm/lightning_indexer_vllm_torch_adpt.h", - "csrc/matmul_allreduce_add_rmsnorm/matmul_allreduce_add_rmsnorm_torch_adpt.h", - "csrc/mla_preprocess/mla_preprocess_torch_adpt.h", - "csrc/moe_combine_normal/moe_combine_normal_torch_adpt.h", - "csrc/moe_gating_top_k/moe_gating_top_k_torch_adpt.h", - "csrc/moe_init_routing_custom/moe_init_routing_custom_torch_adpt.h", - "csrc/sparse_flash_attention/sparse_flash_attention_torch_adpt.h", - "csrc/torch_binding.cpp" - ] - }, - { - "commit": "70e26551cfd8c8ca255df72f2af027a617c29cf4", - "subject": "[Doc] modify glm doc (#6770)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/tutorials/models/GLM5.md" - ] - }, - { - "commit": "e2237819a9a70a49b8c1bfaadaaa0614ca1c840e", - "subject": "[CI]Fixed the spell check function in `typos.toml` (#6753)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/workflows/labled_doctest.yaml", - ".github/workflows/schedule_nightly_test_a2.yaml", - ".pre-commit-config.yaml", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/modeling/adding_a_new_multimodal_model.po", - "docs/source/user_guide/release_notes.md", - "examples/disaggregated_prefill_v1/load_balance_proxy_layerwise_server_example.py", - "examples/disaggregated_prefill_v1/load_balance_proxy_server_example.py", - "examples/eplb/eplb_strategy.py", - "examples/external_online_dp/dp_load_balance_proxy_server.py", - "examples/external_online_dp/launch_online_dp.py", - "setup.py", - "tests/e2e/nightly/multi_node/scripts/multi_node_config.py", - "tests/e2e/nightly/multi_node/scripts/utils.py", - "tests/e2e/nightly/single_node/ops/multicard_ops_a3/test_dispatch_ffn_combine.py", - "tests/e2e/nightly/single_node/ops/multicard_ops_a3/test_dispatch_ffn_combine_bf16.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_penality.py", - "tests/ut/ops/test_moe_mlp.py", - "typos.toml", - "vllm_ascend/ascend_config.py", - "vllm_ascend/compilation/passes/allreduce_rmsnorm_fusion_pass.py", - "vllm_ascend/core/scheduler_dynamic_batch.py", - "vllm_ascend/distributed/device_communicators/pyhccl_wrapper.py", - "vllm_ascend/distributed/kv_transfer/kv_p2p/mooncake_connector.py", - "vllm_ascend/distributed/kv_transfer/kv_p2p/mooncake_layerwise_connector.py", - "vllm_ascend/eplb/core/eplb_device_transfer_loader.py", - "vllm_ascend/eplb/core/policy/policy_flashlb.py", - "vllm_ascend/platform.py", - "vllm_ascend/spec_decode/eagle_proposer.py", - "vllm_ascend/spec_decode/interface.py", - "vllm_ascend/worker/model_runner_v1.py", - "vllm_ascend/worker/pcp_utils.py" - ] - }, - { - "commit": "1e77077788bd9f889dd2b6ab315bd9839444cbaf", - "subject": "[Bugfix][DispatchFFNCombine] resolve vec error caused by unaligned UB access (#6707)", - "topics": [ - "communication", - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "csrc/dispatch_ffn_combine/op_kernel/dispatch_ffn_combine_kernel.hpp", - "csrc/dispatch_ffn_combine_bf16/op_kernel/dispatch_ffn_combine_bf16_kernel.hpp", - "vllm_ascend/ops/fused_moe/moe_comm_method.py" - ] - }, - { - "commit": "f40256b697b9ab533e7e3a9a86a1d2ac8dd7c9b3", - "subject": "[Feat.][310P] addrmsnorm for 300I DUO (#6704)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/ops/test_layernorm.py", - "vllm_ascend/_310p/ops/layernorm.py", - "vllm_ascend/_310p/ops/mm_encoder_attention.py", - "vllm_ascend/utils.py" - ] - }, - { - "commit": "71649909047bdf275edd8db30794f8a7abf49360", - "subject": "[Graph][Fusion] Integrating inductor pass and npugraph ex pass (#6354)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/e2e/singlecard/compile/test_graphex_norm_quant_fusion.py", - "tests/e2e/singlecard/compile/test_graphex_qknorm_rope_fusion.py", - "tests/e2e/singlecard/compile/test_norm_quant_fusion.py", - "tests/ut/compilation/test_npugraph_ex_utils_check.py", - "vllm_ascend/compilation/npu_graph_ex_pass_manager.py", - "vllm_ascend/compilation/npugraph_ex_passes/graphex_allreduce_rmsnorm_fusion_pass.py", - "vllm_ascend/compilation/npugraph_ex_passes/graphex_norm_quant_fusion_pass.py", - "vllm_ascend/compilation/npugraph_ex_passes/graphex_qknorm_rope_fusion_pass.py", - "vllm_ascend/compilation/npugraph_ex_passes/utils/__init__.py", - "vllm_ascend/compilation/passes/allreduce_rmsnorm_fusion_pass.py", - "vllm_ascend/compilation/passes/base_pattern.py", - "vllm_ascend/compilation/passes/norm_quant_fusion_pass.py", - "vllm_ascend/compilation/passes/qknorm_rope_fusion_pass.py", - "vllm_ascend/compilation/passes/utils/__init__.py", - "vllm_ascend/compilation/passes/utils/npugraph_ex_utils_check.py", - "vllm_ascend/platform.py" - ] - }, - { - "commit": "87a0b7b7c7630ae112c8384ad1af11ed29d55801", - "subject": "[bugfix] adapt bugfix for norm_quant_fusion_pass to npugraph_ex (#6726)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/compilation/npugraph_ex_passes/graphex_norm_quant_fusion_pass.py" - ] - }, - { - "commit": "41d056f94716d2ac74b73f26547a109c099e82dd", - "subject": "[doc] add A2 series doc for GLM5.md (#6717)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/tutorials/models/GLM5.md" - ] - }, - { - "commit": "7221045777bd4c3d77037fcb20d9bddd7b4dba3c", - "subject": "[Attention] add gpt-oss support (#5901)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/workflows/scripts/config.yaml", - "tests/e2e/multicard/2-cards/test_offline_inference_distributed.py", - "vllm_ascend/attention/attention_v1.py", - "vllm_ascend/ops/activation.py", - "vllm_ascend/ops/fused_moe/fused_moe.py", - "vllm_ascend/ops/fused_moe/moe_comm_method.py", - "vllm_ascend/ops/fused_moe/moe_mlp.py", - "vllm_ascend/ops/rotary_embedding.py" - ] - }, - { - "commit": "f71812011d49013ca077cc2441412593b7356052", - "subject": "[Feature] DispatchGmmCombineDecode support bf16/float16 gmm1/gmm2 weight and support gmm weight with ND format (#6393)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "csrc/dispatch_gmm_combine_decode/op_host/CMakeLists.txt", - "csrc/dispatch_gmm_combine_decode/op_host/dispatch_gmm_combine_decode_def.cpp", - "csrc/dispatch_gmm_combine_decode/op_host/dispatch_gmm_combine_decode_tiling.cpp", - "csrc/dispatch_gmm_combine_decode/op_kernel/dispatch_gmm_combine_decode.cpp", - "csrc/dispatch_gmm_combine_decode/op_kernel/dispatch_gmm_combine_decode.h", - "csrc/dispatch_gmm_combine_decode/op_kernel/dispatch_gmm_combine_decode/epilogue/block/block_epilogue.h", - "csrc/dispatch_gmm_combine_decode/op_kernel/dispatch_gmm_combine_decode/epilogue/block/block_epilogue_bf16_fp16.hpp", - "csrc/dispatch_gmm_combine_decode/op_kernel/dispatch_gmm_combine_decode/epilogue/block/block_epilogue_swiglu_bf16_fp16.h", - "csrc/dispatch_gmm_combine_decode/op_kernel/dispatch_gmm_combine_decode/epilogue/dispatch_policy.h", - "csrc/dispatch_gmm_combine_decode/op_kernel/dispatch_gmm_combine_decode/gemm/kernel/grouped_matmul_slice_m_multistage_workspace_bf16_fp16.h", - "csrc/dispatch_gmm_combine_decode/op_kernel/dispatch_gmm_combine_decode/gemm/kernel/grouped_matmul_slice_m_per_token_dequant_swiglu_quant_multistage_workspace.h", - "csrc/dispatch_gmm_combine_decode/op_kernel/dispatch_gmm_combine_decode/gemm/kernel/grouped_matmul_slice_m_swiglu_multistage_workspace_bf16_fp16.h", - "csrc/dispatch_gmm_combine_decode/op_kernel/dispatch_gmm_combine_decode/raw_distributed/cam_moe_distribute_combine.h", - "csrc/dispatch_gmm_combine_decode/op_kernel/dispatch_gmm_combine_decode/raw_distributed/cam_moe_distribute_dispatch.h", - "csrc/dispatch_gmm_combine_decode/op_kernel/dispatch_gmm_combine_decode_base.h", - "csrc/dispatch_gmm_combine_decode/op_kernel/dispatch_gmm_combine_decode_bf16_fp16.h", - "csrc/dispatch_gmm_combine_decode/op_kernel/dispatch_gmm_combine_decode_tiling.h", - "tests/e2e/nightly/single_node/ops/multicard_ops_a3/test_dispatch_gmm_combine_decode.py" - ] - }, - { - "commit": "56269eae0e553909ab8dad60deb5c9235988a04e", - "subject": "[BugFix] Fix AddRMSNormQuant not taking effect (#6620)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/e2e/singlecard/compile/test_norm_quant_fusion.py", - "tests/e2e/singlecard/test_quantization.py", - "vllm_ascend/compilation/passes/norm_quant_fusion_pass.py" - ] - }, - { - "commit": "052cc4e61bd7625d47f6e93ff03a272c0520be86", - "subject": "[Docs] Fix GLM-5 deploy command (#6711)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/tutorials/models/GLM5.md" - ] - }, - { - "commit": "a0315f66972d64cd586cb2ef29d8484f51e3194a", - "subject": "[npugraph_ex]enable npugraph_ex by default (#6664)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "docs/source/user_guide/configuration/additional_config.md", - "tests/e2e/singlecard/test_aclgraph_accuracy.py", - "tests/ut/test_ascend_config.py", - "vllm_ascend/ascend_config.py", - "vllm_ascend/compilation/compiler_interface.py", - "vllm_ascend/compilation/npu_graph_ex_pass_manager.py", - "vllm_ascend/patch/__init__.py", - "vllm_ascend/patch/worker/__init__.py", - "vllm_ascend/patch/worker/patch_npugraph_ex_triton.py", - "vllm_ascend/platform.py" - ] - }, - { - "commit": "b86ea66b0a2de3795609804b3f93538563f6dc8d", - "subject": "[doc]add GLM5.md (#6709)", - "topics": [ - "priority-models" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/tutorials/models/GLM5.md", - "docs/source/tutorials/models/index.md" - ] - }, - { - "commit": "ff3a50d011dcbea08f87ebed69ff1bf156dbb01e", - "subject": "[Model] GLM5 adaptation (#6642)", - "topics": [ - "priority-models", - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "csrc/build_aclnn.sh", - "csrc/lightning_indexer_vllm/op_host/CMakeLists.txt", - "csrc/lightning_indexer_vllm/op_host/lightning_indexer_vllm_def.cpp", - "csrc/lightning_indexer_vllm/op_host/lightning_indexer_vllm_proto.cpp", - "csrc/lightning_indexer_vllm/op_host/lightning_indexer_vllm_tiling.cpp", - "csrc/lightning_indexer_vllm/op_host/lightning_indexer_vllm_tiling.h", - "csrc/lightning_indexer_vllm/op_kernel/lightning_indexer_common.h", - "csrc/lightning_indexer_vllm/op_kernel/lightning_indexer_kernel.h", - "csrc/lightning_indexer_vllm/op_kernel/lightning_indexer_service_cube.h", - "csrc/lightning_indexer_vllm/op_kernel/lightning_indexer_service_vector.h", - "csrc/lightning_indexer_vllm/op_kernel/lightning_indexer_template_tiling_key.h", - "csrc/lightning_indexer_vllm/op_kernel/lightning_indexer_vector.h", - "csrc/lightning_indexer_vllm/op_kernel/lightning_indexer_vllm.cpp", - "csrc/torch_binding.cpp", - "vllm_ascend/attention/sfa_v1.py", - "vllm_ascend/quantization/modelslim_config.py", - "vllm_ascend/spec_decode/mtp_proposer.py" - ] - }, - { - "commit": "c0c2eb614e7cbc262b17de9dd6aad42883705a62", - "subject": "[Main][Ops] Make triton rope support index_selecting from cos_sin_cache (#5450)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_rope.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_split_qkv_rmsnorm_rope.py", - "tests/e2e/singlecard/compile/test_graphex_qknorm_rope_fusion.py", - "tests/e2e/singlecard/test_aclgraph_accuracy.py", - "vllm_ascend/ascend_forward_context.py", - "vllm_ascend/compilation/npugraph_ex_passes/graphex_qknorm_rope_fusion_pass.py", - "vllm_ascend/compilation/passes/qknorm_rope_fusion_pass.py", - "vllm_ascend/ops/register_custom_ops.py", - "vllm_ascend/ops/rotary_embedding.py", - "vllm_ascend/ops/triton/linearnorm/split_qkv_rmsnorm_rope.py", - "vllm_ascend/ops/triton/rope.py", - "vllm_ascend/spec_decode/eagle_proposer.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "02886e2641d7a9679f2e587745a172228b0f7c71", - "subject": "[Feat] 310p support MoE W8A8 quantizaition (#6641)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/e2e/310p/multicard/test_moe_model_multicard.py", - "tests/ut/_310p/fused_moe/test_experts_selector_310.py", - "tests/ut/_310p/fused_moe/test_moe_mlp_310.py", - "tests/ut/_310p/quantization/test_modelslim_config_310.py", - "tests/ut/_310p/quantization/test_w8a8_dynamic_310.py", - "tests/ut/_310p/quantization/test_w8a8_static_310.py", - "vllm_ascend/_310p/fused_moe/experts_selector.py", - "vllm_ascend/_310p/fused_moe/fused_moe.py", - "vllm_ascend/_310p/fused_moe/moe_comm_method.py", - "vllm_ascend/_310p/fused_moe/moe_mlp.py", - "vllm_ascend/_310p/fused_moe/token_dispatcher.py", - "vllm_ascend/_310p/quantization/methods/__init__.py", - "vllm_ascend/_310p/quantization/methods/w8a8_dynamic.py", - "vllm_ascend/_310p/quantization/methods/w8a8_static.py", - "vllm_ascend/_310p/quantization/modelslim_config.py" - ] - }, - { - "commit": "7cf285a77a95e485164130937082ed75e18003d5", - "subject": "[MOE Refactor] Remove QuantType in prepare_finalize.py (#6534)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/ops/test_moe_comm_method.py", - "vllm_ascend/ops/fused_moe/fused_moe.py", - "vllm_ascend/ops/fused_moe/moe_comm_method.py", - "vllm_ascend/ops/fused_moe/prepare_finalize.py" - ] - }, - { - "commit": "77305df398a462b90bc22dfc7c17b9d01dea9111", - "subject": "implement batch invariant with ascendc (#6590)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/batch_invariant/test_batch_invariant.py", - "vllm_ascend/batch_invariant.py" - ] - }, - { - "commit": "66b60c9440fe0c3731453ea4fa0da70c594dbfe5", - "subject": "[Refact]Refact MLA/SFA weight prefetch to consist with moe weight prefetch (#6629)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/attention/test_mla_cp.py", - "tests/ut/attention/test_mla_v1.py", - "tests/ut/ops/test_activation.py", - "tests/ut/ops/test_fused_moe.py", - "tests/ut/ops/test_linear.py", - "vllm_ascend/_310p/fused_moe/experts_selector.py", - "vllm_ascend/attention/mla_v1.py", - "vllm_ascend/attention/sfa_v1.py", - "vllm_ascend/ops/activation.py", - "vllm_ascend/ops/fused_moe/experts_selector.py", - "vllm_ascend/ops/fused_moe/moe_mlp.py", - "vllm_ascend/ops/layernorm.py", - "vllm_ascend/ops/linear_op.py", - "vllm_ascend/ops/weight_prefetch.py", - "vllm_ascend/quantization/methods/w8a8_static.py" - ] - }, - { - "commit": "2a826b5fad5283ca4174567835413b8c08493f3e", - "subject": "[Misc] upgrade to vllm main (#6646)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/_unit_test.yaml", - ".github/workflows/bot_pr_create.yaml", - ".github/workflows/dockerfiles/Dockerfile.lint", - ".github/workflows/pr_test_full.yaml", - ".github/workflows/pr_test_light.yaml", - ".github/workflows/schedule_codecov_refresh.yaml", - "docs/source/community/versioning_policy.md", - "tests/e2e/multicard/2-cards/test_aclgraph_capture_replay.py", - "tests/e2e/singlecard/compile/backend.py", - "tests/e2e/singlecard/compile/test_norm_quant_fusion.py", - "tests/e2e/singlecard/test_llama32_lora.py", - "tests/ut/spec_decode/test_eagle_proposer.py", - "tests/ut/spec_decode/test_mtp_proposer.py", - "vllm_ascend/compilation/graph_fusion_pass_manager.py", - "vllm_ascend/compilation/npu_graph_ex_pass_manager.py", - "vllm_ascend/compilation/npugraph_ex_passes/graphex_allreduce_rmsnorm_fusion_pass.py", - "vllm_ascend/compilation/passes/allreduce_rmsnorm_fusion_pass.py", - "vllm_ascend/compilation/passes/norm_quant_fusion_pass.py", - "vllm_ascend/compilation/passes/qknorm_rope_fusion_pass.py" - ] - }, - { - "commit": "d060c797ed8fd22a1ecdf86c5e26f6eeaf7531f2", - "subject": "[fix bug] fix tensor mismatch bug in sigmoid operate test case (#6619)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_fused_sigmoid_gating_delta_rule.py" - ] - }, - { - "commit": "832552836842e00cc9de75bc853a90338694ccc4", - "subject": "[Kernel]: Optimize DispatchFFNCombine performance (#6468)", - "topics": [ - "communication", - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "csrc/dispatch_ffn_combine/op_host/dispatch_ffn_combine_tiling.cpp", - "csrc/dispatch_ffn_combine/op_kernel/dispatch_ffn_combine.cpp", - "csrc/dispatch_ffn_combine/op_kernel/dispatch_ffn_combine.h", - "csrc/dispatch_ffn_combine/op_kernel/dispatch_ffn_combine_kernel.hpp", - "csrc/dispatch_ffn_combine/op_kernel/moe_init_routing_quant_v2/moe_v2_common.h", - "csrc/dispatch_ffn_combine/op_kernel/moe_init_routing_quant_v2/moe_v2_fullload_dynamic_quant.h", - "csrc/dispatch_ffn_combine/op_kernel/moe_init_routing_quant_v2/moe_v2_gather_dynamic_quant.h", - "csrc/dispatch_ffn_combine/op_kernel/unpermute/moe_token_unpermute.h", - "csrc/dispatch_ffn_combine/op_kernel/utils/block_epilogue_pertoken_v2.hpp", - "csrc/dispatch_ffn_combine/op_kernel/utils/block_mmad_preload_async_fixpipe_quant.hpp", - "csrc/dispatch_ffn_combine/op_kernel/utils/const_args.hpp", - "csrc/dispatch_ffn_combine/op_kernel/utils/dispatch_policy_custom.hpp", - "csrc/dispatch_ffn_combine/op_kernel/utils/hccl_shmem.hpp" - ] - }, - { - "commit": "1cc225711d04aa9ee6a499b7965b8193646e760f", - "subject": "[Refactor]310p_e2e test case update (#6539)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/_e2e_test.yaml", - "tests/e2e/310p/multicard/test_dense_model_multicard.py", - "tests/e2e/310p/multicard/test_moe_model_multicard.py", - "tests/e2e/310p/singlecard/test_dense_model_singlecard.py", - "tests/e2e/310p/test_offline_inference_310p.py", - "tests/e2e/310p/test_offline_inference_w8a8_310p.py", - "vllm_ascend/_310p/fused_moe/fused_moe.py", - "vllm_ascend/_310p/worker_310p.py" - ] - }, - { - "commit": "4f33e250466afa4f67ac3a0d8f9dc1882fae8cc6", - "subject": "[Refactor]refactor 310p attention impl and add ut (#6579)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/_310p/attention/test_attention_mask_310.py", - "tests/ut/_310p/attention/test_attention_v1_310.py", - "vllm_ascend/_310p/attention/attention_mask.py", - "vllm_ascend/_310p/attention/attention_v1.py", - "vllm_ascend/_310p/attention/metadata_builder.py" - ] - }, - { - "commit": "6c49f95da2f97fcfbe709d96ad794c2675f42a66", - "subject": "[Ops][Refactor] Remove custom rotary_embedding operator (#6523)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "csrc/kernels/pos_encoding_kernels.cpp", - "csrc/ops.h", - "csrc/torch_binding.cpp", - "csrc/torch_binding_meta.cpp", - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_rotary_embedding.py", - "tests/ut/ops/test_rotary_embedding.py", - "vllm_ascend/meta_registration.py", - "vllm_ascend/ops/rotary_embedding.py" - ] - }, - { - "commit": "06aa6036f6647019f1c44ef7db79169bff069f2e", - "subject": "[Lint]Style: Convert `vllm-ascend/` to ruff format(new Batch #8) (#6604)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "pyproject.toml", - "vllm_ascend/ops/__init__.py", - "vllm_ascend/ops/activation.py", - "vllm_ascend/ops/flashcomm2_oshard_manager.py", - "vllm_ascend/ops/layernorm.py", - "vllm_ascend/ops/mla.py", - "vllm_ascend/ops/mm_encoder_attention.py", - "vllm_ascend/ops/register_custom_ops.py", - "vllm_ascend/ops/vocab_parallel_embedding.py", - "vllm_ascend/ops/weight_prefetch.py", - "vllm_ascend/spec_decode/__init__.py", - "vllm_ascend/spec_decode/eagle_proposer.py", - "vllm_ascend/spec_decode/interface.py", - "vllm_ascend/spec_decode/medusa_proposer.py", - "vllm_ascend/spec_decode/mtp_proposer.py", - "vllm_ascend/spec_decode/ngram_proposer.py", - "vllm_ascend/spec_decode/suffix_proposer.py" - ] - }, - { - "commit": "06c0aed124f94335869d50323c495a2e5fb81f8e", - "subject": "[CI] Fix broken CI (#6599)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "pyproject.toml", - "vllm_ascend/ops/__init__.py", - "vllm_ascend/ops/activation.py", - "vllm_ascend/ops/flashcomm2_oshard_manager.py", - "vllm_ascend/ops/layernorm.py", - "vllm_ascend/ops/mla.py", - "vllm_ascend/ops/mm_encoder_attention.py", - "vllm_ascend/ops/register_custom_ops.py", - "vllm_ascend/ops/vocab_parallel_embedding.py", - "vllm_ascend/ops/weight_prefetch.py", - "vllm_ascend/spec_decode/__init__.py", - "vllm_ascend/spec_decode/eagle_proposer.py", - "vllm_ascend/spec_decode/interface.py", - "vllm_ascend/spec_decode/medusa_proposer.py", - "vllm_ascend/spec_decode/mtp_proposer.py", - "vllm_ascend/spec_decode/ngram_proposer.py", - "vllm_ascend/spec_decode/suffix_proposer.py" - ] - }, - { - "commit": "19b5d44ea859eef468ade11220ebdb74440fc7e6", - "subject": "[Lint]Style: Convert `vllm-ascend/` to ruff format(Batch #10) (#6173)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "pyproject.toml", - "vllm_ascend/distributed/parallel_state.py", - "vllm_ascend/distributed/utils.py", - "vllm_ascend/ops/layer_shard_linear.py", - "vllm_ascend/ops/linear.py", - "vllm_ascend/ops/linear_op.py", - "vllm_ascend/ops/rotary_embedding.py", - "vllm_ascend/patch/worker/patch_bert.py", - "vllm_ascend/patch/worker/patch_deepseek.py", - "vllm_ascend/patch/worker/patch_distributed.py", - "vllm_ascend/patch/worker/patch_huanyuan_vl.py", - "vllm_ascend/patch/worker/patch_module.py", - "vllm_ascend/patch/worker/patch_multimodal_merge.py", - "vllm_ascend/patch/worker/patch_qwen3_next.py", - "vllm_ascend/patch/worker/patch_qwen3_next_mtp.py", - "vllm_ascend/patch/worker/patch_rejection_sampler.py", - "vllm_ascend/patch/worker/patch_rope.py", - "vllm_ascend/patch/worker/patch_triton.py", - "vllm_ascend/patch/worker/patch_unquantized_gemm.py", - "vllm_ascend/patch/worker/patch_v2_egale.py", - "vllm_ascend/worker/npu_input_batch.py", - "vllm_ascend/worker/v2/aclgraph_utils.py", - "vllm_ascend/worker/v2/attn_utils.py", - "vllm_ascend/worker/v2/model_runner.py", - "vllm_ascend/worker/v2/sample/gumbel.py", - "vllm_ascend/worker/v2/sample/penalties.py", - "vllm_ascend/worker/v2/sample/sampler.py", - "vllm_ascend/worker/v2/spec_decode/__init__.py", - "vllm_ascend/worker/v2/spec_decode/eagle.py", - "vllm_ascend/worker/v2/states.py", - "vllm_ascend/worker/worker.py", - "vllm_ascend/xlite/xlite.py", - "vllm_ascend/xlite/xlite_model_runner.py" - ] - }, - { - "commit": "65b7f716e6cf2d9c9d9634af2c10926f6af98ce7", - "subject": "[Lint]Style: Convert `vllm-ascend/` to ruff format(Batch #11) (#6176)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "pyproject.toml", - "vllm_ascend/ops/fused_moe/comm_utils.py", - "vllm_ascend/ops/fused_moe/experts_selector.py", - "vllm_ascend/ops/fused_moe/fused_moe.py", - "vllm_ascend/ops/fused_moe/moe_comm_method.py", - "vllm_ascend/ops/fused_moe/moe_mlp.py", - "vllm_ascend/ops/fused_moe/prepare_finalize.py", - "vllm_ascend/ops/fused_moe/token_dispatcher.py" - ] - }, - { - "commit": "4fb3d5e1b2588e91d3c00a434e1eb3daae220a48", - "subject": "[Lint]Style: Convert `vllm-ascend/` to ruff format(Batch #8) (#6129)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "pyproject.toml", - "vllm_ascend/ops/__init__.py", - "vllm_ascend/ops/activation.py", - "vllm_ascend/ops/flashcomm2_oshard_manager.py", - "vllm_ascend/ops/layernorm.py", - "vllm_ascend/ops/mla.py", - "vllm_ascend/ops/mm_encoder_attention.py", - "vllm_ascend/ops/register_custom_ops.py", - "vllm_ascend/ops/vocab_parallel_embedding.py", - "vllm_ascend/ops/weight_prefetch.py", - "vllm_ascend/spec_decode/__init__.py", - "vllm_ascend/spec_decode/eagle_proposer.py", - "vllm_ascend/spec_decode/interface.py", - "vllm_ascend/spec_decode/medusa_proposer.py", - "vllm_ascend/spec_decode/mtp_proposer.py", - "vllm_ascend/spec_decode/ngram_proposer.py", - "vllm_ascend/spec_decode/suffix_proposer.py" - ] - }, - { - "commit": "99aedaff63299648a6f8f440e84dfdb440bae82f", - "subject": "[Lint]Style: Convert `vllm-ascend/` to ruff format(Batch #7) (#6023)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "pyproject.toml", - "vllm_ascend/quantization/__init__.py", - "vllm_ascend/quantization/compressed_tensors_config.py", - "vllm_ascend/quantization/method_adapters.py", - "vllm_ascend/quantization/methods/__init__.py", - "vllm_ascend/quantization/methods/base.py", - "vllm_ascend/quantization/methods/registry.py", - "vllm_ascend/quantization/methods/w4a16.py", - "vllm_ascend/quantization/methods/w4a4_flatquant.py", - "vllm_ascend/quantization/methods/w4a4_laos_dynamic.py", - "vllm_ascend/quantization/methods/w4a8.py", - "vllm_ascend/quantization/methods/w8a16.py", - "vllm_ascend/quantization/methods/w8a8_dynamic.py", - "vllm_ascend/quantization/methods/w8a8_mxfp8.py", - "vllm_ascend/quantization/methods/w8a8_pdmix.py", - "vllm_ascend/quantization/methods/w8a8_static.py", - "vllm_ascend/quantization/modelslim_config.py", - "vllm_ascend/sample/rejection_sampler.py", - "vllm_ascend/sample/sampler.py", - "vllm_ascend/worker/block_table.py" - ] - }, - { - "commit": "85e33941e848e5a273b243fad0eb685fbfad2a02", - "subject": "[Feat.]: 310p support MOE models (#6530)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/_310p/fused_moe/__init__.py", - "vllm_ascend/_310p/fused_moe/experts_selector.py", - "vllm_ascend/_310p/fused_moe/fused_moe.py", - "vllm_ascend/_310p/fused_moe/moe_comm_method.py", - "vllm_ascend/_310p/fused_moe/token_dispatcher.py", - "vllm_ascend/ascend_forward_context.py", - "vllm_ascend/ops/fused_moe/fused_moe.py", - "vllm_ascend/ops/fused_moe/moe_mlp.py", - "vllm_ascend/ops/rotary_embedding.py", - "vllm_ascend/utils.py" - ] - }, - { - "commit": "8e66299bf16a7b48cf85b080dc3f8927144d5add", - "subject": "[Bugfix] Fix the incorrect use of the output parameter in _forward_fia_slidingwindow (#6469)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/attention/attention_v1.py" - ] - }, - { - "commit": "922e5c163b75850049fab99ba541a07a95f26cb4", - "subject": "[main2main] upgrade vllm main 0202 (#6560)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/_pre_commit.yml", - ".github/workflows/bot_pr_create.yaml", - ".github/workflows/dockerfiles/Dockerfile.lint", - ".github/workflows/pr_test_full.yaml", - ".github/workflows/pr_test_light.yaml", - ".github/workflows/schedule_codecov_refresh.yaml", - "docs/source/community/versioning_policy.md", - "tests/e2e/conftest.py", - "tests/ut/eplb/core/test_eplb_utils.py", - "tests/ut/ops/test_mla.py", - "tests/ut/quantization/test_modelslim_config.py", - "tests/ut/spec_decode/test_eagle_proposer.py", - "tests/ut/spec_decode/test_mtp_proposer.py", - "vllm_ascend/attention/mla_v1.py", - "vllm_ascend/attention/sfa_v1.py", - "vllm_ascend/compilation/npugraph_ex_passes/graphex_qknorm_rope_fusion_pass.py", - "vllm_ascend/compilation/passes/qknorm_rope_fusion_pass.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/cpu_offload/cpu_offload_connector.py", - "vllm_ascend/kv_offload/cpu_npu.py", - "vllm_ascend/ops/fused_moe/fused_moe.py", - "vllm_ascend/ops/mla.py", - "vllm_ascend/patch/worker/__init__.py", - "vllm_ascend/patch/worker/patch_huanyuan_vl.py", - "vllm_ascend/patch/worker/patch_qwen3_next_mtp.py", - "vllm_ascend/quantization/modelslim_config.py", - "vllm_ascend/spec_decode/eagle_proposer.py", - "vllm_ascend/spec_decode/mtp_proposer.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "804a9ec4e6365aa484e3f9db61e226598b406ee6", - "subject": "[Fusion] Add rmsnorm dynamic quant fusion pass (#6274)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/compilation/passes/norm_quant_fusion_pass.py" - ] - }, - { - "commit": "fa56abea9faf2d755ca4a800f59666bcf7dd34a6", - "subject": "[bugfix][npugraph_ex]duplicate pattern issue (#6513)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/compilation/npugraph_ex_passes/graphex_allreduce_rmsnorm_fusion_pass.py", - "vllm_ascend/compilation/npugraph_ex_passes/graphex_norm_quant_fusion_pass.py", - "vllm_ascend/compilation/npugraph_ex_passes/graphex_qknorm_rope_fusion_pass.py", - "vllm_ascend/compilation/npugraph_ex_passes/utils/npugraph_ex_utils_check.py" - ] - }, - { - "commit": "7b3921c498d64081e507e03eef6edba15e8428b1", - "subject": "[bugfix][npugraph_ex]add the extra check for allreduce rmsnorm fusion pass (#6430)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/compilation/npugraph_ex_passes/graphex_allreduce_rmsnorm_fusion_pass.py" - ] - }, - { - "commit": "a80e524fbcaf58576ba7bfe56365a4259a24b692", - "subject": "[Quant] GLM4.7-Flash Support W8A8 (#6492)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/quantization/modelslim_config.py" - ] - }, - { - "commit": "b1de6cbb314a2c140407718553aeefcab2989484", - "subject": "[Bugfix][CI]Add qwen3Next MTP+Full Decode (#6047)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/e2e/multicard/4-cards/spec_decode/test_mtp_qwen3_next.py", - "vllm_ascend/patch/worker/patch_qwen3_next.py" - ] - }, - { - "commit": "39e77fb9e475ac6587c3c5132dbc0a354e11563c", - "subject": "[Feat.]: support 310p w8a8 (#6454)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/workflows/_e2e_test.yaml", - "tests/e2e/310p/test_offline_inference_w8a8_310p.py", - "vllm_ascend/_310p/quantization/__init__.py", - "vllm_ascend/_310p/quantization/methods/__init__.py", - "vllm_ascend/_310p/quantization/methods/registry.py", - "vllm_ascend/_310p/quantization/methods/w8a8_static.py", - "vllm_ascend/_310p/quantization/modelslim_config.py", - "vllm_ascend/platform.py", - "vllm_ascend/quantization/methods/w8a8_static.py" - ] - }, - { - "commit": "79803932e23ef2a09fbc0bc3dfee9a29eb302044", - "subject": "[Kernel] Add AscendC fused op transpose_kv_cache_by_block to speed up GQA transfer (#6366)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "csrc/build_aclnn.sh", - "csrc/torch_binding.cpp", - "csrc/torch_binding_meta.cpp", - "csrc/transpose_kv_cache_by_block/op_host/CMakeLists.txt", - "csrc/transpose_kv_cache_by_block/op_host/transpose_kv_cache_by_block_def.cpp", - "csrc/transpose_kv_cache_by_block/op_host/transpose_kv_cache_by_block_proto.cpp", - "csrc/transpose_kv_cache_by_block/op_host/transpose_kv_cache_by_block_tiling.cpp", - "csrc/transpose_kv_cache_by_block/op_host/transpose_kv_cache_by_block_tiling.h", - "csrc/transpose_kv_cache_by_block/op_kernel/common.h", - "csrc/transpose_kv_cache_by_block/op_kernel/full_load.h", - "csrc/transpose_kv_cache_by_block/op_kernel/general.h", - "csrc/transpose_kv_cache_by_block/op_kernel/transpose_kv_cache_by_block.cpp", - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_transpose_kv_cache_by_block.py", - "vllm_ascend/distributed/kv_transfer/kv_p2p/mooncake_connector.py", - "vllm_ascend/envs.py" - ] - }, - { - "commit": "dffac6db733820862238ef5a25057c3d4135ae54", - "subject": "[Refactor] Add expert processed token count output for DispatchFFNCombine/DispatchFFNCombineBF16 (#6402)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "csrc/dispatch_ffn_combine/op_host/aclnn_dispatch_ffn_combine.cpp", - "csrc/dispatch_ffn_combine/op_host/aclnn_dispatch_ffn_combine.h", - "csrc/dispatch_ffn_combine/op_host/dispatch_ffn_combine_def.cpp", - "csrc/dispatch_ffn_combine/op_kernel/dispatch_ffn_combine.cpp", - "csrc/dispatch_ffn_combine/op_kernel/dispatch_ffn_combine.h", - "csrc/dispatch_ffn_combine/op_kernel/dispatch_ffn_combine_kernel.hpp", - "csrc/dispatch_ffn_combine_bf16/op_host/aclnn_dispatch_ffn_combine_bf16.cpp", - "csrc/dispatch_ffn_combine_bf16/op_host/aclnn_dispatch_ffn_combine_bf16.h", - "csrc/dispatch_ffn_combine_bf16/op_host/dispatch_ffn_combine_bf16_def.cpp", - "csrc/dispatch_ffn_combine_bf16/op_kernel/dispatch_ffn_combine_bf16.cpp", - "csrc/dispatch_ffn_combine_bf16/op_kernel/dispatch_ffn_combine_bf16.h", - "csrc/dispatch_ffn_combine_bf16/op_kernel/dispatch_ffn_combine_bf16_kernel.hpp", - "csrc/dispatch_ffn_combine_bf16/op_kernel/utils/block_epilogue_pertoken_swiglu.hpp", - "csrc/torch_binding.cpp", - "csrc/torch_binding_meta.cpp", - "tests/e2e/nightly/single_node/ops/multicard_ops_a3/test_dispatch_ffn_combine.py", - "tests/e2e/nightly/single_node/ops/multicard_ops_a3/test_dispatch_ffn_combine_bf16.py", - "vllm_ascend/ops/fused_moe/moe_comm_method.py" - ] - }, - { - "commit": "45a573cff1e6ac1ffea571242eacce955bb6f087", - "subject": "[Quantization][Feature] Support compressed tensors moe w4a8 dynamic weight (#5889)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/workflows/misc/model_list.json", - "examples/quantization/llm-compressor/w4a8_dynamic_moe.py", - "tests/e2e/multicard/2-cards/test_quantization.py", - "vllm_ascend/quantization/compressed_tensors_config.py", - "vllm_ascend/quantization/methods/w4a8.py" - ] - }, - { - "commit": "eeedf7c5039cbcea62d187a0396e580df667e66a", - "subject": "[Main2Main][Deps][Misc] Upgrade vLLM to v0.15.0 (#6470)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/workflows/_e2e_nightly_multi_node.yaml", - ".github/workflows/bot_pr_create.yaml", - ".github/workflows/dockerfiles/Dockerfile.lint", - ".github/workflows/pr_test_full.yaml", - ".github/workflows/pr_test_light.yaml", - ".github/workflows/schedule_codecov_refresh.yaml", - ".github/workflows/schedule_nightly_test_a2.yaml", - ".github/workflows/schedule_test_benchmarks.yaml", - "Dockerfile", - "Dockerfile.310p", - "Dockerfile.310p.openEuler", - "Dockerfile.a3", - "Dockerfile.a3.openEuler", - "Dockerfile.openEuler", - "docs/source/community/versioning_policy.md", - "docs/source/conf.py", - "tests/e2e/singlecard/spec_decode/test_mtp_eagle_correctness.py", - "tests/ut/attention/test_mla_v1.py", - "tests/ut/attention/test_sfa_v1.py", - "tests/ut/eplb/core/test_eplb_utils.py", - "vllm_ascend/ascend_forward_context.py", - "vllm_ascend/attention/attention_v1.py", - "vllm_ascend/attention/mla_v1.py", - "vllm_ascend/attention/sfa_v1.py", - "vllm_ascend/ops/fused_moe/fused_moe.py", - "vllm_ascend/ops/mm_encoder_attention.py", - "vllm_ascend/patch/platform/__init__.py", - "vllm_ascend/patch/platform/patch_multiproc_executor.py", - "vllm_ascend/spec_decode/eagle_proposer.py", - "vllm_ascend/spec_decode/mtp_proposer.py", - "vllm_ascend/worker/v2/aclgraph_utils.py", - "vllm_ascend/worker/v2/attn_utils.py" - ] - }, - { - "commit": "347eb36a590dfbac0814bd43f20196d96211a6b7", - "subject": "[Lint]Style: Convert `vllm-ascend/` to ruff format(Batch #9) (#6135)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "pyproject.toml", - "vllm_ascend/worker/model_runner_v1.py", - "vllm_ascend/worker/pcp_utils.py" - ] - }, - { - "commit": "46cee945b31ea5bafff95be62bac5ae52d3a307e", - "subject": "[doc][npugraph_ex]add npugraph_ex introduction doc (#6306)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/developer_guide/feature_guide/index.md", - "docs/source/developer_guide/feature_guide/npugraph_ex.md", - "docs/source/user_guide/feature_guide/index.md", - "docs/source/user_guide/feature_guide/npugraph_ex.md" - ] - }, - { - "commit": "b2857de43fdbfad145380f857fe1f9d31bd99f64", - "subject": "[ST]Add e2e test for Npugraphex_pass (#6388)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/scripts/config.yaml", - "tests/e2e/singlecard/compile/test_graphex_norm_quant_fusion.py", - "tests/e2e/singlecard/compile/test_graphex_qknorm_rope_fusion.py" - ] - }, - { - "commit": "e35f304419825fbf92acf772bf8d7bbe4e43d167", - "subject": "[CI] Auto partition for test cases (#6379)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/READMD.md", - ".github/workflows/_e2e_test.yaml", - ".github/workflows/pr_test_full.yaml", - ".github/workflows/scripts/ci_utils.py", - ".github/workflows/scripts/config.yaml", - ".github/workflows/scripts/run_suite.py", - ".pre-commit-config.yaml" - ] - }, - { - "commit": "39f8af9d96914e2659239046796b13a31215a2e9", - "subject": "[Main2Main][BugFix] Add shared_experts check for AscendSharedFusedMoE (#6335)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/ops/fused_moe/fused_moe.py" - ] - }, - { - "commit": "c498cea22d1aeff9ddfa376db728c616e553628a", - "subject": "[refactor] refactor excute_model and _dymmy_run method (#6043)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/worker/test_pcp_manager.py", - "vllm_ascend/attention/sfa_v1.py", - "vllm_ascend/worker/model_runner_v1.py", - "vllm_ascend/worker/pcp_utils.py" - ] - }, - { - "commit": "54e8389f8e54584358b81caa0a7f35bafc9e059a", - "subject": "[Graph][Fusion] Add MatmulAllReduceAddRMSNorm graph fusion for npugraph_ex. (#6006)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "docs/source/user_guide/configuration/additional_config.md", - "vllm_ascend/ascend_config.py", - "vllm_ascend/compilation/npu_graph_ex_pass_manager.py", - "vllm_ascend/compilation/npugraph_ex_passes/graphex_allreduce_rmsnorm_fusion_pass.py", - "vllm_ascend/compilation/passes/allreduce_rmsnorm_fusion_pass.py" - ] - }, - { - "commit": "5e34c70ffcd2adb4448e836415cf440089bdd039", - "subject": "[Misc] Removes unnecessary graph size re-initialization (#6280)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/test_platform.py", - "vllm_ascend/platform.py", - "vllm_ascend/utils.py" - ] - }, - { - "commit": "fea197ad50556e40f75372368288505cc0353b78", - "subject": "[Main2Main] Upgrade vllm commit to 0123 (#6169)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/bot_pr_create.yaml", - ".github/workflows/dockerfiles/Dockerfile.lint", - ".github/workflows/pr_test_full.yaml", - ".github/workflows/pr_test_light.yaml", - ".github/workflows/schedule_codecov_refresh.yaml", - "docs/source/community/versioning_policy.md", - "tests/e2e/multicard/2-cards/test_aclgraph_capture_replay.py", - "tests/e2e/multicard/2-cards/test_qwen3_performance.py", - "tests/ut/attention/test_mla_v1.py", - "tests/ut/attention/test_sfa_v1.py", - "tests/ut/eplb/core/test_eplb_utils.py", - "tests/ut/spec_decode/test_eagle_proposer.py", - "tests/ut/spec_decode/test_mtp_proposer.py", - "vllm_ascend/ascend_forward_context.py", - "vllm_ascend/attention/mla_v1.py", - "vllm_ascend/attention/sfa_v1.py", - "vllm_ascend/patch/platform/patch_multiproc_executor.py", - "vllm_ascend/patch/worker/patch_v2_egale.py", - "vllm_ascend/spec_decode/eagle_proposer.py", - "vllm_ascend/spec_decode/mtp_proposer.py", - "vllm_ascend/worker/model_runner_v1.py", - "vllm_ascend/worker/v2/aclgraph_utils.py", - "vllm_ascend/worker/v2/attn_utils.py", - "vllm_ascend/worker/v2/sample/penalties.py", - "vllm_ascend/worker/v2/sample/sampler.py" - ] - }, - { - "commit": "29fb27d3bb7a467690ec5d59b2fa857f994830c8", - "subject": "BugFix: Fix moe_load accumulation error in ACL graph mode (#6182)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/ops/fused_moe/fused_moe.py" - ] - }, - { - "commit": "b390e0ef78afb31d5ee26fdd26e4f33e89d645b3", - "subject": "[Bugfix] Fix PP+PCP and PP+flashcomm1 bugs (#5416)", - "topics": [ - "communication" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "611e223b7d495bc7ddac3e7ca44a47935bfa79cb", - "subject": "[EPLB][Bugfix] EPLB support fp/bf16 (#5531)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/eplb/adaptor/test_abstract_adaptor.py", - "tests/ut/eplb/adaptor/test_vllm_adaptor.py", - "vllm_ascend/eplb/adaptor/vllm_adaptor.py", - "vllm_ascend/ops/fused_moe/fused_moe.py" - ] - }, - { - "commit": "c26ad78f863f610c3f494df2d37a5fb4fc71dd3f", - "subject": "[CI][lint] Add rule `codespell` back (#6236)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - ".github/CODEOWNERS", - ".github/ISSUE_TEMPLATE/110-user-story.yml", - ".github/workflows/_schedule_image_build.yaml", - ".pre-commit-config.yaml", - "csrc/add_rms_norm_bias/op_host/add_rms_norm_bias_tiling.cpp", - "csrc/build_aclnn.sh", - "csrc/dispatch_ffn_combine/op_kernel/moe_init_routing_quant_v2/moe_v2_expert_token_out.h", - "csrc/dispatch_ffn_combine/op_kernel/utils/copy_gm_to_l1_custom.hpp", - "csrc/dispatch_ffn_combine_bf16/op_kernel/moe_init_routing_v2/moe_v2_expert_token_out.h", - "csrc/dispatch_ffn_combine_bf16/op_kernel/utils/copy_gm_to_l1_custom.hpp", - "csrc/dispatch_layout/op_host/aclnn_dispatch_layout.h", - "csrc/grouped_matmul_swiglu_quant_weight_nz_tensor_list/op_kernel/grouped_matmul_swiglu_quant_weight_nz_tensor_list.h", - "csrc/lightning_indexer/op_host/lightning_indexer_tiling.cpp", - "csrc/moe_combine_normal/op_host/aclnn_moe_combine_normal.h", - "csrc/moe_combine_normal/op_host/moe_combine_normal_tiling.cpp", - "csrc/moe_gating_top_k/op_host/moe_gating_top_k_proto.h", - "csrc/moe_gating_top_k/op_host/moe_gating_top_k_tiling.cpp", - "csrc/moe_gating_top_k/op_host/moe_gating_top_k_tiling_arch35.cpp", - "csrc/moe_gating_top_k/tiling_base/tiling_key.h", - "csrc/moe_gating_top_k/tiling_base/tiling_type.h", - "csrc/moe_init_routing_custom/op_host/moe_init_routing_custom_tiling.cpp", - "csrc/moe_init_routing_custom/op_kernel/moe_custom_full_load_dynamic_quant.h", - "csrc/moe_init_routing_custom/op_kernel/moe_custom_full_load_static_quant.h", - "csrc/moe_init_routing_custom/op_kernel/moe_custom_full_load_unquantized.h", - "csrc/notify_dispatch/op_host/aclnn_notify_dispatch.h", - "csrc/sparse_flash_attention/op_host/sparse_flash_attention_tiling.cpp", - "csrc/utils/inc/tiling/tiling_type.h", - "docs/source/developer_guide/feature_guide/context_parallel.md", - "docs/source/tutorials/DeepSeek-V3.1.md", - "docs/source/user_guide/release_notes.md", - "examples/external_online_dp/README.md", - "typos.toml", - "vllm_ascend/_310p/worker_310p.py" - ] - }, - { - "commit": "4e3919e965f531a2ac8343821fdf5f1f617911ad", - "subject": "Reapply \"[Refactor] Unify full-graph parameter update logic (#6041)\" (#6227) (#6231)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/compilation/test_acl_graph.py", - "tests/ut/spec_decode/test_eagle_proposer.py", - "vllm_ascend/attention/attention_v1.py", - "vllm_ascend/attention/context_parallel/attention_cp.py", - "vllm_ascend/attention/context_parallel/mla_cp.py", - "vllm_ascend/attention/mla_v1.py", - "vllm_ascend/compilation/acl_graph.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/cpu_offload/cpu_offload_connector.py", - "vllm_ascend/spec_decode/eagle_proposer.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "95649344aa13684fbe01a3c375b201d59a9bfa29", - "subject": "Revert \"[Refactor] Unify full-graph parameter update logic (#6041)\" (#6227)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/compilation/test_acl_graph.py", - "tests/ut/spec_decode/test_eagle_proposer.py", - "vllm_ascend/attention/attention_v1.py", - "vllm_ascend/attention/context_parallel/attention_cp.py", - "vllm_ascend/attention/context_parallel/mla_cp.py", - "vllm_ascend/attention/mla_v1.py", - "vllm_ascend/compilation/acl_graph.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/cpu_offload/cpu_offload_connector.py", - "vllm_ascend/spec_decode/eagle_proposer.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "7799c4ca3be0f6d7541b014a1a4dfbcccb33d8b2", - "subject": "[Fusion] change fusion env variable (#6201)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "docs/source/user_guide/configuration/additional_config.md", - "vllm_ascend/ascend_config.py" - ] - }, - { - "commit": "7faa6878a664f653469e4f77e981286d086b5570", - "subject": "[Lint]Style: Convert `vllm-ascend/` to ruff format(Batch #3) (#5978)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "pyproject.toml", - "vllm_ascend/attention/context_parallel/mla_cp.py", - "vllm_ascend/attention/mla_v1.py", - "vllm_ascend/attention/sfa_v1.py", - "vllm_ascend/core/recompute_scheduler.py", - "vllm_ascend/core/scheduler_dynamic_batch.py", - "vllm_ascend/distributed/device_communicators/npu_communicator.py", - "vllm_ascend/distributed/device_communicators/pyhccl.py", - "vllm_ascend/distributed/device_communicators/pyhccl_wrapper.py" - ] - }, - { - "commit": "5b746f3e831bfdcc5fed9acb2f35c42d3f63b42e", - "subject": "[Inductor]change pass to adapt to new addrmsnormBias operator (#6094)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/e2e/singlecard/compile/test_norm_quant_fusion.py", - "vllm_ascend/compilation/passes/norm_quant_fusion_pass.py" - ] - }, - { - "commit": "8966a99710a4f67995d8f50d875f4e5298e14d8f", - "subject": "[Refactor] Unify full-graph parameter update logic (#6041)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/compilation/test_acl_graph.py", - "tests/ut/spec_decode/test_eagle_proposer.py", - "vllm_ascend/attention/attention_v1.py", - "vllm_ascend/attention/context_parallel/attention_cp.py", - "vllm_ascend/attention/context_parallel/mla_cp.py", - "vllm_ascend/attention/mla_v1.py", - "vllm_ascend/compilation/acl_graph.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/cpu_offload/cpu_offload_connector.py", - "vllm_ascend/spec_decode/eagle_proposer.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "717d299ae5a74c43ed142975f288fef1b4d7983e", - "subject": "[BugFix]bug fix for dispatch_ffn_combine (#6156)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "csrc/dispatch_ffn_combine/op_kernel/dispatch_ffn_combine_kernel.hpp", - "csrc/dispatch_ffn_combine/op_kernel/utils/const_args.hpp" - ] - }, - { - "commit": "44a4ff6960b9d4edbfd8df52695cdb1655009f39", - "subject": "[main][BugFix] Avoided a bug of `torch_npu.npu_mm_reduce_scatter_base` when sp size >= 16 (#6168)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/ascend_forward_context.py" - ] - }, - { - "commit": "6c73b88dd696d2e0417f50458301b7a605efaf9f", - "subject": "[CI] Enable FLASHCOMM1 with layer_sharding and FULL_DECODE_ONLY in ds32 testing (#6115)", - "topics": [ - "communication" - ], - "runtime_patch_match": false, - "paths": [ - "tests/e2e/multicard/2-cards/test_offline_inference_distributed.py", - "tests/e2e/nightly/multi_node/config/DeepSeek-V3_2-W8A8-A3-dual-nodes.yaml", - "tests/e2e/nightly/single_node/models/test_deepseek_v3_2_w8a8.py" - ] - }, - { - "commit": "78af0c30a39800496a1b2415225c2f1515a20bca", - "subject": "[Lint]Style: Convert `vllm-ascend/` to ruff format(Batch #12) (#6177)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "pyproject.toml", - "vllm_ascend/ops/triton/activation/swiglu_quant.py", - "vllm_ascend/ops/triton/batch_invariant/matmul.py", - "vllm_ascend/ops/triton/batch_invariant/mean.py", - "vllm_ascend/ops/triton/batch_invariant/rmsnorm.py", - "vllm_ascend/ops/triton/fla/chunk.py", - "vllm_ascend/ops/triton/fla/chunk_delta_h.py", - "vllm_ascend/ops/triton/fla/chunk_o.py", - "vllm_ascend/ops/triton/fla/chunk_scaled_dot_kkt.py", - "vllm_ascend/ops/triton/fla/cumsum.py", - "vllm_ascend/ops/triton/fla/fused_qkvzba_split_reshape.py", - "vllm_ascend/ops/triton/fla/l2norm.py", - "vllm_ascend/ops/triton/fla/layernorm_guard.py", - "vllm_ascend/ops/triton/fla/sigmoid_gating.py", - "vllm_ascend/ops/triton/fla/solve_tril.py", - "vllm_ascend/ops/triton/fla/utils.py", - "vllm_ascend/ops/triton/fla/wy_fast.py", - "vllm_ascend/ops/triton/fused_gdn_gating.py", - "vllm_ascend/ops/triton/layernorm_gated.py", - "vllm_ascend/ops/triton/linearnorm/split_qkv_rmsnorm_rope.py", - "vllm_ascend/ops/triton/mamba/causal_conv1d.py", - "vllm_ascend/ops/triton/reject_sample.py", - "vllm_ascend/ops/triton/rope.py", - "vllm_ascend/ops/triton/spec_decode/utils.py", - "vllm_ascend/ops/triton/triton_utils.py" - ] - }, - { - "commit": "749e24f81eeda8b6750fe98728af2487cf9a564d", - "subject": "[bugfix] align max_num_batched_tokens with tp*pcp when using FLASHCOMM1 (#6000)", - "topics": [ - "communication" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/attention/test_mla_cp.py", - "tests/ut/attention/test_mla_v1.py", - "tests/ut/attention/test_sfa_v1.py", - "vllm_ascend/ascend_config.py" - ] - }, - { - "commit": "a69ef10c3a67c2b6c0ff83979a3da4d0c2784dc7", - "subject": "[Refactor] Quantization Module Refactor (#5738)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "docs/source/developer_guide/feature_guide/quantization.md", - "tests/ut/ops/test_fused_moe.py", - "tests/ut/quantization/test_modelslim_config.py", - "tests/ut/quantization/test_utils.py", - "tests/ut/quantization/test_w4a16.py", - "tests/ut/quantization/test_w4a4_flatquant_dynamic.py", - "tests/ut/quantization/test_w4a8_dynamic.py", - "tests/ut/quantization/test_w8a16.py", - "tests/ut/quantization/test_w8a8.py", - "tests/ut/quantization/test_w8a8_dynamic.py", - "tests/ut/test_platform.py", - "vllm_ascend/attention/mla_v1.py", - "vllm_ascend/attention/sfa_v1.py", - "vllm_ascend/ops/fused_moe/fused_moe.py", - "vllm_ascend/ops/linear_op.py", - "vllm_ascend/platform.py", - "vllm_ascend/quantization/__init__.py", - "vllm_ascend/quantization/compressed_tensors/__init__.py", - "vllm_ascend/quantization/compressed_tensors_config.py", - "vllm_ascend/quantization/method_adapters.py", - "vllm_ascend/quantization/methods/__init__.py", - "vllm_ascend/quantization/methods/base.py", - "vllm_ascend/quantization/methods/registry.py", - "vllm_ascend/quantization/methods/w4a16.py", - "vllm_ascend/quantization/methods/w4a4_flatquant.py", - "vllm_ascend/quantization/methods/w4a4_laos_dynamic.py", - "vllm_ascend/quantization/methods/w4a8.py", - "vllm_ascend/quantization/methods/w8a16.py", - "vllm_ascend/quantization/methods/w8a8_dynamic.py", - "vllm_ascend/quantization/methods/w8a8_mxfp8.py", - "vllm_ascend/quantization/methods/w8a8_pdmix.py", - "vllm_ascend/quantization/methods/w8a8_static.py", - "vllm_ascend/quantization/modelslim_config.py", - "vllm_ascend/quantization/quant_config.py", - "vllm_ascend/quantization/utils.py", - "vllm_ascend/quantization/w8a8_pdmix.py" - ] - }, - { - "commit": "819a4459ce4e2a42755f9cb9671c8c9ce0034092", - "subject": "Drop vLLM 0.13.0 support (#6069)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/_e2e_nightly_multi_node.yaml", - ".github/workflows/nightly_test_a2.yaml", - ".github/workflows/nightly_test_a3.yaml", - ".github/workflows/pr_test_full.yaml", - ".github/workflows/pr_test_light.yaml", - ".github/workflows/schedule_test_benchmarks.yaml", - "Dockerfile", - "Dockerfile.310p", - "Dockerfile.310p.openEuler", - "Dockerfile.a3", - "Dockerfile.a3.openEuler", - "Dockerfile.openEuler", - "docs/source/community/versioning_policy.md", - "docs/source/conf.py", - "tests/e2e/multicard/4-cards/long_sequence/test_mtp.py", - "tests/ut/attention/test_mla_v1.py", - "tests/ut/test_platform.py", - "tests/ut/worker/test_worker_v1.py", - "vllm_ascend/attention/attention_v1.py", - "vllm_ascend/attention/context_parallel/attention_cp.py", - "vllm_ascend/attention/context_parallel/mla_cp.py", - "vllm_ascend/attention/mla_v1.py", - "vllm_ascend/attention/sfa_v1.py", - "vllm_ascend/distributed/kv_transfer/kv_p2p/mooncake_connector.py", - "vllm_ascend/distributed/kv_transfer/kv_p2p/mooncake_layerwise_connector.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/ascend_store/ascend_store_connector.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/cpu_offload/cpu_offload_connector.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/ucm_connector.py", - "vllm_ascend/kv_offload/cpu_npu.py", - "vllm_ascend/kv_offload/npu.py", - "vllm_ascend/ops/fused_moe/fused_moe.py", - "vllm_ascend/ops/mla.py", - "vllm_ascend/ops/mm_encoder_attention.py", - "vllm_ascend/ops/rotary_embedding.py", - "vllm_ascend/ops/triton/mamba/causal_conv1d.py", - "vllm_ascend/patch/platform/__init__.py", - "vllm_ascend/patch/worker/patch_qwen3_next.py", - "vllm_ascend/worker/model_runner_v1.py", - "vllm_ascend/worker/worker.py" - ] - }, - { - "commit": "7f91ac264919fe6816907e0fd6c97d581bed7174", - "subject": "[CP&SP] Integrate FIA operator in mla_cp._forward_decode (#5641)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/attention/test_mla_cp.py", - "tests/ut/compilation/test_acl_graph.py", - "vllm_ascend/attention/context_parallel/mla_cp.py", - "vllm_ascend/compilation/acl_graph.py" - ] - }, - { - "commit": "1402cf6874e48c1a345834e780c32d1db2a7fb8f", - "subject": "[Graph][Fusion] Add QKVNormRope and QKVNormRopeWithBias (#5721)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/compilation/test_add_rms_norm_quant.py", - "tests/ut/compilation/test_npugraph_ex_utils_check.py", - "vllm_ascend/compilation/compiler_interface.py", - "vllm_ascend/compilation/npu_graph_ex_pass_manager.py", - "vllm_ascend/compilation/npugraph_ex_passes/add_rms_norm_quant.py", - "vllm_ascend/compilation/npugraph_ex_passes/graphex_norm_quant_fusion_pass.py", - "vllm_ascend/compilation/npugraph_ex_passes/graphex_qknorm_rope_fusion_pass.py", - "vllm_ascend/compilation/npugraph_ex_passes/utils/__init__.py", - "vllm_ascend/compilation/npugraph_ex_passes/utils/npugraph_ex_utils_check.py", - "vllm_ascend/platform.py" - ] - }, - { - "commit": "1d3544c8875ed66c7dd7c1235c0825bfe21ad730", - "subject": "[BugFix]converting pa get_workspace back to capturing (#5833)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/compilation/acl_graph.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "08d7014874f4e9d3aa97b5c1ac45d919bd304139", - "subject": "[Feature]Enable DispatchGmmCombineDecode when eagle is moe with w8a8 or not moe [RFC: issue 5476] (#5758)", - "topics": [ - "communication", - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/utils.py" - ] - }, - { - "commit": "dd8571860d9a9db46f2d834e726d5d1cd252e0b0", - "subject": "[Feature] Support DSA-CP for Hybrid scenario (#5702)", - "topics": [ - "communication" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/attention/sfa_v1.py", - "vllm_ascend/distributed/parallel_state.py", - "vllm_ascend/distributed/utils.py", - "vllm_ascend/ops/linear_op.py" - ] - }, - { - "commit": "bec86418764bfe43bcf431e6693976fb024afa92", - "subject": "[BugFix] Fix input parameter bug of dispatch_gmm_combine_decode[RFC: issue 5476] (#5932)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/ops/fused_moe/moe_comm_method.py", - "vllm_ascend/ops/fused_moe/token_dispatcher.py" - ] - }, - { - "commit": "55b20ac63b950d0096c0a84a8aaf9f1e1343e2b4", - "subject": "[Ops] Add layernorm for qwen3Next (#5765)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/workflows/nightly_test_a2.yaml", - "vllm_ascend/ops/layernorm.py", - "vllm_ascend/ops/triton/layernorm_gated.py", - "vllm_ascend/utils.py" - ] - }, - { - "commit": "a8576ec610edf2eca9f98272d42187345277f737", - "subject": "[Refactor][EAGLE] 5/N Update attn_metadata by common_attn_metadata (#5869)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/spec_decode/eagle_proposer.py" - ] - }, - { - "commit": "f58e110afe65653c50f909f701e8ce7fec20054a", - "subject": "\u3010feat\u3011switch for fusion ops gmmswigluquant (#5992)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/test_ascend_config.py", - "vllm_ascend/ascend_config.py", - "vllm_ascend/ops/fused_moe/moe_comm_method.py", - "vllm_ascend/platform.py" - ] - }, - { - "commit": "ebb940691fcc1b6e48bc14e8d180934da4d1542f", - "subject": "[Feature] Adapt DispathGmmCombineDecode opertor to align with weight scale dtype of small operators. [RFC: issue 5476] (#5755)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "csrc/dispatch_gmm_combine_decode/op_host/dispatch_gmm_combine_decode_def.cpp", - "csrc/dispatch_gmm_combine_decode/op_kernel/dispatch_gmm_combine_decode.cpp", - "csrc/dispatch_gmm_combine_decode/op_kernel/dispatch_gmm_combine_decode.h", - "csrc/dispatch_gmm_combine_decode/op_kernel/dispatch_gmm_combine_decode/epilogue/block/block_epilogue_per_token_dequant.hpp", - "csrc/dispatch_gmm_combine_decode/op_kernel/dispatch_gmm_combine_decode/epilogue/block/block_epilogue_per_token_dequant_swiglu.h", - "csrc/dispatch_gmm_combine_decode/op_kernel/dispatch_gmm_combine_decode/gemm/kernel/grouped_matmul_slice_m_per_token_dequant_multistage_workspace.h", - "csrc/dispatch_gmm_combine_decode/op_kernel/dispatch_gmm_combine_decode/gemm/kernel/grouped_matmul_slice_m_per_token_dequant_swiglu_quant_multistage_workspace.h", - "csrc/dispatch_gmm_combine_decode/op_kernel/dispatch_gmm_combine_decode_base.h", - "tests/e2e/nightly/single_node/ops/multicard_ops_a3/test_dispatch_gmm_combine_decode.py", - "vllm_ascend/eplb/adaptor/vllm_adaptor.py", - "vllm_ascend/quantization/w8a8_dynamic.py" - ] - }, - { - "commit": "c929bd1e8d4f955a0b5434bf72ba7061068ccd14", - "subject": "[Fusion] [Graph]Add Matmul Allreduce Rmsnorm fusion Pass (#5034)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/ascend_config.py", - "vllm_ascend/compilation/graph_fusion_pass_manager.py", - "vllm_ascend/compilation/passes/allreduce_rmsnorm_fusion_pass.py", - "vllm_ascend/patch/__init__.py", - "vllm_ascend/patch/worker/__init__.py", - "vllm_ascend/patch/worker/patch_distributed.py", - "vllm_ascend/patch/worker/patch_unquantized_gemm.py", - "vllm_ascend/platform.py" - ] - }, - { - "commit": "bc1f6713e72662d881fd0238b40028ce405688f3", - "subject": "[EPLB][Bugfix] Dispatch Allgather use log2phy if enable eplb (#5933)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/ops/test_token_dispatcher.py", - "vllm_ascend/eplb/adaptor/vllm_adaptor.py", - "vllm_ascend/eplb/eplb_updator.py", - "vllm_ascend/ops/fused_moe/fused_moe.py", - "vllm_ascend/ops/fused_moe/moe_comm_method.py", - "vllm_ascend/ops/fused_moe/token_dispatcher.py" - ] - }, - { - "commit": "9fed2636cbf478ad8a57c4deed62679f2b8d2510", - "subject": "[EPLB][Nightly][Bugfix] Get expert from moe layer only (#5908)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/eplb/core/test_eplb_utils.py", - "vllm_ascend/eplb/adaptor/vllm_adaptor.py", - "vllm_ascend/eplb/core/eplb_utils.py", - "vllm_ascend/ops/fused_moe/fused_moe.py" - ] - }, - { - "commit": "329961b37534f7d57f18732509ad4385612903a7", - "subject": "[Lint]Style: Convert `vllm-ascend/` to ruff format(Batch #2) (#5977)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "pyproject.toml", - "vllm_ascend/attention/attention_mask.py", - "vllm_ascend/attention/attention_v1.py", - "vllm_ascend/attention/context_parallel/attention_cp.py", - "vllm_ascend/attention/context_parallel/common_cp.py", - "vllm_ascend/attention/context_parallel/mla_cp.py", - "vllm_ascend/attention/utils.py", - "vllm_ascend/batch_invariant.py", - "vllm_ascend/device/device_op.py", - "vllm_ascend/device_allocator/camem.py", - "vllm_ascend/envs.py" - ] - }, - { - "commit": "22f253142a9fe13de58fb925b633674ef2dbf85f", - "subject": "[Feature] Support fine-grained shared expert overlap (#5482)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/ops/test_moe_comm_method.py", - "tests/ut/ops/test_token_dispatcher.py", - "vllm_ascend/ops/fused_moe/fused_moe.py", - "vllm_ascend/ops/fused_moe/moe_comm_method.py", - "vllm_ascend/ops/fused_moe/prepare_finalize.py", - "vllm_ascend/ops/fused_moe/token_dispatcher.py", - "vllm_ascend/quantization/w4a16.py", - "vllm_ascend/quantization/w4a8_dynamic.py", - "vllm_ascend/quantization/w8a8_dynamic.py" - ] - }, - { - "commit": "1ffca8673fa460174b891a194560e4fd05eb651d", - "subject": "[Feature]: Support 310P device run qwen2.5/3 dense and qwen2.5vl models (#5776)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "CMakeLists.txt", - "tests/e2e/310p/test_offline_inference_310p.py", - "tests/e2e/310p/test_offline_inference_parallel_310p.py", - "vllm_ascend/_310p/__init__.py", - "vllm_ascend/_310p/attention/__init__.py", - "vllm_ascend/_310p/attention/attention_mask.py", - "vllm_ascend/_310p/attention/attention_v1.py", - "vllm_ascend/_310p/attention/metadata_builder.py", - "vllm_ascend/_310p/modelrunner_310p.py", - "vllm_ascend/_310p/ops/__init__.py", - "vllm_ascend/_310p/ops/activation.py", - "vllm_ascend/_310p/ops/mm_encoder_attention.py", - "vllm_ascend/_310p/ops/rotary_embedding.py", - "vllm_ascend/_310p/worker_310p.py", - "vllm_ascend/patch/platform/patch_distributed.py", - "vllm_ascend/platform.py", - "vllm_ascend/utils.py" - ] - }, - { - "commit": "52086394ae6605ce33186bdfb55a79c29bd2a519", - "subject": "[Lint]Style: Convert `vllm-ascend/compilation` to ruff format (#5912)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "pyproject.toml", - "vllm_ascend/__init__.py", - "vllm_ascend/ascend_config.py", - "vllm_ascend/ascend_forward_context.py", - "vllm_ascend/compilation/acl_graph.py", - "vllm_ascend/compilation/compiler_interface.py", - "vllm_ascend/compilation/graph_fusion_pass_manager.py", - "vllm_ascend/compilation/npugraph_ex_passes/add_rms_norm_quant.py", - "vllm_ascend/compilation/passes/norm_quant_fusion_pass.py", - "vllm_ascend/compilation/passes/qknorm_rope_fusion_pass.py", - "vllm_ascend/cpu_binding.py", - "vllm_ascend/flash_common3_context.py", - "vllm_ascend/meta_registration.py", - "vllm_ascend/platform.py", - "vllm_ascend/profiling_config.py", - "vllm_ascend/utils.py" - ] - }, - { - "commit": "da958ee386e248d0b95d8b6772d579e26aa06b56", - "subject": "[EPLB]Eplb Config Renaming (#5533)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/developer_guide/feature_guide/eplb_swift_balancer.md", - "docs/source/user_guide/configuration/additional_config.md", - "docs/source/user_guide/feature_guide/eplb_swift_balancer.md", - "tests/e2e/multicard/2-cards/test_qwen3_moe.py", - "tests/e2e/nightly/multi_node/config/DeepSeek-R1-W8A8-EPLB.yaml", - "tests/e2e/nightly/multi_node/config/Qwen3-235B-W8A8-EPLB.yaml", - "tests/e2e/nightly/single_node/models/test_deepseek_r1_0528_w8a8_eplb.py", - "tests/e2e/nightly/single_node/models/test_qwen3_235b_a22b_w8a8_eplb.py", - "tests/ut/eplb/core/test_eplb_utils.py", - "tests/ut/quantization/test_w4a16.py", - "tests/ut/quantization/test_w4a8_dynamic.py", - "tests/ut/test_ascend_config.py", - "vllm_ascend/ascend_config.py", - "vllm_ascend/ascend_forward_context.py", - "vllm_ascend/eplb/core/eplb_utils.py", - "vllm_ascend/eplb/eplb_updator.py", - "vllm_ascend/ops/fused_moe/fused_moe.py", - "vllm_ascend/quantization/w4a16.py", - "vllm_ascend/quantization/w4a8_dynamic.py", - "vllm_ascend/quantization/w8a8_dynamic.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "e67608041dc29ca0d9d41210d7f5f6eb812d397e", - "subject": "[main][BugFix]Fix DispatchGmmCombineDecode acc bug when big batch (#5808)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "csrc/dispatch_gmm_combine_decode/op_kernel/dispatch_gmm_combine_decode/gemm/kernel/grouped_matmul_slice_m_per_token_dequant_swiglu_quant_multistage_workspace.h" - ] - }, - { - "commit": "d840f153f42f291ca2078d3fe4d027efdcb3d2db", - "subject": "[Bugfix] Fix acc bug when enbale dispatch_gmm_combine_decode and eplb (#5806)", - "topics": [ - "communication", - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/ops/fused_moe/moe_comm_method.py" - ] - }, - { - "commit": "7078dff6913af3536ef501a1bb57d2dd7220b156", - "subject": "[Feature] implenment set_additional_forward_context for model runner v2 (#5720)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/platform.py" - ] - }, - { - "commit": "a88937f5cb564ace26135abfd400b9a57919b2d3", - "subject": "[bugfix](cp) replace None with zeros/inf tensor to avoid TypeError (#5837)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/e2e/multicard/4-cards/long_sequence/test_chunked_prefill.py", - "vllm_ascend/attention/context_parallel/attention_cp.py" - ] - }, - { - "commit": "2a6d95c389b91774f9fe846f44449c85478f8b4e", - "subject": "[Cleanup] Remove dead code make_attention_mask function (#5818)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/worker/v2/attn_utils.py" - ] - }, - { - "commit": "0415e694cddbe755e38e293c078d0271b2226088", - "subject": "[Quantization] Support compressed tensors moe w8a8 int8 dynamic weight (#5718)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/workflows/misc/model_list.json", - "docs/source/user_guide/feature_guide/quantization.md", - "examples/quantization/llm-compressor/w8a8_int8_dynamic_moe.py", - "tests/e2e/multicard/2-cards/test_quantization.py", - "vllm_ascend/quantization/compressed_tensors/compressed_tensors.py" - ] - }, - { - "commit": "ecf2fa482e0cb777a53da57ae70af659f7769bf0", - "subject": "[EPLB][Bugfix] Get expert map from layers (#5817)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/eplb/adaptor/test_abstract_adaptor.py", - "tests/ut/eplb/core/test_eplb_utils.py", - "vllm_ascend/eplb/adaptor/abstract_adaptor.py", - "vllm_ascend/eplb/adaptor/vllm_adaptor.py", - "vllm_ascend/eplb/core/eplb_utils.py", - "vllm_ascend/eplb/eplb_updator.py", - "vllm_ascend/ops/fused_moe/fused_moe.py" - ] - }, - { - "commit": "f7b904641ef31b214aa36fb726e8d5e3056b3b94", - "subject": "[Main2Main] Upgrade vllm commit to 0109 (#5752)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/_e2e_test.yaml", - ".github/workflows/bot_pr_create.yaml", - ".github/workflows/pr_test_full.yaml", - ".github/workflows/pr_test_light.yaml", - ".github/workflows/schedule_codecov_refresh.yaml", - "docs/source/community/versioning_policy.md", - "tests/e2e/singlecard/compile/test_norm_quant_fusion.py", - "tests/ut/attention/test_attention_cp.py", - "tests/ut/attention/test_attention_v1.py", - "tests/ut/attention/test_mla_v1.py", - "tests/ut/attention/test_sfa_v1.py", - "tests/ut/ops/test_activation.py", - "tests/ut/ops/test_layernorm.py", - "tests/ut/ops/test_rotary_embedding.py", - "tests/ut/ops/test_token_dispatcher.py", - "tests/ut/ops/test_vocab_parallel_embedding.py", - "tests/ut/worker/test_worker_v1.py", - "vllm_ascend/attention/mla_v1.py", - "vllm_ascend/ops/triton/mamba/causal_conv1d.py", - "vllm_ascend/worker/model_runner_v1.py", - "vllm_ascend/worker/worker.py" - ] - }, - { - "commit": "db12c1e2c840a98570bf8a793f2a9786999e8c3f", - "subject": "[Perf] Supports compute-communication overlap in the forward of sfa_v1 in the Sharded-CP feature. (#5701)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/attention/sfa_v1.py", - "vllm_ascend/distributed/utils.py", - "vllm_ascend/utils.py" - ] - }, - { - "commit": "09b3f9d91b162a62f5affce3a25627c8b249382a", - "subject": "[CI]Add Disaggregated PD Nightly Test for Qwen3-235B and Qwen3-VL-235B (#5502)", - "topics": [ - "communication", - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/nightly_test_a3.yaml", - "tests/e2e/nightly/multi_node/config/Qwen3-235B-disagg-pd.yaml", - "tests/e2e/nightly/multi_node/config/Qwen3-VL-235B-disagg-pd.yaml", - "vllm_ascend/ascend_config.py" - ] - }, - { - "commit": "64d29875f9492141491dacafdadc8bbcf5985e16", - "subject": "[Refactor] Replace the implementations of o_proj, q_b_proj, and kv_b_proj with custom_op for sharded CP (#5698)", - "topics": [ - "communication" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/attention/test_sfa_v1.py", - "vllm_ascend/attention/sfa_v1.py", - "vllm_ascend/ops/linear_op.py", - "vllm_ascend/utils.py" - ] - }, - { - "commit": "dc99cfdc15bfb094983e49ec14e4eaa98dee7a7a", - "subject": "[CustomOp] support TensorList for dispatchFFNCombine (#5665)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "csrc/dispatch_ffn_combine/op_host/aclnn_dispatch_ffn_combine.cpp", - "csrc/dispatch_ffn_combine/op_host/aclnn_dispatch_ffn_combine.h", - "csrc/dispatch_ffn_combine/op_host/dispatch_ffn_combine_def.cpp", - "csrc/dispatch_ffn_combine/op_host/dispatch_ffn_combine_tiling.cpp", - "csrc/dispatch_ffn_combine/op_kernel/dispatch_ffn_combine.h", - "csrc/dispatch_ffn_combine/op_kernel/dispatch_ffn_combine_kernel.hpp", - "csrc/dispatch_ffn_combine/op_kernel/dispatch_ffn_combine_tiling.h", - "csrc/dispatch_ffn_combine/op_kernel/utils/block_epilogue_pertoken_row.hpp", - "csrc/dispatch_ffn_combine/op_kernel/utils/block_epilogue_pertoken_swiglu.hpp", - "csrc/dispatch_ffn_combine/op_kernel/utils/const_args.hpp", - "csrc/dispatch_ffn_combine/op_kernel/utils/get_tensor_addr.hpp", - "csrc/dispatch_ffn_combine/op_kernel/utils/hccl_shmem.hpp", - "csrc/torch_binding.cpp", - "csrc/torch_binding_meta.cpp", - "tests/e2e/nightly/single_node/ops/multicard_ops_a3/test_dispatch_ffn_combine.py", - "vllm_ascend/ops/fused_moe/moe_comm_method.py" - ] - }, - { - "commit": "40eb3e18361a1dae229e2d8dae03538845f27471", - "subject": "[OP] Enable custom op aclnnMoeInitRoutingCustom (#5332)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/e2e/singlecard/test_aclgraph_accuracy.py", - "tests/ut/ops/test_token_dispatcher.py", - "vllm_ascend/ops/fused_moe/token_dispatcher.py" - ] - }, - { - "commit": "481138e1d2c7ad7d9818a8d276e96feabc477d1a", - "subject": "[bugfix] adapt to new implemented get_kv_cache_spec in cpuoffload connector (#4311)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/distributed/cpu_offload_connector.py", - "vllm_ascend/distributed/cpu_offload_manager/metadata.py" - ] - }, - { - "commit": "3be8e33fe9a78529e6202cafcfaaffdede414441", - "subject": "[Kernel] Add moe_gating_top_k operator support for Ascend NPU (#5579)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "csrc/build_aclnn.sh", - "csrc/moe_gating_top_k/op_host/CMakeLists.txt", - "csrc/moe_gating_top_k/op_host/error_log.h", - "csrc/moe_gating_top_k/op_host/math_util.h", - "csrc/moe_gating_top_k/op_host/moe_gating_top_k_def.cpp", - "csrc/moe_gating_top_k/op_host/moe_gating_top_k_infershape.cpp", - "csrc/moe_gating_top_k/op_host/moe_gating_top_k_proto.cpp", - "csrc/moe_gating_top_k/op_host/moe_gating_top_k_proto.h", - "csrc/moe_gating_top_k/op_host/moe_gating_top_k_tiling.cpp", - "csrc/moe_gating_top_k/op_host/moe_gating_top_k_tiling.h", - "csrc/moe_gating_top_k/op_host/moe_gating_top_k_tiling_arch35.cpp", - "csrc/moe_gating_top_k/op_host/moe_gating_top_k_tiling_base.cpp", - "csrc/moe_gating_top_k/op_kernel/common.h", - "csrc/moe_gating_top_k/op_kernel/error_log.h", - "csrc/moe_gating_top_k/op_kernel/moe_gating_top_k.cpp", - "csrc/moe_gating_top_k/op_kernel/moe_gating_top_k_apt.cpp", - "csrc/moe_gating_top_k/op_kernel/moe_gating_top_k_e_k_fullload.h", - "csrc/moe_gating_top_k/op_kernel/moe_gating_top_k_generalized.h", - "csrc/moe_gating_top_k/op_kernel/moe_gating_top_k_without_group.h", - "csrc/moe_gating_top_k/tiling_base/data_copy_transpose_tiling.h", - "csrc/moe_gating_top_k/tiling_base/data_copy_transpose_tiling_def.h", - "csrc/moe_gating_top_k/tiling_base/error_log.h", - "csrc/moe_gating_top_k/tiling_base/tiling_base.h", - "csrc/moe_gating_top_k/tiling_base/tiling_key.h", - "csrc/moe_gating_top_k/tiling_base/tiling_templates_registry.h", - "csrc/moe_gating_top_k/tiling_base/tiling_type.h", - "csrc/moe_gating_top_k/tiling_base/tiling_util.h", - "csrc/torch_binding.cpp", - "csrc/torch_binding_meta.cpp", - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_fused_moe.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_npu_moe_gating_top_k.py", - "vllm_ascend/ops/fused_moe/experts_selector.py" - ] - }, - { - "commit": "b94fc13d3fbd5d514edcef83fbf72338b96deef3", - "subject": "[BugFix][Fusion] Fix graph fusion failure problem (#5676)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/compilation/compiler_interface.py", - "vllm_ascend/compilation/graph_fusion_pass_manager.py", - "vllm_ascend/compilation/passes/norm_quant_fusion_pass.py", - "vllm_ascend/compilation/passes/qknorm_rope_fusion_pass.py", - "vllm_ascend/patch/__init__.py", - "vllm_ascend/patch/platform/__init__.py", - "vllm_ascend/patch/platform/patch_compile_backend.py", - "vllm_ascend/worker/worker.py" - ] - }, - { - "commit": "380f089fbf81fcd3987899c741b6c4196cbce2a5", - "subject": "[Refactor] Fix AttentionMaskBuilder singleton and remove redundant pcp_prefill_mask (#4870)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/attention/test_attention_v1.py", - "tests/ut/attention/test_mla_cp.py", - "tests/ut/attention/test_mla_v1.py", - "tests/ut/spec_decode/test_mtp_proposer.py", - "tests/ut/worker/test_pcp_manager.py", - "vllm_ascend/attention/attention_mask.py", - "vllm_ascend/attention/attention_v1.py", - "vllm_ascend/attention/context_parallel/attention_cp.py", - "vllm_ascend/attention/context_parallel/common_cp.py", - "vllm_ascend/attention/context_parallel/mla_cp.py", - "vllm_ascend/attention/mla_v1.py", - "vllm_ascend/attention/sfa_v1.py", - "vllm_ascend/attention/utils.py", - "vllm_ascend/compilation/acl_graph.py", - "vllm_ascend/spec_decode/eagle_proposer.py", - "vllm_ascend/spec_decode/mtp_proposer.py", - "vllm_ascend/utils.py", - "vllm_ascend/worker/model_runner_v1.py", - "vllm_ascend/worker/pcp_utils.py", - "vllm_ascend/worker/v2/attn_utils.py", - "vllm_ascend/worker/v2/model_runner.py" - ] - }, - { - "commit": "6f7a81cd9fb2b70f4e61dfa9f30e6ddbceae7438", - "subject": "[CI] cleanup single/multi-card test (#5623)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/_e2e_test.yaml", - "docs/source/developer_guide/contribution/testing.md", - "tests/e2e/multicard/2-cards/test_aclgraph_capture_replay.py", - "tests/e2e/multicard/2-cards/test_data_parallel.py", - "tests/e2e/multicard/2-cards/test_expert_parallel.py", - "tests/e2e/multicard/2-cards/test_external_launcher.py", - "tests/e2e/multicard/2-cards/test_full_graph_mode.py", - "tests/e2e/multicard/2-cards/test_ilama_lora_tp2.py", - "tests/e2e/multicard/2-cards/test_offline_inference_distributed.py", - "tests/e2e/multicard/2-cards/test_offline_weight_load.py", - "tests/e2e/multicard/2-cards/test_pipeline_parallel.py", - "tests/e2e/multicard/2-cards/test_prefix_caching.py", - "tests/e2e/multicard/2-cards/test_quantization.py", - "tests/e2e/multicard/2-cards/test_qwen3_moe.py", - "tests/e2e/multicard/2-cards/test_shared_expert_dp.py", - "tests/e2e/multicard/2-cards/test_single_request_aclgraph.py", - "tests/e2e/multicard/4-cards/long_sequence/test_accuracy.py", - "tests/e2e/multicard/4-cards/long_sequence/test_basic.py", - "tests/e2e/multicard/4-cards/long_sequence/test_chunked_prefill.py", - "tests/e2e/multicard/4-cards/long_sequence/test_mtp.py", - "tests/e2e/multicard/4-cards/spec_decode/test_mtp_qwen3_next.py", - "tests/e2e/multicard/4-cards/test_data_parallel_tp2.py", - "tests/e2e/multicard/4-cards/test_kimi_k2.py", - "tests/e2e/multicard/4-cards/test_qwen3_next.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_chunk_gated_delta_rule.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_fused_sigmoid_gating_delta_rule.py", - "tests/e2e/singlecard/compile/__init__.py", - "tests/e2e/singlecard/spec_decode/__init__.py", - "tests/e2e/singlecard/test_aclgraph_accuracy.py", - "tests/e2e/singlecard/test_quantization.py" - ] - }, - { - "commit": "cd59323e409d7e337045462f7b767c863f9cf827", - "subject": "[Bugfix] Revert pr4214 multi-stream collect expert hotpot (#5529)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/eplb/eplb_updator.py", - "vllm_ascend/eplb/utils.py", - "vllm_ascend/ops/fused_moe/fused_moe.py" - ] - }, - { - "commit": "25baf6df095f10c6af29fcafd56ad3ff3e83a9b2", - "subject": "[Feature]EPLB:Adapt DispatchGmmCombineDecode operator to eplb tensor list and expert token numbers (#5552)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "csrc/dispatch_gmm_combine_decode/op_host/aclnn_dispatch_gmm_combine_decode.cpp", - "csrc/dispatch_gmm_combine_decode/op_host/aclnn_dispatch_gmm_combine_decode.h", - "csrc/dispatch_gmm_combine_decode/op_host/dispatch_gmm_combine_decode_def.cpp", - "csrc/dispatch_gmm_combine_decode/op_host/dispatch_gmm_combine_decode_proto.cpp", - "csrc/dispatch_gmm_combine_decode/op_host/dispatch_gmm_combine_decode_tiling.cpp", - "csrc/dispatch_gmm_combine_decode/op_kernel/dispatch_gmm_combine_decode.cpp", - "csrc/dispatch_gmm_combine_decode/op_kernel/dispatch_gmm_combine_decode.h", - "csrc/dispatch_gmm_combine_decode/op_kernel/dispatch_gmm_combine_decode/gemm/kernel/grouped_matmul_slice_m_per_token_dequant_multistage_workspace.h", - "csrc/dispatch_gmm_combine_decode/op_kernel/dispatch_gmm_combine_decode/gemm/kernel/grouped_matmul_slice_m_per_token_dequant_swiglu_quant_multistage_workspace.h", - "csrc/dispatch_gmm_combine_decode/op_kernel/dispatch_gmm_combine_decode_tiling.h", - "csrc/torch_binding.cpp", - "csrc/torch_binding_meta.cpp", - "tests/e2e/nightly/single_node/ops/multicard_ops/test_dispatch_gmm_combine_decode.py", - "vllm_ascend/ascend_forward_context.py", - "vllm_ascend/envs.py", - "vllm_ascend/eplb/adaptor/vllm_adaptor.py", - "vllm_ascend/ops/fused_moe/moe_comm_method.py", - "vllm_ascend/quantization/w8a8_dynamic.py" - ] - }, - { - "commit": "11122080523853960b6f2605dd9c308f53bb9e8a", - "subject": "[Refactor] Cleanup platform (#5566)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/attention/test_mla_cp.py", - "tests/ut/device_allocator/test_camem.py", - "tests/ut/test_platform.py", - "tests/ut/worker/test_worker_v1.py", - "vllm_ascend/compilation/compiler_interface.py", - "vllm_ascend/device_allocator/camem.py", - "vllm_ascend/platform.py", - "vllm_ascend/utils.py", - "vllm_ascend/worker/worker.py" - ] - }, - { - "commit": "bdedf3c9f805e3a489109338fe9590dafbfe6331", - "subject": "[Graph][Fusion] Add AddRMSNormSPPattern and AddRMSNormSPPatternWithBias (#5569)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/compilation/test_add_rms_norm_quant.py", - "vllm_ascend/compilation/npugraph_ex_passes/add_rms_norm_quant.py" - ] - }, - { - "commit": "77a029979e38eda029039cd4987f457f0a28ab9e", - "subject": "Revert \"[BugFix][Fusion] Fix graph fusion failure problem (#5253)\" (#5667)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/worker/test_worker_v1.py", - "vllm_ascend/compilation/compiler_interface.py", - "vllm_ascend/compilation/graph_fusion_pass_manager.py", - "vllm_ascend/compilation/passes/norm_quant_fusion_pass.py", - "vllm_ascend/compilation/passes/qknorm_rope_fusion_pass.py", - "vllm_ascend/patch/__init__.py", - "vllm_ascend/patch/platform/__init__.py", - "vllm_ascend/patch/platform/patch_compile_backend.py", - "vllm_ascend/worker/worker.py" - ] - }, - { - "commit": "330e25ab1d83c3429edab25ec6fc25b9c59e2221", - "subject": "[P/D] Performance enhancement of Layerwise connector in TP asymmetric scenarios (#5540)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/kv_connector/test_mooncake_layerwise_connector.py", - "vllm_ascend/distributed/mooncake_layerwise_connector.py" - ] - }, - { - "commit": "29e2f9a43e34af20dfe0f897cc11c2f7c46c321a", - "subject": "Bugfix: Align expert map shapes with redundant experts in EPLB adjustment (#5285)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/eplb/core/test_eplb_device_transfer_loader.py", - "tests/ut/ops/test_moe_comm_method.py", - "tests/ut/ops/test_token_dispatcher.py", - "vllm_ascend/eplb/core/eplb_device_transfer_loader.py", - "vllm_ascend/ops/fused_moe/fused_moe.py", - "vllm_ascend/ops/fused_moe/moe_comm_method.py", - "vllm_ascend/ops/fused_moe/token_dispatcher.py", - "vllm_ascend/quantization/w4a16.py", - "vllm_ascend/quantization/w4a8_dynamic.py", - "vllm_ascend/quantization/w8a8_dynamic.py" - ] - }, - { - "commit": "fe3f2c7702603b5894c96ea9bb4a38e839f27547", - "subject": "[Refactor][EAGLE] 3/N delete redundant methods in mtp_proposer (#5420)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/spec_decode/test_mtp_proposer.py", - "vllm_ascend/spec_decode/eagle_proposer.py", - "vllm_ascend/spec_decode/mtp_proposer.py" - ] - }, - { - "commit": "c5e2f485102397eca26118302681716328046a1e", - "subject": "[CI] mv ops to correct path (#5615)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_fused_qkvzba_split_reshape_cat.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_rejection_sample.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_split_qkv_rmsnorm_rope.py" - ] - }, - { - "commit": "8eae949d11a0c4adbfa50391f100e09c13071099", - "subject": "Revert \"[Feat] enable hierarchical mc2 ops on A2 by default (#5545)\" (#5611)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/ops/fused_moe/token_dispatcher.py", - "vllm_ascend/utils.py" - ] - }, - { - "commit": "a2daacbd7157a315f1dd07e9a0b37f8dda1ea9d2", - "subject": "[perf] Fix MLAPO weight disposal for KV-consumer MLA in PD-mix deploy... (#5192)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/attention/mla_v1.py", - "vllm_ascend/attention/sfa_v1.py" - ] - }, - { - "commit": "473431e7e2bdc6c09f121ff4bb52cf5a214aefca", - "subject": "[P/D]Remove mooncake kvpool unused parameter `local_hostname` (#5574)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/distributed/kvpool/backend/mooncake_backend.py" - ] - }, - { - "commit": "16b1bee804f9af9b48c9ee37909b9fb481fc7fe4", - "subject": "[bugfix] fix test_camem failed with triton-ascend (#5492)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/_e2e_test.yaml", - "vllm_ascend/ops/triton/activation/swiglu_quant.py", - "vllm_ascend/ops/triton/fla/fused_qkvzba_split_reshape.py", - "vllm_ascend/ops/triton/rope.py", - "vllm_ascend/worker/worker.py" - ] - }, - { - "commit": "58e8d19c352460a9d02036d8bd8cea66b83e4d54", - "subject": "[UT]add triton ops ut : test_fused_qkvzba_split_reshape_cat (#5474)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/e2e/nightly/ops/triton/test_fused_qkvzba_split_reshape_cat.py" - ] - }, - { - "commit": "e7b623b363f8d3bb0c574a310a46d20c7b7076c4", - "subject": "[BugFix][Fusion] Fix graph fusion failure problem (#5253)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/worker/test_worker_v1.py", - "vllm_ascend/compilation/compiler_interface.py", - "vllm_ascend/compilation/graph_fusion_pass_manager.py", - "vllm_ascend/compilation/passes/norm_quant_fusion_pass.py", - "vllm_ascend/compilation/passes/qknorm_rope_fusion_pass.py", - "vllm_ascend/patch/__init__.py", - "vllm_ascend/patch/platform/__init__.py", - "vllm_ascend/patch/platform/patch_compile_backend.py", - "vllm_ascend/worker/worker.py" - ] - }, - { - "commit": "755caeb06e6fa3beef83eb3c05b781369ea51aad", - "subject": "[Feat][Spec] Optimize token index calculation in spec decode with Triton kernel (#5356)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_prepare_inputs_padded.py", - "vllm_ascend/ops/triton/spec_decode/__init__.py", - "vllm_ascend/ops/triton/spec_decode/utils.py", - "vllm_ascend/spec_decode/eagle_proposer.py" - ] - }, - { - "commit": "350b95efcf6ebedaae7b87ba621c38c021b471a8", - "subject": "[BugFix]Disable dispatch_gmm_combine_decode operator when mtp drafter model uses non-w8a8 while main model uses w8a8, or drafter model is eagle series (#5293)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/ascend_forward_context.py", - "vllm_ascend/utils.py" - ] - }, - { - "commit": "7c210225a262aa4147a96455d55f24c830d414c6", - "subject": "[Perf][PCP][DCP] add multi-stream for GQA to enable computation-communication overlap (#5382)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/attention/test_attention_cp.py", - "vllm_ascend/attention/attention_cp.py", - "vllm_ascend/attention/utils.py", - "vllm_ascend/utils.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "fb9fdcdbe4f193cddf793dfab9e5666950f3fe4b", - "subject": "[Feat] enable hierarchical mc2 ops on A2 by default (#5545)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/ops/fused_moe/token_dispatcher.py", - "vllm_ascend/utils.py" - ] - }, - { - "commit": "1d7539ab3fd6ec0c68dfd752855de6d157101f78", - "subject": "Cleanup pass config override (#5283)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/platform.py" - ] - }, - { - "commit": "80fc0f5b9e09dfae511345e1e9aaf96b5c0f1b25", - "subject": "[Graph][Fusion] Add AddRMSNorm(with bias) (#5491)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/compilation/test_add_rms_norm_quant.py", - "vllm_ascend/compilation/npugraph_ex_passes/add_rms_norm_quant.py" - ] - }, - { - "commit": "d07d8a4535f499c7af95e1966b86700a693c8253", - "subject": "[Model] Add LongCat-Flash (#3833)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/ops/fused_moe/experts_selector.py", - "vllm_ascend/ops/fused_moe/fused_moe.py", - "vllm_ascend/ops/mla.py", - "vllm_ascend/quantization/quant_config.py", - "vllm_ascend/quantization/w4a16.py", - "vllm_ascend/quantization/w4a8_dynamic.py", - "vllm_ascend/quantization/w8a8_dynamic.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "7d5242facaac708ba924f758e8c1be4322af6d37", - "subject": "[Refactor] Formatting output types related to FuseMoE (#5481)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_fused_moe.py", - "tests/ut/ops/test_moe_comm_method.py", - "tests/ut/ops/test_token_dispatcher.py", - "vllm_ascend/ops/fused_moe/fused_moe.py", - "vllm_ascend/ops/fused_moe/moe_comm_method.py", - "vllm_ascend/ops/fused_moe/token_dispatcher.py" - ] - }, - { - "commit": "bdc721d35a25d0ec9174c3c52881b1656c4a3367", - "subject": "[smoke][bugfix] moe_init_routing_v2 active_expert_range use int type (#5521)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/eplb/core/eplb_utils.py", - "vllm_ascend/ops/fused_moe/fused_moe.py", - "vllm_ascend/ops/fused_moe/token_dispatcher.py" - ] - }, - { - "commit": "e760aae1df7814073a4180172385505c1ec0fd83", - "subject": "[1/N] Refactor nightly test structure (#5479)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/nightly_test_a2.yaml", - ".github/workflows/nightly_test_a3.yaml", - "tests/e2e/conftest.py", - "tests/e2e/nightly/multi_node/config/DeepSeek-R1-W8A8-A2.yaml", - "tests/e2e/nightly/multi_node/config/DeepSeek-R1-W8A8-EPLB.yaml", - "tests/e2e/nightly/multi_node/config/DeepSeek-R1-W8A8.yaml", - "tests/e2e/nightly/multi_node/config/DeepSeek-V3.yaml", - "tests/e2e/nightly/multi_node/config/DeepSeek-V3_2-Exp-bf16.yaml", - "tests/e2e/nightly/multi_node/config/Qwen3-235B-A22B-A2.yaml", - "tests/e2e/nightly/multi_node/config/Qwen3-235B-A22B.yaml", - "tests/e2e/nightly/multi_node/config/Qwen3-235B-W8A8-EPLB.yaml", - "tests/e2e/nightly/multi_node/config/Qwen3-235B-W8A8.yaml", - "tests/e2e/nightly/multi_node/config/multi_node_config.py", - "tests/e2e/nightly/multi_node/scripts/__init__.py", - "tests/e2e/nightly/multi_node/scripts/multi_node_config.py", - "tests/e2e/nightly/multi_node/scripts/run.sh", - "tests/e2e/nightly/multi_node/scripts/test_multi_node.py", - "tests/e2e/nightly/multi_node/scripts/utils.py", - "tests/e2e/nightly/multi_node/test_multi_node.py", - "tests/e2e/nightly/single_node/models/__init__.py", - "tests/e2e/nightly/single_node/models/test_deepseek_r1_0528_w8a8.py", - "tests/e2e/nightly/single_node/models/test_deepseek_r1_0528_w8a8_eplb.py", - "tests/e2e/nightly/single_node/models/test_deepseek_v3_2_exp_w8a8.py", - "tests/e2e/nightly/single_node/models/test_glm4_5.py", - "tests/e2e/nightly/single_node/models/test_mtpx_deepseek_r1_0528_w8a8.py", - "tests/e2e/nightly/single_node/models/test_prefix_cache_deepseek_r1_0528_w8a8.py", - "tests/e2e/nightly/single_node/models/test_prefix_cache_qwen3_32b_int8.py", - "tests/e2e/nightly/single_node/models/test_qwen2_5_vl_32b.py", - "tests/e2e/nightly/single_node/models/test_qwen2_5_vl_7b.py", - "tests/e2e/nightly/single_node/models/test_qwen3_235b_a22b_w8a8_eplb.py", - "tests/e2e/nightly/single_node/models/test_qwen3_235b_w8a8.py", - "tests/e2e/nightly/single_node/models/test_qwen3_30b_w8a8.py", - "tests/e2e/nightly/single_node/models/test_qwen3_32b.py", - "tests/e2e/nightly/single_node/models/test_qwen3_32b_int8.py", - "tests/e2e/nightly/single_node/models/test_qwen3_32b_int8_a3_feature_stack3.py", - "tests/e2e/nightly/single_node/models/test_qwq_32b.py", - "tests/e2e/nightly/single_node/ops/__init__.py", - "tests/e2e/nightly/single_node/ops/multicard_ops/__init__.py", - "tests/e2e/nightly/single_node/ops/multicard_ops/test_dispatch_gmm_combine_decode.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/__init__.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_batch_matmul_transpose.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_bgmv_expand.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_bgmv_shrink.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_dispatch_ffn_combine.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_fused_moe.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_gating_top_k_softmax.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_gmm_swiglu_quant_weight_nz_tensor_list.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_grouped_matmul_swiglu_quant.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_matmul_allreduce_add_rmsnorm.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_mla_preprocess.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_mla_preprocess_nq.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_mla_preprocess_qdown.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_moe_init_routing_custom.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_rotary_embedding.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_vocabparallelembedding.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/__init__.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_causal_conv1d.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_l2norm.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_rope.py" - ] - }, - { - "commit": "71f729a6617383aa510240742f41724531d5fc37", - "subject": "Revert \"moe_gating_top_k\" (#5512)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "csrc/build_aclnn.sh", - "csrc/moe_gating_top_k/op_host/CMakeLists.txt", - "csrc/moe_gating_top_k/op_host/error_log.h", - "csrc/moe_gating_top_k/op_host/math_util.h", - "csrc/moe_gating_top_k/op_host/moe_gating_top_k_def.cpp", - "csrc/moe_gating_top_k/op_host/moe_gating_top_k_infershape.cpp", - "csrc/moe_gating_top_k/op_host/moe_gating_top_k_proto.cpp", - "csrc/moe_gating_top_k/op_host/moe_gating_top_k_proto.h", - "csrc/moe_gating_top_k/op_host/moe_gating_top_k_tiling.cpp", - "csrc/moe_gating_top_k/op_host/moe_gating_top_k_tiling.h", - "csrc/moe_gating_top_k/op_host/moe_gating_top_k_tiling_arch35.cpp", - "csrc/moe_gating_top_k/op_host/moe_gating_top_k_tiling_base.cpp", - "csrc/moe_gating_top_k/op_kernel/common.h", - "csrc/moe_gating_top_k/op_kernel/error_log.h", - "csrc/moe_gating_top_k/op_kernel/moe_gating_top_k.cpp", - "csrc/moe_gating_top_k/op_kernel/moe_gating_top_k_apt.cpp", - "csrc/moe_gating_top_k/op_kernel/moe_gating_top_k_e_k_fullload.h", - "csrc/moe_gating_top_k/op_kernel/moe_gating_top_k_generalized.h", - "csrc/moe_gating_top_k/op_kernel/moe_gating_top_k_without_group.h", - "csrc/moe_gating_top_k/tiling_base/data_copy_transpose_tiling.h", - "csrc/moe_gating_top_k/tiling_base/data_copy_transpose_tiling_def.h", - "csrc/moe_gating_top_k/tiling_base/error_log.h", - "csrc/moe_gating_top_k/tiling_base/tiling_base.h", - "csrc/moe_gating_top_k/tiling_base/tiling_key.h", - "csrc/moe_gating_top_k/tiling_base/tiling_templates_registry.h", - "csrc/moe_gating_top_k/tiling_base/tiling_type.h", - "csrc/moe_gating_top_k/tiling_base/tiling_util.h", - "csrc/torch_binding.cpp", - "csrc/torch_binding_meta.cpp", - "tests/e2e/nightly/multicard_ops/test_dispatch_gmm_combine_decode.py", - "tests/e2e/nightly/ops/test_npu_moe_gating_top_k.py", - "tests/ut/model_loader/netloader/test_netloader_elastic.py", - "tests/ut/worker/test_worker_v1.py", - "vllm_ascend/ops/fused_moe/experts_selector.py" - ] - }, - { - "commit": "45c3c279e2b31c85c8739c45b43d8c47710e447b", - "subject": "moe_gating_top_k (#5271)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "csrc/build_aclnn.sh", - "csrc/moe_gating_top_k/op_host/CMakeLists.txt", - "csrc/moe_gating_top_k/op_host/error_log.h", - "csrc/moe_gating_top_k/op_host/math_util.h", - "csrc/moe_gating_top_k/op_host/moe_gating_top_k_def.cpp", - "csrc/moe_gating_top_k/op_host/moe_gating_top_k_infershape.cpp", - "csrc/moe_gating_top_k/op_host/moe_gating_top_k_proto.cpp", - "csrc/moe_gating_top_k/op_host/moe_gating_top_k_proto.h", - "csrc/moe_gating_top_k/op_host/moe_gating_top_k_tiling.cpp", - "csrc/moe_gating_top_k/op_host/moe_gating_top_k_tiling.h", - "csrc/moe_gating_top_k/op_host/moe_gating_top_k_tiling_arch35.cpp", - "csrc/moe_gating_top_k/op_host/moe_gating_top_k_tiling_base.cpp", - "csrc/moe_gating_top_k/op_kernel/common.h", - "csrc/moe_gating_top_k/op_kernel/error_log.h", - "csrc/moe_gating_top_k/op_kernel/moe_gating_top_k.cpp", - "csrc/moe_gating_top_k/op_kernel/moe_gating_top_k_apt.cpp", - "csrc/moe_gating_top_k/op_kernel/moe_gating_top_k_e_k_fullload.h", - "csrc/moe_gating_top_k/op_kernel/moe_gating_top_k_generalized.h", - "csrc/moe_gating_top_k/op_kernel/moe_gating_top_k_without_group.h", - "csrc/moe_gating_top_k/tiling_base/data_copy_transpose_tiling.h", - "csrc/moe_gating_top_k/tiling_base/data_copy_transpose_tiling_def.h", - "csrc/moe_gating_top_k/tiling_base/error_log.h", - "csrc/moe_gating_top_k/tiling_base/tiling_base.h", - "csrc/moe_gating_top_k/tiling_base/tiling_key.h", - "csrc/moe_gating_top_k/tiling_base/tiling_templates_registry.h", - "csrc/moe_gating_top_k/tiling_base/tiling_type.h", - "csrc/moe_gating_top_k/tiling_base/tiling_util.h", - "csrc/torch_binding.cpp", - "csrc/torch_binding_meta.cpp", - "tests/e2e/nightly/multicard_ops/test_dispatch_gmm_combine_decode.py", - "tests/e2e/nightly/ops/test_npu_moe_gating_top_k.py", - "tests/ut/model_loader/netloader/test_netloader_elastic.py", - "tests/ut/worker/test_worker_v1.py", - "vllm_ascend/ops/fused_moe/experts_selector.py" - ] - }, - { - "commit": "3e67e8276c13ee473417ec74f61aab38627216af", - "subject": "[Feature] Support to use fullgraph with eagle (#5118)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/e2e/singlecard/spec_decode_v1/test_v1_spec_decode.py", - "tests/ut/compilation/test_acl_graph.py", - "tests/ut/spec_decode/test_eagle_proposer.py", - "vllm_ascend/ascend_forward_context.py", - "vllm_ascend/attention/attention_v1.py", - "vllm_ascend/attention/mla_cp.py", - "vllm_ascend/attention/mla_v1.py", - "vllm_ascend/compilation/acl_graph.py", - "vllm_ascend/spec_decode/eagle_proposer.py", - "vllm_ascend/spec_decode/mtp_proposer.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "f81cf694b2c0e5ec2e5cb3c23e267103884bde1a", - "subject": "[EPLB][refactor] Modification of the initialization logic for expert_map and log2phy\uff08depend on pr5285\uff09 (#5311)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/eplb/core/expert_map.json", - "tests/ut/eplb/core/test_eplb_utils.py", - "tests/ut/ops/test_expert_load_balancer.py", - "tests/ut/ops/test_fused_moe.py", - "vllm_ascend/eplb/core/eplb_utils.py", - "vllm_ascend/eplb/core/eplb_worker.py", - "vllm_ascend/ops/expert_load_balancer.py", - "vllm_ascend/ops/fused_moe/fused_moe.py", - "vllm_ascend/quantization/quant_config.py", - "vllm_ascend/quantization/w8a8_dynamic.py" - ] - }, - { - "commit": "23169021d9f5d20649061ab8f9ab734d7d00dd1b", - "subject": "[Refactor]6/N Extract common code of class AscendMLAImpl (#5314)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/attention/test_mla_cp.py", - "vllm_ascend/attention/mla_cp.py", - "vllm_ascend/attention/mla_v1.py" - ] - }, - { - "commit": "09f71c14a666a74f167a8e5c2e5209e7ead65370", - "subject": "Revert \"[feat] enable hierarchical mc2 ops on A2 by default (#5300)\" (#5434)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/ops/fused_moe/token_dispatcher.py", - "vllm_ascend/utils.py" - ] - }, - { - "commit": "12da9f94601e5ec05ea1207e90a48d0e07e3b856", - "subject": "[feat] enable hierarchical mc2 ops on A2 by default (#5300)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/ops/fused_moe/token_dispatcher.py", - "vllm_ascend/utils.py" - ] - }, - { - "commit": "8ed6f98a5af91d023b6faa33a4d27cf426512aa5", - "subject": "[Build] Add installation script of fused_infer_attention_score kernel with flash decoding (#5402)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - ".pre-commit-config.yaml", - "tools/install_flash_infer_attention_score_ops_a2.sh", - "tools/install_flash_infer_attention_score_ops_a3.sh" - ] - }, - { - "commit": "8b9ca868274d169c47424df9b74376d83f441d89", - "subject": "[Feature] Remove the transpose step after attention and switch to transpose_batchmatmul (#5390)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/attention/mla_v1.py" - ] - }, - { - "commit": "0dfdfa952604c4383de7152f22106e7e845931bf", - "subject": "[Feature] Enhance all-reduce skipping logic for MoE models in NPUModelRunner (#5329)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "320877d48888ef8ac2cb3d1079518d12d4a626d8", - "subject": "move contiguous in fused_sigmoid_gating_delta_rule_update to model_runner_v1 (#5274)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/ops/triton/fla/sigmoid_gating.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "9b2a7d8866361d662e63fec5d51c323c3a9eb050", - "subject": "[BugFix][Fusion] Patch compile backend to make fusion available (#5308)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/e2e/singlecard/test_aclgraph_accuracy.py", - "tests/e2e/singlecard/test_xlite.py", - "vllm_ascend/patch/__init__.py", - "vllm_ascend/patch/platform/__init__.py", - "vllm_ascend/patch/platform/patch_compile_backend.py" - ] - }, - { - "commit": "a90482803dc12ede67028d4b83e029fde48f1adf", - "subject": "[Kernel] add l2norm triton kernel (#4595)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/workflows/_e2e_test.yaml", - "tests/e2e/nightly/ops/triton/test_l2norm.py", - "vllm_ascend/ops/triton/fla/chunk.py", - "vllm_ascend/ops/triton/fla/l2norm.py" - ] - }, - { - "commit": "fb3d6ca08cb1b66746a1fad697bae0f58e2c8196", - "subject": "Cleanup uesless env (#5270)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/workflows/_e2e_test.yaml", - "tests/e2e/multicard/test_fused_moe_allgather_ep.py", - "vllm_ascend/envs.py" - ] - }, - { - "commit": "515267de22dd499a768fdcadae79f45e4a58ba49", - "subject": "[perf][bugfix] improve performance of rejection sampler and eliminate HD synchronize in TopKTopPSampler (#4154)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/sample/test_sampler.py", - "vllm_ascend/sample/rejection_sampler.py", - "vllm_ascend/sample/sampler.py" - ] - }, - { - "commit": "9227e6af73a0b13383f1dfcb70ef11f0e4af1ee5", - "subject": "[bugfix] remove the EP buffer allocation introduced by fused-op dispatch_ffn_c\u2026 (#5284)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/utils.py" - ] - }, - { - "commit": "ffe51eedd6e609cd1bcd2ee595eda49f0e271f0a", - "subject": "[Refactor][MoE] Reuse vLLM's all_reduce logic (#5189)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/ops/test_prepare_finalize.py", - "vllm_ascend/ops/fused_moe/fused_moe.py", - "vllm_ascend/ops/fused_moe/prepare_finalize.py" - ] - }, - { - "commit": "29a93daa82c51a5f78bc575b1d106c5d280fb2d1", - "subject": "[CI]refactor: standardize test case naming convention (#5243)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/_e2e_test.yaml", - "tests/e2e/multicard/test_aclgraph_capture_replay.py", - "tests/e2e/multicard/test_data_parallel.py", - "tests/e2e/multicard/test_data_parallel_tp2.py", - "tests/e2e/multicard/test_external_launcher.py", - "tests/e2e/multicard/test_full_graph_mode.py", - "tests/e2e/multicard/test_fused_moe_allgather_ep.py", - "tests/e2e/multicard/test_offline_inference_distributed.py", - "tests/e2e/multicard/test_offline_weight_load.py", - "tests/e2e/multicard/test_pipeline_parallel.py" - ] - }, - { - "commit": "592cfb6a6f5fab48721e16bd473483c5912a08eb", - "subject": "[CI] Add Triton Ascend in CI (#4921)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/_e2e_test.yaml", - ".github/workflows/pr_test_light.yaml", - "tests/e2e/singlecard/test_async_scheduling.py", - "tests/ut/attention/test_sfa_v1.py", - "tests/ut/conftest.py", - "tests/ut/ops/test_fused_moe.py", - "tests/ut/sample/test_rejection_sampler.py", - "tests/ut/worker/test_worker_v1.py" - ] - }, - { - "commit": "c3a8d13ca7894fe323ec37e7b505ff42249249a2", - "subject": "[refactor] Remove unnecessary attributes from set_ascend_forward_context (#5204)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/e2e/nightly/ops/test_fused_moe.py", - "tests/ut/ops/test_fused_moe.py", - "tests/ut/quantization/test_w8a8.py", - "vllm_ascend/ascend_forward_context.py", - "vllm_ascend/ops/fused_moe/experts_selector.py", - "vllm_ascend/ops/fused_moe/moe_mlp.py", - "vllm_ascend/ops/register_custom_ops.py", - "vllm_ascend/quantization/w8a8.py", - "vllm_ascend/utils.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "052e472453ef194ef6f48c88c5626a3d72725ece", - "subject": "[bugfix] fix w8a8dynamic fused_moe trans nz (#5199)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/quantization/test_w8a8_dynamic.py", - "vllm_ascend/quantization/w8a8_dynamic.py" - ] - }, - { - "commit": "12d581605bb5c47f04a5da1283db8622b13e7930", - "subject": "[Triton]support swiglu_quant triton in w4a8 (#5161)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/ops/fused_moe/moe_mlp.py", - "vllm_ascend/ops/triton/activation/__init__.py", - "vllm_ascend/ops/triton/activation/swiglu_quant.py" - ] - }, - { - "commit": "64669c4243080ac39207f3e4228d688976fdc066", - "subject": "[misc][FlashComm1][ACLGraph] Incompatibility between Flashcomm1 and FULL_DECODE_ONLY. (#5200)", - "topics": [ - "communication" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/attention/sfa_v1.py" - ] - }, - { - "commit": "b2c121637fd8b8045e66e24ea0f63cb17ffb3b69", - "subject": "[task] Add fused gdn gating triton kernel (#4304)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/ops/triton/fused_gdn_gating.py", - "vllm_ascend/patch/__init__.py", - "vllm_ascend/patch/worker/patch_qwen3_next.py" - ] - }, - { - "commit": "492173cf89688c6b7f53735a1d3d289d4f841f12", - "subject": "[Misc] Cleanup useless print and logger (#5220)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "examples/eplb/eplb_strategy.py", - "vllm_ascend/compilation/passes/norm_quant_fusion_pass.py", - "vllm_ascend/compilation/passes/qknorm_rope_fusion_pass.py", - "vllm_ascend/eplb/core/policy/policy_dynamic_ep_v2.py", - "vllm_ascend/eplb/core/policy/policy_flashlb.py", - "vllm_ascend/utils.py" - ] - }, - { - "commit": "904c18f929d1ad860e5fa9bb2ea7691928366fe0", - "subject": "[Feature]Use DispatchGmmCombineDecode operator to replace MC2(Optional) (#5040)", - "topics": [ - "communication", - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/ascend_forward_context.py", - "vllm_ascend/envs.py", - "vllm_ascend/ops/fused_moe/fused_moe.py", - "vllm_ascend/ops/fused_moe/moe_comm_method.py", - "vllm_ascend/ops/fused_moe/token_dispatcher.py", - "vllm_ascend/quantization/w8a8_dynamic.py" - ] - }, - { - "commit": "758d81dcb10740b7332973a3169e837584100ee3", - "subject": "Drop 0.12.0 support (#5146)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/_e2e_nightly_multi_node.yaml", - ".github/workflows/nightly_test_a2.yaml", - ".github/workflows/nightly_test_a3.yaml", - ".github/workflows/pr_test_full.yaml", - ".github/workflows/pr_test_light.yaml", - ".github/workflows/schedule_test_benchmarks.yaml", - "Dockerfile", - "Dockerfile.310p", - "Dockerfile.310p.openEuler", - "Dockerfile.a3", - "Dockerfile.a3.openEuler", - "Dockerfile.openEuler", - "docs/source/community/versioning_policy.md", - "docs/source/conf.py", - "tests/ut/test_platform.py", - "vllm_ascend/ops/fused_moe/fused_moe.py", - "vllm_ascend/patch/platform/__init__.py", - "vllm_ascend/patch/platform/patch_ec_connector012.py", - "vllm_ascend/platform.py", - "vllm_ascend/worker/model_runner_v1.py", - "vllm_ascend/worker/npu_input_batch.py" - ] - }, - { - "commit": "0cc3fc357ff7fe1fac675fc8898ad904158ff12a", - "subject": "[pref] qwen3_next add triton ops : fused_sigmoid_gating_delta_rule_update (#4818)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/e2e/singlecard/test_fused_sigmoid_gating_delta_rule.py", - "vllm_ascend/ops/triton/fla/sigmoid_gating.py", - "vllm_ascend/patch/__init__.py", - "vllm_ascend/patch/worker/__init__.py", - "vllm_ascend/patch/worker/patch_qwen3_next.py" - ] - }, - { - "commit": "118b0ed3460f37bd364a2cde868eeb6e9492e126", - "subject": "[Feature] Add token mask for DispatchGmmCombineDecode operator (#5171)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "csrc/dispatch_gmm_combine_decode/op_host/aclnn_dispatch_gmm_combine_decode.cpp", - "csrc/dispatch_gmm_combine_decode/op_host/aclnn_dispatch_gmm_combine_decode.h", - "csrc/dispatch_gmm_combine_decode/op_host/dispatch_gmm_combine_decode_def.cpp", - "csrc/dispatch_gmm_combine_decode/op_host/dispatch_gmm_combine_decode_tiling.cpp", - "csrc/dispatch_gmm_combine_decode/op_kernel/dispatch_gmm_combine_decode.cpp", - "csrc/dispatch_gmm_combine_decode/op_kernel/dispatch_gmm_combine_decode.h", - "csrc/dispatch_gmm_combine_decode/op_kernel/dispatch_gmm_combine_decode/gemm/kernel/grouped_matmul_slice_m_per_token_dequant_swiglu_quant_multistage_workspace.h", - "csrc/dispatch_gmm_combine_decode/op_kernel/dispatch_gmm_combine_decode/raw_distributed/cam_moe_distribute_combine.h", - "csrc/dispatch_gmm_combine_decode/op_kernel/dispatch_gmm_combine_decode/raw_distributed/cam_moe_distribute_dispatch.h", - "csrc/dispatch_gmm_combine_decode/op_kernel/dispatch_gmm_combine_decode_base.h", - "csrc/dispatch_gmm_combine_decode/op_kernel/dispatch_gmm_combine_decode_tiling.h", - "csrc/torch_binding.cpp", - "csrc/torch_binding_meta.cpp", - "tests/e2e/nightly/multicard_ops/test_dispatch_gmm_combine_decode.py" - ] - }, - { - "commit": "cc23067f1ea0dad9d461aaaf816ea4ae2792767f", - "subject": "[refactor] refactor weight trans nz and transpose (#4878)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/attention/test_mla_v1.py", - "tests/ut/ops/test_linear.py", - "tests/ut/quantization/test_w4a4_flatquant_dynamic.py", - "tests/ut/quantization/test_w4a8_dynamic.py", - "tests/ut/quantization/test_w8a8.py", - "tests/ut/test_utils.py", - "vllm_ascend/attention/mla_v1.py", - "vllm_ascend/attention/sfa_v1.py", - "vllm_ascend/envs.py", - "vllm_ascend/ops/fused_moe/fused_moe.py", - "vllm_ascend/ops/linear.py", - "vllm_ascend/quantization/w4a4_flatquant_dynamic.py", - "vllm_ascend/quantization/w4a8_dynamic.py", - "vllm_ascend/quantization/w8a8.py", - "vllm_ascend/quantization/w8a8_dynamic.py", - "vllm_ascend/utils.py", - "vllm_ascend/worker/model_runner_v1.py", - "vllm_ascend/worker/worker.py", - "vllm_ascend/xlite/xlite.py" - ] - }, - { - "commit": "ca6f631cba5140be2ea0901bd7c300c924d9a3cd", - "subject": "[2/N][Pangu][MoE] Remove Pangu Related Code (#5130)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "docs/source/user_guide/configuration/additional_config.md", - "tests/ut/attention/test_attention_v1.py", - "tests/ut/quantization/test_quant_config.py", - "tests/ut/quantization/test_utils.py", - "tests/ut/quantization/test_w8a8.py", - "vllm_ascend/attention/attention_v1.py", - "vllm_ascend/platform.py", - "vllm_ascend/quantization/quant_config.py", - "vllm_ascend/quantization/utils.py", - "vllm_ascend/quantization/w8a8.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "1b47fca0e8e1795414452ca440617f41c9eecc96", - "subject": "[bugfix] Use FUSED_MC2 MoE comm path for the op `dispatch_ffn_combine` (#5156)", - "topics": [ - "communication", - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/ascend_forward_context.py", - "vllm_ascend/envs.py", - "vllm_ascend/ops/fused_moe/fused_moe.py", - "vllm_ascend/ops/fused_moe/moe_comm_method.py", - "vllm_ascend/ops/register_custom_ops.py", - "vllm_ascend/quantization/w8a8_dynamic.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "632eab28b718cfe21dabc6cfdc9b88365c86960f", - "subject": "[BugFix]Fix incorrect get_current_vllm_config (#5121)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/ascend_config.py", - "vllm_ascend/attention/attention_v1.py", - "vllm_ascend/attention/utils.py", - "vllm_ascend/compilation/acl_graph.py", - "vllm_ascend/compilation/passes/qknorm_rope_fusion_pass.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "acc3578f585272aaea6e8bdbbb0a94320a476b51", - "subject": "[Graph][Fusion]Add new pattern for AddRmsnormQuant with SP. (#5077)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/e2e/singlecard/compile/test_norm_quant_fusion.py", - "tests/ut/quantization/test_w8a8.py", - "vllm_ascend/compilation/passes/norm_quant_fusion_pass.py", - "vllm_ascend/ops/layernorm.py", - "vllm_ascend/ops/linear_op.py", - "vllm_ascend/ops/register_custom_ops.py", - "vllm_ascend/quantization/w8a8.py" - ] - }, - { - "commit": "0f571c347b178d15b74583055701d0011d5def7d", - "subject": "Nominate new maintainers @zzzzwwjj @realliujiaxu @LCAIZJ (#5152)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/community/contributors.md" - ] - }, - { - "commit": "b69b04d3a91cdc312031c466556d42001df841f3", - "subject": "implement model runner v2 basic framework (#5051)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "docs/source/developer_guide/contribution/testing.md", - "docs/source/developer_guide/feature_guide/patch.md", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/contribution/testing.po", - "docs/source/locale/zh_CN/LC_MESSAGES/developer_guide/feature_guide/patch.po", - "tests/ut/test_platform.py", - "tests/ut/worker/test_worker_v1.py", - "vllm_ascend/platform.py", - "vllm_ascend/worker/v2/__init__.py", - "vllm_ascend/worker/v2/aclgraph_utils.py", - "vllm_ascend/worker/v2/attn_utils.py", - "vllm_ascend/worker/v2/input_batch.py", - "vllm_ascend/worker/v2/model_runner.py", - "vllm_ascend/worker/v2/states.py", - "vllm_ascend/worker/v2/utils.py", - "vllm_ascend/worker/worker.py", - "vllm_ascend/xlite/xlite_worker.py" - ] - }, - { - "commit": "ef8157a5f256be01d7a602e6ae58817761be68d9", - "subject": "fixed fused alltoall execute all reduce (#5109)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/ops/fused_moe/fused_moe.py" - ] - }, - { - "commit": "39fb9e7c8324c48f967a5f38d77b3e1225ecec2f", - "subject": "qwen3_next add triton ops : fused_qkvzba_split_reshape (#4788)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/ops/triton/fla/fused_qkvzba_split_reshape.py", - "vllm_ascend/patch/__init__.py", - "vllm_ascend/patch/worker/__init__.py", - "vllm_ascend/patch/worker/patch_qwen3_next.py" - ] - }, - { - "commit": "98e6e57622f9ab43ff7ff3aa3b5314cb44ad7b48", - "subject": "[Refactor] 4/N Distinguish the branches based on the applicable scenarios of PA and FIA Ops. (#5081)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/attention/test_attention_v1.py", - "vllm_ascend/attention/attention_v1.py", - "vllm_ascend/attention/utils.py" - ] - }, - { - "commit": "724d04391e89408f3fc3fb0cef13e5f4cf155dde", - "subject": "[model] Support PanguUltraMoE (#4615)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/quantization/quant_config.py", - "vllm_ascend/spec_decode/mtp_proposer.py" - ] - }, - { - "commit": "f0060fc822ff062c439299e899b1990fed2493d3", - "subject": "[Pangu][MoE] Remove PanguProMoEV1 related code (#5088)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/ops/test_fused_moe.py", - "tests/ut/ops/test_moe_comm_method.py", - "vllm_ascend/ascend_forward_context.py", - "vllm_ascend/ops/fused_moe/moe_comm_method.py", - "vllm_ascend/ops/fused_moe/token_dispatcher.py" - ] - }, - { - "commit": "06b82e7503c62d4176b177354fec997481a688f3", - "subject": "[main] rename device type (#5099)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "setup.py", - "tests/e2e/multicard/test_aclgraph_capture_replay.py", - "tests/ut/attention/test_attention_v1.py", - "tests/ut/ops/test_activation.py", - "tests/ut/ops/test_fused_moe.py", - "tests/ut/ops/test_layernorm.py", - "tests/ut/ops/test_rotary_embedding.py", - "tests/ut/ops/test_token_dispatcher.py", - "tests/ut/quantization/test_w8a8.py", - "tests/ut/test_platform.py", - "vllm_ascend/ascend_forward_context.py", - "vllm_ascend/attention/attention_v1.py", - "vllm_ascend/ops/fused_moe/token_dispatcher.py", - "vllm_ascend/ops/rotary_embedding.py", - "vllm_ascend/utils.py", - "vllm_ascend/worker/worker_v1.py" - ] - }, - { - "commit": "cadfa5ddc162a4e4ac0886524fa6400ea1dcd72d", - "subject": "[Fusion] [Graph] Add qknorm rope fusion operator (#4711)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/e2e/singlecard/test_aclgraph_accuracy.py", - "vllm_ascend/ascend_config.py", - "vllm_ascend/ascend_forward_context.py", - "vllm_ascend/attention/mla_cp.py", - "vllm_ascend/attention/mla_v1.py", - "vllm_ascend/attention/sfa_v1.py", - "vllm_ascend/compilation/graph_fusion_pass_manager.py", - "vllm_ascend/compilation/passes/qknorm_rope_fusion_pass.py", - "vllm_ascend/ops/__init__.py", - "vllm_ascend/ops/rotary_embedding.py", - "vllm_ascend/ops/triton/linearnorm/__init__.py", - "vllm_ascend/ops/triton/linearnorm/split_qkv_rmsnorm_rope.py", - "vllm_ascend/spec_decode/eagle_proposer.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "eb4c08f05dac1ae5d12bd9aa7e4aa76a4ab72032", - "subject": "[bugfix] fix mtp accept rate (#5093)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "csrc/matmul_allreduce_add_rmsnorm/op_host/aclnn_matmul_allreduce_add_rmsnorm.cpp", - "vllm_ascend/ascend_forward_context.py", - "vllm_ascend/spec_decode/eagle_proposer.py", - "vllm_ascend/spec_decode/mtp_proposer.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "5b1da4e914e43e30c12aa222ad9b514342db8712", - "subject": "[Feat] Support async_scheduler and disable_padded_drafter_batch in eagle (#4893)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/e2e/singlecard/spec_decode_v1/test_v1_spec_decode.py", - "tests/ut/spec_decode/test_eagle_proposer.py", - "vllm_ascend/attention/attention_v1.py", - "vllm_ascend/attention/utils.py", - "vllm_ascend/spec_decode/eagle_proposer.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "4ed2951400c539d4822f38d615363ad90c63e3b6", - "subject": "\u3010Feature\u3011refactor npu_modelrunner for profile_run (#4993)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/spec_decode/test_eagle_proposer.py", - "tests/ut/spec_decode/test_mtp_proposer.py", - "vllm_ascend/ascend_forward_context.py", - "vllm_ascend/spec_decode/eagle_proposer.py", - "vllm_ascend/spec_decode/mtp_proposer.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "0918de58d5634d468549bd700a5693e1eea4f2ad", - "subject": "[Bugfix] dynamic eplb does't use fused_alltoall (#4919)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "d43cabc2b1b3ea8c308f0323ef60e70b9e2c301e", - "subject": "[Bugfix] Fix precision issues in moe_mlp (vllm-ascend main) (#5025)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/ops/fused_moe/moe_mlp.py" - ] - }, - { - "commit": "a9625851ef39fd331316ee39f4f670f1e7545cbc", - "subject": "[Attention] Temporarily add back pa for small batch sizes. (#4765)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/ascend_config.py", - "vllm_ascend/attention/attention_v1.py", - "vllm_ascend/attention/utils.py", - "vllm_ascend/compilation/acl_graph.py" - ] - }, - { - "commit": "8d2998d0e48c360e22a55b9977cdfc153e110e4b", - "subject": "[Misc] Upgrade vllm hash to 12_14 (#5000)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/pr_test_full.yaml", - ".github/workflows/pr_test_light.yaml", - "docs/source/community/versioning_policy.md", - "tests/ut/compilation/test_acl_graph.py", - "tests/ut/spec_decode/test_eagle_proposer.py", - "tests/ut/worker/test_input_batch.py", - "vllm_ascend/attention/mla_v1.py", - "vllm_ascend/attention/sfa_v1.py", - "vllm_ascend/eplb/utils.py", - "vllm_ascend/ops/fused_moe/fused_moe.py", - "vllm_ascend/patch/platform/__init__.py", - "vllm_ascend/patch/platform/patch_ec_connector.py", - "vllm_ascend/patch/platform/patch_ec_connector012.py", - "vllm_ascend/platform.py", - "vllm_ascend/utils.py", - "vllm_ascend/worker/model_runner_v1.py", - "vllm_ascend/worker/npu_input_batch.py" - ] - }, - { - "commit": "5fae65f3a8612ed4034544a22632b9cdaad32d61", - "subject": "[Graph][Fusion] Add AddRMSNorm(with bias) and Quant Fusion Pattern (#5011)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/e2e/singlecard/compile/test_norm_quant_fusion.py", - "vllm_ascend/compilation/passes/norm_quant_fusion_pass.py" - ] - }, - { - "commit": "aa02a85e4d10294e9f6b4678cbd9d85480f8e975", - "subject": "[bugfix] Fix dummy-run and multi-node issues in MoE routing and MTP (#4947)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "csrc/dispatch_ffn_combine/op_kernel/moe_init_routing_quant_v2/moe_init_routing_quant_v2.cpp", - "vllm_ascend/spec_decode/mtp_proposer.py", - "vllm_ascend/utils.py" - ] - }, - { - "commit": "545e8569719266a6f92417585a91b075d826d2ef", - "subject": "[Refactor]3/N Refactor mla_v1.py & extract mla_cp (#4933)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/attention/test_mla_v1.py", - "vllm_ascend/attention/mla_cp.py", - "vllm_ascend/attention/mla_v1.py" - ] - }, - { - "commit": "ba28d54f359278c83ed0f55ba78de9b1996f2b12", - "subject": "[Perf]enable prefill flashcommon3 (#4065)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/ops/test_prepare_finalize.py", - "vllm_ascend/ascend_config.py", - "vllm_ascend/distributed/parallel_state.py", - "vllm_ascend/distributed/utils.py", - "vllm_ascend/flash_common3_context.py", - "vllm_ascend/ops/fused_moe/fused_moe.py", - "vllm_ascend/ops/fused_moe/prepare_finalize.py", - "vllm_ascend/quantization/w8a8_dynamic.py" - ] - }, - { - "commit": "fd7c92914584f4aad8a03eefaece456775a9c4ab", - "subject": "[perf] replace all_reduce for kv_consumer and support different num_tokens among all ranks (#4983)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/ops/test_token_dispatcher.py", - "vllm_ascend/ops/fused_moe/fused_moe.py", - "vllm_ascend/ops/fused_moe/token_dispatcher.py", - "vllm_ascend/quantization/w4a8_dynamic.py", - "vllm_ascend/quantization/w8a8_dynamic.py", - "vllm_ascend/spec_decode/mtp_proposer.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "0983c5510aa49c7310b79db72657d8a0f92918ec", - "subject": "vllm-ascend support Ascend950 with Qwen dense model. (#4228)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "setup.py", - "vllm_ascend/attention/attention_v1.py", - "vllm_ascend/ops/rotary_embedding.py", - "vllm_ascend/worker/worker_v1.py" - ] - }, - { - "commit": "4984e8a284eaf7713e907dc96efee8f5e21edb3a", - "subject": "[Bugfix] bugfix for moe_mlp (#4822)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/ops/test_fused_moe.py", - "vllm_ascend/ops/fused_moe/moe_mlp.py" - ] - }, - { - "commit": "d54db76dd2df617a7c6b8539f21e646fa3d0bdaa", - "subject": "[MoE][TorchAir] Remove FusedMoEState (#4927)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/ascend_forward_context.py" - ] - }, - { - "commit": "e56dba9b0d44dbdd37f8c57ea89658f6af19b031", - "subject": "[CI]cleanup e2e test (#4800)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/_e2e_test.yaml", - "tests/e2e/multicard/test_aclgraph_capture_replay.py", - "tests/e2e/multicard/test_data_parallel.py", - "tests/e2e/multicard/test_data_parallel_tp2.py", - "tests/e2e/multicard/test_expert_parallel.py", - "tests/e2e/multicard/test_external_launcher.py", - "tests/e2e/multicard/test_full_graph_mode.py", - "tests/e2e/multicard/test_fused_moe_allgather_ep.py", - "tests/e2e/multicard/test_offline_inference_distributed.py", - "tests/e2e/multicard/test_offline_weight_load.py", - "tests/e2e/multicard/test_pipeline_parallel.py" - ] - }, - { - "commit": "18221c0e1d2eac75eea27eca81de651e9035b637", - "subject": "[Fusion] normalize fusion naming and enable e2e test (#4693)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/workflows/_e2e_test.yaml", - "tests/e2e/singlecard/compile/backend.py", - "tests/e2e/singlecard/compile/test_norm_quant_fusion.py", - "tests/ut/test_ascend_config.py", - "vllm_ascend/ascend_config.py", - "vllm_ascend/compilation/graph_fusion_pass_manager.py", - "vllm_ascend/compilation/passes/norm_quant_fusion_pass.py", - "vllm_ascend/platform.py" - ] - }, - { - "commit": "07c71311049ee758fb3feb833ed113a1a1b2c362", - "subject": "[Fix] Delete redundant variable (#4903)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/ascend_forward_context.py" - ] - }, - { - "commit": "a47aa4da2f87dfd2792ef13d31476eccd27685bf", - "subject": "[feat] apply flashcomm1 on bailing (#4868)", - "topics": [ - "communication" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/ops/linear_op.py" - ] - }, - { - "commit": "bb76f7962c65d2f2ccd063812bb3e2f0e6a35021", - "subject": "cleanup useless torchair logic (#4856)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "docs/source/tutorials/index.md", - "docs/source/tutorials/multi_npu_moge.md", - "tests/ut/test_platform.py", - "vllm_ascend/ascend_config.py", - "vllm_ascend/platform.py", - "vllm_ascend/spec_decode/mtp_proposer.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "dd622aa6a6ea34a6bc799d52171ebe6c3e062972", - "subject": "[Feature] Support npuhraph_ex backend (#4700)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/test_ascend_config.py", - "vllm_ascend/ascend_config.py", - "vllm_ascend/compilation/compiler_interface.py", - "vllm_ascend/compilation/npugraph_ex_passes/__init__.py", - "vllm_ascend/compilation/npugraph_ex_passes/add_rms_norm_quant.py", - "vllm_ascend/platform.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "8bb028424bee2dc7e7d260103f0f62d356042141", - "subject": "Fixed the performance degradation issue in post-processing in speculative decoding scenarios. (#4849)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/sample/rejection_sampler.py" - ] - }, - { - "commit": "a82b0fa70ea483449c02ac2b6e50760249ea6406", - "subject": "mooncake connector support pipeline parallel & fix pp with flashcomm1 (#4054)", - "topics": [ - "communication" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/kv_connector/test_mooncake_connector.py", - "tests/ut/worker/test_model_runner_v1.py", - "vllm_ascend/distributed/mooncake_connector.py", - "vllm_ascend/worker/model_runner_v1.py", - "vllm_ascend/worker/worker_v1.py" - ] - }, - { - "commit": "ce5872705e80d3e2fb107808aa296831d93fe6fa", - "subject": "[Feat] Support native Kimi-K2-Thinking native W4A16 quantized experts weights (#4516)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/workflows/_e2e_test.yaml", - "docs/source/tutorials/index.md", - "docs/source/tutorials/multi_npu_kimi-k2-thinking.md", - "tests/e2e/multicard/test_offline_inference_distributed.py", - "tests/ut/quantization/test_w4a16.py", - "vllm_ascend/ops/fused_moe/moe_comm_method.py", - "vllm_ascend/ops/fused_moe/moe_mlp.py", - "vllm_ascend/ops/rotary_embedding.py", - "vllm_ascend/quantization/compressed_tensors/compressed_tensors.py", - "vllm_ascend/quantization/quant_config.py", - "vllm_ascend/quantization/utils.py", - "vllm_ascend/quantization/w4a16.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "e32014ac1da082db9d142515d892a685ba12cd4c", - "subject": "[Model] Support pooling models (#3122)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/workflows/_e2e_test.yaml", - "docs/source/user_guide/support_matrix/supported_models.md", - "tests/e2e/conftest.py", - "tests/e2e/singlecard/pooling/__init__.py", - "tests/e2e/singlecard/pooling/test_classification.py", - "tests/e2e/singlecard/pooling/test_embedding.py", - "tests/e2e/singlecard/pooling/test_scoring.py", - "tests/e2e/singlecard/test_bge_model.py", - "tests/e2e/singlecard/test_embedding_aclgraph.py", - "vllm_ascend/attention/attention_mask.py", - "vllm_ascend/attention/attention_v1.py", - "vllm_ascend/patch/__init__.py", - "vllm_ascend/patch/worker/__init__.py", - "vllm_ascend/patch/worker/patch_bert.py", - "vllm_ascend/patch/worker/patch_roberta.py", - "vllm_ascend/worker/model_runner_v1.py", - "vllm_ascend/worker/npu_input_batch.py" - ] - }, - { - "commit": "835b4c8f1dd6fd485ec2df305c68fae92e24487c", - "subject": "Drop torchair (#4814)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/workflows/_e2e_test.yaml", - ".github/workflows/vllm_ascend_test_nightly_a2.yaml", - ".github/workflows/vllm_ascend_test_pr_light.yaml", - ".pre-commit-config.yaml", - "docs/source/developer_guide/contribution/testing.md", - "docs/source/developer_guide/feature_guide/Multi_Token_Prediction.md", - "docs/source/tutorials/DeepSeek-V3.1.md", - "docs/source/tutorials/DeepSeek-V3.2-Exp.md", - "docs/source/tutorials/multi_node.md", - "docs/source/tutorials/multi_node_kimi.md", - "docs/source/tutorials/multi_npu_moge.md", - "docs/source/user_guide/configuration/additional_config.md", - "docs/source/user_guide/feature_guide/graph_mode.md", - "docs/source/user_guide/feature_guide/quantization.md", - "examples/run_dp_server.sh", - "tests/e2e/310p/test_offline_inference_parallel_310p.py", - "tests/e2e/multicard/test_offline_inference_distributed.py", - "tests/e2e/multicard/test_torchair_graph_mode.py", - "tests/e2e/nightly/features/test_mtpx_deepseek_r1_0528_w8a8.py", - "tests/e2e/nightly/features/test_prefix_cache_deepseek_r1_0528_w8a8.py", - "tests/e2e/nightly/models/test_deepseek_r1_0528_w8a8.py", - "tests/e2e/nightly/models/test_deepseek_r1_w8a8_eplb.py", - "tests/e2e/nightly/models/test_deepseek_v3_2_exp_w8a8.py", - "tests/e2e/nightly/multi_node/config/models/DeepSeek-R1-W8A8-A2-torchair.yaml", - "tests/e2e/nightly/multi_node/config/models/DeepSeek-R1-W8A8-EPLB.yaml", - "tests/e2e/nightly/multi_node/config/models/DeepSeek-R1-W8A8.yaml", - "tests/e2e/nightly/multi_node/config/models/DeepSeek-V3.yaml", - "tests/e2e/nightly/multi_node/config/models/DeepSeek-V3_2-Exp-bf16.yaml", - "tests/e2e/singlecard/spec_decode_v1/test_v1_mtp_torchair_correctness.py", - "tests/ut/ops/test_fused_moe.py", - "tests/ut/ops/test_rotary_embedding.py", - "tests/ut/quantization/test_w8a8_dynamic.py", - "tests/ut/test_ascend_config.py", - "tests/ut/test_platform.py", - "tests/ut/torchair/__init__.py", - "tests/ut/torchair/models/test_qwen3_moe.py", - "tests/ut/torchair/models/test_torchair_deepseek_mtp.py", - "tests/ut/torchair/models/test_torchair_deepseek_v2.py", - "tests/ut/torchair/ops/test_torchair_fused_moe.py", - "tests/ut/torchair/ops/test_torchair_rotary_embedding.py", - "tests/ut/torchair/quantization/test_torchair_w4a8_dynamic.py", - "tests/ut/torchair/quantization/test_torchair_w8a8_dynamic.py", - "tests/ut/torchair/test_torchair_attention.py", - "tests/ut/torchair/test_torchair_mla.py", - "tests/ut/torchair/test_torchair_model_runner.py", - "tests/ut/torchair/test_torchair_mtp_proposer.py", - "tests/ut/torchair/test_torchair_sfa.py", - "tests/ut/torchair/test_torchair_worker.py", - "tests/ut/torchair/test_utils.py", - "vllm_ascend/ascend_config.py", - "vllm_ascend/attention/mla_v1.py", - "vllm_ascend/attention/sfa_v1.py", - "vllm_ascend/distributed/mooncake_connector.py", - "vllm_ascend/ops/fused_moe/moe_comm_method.py", - "vllm_ascend/platform.py", - "vllm_ascend/quantization/w8a8_dynamic.py", - "vllm_ascend/spec_decode/__init__.py", - "vllm_ascend/torchair/__init__.py", - "vllm_ascend/torchair/models/__init__.py", - "vllm_ascend/torchair/models/qwen2.py", - "vllm_ascend/torchair/models/qwen3_moe.py", - "vllm_ascend/torchair/models/torchair_deepseek_mtp.py", - "vllm_ascend/torchair/models/torchair_deepseek_v2.py", - "vllm_ascend/torchair/models/torchair_deepseek_v3.py", - "vllm_ascend/torchair/models/torchair_pangu_moe.py", - "vllm_ascend/torchair/ops/__init__.py", - "vllm_ascend/torchair/ops/sequence_parallel.py", - "vllm_ascend/torchair/ops/shared_weight_layer.py", - "vllm_ascend/torchair/ops/torchair_activation.py", - "vllm_ascend/torchair/ops/torchair_fused_moe.py", - "vllm_ascend/torchair/ops/torchair_layernorm.py", - "vllm_ascend/torchair/ops/torchair_rotary_embedding.py", - "vllm_ascend/torchair/ops/torchair_vocab_parallel_embedding.py", - "vllm_ascend/torchair/quantization/__init__.py", - "vllm_ascend/torchair/quantization/torchair_w4a8_dynamic.py", - "vllm_ascend/torchair/quantization/torchair_w8a8_dynamic.py", - "vllm_ascend/torchair/torchair_attention.py", - "vllm_ascend/torchair/torchair_mla.py", - "vllm_ascend/torchair/torchair_model_runner.py", - "vllm_ascend/torchair/torchair_mtp_proposer.py", - "vllm_ascend/torchair/torchair_sfa.py", - "vllm_ascend/torchair/torchair_worker.py", - "vllm_ascend/torchair/utils.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "ba9cda9dfd83765a7e79a074c033818ff5a76bd6", - "subject": "[Kernel] add custom op MatmulAllreduceAddRmsnorm (#4606)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "csrc/build_aclnn.sh", - "csrc/matmul_allreduce_add_rmsnorm/op_host/CMakeLists.txt", - "csrc/matmul_allreduce_add_rmsnorm/op_host/aclnn_matmul_allreduce_add_rmsnorm.cpp", - "csrc/matmul_allreduce_add_rmsnorm/op_host/aclnn_matmul_allreduce_add_rmsnorm.h", - "csrc/matmul_allreduce_add_rmsnorm/op_host/matmul_allreduce_add_rmsnorm_def.cpp", - "csrc/matmul_allreduce_add_rmsnorm/op_host/matmul_allreduce_add_rmsnorm_proto.cpp", - "csrc/matmul_allreduce_add_rmsnorm/op_host/matmul_allreduce_add_rmsnorm_tiling.cpp", - "csrc/matmul_allreduce_add_rmsnorm/op_host/matmul_allreduce_add_rmsnorm_workspace.h", - "csrc/matmul_allreduce_add_rmsnorm/op_kernel/matmul_allreduce_add_rmsnorm.cpp", - "csrc/matmul_allreduce_add_rmsnorm/op_kernel/matmul_allreduce_add_rmsnorm_aic_kernel.h", - "csrc/matmul_allreduce_add_rmsnorm/op_kernel/matmul_allreduce_add_rmsnorm_aiv_kernel.h", - "csrc/matmul_allreduce_add_rmsnorm/op_kernel/matmul_allreduce_add_rmsnorm_tiling.h", - "csrc/matmul_allreduce_add_rmsnorm/op_kernel/matmul_allreduce_add_rmsnorm_utils.h", - "csrc/torch_binding.cpp", - "csrc/torch_binding_meta.cpp", - "tests/e2e/nightly/ops/test_matmul_allreduce_add_rmsnorm.py" - ] - }, - { - "commit": "848419d1ba89eddea2bcd721b1d59469fb4e68e4", - "subject": "[Bugfix] Disable the dispatch_ffn_combine kernel in MTP path (#4751)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/spec_decode/mtp_proposer.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "c331503677d6c57a2387ed7c5c58a4ce956674d9", - "subject": "[Refactor] 2/N Unify all mask generation methods and cache mask (#4779)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/attention/test_attention_mask.py", - "vllm_ascend/attention/attention_mask.py", - "vllm_ascend/attention/mla_v1.py", - "vllm_ascend/attention/utils.py", - "vllm_ascend/spec_decode/eagle_proposer.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "9567e5dd8c61d2e62b3572c9e98dc4f9eaf51c2a", - "subject": "[kernel] Adapt DispatchGmmCombineDecode operator to parameters of small operators (#4790)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "csrc/dispatch_gmm_combine_decode/op_host/dispatch_gmm_combine_decode_tiling.cpp", - "csrc/dispatch_gmm_combine_decode/op_kernel/dispatch_gmm_combine_decode.h", - "csrc/dispatch_gmm_combine_decode/op_kernel/dispatch_gmm_combine_decode/epilogue/block/block_epilogue_per_token_dequant_swiglu.h", - "csrc/dispatch_gmm_combine_decode/op_kernel/dispatch_gmm_combine_decode/gemm/kernel/grouped_matmul_slice_m_per_token_dequant_swiglu_quant_multistage_workspace.h", - "tests/e2e/nightly/multicard_ops/test_dispatch_gmm_combine_decode.py" - ] - }, - { - "commit": "9a885d08d0064ebbe5b3e9ccfc93681bddb155fd", - "subject": "[Feat] Multi-stream for eplb heat collection and aggregation (#4214)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/eplb/eplb_updator.py", - "vllm_ascend/eplb/utils.py", - "vllm_ascend/ops/fused_moe/fused_moe.py" - ] - }, - { - "commit": "432b861cae054bbf424dcc498ab6abed548549d0", - "subject": "Fix incorrect MLAPO weight release in PD mixex scenarios. (#4774)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/attention/sfa_v1.py" - ] - }, - { - "commit": "b230e7e987da602a3985a4f297b01decfc1ec40f", - "subject": "[MOE]move weight transpose to wakeup for RL secnarios (#4626)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/_e2e_test.yaml", - "examples/offline_weight_load.py", - "tests/e2e/multicard/test_offline_weight_load.py", - "tests/ut/ops/test_fused_moe.py", - "tests/ut/worker/test_worker_v1.py", - "vllm_ascend/ops/fused_moe/fused_moe.py", - "vllm_ascend/worker/worker_v1.py" - ] - }, - { - "commit": "b91a5f096889b2b6e1d09c2137974de13ad9b904", - "subject": "Support DeepSeekV3.2 with MLAPO operator (#4753)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/attention/sfa_v1.py" - ] - }, - { - "commit": "a5163c8c369e046c53e6127bbbcee23392cec069", - "subject": "[Feat]enable sfa cp for dsv3.2 (#4702)", - "topics": [ - "communication" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/attention/sfa_v1.py", - "vllm_ascend/distributed/parallel_state.py", - "vllm_ascend/ops/shared_weight_layer.py", - "vllm_ascend/utils.py" - ] - }, - { - "commit": "4bd1030842a3831109ccfb136056459cd83c1441", - "subject": "[Kernel] add custom op DispatchGmmCombineDecode (#4139)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/workflows/vllm_ascend_test_nightly_a3.yaml", - "csrc/build_aclnn.sh", - "csrc/dispatch_gmm_combine_decode/op_host/CMakeLists.txt", - "csrc/dispatch_gmm_combine_decode/op_host/aclnn_dispatch_gmm_combine_decode.cpp", - "csrc/dispatch_gmm_combine_decode/op_host/aclnn_dispatch_gmm_combine_decode.h", - "csrc/dispatch_gmm_combine_decode/op_host/dispatch_gmm_combine_decode_def.cpp", - "csrc/dispatch_gmm_combine_decode/op_host/dispatch_gmm_combine_decode_proto.cpp", - "csrc/dispatch_gmm_combine_decode/op_host/dispatch_gmm_combine_decode_tiling.cpp", - "csrc/dispatch_gmm_combine_decode/op_kernel/dispatch_gmm_combine_decode.cpp", - "csrc/dispatch_gmm_combine_decode/op_kernel/dispatch_gmm_combine_decode.h", - "csrc/dispatch_gmm_combine_decode/op_kernel/dispatch_gmm_combine_decode/epilogue/block/block_epilogue.h", - "csrc/dispatch_gmm_combine_decode/op_kernel/dispatch_gmm_combine_decode/epilogue/block/block_epilogue_per_token_dequant.hpp", - "csrc/dispatch_gmm_combine_decode/op_kernel/dispatch_gmm_combine_decode/epilogue/block/block_epilogue_per_token_dequant_swiglu.h", - "csrc/dispatch_gmm_combine_decode/op_kernel/dispatch_gmm_combine_decode/epilogue/dispatch_policy.h", - "csrc/dispatch_gmm_combine_decode/op_kernel/dispatch_gmm_combine_decode/epilogue/tile/tile_stride_binary.h", - "csrc/dispatch_gmm_combine_decode/op_kernel/dispatch_gmm_combine_decode/epilogue/tile/tile_stride_muls.h", - "csrc/dispatch_gmm_combine_decode/op_kernel/dispatch_gmm_combine_decode/gemm/block/block_mmad.h", - "csrc/dispatch_gmm_combine_decode/op_kernel/dispatch_gmm_combine_decode/gemm/block/block_mmad_preload_async_with_callback_resident_a.h", - "csrc/dispatch_gmm_combine_decode/op_kernel/dispatch_gmm_combine_decode/gemm/dispatch_policy.h", - "csrc/dispatch_gmm_combine_decode/op_kernel/dispatch_gmm_combine_decode/gemm/kernel/grouped_matmul_slice_m_per_token_dequant_multistage_workspace.h", - "csrc/dispatch_gmm_combine_decode/op_kernel/dispatch_gmm_combine_decode/gemm/kernel/grouped_matmul_slice_m_per_token_dequant_swiglu_quant_multistage_workspace.h", - "csrc/dispatch_gmm_combine_decode/op_kernel/dispatch_gmm_combine_decode/raw_distributed/cam_moe_distribute_combine.h", - "csrc/dispatch_gmm_combine_decode/op_kernel/dispatch_gmm_combine_decode/raw_distributed/cam_moe_distribute_dispatch.h", - "csrc/dispatch_gmm_combine_decode/op_kernel/dispatch_gmm_combine_decode_base.h", - "csrc/dispatch_gmm_combine_decode/op_kernel/dispatch_gmm_combine_decode_tiling.h", - "csrc/torch_binding.cpp", - "csrc/torch_binding_meta.cpp", - "docs/source/installation.md", - "tests/e2e/nightly/multicard_ops/test_dispatch_gmm_combine_decode.py" - ] - }, - { - "commit": "cb42564942b2b2eed11d26c8e86b4c869d5f4c4b", - "subject": "[BugFix] Fix eagle3 accuracy problem when enforce_eager=True (#4521)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/e2e/singlecard/spec_decode_v1/test_v1_spec_decode.py", - "vllm_ascend/spec_decode/eagle_proposer.py" - ] - }, - { - "commit": "a78f49ea578e4e97b647fcbf4f89592fe1a7f5a9", - "subject": "[Refactor] 1/N Refactor attention_v1 & extract attention_cp (#4628)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/attention/attention_cp.py", - "vllm_ascend/attention/attention_v1.py" - ] - }, - { - "commit": "a336543977a731992199d8a0734fd2a51c2b588a", - "subject": "[Bugifx] fix quant_apply_mlp w1_scale type error & fix getting num_local_expert (#4632)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/eplb/adaptor/vllm_adaptor.py", - "vllm_ascend/ops/fused_moe/moe_mlp.py", - "vllm_ascend/quantization/w8a8_dynamic.py" - ] - }, - { - "commit": "b32ef53b3bfe67891ab107d731bf085c0db75f64", - "subject": "[long_seq] remove long_seq env (#4660)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/attention/test_attention_v1.py", - "tests/ut/attention/test_mla_v1.py", - "tests/ut/kv_connector/test_mooncake_connector.py", - "tests/ut/test_platform.py", - "vllm_ascend/attention/attention_v1.py", - "vllm_ascend/attention/mla_v1.py", - "vllm_ascend/distributed/kvpool/pool_worker.py", - "vllm_ascend/distributed/llmdatadist_c_mgr_connector.py", - "vllm_ascend/distributed/mooncake_connector.py", - "vllm_ascend/distributed/parallel_state.py", - "vllm_ascend/ops/fused_moe/prepare_finalize.py", - "vllm_ascend/platform.py", - "vllm_ascend/spec_decode/mtp_proposer.py", - "vllm_ascend/worker/block_table.py", - "vllm_ascend/worker/model_runner_v1.py", - "vllm_ascend/worker/worker_v1.py" - ] - }, - { - "commit": "ad0607f9009a9c2defbfc3e5045735510dc54e0b", - "subject": "add `dispatch_gmm_combine` kernel (#3532)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".gitmodules", - "csrc/build_aclnn.sh", - "csrc/cmake/func.cmake", - "csrc/dispatch_ffn_combine/op_host/CMakeLists.txt", - "csrc/dispatch_ffn_combine/op_host/aclnn_dispatch_ffn_combine.cpp", - "csrc/dispatch_ffn_combine/op_host/aclnn_dispatch_ffn_combine.h", - "csrc/dispatch_ffn_combine/op_host/dispatch_ffn_combine_def.cpp", - "csrc/dispatch_ffn_combine/op_host/dispatch_ffn_combine_proto.cpp", - "csrc/dispatch_ffn_combine/op_host/dispatch_ffn_combine_tiling.cpp", - "csrc/dispatch_ffn_combine/op_host/error_log.h", - "csrc/dispatch_ffn_combine/op_host/hcom_topo_info.h", - "csrc/dispatch_ffn_combine/op_host/tiling_args.h", - "csrc/dispatch_ffn_combine/op_kernel/dispatch_ffn_combine.cpp", - "csrc/dispatch_ffn_combine/op_kernel/dispatch_ffn_combine.h", - "csrc/dispatch_ffn_combine/op_kernel/dispatch_ffn_combine_kernel.hpp", - "csrc/dispatch_ffn_combine/op_kernel/dispatch_ffn_combine_tiling.h", - "csrc/dispatch_ffn_combine/op_kernel/moe_init_routing_quant_v2/moe_init_routing_quant_v2.cpp", - "csrc/dispatch_ffn_combine/op_kernel/moe_init_routing_quant_v2/moe_init_routing_quant_v2_tiling.h", - "csrc/dispatch_ffn_combine/op_kernel/moe_init_routing_quant_v2/moe_init_routing_v2_tiling.h", - "csrc/dispatch_ffn_combine/op_kernel/moe_init_routing_quant_v2/moe_v2_common.h", - "csrc/dispatch_ffn_combine/op_kernel/moe_init_routing_quant_v2/moe_v2_expert_token_out.h", - "csrc/dispatch_ffn_combine/op_kernel/moe_init_routing_quant_v2/moe_v2_fullload_dynamic_quant.h", - "csrc/dispatch_ffn_combine/op_kernel/moe_init_routing_quant_v2/moe_v2_fullload_quant.h", - "csrc/dispatch_ffn_combine/op_kernel/moe_init_routing_quant_v2/moe_v2_fullload_quant_base.h", - "csrc/dispatch_ffn_combine/op_kernel/moe_init_routing_quant_v2/moe_v2_gather_dynamic_quant.h", - "csrc/dispatch_ffn_combine/op_kernel/moe_init_routing_quant_v2/moe_v2_gather_out.h", - "csrc/dispatch_ffn_combine/op_kernel/moe_init_routing_quant_v2/moe_v2_gather_quant.h", - "csrc/dispatch_ffn_combine/op_kernel/moe_init_routing_quant_v2/moe_v2_init_routing_fullload.h", - "csrc/dispatch_ffn_combine/op_kernel/moe_init_routing_quant_v2/moe_v2_mrgsort.h", - "csrc/dispatch_ffn_combine/op_kernel/moe_init_routing_quant_v2/moe_v2_mrgsort_out.h", - "csrc/dispatch_ffn_combine/op_kernel/moe_init_routing_quant_v2/moe_v2_sort_base.h", - "csrc/dispatch_ffn_combine/op_kernel/moe_init_routing_quant_v2/moe_v2_sort_multi_core.h", - "csrc/dispatch_ffn_combine/op_kernel/moe_init_routing_quant_v2/moe_v2_sort_one_core.h", - "csrc/dispatch_ffn_combine/op_kernel/moe_init_routing_quant_v2/moe_v2_src_to_dst_and_gather.h", - "csrc/dispatch_ffn_combine/op_kernel/moe_init_routing_quant_v2/moe_v2_src_to_dst_op.h", - "csrc/dispatch_ffn_combine/op_kernel/moe_init_routing_quant_v2/moe_v2_src_to_dst_with_capacity.h", - "csrc/dispatch_ffn_combine/op_kernel/moe_init_routing_quant_v2/tiling_base.h", - "csrc/dispatch_ffn_combine/op_kernel/unpermute/moe_token_unpermute.h", - "csrc/dispatch_ffn_combine/op_kernel/unpermute/moe_token_unpermute_tiling.h", - "csrc/dispatch_ffn_combine/op_kernel/utils/block_epilogue_pertoken_row.hpp", - "csrc/dispatch_ffn_combine/op_kernel/utils/block_epilogue_pertoken_swiglu.hpp", - "csrc/dispatch_ffn_combine/op_kernel/utils/block_mmad_preload_async_fixpipe_quant.hpp", - "csrc/dispatch_ffn_combine/op_kernel/utils/const_args.hpp", - "csrc/dispatch_ffn_combine/op_kernel/utils/copy_gm_to_l1_custom.hpp", - "csrc/dispatch_ffn_combine/op_kernel/utils/copy_l0c_to_gm_custom.hpp", - "csrc/dispatch_ffn_combine/op_kernel/utils/dispatch_policy_custom.hpp", - "csrc/dispatch_ffn_combine/op_kernel/utils/hccl_shmem.hpp", - "csrc/dispatch_ffn_combine/op_kernel/utils/layout3d.hpp", - "csrc/dispatch_ffn_combine/op_kernel/utils/select_helper.hpp", - "csrc/third_party/catlass", - "csrc/torch_binding.cpp", - "csrc/torch_binding_meta.cpp", - "tests/e2e/nightly/ops/test_dispatch_ffn_combine.py", - "vllm_ascend/ascend_forward_context.py", - "vllm_ascend/ops/fused_moe/fused_moe.py", - "vllm_ascend/ops/fused_moe/moe_comm_method.py", - "vllm_ascend/ops/fused_moe/token_dispatcher.py", - "vllm_ascend/ops/register_custom_ops.py", - "vllm_ascend/quantization/w8a8_dynamic.py", - "vllm_ascend/utils.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "fb15fec66224d828ed89174f23cc4f3b13b2e0ac", - "subject": "[MM][Patch] Remove patch for cos/sin cache (#4672)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/patch/worker/patch_qwen2_5_vl.py", - "vllm_ascend/patch/worker/patch_qwen3_vl.py" - ] - }, - { - "commit": "b3e1377a924b436c9ee86055ee468482d4eb3b4e", - "subject": "\u3010fix\u3011ops gatingtopk fix nightly ci error (#4340)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/e2e/nightly/ops/test_fused_moe.py", - "tests/ut/quantization/test_w8a8.py", - "vllm_ascend/ops/fused_moe/experts_selector.py" - ] - }, - { - "commit": "178ca1607e77454297bea8193cd1e44aff012663", - "subject": "Adopt inductor fusion and define quantization fusion pass (#4168)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/e2e/singlecard/test_quant_fusion.py", - "tests/ut/ops/test_layernorm.py", - "tests/ut/test_ascend_config.py", - "tests/ut/test_platform.py", - "vllm_ascend/ascend_config.py", - "vllm_ascend/ascend_forward_context.py", - "vllm_ascend/compilation/compiler_interface.py", - "vllm_ascend/compilation/graph_fusion_pass_manager.py", - "vllm_ascend/compilation/passes/__init__.py", - "vllm_ascend/compilation/passes/quant_fusion_pass.py", - "vllm_ascend/ops/layernorm.py", - "vllm_ascend/ops/rotary_embedding.py", - "vllm_ascend/platform.py" - ] - }, - { - "commit": "3f4c0ea0a0f09461dc3a3f8d814e9b218bb62c48", - "subject": "upgrade vLLM to 0.12.0 tag (#4647)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/workflows/_e2e_nightly_multi_node.yaml", - ".github/workflows/format_pr_body.yaml", - ".github/workflows/nightly_benchmarks.yaml", - ".github/workflows/vllm_ascend_test_nightly_a2.yaml", - ".github/workflows/vllm_ascend_test_nightly_a3.yaml", - ".github/workflows/vllm_ascend_test_pr_full.yaml", - ".github/workflows/vllm_ascend_test_pr_light.yaml", - ".github/workflows/vllm_ascend_test_report.yaml", - "Dockerfile", - "Dockerfile.310p", - "Dockerfile.310p.openEuler", - "Dockerfile.a3", - "Dockerfile.a3.openEuler", - "Dockerfile.openEuler", - "docs/source/community/versioning_policy.md", - "docs/source/conf.py", - "tests/ut/test_platform.py", - "vllm_ascend/patch/worker/__init__.py", - "vllm_ascend/patch/worker/patch_deepseek.py", - "vllm_ascend/platform.py", - "vllm_ascend/torchair/models/qwen3_moe.py", - "vllm_ascend/utils.py" - ] - }, - { - "commit": "15dc01f0502f7ee2f3efe38c279927988f9001d1", - "subject": "[Fix] Fix FIA `query` and `query_start_loc` shape mismatch error (#4518)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/attention/attention_v1.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "18b90b501d6aad1d9426dcdee1ccfbe8139dd47d", - "subject": "[kernel] add AscendC op: lightning_indexer and sparse_flash_attention (#4625)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "csrc/build_aclnn.sh", - "csrc/lightning_indexer/op_host/CMakeLists.txt", - "csrc/lightning_indexer/op_host/lightning_indexer_def.cpp", - "csrc/lightning_indexer/op_host/lightning_indexer_proto.cpp", - "csrc/lightning_indexer/op_host/lightning_indexer_tiling.cpp", - "csrc/lightning_indexer/op_host/lightning_indexer_tiling.h", - "csrc/lightning_indexer/op_kernel/lightning_indexer.cpp", - "csrc/lightning_indexer/op_kernel/lightning_indexer_common.h", - "csrc/lightning_indexer/op_kernel/lightning_indexer_kernel.h", - "csrc/lightning_indexer/op_kernel/lightning_indexer_service_cube.h", - "csrc/lightning_indexer/op_kernel/lightning_indexer_service_vector.h", - "csrc/lightning_indexer/op_kernel/lightning_indexer_template_tiling_key.h", - "csrc/lightning_indexer/op_kernel/lightning_indexer_vector.h", - "csrc/sparse_flash_attention/op_host/CMakeLists.txt", - "csrc/sparse_flash_attention/op_host/sparse_flash_attention_def.cpp", - "csrc/sparse_flash_attention/op_host/sparse_flash_attention_proto.cpp", - "csrc/sparse_flash_attention/op_host/sparse_flash_attention_tiling.cpp", - "csrc/sparse_flash_attention/op_host/sparse_flash_attention_tiling.h", - "csrc/sparse_flash_attention/op_kernel/sparse_flash_attention.cpp", - "csrc/sparse_flash_attention/op_kernel/sparse_flash_attention_common.h", - "csrc/sparse_flash_attention/op_kernel/sparse_flash_attention_kernel_mla.h", - "csrc/sparse_flash_attention/op_kernel/sparse_flash_attention_service_cube_mla.h", - "csrc/sparse_flash_attention/op_kernel/sparse_flash_attention_service_vector_mla.h", - "csrc/sparse_flash_attention/op_kernel/sparse_flash_attention_template_tiling_key.h", - "csrc/torch_binding.cpp", - "csrc/torch_binding_meta.cpp", - "vllm_ascend/attention/sfa_v1.py", - "vllm_ascend/worker/worker_v1.py" - ] - }, - { - "commit": "7f2673ea2d41244bc4e07f4b6f008c4f021c41d1", - "subject": "upgrade vLLM to main (#4608)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/workflows/_e2e_nightly_multi_node.yaml", - ".github/workflows/format_pr_body.yaml", - ".github/workflows/nightly_benchmarks.yaml", - ".github/workflows/vllm_ascend_test_nightly_a2.yaml", - ".github/workflows/vllm_ascend_test_nightly_a3.yaml", - ".github/workflows/vllm_ascend_test_pr_full.yaml", - ".github/workflows/vllm_ascend_test_pr_light.yaml", - ".github/workflows/vllm_ascend_test_report.yaml", - "Dockerfile", - "Dockerfile.310p", - "Dockerfile.310p.openEuler", - "Dockerfile.a3", - "Dockerfile.a3.openEuler", - "Dockerfile.openEuler", - "docs/source/conf.py", - "tests/ut/attention/test_mla_v1.py", - "tests/ut/compilation/test_acl_graph.py", - "tests/ut/core/test_scheduler.py", - "tests/ut/kv_connector/utils.py", - "tests/ut/spec_decode/test_eagle_proposer.py", - "tests/ut/torchair/models/test_torchair_deepseek_v2.py", - "tests/ut/torchair/test_torchair_mla.py", - "tests/ut/torchair/test_torchair_sfa.py", - "vllm_ascend/attention/attention_v1.py", - "vllm_ascend/attention/mla_v1.py", - "vllm_ascend/core/recompute_scheduler.py", - "vllm_ascend/core/scheduler.py", - "vllm_ascend/core/scheduler_dynamic_batch.py", - "vllm_ascend/distributed/cpu_offload_connector.py", - "vllm_ascend/distributed/cpu_offload_manager/cpu_kv_cache_manager.py", - "vllm_ascend/distributed/cpu_offload_manager/metadata.py", - "vllm_ascend/distributed/kvpool/ascend_store_connector.py", - "vllm_ascend/distributed/kvpool/backend/memcache_backend.py", - "vllm_ascend/distributed/kvpool/backend/mooncake_backend.py", - "vllm_ascend/distributed/kvpool/config_data.py", - "vllm_ascend/distributed/kvpool/kv_transfer.py", - "vllm_ascend/distributed/kvpool/pool_scheduler.py", - "vllm_ascend/distributed/kvpool/pool_worker.py", - "vllm_ascend/distributed/llmdatadist_c_mgr_connector.py", - "vllm_ascend/distributed/mooncake_connector.py", - "vllm_ascend/distributed/mooncake_layerwise_connector.py", - "vllm_ascend/kv_offload/cpu_npu.py", - "vllm_ascend/ops/mla.py", - "vllm_ascend/patch/worker/patch_qwen2_5_vl.py", - "vllm_ascend/patch/worker/patch_qwen3_vl.py", - "vllm_ascend/platform.py", - "vllm_ascend/spec_decode/eagle_proposer.py", - "vllm_ascend/spec_decode/interface.py", - "vllm_ascend/spec_decode/mtp_proposer.py", - "vllm_ascend/spec_decode/ngram_proposer.py", - "vllm_ascend/torchair/models/qwen2.py", - "vllm_ascend/torchair/models/qwen3_moe.py", - "vllm_ascend/torchair/models/torchair_deepseek_v2.py", - "vllm_ascend/torchair/models/torchair_pangu_moe.py", - "vllm_ascend/torchair/ops/torchair_fused_moe.py", - "vllm_ascend/torchair/torchair_mla.py", - "vllm_ascend/torchair/torchair_mtp_proposer.py", - "vllm_ascend/torchair/torchair_sfa.py", - "vllm_ascend/utils.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "4588cdac02449d4c5e6e5d1ac025e0a99cd37726", - "subject": "[Bugfix] fix custom op GmmSwigluQuantWeightNzTensorList (#4593)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "CMakeLists.txt", - "csrc/CMakeLists.txt", - "csrc/build_aclnn.sh", - "csrc/torch_binding.cpp", - "vllm_ascend/platform.py", - "vllm_ascend/utils.py" - ] - }, - { - "commit": "6360eb1deab4e118116f563fc3d39167c3d6e09e", - "subject": "Revert \"[Bugfix] Fix Qwen2.5-Omni-7B accuarcy test (#4556)\" (#4619)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/ops/layernorm.py", - "vllm_ascend/ops/register_custom_ops.py" - ] - }, - { - "commit": "2fa39451121d4ab002a8621fe4c4226abe0db378", - "subject": "[Bugfix]Fix eplb enable when using mtp float weights. (#4571)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/ops/fused_moe/fused_moe.py" - ] - }, - { - "commit": "71e9b379c84d4d68764fcda4ef833e21884f8341", - "subject": "[Bugfix] Fix Qwen2.5-Omni-7B accuarcy test (#4556)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/ops/layernorm.py", - "vllm_ascend/ops/register_custom_ops.py" - ] - }, - { - "commit": "b4bf01ead16958068dac4c2972ac026bfe7e6c53", - "subject": "[Refactor] Remove redundant attention operator branches. (#4531)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/attention/test_attention_v1.py", - "vllm_ascend/attention/attention_v1.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "a9c4b8604a23c87fad744cbe75371bb9dec5d050", - "subject": "[main][bugfix] bugfix for qwen3 moe quantization (#4599)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "examples/offline_data_parallel.py", - "tests/e2e/multicard/test_data_parallel.py", - "vllm_ascend/quantization/quant_config.py", - "vllm_ascend/torchair/ops/torchair_fused_moe.py" - ] - }, - { - "commit": "27b09ca9b9515a271683a626cec85d33534520b4", - "subject": "[CI] drop ascend scheduler test (#4582)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/_e2e_test.yaml", - "tests/e2e/310p/test_offline_inference_parallel_310p.py", - "tests/e2e/multicard/test_expert_parallel.py", - "tests/e2e/multicard/test_fused_moe_allgather_ep.py", - "tests/e2e/multicard/test_offline_inference_distributed.py", - "tests/e2e/multicard/test_prefix_caching.py", - "tests/e2e/multicard/test_qwen3_next.py", - "tests/e2e/multicard/test_torchair_graph_mode.py", - "tests/e2e/nightly/features/test_mtpx_deepseek_r1_0528_w8a8.py", - "tests/e2e/nightly/features/test_prefix_cache_deepseek_r1_0528_w8a8.py", - "tests/e2e/nightly/features/test_prefix_cache_qwen3_32b_int8.py", - "tests/e2e/nightly/features/test_qwen3_32b_int8_a3_feature_stack3.py", - "tests/e2e/nightly/models/test_deepseek_r1_0528_w8a8.py", - "tests/e2e/nightly/models/test_deepseek_r1_w8a8_eplb.py", - "tests/e2e/nightly/models/test_deepseek_v3_2_exp_w8a8.py", - "tests/e2e/nightly/models/test_qwen2_5_vl_32b.py", - "tests/e2e/nightly/models/test_qwen3_235b_a22b_w8a8_eplb.py", - "tests/e2e/nightly/models/test_qwen3_235b_w8a8.py", - "tests/e2e/nightly/models/test_qwq_32b.py", - "tests/e2e/nightly/multi_node/config/models/DeepSeek-R1-W8A8-A2-torchair.yaml", - "tests/e2e/nightly/multi_node/config/models/DeepSeek-R1-W8A8-A2.yaml", - "tests/e2e/nightly/multi_node/config/models/DeepSeek-R1-W8A8-EPLB.yaml", - "tests/e2e/nightly/multi_node/config/models/DeepSeek-R1-W8A8.yaml", - "tests/e2e/nightly/multi_node/config/models/DeepSeek-V3_2-Exp-bf16.yaml", - "tests/e2e/singlecard/spec_decode_v1/test_v1_mtp_correctness.py", - "tests/e2e/singlecard/test_ascend_scheduler.py", - "tests/e2e/singlecard/test_chunked.py", - "tests/e2e/singlecard/test_vlm.py" - ] - }, - { - "commit": "c68ddc11ce53334fc9a17bad58342148cbf14e86", - "subject": "[OPS] add bmm_transpose ops (#3990)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - ".pre-commit-config.yaml", - "CMakeLists.txt", - "csrc/batch_matmul_transpose/op_host/batch_matmul_transpose.h", - "csrc/batch_matmul_transpose/op_host/common.h", - "csrc/batch_matmul_transpose/op_host/common_tiling.h", - "csrc/batch_matmul_transpose/op_host/tiling/tiling_data.cpp", - "csrc/batch_matmul_transpose/op_host/tiling/tiling_data.h", - "csrc/batch_matmul_transpose/op_kernel/batch_matmul_transpose_kernel.cpp", - "csrc/kernels/math_utils.h", - "csrc/ops.h", - "csrc/torch_binding.cpp", - "csrc/torch_binding_meta.cpp", - "tests/e2e/nightly/ops/test_batch_matmul_transpose.py", - "typos.toml", - "vllm_ascend/attention/mla_v1.py" - ] - }, - { - "commit": "bc67696a02fffff3b5d246efe17d0a01e52ab2a4", - "subject": "[EPLB][Ops] Integerate grouped_matmul_swiglu_quant_weight_nz_tensor_list operator into dynamic EPLB (#4216)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/ops/test_moe_comm_method.py", - "vllm_ascend/eplb/adaptor/vllm_adaptor.py", - "vllm_ascend/ops/fused_moe/moe_comm_method.py", - "vllm_ascend/ops/fused_moe/moe_mlp.py", - "vllm_ascend/quantization/w4a8_dynamic.py", - "vllm_ascend/quantization/w8a8_dynamic.py" - ] - }, - { - "commit": "18eefc23c3fd7275c6d2a9f540de80f2bf5e7f5f", - "subject": "[feature] Support W8A8 PD-Mix Quantization (#4235)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/ops/fused_moe/fused_moe.py", - "vllm_ascend/ops/linear.py", - "vllm_ascend/quantization/utils.py", - "vllm_ascend/quantization/w8a8.py", - "vllm_ascend/quantization/w8a8_dynamic.py", - "vllm_ascend/quantization/w8a8_pdmix.py" - ] - }, - { - "commit": "2b3bfe432e886b4773ef5cfa33a0e69b2c7d5b6d", - "subject": "[bugfix] Repair the problem of moe model accuracy caused by version upgrade. (#4562)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/ops/fused_moe/fused_moe.py" - ] - }, - { - "commit": "517fd9272d911156d563a71ba2bec0a7647b379b", - "subject": "Revert \"drop ascend scheduler\" (#4580)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/_e2e_test.yaml", - "docs/source/tutorials/DeepSeek-V3.2-Exp.md", - "docs/source/tutorials/multi_node.md", - "docs/source/tutorials/multi_node_kimi.md", - "docs/source/tutorials/multi_npu_moge.md", - "docs/source/user_guide/configuration/additional_config.md", - "docs/source/user_guide/feature_guide/graph_mode.md", - "examples/offline_inference_npu_long_seq.py", - "examples/run_dp_server.sh", - "tests/e2e/310p/test_offline_inference_parallel_310p.py", - "tests/e2e/multicard/test_expert_parallel.py", - "tests/e2e/multicard/test_fused_moe_allgather_ep.py", - "tests/e2e/multicard/test_offline_inference_distributed.py", - "tests/e2e/multicard/test_prefix_caching.py", - "tests/e2e/multicard/test_qwen3_next.py", - "tests/e2e/multicard/test_torchair_graph_mode.py", - "tests/e2e/nightly/features/test_mtpx_deepseek_r1_0528_w8a8.py", - "tests/e2e/nightly/features/test_prefix_cache_deepseek_r1_0528_w8a8.py", - "tests/e2e/nightly/features/test_prefix_cache_qwen3_32b_int8.py", - "tests/e2e/nightly/features/test_qwen3_32b_int8_a3_feature_stack3.py", - "tests/e2e/nightly/models/test_deepseek_r1_0528_w8a8.py", - "tests/e2e/nightly/models/test_deepseek_r1_w8a8_eplb.py", - "tests/e2e/nightly/models/test_deepseek_v3_2_exp_w8a8.py", - "tests/e2e/nightly/models/test_qwen2_5_vl_32b.py", - "tests/e2e/nightly/models/test_qwen3_235b_a22b_w8a8_eplb.py", - "tests/e2e/nightly/models/test_qwen3_235b_w8a8.py", - "tests/e2e/nightly/models/test_qwq_32b.py", - "tests/e2e/nightly/multi_node/config/models/DeepSeek-R1-W8A8-A2-torchair.yaml", - "tests/e2e/nightly/multi_node/config/models/DeepSeek-R1-W8A8-A2.yaml", - "tests/e2e/nightly/multi_node/config/models/DeepSeek-R1-W8A8-EPLB.yaml", - "tests/e2e/nightly/multi_node/config/models/DeepSeek-R1-W8A8.yaml", - "tests/e2e/nightly/multi_node/config/models/DeepSeek-V3_2-Exp-bf16.yaml", - "tests/e2e/singlecard/spec_decode_v1/test_v1_mtp_correctness.py", - "tests/e2e/singlecard/test_ascend_scheduler.py", - "tests/e2e/singlecard/test_chunked.py", - "tests/e2e/singlecard/test_vlm.py", - "tests/ut/core/test_schedule_config.py", - "tests/ut/core/test_scheduler.py", - "tests/ut/ops/test_linear.py", - "tests/ut/ops/test_vocab_parallel_embedding.py", - "tests/ut/quantization/test_w8a8_dynamic.py", - "tests/ut/test_ascend_config.py", - "tests/ut/test_platform.py", - "tests/ut/test_utils.py", - "tests/ut/torchair/models/test_torchair_deepseek_v2.py", - "vllm_ascend/ascend_config.py", - "vllm_ascend/core/schedule_config.py", - "vllm_ascend/core/scheduler.py", - "vllm_ascend/platform.py", - "vllm_ascend/profiling_config.py", - "vllm_ascend/torchair/torchair_attention.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "1eb5295a1beb95d626e3e7eb56ad74a4b76aa552", - "subject": "remove qwen3-next model file (#4573)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/workflows/_e2e_test.yaml", - "vllm_ascend/models/__init__.py", - "vllm_ascend/models/qwen3_next.py", - "vllm_ascend/models/qwen3_next_mtp.py", - "vllm_ascend/spec_decode/mtp_proposer.py" - ] - }, - { - "commit": "f10acddb78d5ba24ef29e18d8663e6d9a68583f3", - "subject": "drop ascend scheduler (#4498)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/_e2e_test.yaml", - "docs/source/tutorials/DeepSeek-V3.2-Exp.md", - "docs/source/tutorials/multi_node.md", - "docs/source/tutorials/multi_node_kimi.md", - "docs/source/tutorials/multi_npu_moge.md", - "docs/source/user_guide/configuration/additional_config.md", - "docs/source/user_guide/feature_guide/graph_mode.md", - "examples/offline_inference_npu_long_seq.py", - "examples/run_dp_server.sh", - "tests/e2e/310p/test_offline_inference_parallel_310p.py", - "tests/e2e/multicard/test_expert_parallel.py", - "tests/e2e/multicard/test_fused_moe_allgather_ep.py", - "tests/e2e/multicard/test_offline_inference_distributed.py", - "tests/e2e/multicard/test_prefix_caching.py", - "tests/e2e/multicard/test_qwen3_next.py", - "tests/e2e/multicard/test_torchair_graph_mode.py", - "tests/e2e/nightly/features/test_mtpx_deepseek_r1_0528_w8a8.py", - "tests/e2e/nightly/features/test_prefix_cache_deepseek_r1_0528_w8a8.py", - "tests/e2e/nightly/features/test_prefix_cache_qwen3_32b_int8.py", - "tests/e2e/nightly/features/test_qwen3_32b_int8_a3_feature_stack3.py", - "tests/e2e/nightly/models/test_deepseek_r1_0528_w8a8.py", - "tests/e2e/nightly/models/test_deepseek_r1_w8a8_eplb.py", - "tests/e2e/nightly/models/test_deepseek_v3_2_exp_w8a8.py", - "tests/e2e/nightly/models/test_qwen2_5_vl_32b.py", - "tests/e2e/nightly/models/test_qwen3_235b_a22b_w8a8_eplb.py", - "tests/e2e/nightly/models/test_qwen3_235b_w8a8.py", - "tests/e2e/nightly/models/test_qwq_32b.py", - "tests/e2e/nightly/multi_node/config/models/DeepSeek-R1-W8A8-A2-torchair.yaml", - "tests/e2e/nightly/multi_node/config/models/DeepSeek-R1-W8A8-A2.yaml", - "tests/e2e/nightly/multi_node/config/models/DeepSeek-R1-W8A8-EPLB.yaml", - "tests/e2e/nightly/multi_node/config/models/DeepSeek-R1-W8A8.yaml", - "tests/e2e/nightly/multi_node/config/models/DeepSeek-V3_2-Exp-bf16.yaml", - "tests/e2e/singlecard/spec_decode_v1/test_v1_mtp_correctness.py", - "tests/e2e/singlecard/test_ascend_scheduler.py", - "tests/e2e/singlecard/test_chunked.py", - "tests/e2e/singlecard/test_vlm.py", - "tests/ut/core/test_schedule_config.py", - "tests/ut/core/test_scheduler.py", - "tests/ut/ops/test_linear.py", - "tests/ut/ops/test_vocab_parallel_embedding.py", - "tests/ut/quantization/test_w8a8_dynamic.py", - "tests/ut/test_ascend_config.py", - "tests/ut/test_platform.py", - "tests/ut/test_utils.py", - "tests/ut/torchair/models/test_torchair_deepseek_v2.py", - "vllm_ascend/ascend_config.py", - "vllm_ascend/core/schedule_config.py", - "vllm_ascend/core/scheduler.py", - "vllm_ascend/platform.py", - "vllm_ascend/profiling_config.py", - "vllm_ascend/torchair/torchair_attention.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "0151022ab8621d1672b50b7a60735d0ddd2e064c", - "subject": "[bugfix] dep ineffective (#4417)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/eplb/adaptor/vllm_adaptor.py", - "vllm_ascend/ops/fused_moe/fused_moe.py", - "vllm_ascend/torchair/ops/torchair_fused_moe.py" - ] - }, - { - "commit": "9af34755ff87c4884e1b949311fabe7797703a74", - "subject": "[Bugfix] Fix model run _npu_flash_attention hang issue (#4410)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/attention/test_attention_mask.py", - "vllm_ascend/attention/attention_mask.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "1c4a0468eecc30de070d1d487db1dd29388b131a", - "subject": "\u3010OPS\u3011qwen3-next support triton chunk_gated_delta_rule ops (#4070)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/workflows/_e2e_test.yaml", - "tests/e2e/multicard/test_chunk_gated_delta_rule.py", - "vllm_ascend/models/qwen3_next.py", - "vllm_ascend/ops/triton/fla/chunk.py", - "vllm_ascend/ops/triton/fla/chunk_delta_h.py", - "vllm_ascend/ops/triton/fla/chunk_o.py", - "vllm_ascend/ops/triton/fla/chunk_scaled_dot_kkt.py", - "vllm_ascend/ops/triton/fla/cumsum.py", - "vllm_ascend/ops/triton/fla/layernorm_guard.py", - "vllm_ascend/ops/triton/fla/solve_tril.py", - "vllm_ascend/ops/triton/fla/utils.py", - "vllm_ascend/ops/triton/fla/wy_fast.py", - "vllm_ascend/patch/worker/patch_triton.py" - ] - }, - { - "commit": "554f16ae1fb89b35ad82b24e7f7fe5eaba0e80d0", - "subject": "[Kernel] add custom op GmmSwigluQuantWeightNzTensorList (#3804)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/release_whl.yml", - ".pre-commit-config.yaml", - "CMakeLists.txt", - "csrc/CMakeLists.txt", - "csrc/build.sh", - "csrc/build_aclnn.sh", - "csrc/cmake/config.cmake", - "csrc/cmake/func.cmake", - "csrc/cmake/intf.cmake", - "csrc/cmake/intf_pub.cmake", - "csrc/cmake/modules/Findalog.cmake", - "csrc/cmake/scripts/prepare.sh", - "csrc/grouped_matmul_swiglu_quant_weight_nz_tensor_list/op_host/CMakeLists.txt", - "csrc/grouped_matmul_swiglu_quant_weight_nz_tensor_list/op_host/aclnn_grouped_matmul_swiglu_quant_weight_nz_tensor_list.cpp", - "csrc/grouped_matmul_swiglu_quant_weight_nz_tensor_list/op_host/aclnn_grouped_matmul_swiglu_quant_weight_nz_tensor_list.h", - "csrc/grouped_matmul_swiglu_quant_weight_nz_tensor_list/op_host/grouped_matmul_swiglu_quant_weight_nz_tensor_list.cpp", - "csrc/grouped_matmul_swiglu_quant_weight_nz_tensor_list/op_host/grouped_matmul_swiglu_quant_weight_nz_tensor_list.h", - "csrc/grouped_matmul_swiglu_quant_weight_nz_tensor_list/op_host/grouped_matmul_swiglu_quant_weight_nz_tensor_list_def.cpp", - "csrc/grouped_matmul_swiglu_quant_weight_nz_tensor_list/op_host/grouped_matmul_swiglu_quant_weight_nz_tensor_list_proto.cpp", - "csrc/grouped_matmul_swiglu_quant_weight_nz_tensor_list/op_host/grouped_matmul_swiglu_quant_weight_nz_tensor_list_tiling.cpp", - "csrc/grouped_matmul_swiglu_quant_weight_nz_tensor_list/op_host/grouped_matmul_swiglu_quant_weight_nz_tensor_list_tiling.h", - "csrc/grouped_matmul_swiglu_quant_weight_nz_tensor_list/op_kernel/grouped_matmul_swiglu_quant_weight_nz_tensor_list.cpp", - "csrc/grouped_matmul_swiglu_quant_weight_nz_tensor_list/op_kernel/grouped_matmul_swiglu_quant_weight_nz_tensor_list.h", - "csrc/grouped_matmul_swiglu_quant_weight_nz_tensor_list/op_kernel/grouped_matmul_swiglu_quant_weight_nz_tensor_list_split_ws.h", - "csrc/grouped_matmul_swiglu_quant_weight_nz_tensor_list/op_kernel/grouped_matmul_swiglu_quant_weight_nz_tensor_list_utils.h", - "csrc/torch_binding.cpp", - "csrc/torch_binding_meta.cpp", - "csrc/utils/CMakeLists.txt", - "csrc/utils/inc/aclnn_util.h", - "csrc/utils/inc/error/ops_error.h", - "csrc/utils/inc/fallback.h", - "csrc/utils/inc/fallback_comm.h", - "csrc/utils/inc/kernel/dropmask.h", - "csrc/utils/inc/kernel/pse.h", - "csrc/utils/inc/kernel/util.h", - "csrc/utils/inc/log/inner/dfx_base.h", - "csrc/utils/inc/log/ops_log.h", - "csrc/utils/inc/tiling/data_copy_transpose_tiling.h", - "csrc/utils/inc/tiling/data_copy_transpose_tiling_def.h", - "csrc/utils/inc/tiling/tiling_base.h", - "csrc/utils/inc/tiling/tiling_templates_registry.h", - "csrc/utils/inc/tiling/tiling_type.h", - "csrc/utils/src/fallback_comm.cpp", - "docs/source/developer_guide/feature_guide/add_custom_aclnn_op.md", - "docs/source/developer_guide/feature_guide/index.md", - "pyproject.toml", - "setup.py", - "tests/e2e/nightly/ops/test_gmm_swiglu_quant_weight_nz_tensor_list.py", - "vllm_ascend/_cann_ops_custom/.gitkeep", - "vllm_ascend/platform.py" - ] - }, - { - "commit": "e52ebf86749a727b0003c0d912f641ef996af198", - "subject": "[MM][Model][Perf] Remove Qwen2.5-VL modeling files and add patch for VisionAttention (#4349)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/models/test_qwen2_5_vl.py", - "tests/ut/models/test_qwen2_5_vl_without_padding.py", - "vllm_ascend/models/__init__.py", - "vllm_ascend/models/qwen2_5_vl.py", - "vllm_ascend/models/qwen2_5_vl_without_padding.py", - "vllm_ascend/models/qwen3_vl.py", - "vllm_ascend/patch/worker/__init__.py", - "vllm_ascend/patch/worker/patch_qwen2_5_vl.py", - "vllm_ascend/patch/worker/patch_rope.py" - ] - }, - { - "commit": "bdc66972db12cc9f4b63721495fb2b103942dd25", - "subject": "[Quantization] Support compressed tensors w8a8 static and w8a8 dynamic weight (#4036)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/workflows/_e2e_test.yaml", - "docs/source/user_guide/feature_guide/index.md", - "docs/source/user_guide/feature_guide/quantization-llm-compressor.md", - "examples/quantization/llm-compressor/w8a8_int8.py", - "examples/quantization/llm-compressor/w8a8_int8_dynamic.py", - "mypy.ini", - "pyproject.toml", - "requirements.txt", - "tests/e2e/multicard/test_quantization.py", - "tests/ut/quantization/test_quant_config.py", - "tests/ut/test_platform.py", - "vllm_ascend/platform.py", - "vllm_ascend/quantization/compressed_tensors/__init__.py", - "vllm_ascend/quantization/compressed_tensors/compressed_tensors.py", - "vllm_ascend/quantization/quant_config.py", - "vllm_ascend/quantization/utils.py", - "vllm_ascend/quantization/w8a8.py", - "vllm_ascend/utils.py" - ] - }, - { - "commit": "9fdabb7b600660d1e2106907d85a1800e7cd1afe", - "subject": "[feature] Add Custom Op grouped_matmul_swiglu_quant (#4431)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/release_whl.yml", - "CMakeLists.txt", - "csrc/aclnn_torch_adapter/NPUBridge.cpp", - "csrc/aclnn_torch_adapter/NPUBridge.h", - "csrc/aclnn_torch_adapter/NPUStorageImpl.cpp", - "csrc/aclnn_torch_adapter/NPUStorageImpl.h", - "csrc/aclnn_torch_adapter/op_api_common.h", - "csrc/torch_binding.cpp", - "csrc/torch_binding_meta.cpp", - "tests/e2e/nightly/ops/test_grouped_matmul_swiglu_quant.py" - ] - }, - { - "commit": "136ea9ff560b610f0db0065dc520062acbbe4283", - "subject": "[refact] unified soc_version code (#4359)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "examples/disaggregated_prefill_v1/gen_ranktable.py", - "setup.py", - "tests/ut/attention/test_attention_v1.py", - "tests/ut/models/conftest.py", - "tests/ut/ops/test_activation.py", - "tests/ut/ops/test_fused_moe.py", - "tests/ut/ops/test_layernorm.py", - "tests/ut/ops/test_rotary_embedding.py", - "tests/ut/ops/test_token_dispatcher.py", - "tests/ut/quantization/test_w8a8.py", - "tests/ut/test_platform.py", - "tests/ut/test_utils.py", - "tests/ut/torchair/ops/test_torchair_fused_moe.py", - "tests/ut/torchair/ops/test_torchair_rotary_embedding.py", - "tests/ut/torchair/quantization/test_torchair_w8a8_dynamic.py", - "tests/ut/worker/test_model_runner_v1.py", - "tests/ut/worker/test_worker_v1.py", - "vllm_ascend/attention/attention_v1.py", - "vllm_ascend/distributed/llmdatadist_c_mgr_connector.py", - "vllm_ascend/envs.py", - "vllm_ascend/lora/punica_npu.py", - "vllm_ascend/ops/activation.py", - "vllm_ascend/ops/fused_moe/fused_moe.py", - "vllm_ascend/ops/fused_moe/moe_mlp.py", - "vllm_ascend/ops/fused_moe/token_dispatcher.py", - "vllm_ascend/ops/layernorm.py", - "vllm_ascend/ops/rotary_embedding.py", - "vllm_ascend/patch/platform/patch_distributed.py", - "vllm_ascend/platform.py", - "vllm_ascend/quantization/w8a8.py", - "vllm_ascend/sample/sampler.py", - "vllm_ascend/torchair/models/torchair_pangu_moe.py", - "vllm_ascend/torchair/ops/torchair_activation.py", - "vllm_ascend/torchair/ops/torchair_fused_moe.py", - "vllm_ascend/torchair/ops/torchair_layernorm.py", - "vllm_ascend/torchair/ops/torchair_rotary_embedding.py", - "vllm_ascend/torchair/quantization/torchair_w8a8_dynamic.py", - "vllm_ascend/torchair/torchair_attention.py", - "vllm_ascend/torchair/torchair_model_runner.py", - "vllm_ascend/utils.py", - "vllm_ascend/worker/model_runner_v1.py", - "vllm_ascend/worker/worker_v1.py" - ] - }, - { - "commit": "bc69d7cfe1beb68f262a515e97175dcfa1e0baab", - "subject": "upgrade to vllm 0.11.2 (#4400)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/workflows/_e2e_nightly_multi_node.yaml", - ".github/workflows/format_pr_body.yaml", - ".github/workflows/nightly_benchmarks.yaml", - ".github/workflows/vllm_ascend_test.yaml", - ".github/workflows/vllm_ascend_test_full.yaml", - ".github/workflows/vllm_ascend_test_nightly_a2.yaml", - ".github/workflows/vllm_ascend_test_nightly_a3.yaml", - ".github/workflows/vllm_ascend_test_report.yaml", - "Dockerfile", - "Dockerfile.310p", - "Dockerfile.310p.openEuler", - "Dockerfile.a3", - "Dockerfile.a3.openEuler", - "Dockerfile.openEuler", - "docs/source/community/versioning_policy.md", - "docs/source/conf.py", - "tests/e2e/multicard/test_prefix_caching.py", - "tests/e2e/multicard/test_qwen3_next.py", - "tests/e2e/singlecard/test_ascend_scheduler.py", - "tests/ut/attention/test_attention_v1.py", - "tests/ut/compilation/test_acl_graph.py", - "tests/ut/core/test_schedule_config.py", - "tests/ut/core/test_scheduler.py", - "tests/ut/kv_connector/utils.py", - "tests/ut/test_platform.py", - "tests/ut/torchair/test_torchair_worker.py", - "tests/ut/worker/test_worker_v1.py", - "vllm_ascend/attention/attention_v1.py", - "vllm_ascend/compilation/acl_graph.py", - "vllm_ascend/core/recompute_scheduler.py", - "vllm_ascend/core/scheduler.py", - "vllm_ascend/core/scheduler_dynamic_batch.py", - "vllm_ascend/models/qwen2_5_vl.py", - "vllm_ascend/models/qwen2_5_vl_without_padding.py", - "vllm_ascend/models/qwen3_next.py", - "vllm_ascend/ops/fused_moe/fused_moe.py", - "vllm_ascend/ops/triton/fla/sigmoid_gating.py", - "vllm_ascend/patch/platform/__init__.py", - "vllm_ascend/patch/platform/patch_dynamo_vllm_backend.py", - "vllm_ascend/patch/platform/patch_multiproc_executor.py", - "vllm_ascend/platform.py", - "vllm_ascend/spec_decode/eagle_proposer.py", - "vllm_ascend/spec_decode/interface.py", - "vllm_ascend/spec_decode/mtp_proposer.py", - "vllm_ascend/spec_decode/ngram_proposer.py", - "vllm_ascend/torchair/models/qwen2.py", - "vllm_ascend/torchair/models/qwen3_moe.py", - "vllm_ascend/torchair/models/torchair_deepseek_v2.py", - "vllm_ascend/torchair/models/torchair_pangu_moe.py", - "vllm_ascend/torchair/torchair_mla.py", - "vllm_ascend/torchair/torchair_mtp_proposer.py", - "vllm_ascend/worker/model_runner_v1.py", - "vllm_ascend/worker/npu_input_batch.py", - "vllm_ascend/worker/worker_v1.py" - ] - }, - { - "commit": "5b1a7514eb101c69e4925c22d83d02f3219f2d20", - "subject": "[Bugfix][MoE] enable force_load_balance in aclgraph (#4366)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/ops/fused_moe/fused_moe.py" - ] - }, - { - "commit": "ae068a334297efcb7cb54e51bfa2d7c3a00ddbd6", - "subject": "[Refactor] remove moe type of multicast. (#4224)", - "topics": [ - "communication", - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/ops/test_prepare_finalize.py", - "tests/ut/torchair/ops/test_torchair_fused_moe.py", - "tests/ut/worker/test_model_runner_v1.py", - "vllm_ascend/ascend_forward_context.py", - "vllm_ascend/ops/fused_moe/moe_comm_method.py", - "vllm_ascend/ops/fused_moe/prepare_finalize.py", - "vllm_ascend/spec_decode/eagle_proposer.py", - "vllm_ascend/spec_decode/mtp_proposer.py", - "vllm_ascend/torchair/torchair_mtp_proposer.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "a1f142b7ad5ba6f0c0b6e52b3ee99bf3ce985f1a", - "subject": "Drop 0.11.0 support (#4377)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/workflows/_e2e_nightly_multi_node.yaml", - ".github/workflows/nightly_benchmarks.yaml", - ".github/workflows/vllm_ascend_test.yaml", - ".github/workflows/vllm_ascend_test_full.yaml", - ".github/workflows/vllm_ascend_test_nightly_a2.yaml", - ".github/workflows/vllm_ascend_test_nightly_a3.yaml", - ".github/workflows/vllm_ascend_test_report.yaml", - "Dockerfile", - "Dockerfile.310p", - "Dockerfile.310p.openEuler", - "Dockerfile.a3", - "Dockerfile.a3.openEuler", - "Dockerfile.openEuler", - "examples/offline_data_parallel.py", - "examples/offline_external_launcher.py", - "examples/offline_inference_sleep_mode_npu.py", - "examples/offline_weight_load.py", - "tests/e2e/conftest.py", - "tests/e2e/multicard/test_aclgraph_capture_replay.py", - "tests/e2e/multicard/test_single_request_aclgraph.py", - "tests/e2e/nightly/models/test_qwen3_32b.py", - "tests/e2e/singlecard/multi-modal/test_internvl.py", - "tests/e2e/singlecard/test_camem.py", - "tests/ut/core/test_scheduler.py", - "tests/ut/kv_connector/test_mooncake_connector.py", - "tests/ut/kv_connector/utils.py", - "tests/ut/model_loader/netloader/test_netloader.py", - "tests/ut/models/test_mla.py", - "tests/ut/test_platform.py", - "tests/ut/test_utils.py", - "tests/ut/torchair/test_torchair_mtp_proposer.py", - "tests/ut/torchair/test_torchair_worker.py", - "tests/ut/worker/test_input_batch.py", - "tests/ut/worker/test_worker_v1.py", - "vllm_ascend/attention/attention_v1.py", - "vllm_ascend/attention/mla_v1.py", - "vllm_ascend/core/recompute_scheduler.py", - "vllm_ascend/core/scheduler.py", - "vllm_ascend/core/scheduler_dynamic_batch.py", - "vllm_ascend/distributed/cpu_offload_manager/metadata.py", - "vllm_ascend/distributed/llmdatadist_c_mgr_connector.py", - "vllm_ascend/distributed/mooncake/config_data.py", - "vllm_ascend/distributed/mooncake/mooncake_engine.py", - "vllm_ascend/distributed/mooncake/mooncake_store.py", - "vllm_ascend/distributed/mooncake/mooncake_store_connector_v1.py", - "vllm_ascend/distributed/mooncake_connector.py", - "vllm_ascend/distributed/mooncake_layerwise_connector.py", - "vllm_ascend/kv_offload/cpu_npu.py", - "vllm_ascend/lora/punica_npu.py", - "vllm_ascend/model_loader/netloader/netloader.py", - "vllm_ascend/models/layers/mla.py", - "vllm_ascend/models/qwen2_5_vl.py", - "vllm_ascend/models/qwen2_vl.py", - "vllm_ascend/models/qwen3_next.py", - "vllm_ascend/ops/fused_moe/fused_moe.py", - "vllm_ascend/ops/register_custom_ops.py", - "vllm_ascend/patch/platform/patch_mamba_config.py", - "vllm_ascend/patch/platform/patch_multiproc_executor.py", - "vllm_ascend/patch/worker/__init__.py", - "vllm_ascend/patch/worker/patch_deepseek_mtp.py", - "vllm_ascend/patch/worker/patch_deepseek_v3_2.py", - "vllm_ascend/patch/worker/patch_triton.py", - "vllm_ascend/patch/worker/patch_weight_loader.py", - "vllm_ascend/platform.py", - "vllm_ascend/quantization/w8a8_dynamic.py", - "vllm_ascend/sample/rejection_sampler.py", - "vllm_ascend/spec_decode/eagle_proposer.py", - "vllm_ascend/spec_decode/mtp_proposer.py", - "vllm_ascend/torchair/models/qwen3_moe.py", - "vllm_ascend/torchair/models/torchair_deepseek_v2.py", - "vllm_ascend/torchair/ops/torchair_fused_moe.py", - "vllm_ascend/torchair/torchair_attention.py", - "vllm_ascend/torchair/torchair_mla.py", - "vllm_ascend/torchair/torchair_mtp_proposer.py", - "vllm_ascend/torchair/torchair_sfa.py", - "vllm_ascend/utils.py", - "vllm_ascend/worker/block_table.py", - "vllm_ascend/worker/model_runner_v1.py", - "vllm_ascend/worker/npu_input_batch.py", - "vllm_ascend/worker/worker_v1.py" - ] - }, - { - "commit": "3955bf29082f1e57e88ca58d2bfce63ba28301c6", - "subject": "[EPLB] Eplb Verify Fix (#4333)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/ops/fused_moe/fused_moe.py" - ] - }, - { - "commit": "8e3b834bf723f91d1a825559bfbaf3eff4b3d281", - "subject": "[MM][Bugfix] Add error log for VL models when enabling FLASHCOMM (#4272)", - "topics": [ - "communication" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/platform.py", - "vllm_ascend/utils.py" - ] - }, - { - "commit": "4573c855b7f8903321573470388c723aab41c8d8", - "subject": "[Readme] EPLB Support Scenarios (#4314)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/ops/fused_moe/fused_moe.py" - ] - }, - { - "commit": "019c7ded91f0c2a9533ace8d87f0cf56e396f979", - "subject": "eplb redundant expert bugfix (#4291)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/user_guide/feature_guide/eplb_swift_balancer.md", - "tests/ut/eplb/core/test_eplb_utils.py", - "tests/ut/ops/test_expert_load_balancer.py", - "vllm_ascend/eplb/core/eplb_utils.py", - "vllm_ascend/ops/expert_load_balancer.py", - "vllm_ascend/ops/fused_moe/fused_moe.py", - "vllm_ascend/quantization/w4a8_dynamic.py", - "vllm_ascend/quantization/w8a8_dynamic.py", - "vllm_ascend/torchair/ops/torchair_fused_moe.py", - "vllm_ascend/torchair/quantization/torchair_w8a8_dynamic.py" - ] - }, - { - "commit": "5c9f4a40c64938af19bcb3368201c02118dc8bf4", - "subject": "[Feat] Support MTP to running in full graph mode (#3892)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/attention/test_mla_v1.py", - "tests/ut/compilation/test_acl_graph.py", - "vllm_ascend/ascend_forward_context.py", - "vllm_ascend/attention/mla_v1.py", - "vllm_ascend/compilation/acl_graph.py", - "vllm_ascend/patch/worker/patch_deepseek_mtp.py", - "vllm_ascend/spec_decode/mtp_proposer.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "3653f33878d025a5d2b641f930fa98dee9288ed6", - "subject": "avoid mrope fusion op when running qwen2.5-vl on a+x machine (#4270)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/ops/test_rotary_embedding.py", - "vllm_ascend/ops/rotary_embedding.py" - ] - }, - { - "commit": "a3e9673137dd31d77b7441d657a0fc394ac7bc59", - "subject": "[long seq feat]GQA support long-prefill-token-threshold and fixbug (#4209)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/attention/attention_v1.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "9328f377b4dd3c6dd8915b7c60ae01335fc554c4", - "subject": "[refactor]support gatingtopk operator generalization (#2958)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/quantization/test_w8a8.py", - "vllm_ascend/ascend_forward_context.py", - "vllm_ascend/ops/fused_moe/experts_selector.py" - ] - }, - { - "commit": "10a046ddce2870f5d6c2a77214e6e010cc4dab45", - "subject": "[main][misc]change default capture size for Qwen3-MoE when using full dp (#4199)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/test_platform.py", - "vllm_ascend/platform.py", - "vllm_ascend/utils.py" - ] - }, - { - "commit": "e38ef2c4344e11130a8fae5260b9dad6962d1fd1", - "subject": "support FULL graph mode for GQA (#3970)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/workflows/_e2e_test.yaml", - "tests/e2e/multicard/test_full_graph_mode.py", - "tests/e2e/singlecard/spec_decode_v1/test_v1_mtp_correctness.py", - "tests/ut/attention/test_attention_v1.py", - "vllm_ascend/attention/attention_v1.py", - "vllm_ascend/attention/mla_v1.py", - "vllm_ascend/attention/utils.py", - "vllm_ascend/compilation/acl_graph.py", - "vllm_ascend/platform.py", - "vllm_ascend/torchair/torchair_model_runner.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "5ec96fd46cdf8e6355c3b534cb3b8d29bcf4acc3", - "subject": "[long_seq_Feat] support chunk prefill (#4158)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/attention/test_mla_v1.py", - "vllm_ascend/attention/attention_v1.py", - "vllm_ascend/attention/mla_v1.py", - "vllm_ascend/attention/utils.py", - "vllm_ascend/worker/model_runner_v1.py", - "vllm_ascend/worker/npu_input_batch.py" - ] - }, - { - "commit": "9d84172359d6f1a23cc6be7c0cc619f943529f07", - "subject": "[BugFix] adapted e2e tests for Qwen3-next-mtp (#4160)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/e2e/multicard/test_qwen3_next.py" - ] - }, - { - "commit": "509319276940b405ebdb53d40b8827d10034ece8", - "subject": "[Bugfix] fix mtp profile run error where main model and mtp model use different quantization (#4102)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/ops/test_moe_comm_method.py", - "vllm_ascend/ops/fused_moe/fused_moe.py", - "vllm_ascend/ops/fused_moe/moe_comm_method.py", - "vllm_ascend/ops/fused_moe/prepare_finalize.py", - "vllm_ascend/quantization/w4a8_dynamic.py", - "vllm_ascend/quantization/w8a8_dynamic.py" - ] - }, - { - "commit": "17259cb265da85cd5f14ac87887b4a17047be61e", - "subject": "[Perf] [MoE] optimize all2allv (#3738)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/ops/fused_moe/token_dispatcher.py" - ] - }, - { - "commit": "c272747d13c955126a59f7538a2054a24d31a285", - "subject": "Upgrade to 0.11.1 newest vllm commit (#3982)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/workflows/format_pr_body.yaml", - ".github/workflows/vllm_ascend_test.yaml", - ".github/workflows/vllm_ascend_test_full.yaml", - "docs/source/community/versioning_policy.md", - "tests/ut/worker/test_worker_v1.py", - "vllm_ascend/attention/attention_v1.py", - "vllm_ascend/attention/mla_v1.py", - "vllm_ascend/core/scheduler.py", - "vllm_ascend/distributed/mooncake/config_data.py", - "vllm_ascend/models/qwen2_5_vl.py", - "vllm_ascend/ops/sigmoid_gating.py", - "vllm_ascend/patch/platform/patch_mamba_config.py", - "vllm_ascend/patch/worker/patch_triton.py", - "vllm_ascend/spec_decode/mtp_proposer.py", - "vllm_ascend/torchair/models/torchair_deepseek_v2.py", - "vllm_ascend/torchair/torchair_attention.py", - "vllm_ascend/torchair/torchair_mla.py", - "vllm_ascend/torchair/torchair_sfa.py", - "vllm_ascend/worker/block_table.py", - "vllm_ascend/worker/model_runner_v1.py", - "vllm_ascend/worker/worker_v1.py" - ] - }, - { - "commit": "71866d531151b36a9af4db36ba8b3b74162f7028", - "subject": "[feature] chunkprefill support pcp&dcp (#3801)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/attention/test_attention_v1.py", - "tests/ut/attention/test_mla_v1.py", - "vllm_ascend/attention/attention_v1.py", - "vllm_ascend/attention/mla_v1.py", - "vllm_ascend/attention/utils.py", - "vllm_ascend/worker/block_table.py", - "vllm_ascend/worker/model_runner_v1.py", - "vllm_ascend/worker/npu_input_batch.py" - ] - }, - { - "commit": "0a62e671fb0244f799137d3b4fae471dfd74f91d", - "subject": "[Feat] flashcomm_v2 optim solution (#3232)", - "topics": [ - "communication" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/_e2e_test.yaml", - "tests/e2e/multicard/test_offline_inference_distributed.py", - "tests/e2e/singlecard/test_aclgraph_mem.py", - "tests/ut/distributed/test_parallel_state.py", - "vllm_ascend/ascend_config.py", - "vllm_ascend/ascend_forward_context.py", - "vllm_ascend/distributed/parallel_state.py", - "vllm_ascend/envs.py", - "vllm_ascend/ops/linear_op.py", - "vllm_ascend/quantization/quant_config.py", - "vllm_ascend/quantization/w8a8.py", - "vllm_ascend/utils.py" - ] - }, - { - "commit": "e687d6af855aa5475c95094d43ea1d08497a739f", - "subject": "[BugFix]Fix group list type of mc2. (#4047)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/ops/fused_moe/token_dispatcher.py" - ] - }, - { - "commit": "23b785fdfbf14d6b4eaafde66217545dacfb4863", - "subject": "[Feat] Adapted mtp function to Qwen3-next (#3918)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/e2e/multicard/test_qwen3_next.py", - "tests/ut/attention/test_attention_v1.py", - "vllm_ascend/attention/attention_v1.py", - "vllm_ascend/models/__init__.py", - "vllm_ascend/models/qwen3_next.py", - "vllm_ascend/models/qwen3_next_mtp.py", - "vllm_ascend/ops/casual_conv1d.py", - "vllm_ascend/spec_decode/__init__.py", - "vllm_ascend/spec_decode/mtp_proposer.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "f8610b7d6702d21f9e932f8d1b4f16061da1806c", - "subject": "[long_seq] fix A2 accuracy problem (#4030)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/ops/fused_moe/prepare_finalize.py" - ] - }, - { - "commit": "e0d58d543b7119980c0326fc37664be7ff2ca79f", - "subject": "[main][bugfix] Fix a rare bug triggered by _npu_paged_attention in FULL_DECODE_ONLY mode (#3986)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/compilation/acl_graph.py" - ] - }, - { - "commit": "1804b60ec8d93a8b610b7cb543ee1f493cbd8b19", - "subject": "[BugFix][main] Adapted to torch_npu.npu_fused_infer_attention_score (#4025)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/attention/attention_v1.py", - "vllm_ascend/patch/platform/patch_mamba_config.py" - ] - }, - { - "commit": "bedf22377139cd0bceaa089afc8b16ceaddee0a3", - "subject": "[Perf] move quant before allgather in Allgather EP (#3420)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/e2e/multicard/test_offline_inference_distributed.py", - "tests/ut/ops/test_fused_moe.py", - "vllm_ascend/ops/fused_moe/fused_moe.py", - "vllm_ascend/ops/fused_moe/moe_comm_method.py", - "vllm_ascend/ops/fused_moe/moe_mlp.py", - "vllm_ascend/ops/fused_moe/prepare_finalize.py", - "vllm_ascend/ops/fused_moe/token_dispatcher.py", - "vllm_ascend/ops/register_custom_ops.py", - "vllm_ascend/quantization/w4a8_dynamic.py", - "vllm_ascend/quantization/w8a8_dynamic.py" - ] - }, - { - "commit": "40c7db65599534e92a6f17cbacf2dc784bcc20b7", - "subject": "[MM][Bugfix] Add MoE verification for multi-modal models (#3897)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/utils.py" - ] - }, - { - "commit": "cc2cd42ad398f407dbdfc664522d57ba6996d8ff", - "subject": "Upgrade CANN to 8.3.rc1 (#3945)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/Dockerfile.buildwheel", - ".github/workflows/_accuracy_test.yaml", - ".github/workflows/_e2e_nightly_multi_node.yaml", - ".github/workflows/_e2e_nightly_single_node.yaml", - ".github/workflows/_kill_lws_resources.yaml", - ".github/workflows/accuracy_test.yaml", - ".github/workflows/nightly_benchmarks.yaml", - ".github/workflows/vllm_ascend_dist.yaml", - ".github/workflows/vllm_ascend_test.yaml", - ".github/workflows/vllm_ascend_test_310p.yaml", - ".github/workflows/vllm_ascend_test_full.yaml", - ".github/workflows/vllm_ascend_test_full_vllm_main.yaml", - ".github/workflows/vllm_ascend_test_models.yaml", - ".github/workflows/vllm_ascend_test_nightly_a2.yaml", - ".github/workflows/vllm_ascend_test_nightly_a3.yaml", - ".github/workflows/vllm_ascend_test_pd.yaml", - "Dockerfile", - "Dockerfile.310p", - "Dockerfile.310p.openEuler", - "Dockerfile.a3", - "Dockerfile.a3.openEuler", - "Dockerfile.openEuler", - "README.md", - "README.zh.md", - "docs/source/conf.py", - "docs/source/developer_guide/contribution/multi_node_test.md", - "docs/source/installation.md", - "examples/disaggregated_prefill_v1/mooncake_connector_deployment_guide.md", - "examples/disaggregated_prefill_v1/mooncake_connector_store_deployment_guide.md", - "tests/e2e/nightly/multi_node/scripts/lws.yaml.jinja2", - "tests/e2e/vllm_interface/vllm_test.cfg", - "tests/ut/attention/test_attention_mask.py", - "tests/ut/attention/test_attention_v1.py", - "tests/ut/ops/test_linear.py", - "vllm_ascend/attention/attention_mask.py", - "vllm_ascend/attention/attention_v1.py", - "vllm_ascend/ops/linear.py", - "vllm_ascend/ops/linear_op.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "fcc9a0eaeba43c455535dcff5c805757a95d8db4", - "subject": "Update torch-npu version to 2.7.1 (#3896)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "README.md", - "README.zh.md", - "docs/source/installation.md", - "examples/disaggregated_prefill_v1/mooncake_connector_deployment_guide.md", - "examples/disaggregated_prefill_v1/mooncake_connector_store_deployment_guide.md", - "pyproject.toml", - "requirements.txt", - "tests/ut/ops/test_layernorm.py", - "tests/ut/torchair/quantization/test_torchair_w8a8_dynamic.py", - "vllm_ascend/ascend_forward_context.py", - "vllm_ascend/attention/attention_v1.py", - "vllm_ascend/compilation/acl_graph.py", - "vllm_ascend/ops/layernorm.py", - "vllm_ascend/ops/weight_prefetch.py", - "vllm_ascend/utils.py" - ] - }, - { - "commit": "627f20ce261d7126c2607b0a7fe0f7adcef839eb", - "subject": "[BugFix]Fix group list type of mc2. (#3864)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/ops/fused_moe/fused_moe.py", - "vllm_ascend/ops/fused_moe/moe_comm_method.py", - "vllm_ascend/ops/fused_moe/token_dispatcher.py" - ] - }, - { - "commit": "f6149f3894405aa643a584851cafdceb768ffc18", - "subject": "[Model][3/N] Refactor sfa into mla and remove deepseek_v3_2.py (#3769)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/attention/mla_v1.py", - "vllm_ascend/attention/sfa_v1.py", - "vllm_ascend/models/__init__.py", - "vllm_ascend/models/deepseek_v3_2.py", - "vllm_ascend/models/layers/mla.py", - "vllm_ascend/models/layers/sfa.py", - "vllm_ascend/patch/worker/__init__.py", - "vllm_ascend/patch/worker/patch_deepseek_v3_2.py", - "vllm_ascend/torchair/models/torchair_deepseek_v2.py", - "vllm_ascend/utils.py" - ] - }, - { - "commit": "eff3e5fc6f9c5f7956f1a04c86f16c76c6256cfb", - "subject": "[FEAT] Refactor spec decode to support efficient padded speculation (#3528)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/e2e/singlecard/spec_decode_v1/test_v1_mtp_correctness.py", - "vllm_ascend/spec_decode/__init__.py", - "vllm_ascend/spec_decode/mtp_proposer.py", - "vllm_ascend/torchair/torchair_model_runner.py", - "vllm_ascend/torchair/torchair_mtp_proposer.py", - "vllm_ascend/worker/model_runner_v1.py", - "vllm_ascend/worker/npu_input_batch.py" - ] - }, - { - "commit": "74191864b7287f72812c9e67e5668c84d06dbd83", - "subject": "[Perf] Delete redundant operations in model_runner and forward_context (#3677)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/worker/test_model_runner_v1.py", - "vllm_ascend/ascend_forward_context.py", - "vllm_ascend/ops/fused_moe/moe_comm_method.py", - "vllm_ascend/utils.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "0d1859af0831e149153d71a0ab9141d6d4ac9b20", - "subject": "[Bugfix] [MoE] fix error in deepseek when using allgather (#3824)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/ops/test_moe_comm_method.py", - "tests/ut/ops/test_prepare_finalize.py", - "vllm_ascend/ops/fused_moe/moe_comm_method.py", - "vllm_ascend/ops/fused_moe/prepare_finalize.py", - "vllm_ascend/torchair/ops/torchair_fused_moe.py", - "vllm_ascend/torchair/utils.py", - "vllm_ascend/utils.py" - ] - }, - { - "commit": "f57bdb09fc3126a1857edf95a3bbc208312678a0", - "subject": "[long_seq_optim] BSND to TND and FA_UPDATE replacement (#3778)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/attention/attention_v1.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "d08401d1e78722fa723405c6ad6709bb56d04fcd", - "subject": "[Main][Bugfix]Avoid using the fusion operator in the MOE model (#3834)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/ascend_forward_context.py", - "vllm_ascend/ops/linear_op.py" - ] - }, - { - "commit": "00aa0bf33e72aa376501195b9df4dcfa11c41b2e", - "subject": "support prefill cache mode use fia op (#3696)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/attention/attention_mask.py", - "vllm_ascend/attention/attention_v1.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "60ee4af6d0952baffe451fc9fa4f735b96d927df", - "subject": "[CI] Add custom op to nightly (#3765)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/_e2e_nightly_multi_node.yaml", - ".github/workflows/_e2e_nightly_single_node.yaml", - ".github/workflows/vllm_ascend_test_nightly_a2.yaml", - ".github/workflows/vllm_ascend_test_nightly_a3.yaml", - "tests/e2e/nightly/ops/__init__.py", - "tests/e2e/nightly/ops/test_bgmv_expand.py", - "tests/e2e/nightly/ops/test_bgmv_shrink.py", - "tests/e2e/nightly/ops/test_fused_moe.py", - "tests/e2e/nightly/ops/test_gating_top_k_softmax.py", - "tests/e2e/nightly/ops/test_mla_preprocess.py", - "tests/e2e/nightly/ops/test_rotary_embedding.py", - "tests/e2e/nightly/ops/test_vocabparallelembedding.py" - ] - }, - { - "commit": "4312a92a4f480f2a4ba79baf184be1b55e6da139", - "subject": "[feat]dcp pcp support aclgraph (#3731)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/attention/test_mla_v1.py", - "vllm_ascend/attention/attention_v1.py", - "vllm_ascend/attention/mla_v1.py", - "vllm_ascend/compilation/acl_graph.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "d9cdc658545695bd0769f6c762317ebc90c94348", - "subject": "Upgrade to new vllm commit (#3719)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/format_pr_body.yaml", - ".github/workflows/vllm_ascend_test.yaml", - ".github/workflows/vllm_ascend_test_full.yaml", - "examples/offline_data_parallel.py", - "examples/offline_external_launcher.py", - "examples/offline_inference_sleep_mode_npu.py", - "examples/offline_weight_load.py", - "tests/e2e/conftest.py", - "tests/e2e/multicard/test_single_request_aclgraph.py", - "tests/e2e/nightly/models/test_qwen3_32b.py", - "tests/e2e/singlecard/test_camem.py", - "tests/ut/core/test_schedule_config.py", - "tests/ut/core/test_scheduler.py", - "tests/ut/kv_connector/test_mooncake_connector.py", - "tests/ut/kv_connector/utils.py", - "tests/ut/model_loader/netloader/test_netloader.py", - "tests/ut/worker/test_input_batch.py", - "tests/ut/worker/test_worker_v1.py", - "vllm_ascend/core/schedule_config.py", - "vllm_ascend/distributed/cpu_offload_manager/metadata.py", - "vllm_ascend/distributed/llmdatadist_c_mgr_connector.py", - "vllm_ascend/distributed/mooncake/mooncake_engine.py", - "vllm_ascend/distributed/mooncake_connector.py", - "vllm_ascend/distributed/mooncake_layerwise_connector.py", - "vllm_ascend/model_loader/netloader/netloader.py", - "vllm_ascend/models/layers/mla.py", - "vllm_ascend/models/layers/sfa.py", - "vllm_ascend/ops/fused_moe/fused_moe.py", - "vllm_ascend/ops/register_custom_ops.py", - "vllm_ascend/patch/platform/patch_mamba_config.py", - "vllm_ascend/patch/platform/patch_multiproc_executor.py", - "vllm_ascend/patch/worker/patch_weight_loader.py", - "vllm_ascend/platform.py", - "vllm_ascend/spec_decode/mtp_proposer.py", - "vllm_ascend/worker/model_runner_v1.py", - "vllm_ascend/worker/npu_input_batch.py", - "vllm_ascend/worker/worker_v1.py" - ] - }, - { - "commit": "63c363d3de1da8988caa05f293bd36cd956cbfde", - "subject": "[Refactor] [MoE] Rename moe-related classes & files (#3646)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/e2e/singlecard/ops/test_fused_moe.py", - "tests/ut/models/conftest.py", - "tests/ut/ops/test_comm_utils.py", - "tests/ut/ops/test_common_fused_moe.py", - "tests/ut/ops/test_fused_moe.py", - "tests/ut/ops/test_moe_comm_method.py", - "tests/ut/ops/test_prepare_finalize.py", - "tests/ut/ops/test_token_dispatcher.py", - "tests/ut/quantization/test_w8a8.py", - "vllm_ascend/ascend_forward_context.py", - "vllm_ascend/models/deepseek_v3_2.py", - "vllm_ascend/ops/__init__.py", - "vllm_ascend/ops/fused_moe/__init__.py", - "vllm_ascend/ops/fused_moe/comm_utils.py", - "vllm_ascend/ops/fused_moe/experts_selector.py", - "vllm_ascend/ops/fused_moe/fused_moe.py", - "vllm_ascend/ops/fused_moe/moe_comm_method.py", - "vllm_ascend/ops/fused_moe/moe_mlp.py", - "vllm_ascend/ops/fused_moe/prepare_finalize.py", - "vllm_ascend/ops/fused_moe/token_dispatcher.py", - "vllm_ascend/quantization/quant_config.py", - "vllm_ascend/quantization/w4a8_dynamic.py", - "vllm_ascend/quantization/w8a8.py", - "vllm_ascend/quantization/w8a8_dynamic.py", - "vllm_ascend/utils.py" - ] - }, - { - "commit": "39b994a987888f7ba78df28b1ccb41a5e8d6eaf5", - "subject": "[Feat] Add mrope fusion op (#3708)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/ops/test_rotary_embedding.py", - "vllm_ascend/ops/rotary_embedding.py", - "vllm_ascend/utils.py" - ] - }, - { - "commit": "3158742a9751234434b1f9caeefe4f9b85f530d9", - "subject": "[Refactor] Refactor Ascend attention implementation forward (#3714)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/attention/test_attention_v1.py", - "tests/ut/torchair/test_torchair_attention.py", - "vllm_ascend/attention/attention_v1.py", - "vllm_ascend/platform.py", - "vllm_ascend/torchair/models/qwen2.py", - "vllm_ascend/torchair/models/qwen3_moe.py", - "vllm_ascend/torchair/models/torchair_pangu_moe.py", - "vllm_ascend/torchair/torchair_attention.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "0b1da2474260bd30c5a70f8090c2e6d14ee5d684", - "subject": "[Main][Perf] Add fused matmul/reduce-scatter kernel for performance optimization. (#3693)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/ops/linear_op.py" - ] - }, - { - "commit": "9b0baa1182bc4c3ba27b33a739d95dfda161e555", - "subject": "[BugFix] Check all expert maps when using muilty instance. (#3576)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/_e2e_test.yaml", - "vllm_ascend/ops/common_fused_moe.py", - "vllm_ascend/ops/expert_load_balancer.py", - "vllm_ascend/torchair/ops/torchair_fused_moe.py" - ] - }, - { - "commit": "cea0755b071ab26486f93881186749731d290a57", - "subject": "[1/N][Refactor] Refactor code to adapt with vllm main (#3612)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/workflows/_e2e_test.yaml", - ".github/workflows/format_pr_body.yaml", - ".github/workflows/vllm_ascend_test.yaml", - ".github/workflows/vllm_ascend_test_full.yaml", - ".pre-commit-config.yaml", - "tests/e2e/singlecard/spec_decode_v1/test_v1_mtp_correctness.py", - "tests/ut/attention/test_mla_v1.py", - "tests/ut/core/test_scheduler.py", - "tests/ut/kv_connector/utils.py", - "tests/ut/ops/test_linear.py", - "tests/ut/test_platform.py", - "tests/ut/test_utils.py", - "tests/ut/torchair/models/test_torchair_deepseek_mtp.py", - "tests/ut/torchair/models/test_torchair_deepseek_v2.py", - "tests/ut/torchair/ops/test_torchair_fused_moe.py", - "vllm_ascend/__init__.py", - "vllm_ascend/ascend_config.py", - "vllm_ascend/attention/attention_v1.py", - "vllm_ascend/attention/mla_v1.py", - "vllm_ascend/attention/sfa_v1.py", - "vllm_ascend/core/recompute_scheduler.py", - "vllm_ascend/core/schedule_config.py", - "vllm_ascend/core/scheduler.py", - "vllm_ascend/core/scheduler_dynamic_batch.py", - "vllm_ascend/lora/punica_npu.py", - "vllm_ascend/models/deepseek_v3_2.py", - "vllm_ascend/models/layers/mla.py", - "vllm_ascend/models/layers/sfa.py", - "vllm_ascend/ops/common_fused_moe.py", - "vllm_ascend/patch/worker/patch_roberta.py", - "vllm_ascend/platform.py", - "vllm_ascend/quantization/quant_config.py", - "vllm_ascend/quantization/w8a8_dynamic.py", - "vllm_ascend/spec_decode/eagle_proposer.py", - "vllm_ascend/spec_decode/mtp_proposer.py", - "vllm_ascend/torchair/models/qwen3_moe.py", - "vllm_ascend/torchair/models/torchair_deepseek_mtp.py", - "vllm_ascend/torchair/models/torchair_deepseek_v2.py", - "vllm_ascend/torchair/ops/torchair_fused_moe.py", - "vllm_ascend/torchair/torchair_attention.py", - "vllm_ascend/torchair/torchair_mla.py", - "vllm_ascend/torchair/torchair_model_runner.py", - "vllm_ascend/torchair/torchair_sfa.py", - "vllm_ascend/utils.py", - "vllm_ascend/worker/model_runner_v1.py", - "vllm_ascend/worker/npu_input_batch.py", - "vllm_ascend/worker/worker_v1.py" - ] - }, - { - "commit": "1b270a64bd8bf6766fec09a776de4c1efb88bb32", - "subject": "[MoE][Multistream] Avoid performing communication in extra stream. (#3582)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/e2e/singlecard/test_multistream_overlap_shared_expert.py", - "vllm_ascend/ops/common_fused_moe.py", - "vllm_ascend/utils.py" - ] - }, - { - "commit": "b54d44e6647c102614468b11bb034bc8e7b8f8fa", - "subject": "support cp&dcp (#3260)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "examples/offline_inference_npu_long_seq.py", - "tests/ut/attention/test_attention_v1.py", - "tests/ut/attention/test_mla_v1.py", - "tests/ut/kv_connector/test_llmdatadist_connector.py", - "tests/ut/kv_connector/utils.py", - "tests/ut/torchair/models/test_torchair_deepseek_v2.py", - "vllm_ascend/attention/attention_v1.py", - "vllm_ascend/attention/mla_v1.py", - "vllm_ascend/attention/utils.py", - "vllm_ascend/distributed/llmdatadist_c_mgr_connector.py", - "vllm_ascend/distributed/parallel_state.py", - "vllm_ascend/envs.py", - "vllm_ascend/platform.py", - "vllm_ascend/utils.py", - "vllm_ascend/worker/block_table.py", - "vllm_ascend/worker/model_runner_v1.py", - "vllm_ascend/worker/npu_input_batch.py", - "vllm_ascend/worker/worker_v1.py" - ] - }, - { - "commit": "292e213dd225c5bac04605e15edd2e5619ef6699", - "subject": "[main][refactor] refactor SequenceRowParallelOp forward (#3616)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/ops/test_linear.py", - "vllm_ascend/ops/linear.py", - "vllm_ascend/ops/linear_op.py", - "vllm_ascend/ops/register_custom_ops.py" - ] - }, - { - "commit": "dd7a25063ce2e7bd013e5055356a0650036ad93a", - "subject": "[Feat] Prefetching Attention QKV Linear Weight With `AddRmsNormQuant` Custom Op (#3517)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/ops/test_layernorm.py", - "vllm_ascend/ops/layernorm.py" - ] - }, - { - "commit": "2f1b9a7a641b58e49b760311d06cd81091faff12", - "subject": "Reapply \"[MoE] [Refactor] Remove manual memory cleanup (#3365)\" (#3483) (#3512)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/workflows/vllm_ascend_dist.yaml", - "tests/e2e/multicard/test_offline_inference_distributed.py", - "tests/e2e/multicard/test_qwen3_moe.py", - "tests/e2e/singlecard/ops/test_fused_moe.py", - "tests/ut/ops/test_fused_moe_prepare_and_finalize.py", - "tests/ut/ops/test_moe_comm_method.py", - "tests/ut/ops/test_token_dispatcher.py", - "vllm_ascend/ops/common_fused_moe.py", - "vllm_ascend/ops/moe/fused_moe_prepare_and_finalize.py", - "vllm_ascend/ops/moe/moe_comm_method.py", - "vllm_ascend/ops/moe/token_dispatcher.py", - "vllm_ascend/quantization/w4a8_dynamic.py", - "vllm_ascend/quantization/w8a8_dynamic.py" - ] - }, - { - "commit": "220df60c617a7a7256c820033412671dc04b2781", - "subject": "[Model][2/N] Remove deepseek_mtp modeling. (#3561)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/attention/test_mla_v1.py", - "tests/ut/models/test_deepseek_mtp.py", - "vllm_ascend/attention/mla_v1.py", - "vllm_ascend/models/__init__.py", - "vllm_ascend/models/deepseek_mtp.py", - "vllm_ascend/quantization/quant_config.py", - "vllm_ascend/spec_decode/mtp_proposer.py" - ] - }, - { - "commit": "6b290acfe109bdfd9225a6c06a89f2dcba7a4156", - "subject": "remove redundant params in mla_preprocess kernel (#3530)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "csrc/mla_preprocess/op_kernel/mla_preprocess_kernel.cpp", - "csrc/mla_preprocess/op_kernel/mla_preprocess_mix_bf16.hpp", - "csrc/mla_preprocess/op_kernel/mla_preprocess_mix_fp16.hpp", - "csrc/ops.h", - "csrc/torch_binding.cpp", - "csrc/torch_binding_meta.cpp", - "tests/e2e/singlecard/ops/test_mla_preprocess.py", - "tests/e2e/singlecard/spec_decode_v1/test_v1_mtp_correctness.py", - "vllm_ascend/attention/mla_v1.py" - ] - }, - { - "commit": "0bf3f21a987aede366ec4629ad0ffec8e32fe90d", - "subject": "Revert \"Add mrope op fusion (#3509)\" (#3562)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/ops/test_rotary_embedding.py", - "vllm_ascend/ops/rotary_embedding.py", - "vllm_ascend/utils.py" - ] - }, - { - "commit": "068ed706c8a4e6d2c35bc215b95d0ff0a76c5692", - "subject": "[feat][torchair] support super kernel feat for quantized dsr1 (#3485)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "docs/source/user_guide/configuration/additional_config.md", - "tests/ut/ops/test_linear.py", - "vllm_ascend/ascend_config.py", - "vllm_ascend/ops/linear.py", - "vllm_ascend/torchair/models/torchair_deepseek_v2.py", - "vllm_ascend/torchair/ops/torchair_fused_moe.py", - "vllm_ascend/torchair/quantization/torchair_w8a8_dynamic.py", - "vllm_ascend/torchair/utils.py" - ] - }, - { - "commit": "b9e2896eb10927f0495856b1f5b1b9071bf506a4", - "subject": "Revert \"[Perf] Add FIA interface in FA case\" (#3553)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/attention/attention_v1.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "f8b52fe9509e4b90e384d9780042765c99b50c63", - "subject": "[Model][1/N] Delete deepseek v2/v3 modeling codes. (#3189)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/attention/test_mla_v1.py", - "tests/ut/models/conftest.py", - "tests/ut/models/test_deepseek_mtp.py", - "tests/ut/models/test_deepseek_v2.py", - "tests/ut/torchair/test_torchair_mla.py", - "vllm_ascend/attention/mla_v1.py", - "vllm_ascend/models/__init__.py", - "vllm_ascend/models/deepseek_v2.py", - "vllm_ascend/models/layers/mla.py", - "vllm_ascend/ops/common_fused_moe.py", - "vllm_ascend/quantization/quant_config.py", - "vllm_ascend/torchair/models/torchair_deepseek_v2.py", - "vllm_ascend/torchair/torchair_mla.py" - ] - }, - { - "commit": "daa4dd0a577e4e32eb5c48623046f3ede93caf2f", - "subject": "[DeepSeek] Seperate deepseek v3.2 modeling form deepseek v2 (#3531)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/models/__init__.py", - "vllm_ascend/models/deepseek_v2.py", - "vllm_ascend/models/deepseek_v3.py", - "vllm_ascend/models/deepseek_v3_2.py" - ] - }, - { - "commit": "1e78ecbad626968c7e65fab5596c90ea79e7454d", - "subject": "[Perf] Add FIA interface in FA case (#3321)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/attention/attention_v1.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "6c9909c861aab7fd593f5f53e87c1c959183811c", - "subject": "[Patch]patch of v1 executor when enable eplb. (#3511)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/user_guide/feature_guide/eplb_swift_balancer.md", - "vllm_ascend/patch/platform/patch_common/__init__.py", - "vllm_ascend/patch/platform/patch_common/patch_multiproc_executor.py", - "vllm_ascend/torchair/ops/torchair_fused_moe.py", - "vllm_ascend/torchair/quantization/torchair_w8a8_dynamic.py" - ] - }, - { - "commit": "646c1db5d707969307f0b7065a98d4cdd22a3949", - "subject": "Add mrope op fusion (#3509)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/ops/test_rotary_embedding.py", - "vllm_ascend/ops/rotary_embedding.py", - "vllm_ascend/utils.py" - ] - }, - { - "commit": "4750d45d86632b7085ab9f0b57070cd17aa9d108", - "subject": "[BugFix]Support redundant experts in EPLB (#3473)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/eplb/core/test_eplb_utils.py", - "tests/ut/ops/test_fused_ops.py", - "tests/ut/torchair/ops/test_torchair_fused_moe.py", - "vllm_ascend/eplb/core/eplb_utils.py", - "vllm_ascend/ops/moe/experts_selector.py", - "vllm_ascend/ops/moe/token_dispatcher.py", - "vllm_ascend/quantization/w4a8_dynamic.py", - "vllm_ascend/quantization/w8a8.py", - "vllm_ascend/quantization/w8a8_dynamic.py", - "vllm_ascend/torchair/ops/torchair_fused_moe.py", - "vllm_ascend/torchair/quantization/torchair_w4a8_dynamic.py", - "vllm_ascend/torchair/quantization/torchair_w8a8_dynamic.py" - ] - }, - { - "commit": "21769e8f44fb017a492ecbd95df3402ba889078a", - "subject": "[BUGFIX] Mtp torchair pd fix (#3506)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/torchair/torchair_model_runner.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "3a53bbc508d80fc8148ab66662716e4370d83e68", - "subject": "[Feat]Qwen3 Moe supports npu_add_rms_norm_quant op by default, update op with bias, resolve conflict with weight prefetch (#3465)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/ops/test_layernorm.py", - "vllm_ascend/ascend_forward_context.py", - "vllm_ascend/attention/attention_v1.py", - "vllm_ascend/attention/utils.py", - "vllm_ascend/compilation/acl_graph.py", - "vllm_ascend/ops/layernorm.py", - "vllm_ascend/ops/moe/moe_mlp.py", - "vllm_ascend/ops/weight_prefetch.py", - "vllm_ascend/utils.py" - ] - }, - { - "commit": "cec1fab5099dde2cf9247a1993c3e97761d9f7ab", - "subject": "Revert \"[MoE] [Refactor] Remove manual memory cleanup (#3365)\" (#3483)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/e2e/singlecard/ops/test_fused_moe.py", - "tests/ut/ops/test_fused_moe_prepare_and_finalize.py", - "tests/ut/ops/test_moe_comm_method.py", - "tests/ut/ops/test_token_dispatcher.py", - "vllm_ascend/ops/common_fused_moe.py", - "vllm_ascend/ops/moe/fused_moe_prepare_and_finalize.py", - "vllm_ascend/ops/moe/moe_comm_method.py", - "vllm_ascend/ops/moe/token_dispatcher.py" - ] - }, - { - "commit": "f69a83b7ba7af657dbd2616bc11eb36a41aaf1b8", - "subject": "[Feat] Flash comm allgher ep (#3334)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/e2e/multicard/test_offline_inference_distributed.py", - "tests/ut/attention/test_mla_v1.py", - "tests/ut/models/test_deepseek_v2.py", - "vllm_ascend/ascend_forward_context.py", - "vllm_ascend/attention/mla_v1.py", - "vllm_ascend/envs.py", - "vllm_ascend/models/deepseek_v2.py", - "vllm_ascend/models/layers/mla.py", - "vllm_ascend/ops/common_fused_moe.py", - "vllm_ascend/ops/linear_op.py", - "vllm_ascend/ops/moe/fused_moe_prepare_and_finalize.py", - "vllm_ascend/ops/register_custom_ops.py", - "vllm_ascend/platform.py", - "vllm_ascend/utils.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "5a3082cd1533a4aa97611e7e8edf7f22841f4f88", - "subject": "[EPLB]Record expert map without dynamic eplb. (#3409)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "docs/source/user_guide/feature_guide/eplb_swift_balancer.md", - "vllm_ascend/eplb/eplb_updator.py", - "vllm_ascend/ops/common_fused_moe.py", - "vllm_ascend/quantization/w4a8_dynamic.py", - "vllm_ascend/quantization/w8a8_dynamic.py", - "vllm_ascend/torchair/ops/torchair_fused_moe.py", - "vllm_ascend/torchair/quantization/torchair_w4a8_dynamic.py", - "vllm_ascend/torchair/quantization/torchair_w8a8_dynamic.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "4f937f561d573ae97f953169865cfbf70d0c220b", - "subject": "[MoE] [Refactor] Remove manual memory cleanup (#3365)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/e2e/singlecard/ops/test_fused_moe.py", - "tests/ut/ops/test_fused_moe_prepare_and_finalize.py", - "tests/ut/ops/test_moe_comm_method.py", - "tests/ut/ops/test_token_dispatcher.py", - "vllm_ascend/ops/common_fused_moe.py", - "vllm_ascend/ops/moe/fused_moe_prepare_and_finalize.py", - "vllm_ascend/ops/moe/moe_comm_method.py", - "vllm_ascend/ops/moe/token_dispatcher.py" - ] - }, - { - "commit": "16cb3cc45de5c579d003e6516b3a6460863af235", - "subject": "adapt the mla_v1 with the `mla_preprocess` kernel (#3397)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/attention/mla_v1.py", - "vllm_ascend/attention/utils.py" - ] - }, - { - "commit": "15b2e5c9959a22e326f132b6964e10f9d3c7e26a", - "subject": "Remove unused row_idx in token_dispatcher (#3442)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/e2e/singlecard/ops/test_fused_moe.py", - "tests/ut/ops/test_fused_ops.py", - "tests/ut/ops/test_moe_comm_method.py", - "tests/ut/ops/test_token_dispatcher.py", - "tests/ut/quantization/test_w8a8.py", - "vllm_ascend/ops/common_fused_moe.py", - "vllm_ascend/ops/moe/experts_selector.py", - "vllm_ascend/ops/moe/moe_comm_method.py", - "vllm_ascend/ops/moe/token_dispatcher.py", - "vllm_ascend/quantization/w4a8_dynamic.py", - "vllm_ascend/quantization/w8a8_dynamic.py" - ] - }, - { - "commit": "02c26dcfc7632e90b280a1d20481826b442b9c69", - "subject": "[Feat] Supports Aclgraph for bge-m3 (#3171)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/workflows/_e2e_test.yaml", - "tests/e2e/singlecard/test_bge_model.py", - "tests/e2e/singlecard/test_embedding_aclgraph.py", - "vllm_ascend/attention/attention_mask.py", - "vllm_ascend/attention/attention_v1.py", - "vllm_ascend/patch/__init__.py", - "vllm_ascend/patch/worker/patch_common/__init__.py", - "vllm_ascend/patch/worker/patch_common/patch_attention_selector.py", - "vllm_ascend/patch/worker/patch_common/patch_roberta.py", - "vllm_ascend/platform.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "78777237a9ae95af88ce6068b000635f96aafe25", - "subject": "[2/N][Feat] Attention and MoE weight prefetch in Qwen3MoE models (#3203)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/user_guide/configuration/additional_config.md", - "tests/e2e/singlecard/ops/test_fused_moe.py", - "tests/ut/ops/test_fused_ops.py", - "tests/ut/quantization/test_w8a8.py", - "vllm_ascend/ascend_config.py", - "vllm_ascend/ascend_forward_context.py", - "vllm_ascend/ops/moe/experts_selector.py", - "vllm_ascend/ops/moe/moe_mlp.py", - "vllm_ascend/ops/weight_prefetch.py" - ] - }, - { - "commit": "07e39620ea5685215da585d6ff7e992f9d58ab89", - "subject": "[Feat] Unquantized Linear to nz and control all nz-cast (#3356)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/attention/test_mla_v1.py", - "tests/ut/models/test_deepseek_v2.py", - "tests/ut/ops/test_linear.py", - "tests/ut/quantization/test_quant_config.py", - "tests/ut/quantization/test_w8a8.py", - "tests/ut/test_utils.py", - "tests/ut/torchair/test_utils.py", - "vllm_ascend/attention/mla_v1.py", - "vllm_ascend/envs.py", - "vllm_ascend/models/deepseek_v2.py", - "vllm_ascend/ops/common_fused_moe.py", - "vllm_ascend/ops/linear.py", - "vllm_ascend/ops/linear_op.py", - "vllm_ascend/quantization/quant_config.py", - "vllm_ascend/quantization/w4a8_dynamic.py", - "vllm_ascend/quantization/w8a8.py", - "vllm_ascend/quantization/w8a8_dynamic.py", - "vllm_ascend/torchair/quantization/torchair_w8a8_dynamic.py", - "vllm_ascend/torchair/torchair_sfa.py", - "vllm_ascend/torchair/utils.py", - "vllm_ascend/utils.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "6972df59514c2f04eb8e5dd6b7b2c25e0276a230", - "subject": "[Feature] optimize sp & qwen3 next support sp. (#3225)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/ops/test_layernorm.py", - "tests/ut/ops/test_vocab_parallel_embedding.py", - "tests/ut/torchair/models/test_torchair_deepseek_v2.py", - "vllm_ascend/ops/layernorm.py", - "vllm_ascend/ops/linear.py", - "vllm_ascend/ops/linear_op.py", - "vllm_ascend/ops/register_custom_ops.py", - "vllm_ascend/ops/vocab_parallel_embedding.py", - "vllm_ascend/torchair/ops/torchair_vocab_parallel_embedding.py", - "vllm_ascend/torchair/utils.py" - ] - }, - { - "commit": "31682961af1a0c625e8d19fabad089aecc99e88d", - "subject": "[Feat] enable hierarchical communication for mc2 ops on A2 (#3015)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/ops/test_token_dispatcher.py", - "tests/ut/torchair/quantization/test_torchair_w8a8_dynamic.py", - "vllm_ascend/ops/moe/token_dispatcher.py", - "vllm_ascend/torchair/ops/torchair_fused_moe.py", - "vllm_ascend/torchair/quantization/torchair_w8a8_dynamic.py", - "vllm_ascend/utils.py" - ] - }, - { - "commit": "847d12a389217e4cbcc5fff70abd72d9b15ad5c4", - "subject": "[BugFix]Fix moe load problems in torchair when using dynamic eplb (#3381)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/torchair/ops/torchair_fused_moe.py", - "vllm_ascend/torchair/quantization/torchair_w8a8_dynamic.py" - ] - }, - { - "commit": "82b6c846ca7ad905d4a244f353c896f46910b0ca", - "subject": "[BugFix]Fix eplb problems when using dynamic eplb. (#3364)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/eplb/core/test_eplb_device_transfer_loader.py", - "vllm_ascend/eplb/adaptor/vllm_adaptor.py", - "vllm_ascend/eplb/core/eplb_device_transfer_loader.py", - "vllm_ascend/ops/common_fused_moe.py", - "vllm_ascend/ops/moe/moe_comm_method.py", - "vllm_ascend/ops/moe/moe_mlp.py", - "vllm_ascend/quantization/w8a8_dynamic.py", - "vllm_ascend/torchair/ops/torchair_fused_moe.py" - ] - }, - { - "commit": "ee0a95e47f98c98e13f85a37c0ca9515d71f1537", - "subject": "bugfix for mtp when running torchair in a2 (#3354)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/torchair/ops/torchair_fused_moe.py" - ] - }, - { - "commit": "1756efa5fd25cc78ff5086c6ab1ad07719a781fa", - "subject": "[Feat][Graph]Support FULL_DECEDE_ONLY mode for MLA models (#3125)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/e2e/singlecard/test_aclgraph.py", - "tests/ut/attention/test_mla_v1.py", - "vllm_ascend/attention/attention_v1.py", - "vllm_ascend/attention/mla_v1.py", - "vllm_ascend/attention/utils.py", - "vllm_ascend/compilation/acl_graph.py", - "vllm_ascend/platform.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "ba19dd3183d31be8e840d9f191a459589119cbda", - "subject": "Revert PTA upgrade PR (#3352)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/workflows/_e2e_test.yaml", - "README.md", - "README.zh.md", - "docs/source/installation.md", - "pyproject.toml", - "requirements.txt", - "tests/e2e/multicard/test_full_graph_mode.py", - "tests/ut/attention/test_attention_v1.py", - "tests/ut/ops/test_layernorm.py", - "tests/ut/torchair/quantization/test_torchair_w8a8_dynamic.py", - "vllm_ascend/ascend_forward_context.py", - "vllm_ascend/attention/attention_v1.py", - "vllm_ascend/compilation/acl_graph.py", - "vllm_ascend/ops/layernorm.py", - "vllm_ascend/utils.py" - ] - }, - { - "commit": "6ae75933daad0b309cebfcd833e136e8d767e7d1", - "subject": "[Feat] Load balance of tokens across experts in dummy_run (#3184)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/ops/common_fused_moe.py", - "vllm_ascend/quantization/w8a8_dynamic.py" - ] - }, - { - "commit": "1c2c72af8d5f022083e909c7146b231869f1a71a", - "subject": "[bugfix]change log2phy map to npu (#3339)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/torchair/ops/test_torchair_fused_moe.py", - "vllm_ascend/ops/common_fused_moe.py", - "vllm_ascend/torchair/ops/torchair_fused_moe.py" - ] - }, - { - "commit": "23db56a34063c5285f7d34e80e45c1888b877bd3", - "subject": "[Feat]Qwen3 Moe supports npu_add_rms_norm_quant op by default, update op with norm bias (#3205)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/ops/test_layernorm.py", - "vllm_ascend/ascend_forward_context.py", - "vllm_ascend/ops/layernorm.py", - "vllm_ascend/utils.py" - ] - }, - { - "commit": "94dd8328159c22f950089b24eea701c120cbf5a5", - "subject": "[MoE] [Refactor] Combine common_fused_moe and fused_moe (#3176)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/models/conftest.py", - "tests/ut/models/test_qwen3_moe.py", - "tests/ut/ops/test_fused_moe_prepare_and_finalize.py", - "tests/ut/ops/test_fused_ops.py", - "tests/ut/ops/test_moe_comm_method.py", - "vllm_ascend/models/__init__.py", - "vllm_ascend/models/deepseek_v2.py", - "vllm_ascend/models/qwen3_moe.py", - "vllm_ascend/ops/__init__.py", - "vllm_ascend/ops/common_fused_moe.py", - "vllm_ascend/ops/fused_moe.py", - "vllm_ascend/ops/moe/fused_moe_prepare_and_finalize.py", - "vllm_ascend/ops/moe/moe_comm_method.py", - "vllm_ascend/ops/moe/token_dispatcher.py", - "vllm_ascend/quantization/quant_config.py", - "vllm_ascend/torchair/models/qwen3_moe.py", - "vllm_ascend/torchair/ops/torchair_fused_moe.py" - ] - }, - { - "commit": "1c5b302f0d2920485b519b9a08531fe28dd590ed", - "subject": "[Misc] Clean up useless patch (#3320)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/vllm_ascend_test_full_vllm_0.11.0.yaml", - "tests/e2e/conftest.py", - "tests/e2e/model_utils.py", - "tests/e2e/singlecard/test_guided_decoding.py", - "tests/ut/ops/test_fused_moe_prepare_and_finalize.py", - "tests/ut/ops/test_fused_ops.py", - "tests/ut/torchair/ops/test_torchair_fused_moe.py", - "vllm_ascend/patch/platform/patch_common/__init__.py", - "vllm_ascend/patch/platform/patch_common/patch_config.py", - "vllm_ascend/patch/platform/patch_common/patch_transformers_utils.py" - ] - }, - { - "commit": "f12f76d7baeaaefd239f2035313732f47144d326", - "subject": "Drop 0.10.2 (#3284)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/models/qwen2_5_vl.py", - "vllm_ascend/models/qwen2_5_vl_without_padding.py", - "vllm_ascend/models/qwen2_vl.py", - "vllm_ascend/models/qwen3_moe.py", - "vllm_ascend/ops/fused_moe.py", - "vllm_ascend/ops/moe/fused_moe_prepare_and_finalize.py", - "vllm_ascend/patch/worker/patch_common/patch_attention_layer.py", - "vllm_ascend/patch/worker/patch_common/patch_attention_selector.py", - "vllm_ascend/patch/worker/patch_common/patch_weight_loader.py", - "vllm_ascend/platform.py", - "vllm_ascend/sample/sampler.py", - "vllm_ascend/spec_decode/eagle_proposer.py", - "vllm_ascend/spec_decode/mtp_proposer.py", - "vllm_ascend/torchair/models/qwen3_moe.py", - "vllm_ascend/torchair/ops/torchair_fused_moe.py", - "vllm_ascend/worker/model_runner_v1.py", - "vllm_ascend/worker/npu_input_batch.py" - ] - }, - { - "commit": "81bd6e4c99c15c874ab6bea2dc88d8b39bed8056", - "subject": "Add DeepSeek V3.2 support (#3270)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/workflows/vllm_ascend_test.yaml", - "vllm_ascend/__init__.py", - "vllm_ascend/ascend_config.py", - "vllm_ascend/attention/attention_mask.py", - "vllm_ascend/attention/sfa_v1.py", - "vllm_ascend/distributed/llmdatadist_c_mgr_connector.py", - "vllm_ascend/distributed/mooncake_connector.py", - "vllm_ascend/envs.py", - "vllm_ascend/models/__init__.py", - "vllm_ascend/models/deepseek_v2.py", - "vllm_ascend/models/layers/sfa.py", - "vllm_ascend/patch/platform/patch_common/__init__.py", - "vllm_ascend/patch/platform/patch_common/patch_config.py", - "vllm_ascend/patch/platform/patch_common/patch_mamba_config.py", - "vllm_ascend/patch/platform/patch_common/patch_transformers_utils.py", - "vllm_ascend/patch/worker/patch_common/__init__.py", - "vllm_ascend/patch/worker/patch_common/patch_attention_layer.py", - "vllm_ascend/patch/worker/patch_common/patch_attention_selector.py", - "vllm_ascend/patch/worker/patch_common/patch_attentionspec.py", - "vllm_ascend/platform.py", - "vllm_ascend/spec_decode/mtp_proposer.py", - "vllm_ascend/torchair/models/torchair_deepseek_v2.py", - "vllm_ascend/torchair/torchair_model_runner.py", - "vllm_ascend/torchair/torchair_sfa.py", - "vllm_ascend/torchair/utils.py", - "vllm_ascend/worker/model_runner_v1.py", - "vllm_ascend/worker/worker_v1.py" - ] - }, - { - "commit": "83092d9b8b485f4cd3fea22f738994054b0a33e5", - "subject": "[BugFix] Fix Qwen3-Next because of vllm #24982 (#3221)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/models/qwen3_next.py" - ] - }, - { - "commit": "050d202bb97db481db2db3c142f5eb08608e3c7a", - "subject": "[Quickfix] Fix dp+ep+tp error when sp chunked the hidden_states (#3246)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/ops/fused_moe.py", - "vllm_ascend/platform.py" - ] - }, - { - "commit": "14d4ed5f0cfc5976a7cb42984f570d8e66655430", - "subject": "[BugFix] Fix aclgraph accu problem in A2. (#3163)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/ops/common_fused_moe.py", - "vllm_ascend/ops/register_custom_ops.py" - ] - }, - { - "commit": "dd56e9306b0780cc0fa7b94b1c87cde6c12e0d3d", - "subject": "[3/N][Refactor][Qwen3-Next] Refacotr model structure and fix bug by vllm #25400 (#3142)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/models/__init__.py", - "vllm_ascend/models/qwen3_next.py" - ] - }, - { - "commit": "4ff422c7308af25ad8c2352392fb1b4b78280e65", - "subject": "[CI][Bugfix] Quickfix for DPMetaData (#3234)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/ops/test_fused_moe_prepare_and_finalize.py", - "tests/ut/ops/test_fused_ops.py", - "tests/ut/torchair/ops/test_torchair_fused_moe.py", - "vllm_ascend/models/qwen3_moe.py", - "vllm_ascend/ops/moe/fused_moe_prepare_and_finalize.py", - "vllm_ascend/torchair/models/qwen3_moe.py", - "vllm_ascend/torchair/ops/torchair_fused_moe.py" - ] - }, - { - "commit": "07f4710216da1d61c521fb3bdf4ba90ea1794474", - "subject": "[BugFix] Fix dummy_run memory explosion in eager mode (#3132)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/ops/moe/fused_moe_prepare_and_finalize.py" - ] - }, - { - "commit": "2a9d02e08039749cf811c5fb190d4c8a950d792d", - "subject": "[Bugfix] eagle and eagle3 spec decode failures and enable e2e test (#2979)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/e2e/singlecard/spec_decode_v1/test_v1_spec_decode.py", - "vllm_ascend/spec_decode/eagle_proposer.py" - ] - }, - { - "commit": "ac1c2cd9ac6f7ed6ba7505065959393717e9b903", - "subject": "[CI] Upgrade vllm version - 0925 (#3167)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/format_pr_body.yaml", - ".github/workflows/vllm_ascend_test.yaml", - ".github/workflows/vllm_ascend_test_full.yaml", - "tests/ut/attention/test_attention_v1.py", - "tests/ut/ops/test_fused_ops.py", - "tests/ut/torchair/ops/test_torchair_fused_moe.py", - "vllm_ascend/attention/attention_v1.py" - ] - }, - { - "commit": "6aa425379867b7c846938416f3dd8b27c66c7b28", - "subject": "[Refactor] [SP]The sequence parallelism characteristics in the MoE and Dense models are integrated into a single solution. (#3085)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/ascend_forward_context.py", - "vllm_ascend/attention/attention_v1.py", - "vllm_ascend/models/qwen3_moe.py", - "vllm_ascend/ops/common_fused_moe.py", - "vllm_ascend/ops/fused_moe.py", - "vllm_ascend/ops/linear_op.py", - "vllm_ascend/ops/moe/fused_moe_prepare_and_finalize.py", - "vllm_ascend/ops/register_custom_ops.py", - "vllm_ascend/platform.py", - "vllm_ascend/torchair/models/qwen3_moe.py", - "vllm_ascend/torchair/ops/sequence_parallel.py", - "vllm_ascend/torchair/ops/torchair_fused_moe.py", - "vllm_ascend/utils.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "e7618d94147f4749c1ea85d1264c7c1d00437008", - "subject": "[2/N][Refactor][Qwen3-Next] remove redundant methods and patch methods in Qwen3NextGatedDeltaNet (#3082)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/models/qwen3_next.py", - "vllm_ascend/ops/casual_conv1d.py", - "vllm_ascend/ops/fla.py", - "vllm_ascend/ops/sigmoid_gating.py", - "vllm_ascend/patch/worker/patch_common/__init__.py", - "vllm_ascend/patch/worker/patch_common/patch_triton.py" - ] - }, - { - "commit": "d01fd1d1c3526ef038f05edc2464a1e77dfb200d", - "subject": "[misc][torchair] fix bugs around `deepseek mtp`, `enable_shared_expert_dp` and `use_cached_kv_cache_bytes` (#3074)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/torchair/ops/test_torchair_fused_moe.py", - "vllm_ascend/spec_decode/mtp_proposer.py", - "vllm_ascend/torchair/models/torchair_deepseek_v2.py", - "vllm_ascend/torchair/ops/torchair_fused_moe.py", - "vllm_ascend/torchair/quantization/torchair_w8a8_dynamic.py", - "vllm_ascend/torchair/torchair_model_runner.py", - "vllm_ascend/torchair/torchair_worker.py" - ] - }, - { - "commit": "0f3939e5a9b3ce01602c5d7fc269dce08fb720b0", - "subject": "[Feature]cpu offload connector (#1659)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/attention/test_mla_v1.py", - "tests/ut/torchair/models/test_torchair_deepseek_v2.py", - "vllm_ascend/attention/attention_v1.py", - "vllm_ascend/attention/mla_v1.py", - "vllm_ascend/attention/utils.py", - "vllm_ascend/distributed/cpu_offload_connector.py", - "vllm_ascend/distributed/cpu_offload_manager/__init__.py", - "vllm_ascend/distributed/cpu_offload_manager/cpu_kv_cache_manager.py", - "vllm_ascend/distributed/cpu_offload_manager/metadata.py", - "vllm_ascend/models/layers/mla.py" - ] - }, - { - "commit": "37a0715edade3efae975c02ec8edf6f7a5d07530", - "subject": "[Refactor] Adjustments to moe_comm_method selection process (#3001)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/ops/test_common_fused_moe.py", - "tests/ut/ops/test_fused_ops.py", - "tests/ut/ops/test_moe_comm_method.py", - "tests/ut/quantization/test_w4a8_dynamic.py", - "tests/ut/worker/test_model_runner_v1.py", - "vllm_ascend/ascend_forward_context.py", - "vllm_ascend/ops/common_fused_moe.py", - "vllm_ascend/ops/fused_moe.py", - "vllm_ascend/ops/moe/moe_comm_method.py", - "vllm_ascend/ops/moe/moe_mlp.py", - "vllm_ascend/ops/moe/token_dispatcher.py", - "vllm_ascend/spec_decode/eagle_proposer.py", - "vllm_ascend/spec_decode/mtp_proposer.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "f1f2c8f5e5d1659a97edc5cde7b7d80b8426416a", - "subject": "[Perf] Add new npu_fused_infer_attention_score op to improve perfomance in splitfuse cases and resolve long-seq mask problems (#2962)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/attention/attention_mask.py", - "vllm_ascend/attention/attention_v1.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "14b39d3c700c543e0f47bd5ccf1fb72ccae98c71", - "subject": "[1/N][Refactor][Qwen3-Next] remove redundant Qwen3NextSparseMoeBlock and Qwen3NextAttention (#3019)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/models/qwen3_next.py" - ] - }, - { - "commit": "12bcbd02bb52e1a47f9c93b49af1a24eb32d175a", - "subject": "[CI] Upgrade vLLM to 20250919 (6d8246aa) and fix some broken issue (#2907)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/workflows/format_pr_body.yaml", - ".github/workflows/vllm_ascend_test.yaml", - ".github/workflows/vllm_ascend_test_full.yaml", - "tests/e2e/singlecard/test_guided_decoding.py", - "tests/ut/ops/test_fused_ops.py", - "tests/ut/torchair/ops/test_torchair_fused_moe.py", - "vllm_ascend/ops/fused_moe.py", - "vllm_ascend/patch/worker/patch_common/__init__.py", - "vllm_ascend/platform.py", - "vllm_ascend/quantization/quant_config.py", - "vllm_ascend/spec_decode/mtp_proposer.py", - "vllm_ascend/torchair/ops/torchair_fused_moe.py", - "vllm_ascend/worker/model_runner_v1.py", - "vllm_ascend/worker/npu_input_batch.py" - ] - }, - { - "commit": "8326f15ecfb4e0139c8d11ee928742d91fc89ee9", - "subject": "[CustomOp] Register AscendSharedFusedMoE custom op (#2980)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/ops/common_fused_moe.py", - "vllm_ascend/patch/worker/patch_common/__init__.py", - "vllm_ascend/patch/worker/patch_common/patch_shared_fused_moe.py", - "vllm_ascend/utils.py" - ] - }, - { - "commit": "0a526768f55cc60b5b870adce92affdfcfea8523", - "subject": "[Feature] Support moe multi-stream for aclgraph. (#2946)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "docs/source/locale/zh_CN/LC_MESSAGES/user_guide/configuration/additional_config.po", - "docs/source/user_guide/configuration/additional_config.md", - "examples/external_online_dp/run_dp_template.sh", - "examples/run_dp_server.sh", - "tests/e2e/multicard/test_offline_inference_distributed.py", - "tests/e2e/singlecard/test_multistream_overlap_shared_expert.py", - "tests/ut/ops/test_fused_ops.py", - "tests/ut/test_ascend_config.py", - "tests/ut/torchair/ops/test_torchair_fused_moe.py", - "vllm_ascend/ascend_config.py", - "vllm_ascend/ops/common_fused_moe.py", - "vllm_ascend/torchair/models/torchair_deepseek_v2.py", - "vllm_ascend/torchair/ops/torchair_fused_moe.py", - "vllm_ascend/utils.py" - ] - }, - { - "commit": "79a910ef4730d3f1be14496a1681eee2566f64a0", - "subject": "[bugfix][torchair] fix multistream_moe problems in torchair graph mode (#2681)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/torchair/ops/torchair_fused_moe.py" - ] - }, - { - "commit": "af2a886814dbc1ce512ed9997c3e0b2bb3dcf01a", - "subject": "refactor linear (#2867)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/models/test_qwen2_5_vl.py", - "tests/ut/ops/test_linear.py", - "vllm_ascend/ops/linear.py", - "vllm_ascend/ops/linear_op.py" - ] - }, - { - "commit": "a7f8ed38ed0681a0c3e29d848b04db4c7e972e06", - "subject": "[Bugfix]:replace npu_incre_flash_attention with npu_fused_infer_atten\u2026 (#2901)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/torchair/test_torchair_attention.py", - "vllm_ascend/torchair/torchair_attention.py" - ] - }, - { - "commit": "6681dde9028b2ae8ab0e56158a117c5db7f0bc7a", - "subject": "[Feat][Graph] Support MTP for ACL Graph (#2932)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/e2e/singlecard/spec_decode_v1/test_v1_mtp_correctness.py", - "tests/ut/attention/test_mla_v1.py", - "tests/ut/torchair/test_torchair_mla.py", - "vllm_ascend/attention/mla_v1.py", - "vllm_ascend/spec_decode/mtp_proposer.py", - "vllm_ascend/torchair/torchair_model_runner.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "76844eec78a23f482a4e0dfe9684898a6ef35fb2", - "subject": "Dynamic Expert Load Balance with Zero-like-overhead (#2956)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "docs/source/user_guide/feature_guide/eplb_swift_balancer.md", - "docs/source/user_guide/feature_guide/images/eplb_img.png", - "docs/source/user_guide/feature_guide/index.md", - "tests/ut/distributed/test_determin_expert_map_all.py", - "vllm_ascend/ascend_config.py", - "vllm_ascend/eplb/__init__.py", - "vllm_ascend/eplb/adaptor/__init__.py", - "vllm_ascend/eplb/adaptor/abstract_adaptor.py", - "vllm_ascend/eplb/adaptor/vllm_adaptor.py", - "vllm_ascend/eplb/core/__init__.py", - "vllm_ascend/eplb/core/eplb_device_transfer_loader.py", - "vllm_ascend/eplb/core/eplb_utils.py", - "vllm_ascend/eplb/core/eplb_worker.py", - "vllm_ascend/eplb/core/policy/__init__.py", - "vllm_ascend/eplb/core/policy/policy_abstract.py", - "vllm_ascend/eplb/core/policy/policy_dynamic_ep.py", - "vllm_ascend/eplb/core/policy/policy_dynamic_ep_v2.py", - "vllm_ascend/eplb/core/policy/policy_factory.py", - "vllm_ascend/eplb/core/policy/policy_random.py", - "vllm_ascend/eplb/eplb_updator.py", - "vllm_ascend/eplb/utils.py", - "vllm_ascend/ops/common_fused_moe.py", - "vllm_ascend/ops/fused_moe.py", - "vllm_ascend/ops/moe/moe_comm_method.py", - "vllm_ascend/quantization/w4a8_dynamic.py", - "vllm_ascend/quantization/w8a8_dynamic.py", - "vllm_ascend/torchair/models/torchair_deepseek_v2.py", - "vllm_ascend/torchair/ops/torchair_fused_moe.py", - "vllm_ascend/worker/model_runner_v1.py", - "vllm_ascend/worker/worker_v1.py" - ] - }, - { - "commit": "ae758dda05b57adaf70af44122e6bdc54fbb88ff", - "subject": "[Bugfix] Fix mtp torchair in pd Disaggregation scenario (#2951)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/attention/mla_v1.py", - "vllm_ascend/torchair/torchair_mla.py", - "vllm_ascend/torchair/torchair_model_runner.py" - ] - }, - { - "commit": "6b7117dbb74e6c46da110d74014af811be6323ec", - "subject": "[main] addrmsnorm + quant fusion optim in Dense Models (#2772)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/ops/test_layernorm.py", - "vllm_ascend/ascend_forward_context.py", - "vllm_ascend/models/__init__.py", - "vllm_ascend/models/qwen3.py", - "vllm_ascend/ops/layernorm.py" - ] - }, - { - "commit": "1c5900327b67015e5707d3879ccf5fa5ab622832", - "subject": "[refactor] refactor deepseek-related files (#2849)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/e2e/multicard/test_expert_parallel.py", - "tests/e2e/multicard/test_torchair_graph_mode.py", - "tests/ut/models/conftest.py", - "tests/ut/models/test_deepseek_mtp.py", - "tests/ut/models/test_deepseek_v2.py", - "tests/ut/ops/test_common_fused_moe.py", - "tests/ut/test_platform.py", - "vllm_ascend/models/__init__.py", - "vllm_ascend/models/deepseek_dbo.py", - "vllm_ascend/models/deepseek_mtp.py", - "vllm_ascend/models/deepseek_v2.py", - "vllm_ascend/ops/common_fused_moe.py", - "vllm_ascend/patch/platform/patch_common/__init__.py", - "vllm_ascend/patch/worker/patch_common/__init__.py", - "vllm_ascend/patch/worker/patch_common/patch_shared_fused_moe.py", - "vllm_ascend/platform.py", - "vllm_ascend/torchair/torchair_model_runner.py", - "vllm_ascend/torchair/torchair_worker.py" - ] - }, - { - "commit": "18ca7861f6e4f27cea58cc1d70a8c3081422081c", - "subject": "[Main] [Refactor] Enable MoECommMethod in Eager Mode (#2791)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/models/test_deepseek_v2.py", - "tests/ut/ops/test_ascend_forwad_context.py", - "tests/ut/ops/test_fused_moe_prepare_and_finalize.py", - "tests/ut/ops/test_fused_ops.py", - "tests/ut/ops/test_token_dispatcher.py", - "tests/ut/worker/test_model_runner_v1.py", - "vllm_ascend/ascend_forward_context.py", - "vllm_ascend/ops/common_fused_moe.py", - "vllm_ascend/ops/fused_moe.py", - "vllm_ascend/ops/moe/fused_moe_prepare_and_finalize.py", - "vllm_ascend/ops/moe/moe_comm_method.py", - "vllm_ascend/ops/moe/moe_mlp.py", - "vllm_ascend/ops/moe/token_dispatcher.py", - "vllm_ascend/quantization/w4a8_dynamic.py", - "vllm_ascend/quantization/w8a8_dynamic.py", - "vllm_ascend/spec_decode/eagle_proposer.py", - "vllm_ascend/spec_decode/mtp_proposer.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "c556038ef0b8580cb9079823bd9f06263ee7d731", - "subject": "[New model] Qwen3-next support (#2917)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/workflows/vllm_ascend_test_full.yaml", - "tests/e2e/multicard/test_prefix_caching.py", - "tests/ut/attention/test_attention_v1.py", - "tests/ut/attention/test_mla_v1.py", - "tests/ut/torchair/test_torchair_mla.py", - "tests/ut/worker/test_input_batch.py", - "vllm_ascend/attention/attention_v1.py", - "vllm_ascend/attention/mla_v1.py", - "vllm_ascend/attention/utils.py", - "vllm_ascend/models/__init__.py", - "vllm_ascend/models/layers/mla.py", - "vllm_ascend/models/qwen3_next.py", - "vllm_ascend/ops/casual_conv1d.py", - "vllm_ascend/ops/fla.py", - "vllm_ascend/ops/sigmoid_gating.py", - "vllm_ascend/patch/platform/patch_common/__init__.py", - "vllm_ascend/patch/platform/patch_common/patch_mamba_config.py", - "vllm_ascend/platform.py", - "vllm_ascend/spec_decode/eagle_proposer.py", - "vllm_ascend/spec_decode/mtp_proposer.py", - "vllm_ascend/torchair/torchair_attention.py", - "vllm_ascend/torchair/torchair_mla.py", - "vllm_ascend/torchair/torchair_model_runner.py", - "vllm_ascend/worker/block_table.py", - "vllm_ascend/worker/model_runner_v1.py", - "vllm_ascend/worker/npu_input_batch.py" - ] - }, - { - "commit": "e57cca971c4db11d8ce9da6b008bd655ada8c77e", - "subject": "Fix the bugs about operator registration by PyTorch Dispatcher (#2786)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "benchmarks/ops/ben_vocabparallelembedding.py", - "csrc/torch_binding.cpp", - "csrc/torch_binding_meta.cpp", - "tests/e2e/singlecard/ops/test_bgmv_expand.py", - "tests/e2e/singlecard/ops/test_bgmv_shrink.py", - "tests/e2e/singlecard/ops/test_rotary_embedding.py", - "tests/e2e/singlecard/ops/test_vocabparallelembedding.py", - "tests/ut/ops/test_rotary_embedding.py", - "tests/ut/torchair/ops/test_torchair_rotary_embedding.py", - "vllm_ascend/compilation/acl_graph.py", - "vllm_ascend/lora/lora_ops.py", - "vllm_ascend/meta_registration.py", - "vllm_ascend/ops/__init__.py", - "vllm_ascend/ops/rotary_embedding.py", - "vllm_ascend/torchair/ops/torchair_rotary_embedding.py", - "vllm_ascend/utils.py" - ] - }, - { - "commit": "aeffe27b3089cf12b743806fd922c7d1fd455ac2", - "subject": "[Perf]set moe w2_weight default to be nz (#2842)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/quantization/w8a8_dynamic.py", - "vllm_ascend/torchair/quantization/torchair_w8a8_dynamic.py" - ] - }, - { - "commit": "c3c222150363900e5f0e87ae2abb4868dacf6a1c", - "subject": "[Feat]support dynamic quantization in allgather (#2841)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/e2e/multicard/test_qwen3_moe.py", - "tests/e2e/singlecard/ops/test_fused_moe.py", - "tests/ut/ops/test_token_dispatcher.py", - "vllm_ascend/ops/moe/token_dispatcher.py" - ] - }, - { - "commit": "923cdaeba389ea5b895acb25a7c755ccb830cee2", - "subject": "fix ascend fused moe spelling error (#2863)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/ops/common_fused_moe.py" - ] - }, - { - "commit": "c2fdd4b8bc9ce859343909caebf331e0cd047908", - "subject": "[CI/UT] Fix UTs on register customop and warm up model (#2862)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/test_utils.py", - "tests/ut/worker/test_worker_v1.py", - "vllm_ascend/utils.py" - ] - }, - { - "commit": "edf1f600ad30120a7d870a38b65520616f3131ae", - "subject": "[CI] Remove compatibility maintenance for vllm v0.10.1 and v0.10.1.1 (#2840)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/workflows/accuracy_test.yaml", - ".github/workflows/nightly_benchmarks.yaml", - ".github/workflows/vllm_ascend_dist.yaml", - ".github/workflows/vllm_ascend_test.yaml", - ".github/workflows/vllm_ascend_test_310p.yaml", - ".github/workflows/vllm_ascend_test_full.yaml", - "Dockerfile", - "Dockerfile.310p", - "Dockerfile.310p.openEuler", - "Dockerfile.a3", - "Dockerfile.a3.openEuler", - "Dockerfile.openEuler", - "docs/source/conf.py", - "tests/ut/core/test_scheduler.py", - "tests/ut/kv_connector/utils.py", - "vllm_ascend/core/scheduler.py", - "vllm_ascend/models/qwen3_moe.py", - "vllm_ascend/ops/common_fused_moe.py", - "vllm_ascend/sample/sampler.py", - "vllm_ascend/worker/model_runner_v1.py", - "vllm_ascend/worker/npu_input_batch.py", - "vllm_ascend/worker/worker_v1.py" - ] - }, - { - "commit": "93e28e6862669e3b5cf47cea9f782a65ec47e155", - "subject": "add weight transpose check. (#2756)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/ops/test_common_fused_moe.py", - "vllm_ascend/ops/common_fused_moe.py" - ] - }, - { - "commit": "e13c4ddb4290112e5869a26f97fbfd02075fbc67", - "subject": "[Fix] Fix SharedFusedMoE (#2817)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/ops/common_fused_moe.py", - "vllm_ascend/patch/platform/patch_common/__init__.py", - "vllm_ascend/patch/platform/patch_common/patch_shared_fused_moe.py" - ] - }, - { - "commit": "1bbb20ea13e2b9a47936abebdcfb6143fdce8079", - "subject": "[main] flashcomm_v1 optim in Qwen Dense Models (#2802)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/e2e/multicard/test_offline_inference_distributed.py", - "tests/ut/ops/test_layernorm.py", - "tests/ut/test_utils.py", - "vllm_ascend/ascend_forward_context.py", - "vllm_ascend/envs.py", - "vllm_ascend/ops/__init__.py", - "vllm_ascend/ops/layernorm.py", - "vllm_ascend/ops/linear.py", - "vllm_ascend/ops/register_custom_ops.py", - "vllm_ascend/utils.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "7d6d9449a83d661c229b93c17a998e67d01bafa7", - "subject": "[Misc] Move lora patch file into lora module (#2797)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/test_platform.py", - "vllm_ascend/lora/lora_ops.py", - "vllm_ascend/lora/punica_npu.py", - "vllm_ascend/lora/punica_wrapper/__init__.py", - "vllm_ascend/lora/utils.py", - "vllm_ascend/patch/__init__.py", - "vllm_ascend/patch/worker/patch_common/__init__.py", - "vllm_ascend/patch/worker/patch_common/patch_lora.py", - "vllm_ascend/patch/worker/patch_common/patch_lora_embedding.py", - "vllm_ascend/platform.py" - ] - }, - { - "commit": "85d989a3b93af7f566c2c52210c3576a665c5e66", - "subject": "[Misc] Remove pangu model file (#2798)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/models/__init__.py", - "vllm_ascend/models/pangu_moe.py" - ] - }, - { - "commit": "a041d4f32852289a28ec7c72f5f2d8c3ac142457", - "subject": "[main] [refactor] refactor common_fused_moe.py (#2706)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/e2e/singlecard/ops/test_fused_moe.py", - "tests/e2e/singlecard/ops/test_moe_comm.py", - "tests/ut/ops/test_fused_moe_prepare_and_finalize.py", - "tests/ut/ops/test_fused_ops.py", - "tests/ut/ops/test_moe_comm_method.py", - "tests/ut/ops/test_token_dispatcher.py", - "tests/ut/quantization/test_w8a8.py", - "vllm_ascend/ascend_forward_context.py", - "vllm_ascend/distributed/moe_comm_method.py", - "vllm_ascend/ops/common_fused_moe.py", - "vllm_ascend/ops/fused_moe.py", - "vllm_ascend/ops/moe/__init__.py", - "vllm_ascend/ops/moe/experts_selector.py", - "vllm_ascend/ops/moe/fused_moe_prepare_and_finalize.py", - "vllm_ascend/ops/moe/moe_comm_method.py", - "vllm_ascend/ops/moe/moe_mlp.py", - "vllm_ascend/ops/moe/token_dispatcher.py", - "vllm_ascend/ops/moe_dispatcher/__init__.py", - "vllm_ascend/quantization/w4a8_dynamic.py", - "vllm_ascend/quantization/w8a8.py", - "vllm_ascend/quantization/w8a8_dynamic.py" - ] - }, - { - "commit": "1a82b16355d2ec0ba01c23935092dc0af323b820", - "subject": "Remove unused code in fused_moe.py (#2805)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/ops/fused_moe.py" - ] - }, - { - "commit": "2693196ef8382761ae7858b9fcafe3866bf7287a", - "subject": "add gatherep select. (#2740)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/ops/test_ascend_forwad_context.py", - "vllm_ascend/ascend_forward_context.py", - "vllm_ascend/ops/moe_dispatcher/token_dispatcher.py" - ] - }, - { - "commit": "b2f77d3aa8f384488b56f5787928e84b1f347999", - "subject": "[fix] prefill unsupport sliding window attention (#2758)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/attention/test_attention_v1.py", - "vllm_ascend/attention/attention_v1.py" - ] - }, - { - "commit": "51a2aec1154e5e2ab9cbbba53d830d8d21ab3571", - "subject": "Delete redundant codes related to communication (#2717)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/ops/test_fused_ops.py", - "tests/ut/torchair/ops/test_torchair_fused_moe.py", - "vllm_ascend/distributed/communication_op.py", - "vllm_ascend/distributed/moe_comm_method.py", - "vllm_ascend/ops/fused_moe.py", - "vllm_ascend/torchair/ops/torchair_fused_moe.py" - ] - }, - { - "commit": "83eb40a51cb30c654c80a9203b3d9d7bd0351e4e", - "subject": "[Fix][MoE] Refine MoE communication strategy (#2734)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/worker/test_model_runner_v1.py", - "vllm_ascend/ops/common_fused_moe.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "f86596a66cc0aff2b05280303212758380f0ec9a", - "subject": "allgather use fusedop. (#2689)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/e2e/singlecard/ops/test_fused_moe.py", - "tests/ut/ops/test_token_dispatcher.py", - "vllm_ascend/ops/moe_dispatcher/token_dispatcher.py" - ] - }, - { - "commit": "aff5189c8781f1127d7dbca6c59a9ec44a427100", - "subject": "[main] Fuse GroupedMatmul, Swiglu and DynamicQuant in `W8A8_DYNAMIC` quantized MoE layers (#2275)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/ops/test_fused_ops.py", - "tests/ut/quantization/test_w8a8_dynamic.py", - "vllm_ascend/ops/fused_moe.py", - "vllm_ascend/ops/layers/moe_mlp.py", - "vllm_ascend/quantization/w8a8_dynamic.py" - ] - }, - { - "commit": "37f5a29cd4f84fa0beee236dadf070b41e4b5403", - "subject": "[1/N][Refactor][Quantization] remove redundant quantizer class (#2680)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/quantization/test_quant_config.py", - "tests/ut/quantization/test_quantizer.py", - "tests/ut/quantization/test_utils.py", - "tests/ut/torchair/ops/test_torchair_fused_moe.py", - "tests/ut/torchair/test_utils.py", - "vllm_ascend/quantization/quant_config.py", - "vllm_ascend/quantization/quantizer.py", - "vllm_ascend/quantization/utils.py", - "vllm_ascend/torchair/quantization/torchair_quantizer.py", - "vllm_ascend/torchair/utils.py" - ] - }, - { - "commit": "d4370ebc42f8a2cecbb7ad4b199ac3f840ca3b28", - "subject": "[Refactor] Refactor Spec Decode (#2668)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/spec_decode/__init__.py", - "vllm_ascend/spec_decode/eagle_proposer.py", - "vllm_ascend/spec_decode/interface.py", - "vllm_ascend/spec_decode/mtp_proposer.py", - "vllm_ascend/spec_decode/ngram_proposer.py", - "vllm_ascend/worker/eagle_proposer_v1.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "e7409e95ee73fb3bb7bf8b23f26c16620ed94543", - "subject": "[1/N][Draft][Refactor]torchair pangu_moe modeling refactor (#2437)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/e2e/multicard/test_torchair_graph_mode.py", - "tests/ut/torchair/test_utils.py", - "vllm_ascend/models/pangu_moe.py", - "vllm_ascend/ops/common_fused_moe.py", - "vllm_ascend/torchair/models/torchair_pangu_moe.py", - "vllm_ascend/torchair/utils.py" - ] - }, - { - "commit": "b84465c52564134d217a0119e34a68ef86b6d35d", - "subject": "[Perf]Enable npu_moe_gating_top_k_softmax on quantized scenarios (#2633)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/quantization/test_w8a8.py", - "vllm_ascend/ops/fused_moe.py", - "vllm_ascend/ops/layers/experts_selector.py" - ] - }, - { - "commit": "214b32a3460809148271d66774a457e3c750d79e", - "subject": "[V1][BUGFIX][0.10.1] FIX mtp on main branch (#2632)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/e2e/singlecard/spec_decode_v1/test_v1_mtp_torchair_correctness.py", - "tests/ut/torchair/ops/test_torchair_fused_moe.py", - "vllm_ascend/torchair/ops/torchair_fused_moe.py", - "vllm_ascend/worker/mtp_proposer_v1.py" - ] - }, - { - "commit": "fef18b60bc056a08b67ab95fb798052c51839cc6", - "subject": "Refactor e2e CI (#2276)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/vllm_ascend_test.yaml", - ".github/workflows/vllm_ascend_test_310p.yaml", - "tests/e2e/310p/test_offline_inference_310p.py", - "tests/e2e/310p/test_offline_inference_parallel_310p.py", - "tests/e2e/conftest.py", - "tests/e2e/model_utils.py", - "tests/e2e/multicard/test_data_parallel.py", - "tests/e2e/multicard/test_dynamic_npugraph_batchsize.py", - "tests/e2e/multicard/test_expert_parallel.py", - "tests/e2e/multicard/test_fused_moe_allgather_ep.py", - "tests/e2e/multicard/test_ilama_lora_tp2.py", - "tests/e2e/multicard/test_offline_inference_distributed.py", - "tests/e2e/multicard/test_pipeline_parallel.py", - "tests/e2e/multicard/test_prefix_caching.py", - "tests/e2e/multicard/test_pyhccl_distributed.py", - "tests/e2e/multicard/test_qwen3_moe.py", - "tests/e2e/multicard/test_torchair_graph_mode.py", - "tests/e2e/singlecard/compile/__init__.py", - "tests/e2e/singlecard/compile/test_simple.py", - "tests/e2e/singlecard/ops/test_bgmv_expand.py", - "tests/e2e/singlecard/ops/test_bgmv_shrink.py", - "tests/e2e/singlecard/ops/test_fused_moe.py", - "tests/e2e/singlecard/ops/test_moe_comm.py", - "tests/e2e/singlecard/ops/test_rotary_embedding.py", - "tests/e2e/singlecard/ops/test_vocabparallelembedding.py", - "tests/e2e/singlecard/sample/__init__.py", - "tests/e2e/singlecard/sample/test_rejection_sampler.py", - "tests/e2e/singlecard/spec_decode_v1/test_v1_spec_decode.py", - "tests/e2e/singlecard/test_aclgraph.py", - "tests/e2e/singlecard/test_ascend_scheduler.py", - "tests/e2e/singlecard/test_camem.py", - "tests/e2e/singlecard/test_chunked.py", - "tests/e2e/singlecard/test_embedding.py", - "tests/e2e/singlecard/test_guided_decoding.py", - "tests/e2e/singlecard/test_ilama_lora.py", - "tests/e2e/singlecard/test_offline_inference.py", - "tests/e2e/singlecard/test_profile_execute_duration.py", - "tests/e2e/singlecard/test_pyhccl.py", - "tests/e2e/singlecard/test_quantization.py", - "tests/e2e/singlecard/test_sampler.py", - "tests/e2e/singlecard/test_vlm.py" - ] - }, - { - "commit": "0df059f41a4c45ac98b0b6be9d5b50f989893905", - "subject": "[CI] Fix CI Break: upstream adds routed_scaling_factor in forward_oot interface (#2675)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/ops/common_fused_moe.py", - "vllm_ascend/ops/layers/experts_selector.py" - ] - }, - { - "commit": "3a5fc5ee01edb3e6c88774edfea2a85eed1ff990", - "subject": "[Refactor][MoE] remove redundant code after refactoring fused_moe (#2612)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/workflows/vllm_ascend_test.yaml", - "tests/e2e/multicard/test_offline_inference_distributed.py", - "tests/ut/ops/test_common_fused_moe.py", - "tests/ut/ops/test_fused_ops.py", - "tests/ut/ops/test_token_dispatcher.py", - "tests/ut/torchair/ops/test_torchair_fused_moe.py", - "vllm_ascend/ascend_forward_context.py", - "vllm_ascend/envs.py", - "vllm_ascend/ops/common_fused_moe.py", - "vllm_ascend/ops/fused_moe.py", - "vllm_ascend/ops/layers/moe_mlp.py", - "vllm_ascend/ops/moe_dispatcher/token_dispatcher.py", - "vllm_ascend/torchair/ops/torchair_fused_moe.py" - ] - }, - { - "commit": "d3c93fba5ca9279ec9f0ebb9026f90abf6132f38", - "subject": "[3/N][Feat][Graph] Support `all-to-all` and quantized models with ACL Graph (#2614)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/e2e/multicard/moe/test_moe_comm.py", - "tests/e2e/multicard/test_qwen3_moe.py", - "vllm_ascend/distributed/moe_comm_method.py", - "vllm_ascend/ops/common_fused_moe.py", - "vllm_ascend/ops/fused_moe.py", - "vllm_ascend/quantization/w8a8_dynamic.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "91c35d765aa2edeb3e9c805f2fe3c330320fe696", - "subject": "[Bugfix] Fix mc2 operator error in aclgraph + ep<16 scenario (#2609)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/e2e/multicard/test_qwen3_moe.py", - "vllm_ascend/ops/common_fused_moe.py" - ] - }, - { - "commit": "52aff9e229b8c5557bb2ee485ec381e0886d3701", - "subject": "[main] [bugfix] Fix misjudging quantized/unquantized scenarios (#2627)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/ops/test_fused_ops.py", - "tests/ut/ops/test_token_dispatcher.py", - "vllm_ascend/ascend_forward_context.py", - "vllm_ascend/ops/fused_moe.py", - "vllm_ascend/ops/moe_dispatcher/token_dispatcher.py", - "vllm_ascend/quantization/w4a8_dynamic.py", - "vllm_ascend/quantization/w8a8_dynamic.py" - ] - }, - { - "commit": "dfc7eb39ada3f86f5c15425ba759ecfaa8f5c9a8", - "subject": "[Fix] Fix DP-related padding logic (#2582)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/distributed/moe_comm_method.py", - "vllm_ascend/ops/common_fused_moe.py", - "vllm_ascend/torchair/torchair_model_runner.py", - "vllm_ascend/worker/model_runner_v1.py", - "vllm_ascend/worker/mtp_proposer_v1.py" - ] - }, - { - "commit": "1191a64ae508183d5613711bc98a90250963f83a", - "subject": "[Feat]attention add sliding windows size (#2528)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/attention/test_attention_v1.py", - "vllm_ascend/attention/attention_v1.py" - ] - }, - { - "commit": "c8d1df3a3fa803a8a0742df80cf7601a8b15ef7a", - "subject": "[Refactor][WIP] Refactor mla_v1 by moving all MLA preprocessing ops into mla_v1 attention impl (#2465)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "docs/source/user_guide/configuration/additional_config.md", - "tests/ut/attention/test_mla_v1.py", - "vllm_ascend/ascend_config.py", - "vllm_ascend/attention/mla_v1.py", - "vllm_ascend/models/deepseek_v2.py" - ] - }, - { - "commit": "320edde2df14a16436d7094012c12592b2e16266", - "subject": "[main] [refactor] refactor fused_moe.py to enable token_dispatchers (#2570)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/e2e/singlecard/ops/test_fused_moe.py", - "tests/ut/models/test_deepseek_v2.py", - "tests/ut/ops/test_fused_ops.py", - "tests/ut/ops/test_token_dispatcher.py", - "tests/ut/quantization/test_w8a8_dynamic.py", - "vllm_ascend/ascend_forward_context.py", - "vllm_ascend/ops/fused_moe.py", - "vllm_ascend/ops/moe_dispatcher/token_dispatcher.py", - "vllm_ascend/quantization/w4a8_dynamic.py", - "vllm_ascend/quantization/w8a8_dynamic.py" - ] - }, - { - "commit": "936c102105b72a4e36dd284f900f32338a232696", - "subject": "[bugfix][refactor]fix torchair_w8a8 (#2569)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/torchair/quantization/torchair_w4a8_dynamic.py", - "vllm_ascend/torchair/quantization/torchair_w8a8_dynamic.py" - ] - }, - { - "commit": "2bfbf9b9b3f6cd332fc438fc322b00b6053a043c", - "subject": "[main][bugfix] Fix bugs and refactor cached mask generation logic (#2442)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/attention/test_attention_mask.py", - "vllm_ascend/attention/attention_mask.py", - "vllm_ascend/worker/eagle_proposer_v1.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "6881c194580b800cb233632300575b9b387778ef", - "subject": "[main] convert the format of gmm to nz (#2474)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/ops/test_fused_ops.py", - "vllm_ascend/ops/fused_moe.py" - ] - }, - { - "commit": "20a7bc4b71827a39c21ecac95536183468ad90a7", - "subject": "[3/N][refactor] refactoer quantization (#2504)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/torchair/quantization/test_torchair_w4a8_dynamic.py", - "tests/ut/torchair/quantization/test_torchair_w8a8_dynamic.py", - "tests/ut/torchair/test_utils.py", - "vllm_ascend/torchair/models/torchair_deepseek_v2.py", - "vllm_ascend/torchair/quantization/__init__.py", - "vllm_ascend/torchair/quantization/torchair_quantizer.py", - "vllm_ascend/torchair/quantization/torchair_w4a8_dynamic.py", - "vllm_ascend/torchair/quantization/torchair_w8a8_dynamic.py", - "vllm_ascend/torchair/torchair_model_runner.py", - "vllm_ascend/torchair/utils.py" - ] - }, - { - "commit": "6a4ec186e731b9516235f4fd30b5b98227513fe7", - "subject": "[Qwen-moe] Remove the minor operation arange (#2373)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/e2e/singlecard/ops/test_fused_moe.py", - "tests/ut/ops/test_fused_ops.py", - "tests/ut/quantization/test_w8a8.py", - "tests/ut/quantization/test_w8a8_dynamic.py", - "vllm_ascend/ops/common_fused_moe.py", - "vllm_ascend/ops/fused_moe.py", - "vllm_ascend/ops/layers/experts_selector.py", - "vllm_ascend/quantization/w4a8_dynamic.py", - "vllm_ascend/quantization/w8a8_dynamic.py" - ] - }, - { - "commit": "a6bb502e70b7554b2a0342565348b1a191cd0aa0", - "subject": "[2/N][Feat] Add MC2 communication method for MoE layers (#2469)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/e2e/multicard/moe/test_moe_comm.py", - "tests/ut/distributed/test_communicator.py", - "tests/ut/test_utils.py", - "vllm_ascend/ascend_forward_context.py", - "vllm_ascend/distributed/communicator.py", - "vllm_ascend/distributed/moe_comm_method.py", - "vllm_ascend/models/pangu_moe.py", - "vllm_ascend/ops/common_fused_moe.py", - "vllm_ascend/ops/fused_moe.py", - "vllm_ascend/utils.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "5d8ec280090b4a7567fb2b50a7cedda44902c37f", - "subject": "[2/N][refactor] split torchair from fused_moe (#2503)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/ops/fused_moe.py" - ] - }, - { - "commit": "de7649492ddcbdb7c818665f0b81cc8fbaaaa4b7", - "subject": "[Refactor] cleanup converting_weight_acl_format_format (#2482)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/test_utils.py", - "tests/ut/torchair/test_utils.py", - "vllm_ascend/torchair/torchair_model_runner.py", - "vllm_ascend/torchair/utils.py", - "vllm_ascend/utils.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "0f81e032f04b72f4dd0c7fefd62b7220942c545a", - "subject": "[1/N][refactor] torchair fused_moe refactor (#2438)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/torchair/models/test_torchair_deepseek_v2.py", - "tests/ut/torchair/ops/test_torchair_fused_moe.py", - "vllm_ascend/torchair/models/torchair_deepseek_v2.py", - "vllm_ascend/torchair/ops/__init__.py", - "vllm_ascend/torchair/ops/torchair_fused_moe.py" - ] - }, - { - "commit": "950c4b219a8fe5d4339de3acd535b32b99e790ad", - "subject": "[main] refactor alltoallv in fused_moe (#2487)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/ops/test_token_dispatcher.py", - "vllm_ascend/ops/moe_dispatcher/token_dispatcher.py" - ] - }, - { - "commit": "3629bc4431d3edb4224761f9036b3bddb16158d6", - "subject": "feat: add mtp ut and fix some bugs (#2453)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/e2e/singlecard/spec_decode_v1/test_v1_mtp_correctness.py", - "tests/ut/quantization/test_quant_config.py", - "tests/ut/torchair/test_torchair_mla.py", - "vllm_ascend/attention/mla_v1.py", - "vllm_ascend/models/deepseek_mtp.py", - "vllm_ascend/ops/fused_moe.py", - "vllm_ascend/quantization/quant_config.py", - "vllm_ascend/torchair/torchair_mla.py", - "vllm_ascend/worker/mtp_proposer_v1.py", - "vllm_ascend/worker/worker_v1.py" - ] - }, - { - "commit": "0ca3f48c900b333673830e8307c259acc684c1a3", - "subject": "[2/N][refactor] torchair deepseek mla backend refactor (#2459)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/attention/test_mla_v1.py", - "tests/ut/test_platform.py", - "tests/ut/torchair/test_torchair_mla.py", - "vllm_ascend/attention/mla_v1.py", - "vllm_ascend/platform.py", - "vllm_ascend/torchair/torchair_mla.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "3f867ee7081f6f041652180ccf06d0c70fd44429", - "subject": "refactor allgather/mc2-related fused_experts (#2369)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/ops/test_token_dispatcher.py", - "vllm_ascend/ops/moe_dispatcher/token_dispatcher.py" - ] - }, - { - "commit": "7bec1a9b9c372785551d45682bf11063ec42b216", - "subject": "qwen3_moe/qwen25 support torchair graph (#2403)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/e2e/multicard/test_torchair_graph_mode.py", - "tests/ut/models/test_qwen3_moe.py", - "tests/ut/test_ascend_config.py", - "vllm_ascend/ascend_config.py", - "vllm_ascend/ops/rotary_embedding.py", - "vllm_ascend/torchair/models/qwen2.py", - "vllm_ascend/torchair/models/qwen3_moe.py", - "vllm_ascend/torchair/torchair_attention.py", - "vllm_ascend/torchair/utils.py" - ] - }, - { - "commit": "1327f9be1cea85b2750ac4145981ddd732064bb9", - "subject": "Fix some ci issue and refactor modelrunner (#2445)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/vllm_ascend_test.yaml", - "examples/offline_inference_audio_language.py", - "tests/e2e/singlecard/sample/test_rejection_sampler.py", - "tests/ut/attention/test_attention_v1.py", - "tests/ut/attention/test_mla_v1.py", - "tests/ut/core/test_scheduler.py", - "tests/ut/kv_connector/test_remote_prefill_lifecycle.py", - "tests/ut/kv_connector/utils.py", - "tests/ut/ops/test_fused_ops.py", - "tests/ut/test_platform.py", - "tests/ut/worker/test_input_batch.py", - "vllm_ascend/ascend_forward_context.py", - "vllm_ascend/attention/attention_v1.py", - "vllm_ascend/attention/mla_v1.py", - "vllm_ascend/attention/utils.py", - "vllm_ascend/compilation/acl_graph.py", - "vllm_ascend/compilation/piecewise_backend.py", - "vllm_ascend/lora/punica_wrapper/lora_ops.py", - "vllm_ascend/meta_registration.py", - "vllm_ascend/platform.py", - "vllm_ascend/torchair/torchair_attention.py", - "vllm_ascend/torchair/torchair_model_runner.py", - "vllm_ascend/torchair/utils.py", - "vllm_ascend/worker/eagle_proposer_v1.py", - "vllm_ascend/worker/model_runner_v1.py", - "vllm_ascend/worker/mtp_proposer_v1.py", - "vllm_ascend/worker/npu_input_batch.py", - "vllm_ascend/worker/worker_v1.py" - ] - }, - { - "commit": "3fc31ee1cbdf0c0d11efc4da5fd865bb2d077c4b", - "subject": "[1/N][refactor] torchair deepseek modeling refactor (#2384)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/torchair/models/test_torchair_deepseek_mtp.py", - "tests/ut/torchair/models/test_torchair_deepseek_v2.py", - "tests/ut/torchair/test_utils.py", - "vllm_ascend/torchair/models/__init__.py", - "vllm_ascend/torchair/models/torchair_deepseek_mtp.py", - "vllm_ascend/torchair/models/torchair_deepseek_v2.py", - "vllm_ascend/torchair/models/torchair_deepseek_v3.py", - "vllm_ascend/torchair/torchair_model_runner.py", - "vllm_ascend/torchair/utils.py" - ] - }, - { - "commit": "e14f2ef6690a04a3359220c94d6ea3eed7af5049", - "subject": "refactor select_experts of moe module (#2150)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/e2e/singlecard/ops/test_fused_moe.py", - "tests/ut/ops/test_fused_ops.py", - "tests/ut/quantization/test_w8a8.py", - "vllm_ascend/ops/common_fused_moe.py", - "vllm_ascend/ops/fused_moe.py", - "vllm_ascend/ops/layers/__init__.py", - "vllm_ascend/ops/layers/experts_selector.py", - "vllm_ascend/quantization/w4a8_dynamic.py", - "vllm_ascend/quantization/w8a8.py", - "vllm_ascend/quantization/w8a8_dynamic.py" - ] - }, - { - "commit": "103654ccd6f90d76b49354a00185601ac2d9ef7c", - "subject": "[Misc] Remove redundant imported `envs`, using `envs_ascend` instead (#2193)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/patch/worker/patch_common/test_patch_linear.py", - "tests/ut/test_envs.py", - "vllm_ascend/ascend_forward_context.py", - "vllm_ascend/attention/mla_v1.py", - "vllm_ascend/compilation/piecewise_backend.py", - "vllm_ascend/distributed/llmdatadist_c_mgr_connector.py", - "vllm_ascend/models/__init__.py", - "vllm_ascend/patch/platform/patch_common/patch_distributed.py", - "vllm_ascend/patch/worker/patch_common/patch_linear.py", - "vllm_ascend/platform.py", - "vllm_ascend/quantization/w8a8_dynamic.py", - "vllm_ascend/utils.py", - "vllm_ascend/worker/model_runner_v1.py", - "vllm_ascend/worker/worker_v1.py" - ] - }, - { - "commit": "8914d5a4b27ba3e6c0b908e6acfcaa228e015e97", - "subject": "[Quickfix] Add the missing `apply_router_weight_on_input` in FusedMoE init (#2348)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/ops/fused_moe.py" - ] - }, - { - "commit": "0f7492d18e572f53cce67615aa714ad09090f859", - "subject": "[Bugfix] fix the oom when chunkprefill with long context like 64k (#2319)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "992271b027770257915386d06a715eba1dbc9c12", - "subject": "[1/N][Feat] Support MoE models with ACL Graph and refactor MoE communication logic (#2125)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/e2e/multicard/moe/test_moe_comm.py", - "vllm_ascend/ascend_forward_context.py", - "vllm_ascend/distributed/moe_comm_method.py", - "vllm_ascend/ops/common_fused_moe.py", - "vllm_ascend/ops/fused_moe.py", - "vllm_ascend/platform.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "dc585f148acc91916916556ffd37fda708657cd3", - "subject": "[main][prefill optimization] Optimize parallel strategies to reduce communication overhead (#2198)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/user_guide/configuration/additional_config.md", - "tests/ut/attention/test_mla_v1.py", - "vllm_ascend/ascend_config.py", - "vllm_ascend/attention/mla_v1.py", - "vllm_ascend/models/deepseek_v2.py", - "vllm_ascend/ops/fused_moe.py" - ] - }, - { - "commit": "ad1083761f06737eb1ad2424dcd74aab47b559cb", - "subject": "[CI][Quickfix] Fix AscendFusedMoE init error (#2268)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/ops/test_fused_ops.py", - "vllm_ascend/ops/fused_moe.py" - ] - }, - { - "commit": "dceef080b140305841a98d71cd8cfdeccbd390af", - "subject": "[main] remove torch.cat and replace it by List[0] (#2153)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/ops/test_fused_ops.py", - "vllm_ascend/ops/fused_moe.py" - ] - }, - { - "commit": "b2598c3271af2d55a948e29799e90bc0cfcda188", - "subject": "enable mm allreduce test (#2192)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/e2e/multicard/test_external_launcher.py", - "vllm_ascend/patch/worker/patch_common/patch_linear.py" - ] - }, - { - "commit": "c6112916614b3be200bb49f66c89181be055d10a", - "subject": "\u3010main\u3011SP For Qwen3 MoE (#2209)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/vllm_ascend_test.yaml", - "tests/e2e/multicard/test_offline_inference_distributed.py", - "tests/ut/test_platform.py", - "vllm_ascend/attention/attention_v1.py", - "vllm_ascend/attention/attention_v1_torchair.py", - "vllm_ascend/attention/mla_v1.py", - "vllm_ascend/models/qwen3_moe.py", - "vllm_ascend/ops/fused_moe.py", - "vllm_ascend/ops/sequence_parallel.py", - "vllm_ascend/platform.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "26fc36b0e026f854f703d7ffc113c682985f2319", - "subject": "[V1] MTP supports torchair (#2145)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/attention/test_mla_v1.py", - "tests/ut/models/test_deepseek_mtp.py", - "tests/ut/quantization/test_quant_config.py", - "vllm_ascend/attention/attention_v1_torchair.py", - "vllm_ascend/attention/mla_v1.py", - "vllm_ascend/models/deepseek_mtp.py", - "vllm_ascend/ops/fused_moe.py", - "vllm_ascend/ops/vocab_parallel_embedding.py", - "vllm_ascend/quantization/quant_config.py", - "vllm_ascend/sample/rejection_sampler.py", - "vllm_ascend/worker/model_runner_v1.py", - "vllm_ascend/worker/mtp_proposer_v1.py" - ] - }, - { - "commit": "8a59367d0c249b8242bbaa9fb69a2713ea596567", - "subject": "[main][Feature] Support deepseek w4a8 quantization (#2172)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/workflows/vllm_ascend_test.yaml", - "tests/e2e/multicard/test_offline_inference_distributed.py", - "tests/ut/quantization/test_quantizer.py", - "tests/ut/quantization/test_w4a8_dynamic.py", - "vllm_ascend/models/deepseek_v2.py", - "vllm_ascend/quantization/quant_config.py", - "vllm_ascend/quantization/quantizer.py", - "vllm_ascend/quantization/w4a8_dynamic.py", - "vllm_ascend/quantization/w8a8_dynamic.py" - ] - }, - { - "commit": "e31b31f9c3fa2f758724746c29a8b5876af38cac", - "subject": "[main][Bugfix] Fix unable to load qwen3_moe quantized weights (#2219)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/e2e/multicard/test_qwen3_moe.py", - "vllm_ascend/models/qwen3_moe.py" - ] - }, - { - "commit": "458ab2db12ad3d4104eadf0e13a452464b09765d", - "subject": "[BugFix] Fix the bug that qwen3 moe doesn't work with aclgraph (#2183)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/e2e/multicard/test_qwen3_moe.py", - "vllm_ascend/models/qwen3_moe.py", - "vllm_ascend/ops/fused_moe.py" - ] - }, - { - "commit": "807f0895b2cf4f569f0c86aa808149bef70493b1", - "subject": "Bump torch version to 2.7.1 (#1562)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/release_whl.yml", - "README.md", - "README.zh.md", - "docs/source/installation.md", - "pyproject.toml", - "requirements.txt", - "tests/e2e/singlecard/ops/test_fused_moe.py", - "tests/ut/patch/worker/patch_common/test_patch_utils.py", - "tests/ut/test_platform.py", - "vllm_ascend/__init__.py", - "vllm_ascend/patch/__init__.py", - "vllm_ascend/patch/worker/patch_common/__init__.py", - "vllm_ascend/patch/worker/patch_common/patch_utils.py", - "vllm_ascend/platform.py", - "vllm_ascend/quantization/quant_config.py" - ] - }, - { - "commit": "36e450eb0f4fb5e8a262384fb302a5b7dc83b582", - "subject": "[Misc] Nit fix for disaggregated_prefill and ascend_forward_context (#2097)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "examples/disaggregated_prefill_v1/gen_ranktable.py", - "tests/ut/ops/test_fused_ops.py", - "vllm_ascend/ascend_forward_context.py", - "vllm_ascend/distributed/llmdatadist_c_mgr_connector.py", - "vllm_ascend/soc_info.py", - "vllm_ascend/utils.py", - "vllm_ascend/worker/worker_v1.py" - ] - }, - { - "commit": "ad366bf90809ee60b80a2e768a47ed4ab2b66b52", - "subject": "[Bugfix] Follow vLLM Qwen-Moe/VL and KV Connector change to fix broken CI (#2181)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/kv_connector/test_remote_decode_lifecycle.py", - "tests/ut/kv_connector/test_remote_prefill_lifecycle.py", - "tests/ut/kv_connector/utils.py", - "vllm_ascend/models/qwen2_5_vl.py", - "vllm_ascend/models/qwen2_5_vl_without_padding.py", - "vllm_ascend/models/qwen3_moe.py", - "vllm_ascend/worker/model_runner_v1.py", - "vllm_ascend/worker/worker_v1.py" - ] - }, - { - "commit": "a9480d5f0a031919d151a842c56563d318e4ecf6", - "subject": "[Fix] Adjust use_aclgraph logic (#2156)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/ops/common_fused_moe.py", - "vllm_ascend/ops/fused_moe.py" - ] - }, - { - "commit": "af04ee9e7a53dfab5ddbcee5535e8645cc84c169", - "subject": "[MoE][Dist] Fix Qwen MoE accuracy bug in DP scenario (#1856)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/e2e/long_term/accuracy/accuracy_multicard.py", - "tests/e2e/multicard/test_data_parallel.py", - "vllm_ascend/distributed/communicator.py" - ] - }, - { - "commit": "6e00aed4d540432ffe2317601d47039d4ef2332b", - "subject": "[main][Feature]Moe alltoallv communication optimization for unquantized RL training sence (#2088)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/workflows/vllm_ascend_test.yaml", - "requirements-dev.txt", - "tests/e2e/multicard/test_offline_inference_distributed.py", - "tests/ut/distributed/test_distributed_tensor_parallel.py", - "tests/ut/ops/test_token_dispatcher.py", - "vllm_ascend/ascend_forward_context.py", - "vllm_ascend/distributed/tensor_parallel.py", - "vllm_ascend/envs.py", - "vllm_ascend/models/__init__.py", - "vllm_ascend/models/qwen3_moe.py", - "vllm_ascend/ops/comm_utils.py", - "vllm_ascend/ops/fused_moe.py", - "vllm_ascend/ops/moe_dispatcher/__init__.py", - "vllm_ascend/ops/moe_dispatcher/token_dispatcher.py" - ] - }, - { - "commit": "8cf97d8310cfdad33efc1af24c3251fe6c9c973e", - "subject": "[Misc] Add extra checking to torchair_graph_config. (#1939)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "examples/offline_data_parallel.py", - "tests/ut/test_ascend_config.py", - "vllm_ascend/ascend_config.py", - "vllm_ascend/models/deepseek_v2.py", - "vllm_ascend/ops/fused_moe.py" - ] - }, - { - "commit": "9c9a7cd90bf396eaac09402b10382cd61fc921e4", - "subject": "[main] adapt usage of npu_moe_gating_top_k_softmax and remove envs.SELECT_GATING_TOPK_SOTFMAX_EXPERTS (#2112)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/e2e/singlecard/ops/test_fused_moe.py", - "tests/ut/ops/test_fused_ops.py", - "vllm_ascend/envs.py", - "vllm_ascend/ops/common_fused_moe.py", - "vllm_ascend/ops/fused_moe.py" - ] - }, - { - "commit": "2008152c48c976a3d2858cccc9f8d301b0915df6", - "subject": "[main][bugfix]Fix vLLM startup failure when inferring DeepSeek R1 model in DP scenario (#2020)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/ops/fused_moe.py" - ] - }, - { - "commit": "b6a7f07c701984eb5c76e474a74f8889f9c300c5", - "subject": "[Perf][MoE] Improve MoE multistream parallel performace. (#1891)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/models/deepseek_v2.py", - "vllm_ascend/ops/fused_moe.py", - "vllm_ascend/quantization/w8a8_dynamic.py" - ] - }, - { - "commit": "f60bb474f98dff267d9e737a396f65f2430aba09", - "subject": "[CI] Enable linux-aarch64-a2 (64GB) and tp2 * 2 max-parallel to speed up CI (#2065)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - ".github/actionlint.yaml", - ".github/workflows/accuracy_test.yaml", - ".github/workflows/vllm_ascend_doctest.yaml", - ".github/workflows/vllm_ascend_test.yaml", - ".github/workflows/vllm_ascend_test_long_term.yaml", - "benchmarks/scripts/run_accuracy.py", - "examples/disaggregated_prefill_v1/gen_ranktable.sh", - "tests/e2e/long_term/accuracy/accuracy_multicard.py", - "tests/e2e/multicard/test_fused_moe_allgather_ep.py", - "tests/e2e/multicard/test_offline_inference_distributed.py", - "tests/e2e/multicard/test_pipeline_parallel.py", - "tests/e2e/multicard/test_torchair_graph_mode.py", - "tests/e2e/singlecard/quant/test_w8a8.py", - "tests/e2e/singlecard/test_offline_inference.py" - ] - }, - { - "commit": "32a9c5f69430c326a63f930c820386b3b3ae66c2", - "subject": "[Feature]: implement the fusion of allreduce and matmul in prefill phase when tp is enabled (#1926)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/base.py", - "tests/ut/patch/worker/patch_common/test_patch_linear.py", - "vllm_ascend/envs.py", - "vllm_ascend/patch/__init__.py", - "vllm_ascend/patch/worker/patch_common/__init__.py", - "vllm_ascend/patch/worker/patch_common/patch_linear.py" - ] - }, - { - "commit": "ba3dfbd59e43b9071895f483d12c034d8538ced0", - "subject": "[main][refactor] Refactoring forward_context and model_runner_v1 (#1979)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "examples/offline_dualbatch_overlap_npu.py", - "examples/run_dp_server.sh", - "tests/ut/models/test_deepseek_v2.py", - "tests/ut/ops/test_fused_ops.py", - "vllm_ascend/ascend_forward_context.py", - "vllm_ascend/attention/attention_v1.py", - "vllm_ascend/attention/attention_v1_torchair.py", - "vllm_ascend/attention/mla_v1.py", - "vllm_ascend/distributed/parallel_state.py", - "vllm_ascend/models/deepseek_dbo.py", - "vllm_ascend/models/deepseek_v2.py", - "vllm_ascend/models/pangu_moe.py", - "vllm_ascend/multistream/ms_split.py", - "vllm_ascend/ops/fused_moe.py", - "vllm_ascend/quantization/quantizer.py", - "vllm_ascend/quantization/w8a8_dynamic.py", - "vllm_ascend/torchair/torchair_worker.py", - "vllm_ascend/utils.py", - "vllm_ascend/worker/eagle_proposer_v1.py", - "vllm_ascend/worker/model_runner_v1.py", - "vllm_ascend/worker/mtp_proposer_v1.py", - "vllm_ascend/worker/worker_v1.py" - ] - }, - { - "commit": "df0ec55162339c08b0bfcb79a2dcfac89f8c6e33", - "subject": "Disaggregate prefill for kv cache register style (#950)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/vllm_ascend_test_pd.yaml", - "examples/disaggregated_prefill_v1/README.md", - "examples/disaggregated_prefill_v1/gen_ranktable.py", - "examples/disaggregated_prefill_v1/gen_ranktable.sh", - "examples/disaggregated_prefill_v1/run_server.sh", - "examples/disaggregated_prefill_v1/toy_proxy_server.py", - "tests/e2e/multicard/test_fused_moe_allgather_ep.py", - "tests/e2e/multicard/test_offline_inference_distributed.py", - "tests/e2e/pd_disaggreate/run_edge_case_test.sh", - "tests/e2e/pd_disaggreate/test_edge_cases.py", - "tests/ut/attention/test_attention_v1.py", - "tests/ut/kv_connector/test_llmdatadist_connector.py", - "tests/ut/kv_connector/test_remote_decode_lifecycle.py", - "tests/ut/kv_connector/test_remote_prefill_lifecycle.py", - "tests/ut/kv_connector/utils.py", - "vllm_ascend/attention/attention_v1.py", - "vllm_ascend/attention/attention_v1_torchair.py", - "vllm_ascend/attention/mla_v1.py", - "vllm_ascend/core/scheduler.py", - "vllm_ascend/distributed/__init__.py", - "vllm_ascend/distributed/llmdatadist_c_mgr_connector.py", - "vllm_ascend/envs.py", - "vllm_ascend/models/deepseek_v2.py", - "vllm_ascend/ops/attention.py", - "vllm_ascend/quantization/w8a8_dynamic.py", - "vllm_ascend/soc_info.py", - "vllm_ascend/worker/model_runner_v1.py", - "vllm_ascend/worker/worker_v1.py" - ] - }, - { - "commit": "2ffe051859d585df8353d1b9eefb64c44078175a", - "subject": "[Test]add ut for deepseek_v2. (#1964)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/models/test_deepseek_v2.py", - "tests/ut/ops/test_fused_ops.py" - ] - }, - { - "commit": "ac0bf133f47ead20f18bf71f9be6dbe05fbd218f", - "subject": "add ut of fused_moe.py (#1930)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/ops/test_fused_ops.py" - ] - }, - { - "commit": "9a3bdf21628e5942437af72698c4efd23ad1bc96", - "subject": "[main] Use AddRmsNormQuant ops in the custom model to optimize Qwen3's performance (#1806)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/e2e/multicard/test_offline_inference_distributed.py", - "vllm_ascend/models/__init__.py", - "vllm_ascend/models/qwen3.py", - "vllm_ascend/ops/layernorm.py", - "vllm_ascend/quantization/w8a8.py" - ] - }, - { - "commit": "7265dc090d0ef0d313d7a3ea6ddf0bc64cb2de07", - "subject": "[2/4][Refactor] Refactor torchair utils (#1892)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/__init__.py", - "tests/ut/test_utils.py", - "tests/ut/torchair/__init__.py", - "tests/ut/torchair/test_utils.py", - "vllm_ascend/attention/mla_v1.py", - "vllm_ascend/ops/fused_moe.py", - "vllm_ascend/quantization/w8a8_dynamic.py", - "vllm_ascend/torchair/torchair_worker.py", - "vllm_ascend/torchair/utils.py", - "vllm_ascend/utils.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "8cfd257992c116fe8d186e17d948528f4e8fe10b", - "subject": "[Dist][EP] Remove ETP/EP maintained in vllm-ascend (#1681)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "docs/source/tutorials/multi_npu_moge.md", - "docs/source/user_guide/configuration/additional_config.md", - "examples/run_dp_attention_etp16.sh", - "examples/run_dp_attention_etp16_benmark.sh", - "tests/e2e/long_term/accuracy/accuracy_multicard.py", - "tests/e2e/multicard/test_expert_parallel.py", - "tests/e2e/multicard/test_fused_moe_allgather_ep.py", - "tests/e2e/multicard/test_torchair_graph_mode.py", - "tests/ut/distributed/test_parallel_state.py", - "tests/ut/test_ascend_config.py", - "tests/ut/test_platform.py", - "vllm_ascend/ascend_config.py", - "vllm_ascend/distributed/parallel_state.py", - "vllm_ascend/models/deepseek_v2.py", - "vllm_ascend/models/pangu_moe.py", - "vllm_ascend/ops/common_fused_moe.py", - "vllm_ascend/ops/fused_moe.py", - "vllm_ascend/patch/__init__.py", - "vllm_ascend/patch/platform/patch_common/patch_distributed.py", - "vllm_ascend/platform.py", - "vllm_ascend/quantization/w8a8.py", - "vllm_ascend/quantization/w8a8_dynamic.py", - "vllm_ascend/utils.py", - "vllm_ascend/worker/worker_v1.py" - ] - }, - { - "commit": "2b726d8f905b29d776eb1fea18c37e7a16522fc3", - "subject": "[CI] Fix broken CI (#1889)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "pyproject.toml", - "requirements.txt", - "vllm_ascend/ops/common_fused_moe.py", - "vllm_ascend/worker/model_runner_v1.py", - "vllm_ascend/worker/npu_input_batch.py", - "vllm_ascend/worker/worker_v1.py" - ] - }, - { - "commit": "d08ff304cdd0642c89ac64ce948a1d94ae397a50", - "subject": "[Misc][V0 Deprecation] Remove V0 Attention (#1835)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/attention/attention.py", - "vllm_ascend/attention/mla_v1.py" - ] - }, - { - "commit": "19e37cd3792dc9fba48226139d5197e4863fa677", - "subject": "[Misc] Add `fusion_result.json` to `.gitignore` (#1836)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - ".gitignore" - ] - }, - { - "commit": "787010a637ad110b2ef3f12fab9e57f6607938e5", - "subject": "[Test] Remove VLLM_USE_V1 in example and tests (#1733)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/vllm_ascend_test.yaml", - "examples/offline_data_parallel.py", - "examples/offline_dualbatch_overlap_npu.py", - "examples/offline_inference_sleep_mode_npu.py", - "examples/run_dp_attention_etp16.sh", - "requirements-dev.txt", - "requirements-lint.txt", - "tests/e2e/conftest.py", - "tests/e2e/model_utils.py", - "tests/e2e/multicard/test_fused_moe_allgather_ep.py", - "tests/e2e/multicard/test_ilama_lora_tp2.py", - "tests/e2e/multicard/test_offline_inference_distributed.py", - "tests/e2e/multicard/test_pipeline_parallel.py", - "tests/e2e/multicard/test_prefix_caching.py", - "tests/e2e/multicard/test_torchair_graph_mode.py", - "tests/e2e/singlecard/core/ascend_scheduler/test_ascend_scheduler_e2e.py", - "tests/e2e/singlecard/core/ascend_scheduler/test_chunk_prefill.py", - "tests/e2e/singlecard/spec_decode_v1/test_v1_mtp_correctness.py", - "tests/e2e/singlecard/spec_decode_v1/test_v1_spec_decode.py", - "tests/e2e/singlecard/test_aclgraph.py", - "tests/e2e/singlecard/test_camem.py", - "tests/e2e/singlecard/test_chunked.py", - "tests/e2e/singlecard/test_embedding.py", - "tests/e2e/singlecard/test_guided_decoding.py", - "tests/e2e/singlecard/test_ilama_lora.py", - "tests/e2e/singlecard/test_offline_inference.py", - "tests/e2e/utils.py", - "tests/ut/test_ascend_config.py", - "tests/ut/test_platform.py" - ] - }, - { - "commit": "ee40d3d850bc906e06e66b81b3bb233f38b16546", - "subject": "use npu_moe_gating_top_k_softmax (#1355)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/e2e/singlecard/ops/test_gating_top_k_softmax.py", - "vllm_ascend/envs.py", - "vllm_ascend/ops/common_fused_moe.py", - "vllm_ascend/ops/fused_moe.py" - ] - }, - { - "commit": "9d16c9982e7bb67053e828b1cd8ba734b11670b2", - "subject": "rm router logits Improve TTOP 3ms (#1407)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/models/deepseek_v2.py", - "vllm_ascend/ops/fused_moe.py", - "vllm_ascend/utils.py" - ] - }, - { - "commit": "c7446438a98b56a2b7aec5dbb0a92c7dc75138e9", - "subject": "[1/N][CI] Move linting system to pre-commits hooks (#1256)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - ".github/ISSUE_TEMPLATE/900-release-checklist.yml", - ".github/workflows/doc_codespell.yaml", - ".github/workflows/pre-commit.yml", - ".github/workflows/shellcheck.yml", - ".github/workflows/vllm_ascend_test.yaml", - ".pre-commit-config.yaml", - "benchmarks/ops/ben_vocabparallelembedding.py", - "benchmarks/scripts/convert_json_to_markdown.py", - "benchmarks/scripts/patch_benchmark_dataset.py", - "benchmarks/scripts/run_accuracy.py", - "docs/source/developer_guide/contribution/index.md", - "docs/source/user_guide/additional_config.md", - "format.sh", - "requirements-lint.txt", - "tests/e2e/singlecard/core/ascend_scheduler/test_ascend_scheduler.py", - "tests/e2e/singlecard/ops/test_vocabparallelembedding.py", - "tests/e2e/singlecard/sample/test_rejection_sampler.py", - "tests/e2e/singlecard/test_scheduler.py", - "tools/enforce_regex_import.py", - "tools/mypy.sh", - "tools/shellcheck.sh", - "typos.toml", - "vllm_ascend/core/scheduler.py", - "vllm_ascend/distributed/kv_transfer/simple_connector.py", - "vllm_ascend/models/deepseek_mtp.py", - "vllm_ascend/ops/fused_moe.py", - "vllm_ascend/ops/vocab_parallel_embedding.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "60519c71bd1f1395989693e755fd3fc6a1a985c6", - "subject": "shared_experts+router_experts merge all_reduce(Improve TTOP 5ms) (#1395)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "examples/run_dp_attention_etp16.sh", - "examples/run_dp_attention_etp16_benmark.sh", - "vllm_ascend/models/deepseek_v2.py", - "vllm_ascend/ops/fused_moe.py", - "vllm_ascend/utils.py" - ] - }, - { - "commit": "89c1a0f00691ce4cda87cc7eb6f5e558e926a455", - "subject": "[Bugfix] Fix memory-leak caused by dist._functional_collectives.reduce_scatter_tensor (#1380)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/distributed/communication_op.py", - "vllm_ascend/ops/fused_moe.py" - ] - }, - { - "commit": "392fd7239bb8ab12e0ad45c454f6941ead6a2a56", - "subject": "[Misc] Add attention mask (#1673)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/attention/test_attention_mask.py", - "vllm_ascend/attention/attention.py", - "vllm_ascend/attention/attention_mask.py", - "vllm_ascend/worker/eagle_proposer_v1.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "830332ebfc649e5f0604513460e77207151c7545", - "subject": "Clean up v0.9.1 code (#1672)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/workflows/vllm_ascend_test.yaml", - "docs/source/developer_guide/feature_guide/patch.md", - "tests/e2e/multicard/test_offline_inference_distributed.py", - "tests/e2e/singlecard/core/ascend_scheduler/test_ascend_scheduler.py", - "tests/e2e/singlecard/sample/test_rejection_sampler.py", - "tests/e2e/singlecard/test_embedding.py", - "tests/e2e/singlecard/test_offline_inference.py", - "tests/e2e/singlecard/test_sampler.py", - "tests/e2e/singlecard/test_scheduler.py", - "tests/ut/patch/worker/patch_common/test_patch_sampler.py", - "vllm_ascend/core/scheduler.py", - "vllm_ascend/envs.py", - "vllm_ascend/models/deepseek_dbo.py", - "vllm_ascend/models/deepseek_v2.py", - "vllm_ascend/ops/fused_moe.py", - "vllm_ascend/patch/__init__.py", - "vllm_ascend/patch/platform/__init__.py", - "vllm_ascend/patch/platform/patch_0_9_2/__init__.py", - "vllm_ascend/patch/worker/__init__.py", - "vllm_ascend/patch/worker/patch_0_9_1/patch_sampler.py", - "vllm_ascend/patch/worker/patch_0_9_2/__init__.py", - "vllm_ascend/worker/model_runner_v1.py", - "vllm_ascend/worker/npu_input_batch.py" - ] - }, - { - "commit": "df84cceca841eb69b7a73e1a0941dc74084a3ba5", - "subject": "perf: use multicast to avoid padding decode request to prefill size (#1555)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/ops/fused_moe.py", - "vllm_ascend/quantization/w8a8_dynamic.py", - "vllm_ascend/utils.py" - ] - }, - { - "commit": "c58accc15e2a4e8a672c754035168a09ccf421f6", - "subject": "[Bugfix] Support Qwen3-MOE on aclgraph mode (#1381)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/e2e/singlecard/test_aclgraph.py", - "vllm_ascend/ops/common_fused_moe.py", - "vllm_ascend/ops/fused_moe.py" - ] - }, - { - "commit": "343955c7ac7c6615223366ad48fbd11ee74969e3", - "subject": "[CI] Follow vLLM FusedMoEParallelConfig interface change and clean up unused config (#1625)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/test_ascend_config.py", - "vllm_ascend/ascend_config.py", - "vllm_ascend/attention/attention_v1_torchair.py", - "vllm_ascend/ops/fused_moe.py" - ] - }, - { - "commit": "9fbd8017c0d1e6e09ac4568ff16931118a56ab12", - "subject": "[Quantization]300I Duo support w8a8 quantization (#1560)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/quantization/test_w8a8.py", - "tests/ut/test_utils.py", - "vllm_ascend/quantization/w8a8.py", - "vllm_ascend/utils.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "a45dfde283dfa555cec76f282b1e0a360fa2743d", - "subject": "[CI] Fix FusedMoEConfig and input batch failure to recover CI (#1602)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/e2e/singlecard/core/ascend_scheduler/test_ascend_scheduler.py", - "tests/e2e/singlecard/core/ascend_scheduler/test_chunk_prefill.py", - "tests/e2e/singlecard/sample/test_rejection_sampler.py", - "tests/e2e/singlecard/test_sampler.py", - "tests/ut/patch/worker/patch_common/test_patch_sampler.py", - "vllm_ascend/ops/fused_moe.py", - "vllm_ascend/patch/worker/patch_0_9_1/__init__.py", - "vllm_ascend/patch/worker/patch_0_9_1/patch_sampler.py", - "vllm_ascend/patch/worker/patch_common/__init__.py", - "vllm_ascend/worker/model_runner_v1.py", - "vllm_ascend/worker/npu_input_batch.py" - ] - }, - { - "commit": "6b80c5acbad0224477b8de4e202f2853ce992eb1", - "subject": "Fix W8A8 fused moe bug (#1529)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ut/attention/test_attention_v1.py", - "tests/ut/quantization/test_quant_config.py", - "tests/ut/quantization/test_quantizer.py", - "tests/ut/quantization/test_w8a8.py", - "vllm_ascend/attention/attention_v1.py", - "vllm_ascend/quantization/quantizer.py", - "vllm_ascend/quantization/w8a8.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "a054f0f4cae72d94db2f1f56c9f5605eb436682e", - "subject": "[CI] change to new ds model (#1513)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/e2e/multicard/test_fused_moe_allgather_ep.py", - "tests/e2e/multicard/test_offline_inference_distributed.py", - "tests/e2e/multicard/test_torchair_graph_mode.py" - ] - }, - { - "commit": "b308a7a25897b88d4a23a9e3d583f4ec6de256ac", - "subject": "support pangumoe w8a8c8 and docs (#1477)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "docs/source/user_guide/additional_config.md", - "vllm_ascend/attention/attention_v1.py", - "vllm_ascend/models/pangu_moe.py", - "vllm_ascend/platform.py", - "vllm_ascend/quantization/quant_config.py", - "vllm_ascend/quantization/quantizer.py", - "vllm_ascend/quantization/w8a8.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "c59d69d9e65de4b91628411ef415eca6bf512b44", - "subject": "[PERF]support MERRouter (#1421)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/models/pangu_moe.py", - "vllm_ascend/ops/common_fused_moe.py", - "vllm_ascend/ops/fused_moe.py" - ] - }, - { - "commit": "8fa188111da3a8f752dc309330d9bd3ec18194e6", - "subject": "[PERF]support H2P communication optimization for PanguProMoe (#1463)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/models/pangu_moe.py" - ] - }, - { - "commit": "d59e7fa0959a5571e7debd884d27ea2e6d9cb582", - "subject": "[CI] Pin transformers<4.53.0 and fix EPLB load_weights to make CI passed (#1482)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "pyproject.toml", - "requirements.txt", - "vllm_ascend/models/deepseek_dbo.py", - "vllm_ascend/models/deepseek_v2.py" - ] - }, - { - "commit": "5968dff4e000f8c4b00751d963898f1f0619b164", - "subject": "[Build] Add build info (#1386)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - ".gitignore", - "setup.py", - "tests/ut/base.py", - "tests/ut/patch/worker/patch_common/test_patch_distributed.py", - "tests/ut/patch/worker/patch_common/test_patch_sampler.py", - "tests/ut/test_utils.py", - "vllm_ascend/device_allocator/camem.py", - "vllm_ascend/ops/fused_moe.py", - "vllm_ascend/patch/platform/patch_common/patch_distributed.py", - "vllm_ascend/utils.py", - "vllm_ascend/worker/worker_v1.py" - ] - }, - { - "commit": "2690697caa47ab8daee4083778020ea7c13c16c7", - "subject": "[Bugfix] Reset all unused positions to prevent out-of-bounds in GatherV3 (#1416)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "2fda60464c287fe456b4a2f27e63996edc65dd40", - "subject": "[Perf] Use fused ops npu_top_k_top_p (#1308)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ut/worker/patch_common/test_patch_sampler.py", - "vllm_ascend/patch/worker/patch_common/patch_sampler.py" - ] - }, - { - "commit": "e7efc7e7e7675b5edf09e2e562c05c560df62114", - "subject": "[BugFix] Remove not using patch_eagle.py for CI. (#1385)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/patch/__init__.py", - "vllm_ascend/patch/worker/patch_common/__init__.py", - "vllm_ascend/patch/worker/patch_common/patch_eagle.py" - ] - }, - { - "commit": "941269a6c5bbc79f6c1b6abd4680dc5802dd8666", - "subject": "adjusting the communication method in graph mode (#1194)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/workflows/vllm_ascend_test.yaml", - "tests/e2e/multicard/test_offline_inference_distributed.py", - "vllm_ascend/attention/mla_v1.py", - "vllm_ascend/models/deepseek_dbo.py", - "vllm_ascend/models/deepseek_v2.py", - "vllm_ascend/ops/fused_moe.py" - ] - }, - { - "commit": "5177bef87a21331dcca11159d3d1438075cbd74e", - "subject": "support fused_moe_allgather_ep (#1335)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/multicard/test_fused_moe_allgather_ep.py", - "vllm_ascend/envs.py", - "vllm_ascend/ops/fused_moe.py", - "vllm_ascend/quantization/w8a8_dynamic.py", - "vllm_ascend/utils.py" - ] - }, - { - "commit": "097e7149f75c0806774bc68207f0f6270bc7d392", - "subject": "[Platform] Add initial experimental support for Altlas 300I series (#1333)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/workflows/doc_codespell.yaml", - ".github/workflows/image_310p_openeuler.yml", - ".github/workflows/image_310p_ubuntu.yml", - ".github/workflows/image_openeuler.yml", - ".github/workflows/image_ubuntu.yml", - "Dockerfile.310p", - "Dockerfile.310p.openEuler", - "csrc/kernels/get_masked_input_and_mask_kernel.cpp", - "csrc/kernels/pos_encoding_kernels.cpp", - "csrc/kernels/utils.h", - "format.sh", - "vllm_ascend/attention/attention.py", - "vllm_ascend/attention/attention_v1.py", - "vllm_ascend/ops/activation.py", - "vllm_ascend/ops/common_fused_moe.py", - "vllm_ascend/ops/fused_moe.py", - "vllm_ascend/ops/layernorm.py", - "vllm_ascend/ops/rotary_embedding.py", - "vllm_ascend/patch/platform/patch_common/patch_distributed.py", - "vllm_ascend/platform.py", - "vllm_ascend/utils.py", - "vllm_ascend/worker/model_runner_v1.py", - "vllm_ascend/worker/worker.py" - ] - }, - { - "commit": "2f1266d451b4576c297d6d65b508487a4f188303", - "subject": "Support Pangu Pro MoE model (#1204)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/models/__init__.py", - "vllm_ascend/models/pangu_moe.py", - "vllm_ascend/ops/common_fused_moe.py", - "vllm_ascend/ops/fused_moe.py" - ] - }, - { - "commit": "00ae250f3ced68317bc91c93dc1f1a0977aa0b94", - "subject": "[V1][eagle3] Support eagle3 proposer for v1 (#1032)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/workflows/vllm_ascend_test_long_term.yaml", - "tests/e2e/long_term/spec_decode/e2e/test_v1_spec_decode.py", - "tests/e2e/long_term/test_deepseek_v2_lite_tp2_accuracy.py", - "vllm_ascend/worker/eagle_proposer_v1.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "db2f630aebb0cad44f9705ac028993233a00c82e", - "subject": "[bugfix] fix deepseek with mc2 (#1268)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/ops/fused_moe.py" - ] - }, - { - "commit": "23ca68d0c8557a91b7213782de5c7fafa0cfb985", - "subject": "[refactor] Refactoring AscendFusedMoE (#1229)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/attention/mla_v1.py", - "vllm_ascend/envs.py", - "vllm_ascend/models/deepseek_dbo.py", - "vllm_ascend/models/deepseek_v2.py", - "vllm_ascend/ops/fused_moe.py", - "vllm_ascend/quantization/w8a8_dynamic.py", - "vllm_ascend/utils.py", - "vllm_ascend/worker/model_runner_v1.py", - "vllm_ascend/worker/worker_v1.py" - ] - }, - { - "commit": "69b817ed654b9dbff9e614c69752c2b76d9fe7d7", - "subject": "[CI] Add unit test framework (#1201)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/vllm_ascend_test.yaml", - ".github/workflows/vllm_ascend_test_long_term.yaml", - "format.sh", - "tests/e2e/long_term/spec_decode/__init__.py", - "tests/e2e/long_term/spec_decode/conftest.py", - "tests/e2e/long_term/spec_decode/e2e/__init__.py", - "tests/e2e/long_term/spec_decode/e2e/conftest.py", - "tests/e2e/long_term/spec_decode/e2e/test_medusa_correctness.py", - "tests/e2e/long_term/spec_decode/e2e/test_mlp_correctness.py", - "tests/e2e/long_term/spec_decode/e2e/test_mtp_correctness.py", - "tests/e2e/long_term/spec_decode/e2e/test_ngram_correctness.py", - "tests/e2e/long_term/spec_decode/e2e/test_v1_mtp_correctness.py", - "tests/e2e/long_term/spec_decode/e2e/test_v1_spec_decode.py", - "tests/e2e/long_term/spec_decode/test_dynamic_spec_decode.py", - "tests/e2e/long_term/spec_decode/test_multi_step_worker.py", - "tests/e2e/long_term/spec_decode/test_ngram_worker.py", - "tests/e2e/long_term/spec_decode/test_spec_decode_worker.py", - "tests/e2e/long_term/spec_decode/test_utils.py", - "tests/e2e/long_term/spec_decode/utils.py", - "tests/e2e/long_term/test_accuracy.py", - "tests/e2e/long_term/test_deepseek_v2_lite_tp2_accuracy.py", - "tests/e2e/multicard/test_dynamic_npugraph_batchsize.py", - "tests/e2e/multicard/test_ilama_lora_tp2.py", - "tests/e2e/multicard/test_offline_inference_distributed.py", - "tests/e2e/multicard/test_pyhccl_distributed.py", - "tests/e2e/multicard/test_torchair_graph_mode.py", - "tests/e2e/singlecard/__init__.py", - "tests/e2e/singlecard/compile/__init__.py", - "tests/e2e/singlecard/compile/test_simple.py", - "tests/e2e/singlecard/core/__init__.py", - "tests/e2e/singlecard/core/test_ascend_scheduler.py", - "tests/e2e/singlecard/core/test_ascend_scheduler_e2e.py", - "tests/e2e/singlecard/ops/__init__.py", - "tests/e2e/singlecard/ops/test_fused_moe.py", - "tests/e2e/singlecard/ops/test_multi_step.py", - "tests/e2e/singlecard/ops/test_rotary_embedding.py", - "tests/e2e/singlecard/ops/test_vocabparallelembedding.py", - "tests/e2e/singlecard/sample/__init__.py", - "tests/e2e/singlecard/sample/test_rejection_sampler.py", - "tests/e2e/singlecard/test_aclgraph.py", - "tests/e2e/singlecard/test_camem.py", - "tests/e2e/singlecard/test_chunked.py", - "tests/e2e/singlecard/test_guided_decoding.py", - "tests/e2e/singlecard/test_ilama_lora.py", - "tests/e2e/singlecard/test_offline_inference.py", - "tests/e2e/singlecard/test_profile_execute_duration.py", - "tests/e2e/singlecard/test_prompt_embedding.py", - "tests/e2e/singlecard/test_pyhccl.py", - "tests/e2e/singlecard/test_sampler.py", - "tests/e2e/singlecard/test_scheduler.py", - "tests/singlecard/test_ascend_config.py", - "tests/ut/fake_weight/config.json", - "tests/ut/test_ascend_config.py", - "tests/ut/worker/test_worker_v1.py", - "vllm_ascend/ascend_config.py", - "vllm_ascend/platform.py", - "vllm_ascend/utils.py" - ] - }, - { - "commit": "4270682383b4f4876296565bbb1227c1111b1d5e", - "subject": "Waiting for BMM NZ support(Improve TPOP 2ms performance) (#1131)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/attention/mla_v1.py" - ] - }, - { - "commit": "e72f94e38f4208d36a20df1ff85ad4abf430447d", - "subject": "Support multistream of MLA vector operations (#1135)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/user_guide/additional_config.md", - "tests/singlecard/test_ascend_config.py", - "vllm_ascend/ascend_config.py", - "vllm_ascend/attention/mla_v1.py", - "vllm_ascend/models/deepseek_v2.py" - ] - }, - { - "commit": "e46dc142bf1180453c64226d76854fc1ec696169", - "subject": "Enable kvcache_nz for the decode process in torchair graph mode (#1098)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "docs/source/user_guide/additional_config.md", - "vllm_ascend/ascend_config.py", - "vllm_ascend/attention/mla_v1.py" - ] - }, - { - "commit": "980cd814660791b1af7ab8b7e5d1c548359224e7", - "subject": "etp best a2 (#1101)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "examples/run_dp_attention_etp16.sh", - "examples/run_dp_attention_etp16_benmark.sh" - ] - }, - { - "commit": "7bdc606677705d072c1dc45f050a3c3471d6d379", - "subject": "Support multistream of shared experts in FusedMoE (#997)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/workflows/vllm_ascend_test.yaml", - "docs/source/user_guide/additional_config.md", - "mypy.ini", - "tests/multicard/test_offline_inference_distributed.py", - "tests/singlecard/test_ascend_config.py", - "vllm_ascend/ascend_config.py", - "vllm_ascend/models/deepseek_dbo.py", - "vllm_ascend/models/deepseek_v2.py", - "vllm_ascend/ops/fused_moe.py", - "vllm_ascend/quantization/w8a8_dynamic.py", - "vllm_ascend/utils.py" - ] - }, - { - "commit": "8dd686dfa20d8f577cd63fb751ce0a7a98f2f516", - "subject": "[MLA][Graph] Improve assertion on Graph mode with MLA (#933)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/faqs.md", - "vllm_ascend/attention/attention.py", - "vllm_ascend/attention/mla_v1.py", - "vllm_ascend/worker/multi_step_worker.py" - ] - }, - { - "commit": "95414bae7046eb4bf2323f5f21473c3b1cd158af", - "subject": "[CI] Run e2e after pre check pass (#1132)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/workflows/codespell.yml", - ".github/workflows/mypy.yaml", - ".github/workflows/ruff.yml", - ".github/workflows/vllm_ascend_test.yaml", - ".github/workflows/yapf.yml", - "vllm_ascend/patch/__init__.py" - ] - }, - { - "commit": "6b853f15fe69ba335d2745ebcf14a164d0bcc505", - "subject": "Add static EPLB (#1116)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "docs/source/user_guide/additional_config.md", - "vllm_ascend/ascend_config.py", - "vllm_ascend/ops/expert_load_balancer.py", - "vllm_ascend/ops/fused_moe.py", - "vllm_ascend/quantization/quant_config.py", - "vllm_ascend/quantization/w8a8_dynamic.py" - ] - }, - { - "commit": "f1543d5e0d71de4fc4a649c1d20d2d2ac2eb5d7e", - "subject": "[bugfix] fix deeepseek accuracy (#1118)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/models/deepseek_v2.py", - "vllm_ascend/ops/fused_moe.py", - "vllm_ascend/platform.py" - ] - }, - { - "commit": "87ebaef4e4e519988f27a6aa378f614642202ecf", - "subject": "[perf]: support dual-batch overlap(dbo) for deepseek (#941)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "examples/offline_dualbatch_overlap_npu.py", - "tests/multicard/test_offline_inference_distributed.py", - "vllm_ascend/attention/mla_v1.py", - "vllm_ascend/envs.py", - "vllm_ascend/models/__init__.py", - "vllm_ascend/models/deepseek_dbo.py", - "vllm_ascend/multistream/__init__.py", - "vllm_ascend/multistream/base.py", - "vllm_ascend/multistream/context.py", - "vllm_ascend/multistream/decorator.py", - "vllm_ascend/multistream/layers.py", - "vllm_ascend/multistream/metadata.py", - "vllm_ascend/multistream/ms_split.py", - "vllm_ascend/ops/fused_moe.py" - ] - }, - { - "commit": "3640c60b0eb4d4cb104e20bfa406d3f1d17920a7", - "subject": "Avoid unfused Transpose in DeepSeekV3 EP256 MoE layer (#1091)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "docs/source/user_guide/additional_config.md", - "vllm_ascend/ascend_config.py", - "vllm_ascend/worker/model_runner.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "e9ada685ece798f9fe0d4a287e3f5246a8a7207b", - "subject": "[CI]Moe alltoall communication optimization (#1067)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/envs.py", - "vllm_ascend/ops/fused_moe.py" - ] - }, - { - "commit": "0b12c2acf7d9fd192beebebf662298067d9a5435", - "subject": "[Kernel] Remove cumsum in groupedmatmul (#987)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/ops/fused_moe.py" - ] - }, - { - "commit": "c94afd79ceeb19424ed0d6da4c4fe82d29b5242b", - "subject": "[Doc] Update the description for env (#1079)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/envs.py" - ] - }, - { - "commit": "78431b34694dfa3c8f54ed7cc626660318557927", - "subject": "[perf]Support MOE Multi-stream in Deepseek (#947)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/singlecard/test_ascend_config.py", - "vllm_ascend/ascend_config.py", - "vllm_ascend/models/deepseek_v2.py", - "vllm_ascend/ops/fused_moe.py", - "vllm_ascend/quantization/quant_config.py", - "vllm_ascend/quantization/w8a8_dynamic.py" - ] - }, - { - "commit": "908a851a776cfd9051cc062119e6ec481561c6f7", - "subject": "optimize the funtion of computing topk and topp in sampler. (#970)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/multicard/test_offline_inference_distributed.py", - "tests/singlecard/test_offline_inference.py", - "tests/singlecard/test_sampler.py", - "vllm_ascend/envs.py", - "vllm_ascend/models/deepseek_v2.py", - "vllm_ascend/ops/fused_moe.py", - "vllm_ascend/patch/__init__.py", - "vllm_ascend/patch/worker/patch_common/__init__.py", - "vllm_ascend/patch/worker/patch_common/patch_sampler.py" - ] - }, - { - "commit": "e1ab6d318ee92c5b34ab3d7374bdd288fce0375f", - "subject": "[Misc] Refactor additional_config (#1029)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/vllm_ascend_test.yaml", - "docs/source/index.md", - "docs/source/user_guide/additional_config.md", - "examples/dp_offline/data_parallel.py", - "tests/long_term/spec_decode/e2e/conftest.py", - "tests/long_term/spec_decode/e2e/test_mtp_correctness.py", - "tests/multicard/test_dynamic_npugraph_batchsize.py", - "tests/multicard/test_offline_inference_distributed.py", - "tests/singlecard/test_aclgraph.py", - "tests/singlecard/test_ascend_config.py", - "vllm_ascend/ascend_config.py", - "vllm_ascend/attention/attention.py", - "vllm_ascend/attention/mla_v1.py", - "vllm_ascend/core/schedule_config.py", - "vllm_ascend/models/deepseek_v2.py", - "vllm_ascend/ops/fused_moe.py", - "vllm_ascend/platform.py", - "vllm_ascend/quantization/w8a8_dynamic.py", - "vllm_ascend/worker/cache_engine.py", - "vllm_ascend/worker/model_runner.py", - "vllm_ascend/worker/model_runner_v1.py", - "vllm_ascend/worker/worker.py", - "vllm_ascend/worker/worker_v1.py" - ] - }, - { - "commit": "afc4c0cd035ee22d51950d24b7f642abcaa0b176", - "subject": "[Bugfix] Fix deepseek percision issue and add acc ci for it (#905)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/vllm_ascend_test_long_term.yaml", - "tests/conftest.py", - "tests/long_term/test_deepseek_v2_lite_tp2_accuracy.py", - "tests/multicard/test_offline_inference_distributed.py", - "vllm_ascend/attention/attention.py", - "vllm_ascend/attention/attention_v1.py", - "vllm_ascend/attention/mla_v1.py", - "vllm_ascend/ops/fused_moe.py", - "vllm_ascend/quantization/w8a8_dynamic.py" - ] - }, - { - "commit": "da9acfca6053352730fce75fb772e214755d0341", - "subject": "feat: support data parallel for deepseek (#1012)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/attention/mla_v1.py", - "vllm_ascend/models/deepseek_v2.py", - "vllm_ascend/ops/fused_moe.py", - "vllm_ascend/platform.py", - "vllm_ascend/quantization/w8a8_dynamic.py", - "vllm_ascend/worker/model_runner_v1.py", - "vllm_ascend/worker/worker.py", - "vllm_ascend/worker/worker_v1.py" - ] - }, - { - "commit": "6ec64a3f9686df65b5a23a41aa301e669db19099", - "subject": "[bugfix] some bugs maybe fail to run (#896)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/long_term/spec_decode/e2e/test_v1_mtp_correctness.py", - "vllm_ascend/attention/attention_v1.py", - "vllm_ascend/distributed/parallel_state.py", - "vllm_ascend/ops/fused_moe.py", - "vllm_ascend/patch/platform/patch_common/patch_distributed.py", - "vllm_ascend/worker/mtp_proposer_v1.py", - "vllm_ascend/worker/worker_v1.py" - ] - }, - { - "commit": "f6e5decc109663f39e44e5fb792988b18a74fd53", - "subject": "[CI] upgrade to vllm 0.9.0 (#959)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/workflows/vllm_ascend_test.yaml", - ".github/workflows/vllm_ascend_test_long_term.yaml", - ".github/workflows/vllm_ascend_test_pd.yaml", - "Dockerfile", - "Dockerfile.openEuler", - "tests/long_term/spec_decode/e2e/conftest.py", - "vllm_ascend/attention/attention_v1.py", - "vllm_ascend/attention/mla_v1.py", - "vllm_ascend/ops/fused_moe.py", - "vllm_ascend/patch/__init__.py", - "vllm_ascend/patch/platform/__init__.py", - "vllm_ascend/patch/platform/patch_0_9_0/__init__.py", - "vllm_ascend/patch/worker/__init__.py", - "vllm_ascend/patch/worker/patch_0_9_0/__init__.py", - "vllm_ascend/worker/model_runner.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "e2a0c19cead28c06c6c4429e8bdbd3e6d5a54f65", - "subject": "[CI] Refactor CI (#952)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/vllm_ascend_test.yaml", - ".github/workflows/vllm_ascend_test_long_term.yaml", - ".github/workflows/vllm_ascend_test_pd.yaml", - "format.sh", - "tests/long_term/spec_decode/__init__.py", - "tests/long_term/spec_decode/conftest.py", - "tests/long_term/spec_decode/e2e/__init__.py", - "tests/long_term/spec_decode/e2e/conftest.py", - "tests/long_term/spec_decode/e2e/test_medusa_correctness.py", - "tests/long_term/spec_decode/e2e/test_mlp_correctness.py", - "tests/long_term/spec_decode/e2e/test_mtp_correctness.py", - "tests/long_term/spec_decode/e2e/test_ngram_correctness.py", - "tests/long_term/spec_decode/e2e/test_v1_spec_decode.py", - "tests/long_term/spec_decode/test_dynamic_spec_decode.py", - "tests/long_term/spec_decode/test_multi_step_worker.py", - "tests/long_term/spec_decode/test_ngram_worker.py", - "tests/long_term/spec_decode/test_spec_decode_worker.py", - "tests/long_term/spec_decode/test_utils.py", - "tests/long_term/spec_decode/utils.py", - "tests/long_term/test_accuracy.py", - "tests/model_utils.py", - "tests/multicard/test_offline_inference_distributed.py", - "tests/scheduler/test_scheduler.py", - "tests/singlecard/compile/__init__.py", - "tests/singlecard/compile/test_simple.py", - "tests/singlecard/ops/__init__.py", - "tests/singlecard/ops/test_fused_moe.py", - "tests/singlecard/ops/test_multi_step.py", - "tests/singlecard/ops/test_rotary_embedding.py", - "tests/singlecard/sample/__init__.py", - "tests/singlecard/sample/test_rejection_sampler.py", - "tests/singlecard/test_camem.py", - "tests/singlecard/test_offline_inference.py", - "tests/utils.py" - ] - }, - { - "commit": "9f5ab59e307a66fd0b17916218c9387c328b1c59", - "subject": "[WIP][BugFix]Fix accuracy issues caused by wrong etp_size passed into FusedMoEParallelConfig when using vLLM 0.9.0 (#961)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/ops/fused_moe.py" - ] - }, - { - "commit": "1f9fb869ad8832fefed92ae6ddaf36552d694c89", - "subject": "[BugFix] Fix accuracy bugs for unquantized deepseekv3 models (#897)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/ops/fused_moe.py", - "vllm_ascend/quantization/quant_config.py", - "vllm_ascend/quantization/w8a8_dynamic.py" - ] - }, - { - "commit": "df58fb80eee24139fc61c495be3ce79cf81b3f73", - "subject": "Spec decode support for V1 Engine (#874)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/workflows/vllm_ascend_test.yaml", - "requirements-dev.txt", - "tests/sample/__init__.py", - "tests/sample/test_rejection_sampler.py", - "tests/singlecard/spec_decode/e2e/test_v1_spec_decode.py", - "vllm_ascend/attention/attention_v1.py", - "vllm_ascend/patch/__init__.py", - "vllm_ascend/patch/worker/patch_common/__init__.py", - "vllm_ascend/patch/worker/patch_common/patch_eagle.py", - "vllm_ascend/sample/__init__.py", - "vllm_ascend/sample/rejection_sampler.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "7a325b2e2d1001a6341ba71eebd8bd8d4458d12b", - "subject": "[Bugfix][Model] Fix fusedmoe and make modelrunner_v1 compatible with latest vllm (#867)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/attention/attention_v1.py", - "vllm_ascend/attention/mla_v1.py", - "vllm_ascend/ops/fused_moe.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "1e67089bc970bfe2f667f321fd0c0777a0c55a26", - "subject": "[BugFix]add all2all when dp_size > 1 && downgrade npu_dequant_swiglu_quant (#819)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/distributed/communicator.py", - "vllm_ascend/models/deepseek_v2.py", - "vllm_ascend/ops/fused_moe.py", - "vllm_ascend/patch/worker/patch_common/__init__.py", - "vllm_ascend/patch/worker/patch_common/patch_distributed.py", - "vllm_ascend/quantization/quant_config.py", - "vllm_ascend/quantization/w8a8_dynamic.py" - ] - }, - { - "commit": "68fb63428b8b972dd60ad9189538909b0eb1fcc8", - "subject": "[CI] Patch torch.library.infer_schema for fused moe ops to fix CI (#854)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/ops/test_fused_moe.py", - "vllm_ascend/__init__.py", - "vllm_ascend/quantization/quant_config.py" - ] - }, - { - "commit": "e56447033889ca95df512208cab22ef832bfdf07", - "subject": "[Attention][Kernel]moe support for llama4 and mllama4 (#740)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/attention/attention.py", - "vllm_ascend/attention/attention_v1.py", - "vllm_ascend/ops/common_fused_moe.py", - "vllm_ascend/ops/fused_moe.py" - ] - }, - { - "commit": "c6ac399091f927dd743267ed7cebbba957e1a92f", - "subject": "[Bugfix] Fix the method of importing environment variables in DeepSee\u2026 (#817)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/envs.py", - "vllm_ascend/models/deepseek_v2.py", - "vllm_ascend/ops/fused_moe.py" - ] - }, - { - "commit": "efabd722eb757e49aa309c173bbec91ca8c4ced1", - "subject": "feat: support torchair graph mode in v1 engine (#789)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/attention/mla_v1.py", - "vllm_ascend/core/scheduler.py", - "vllm_ascend/models/deepseek_v2.py", - "vllm_ascend/platform.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "324f819b929ae2c30fe629e27df0c37c2cc607b7", - "subject": "[Perf] Optimize fused_experts quantization code to save npu memory (#784)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/models/deepseek_v2.py", - "vllm_ascend/quantization/w8a8_dynamic.py" - ] - }, - { - "commit": "84e2ed898b7edec4f358d0809a2270dfe7bacfd5", - "subject": "performance optimization, usability optimization and API compatibility adjustments for deepseek with npu graph mode (#731)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/attention/attention.py", - "vllm_ascend/models/deepseek_v2.py", - "vllm_ascend/ops/rotary_embedding.py", - "vllm_ascend/platform.py", - "vllm_ascend/quantization/w8a8_dynamic.py", - "vllm_ascend/worker/model_runner.py" - ] - }, - { - "commit": "f8350569e6267675861a8e8e4b975400268e7d2b", - "subject": "[CI] upgrade vllm to 0.8.5 (#715)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/workflows/vllm_ascend_test.yaml", - "Dockerfile", - "Dockerfile.openEuler", - "vllm_ascend/__init__.py", - "vllm_ascend/patch/__init__.py", - "vllm_ascend/patch/platform/__init__.py", - "vllm_ascend/patch/platform/patch_0_8_4/patch_config.py", - "vllm_ascend/patch/platform/patch_0_8_5/__init__.py", - "vllm_ascend/patch/worker/__init__.py", - "vllm_ascend/patch/worker/patch_0_8_4/patch_metrics.py", - "vllm_ascend/patch/worker/patch_0_8_4/patch_spec_decode_worker.py", - "vllm_ascend/patch/worker/patch_0_8_4/patch_tritonplaceholder.py", - "vllm_ascend/patch/worker/patch_0_8_5/__init__.py", - "vllm_ascend/patch/worker/patch_common/patch_multi_step_worker.py", - "vllm_ascend/worker/draft_model_runner.py", - "vllm_ascend/worker/model_runner.py", - "vllm_ascend/worker/model_runner_v1.py", - "vllm_ascend/worker/multi_step_runner.py", - "vllm_ascend/worker/worker.py", - "vllm_ascend/worker/worker_v1.py" - ] - }, - { - "commit": "0dae55a9a3deebdb4f2263011154d886c525fc13", - "subject": "[MISC] fix format check error (#654)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "csrc/kernels/pos_encoding_kernels.cpp", - "docs/source/faqs.md", - "docs/source/installation.md", - "examples/disaggregated_prefill_hccl.py", - "examples/offline_inference_audio_language.py", - "format.sh", - "tests/singlecard/spec_decode/e2e/test_medusa_correctness.py", - "tests/singlecard/spec_decode/e2e/test_mlp_correctness.py", - "tests/singlecard/spec_decode/e2e/test_ngram_correctness.py", - "tools/actionlint.sh", - "tools/mypy.sh", - "vllm_ascend/attention/attention.py", - "vllm_ascend/core/schedule_config.py", - "vllm_ascend/distributed/llmdatadist_connector.py", - "vllm_ascend/distributed/parallel_state.py", - "vllm_ascend/ops/fused_moe.py", - "vllm_ascend/ops/rotary_embedding.py" - ] - }, - { - "commit": "1fce70a2fb2602170781773104a69c69beb16161", - "subject": "[Model] Support common fused moe ops for moe model, such as Qwen3Moe (#709)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/ops/__init__.py", - "vllm_ascend/ops/common_fused_moe.py" - ] - }, - { - "commit": "40bd6024856b340dfb0ad80f101d96f670c72991", - "subject": "[Feature] Use reshape_and_cache fused op (#706)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/attention/mla_v1.py" - ] - }, - { - "commit": "54c0e63df7dc513a2c2316e2ccd6ed1dfdefdff3", - "subject": "[MTP] follow custom deepseek modeling changes to support graph mode (#636)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/workflows/vllm_ascend_test.yaml", - "tests/singlecard/spec_decode/__init__.py", - "tests/singlecard/spec_decode/e2e/conftest.py", - "tests/singlecard/spec_decode/e2e/test_mtp_correctness.py", - "tests/singlecard/spec_decode/test_dynamic_spec_decode.py", - "tests/singlecard/spec_decode/test_multi_step_worker.py", - "tests/singlecard/spec_decode/test_ngram_worker.py", - "tests/singlecard/spec_decode/test_spec_decode_worker.py", - "vllm_ascend/models/deepseek_mtp.py", - "vllm_ascend/patch/__init__.py", - "vllm_ascend/patch/worker/patch_0_8_4/patch_spec_decode_worker.py", - "vllm_ascend/patch/worker/patch_common/patch_multi_step_worker.py", - "vllm_ascend/patch/worker/patch_common/patch_spec_decode_worker.py", - "vllm_ascend/worker/draft_model_runner.py", - "vllm_ascend/worker/multi_step_runner.py" - ] - }, - { - "commit": "38f34e359f08bcf652d2a95b2a2521880286d710", - "subject": "[Fix] fix deepseek v0 attention eager mode (#671)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/attention/attention.py" - ] - }, - { - "commit": "ba3d8aae943271221ff7b1ae74551939fd935ee5", - "subject": "[Model][MiniCPM] support MiniCPM (#645)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/patch/__init__.py", - "vllm_ascend/patch/worker/patch_0_8_4/__init__.py", - "vllm_ascend/patch/worker/patch_0_8_4/patch_tritonplaceholder.py", - "vllm_ascend/patch/worker/patch_common/__init__.py", - "vllm_ascend/patch/worker/patch_common/patch_minicpm.py" - ] - }, - { - "commit": "d785e785639a0ebcf21c0b5e46ab47a3b041344c", - "subject": "[V1] Make V1 engine backward compatible (#637)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "tests/multicard/test_offline_inference_distributed.py", - "tests/ops/test_fused_moe.py", - "tests/singlecard/test_offline_inference.py", - "vllm_ascend/platform.py" - ] - }, - { - "commit": "05bdcbeae47c7fcb9b1c30cad059abf1d40b5421", - "subject": "support aclgraph (#426)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/workflows/vllm_ascend_test.yaml", - "csrc/ops.h", - "csrc/torch_binding.cpp", - "pyproject.toml", - "requirements-dev.txt", - "requirements.txt", - "tests/compile/__init__.py", - "tests/compile/test_simple.py", - "tests/multicard/test_offline_inference_distributed.py", - "tests/singlecard/test_offline_inference.py", - "vllm_ascend/attention/attention_v1.py", - "vllm_ascend/ops/__init__.py", - "vllm_ascend/platform.py", - "vllm_ascend/worker/model_runner_v1.py", - "vllm_ascend/worker/worker_v1.py" - ] - }, - { - "commit": "5c6d05a59e996ab0ce6b91e7d4e267d7be1157f8", - "subject": "support deepseek quant & mix-parallel with graphmode (#585)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "examples/dp_offline/data_parallel.py", - "examples/dp_offline/run_dp.sh", - "vllm_ascend/attention/attention.py", - "vllm_ascend/models/deepseek_v2.py", - "vllm_ascend/ops/fused_moe.py", - "vllm_ascend/platform.py", - "vllm_ascend/quantization/quant_config.py", - "vllm_ascend/quantization/w8a8.py", - "vllm_ascend/quantization/w8a8_dynamic.py", - "vllm_ascend/utils.py", - "vllm_ascend/worker/cache_engine.py", - "vllm_ascend/worker/model_runner.py", - "vllm_ascend/worker/worker.py" - ] - }, - { - "commit": "538a69c1459cc8fde032b8db211ea215c78063b9", - "subject": "[Patch] format patch module to make it more clear (#601)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/patch/__init__.py", - "vllm_ascend/patch/platform/patch_0_8_4/__init__.py", - "vllm_ascend/patch/platform/patch_common/__init__.py", - "vllm_ascend/patch/platform/patch_common/patch_distributed.py", - "vllm_ascend/patch/platform/patch_main/__init__.py", - "vllm_ascend/patch/platform/patch_main/patch_distributed.py", - "vllm_ascend/patch/worker/patch_common/__init__.py" - ] - }, - { - "commit": "d12a057df850f2664f4388397646ec9c32beb88e", - "subject": "Add note for deepseek related docs and remove unnecessary comments (#590)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/models/deepseek_v2.py", - "vllm_ascend/ops/fused_moe.py", - "vllm_ascend/ops/rotary_embedding.py", - "vllm_ascend/patch/platform/patch_0_8_4/patch_distributed.py", - "vllm_ascend/patch/platform/patch_main/patch_distributed.py", - "vllm_ascend/worker/model_runner_v1.py" - ] - }, - { - "commit": "0ae9ee0f8a0fb8f8c20dd5005932a68550a15f89", - "subject": "[BUGFIX] main-sd-bugfix && [UT] add mtp UT (#593)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/workflows/vllm_ascend_test.yaml", - "tests/singlecard/spec_decode/e2e/test_mtp_correctness.py", - "vllm_ascend/attention/attention.py", - "vllm_ascend/attention/mla_v1.py", - "vllm_ascend/core/scheduler.py", - "vllm_ascend/distributed/parallel_state.py", - "vllm_ascend/models/deepseek_mtp.py", - "vllm_ascend/patch/worker/patch_common/patch_spec_decode_worker.py", - "vllm_ascend/quantization/quant_config.py", - "vllm_ascend/worker/draft_model_runner.py" - ] - }, - { - "commit": "5442b463fd7b232fc560dd549e50b737043992cb", - "subject": "add doc for patch_config (#574)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/patch/platform/patch_0_8_4/__init__.py" - ] - }, - { - "commit": "12cae04db9ebe6cd70dede409c63ed5d326537fb", - "subject": "[quantization] Support w8a8 quantization (#580)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "docs/source/installation.md", - "tests/singlecard/test_offline_inference.py", - "vllm_ascend/quantization/func_wrapper.py", - "vllm_ascend/quantization/quant_config.py", - "vllm_ascend/quantization/quantizer.py", - "vllm_ascend/quantization/w8a8.py", - "vllm_ascend/quantization/w8a8_dynamic.py" - ] - }, - { - "commit": "1a1f9a6d894a3947fcff4c5c52fa0846c35e5759", - "subject": "port deepseekv2 and mtp to main branch (#429)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "examples/disaggregated_prefill_hccl.py", - "examples/dp_offline/data_parallel.py", - "examples/dp_offline/run_dp.sh", - "examples/offline_inference_npu_v1.py", - "tests/conftest.py", - "vllm_ascend/attention/attention.py", - "vllm_ascend/attention/attention_v1.py", - "vllm_ascend/attention/mla_v1.py", - "vllm_ascend/distributed/llmdatadist_connector.py", - "vllm_ascend/distributed/parallel_state.py", - "vllm_ascend/models/__init__.py", - "vllm_ascend/models/deepseek_mtp.py", - "vllm_ascend/models/deepseek_v2.py", - "vllm_ascend/ops/__init__.py", - "vllm_ascend/ops/attention.py", - "vllm_ascend/ops/cache.py", - "vllm_ascend/ops/fused_moe.py", - "vllm_ascend/ops/rotary_embedding.py", - "vllm_ascend/ops/vocab_parallel_embedding.py", - "vllm_ascend/patch/__init__.py", - "vllm_ascend/patch/platform/patch_0_8_4/__init__.py", - "vllm_ascend/patch/platform/patch_0_8_4/patch_distributed.py", - "vllm_ascend/patch/platform/patch_main/__init__.py", - "vllm_ascend/patch/platform/patch_main/patch_distributed.py", - "vllm_ascend/platform.py", - "vllm_ascend/quantization/quant_config.py", - "vllm_ascend/utils.py", - "vllm_ascend/worker/__init__.py", - "vllm_ascend/worker/cache_engine.py", - "vllm_ascend/worker/model_runner.py", - "vllm_ascend/worker/model_runner_v1.py", - "vllm_ascend/worker/worker.py", - "vllm_ascend/worker/worker_v1.py" - ] - }, - { - "commit": "6ee7f5cf711509349fc06f7871659174395d1801", - "subject": "[SpecDecode] Add spec decode support (#500)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/workflows/vllm_ascend_test.yaml", - "requirements-dev.txt", - "tests/__init__.py", - "tests/multicard/test_offline_inference_distributed.py", - "tests/ops/__init__.py", - "tests/singlecard/__init__.py", - "tests/singlecard/spec_decode/__init__.py", - "tests/singlecard/spec_decode/conftest.py", - "tests/singlecard/spec_decode/e2e/__init__.py", - "tests/singlecard/spec_decode/e2e/conftest.py", - "tests/singlecard/spec_decode/e2e/test_medusa_correctness.py", - "tests/singlecard/spec_decode/e2e/test_mlp_correctness.py", - "tests/singlecard/spec_decode/e2e/test_ngram_correctness.py", - "tests/singlecard/spec_decode/test_dynamic_spec_decode.py", - "tests/singlecard/spec_decode/test_multi_step_worker.py", - "tests/singlecard/spec_decode/test_ngram_worker.py", - "tests/singlecard/spec_decode/test_spec_decode_worker.py", - "tests/singlecard/spec_decode/test_utils.py", - "tests/singlecard/spec_decode/utils.py", - "tests/singlecard/test_offline_inference.py", - "tests/utils.py", - "vllm_ascend/attention/attention.py", - "vllm_ascend/patch/worker/patch_common/__init__.py", - "vllm_ascend/patch/worker/patch_common/patch_metrics.py", - "vllm_ascend/patch/worker/patch_common/patch_multi_step_worker.py", - "vllm_ascend/patch/worker/patch_common/patch_spec_decode_worker.py", - "vllm_ascend/worker/draft_model_runner.py" - ] - }, - { - "commit": "20dff4deffc90dfcb96472de5ae36737e78cba96", - "subject": "[Scheduler] Add AscendScheduler. (#543)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/scheduler/test_scheduler.py", - "vllm_ascend/attention/attention.py", - "vllm_ascend/attention/attention_v1.py", - "vllm_ascend/core/__init__.py", - "vllm_ascend/core/schedule_config.py", - "vllm_ascend/core/scheduler.py", - "vllm_ascend/platform.py", - "vllm_ascend/worker/model_runner_v1.py", - "vllm_ascend/worker/worker_v1.py" - ] - }, - { - "commit": "9935d457289ae85c0bf3ffbe5496875c6eb34782", - "subject": "[CI]Add model basic accuracy test(Qwen2.5-0.5B-Instruct) (#460)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/actionlint.yml", - ".github/workflows/mypy.yaml", - ".github/workflows/ruff.yml", - ".github/workflows/shellcheck.yml", - ".github/workflows/vllm_ascend_test.yaml", - ".github/workflows/yapf.yml", - "Dockerfile", - "Dockerfile.openEuler", - "collect_env.py", - "docs/source/conf.py", - "examples/offline_distributed_inference_npu.py", - "examples/offline_inference_audio_language.py", - "examples/offline_inference_npu.py", - "format.sh", - "mypy.ini", - "requirements-dev.txt", - "setup.py", - "tests/conftest.py", - "tests/model_utils.py", - "tests/multicard/test_offline_inference_distributed.py", - "tests/ops/test_fused_moe.py", - "tests/ops/test_rotary_embedding.py", - "tests/singlecard/test_accuracy.py", - "tests/singlecard/test_offline_inference.py", - "tools/actionlint.sh", - "tools/check_repo.sh", - "tools/mypy.sh", - "tools/png-lint.sh", - "tools/shellcheck.sh", - "tools/sphinx-lint.sh", - "vllm_ascend/__init__.py", - "vllm_ascend/attention/attention.py", - "vllm_ascend/attention/attention_v1.py", - "vllm_ascend/distributed/communicator.py", - "vllm_ascend/models/deepseek_v2.py", - "vllm_ascend/models/qwen2_vl.py", - "vllm_ascend/ops/__init__.py", - "vllm_ascend/ops/activation.py", - "vllm_ascend/ops/fused_moe.py", - "vllm_ascend/ops/layernorm.py", - "vllm_ascend/ops/rotary_embedding.py", - "vllm_ascend/platform.py", - "vllm_ascend/quantization/quant_config.py", - "vllm_ascend/utils.py", - "vllm_ascend/worker/model_runner.py", - "vllm_ascend/worker/model_runner_v1.py", - "vllm_ascend/worker/pooling_model_runner.py", - "vllm_ascend/worker/worker.py", - "vllm_ascend/worker/worker_v1.py" - ] - }, - { - "commit": "f6cf92e7d55004bf8eb8729d71acaf0635f88883", - "subject": "[quant][bugfix] fix deepseek quant bug (#478)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/models/deepseek_v2.py", - "vllm_ascend/quantization/quant_config.py" - ] - }, - { - "commit": "344228a5da6130bf1d7e5f3c3d04cc8b661748ff", - "subject": "[deepseek][bugfix] support deepseek quant (#469)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "vllm_ascend/models/__init__.py", - "vllm_ascend/models/deepseek_v2.py", - "vllm_ascend/quantization/quant_config.py" - ] - }, - { - "commit": "31f29b9f30eb65f0a38d30ffe613282ce2f1130a", - "subject": "[Core] Make V1 work and enable V1 engine test (#389)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/vllm_ascend_test_main.yaml", - "collect_env.py", - "tests/ops/test_fused_moe.py", - "tests/test_offline_inference.py", - "vllm_ascend/attention/__init__.py", - "vllm_ascend/platform.py", - "vllm_ascend/worker/model_runner_v1.py", - "vllm_ascend/worker/worker_v1.py" - ] - }, - { - "commit": "c06af8b2e0f4ace8caf20f4cf4fdbb1978647df6", - "subject": "[V1][Core] Add support for V1 Engine (#295)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "README.md", - "docs/source/installation.md", - "docs/source/tutorials/multi_node.md", - "docs/source/user_guide/release_notes.md", - "docs/source/user_guide/suppoted_features.md", - "format.sh", - "test.py", - "tools/actionlint.sh", - "vllm_ascend/attention/attention.py", - "vllm_ascend/attention/attention_v1.py", - "vllm_ascend/platform.py", - "vllm_ascend/worker/model_runner_v1.py", - "vllm_ascend/worker/worker_v1.py" - ] - }, - { - "commit": "663dca7578b9f6573a7701e257b45b4d9a700a3d", - "subject": "[CI] fix race condition problem (#353)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - ".github/workflows/actionlint.yml", - ".github/workflows/mypy.yaml", - ".github/workflows/ruff.yml", - ".github/workflows/shellcheck.yml", - ".github/workflows/vllm_ascend_test.yaml", - ".github/workflows/yapf.yml", - "tests/ops/test_fused_moe.py" - ] - }, - { - "commit": "0db6670bfab8cb1d84c9e7270df0a1d42d6ce7ca", - "subject": "[Feature] Implement EP-compatible fused_moe (#121)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - "tests/ops/test_fused_moe.py", - "vllm_ascend/ops/fused_moe.py" - ] - }, - { - "commit": "dcd0005058dbd6fd8672378565890cbda924b792", - "subject": "[Fix] Remove npu_group_topk before CANN version update (#242)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/ops/fused_moe.py" - ] - }, - { - "commit": "1715230867048aaf3102dbe6448b3c476db74c9e", - "subject": "[CI] Upgrade to newest pta.(MLA and FusedMoE) (#189)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/attention.py", - "vllm_ascend/ops/fused_moe.py" - ] - }, - { - "commit": "fd18ae649453fa4c31b58b04ba75d3fd9ed0b3d4", - "subject": "[MOE] fix #176 (#179)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/ops/fused_moe.py" - ] - }, - { - "commit": "d0b3cb4fa79d5fc7f8245a3c68885ce1fa030ba4", - "subject": "modify:Eliminate redundant operations in the code to improve performance (#137)", - "topics": [ - "fusion" - ], - "runtime_patch_match": false, - "paths": [ - "vllm_ascend/attention.py", - "vllm_ascend/model_runner.py", - "vllm_ascend/ops/fused_moe.py", - "vllm_ascend/ops/rotary_embedding.py" - ] - }, - { - "commit": "5f465010deef1a2b507a107e720c1c366161d820", - "subject": "[Core] Cherry pick from 0.7.1 to keep the main code newest (#127)", - "topics": [ - "fusion" - ], - "runtime_patch_match": true, - "paths": [ - ".github/workflows/vllm_ascend_test.yaml", - "vllm_ascend/attention.py", - "vllm_ascend/model_runner.py", - "vllm_ascend/ops/__init__.py", - "vllm_ascend/ops/activation.py", - "vllm_ascend/ops/fused_moe.py", - "vllm_ascend/ops/rotary_embedding.py", - "vllm_ascend/platform.py", - "vllm_ascend/quantization/__init__.py", - "vllm_ascend/quantization/quant_config.py", - "vllm_ascend/quantization/quantizer.py" - ] - } - ] -} diff --git a/evidence/README.md b/evidence/README.md index edbf2fc45..68afd9eea 100644 --- a/evidence/README.md +++ b/evidence/README.md @@ -5,7 +5,7 @@ First-class paths for every campaign (see [contribution guide](../docs/contribut | Segment | Purpose | | --- | --- | | `plan/` | Hypothesis, RunSpec snapshot, gate checklist | -| `results/` | Metrics, logs, verdict rationale | +| `results/` | Key metrics, complete trial index, archive references, verdict rationale | | `patch/` | Candidate diff or revision pointer | | `validator/` | Frozen validator definition + inputs | @@ -20,7 +20,13 @@ evidence/ validator/ ``` -Populate per acceptance issue. This scaffold keeps only `.gitkeep` placeholders—no campaign artifacts or pinned SHAs in the template. +Populate per acceptance issue using [the bundle template](_template/README.md) +and [archiving contract](../docs/evidence-archiving.md). Large raw records, logs, +profiler output and generated inventories belong in external versioned archives. +The template has placeholders only, with no actual campaign pins or results. +`results/artifacts.json` is the archive authority; the repository inventory +references it. Validate metadata offline with `tools/validate_evidence.py` and +explicitly verify archive content with `tools/verify_evidence_archive.py`. When archiving a round that studied knowledge retrieval or adoption, copy `search/archive/round-.json` from the local campaign directory to diff --git a/evidence/_template/README.md b/evidence/_template/README.md new file mode 100644 index 000000000..1b6ceeed2 --- /dev/null +++ b/evidence/_template/README.md @@ -0,0 +1,21 @@ +# + +Copy this directory for a new campaign, then replace all placeholders. The +`*.example.json` files describe fields; they are not runnable evidence. +Follow the [archiving contract](../../docs/evidence-archiving.md). + +- Acceptance issue and owner: , +- Goal, compared variants and execution conditions: +- Source/configuration/input/validator identities: [plan/](plan/) +- Conclusion and limitations: +- Performance and quality: [results/summary.json](results/summary.json) +- Every attempted trial: [results/trials.jsonl](results/trials.jsonl) +- Archive authority: [results/artifacts.json](results/artifacts.json) +- Verification commands, identity pins and comparison rules: [validator/](validator/) + +State sample/failure counts, repetitions and variability, not only best results. +Generate the trial index from the existing controller or frozen source records; +retain failed, regressing, interrupted and repeated attempts with distinct IDs. +Do not create a second controller state. Explain unavailable/private originals +and the resulting verification limits. Archive availability is separate from +the historical experiment verdict. diff --git a/evidence/_template/plan/storage-exception.md b/evidence/_template/plan/storage-exception.md new file mode 100644 index 000000000..5ab05d53f --- /dev/null +++ b/evidence/_template/plan/storage-exception.md @@ -0,0 +1,17 @@ +# Repository storage exception: + +Fill only when an oversized artifact must remain in Git; otherwise remove this +unused template. Approval applies to the named artifact, not future additions. + +| Required information | Record | +| --- | --- | +| Artifact path and exact identity | | +| Bytes and affected reporting threshold | | +| Purpose and concrete consumers | | +| Obstacle to external storage | | +| Maintenance and retention owner | | +| Explicit maintainer agreement | | +| Review/expiry condition | | + +R6 remains report-only. This record grants no experiment acceptance and does +not modify frozen data or authorize rewriting repository history. diff --git a/evidence/_template/results/README.md b/evidence/_template/results/README.md index e57d85117..64f8bb6ed 100644 --- a/evidence/_template/results/README.md +++ b/evidence/_template/results/README.md @@ -1,11 +1,25 @@ # results/ (template) -- Metrics, logs, profiler exports -- Verdict rationale: `PASS`, `FAIL`, or `INCONCLUSIVE` -- Links to baseline vs candidate comparison artifacts +- `summary.json`: compared variants, metric definitions, conditions, sample and + failure counts, repetitions, variability, performance/quality and limitations. +- `trials.jsonl`: one row per attempted trial, including failures and interruptions. +- `artifacts.json`: authoritative versioned archive metadata, based on + [artifacts.example.json](artifacts.example.json). +- Verdict rationale: `PASS`, `FAIL`, or `INCONCLUSIVE`. +- Raw requests, logs, profiler exports and media belong in the external archive. + +Example trial row (replace all placeholders; regenerate it from control records): + +```json +{"id":"","configuration":{},"status":"interrupted","conclusion":"INCONCLUSIVE: ","artifact_id":"","source_record":"control/trials.jsonl#"} +``` + +`source_record` identifies the original control record, not just a configuration +name. Repeated configurations remain separate trials. Compare the regenerated +index and summary using the [verification procedure](../validator/README.md). A campaign-level figure of every trial's gain comes from -[tools/campaign_summary_chart.py](../../tools/campaign_summary_chart.py). It also +[tools/campaign_summary_chart.py](../../../tools/campaign_summary_chart.py). It also renders automatically into the experiment report, and reads a committed `summary-chart.json` beside the run when a campaign has several steps: @@ -17,4 +31,4 @@ renders automatically into the experiment report, and reads a committed Copy the shape that matches the campaign, replace the values with measured results, and commit the specification with the figure. See the -[campaign summary chart guide](../../docs/campaign-summary-chart.md). +[campaign summary chart guide](../../../docs/campaign-summary-chart.md). diff --git a/evidence/_template/results/artifacts.example.json b/evidence/_template/results/artifacts.example.json new file mode 100644 index 000000000..1834bee3e --- /dev/null +++ b/evidence/_template/results/artifacts.example.json @@ -0,0 +1,35 @@ +{ + "schema_version": 2, + "archive": { + "id": "-v1", + "file": "-v1.tar.gz", + "url": "https://github.com///releases/download//-v1.tar.gz", + "sha256": "", + "bytes": "", + "format": "tar+gzip", + "retention_owner": "", + "source_revision": "", + "root": "", + "files": "", + "uncompressed_bytes": "", + "key_files": { + "control": "/control/trials.jsonl", + "configuration": "/plan/effective.json", + "validator": "/validator/recompute.py" + } + }, + "verification": { + "summary": "results/summary.json", + "trials": "results/trials.jsonl", + "command": ["python", "validator/recompute.py", "--archive", "", "--output", ""], + "inputs": [ + {"role": "control", "path": "/control/trials.jsonl", "sha256": ""}, + {"role": "configuration", "path": "/plan/effective.json", "sha256": ""}, + {"role": "validator", "path": "/validator/recompute.py", "sha256": ""} + ], + "absolute_tolerance": 0, + "relative_tolerance": 0 + }, + "availability_limits": [""], + "retention_policy": "" +} diff --git a/evidence/_template/validator/README.md b/evidence/_template/validator/README.md index 88068d778..ba014552e 100644 --- a/evidence/_template/validator/README.md +++ b/evidence/_template/validator/README.md @@ -2,3 +2,30 @@ - Frozen validator definition and inputs (version-pinned) - Must be fixed **before** campaign runs; missing freeze → `INCONCLUSIVE` + +## Archive replay contract + +Keep a reviewed recomputation entry point here or reference its immutable source +revision. Declare its command, input identities and numerical comparison rules +in `results/artifacts.json` (v2). The generic verifier never executes that command. + +The entry point must independently enumerate every original trial from the +archived control records, including failed/interrupted and repeated attempts. +It writes `results/trials.jsonl` and `results/summary.json` to a new output +directory. Do not read retained summaries/indexes as inputs to regeneration. +Document each metric's source rows, formula, sample/failure counts and validator +identity. Review this derivation before accepting completeness. + +1. Run the shared verifier with `--download` or a locally retrieved `--archive`. +2. Inspect and run the declared, identity-pinned replay command separately. +3. Run the shared verifier with the same archive and `--recomputed `. + +Structure, trial IDs, statuses, strings, integers and booleans compare exactly. +Only finite JSON floats use the predeclared absolute/relative tolerances. Freeze +those tolerances before archiving; never adjust them just to make replay pass. +Matching outputs do not prove the replay algorithm enumerated all control +records; the campaign review and adversarial coverage tests establish that. + +Record commands, input digests, completeness checks, comparison results and gaps. +Download/hash success alone does not establish experiment reproducibility or +campaign acceptance. See [the shared tool](../../../tools/verify_evidence_archive.py). diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/README.md b/evidence/h3-ref2va-910c-4logical-20261101-297/README.md index 54b3f6c4d..b459b0890 100644 --- a/evidence/h3-ref2va-910c-4logical-20261101-297/README.md +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/README.md @@ -1,50 +1,33 @@ -# MiniMax-H3 Ref2VA: 910C four-logical-device campaign - -Governing acceptance issue: [InferRSI #297](https://github.com/InferMatrixLab/InferRSI/issues/297). -Accountable owner: the requester's Codex H3 session, operating under Dong1017. -Claim and plan: [owner receipt](https://github.com/InferMatrixLab/InferRSI/issues/297#issuecomment-6555458151). -Final campaign acceptance remains with the requester. - -**Status: Draft tracking snapshot; campaign verdict INCONCLUSIVE.** -No accepted 50% complete-HTTP latency reduction exists. All four final tracks -remain incomplete. C1 passed 439 strict checks on each fixed 5 s/15 s request; -one clean 5 s screening pair measured 291.248→286.667 s, a 1.5729% reduction, -below the unchanged 50% bar. This is not final confirmation. - -C0–C23 cover 24 charged direction families. The tested fast lossy profiles have -observed frozen quality failures. Most recently, C21/C22/C23 completed 5s-00 -but failed seven/nine/eight quality checks. C23 V2 passed 44 native tests and -reduced a static 50-call attention cycle by 66.46%; its full 5 s diagnostic HTTP -was 199.065 s. These observations do not qualify model quality or HTTP gains. -No 15 s or clean timing extension follows for those rejected profiles. - -Start with the [experiment report and 24-direction ledger](results/report.md), -[four-track execution snapshot](plan/execution-checkpoint.json), and -[replay/availability notes](plan/reproduction.md). The inventory's publication -hash map verifies available files; it does not qualify device or model behavior. - -Downstream code is tracked in [vLLM-Omni Draft PR #9745](https://github.com/vllm-project/vllm-omni/pull/9745). -It ports only C1 to main, preserving newer piecewise/query-range dispatch, and -passed [12 source-bound native tests](results/c1-upstream-main-native.json). -The historical full-model C1 results remain tied to `be319692`; full-model -quality and clean HTTP have not been rerun on the new upstream base. - -The scope is 5s/15s × lossless/lossy, 1344×768 at 24 FPS, 124/362 frames, -original BF16 dense Ref2VA, 50 steps, four logical 910C devices. This is separate -from the eight-device 910B1 campaign in #294 and the original eight-device requirement. - -- `plan/`: execution identity, original budget, protocol and frozen contract bytes. -- `results/`: scoped numerical results, rejected operator variants and automatic quality comparisons. -- `patch/`: tested private C1 and C10–C23 identities and their exact engine patches. -- `validator/`: byte-preserved validator/scorer snapshots and checksums. - -The original B0 is retained. Migration and candidate sources are separate. -Diagnostic HTTP values, operator improvements, unit tests and numerical equality -are distinct conclusions. Missing quality samples remain incomplete; failures are -not removed because other samples pass. The heldout clips have not been generated. -See [execution checkpoint](plan/execution-checkpoint.json) and [results](results/README.md). - -Private connection details and original raw captures stay outside this repository. -This Draft tracking snapshot contains sanitized projections and engine patch -artifacts. Independent review, four-track acceptance and production qualification -remain incomplete. The inventory registration checks layout/bytes only. +# MiniMax-H3 Ref2VA on four logical Ascend devices + +Acceptance: [#297](https://github.com/InferMatrixLab/InferRSI/issues/297). +**Campaign verdict remains INCONCLUSIVE.** The required complete-HTTP gain and four final tracks remain unqualified; rejected profiles stay rejected. + +This explicitly requested cleanup preserves the 96 previously public +files (1,259,483 bytes) at `6c7e237935a22cc5e90e954d4f397376e753e7d8` unchanged in a +versioned archive. It moves cumulative checkpoints, repeated diagnostic source +copies and generated details out of the current tree. The original README, +failed/interrupted attempts, superseded observations, all patches and frozen +validators remain at their original paths inside the archive. No model run, +historical gate, result or original byte was changed. + +## Retained evidence + +- [Summary](results/summary.json): key recorded conclusions, sample distributions + recomputed from published samples, and explicit coverage/availability limits. +- [Index](results/trials.jsonl): 95 published record/attempt observations; + every row has configuration references, status, conclusion, archive identity + and an archive-relative JSON Pointer. These are not independent trial counts. +- [Archive authority](results/artifacts.json): v2 URL/hash/size/source/retention + and pinned replay inputs. The inventory references this document. +- [Plan](plan/README.md), [patch](patch/README.md) and [validator](validator/README.md). + +Identical observations of the same named attempt collapse to one row with all +`source_records`. Different observations of that attempt, retries and anonymous +records remain separate. Historical revisions are not replaced by the latest +checkpoint. Unpublished attempts cannot be reconstructed; Markdown-only history +and per-request JSONL remain in the original archive. Passing archive checks +does not complete the campaign or turn individual engineering results into PASS. + +The [original tree](https://github.com/InferMatrixLab/InferRSI/tree/6c7e237935a22cc5e90e954d4f397376e753e7d8/evidence/h3-ref2va-910c-4logical-20261101-297/) +is another immutable reference. This change does not rewrite Git history. diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/patch/README.md b/evidence/h3-ref2va-910c-4logical-20261101-297/patch/README.md index e7820c539..9f6000076 100644 --- a/evidence/h3-ref2va-910c-4logical-20261101-297/patch/README.md +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/patch/README.md @@ -1,78 +1,10 @@ -# Patch status +# Source and patch identities -[C1 patch](c1-h3-bf16-prefix.patch) changes exactly three H3 source/test files. -The [identity record](c1-source-identity.json) binds the tested private commit, -parent, Git tree and file hashes. The engine commit remains private/unmerged; -its diff is published here as an experiment artifact. Native focused -tests passed (10 tests). Two complete fixed requests per duration passed strict -comparison for both 5 s and 15 s, 439 checks each. The 15 s audit also recorded -2,500 actual prefix-attention calls per rank per request. Clean performance -screening measured a 1.5729% reduction at5s and failed the50% performance bar. -Final qualification remains pending; no50% HTTP gain is accepted. +All original patch/source-identity records remain under `h3-ref2va-910c-4logical-20261101-297/patch/` in the +archive declared by `../results/artifacts.json`, with exact per-file SHA-256 in +its `snapshot.json`. The original README retains upstream implementation links +and the distinction between tested, rejected and proposed variants. A later +source revision does not inherit an earlier quality or performance verdict. -The H3-owned override handles the explicit long-query BF16 fused selection with -independent BNSD head batches and valid K/V-prefix slicing. Default and unsupported -paths retain their parent behavior. It adds no generic backend API, MindIE binary -or common environment change. Other failed candidates are retained in results; -their existence does not establish upstream support. Any eventual engine PR -needs current author checks and the upstream human-review/AI-disclosure gates. - -[C10 patch](c10-h3-audio-cache-guard.patch) changes four H3 source/test files. -Its [identity](c10-source-identity.json) binds private commit `0af99184` to the -native-tested bytes. Fifteen focused tests passed, including native NPU execution. -The first full5s-01 request had no failures among39 observed gates. Independent -5s-00 failed CLAP and audio-caption gates; the original C10 configuration is -rejected for5s. Diagnostic HTTP was236.9/243.0s, with no clean performance claim. -The second audit directly recorded50 actual wrapper forwards per rank and -42checks/27vetoes agreeing across all4ranks. The first audit's missing runtime -counters remain a historical evidence gap. No public API, dependency or frozen -quality-gate change is included. - -## C12–C23 source artifacts - -Each C12/C13/C15–C23 `*-source-identity.json` records its tested engine head, -parent, tree, raw file hashes and `patch_base`. Each matching -`*-candidate.patch` is cumulative over that recorded `patch_base`: - -- C12, C13, C15, C16, C21, C22 and C23 use C1 - `be319692b96413918973b5bd399cdda3f320c574`. Apply C1, then exactly one of - these patches. -- C17, C18, C19 and C20 use C13 `e89b1d40d37f7f72767892a56fa7980ea572ba8e`. - Each of these patches changes only two H3 paths and omits C13's ten - DLO/collective paths. Apply C1, then C13, then exactly one of C17–C20. - -Patches that share a base are alternatives, not a sequence to stack. -`published: false` in an archived identity describes the engine commit's status -at capture, not this evidence PR's publication state. - -C13 adapts [vLLM-Omni PR #9351](https://github.com/vllm-project/vllm-omni/pull/9351) -ordering to the pinned chunked DLO implementation, retaining current manifests -and double staging slots. C15–C23 are local experiments; their native tests and -primitive results do not establish upstream support or complete model quality. -C14 is a prerequisite probe and has no adopted engine patch. - -[The experiment report](../results/report.md) records dispositions and limits. -In particular, C21/C22/C23 failed frozen quality and are not promoted for use. -The separate [vLLM-Omni Draft PR #9745](https://github.com/vllm-project/vllm-omni/pull/9745) -ports C1 to main; [its native checks](../results/c1-upstream-main-native.json) -passed 12 tests. New-base full-model/HTTP qualification remains pending. These -historical patch files are unchanged and retain their original tested revisions. -No MindIE-SD code contribution is included. -MindIE-SD source, installed provider binaries and shared environments were not -modified. Future upstream engine contributions require their own focused checks -and review. [Source reconstruction notes](../plan/reproduction.md) state the -external artifacts needed for replay. - - -[C11 dual-sigma patch](c11-modality-sigma-residual.patch) changes five H3 paths. -Its [identity](c11-source-identity.json) binds61b8efa and20 native-focused tests. -Fn1/Bn0 failed5s-00 audio adherence; Fn1/Bn1 failed its frozen audio-level gate. -These are rejected configurations, not production acceleration claims. - -[FP32 residual revision](c11-sigma-fp32-residual.patch) preserves the same -five-path overall boundary and adds a two-file delta over61b8efa. Its -[identity](c11-fp32-source-identity.json) bindsd4d25b4 and23 passed tests, -including three executed NPU tests. The native attempt projection retains two -test-reference failures and the later correction. The production bytes stayed -unchanged across these native attempts. Full5s-00 and15s-00 still failed frozen -quality guards; the patch is not promoted. No frozen scorer/threshold changed. +The immutable original source tree is +https://github.com/InferMatrixLab/InferRSI/tree/6c7e237935a22cc5e90e954d4f397376e753e7d8/evidence/h3-ref2va-910c-4logical-20261101-297/patch diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c10-h3-audio-cache-guard.patch b/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c10-h3-audio-cache-guard.patch deleted file mode 100644 index a92ab501c..000000000 --- a/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c10-h3-audio-cache-guard.patch +++ /dev/null @@ -1,412 +0,0 @@ -diff --git a/tests/diffusion/models/minimax_h3/test_minimax_h3_audio_cache_guard.py b/tests/diffusion/models/minimax_h3/test_minimax_h3_audio_cache_guard.py -new file mode 100644 -index 0000000..c973180 ---- /dev/null -+++ b/tests/diffusion/models/minimax_h3/test_minimax_h3_audio_cache_guard.py -@@ -0,0 +1,185 @@ -+# SPDX-License-Identifier: Apache-2.0 -+# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project -+ -+from types import SimpleNamespace -+ -+import pytest -+import torch -+import torch.nn as nn -+ -+from tests.helpers.mark import hardware_marks -+from vllm_omni.diffusion.models.minimax_h3 import cache_audio_guard as guard -+from vllm_omni.diffusion.models.minimax_h3.reference_kv_cache_dit import ( -+ MiniMaxH3CachedAdapter, -+ MiniMaxH3CachedBlocks, -+) -+from vllm_omni.platforms import current_omni_platform -+ -+pytestmark = [pytest.mark.core_model, pytest.mark.diffusion, *hardware_marks(res={"npu": "A3"}, num_cards=1)] -+ -+ -+def decision(previous, current, indices, **kwargs): -+ return guard._modality_changes_allow_reuse( -+ previous, -+ current, -+ indices, -+ max_seqlen=kwargs.pop("max_seqlen", current.shape[0]), -+ packed_total=kwargs.pop("packed_total", current.shape[0]), -+ video_limit=kwargs.pop("video_limit", 0.08), -+ parallelized=kwargs.pop("parallelized", False), -+ **kwargs, -+ ) -+ -+ -+def test_audio_change_hidden_by_global_mean_vetoes(): -+ before = torch.ones(128, 8) -+ after = before.clone() -+ after[-2:] = 1.3 -+ indices = torch.zeros(128, dtype=torch.long) -+ indices[-2:] = 17 # timestep-index5, audio-modality2 -+ assert ((before - after).abs().mean() / before.abs().mean()).item() < 0.08 -+ assert not decision(before, after, indices) -+ after[-2:] = 1.02 -+ assert decision(before, after, indices) -+ assert not decision(before, after, indices, video_limit=0.01) -+ -+ -+def test_video_change_and_structural_padding(): -+ before = torch.ones(8, 4) -+ after = before.clone() -+ indices = torch.tensor([0, 0, 2, 2, 0, 0, 0, 0]) -+ after[4:] = float("nan") -+ assert decision(before, after, indices, max_seqlen=4) -+ after[:2] = 1.2 -+ assert not decision(before, after, indices, max_seqlen=4) -+ -+ -+def test_zero_denominator_and_invalid_layout(): -+ before = torch.zeros(4, 2) -+ after = before.clone() -+ indices = torch.full((4,), 2) -+ assert decision(before, after, indices) -+ after[0] = 1 -+ assert not decision(before, after, indices) -+ with pytest.raises(ValueError, match="valid packed prefix"): -+ decision(before, after, indices, packed_total=5) -+ with pytest.raises(ValueError, match="modality index"): -+ decision(before, after, indices[:2]) -+ -+ -+def test_remote_audio_participates_when_local_rank_has_none(monkeypatch): -+ from vllm_omni.diffusion.distributed import parallel_state -+ -+ monkeypatch.setattr(guard.dist, "is_initialized", lambda: True) -+ monkeypatch.setattr(parallel_state, "get_sequence_parallel_rank", lambda: 0) -+ monkeypatch.setattr(parallel_state, "get_sequence_parallel_world_size", lambda: 2) -+ calls = [] -+ -+ def reduce(values, op): -+ assert op == guard.dist.ReduceOp.SUM -+ calls.append(values.clone()) -+ values.add_(torch.tensor([0.0, 2.0, 1.0, 0.8, 2.0, 1.0])) -+ -+ monkeypatch.setattr(guard.dist, "all_reduce", reduce) -+ before = torch.ones(2, 8) -+ assert not decision(before, before, torch.zeros(2, dtype=torch.long), packed_total=4, parallelized=True) -+ assert len(calls) == 1 and calls[0][5].item() == 0 -+ -+ -+@pytest.mark.parametrize("change", ["default", "task", "cache", "refkv", "dp", "cfg"]) -+def test_adapter_selection_preserves_other_modes(change): -+ config = SimpleNamespace( -+ task_type="ref2va", -+ cache_backend="cache_dit", -+ kv_offload_config=None, -+ parallel_config=SimpleNamespace(data_parallel_size=1, cfg_parallel_size=1), -+ ) -+ if change == "task": -+ config.task_type = "t2va" -+ elif change == "cache": -+ config.cache_backend = "none" -+ elif change == "refkv": -+ config.kv_offload_config = {"mode": "on"} -+ elif change == "dp": -+ config.parallel_config.data_parallel_size = 2 -+ elif change == "cfg": -+ config.parallel_config.cfg_parallel_size = 2 -+ assert guard.audio_cache_guard_enabled(config) == (change == "default") -+ assert MiniMaxH3CachedAdapter.block_class is MiniMaxH3CachedBlocks -+ assert guard.MiniMaxH3AudioGuardCachedAdapter.block_class is guard.MiniMaxH3AudioGuardCachedBlocks -+ -+ -+def test_parent_denial_does_not_run_guard(): -+ class Owner: -+ _audio_guard_bypass = False -+ _audio_guard_layout = None -+ _audio_guard_checks = 0 -+ _audio_guard_vetoes = 0 -+ -+ owner = Owner() -+ proxy = guard._AudioCacheManager(SimpleNamespace(can_cache=lambda *args, **kwargs: False), owner) -+ assert proxy.can_cache(torch.ones(4, 2)) is False -+ assert owner._audio_guard_checks == 0 -+ -+ -+def test_allowed_parent_hit_is_vetoed_without_changing_buffers(): -+ class Owner: -+ _audio_guard_bypass = False -+ _audio_guard_checks = 0 -+ _audio_guard_vetoes = 0 -+ -+ previous = torch.ones(128, 8) -+ current = previous.clone() -+ current[-2:] = 1.3 -+ indices = torch.zeros(128, dtype=torch.long) -+ indices[-2:] = 2 -+ owner = Owner() -+ owner._audio_guard_layout = (indices, 128, 128) -+ manager = SimpleNamespace( -+ can_cache=lambda *args, **kwargs: True, -+ get_Fn_buffer=lambda prefix: previous, -+ get_residual_diff_threshold=lambda: 0.08, -+ ) -+ proxy = guard._AudioCacheManager(manager, owner) -+ assert proxy.can_cache(current, prefix="blocks_Fn_residual", parallelized=False) is False -+ assert (owner._audio_guard_checks, owner._audio_guard_vetoes) == (1, 1) -+ current[-2:] = 1.01 -+ assert proxy.can_cache(current, prefix="blocks_Fn_residual", parallelized=False) is True -+ assert (owner._audio_guard_checks, owner._audio_guard_vetoes) == (2, 1) -+ assert torch.equal(previous, torch.ones_like(previous)) -+ owner._audio_guard_bypass, owner._audio_guard_layout = True, None -+ assert proxy.can_cache(current) is True -+ assert owner._audio_guard_checks == 2 -+ -+ -+@pytest.mark.parametrize("failure", ["metadata", "parent"]) -+def test_forward_releases_layout_on_failure(monkeypatch, failure): -+ wrapper = object.__new__(guard.MiniMaxH3AudioGuardCachedBlocks) -+ nn.Module.__init__(wrapper) -+ wrapper._audio_guard_layout, wrapper._audio_guard_bypass = None, False -+ -+ def broken(self, *args, **kwargs): -+ raise RuntimeError("parent failed") -+ -+ monkeypatch.setattr(MiniMaxH3CachedBlocks, "forward", broken) -+ kwargs = ( -+ {} -+ if failure == "metadata" -+ else {"combined_indices": torch.zeros(4, dtype=torch.long), "max_seqlen": 4, "packed_total": 4} -+ ) -+ with pytest.raises((KeyError, RuntimeError)): -+ wrapper.forward(torch.ones(4, 2), **kwargs) -+ assert wrapper._audio_guard_layout is None and wrapper._audio_guard_bypass is False -+ -+ -+@pytest.mark.skipif(not current_omni_platform.is_npu(), reason="Requires native Ascend reductions") -+def test_native_audio_guard_preserves_input_values(): -+ device = torch.device(current_omni_platform.device_type) -+ before = torch.ones(128, 64, dtype=torch.bfloat16, device=device) -+ after = before.clone() -+ indices = torch.zeros(128, dtype=torch.long, device=device) -+ indices[-2:] = 2 -+ after[-2:] = 1.3 -+ saved = [tensor.clone() for tensor in (before, after, indices)] -+ assert not decision(before, after, indices) -+ assert all(torch.equal(x, y) for x, y in zip(saved, (before, after, indices))) -diff --git a/vllm_omni/diffusion/models/minimax_h3/cache_audio_guard.py b/vllm_omni/diffusion/models/minimax_h3/cache_audio_guard.py -new file mode 100644 -index 0000000..7472404 ---- /dev/null -+++ b/vllm_omni/diffusion/models/minimax_h3/cache_audio_guard.py -@@ -0,0 +1,174 @@ -+# SPDX-License-Identifier: Apache-2.0 -+# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project -+"""Ref2VA-owned veto for cache hits with a changing audio modality.""" -+ -+from __future__ import annotations -+ -+import math -+from typing import Any -+from weakref import ref -+ -+import torch -+import torch.distributed as dist -+ -+from .reference_kv_cache_dit import MiniMaxH3CachedAdapter, MiniMaxH3CachedBlocks -+ -+_AUDIO_REUSE_LIMIT = 0.04 -+ -+ -+def audio_cache_guard_enabled(config: Any) -> bool: -+ parallel = getattr(config, "parallel_config", None) -+ return ( -+ getattr(config, "task_type", None) == "ref2va" -+ and getattr(config, "cache_backend", None) == "cache_dit" -+ and getattr(config, "kv_offload_config", None) is None -+ and (getattr(parallel, "data_parallel_size", 1) or 1) == 1 -+ and (getattr(parallel, "cfg_parallel_size", 1) or 1) == 1 -+ ) -+ -+ -+def _modality_changes_allow_reuse( -+ previous: torch.Tensor, -+ current: torch.Tensor, -+ combined_indices: torch.Tensor, -+ *, -+ max_seqlen: int, -+ packed_total: int, -+ video_limit: float, -+ parallelized: bool, -+) -> bool: -+ if previous.shape != current.shape or current.ndim != 2: -+ raise ValueError("H3 audio cache guard requires matching [local_rows, hidden] states") -+ if combined_indices.ndim != 1 or combined_indices.shape[0] != current.shape[0]: -+ raise ValueError("H3 audio cache guard requires one modality index per local row") -+ if previous.device != current.device or combined_indices.device != current.device: -+ raise ValueError("H3 audio cache guard state and modality devices must match") -+ if not math.isfinite(video_limit) or video_limit <= 0: -+ return False -+ -+ sp_rank, sp_size = 0, 1 -+ if dist.is_initialized(): -+ from vllm_omni.diffusion.distributed.parallel_state import ( -+ get_sequence_parallel_rank, -+ get_sequence_parallel_world_size, -+ ) -+ -+ sp_rank, sp_size = get_sequence_parallel_rank(), get_sequence_parallel_world_size() -+ rows = current.shape[0] -+ if packed_total != rows * sp_size or not 0 < max_seqlen <= packed_total: -+ raise ValueError("H3 audio cache guard requires equal SP shards and a valid packed prefix") -+ -+ # Padding tags were clamped before combined_indices was built. Exclude the -+ # structural suffix using positions rather than interpreting those tags. -+ valid = torch.arange(rows, device=current.device) + sp_rank * rows < max_seqlen -+ modalities = combined_indices.remainder(3) -+ before, after = previous.float(), current.float() -+ delta = (before - after).abs().mean(dim=-1) -+ magnitude = before.abs().mean(dim=-1) -+ scalars = [] -+ for modality in (0, 2): # Video and audio; the parent still checks all tokens. -+ mask = valid & (modalities == modality) -+ scalars.extend( -+ [ -+ torch.where(mask, delta, 0.0).sum(), -+ torch.where(mask, magnitude, 0.0).sum(), -+ mask.sum(dtype=torch.float32), -+ ] -+ ) -+ totals = torch.stack(scalars) -+ if parallelized: -+ if not dist.is_initialized(): -+ raise RuntimeError("Parallel H3 audio cache guard requires an initialized process group") -+ # Every rank participates, including ranks with no local audio rows. -+ # TP replicas duplicate both sums equally, so their ratio is preserved. -+ dist.all_reduce(totals, op=dist.ReduceOp.SUM) -+ values = totals.cpu().tolist() -+ for start, limit in ((0, video_limit), (3, min(video_limit, _AUDIO_REUSE_LIMIT))): -+ numerator, denominator, count = values[start : start + 3] -+ if count == 0: -+ continue -+ if not math.isfinite(numerator) or not math.isfinite(denominator): -+ return False -+ if denominator == 0: -+ if numerator != 0: -+ return False -+ elif numerator / denominator > limit: -+ return False -+ return True -+ -+ -+class _AudioCacheManager: -+ def __init__(self, manager: Any, owner: MiniMaxH3AudioGuardCachedBlocks) -> None: -+ self._manager = manager -+ self._owner = ref(owner) -+ -+ def __getattr__(self, name: str) -> Any: -+ return getattr(self._manager, name) -+ -+ def can_cache(self, states_tensor: torch.Tensor, *args, **kwargs) -> bool: -+ allowed = self._manager.can_cache(states_tensor, *args, **kwargs) -+ owner = self._owner() -+ if not allowed or owner is None or owner._audio_guard_bypass: -+ return allowed -+ if args: -+ raise TypeError("H3 audio cache guard expects named cache-decision options") -+ layout = owner._audio_guard_layout -+ if layout is None: -+ raise RuntimeError("H3 audio cache guard has no active block layout") -+ prefix = kwargs.get("prefix", "Fn") -+ previous = self._manager.get_Fn_buffer(prefix) -+ if previous is None: -+ return False -+ threshold = kwargs.get("threshold") -+ if threshold is None or threshold <= 0: -+ threshold = self._manager.get_residual_diff_threshold() -+ owner._audio_guard_checks += 1 -+ accepted = _modality_changes_allow_reuse( -+ previous, -+ states_tensor, -+ layout[0], -+ max_seqlen=layout[1], -+ packed_total=layout[2], -+ video_limit=float(threshold), -+ parallelized=kwargs.get("parallelized", False), -+ ) -+ owner._audio_guard_vetoes += int(not accepted) -+ return accepted -+ -+ -+class MiniMaxH3AudioGuardCachedBlocks(MiniMaxH3CachedBlocks): -+ def __init__(self, *args, **kwargs) -> None: -+ super().__init__(*args, **kwargs) -+ self._audio_guard_layout = None -+ self._audio_guard_bypass = False -+ self._audio_guard_checks = 0 -+ self._audio_guard_vetoes = 0 -+ self.context_manager = _AudioCacheManager(self.context_manager, self) -+ -+ def audio_cache_guard_stats(self) -> dict[str, int | float]: -+ """Diagnostic counters for this wrapper's lifetime, not quality metrics.""" -+ return { -+ "checks": self._audio_guard_checks, -+ "vetoes": self._audio_guard_vetoes, -+ "audio_reuse_limit": _AUDIO_REUSE_LIMIT, -+ } -+ -+ def forward(self, hidden_states: torch.Tensor, *args, **kwargs): -+ old_layout, old_bypass = self._audio_guard_layout, self._audio_guard_bypass -+ try: -+ self._audio_guard_bypass = ( -+ kwargs.get("num_requests", 1) != 1 or kwargs.get("reference_kv_state") is not None -+ ) -+ self._audio_guard_layout = ( -+ None -+ if self._audio_guard_bypass -+ else (kwargs["combined_indices"], kwargs["max_seqlen"], kwargs["packed_total"]) -+ ) -+ return super().forward(hidden_states, *args, **kwargs) -+ finally: -+ # Do not retain request metadata in a persistent cached model. -+ self._audio_guard_layout, self._audio_guard_bypass = old_layout, old_bypass -+ -+ -+class MiniMaxH3AudioGuardCachedAdapter(MiniMaxH3CachedAdapter): -+ block_class = MiniMaxH3AudioGuardCachedBlocks -diff --git a/vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py b/vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py -index 9518828..1669757 100644 ---- a/vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py -+++ b/vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py -@@ -784,6 +784,14 @@ class MiniMaxH3DiTModel(nn.Module): - self._cache_dit_adapter_config, - cached_adapter_cls=MiniMaxH3CachedAdapter, - ) -+ elif getattr(od_config, "cache_backend", None) == "cache_dit": -+ from .cache_audio_guard import MiniMaxH3AudioGuardCachedAdapter, audio_cache_guard_enabled -+ -+ if audio_cache_guard_enabled(od_config): -+ self._cache_dit_adapter_config = replace( -+ self._cache_dit_adapter_config, -+ cached_adapter_cls=MiniMaxH3AudioGuardCachedAdapter, -+ ) - self.parallel_config = od_config.parallel_config - self.hidden_size = arch.hidden_size - self.num_attention_heads = arch.num_attention_heads -diff --git a/vllm_omni/diffusion/models/minimax_h3/reference_kv_cache_dit.py b/vllm_omni/diffusion/models/minimax_h3/reference_kv_cache_dit.py -index f14dcc9..f78598b 100644 ---- a/vllm_omni/diffusion/models/minimax_h3/reference_kv_cache_dit.py -+++ b/vllm_omni/diffusion/models/minimax_h3/reference_kv_cache_dit.py -@@ -125,6 +125,8 @@ class MiniMaxH3CachedBlocks(CachedBlocks_Pattern_3_4_5): - - - class MiniMaxH3CachedAdapter(CachedAdapter): -+ block_class = MiniMaxH3CachedBlocks -+ - @classmethod - def collect_unified_blocks( - cls, block_adapter: BlockAdapter, contexts_kwargs: list[dict] -@@ -139,7 +141,7 @@ class MiniMaxH3CachedAdapter(CachedAdapter): - context_index += 1 - contexts[block_adapter.unique_blocks_name[i][j]] = nn.ModuleList( - [ -- MiniMaxH3CachedBlocks( -+ cls.block_class( - blocks, - transformer=transformer, - forward_pattern=block_adapter.forward_pattern[i][j], diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c10-source-identity.json b/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c10-source-identity.json deleted file mode 100644 index 2fa4f956f..000000000 --- a/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c10-source-identity.json +++ /dev/null @@ -1,13 +0,0 @@ -{ - "head": "0af99184bc860cffa8d1e5343c2bff4664f667c5", - "tree": "3e3971e6a1c63e70116cef7dbcb5da2678ff514f", - "base": "922f1e9f1cb85b9f2b66d86e1eb500e6872b2dcb", - "branch": "rsi/h3-297-audio-cache-guard", - "files": { - "vllm_omni/diffusion/models/minimax_h3/cache_audio_guard.py": "2844e749c9c876e52bffeef565f2c69904693e2d62c359da6abe66844930daed", - "vllm_omni/diffusion/models/minimax_h3/reference_kv_cache_dit.py": "d3381079ff012f32176a917bb0825d12288c058d3a058f5169ecc97d1e2c675c", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "1d5e806489de94cec2f9224f7d31ff96d9ace0a3fb80c23d8f8ba25f262d275d", - "tests/diffusion/models/minimax_h3/test_minimax_h3_audio_cache_guard.py": "c13e1ec677531457e92853d3a97e1e3fdcccb9547883ef0088f75852473d523d" - }, - "published": false -} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c11-fp32-source-identity.json b/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c11-fp32-source-identity.json deleted file mode 100644 index 54eb936e1..000000000 --- a/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c11-fp32-source-identity.json +++ /dev/null @@ -1,24 +0,0 @@ -{ - "parent": "61b8efa06e88372bc5ab87b9e2668a133eaac282", - "cumulative_patch_base": "922f1e9f1cb85b9f2b66d86e1eb500e6872b2dcb", - "head": "d4d25b420889965ab42459c96ab4a82389c9fa64", - "tree": "a1aa28529bd71b733914d82a80aa76e12f5a76f7", - "files": { - "vllm_omni/diffusion/models/minimax_h3/cache_sigma_prediction.py": "fb27fdebdf0cb45e678243c0180d56cf3e7a7470ff424bdf2d3bf144c8e2aee3", - "vllm_omni/diffusion/models/minimax_h3/reference_kv_cache_dit.py": "d3381079ff012f32176a917bb0825d12288c058d3a058f5169ecc97d1e2c675c", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "a649a09e5b59256b5887c12fe508d8e14e3dedd9019bdb82c65db2d59f11de91", - "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "a9524734a4f3410a0c6f766fa64fa9e5bd5a4d41bd7481738e9024cac54ce7f9", - "tests/diffusion/models/minimax_h3/test_minimax_h3_sigma_cache.py": "16d1e9329c0088eabf4bb17214fc364ec0c6176d92f871786456a2b92aaa418f" - }, - "native_tests_passed": 23, - "native_test_names": [ - "test_native_two_clock_forecast_and_input_immutability", - "test_native_real_cache_manager_computes_then_forecasts", - "test_native_bf16_blocks_use_fp32_anchors_and_keep_output_dtype" - ], - "cleanup_passed": true, - "source_unchanged": true, - "published": false, - "full_model_quality": "5s-00 FAIL adherence;15s-00 FAIL five visual/audio guards", - "performance_acceptance": "PENDING" -} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c11-modality-sigma-residual.patch b/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c11-modality-sigma-residual.patch deleted file mode 100644 index 18c25cfdb..000000000 --- a/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c11-modality-sigma-residual.patch +++ /dev/null @@ -1,620 +0,0 @@ -diff --git a/tests/diffusion/models/minimax_h3/test_minimax_h3_sigma_cache.py b/tests/diffusion/models/minimax_h3/test_minimax_h3_sigma_cache.py -new file mode 100644 -index 0000000..c5a91cc ---- /dev/null -+++ b/tests/diffusion/models/minimax_h3/test_minimax_h3_sigma_cache.py -@@ -0,0 +1,273 @@ -+# SPDX-License-Identifier: Apache-2.0 -+# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project -+ -+from types import SimpleNamespace -+ -+import pytest -+import torch -+import torch.nn as nn -+ -+from tests.helpers.mark import hardware_marks -+from vllm_omni.diffusion.models.minimax_h3 import cache_sigma_prediction as sigma -+from vllm_omni.diffusion.models.minimax_h3.reference_kv_cache_dit import MiniMaxH3CachedBlocks -+from vllm_omni.platforms import current_omni_platform -+ -+pytestmark = [pytest.mark.core_model, pytest.mark.diffusion, *hardware_marks(res={"npu": "A3"}, num_cards=1)] -+ -+ -+def config(**changes): -+ values = dict( -+ cache_config={"minimax_h3_sigma_cache": True, "enable_taylorseer": False}, -+ task_type="ref2va", -+ cache_backend="cache_dit", -+ kv_offload_config=None, -+ parallel_config=SimpleNamespace(data_parallel_size=1, cfg_parallel_size=1), -+ ) -+ values.update(changes) -+ return SimpleNamespace(**values) -+ -+ -+def sample(video, audio, *, device="cpu", dtype=torch.float32): -+ return torch.tensor([[1 + 2 * video + video**2], [4 - audio + 3 * audio**2]], device=device, dtype=dtype) -+ -+ -+def anchored(*, device="cpu", dtype=torch.float32): -+ state = sigma.MiniMaxH3SigmaCacheState() -+ originals = [] -+ for step, (video, audio) in enumerate([(1.0, 1.0), (0.9, 0.8), (0.7, 0.5)]): -+ state.begin_step(step, video, audio) -+ residual = sample(video, audio, device=device, dtype=dtype) -+ state.record(residual) -+ originals.append(residual) -+ state.begin_step(3, 0.5, 0.3) -+ return state, originals -+ -+ -+def test_distinct_nonuniform_clocks_recover_quadratic_and_own_anchors(): -+ state, originals = anchored() -+ indices = torch.tensor([0, 17]) -+ expected = sample(0.5, 0.3) -+ assert state.can_predict(expected) -+ torch.testing.assert_close(state.predict(indices), expected) -+ saved = [value.clone() for _, value in state.history] -+ originals[-1].fill_(99) -+ assert all(torch.equal(old, value) for old, (_, value) in zip(saved, state.history)) -+ torch.testing.assert_close(state.predict(indices), expected) -+ # One shared video clock gives the wrong audio result on this fixture. -+ video_weights = sigma._quadratic_weights([1.0, 0.9, 0.7], 0.5) -+ wrong_audio = sum(weight * value[1] for weight, (_, value) in zip(video_weights, state.history)) -+ assert not torch.allclose(wrong_audio, expected[1]) -+ assert state.stats() == {"computed": 3, "predicted": 2, "history_vetoes": 0} -+ state.close() -+ assert state.closed and not state.history and not state.can_predict(expected) -+ -+ -+@pytest.mark.parametrize( -+ "anchors,target", -+ [([1.0, 0.8], 0.7), ([1.0, 0.8, 0.8], 0.7), ([1.0, 0.9, 0.8], 0.1), ([1.0, 0.9, 0.8], float("nan"))], -+) -+def test_unstable_or_insufficient_history_vetoes(anchors, target): -+ assert sigma._quadratic_weights(anchors, target) is None -+ -+ -+@pytest.mark.parametrize( -+ "changes", -+ [ -+ {"task_type": "t2va"}, -+ {"cache_backend": "none"}, -+ {"kv_offload_config": {}}, -+ {"parallel_config": SimpleNamespace(data_parallel_size=2)}, -+ {"cache_config": {"minimax_h3_sigma_cache": 1}}, -+ {"cache_config": {"minimax_h3_sigma_cache": True, "enable_taylorseer": True}}, -+ ], -+) -+def test_explicit_unsupported_selection_fails(changes): -+ with pytest.raises(ValueError): -+ sigma.sigma_cache_enabled(config(**changes)) -+ -+ -+def test_default_and_dataclass_extra_selection(): -+ from vllm_omni.diffusion.data import DiffusionCacheConfig -+ -+ assert not sigma.sigma_cache_enabled(config(cache_config={})) -+ cache = DiffusionCacheConfig.from_dict({"minimax_h3_sigma_cache": True}) -+ assert sigma.sigma_cache_enabled(config(cache_config=cache)) -+ assert sigma.MiniMaxH3SigmaCachedAdapter.block_class is sigma.MiniMaxH3SigmaCachedBlocks -+ -+ -+def test_parent_denial_and_request_prediction_do_not_mutate_inputs(): -+ state, _ = anchored() -+ owner = SimpleOwner(state, torch.tensor([0, 2])) -+ calls = [] -+ manager = SimpleNamespace( -+ can_cache=lambda *args, **kwargs: False, -+ is_cache_residual=lambda: True, -+ set_Bn_buffer=lambda *args, **kwargs: calls.append((args, kwargs)), -+ ) -+ proxy = sigma._SigmaCacheManager(manager, owner) -+ hidden = torch.ones(2, 1) -+ assert not proxy.can_cache(hidden) -+ assert state.history_vetoes == 0 -+ manager.can_cache = lambda *args, **kwargs: True -+ assert proxy.can_cache(hidden) -+ actual, encoder = proxy.apply_cache(hidden, prefix="blocks_Bn_residual") -+ torch.testing.assert_close(actual, sample(0.5, 0.3) + 1) -+ assert encoder is None and torch.equal(hidden, torch.ones_like(hidden)) -+ owner._sigma_state = sigma.MiniMaxH3SigmaCacheState() -+ assert not proxy.can_cache(hidden) -+ assert owner._sigma_state.history_vetoes == 1 -+ -+ -+class SimpleOwner: -+ def __init__(self, state, indices): -+ self._sigma_state, self._sigma_indices = state, indices -+ -+ -+@pytest.mark.parametrize("failure", [False, True]) -+def test_wrapper_releases_transient_request_metadata(monkeypatch, failure): -+ wrapper = object.__new__(sigma.MiniMaxH3SigmaCachedBlocks) -+ nn.Module.__init__(wrapper) -+ wrapper._sigma_state, wrapper._sigma_indices = None, None -+ state = sigma.MiniMaxH3SigmaCacheState() -+ -+ def parent(self, hidden, *args, **kwargs): -+ assert "sigma_cache_state" not in kwargs -+ assert self._sigma_state is state -+ if failure: -+ raise RuntimeError("block failed") -+ return hidden -+ -+ monkeypatch.setattr(MiniMaxH3CachedBlocks, "forward", parent) -+ args = dict(sigma_cache_state=state, combined_indices=torch.tensor([0, 2])) -+ if failure: -+ with pytest.raises(RuntimeError, match="block failed"): -+ wrapper.forward(torch.ones(2, 1), **args) -+ else: -+ wrapper.forward(torch.ones(2, 1), **args) -+ assert wrapper._sigma_state is None and wrapper._sigma_indices is None -+ -+ -+@pytest.mark.parametrize("failure", ["success", "model", "cancel"]) -+def test_denoise_request_closes_history_on_all_exit_paths(monkeypatch, failure): -+ from vllm_omni.diffusion.models.minimax_h3 import denoise_loop as loop -+ -+ monkeypatch.setattr(loop.current_omni_platform, "is_npu", lambda: False) -+ monkeypatch.setattr(loop, "minimax_h3_publish_denoise_progress", lambda *args: None) -+ state = sigma.MiniMaxH3SigmaCacheState() -+ -+ class Model: -+ def create_sigma_cache_state(self): -+ return state -+ -+ def __call__(self, **kwargs): -+ assert kwargs["sigma_cache_state"] is state -+ state.record(torch.ones(2, 1)) -+ if failure == "model": -+ raise RuntimeError("model failed") -+ return torch.zeros(1, 96), torch.zeros(1, 32) -+ -+ class Branch: -+ update_mask = torch.ones(1, dtype=torch.bool) -+ update_mask_dev = update_mask -+ audio_update_mask_dev = update_mask -+ locked_audio_rows = None -+ -+ def forward_kwargs(self, **kwargs): -+ return {} -+ -+ monkeypatch.setattr( -+ loop, -+ "minimax_h3_prepare_denoise_rows", -+ lambda **kwargs: (kwargs["initial_video_rows"], kwargs["initial_audio_rows"], None, None), -+ ) -+ -+ def cancellation(**kwargs): -+ if failure == "cancel": -+ raise RuntimeError("request cancelled") -+ -+ monkeypatch.setattr(loop, "check_request_cancellation", cancellation) -+ model = Model() -+ kwargs = dict( -+ model=model, -+ positive=Branch(), -+ initial_video_rows=torch.zeros(1, 96), -+ initial_audio_rows=torch.zeros(1, 32), -+ keyframe_cond_rows=None, -+ sigmas_video=[1.0, 0.0], -+ sigmas_audio=[1.0, 0.0], -+ device=torch.device("cpu"), -+ ) -+ if failure == "success": -+ loop.minimax_h3_denoise_loop(**kwargs) -+ else: -+ with pytest.raises(RuntimeError): -+ loop.minimax_h3_denoise_loop(**kwargs) -+ assert state.closed and not state.history -+ assert model._sigma_cache_stats["computed"] == (0 if failure == "cancel" else 1) -+ -+ -+@pytest.mark.skipif(not current_omni_platform.is_npu(), reason="Requires native Ascend forecasting") -+def test_native_two_clock_forecast_and_input_immutability(): -+ device = torch.device(current_omni_platform.device_type) -+ state, originals = anchored(device=device, dtype=torch.bfloat16) -+ indices = torch.tensor([0, 2], device=device) -+ saved = [value.clone() for value in originals] -+ expected = sample(0.5, 0.3, device=device, dtype=torch.bfloat16) -+ assert state.can_predict(expected) -+ actual = state.predict(indices) -+ # BF16 anchor quantization is separate from exact polynomial algebra. -+ torch.testing.assert_close(actual.float(), expected.float(), atol=0.0625, rtol=0) -+ assert all(torch.equal(value, old) for value, old in zip(originals, saved)) -+ assert state.stats()["predicted"] == 1 -+ state.close() -+ assert not state.history -+ -+ -+@pytest.mark.skipif(not current_omni_platform.is_npu(), reason="Requires native Ascend cached block execution") -+def test_native_real_cache_manager_computes_then_forecasts(): -+ from cache_dit import DBCacheConfig, ForwardPattern -+ from cache_dit.caching.cache_contexts.cache_manager import CachedContextManager -+ -+ class Block(nn.Module): -+ def __init__(self, prefix=False): -+ super().__init__() -+ self.prefix = prefix -+ self.calls = 0 -+ -+ def forward(self, hidden_states, *, test_sigmas, **kwargs): -+ self.calls += 1 -+ return hidden_states + (0.125 if self.prefix else sample(*test_sigmas, device=hidden_states.device)) -+ -+ manager = CachedContextManager() -+ context = manager.new_context( -+ name="h3-sigma-native", -+ cache_config=DBCacheConfig( -+ Fn_compute_blocks=1, -+ Bn_compute_blocks=0, -+ max_warmup_steps=4, -+ num_inference_steps=5, -+ residual_diff_threshold=0.08, -+ max_continuous_cached_steps=3, -+ ), -+ ) -+ blocks = nn.ModuleList([Block(prefix=True), Block(), Block()]) -+ wrapper = sigma.MiniMaxH3SigmaCachedBlocks( -+ blocks, -+ forward_pattern=ForwardPattern.Pattern_3, -+ check_forward_pattern=False, -+ cache_prefix="blocks", -+ cache_context=context, -+ context_manager=manager, -+ ) -+ device = torch.device(current_omni_platform.device_type) -+ indices = torch.tensor([0, 2], device=device) -+ state = sigma.MiniMaxH3SigmaCacheState() -+ for step, clocks in enumerate([(1.0, 1.0), (0.95, 0.9), (0.85, 0.75), (0.7, 0.55), (0.5, 0.35)]): -+ state.begin_step(step, *clocks) -+ hidden = torch.ones(2, 1, device=device) -+ actual = wrapper(hidden, sigma_cache_state=state, combined_indices=indices, test_sigmas=clocks) -+ torch.testing.assert_close(actual, hidden + 0.125 + 2 * sample(*clocks, device=device), atol=1e-5, rtol=1e-5) -+ assert wrapper._sigma_state is None and wrapper._sigma_indices is None -+ assert blocks[0].calls == 5 and blocks[1].calls == blocks[2].calls == 4 -+ assert state.stats()["computed"] == 4 and state.stats()["predicted"] == 1 -+ state.close() -diff --git a/vllm_omni/diffusion/models/minimax_h3/cache_sigma_prediction.py b/vllm_omni/diffusion/models/minimax_h3/cache_sigma_prediction.py -new file mode 100644 -index 0000000..50626a9 ---- /dev/null -+++ b/vllm_omni/diffusion/models/minimax_h3/cache_sigma_prediction.py -@@ -0,0 +1,194 @@ -+# SPDX-License-Identifier: Apache-2.0 -+# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project -+"""Opt-in, request-owned residual forecasting on H3's two sigma clocks.""" -+ -+from __future__ import annotations -+ -+import math -+from collections.abc import Mapping -+from typing import Any -+from weakref import ref -+ -+import torch -+ -+from .reference_kv_cache_dit import MiniMaxH3CachedAdapter, MiniMaxH3CachedBlocks -+ -+ -+def sigma_cache_enabled(config: Any) -> bool: -+ cache = getattr(config, "cache_config", {}) -+ enabled = ( -+ cache.get("minimax_h3_sigma_cache", False) -+ if isinstance(cache, Mapping) -+ else getattr(cache, "minimax_h3_sigma_cache", False) -+ ) -+ if type(enabled) is not bool: -+ raise ValueError("minimax_h3_sigma_cache must be a boolean") -+ if not enabled: -+ return False -+ parallel = getattr(config, "parallel_config", None) -+ if ( -+ getattr(config, "task_type", None) != "ref2va" -+ or getattr(config, "cache_backend", None) != "cache_dit" -+ or getattr(config, "kv_offload_config", None) is not None -+ or (getattr(parallel, "data_parallel_size", 1) or 1) != 1 -+ or (getattr(parallel, "cfg_parallel_size", 1) or 1) != 1 -+ ): -+ raise ValueError("H3 sigma cache requires Ref2VA Cache-DiT, DP1/CFG1 and no reference-KV cache") -+ taylor = cache.get("enable_taylorseer", False) if isinstance(cache, Mapping) else cache.enable_taylorseer -+ if taylor: -+ raise ValueError("H3 sigma cache replaces TaylorSeer; disable enable_taylorseer") -+ return True -+ -+ -+def _quadratic_weights(anchors: list[float], target: float) -> tuple[float, ...] | None: -+ if len(anchors) != 3 or not all(math.isfinite(x) for x in [*anchors, target]): -+ return None -+ if not anchors[0] > anchors[1] > anchors[2] > target: -+ return None -+ weights = [] -+ for i, anchor in enumerate(anchors): -+ weight = 1.0 -+ for j, other in enumerate(anchors): -+ if i != j: -+ weight *= (target - other) / (anchor - other) -+ weights.append(weight) -+ # A host-only veto: all SP ranks have the same clocks and anchor steps. -+ if not all(math.isfinite(x) for x in weights) or sum(abs(x) for x in weights) > 64.0: -+ return None -+ return tuple(weights) -+ -+ -+class MiniMaxH3SigmaCacheState: -+ """Three real middle-residual anchors, owned by one denoise request.""" -+ -+ def __init__(self) -> None: -+ self.step = -1 -+ self.sigmas = (1.0, 1.0) -+ self.history: list[tuple[tuple[float, float], torch.Tensor]] = [] -+ self.closed = False -+ self.computed = 0 -+ self.predicted = 0 -+ self.history_vetoes = 0 -+ -+ def begin_step(self, step: int, video_sigma: float, audio_sigma: float) -> None: -+ if self.closed or step != self.step + 1: -+ raise RuntimeError("H3 sigma cache requires consecutive steps in an open request") -+ sigmas = (float(video_sigma), float(audio_sigma)) -+ if not all(math.isfinite(x) for x in sigmas): -+ raise ValueError("H3 sigma cache requires finite sigma clocks") -+ if self.step >= 0 and not all(x < old for x, old in zip(sigmas, self.sigmas)): -+ raise ValueError("H3 sigma cache requires descending sigma clocks") -+ self.step, self.sigmas = step, sigmas -+ -+ def _weights(self) -> tuple[tuple[float, ...], tuple[float, ...]] | None: -+ if self.closed or len(self.history) != 3: -+ return None -+ video = _quadratic_weights([clock[0] for clock, _ in self.history], self.sigmas[0]) -+ audio = _quadratic_weights([clock[1] for clock, _ in self.history], self.sigmas[1]) -+ return None if video is None or audio is None else (video, audio) -+ -+ def can_predict(self, current: torch.Tensor) -> bool: -+ if self._weights() is None: -+ self.history_vetoes += 1 -+ return False -+ if any( -+ tensor.shape != current.shape or tensor.device != current.device or tensor.dtype != current.dtype -+ for _, tensor in self.history -+ ): -+ self.history.clear() -+ self.history_vetoes += 1 -+ return False -+ return True -+ -+ def record(self, residual: torch.Tensor) -> None: -+ if self.closed or self.step < 0 or residual.ndim != 2: -+ raise RuntimeError("H3 sigma cache requires an open step and a two-dimensional residual") -+ if self.history and self.history[-1][0] == self.sigmas: -+ raise RuntimeError("H3 sigma cache received duplicate middle-block residuals") -+ # Own the anchor: neither Cache-DiT nor the model may mutate our history. -+ self.history.append((self.sigmas, residual.detach().clone())) -+ del self.history[:-3] -+ self.computed += 1 -+ -+ def predict(self, combined_indices: torch.Tensor) -> torch.Tensor: -+ weights = self._weights() -+ if weights is None: -+ raise RuntimeError("H3 sigma cache cannot forecast without three stable anchors") -+ last = self.history[-1][1] -+ if combined_indices.ndim != 1 or combined_indices.shape[0] != last.shape[0]: -+ raise ValueError("H3 sigma cache requires one modality index per local residual row") -+ if combined_indices.device != last.device: -+ raise ValueError("H3 sigma cache modality and residual devices must match") -+ audio_rows = combined_indices.remainder(3).eq(2).unsqueeze(-1) -+ output = torch.zeros_like(last, dtype=torch.float32) -+ for i, (_, residual) in enumerate(self.history): -+ coefficient = torch.where(audio_rows, weights[1][i], weights[0][i]) -+ output.add_(residual.float() * coefficient) -+ self.predicted += 1 -+ return output.to(last.dtype) -+ -+ def stats(self) -> dict[str, int]: -+ return {"computed": self.computed, "predicted": self.predicted, "history_vetoes": self.history_vetoes} -+ -+ def close(self) -> None: -+ self.history.clear() -+ self.closed = True -+ -+ -+class _SigmaCacheManager: -+ def __init__(self, manager: Any, owner: MiniMaxH3SigmaCachedBlocks) -> None: -+ self._manager = manager -+ self._owner = ref(owner) -+ -+ def __getattr__(self, name: str) -> Any: -+ return getattr(self._manager, name) -+ -+ def _state(self) -> MiniMaxH3SigmaCacheState: -+ owner = self._owner() -+ if owner is None or owner._sigma_state is None: -+ raise RuntimeError("H3 sigma cache manager has no active request") -+ return owner._sigma_state -+ -+ def can_cache(self, states_tensor: torch.Tensor, *args, **kwargs) -> bool: -+ allowed = self._manager.can_cache(states_tensor, *args, **kwargs) -+ return allowed and self._state().can_predict(states_tensor) -+ -+ def set_Bn_buffer(self, buffer: torch.Tensor, prefix: str = "Bn") -> None: -+ if not self._manager.is_cache_residual(): -+ raise ValueError("H3 sigma cache requires middle-block residual caching") -+ self._state().record(buffer) -+ self._manager.set_Bn_buffer(buffer, prefix=prefix) -+ -+ def apply_cache(self, hidden_states: torch.Tensor, encoder_hidden_states=None, **kwargs): -+ if encoder_hidden_states is not None or not self._manager.is_cache_residual(): -+ raise ValueError("H3 sigma cache supports only the packed H3 residual stream") -+ owner = self._owner() -+ if owner is None or owner._sigma_indices is None: -+ raise RuntimeError("H3 sigma cache has no active modality layout") -+ prediction = self._state().predict(owner._sigma_indices) -+ return (prediction + hidden_states).contiguous(), None -+ -+ -+class MiniMaxH3SigmaCachedBlocks(MiniMaxH3CachedBlocks): -+ def __init__(self, *args, **kwargs) -> None: -+ super().__init__(*args, **kwargs) -+ self._sigma_state = None -+ self._sigma_indices = None -+ self.context_manager = _SigmaCacheManager(self.context_manager, self) -+ -+ def forward(self, hidden_states: torch.Tensor, *args, sigma_cache_state=None, **kwargs): -+ if not isinstance(sigma_cache_state, MiniMaxH3SigmaCacheState) or sigma_cache_state.closed: -+ raise RuntimeError("H3 sigma cache requires a live denoise request state") -+ if kwargs.get("num_requests", 1) != 1 or kwargs.get("reference_kv_state") is not None: -+ raise ValueError("H3 sigma cache requires a single request without reference-KV reuse") -+ old_state, old_indices = self._sigma_state, self._sigma_indices -+ try: -+ self._sigma_state = sigma_cache_state -+ self._sigma_indices = kwargs["combined_indices"] -+ return super().forward(hidden_states, *args, **kwargs) -+ finally: -+ self._sigma_state, self._sigma_indices = old_state, old_indices -+ -+ -+class MiniMaxH3SigmaCachedAdapter(MiniMaxH3CachedAdapter): -+ block_class = MiniMaxH3SigmaCachedBlocks -diff --git a/vllm_omni/diffusion/models/minimax_h3/denoise_loop.py b/vllm_omni/diffusion/models/minimax_h3/denoise_loop.py -index 8cca219..ddbdace 100644 ---- a/vllm_omni/diffusion/models/minimax_h3/denoise_loop.py -+++ b/vllm_omni/diffusion/models/minimax_h3/denoise_loop.py -@@ -379,13 +379,21 @@ def minimax_h3_denoise_loop( - if reference_kv_state is not None: - positive.static_kwargs["reference_kv_state"] = reference_kv_state - -+ sigma_cache_state = None - try: -+ create_sigma_cache_state = getattr(model, "create_sigma_cache_state", None) -+ if callable(create_sigma_cache_state): -+ sigma_cache_state = create_sigma_cache_state() -+ if sigma_cache_state is not None and (video_edit is not None or audio_edit is not None): -+ raise ValueError("H3 sigma cache does not support latent edits") - for step in range(num_steps): - check_request_cancellation() - step_cm = step_profiler(step) if step_profiler is not None else nullcontext() - with step_cm: - s_v = sigmas_video[step] - s_a = sigmas_audio[step] -+ if sigma_cache_state is not None: -+ sigma_cache_state.begin_step(step, s_v, s_a) - if reference_kv_state is not None: - reference_kv_state.begin_step(step, sigma=float(s_v)) - # Publish where we are so step-gated attention features (the dense -@@ -426,6 +434,8 @@ def minimax_h3_denoise_loop( - ) - if getattr(getattr(model, "arch", None), "pdd_steps", None) is not None: - fk["pdd_step"] = step -+ if sigma_cache_state is not None: -+ fk["sigma_cache_state"] = sigma_cache_state - with torch.inference_mode(): - v_video, v_audio = model(**fk) - if reference_kv_state is not None: -@@ -475,6 +485,9 @@ def minimax_h3_denoise_loop( - check_request_cancellation(synchronize=True) - - finally: -+ if sigma_cache_state is not None: -+ model._sigma_cache_stats = sigma_cache_state.stats() -+ sigma_cache_state.close() - minimax_h3_publish_denoise_progress(None, None, None) - if reference_kv_state is not None: - positive.static_kwargs.pop("reference_kv_state", None) -diff --git a/vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py b/vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py -index 9518828..ca727c6 100644 ---- a/vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py -+++ b/vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py -@@ -245,6 +245,7 @@ _FORWARD_SUPPORTED_KWARGS = frozenset( - "control_rows", - "control_context_scale", - "reference_kv_state", -+ "sigma_cache_state", - } - ) - -@@ -777,6 +778,23 @@ class MiniMaxH3DiTModel(nn.Module): - max_bytes=256 * 1024**2 if enabled else 0, offload_weights=offload_adaln - ) - self.od_config = od_config -+ sigma_enabled = ( -+ cache_config.get("minimax_h3_sigma_cache", False) -+ if isinstance(cache_config, Mapping) -+ else getattr(cache_config, "minimax_h3_sigma_cache", False) -+ ) -+ if type(sigma_enabled) is not bool: -+ raise ValueError("minimax_h3_sigma_cache must be a boolean") -+ self._sigma_cache_enabled = False -+ if sigma_enabled: -+ from .cache_sigma_prediction import MiniMaxH3SigmaCachedAdapter, sigma_cache_enabled -+ -+ self._sigma_cache_enabled = sigma_cache_enabled(od_config) -+ if not current_omni_platform.is_npu(): -+ raise ValueError("H3 sigma cache is currently restricted to the Ascend Ref2VA experiment") -+ self._cache_dit_adapter_config = replace( -+ self._cache_dit_adapter_config, cached_adapter_cls=MiniMaxH3SigmaCachedAdapter -+ ) - if getattr(od_config, "kv_offload_config", None) is not None: - from vllm_omni.diffusion.models.minimax_h3.reference_kv_cache_dit import MiniMaxH3CachedAdapter - -@@ -1257,6 +1275,14 @@ class MiniMaxH3DiTModel(nn.Module): - device=device, - ) - -+ def create_sigma_cache_state(self): -+ if not getattr(self, "_sigma_cache_enabled", False): -+ return None -+ from .cache_sigma_prediction import MiniMaxH3SigmaCacheState -+ -+ self._sigma_cache_stats = None -+ return MiniMaxH3SigmaCacheState() -+ - def forward(self, *, pdd_step: int | None = None, **kwargs: Any) -> tuple[torch.Tensor, torch.Tensor]: - """Packed inference forward. - -@@ -1490,6 +1516,14 @@ class MiniMaxH3DiTModel(nn.Module): - if getattr(self, "adaln_cache", None) is not None: - self.adaln_cache.prepare(t_emb) - -+ sigma_state = kwargs.get("sigma_cache_state") -+ sigma_options = {} -+ if sigma_state is not None: -+ if not getattr(self, "_sigma_cache_enabled", False): -+ raise ValueError("H3 sigma cache state was supplied while the feature is disabled") -+ if num_requests != 1 or reference_kv_state is not None or control_rows is not None: -+ raise ValueError("H3 sigma cache does not support packed batches, control or reference-KV reuse") -+ sigma_options["sigma_cache_state"] = sigma_state - for block_index, block in enumerate(self.blocks): - hidden = block( - hidden, -@@ -1505,6 +1539,7 @@ class MiniMaxH3DiTModel(nn.Module): - reference_kv_state=reference_kv_state, - reference_kv_compact=compact_reference, - vdn_window=vdn_window, -+ **sigma_options, - ) - if block_index in hints: - hidden = hidden + hints.pop(block_index) * control_scale -diff --git a/vllm_omni/diffusion/models/minimax_h3/reference_kv_cache_dit.py b/vllm_omni/diffusion/models/minimax_h3/reference_kv_cache_dit.py -index f14dcc9..f78598b 100644 ---- a/vllm_omni/diffusion/models/minimax_h3/reference_kv_cache_dit.py -+++ b/vllm_omni/diffusion/models/minimax_h3/reference_kv_cache_dit.py -@@ -125,6 +125,8 @@ class MiniMaxH3CachedBlocks(CachedBlocks_Pattern_3_4_5): - - - class MiniMaxH3CachedAdapter(CachedAdapter): -+ block_class = MiniMaxH3CachedBlocks -+ - @classmethod - def collect_unified_blocks( - cls, block_adapter: BlockAdapter, contexts_kwargs: list[dict] -@@ -139,7 +141,7 @@ class MiniMaxH3CachedAdapter(CachedAdapter): - context_index += 1 - contexts[block_adapter.unique_blocks_name[i][j]] = nn.ModuleList( - [ -- MiniMaxH3CachedBlocks( -+ cls.block_class( - blocks, - transformer=transformer, - forward_pattern=block_adapter.forward_pattern[i][j], diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c11-sigma-fp32-residual.patch b/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c11-sigma-fp32-residual.patch deleted file mode 100644 index fdbb14d3c..000000000 --- a/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c11-sigma-fp32-residual.patch +++ /dev/null @@ -1,685 +0,0 @@ -diff --git a/tests/diffusion/models/minimax_h3/test_minimax_h3_sigma_cache.py b/tests/diffusion/models/minimax_h3/test_minimax_h3_sigma_cache.py -new file mode 100644 -index 0000000..c2d577f ---- /dev/null -+++ b/tests/diffusion/models/minimax_h3/test_minimax_h3_sigma_cache.py -@@ -0,0 +1,333 @@ -+# SPDX-License-Identifier: Apache-2.0 -+# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project -+ -+from types import SimpleNamespace -+ -+import pytest -+import torch -+import torch.nn as nn -+ -+from tests.helpers.mark import hardware_marks -+from vllm_omni.diffusion.models.minimax_h3 import cache_sigma_prediction as sigma -+from vllm_omni.diffusion.models.minimax_h3.reference_kv_cache_dit import MiniMaxH3CachedBlocks -+from vllm_omni.platforms import current_omni_platform -+ -+pytestmark = [pytest.mark.core_model, pytest.mark.diffusion, *hardware_marks(res={"npu": "A3"}, num_cards=1)] -+ -+ -+def config(**changes): -+ values = dict( -+ cache_config={"minimax_h3_sigma_cache": True, "enable_taylorseer": False}, -+ task_type="ref2va", -+ cache_backend="cache_dit", -+ kv_offload_config=None, -+ parallel_config=SimpleNamespace(data_parallel_size=1, cfg_parallel_size=1), -+ ) -+ values.update(changes) -+ return SimpleNamespace(**values) -+ -+ -+def sample(video, audio, *, device="cpu", dtype=torch.float32): -+ return torch.tensor([[1 + 2 * video + video**2], [4 - audio + 3 * audio**2]], device=device, dtype=dtype) -+ -+ -+def anchored(*, device="cpu", dtype=torch.float32): -+ state = sigma.MiniMaxH3SigmaCacheState() -+ originals = [] -+ for step, (video, audio) in enumerate([(1.0, 1.0), (0.9, 0.8), (0.7, 0.5)]): -+ state.begin_step(step, video, audio) -+ residual = sample(video, audio, device=device, dtype=dtype) -+ state.record(residual) -+ originals.append(residual) -+ state.begin_step(3, 0.5, 0.3) -+ return state, originals -+ -+ -+def test_distinct_nonuniform_clocks_recover_quadratic_and_own_anchors(): -+ state, originals = anchored() -+ indices = torch.tensor([0, 17]) -+ expected = sample(0.5, 0.3) -+ assert state.can_predict(expected) -+ torch.testing.assert_close(state.predict(indices), expected) -+ saved = [value.clone() for _, value in state.history] -+ originals[-1].fill_(99) -+ assert all(torch.equal(old, value) for old, (_, value) in zip(saved, state.history)) -+ torch.testing.assert_close(state.predict(indices), expected) -+ # One shared video clock gives the wrong audio result on this fixture. -+ video_weights = sigma._quadratic_weights([1.0, 0.9, 0.7], 0.5) -+ wrong_audio = sum(weight * value[1] for weight, (_, value) in zip(video_weights, state.history)) -+ assert not torch.allclose(wrong_audio, expected[1]) -+ assert state.stats() == {"computed": 3, "predicted": 2, "history_vetoes": 0} -+ state.close() -+ assert state.closed and not state.history and not state.can_predict(expected) -+ -+ -+@pytest.mark.parametrize( -+ "anchors,target", -+ [([1.0, 0.8], 0.7), ([1.0, 0.8, 0.8], 0.7), ([1.0, 0.9, 0.8], 0.1), ([1.0, 0.9, 0.8], float("nan"))], -+) -+def test_unstable_or_insufficient_history_vetoes(anchors, target): -+ assert sigma._quadratic_weights(anchors, target) is None -+ -+ -+@pytest.mark.parametrize( -+ "changes", -+ [ -+ {"task_type": "t2va"}, -+ {"cache_backend": "none"}, -+ {"kv_offload_config": {}}, -+ {"parallel_config": SimpleNamespace(data_parallel_size=2)}, -+ {"cache_config": {"minimax_h3_sigma_cache": 1}}, -+ {"cache_config": {"minimax_h3_sigma_cache": True, "enable_taylorseer": True}}, -+ ], -+) -+def test_explicit_unsupported_selection_fails(changes): -+ with pytest.raises(ValueError): -+ sigma.sigma_cache_enabled(config(**changes)) -+ -+ -+def test_default_and_dataclass_extra_selection(): -+ from vllm_omni.diffusion.data import DiffusionCacheConfig -+ -+ assert not sigma.sigma_cache_enabled(config(cache_config={})) -+ cache = DiffusionCacheConfig.from_dict({"minimax_h3_sigma_cache": True}) -+ assert sigma.sigma_cache_enabled(config(cache_config=cache)) -+ assert sigma.MiniMaxH3SigmaCachedAdapter.block_class is sigma.MiniMaxH3SigmaCachedBlocks -+ -+ -+def test_parent_denial_and_request_prediction_do_not_mutate_inputs(): -+ state, _ = anchored() -+ owner = SimpleOwner(state, torch.tensor([0, 2])) -+ calls = [] -+ manager = SimpleNamespace( -+ can_cache=lambda *args, **kwargs: False, -+ is_cache_residual=lambda: True, -+ set_Bn_buffer=lambda *args, **kwargs: calls.append((args, kwargs)), -+ ) -+ proxy = sigma._SigmaCacheManager(manager, owner) -+ hidden = torch.ones(2, 1) -+ assert not proxy.can_cache(hidden) -+ assert state.history_vetoes == 0 -+ manager.can_cache = lambda *args, **kwargs: True -+ assert proxy.can_cache(hidden) -+ actual, encoder = proxy.apply_cache(hidden, prefix="blocks_Bn_residual") -+ torch.testing.assert_close(actual, sample(0.5, 0.3) + 1) -+ assert encoder is None and torch.equal(hidden, torch.ones_like(hidden)) -+ owner._sigma_state = sigma.MiniMaxH3SigmaCacheState() -+ assert not proxy.can_cache(hidden) -+ assert owner._sigma_state.history_vetoes == 1 -+ -+ -+class SimpleOwner: -+ def __init__(self, state, indices): -+ self._sigma_state, self._sigma_indices = state, indices -+ -+ -+def test_fp32_history_accepts_bf16_prefix_and_preserves_output_dtype(): -+ state, _ = anchored() -+ hidden = torch.ones(2, 1, dtype=torch.bfloat16) -+ assert state.can_predict(hidden) -+ owner = SimpleOwner(state, torch.tensor([0, 2])) -+ proxy = sigma._SigmaCacheManager(SimpleNamespace(is_cache_residual=lambda: True), owner) -+ actual, _ = proxy.apply_cache(hidden) -+ assert actual.dtype == hidden.dtype -+ torch.testing.assert_close(actual, (sample(0.5, 0.3) + hidden.float()).to(hidden.dtype)) -+ assert all(value.dtype == torch.float32 for _, value in state.history) -+ -+ -+def test_middle_anchor_retains_unrounded_endpoint_difference(monkeypatch): -+ before = torch.tensor([[0.0712890625]], dtype=torch.bfloat16) -+ computed = torch.tensor([[7.0]], dtype=torch.bfloat16) -+ -+ def parent(self, hidden, *args, **kwargs): -+ return computed, None, computed - hidden -+ -+ monkeypatch.setattr(MiniMaxH3CachedBlocks, "call_Mn_blocks", parent) -+ wrapper = object.__new__(sigma.MiniMaxH3SigmaCachedBlocks) -+ nn.Module.__init__(wrapper) -+ end, encoder, residual = wrapper.call_Mn_blocks(before) -+ assert end is computed and encoder is None and residual.dtype == torch.float32 -+ assert torch.equal(residual, computed.float() - before.float()) -+ assert not torch.equal((computed - before).float(), residual) -+ assert torch.equal((residual + before.float()).to(before.dtype), computed) -+ -+ -+@pytest.mark.parametrize("failure", [False, True]) -+def test_wrapper_releases_transient_request_metadata(monkeypatch, failure): -+ wrapper = object.__new__(sigma.MiniMaxH3SigmaCachedBlocks) -+ nn.Module.__init__(wrapper) -+ wrapper._sigma_state, wrapper._sigma_indices = None, None -+ state = sigma.MiniMaxH3SigmaCacheState() -+ -+ def parent(self, hidden, *args, **kwargs): -+ assert "sigma_cache_state" not in kwargs -+ assert self._sigma_state is state -+ if failure: -+ raise RuntimeError("block failed") -+ return hidden -+ -+ monkeypatch.setattr(MiniMaxH3CachedBlocks, "forward", parent) -+ args = dict(sigma_cache_state=state, combined_indices=torch.tensor([0, 2])) -+ if failure: -+ with pytest.raises(RuntimeError, match="block failed"): -+ wrapper.forward(torch.ones(2, 1), **args) -+ else: -+ wrapper.forward(torch.ones(2, 1), **args) -+ assert wrapper._sigma_state is None and wrapper._sigma_indices is None -+ -+ -+@pytest.mark.parametrize("failure", ["success", "model", "cancel"]) -+def test_denoise_request_closes_history_on_all_exit_paths(monkeypatch, failure): -+ from vllm_omni.diffusion.models.minimax_h3 import denoise_loop as loop -+ -+ monkeypatch.setattr(loop.current_omni_platform, "is_npu", lambda: False) -+ monkeypatch.setattr(loop, "minimax_h3_publish_denoise_progress", lambda *args: None) -+ state = sigma.MiniMaxH3SigmaCacheState() -+ -+ class Model: -+ def create_sigma_cache_state(self): -+ return state -+ -+ def __call__(self, **kwargs): -+ assert kwargs["sigma_cache_state"] is state -+ state.record(torch.ones(2, 1)) -+ if failure == "model": -+ raise RuntimeError("model failed") -+ return torch.zeros(1, 96), torch.zeros(1, 32) -+ -+ class Branch: -+ update_mask = torch.ones(1, dtype=torch.bool) -+ update_mask_dev = update_mask -+ audio_update_mask_dev = update_mask -+ locked_audio_rows = None -+ -+ def forward_kwargs(self, **kwargs): -+ return {} -+ -+ monkeypatch.setattr( -+ loop, -+ "minimax_h3_prepare_denoise_rows", -+ lambda **kwargs: (kwargs["initial_video_rows"], kwargs["initial_audio_rows"], None, None), -+ ) -+ -+ def cancellation(**kwargs): -+ if failure == "cancel": -+ raise RuntimeError("request cancelled") -+ -+ monkeypatch.setattr(loop, "check_request_cancellation", cancellation) -+ model = Model() -+ kwargs = dict( -+ model=model, -+ positive=Branch(), -+ initial_video_rows=torch.zeros(1, 96), -+ initial_audio_rows=torch.zeros(1, 32), -+ keyframe_cond_rows=None, -+ sigmas_video=[1.0, 0.0], -+ sigmas_audio=[1.0, 0.0], -+ device=torch.device("cpu"), -+ ) -+ if failure == "success": -+ loop.minimax_h3_denoise_loop(**kwargs) -+ else: -+ with pytest.raises(RuntimeError): -+ loop.minimax_h3_denoise_loop(**kwargs) -+ assert state.closed and not state.history -+ assert model._sigma_cache_stats["computed"] == (0 if failure == "cancel" else 1) -+ -+ -+@pytest.mark.skipif(not current_omni_platform.is_npu(), reason="Requires native Ascend forecasting") -+def test_native_two_clock_forecast_and_input_immutability(): -+ device = torch.device(current_omni_platform.device_type) -+ state, originals = anchored(device=device, dtype=torch.bfloat16) -+ indices = torch.tensor([0, 2], device=device) -+ saved = [value.clone() for value in originals] -+ expected = sample(0.5, 0.3, device=device, dtype=torch.bfloat16) -+ assert state.can_predict(expected) -+ actual = state.predict(indices) -+ # BF16 anchor quantization is separate from exact polynomial algebra. -+ torch.testing.assert_close(actual.float(), expected.float(), atol=0.0625, rtol=0) -+ assert all(torch.equal(value, old) for value, old in zip(originals, saved)) -+ assert state.stats()["predicted"] == 1 -+ state.close() -+ assert not state.history -+ -+ -+def _native_real_cache_manager(dtype): -+ from cache_dit import DBCacheConfig, ForwardPattern -+ from cache_dit.caching.cache_contexts.cache_manager import CachedContextManager -+ -+ class Block(nn.Module): -+ def __init__(self, prefix=False): -+ super().__init__() -+ self.prefix = prefix -+ self.calls = 0 -+ -+ def forward(self, hidden_states, *, test_sigmas, **kwargs): -+ self.calls += 1 -+ return hidden_states + ( -+ 0.125 if self.prefix else sample(*test_sigmas, device=hidden_states.device, dtype=hidden_states.dtype) -+ ) -+ -+ manager = CachedContextManager() -+ context = manager.new_context( -+ name="h3-sigma-native", -+ cache_config=DBCacheConfig( -+ Fn_compute_blocks=1, -+ Bn_compute_blocks=0, -+ max_warmup_steps=4, -+ num_inference_steps=5, -+ residual_diff_threshold=0.08, -+ max_continuous_cached_steps=3, -+ ), -+ ) -+ blocks = nn.ModuleList([Block(prefix=True), Block(), Block()]) -+ wrapper = sigma.MiniMaxH3SigmaCachedBlocks( -+ blocks, -+ forward_pattern=ForwardPattern.Pattern_3, -+ check_forward_pattern=False, -+ cache_prefix="blocks", -+ cache_context=context, -+ context_manager=manager, -+ ) -+ device = torch.device(current_omni_platform.device_type) -+ indices = torch.tensor([0, 2], device=device) -+ state = sigma.MiniMaxH3SigmaCacheState() -+ for step, clocks in enumerate([(1.0, 1.0), (0.95, 0.9), (0.85, 0.75), (0.7, 0.55), (0.5, 0.35)]): -+ state.begin_step(step, *clocks) -+ hidden = torch.ones(2, 1, device=device, dtype=dtype) -+ actual = wrapper(hidden, sigma_cache_state=state, combined_indices=indices, test_sigmas=clocks) -+ assert actual.dtype == hidden.dtype -+ assert all(value.dtype == torch.float32 for _, value in state.history) -+ if step < 4: -+ delta = sample(*clocks, device=device, dtype=dtype) -+ expected = hidden + 0.125 + delta + delta -+ torch.testing.assert_close(actual, expected, atol=0, rtol=0) -+ else: -+ # BF16 full-compute endpoints are quantized, so they do not lie -+ # exactly on the analytic polynomial. Fit the real anchors with an -+ # independent FP64 Vandermonde solve rather than ignoring that error. -+ forecast = torch.empty(2, 1, dtype=torch.float64) -+ for row, modality in enumerate((0, 1)): -+ anchors = [clock[modality] for clock, _ in state.history] -+ matrix = torch.tensor([[1, x, x * x] for x in anchors], dtype=torch.float64) -+ values = torch.stack([value[row].cpu().double() for _, value in state.history]) -+ coefficients = torch.linalg.solve(matrix, values) -+ target = clocks[modality] -+ forecast[row] = torch.tensor([1, target, target * target], dtype=torch.float64) @ coefficients -+ expected = (hidden.cpu().double() + 0.125 + forecast).to(device=device, dtype=dtype) -+ tolerance = 0.03125 if dtype == torch.bfloat16 else 1e-5 -+ torch.testing.assert_close(actual, expected, atol=tolerance, rtol=1e-5) -+ assert wrapper._sigma_state is None and wrapper._sigma_indices is None -+ assert blocks[0].calls == 5 and blocks[1].calls == blocks[2].calls == 4 -+ assert state.stats()["computed"] == 4 and state.stats()["predicted"] == 1 -+ state.close() -+ -+ -+@pytest.mark.skipif(not current_omni_platform.is_npu(), reason="Requires native Ascend cached block execution") -+def test_native_real_cache_manager_computes_then_forecasts(): -+ _native_real_cache_manager(torch.float32) -+ -+ -+@pytest.mark.skipif(not current_omni_platform.is_npu(), reason="Requires native BF16 blocks with FP32 residual history") -+def test_native_bf16_blocks_use_fp32_anchors_and_keep_output_dtype(): -+ _native_real_cache_manager(torch.bfloat16) -diff --git a/vllm_omni/diffusion/models/minimax_h3/cache_sigma_prediction.py b/vllm_omni/diffusion/models/minimax_h3/cache_sigma_prediction.py -new file mode 100644 -index 0000000..cfea2a8 ---- /dev/null -+++ b/vllm_omni/diffusion/models/minimax_h3/cache_sigma_prediction.py -@@ -0,0 +1,199 @@ -+# SPDX-License-Identifier: Apache-2.0 -+# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project -+"""Opt-in, request-owned residual forecasting on H3's two sigma clocks.""" -+ -+from __future__ import annotations -+ -+import math -+from collections.abc import Mapping -+from typing import Any -+from weakref import ref -+ -+import torch -+ -+from .reference_kv_cache_dit import MiniMaxH3CachedAdapter, MiniMaxH3CachedBlocks -+ -+ -+def sigma_cache_enabled(config: Any) -> bool: -+ cache = getattr(config, "cache_config", {}) -+ enabled = ( -+ cache.get("minimax_h3_sigma_cache", False) -+ if isinstance(cache, Mapping) -+ else getattr(cache, "minimax_h3_sigma_cache", False) -+ ) -+ if type(enabled) is not bool: -+ raise ValueError("minimax_h3_sigma_cache must be a boolean") -+ if not enabled: -+ return False -+ parallel = getattr(config, "parallel_config", None) -+ if ( -+ getattr(config, "task_type", None) != "ref2va" -+ or getattr(config, "cache_backend", None) != "cache_dit" -+ or getattr(config, "kv_offload_config", None) is not None -+ or (getattr(parallel, "data_parallel_size", 1) or 1) != 1 -+ or (getattr(parallel, "cfg_parallel_size", 1) or 1) != 1 -+ ): -+ raise ValueError("H3 sigma cache requires Ref2VA Cache-DiT, DP1/CFG1 and no reference-KV cache") -+ taylor = cache.get("enable_taylorseer", False) if isinstance(cache, Mapping) else cache.enable_taylorseer -+ if taylor: -+ raise ValueError("H3 sigma cache replaces TaylorSeer; disable enable_taylorseer") -+ return True -+ -+ -+def _quadratic_weights(anchors: list[float], target: float) -> tuple[float, ...] | None: -+ if len(anchors) != 3 or not all(math.isfinite(x) for x in [*anchors, target]): -+ return None -+ if not anchors[0] > anchors[1] > anchors[2] > target: -+ return None -+ weights = [] -+ for i, anchor in enumerate(anchors): -+ weight = 1.0 -+ for j, other in enumerate(anchors): -+ if i != j: -+ weight *= (target - other) / (anchor - other) -+ weights.append(weight) -+ # A host-only veto: all SP ranks have the same clocks and anchor steps. -+ if not all(math.isfinite(x) for x in weights) or sum(abs(x) for x in weights) > 64.0: -+ return None -+ return tuple(weights) -+ -+ -+class MiniMaxH3SigmaCacheState: -+ """Three real middle-residual anchors, owned by one denoise request.""" -+ -+ def __init__(self) -> None: -+ self.step = -1 -+ self.sigmas = (1.0, 1.0) -+ self.history: list[tuple[tuple[float, float], torch.Tensor]] = [] -+ self.closed = False -+ self.computed = 0 -+ self.predicted = 0 -+ self.history_vetoes = 0 -+ -+ def begin_step(self, step: int, video_sigma: float, audio_sigma: float) -> None: -+ if self.closed or step != self.step + 1: -+ raise RuntimeError("H3 sigma cache requires consecutive steps in an open request") -+ sigmas = (float(video_sigma), float(audio_sigma)) -+ if not all(math.isfinite(x) for x in sigmas): -+ raise ValueError("H3 sigma cache requires finite sigma clocks") -+ if self.step >= 0 and not all(x < old for x, old in zip(sigmas, self.sigmas)): -+ raise ValueError("H3 sigma cache requires descending sigma clocks") -+ self.step, self.sigmas = step, sigmas -+ -+ def _weights(self) -> tuple[tuple[float, ...], tuple[float, ...]] | None: -+ if self.closed or len(self.history) != 3: -+ return None -+ video = _quadratic_weights([clock[0] for clock, _ in self.history], self.sigmas[0]) -+ audio = _quadratic_weights([clock[1] for clock, _ in self.history], self.sigmas[1]) -+ return None if video is None or audio is None else (video, audio) -+ -+ def can_predict(self, current: torch.Tensor) -> bool: -+ if self._weights() is None: -+ self.history_vetoes += 1 -+ return False -+ if any(tensor.shape != current.shape or tensor.device != current.device for _, tensor in self.history): -+ self.history.clear() -+ self.history_vetoes += 1 -+ return False -+ return True -+ -+ def record(self, residual: torch.Tensor) -> None: -+ if self.closed or self.step < 0 or residual.ndim != 2: -+ raise RuntimeError("H3 sigma cache requires an open step and a two-dimensional residual") -+ if self.history and self.history[-1][0] == self.sigmas: -+ raise RuntimeError("H3 sigma cache received duplicate middle-block residuals") -+ # Own the anchor: neither Cache-DiT nor the model may mutate our history. -+ self.history.append((self.sigmas, residual.detach().clone())) -+ del self.history[:-3] -+ self.computed += 1 -+ -+ def predict(self, combined_indices: torch.Tensor) -> torch.Tensor: -+ weights = self._weights() -+ if weights is None: -+ raise RuntimeError("H3 sigma cache cannot forecast without three stable anchors") -+ last = self.history[-1][1] -+ if combined_indices.ndim != 1 or combined_indices.shape[0] != last.shape[0]: -+ raise ValueError("H3 sigma cache requires one modality index per local residual row") -+ if combined_indices.device != last.device: -+ raise ValueError("H3 sigma cache modality and residual devices must match") -+ audio_rows = combined_indices.remainder(3).eq(2).unsqueeze(-1) -+ output = torch.zeros_like(last, dtype=torch.float32) -+ for i, (_, residual) in enumerate(self.history): -+ coefficient = torch.where(audio_rows, weights[1][i], weights[0][i]) -+ output.add_(residual.float() * coefficient) -+ self.predicted += 1 -+ return output -+ -+ def stats(self) -> dict[str, int]: -+ return {"computed": self.computed, "predicted": self.predicted, "history_vetoes": self.history_vetoes} -+ -+ def close(self) -> None: -+ self.history.clear() -+ self.closed = True -+ -+ -+class _SigmaCacheManager: -+ def __init__(self, manager: Any, owner: MiniMaxH3SigmaCachedBlocks) -> None: -+ self._manager = manager -+ self._owner = ref(owner) -+ -+ def __getattr__(self, name: str) -> Any: -+ return getattr(self._manager, name) -+ -+ def _state(self) -> MiniMaxH3SigmaCacheState: -+ owner = self._owner() -+ if owner is None or owner._sigma_state is None: -+ raise RuntimeError("H3 sigma cache manager has no active request") -+ return owner._sigma_state -+ -+ def can_cache(self, states_tensor: torch.Tensor, *args, **kwargs) -> bool: -+ allowed = self._manager.can_cache(states_tensor, *args, **kwargs) -+ return allowed and self._state().can_predict(states_tensor) -+ -+ def set_Bn_buffer(self, buffer: torch.Tensor, prefix: str = "Bn") -> None: -+ if not self._manager.is_cache_residual(): -+ raise ValueError("H3 sigma cache requires middle-block residual caching") -+ self._state().record(buffer) -+ self._manager.set_Bn_buffer(buffer, prefix=prefix) -+ -+ def apply_cache(self, hidden_states: torch.Tensor, encoder_hidden_states=None, **kwargs): -+ if encoder_hidden_states is not None or not self._manager.is_cache_residual(): -+ raise ValueError("H3 sigma cache supports only the packed H3 residual stream") -+ owner = self._owner() -+ if owner is None or owner._sigma_indices is None: -+ raise RuntimeError("H3 sigma cache has no active modality layout") -+ prediction = self._state().predict(owner._sigma_indices) -+ output = (prediction + hidden_states.float()).to(hidden_states.dtype) -+ return output.contiguous(), None -+ -+ -+class MiniMaxH3SigmaCachedBlocks(MiniMaxH3CachedBlocks): -+ def __init__(self, *args, **kwargs) -> None: -+ super().__init__(*args, **kwargs) -+ self._sigma_state = None -+ self._sigma_indices = None -+ self.context_manager = _SigmaCacheManager(self.context_manager, self) -+ -+ def call_Mn_blocks(self, hidden_states: torch.Tensor, *args, **kwargs): -+ computed, encoder, _ = super().call_Mn_blocks(hidden_states, *args, **kwargs) -+ # Fit the exact difference of the computed BF16 endpoints. Reconstruct -+ # in FP32 and cast once so downstream blocks retain their input dtype. -+ residual = computed.float() - hidden_states.float() -+ return computed, encoder, residual -+ -+ def forward(self, hidden_states: torch.Tensor, *args, sigma_cache_state=None, **kwargs): -+ if not isinstance(sigma_cache_state, MiniMaxH3SigmaCacheState) or sigma_cache_state.closed: -+ raise RuntimeError("H3 sigma cache requires a live denoise request state") -+ if kwargs.get("num_requests", 1) != 1 or kwargs.get("reference_kv_state") is not None: -+ raise ValueError("H3 sigma cache requires a single request without reference-KV reuse") -+ old_state, old_indices = self._sigma_state, self._sigma_indices -+ try: -+ self._sigma_state = sigma_cache_state -+ self._sigma_indices = kwargs["combined_indices"] -+ return super().forward(hidden_states, *args, **kwargs) -+ finally: -+ self._sigma_state, self._sigma_indices = old_state, old_indices -+ -+ -+class MiniMaxH3SigmaCachedAdapter(MiniMaxH3CachedAdapter): -+ block_class = MiniMaxH3SigmaCachedBlocks -diff --git a/vllm_omni/diffusion/models/minimax_h3/denoise_loop.py b/vllm_omni/diffusion/models/minimax_h3/denoise_loop.py -index 8cca219..ddbdace 100644 ---- a/vllm_omni/diffusion/models/minimax_h3/denoise_loop.py -+++ b/vllm_omni/diffusion/models/minimax_h3/denoise_loop.py -@@ -379,13 +379,21 @@ def minimax_h3_denoise_loop( - if reference_kv_state is not None: - positive.static_kwargs["reference_kv_state"] = reference_kv_state - -+ sigma_cache_state = None - try: -+ create_sigma_cache_state = getattr(model, "create_sigma_cache_state", None) -+ if callable(create_sigma_cache_state): -+ sigma_cache_state = create_sigma_cache_state() -+ if sigma_cache_state is not None and (video_edit is not None or audio_edit is not None): -+ raise ValueError("H3 sigma cache does not support latent edits") - for step in range(num_steps): - check_request_cancellation() - step_cm = step_profiler(step) if step_profiler is not None else nullcontext() - with step_cm: - s_v = sigmas_video[step] - s_a = sigmas_audio[step] -+ if sigma_cache_state is not None: -+ sigma_cache_state.begin_step(step, s_v, s_a) - if reference_kv_state is not None: - reference_kv_state.begin_step(step, sigma=float(s_v)) - # Publish where we are so step-gated attention features (the dense -@@ -426,6 +434,8 @@ def minimax_h3_denoise_loop( - ) - if getattr(getattr(model, "arch", None), "pdd_steps", None) is not None: - fk["pdd_step"] = step -+ if sigma_cache_state is not None: -+ fk["sigma_cache_state"] = sigma_cache_state - with torch.inference_mode(): - v_video, v_audio = model(**fk) - if reference_kv_state is not None: -@@ -475,6 +485,9 @@ def minimax_h3_denoise_loop( - check_request_cancellation(synchronize=True) - - finally: -+ if sigma_cache_state is not None: -+ model._sigma_cache_stats = sigma_cache_state.stats() -+ sigma_cache_state.close() - minimax_h3_publish_denoise_progress(None, None, None) - if reference_kv_state is not None: - positive.static_kwargs.pop("reference_kv_state", None) -diff --git a/vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py b/vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py -index 9518828..ca727c6 100644 ---- a/vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py -+++ b/vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py -@@ -245,6 +245,7 @@ _FORWARD_SUPPORTED_KWARGS = frozenset( - "control_rows", - "control_context_scale", - "reference_kv_state", -+ "sigma_cache_state", - } - ) - -@@ -777,6 +778,23 @@ class MiniMaxH3DiTModel(nn.Module): - max_bytes=256 * 1024**2 if enabled else 0, offload_weights=offload_adaln - ) - self.od_config = od_config -+ sigma_enabled = ( -+ cache_config.get("minimax_h3_sigma_cache", False) -+ if isinstance(cache_config, Mapping) -+ else getattr(cache_config, "minimax_h3_sigma_cache", False) -+ ) -+ if type(sigma_enabled) is not bool: -+ raise ValueError("minimax_h3_sigma_cache must be a boolean") -+ self._sigma_cache_enabled = False -+ if sigma_enabled: -+ from .cache_sigma_prediction import MiniMaxH3SigmaCachedAdapter, sigma_cache_enabled -+ -+ self._sigma_cache_enabled = sigma_cache_enabled(od_config) -+ if not current_omni_platform.is_npu(): -+ raise ValueError("H3 sigma cache is currently restricted to the Ascend Ref2VA experiment") -+ self._cache_dit_adapter_config = replace( -+ self._cache_dit_adapter_config, cached_adapter_cls=MiniMaxH3SigmaCachedAdapter -+ ) - if getattr(od_config, "kv_offload_config", None) is not None: - from vllm_omni.diffusion.models.minimax_h3.reference_kv_cache_dit import MiniMaxH3CachedAdapter - -@@ -1257,6 +1275,14 @@ class MiniMaxH3DiTModel(nn.Module): - device=device, - ) - -+ def create_sigma_cache_state(self): -+ if not getattr(self, "_sigma_cache_enabled", False): -+ return None -+ from .cache_sigma_prediction import MiniMaxH3SigmaCacheState -+ -+ self._sigma_cache_stats = None -+ return MiniMaxH3SigmaCacheState() -+ - def forward(self, *, pdd_step: int | None = None, **kwargs: Any) -> tuple[torch.Tensor, torch.Tensor]: - """Packed inference forward. - -@@ -1490,6 +1516,14 @@ class MiniMaxH3DiTModel(nn.Module): - if getattr(self, "adaln_cache", None) is not None: - self.adaln_cache.prepare(t_emb) - -+ sigma_state = kwargs.get("sigma_cache_state") -+ sigma_options = {} -+ if sigma_state is not None: -+ if not getattr(self, "_sigma_cache_enabled", False): -+ raise ValueError("H3 sigma cache state was supplied while the feature is disabled") -+ if num_requests != 1 or reference_kv_state is not None or control_rows is not None: -+ raise ValueError("H3 sigma cache does not support packed batches, control or reference-KV reuse") -+ sigma_options["sigma_cache_state"] = sigma_state - for block_index, block in enumerate(self.blocks): - hidden = block( - hidden, -@@ -1505,6 +1539,7 @@ class MiniMaxH3DiTModel(nn.Module): - reference_kv_state=reference_kv_state, - reference_kv_compact=compact_reference, - vdn_window=vdn_window, -+ **sigma_options, - ) - if block_index in hints: - hidden = hidden + hints.pop(block_index) * control_scale -diff --git a/vllm_omni/diffusion/models/minimax_h3/reference_kv_cache_dit.py b/vllm_omni/diffusion/models/minimax_h3/reference_kv_cache_dit.py -index f14dcc9..f78598b 100644 ---- a/vllm_omni/diffusion/models/minimax_h3/reference_kv_cache_dit.py -+++ b/vllm_omni/diffusion/models/minimax_h3/reference_kv_cache_dit.py -@@ -125,6 +125,8 @@ class MiniMaxH3CachedBlocks(CachedBlocks_Pattern_3_4_5): - - - class MiniMaxH3CachedAdapter(CachedAdapter): -+ block_class = MiniMaxH3CachedBlocks -+ - @classmethod - def collect_unified_blocks( - cls, block_adapter: BlockAdapter, contexts_kwargs: list[dict] -@@ -139,7 +141,7 @@ class MiniMaxH3CachedAdapter(CachedAdapter): - context_index += 1 - contexts[block_adapter.unique_blocks_name[i][j]] = nn.ModuleList( - [ -- MiniMaxH3CachedBlocks( -+ cls.block_class( - blocks, - transformer=transformer, - forward_pattern=block_adapter.forward_pattern[i][j], diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c11-source-identity.json b/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c11-source-identity.json deleted file mode 100644 index 47ec5b291..000000000 --- a/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c11-source-identity.json +++ /dev/null @@ -1,22 +0,0 @@ -{ - "base": "922f1e9f1cb85b9f2b66d86e1eb500e6872b2dcb", - "head": "61b8efa06e88372bc5ab87b9e2668a133eaac282", - "tree": "b5f29f2e136e17b23744f07fe65664237fdae627", - "files": { - "vllm_omni/diffusion/models/minimax_h3/cache_sigma_prediction.py": "ac29ef338d569529e40f59210c89a57dd2e07c510b5e2952c0da451f183da52a", - "vllm_omni/diffusion/models/minimax_h3/reference_kv_cache_dit.py": "d3381079ff012f32176a917bb0825d12288c058d3a058f5169ecc97d1e2c675c", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "a649a09e5b59256b5887c12fe508d8e14e3dedd9019bdb82c65db2d59f11de91", - "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "a9524734a4f3410a0c6f766fa64fa9e5bd5a4d41bd7481738e9024cac54ce7f9", - "tests/diffusion/models/minimax_h3/test_minimax_h3_sigma_cache.py": "9ba094c0bc0686e8d044ebe064beaa2606a6cad496449e63ec282411bb2e727a" - }, - "native_tests_passed": 20, - "native_test_names": [ - "test_native_two_clock_forecast_and_input_immutability", - "test_native_real_cache_manager_computes_then_forecasts" - ], - "source_unchanged": true, - "cleanup_passed": true, - "published": false, - "full_model_quality": "original Fn1Bn0 failed5s-00 adherence; Fn1Bn1 failed5s-00 audiolevel", - "performance_acceptance": "PENDING" -} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c12-candidate.patch b/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c12-candidate.patch deleted file mode 100644 index f18f79855..000000000 --- a/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c12-candidate.patch +++ /dev/null @@ -1,633 +0,0 @@ -diff --git a/tests/diffusion/models/minimax_h3/test_minimax_h3_state_cache.py b/tests/diffusion/models/minimax_h3/test_minimax_h3_state_cache.py -new file mode 100644 -index 0000000..6896e2c ---- /dev/null -+++ b/tests/diffusion/models/minimax_h3/test_minimax_h3_state_cache.py -@@ -0,0 +1,252 @@ -+# SPDX-License-Identifier: Apache-2.0 -+# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project -+ -+from datetime import timedelta -+from types import SimpleNamespace -+ -+import pytest -+import torch -+import torch.distributed as dist -+import torch.nn as nn -+ -+from tests.helpers.mark import hardware_marks -+from vllm_omni.diffusion.models.minimax_h3 import cache_state_alignment as aligned -+from vllm_omni.diffusion.models.minimax_h3.reference_kv_cache_dit import MiniMaxH3CachedBlocks -+from vllm_omni.platforms import current_omni_platform -+ -+pytestmark = [pytest.mark.core_model, pytest.mark.diffusion, *hardware_marks(res={"npu": "A3"}, num_cards=4)] -+ -+ -+def feature(t, device="cpu", dtype=torch.float32): -+ return torch.tensor([[t, t * t, 5.0]], device=device, dtype=dtype) -+ -+ -+def history(device="cpu", group=2): -+ state = aligned.MiniMaxH3AlignedCacheState() -+ state.set_groups(torch.tensor([group], device=device)) -+ for step in range(3): -+ state.begin_step(step) -+ f = feature(step, device) -+ state.record(f, 2 * f + 3) -+ state.begin_step(3) -+ return state -+ -+ -+def test_feature_forecast_uses_current_state_and_owns_anchors(): -+ state = history() -+ current = feature(2.5) -+ assert state.can_predict(current, parallelized=False) -+ torch.testing.assert_close(state.predict(current), 3 * current + 3, atol=0.01, rtol=0) -+ saved = [f.clone() for f, _ in state.history] -+ current.fill_(99) -+ assert all(torch.equal(a, b) for a, (b, _) in zip(saved, state.history)) -+ state.close() -+ assert state.closed and not state.history and state.groups is None and state.coefficients is None -+ -+ -+@pytest.mark.parametrize("kind", ["missing", "out_of_span", "unstable", "nonfinite"]) -+def test_invalid_prediction_is_vetoed(kind): -+ state = history() -+ current = feature(2.5) -+ if kind == "missing": -+ state.history.pop() -+ elif kind == "out_of_span": -+ current[0, 2] = 100 -+ elif kind == "unstable": -+ current = feature(30) -+ else: -+ current[0, 0] = float("nan") -+ assert not state.can_predict(current, parallelized=False) -+ assert state.vetoes == 1 -+ -+ -+def test_structural_groups_preserve_reference_and_exclude_padding(): -+ state = aligned.MiniMaxH3AlignedCacheState() -+ branch = SimpleNamespace( -+ seq_len=7, -+ device="cpu", -+ text_pos_dev=torch.tensor([0]), -+ img_pos_dev=torch.tensor([1, 2]), -+ update_mask_dev=torch.tensor([False, True]), -+ audio_pos_dev=torch.tensor([3, 4]), -+ audio_update_mask_dev=torch.tensor([False, True]), -+ ) -+ state.configure(branch) -+ assert state.groups.tolist() == [0, 1, 2, 3, 4, -1, -1] -+ for step in range(3): -+ state.begin_step(step) -+ f = feature(step).expand(7, -1).clone() -+ state.record(f, 2 * f + 3) -+ state.begin_step(3) -+ current = feature(2.5).expand(7, -1).clone() -+ current[5:] = 1000 -+ assert state.can_predict(current, parallelized=False) -+ torch.testing.assert_close(state.predict(current)[:5], (3 * current + 3)[:5], atol=0.01, rtol=0) -+ -+ -+@pytest.mark.parametrize("failure", [False, True]) -+def test_wrapper_releases_request_and_does_not_forward_private_state(monkeypatch, failure): -+ wrapper = object.__new__(aligned.MiniMaxH3AlignedCachedBlocks) -+ nn.Module.__init__(wrapper) -+ wrapper._alignment_state = wrapper._alignment_prefix = None -+ state = history() -+ -+ def parent(self, hidden, **kwargs): -+ assert "alignment_cache_state" not in kwargs and self._alignment_state is state -+ if failure: -+ raise RuntimeError("test failure") -+ return hidden -+ -+ monkeypatch.setattr(MiniMaxH3CachedBlocks, "forward", parent) -+ if failure: -+ with pytest.raises(RuntimeError, match="test failure"): -+ wrapper.forward(feature(2.5), alignment_cache_state=state) -+ else: -+ wrapper.forward(feature(2.5), alignment_cache_state=state) -+ assert wrapper._alignment_state is None and wrapper._alignment_prefix is None -+ -+ -+def test_default_and_unsupported_configuration(): -+ config = SimpleNamespace(cache_config={}) -+ assert not aligned.alignment_cache_enabled(config) -+ config.cache_config = {"minimax_h3_alignment_cache": True} -+ with pytest.raises(ValueError): -+ aligned.alignment_cache_enabled(config) -+ -+ -+@pytest.mark.skipif(not current_omni_platform.is_npu(), reason="Requires native Ascend execution") -+def test_native_bf16_and_actual_cache_wrapper(): -+ from cache_dit import DBCacheConfig, ForwardPattern -+ from cache_dit.caching.cache_contexts.cache_manager import CachedContextManager -+ -+ class Block(nn.Module): -+ def __init__(self, prefix=False): -+ super().__init__() -+ self.prefix = prefix -+ self.calls = 0 -+ -+ def forward(self, hidden, **kwargs): -+ self.calls += 1 -+ return hidden + (0.125 if self.prefix else 1.0) -+ -+ manager = CachedContextManager() -+ context = manager.new_context( -+ name="h3-alignment-native", -+ cache_config=DBCacheConfig( -+ Fn_compute_blocks=1, -+ Bn_compute_blocks=0, -+ max_warmup_steps=4, -+ num_inference_steps=5, -+ residual_diff_threshold=0.15, -+ max_continuous_cached_steps=4, -+ ), -+ ) -+ blocks = nn.ModuleList([Block(True), Block(), Block()]) -+ wrapper = aligned.MiniMaxH3AlignedCachedBlocks( -+ blocks, -+ forward_pattern=ForwardPattern.Pattern_3, -+ check_forward_pattern=False, -+ cache_prefix="blocks", -+ cache_context=context, -+ context_manager=manager, -+ ) -+ state = aligned.MiniMaxH3AlignedCacheState() -+ state.set_groups(torch.tensor([2], device="npu")) -+ for step in range(5): -+ state.begin_step(step) -+ f = torch.ones(1, 3, dtype=torch.bfloat16, device="npu") -+ output = wrapper(f, alignment_cache_state=state) -+ assert torch.equal(output, f + 2.125) and output.dtype == f.dtype -+ assert wrapper._alignment_state is None and wrapper._alignment_prefix is None -+ assert state.computed == 4 and state.predicted == 1 -+ assert blocks[1].calls == 4 and blocks[0].calls == 5 -+ state.close() -+ -+ -+def _distributed_worker(rank): -+ import torch_npu # noqa: F401 -+ -+ torch.npu.set_device(rank) -+ dist.init_process_group( -+ "hccl", init_method="tcp://127.0.0.1:29597", rank=rank, world_size=4, timeout=timedelta(seconds=120) -+ ) -+ try: -+ # Only rank3 has audio; all ranks have empty reference-audio groups. -+ state = history(device=f"npu:{rank}", group=[0, 1, 2, 4][rank]) -+ current = feature(2.5, device=f"npu:{rank}", dtype=torch.bfloat16) -+ assert state.can_predict(current, parallelized=True) -+ torch.testing.assert_close(state.predict(current).float(), 3 * current.float() + 3, atol=0.0625, rtol=0) -+ state.close() -+ assert state.groups is None and not state.history -+ dist.barrier() -+ finally: -+ dist.destroy_process_group() -+ -+ -+@pytest.mark.skipif(not current_omni_platform.is_npu(), reason="Requires four native Ascend devices") -+def test_native_four_rank_empty_group_collectives(): -+ torch.multiprocessing.spawn(_distributed_worker, nprocs=4, join=True) -+ -+ -+@pytest.mark.parametrize("failure", ["success", "model", "cancel"]) -+def test_denoise_request_closes_history_on_all_exit_paths(monkeypatch, failure): -+ from vllm_omni.diffusion.models.minimax_h3 import denoise_loop as loop -+ -+ monkeypatch.setattr(loop.current_omni_platform, "is_npu", lambda: False) -+ monkeypatch.setattr(loop, "minimax_h3_publish_denoise_progress", lambda *args: None) -+ state = aligned.MiniMaxH3AlignedCacheState() -+ -+ class Model: -+ def create_alignment_cache_state(self): -+ return state -+ -+ def __call__(self, **kwargs): -+ assert kwargs["alignment_cache_state"] is state -+ state.record(torch.ones(2, 1), torch.ones(2, 1)) -+ if failure == "model": -+ raise RuntimeError("model failed") -+ return torch.zeros(1, 96), torch.zeros(1, 32) -+ -+ class Branch: -+ seq_len = 2 -+ device = torch.device("cpu") -+ text_pos_dev = torch.tensor([], dtype=torch.long) -+ img_pos_dev = torch.tensor([0]) -+ audio_pos_dev = torch.tensor([1]) -+ update_mask = torch.ones(1, dtype=torch.bool) -+ update_mask_dev = update_mask -+ audio_update_mask_dev = update_mask -+ locked_audio_rows = None -+ -+ def forward_kwargs(self, **kwargs): -+ return {} -+ -+ monkeypatch.setattr( -+ loop, -+ "minimax_h3_prepare_denoise_rows", -+ lambda **kwargs: (kwargs["initial_video_rows"], kwargs["initial_audio_rows"], None, None), -+ ) -+ -+ def cancellation(**kwargs): -+ if failure == "cancel": -+ raise RuntimeError("request cancelled") -+ -+ monkeypatch.setattr(loop, "check_request_cancellation", cancellation) -+ model = Model() -+ kwargs = dict( -+ model=model, -+ positive=Branch(), -+ initial_video_rows=torch.zeros(1, 96), -+ initial_audio_rows=torch.zeros(1, 32), -+ keyframe_cond_rows=None, -+ sigmas_video=[1.0, 0.0], -+ sigmas_audio=[1.0, 0.0], -+ device=torch.device("cpu"), -+ ) -+ if failure == "success": -+ loop.minimax_h3_denoise_loop(**kwargs) -+ else: -+ with pytest.raises(RuntimeError): -+ loop.minimax_h3_denoise_loop(**kwargs) -+ assert state.closed and not state.history -+ assert model._alignment_cache_stats["computed"] == (0 if failure == "cancel" else 1) -diff --git a/vllm_omni/diffusion/models/minimax_h3/cache_state_alignment.py b/vllm_omni/diffusion/models/minimax_h3/cache_state_alignment.py -new file mode 100644 -index 0000000..af1c360 ---- /dev/null -+++ b/vllm_omni/diffusion/models/minimax_h3/cache_state_alignment.py -@@ -0,0 +1,226 @@ -+# SPDX-License-Identifier: Apache-2.0 -+# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project -+"""Experimental request-owned residual prediction from current H3 features.""" -+ -+from __future__ import annotations -+ -+import math -+from collections.abc import Mapping -+from typing import Any -+from weakref import ref -+ -+import torch -+import torch.distributed as dist -+ -+from .reference_kv_cache_dit import MiniMaxH3CachedAdapter, MiniMaxH3CachedBlocks -+ -+ -+def alignment_cache_enabled(config: Any) -> bool: -+ cache = getattr(config, "cache_config", {}) -+ enabled = ( -+ cache.get("minimax_h3_alignment_cache", False) -+ if isinstance(cache, Mapping) -+ else getattr(cache, "minimax_h3_alignment_cache", False) -+ ) -+ if type(enabled) is not bool: -+ raise ValueError("minimax_h3_alignment_cache must be boolean") -+ if not enabled: -+ return False -+ parallel = getattr(config, "parallel_config", None) -+ taylor = cache.get("enable_taylorseer", False) if isinstance(cache, Mapping) else cache.enable_taylorseer -+ if ( -+ getattr(config, "task_type", None) != "ref2va" -+ or getattr(config, "cache_backend", None) != "cache_dit" -+ or getattr(config, "kv_offload_config", None) is not None -+ or taylor -+ or (getattr(parallel, "data_parallel_size", 1) or 1) != 1 -+ or (getattr(parallel, "cfg_parallel_size", 1) or 1) != 1 -+ ): -+ raise ValueError("H3 alignment cache requires Ref2VA Cache-DiT, DP1/CFG1, no Taylor/RefKV") -+ return True -+ -+ -+class MiniMaxH3AlignedCacheState: -+ """Own three real prefix/residual pairs and structural row identities.""" -+ -+ def __init__(self) -> None: -+ self.history: list[tuple[torch.Tensor, torch.Tensor]] = [] -+ self.groups: torch.Tensor | None = None -+ self.coefficients: torch.Tensor | None = None -+ self.step = -1 -+ self.closed = False -+ self.computed = self.predicted = self.vetoes = 0 -+ -+ def configure(self, branch) -> None: -+ groups = torch.full((branch.seq_len,), -1, dtype=torch.long, device=branch.device) -+ groups[branch.text_pos_dev] = 0 -+ groups[branch.img_pos_dev] = torch.where(branch.update_mask_dev, 2, 1) -+ groups[branch.audio_pos_dev] = torch.where(branch.audio_update_mask_dev, 4, 3) -+ self.set_groups(groups) -+ -+ def set_groups(self, groups: torch.Tensor) -> None: -+ if self.groups is not None or self.closed or groups.ndim != 1: -+ raise ValueError("H3 alignment groups must be initialized once per request") -+ self.groups = groups.detach().clone() -+ -+ def begin_step(self, step: int, *_clocks) -> None: -+ if self.closed or step != self.step + 1: -+ raise RuntimeError("H3 alignment cache requires consecutive request steps") -+ self.step = step -+ self.coefficients = None -+ -+ def _local_groups(self, rows: int) -> torch.Tensor: -+ if self.groups is None: -+ raise RuntimeError("H3 alignment cache has no structural groups") -+ if self.groups.numel() == rows: -+ return self.groups -+ from vllm_omni.diffusion.distributed.parallel_state import ( -+ get_sequence_parallel_rank, -+ get_sequence_parallel_world_size, -+ ) -+ -+ size, rank = get_sequence_parallel_world_size(), get_sequence_parallel_rank() -+ if self.groups.numel() != rows * size: -+ raise ValueError("H3 alignment cache requires equal pure-Ulysses shards") -+ return self.groups.narrow(0, rows * rank, rows) -+ -+ def record(self, prefix: torch.Tensor, residual: torch.Tensor) -> None: -+ if self.closed or self.step < 0 or prefix.ndim != 2 or prefix.shape != residual.shape: -+ raise RuntimeError("H3 alignment cache requires an open step and matching matrices") -+ self.history.append((prefix.detach().float().clone(), residual.detach().float().clone())) -+ del self.history[:-3] -+ self.computed += 1 -+ -+ def can_predict(self, current: torch.Tensor, *, parallelized: bool) -> bool: -+ if len(self.history) != 3 or self.closed: -+ self.vetoes += 1 -+ return False -+ if any(f.shape != current.shape or f.device != current.device for f, _ in self.history): -+ self.history.clear() -+ self.vetoes += 1 -+ return False -+ groups = self._local_groups(current.shape[0]) -+ f0, f1, f2 = [item[0] for item in self.history] -+ d0, d1, u = f2 - f1, f1 - f0, current.float() - f2 -+ # Reduce feature products per row once; grouping never materializes -+ # a [groups, rows, hidden] tensor and padding contributes no products. -+ products = torch.stack( -+ [ -+ (d0 * d0).sum(-1), -+ (d0 * d1).sum(-1), -+ (d1 * d1).sum(-1), -+ (d0 * u).sum(-1), -+ (d1 * u).sum(-1), -+ (u * u).sum(-1), -+ ], -+ dim=-1, -+ ) -+ totals = torch.stack([torch.where((groups == group)[:, None], products, 0.0).sum(0) for group in range(5)]) -+ if parallelized: -+ if not dist.is_initialized(): -+ raise RuntimeError("H3 alignment requires initialized collectives") -+ # DP1/CFG1, pure USP: WORLD is the four-rank model group. Empty -+ # local audio/reference groups must still enter this collective. -+ dist.all_reduce(totals, op=dist.ReduceOp.SUM) -+ coefficients = [] -+ valid = True -+ for g00, g01, g11, h0, h1, uu in totals.cpu().double().tolist(): -+ if not all(math.isfinite(x) for x in [g00, g01, g11, h0, h1, uu]): -+ valid = False -+ coefficients.append((0.0, 0.0)) -+ continue -+ if uu == 0: -+ coefficients.append((0.0, 0.0)) -+ continue -+ ridge = 1e-4 * (g00 + g11) -+ determinant = (g00 + ridge) * (g11 + ridge) - g01 * g01 -+ if determinant <= 0: -+ valid = False -+ coefficients.append((0.0, 0.0)) -+ continue -+ a0 = ((g11 + ridge) * h0 - g01 * h1) / determinant -+ a1 = ((g00 + ridge) * h1 - g01 * h0) / determinant -+ error = max(0.0, uu - 2 * (a0 * h0 + a1 * h1) + a0 * a0 * g00 + 2 * a0 * a1 * g01 + a1 * a1 * g11) -+ valid &= abs(a0) + abs(a1) <= 8.0 and error <= 0.25**2 * uu -+ coefficients.append((a0, a1)) -+ if not valid: -+ self.vetoes += 1 -+ return False -+ self.coefficients = torch.tensor(coefficients, dtype=torch.float32, device=current.device) -+ return True -+ -+ def predict(self, current: torch.Tensor) -> torch.Tensor: -+ if self.coefficients is None or self.closed: -+ raise RuntimeError("H3 alignment prediction has not been admitted") -+ groups = self._local_groups(current.shape[0]) -+ coefficients = self.coefficients[groups.clamp(min=0)] -+ coefficients = torch.where((groups >= 0)[:, None], coefficients, 0.0) -+ b0, b1, b2 = [item[1] for item in self.history] -+ predicted = b2 + coefficients[:, :1] * (b2 - b1) + coefficients[:, 1:] * (b1 - b0) -+ self.predicted += 1 -+ return (current.float() + predicted).to(current.dtype).contiguous() -+ -+ def stats(self) -> dict[str, int]: -+ return {"computed": self.computed, "predicted": self.predicted, "vetoes": self.vetoes} -+ -+ def close(self) -> None: -+ self.history.clear() -+ self.groups = self.coefficients = None -+ self.closed = True -+ -+ -+class _AlignmentManager: -+ def __init__(self, manager: Any, owner) -> None: -+ self._manager, self._owner = manager, ref(owner) -+ -+ def __getattr__(self, name): -+ return getattr(self._manager, name) -+ -+ def can_cache(self, states, *args, **kwargs): -+ allowed = self._manager.can_cache(states, *args, **kwargs) -+ owner = self._owner() -+ return allowed and owner._alignment_state.can_predict( -+ owner._alignment_prefix, parallelized=kwargs.get("parallelized", False) -+ ) -+ -+ def set_Bn_buffer(self, buffer, prefix="Bn"): -+ owner = self._owner() -+ owner._alignment_state.record(owner._alignment_prefix, buffer) -+ self._manager.set_Bn_buffer(buffer, prefix=prefix) -+ -+ def apply_cache(self, hidden_states, encoder_hidden_states=None, **kwargs): -+ if encoder_hidden_states is not None or not self._manager.is_cache_residual(): -+ raise ValueError("H3 alignment supports only residual-cached packed hidden states") -+ return self._owner()._alignment_state.predict(hidden_states), None -+ -+ -+class MiniMaxH3AlignedCachedBlocks(MiniMaxH3CachedBlocks): -+ def __init__(self, *args, **kwargs) -> None: -+ super().__init__(*args, **kwargs) -+ self._alignment_state = self._alignment_prefix = None -+ self.context_manager = _AlignmentManager(self.context_manager, self) -+ -+ def call_Fn_blocks(self, hidden_states, *args, **kwargs): -+ output = super().call_Fn_blocks(hidden_states, *args, **kwargs) -+ self._alignment_prefix = output[0] -+ return output -+ -+ def call_Mn_blocks(self, hidden_states, *args, **kwargs): -+ computed, encoder, _ = super().call_Mn_blocks(hidden_states, *args, **kwargs) -+ return computed, encoder, computed.float() - hidden_states.float() -+ -+ def forward(self, hidden_states, *args, alignment_cache_state=None, **kwargs): -+ if not isinstance(alignment_cache_state, MiniMaxH3AlignedCacheState) or alignment_cache_state.closed: -+ raise RuntimeError("H3 alignment requires an active request state") -+ if kwargs.get("num_requests", 1) != 1 or kwargs.get("reference_kv_state") is not None: -+ raise ValueError("H3 alignment requires one request without reference-KV reuse") -+ old_state, old_prefix = self._alignment_state, self._alignment_prefix -+ try: -+ self._alignment_state = alignment_cache_state -+ return super().forward(hidden_states, *args, **kwargs) -+ finally: -+ self._alignment_state, self._alignment_prefix = old_state, old_prefix -+ -+ -+class MiniMaxH3AlignedCachedAdapter(MiniMaxH3CachedAdapter): -+ block_class = MiniMaxH3AlignedCachedBlocks -diff --git a/vllm_omni/diffusion/models/minimax_h3/denoise_loop.py b/vllm_omni/diffusion/models/minimax_h3/denoise_loop.py -index 8cca219..716b931 100644 ---- a/vllm_omni/diffusion/models/minimax_h3/denoise_loop.py -+++ b/vllm_omni/diffusion/models/minimax_h3/denoise_loop.py -@@ -379,13 +379,23 @@ def minimax_h3_denoise_loop( - if reference_kv_state is not None: - positive.static_kwargs["reference_kv_state"] = reference_kv_state - -+ alignment_cache_state = None - try: -+ create_alignment_cache_state = getattr(model, "create_alignment_cache_state", None) -+ if callable(create_alignment_cache_state): -+ alignment_cache_state = create_alignment_cache_state() -+ if alignment_cache_state is not None: -+ alignment_cache_state.configure(positive) -+ if alignment_cache_state is not None and (video_edit is not None or audio_edit is not None): -+ raise ValueError("H3 alignment cache does not support latent edits") - for step in range(num_steps): - check_request_cancellation() - step_cm = step_profiler(step) if step_profiler is not None else nullcontext() - with step_cm: - s_v = sigmas_video[step] - s_a = sigmas_audio[step] -+ if alignment_cache_state is not None: -+ alignment_cache_state.begin_step(step, s_v, s_a) - if reference_kv_state is not None: - reference_kv_state.begin_step(step, sigma=float(s_v)) - # Publish where we are so step-gated attention features (the dense -@@ -426,6 +436,8 @@ def minimax_h3_denoise_loop( - ) - if getattr(getattr(model, "arch", None), "pdd_steps", None) is not None: - fk["pdd_step"] = step -+ if alignment_cache_state is not None: -+ fk["alignment_cache_state"] = alignment_cache_state - with torch.inference_mode(): - v_video, v_audio = model(**fk) - if reference_kv_state is not None: -@@ -475,6 +487,9 @@ def minimax_h3_denoise_loop( - check_request_cancellation(synchronize=True) - - finally: -+ if alignment_cache_state is not None: -+ model._alignment_cache_stats = alignment_cache_state.stats() -+ alignment_cache_state.close() - minimax_h3_publish_denoise_progress(None, None, None) - if reference_kv_state is not None: - positive.static_kwargs.pop("reference_kv_state", None) -diff --git a/vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py b/vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py -index 9518828..63f1530 100644 ---- a/vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py -+++ b/vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py -@@ -245,6 +245,7 @@ _FORWARD_SUPPORTED_KWARGS = frozenset( - "control_rows", - "control_context_scale", - "reference_kv_state", -+ "alignment_cache_state", - } - ) - -@@ -777,6 +778,23 @@ class MiniMaxH3DiTModel(nn.Module): - max_bytes=256 * 1024**2 if enabled else 0, offload_weights=offload_adaln - ) - self.od_config = od_config -+ alignment_enabled = ( -+ cache_config.get("minimax_h3_alignment_cache", False) -+ if isinstance(cache_config, Mapping) -+ else getattr(cache_config, "minimax_h3_alignment_cache", False) -+ ) -+ if type(alignment_enabled) is not bool: -+ raise ValueError("minimax_h3_alignment_cache must be a boolean") -+ self._alignment_cache_enabled = False -+ if alignment_enabled: -+ from .cache_state_alignment import MiniMaxH3AlignedCachedAdapter, alignment_cache_enabled -+ -+ self._alignment_cache_enabled = alignment_cache_enabled(od_config) -+ if not current_omni_platform.is_npu(): -+ raise ValueError("H3 alignment cache is currently restricted to the Ascend Ref2VA experiment") -+ self._cache_dit_adapter_config = replace( -+ self._cache_dit_adapter_config, cached_adapter_cls=MiniMaxH3AlignedCachedAdapter -+ ) - if getattr(od_config, "kv_offload_config", None) is not None: - from vllm_omni.diffusion.models.minimax_h3.reference_kv_cache_dit import MiniMaxH3CachedAdapter - -@@ -1257,6 +1275,14 @@ class MiniMaxH3DiTModel(nn.Module): - device=device, - ) - -+ def create_alignment_cache_state(self): -+ if not getattr(self, "_alignment_cache_enabled", False): -+ return None -+ from .cache_state_alignment import MiniMaxH3AlignedCacheState -+ -+ self._alignment_cache_stats = None -+ return MiniMaxH3AlignedCacheState() -+ - def forward(self, *, pdd_step: int | None = None, **kwargs: Any) -> tuple[torch.Tensor, torch.Tensor]: - """Packed inference forward. - -@@ -1490,6 +1516,14 @@ class MiniMaxH3DiTModel(nn.Module): - if getattr(self, "adaln_cache", None) is not None: - self.adaln_cache.prepare(t_emb) - -+ alignment_state = kwargs.get("alignment_cache_state") -+ alignment_options = {} -+ if alignment_state is not None: -+ if not getattr(self, "_alignment_cache_enabled", False): -+ raise ValueError("H3 alignment cache state was supplied while the feature is disabled") -+ if num_requests != 1 or reference_kv_state is not None or control_rows is not None: -+ raise ValueError("H3 alignment cache does not support packed batches, control or reference-KV reuse") -+ alignment_options["alignment_cache_state"] = alignment_state - for block_index, block in enumerate(self.blocks): - hidden = block( - hidden, -@@ -1505,6 +1539,7 @@ class MiniMaxH3DiTModel(nn.Module): - reference_kv_state=reference_kv_state, - reference_kv_compact=compact_reference, - vdn_window=vdn_window, -+ **alignment_options, - ) - if block_index in hints: - hidden = hidden + hints.pop(block_index) * control_scale -diff --git a/vllm_omni/diffusion/models/minimax_h3/reference_kv_cache_dit.py b/vllm_omni/diffusion/models/minimax_h3/reference_kv_cache_dit.py -index f14dcc9..f78598b 100644 ---- a/vllm_omni/diffusion/models/minimax_h3/reference_kv_cache_dit.py -+++ b/vllm_omni/diffusion/models/minimax_h3/reference_kv_cache_dit.py -@@ -125,6 +125,8 @@ class MiniMaxH3CachedBlocks(CachedBlocks_Pattern_3_4_5): - - - class MiniMaxH3CachedAdapter(CachedAdapter): -+ block_class = MiniMaxH3CachedBlocks -+ - @classmethod - def collect_unified_blocks( - cls, block_adapter: BlockAdapter, contexts_kwargs: list[dict] -@@ -139,7 +141,7 @@ class MiniMaxH3CachedAdapter(CachedAdapter): - context_index += 1 - contexts[block_adapter.unique_blocks_name[i][j]] = nn.ModuleList( - [ -- MiniMaxH3CachedBlocks( -+ cls.block_class( - blocks, - transformer=transformer, - forward_pattern=block_adapter.forward_pattern[i][j], diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c12-source-identity.json b/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c12-source-identity.json deleted file mode 100644 index 331f49363..000000000 --- a/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c12-source-identity.json +++ /dev/null @@ -1,15 +0,0 @@ -{ - "head": "a29648e5acd3077b7b22739bc9522017782fe1ac", - "tree": "3490989d041cc8fca66a4fc3a234d71a3366a352", - "base": "be319692b96413918973b5bd399cdda3f320c574", - "branch": "rsi/h3-297-state-aligned-residual", - "files": { - "vllm_omni/diffusion/models/minimax_h3/cache_state_alignment.py": "3340ec21a77aef78531c601be241bcfb1e44107c795e8c8649907d1a18ed377f", - "vllm_omni/diffusion/models/minimax_h3/reference_kv_cache_dit.py": "f4cd0e9aa9e824c92ec1acd02c57a8f7d8684c35d4edf92ffdfae3345ff44e99", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "436ec52cd6329cbc64e91d94b450bfa6f773a9a953b8f974db73a5eb986a98bb", - "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "4dec8f02a04bfe7e8d2ada24fc6d5ae40a0058671d8515c8a266ecf88824b7b0", - "tests/diffusion/models/minimax_h3/test_minimax_h3_state_cache.py": "89cf57482b35d8d8ad8d4978aec87e9ee3ef45330e5b911161adef4379950a65" - }, - "published": false, - "patch_base": "be319692b96413918973b5bd399cdda3f320c574" -} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c13-candidate.patch b/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c13-candidate.patch deleted file mode 100644 index 09b2049e6..000000000 --- a/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c13-candidate.patch +++ /dev/null @@ -1,817 +0,0 @@ -diff --git a/tests/diffusion/distributed/test_collective_gate.py b/tests/diffusion/distributed/test_collective_gate.py -new file mode 100644 -index 0000000..e66fdae ---- /dev/null -+++ b/tests/diffusion/distributed/test_collective_gate.py -@@ -0,0 +1,205 @@ -+# SPDX-License-Identifier: Apache-2.0 -+# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project -+ -+"""Unit tests for the DLO/QKV-AllToAll ordering gate. -+ -+Covers the barrier rendezvous and the hook's deferred-AllGather state machine -+that keeps the background weight gather from taking the links ahead of the -+block's foreground AllToAll. -+""" -+ -+from unittest.mock import Mock -+ -+import pytest -+import torch -+from torch import nn -+ -+import vllm_omni.diffusion.offloader.distributed_layerwise_backend as dist_backend_module -+from tests.diffusion.offloader.helpers import DummyStream, patch_offload_runtime -+from vllm_omni.diffusion.distributed.collective_gate import QKVAllToAllBarrier -+from vllm_omni.diffusion.offloader.distributed_layerwise_backend import ( -+ DistributedLayerwiseOffloadHook, -+ attach_collective_gate, -+) -+ -+pytestmark = [pytest.mark.diffusion, pytest.mark.cpu, pytest.mark.core_model] -+ -+ -+@pytest.fixture -+def patched_offload_runtime(monkeypatch): -+ patch_offload_runtime(monkeypatch, dist_backend_module.current_omni_platform, synchronize=True) -+ -+ -+class TestQKVAllToAllBarrier: -+ def test_signal_without_arm_is_a_noop(self): -+ on_signal = Mock() -+ barrier = QKVAllToAllBarrier(on_signal) -+ -+ barrier.signal() -+ -+ on_signal.assert_not_called() -+ assert not barrier.armed -+ -+ def test_arm_then_signal_fires_once(self): -+ on_signal = Mock() -+ barrier = QKVAllToAllBarrier(on_signal) -+ -+ barrier.arm() -+ assert barrier.armed -+ barrier.signal() -+ -+ on_signal.assert_called_once_with() -+ assert not barrier.armed -+ -+ # A second signal in the same forward must not re-issue the gather. -+ barrier.signal() -+ on_signal.assert_called_once_with() -+ -+ def test_disarm_suppresses_a_pending_signal(self): -+ on_signal = Mock() -+ barrier = QKVAllToAllBarrier(on_signal) -+ -+ barrier.arm() -+ barrier.disarm() -+ barrier.signal() -+ -+ on_signal.assert_not_called() -+ -+ -+def _gated_hook() -> DistributedLayerwiseOffloadHook: -+ """A hook wired for the gated path, with hand-set shard buffers.""" -+ hook = DistributedLayerwiseOffloadHook( -+ next_block=nn.Linear(2, 2, bias=False), -+ device=torch.device("cpu"), -+ dp_group=object(), -+ dp_size=2, -+ rank=0, -+ copy_stream=DummyStream(), -+ comm_stream=DummyStream(), -+ pin_memory=False, -+ order_allgather_after_attn_a2a=True, -+ ) -+ device_buf = {torch.float32: torch.zeros(4)} -+ shard_buf = {torch.float32: torch.zeros(2)} -+ hook.gpu_buffers = [dict(device_buf), dict(device_buf)] -+ hook.gpu_shard_buffers = [dict(shard_buf), dict(shard_buf)] -+ hook.cpu_shards = {torch.float32: torch.zeros(2)} -+ hook._ag_output_sizes = {torch.float32: 4} -+ hook.metadata = {} -+ hook._cached_repoint = () -+ hook.rank_local_mmap = False -+ return hook -+ -+ -+class TestDeferredAllGather: -+ def test_prefetch_defers_the_gather_until_the_barrier_signals(self, patched_offload_runtime, monkeypatch): -+ hook = _gated_hook() -+ gather = Mock() -+ monkeypatch.setattr(torch.distributed, "all_gather_into_tensor", gather) -+ -+ hook.prefetch_layer(0, non_blocking=True) -+ -+ # H2D ran; the collective is held back for the block's AllToAll. -+ gather.assert_not_called() -+ assert hook._a2a_barrier.armed -+ assert hook._deferred_allgather is not None -+ # The ready event must still be published for the consumer. -+ assert hook.ready_events[0] is hook._prefetch_done -+ -+ hook._a2a_barrier.signal() -+ -+ gather.assert_called_once() -+ assert hook._deferred_allgather is None -+ assert not hook._a2a_barrier.armed -+ -+ def test_post_forward_flush_issues_an_unsignalled_gather(self, patched_offload_runtime, monkeypatch): -+ """A block whose attention never ran must still get its weights.""" -+ hook = _gated_hook() -+ gather = Mock() -+ monkeypatch.setattr(torch.distributed, "all_gather_into_tensor", gather) -+ -+ hook.prefetch_layer(0, non_blocking=True) -+ gather.assert_not_called() -+ -+ # pre_forward ran, attention did not: the pairing flush must fire. -+ hook.post_forward(hook.next_block, None) -+ -+ gather.assert_called_once() -+ assert hook._deferred_allgather is None -+ assert not hook._a2a_barrier.armed -+ -+ def test_sync_prefetch_issues_immediately_without_arming(self, patched_offload_runtime, monkeypatch): -+ """The cache-dit sync fallback has no attention signal to wait for.""" -+ hook = _gated_hook() -+ gather = Mock() -+ monkeypatch.setattr(torch.distributed, "all_gather_into_tensor", gather) -+ -+ hook.prefetch_layer(0, non_blocking=False) -+ -+ gather.assert_called_once() -+ assert not hook._a2a_barrier.armed -+ assert hook._deferred_allgather is None -+ -+ def test_disabled_by_default(self, patched_offload_runtime, monkeypatch): -+ hook = DistributedLayerwiseOffloadHook( -+ next_block=nn.Linear(2, 2, bias=False), -+ device=torch.device("cpu"), -+ dp_group=object(), -+ dp_size=2, -+ rank=0, -+ copy_stream=DummyStream(), -+ comm_stream=DummyStream(), -+ pin_memory=False, -+ ) -+ hook.gpu_buffers = [{torch.float32: torch.zeros(4)}] * 2 -+ hook.gpu_shard_buffers = [{torch.float32: torch.zeros(2)}] * 2 -+ hook.cpu_shards = {torch.float32: torch.zeros(2)} -+ hook._ag_output_sizes = {torch.float32: 4} -+ hook.metadata = {} -+ hook._cached_repoint = () -+ gather = Mock() -+ monkeypatch.setattr(torch.distributed, "all_gather_into_tensor", gather) -+ -+ hook.prefetch_layer(0, non_blocking=True) -+ -+ assert hook._a2a_barrier is None -+ gather.assert_called_once() -+ -+ -+class TestAttachCollectiveGate: -+ def test_block_without_attention_reports_zero(self): -+ hook = _gated_hook() -+ -+ assert attach_collective_gate(nn.Linear(2, 2), hook) == 0 -+ -+ -+@pytest.mark.parametrize("flush", [False, True]) -+def test_chunked_gate_defers_copies_and_reuses_bounded_slots(patched_offload_runtime, monkeypatch, flush): -+ from types import SimpleNamespace -+ -+ hook = _gated_hook() -+ chunks = tuple( -+ SimpleNamespace(chunk_id=i, local_numel=1, cpu_offset=i, full_offset=2 * i, padded_numel=2) for i in range(3) -+ ) -+ hook.manifest = SimpleNamespace(dtypes=(SimpleNamespace(dtype=torch.float32, chunks=chunks),)) -+ hook.cpu_shards = {torch.float32: torch.tensor([11.0, 22.0, 33.0])} -+ hook.gpu_buffers = [{torch.float32: torch.zeros(6)} for _ in range(2)] -+ calls = [] -+ -+ def gather(output, local, **kwargs): -+ calls.append(local.clone()) -+ output.copy_(local.repeat(2)) -+ -+ monkeypatch.setattr(torch.distributed, "all_gather_into_tensor", gather) -+ hook.prefetch_layer(0, non_blocking=True) -+ assert not calls and hook._deferred_chunked_allgather is not None -+ assert torch.equal(hook.gpu_shard_buffers[0][torch.float32], torch.zeros(2)) -+ if flush: -+ hook._flush_deferred_allgather() -+ else: -+ hook._a2a_barrier.signal() -+ assert len(calls) == 3 and hook._deferred_chunked_allgather is None -+ assert not hook._a2a_barrier.armed -+ assert torch.equal(hook.gpu_buffers[0][torch.float32], torch.tensor([11.0, 11.0, 22.0, 22.0, 33.0, 33.0])) -+ hook._a2a_barrier.signal() -+ assert len(calls) == 3 -diff --git a/vllm_omni/config/omni_config.py b/vllm_omni/config/omni_config.py -index 0a803cf..44fc48c 100644 ---- a/vllm_omni/config/omni_config.py -+++ b/vllm_omni/config/omni_config.py -@@ -863,6 +863,7 @@ class _DiffusionConfigProjection(_TrackExplicitConfigFields): - dlo_use_allgather: bool = True - dlo_chunk_size_mb: int = Field(default=64, gt=0, strict=True) - dlo_resident_layers: int = Field(default=0, ge=0) -+ dlo_order_allgather_after_attn_a2a: bool = False - host_weight_runtime_mode: Literal["disabled", "preferred", "required"] = "disabled" - host_weight_runtime_root: str | None = None - dlo_host_registration_limit_gib: float = Field(default=0.0, ge=0) -diff --git a/vllm_omni/config/stage_config.py b/vllm_omni/config/stage_config.py -index 76a58a9..3c69986 100644 ---- a/vllm_omni/config/stage_config.py -+++ b/vllm_omni/config/stage_config.py -@@ -672,6 +672,7 @@ class StageDeployConfig: - dlo_use_allgather: bool | None = None - dlo_chunk_size_mb: int | None = None - dlo_resident_layers: int | None = None -+ dlo_order_allgather_after_attn_a2a: bool | None = None - host_weight_runtime_mode: str | None = None - host_weight_runtime_root: str | None = None - dlo_host_registration_limit_gib: float | None = None -diff --git a/vllm_omni/diffusion/attention/layer.py b/vllm_omni/diffusion/attention/layer.py -index 5520390..574eafa 100644 ---- a/vllm_omni/diffusion/attention/layer.py -+++ b/vllm_omni/diffusion/attention/layer.py -@@ -9,7 +9,7 @@ - - from collections.abc import Mapping - from dataclasses import replace --from typing import cast -+from typing import Any, cast - - import torch - import torch.nn as nn -@@ -333,6 +333,11 @@ class Attention(nn.Module): - # Local strategy when SP is intentionally inactive outside sharded regions. - self._no_parallel_strategy = NoParallelAttention() - -+ # Optional cross-module rendezvous (see distributed/collective_gate.py). -+ # Installed by the DLO backend when it must keep its background weight -+ # AllGather from taking the links ahead of this layer's QKV AllToAll. -+ self._collective_gate: Any | None = None -+ - self.layer_idx: int | None = _try_extract_layer_index(prefix) - - self._kv_cache_dtype: str | None = None -@@ -506,10 +511,12 @@ class Attention(nn.Module): - reason = self._cross_backend_limitation(attn_metadata) - if reason is not None: - blocked = self._cross_fallback_blocker(attn_metadata) -- result = self.sdpa_fallback.resolve_execution_path(resolved_context, query, key, value, attn_metadata) -+ result = cast(SDPAImpl, self.sdpa_fallback).resolve_execution_path( -+ resolved_context, query, key, value, attn_metadata -+ ) - return replace( - result, -- backend=self.attn_backend.get_name() if blocked else "SDPA", -+ backend=cast(type[AttentionBackend], self.attn_backend).get_name() if blocked else "SDPA", - path="cross_unsupported" if blocked else "cross_sdpa_fallback", - support=CapabilityResult.unsupported(blocked) if blocked else CapabilityResult.unmigrated(reason), - ) -@@ -591,6 +598,12 @@ class Attention(nn.Module): - # For Ring: Concat joint_q - query, key, value, attn_metadata, ctx = strategy.pre_attention(query, key, value, attn_metadata) - -+ # The QKV AllToAll is now enqueued on this stream. Release any gather -+ # that was deferred to give this collective the links first. -+ gate = self._collective_gate -+ if gate is not None: -+ gate.signal() -+ - # MiniMax-H3 Tier2 stores reference K/V in the balanced layout produced - # by Ulysses. Keeping this hook at the generic post-parallel boundary - # avoids a model import in the parallel strategy and leaves other models -@@ -760,7 +773,7 @@ class Attention(nn.Module): - def _cross_fallback_blocker(self, attn_metadata: AttentionMetadata | None) -> str | None: - if self.backend_explicit: - return ( -- f"Explicit backend {self.attn_backend.get_name()} does not support " -+ f"Explicit backend {cast(type[AttentionBackend], self.attn_backend).get_name()} does not support " - f"{self._cross_backend_limitation(attn_metadata)}" - ) - if attn_metadata is not None and attn_metadata.extra.get("kv_cache_dtype") not in (None, "float"): -@@ -793,7 +806,7 @@ class Attention(nn.Module): - blocked = self._cross_fallback_blocker(attn_metadata) - if blocked is not None: - raise NotImplementedError(blocked) -- return self.sdpa_fallback.forward(query, key, value, attn_metadata) -+ return cast(SDPAImpl, self.sdpa_fallback).forward(query, key, value, attn_metadata) - dense_mask = attn_metadata.attn_mask if attn_metadata is not None else None - # Piecewise backends honor full_attn_spans ahead of attn_mask and never read the - # dense mask (e.g. HunyuanImage3, which sends both), so there is nothing to divert. -diff --git a/vllm_omni/diffusion/data.py b/vllm_omni/diffusion/data.py -index 8528574..fcbeec0 100644 ---- a/vllm_omni/diffusion/data.py -+++ b/vllm_omni/diffusion/data.py -@@ -962,6 +962,10 @@ class OmniDiffusionConfig: - dlo_chunk_size_mb: int = 64 - # Leading main-DiT blocks kept resident by distributed layerwise offload. - dlo_resident_layers: int = 0 -+ # Hold each DiT block's weight AllGather until that block's QKV AllToAll has -+ # been enqueued, so the background collective does not take the links ahead -+ # of the critical-path all-to-all. -+ dlo_order_allgather_after_attn_a2a: bool = False - # Final-layout Host Weight Runtime policy. The loader only activates this - # for eligible no-AllGather DLO; all other configurations preserve their - # existing loader/storage path. -diff --git a/vllm_omni/diffusion/distributed/collective_gate.py b/vllm_omni/diffusion/distributed/collective_gate.py -new file mode 100644 -index 0000000..1386d89 ---- /dev/null -+++ b/vllm_omni/diffusion/distributed/collective_gate.py -@@ -0,0 +1,72 @@ -+# SPDX-License-Identifier: Apache-2.0 -+# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project -+"""Cross-module rendezvous that orders background collectives after the -+foreground all-to-all of a transformer block. -+ -+Distributed layerwise offload (DLO) issues the *next* block's weight AllGather -+on a dedicated ``comm_stream`` while the *current* block computes. Both that -+AllGather and the block's QKV sequence-parallel AllToAll are collectives over -+the same ranks, so whichever reaches the links first wins: the AllGather is -+submitted at block entry, well before the block reaches its attention, and the -+critical-path AllToAll ends up queued behind it. -+ -+A plain event wait cannot fix this by itself. ``wait_event`` snapshots the -+records enqueued *before* the call, so "enqueue the wait at block entry, record -+the event later in the block" is a no-op -- the wait resolves against the -+previous iteration's record. The gather therefore has to be *issued* after the -+AllToAll: the hook arms this barrier at block entry and defers the gather, the -+attention signals it right after ``strategy.pre_attention``, and the signal -+records the event and lets the hook submit the gather gated on it. -+ -+The gather still overlaps the rest of the block (attention output, MLP, the -+next block's projections); it simply cannot start before the AllToAll finished. -+""" -+ -+from __future__ import annotations -+ -+from collections.abc import Callable -+ -+__all__ = ["QKVAllToAllBarrier"] -+ -+ -+class QKVAllToAllBarrier: -+ """One-block rendezvous between a DLO hook and the block's attention. -+ -+ Owned by a single :class:`DistributedLayerwiseOffloadHook` (one per -+ streaming block) and shared with the attention layer inside that block. -+ Both live on the same block, so the arming and the signalling always come -+ from the same module forward. -+ -+ Args: -+ on_signal: Invoked at most once per :meth:`arm` when the block's -+ attention reports its QKV AllToAll. It must submit the deferred -+ weight AllGather. -+ """ -+ -+ def __init__(self, on_signal: Callable[[], None]) -> None: -+ self._on_signal = on_signal -+ self._armed = False -+ -+ @property -+ def armed(self) -> bool: -+ """True between :meth:`arm` and the matching :meth:`signal`.""" -+ return self._armed -+ -+ def arm(self) -> None: -+ """Mark the next attention QKV AllToAll as the gather's gate.""" -+ self._armed = True -+ -+ def disarm(self) -> None: -+ """Drop a pending arm without firing it (the caller flushes instead).""" -+ self._armed = False -+ -+ def signal(self) -> None: -+ """Report that the QKV AllToAll is now enqueued on the compute stream. -+ -+ A no-op unless armed, so it is cheap to call unconditionally from the -+ attention hot path. -+ """ -+ if not self._armed: -+ return -+ self._armed = False -+ self._on_signal() -diff --git a/vllm_omni/diffusion/offloader/base.py b/vllm_omni/diffusion/offloader/base.py -index 2505d48..50b73e5 100644 ---- a/vllm_omni/diffusion/offloader/base.py -+++ b/vllm_omni/diffusion/offloader/base.py -@@ -71,6 +71,10 @@ class OffloadConfig: - dlo_use_allgather: bool = True - dlo_resident_layers: int = 0 # leading DiT layers kept on device - chunk_size_bytes: int = 64 * 1024 * 1024 -+ # Hold each DiT block's weight AllGather until that block's QKV AllToAll has -+ # been enqueued, so the background collective does not take the links ahead -+ # of the critical-path all-to-all. See ``collective_gate.py``. -+ dlo_order_allgather_after_attn_a2a: bool = False - # Optional per-worker ceiling for registering an HWR mmap. Zero means no - # additional ceiling; pin_cpu_memory controls whether registration is tried. - dlo_host_registration_limit_gib: float = 0.0 -@@ -216,6 +220,7 @@ class OffloadConfig: - dlo_use_allgather=dit_uses_allgather, - dlo_resident_layers=dlo_resident_layers, - chunk_size_bytes=chunk_size_bytes, -+ dlo_order_allgather_after_attn_a2a=bool(getattr(od_config, "dlo_order_allgather_after_attn_a2a", False)), - dlo_host_registration_limit_gib=dlo_host_registration_limit_gib, - components=components, - dlo_transfers=dlo_transfers, -diff --git a/vllm_omni/diffusion/offloader/distributed_layerwise_backend.py b/vllm_omni/diffusion/offloader/distributed_layerwise_backend.py -index 7f0f081..8c4f87f 100644 ---- a/vllm_omni/diffusion/offloader/distributed_layerwise_backend.py -+++ b/vllm_omni/diffusion/offloader/distributed_layerwise_backend.py -@@ -31,6 +31,7 @@ import torch.distributed - from torch import nn - from vllm.logger import init_logger - -+from vllm_omni.diffusion.distributed.collective_gate import QKVAllToAllBarrier - from vllm_omni.diffusion.hooks import HookRegistry, ModelHook - from vllm_omni.diffusion.model_loader.host_weight_plan import ( - HostWeightPlan, -@@ -142,6 +143,7 @@ class DistributedLayerwiseOffloadHook(ModelHook): - tensor_transforms: dict[int, Any] | None = None, - materialization_probe_tensor: torch.Tensor | None = None, - chunk_size_bytes: int = 64 * 1024 * 1024, -+ order_allgather_after_attn_a2a: bool = False, - ): - assert isinstance(next_block, nn.Module), "transformer block must be type `torch.nn.Module`" - if type(dp_size) is not int or dp_size < 1: -@@ -167,6 +169,7 @@ class DistributedLayerwiseOffloadHook(ModelHook): - - self.chunk_size_bytes = chunk_size_bytes - self.manifest: PartManifest | None = None -+ self._deferred_chunked_allgather = None - self._h2d_done_events = [current_omni_platform.Event(), current_omni_platform.Event()] - self._ag_done_events = [current_omni_platform.Event(), current_omni_platform.Event()] - # Hook-local events protect chunk-input reuse within this hook. Across -@@ -239,6 +242,19 @@ class DistributedLayerwiseOffloadHook(ModelHook): - # Per-block synchronization primitive: set after H2D copy completes. - self._prefetch_done: Any | None = None - -+ # Optional ordering: hold the background weight AllGather until the -+ # block's QKV AllToAll is on the compute stream. See -+ # distributed/collective_gate.py. Off by default; the attention layer -+ # inside this block is handed the same barrier at install time. -+ self._a2a_barrier: QKVAllToAllBarrier | None = None -+ self._a2a_event: Any | None = None -+ # ``(slot, gpu_shards, gpu_weights, evt)`` of a gather armed but not yet -+ # issued, so ``post_forward`` can flush it if the attention never ran. -+ self._deferred_allgather: tuple[int, dict, dict, Any] | None = None -+ if order_allgather_after_attn_a2a: -+ self._a2a_event = current_omni_platform.Event() -+ self._a2a_barrier = QKVAllToAllBarrier(self._allgather_after_attn_a2a) -+ - # Shared shard (AllGather input) buffers — assigned by backend. - self.gpu_shard_buffers: list[dict[torch.dtype, torch.Tensor] | None] = [None, None] - -@@ -653,49 +669,23 @@ class DistributedLayerwiseOffloadHook(ModelHook): - gpu_shard = shard_bufs[dtype][: cpu_shard.numel()] - gpu_shard.copy_(cpu_shard, non_blocking=non_blocking) - gpu_shards[dtype] = gpu_shard -- self.comm_stream.wait_stream(self.copy_stream) -- with current_omni_platform.stream(self.comm_stream): -- for dtype, local_shard in gpu_shards.items(): -- gw = gpu_weights[dtype][: self._ag_output_sizes[dtype]] -- torch.distributed.all_gather_into_tensor( -- gw, -- local_shard, -- group=self.dp_group, -- ) -- evt.record(self.comm_stream) -+ -+ if self._a2a_barrier is not None and non_blocking and getattr(self, "_nz_cast_fn", None) is None: -+ # Hold the collective until the block's attention reports its -+ # QKV AllToAll, so the background gather cannot take the links -+ # ahead of the critical-path all-to-all. The H2D above has -+ # already been enqueued; only the gather moves, so it still -+ # overlaps the rest of the block. Deferral needs a repoint that -+ # emits no stream work, hence the NZ-cast guard. -+ self._deferred_allgather = (slot, gpu_shards, gpu_weights, evt) -+ self._a2a_barrier.arm() -+ else: -+ self._issue_allgather(slot, gpu_shards, gpu_weights, evt) -+ elif self._a2a_barrier is not None and non_blocking: -+ self._deferred_chunked_allgather = (slot, gpu_weights, evt) -+ self._a2a_barrier.arm() - else: -- for dtype_manifest in self.manifest.dtypes: -- cpu_shard = self.cpu_shards[dtype_manifest.dtype] -- buf = shard_bufs[dtype_manifest.dtype] -- for chunk in dtype_manifest.chunks: -- if buf.numel() >= 2 * chunk.local_numel: -- width = buf.numel() // 2 -- input_slot = chunk.chunk_id % 2 -- else: -- width = buf.numel() -- input_slot = 0 -- reuse = self._chunk_reuse_events[input_slot] -- if reuse is not None: -- self.copy_stream.wait_event(reuse) -- local_input = buf[input_slot * width : input_slot * width + chunk.local_numel] -- source = cpu_shard[chunk.cpu_offset : chunk.cpu_offset + chunk.local_numel] -- with current_omni_platform.stream(self.copy_stream): -- local_input.copy_(source, non_blocking=non_blocking) -- self._h2d_done_events[input_slot].record(self.copy_stream) -- self.comm_stream.wait_event(self._h2d_done_events[input_slot]) -- with current_omni_platform.stream(self.comm_stream): -- full_out = gpu_weights[dtype_manifest.dtype][ -- chunk.full_offset : chunk.full_offset + chunk.padded_numel -- ] -- torch.distributed.all_gather_into_tensor( -- full_out, -- local_input, -- group=self.dp_group, -- ) -- self._ag_done_events[input_slot].record(self.comm_stream) -- self._chunk_reuse_events[input_slot] = self._ag_done_events[input_slot] -- with current_omni_platform.stream(self.comm_stream): -- evt.record(self.comm_stream) -+ self._issue_chunked_allgather(slot, gpu_weights, evt, non_blocking=non_blocking) - - self.ready_events[slot] = evt - self._prefetch_done = evt -@@ -719,6 +709,131 @@ class DistributedLayerwiseOffloadHook(ModelHook): - ), - ) - -+ def _issue_chunked_allgather(self, slot, gpu_weights, evt, *, gated=False, non_blocking=True) -> None: -+ """Submit chunk copies and gathers together after the foreground gate. -+ -+ Chunk staging slots are bounded and reused. Deferring only gathers -+ would overwrite an input slot before its collective consumes it. -+ """ -+ shard_bufs = self.gpu_shard_buffers[slot] -+ assert shard_bufs is not None and self.manifest is not None -+ if gated: -+ self.comm_stream.wait_event(self._a2a_event) -+ for dtype_manifest in self.manifest.dtypes: -+ cpu_shard = self.cpu_shards[dtype_manifest.dtype] -+ buf = shard_bufs[dtype_manifest.dtype] -+ for chunk in dtype_manifest.chunks: -+ if buf.numel() >= 2 * chunk.local_numel: -+ width = buf.numel() // 2 -+ input_slot = chunk.chunk_id % 2 -+ else: -+ width = buf.numel() -+ input_slot = 0 -+ reuse = self._chunk_reuse_events[input_slot] -+ if reuse is not None: -+ self.copy_stream.wait_event(reuse) -+ local_input = buf[input_slot * width : input_slot * width + chunk.local_numel] -+ source = cpu_shard[chunk.cpu_offset : chunk.cpu_offset + chunk.local_numel] -+ with current_omni_platform.stream(self.copy_stream): -+ local_input.copy_(source, non_blocking=non_blocking) -+ self._h2d_done_events[input_slot].record(self.copy_stream) -+ self.comm_stream.wait_event(self._h2d_done_events[input_slot]) -+ with current_omni_platform.stream(self.comm_stream): -+ full_out = gpu_weights[dtype_manifest.dtype][ -+ chunk.full_offset : chunk.full_offset + chunk.padded_numel -+ ] -+ torch.distributed.all_gather_into_tensor( -+ full_out, -+ local_input, -+ group=self.dp_group, -+ ) -+ self._ag_done_events[input_slot].record(self.comm_stream) -+ self._chunk_reuse_events[input_slot] = self._ag_done_events[input_slot] -+ with current_omni_platform.stream(self.comm_stream): -+ evt.record(self.comm_stream) -+ -+ def _issue_allgather( -+ self, -+ slot: int, -+ gpu_shards: dict[torch.dtype, torch.Tensor], -+ gpu_weights: dict[torch.dtype, torch.Tensor], -+ evt: Any, -+ *, -+ gated: bool = False, -+ ) -> None: -+ """Submit this block's weight AllGather on ``comm_stream``. -+ -+ ``gated`` additionally waits on the QKV AllToAll barrier event recorded -+ by the block's attention, so the foreground collective gets the links -+ first and this gather queues behind it. -+ """ -+ self.comm_stream.wait_stream(self.copy_stream) -+ if gated: -+ self.comm_stream.wait_event(self._a2a_event) -+ with current_omni_platform.stream(self.comm_stream): -+ for dtype, local_shard in gpu_shards.items(): -+ # Slice the shared (max-sized) output buffer down to this -+ # block's actual AllGather output size. The buffers are -+ # sized to the *largest* block across all groups, so for any -+ # smaller block the full buffer would violate the -+ # all_gather_into_tensor contract -+ # (output.numel() == world_size * input.numel()). -+ # Repoint offsets are relative to the block's flattened -+ # buffer, so a prefix slice is safe. -+ gw = gpu_weights[dtype][: self._ag_output_sizes[dtype]] -+ torch.distributed.all_gather_into_tensor( -+ gw, -+ local_shard, -+ group=self.dp_group, -+ ) -+ evt.record(self.comm_stream) -+ -+ def _allgather_after_attn_a2a(self) -> None: -+ """Barrier callback: the block's QKV AllToAll is now enqueued. -+ -+ Recording on the compute stream here (rather than at block entry) is -+ what makes the gate effective: an event wait snapshots the records -+ enqueued before it, so the wait must be enqueued *after* the all-to-all. -+ """ -+ chunked = self._deferred_chunked_allgather -+ if chunked is not None: -+ self._deferred_chunked_allgather = None -+ assert self._a2a_event is not None -+ self._a2a_event.record(current_omni_platform.current_stream()) -+ self._issue_chunked_allgather(*chunked, gated=True) -+ return -+ deferred = self._deferred_allgather -+ if deferred is None: -+ return -+ self._deferred_allgather = None -+ assert self._a2a_event is not None -+ self._a2a_event.record(current_omni_platform.current_stream()) -+ slot, gpu_shards, gpu_weights, evt = deferred -+ self._issue_allgather(slot, gpu_shards, gpu_weights, evt, gated=True) -+ -+ def _flush_deferred_allgather(self) -> None: -+ """Issue a deferred gather whose attention never signalled. -+ -+ Keeps the double buffer live: a block whose attention is skipped -+ (cache-dit, an all-resident stage) still needs its weights, so the -+ gather is submitted ungated at the end of that block's forward. -+ """ -+ chunked = self._deferred_chunked_allgather -+ if chunked is not None: -+ self._deferred_chunked_allgather = None -+ if self._a2a_barrier is not None: -+ self._a2a_barrier.disarm() -+ self._issue_chunked_allgather(*chunked) -+ return -+ deferred = self._deferred_allgather -+ if deferred is None: -+ return -+ self._deferred_allgather = None -+ if self._a2a_barrier is not None: -+ self._a2a_barrier.disarm() -+ slot, gpu_shards, gpu_weights, evt = deferred -+ self._issue_allgather(slot, gpu_shards, gpu_weights, evt) -+ - def _wait_for_weights(self, slot: int) -> None: - """Wait until the slot holding this block is ready for compute. - -@@ -840,6 +955,10 @@ class DistributedLayerwiseOffloadHook(ModelHook): - return args, kwargs - - def post_forward(self, module: nn.Module, output: Any) -> Any: -+ # Pair with pre_forward: if the block's attention never signalled the -+ # gate, the deferred gather must still go out before this block's -+ # storage is released (offload_layer waits on _prefetch_done). -+ self._flush_deferred_allgather() - self.offload_layer() - return output - -@@ -864,6 +983,7 @@ def apply_distributed_block_hook( - tensor_transforms: dict[int, Any] | None = None, - materialization_probe_tensor: torch.Tensor | None = None, - chunk_size_bytes: int = 64 * 1024 * 1024, -+ order_allgather_after_attn_a2a: bool = False, - ) -> DistributedLayerwiseOffloadHook: - """Register a DistributedLayerwiseOffloadHook on *module*.""" - registry = HookRegistry.get_or_create(module) -@@ -881,11 +1001,32 @@ def apply_distributed_block_hook( - tensor_transforms=tensor_transforms, - materialization_probe_tensor=materialization_probe_tensor, - chunk_size_bytes=chunk_size_bytes, -+ order_allgather_after_attn_a2a=order_allgather_after_attn_a2a, - ) - registry.register_hook(DistributedLayerwiseOffloadHook._HOOK_NAME, hook) - return hook - - -+def attach_collective_gate(block: nn.Module, hook: DistributedLayerwiseOffloadHook) -> int: -+ """Hand ``hook``'s barrier to every Attention layer inside *block*. -+ -+ The attention signals the barrier right after its QKV AllToAll. The import -+ is local to keep this module independent of the attention package at import -+ time. -+ -+ Returns the number of attention layers wired (0 means the hook will rely on -+ the ``post_forward`` flush instead). -+ """ -+ from vllm_omni.diffusion.attention.layer import Attention -+ -+ attached = 0 -+ for module in block.modules(): -+ if isinstance(module, Attention): -+ module._collective_gate = hook._a2a_barrier -+ attached += 1 -+ return attached -+ -+ - def remove_distributed_block_hook(module: nn.Module) -> None: - """Remove the distributed layerwise offload hook from *module*.""" - registry: HookRegistry | None = getattr(module, "_hook_registry", None) -@@ -1566,29 +1707,45 @@ class DistributedLayerwiseOffloadBackend(OffloadBackend): - self._all_hook_groups.append(hooks) - self._blocks.append(block_list) - probes = {id(block): module_materialization_probe(block) for block in block_list} -+ # Only the DiT blocks issue a QKV AllToAll the gather can queue behind. -+ order_gate = bool(getattr(self.config, "dlo_order_allgather_after_attn_a2a", False)) and ( -+ component == DIT_COMPONENT -+ ) -+ if order_gate and not getattr(self, "_collective_gate_logged", False): -+ self._collective_gate_logged = True -+ logger.info( -+ "Distributed layerwise offload will hold each DiT block's weight AllGather " -+ "until that block's QKV AllToAll is on the compute stream" -+ ) - for block, next_block in zip( - chain((block_list[-1],), block_list[:-1]), - block_list, - strict=True, - ): -- hooks.append( -- apply_distributed_block_hook( -- block, -- next_block, -- self.device, -- group, -- group_size, -- group_rank, -- self.copy_stream, -- self.comm_stream, -- self.config.pin_cpu_memory, -- shared_buffers=[None, None], -- rank_local_mmap=self._using_rank_local_mmap if use_dit_mmap else False, -- tensor_transforms=self._mmap_transforms_by_tensor_id if use_dit_mmap else None, -- materialization_probe_tensor=probes[id(block)], -- chunk_size_bytes=self.config.chunk_size_bytes, -- ) -+ hook = apply_distributed_block_hook( -+ block, -+ next_block, -+ self.device, -+ group, -+ group_size, -+ group_rank, -+ self.copy_stream, -+ self.comm_stream, -+ self.config.pin_cpu_memory, -+ shared_buffers=[None, None], -+ rank_local_mmap=self._using_rank_local_mmap if use_dit_mmap else False, -+ tensor_transforms=self._mmap_transforms_by_tensor_id if use_dit_mmap else None, -+ materialization_probe_tensor=probes[id(block)], -+ chunk_size_bytes=self.config.chunk_size_bytes, -+ order_allgather_after_attn_a2a=order_gate, - ) -+ if hook._a2a_barrier is not None and attach_collective_gate(block, hook) == 0: -+ logger.warning( -+ "DLO AllGather ordering is enabled but block %s exposes no Attention layer; " -+ "its gather will fall back to the ungated path", -+ block.__class__.__name__, -+ ) -+ hooks.append(hook) - - # hooks = [last -> first, block0 -> block1, ...]. Alternating slots - # keep a prefetch from overwriting the current block for any ring size. -diff --git a/vllm_omni/engine/arg_utils.py b/vllm_omni/engine/arg_utils.py -index 0a876b9..bf8dcdc 100644 ---- a/vllm_omni/engine/arg_utils.py -+++ b/vllm_omni/engine/arg_utils.py -@@ -807,6 +807,7 @@ class OrchestratorArgs: - dlo_use_allgather: bool = True - dlo_chunk_size_mb: int = 64 - dlo_resident_layers: int = 0 -+ dlo_order_allgather_after_attn_a2a: bool = False - host_weight_runtime_mode: str = "disabled" - host_weight_runtime_root: str | None = None - dlo_host_registration_limit_gib: float = 0.0 -diff --git a/vllm_omni/entrypoints/cli/serve.py b/vllm_omni/entrypoints/cli/serve.py -index f2b55e7..aff71b5 100644 ---- a/vllm_omni/entrypoints/cli/serve.py -+++ b/vllm_omni/entrypoints/cli/serve.py -@@ -1041,6 +1041,17 @@ class OmniServeCommand(CLISubcommand): - help="Full-chunk target size in MiB for distributed layerwise offload " - "(each chunk is one host-to-device copy + AllGather unit).", - ) -+ omni_config_group.add_argument( -+ "--dlo-order-allgather-after-attn-a2a", -+ action="store_true", -+ default=False, -+ help=( -+ "Hold each DiT block's weight AllGather until that block's QKV AllToAll " -+ "has been enqueued, so the background gather does not take the collective " -+ "links ahead of the critical-path all-to-all. Requires distributed " -+ "layerwise offload with AllGather." -+ ), -+ ) - omni_config_group.add_argument( - "--host-weight-runtime-mode", - choices=("disabled", "preferred", "required"), diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c13-source-identity.json b/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c13-source-identity.json deleted file mode 100644 index 939545aaf..000000000 --- a/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c13-source-identity.json +++ /dev/null @@ -1,21 +0,0 @@ -{ - "head": "e89b1d40d37f7f72767892a56fa7980ea572ba8e", - "tree": "cc5924c4270350d34c8d3f2213bce95d88fa2210", - "base": "be319692b96413918973b5bd399cdda3f320c574", - "branch": "rsi/h3-297-c1-dlo-stack", - "files": { - "tests/diffusion/distributed/test_collective_gate.py": "2b33f73dff9c79118cc2b1bd4d50e3542ebc45b7bf029b38c17abbef37e4b917", - "vllm_omni/config/omni_config.py": "6894120a330474f718d43bf2bf7928892eb996152839ce6469c34d18a7df09d1", - "vllm_omni/config/stage_config.py": "0752e8cc1c6f1021f737830d08663cfc118923a83fd909c607f9ec3ab2a3d2fb", - "vllm_omni/diffusion/attention/layer.py": "76c5fce3dabd23e39a8890316e1691c03bc203066cb60af5611ec4790eefc632", - "vllm_omni/diffusion/data.py": "5db83d5c71cb27cc7810d35fcfc891da80f0b003adbb16bdecc3816a63305e11", - "vllm_omni/diffusion/distributed/collective_gate.py": "01c6317cfa615840681b8bb8b8513fa3736d26e845a24ad9e1ab7f87489b962f", - "vllm_omni/diffusion/offloader/base.py": "533c793119e4cdc14a2ea3c6cb5bf97ffede673befeab3f0fb8dbbf51e655379", - "vllm_omni/diffusion/offloader/distributed_layerwise_backend.py": "d441481513aba9e90a91265f4bba3f53f8766197be8c1302c3c714e690d5ea76", - "vllm_omni/engine/arg_utils.py": "52569e3688cc708729a48ca0a2c875c28193733c2f934f6ff89f6c815e4009c7", - "vllm_omni/entrypoints/cli/serve.py": "350461dfca565bc0b07087ac7af1e27b0e247f5eac717e350b637e37f62dd088", - "tests/diffusion/offloader/helpers.py": "729c73abba3b9a12911282d4bcd3cf8bfa148fbcbfdc2e2b7c86000e387fa687" - }, - "published": false, - "patch_base": "be319692b96413918973b5bd399cdda3f320c574" -} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c15-candidate.patch b/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c15-candidate.patch deleted file mode 100644 index 5410afa5b..000000000 --- a/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c15-candidate.patch +++ /dev/null @@ -1,890 +0,0 @@ -diff --git a/tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py b/tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py -new file mode 100644 -index 0000000..ce8464c ---- /dev/null -+++ b/tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py -@@ -0,0 +1,380 @@ -+# SPDX-License-Identifier: Apache-2.0 -+# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project -+ -+from datetime import timedelta -+from types import SimpleNamespace -+ -+import pytest -+import torch -+import torch.distributed as dist -+import torch.nn as nn -+ -+from tests.helpers.mark import hardware_marks -+from vllm_omni.diffusion.models.minimax_h3 import minimax_h3_blocks as blocks -+from vllm_omni.diffusion.models.minimax_h3.attention.flash_h3 import _prefix_bnsd, _subset_prefix_bnsd -+from vllm_omni.diffusion.models.minimax_h3.video_row_cache import MiniMaxH3VideoRowCache -+from vllm_omni.platforms import current_omni_platform -+ -+pytestmark = [pytest.mark.core_model, pytest.mark.diffusion, *hardware_marks(res={"npu": "A3"}, num_cards=4)] -+ -+ -+def state_for_rows(device="cpu", start=0, length=4): -+ state = MiniMaxH3VideoRowCache(torch.tensor([1, 2], device=device), seq_len=4, used_len=3, num_layers=1) -+ state.prepare_local_span(start, length) -+ return state -+ -+ -+def anchor(state, x): -+ for step in range(5): -+ state.begin_step(step) -+ state.capture(0, x, x + 0.125) -+ state.end_step() -+ state.begin_step(5) -+ -+ -+def test_fixed_schedule_and_row_ownership_excludes_padding(): -+ assert [i for i in range(50) if MiniMaxH3VideoRowCache.is_full_step(i)] == [ -+ 0, -+ 1, -+ 2, -+ 3, -+ 4, -+ 9, -+ 14, -+ 19, -+ 24, -+ 29, -+ 34, -+ 39, -+ 44, -+ 46, -+ 47, -+ 48, -+ 49, -+ ] -+ state = state_for_rows() -+ assert state.global_dense_positions.tolist() == [0] -+ assert state.local_dense_positions.tolist() == [0] -+ assert state.local_video_positions.tolist() == [1, 2] -+ with pytest.raises(RuntimeError, match="ownership"): -+ state.prepare_local_span(0, 3) -+ -+ -+@pytest.mark.parametrize("positions", [[1, 1], [2, 1], [-1], [3]]) -+def test_invalid_target_positions_are_rejected(positions): -+ with pytest.raises(ValueError): -+ MiniMaxH3VideoRowCache(torch.tensor(positions), seq_len=4, used_len=3, num_layers=1) -+ -+ -+def test_delta_is_fp16_and_restore_owns_output(): -+ state = state_for_rows() -+ x = torch.tensor([[1, 2], [3, 4], [5, 6], [7, 8]], dtype=torch.bfloat16) -+ anchor(state, x) -+ assert state._deltas[0].dtype == torch.float16 -+ before = x.clone() -+ output = state.restore(0, x) -+ assert torch.equal(output[1:3], x[1:3] + 0.125) -+ assert torch.equal(output[[0, 3]], x[[0, 3]]) -+ output.zero_() -+ assert torch.equal(x, before) -+ state.end_step() -+ state.close() -+ assert state.closed and not state._deltas and state.global_dense_positions.numel() == 0 -+ -+ -+class CountMLP(nn.Module): -+ def __init__(self): -+ super().__init__() -+ self.rows = [] -+ -+ def forward(self, x): -+ self.rows.append(x.shape[0]) -+ return x + 0.25 -+ -+ -+def tiny_block(): -+ block = object.__new__(blocks.MiniMaxH3DiTBlock) -+ nn.Module.__init__(block) -+ block.layer_index = 0 -+ block.norm1 = SimpleNamespace(weight=torch.ones(2), variance_epsilon=1e-6) -+ block.norm2 = SimpleNamespace(weight=torch.ones(2), variance_epsilon=1e-6) -+ block.adaln_proj = lambda t: ( -+ torch.zeros(3, 2), -+ torch.zeros(3, 2), -+ torch.ones(3, 2), -+ torch.zeros(3, 2), -+ torch.zeros(3, 2), -+ torch.ones(3, 2), -+ ) -+ block.attn = lambda x, **kwargs: x * 0.5 -+ block.mlp = CountMLP() -+ return block -+ -+ -+@pytest.mark.parametrize("start,length", [(0, 4), (1, 2)]) -+def test_real_block_dense_path_and_partial_mlp_empty_rows(start, length): -+ state = state_for_rows(start=start, length=length) -+ block = tiny_block() -+ x = torch.arange(length * 2).reshape(length, 2).to(torch.bfloat16) -+ before = x.clone() -+ kwargs = dict( -+ t_emb=torch.zeros(1), -+ combined_indices=torch.zeros(length, dtype=torch.long), -+ rope_table=None, -+ cu_seqlens=torch.tensor([0, 3, 4], dtype=torch.int32), -+ max_seqlen=3, -+ packed_total=4, -+ ) -+ expected = block(x, **kwargs) -+ for step in range(5): -+ state.begin_step(step) -+ actual = block(x, video_row_cache=state, **kwargs) -+ assert torch.equal(actual.view(torch.uint8), expected.view(torch.uint8)) -+ state.end_step() -+ block.mlp.rows.clear() -+ state.begin_step(5) -+ actual = block(x, video_row_cache=state, **kwargs) -+ state.end_step() -+ rows = state.local_dense_positions -+ assert block.mlp.rows == ([rows.numel()] if rows.numel() else []) -+ assert torch.equal(actual.index_select(0, rows), expected.index_select(0, rows)) -+ assert torch.equal(x, before) -+ -+ -+@pytest.mark.parametrize("failure", ["success", "model", "cancel"]) -+def test_request_state_closed_and_removed_on_every_exit(monkeypatch, failure): -+ from vllm_omni.diffusion.models.minimax_h3 import denoise_loop as loop -+ -+ monkeypatch.setenv("H3_VIDEO_ROW_CACHE", "1") -+ monkeypatch.setattr(loop.current_omni_platform, "is_npu", lambda: False) -+ monkeypatch.setattr(loop, "minimax_h3_publish_denoise_progress", lambda *args: None) -+ state = MiniMaxH3VideoRowCache(torch.tensor([1]), seq_len=4, used_len=3, num_layers=1) -+ -+ class Model: -+ def create_video_row_cache(self, *args, **kwargs): -+ return state -+ -+ def __call__(self, **kwargs): -+ assert kwargs["video_row_cache"] is state -+ if failure == "model": -+ raise RuntimeError("model failed") -+ state.prepare_local_span(0, 4) -+ x = torch.ones(4, 2, dtype=torch.bfloat16) -+ if state.partial: -+ state.restore(0, x) -+ else: -+ state.capture(0, x, x + 0.125) -+ return torch.zeros(2, 96), torch.zeros(1, 32) -+ -+ class Branch: -+ task = "ref2va" -+ seq_len = 4 -+ used_len = 3 -+ img_pos = torch.tensor([0, 1]) -+ update_mask = torch.tensor([False, True]) -+ update_mask_dev = update_mask -+ audio_update_mask_dev = torch.tensor([True]) -+ locked_audio_rows = None -+ static_kwargs = {"video_token_layout": SimpleNamespace(video_spans=[SimpleNamespace(role="target", start=1)])} -+ -+ def forward_kwargs(self, **kwargs): -+ return dict(self.static_kwargs) -+ -+ monkeypatch.setattr( -+ loop, -+ "minimax_h3_prepare_denoise_rows", -+ lambda **kwargs: (kwargs["initial_video_rows"], kwargs["initial_audio_rows"], torch.zeros(1, 96), None), -+ ) -+ -+ def cancellation(**kwargs): -+ if failure == "cancel": -+ raise RuntimeError("request cancelled") -+ -+ monkeypatch.setattr(loop, "check_request_cancellation", cancellation) -+ branch, model = Branch(), Model() -+ kwargs = dict( -+ model=model, -+ positive=branch, -+ initial_video_rows=torch.zeros(2, 96), -+ initial_audio_rows=torch.zeros(1, 32), -+ keyframe_cond_rows=torch.zeros(1, 96), -+ sigmas_video=[1 - i / 50 for i in range(51)], -+ sigmas_audio=[1 - i / 50 for i in range(51)], -+ device=torch.device("cpu"), -+ ) -+ if failure == "success": -+ loop.minimax_h3_denoise_loop(**kwargs) -+ assert state.full_steps == 17 and state.partial_steps == 33 -+ else: -+ with pytest.raises(RuntimeError): -+ loop.minimax_h3_denoise_loop(**kwargs) -+ assert state.closed and not state._deltas and "video_row_cache" not in branch.static_kwargs -+ assert model._video_row_cache_stats["closed"] -+ -+ -+def test_default_factory_does_not_create_state(monkeypatch): -+ from vllm_omni.diffusion.models.minimax_h3.minimax_h3_transformer import MiniMaxH3DiTModel -+ -+ monkeypatch.delenv("H3_VIDEO_ROW_CACHE", raising=False) -+ assert ( -+ MiniMaxH3DiTModel.create_video_row_cache(None, torch.tensor([1]), seq_len=4, used_len=3, num_steps=50) is None -+ ) -+ -+ -+@pytest.mark.parametrize("ref2va", [False, True]) -+def test_real_packing_task_selects_ref2va_and_rejects_fl2va(monkeypatch, ref2va): -+ from vllm_omni.diffusion.models.minimax_h3 import denoise_loop as loop -+ from vllm_omni.diffusion.models.minimax_h3.packed_sequence import ( -+ minimax_h3_packed_sequence, -+ minimax_h3_packed_sequence_ref2va_blocks, -+ ) -+ -+ common = dict(text_len=2, latent_t=2, latent_h=4, latent_w=4, audio_t=2) -+ packed = ( -+ minimax_h3_packed_sequence_ref2va_blocks(**common, ref_blocks=[{"kind": "image", "latent_h": 4, "latent_w": 4}]) -+ if ref2va -+ else minimax_h3_packed_sequence(**common, include_keyframe_cond=True, keyframe_frame_indices=[0], frame_count=5) -+ ) -+ branch = loop.MiniMaxH3DenoiseBranch( -+ packed=packed, -+ text_embeddings=torch.zeros(2, 5120), -+ token_tags=packed["token_tags"], -+ device=torch.device("cpu"), -+ task="ref2va" if ref2va else "fl2va", -+ ) -+ layout = branch.static_kwargs["video_token_layout"] -+ assert not any(span.role == "reference" for span in layout.video_spans) -+ monkeypatch.setenv("H3_VIDEO_ROW_CACHE", "1") -+ monkeypatch.setattr(loop.current_omni_platform, "is_npu", lambda: False) -+ monkeypatch.setattr(loop, "minimax_h3_publish_denoise_progress", lambda *args: None) -+ monkeypatch.setattr(loop, "check_request_cancellation", lambda **kwargs: None) -+ states = [] -+ -+ class Model: -+ def create_video_row_cache(self, positions, **kwargs): -+ state = MiniMaxH3VideoRowCache( -+ positions, seq_len=kwargs["seq_len"], used_len=kwargs["used_len"], num_layers=1 -+ ) -+ state.prepare_local_span(0, branch.seq_len) -+ states.append(state) -+ return state -+ -+ def __call__(self, **kwargs): -+ state = kwargs["video_row_cache"] -+ x = torch.ones(branch.seq_len, 2, dtype=torch.bfloat16) -+ if state.partial: -+ state.restore(0, x) -+ else: -+ state.capture(0, x, x + 0.125) -+ return torch.zeros(branch.img_pos.numel(), 96), torch.zeros(branch.audio_pos.numel(), 32) -+ -+ kwargs = dict( -+ model=Model(), -+ positive=branch, -+ initial_video_rows=torch.zeros(branch.img_pos.numel(), 96), -+ initial_audio_rows=torch.zeros(branch.audio_pos.numel(), 32), -+ keyframe_cond_rows=torch.zeros(int((~branch.update_mask).sum()), 96), -+ sigmas_video=[1 - i / 50 for i in range(51)], -+ sigmas_audio=[1 - i / 50 for i in range(51)], -+ device=torch.device("cpu"), -+ ) -+ if ref2va: -+ loop.minimax_h3_denoise_loop(**kwargs) -+ assert len(states) == 1 and states[0].closed -+ assert states[0].full_steps == 17 and states[0].partial_steps == 33 -+ else: -+ with pytest.raises(ValueError, match="requires Ref2VA"): -+ loop.minimax_h3_denoise_loop(**kwargs) -+ assert not states -+ -+ -+@pytest.mark.parametrize("start,length", [(0, 4), (1, 2)]) -+def test_attention_keeps_collective_path_and_projects_only_dense_local_rows(start, length): -+ state = state_for_rows(start=start, length=length) -+ x = torch.ones(length, 2, dtype=torch.bfloat16) -+ anchor(state, x) -+ attention = object.__new__(blocks.MiniMaxH3Attention) -+ nn.Module.__init__(attention) -+ attention.num_heads = 1 -+ attention.head_dim = 2 -+ attention.vdn = None -+ attention.to_gate_compress = None -+ calls, projected_rows = [], [] -+ attention._project_qkv = lambda *args: (x.reshape(length, 1, 2),) * 5 -+ -+ def packed(q, k, v, **kwargs): -+ calls.append(q.shape[0]) -+ assert kwargs["video_row_cache"] is state -+ return q -+ -+ def project(rows): -+ projected_rows.append(rows.shape[0]) -+ return rows * 2, None -+ -+ attention._run_packed_attention = packed -+ attention.out_proj = project -+ output = attention( -+ x, -+ rope_table=None, -+ cu_seqlens=torch.tensor([0, 3, 4], dtype=torch.int32), -+ max_seqlen=3, -+ packed_total=4, -+ video_row_cache=state, -+ ) -+ rows = state.local_dense_positions -+ assert calls == [length] -+ assert projected_rows == ([rows.numel()] if rows.numel() else []) -+ assert torch.equal(output.index_select(0, rows), (x * 2).index_select(0, rows)) -+ assert torch.count_nonzero(output.index_select(0, state.local_video_positions)) == 0 -+ -+ -+def _native_worker(rank): -+ import torch_npu # noqa: F401 -+ -+ torch.npu.set_device(rank) -+ dist.init_process_group( -+ "hccl", init_method="tcp://127.0.0.1:29597", rank=rank, world_size=4, timeout=timedelta(seconds=120) -+ ) -+ try: -+ torch.manual_seed(19) -+ full = [torch.randn(32, 4, 128, dtype=torch.bfloat16).npu() for _ in range(3)] -+ exchanged = [] -+ for tensor in full: -+ local = tensor[rank * 8 : (rank + 1) * 8] -+ send = local.reshape(8, 4, 1, 128).permute(1, 0, 2, 3).contiguous() -+ receive = torch.empty_like(send) -+ dist.all_to_all_single(receive, send) -+ exchanged.append(receive.reshape(1, 32, 1, 128)) -+ state = MiniMaxH3VideoRowCache(torch.arange(8, 24, device="npu"), seq_len=32, used_len=30, num_layers=1) -+ state.prepare_local_span(rank * 8, 8) -+ x = torch.ones(8, 2, dtype=torch.bfloat16, device="npu") -+ anchor(state, x) -+ output = _subset_prefix_bnsd(exchanged[0], exchanged[1][:, :30], exchanged[2][:, :30], 128**-0.5, state) -+ send = output.reshape(4, 8, 1, 128).contiguous() -+ receive = torch.empty_like(send) -+ dist.all_to_all_single(receive, send) -+ local_output = receive.permute(1, 0, 2, 3).reshape(8, 4, 128) -+ expected = _prefix_bnsd(full[0].unsqueeze(0), full[1][:30].unsqueeze(0), full[2][:30].unsqueeze(0), 128**-0.5)[ -+ 0 -+ ] -+ rows = state.local_dense_positions -+ torch.testing.assert_close( -+ local_output.index_select(0, rows).float(), -+ expected[rank * 8 : (rank + 1) * 8].index_select(0, rows).float(), -+ atol=0.03125, -+ rtol=0.02, -+ ) -+ assert state.subset_attention_calls == 1 and state.subset_query_rows == 14 -+ assert rows.numel() == [8, 0, 0, 6][rank] -+ state.restore(0, x) -+ state.end_step() -+ state.close() -+ torch.npu.synchronize() -+ dist.barrier() -+ finally: -+ dist.destroy_process_group() -+ -+ -+@pytest.mark.skipif(not current_omni_platform.is_npu(), reason="Requires four native Ascend devices") -+def test_native_four_rank_subset_attention_with_empty_local_dense_rows(): -+ torch.multiprocessing.spawn(_native_worker, nprocs=4, join=True) -diff --git a/vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py b/vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py -index 6ebfb10..044c1f0 100644 ---- a/vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py -+++ b/vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py -@@ -36,6 +36,16 @@ def _prefix_bnsd(query: torch.Tensor, key: torch.Tensor, value: torch.Tensor, sc - return output.reshape(batch, heads, rows, dim).transpose(1, 2).contiguous() - - -+def _subset_prefix_bnsd(query, key, value, scale, state): -+ """Compute only fresh conditioning queries; preserve the shared SP shape.""" -+ rows = state.global_dense_positions -+ dense = _prefix_bnsd(query.index_select(1, rows), key, value, scale) -+ output = torch.zeros_like(query) -+ output.index_copy_(1, rows, dense) -+ state.record_subset_attention(rows.numel()) -+ return output -+ -+ - class MiniMaxH3FlashAttentionImpl(FlashAttentionImpl): - """Use independent head batches for explicit long BF16 prefix attention. - -@@ -51,7 +61,10 @@ class MiniMaxH3FlashAttentionImpl(FlashAttentionImpl): - value: torch.Tensor, - attn_metadata: AttentionMetadata | None = None, - ) -> torch.Tensor: -+ state = attn_metadata.extra.get("minimax_h3_video_row_cache") if attn_metadata is not None else None - if os.environ.get("MINDIE_SD_FA_TYPE") != "fused_attn_score" or attn_metadata is None: -+ if state is not None: -+ raise RuntimeError("H3 video-row cache requires the explicit fused attention path") - return super().forward_fa_npu(query, key, value, attn_metadata) - extra = attn_metadata.extra - if ( -@@ -68,9 +81,17 @@ class MiniMaxH3FlashAttentionImpl(FlashAttentionImpl): - or query.shape[2:] != value.shape[2:] - or key.shape[1] != value.shape[1] - ): -+ if state is not None: -+ raise RuntimeError("H3 video-row cache received unsupported fused attention inputs") - return super().forward_fa_npu(query, key, value, attn_metadata) - resolved = self._resolve_packed_seq_npu(query, key, extra) - if resolved is None: -+ if state is not None: -+ raise RuntimeError("H3 video-row cache requires a resolved packed prefix") - return super().forward_fa_npu(query, key, value, attn_metadata) - used_k = resolved[1][0] -+ if state is not None: -+ if not state.partial or used_k != state.used_len or query.shape[1] != state.seq_len: -+ raise RuntimeError("H3 video-row cache attention layout changed") -+ return _subset_prefix_bnsd(query, key[:, :used_k], value[:, :used_k], self.softmax_scale, state) - return _prefix_bnsd(query, key[:, :used_k], value[:, :used_k], self.softmax_scale) -diff --git a/vllm_omni/diffusion/models/minimax_h3/denoise_loop.py b/vllm_omni/diffusion/models/minimax_h3/denoise_loop.py -index 8cca219..07fb372 100644 ---- a/vllm_omni/diffusion/models/minimax_h3/denoise_loop.py -+++ b/vllm_omni/diffusion/models/minimax_h3/denoise_loop.py -@@ -9,6 +9,7 @@ rows stay pinned to their noised step-0 anchors. - - from __future__ import annotations - -+import os - from collections.abc import Callable - from contextlib import AbstractContextManager, nullcontext - from typing import Any -@@ -69,7 +70,9 @@ class MiniMaxH3DenoiseBranch: - text_embeddings: torch.Tensor, - token_tags: torch.Tensor, - device: torch.device, -+ task: str | None = None, - ) -> None: -+ self.task = task - seq_len = int(packed["seq_len"]) - self.seq_len = seq_len - self.img_pos = packed["img_pos"].view(-1).to(torch.long) -@@ -380,6 +383,24 @@ def minimax_h3_denoise_loop( - positive.static_kwargs["reference_kv_state"] = reference_kv_state - - try: -+ video_row_cache = None -+ create_video_cache = getattr(model, "create_video_row_cache", None) -+ if os.environ.get("H3_VIDEO_ROW_CACHE") == "1": -+ if ( -+ not callable(create_video_cache) -+ or video_edit is not None -+ or audio_edit is not None -+ or keyframe_cond_rows is None -+ or positive.task != "ref2va" -+ ): -+ raise ValueError("H3 video-row cache requires Ref2VA image references without editing") -+ video_row_cache = create_video_cache( -+ positive.img_pos[positive.update_mask].to(device), -+ seq_len=positive.seq_len, -+ used_len=positive.used_len, -+ num_steps=num_steps, -+ ) -+ positive.static_kwargs["video_row_cache"] = video_row_cache - for step in range(num_steps): - check_request_cancellation() - step_cm = step_profiler(step) if step_profiler is not None else nullcontext() -@@ -388,6 +409,8 @@ def minimax_h3_denoise_loop( - s_a = sigmas_audio[step] - if reference_kv_state is not None: - reference_kv_state.begin_step(step, sigma=float(s_v)) -+ if video_row_cache is not None: -+ video_row_cache.begin_step(step) - # Publish where we are so step-gated attention features (the dense - # warmup of RAINFUSION_ATTN, the timestep gate of TRTLLM_ATTN) can - # see it. Gates use the scheduler-style descending timestep, which -@@ -430,6 +453,8 @@ def minimax_h3_denoise_loop( - v_video, v_audio = model(**fk) - if reference_kv_state is not None: - reference_kv_state.end_step() -+ if video_row_cache is not None: -+ video_row_cache.end_step() - mv_video_t = v_video.float()[update] - mv_audio_t = v_audio.float()[audio_update] - -@@ -476,6 +501,10 @@ def minimax_h3_denoise_loop( - - finally: - minimax_h3_publish_denoise_progress(None, None, None) -+ if video_row_cache is not None: -+ positive.static_kwargs.pop("video_row_cache", None) -+ video_row_cache.close() -+ model._video_row_cache_stats = video_row_cache.stats() - if reference_kv_state is not None: - positive.static_kwargs.pop("reference_kv_state", None) - reference_kv_state.close() -diff --git a/vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py b/vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py -index 09f2e7d..94ef291 100644 ---- a/vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py -+++ b/vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py -@@ -45,6 +45,7 @@ if TYPE_CHECKING: - from vllm.model_executor.layers.quantization.base_config import QuantizationConfig - - from .reference_kv_cache import MiniMaxH3ReferenceKVTier2State -+ from .video_row_cache import MiniMaxH3VideoRowCache - - _BF16_DTYPE = torch.bfloat16 - _FP32_DTYPE = torch.float32 -@@ -291,6 +292,7 @@ class MiniMaxH3Attention(nn.Module): - reference_kv_layer_index: int | None = None, - reference_kv_compact: bool = False, - vdn_window: VDNLayout | None = None, -+ video_row_cache: MiniMaxH3VideoRowCache | None = None, - ) -> torch.Tensor: - """Run packed attention as a small eager island. - -@@ -368,6 +370,8 @@ class MiniMaxH3Attention(nn.Module): - "vsa_h3_ref_policy": self.vsa_ref_policy, - "vsa_h3_ref_keep_rate": self.vsa_ref_keep_rate, - } -+ if video_row_cache is not None and video_row_cache.partial: -+ extra["minimax_h3_video_row_cache"] = video_row_cache - if gate_compress is not None: - extra["gate_compress"] = gate_compress.unsqueeze(0) - if video_layout is not None and video_layout.video_spans: -@@ -455,6 +459,7 @@ class MiniMaxH3Attention(nn.Module): - reference_kv_layer_index: int | None = None, - reference_kv_compact: bool = False, - vdn_window: VDNLayout | None = None, -+ video_row_cache: MiniMaxH3VideoRowCache | None = None, - ) -> torch.Tensor: - """x: [T, hidden] packed thd rows -> [T, hidden]. - -@@ -508,10 +513,19 @@ class MiniMaxH3Attention(nn.Module): - reference_kv_layer_index=reference_kv_layer_index, - reference_kv_compact=reference_kv_compact, - vdn_window=vdn_window, -+ **({"video_row_cache": video_row_cache} if video_row_cache is not None else {}), - ) - if self.vdn is not None: - out = self.vdn.gate_softmax(out, x) - out = out.reshape(total, self.num_heads * self.head_dim) -+ if video_row_cache is not None and video_row_cache.partial: -+ rows = video_row_cache.local_dense_positions -+ assert rows is not None -+ projected = x.new_zeros((total, x.shape[-1])) -+ if rows.numel(): -+ dense, _ = self.out_proj(out.index_select(0, rows)) -+ projected.index_copy_(0, rows, dense) -+ return projected - out, _ = self.out_proj(out) - if self.vdn is not None: - out = self.vdn.add_linear(out, x, q_raw, k_raw, v, vdn_window) -@@ -742,6 +756,7 @@ class MiniMaxH3DiTBlock(nn.Module): - reference_kv_state: MiniMaxH3ReferenceKVTier2State | None = None, - reference_kv_compact: bool = False, - vdn_window: VDNLayout | None = None, -+ video_row_cache: MiniMaxH3VideoRowCache | None = None, - ) -> torch.Tensor: - """x: [T, H]; t_emb: [M, t_dim]; combined_indices: [T] - (= inverse_indices * modality_num + token_tags.clamp(min=0)). -@@ -759,6 +774,7 @@ class MiniMaxH3DiTBlock(nn.Module): - gate_mlp, - ) = self.adaln_proj(t_emb) - -+ block_input = x - residual = x - h = rms_norm_indexed_scale_shift( - x, -@@ -782,6 +798,7 @@ class MiniMaxH3DiTBlock(nn.Module): - reference_kv_layer_index=self.layer_index, - reference_kv_compact=reference_kv_compact, - vdn_window=vdn_window, -+ **({"video_row_cache": video_row_cache} if video_row_cache is not None else {}), - ) - x, h = indexed_gate_rms_norm_scale_shift( - residual, -@@ -794,5 +811,22 @@ class MiniMaxH3DiTBlock(nn.Module): - self.norm2.variance_epsilon, - ) - residual = x -+ if video_row_cache is not None and video_row_cache.partial: -+ rows = video_row_cache.local_dense_positions -+ assert rows is not None -+ output = video_row_cache.restore(self.layer_index, block_input) -+ if rows.numel(): -+ dense = indexed_gate( -+ residual.index_select(0, rows), -+ gate_mlp, -+ self.mlp(h.index_select(0, rows)), -+ combined_indices.index_select(0, rows), -+ ) -+ output.index_copy_(0, rows, dense) -+ video_row_cache.partial_mlp_rows += rows.numel() -+ return output - h = self.mlp(h) -- return indexed_gate(residual, gate_mlp, h, combined_indices) -+ output = indexed_gate(residual, gate_mlp, h, combined_indices) -+ if video_row_cache is not None: -+ video_row_cache.capture(self.layer_index, block_input, output) -+ return output -diff --git a/vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py b/vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py -index 9518828..2cf7aae 100644 ---- a/vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py -+++ b/vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py -@@ -10,6 +10,7 @@ layout. - from __future__ import annotations - - import math -+import os - from bisect import bisect_left - from collections.abc import Iterable, Mapping - from dataclasses import replace -@@ -66,6 +67,8 @@ if TYPE_CHECKING: - from vllm_omni.diffusion.data import OmniDiffusionConfig - from vllm_omni.diffusion.models.minimax_h3.reference_kv_cache import MiniMaxH3ReferenceKVTier2State - -+ from .video_row_cache import MiniMaxH3VideoRowCache -+ - logger = init_logger(__name__) - - -@@ -245,6 +248,7 @@ _FORWARD_SUPPORTED_KWARGS = frozenset( - "control_rows", - "control_context_scale", - "reference_kv_state", -+ "video_row_cache", - } - ) - -@@ -1232,6 +1236,35 @@ class MiniMaxH3DiTModel(nn.Module): - t_emb = self.time_embedder(unique_timesteps) - return embeddings, t_emb - -+ def create_video_row_cache( -+ self, target_positions: torch.Tensor, *, seq_len: int, used_len: int, num_steps: int -+ ) -> MiniMaxH3VideoRowCache | None: -+ if os.environ.get("H3_VIDEO_ROW_CACHE") != "1": -+ return None -+ p = self.parallel_config -+ if ( -+ not current_omni_platform.is_npu() -+ or num_steps != 50 -+ or p.tensor_parallel_size != 1 -+ or p.data_parallel_size not in (None, 1) -+ or p.cfg_parallel_size != 1 -+ or p.pipeline_parallel_size != 1 -+ or p.ulysses_degree != 4 -+ or p.ring_degree != 1 -+ or p.allgather_degree != 1 -+ or p.ulysses_mode != "strict" -+ or self.od_config.cache_backend not in (None, "none") -+ or self.od_config.kv_offload_config is not None -+ or self.vdn_config is not None -+ or os.environ.get("MINDIE_SD_FA_TYPE") != "fused_attn_score" -+ or any(block.attn.attention.attn_backend.get_name() != "FLASH_ATTN" for block in self.blocks) -+ ): -+ raise ValueError("H3 video-row cache requires dense NPU BF16 Ref2VA with strict Ulysses4, TP/DP/CFG/PP1") -+ from .video_row_cache import MiniMaxH3VideoRowCache -+ -+ self._video_row_cache_stats = None -+ return MiniMaxH3VideoRowCache(target_positions, seq_len=seq_len, used_len=used_len, num_layers=len(self.blocks)) -+ - def create_reference_kv_state( - self, - *, -@@ -1308,6 +1341,14 @@ class MiniMaxH3DiTModel(nn.Module): - refiner_max = int(self._psp_field(refiner_psp, "refiner_packed_seq_params", "max_seqlen_q")) - video_layout = kwargs.get("video_token_layout") - reference_kv_state = kwargs.get("reference_kv_state") -+ video_row_cache = kwargs.get("video_row_cache") -+ if video_row_cache is not None and ( -+ num_requests != 1 -+ or reference_kv_state is not None -+ or self.vdn_config is not None -+ or kwargs.get("control_rows") is not None -+ ): -+ raise ValueError("H3 video-row cache cannot combine with packed requests, reference KV, VDN or control") - vdn_window = None - if self.vdn_config is not None: - # Text rows lead the packed document. -@@ -1442,6 +1483,11 @@ class MiniMaxH3DiTModel(nn.Module): - if reference_kv_state is not None: - reference_kv_state.set_local_sequence_rows(hidden.shape[0]) - -+ if video_row_cache is not None: -+ if hidden.shape[0] != local_len: -+ raise RuntimeError("H3 video-row cache requires the strict local sequence shard") -+ video_row_cache.prepare_local_span(local_start, local_len) -+ - block_video_layout = video_layout - if compact_reference: - block_video_layout = None -@@ -1505,6 +1551,7 @@ class MiniMaxH3DiTModel(nn.Module): - reference_kv_state=reference_kv_state, - reference_kv_compact=compact_reference, - vdn_window=vdn_window, -+ **({"video_row_cache": video_row_cache} if video_row_cache is not None else {}), - ) - if block_index in hints: - hidden = hidden + hints.pop(block_index) * control_scale -diff --git a/vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py b/vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py -index 476c3cc..ebce8fc 100644 ---- a/vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py -+++ b/vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py -@@ -2115,6 +2115,7 @@ class MiniMaxH3Pipeline( - text_embeddings=text_embeddings, - token_tags=tags, - device=self.device, -+ task=task, - ) - - if control_rows is not None: -diff --git a/vllm_omni/diffusion/models/minimax_h3/video_row_cache.py b/vllm_omni/diffusion/models/minimax_h3/video_row_cache.py -new file mode 100644 -index 0000000..f8437f7 ---- /dev/null -+++ b/vllm_omni/diffusion/models/minimax_h3/video_row_cache.py -@@ -0,0 +1,155 @@ -+# SPDX-License-Identifier: Apache-2.0 -+# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project -+"""Request-owned, experimental target-video block residuals for H3 Ref2VA.""" -+ -+from __future__ import annotations -+ -+import torch -+ -+ -+class MiniMaxH3VideoRowCache: -+ """Keep non-video computation dense while reusing video block updates. -+ -+ The first experimental profile is deliberately fixed. No state is shared -+ across requests, and every rank retains all attention collectives. -+ """ -+ -+ def __init__(self, target_positions: torch.Tensor, *, seq_len: int, used_len: int, num_layers: int) -> None: -+ positions = target_positions.detach().cpu().tolist() -+ if not 0 < used_len <= seq_len or not positions or num_layers < 1: -+ raise ValueError("video-row cache requires a nonempty valid packed document") -+ if positions != sorted(set(positions)) or positions[0] < 0 or positions[-1] >= used_len: -+ raise ValueError("target-video positions must be sorted unique valid rows") -+ target_set = set(positions) -+ dense = [i for i in range(used_len) if i not in target_set] -+ if not dense: -+ raise ValueError("video-row cache requires non-video conditioning rows") -+ self.seq_len = seq_len -+ self.used_len = used_len -+ self.num_layers = num_layers -+ self._target_positions = positions -+ self.global_dense_positions = torch.tensor(dense, dtype=torch.long, device=target_positions.device) -+ self.local_dense_positions: torch.Tensor | None = None -+ self.local_video_positions: torch.Tensor | None = None -+ self._local_span: tuple[int, int] | None = None -+ self._deltas: dict[int, torch.Tensor] = {} -+ self._step = -1 -+ self._blocks_this_step = 0 -+ self.partial = False -+ self.closed = False -+ self.full_steps = 0 -+ self.partial_steps = 0 -+ self.full_blocks = 0 -+ self.partial_blocks = 0 -+ self.subset_attention_calls = 0 -+ self.subset_query_rows = 0 -+ self.omitted_query_rows = 0 -+ self.partial_mlp_rows = 0 -+ self.cache_bytes = 0 -+ -+ @staticmethod -+ def is_full_step(step: int) -> bool: -+ if not 0 <= step < 50: -+ raise ValueError("the video-row cache profile requires 50 steps") -+ return step < 4 or step >= 46 or (step - 4) % 5 == 0 -+ -+ def begin_step(self, step: int) -> None: -+ if self.closed or step != self._step + 1: -+ raise RuntimeError("video-row cache step order changed") -+ if self._step >= 0 and self._blocks_this_step != self.num_layers: -+ raise RuntimeError("previous video-row cache step is incomplete") -+ self.partial = not self.is_full_step(step) -+ if self.partial and len(self._deltas) != self.num_layers: -+ raise RuntimeError("partial video step has no complete dense anchor") -+ self._step = step -+ self._blocks_this_step = 0 -+ if self.partial: -+ self.partial_steps += 1 -+ else: -+ self.full_steps += 1 -+ -+ def prepare_local_span(self, start: int, length: int) -> None: -+ if self.closed or start < 0 or length < 1 or start + length > self.seq_len: -+ raise ValueError("invalid local packed span") -+ if self._local_span is not None: -+ if self._local_span != (start, length): -+ raise RuntimeError("video-row cache local ownership changed") -+ return -+ device = self.global_dense_positions.device -+ target = set(self._target_positions) -+ self.local_video_positions = torch.tensor( -+ [i - start for i in self._target_positions if start <= i < start + length], dtype=torch.long, device=device -+ ) -+ self.local_dense_positions = torch.tensor( -+ [i - start for i in range(start, min(start + length, self.used_len)) if i not in target], -+ dtype=torch.long, -+ device=device, -+ ) -+ self._local_span = (start, length) -+ -+ def _visit(self, layer: int, x: torch.Tensor) -> None: -+ if self.closed or self._step < 0 or layer != self._blocks_this_step: -+ raise RuntimeError("video-row cache block order changed") -+ if self._local_span is None or x.shape[0] != self._local_span[1] or x.dtype != torch.bfloat16: -+ raise ValueError("video-row cache requires the prepared BF16 row shard") -+ self._blocks_this_step += 1 -+ -+ def capture(self, layer: int, x: torch.Tensor, output: torch.Tensor) -> None: -+ if self.partial: -+ raise RuntimeError("cannot replace the dense anchor during a partial step") -+ self._visit(layer, x) -+ rows = self.local_video_positions -+ assert rows is not None -+ self._deltas[layer] = (output.index_select(0, rows).float() - x.index_select(0, rows).float()).to(torch.float16) -+ self.cache_bytes = sum(t.numel() * t.element_size() for t in self._deltas.values()) -+ self.full_blocks += 1 -+ -+ def restore(self, layer: int, x: torch.Tensor) -> torch.Tensor: -+ if not self.partial: -+ raise RuntimeError("cannot reuse a video residual on a full step") -+ self._visit(layer, x) -+ rows = self.local_video_positions -+ assert rows is not None -+ # This copy owns the returned full shard, preserving caller input and -+ # unused padding while fresh non-video rows are filled by the block. -+ output = x.clone() -+ if rows.numel(): -+ output.index_copy_(0, rows, (x.index_select(0, rows).float() + self._deltas[layer].float()).to(x.dtype)) -+ self.partial_blocks += 1 -+ return output -+ -+ def record_subset_attention(self, rows: int) -> None: -+ if not self.partial or rows != self.global_dense_positions.numel(): -+ raise RuntimeError("unexpected subset attention query count") -+ self.subset_attention_calls += 1 -+ self.subset_query_rows += rows -+ self.omitted_query_rows += len(self._target_positions) -+ -+ def end_step(self) -> None: -+ if self._blocks_this_step != self.num_layers: -+ raise RuntimeError("video-row cache step did not visit every main block") -+ -+ def stats(self) -> dict[str, int | bool]: -+ return { -+ name: getattr(self, name) -+ for name in [ -+ "full_steps", -+ "partial_steps", -+ "full_blocks", -+ "partial_blocks", -+ "subset_attention_calls", -+ "subset_query_rows", -+ "omitted_query_rows", -+ "partial_mlp_rows", -+ "cache_bytes", -+ "closed", -+ ] -+ } -+ -+ def close(self) -> None: -+ self._deltas.clear() -+ self._target_positions.clear() -+ self.global_dense_positions = torch.empty(0, dtype=torch.long) -+ self.local_dense_positions = None -+ self.local_video_positions = None -+ self.closed = True diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c15-source-identity.json b/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c15-source-identity.json deleted file mode 100644 index 80157f1a1..000000000 --- a/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c15-source-identity.json +++ /dev/null @@ -1,17 +0,0 @@ -{ - "head": "81c1beb6e47ecfa7602e4309a7342a38108aabc8", - "tree": "590f9798d83a2e11cf001c4b7ac1043bae1c5476", - "base": "be319692b96413918973b5bd399cdda3f320c574", - "branch": "rsi/h3-297-video-row-cache", - "files": { - "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "cf9fb13c5f24c7595d62d13cae6e283cafb2178612b20028d35357f0cafaac83", - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ca87df1936fd2694fcc768079e9e8f926b6d45969ddc61ebf353f5e726d70b8b", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "1258655fcdca4692c2306100846761a2b0b149bbcaa5cde48a9abaa559913bc3", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "a8257f98c2e30c7bc7978bf645966d2eaa98cd49a42a8a923c062b5808dcdcdf", - "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "c5fab405ee82bc553e9a902fa27e6dfbc5213ec18311e17b6415396a69b50f43", - "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "8ec1b3ae4cd252f5d95ab23e0232b983900939e8cb15d1e64f4740f7204abf39", - "vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py": "e8e2e74fb8edb05e02d3e2328ea1f3ad5ddcfd7a10ed09c5cf3911b1d063e849" - }, - "published": false, - "patch_base": "be319692b96413918973b5bd399cdda3f320c574" -} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c16-candidate.patch b/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c16-candidate.patch deleted file mode 100644 index 36082d6ac..000000000 --- a/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c16-candidate.patch +++ /dev/null @@ -1,924 +0,0 @@ -diff --git a/tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py b/tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py -new file mode 100644 -index 0000000..94ff958 ---- /dev/null -+++ b/tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py -@@ -0,0 +1,406 @@ -+# SPDX-License-Identifier: Apache-2.0 -+# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project -+ -+from datetime import timedelta -+from types import SimpleNamespace -+ -+import pytest -+import torch -+import torch.distributed as dist -+import torch.nn as nn -+ -+from tests.helpers.mark import hardware_marks -+from vllm_omni.diffusion.models.minimax_h3 import minimax_h3_blocks as blocks -+from vllm_omni.diffusion.models.minimax_h3.attention.flash_h3 import _prefix_bnsd, _subset_prefix_bnsd -+from vllm_omni.diffusion.models.minimax_h3.video_row_cache import MiniMaxH3VideoRowCache -+from vllm_omni.platforms import current_omni_platform -+ -+pytestmark = [pytest.mark.core_model, pytest.mark.diffusion, *hardware_marks(res={"npu": "A3"}, num_cards=4)] -+ -+ -+def state_for_rows(device="cpu", start=0, length=4): -+ state = MiniMaxH3VideoRowCache(torch.tensor([1, 2], device=device), seq_len=4, used_len=3, num_layers=1) -+ state.prepare_local_span(start, length) -+ return state -+ -+ -+def anchor(state, x): -+ for step in range(5): -+ state.begin_step(step) -+ state.capture(0, x, x + 0.125) -+ state.end_step() -+ state.begin_step(5) -+ -+ -+def test_fixed_schedule_and_row_ownership_excludes_padding(): -+ assert [i for i in range(50) if MiniMaxH3VideoRowCache.is_full_step(i)] == [ -+ 0, -+ 1, -+ 2, -+ 3, -+ 4, -+ 9, -+ 14, -+ 19, -+ 24, -+ 29, -+ 34, -+ 39, -+ 44, -+ 46, -+ 47, -+ 48, -+ 49, -+ ] -+ state = state_for_rows() -+ assert state.global_dense_positions.tolist() == [0] -+ assert state.local_dense_positions.tolist() == [0] -+ assert state.local_video_positions.tolist() == [1, 2] -+ with pytest.raises(RuntimeError, match="ownership"): -+ state.prepare_local_span(0, 3) -+ -+ -+@pytest.mark.parametrize("positions", [[1, 1], [2, 1], [-1], [3]]) -+def test_invalid_target_positions_are_rejected(positions): -+ with pytest.raises(ValueError): -+ MiniMaxH3VideoRowCache(torch.tensor(positions), seq_len=4, used_len=3, num_layers=1) -+ -+ -+def test_delta_is_fp32_and_restore_owns_output(): -+ state = state_for_rows() -+ x = torch.tensor([[1, 2], [3, 4], [5, 6], [7, 8]], dtype=torch.bfloat16) -+ anchor(state, x) -+ assert state._deltas[0].dtype == torch.float32 -+ before = x.clone() -+ output = state.restore(0, x) -+ assert torch.equal(output[1:3], x[1:3] + 0.125) -+ assert torch.equal(output[[0, 3]], x[[0, 3]]) -+ output.zero_() -+ assert torch.equal(x, before) -+ state.end_step() -+ state.close() -+ assert state.closed and not state._deltas and state.global_dense_positions.numel() == 0 -+ -+ -+def large_delta_roundtrip(device="cpu"): -+ state = state_for_rows(device=device) -+ x = torch.full((4, 2), 100000, dtype=torch.bfloat16, device=device) -+ expected = -x -+ delta = expected.float() - x.float() -+ assert torch.isfinite(delta).all() -+ assert not torch.isfinite(delta.to(torch.float16)).all() -+ for step in range(5): -+ state.begin_step(step) -+ state.capture(0, x, expected) -+ state.end_step() -+ state.begin_step(5) -+ actual = state.restore(0, x) -+ assert torch.isfinite(actual).all() -+ assert torch.equal(actual[1:3].cpu().view(torch.uint8), expected[1:3].cpu().view(torch.uint8)) -+ assert torch.equal(actual[[0, 3]], x[[0, 3]]) -+ state.end_step() -+ state.close() -+ assert state.stats()["live_cache_bytes"] == 0 -+ -+ -+def test_large_finite_bf16_delta_roundtrip_exceeds_fp16_storage_range(): -+ large_delta_roundtrip() -+ -+ -+class CountMLP(nn.Module): -+ def __init__(self): -+ super().__init__() -+ self.rows = [] -+ -+ def forward(self, x): -+ self.rows.append(x.shape[0]) -+ return x + 0.25 -+ -+ -+def tiny_block(): -+ block = object.__new__(blocks.MiniMaxH3DiTBlock) -+ nn.Module.__init__(block) -+ block.layer_index = 0 -+ block.norm1 = SimpleNamespace(weight=torch.ones(2), variance_epsilon=1e-6) -+ block.norm2 = SimpleNamespace(weight=torch.ones(2), variance_epsilon=1e-6) -+ block.adaln_proj = lambda t: ( -+ torch.zeros(3, 2), -+ torch.zeros(3, 2), -+ torch.ones(3, 2), -+ torch.zeros(3, 2), -+ torch.zeros(3, 2), -+ torch.ones(3, 2), -+ ) -+ block.attn = lambda x, **kwargs: x * 0.5 -+ block.mlp = CountMLP() -+ return block -+ -+ -+@pytest.mark.parametrize("start,length", [(0, 4), (1, 2)]) -+def test_real_block_dense_path_and_partial_mlp_empty_rows(start, length): -+ state = state_for_rows(start=start, length=length) -+ block = tiny_block() -+ x = torch.arange(length * 2).reshape(length, 2).to(torch.bfloat16) -+ before = x.clone() -+ kwargs = dict( -+ t_emb=torch.zeros(1), -+ combined_indices=torch.zeros(length, dtype=torch.long), -+ rope_table=None, -+ cu_seqlens=torch.tensor([0, 3, 4], dtype=torch.int32), -+ max_seqlen=3, -+ packed_total=4, -+ ) -+ expected = block(x, **kwargs) -+ for step in range(5): -+ state.begin_step(step) -+ actual = block(x, video_row_cache=state, **kwargs) -+ assert torch.equal(actual.view(torch.uint8), expected.view(torch.uint8)) -+ state.end_step() -+ block.mlp.rows.clear() -+ state.begin_step(5) -+ actual = block(x, video_row_cache=state, **kwargs) -+ state.end_step() -+ rows = state.local_dense_positions -+ assert block.mlp.rows == ([rows.numel()] if rows.numel() else []) -+ assert torch.equal(actual.index_select(0, rows), expected.index_select(0, rows)) -+ assert torch.equal(x, before) -+ -+ -+@pytest.mark.parametrize("failure", ["success", "model", "cancel"]) -+def test_request_state_closed_and_removed_on_every_exit(monkeypatch, failure): -+ from vllm_omni.diffusion.models.minimax_h3 import denoise_loop as loop -+ -+ monkeypatch.setenv("H3_VIDEO_ROW_CACHE", "1") -+ monkeypatch.setattr(loop.current_omni_platform, "is_npu", lambda: False) -+ monkeypatch.setattr(loop, "minimax_h3_publish_denoise_progress", lambda *args: None) -+ state = MiniMaxH3VideoRowCache(torch.tensor([1]), seq_len=4, used_len=3, num_layers=1) -+ -+ class Model: -+ def create_video_row_cache(self, *args, **kwargs): -+ return state -+ -+ def __call__(self, **kwargs): -+ assert kwargs["video_row_cache"] is state -+ if failure == "model": -+ raise RuntimeError("model failed") -+ state.prepare_local_span(0, 4) -+ x = torch.ones(4, 2, dtype=torch.bfloat16) -+ if state.partial: -+ state.restore(0, x) -+ else: -+ state.capture(0, x, x + 0.125) -+ return torch.zeros(2, 96), torch.zeros(1, 32) -+ -+ class Branch: -+ task = "ref2va" -+ seq_len = 4 -+ used_len = 3 -+ img_pos = torch.tensor([0, 1]) -+ update_mask = torch.tensor([False, True]) -+ update_mask_dev = update_mask -+ audio_update_mask_dev = torch.tensor([True]) -+ locked_audio_rows = None -+ static_kwargs = {"video_token_layout": SimpleNamespace(video_spans=[SimpleNamespace(role="target", start=1)])} -+ -+ def forward_kwargs(self, **kwargs): -+ return dict(self.static_kwargs) -+ -+ monkeypatch.setattr( -+ loop, -+ "minimax_h3_prepare_denoise_rows", -+ lambda **kwargs: (kwargs["initial_video_rows"], kwargs["initial_audio_rows"], torch.zeros(1, 96), None), -+ ) -+ -+ def cancellation(**kwargs): -+ if failure == "cancel": -+ raise RuntimeError("request cancelled") -+ -+ monkeypatch.setattr(loop, "check_request_cancellation", cancellation) -+ branch, model = Branch(), Model() -+ kwargs = dict( -+ model=model, -+ positive=branch, -+ initial_video_rows=torch.zeros(2, 96), -+ initial_audio_rows=torch.zeros(1, 32), -+ keyframe_cond_rows=torch.zeros(1, 96), -+ sigmas_video=[1 - i / 50 for i in range(51)], -+ sigmas_audio=[1 - i / 50 for i in range(51)], -+ device=torch.device("cpu"), -+ ) -+ if failure == "success": -+ loop.minimax_h3_denoise_loop(**kwargs) -+ assert state.full_steps == 17 and state.partial_steps == 33 -+ else: -+ with pytest.raises(RuntimeError): -+ loop.minimax_h3_denoise_loop(**kwargs) -+ assert state.closed and not state._deltas and "video_row_cache" not in branch.static_kwargs -+ assert model._video_row_cache_stats["closed"] -+ -+ -+def test_default_factory_does_not_create_state(monkeypatch): -+ from vllm_omni.diffusion.models.minimax_h3.minimax_h3_transformer import MiniMaxH3DiTModel -+ -+ monkeypatch.delenv("H3_VIDEO_ROW_CACHE", raising=False) -+ assert ( -+ MiniMaxH3DiTModel.create_video_row_cache(None, torch.tensor([1]), seq_len=4, used_len=3, num_steps=50) is None -+ ) -+ -+ -+@pytest.mark.parametrize("ref2va", [False, True]) -+def test_real_packing_task_selects_ref2va_and_rejects_fl2va(monkeypatch, ref2va): -+ from vllm_omni.diffusion.models.minimax_h3 import denoise_loop as loop -+ from vllm_omni.diffusion.models.minimax_h3.packed_sequence import ( -+ minimax_h3_packed_sequence, -+ minimax_h3_packed_sequence_ref2va_blocks, -+ ) -+ -+ common = dict(text_len=2, latent_t=2, latent_h=4, latent_w=4, audio_t=2) -+ packed = ( -+ minimax_h3_packed_sequence_ref2va_blocks(**common, ref_blocks=[{"kind": "image", "latent_h": 4, "latent_w": 4}]) -+ if ref2va -+ else minimax_h3_packed_sequence(**common, include_keyframe_cond=True, keyframe_frame_indices=[0], frame_count=5) -+ ) -+ branch = loop.MiniMaxH3DenoiseBranch( -+ packed=packed, -+ text_embeddings=torch.zeros(2, 5120), -+ token_tags=packed["token_tags"], -+ device=torch.device("cpu"), -+ task="ref2va" if ref2va else "fl2va", -+ ) -+ layout = branch.static_kwargs["video_token_layout"] -+ assert not any(span.role == "reference" for span in layout.video_spans) -+ monkeypatch.setenv("H3_VIDEO_ROW_CACHE", "1") -+ monkeypatch.setattr(loop.current_omni_platform, "is_npu", lambda: False) -+ monkeypatch.setattr(loop, "minimax_h3_publish_denoise_progress", lambda *args: None) -+ monkeypatch.setattr(loop, "check_request_cancellation", lambda **kwargs: None) -+ states = [] -+ -+ class Model: -+ def create_video_row_cache(self, positions, **kwargs): -+ state = MiniMaxH3VideoRowCache( -+ positions, seq_len=kwargs["seq_len"], used_len=kwargs["used_len"], num_layers=1 -+ ) -+ state.prepare_local_span(0, branch.seq_len) -+ states.append(state) -+ return state -+ -+ def __call__(self, **kwargs): -+ state = kwargs["video_row_cache"] -+ x = torch.ones(branch.seq_len, 2, dtype=torch.bfloat16) -+ if state.partial: -+ state.restore(0, x) -+ else: -+ state.capture(0, x, x + 0.125) -+ return torch.zeros(branch.img_pos.numel(), 96), torch.zeros(branch.audio_pos.numel(), 32) -+ -+ kwargs = dict( -+ model=Model(), -+ positive=branch, -+ initial_video_rows=torch.zeros(branch.img_pos.numel(), 96), -+ initial_audio_rows=torch.zeros(branch.audio_pos.numel(), 32), -+ keyframe_cond_rows=torch.zeros(int((~branch.update_mask).sum()), 96), -+ sigmas_video=[1 - i / 50 for i in range(51)], -+ sigmas_audio=[1 - i / 50 for i in range(51)], -+ device=torch.device("cpu"), -+ ) -+ if ref2va: -+ loop.minimax_h3_denoise_loop(**kwargs) -+ assert len(states) == 1 and states[0].closed -+ assert states[0].full_steps == 17 and states[0].partial_steps == 33 -+ else: -+ with pytest.raises(ValueError, match="requires Ref2VA"): -+ loop.minimax_h3_denoise_loop(**kwargs) -+ assert not states -+ -+ -+@pytest.mark.parametrize("start,length", [(0, 4), (1, 2)]) -+def test_attention_keeps_collective_path_and_projects_only_dense_local_rows(start, length): -+ state = state_for_rows(start=start, length=length) -+ x = torch.ones(length, 2, dtype=torch.bfloat16) -+ anchor(state, x) -+ attention = object.__new__(blocks.MiniMaxH3Attention) -+ nn.Module.__init__(attention) -+ attention.num_heads = 1 -+ attention.head_dim = 2 -+ attention.vdn = None -+ attention.to_gate_compress = None -+ calls, projected_rows = [], [] -+ attention._project_qkv = lambda *args: (x.reshape(length, 1, 2),) * 5 -+ -+ def packed(q, k, v, **kwargs): -+ calls.append(q.shape[0]) -+ assert kwargs["video_row_cache"] is state -+ return q -+ -+ def project(rows): -+ projected_rows.append(rows.shape[0]) -+ return rows * 2, None -+ -+ attention._run_packed_attention = packed -+ attention.out_proj = project -+ output = attention( -+ x, -+ rope_table=None, -+ cu_seqlens=torch.tensor([0, 3, 4], dtype=torch.int32), -+ max_seqlen=3, -+ packed_total=4, -+ video_row_cache=state, -+ ) -+ rows = state.local_dense_positions -+ assert calls == [length] -+ assert projected_rows == ([rows.numel()] if rows.numel() else []) -+ assert torch.equal(output.index_select(0, rows), (x * 2).index_select(0, rows)) -+ assert torch.count_nonzero(output.index_select(0, state.local_video_positions)) == 0 -+ -+ -+def _native_worker(rank): -+ import torch_npu # noqa: F401 -+ -+ torch.npu.set_device(rank) -+ dist.init_process_group( -+ "hccl", init_method="tcp://127.0.0.1:29597", rank=rank, world_size=4, timeout=timedelta(seconds=120) -+ ) -+ try: -+ large_delta_roundtrip(device=f"npu:{rank}") -+ torch.manual_seed(19) -+ full = [torch.randn(32, 4, 128, dtype=torch.bfloat16).npu() for _ in range(3)] -+ exchanged = [] -+ for tensor in full: -+ local = tensor[rank * 8 : (rank + 1) * 8] -+ send = local.reshape(8, 4, 1, 128).permute(1, 0, 2, 3).contiguous() -+ receive = torch.empty_like(send) -+ dist.all_to_all_single(receive, send) -+ exchanged.append(receive.reshape(1, 32, 1, 128)) -+ state = MiniMaxH3VideoRowCache(torch.arange(8, 24, device="npu"), seq_len=32, used_len=30, num_layers=1) -+ state.prepare_local_span(rank * 8, 8) -+ x = torch.ones(8, 2, dtype=torch.bfloat16, device="npu") -+ anchor(state, x) -+ output = _subset_prefix_bnsd(exchanged[0], exchanged[1][:, :30], exchanged[2][:, :30], 128**-0.5, state) -+ send = output.reshape(4, 8, 1, 128).contiguous() -+ receive = torch.empty_like(send) -+ dist.all_to_all_single(receive, send) -+ local_output = receive.permute(1, 0, 2, 3).reshape(8, 4, 128) -+ expected = _prefix_bnsd(full[0].unsqueeze(0), full[1][:30].unsqueeze(0), full[2][:30].unsqueeze(0), 128**-0.5)[ -+ 0 -+ ] -+ rows = state.local_dense_positions -+ torch.testing.assert_close( -+ local_output.index_select(0, rows).float(), -+ expected[rank * 8 : (rank + 1) * 8].index_select(0, rows).float(), -+ atol=0.03125, -+ rtol=0.02, -+ ) -+ assert state.subset_attention_calls == 1 and state.subset_query_rows == 14 -+ assert rows.numel() == [8, 0, 0, 6][rank] -+ state.restore(0, x) -+ state.end_step() -+ state.close() -+ torch.npu.synchronize() -+ dist.barrier() -+ finally: -+ dist.destroy_process_group() -+ -+ -+@pytest.mark.skipif(not current_omni_platform.is_npu(), reason="Requires four native Ascend devices") -+def test_native_four_rank_subset_attention_with_empty_local_dense_rows(): -+ torch.multiprocessing.spawn(_native_worker, nprocs=4, join=True) -diff --git a/vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py b/vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py -index 6ebfb10..044c1f0 100644 ---- a/vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py -+++ b/vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py -@@ -36,6 +36,16 @@ def _prefix_bnsd(query: torch.Tensor, key: torch.Tensor, value: torch.Tensor, sc - return output.reshape(batch, heads, rows, dim).transpose(1, 2).contiguous() - - -+def _subset_prefix_bnsd(query, key, value, scale, state): -+ """Compute only fresh conditioning queries; preserve the shared SP shape.""" -+ rows = state.global_dense_positions -+ dense = _prefix_bnsd(query.index_select(1, rows), key, value, scale) -+ output = torch.zeros_like(query) -+ output.index_copy_(1, rows, dense) -+ state.record_subset_attention(rows.numel()) -+ return output -+ -+ - class MiniMaxH3FlashAttentionImpl(FlashAttentionImpl): - """Use independent head batches for explicit long BF16 prefix attention. - -@@ -51,7 +61,10 @@ class MiniMaxH3FlashAttentionImpl(FlashAttentionImpl): - value: torch.Tensor, - attn_metadata: AttentionMetadata | None = None, - ) -> torch.Tensor: -+ state = attn_metadata.extra.get("minimax_h3_video_row_cache") if attn_metadata is not None else None - if os.environ.get("MINDIE_SD_FA_TYPE") != "fused_attn_score" or attn_metadata is None: -+ if state is not None: -+ raise RuntimeError("H3 video-row cache requires the explicit fused attention path") - return super().forward_fa_npu(query, key, value, attn_metadata) - extra = attn_metadata.extra - if ( -@@ -68,9 +81,17 @@ class MiniMaxH3FlashAttentionImpl(FlashAttentionImpl): - or query.shape[2:] != value.shape[2:] - or key.shape[1] != value.shape[1] - ): -+ if state is not None: -+ raise RuntimeError("H3 video-row cache received unsupported fused attention inputs") - return super().forward_fa_npu(query, key, value, attn_metadata) - resolved = self._resolve_packed_seq_npu(query, key, extra) - if resolved is None: -+ if state is not None: -+ raise RuntimeError("H3 video-row cache requires a resolved packed prefix") - return super().forward_fa_npu(query, key, value, attn_metadata) - used_k = resolved[1][0] -+ if state is not None: -+ if not state.partial or used_k != state.used_len or query.shape[1] != state.seq_len: -+ raise RuntimeError("H3 video-row cache attention layout changed") -+ return _subset_prefix_bnsd(query, key[:, :used_k], value[:, :used_k], self.softmax_scale, state) - return _prefix_bnsd(query, key[:, :used_k], value[:, :used_k], self.softmax_scale) -diff --git a/vllm_omni/diffusion/models/minimax_h3/denoise_loop.py b/vllm_omni/diffusion/models/minimax_h3/denoise_loop.py -index 8cca219..07fb372 100644 ---- a/vllm_omni/diffusion/models/minimax_h3/denoise_loop.py -+++ b/vllm_omni/diffusion/models/minimax_h3/denoise_loop.py -@@ -9,6 +9,7 @@ rows stay pinned to their noised step-0 anchors. - - from __future__ import annotations - -+import os - from collections.abc import Callable - from contextlib import AbstractContextManager, nullcontext - from typing import Any -@@ -69,7 +70,9 @@ class MiniMaxH3DenoiseBranch: - text_embeddings: torch.Tensor, - token_tags: torch.Tensor, - device: torch.device, -+ task: str | None = None, - ) -> None: -+ self.task = task - seq_len = int(packed["seq_len"]) - self.seq_len = seq_len - self.img_pos = packed["img_pos"].view(-1).to(torch.long) -@@ -380,6 +383,24 @@ def minimax_h3_denoise_loop( - positive.static_kwargs["reference_kv_state"] = reference_kv_state - - try: -+ video_row_cache = None -+ create_video_cache = getattr(model, "create_video_row_cache", None) -+ if os.environ.get("H3_VIDEO_ROW_CACHE") == "1": -+ if ( -+ not callable(create_video_cache) -+ or video_edit is not None -+ or audio_edit is not None -+ or keyframe_cond_rows is None -+ or positive.task != "ref2va" -+ ): -+ raise ValueError("H3 video-row cache requires Ref2VA image references without editing") -+ video_row_cache = create_video_cache( -+ positive.img_pos[positive.update_mask].to(device), -+ seq_len=positive.seq_len, -+ used_len=positive.used_len, -+ num_steps=num_steps, -+ ) -+ positive.static_kwargs["video_row_cache"] = video_row_cache - for step in range(num_steps): - check_request_cancellation() - step_cm = step_profiler(step) if step_profiler is not None else nullcontext() -@@ -388,6 +409,8 @@ def minimax_h3_denoise_loop( - s_a = sigmas_audio[step] - if reference_kv_state is not None: - reference_kv_state.begin_step(step, sigma=float(s_v)) -+ if video_row_cache is not None: -+ video_row_cache.begin_step(step) - # Publish where we are so step-gated attention features (the dense - # warmup of RAINFUSION_ATTN, the timestep gate of TRTLLM_ATTN) can - # see it. Gates use the scheduler-style descending timestep, which -@@ -430,6 +453,8 @@ def minimax_h3_denoise_loop( - v_video, v_audio = model(**fk) - if reference_kv_state is not None: - reference_kv_state.end_step() -+ if video_row_cache is not None: -+ video_row_cache.end_step() - mv_video_t = v_video.float()[update] - mv_audio_t = v_audio.float()[audio_update] - -@@ -476,6 +501,10 @@ def minimax_h3_denoise_loop( - - finally: - minimax_h3_publish_denoise_progress(None, None, None) -+ if video_row_cache is not None: -+ positive.static_kwargs.pop("video_row_cache", None) -+ video_row_cache.close() -+ model._video_row_cache_stats = video_row_cache.stats() - if reference_kv_state is not None: - positive.static_kwargs.pop("reference_kv_state", None) - reference_kv_state.close() -diff --git a/vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py b/vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py -index 09f2e7d..94ef291 100644 ---- a/vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py -+++ b/vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py -@@ -45,6 +45,7 @@ if TYPE_CHECKING: - from vllm.model_executor.layers.quantization.base_config import QuantizationConfig - - from .reference_kv_cache import MiniMaxH3ReferenceKVTier2State -+ from .video_row_cache import MiniMaxH3VideoRowCache - - _BF16_DTYPE = torch.bfloat16 - _FP32_DTYPE = torch.float32 -@@ -291,6 +292,7 @@ class MiniMaxH3Attention(nn.Module): - reference_kv_layer_index: int | None = None, - reference_kv_compact: bool = False, - vdn_window: VDNLayout | None = None, -+ video_row_cache: MiniMaxH3VideoRowCache | None = None, - ) -> torch.Tensor: - """Run packed attention as a small eager island. - -@@ -368,6 +370,8 @@ class MiniMaxH3Attention(nn.Module): - "vsa_h3_ref_policy": self.vsa_ref_policy, - "vsa_h3_ref_keep_rate": self.vsa_ref_keep_rate, - } -+ if video_row_cache is not None and video_row_cache.partial: -+ extra["minimax_h3_video_row_cache"] = video_row_cache - if gate_compress is not None: - extra["gate_compress"] = gate_compress.unsqueeze(0) - if video_layout is not None and video_layout.video_spans: -@@ -455,6 +459,7 @@ class MiniMaxH3Attention(nn.Module): - reference_kv_layer_index: int | None = None, - reference_kv_compact: bool = False, - vdn_window: VDNLayout | None = None, -+ video_row_cache: MiniMaxH3VideoRowCache | None = None, - ) -> torch.Tensor: - """x: [T, hidden] packed thd rows -> [T, hidden]. - -@@ -508,10 +513,19 @@ class MiniMaxH3Attention(nn.Module): - reference_kv_layer_index=reference_kv_layer_index, - reference_kv_compact=reference_kv_compact, - vdn_window=vdn_window, -+ **({"video_row_cache": video_row_cache} if video_row_cache is not None else {}), - ) - if self.vdn is not None: - out = self.vdn.gate_softmax(out, x) - out = out.reshape(total, self.num_heads * self.head_dim) -+ if video_row_cache is not None and video_row_cache.partial: -+ rows = video_row_cache.local_dense_positions -+ assert rows is not None -+ projected = x.new_zeros((total, x.shape[-1])) -+ if rows.numel(): -+ dense, _ = self.out_proj(out.index_select(0, rows)) -+ projected.index_copy_(0, rows, dense) -+ return projected - out, _ = self.out_proj(out) - if self.vdn is not None: - out = self.vdn.add_linear(out, x, q_raw, k_raw, v, vdn_window) -@@ -742,6 +756,7 @@ class MiniMaxH3DiTBlock(nn.Module): - reference_kv_state: MiniMaxH3ReferenceKVTier2State | None = None, - reference_kv_compact: bool = False, - vdn_window: VDNLayout | None = None, -+ video_row_cache: MiniMaxH3VideoRowCache | None = None, - ) -> torch.Tensor: - """x: [T, H]; t_emb: [M, t_dim]; combined_indices: [T] - (= inverse_indices * modality_num + token_tags.clamp(min=0)). -@@ -759,6 +774,7 @@ class MiniMaxH3DiTBlock(nn.Module): - gate_mlp, - ) = self.adaln_proj(t_emb) - -+ block_input = x - residual = x - h = rms_norm_indexed_scale_shift( - x, -@@ -782,6 +798,7 @@ class MiniMaxH3DiTBlock(nn.Module): - reference_kv_layer_index=self.layer_index, - reference_kv_compact=reference_kv_compact, - vdn_window=vdn_window, -+ **({"video_row_cache": video_row_cache} if video_row_cache is not None else {}), - ) - x, h = indexed_gate_rms_norm_scale_shift( - residual, -@@ -794,5 +811,22 @@ class MiniMaxH3DiTBlock(nn.Module): - self.norm2.variance_epsilon, - ) - residual = x -+ if video_row_cache is not None and video_row_cache.partial: -+ rows = video_row_cache.local_dense_positions -+ assert rows is not None -+ output = video_row_cache.restore(self.layer_index, block_input) -+ if rows.numel(): -+ dense = indexed_gate( -+ residual.index_select(0, rows), -+ gate_mlp, -+ self.mlp(h.index_select(0, rows)), -+ combined_indices.index_select(0, rows), -+ ) -+ output.index_copy_(0, rows, dense) -+ video_row_cache.partial_mlp_rows += rows.numel() -+ return output - h = self.mlp(h) -- return indexed_gate(residual, gate_mlp, h, combined_indices) -+ output = indexed_gate(residual, gate_mlp, h, combined_indices) -+ if video_row_cache is not None: -+ video_row_cache.capture(self.layer_index, block_input, output) -+ return output -diff --git a/vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py b/vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py -index 9518828..2cf7aae 100644 ---- a/vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py -+++ b/vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py -@@ -10,6 +10,7 @@ layout. - from __future__ import annotations - - import math -+import os - from bisect import bisect_left - from collections.abc import Iterable, Mapping - from dataclasses import replace -@@ -66,6 +67,8 @@ if TYPE_CHECKING: - from vllm_omni.diffusion.data import OmniDiffusionConfig - from vllm_omni.diffusion.models.minimax_h3.reference_kv_cache import MiniMaxH3ReferenceKVTier2State - -+ from .video_row_cache import MiniMaxH3VideoRowCache -+ - logger = init_logger(__name__) - - -@@ -245,6 +248,7 @@ _FORWARD_SUPPORTED_KWARGS = frozenset( - "control_rows", - "control_context_scale", - "reference_kv_state", -+ "video_row_cache", - } - ) - -@@ -1232,6 +1236,35 @@ class MiniMaxH3DiTModel(nn.Module): - t_emb = self.time_embedder(unique_timesteps) - return embeddings, t_emb - -+ def create_video_row_cache( -+ self, target_positions: torch.Tensor, *, seq_len: int, used_len: int, num_steps: int -+ ) -> MiniMaxH3VideoRowCache | None: -+ if os.environ.get("H3_VIDEO_ROW_CACHE") != "1": -+ return None -+ p = self.parallel_config -+ if ( -+ not current_omni_platform.is_npu() -+ or num_steps != 50 -+ or p.tensor_parallel_size != 1 -+ or p.data_parallel_size not in (None, 1) -+ or p.cfg_parallel_size != 1 -+ or p.pipeline_parallel_size != 1 -+ or p.ulysses_degree != 4 -+ or p.ring_degree != 1 -+ or p.allgather_degree != 1 -+ or p.ulysses_mode != "strict" -+ or self.od_config.cache_backend not in (None, "none") -+ or self.od_config.kv_offload_config is not None -+ or self.vdn_config is not None -+ or os.environ.get("MINDIE_SD_FA_TYPE") != "fused_attn_score" -+ or any(block.attn.attention.attn_backend.get_name() != "FLASH_ATTN" for block in self.blocks) -+ ): -+ raise ValueError("H3 video-row cache requires dense NPU BF16 Ref2VA with strict Ulysses4, TP/DP/CFG/PP1") -+ from .video_row_cache import MiniMaxH3VideoRowCache -+ -+ self._video_row_cache_stats = None -+ return MiniMaxH3VideoRowCache(target_positions, seq_len=seq_len, used_len=used_len, num_layers=len(self.blocks)) -+ - def create_reference_kv_state( - self, - *, -@@ -1308,6 +1341,14 @@ class MiniMaxH3DiTModel(nn.Module): - refiner_max = int(self._psp_field(refiner_psp, "refiner_packed_seq_params", "max_seqlen_q")) - video_layout = kwargs.get("video_token_layout") - reference_kv_state = kwargs.get("reference_kv_state") -+ video_row_cache = kwargs.get("video_row_cache") -+ if video_row_cache is not None and ( -+ num_requests != 1 -+ or reference_kv_state is not None -+ or self.vdn_config is not None -+ or kwargs.get("control_rows") is not None -+ ): -+ raise ValueError("H3 video-row cache cannot combine with packed requests, reference KV, VDN or control") - vdn_window = None - if self.vdn_config is not None: - # Text rows lead the packed document. -@@ -1442,6 +1483,11 @@ class MiniMaxH3DiTModel(nn.Module): - if reference_kv_state is not None: - reference_kv_state.set_local_sequence_rows(hidden.shape[0]) - -+ if video_row_cache is not None: -+ if hidden.shape[0] != local_len: -+ raise RuntimeError("H3 video-row cache requires the strict local sequence shard") -+ video_row_cache.prepare_local_span(local_start, local_len) -+ - block_video_layout = video_layout - if compact_reference: - block_video_layout = None -@@ -1505,6 +1551,7 @@ class MiniMaxH3DiTModel(nn.Module): - reference_kv_state=reference_kv_state, - reference_kv_compact=compact_reference, - vdn_window=vdn_window, -+ **({"video_row_cache": video_row_cache} if video_row_cache is not None else {}), - ) - if block_index in hints: - hidden = hidden + hints.pop(block_index) * control_scale -diff --git a/vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py b/vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py -index 476c3cc..ebce8fc 100644 ---- a/vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py -+++ b/vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py -@@ -2115,6 +2115,7 @@ class MiniMaxH3Pipeline( - text_embeddings=text_embeddings, - token_tags=tags, - device=self.device, -+ task=task, - ) - - if control_rows is not None: -diff --git a/vllm_omni/diffusion/models/minimax_h3/video_row_cache.py b/vllm_omni/diffusion/models/minimax_h3/video_row_cache.py -new file mode 100644 -index 0000000..5ccfa35 ---- /dev/null -+++ b/vllm_omni/diffusion/models/minimax_h3/video_row_cache.py -@@ -0,0 +1,163 @@ -+# SPDX-License-Identifier: Apache-2.0 -+# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project -+"""Request-owned, experimental target-video block residuals for H3 Ref2VA.""" -+ -+from __future__ import annotations -+ -+import torch -+ -+ -+class MiniMaxH3VideoRowCache: -+ """Keep non-video computation dense while reusing video block updates. -+ -+ The first experimental profile is deliberately fixed. No state is shared -+ across requests, and every rank retains all attention collectives. -+ """ -+ -+ def __init__(self, target_positions: torch.Tensor, *, seq_len: int, used_len: int, num_layers: int) -> None: -+ positions = target_positions.detach().cpu().tolist() -+ if not 0 < used_len <= seq_len or not positions or num_layers < 1: -+ raise ValueError("video-row cache requires a nonempty valid packed document") -+ if positions != sorted(set(positions)) or positions[0] < 0 or positions[-1] >= used_len: -+ raise ValueError("target-video positions must be sorted unique valid rows") -+ target_set = set(positions) -+ dense = [i for i in range(used_len) if i not in target_set] -+ if not dense: -+ raise ValueError("video-row cache requires non-video conditioning rows") -+ self.seq_len = seq_len -+ self.used_len = used_len -+ self.num_layers = num_layers -+ self._target_positions = positions -+ self.global_dense_positions = torch.tensor(dense, dtype=torch.long, device=target_positions.device) -+ self.local_dense_positions: torch.Tensor | None = None -+ self.local_video_positions: torch.Tensor | None = None -+ self._local_span: tuple[int, int] | None = None -+ self._deltas: dict[int, torch.Tensor] = {} -+ self._step = -1 -+ self._blocks_this_step = 0 -+ self.partial = False -+ self.closed = False -+ self.full_steps = 0 -+ self.partial_steps = 0 -+ self.full_blocks = 0 -+ self.partial_blocks = 0 -+ self.subset_attention_calls = 0 -+ self.subset_query_rows = 0 -+ self.omitted_query_rows = 0 -+ self.partial_mlp_rows = 0 -+ self.peak_cache_bytes = 0 -+ -+ @staticmethod -+ def is_full_step(step: int) -> bool: -+ if not 0 <= step < 50: -+ raise ValueError("the video-row cache profile requires 50 steps") -+ return step < 4 or step >= 46 or (step - 4) % 5 == 0 -+ -+ def begin_step(self, step: int) -> None: -+ if self.closed or step != self._step + 1: -+ raise RuntimeError("video-row cache step order changed") -+ if self._step >= 0 and self._blocks_this_step != self.num_layers: -+ raise RuntimeError("previous video-row cache step is incomplete") -+ self.partial = not self.is_full_step(step) -+ if self.partial and len(self._deltas) != self.num_layers: -+ raise RuntimeError("partial video step has no complete dense anchor") -+ self._step = step -+ self._blocks_this_step = 0 -+ if self.partial: -+ self.partial_steps += 1 -+ else: -+ self.full_steps += 1 -+ -+ def prepare_local_span(self, start: int, length: int) -> None: -+ if self.closed or start < 0 or length < 1 or start + length > self.seq_len: -+ raise ValueError("invalid local packed span") -+ if self._local_span is not None: -+ if self._local_span != (start, length): -+ raise RuntimeError("video-row cache local ownership changed") -+ return -+ device = self.global_dense_positions.device -+ target = set(self._target_positions) -+ self.local_video_positions = torch.tensor( -+ [i - start for i in self._target_positions if start <= i < start + length], dtype=torch.long, device=device -+ ) -+ self.local_dense_positions = torch.tensor( -+ [i - start for i in range(start, min(start + length, self.used_len)) if i not in target], -+ dtype=torch.long, -+ device=device, -+ ) -+ self._local_span = (start, length) -+ -+ def _visit(self, layer: int, x: torch.Tensor) -> None: -+ if self.closed or self._step < 0 or layer != self._blocks_this_step: -+ raise RuntimeError("video-row cache block order changed") -+ if self._local_span is None or x.shape[0] != self._local_span[1] or x.dtype != torch.bfloat16: -+ raise ValueError("video-row cache requires the prepared BF16 row shard") -+ self._blocks_this_step += 1 -+ -+ def capture(self, layer: int, x: torch.Tensor, output: torch.Tensor) -> None: -+ if self.partial: -+ raise RuntimeError("cannot replace the dense anchor during a partial step") -+ self._visit(layer, x) -+ rows = self.local_video_positions -+ assert rows is not None -+ # BF16 endpoints may have a finite difference outside FP16's range. -+ # Keep that difference in FP32; only the reconstructed block output is -+ # rounded to the original BF16 activation dtype. -+ self._deltas[layer] = output.index_select(0, rows).float() - x.index_select(0, rows).float() -+ self.peak_cache_bytes = max( -+ self.peak_cache_bytes, sum(t.numel() * t.element_size() for t in self._deltas.values()) -+ ) -+ self.full_blocks += 1 -+ -+ def restore(self, layer: int, x: torch.Tensor) -> torch.Tensor: -+ if not self.partial: -+ raise RuntimeError("cannot reuse a video residual on a full step") -+ self._visit(layer, x) -+ rows = self.local_video_positions -+ assert rows is not None -+ # This copy owns the returned full shard, preserving caller input and -+ # unused padding while fresh non-video rows are filled by the block. -+ output = x.clone() -+ if rows.numel(): -+ output.index_copy_(0, rows, (x.index_select(0, rows).float() + self._deltas[layer].float()).to(x.dtype)) -+ self.partial_blocks += 1 -+ return output -+ -+ def record_subset_attention(self, rows: int) -> None: -+ if not self.partial or rows != self.global_dense_positions.numel(): -+ raise RuntimeError("unexpected subset attention query count") -+ self.subset_attention_calls += 1 -+ self.subset_query_rows += rows -+ self.omitted_query_rows += len(self._target_positions) -+ -+ def end_step(self) -> None: -+ if self._blocks_this_step != self.num_layers: -+ raise RuntimeError("video-row cache step did not visit every main block") -+ -+ def stats(self) -> dict[str, int | bool | str]: -+ counters: dict[str, int | bool | str] = { -+ name: getattr(self, name) -+ for name in [ -+ "full_steps", -+ "partial_steps", -+ "full_blocks", -+ "partial_blocks", -+ "subset_attention_calls", -+ "subset_query_rows", -+ "omitted_query_rows", -+ "partial_mlp_rows", -+ "peak_cache_bytes", -+ "closed", -+ ] -+ } -+ counters["storage_dtype"] = "torch.float32" -+ counters["live_cache_bytes"] = sum(t.numel() * t.element_size() for t in self._deltas.values()) -+ return counters -+ -+ def close(self) -> None: -+ self._deltas.clear() -+ self._target_positions.clear() -+ self.global_dense_positions = torch.empty(0, dtype=torch.long) -+ self.local_dense_positions = None -+ self.local_video_positions = None -+ self.closed = True diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c16-source-identity.json b/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c16-source-identity.json deleted file mode 100644 index e94e5aa8f..000000000 --- a/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c16-source-identity.json +++ /dev/null @@ -1,17 +0,0 @@ -{ - "head": "e0a3462fcc44de698b5dedcf8ff2c99f9802f757", - "tree": "830fab783d4750193c321611272cdd087634fb39", - "base": "81c1beb6e47ecfa7602e4309a7342a38108aabc8", - "branch": "rsi/h3-297-video-row-fp32", - "files": { - "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "c555d76f7bd6457a81238fca56100d11cb7ab043a4a649d55e23b6203d09bde2", - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ca87df1936fd2694fcc768079e9e8f926b6d45969ddc61ebf353f5e726d70b8b", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "1258655fcdca4692c2306100846761a2b0b149bbcaa5cde48a9abaa559913bc3", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "a8257f98c2e30c7bc7978bf645966d2eaa98cd49a42a8a923c062b5808dcdcdf", - "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "c5fab405ee82bc553e9a902fa27e6dfbc5213ec18311e17b6415396a69b50f43", - "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "e09b872f6900e821d42632a0ca09595f2affe6f9794049dbc74d69ba6a52b2b0", - "vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py": "e8e2e74fb8edb05e02d3e2328ea1f3ad5ddcfd7a10ed09c5cf3911b1d063e849" - }, - "published": false, - "patch_base": "be319692b96413918973b5bd399cdda3f320c574" -} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c17-candidate.patch b/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c17-candidate.patch deleted file mode 100644 index 4a7c562dd..000000000 --- a/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c17-candidate.patch +++ /dev/null @@ -1,282 +0,0 @@ -diff --git a/tests/diffusion/models/minimax_h3/test_minimax_h3_video_window.py b/tests/diffusion/models/minimax_h3/test_minimax_h3_video_window.py -new file mode 100644 -index 0000000..b29edf8 ---- /dev/null -+++ b/tests/diffusion/models/minimax_h3/test_minimax_h3_video_window.py -@@ -0,0 +1,164 @@ -+# SPDX-License-Identifier: Apache-2.0 -+# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project -+ -+from types import SimpleNamespace -+ -+import pytest -+import torch -+ -+from tests.helpers.mark import hardware_marks -+from vllm_omni.diffusion.models.minimax_h3.attention import flash_h3 as flash -+from vllm_omni.platforms import current_omni_platform -+ -+pytestmark = [pytest.mark.core_model, pytest.mark.diffusion, *hardware_marks(res={"npu": "A3"}, num_cards=1)] -+ -+ -+def reference(query, key, value, scale): -+ q, k, v = [tensor.transpose(1, 2).float() for tensor in (query, key, value)] -+ return (torch.softmax(q @ k.transpose(-2, -1) * scale, dim=-1) @ v).transpose(1, 2).to(query.dtype) -+ -+ -+def fixture(frames=5, prefix=4, rows=3, padding=2, heads=2, dim=8, device="cpu"): -+ torch.manual_seed(19) -+ used = prefix + frames * rows -+ q = torch.randn(1, used + padding, heads, dim, dtype=torch.bfloat16).to(device) -+ k, v = [torch.randn(1, used, heads, dim, dtype=torch.bfloat16).to(device) for _ in range(2)] -+ return q, k, v -+ -+ -+@pytest.mark.parametrize("frames,prefix", [(3, 0), (3, 4), (5, 4), (7, 2)]) -+def test_window_matches_explicit_per_frame_key_sets(monkeypatch, frames, prefix): -+ q, k, v = fixture(frames=frames, prefix=prefix) -+ before = [tensor.clone() for tensor in (q, k, v)] -+ calls = [] -+ -+ def primitive(query, key, value, scale): -+ calls.append((query.shape, key.shape)) -+ return reference(query, key, value, scale) -+ -+ monkeypatch.setattr(flash, "_prefix_bnsd", primitive) -+ actual = flash._video_temporal_window_bnsd(q, k, v, 0.25, target_start=prefix, frames=frames, rows_per_frame=3) -+ if prefix: -+ torch.testing.assert_close(actual[:, :prefix], reference(q[:, :prefix], k, v, 0.25), rtol=0, atol=0) -+ for t in range(frames): -+ start = min(max(t - 1, 0), frames - 3) -+ keys = torch.cat((k[:, :prefix], k[:, prefix + start * 3 : prefix + (start + 3) * 3]), dim=1) -+ values = torch.cat((v[:, :prefix], v[:, prefix + start * 3 : prefix + (start + 3) * 3]), dim=1) -+ expected = reference(q[:, prefix + t * 3 : prefix + (t + 1) * 3], keys, values, 0.25) -+ torch.testing.assert_close(actual[:, prefix + t * 3 : prefix + (t + 1) * 3], expected, rtol=0, atol=0) -+ assert torch.count_nonzero(actual[:, k.shape[1] :]) == 0 -+ assert all(torch.equal(a, b) for a, b in zip(before, (q, k, v))) -+ assert len(calls) == (2 if prefix else 1) -+ assert calls[-1][0] == (frames, 3, 2, 8) -+ assert calls[-1][1] == (frames, prefix + 9, 2, 8) -+ -+ -+@pytest.mark.parametrize("kind", ["short", "tail", "batch", "query"]) -+def test_requested_unsupported_geometry_raises(kind): -+ q, k, v = fixture() -+ kwargs = dict(target_start=4, frames=5, rows_per_frame=3) -+ if kind == "short": -+ kwargs["frames"] = 2 -+ elif kind == "tail": -+ kwargs["target_start"] = 3 -+ elif kind == "batch": -+ q, k, v = [tensor.expand(2, -1, -1, -1) for tensor in (q, k, v)] -+ else: -+ q = q[:, :2] -+ with pytest.raises(ValueError, match="temporal window"): -+ flash._video_temporal_window_bnsd(q, k, v, 0.25, **kwargs) -+ -+ -+def test_fixed_step_profile_and_invalid_progress(monkeypatch): -+ monkeypatch.setenv("H3_VIDEO_TEMPORAL_WINDOW", "3") -+ for step in range(50): -+ monkeypatch.setattr( -+ flash, "get_forward_context", lambda: SimpleNamespace(denoise_step_idx=step, total_denoise_steps=50) -+ ) -+ assert flash._use_video_temporal_window() == (step not in (0, 49)) -+ monkeypatch.setattr( -+ flash, "get_forward_context", lambda: SimpleNamespace(denoise_step_idx=None, total_denoise_steps=50) -+ ) -+ with pytest.raises(ValueError, match="published"): -+ flash._use_video_temporal_window() -+ monkeypatch.setenv("H3_VIDEO_TEMPORAL_WINDOW", "5") -+ with pytest.raises(ValueError, match="window=3"): -+ flash._use_video_temporal_window() -+ -+ -+def test_default_long_prefix_path_does_not_read_window_progress(monkeypatch): -+ from vllm_omni.diffusion.attention.backends.abstract import AttentionMetadata -+ -+ monkeypatch.setenv("MINDIE_SD_FA_TYPE", "fused_attn_score") -+ monkeypatch.delenv("H3_VIDEO_TEMPORAL_WINDOW", raising=False) -+ impl = object.__new__(flash.MiniMaxH3FlashAttentionImpl) -+ impl.causal, impl.softmax_scale = False, 0.25 -+ impl._resolve_packed_seq_npu = lambda *args: ([4096], [4093]) -+ monkeypatch.setattr(flash, "get_forward_context", lambda: pytest.fail("default path read window progress")) -+ calls = [] -+ monkeypatch.setattr(flash, "_prefix_bnsd", lambda q, k, v, scale: calls.append(k.shape[1]) or q) -+ q = torch.ones(1, 4096, 1, 8, dtype=torch.bfloat16) -+ metadata = AttentionMetadata(extra={"npu_attn_varlen": True}) -+ assert impl.forward_fa_npu(q, q, q, metadata) is q -+ assert calls == [4093] -+ -+ -+@pytest.mark.parametrize("step", [0, 1, 49]) -+def test_configured_long_path_routes_published_geometry(monkeypatch, step): -+ from vllm_omni.diffusion.attention.backends.abstract import AttentionMetadata, VideoTokenLayout, VideoTokenSpan -+ -+ monkeypatch.setenv("MINDIE_SD_FA_TYPE", "fused_attn_score") -+ monkeypatch.setenv("H3_VIDEO_TEMPORAL_WINDOW", "3") -+ monkeypatch.setattr( -+ flash, "get_forward_context", lambda: SimpleNamespace(denoise_step_idx=step, total_denoise_steps=50) -+ ) -+ impl = object.__new__(flash.MiniMaxH3FlashAttentionImpl) -+ impl.causal, impl.softmax_scale = False, 0.25 -+ impl._resolve_packed_seq_npu = lambda *args: ([4160], [4160]) -+ q = torch.ones(1, 4192, 1, 8, dtype=torch.bfloat16) -+ layout = VideoTokenLayout( -+ used_len=4160, video_spans=(VideoTokenSpan(start=64, latent_grid=(4, 32, 32), role="target"),) -+ ) -+ metadata = AttentionMetadata(extra={"npu_attn_varlen": True}, video_layout=layout) -+ calls = [] -+ monkeypatch.setattr(flash, "_prefix_bnsd", lambda *args: calls.append("dense") or q) -+ -+ def window(query, key, value, scale, **kwargs): -+ assert key.shape[1] == value.shape[1] == 4160 -+ assert kwargs == {"target_start": 64, "frames": 4, "rows_per_frame": 1024} -+ calls.append("window") -+ return q -+ -+ monkeypatch.setattr(flash, "_video_temporal_window_bnsd", window) -+ assert impl.forward_fa_npu(q, q, q, metadata) is q -+ assert calls == (["dense"] if step in (0, 49) else ["window"]) -+ -+ -+@pytest.mark.skipif(not current_omni_platform.is_npu(), reason="Requires native Ascend FAS") -+def test_native_bf16_window_fas_executes_real_batched_geometry(): -+ import torch_npu # noqa: F401 -+ -+ q, k, v = fixture(rows=8, heads=14, dim=128, device="npu") -+ output = flash._video_temporal_window_bnsd(q, k, v, 128**-0.5, target_start=4, frames=5, rows_per_frame=8) -+ torch.npu.synchronize() -+ assert output.dtype == torch.bfloat16 and torch.isfinite(output).all() -+ for t in range(5): -+ start = min(max(t - 1, 0), 2) -+ keys = torch.cat((k[:, :4], k[:, 4 + start * 8 : 4 + (start + 3) * 8]), dim=1) -+ values = torch.cat((v[:, :4], v[:, 4 + start * 8 : 4 + (start + 3) * 8]), dim=1) -+ expected = reference(q[:, 4 + t * 8 : 4 + (t + 1) * 8].cpu(), keys.cpu(), values.cpu(), 128**-0.5) -+ torch.testing.assert_close( -+ output[:, 4 + t * 8 : 4 + (t + 1) * 8].cpu().float(), expected.float(), atol=0.03125, rtol=0.02 -+ ) -+ -+ -+def test_requested_main_geometry_does_not_silently_fallback(monkeypatch): -+ from vllm_omni.diffusion.attention.backends.abstract import AttentionMetadata -+ -+ monkeypatch.setenv("MINDIE_SD_FA_TYPE", "fused_attn_score") -+ monkeypatch.setenv("H3_VIDEO_TEMPORAL_WINDOW", "3") -+ impl = object.__new__(flash.MiniMaxH3FlashAttentionImpl) -+ impl.causal = True -+ q = torch.ones(1, 4096, 1, 8, dtype=torch.bfloat16) -+ with pytest.raises(ValueError, match="unsupported"): -+ impl.forward_fa_npu(q, q, q, AttentionMetadata(extra={"npu_attn_varlen": True})) -diff --git a/vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py b/vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py -index 6ebfb10..0f0024b 100644 ---- a/vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py -+++ b/vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py -@@ -10,6 +10,7 @@ import torch - - from vllm_omni.diffusion.attention.backends.abstract import AttentionMetadata - from vllm_omni.diffusion.attention.backends.flash_attn import FlashAttentionImpl -+from vllm_omni.diffusion.forward_context import get_forward_context - - - def _prefix_bnsd(query: torch.Tensor, key: torch.Tensor, value: torch.Tensor, scale: float) -> torch.Tensor: -@@ -36,6 +37,58 @@ def _prefix_bnsd(query: torch.Tensor, key: torch.Tensor, value: torch.Tensor, sc - return output.reshape(batch, heads, rows, dim).transpose(1, 2).contiguous() - - -+def _video_temporal_window_bnsd( -+ query: torch.Tensor, -+ key: torch.Tensor, -+ value: torch.Tensor, -+ scale: float, -+ *, -+ target_start: int, -+ frames: int, -+ rows_per_frame: int, -+) -> torch.Tensor: -+ """Fresh global-prefix queries and local video queries, in original row order.""" -+ if ( -+ query.shape[0] != 1 -+ or key.shape[0] != 1 -+ or value.shape != key.shape -+ or query.shape[2:] != key.shape[2:] -+ or target_start < 0 -+ or frames < 3 -+ or rows_per_frame < 1 -+ or target_start + frames * rows_per_frame != key.shape[1] -+ or query.shape[1] < key.shape[1] -+ ): -+ raise ValueError("H3 temporal window requires one final contiguous target-video span and a valid KV prefix") -+ heads, dim = query.shape[2:] -+ starts = (torch.arange(frames, device=key.device) - 1).clamp(0, frames - 3) -+ -+ def pack_windows(tensor: torch.Tensor) -> torch.Tensor: -+ video = tensor[:, target_start:].reshape(frames, rows_per_frame, heads, dim) -+ windows = video.unfold(0, 3, 1).permute(0, 4, 1, 2, 3).reshape(frames - 2, 3 * rows_per_frame, heads, dim) -+ selected = windows.index_select(0, starts) -+ prefix = tensor[:, :target_start].expand(frames, -1, -1, -1) -+ return torch.cat((prefix, selected), dim=1) -+ -+ output = torch.zeros_like(query) -+ if target_start: -+ output[:, :target_start] = _prefix_bnsd(query[:, :target_start], key, value, scale) -+ video_query = query[:, target_start : key.shape[1]].reshape(frames, rows_per_frame, heads, dim) -+ video_output = _prefix_bnsd(video_query, pack_windows(key), pack_windows(value), scale) -+ output[:, target_start : key.shape[1]] = video_output.reshape(1, frames * rows_per_frame, heads, dim) -+ return output -+ -+ -+def _use_video_temporal_window() -> bool: -+ if os.environ.get("H3_VIDEO_TEMPORAL_WINDOW") != "3": -+ raise ValueError("The experimental H3 temporal-window profile requires window=3") -+ context = get_forward_context() -+ step, total = context.denoise_step_idx, context.total_denoise_steps -+ if total != 50 or step is None or not 0 <= step < 50: -+ raise ValueError("The experimental H3 temporal-window profile requires published 50-step progress") -+ return step not in (0, 49) -+ -+ - class MiniMaxH3FlashAttentionImpl(FlashAttentionImpl): - """Use independent head batches for explicit long BF16 prefix attention. - -@@ -51,7 +104,10 @@ class MiniMaxH3FlashAttentionImpl(FlashAttentionImpl): - value: torch.Tensor, - attn_metadata: AttentionMetadata | None = None, - ) -> torch.Tensor: -+ window_requested = os.environ.get("H3_VIDEO_TEMPORAL_WINDOW") is not None and query.shape[1] >= 4096 - if os.environ.get("MINDIE_SD_FA_TYPE") != "fused_attn_score" or attn_metadata is None: -+ if window_requested: -+ raise ValueError("H3 temporal window requires explicit fused attention and packed metadata") - return super().forward_fa_npu(query, key, value, attn_metadata) - extra = attn_metadata.extra - if ( -@@ -68,9 +124,29 @@ class MiniMaxH3FlashAttentionImpl(FlashAttentionImpl): - or query.shape[2:] != value.shape[2:] - or key.shape[1] != value.shape[1] - ): -+ if window_requested: -+ raise ValueError("H3 temporal window received unsupported packed fused-attention inputs") - return super().forward_fa_npu(query, key, value, attn_metadata) - resolved = self._resolve_packed_seq_npu(query, key, extra) - if resolved is None: -+ if window_requested: -+ raise ValueError("H3 temporal window requires a resolved packed prefix") - return super().forward_fa_npu(query, key, value, attn_metadata) - used_k = resolved[1][0] -+ if os.environ.get("H3_VIDEO_TEMPORAL_WINDOW") is not None and _use_video_temporal_window(): -+ layout = attn_metadata.video_layout -+ targets = [] if layout is None else [span for span in layout.video_spans if span.role == "target"] -+ if len(targets) != 1: -+ raise ValueError("H3 temporal window requires exactly one target-video span") -+ target = targets[0] -+ frames, height, width = target.latent_grid -+ return _video_temporal_window_bnsd( -+ query, -+ key[:, :used_k], -+ value[:, :used_k], -+ self.softmax_scale, -+ target_start=target.start, -+ frames=frames, -+ rows_per_frame=height * width, -+ ) - return _prefix_bnsd(query, key[:, :used_k], value[:, :used_k], self.softmax_scale) diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c17-source-identity.json b/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c17-source-identity.json deleted file mode 100644 index ff1baa78f..000000000 --- a/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c17-source-identity.json +++ /dev/null @@ -1,12 +0,0 @@ -{ - "head": "c1589a2b5e4ada617e7cca38ef7c300fe9495c58", - "tree": "47307747212d03fc20f4739ea9e0f3e3ec08f2ee", - "base": "e89b1d40d37f7f72767892a56fa7980ea572ba8e", - "branch": "rsi/h3-297-video-temporal-window", - "files": { - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "66ca27991ce42ce302e5de078be6395c9d1ab53752378babef25089f3f75c3aa", - "tests/diffusion/models/minimax_h3/test_minimax_h3_video_window.py": "0bc59f35eff41164e125a5070e034c94ea60bf0112733ebe59eb6378eccbfed8" - }, - "published": false, - "patch_base": "e89b1d40d37f7f72767892a56fa7980ea572ba8e" -} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c18-candidate.patch b/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c18-candidate.patch deleted file mode 100644 index 0bf4eade1..000000000 --- a/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c18-candidate.patch +++ /dev/null @@ -1,328 +0,0 @@ -diff --git a/tests/diffusion/models/minimax_h3/test_minimax_h3_pooled_context.py b/tests/diffusion/models/minimax_h3/test_minimax_h3_pooled_context.py -new file mode 100644 -index 0000000..5cffb88 ---- /dev/null -+++ b/tests/diffusion/models/minimax_h3/test_minimax_h3_pooled_context.py -@@ -0,0 +1,183 @@ -+# SPDX-License-Identifier: Apache-2.0 -+# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project -+ -+import math -+from types import SimpleNamespace -+ -+import pytest -+import torch -+ -+from tests.helpers.mark import hardware_marks -+from vllm_omni.diffusion.models.minimax_h3.attention import flash_h3 as flash -+from vllm_omni.platforms import current_omni_platform -+ -+pytestmark = [pytest.mark.core_model, pytest.mark.diffusion, *hardware_marks(res={"npu": "A3"}, num_cards=1)] -+ -+ -+def dense(query, key, value, scale): -+ q, k, v = [x.transpose(1, 2).float() for x in (query, key, value)] -+ return (torch.softmax(q @ k.transpose(-2, -1) * scale, dim=-1) @ v).transpose(1, 2).to(query.dtype) -+ -+ -+def expected_video(q, k, v, scale, frames, height, width, prefix): -+ rows = height * width -+ result = torch.zeros_like(q) -+ result[:, :prefix] = dense(q[:, :prefix], k, v, scale) -+ for t in range(frames): -+ key_parts, value_parts = ( -+ [k[:, :prefix], k[:, prefix + t * rows : prefix + (t + 1) * rows]], -+ [v[:, :prefix], v[:, prefix + t * rows : prefix + (t + 1) * rows]], -+ ) -+ for f in range(frames): -+ if f == t: -+ continue -+ for y in range(0, height, 3): -+ for x in range(0, width, 3): -+ indices = [prefix + f * rows + (y + dy) * width + x + dx for dy in range(3) for dx in range(3)] -+ key_parts.append(k[:, indices].float().mean(1, keepdim=True).to(k.dtype)) -+ value_parts.append(v[:, indices].float().mean(1, keepdim=True).to(v.dtype)) -+ keys, values = torch.cat(key_parts, 1), torch.cat(value_parts, 1) -+ queries = q[:, prefix + t * rows : prefix + (t + 1) * rows] -+ logits = queries.transpose(1, 2).float() @ keys.transpose(1, 2).float().transpose(-2, -1) * scale -+ logits[..., prefix + rows :] += math.log(9) -+ output = torch.softmax(logits, -1) @ values.transpose(1, 2).float() -+ result[:, prefix + t * rows : prefix + (t + 1) * rows] = output.transpose(1, 2).to(q.dtype) -+ return result -+ -+ -+def fixture(frames=3, height=3, width=6, prefix=4, heads=2, dim=8, device="cpu"): -+ torch.manual_seed(217) -+ used = prefix + frames * height * width -+ q = torch.randn(1, used + 2, heads, dim, dtype=torch.bfloat16).to(device) -+ k, v = [torch.randn(1, used, heads, dim, dtype=torch.bfloat16).to(device) for _ in range(2)] -+ return q, k, v -+ -+ -+@pytest.mark.parametrize("frames,height,width,prefix", [(2, 3, 3, 0), (3, 3, 6, 4), (5, 6, 3, 2)]) -+def test_pooling_matches_independent_weighted_key_reference(monkeypatch, frames, height, width, prefix): -+ q, k, v = fixture(frames, height, width, prefix) -+ saved = [x.clone() for x in (q, k, v)] -+ monkeypatch.setattr(flash, "_prefix_bnsd", dense) -+ actual = flash._video_pooled_context_bnsd( -+ q, k, v, 0.25, target_start=prefix, frames=frames, height=height, width=width -+ ) -+ expected = expected_video(q, k, v, 0.25, frames, height, width, prefix) -+ torch.testing.assert_close(actual.float(), expected.float(), atol=0.0078125, rtol=0.02) -+ assert torch.count_nonzero(actual[:, k.shape[1] :]) == 0 -+ assert all(torch.equal(a, b) for a, b in zip(saved, (q, k, v))) -+ -+ -+def test_nine_identical_keys_retain_original_softmax_mass(monkeypatch): -+ q, k, v = fixture(height=3, width=3) -+ for tensor in (k, v): -+ for f in range(3): -+ tensor[:, 4 + f * 9 : 4 + (f + 1) * 9] = tensor[:, 4 + f * 9 : 5 + f * 9].clone() -+ monkeypatch.setattr(flash, "_prefix_bnsd", dense) -+ actual = flash._video_pooled_context_bnsd(q, k, v, 0.25, target_start=4, frames=3, height=3, width=3) -+ expected = dense(q[:, : k.shape[1]], k, v, 0.25) -+ torch.testing.assert_close(actual[:, : k.shape[1]].float(), expected.float(), atol=0.0078125, rtol=0.02) -+ -+ -+@pytest.mark.parametrize("kind", ["height", "width", "tail", "frames", "batch", "query", "scale"]) -+def test_invalid_requested_geometry_raises(kind): -+ q, k, v = fixture() -+ kwargs = dict(target_start=4, frames=3, height=3, width=6, scale=0.25) -+ if kind in ("height", "width"): -+ kwargs[kind] += 1 -+ elif kind == "tail": -+ kwargs["target_start"] += 1 -+ elif kind == "frames": -+ kwargs["frames"] = 1 -+ elif kind == "batch": -+ q, k, v = [tensor.expand(2, -1, -1, -1) for tensor in (q, k, v)] -+ elif kind == "query": -+ q = q[:, :2] -+ else: -+ kwargs["scale"] = 0 -+ with pytest.raises(ValueError, match="pooled context"): -+ flash._video_pooled_context_bnsd(q, k, v, **kwargs) -+ -+ -+def test_fixed_progress_and_default_off(monkeypatch): -+ monkeypatch.setenv("H3_VIDEO_POOLED_CONTEXT", "3x3-local1") -+ for step in range(50): -+ monkeypatch.setattr( -+ flash, "get_forward_context", lambda: SimpleNamespace(denoise_step_idx=step, total_denoise_steps=50) -+ ) -+ assert flash._use_video_pooled_context() == (step not in (0, 49)) -+ monkeypatch.setattr( -+ flash, "get_forward_context", lambda: SimpleNamespace(denoise_step_idx=None, total_denoise_steps=50) -+ ) -+ with pytest.raises(ValueError, match="published"): -+ flash._use_video_pooled_context() -+ monkeypatch.setenv("H3_VIDEO_POOLED_CONTEXT", "other") -+ with pytest.raises(ValueError, match="3x3-local1"): -+ flash._use_video_pooled_context() -+ monkeypatch.delenv("H3_VIDEO_POOLED_CONTEXT") -+ monkeypatch.setenv("MINDIE_SD_FA_TYPE", "fused_attn_score") -+ impl = object.__new__(flash.MiniMaxH3FlashAttentionImpl) -+ impl.causal, impl.softmax_scale = False, 0.25 -+ impl._resolve_packed_seq_npu = lambda *args: ([4096], [4093]) -+ monkeypatch.setattr(flash, "get_forward_context", lambda: pytest.fail("default path read experimental progress")) -+ monkeypatch.setattr(flash, "_prefix_bnsd", lambda q, k, v, scale: q) -+ q = torch.ones(1, 4096, 1, 8, dtype=torch.bfloat16) -+ from vllm_omni.diffusion.attention.backends.abstract import AttentionMetadata -+ -+ assert impl.forward_fa_npu(q, q, q, AttentionMetadata(extra={"npu_attn_varlen": True})) is q -+ -+ -+@pytest.mark.parametrize("step", [0, 1, 49]) -+def test_requested_dispatch_uses_published_grid(monkeypatch, step): -+ from vllm_omni.diffusion.attention.backends.abstract import AttentionMetadata, VideoTokenLayout, VideoTokenSpan -+ -+ monkeypatch.setenv("MINDIE_SD_FA_TYPE", "fused_attn_score") -+ monkeypatch.setenv("H3_VIDEO_POOLED_CONTEXT", "3x3-local1") -+ monkeypatch.setattr( -+ flash, "get_forward_context", lambda: SimpleNamespace(denoise_step_idx=step, total_denoise_steps=50) -+ ) -+ impl = object.__new__(flash.MiniMaxH3FlashAttentionImpl) -+ impl.causal, impl.softmax_scale = False, 0.25 -+ impl._resolve_packed_seq_npu = lambda *args: ([4096], [4096]) -+ q = torch.ones(1, 4192, 1, 8, dtype=torch.bfloat16) -+ metadata = AttentionMetadata( -+ extra={"npu_attn_varlen": True}, -+ video_layout=VideoTokenLayout( -+ used_len=4096, video_spans=(VideoTokenSpan(start=64, latent_grid=(4, 24, 42), role="target"),) -+ ), -+ ) -+ calls = [] -+ monkeypatch.setattr(flash, "_prefix_bnsd", lambda *args: calls.append("dense") or q) -+ -+ def pooled(query, key, value, scale, **kwargs): -+ assert key.shape[1] == value.shape[1] == 4096 -+ assert kwargs == {"target_start": 64, "frames": 4, "height": 24, "width": 42} -+ calls.append("pooled") -+ return q -+ -+ monkeypatch.setattr(flash, "_video_pooled_context_bnsd", pooled) -+ assert impl.forward_fa_npu(q, q, q, metadata) is q -+ assert calls == (["dense"] if step in (0, 49) else ["pooled"]) -+ -+ -+def test_requested_unsupported_main_geometry_does_not_fallback(monkeypatch): -+ from vllm_omni.diffusion.attention.backends.abstract import AttentionMetadata -+ -+ monkeypatch.setenv("MINDIE_SD_FA_TYPE", "fused_attn_score") -+ monkeypatch.setenv("H3_VIDEO_POOLED_CONTEXT", "3x3-local1") -+ impl = object.__new__(flash.MiniMaxH3FlashAttentionImpl) -+ impl.causal = True -+ q = torch.ones(1, 4096, 1, 8, dtype=torch.bfloat16) -+ with pytest.raises(ValueError, match="unsupported"): -+ impl.forward_fa_npu(q, q, q, AttentionMetadata(extra={"npu_attn_varlen": True})) -+ -+ -+@pytest.mark.skipif(not current_omni_platform.is_npu(), reason="Requires native Ascend FAS") -+def test_native_pooled_fas_executes_padded_head_with_mass_compensation(): -+ import torch_npu # noqa: F401 -+ -+ q, k, v = fixture(height=6, width=6, heads=14, dim=128, device="npu") -+ actual = flash._video_pooled_context_bnsd(q, k, v, 128**-0.5, target_start=4, frames=3, height=6, width=6) -+ torch.npu.synchronize() -+ assert actual.dtype == torch.bfloat16 and torch.isfinite(actual).all() -+ expected = expected_video(q.cpu(), k.cpu(), v.cpu(), 128**-0.5, 3, 6, 6, 4) -+ torch.testing.assert_close(actual.cpu().float(), expected.float(), atol=0.03125, rtol=0.025) -diff --git a/vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py b/vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py -index 6ebfb10..e2b46bc 100644 ---- a/vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py -+++ b/vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py -@@ -4,12 +4,14 @@ - - from __future__ import annotations - -+import math - import os - - import torch - - from vllm_omni.diffusion.attention.backends.abstract import AttentionMetadata - from vllm_omni.diffusion.attention.backends.flash_attn import FlashAttentionImpl -+from vllm_omni.diffusion.forward_context import get_forward_context - - - def _prefix_bnsd(query: torch.Tensor, key: torch.Tensor, value: torch.Tensor, scale: float) -> torch.Tensor: -@@ -36,6 +38,77 @@ def _prefix_bnsd(query: torch.Tensor, key: torch.Tensor, value: torch.Tensor, sc - return output.reshape(batch, heads, rows, dim).transpose(1, 2).contiguous() - - -+def _video_pooled_context_bnsd( -+ query: torch.Tensor, -+ key: torch.Tensor, -+ value: torch.Tensor, -+ scale: float, -+ *, -+ target_start: int, -+ frames: int, -+ height: int, -+ width: int, -+) -> torch.Tensor: -+ """Keep current-frame detail and mass-compensated global spatial summaries.""" -+ rows = height * width -+ if ( -+ query.shape[0] != 1 -+ or key.shape[0] != 1 -+ or value.shape != key.shape -+ or query.shape[2:] != key.shape[2:] -+ or target_start < 0 -+ or frames < 2 -+ or height < 3 -+ or width < 3 -+ or height % 3 -+ or width % 3 -+ or target_start + frames * rows != key.shape[1] -+ or query.shape[1] < key.shape[1] -+ or scale <= 0 -+ ): -+ raise ValueError("H3 pooled context requires a final video grid divisible by three") -+ heads, dim = query.shape[2:] -+ indices = torch.arange(frames, device=key.device).expand(frames, -1) -+ others = indices[~torch.eye(frames, dtype=torch.bool, device=key.device)].reshape(frames, frames - 1) -+ -+ def pack(tensor: torch.Tensor) -> torch.Tensor: -+ video = tensor[:, target_start:].reshape(frames, rows, heads, dim) -+ pooled = ( -+ video.reshape(frames, height // 3, 3, width // 3, 3, heads, dim) -+ .float() -+ .mean(dim=(2, 4)) -+ .to(tensor.dtype) -+ .reshape(frames, rows // 9, heads, dim) -+ ) -+ global_context = pooled.index_select(0, others.flatten()).reshape(frames, -1, heads, dim) -+ return torch.cat((tensor[:, :target_start].expand(frames, -1, -1, -1), video, global_context), dim=1) -+ -+ output = torch.zeros_like(query) -+ if target_start: -+ output[:, :target_start] = _prefix_bnsd(query[:, :target_start], key, value, scale) -+ video_query = query[:, target_start : key.shape[1]].reshape(frames, rows, heads, dim) -+ # An extra dot-product coordinate adds log(9) to each nine-token summary. -+ # Keep the original scale; changing the padded head size must not rescale QK. -+ padded_q = torch.nn.functional.pad(video_query, (0, 16)) -+ padded_k = torch.nn.functional.pad(pack(key), (0, 16)) -+ padded_v = torch.nn.functional.pad(pack(value), (0, 16)) -+ padded_q[..., dim] = 1 -+ padded_k[:, target_start + rows :, :, dim] = math.log(9) / scale -+ attended = _prefix_bnsd(padded_q, padded_k, padded_v, scale)[..., :dim] -+ output[:, target_start : key.shape[1]] = attended.reshape(1, frames * rows, heads, dim) -+ return output -+ -+ -+def _use_video_pooled_context() -> bool: -+ if os.environ.get("H3_VIDEO_POOLED_CONTEXT") != "3x3-local1": -+ raise ValueError("The experimental H3 pooled-context profile requires 3x3-local1") -+ context = get_forward_context() -+ step, total = context.denoise_step_idx, context.total_denoise_steps -+ if total != 50 or step is None or not 0 <= step < 50: -+ raise ValueError("The experimental H3 pooled-context profile requires published 50-step progress") -+ return step not in (0, 49) -+ -+ - class MiniMaxH3FlashAttentionImpl(FlashAttentionImpl): - """Use independent head batches for explicit long BF16 prefix attention. - -@@ -51,7 +124,10 @@ class MiniMaxH3FlashAttentionImpl(FlashAttentionImpl): - value: torch.Tensor, - attn_metadata: AttentionMetadata | None = None, - ) -> torch.Tensor: -+ requested = os.environ.get("H3_VIDEO_POOLED_CONTEXT") is not None and query.shape[1] >= 4096 - if os.environ.get("MINDIE_SD_FA_TYPE") != "fused_attn_score" or attn_metadata is None: -+ if requested: -+ raise ValueError("H3 pooled context requires explicit fused attention and packed metadata") - return super().forward_fa_npu(query, key, value, attn_metadata) - extra = attn_metadata.extra - if ( -@@ -68,9 +144,30 @@ class MiniMaxH3FlashAttentionImpl(FlashAttentionImpl): - or query.shape[2:] != value.shape[2:] - or key.shape[1] != value.shape[1] - ): -+ if requested: -+ raise ValueError("H3 pooled context received unsupported packed fused-attention inputs") - return super().forward_fa_npu(query, key, value, attn_metadata) - resolved = self._resolve_packed_seq_npu(query, key, extra) - if resolved is None: -+ if requested: -+ raise ValueError("H3 pooled context requires a resolved packed prefix") - return super().forward_fa_npu(query, key, value, attn_metadata) - used_k = resolved[1][0] -+ if os.environ.get("H3_VIDEO_POOLED_CONTEXT") is not None and _use_video_pooled_context(): -+ layout = attn_metadata.video_layout -+ targets = [] if layout is None else [span for span in layout.video_spans if span.role == "target"] -+ if len(targets) != 1: -+ raise ValueError("H3 pooled context requires exactly one target-video span") -+ target = targets[0] -+ frames, height, width = target.latent_grid -+ return _video_pooled_context_bnsd( -+ query, -+ key[:, :used_k], -+ value[:, :used_k], -+ self.softmax_scale, -+ target_start=target.start, -+ frames=frames, -+ height=height, -+ width=width, -+ ) - return _prefix_bnsd(query, key[:, :used_k], value[:, :used_k], self.softmax_scale) diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c18-source-identity.json b/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c18-source-identity.json deleted file mode 100644 index cdd19ca7f..000000000 --- a/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c18-source-identity.json +++ /dev/null @@ -1,12 +0,0 @@ -{ - "head": "f278cb2bf4647aeb852d61bcc82d9fcc1420d64d", - "tree": "626aacaf0da39c527e1c20c09008e6e942119c5f", - "base": "e89b1d40d37f7f72767892a56fa7980ea572ba8e", - "branch": "rsi/h3-297-video-pooled-context", - "files": { - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "1febc030a7917b6e10fe287014fe2aacef76ade7bf58ebb604fdedc36fb7c21d", - "tests/diffusion/models/minimax_h3/test_minimax_h3_pooled_context.py": "45a9ae5ba56858daee6a1088a8c8b1ed612ea9afd3c229c90a9b6db3c80a06b2" - }, - "published": false, - "patch_base": "e89b1d40d37f7f72767892a56fa7980ea572ba8e" -} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c19-candidate.patch b/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c19-candidate.patch deleted file mode 100644 index 9d511c4e9..000000000 --- a/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c19-candidate.patch +++ /dev/null @@ -1,317 +0,0 @@ -diff --git a/tests/diffusion/models/minimax_h3/test_minimax_h3_global_pool.py b/tests/diffusion/models/minimax_h3/test_minimax_h3_global_pool.py -new file mode 100644 -index 0000000..b0c13de ---- /dev/null -+++ b/tests/diffusion/models/minimax_h3/test_minimax_h3_global_pool.py -@@ -0,0 +1,177 @@ -+# SPDX-License-Identifier: Apache-2.0 -+# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project -+ -+import math -+from types import SimpleNamespace -+ -+import pytest -+import torch -+ -+from tests.helpers.mark import hardware_marks -+from vllm_omni.diffusion.models.minimax_h3.attention import flash_h3 as flash -+from vllm_omni.platforms import current_omni_platform -+ -+pytestmark = [pytest.mark.core_model, pytest.mark.diffusion, *hardware_marks(res={"npu": "A3"}, num_cards=1)] -+ -+ -+def dense(query, key, value, scale): -+ q, k, v = [x.transpose(1, 2).float() for x in (query, key, value)] -+ return (torch.softmax(q @ k.transpose(-2, -1) * scale, dim=-1) @ v).transpose(1, 2).to(query.dtype) -+ -+ -+def expected_video(q, k, v, scale, frames, height, width, prefix): -+ rows = height * width -+ result = torch.zeros_like(q) -+ result[:, :prefix] = dense(q[:, :prefix], k, v, scale) -+ key_parts, value_parts = [k[:, :prefix]], [v[:, :prefix]] -+ for f in range(frames): -+ for y in range(0, height, 3): -+ for x in range(0, width, 3): -+ indices = [prefix + f * rows + (y + dy) * width + x + dx for dy in range(3) for dx in range(3)] -+ key_parts.append(k[:, indices].float().mean(1, keepdim=True).to(k.dtype)) -+ value_parts.append(v[:, indices].float().mean(1, keepdim=True).to(v.dtype)) -+ keys, values = torch.cat(key_parts, 1), torch.cat(value_parts, 1) -+ queries = q[:, prefix : k.shape[1]] -+ logits = queries.transpose(1, 2).float() @ keys.transpose(1, 2).float().transpose(-2, -1) * scale -+ logits[..., prefix:] += math.log(9) -+ output = torch.softmax(logits, -1) @ values.transpose(1, 2).float() -+ result[:, prefix : k.shape[1]] = output.transpose(1, 2).to(q.dtype) -+ return result -+ -+ -+def fixture(frames=3, height=3, width=6, prefix=4, heads=2, dim=8, device="cpu"): -+ torch.manual_seed(217) -+ used = prefix + frames * height * width -+ q = torch.randn(1, used + 2, heads, dim, dtype=torch.bfloat16).to(device) -+ k, v = [torch.randn(1, used, heads, dim, dtype=torch.bfloat16).to(device) for _ in range(2)] -+ return q, k, v -+ -+ -+@pytest.mark.parametrize("frames,height,width,prefix", [(2, 3, 3, 0), (3, 3, 6, 4), (5, 6, 3, 2)]) -+def test_pooling_matches_independent_weighted_key_reference(monkeypatch, frames, height, width, prefix): -+ q, k, v = fixture(frames, height, width, prefix) -+ saved = [x.clone() for x in (q, k, v)] -+ monkeypatch.setattr(flash, "_prefix_bnsd", dense) -+ actual = flash._video_global_pool_bnsd( -+ q, k, v, 0.25, target_start=prefix, frames=frames, height=height, width=width -+ ) -+ expected = expected_video(q, k, v, 0.25, frames, height, width, prefix) -+ torch.testing.assert_close(actual.float(), expected.float(), atol=0.0078125, rtol=0.02) -+ assert torch.count_nonzero(actual[:, k.shape[1] :]) == 0 -+ assert all(torch.equal(a, b) for a, b in zip(saved, (q, k, v))) -+ -+ -+def test_nine_identical_keys_retain_original_softmax_mass(monkeypatch): -+ q, k, v = fixture(height=3, width=3) -+ for tensor in (k, v): -+ for f in range(3): -+ tensor[:, 4 + f * 9 : 4 + (f + 1) * 9] = tensor[:, 4 + f * 9 : 5 + f * 9].clone() -+ monkeypatch.setattr(flash, "_prefix_bnsd", dense) -+ actual = flash._video_global_pool_bnsd(q, k, v, 0.25, target_start=4, frames=3, height=3, width=3) -+ expected = dense(q[:, : k.shape[1]], k, v, 0.25) -+ torch.testing.assert_close(actual[:, : k.shape[1]].float(), expected.float(), atol=0.0078125, rtol=0.02) -+ -+ -+@pytest.mark.parametrize("kind", ["height", "width", "tail", "frames", "batch", "query", "scale"]) -+def test_invalid_requested_geometry_raises(kind): -+ q, k, v = fixture() -+ kwargs = dict(target_start=4, frames=3, height=3, width=6, scale=0.25) -+ if kind in ("height", "width"): -+ kwargs[kind] += 1 -+ elif kind == "tail": -+ kwargs["target_start"] += 1 -+ elif kind == "frames": -+ kwargs["frames"] = 1 -+ elif kind == "batch": -+ q, k, v = [tensor.expand(2, -1, -1, -1) for tensor in (q, k, v)] -+ elif kind == "query": -+ q = q[:, :2] -+ else: -+ kwargs["scale"] = 0 -+ with pytest.raises(ValueError, match="pooled context"): -+ flash._video_global_pool_bnsd(q, k, v, **kwargs) -+ -+ -+def test_fixed_progress_and_default_off(monkeypatch): -+ monkeypatch.setenv("H3_VIDEO_GLOBAL_POOL", "3x3-global") -+ for step in range(50): -+ monkeypatch.setattr( -+ flash, "get_forward_context", lambda: SimpleNamespace(denoise_step_idx=step, total_denoise_steps=50) -+ ) -+ assert flash._use_video_global_pool() == (step not in (0, 49)) -+ monkeypatch.setattr( -+ flash, "get_forward_context", lambda: SimpleNamespace(denoise_step_idx=None, total_denoise_steps=50) -+ ) -+ with pytest.raises(ValueError, match="published"): -+ flash._use_video_global_pool() -+ monkeypatch.setenv("H3_VIDEO_GLOBAL_POOL", "other") -+ with pytest.raises(ValueError, match="3x3-global"): -+ flash._use_video_global_pool() -+ monkeypatch.delenv("H3_VIDEO_GLOBAL_POOL") -+ monkeypatch.setenv("MINDIE_SD_FA_TYPE", "fused_attn_score") -+ impl = object.__new__(flash.MiniMaxH3FlashAttentionImpl) -+ impl.causal, impl.softmax_scale = False, 0.25 -+ impl._resolve_packed_seq_npu = lambda *args: ([4096], [4093]) -+ monkeypatch.setattr(flash, "get_forward_context", lambda: pytest.fail("default path read experimental progress")) -+ monkeypatch.setattr(flash, "_prefix_bnsd", lambda q, k, v, scale: q) -+ q = torch.ones(1, 4096, 1, 8, dtype=torch.bfloat16) -+ from vllm_omni.diffusion.attention.backends.abstract import AttentionMetadata -+ -+ assert impl.forward_fa_npu(q, q, q, AttentionMetadata(extra={"npu_attn_varlen": True})) is q -+ -+ -+@pytest.mark.parametrize("step", [0, 1, 49]) -+def test_requested_dispatch_uses_published_grid(monkeypatch, step): -+ from vllm_omni.diffusion.attention.backends.abstract import AttentionMetadata, VideoTokenLayout, VideoTokenSpan -+ -+ monkeypatch.setenv("MINDIE_SD_FA_TYPE", "fused_attn_score") -+ monkeypatch.setenv("H3_VIDEO_GLOBAL_POOL", "3x3-global") -+ monkeypatch.setattr( -+ flash, "get_forward_context", lambda: SimpleNamespace(denoise_step_idx=step, total_denoise_steps=50) -+ ) -+ impl = object.__new__(flash.MiniMaxH3FlashAttentionImpl) -+ impl.causal, impl.softmax_scale = False, 0.25 -+ impl._resolve_packed_seq_npu = lambda *args: ([4096], [4096]) -+ q = torch.ones(1, 4192, 1, 8, dtype=torch.bfloat16) -+ metadata = AttentionMetadata( -+ extra={"npu_attn_varlen": True}, -+ video_layout=VideoTokenLayout( -+ used_len=4096, video_spans=(VideoTokenSpan(start=64, latent_grid=(4, 24, 42), role="target"),) -+ ), -+ ) -+ calls = [] -+ monkeypatch.setattr(flash, "_prefix_bnsd", lambda *args: calls.append("dense") or q) -+ -+ def pooled(query, key, value, scale, **kwargs): -+ assert key.shape[1] == value.shape[1] == 4096 -+ assert kwargs == {"target_start": 64, "frames": 4, "height": 24, "width": 42} -+ calls.append("pooled") -+ return q -+ -+ monkeypatch.setattr(flash, "_video_global_pool_bnsd", pooled) -+ assert impl.forward_fa_npu(q, q, q, metadata) is q -+ assert calls == (["dense"] if step in (0, 49) else ["pooled"]) -+ -+ -+def test_requested_unsupported_main_geometry_does_not_fallback(monkeypatch): -+ from vllm_omni.diffusion.attention.backends.abstract import AttentionMetadata -+ -+ monkeypatch.setenv("MINDIE_SD_FA_TYPE", "fused_attn_score") -+ monkeypatch.setenv("H3_VIDEO_GLOBAL_POOL", "3x3-global") -+ impl = object.__new__(flash.MiniMaxH3FlashAttentionImpl) -+ impl.causal = True -+ q = torch.ones(1, 4096, 1, 8, dtype=torch.bfloat16) -+ with pytest.raises(ValueError, match="unsupported"): -+ impl.forward_fa_npu(q, q, q, AttentionMetadata(extra={"npu_attn_varlen": True})) -+ -+ -+@pytest.mark.skipif(not current_omni_platform.is_npu(), reason="Requires native Ascend FAS") -+def test_native_global_pool_fas_executes_shared_kv_with_mass_compensation(): -+ import torch_npu # noqa: F401 -+ -+ q, k, v = fixture(height=6, width=6, heads=14, dim=128, device="npu") -+ actual = flash._video_global_pool_bnsd(q, k, v, 128**-0.5, target_start=4, frames=3, height=6, width=6) -+ torch.npu.synchronize() -+ assert actual.dtype == torch.bfloat16 and torch.isfinite(actual).all() -+ expected = expected_video(q.cpu(), k.cpu(), v.cpu(), 128**-0.5, 3, 6, 6, 4) -+ torch.testing.assert_close(actual.cpu().float(), expected.float(), atol=0.03125, rtol=0.025) -diff --git a/vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py b/vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py -index 6ebfb10..4932926 100644 ---- a/vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py -+++ b/vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py -@@ -4,12 +4,14 @@ - - from __future__ import annotations - -+import math - import os - - import torch - - from vllm_omni.diffusion.attention.backends.abstract import AttentionMetadata - from vllm_omni.diffusion.attention.backends.flash_attn import FlashAttentionImpl -+from vllm_omni.diffusion.forward_context import get_forward_context - - - def _prefix_bnsd(query: torch.Tensor, key: torch.Tensor, value: torch.Tensor, scale: float) -> torch.Tensor: -@@ -36,6 +38,72 @@ def _prefix_bnsd(query: torch.Tensor, key: torch.Tensor, value: torch.Tensor, sc - return output.reshape(batch, heads, rows, dim).transpose(1, 2).contiguous() - - -+def _video_global_pool_bnsd( -+ query: torch.Tensor, -+ key: torch.Tensor, -+ value: torch.Tensor, -+ scale: float, -+ *, -+ target_start: int, -+ frames: int, -+ height: int, -+ width: int, -+) -> torch.Tensor: -+ """Share mass-compensated global spatial summaries across all video queries.""" -+ rows = height * width -+ if ( -+ query.shape[0] != 1 -+ or key.shape[0] != 1 -+ or value.shape != key.shape -+ or query.shape[2:] != key.shape[2:] -+ or target_start < 0 -+ or frames < 2 -+ or height < 3 -+ or width < 3 -+ or height % 3 -+ or width % 3 -+ or target_start + frames * rows != key.shape[1] -+ or query.shape[1] < key.shape[1] -+ or scale <= 0 -+ ): -+ raise ValueError("H3 pooled context requires a final video grid divisible by three") -+ heads, dim = query.shape[2:] -+ -+ def pack(tensor: torch.Tensor) -> torch.Tensor: -+ pooled = ( -+ tensor[:, target_start:] -+ .reshape(frames, height // 3, 3, width // 3, 3, heads, dim) -+ .float() -+ .mean(dim=(2, 4)) -+ .to(tensor.dtype) -+ .reshape(1, frames * rows // 9, heads, dim) -+ ) -+ return torch.cat((tensor[:, :target_start], pooled), dim=1) -+ -+ output = torch.zeros_like(query) -+ if target_start: -+ output[:, :target_start] = _prefix_bnsd(query[:, :target_start], key, value, scale) -+ video_query = query[:, target_start : key.shape[1]] -+ padded_q = torch.nn.functional.pad(video_query, (0, 16)) -+ padded_k = torch.nn.functional.pad(pack(key), (0, 16)) -+ padded_v = torch.nn.functional.pad(pack(value), (0, 16)) -+ padded_q[..., dim] = 1 -+ padded_k[:, target_start:, :, dim] = math.log(9) / scale -+ attended = _prefix_bnsd(padded_q, padded_k, padded_v, scale)[..., :dim] -+ output[:, target_start : key.shape[1]] = attended -+ return output -+ -+ -+def _use_video_global_pool() -> bool: -+ if os.environ.get("H3_VIDEO_GLOBAL_POOL") != "3x3-global": -+ raise ValueError("The experimental H3 pooled-context profile requires 3x3-global") -+ context = get_forward_context() -+ step, total = context.denoise_step_idx, context.total_denoise_steps -+ if total != 50 or step is None or not 0 <= step < 50: -+ raise ValueError("The experimental H3 pooled-context profile requires published 50-step progress") -+ return step not in (0, 49) -+ -+ - class MiniMaxH3FlashAttentionImpl(FlashAttentionImpl): - """Use independent head batches for explicit long BF16 prefix attention. - -@@ -51,7 +119,10 @@ class MiniMaxH3FlashAttentionImpl(FlashAttentionImpl): - value: torch.Tensor, - attn_metadata: AttentionMetadata | None = None, - ) -> torch.Tensor: -+ requested = os.environ.get("H3_VIDEO_GLOBAL_POOL") is not None and query.shape[1] >= 4096 - if os.environ.get("MINDIE_SD_FA_TYPE") != "fused_attn_score" or attn_metadata is None: -+ if requested: -+ raise ValueError("H3 pooled context requires explicit fused attention and packed metadata") - return super().forward_fa_npu(query, key, value, attn_metadata) - extra = attn_metadata.extra - if ( -@@ -68,9 +139,30 @@ class MiniMaxH3FlashAttentionImpl(FlashAttentionImpl): - or query.shape[2:] != value.shape[2:] - or key.shape[1] != value.shape[1] - ): -+ if requested: -+ raise ValueError("H3 pooled context received unsupported packed fused-attention inputs") - return super().forward_fa_npu(query, key, value, attn_metadata) - resolved = self._resolve_packed_seq_npu(query, key, extra) - if resolved is None: -+ if requested: -+ raise ValueError("H3 pooled context requires a resolved packed prefix") - return super().forward_fa_npu(query, key, value, attn_metadata) - used_k = resolved[1][0] -+ if os.environ.get("H3_VIDEO_GLOBAL_POOL") is not None and _use_video_global_pool(): -+ layout = attn_metadata.video_layout -+ targets = [] if layout is None else [span for span in layout.video_spans if span.role == "target"] -+ if len(targets) != 1: -+ raise ValueError("H3 pooled context requires exactly one target-video span") -+ target = targets[0] -+ frames, height, width = target.latent_grid -+ return _video_global_pool_bnsd( -+ query, -+ key[:, :used_k], -+ value[:, :used_k], -+ self.softmax_scale, -+ target_start=target.start, -+ frames=frames, -+ height=height, -+ width=width, -+ ) - return _prefix_bnsd(query, key[:, :used_k], value[:, :used_k], self.softmax_scale) diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c19-source-identity.json b/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c19-source-identity.json deleted file mode 100644 index bde8516a5..000000000 --- a/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c19-source-identity.json +++ /dev/null @@ -1,12 +0,0 @@ -{ - "head": "c896a8c98493d50e51ed6fa167e4b8d8c59b3ecb", - "tree": "367e7f05c2d640211ad4713cc0b62742b0b25a0a", - "base": "e89b1d40d37f7f72767892a56fa7980ea572ba8e", - "branch": "rsi/h3-297-video-global-pool", - "files": { - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ddfd6d6822cc6a5528fc95539f7f98dfdeed25d5489da3ed5cf76e6076a5ec94", - "tests/diffusion/models/minimax_h3/test_minimax_h3_global_pool.py": "5457db550b59e3373be8f5f21873c1b78ac357a6ac170c2603ee531b50e91467" - }, - "published": false, - "patch_base": "e89b1d40d37f7f72767892a56fa7980ea572ba8e" -} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c20-candidate.patch b/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c20-candidate.patch deleted file mode 100644 index 34957ab40..000000000 --- a/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c20-candidate.patch +++ /dev/null @@ -1,328 +0,0 @@ -diff --git a/tests/diffusion/models/minimax_h3/test_minimax_h3_variance_pool.py b/tests/diffusion/models/minimax_h3/test_minimax_h3_variance_pool.py -new file mode 100644 -index 0000000..cda0c4a ---- /dev/null -+++ b/tests/diffusion/models/minimax_h3/test_minimax_h3_variance_pool.py -@@ -0,0 +1,182 @@ -+# SPDX-License-Identifier: Apache-2.0 -+# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project -+ -+import math -+from types import SimpleNamespace -+ -+import pytest -+import torch -+ -+from tests.helpers.mark import hardware_marks -+from vllm_omni.diffusion.models.minimax_h3.attention import flash_h3 as flash -+from vllm_omni.platforms import current_omni_platform -+ -+pytestmark = [pytest.mark.core_model, pytest.mark.diffusion, *hardware_marks(res={"npu": "A3"}, num_cards=1)] -+ -+ -+def dense(query, key, value, scale): -+ q, k, v = [x.transpose(1, 2).float() for x in (query, key, value)] -+ return (torch.softmax(q @ k.transpose(-2, -1) * scale, dim=-1) @ v).transpose(1, 2).to(query.dtype) -+ -+ -+def expected_video(q, k, v, scale, frames, height, width, prefix): -+ rows = height * width -+ result = torch.zeros_like(q) -+ result[:, :prefix] = dense(q[:, :prefix], k, v, scale) -+ key_parts, value_parts = [k[:, :prefix]], [v[:, :prefix]] -+ variances = [torch.zeros(1, prefix, k.shape[2])] -+ for f in range(frames): -+ for y in range(0, height, 3): -+ for x in range(0, width, 3): -+ indices = [prefix + f * rows + (y + dy) * width + x + dx for dy in range(3) for dx in range(3)] -+ key_parts.append(k[:, indices].float().mean(1, keepdim=True).to(k.dtype)) -+ value_parts.append(v[:, indices].float().mean(1, keepdim=True).to(v.dtype)) -+ variances.append(k[:, indices].float().var(dim=1, unbiased=False).mean(-1).unsqueeze(1)) -+ keys, values = torch.cat(key_parts, 1), torch.cat(value_parts, 1) -+ queries = q[:, prefix : k.shape[1]] -+ logits = queries.transpose(1, 2).float() @ keys.transpose(1, 2).float().transpose(-2, -1) * scale -+ logits[..., prefix:] += math.log(9) -+ norm_sq = queries.transpose(1, 2).float().square().sum(-1, keepdim=True) -+ variance = torch.cat(variances, 1).transpose(1, 2).unsqueeze(2) -+ logits += 0.5 * scale**2 * norm_sq * variance -+ output = torch.softmax(logits, -1) @ values.transpose(1, 2).float() -+ result[:, prefix : k.shape[1]] = output.transpose(1, 2).to(q.dtype) -+ return result -+ -+ -+def fixture(frames=3, height=3, width=6, prefix=4, heads=2, dim=8, device="cpu"): -+ torch.manual_seed(217) -+ used = prefix + frames * height * width -+ q = torch.randn(1, used + 2, heads, dim, dtype=torch.bfloat16).to(device) -+ k, v = [torch.randn(1, used, heads, dim, dtype=torch.bfloat16).to(device) for _ in range(2)] -+ return q, k, v -+ -+ -+@pytest.mark.parametrize("frames,height,width,prefix", [(2, 3, 3, 0), (3, 3, 6, 4), (5, 6, 3, 2)]) -+def test_pooling_matches_independent_weighted_key_reference(monkeypatch, frames, height, width, prefix): -+ q, k, v = fixture(frames, height, width, prefix) -+ saved = [x.clone() for x in (q, k, v)] -+ monkeypatch.setattr(flash, "_prefix_bnsd", dense) -+ actual = flash._video_variance_pool_bnsd( -+ q, k, v, 0.25, target_start=prefix, frames=frames, height=height, width=width -+ ) -+ expected = expected_video(q, k, v, 0.25, frames, height, width, prefix) -+ torch.testing.assert_close(actual.float(), expected.float(), atol=0.0078125, rtol=0.02) -+ assert torch.count_nonzero(actual[:, k.shape[1] :]) == 0 -+ assert all(torch.equal(a, b) for a, b in zip(saved, (q, k, v))) -+ -+ -+def test_nine_identical_keys_retain_original_softmax_mass(monkeypatch): -+ q, k, v = fixture(height=3, width=3) -+ for tensor in (k, v): -+ for f in range(3): -+ tensor[:, 4 + f * 9 : 4 + (f + 1) * 9] = tensor[:, 4 + f * 9 : 5 + f * 9].clone() -+ monkeypatch.setattr(flash, "_prefix_bnsd", dense) -+ actual = flash._video_variance_pool_bnsd(q, k, v, 0.25, target_start=4, frames=3, height=3, width=3) -+ expected = dense(q[:, : k.shape[1]], k, v, 0.25) -+ torch.testing.assert_close(actual[:, : k.shape[1]].float(), expected.float(), atol=0.0078125, rtol=0.02) -+ -+ -+@pytest.mark.parametrize("kind", ["height", "width", "tail", "frames", "batch", "query", "scale"]) -+def test_invalid_requested_geometry_raises(kind): -+ q, k, v = fixture() -+ kwargs = dict(target_start=4, frames=3, height=3, width=6, scale=0.25) -+ if kind in ("height", "width"): -+ kwargs[kind] += 1 -+ elif kind == "tail": -+ kwargs["target_start"] += 1 -+ elif kind == "frames": -+ kwargs["frames"] = 1 -+ elif kind == "batch": -+ q, k, v = [tensor.expand(2, -1, -1, -1) for tensor in (q, k, v)] -+ elif kind == "query": -+ q = q[:, :2] -+ else: -+ kwargs["scale"] = 0 -+ with pytest.raises(ValueError, match="pooled context"): -+ flash._video_variance_pool_bnsd(q, k, v, **kwargs) -+ -+ -+def test_fixed_progress_and_default_off(monkeypatch): -+ monkeypatch.setenv("H3_VIDEO_VARIANCE_POOL", "3x3-variance") -+ for step in range(50): -+ monkeypatch.setattr( -+ flash, "get_forward_context", lambda: SimpleNamespace(denoise_step_idx=step, total_denoise_steps=50) -+ ) -+ assert flash._use_video_variance_pool() == (step not in (0, 49)) -+ monkeypatch.setattr( -+ flash, "get_forward_context", lambda: SimpleNamespace(denoise_step_idx=None, total_denoise_steps=50) -+ ) -+ with pytest.raises(ValueError, match="published"): -+ flash._use_video_variance_pool() -+ monkeypatch.setenv("H3_VIDEO_VARIANCE_POOL", "other") -+ with pytest.raises(ValueError, match="3x3-variance"): -+ flash._use_video_variance_pool() -+ monkeypatch.delenv("H3_VIDEO_VARIANCE_POOL") -+ monkeypatch.setenv("MINDIE_SD_FA_TYPE", "fused_attn_score") -+ impl = object.__new__(flash.MiniMaxH3FlashAttentionImpl) -+ impl.causal, impl.softmax_scale = False, 0.25 -+ impl._resolve_packed_seq_npu = lambda *args: ([4096], [4093]) -+ monkeypatch.setattr(flash, "get_forward_context", lambda: pytest.fail("default path read experimental progress")) -+ monkeypatch.setattr(flash, "_prefix_bnsd", lambda q, k, v, scale: q) -+ q = torch.ones(1, 4096, 1, 8, dtype=torch.bfloat16) -+ from vllm_omni.diffusion.attention.backends.abstract import AttentionMetadata -+ -+ assert impl.forward_fa_npu(q, q, q, AttentionMetadata(extra={"npu_attn_varlen": True})) is q -+ -+ -+@pytest.mark.parametrize("step", [0, 1, 49]) -+def test_requested_dispatch_uses_published_grid(monkeypatch, step): -+ from vllm_omni.diffusion.attention.backends.abstract import AttentionMetadata, VideoTokenLayout, VideoTokenSpan -+ -+ monkeypatch.setenv("MINDIE_SD_FA_TYPE", "fused_attn_score") -+ monkeypatch.setenv("H3_VIDEO_VARIANCE_POOL", "3x3-variance") -+ monkeypatch.setattr( -+ flash, "get_forward_context", lambda: SimpleNamespace(denoise_step_idx=step, total_denoise_steps=50) -+ ) -+ impl = object.__new__(flash.MiniMaxH3FlashAttentionImpl) -+ impl.causal, impl.softmax_scale = False, 0.25 -+ impl._resolve_packed_seq_npu = lambda *args: ([4096], [4096]) -+ q = torch.ones(1, 4192, 1, 8, dtype=torch.bfloat16) -+ metadata = AttentionMetadata( -+ extra={"npu_attn_varlen": True}, -+ video_layout=VideoTokenLayout( -+ used_len=4096, video_spans=(VideoTokenSpan(start=64, latent_grid=(4, 24, 42), role="target"),) -+ ), -+ ) -+ calls = [] -+ monkeypatch.setattr(flash, "_prefix_bnsd", lambda *args: calls.append("dense") or q) -+ -+ def pooled(query, key, value, scale, **kwargs): -+ assert key.shape[1] == value.shape[1] == 4096 -+ assert kwargs == {"target_start": 64, "frames": 4, "height": 24, "width": 42} -+ calls.append("pooled") -+ return q -+ -+ monkeypatch.setattr(flash, "_video_variance_pool_bnsd", pooled) -+ assert impl.forward_fa_npu(q, q, q, metadata) is q -+ assert calls == (["dense"] if step in (0, 49) else ["pooled"]) -+ -+ -+def test_requested_unsupported_main_geometry_does_not_fallback(monkeypatch): -+ from vllm_omni.diffusion.attention.backends.abstract import AttentionMetadata -+ -+ monkeypatch.setenv("MINDIE_SD_FA_TYPE", "fused_attn_score") -+ monkeypatch.setenv("H3_VIDEO_VARIANCE_POOL", "3x3-variance") -+ impl = object.__new__(flash.MiniMaxH3FlashAttentionImpl) -+ impl.causal = True -+ q = torch.ones(1, 4096, 1, 8, dtype=torch.bfloat16) -+ with pytest.raises(ValueError, match="unsupported"): -+ impl.forward_fa_npu(q, q, q, AttentionMetadata(extra={"npu_attn_varlen": True})) -+ -+ -+@pytest.mark.skipif(not current_omni_platform.is_npu(), reason="Requires native Ascend FAS") -+def test_native_variance_pool_fas_executes_second_moment_compensation(): -+ import torch_npu # noqa: F401 -+ -+ q, k, v = fixture(height=6, width=6, heads=14, dim=128, device="npu") -+ actual = flash._video_variance_pool_bnsd(q, k, v, 128**-0.5, target_start=4, frames=3, height=6, width=6) -+ torch.npu.synchronize() -+ assert actual.dtype == torch.bfloat16 and torch.isfinite(actual).all() -+ expected = expected_video(q.cpu(), k.cpu(), v.cpu(), 128**-0.5, 3, 6, 6, 4) -+ torch.testing.assert_close(actual.cpu().float(), expected.float(), atol=0.03125, rtol=0.025) -diff --git a/vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py b/vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py -index 6ebfb10..9dce8b9 100644 ---- a/vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py -+++ b/vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py -@@ -4,12 +4,14 @@ - - from __future__ import annotations - -+import math - import os - - import torch - - from vllm_omni.diffusion.attention.backends.abstract import AttentionMetadata - from vllm_omni.diffusion.attention.backends.flash_attn import FlashAttentionImpl -+from vllm_omni.diffusion.forward_context import get_forward_context - - - def _prefix_bnsd(query: torch.Tensor, key: torch.Tensor, value: torch.Tensor, scale: float) -> torch.Tensor: -@@ -36,6 +38,78 @@ def _prefix_bnsd(query: torch.Tensor, key: torch.Tensor, value: torch.Tensor, sc - return output.reshape(batch, heads, rows, dim).transpose(1, 2).contiguous() - - -+def _video_variance_pool_bnsd( -+ query: torch.Tensor, -+ key: torch.Tensor, -+ value: torch.Tensor, -+ scale: float, -+ *, -+ target_start: int, -+ frames: int, -+ height: int, -+ width: int, -+) -> torch.Tensor: -+ """Share global summaries with isotropic key-variance mass compensation.""" -+ rows = height * width -+ if ( -+ query.shape[0] != 1 -+ or key.shape[0] != 1 -+ or value.shape != key.shape -+ or query.shape[2:] != key.shape[2:] -+ or target_start < 0 -+ or frames < 2 -+ or height < 3 -+ or width < 3 -+ or height % 3 -+ or width % 3 -+ or target_start + frames * rows != key.shape[1] -+ or query.shape[1] < key.shape[1] -+ or scale <= 0 -+ ): -+ raise ValueError("H3 pooled context requires a final video grid divisible by three") -+ heads, dim = query.shape[2:] -+ -+ def pack(tensor: torch.Tensor) -> torch.Tensor: -+ pooled = ( -+ tensor[:, target_start:] -+ .reshape(frames, height // 3, 3, width // 3, 3, heads, dim) -+ .float() -+ .mean(dim=(2, 4)) -+ .to(tensor.dtype) -+ .reshape(1, frames * rows // 9, heads, dim) -+ ) -+ return torch.cat((tensor[:, :target_start], pooled), dim=1) -+ -+ output = torch.zeros_like(query) -+ if target_start: -+ output[:, :target_start] = _prefix_bnsd(query[:, :target_start], key, value, scale) -+ video_query = query[:, target_start : key.shape[1]] -+ grouped_key = key[:, target_start:].reshape(frames, height // 3, 3, width // 3, 3, heads, dim).float() -+ variance = grouped_key.var(dim=(2, 4), unbiased=False).mean(dim=-1).reshape(1, frames * rows // 9, heads) -+ query_norm_sq = video_query.float().square().sum(dim=-1) -+ padded_q = torch.nn.functional.pad(video_query, (0, 16)) -+ padded_k = torch.nn.functional.pad(pack(key), (0, 16)) -+ padded_v = torch.nn.functional.pad(pack(value), (0, 16)) -+ padded_q[..., dim] = 1 -+ padded_k[:, target_start:, :, dim] = math.log(9) / scale -+ # Isotropic second-moment approximation to the pooled log-partition. -+ padded_q[..., dim + 1] = 0.5 * scale * query_norm_sq -+ padded_k[:, target_start:, :, dim + 1] = variance -+ attended = _prefix_bnsd(padded_q, padded_k, padded_v, scale)[..., :dim] -+ output[:, target_start : key.shape[1]] = attended -+ return output -+ -+ -+def _use_video_variance_pool() -> bool: -+ if os.environ.get("H3_VIDEO_VARIANCE_POOL") != "3x3-variance": -+ raise ValueError("The experimental H3 pooled-context profile requires 3x3-variance") -+ context = get_forward_context() -+ step, total = context.denoise_step_idx, context.total_denoise_steps -+ if total != 50 or step is None or not 0 <= step < 50: -+ raise ValueError("The experimental H3 pooled-context profile requires published 50-step progress") -+ return step not in (0, 49) -+ -+ - class MiniMaxH3FlashAttentionImpl(FlashAttentionImpl): - """Use independent head batches for explicit long BF16 prefix attention. - -@@ -51,7 +125,10 @@ class MiniMaxH3FlashAttentionImpl(FlashAttentionImpl): - value: torch.Tensor, - attn_metadata: AttentionMetadata | None = None, - ) -> torch.Tensor: -+ requested = os.environ.get("H3_VIDEO_VARIANCE_POOL") is not None and query.shape[1] >= 4096 - if os.environ.get("MINDIE_SD_FA_TYPE") != "fused_attn_score" or attn_metadata is None: -+ if requested: -+ raise ValueError("H3 pooled context requires explicit fused attention and packed metadata") - return super().forward_fa_npu(query, key, value, attn_metadata) - extra = attn_metadata.extra - if ( -@@ -68,9 +145,30 @@ class MiniMaxH3FlashAttentionImpl(FlashAttentionImpl): - or query.shape[2:] != value.shape[2:] - or key.shape[1] != value.shape[1] - ): -+ if requested: -+ raise ValueError("H3 pooled context received unsupported packed fused-attention inputs") - return super().forward_fa_npu(query, key, value, attn_metadata) - resolved = self._resolve_packed_seq_npu(query, key, extra) - if resolved is None: -+ if requested: -+ raise ValueError("H3 pooled context requires a resolved packed prefix") - return super().forward_fa_npu(query, key, value, attn_metadata) - used_k = resolved[1][0] -+ if os.environ.get("H3_VIDEO_VARIANCE_POOL") is not None and _use_video_variance_pool(): -+ layout = attn_metadata.video_layout -+ targets = [] if layout is None else [span for span in layout.video_spans if span.role == "target"] -+ if len(targets) != 1: -+ raise ValueError("H3 pooled context requires exactly one target-video span") -+ target = targets[0] -+ frames, height, width = target.latent_grid -+ return _video_variance_pool_bnsd( -+ query, -+ key[:, :used_k], -+ value[:, :used_k], -+ self.softmax_scale, -+ target_start=target.start, -+ frames=frames, -+ height=height, -+ width=width, -+ ) - return _prefix_bnsd(query, key[:, :used_k], value[:, :used_k], self.softmax_scale) diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c20-source-identity.json b/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c20-source-identity.json deleted file mode 100644 index 19b78b6bc..000000000 --- a/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c20-source-identity.json +++ /dev/null @@ -1,12 +0,0 @@ -{ - "head": "bba4f7e8802497772f46628b6f408391add84679", - "tree": "5a7ed678aa86cf2555a80f64a5f21ee9bda540fe", - "base": "e89b1d40d37f7f72767892a56fa7980ea572ba8e", - "branch": "rsi/h3-297-video-variance-pool", - "files": { - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "4390450bc5324f7028125101cd4bb32e9de237b04bc12fa879b48110d7122e3c", - "tests/diffusion/models/minimax_h3/test_minimax_h3_variance_pool.py": "e871fb4b222d61d10b6c442a333a52c976cceaa0d406233502ecbcaddfeba8c9" - }, - "published": false, - "patch_base": "e89b1d40d37f7f72767892a56fa7980ea572ba8e" -} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c21-candidate.patch b/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c21-candidate.patch deleted file mode 100644 index 017fbf116..000000000 --- a/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c21-candidate.patch +++ /dev/null @@ -1,2041 +0,0 @@ -diff --git a/tests/diffusion/distributed/test_collective_gate.py b/tests/diffusion/distributed/test_collective_gate.py -new file mode 100644 -index 0000000..e66fdae ---- /dev/null -+++ b/tests/diffusion/distributed/test_collective_gate.py -@@ -0,0 +1,205 @@ -+# SPDX-License-Identifier: Apache-2.0 -+# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project -+ -+"""Unit tests for the DLO/QKV-AllToAll ordering gate. -+ -+Covers the barrier rendezvous and the hook's deferred-AllGather state machine -+that keeps the background weight gather from taking the links ahead of the -+block's foreground AllToAll. -+""" -+ -+from unittest.mock import Mock -+ -+import pytest -+import torch -+from torch import nn -+ -+import vllm_omni.diffusion.offloader.distributed_layerwise_backend as dist_backend_module -+from tests.diffusion.offloader.helpers import DummyStream, patch_offload_runtime -+from vllm_omni.diffusion.distributed.collective_gate import QKVAllToAllBarrier -+from vllm_omni.diffusion.offloader.distributed_layerwise_backend import ( -+ DistributedLayerwiseOffloadHook, -+ attach_collective_gate, -+) -+ -+pytestmark = [pytest.mark.diffusion, pytest.mark.cpu, pytest.mark.core_model] -+ -+ -+@pytest.fixture -+def patched_offload_runtime(monkeypatch): -+ patch_offload_runtime(monkeypatch, dist_backend_module.current_omni_platform, synchronize=True) -+ -+ -+class TestQKVAllToAllBarrier: -+ def test_signal_without_arm_is_a_noop(self): -+ on_signal = Mock() -+ barrier = QKVAllToAllBarrier(on_signal) -+ -+ barrier.signal() -+ -+ on_signal.assert_not_called() -+ assert not barrier.armed -+ -+ def test_arm_then_signal_fires_once(self): -+ on_signal = Mock() -+ barrier = QKVAllToAllBarrier(on_signal) -+ -+ barrier.arm() -+ assert barrier.armed -+ barrier.signal() -+ -+ on_signal.assert_called_once_with() -+ assert not barrier.armed -+ -+ # A second signal in the same forward must not re-issue the gather. -+ barrier.signal() -+ on_signal.assert_called_once_with() -+ -+ def test_disarm_suppresses_a_pending_signal(self): -+ on_signal = Mock() -+ barrier = QKVAllToAllBarrier(on_signal) -+ -+ barrier.arm() -+ barrier.disarm() -+ barrier.signal() -+ -+ on_signal.assert_not_called() -+ -+ -+def _gated_hook() -> DistributedLayerwiseOffloadHook: -+ """A hook wired for the gated path, with hand-set shard buffers.""" -+ hook = DistributedLayerwiseOffloadHook( -+ next_block=nn.Linear(2, 2, bias=False), -+ device=torch.device("cpu"), -+ dp_group=object(), -+ dp_size=2, -+ rank=0, -+ copy_stream=DummyStream(), -+ comm_stream=DummyStream(), -+ pin_memory=False, -+ order_allgather_after_attn_a2a=True, -+ ) -+ device_buf = {torch.float32: torch.zeros(4)} -+ shard_buf = {torch.float32: torch.zeros(2)} -+ hook.gpu_buffers = [dict(device_buf), dict(device_buf)] -+ hook.gpu_shard_buffers = [dict(shard_buf), dict(shard_buf)] -+ hook.cpu_shards = {torch.float32: torch.zeros(2)} -+ hook._ag_output_sizes = {torch.float32: 4} -+ hook.metadata = {} -+ hook._cached_repoint = () -+ hook.rank_local_mmap = False -+ return hook -+ -+ -+class TestDeferredAllGather: -+ def test_prefetch_defers_the_gather_until_the_barrier_signals(self, patched_offload_runtime, monkeypatch): -+ hook = _gated_hook() -+ gather = Mock() -+ monkeypatch.setattr(torch.distributed, "all_gather_into_tensor", gather) -+ -+ hook.prefetch_layer(0, non_blocking=True) -+ -+ # H2D ran; the collective is held back for the block's AllToAll. -+ gather.assert_not_called() -+ assert hook._a2a_barrier.armed -+ assert hook._deferred_allgather is not None -+ # The ready event must still be published for the consumer. -+ assert hook.ready_events[0] is hook._prefetch_done -+ -+ hook._a2a_barrier.signal() -+ -+ gather.assert_called_once() -+ assert hook._deferred_allgather is None -+ assert not hook._a2a_barrier.armed -+ -+ def test_post_forward_flush_issues_an_unsignalled_gather(self, patched_offload_runtime, monkeypatch): -+ """A block whose attention never ran must still get its weights.""" -+ hook = _gated_hook() -+ gather = Mock() -+ monkeypatch.setattr(torch.distributed, "all_gather_into_tensor", gather) -+ -+ hook.prefetch_layer(0, non_blocking=True) -+ gather.assert_not_called() -+ -+ # pre_forward ran, attention did not: the pairing flush must fire. -+ hook.post_forward(hook.next_block, None) -+ -+ gather.assert_called_once() -+ assert hook._deferred_allgather is None -+ assert not hook._a2a_barrier.armed -+ -+ def test_sync_prefetch_issues_immediately_without_arming(self, patched_offload_runtime, monkeypatch): -+ """The cache-dit sync fallback has no attention signal to wait for.""" -+ hook = _gated_hook() -+ gather = Mock() -+ monkeypatch.setattr(torch.distributed, "all_gather_into_tensor", gather) -+ -+ hook.prefetch_layer(0, non_blocking=False) -+ -+ gather.assert_called_once() -+ assert not hook._a2a_barrier.armed -+ assert hook._deferred_allgather is None -+ -+ def test_disabled_by_default(self, patched_offload_runtime, monkeypatch): -+ hook = DistributedLayerwiseOffloadHook( -+ next_block=nn.Linear(2, 2, bias=False), -+ device=torch.device("cpu"), -+ dp_group=object(), -+ dp_size=2, -+ rank=0, -+ copy_stream=DummyStream(), -+ comm_stream=DummyStream(), -+ pin_memory=False, -+ ) -+ hook.gpu_buffers = [{torch.float32: torch.zeros(4)}] * 2 -+ hook.gpu_shard_buffers = [{torch.float32: torch.zeros(2)}] * 2 -+ hook.cpu_shards = {torch.float32: torch.zeros(2)} -+ hook._ag_output_sizes = {torch.float32: 4} -+ hook.metadata = {} -+ hook._cached_repoint = () -+ gather = Mock() -+ monkeypatch.setattr(torch.distributed, "all_gather_into_tensor", gather) -+ -+ hook.prefetch_layer(0, non_blocking=True) -+ -+ assert hook._a2a_barrier is None -+ gather.assert_called_once() -+ -+ -+class TestAttachCollectiveGate: -+ def test_block_without_attention_reports_zero(self): -+ hook = _gated_hook() -+ -+ assert attach_collective_gate(nn.Linear(2, 2), hook) == 0 -+ -+ -+@pytest.mark.parametrize("flush", [False, True]) -+def test_chunked_gate_defers_copies_and_reuses_bounded_slots(patched_offload_runtime, monkeypatch, flush): -+ from types import SimpleNamespace -+ -+ hook = _gated_hook() -+ chunks = tuple( -+ SimpleNamespace(chunk_id=i, local_numel=1, cpu_offset=i, full_offset=2 * i, padded_numel=2) for i in range(3) -+ ) -+ hook.manifest = SimpleNamespace(dtypes=(SimpleNamespace(dtype=torch.float32, chunks=chunks),)) -+ hook.cpu_shards = {torch.float32: torch.tensor([11.0, 22.0, 33.0])} -+ hook.gpu_buffers = [{torch.float32: torch.zeros(6)} for _ in range(2)] -+ calls = [] -+ -+ def gather(output, local, **kwargs): -+ calls.append(local.clone()) -+ output.copy_(local.repeat(2)) -+ -+ monkeypatch.setattr(torch.distributed, "all_gather_into_tensor", gather) -+ hook.prefetch_layer(0, non_blocking=True) -+ assert not calls and hook._deferred_chunked_allgather is not None -+ assert torch.equal(hook.gpu_shard_buffers[0][torch.float32], torch.zeros(2)) -+ if flush: -+ hook._flush_deferred_allgather() -+ else: -+ hook._a2a_barrier.signal() -+ assert len(calls) == 3 and hook._deferred_chunked_allgather is None -+ assert not hook._a2a_barrier.armed -+ assert torch.equal(hook.gpu_buffers[0][torch.float32], torch.tensor([11.0, 11.0, 22.0, 22.0, 33.0, 33.0])) -+ hook._a2a_barrier.signal() -+ assert len(calls) == 3 -diff --git a/tests/diffusion/models/minimax_h3/test_minimax_h3_pre_gate_cache.py b/tests/diffusion/models/minimax_h3/test_minimax_h3_pre_gate_cache.py -new file mode 100644 -index 0000000..4e9979e ---- /dev/null -+++ b/tests/diffusion/models/minimax_h3/test_minimax_h3_pre_gate_cache.py -@@ -0,0 +1,198 @@ -+# SPDX-License-Identifier: Apache-2.0 -+# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project -+ -+from datetime import timedelta -+ -+import pytest -+import torch -+import torch.distributed as dist -+import torch.nn as nn -+ -+from tests.diffusion.models.minimax_h3.test_minimax_h3_video_row_cache import tiny_block -+from tests.helpers.mark import hardware_marks -+from vllm_omni.diffusion.layers.indexed_modulation import indexed_gate -+from vllm_omni.diffusion.models.minimax_h3.attention.flash_h3 import _prefix_bnsd, _subset_prefix_bnsd -+from vllm_omni.diffusion.models.minimax_h3.video_row_cache import MiniMaxH3VideoRowCache -+from vllm_omni.platforms import current_omni_platform -+ -+pytestmark = [pytest.mark.core_model, pytest.mark.diffusion, *hardware_marks(res={"npu": "A3"}, num_cards=4)] -+ -+ -+def make_state(start=0, length=4): -+ state = MiniMaxH3VideoRowCache(torch.tensor([1, 2]), seq_len=4, used_len=3, num_layers=1, pre_gate=True) -+ state.prepare_local_span(start, length) -+ return state -+ -+ -+def test_fixed_pre_gate_schedule_storage_and_request_cleanup(): -+ state = make_state() -+ x = torch.ones(4, 2, dtype=torch.bfloat16) -+ full = [] -+ for step in range(50): -+ state.begin_step(step) -+ if not state.partial: -+ full.append(step) -+ for stage in ("attn", "mlp"): -+ out = state.pre_gate_output(0, x, stage) -+ assert out.dtype == torch.bfloat16 -+ if state.partial: -+ state.record_fresh_gate(stage) -+ state.end_step() -+ assert full == [0, 1, 4, 9, 19, 29, 39, 49] -+ assert state.full_blocks == 8 and state.partial_blocks == 42 -+ assert state.fresh_attention_gates == state.fresh_mlp_gates == 42 -+ assert all(t.dtype == torch.bfloat16 for t in [*state._attention_outputs.values(), *state._mlp_outputs.values()]) -+ assert state.stats()["live_cache_bytes"] == 16 -+ state.close() -+ assert state.stats()["live_cache_bytes"] == 0 and state.closed -+ -+ -+@pytest.mark.parametrize("failure", ["missing_mlp", "mlp_first", "gated_capture", "gated_restore"]) -+def test_pre_gate_stage_order_cannot_silently_use_block_residual(failure): -+ state = make_state() -+ x = torch.ones(4, 2, dtype=torch.bfloat16) -+ state.begin_step(0) -+ with pytest.raises(RuntimeError): -+ if failure == "missing_mlp": -+ state.pre_gate_output(0, x, "attn") -+ state.end_step() -+ elif failure == "mlp_first": -+ state.pre_gate_output(0, x, "mlp") -+ elif failure == "gated_capture": -+ state.capture(0, x, x) -+ else: -+ state.restore(0, x) -+ state.close() -+ -+ -+class ConstantMLP(nn.Module): -+ def __init__(self): -+ super().__init__() -+ self.rows = [] -+ -+ def forward(self, x): -+ self.rows.append(x.shape[0]) -+ return torch.full_like(x, 3) -+ -+ -+@pytest.mark.parametrize("start,length", [(0, 4), (1, 2)]) -+def test_real_block_reapplies_changed_gates_and_preserves_empty_dense_shards(start, length): -+ state = make_state(start, length) -+ block = tiny_block() -+ gates = [1.0, 1.0] -+ block.adaln_proj = lambda t: ( -+ torch.zeros(3, 2), -+ torch.zeros(3, 2), -+ torch.full((3, 2), gates[0]), -+ torch.zeros(3, 2), -+ torch.zeros(3, 2), -+ torch.full((3, 2), gates[1]), -+ ) -+ block.attn = lambda x, **kwargs: torch.full_like(x, 2) -+ block.mlp = ConstantMLP() -+ x = torch.ones(length, 2, dtype=torch.bfloat16) -+ kwargs = dict( -+ t_emb=torch.zeros(1), -+ combined_indices=torch.zeros(length, dtype=torch.long), -+ rope_table=None, -+ cu_seqlens=torch.tensor([0, 3, 4], dtype=torch.int32), -+ max_seqlen=3, -+ packed_total=4, -+ ) -+ for step in (0, 1): -+ state.begin_step(step) -+ expected = block(x, **kwargs) -+ actual = block(x, video_row_cache=state, **kwargs) -+ assert torch.equal(actual.view(torch.uint8), expected.view(torch.uint8)) -+ state.end_step() -+ gates[:] = [0.5, -0.25] -+ current = x * 2 -+ saved = current.clone() -+ expected = block(current, **kwargs) -+ block.mlp.rows.clear() -+ state.begin_step(2) -+ actual = block(current, video_row_cache=state, **kwargs) -+ state.end_step() -+ rows = state.local_dense_positions -+ assert block.mlp.rows == ([rows.numel()] if rows.numel() else []) -+ assert torch.equal( -+ actual[: min(length, 3 - start)].view(torch.uint8), expected[: min(length, 3 - start)].view(torch.uint8) -+ ) -+ assert torch.equal( -+ actual.index_select(0, state.local_video_positions), torch.full((state.local_video_positions.numel(), 2), 2.25) -+ ) -+ assert not torch.equal( -+ actual.index_select(0, state.local_video_positions), current.index_select(0, state.local_video_positions) + 5 -+ ) -+ assert torch.equal(current, saved) -+ assert state.fresh_attention_gates == state.fresh_mlp_gates == 1 -+ state.close() -+ -+ -+def _native_pre_gate_worker(rank): -+ import torch_npu # noqa: F401 -+ -+ torch.npu.set_device(rank) -+ dist.init_process_group( -+ "hccl", init_method="tcp://127.0.0.1:29597", rank=rank, world_size=4, timeout=timedelta(seconds=120) -+ ) -+ try: -+ torch.manual_seed(219) -+ full = [torch.randn(32, 4, 128, dtype=torch.bfloat16).npu() for _ in range(3)] -+ exchanged = [] -+ for tensor in full: -+ local = tensor[rank * 8 : (rank + 1) * 8] -+ send = local.reshape(8, 4, 1, 128).permute(1, 0, 2, 3).contiguous() -+ receive = torch.empty_like(send) -+ dist.all_to_all_single(receive, send) -+ exchanged.append(receive.reshape(1, 32, 1, 128)) -+ state = MiniMaxH3VideoRowCache( -+ torch.arange(8, 24, device="npu"), seq_len=32, used_len=30, num_layers=1, pre_gate=True -+ ) -+ state.prepare_local_span(rank * 8, 8) -+ x = torch.full((8, 512), 2, dtype=torch.bfloat16, device="npu") -+ for step in (0, 1): -+ state.begin_step(step) -+ state.pre_gate_output(0, torch.full_like(x, 0.5), "attn") -+ state.pre_gate_output(0, torch.full_like(x, 0.25), "mlp") -+ state.end_step() -+ state.begin_step(2) -+ out = _subset_prefix_bnsd(exchanged[0], exchanged[1][:, :30], exchanged[2][:, :30], 128**-0.5, state) -+ send, receive = out.reshape(4, 8, 1, 128).contiguous(), torch.empty_like(out.reshape(4, 8, 1, 128)) -+ dist.all_to_all_single(receive, send) -+ local_out = receive.permute(1, 0, 2, 3).reshape(8, 512) -+ expected = _prefix_bnsd(full[0].unsqueeze(0), full[1][:30].unsqueeze(0), full[2][:30].unsqueeze(0), 128**-0.5)[ -+ 0 -+ ].reshape(32, 512) -+ dense = state.local_dense_positions -+ torch.testing.assert_close( -+ local_out.index_select(0, dense).float(), -+ expected[rank * 8 : (rank + 1) * 8].index_select(0, dense).float(), -+ atol=0.03125, -+ rtol=0.02, -+ ) -+ attn = state.pre_gate_output(0, local_out, "attn") -+ indices = torch.zeros(8, dtype=torch.long, device="npu") -+ gate = torch.full((1, 512), 0.5, dtype=torch.float32, device="npu") -+ after_attn = indexed_gate(x, gate, attn, indices) -+ state.record_fresh_gate("attn") -+ mlp = state.pre_gate_output(0, torch.zeros_like(x), "mlp") -+ result = indexed_gate(after_attn, gate, mlp, indices) -+ state.record_fresh_gate("mlp") -+ video = state.local_video_positions -+ assert torch.equal( -+ result.index_select(0, video), torch.full((video.numel(), 512), 2.375, dtype=torch.bfloat16, device="npu") -+ ) -+ assert dense.numel() == [8, 0, 0, 6][rank] -+ state.end_step() -+ state.close() -+ assert state.stats()["live_cache_bytes"] == 0 -+ torch.npu.synchronize() -+ dist.barrier() -+ finally: -+ dist.destroy_process_group() -+ -+ -+@pytest.mark.skipif(not current_omni_platform.is_npu(), reason="Requires four native Ascend devices") -+def test_native_four_rank_pre_gate_subset_attention_and_current_gates(): -+ torch.multiprocessing.spawn(_native_pre_gate_worker, nprocs=4, join=True) -diff --git a/tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py b/tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py -new file mode 100644 -index 0000000..94ff958 ---- /dev/null -+++ b/tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py -@@ -0,0 +1,406 @@ -+# SPDX-License-Identifier: Apache-2.0 -+# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project -+ -+from datetime import timedelta -+from types import SimpleNamespace -+ -+import pytest -+import torch -+import torch.distributed as dist -+import torch.nn as nn -+ -+from tests.helpers.mark import hardware_marks -+from vllm_omni.diffusion.models.minimax_h3 import minimax_h3_blocks as blocks -+from vllm_omni.diffusion.models.minimax_h3.attention.flash_h3 import _prefix_bnsd, _subset_prefix_bnsd -+from vllm_omni.diffusion.models.minimax_h3.video_row_cache import MiniMaxH3VideoRowCache -+from vllm_omni.platforms import current_omni_platform -+ -+pytestmark = [pytest.mark.core_model, pytest.mark.diffusion, *hardware_marks(res={"npu": "A3"}, num_cards=4)] -+ -+ -+def state_for_rows(device="cpu", start=0, length=4): -+ state = MiniMaxH3VideoRowCache(torch.tensor([1, 2], device=device), seq_len=4, used_len=3, num_layers=1) -+ state.prepare_local_span(start, length) -+ return state -+ -+ -+def anchor(state, x): -+ for step in range(5): -+ state.begin_step(step) -+ state.capture(0, x, x + 0.125) -+ state.end_step() -+ state.begin_step(5) -+ -+ -+def test_fixed_schedule_and_row_ownership_excludes_padding(): -+ assert [i for i in range(50) if MiniMaxH3VideoRowCache.is_full_step(i)] == [ -+ 0, -+ 1, -+ 2, -+ 3, -+ 4, -+ 9, -+ 14, -+ 19, -+ 24, -+ 29, -+ 34, -+ 39, -+ 44, -+ 46, -+ 47, -+ 48, -+ 49, -+ ] -+ state = state_for_rows() -+ assert state.global_dense_positions.tolist() == [0] -+ assert state.local_dense_positions.tolist() == [0] -+ assert state.local_video_positions.tolist() == [1, 2] -+ with pytest.raises(RuntimeError, match="ownership"): -+ state.prepare_local_span(0, 3) -+ -+ -+@pytest.mark.parametrize("positions", [[1, 1], [2, 1], [-1], [3]]) -+def test_invalid_target_positions_are_rejected(positions): -+ with pytest.raises(ValueError): -+ MiniMaxH3VideoRowCache(torch.tensor(positions), seq_len=4, used_len=3, num_layers=1) -+ -+ -+def test_delta_is_fp32_and_restore_owns_output(): -+ state = state_for_rows() -+ x = torch.tensor([[1, 2], [3, 4], [5, 6], [7, 8]], dtype=torch.bfloat16) -+ anchor(state, x) -+ assert state._deltas[0].dtype == torch.float32 -+ before = x.clone() -+ output = state.restore(0, x) -+ assert torch.equal(output[1:3], x[1:3] + 0.125) -+ assert torch.equal(output[[0, 3]], x[[0, 3]]) -+ output.zero_() -+ assert torch.equal(x, before) -+ state.end_step() -+ state.close() -+ assert state.closed and not state._deltas and state.global_dense_positions.numel() == 0 -+ -+ -+def large_delta_roundtrip(device="cpu"): -+ state = state_for_rows(device=device) -+ x = torch.full((4, 2), 100000, dtype=torch.bfloat16, device=device) -+ expected = -x -+ delta = expected.float() - x.float() -+ assert torch.isfinite(delta).all() -+ assert not torch.isfinite(delta.to(torch.float16)).all() -+ for step in range(5): -+ state.begin_step(step) -+ state.capture(0, x, expected) -+ state.end_step() -+ state.begin_step(5) -+ actual = state.restore(0, x) -+ assert torch.isfinite(actual).all() -+ assert torch.equal(actual[1:3].cpu().view(torch.uint8), expected[1:3].cpu().view(torch.uint8)) -+ assert torch.equal(actual[[0, 3]], x[[0, 3]]) -+ state.end_step() -+ state.close() -+ assert state.stats()["live_cache_bytes"] == 0 -+ -+ -+def test_large_finite_bf16_delta_roundtrip_exceeds_fp16_storage_range(): -+ large_delta_roundtrip() -+ -+ -+class CountMLP(nn.Module): -+ def __init__(self): -+ super().__init__() -+ self.rows = [] -+ -+ def forward(self, x): -+ self.rows.append(x.shape[0]) -+ return x + 0.25 -+ -+ -+def tiny_block(): -+ block = object.__new__(blocks.MiniMaxH3DiTBlock) -+ nn.Module.__init__(block) -+ block.layer_index = 0 -+ block.norm1 = SimpleNamespace(weight=torch.ones(2), variance_epsilon=1e-6) -+ block.norm2 = SimpleNamespace(weight=torch.ones(2), variance_epsilon=1e-6) -+ block.adaln_proj = lambda t: ( -+ torch.zeros(3, 2), -+ torch.zeros(3, 2), -+ torch.ones(3, 2), -+ torch.zeros(3, 2), -+ torch.zeros(3, 2), -+ torch.ones(3, 2), -+ ) -+ block.attn = lambda x, **kwargs: x * 0.5 -+ block.mlp = CountMLP() -+ return block -+ -+ -+@pytest.mark.parametrize("start,length", [(0, 4), (1, 2)]) -+def test_real_block_dense_path_and_partial_mlp_empty_rows(start, length): -+ state = state_for_rows(start=start, length=length) -+ block = tiny_block() -+ x = torch.arange(length * 2).reshape(length, 2).to(torch.bfloat16) -+ before = x.clone() -+ kwargs = dict( -+ t_emb=torch.zeros(1), -+ combined_indices=torch.zeros(length, dtype=torch.long), -+ rope_table=None, -+ cu_seqlens=torch.tensor([0, 3, 4], dtype=torch.int32), -+ max_seqlen=3, -+ packed_total=4, -+ ) -+ expected = block(x, **kwargs) -+ for step in range(5): -+ state.begin_step(step) -+ actual = block(x, video_row_cache=state, **kwargs) -+ assert torch.equal(actual.view(torch.uint8), expected.view(torch.uint8)) -+ state.end_step() -+ block.mlp.rows.clear() -+ state.begin_step(5) -+ actual = block(x, video_row_cache=state, **kwargs) -+ state.end_step() -+ rows = state.local_dense_positions -+ assert block.mlp.rows == ([rows.numel()] if rows.numel() else []) -+ assert torch.equal(actual.index_select(0, rows), expected.index_select(0, rows)) -+ assert torch.equal(x, before) -+ -+ -+@pytest.mark.parametrize("failure", ["success", "model", "cancel"]) -+def test_request_state_closed_and_removed_on_every_exit(monkeypatch, failure): -+ from vllm_omni.diffusion.models.minimax_h3 import denoise_loop as loop -+ -+ monkeypatch.setenv("H3_VIDEO_ROW_CACHE", "1") -+ monkeypatch.setattr(loop.current_omni_platform, "is_npu", lambda: False) -+ monkeypatch.setattr(loop, "minimax_h3_publish_denoise_progress", lambda *args: None) -+ state = MiniMaxH3VideoRowCache(torch.tensor([1]), seq_len=4, used_len=3, num_layers=1) -+ -+ class Model: -+ def create_video_row_cache(self, *args, **kwargs): -+ return state -+ -+ def __call__(self, **kwargs): -+ assert kwargs["video_row_cache"] is state -+ if failure == "model": -+ raise RuntimeError("model failed") -+ state.prepare_local_span(0, 4) -+ x = torch.ones(4, 2, dtype=torch.bfloat16) -+ if state.partial: -+ state.restore(0, x) -+ else: -+ state.capture(0, x, x + 0.125) -+ return torch.zeros(2, 96), torch.zeros(1, 32) -+ -+ class Branch: -+ task = "ref2va" -+ seq_len = 4 -+ used_len = 3 -+ img_pos = torch.tensor([0, 1]) -+ update_mask = torch.tensor([False, True]) -+ update_mask_dev = update_mask -+ audio_update_mask_dev = torch.tensor([True]) -+ locked_audio_rows = None -+ static_kwargs = {"video_token_layout": SimpleNamespace(video_spans=[SimpleNamespace(role="target", start=1)])} -+ -+ def forward_kwargs(self, **kwargs): -+ return dict(self.static_kwargs) -+ -+ monkeypatch.setattr( -+ loop, -+ "minimax_h3_prepare_denoise_rows", -+ lambda **kwargs: (kwargs["initial_video_rows"], kwargs["initial_audio_rows"], torch.zeros(1, 96), None), -+ ) -+ -+ def cancellation(**kwargs): -+ if failure == "cancel": -+ raise RuntimeError("request cancelled") -+ -+ monkeypatch.setattr(loop, "check_request_cancellation", cancellation) -+ branch, model = Branch(), Model() -+ kwargs = dict( -+ model=model, -+ positive=branch, -+ initial_video_rows=torch.zeros(2, 96), -+ initial_audio_rows=torch.zeros(1, 32), -+ keyframe_cond_rows=torch.zeros(1, 96), -+ sigmas_video=[1 - i / 50 for i in range(51)], -+ sigmas_audio=[1 - i / 50 for i in range(51)], -+ device=torch.device("cpu"), -+ ) -+ if failure == "success": -+ loop.minimax_h3_denoise_loop(**kwargs) -+ assert state.full_steps == 17 and state.partial_steps == 33 -+ else: -+ with pytest.raises(RuntimeError): -+ loop.minimax_h3_denoise_loop(**kwargs) -+ assert state.closed and not state._deltas and "video_row_cache" not in branch.static_kwargs -+ assert model._video_row_cache_stats["closed"] -+ -+ -+def test_default_factory_does_not_create_state(monkeypatch): -+ from vllm_omni.diffusion.models.minimax_h3.minimax_h3_transformer import MiniMaxH3DiTModel -+ -+ monkeypatch.delenv("H3_VIDEO_ROW_CACHE", raising=False) -+ assert ( -+ MiniMaxH3DiTModel.create_video_row_cache(None, torch.tensor([1]), seq_len=4, used_len=3, num_steps=50) is None -+ ) -+ -+ -+@pytest.mark.parametrize("ref2va", [False, True]) -+def test_real_packing_task_selects_ref2va_and_rejects_fl2va(monkeypatch, ref2va): -+ from vllm_omni.diffusion.models.minimax_h3 import denoise_loop as loop -+ from vllm_omni.diffusion.models.minimax_h3.packed_sequence import ( -+ minimax_h3_packed_sequence, -+ minimax_h3_packed_sequence_ref2va_blocks, -+ ) -+ -+ common = dict(text_len=2, latent_t=2, latent_h=4, latent_w=4, audio_t=2) -+ packed = ( -+ minimax_h3_packed_sequence_ref2va_blocks(**common, ref_blocks=[{"kind": "image", "latent_h": 4, "latent_w": 4}]) -+ if ref2va -+ else minimax_h3_packed_sequence(**common, include_keyframe_cond=True, keyframe_frame_indices=[0], frame_count=5) -+ ) -+ branch = loop.MiniMaxH3DenoiseBranch( -+ packed=packed, -+ text_embeddings=torch.zeros(2, 5120), -+ token_tags=packed["token_tags"], -+ device=torch.device("cpu"), -+ task="ref2va" if ref2va else "fl2va", -+ ) -+ layout = branch.static_kwargs["video_token_layout"] -+ assert not any(span.role == "reference" for span in layout.video_spans) -+ monkeypatch.setenv("H3_VIDEO_ROW_CACHE", "1") -+ monkeypatch.setattr(loop.current_omni_platform, "is_npu", lambda: False) -+ monkeypatch.setattr(loop, "minimax_h3_publish_denoise_progress", lambda *args: None) -+ monkeypatch.setattr(loop, "check_request_cancellation", lambda **kwargs: None) -+ states = [] -+ -+ class Model: -+ def create_video_row_cache(self, positions, **kwargs): -+ state = MiniMaxH3VideoRowCache( -+ positions, seq_len=kwargs["seq_len"], used_len=kwargs["used_len"], num_layers=1 -+ ) -+ state.prepare_local_span(0, branch.seq_len) -+ states.append(state) -+ return state -+ -+ def __call__(self, **kwargs): -+ state = kwargs["video_row_cache"] -+ x = torch.ones(branch.seq_len, 2, dtype=torch.bfloat16) -+ if state.partial: -+ state.restore(0, x) -+ else: -+ state.capture(0, x, x + 0.125) -+ return torch.zeros(branch.img_pos.numel(), 96), torch.zeros(branch.audio_pos.numel(), 32) -+ -+ kwargs = dict( -+ model=Model(), -+ positive=branch, -+ initial_video_rows=torch.zeros(branch.img_pos.numel(), 96), -+ initial_audio_rows=torch.zeros(branch.audio_pos.numel(), 32), -+ keyframe_cond_rows=torch.zeros(int((~branch.update_mask).sum()), 96), -+ sigmas_video=[1 - i / 50 for i in range(51)], -+ sigmas_audio=[1 - i / 50 for i in range(51)], -+ device=torch.device("cpu"), -+ ) -+ if ref2va: -+ loop.minimax_h3_denoise_loop(**kwargs) -+ assert len(states) == 1 and states[0].closed -+ assert states[0].full_steps == 17 and states[0].partial_steps == 33 -+ else: -+ with pytest.raises(ValueError, match="requires Ref2VA"): -+ loop.minimax_h3_denoise_loop(**kwargs) -+ assert not states -+ -+ -+@pytest.mark.parametrize("start,length", [(0, 4), (1, 2)]) -+def test_attention_keeps_collective_path_and_projects_only_dense_local_rows(start, length): -+ state = state_for_rows(start=start, length=length) -+ x = torch.ones(length, 2, dtype=torch.bfloat16) -+ anchor(state, x) -+ attention = object.__new__(blocks.MiniMaxH3Attention) -+ nn.Module.__init__(attention) -+ attention.num_heads = 1 -+ attention.head_dim = 2 -+ attention.vdn = None -+ attention.to_gate_compress = None -+ calls, projected_rows = [], [] -+ attention._project_qkv = lambda *args: (x.reshape(length, 1, 2),) * 5 -+ -+ def packed(q, k, v, **kwargs): -+ calls.append(q.shape[0]) -+ assert kwargs["video_row_cache"] is state -+ return q -+ -+ def project(rows): -+ projected_rows.append(rows.shape[0]) -+ return rows * 2, None -+ -+ attention._run_packed_attention = packed -+ attention.out_proj = project -+ output = attention( -+ x, -+ rope_table=None, -+ cu_seqlens=torch.tensor([0, 3, 4], dtype=torch.int32), -+ max_seqlen=3, -+ packed_total=4, -+ video_row_cache=state, -+ ) -+ rows = state.local_dense_positions -+ assert calls == [length] -+ assert projected_rows == ([rows.numel()] if rows.numel() else []) -+ assert torch.equal(output.index_select(0, rows), (x * 2).index_select(0, rows)) -+ assert torch.count_nonzero(output.index_select(0, state.local_video_positions)) == 0 -+ -+ -+def _native_worker(rank): -+ import torch_npu # noqa: F401 -+ -+ torch.npu.set_device(rank) -+ dist.init_process_group( -+ "hccl", init_method="tcp://127.0.0.1:29597", rank=rank, world_size=4, timeout=timedelta(seconds=120) -+ ) -+ try: -+ large_delta_roundtrip(device=f"npu:{rank}") -+ torch.manual_seed(19) -+ full = [torch.randn(32, 4, 128, dtype=torch.bfloat16).npu() for _ in range(3)] -+ exchanged = [] -+ for tensor in full: -+ local = tensor[rank * 8 : (rank + 1) * 8] -+ send = local.reshape(8, 4, 1, 128).permute(1, 0, 2, 3).contiguous() -+ receive = torch.empty_like(send) -+ dist.all_to_all_single(receive, send) -+ exchanged.append(receive.reshape(1, 32, 1, 128)) -+ state = MiniMaxH3VideoRowCache(torch.arange(8, 24, device="npu"), seq_len=32, used_len=30, num_layers=1) -+ state.prepare_local_span(rank * 8, 8) -+ x = torch.ones(8, 2, dtype=torch.bfloat16, device="npu") -+ anchor(state, x) -+ output = _subset_prefix_bnsd(exchanged[0], exchanged[1][:, :30], exchanged[2][:, :30], 128**-0.5, state) -+ send = output.reshape(4, 8, 1, 128).contiguous() -+ receive = torch.empty_like(send) -+ dist.all_to_all_single(receive, send) -+ local_output = receive.permute(1, 0, 2, 3).reshape(8, 4, 128) -+ expected = _prefix_bnsd(full[0].unsqueeze(0), full[1][:30].unsqueeze(0), full[2][:30].unsqueeze(0), 128**-0.5)[ -+ 0 -+ ] -+ rows = state.local_dense_positions -+ torch.testing.assert_close( -+ local_output.index_select(0, rows).float(), -+ expected[rank * 8 : (rank + 1) * 8].index_select(0, rows).float(), -+ atol=0.03125, -+ rtol=0.02, -+ ) -+ assert state.subset_attention_calls == 1 and state.subset_query_rows == 14 -+ assert rows.numel() == [8, 0, 0, 6][rank] -+ state.restore(0, x) -+ state.end_step() -+ state.close() -+ torch.npu.synchronize() -+ dist.barrier() -+ finally: -+ dist.destroy_process_group() -+ -+ -+@pytest.mark.skipif(not current_omni_platform.is_npu(), reason="Requires four native Ascend devices") -+def test_native_four_rank_subset_attention_with_empty_local_dense_rows(): -+ torch.multiprocessing.spawn(_native_worker, nprocs=4, join=True) -diff --git a/vllm_omni/config/omni_config.py b/vllm_omni/config/omni_config.py -index 0a803cf..44fc48c 100644 ---- a/vllm_omni/config/omni_config.py -+++ b/vllm_omni/config/omni_config.py -@@ -863,6 +863,7 @@ class _DiffusionConfigProjection(_TrackExplicitConfigFields): - dlo_use_allgather: bool = True - dlo_chunk_size_mb: int = Field(default=64, gt=0, strict=True) - dlo_resident_layers: int = Field(default=0, ge=0) -+ dlo_order_allgather_after_attn_a2a: bool = False - host_weight_runtime_mode: Literal["disabled", "preferred", "required"] = "disabled" - host_weight_runtime_root: str | None = None - dlo_host_registration_limit_gib: float = Field(default=0.0, ge=0) -diff --git a/vllm_omni/config/stage_config.py b/vllm_omni/config/stage_config.py -index 76a58a9..3c69986 100644 ---- a/vllm_omni/config/stage_config.py -+++ b/vllm_omni/config/stage_config.py -@@ -672,6 +672,7 @@ class StageDeployConfig: - dlo_use_allgather: bool | None = None - dlo_chunk_size_mb: int | None = None - dlo_resident_layers: int | None = None -+ dlo_order_allgather_after_attn_a2a: bool | None = None - host_weight_runtime_mode: str | None = None - host_weight_runtime_root: str | None = None - dlo_host_registration_limit_gib: float | None = None -diff --git a/vllm_omni/diffusion/attention/layer.py b/vllm_omni/diffusion/attention/layer.py -index 5520390..574eafa 100644 ---- a/vllm_omni/diffusion/attention/layer.py -+++ b/vllm_omni/diffusion/attention/layer.py -@@ -9,7 +9,7 @@ - - from collections.abc import Mapping - from dataclasses import replace --from typing import cast -+from typing import Any, cast - - import torch - import torch.nn as nn -@@ -333,6 +333,11 @@ class Attention(nn.Module): - # Local strategy when SP is intentionally inactive outside sharded regions. - self._no_parallel_strategy = NoParallelAttention() - -+ # Optional cross-module rendezvous (see distributed/collective_gate.py). -+ # Installed by the DLO backend when it must keep its background weight -+ # AllGather from taking the links ahead of this layer's QKV AllToAll. -+ self._collective_gate: Any | None = None -+ - self.layer_idx: int | None = _try_extract_layer_index(prefix) - - self._kv_cache_dtype: str | None = None -@@ -506,10 +511,12 @@ class Attention(nn.Module): - reason = self._cross_backend_limitation(attn_metadata) - if reason is not None: - blocked = self._cross_fallback_blocker(attn_metadata) -- result = self.sdpa_fallback.resolve_execution_path(resolved_context, query, key, value, attn_metadata) -+ result = cast(SDPAImpl, self.sdpa_fallback).resolve_execution_path( -+ resolved_context, query, key, value, attn_metadata -+ ) - return replace( - result, -- backend=self.attn_backend.get_name() if blocked else "SDPA", -+ backend=cast(type[AttentionBackend], self.attn_backend).get_name() if blocked else "SDPA", - path="cross_unsupported" if blocked else "cross_sdpa_fallback", - support=CapabilityResult.unsupported(blocked) if blocked else CapabilityResult.unmigrated(reason), - ) -@@ -591,6 +598,12 @@ class Attention(nn.Module): - # For Ring: Concat joint_q - query, key, value, attn_metadata, ctx = strategy.pre_attention(query, key, value, attn_metadata) - -+ # The QKV AllToAll is now enqueued on this stream. Release any gather -+ # that was deferred to give this collective the links first. -+ gate = self._collective_gate -+ if gate is not None: -+ gate.signal() -+ - # MiniMax-H3 Tier2 stores reference K/V in the balanced layout produced - # by Ulysses. Keeping this hook at the generic post-parallel boundary - # avoids a model import in the parallel strategy and leaves other models -@@ -760,7 +773,7 @@ class Attention(nn.Module): - def _cross_fallback_blocker(self, attn_metadata: AttentionMetadata | None) -> str | None: - if self.backend_explicit: - return ( -- f"Explicit backend {self.attn_backend.get_name()} does not support " -+ f"Explicit backend {cast(type[AttentionBackend], self.attn_backend).get_name()} does not support " - f"{self._cross_backend_limitation(attn_metadata)}" - ) - if attn_metadata is not None and attn_metadata.extra.get("kv_cache_dtype") not in (None, "float"): -@@ -793,7 +806,7 @@ class Attention(nn.Module): - blocked = self._cross_fallback_blocker(attn_metadata) - if blocked is not None: - raise NotImplementedError(blocked) -- return self.sdpa_fallback.forward(query, key, value, attn_metadata) -+ return cast(SDPAImpl, self.sdpa_fallback).forward(query, key, value, attn_metadata) - dense_mask = attn_metadata.attn_mask if attn_metadata is not None else None - # Piecewise backends honor full_attn_spans ahead of attn_mask and never read the - # dense mask (e.g. HunyuanImage3, which sends both), so there is nothing to divert. -diff --git a/vllm_omni/diffusion/data.py b/vllm_omni/diffusion/data.py -index 8528574..fcbeec0 100644 ---- a/vllm_omni/diffusion/data.py -+++ b/vllm_omni/diffusion/data.py -@@ -962,6 +962,10 @@ class OmniDiffusionConfig: - dlo_chunk_size_mb: int = 64 - # Leading main-DiT blocks kept resident by distributed layerwise offload. - dlo_resident_layers: int = 0 -+ # Hold each DiT block's weight AllGather until that block's QKV AllToAll has -+ # been enqueued, so the background collective does not take the links ahead -+ # of the critical-path all-to-all. -+ dlo_order_allgather_after_attn_a2a: bool = False - # Final-layout Host Weight Runtime policy. The loader only activates this - # for eligible no-AllGather DLO; all other configurations preserve their - # existing loader/storage path. -diff --git a/vllm_omni/diffusion/distributed/collective_gate.py b/vllm_omni/diffusion/distributed/collective_gate.py -new file mode 100644 -index 0000000..1386d89 ---- /dev/null -+++ b/vllm_omni/diffusion/distributed/collective_gate.py -@@ -0,0 +1,72 @@ -+# SPDX-License-Identifier: Apache-2.0 -+# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project -+"""Cross-module rendezvous that orders background collectives after the -+foreground all-to-all of a transformer block. -+ -+Distributed layerwise offload (DLO) issues the *next* block's weight AllGather -+on a dedicated ``comm_stream`` while the *current* block computes. Both that -+AllGather and the block's QKV sequence-parallel AllToAll are collectives over -+the same ranks, so whichever reaches the links first wins: the AllGather is -+submitted at block entry, well before the block reaches its attention, and the -+critical-path AllToAll ends up queued behind it. -+ -+A plain event wait cannot fix this by itself. ``wait_event`` snapshots the -+records enqueued *before* the call, so "enqueue the wait at block entry, record -+the event later in the block" is a no-op -- the wait resolves against the -+previous iteration's record. The gather therefore has to be *issued* after the -+AllToAll: the hook arms this barrier at block entry and defers the gather, the -+attention signals it right after ``strategy.pre_attention``, and the signal -+records the event and lets the hook submit the gather gated on it. -+ -+The gather still overlaps the rest of the block (attention output, MLP, the -+next block's projections); it simply cannot start before the AllToAll finished. -+""" -+ -+from __future__ import annotations -+ -+from collections.abc import Callable -+ -+__all__ = ["QKVAllToAllBarrier"] -+ -+ -+class QKVAllToAllBarrier: -+ """One-block rendezvous between a DLO hook and the block's attention. -+ -+ Owned by a single :class:`DistributedLayerwiseOffloadHook` (one per -+ streaming block) and shared with the attention layer inside that block. -+ Both live on the same block, so the arming and the signalling always come -+ from the same module forward. -+ -+ Args: -+ on_signal: Invoked at most once per :meth:`arm` when the block's -+ attention reports its QKV AllToAll. It must submit the deferred -+ weight AllGather. -+ """ -+ -+ def __init__(self, on_signal: Callable[[], None]) -> None: -+ self._on_signal = on_signal -+ self._armed = False -+ -+ @property -+ def armed(self) -> bool: -+ """True between :meth:`arm` and the matching :meth:`signal`.""" -+ return self._armed -+ -+ def arm(self) -> None: -+ """Mark the next attention QKV AllToAll as the gather's gate.""" -+ self._armed = True -+ -+ def disarm(self) -> None: -+ """Drop a pending arm without firing it (the caller flushes instead).""" -+ self._armed = False -+ -+ def signal(self) -> None: -+ """Report that the QKV AllToAll is now enqueued on the compute stream. -+ -+ A no-op unless armed, so it is cheap to call unconditionally from the -+ attention hot path. -+ """ -+ if not self._armed: -+ return -+ self._armed = False -+ self._on_signal() -diff --git a/vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py b/vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py -index 6ebfb10..044c1f0 100644 ---- a/vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py -+++ b/vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py -@@ -36,6 +36,16 @@ def _prefix_bnsd(query: torch.Tensor, key: torch.Tensor, value: torch.Tensor, sc - return output.reshape(batch, heads, rows, dim).transpose(1, 2).contiguous() - - -+def _subset_prefix_bnsd(query, key, value, scale, state): -+ """Compute only fresh conditioning queries; preserve the shared SP shape.""" -+ rows = state.global_dense_positions -+ dense = _prefix_bnsd(query.index_select(1, rows), key, value, scale) -+ output = torch.zeros_like(query) -+ output.index_copy_(1, rows, dense) -+ state.record_subset_attention(rows.numel()) -+ return output -+ -+ - class MiniMaxH3FlashAttentionImpl(FlashAttentionImpl): - """Use independent head batches for explicit long BF16 prefix attention. - -@@ -51,7 +61,10 @@ class MiniMaxH3FlashAttentionImpl(FlashAttentionImpl): - value: torch.Tensor, - attn_metadata: AttentionMetadata | None = None, - ) -> torch.Tensor: -+ state = attn_metadata.extra.get("minimax_h3_video_row_cache") if attn_metadata is not None else None - if os.environ.get("MINDIE_SD_FA_TYPE") != "fused_attn_score" or attn_metadata is None: -+ if state is not None: -+ raise RuntimeError("H3 video-row cache requires the explicit fused attention path") - return super().forward_fa_npu(query, key, value, attn_metadata) - extra = attn_metadata.extra - if ( -@@ -68,9 +81,17 @@ class MiniMaxH3FlashAttentionImpl(FlashAttentionImpl): - or query.shape[2:] != value.shape[2:] - or key.shape[1] != value.shape[1] - ): -+ if state is not None: -+ raise RuntimeError("H3 video-row cache received unsupported fused attention inputs") - return super().forward_fa_npu(query, key, value, attn_metadata) - resolved = self._resolve_packed_seq_npu(query, key, extra) - if resolved is None: -+ if state is not None: -+ raise RuntimeError("H3 video-row cache requires a resolved packed prefix") - return super().forward_fa_npu(query, key, value, attn_metadata) - used_k = resolved[1][0] -+ if state is not None: -+ if not state.partial or used_k != state.used_len or query.shape[1] != state.seq_len: -+ raise RuntimeError("H3 video-row cache attention layout changed") -+ return _subset_prefix_bnsd(query, key[:, :used_k], value[:, :used_k], self.softmax_scale, state) - return _prefix_bnsd(query, key[:, :used_k], value[:, :used_k], self.softmax_scale) -diff --git a/vllm_omni/diffusion/models/minimax_h3/denoise_loop.py b/vllm_omni/diffusion/models/minimax_h3/denoise_loop.py -index 8cca219..07fb372 100644 ---- a/vllm_omni/diffusion/models/minimax_h3/denoise_loop.py -+++ b/vllm_omni/diffusion/models/minimax_h3/denoise_loop.py -@@ -9,6 +9,7 @@ rows stay pinned to their noised step-0 anchors. - - from __future__ import annotations - -+import os - from collections.abc import Callable - from contextlib import AbstractContextManager, nullcontext - from typing import Any -@@ -69,7 +70,9 @@ class MiniMaxH3DenoiseBranch: - text_embeddings: torch.Tensor, - token_tags: torch.Tensor, - device: torch.device, -+ task: str | None = None, - ) -> None: -+ self.task = task - seq_len = int(packed["seq_len"]) - self.seq_len = seq_len - self.img_pos = packed["img_pos"].view(-1).to(torch.long) -@@ -380,6 +383,24 @@ def minimax_h3_denoise_loop( - positive.static_kwargs["reference_kv_state"] = reference_kv_state - - try: -+ video_row_cache = None -+ create_video_cache = getattr(model, "create_video_row_cache", None) -+ if os.environ.get("H3_VIDEO_ROW_CACHE") == "1": -+ if ( -+ not callable(create_video_cache) -+ or video_edit is not None -+ or audio_edit is not None -+ or keyframe_cond_rows is None -+ or positive.task != "ref2va" -+ ): -+ raise ValueError("H3 video-row cache requires Ref2VA image references without editing") -+ video_row_cache = create_video_cache( -+ positive.img_pos[positive.update_mask].to(device), -+ seq_len=positive.seq_len, -+ used_len=positive.used_len, -+ num_steps=num_steps, -+ ) -+ positive.static_kwargs["video_row_cache"] = video_row_cache - for step in range(num_steps): - check_request_cancellation() - step_cm = step_profiler(step) if step_profiler is not None else nullcontext() -@@ -388,6 +409,8 @@ def minimax_h3_denoise_loop( - s_a = sigmas_audio[step] - if reference_kv_state is not None: - reference_kv_state.begin_step(step, sigma=float(s_v)) -+ if video_row_cache is not None: -+ video_row_cache.begin_step(step) - # Publish where we are so step-gated attention features (the dense - # warmup of RAINFUSION_ATTN, the timestep gate of TRTLLM_ATTN) can - # see it. Gates use the scheduler-style descending timestep, which -@@ -430,6 +453,8 @@ def minimax_h3_denoise_loop( - v_video, v_audio = model(**fk) - if reference_kv_state is not None: - reference_kv_state.end_step() -+ if video_row_cache is not None: -+ video_row_cache.end_step() - mv_video_t = v_video.float()[update] - mv_audio_t = v_audio.float()[audio_update] - -@@ -476,6 +501,10 @@ def minimax_h3_denoise_loop( - - finally: - minimax_h3_publish_denoise_progress(None, None, None) -+ if video_row_cache is not None: -+ positive.static_kwargs.pop("video_row_cache", None) -+ video_row_cache.close() -+ model._video_row_cache_stats = video_row_cache.stats() - if reference_kv_state is not None: - positive.static_kwargs.pop("reference_kv_state", None) - reference_kv_state.close() -diff --git a/vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py b/vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py -index 09f2e7d..601e2ae 100644 ---- a/vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py -+++ b/vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py -@@ -45,6 +45,7 @@ if TYPE_CHECKING: - from vllm.model_executor.layers.quantization.base_config import QuantizationConfig - - from .reference_kv_cache import MiniMaxH3ReferenceKVTier2State -+ from .video_row_cache import MiniMaxH3VideoRowCache - - _BF16_DTYPE = torch.bfloat16 - _FP32_DTYPE = torch.float32 -@@ -291,6 +292,7 @@ class MiniMaxH3Attention(nn.Module): - reference_kv_layer_index: int | None = None, - reference_kv_compact: bool = False, - vdn_window: VDNLayout | None = None, -+ video_row_cache: MiniMaxH3VideoRowCache | None = None, - ) -> torch.Tensor: - """Run packed attention as a small eager island. - -@@ -368,6 +370,8 @@ class MiniMaxH3Attention(nn.Module): - "vsa_h3_ref_policy": self.vsa_ref_policy, - "vsa_h3_ref_keep_rate": self.vsa_ref_keep_rate, - } -+ if video_row_cache is not None and video_row_cache.partial: -+ extra["minimax_h3_video_row_cache"] = video_row_cache - if gate_compress is not None: - extra["gate_compress"] = gate_compress.unsqueeze(0) - if video_layout is not None and video_layout.video_spans: -@@ -455,6 +459,7 @@ class MiniMaxH3Attention(nn.Module): - reference_kv_layer_index: int | None = None, - reference_kv_compact: bool = False, - vdn_window: VDNLayout | None = None, -+ video_row_cache: MiniMaxH3VideoRowCache | None = None, - ) -> torch.Tensor: - """x: [T, hidden] packed thd rows -> [T, hidden]. - -@@ -508,10 +513,19 @@ class MiniMaxH3Attention(nn.Module): - reference_kv_layer_index=reference_kv_layer_index, - reference_kv_compact=reference_kv_compact, - vdn_window=vdn_window, -+ **({"video_row_cache": video_row_cache} if video_row_cache is not None else {}), - ) - if self.vdn is not None: - out = self.vdn.gate_softmax(out, x) - out = out.reshape(total, self.num_heads * self.head_dim) -+ if video_row_cache is not None and video_row_cache.partial: -+ rows = video_row_cache.local_dense_positions -+ assert rows is not None -+ projected = x.new_zeros((total, x.shape[-1])) -+ if rows.numel(): -+ dense, _ = self.out_proj(out.index_select(0, rows)) -+ projected.index_copy_(0, rows, dense) -+ return projected - out, _ = self.out_proj(out) - if self.vdn is not None: - out = self.vdn.add_linear(out, x, q_raw, k_raw, v, vdn_window) -@@ -742,6 +756,7 @@ class MiniMaxH3DiTBlock(nn.Module): - reference_kv_state: MiniMaxH3ReferenceKVTier2State | None = None, - reference_kv_compact: bool = False, - vdn_window: VDNLayout | None = None, -+ video_row_cache: MiniMaxH3VideoRowCache | None = None, - ) -> torch.Tensor: - """x: [T, H]; t_emb: [M, t_dim]; combined_indices: [T] - (= inverse_indices * modality_num + token_tags.clamp(min=0)). -@@ -759,6 +774,7 @@ class MiniMaxH3DiTBlock(nn.Module): - gate_mlp, - ) = self.adaln_proj(t_emb) - -+ block_input = x - residual = x - h = rms_norm_indexed_scale_shift( - x, -@@ -782,7 +798,10 @@ class MiniMaxH3DiTBlock(nn.Module): - reference_kv_layer_index=self.layer_index, - reference_kv_compact=reference_kv_compact, - vdn_window=vdn_window, -+ **({"video_row_cache": video_row_cache} if video_row_cache is not None else {}), - ) -+ if video_row_cache is not None and video_row_cache.pre_gate: -+ h = video_row_cache.pre_gate_output(self.layer_index, h, "attn") - x, h = indexed_gate_rms_norm_scale_shift( - residual, - gate_msa, -@@ -793,6 +812,36 @@ class MiniMaxH3DiTBlock(nn.Module): - combined_indices, - self.norm2.variance_epsilon, - ) -+ if video_row_cache is not None and video_row_cache.pre_gate and video_row_cache.partial: -+ video_row_cache.record_fresh_gate("attn") - residual = x -+ if video_row_cache is not None and video_row_cache.partial: -+ rows = video_row_cache.local_dense_positions -+ assert rows is not None -+ if video_row_cache.pre_gate: -+ mlp_output = torch.zeros_like(h) -+ if rows.numel(): -+ mlp_output.index_copy_(0, rows, self.mlp(h.index_select(0, rows))) -+ mlp_output = video_row_cache.pre_gate_output(self.layer_index, mlp_output, "mlp") -+ output = indexed_gate(residual, gate_mlp, mlp_output, combined_indices) -+ video_row_cache.record_fresh_gate("mlp") -+ video_row_cache.partial_mlp_rows += rows.numel() -+ return output -+ output = video_row_cache.restore(self.layer_index, block_input) -+ if rows.numel(): -+ dense = indexed_gate( -+ residual.index_select(0, rows), -+ gate_mlp, -+ self.mlp(h.index_select(0, rows)), -+ combined_indices.index_select(0, rows), -+ ) -+ output.index_copy_(0, rows, dense) -+ video_row_cache.partial_mlp_rows += rows.numel() -+ return output - h = self.mlp(h) -- return indexed_gate(residual, gate_mlp, h, combined_indices) -+ if video_row_cache is not None and video_row_cache.pre_gate: -+ h = video_row_cache.pre_gate_output(self.layer_index, h, "mlp") -+ output = indexed_gate(residual, gate_mlp, h, combined_indices) -+ if video_row_cache is not None and not video_row_cache.pre_gate: -+ video_row_cache.capture(self.layer_index, block_input, output) -+ return output -diff --git a/vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py b/vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py -index 9518828..b9e07dc 100644 ---- a/vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py -+++ b/vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py -@@ -10,6 +10,7 @@ layout. - from __future__ import annotations - - import math -+import os - from bisect import bisect_left - from collections.abc import Iterable, Mapping - from dataclasses import replace -@@ -66,6 +67,8 @@ if TYPE_CHECKING: - from vllm_omni.diffusion.data import OmniDiffusionConfig - from vllm_omni.diffusion.models.minimax_h3.reference_kv_cache import MiniMaxH3ReferenceKVTier2State - -+ from .video_row_cache import MiniMaxH3VideoRowCache -+ - logger = init_logger(__name__) - - -@@ -245,6 +248,7 @@ _FORWARD_SUPPORTED_KWARGS = frozenset( - "control_rows", - "control_context_scale", - "reference_kv_state", -+ "video_row_cache", - } - ) - -@@ -1232,6 +1236,41 @@ class MiniMaxH3DiTModel(nn.Module): - t_emb = self.time_embedder(unique_timesteps) - return embeddings, t_emb - -+ def create_video_row_cache( -+ self, target_positions: torch.Tensor, *, seq_len: int, used_len: int, num_steps: int -+ ) -> MiniMaxH3VideoRowCache | None: -+ if os.environ.get("H3_VIDEO_ROW_CACHE") != "1": -+ return None -+ p = self.parallel_config -+ if ( -+ not current_omni_platform.is_npu() -+ or num_steps != 50 -+ or p.tensor_parallel_size != 1 -+ or p.data_parallel_size not in (None, 1) -+ or p.cfg_parallel_size != 1 -+ or p.pipeline_parallel_size != 1 -+ or p.ulysses_degree != 4 -+ or p.ring_degree != 1 -+ or p.allgather_degree != 1 -+ or p.ulysses_mode != "strict" -+ or self.od_config.cache_backend not in (None, "none") -+ or self.od_config.kv_offload_config is not None -+ or self.vdn_config is not None -+ or os.environ.get("MINDIE_SD_FA_TYPE") != "fused_attn_score" -+ or any(block.attn.attention.attn_backend.get_name() != "FLASH_ATTN" for block in self.blocks) -+ ): -+ raise ValueError("H3 video-row cache requires dense NPU BF16 Ref2VA with strict Ulysses4, TP/DP/CFG/PP1") -+ from .video_row_cache import MiniMaxH3VideoRowCache -+ -+ self._video_row_cache_stats = None -+ return MiniMaxH3VideoRowCache( -+ target_positions, -+ seq_len=seq_len, -+ used_len=used_len, -+ num_layers=len(self.blocks), -+ pre_gate=os.environ.get("H3_VIDEO_PRE_GATE_CACHE") == "1", -+ ) -+ - def create_reference_kv_state( - self, - *, -@@ -1308,6 +1347,14 @@ class MiniMaxH3DiTModel(nn.Module): - refiner_max = int(self._psp_field(refiner_psp, "refiner_packed_seq_params", "max_seqlen_q")) - video_layout = kwargs.get("video_token_layout") - reference_kv_state = kwargs.get("reference_kv_state") -+ video_row_cache = kwargs.get("video_row_cache") -+ if video_row_cache is not None and ( -+ num_requests != 1 -+ or reference_kv_state is not None -+ or self.vdn_config is not None -+ or kwargs.get("control_rows") is not None -+ ): -+ raise ValueError("H3 video-row cache cannot combine with packed requests, reference KV, VDN or control") - vdn_window = None - if self.vdn_config is not None: - # Text rows lead the packed document. -@@ -1442,6 +1489,11 @@ class MiniMaxH3DiTModel(nn.Module): - if reference_kv_state is not None: - reference_kv_state.set_local_sequence_rows(hidden.shape[0]) - -+ if video_row_cache is not None: -+ if hidden.shape[0] != local_len: -+ raise RuntimeError("H3 video-row cache requires the strict local sequence shard") -+ video_row_cache.prepare_local_span(local_start, local_len) -+ - block_video_layout = video_layout - if compact_reference: - block_video_layout = None -@@ -1505,6 +1557,7 @@ class MiniMaxH3DiTModel(nn.Module): - reference_kv_state=reference_kv_state, - reference_kv_compact=compact_reference, - vdn_window=vdn_window, -+ **({"video_row_cache": video_row_cache} if video_row_cache is not None else {}), - ) - if block_index in hints: - hidden = hidden + hints.pop(block_index) * control_scale -diff --git a/vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py b/vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py -index 476c3cc..ebce8fc 100644 ---- a/vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py -+++ b/vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py -@@ -2115,6 +2115,7 @@ class MiniMaxH3Pipeline( - text_embeddings=text_embeddings, - token_tags=tags, - device=self.device, -+ task=task, - ) - - if control_rows is not None: -diff --git a/vllm_omni/diffusion/models/minimax_h3/video_row_cache.py b/vllm_omni/diffusion/models/minimax_h3/video_row_cache.py -new file mode 100644 -index 0000000..68f9501 ---- /dev/null -+++ b/vllm_omni/diffusion/models/minimax_h3/video_row_cache.py -@@ -0,0 +1,236 @@ -+# SPDX-License-Identifier: Apache-2.0 -+# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project -+"""Request-owned, experimental target-video block residuals for H3 Ref2VA.""" -+ -+from __future__ import annotations -+ -+import torch -+ -+ -+class MiniMaxH3VideoRowCache: -+ """Keep non-video computation dense while reusing video block updates. -+ -+ The first experimental profile is deliberately fixed. No state is shared -+ across requests, and every rank retains all attention collectives. -+ """ -+ -+ def __init__( -+ self, target_positions: torch.Tensor, *, seq_len: int, used_len: int, num_layers: int, pre_gate: bool = False -+ ) -> None: -+ positions = target_positions.detach().cpu().tolist() -+ if not 0 < used_len <= seq_len or not positions or num_layers < 1: -+ raise ValueError("video-row cache requires a nonempty valid packed document") -+ if positions != sorted(set(positions)) or positions[0] < 0 or positions[-1] >= used_len: -+ raise ValueError("target-video positions must be sorted unique valid rows") -+ target_set = set(positions) -+ dense = [i for i in range(used_len) if i not in target_set] -+ if not dense: -+ raise ValueError("video-row cache requires non-video conditioning rows") -+ self.seq_len = seq_len -+ self.used_len = used_len -+ self.num_layers = num_layers -+ self._target_positions = positions -+ self.global_dense_positions = torch.tensor(dense, dtype=torch.long, device=target_positions.device) -+ self.local_dense_positions: torch.Tensor | None = None -+ self.local_video_positions: torch.Tensor | None = None -+ self._local_span: tuple[int, int] | None = None -+ self._deltas: dict[int, torch.Tensor] = {} -+ self.pre_gate = pre_gate -+ self._attention_outputs: dict[int, torch.Tensor] = {} -+ self._mlp_outputs: dict[int, torch.Tensor] = {} -+ self._awaiting_mlp_layer: int | None = None -+ self.fresh_attention_gates = 0 -+ self.fresh_mlp_gates = 0 -+ self._step = -1 -+ self._blocks_this_step = 0 -+ self.partial = False -+ self.closed = False -+ self.full_steps = 0 -+ self.partial_steps = 0 -+ self.full_blocks = 0 -+ self.partial_blocks = 0 -+ self.subset_attention_calls = 0 -+ self.subset_query_rows = 0 -+ self.omitted_query_rows = 0 -+ self.partial_mlp_rows = 0 -+ self.peak_cache_bytes = 0 -+ -+ @staticmethod -+ def is_full_step(step: int) -> bool: -+ if not 0 <= step < 50: -+ raise ValueError("the video-row cache profile requires 50 steps") -+ return step < 4 or step >= 46 or (step - 4) % 5 == 0 -+ -+ def begin_step(self, step: int) -> None: -+ if self.closed or step != self._step + 1: -+ raise RuntimeError("video-row cache step order changed") -+ if self._step >= 0 and self._blocks_this_step != self.num_layers: -+ raise RuntimeError("previous video-row cache step is incomplete") -+ full = self.is_full_step(step) -+ if self.pre_gate: -+ full = step in (0, 1, 4, 9, 19, 29, 39, 49) -+ self.partial = not full -+ if self._awaiting_mlp_layer is not None: -+ raise RuntimeError("previous video pre-gate sublayer is incomplete") -+ complete = ( -+ len(self._attention_outputs) == len(self._mlp_outputs) == self.num_layers -+ if self.pre_gate -+ else len(self._deltas) == self.num_layers -+ ) -+ if self.partial and not complete: -+ raise RuntimeError("partial video step has no complete dense anchor") -+ self._step = step -+ self._blocks_this_step = 0 -+ if self.partial: -+ self.partial_steps += 1 -+ else: -+ self.full_steps += 1 -+ -+ def prepare_local_span(self, start: int, length: int) -> None: -+ if self.closed or start < 0 or length < 1 or start + length > self.seq_len: -+ raise ValueError("invalid local packed span") -+ if self._local_span is not None: -+ if self._local_span != (start, length): -+ raise RuntimeError("video-row cache local ownership changed") -+ return -+ device = self.global_dense_positions.device -+ target = set(self._target_positions) -+ self.local_video_positions = torch.tensor( -+ [i - start for i in self._target_positions if start <= i < start + length], dtype=torch.long, device=device -+ ) -+ self.local_dense_positions = torch.tensor( -+ [i - start for i in range(start, min(start + length, self.used_len)) if i not in target], -+ dtype=torch.long, -+ device=device, -+ ) -+ self._local_span = (start, length) -+ -+ def _visit(self, layer: int, x: torch.Tensor) -> None: -+ if self.closed or self._step < 0 or layer != self._blocks_this_step: -+ raise RuntimeError("video-row cache block order changed") -+ if self._local_span is None or x.shape[0] != self._local_span[1] or x.dtype != torch.bfloat16: -+ raise ValueError("video-row cache requires the prepared BF16 row shard") -+ self._blocks_this_step += 1 -+ -+ def capture(self, layer: int, x: torch.Tensor, output: torch.Tensor) -> None: -+ if self.pre_gate: -+ raise RuntimeError("pre-gate state cannot capture a gated block residual") -+ if self.partial: -+ raise RuntimeError("cannot replace the dense anchor during a partial step") -+ self._visit(layer, x) -+ rows = self.local_video_positions -+ assert rows is not None -+ # BF16 endpoints may have a finite difference outside FP16's range. -+ # Keep that difference in FP32; only the reconstructed block output is -+ # rounded to the original BF16 activation dtype. -+ self._deltas[layer] = output.index_select(0, rows).float() - x.index_select(0, rows).float() -+ self.peak_cache_bytes = max( -+ self.peak_cache_bytes, sum(t.numel() * t.element_size() for t in self._deltas.values()) -+ ) -+ self.full_blocks += 1 -+ -+ def restore(self, layer: int, x: torch.Tensor) -> torch.Tensor: -+ if self.pre_gate: -+ raise RuntimeError("pre-gate state cannot restore a gated block residual") -+ if not self.partial: -+ raise RuntimeError("cannot reuse a video residual on a full step") -+ self._visit(layer, x) -+ rows = self.local_video_positions -+ assert rows is not None -+ # This copy owns the returned full shard, preserving caller input and -+ # unused padding while fresh non-video rows are filled by the block. -+ output = x.clone() -+ if rows.numel(): -+ output.index_copy_(0, rows, (x.index_select(0, rows).float() + self._deltas[layer].float()).to(x.dtype)) -+ self.partial_blocks += 1 -+ return output -+ -+ def _cache_bytes(self) -> int: -+ return sum( -+ tensor.numel() * tensor.element_size() -+ for cache in (self._deltas, self._attention_outputs, self._mlp_outputs) -+ for tensor in cache.values() -+ ) -+ -+ def pre_gate_output(self, layer: int, output: torch.Tensor, stage: str) -> torch.Tensor: -+ """Capture or fill only target rows before applying the current gate.""" -+ if not self.pre_gate or stage not in ("attn", "mlp"): -+ raise ValueError("pre-gate output requires the selected attention or MLP stage") -+ if stage == "attn": -+ if self._awaiting_mlp_layer is not None: -+ raise RuntimeError("previous video pre-gate sublayer is incomplete") -+ self._visit(layer, output) -+ self._awaiting_mlp_layer = layer -+ if self.partial: -+ self.partial_blocks += 1 -+ else: -+ self.full_blocks += 1 -+ else: -+ if self.closed or self._awaiting_mlp_layer != layer: -+ raise RuntimeError("video pre-gate MLP order changed") -+ if output.dtype != torch.bfloat16 or output.shape[0] != self._local_span[1]: -+ raise ValueError("video pre-gate output must retain the BF16 local shard") -+ self._awaiting_mlp_layer = None -+ rows = self.local_video_positions -+ assert rows is not None -+ cache = self._attention_outputs if stage == "attn" else self._mlp_outputs -+ if self.partial: -+ restored = output.clone() -+ restored.index_copy_(0, rows, cache[layer]) -+ return restored -+ cache[layer] = output.index_select(0, rows) -+ self.peak_cache_bytes = max(self.peak_cache_bytes, self._cache_bytes()) -+ return output -+ -+ def record_fresh_gate(self, stage: str) -> None: -+ if not self.pre_gate or not self.partial or stage not in ("attn", "mlp"): -+ raise RuntimeError("unexpected fresh gate record") -+ if stage == "attn": -+ self.fresh_attention_gates += 1 -+ else: -+ self.fresh_mlp_gates += 1 -+ -+ def record_subset_attention(self, rows: int) -> None: -+ if not self.partial or rows != self.global_dense_positions.numel(): -+ raise RuntimeError("unexpected subset attention query count") -+ self.subset_attention_calls += 1 -+ self.subset_query_rows += rows -+ self.omitted_query_rows += len(self._target_positions) -+ -+ def end_step(self) -> None: -+ if self._blocks_this_step != self.num_layers or self._awaiting_mlp_layer is not None: -+ raise RuntimeError("video-row cache step did not visit every main block") -+ -+ def stats(self) -> dict[str, int | bool | str]: -+ counters: dict[str, int | bool | str] = { -+ name: getattr(self, name) -+ for name in [ -+ "full_steps", -+ "partial_steps", -+ "full_blocks", -+ "partial_blocks", -+ "subset_attention_calls", -+ "subset_query_rows", -+ "omitted_query_rows", -+ "partial_mlp_rows", -+ "peak_cache_bytes", -+ "closed", -+ ] -+ } -+ counters["storage_dtype"] = "torch.bfloat16" if self.pre_gate else "torch.float32" -+ counters["cache_stage"] = "pre_gate" if self.pre_gate else "block_residual" -+ counters["fresh_attention_gates"] = self.fresh_attention_gates -+ counters["fresh_mlp_gates"] = self.fresh_mlp_gates -+ counters["live_cache_bytes"] = self._cache_bytes() -+ return counters -+ -+ def close(self) -> None: -+ self._deltas.clear() -+ self._attention_outputs.clear() -+ self._mlp_outputs.clear() -+ self._awaiting_mlp_layer = None -+ self._target_positions.clear() -+ self.global_dense_positions = torch.empty(0, dtype=torch.long) -+ self.local_dense_positions = None -+ self.local_video_positions = None -+ self.closed = True -diff --git a/vllm_omni/diffusion/offloader/base.py b/vllm_omni/diffusion/offloader/base.py -index 2505d48..50b73e5 100644 ---- a/vllm_omni/diffusion/offloader/base.py -+++ b/vllm_omni/diffusion/offloader/base.py -@@ -71,6 +71,10 @@ class OffloadConfig: - dlo_use_allgather: bool = True - dlo_resident_layers: int = 0 # leading DiT layers kept on device - chunk_size_bytes: int = 64 * 1024 * 1024 -+ # Hold each DiT block's weight AllGather until that block's QKV AllToAll has -+ # been enqueued, so the background collective does not take the links ahead -+ # of the critical-path all-to-all. See ``collective_gate.py``. -+ dlo_order_allgather_after_attn_a2a: bool = False - # Optional per-worker ceiling for registering an HWR mmap. Zero means no - # additional ceiling; pin_cpu_memory controls whether registration is tried. - dlo_host_registration_limit_gib: float = 0.0 -@@ -216,6 +220,7 @@ class OffloadConfig: - dlo_use_allgather=dit_uses_allgather, - dlo_resident_layers=dlo_resident_layers, - chunk_size_bytes=chunk_size_bytes, -+ dlo_order_allgather_after_attn_a2a=bool(getattr(od_config, "dlo_order_allgather_after_attn_a2a", False)), - dlo_host_registration_limit_gib=dlo_host_registration_limit_gib, - components=components, - dlo_transfers=dlo_transfers, -diff --git a/vllm_omni/diffusion/offloader/distributed_layerwise_backend.py b/vllm_omni/diffusion/offloader/distributed_layerwise_backend.py -index 7f0f081..8c4f87f 100644 ---- a/vllm_omni/diffusion/offloader/distributed_layerwise_backend.py -+++ b/vllm_omni/diffusion/offloader/distributed_layerwise_backend.py -@@ -31,6 +31,7 @@ import torch.distributed - from torch import nn - from vllm.logger import init_logger - -+from vllm_omni.diffusion.distributed.collective_gate import QKVAllToAllBarrier - from vllm_omni.diffusion.hooks import HookRegistry, ModelHook - from vllm_omni.diffusion.model_loader.host_weight_plan import ( - HostWeightPlan, -@@ -142,6 +143,7 @@ class DistributedLayerwiseOffloadHook(ModelHook): - tensor_transforms: dict[int, Any] | None = None, - materialization_probe_tensor: torch.Tensor | None = None, - chunk_size_bytes: int = 64 * 1024 * 1024, -+ order_allgather_after_attn_a2a: bool = False, - ): - assert isinstance(next_block, nn.Module), "transformer block must be type `torch.nn.Module`" - if type(dp_size) is not int or dp_size < 1: -@@ -167,6 +169,7 @@ class DistributedLayerwiseOffloadHook(ModelHook): - - self.chunk_size_bytes = chunk_size_bytes - self.manifest: PartManifest | None = None -+ self._deferred_chunked_allgather = None - self._h2d_done_events = [current_omni_platform.Event(), current_omni_platform.Event()] - self._ag_done_events = [current_omni_platform.Event(), current_omni_platform.Event()] - # Hook-local events protect chunk-input reuse within this hook. Across -@@ -239,6 +242,19 @@ class DistributedLayerwiseOffloadHook(ModelHook): - # Per-block synchronization primitive: set after H2D copy completes. - self._prefetch_done: Any | None = None - -+ # Optional ordering: hold the background weight AllGather until the -+ # block's QKV AllToAll is on the compute stream. See -+ # distributed/collective_gate.py. Off by default; the attention layer -+ # inside this block is handed the same barrier at install time. -+ self._a2a_barrier: QKVAllToAllBarrier | None = None -+ self._a2a_event: Any | None = None -+ # ``(slot, gpu_shards, gpu_weights, evt)`` of a gather armed but not yet -+ # issued, so ``post_forward`` can flush it if the attention never ran. -+ self._deferred_allgather: tuple[int, dict, dict, Any] | None = None -+ if order_allgather_after_attn_a2a: -+ self._a2a_event = current_omni_platform.Event() -+ self._a2a_barrier = QKVAllToAllBarrier(self._allgather_after_attn_a2a) -+ - # Shared shard (AllGather input) buffers — assigned by backend. - self.gpu_shard_buffers: list[dict[torch.dtype, torch.Tensor] | None] = [None, None] - -@@ -653,49 +669,23 @@ class DistributedLayerwiseOffloadHook(ModelHook): - gpu_shard = shard_bufs[dtype][: cpu_shard.numel()] - gpu_shard.copy_(cpu_shard, non_blocking=non_blocking) - gpu_shards[dtype] = gpu_shard -- self.comm_stream.wait_stream(self.copy_stream) -- with current_omni_platform.stream(self.comm_stream): -- for dtype, local_shard in gpu_shards.items(): -- gw = gpu_weights[dtype][: self._ag_output_sizes[dtype]] -- torch.distributed.all_gather_into_tensor( -- gw, -- local_shard, -- group=self.dp_group, -- ) -- evt.record(self.comm_stream) -+ -+ if self._a2a_barrier is not None and non_blocking and getattr(self, "_nz_cast_fn", None) is None: -+ # Hold the collective until the block's attention reports its -+ # QKV AllToAll, so the background gather cannot take the links -+ # ahead of the critical-path all-to-all. The H2D above has -+ # already been enqueued; only the gather moves, so it still -+ # overlaps the rest of the block. Deferral needs a repoint that -+ # emits no stream work, hence the NZ-cast guard. -+ self._deferred_allgather = (slot, gpu_shards, gpu_weights, evt) -+ self._a2a_barrier.arm() -+ else: -+ self._issue_allgather(slot, gpu_shards, gpu_weights, evt) -+ elif self._a2a_barrier is not None and non_blocking: -+ self._deferred_chunked_allgather = (slot, gpu_weights, evt) -+ self._a2a_barrier.arm() - else: -- for dtype_manifest in self.manifest.dtypes: -- cpu_shard = self.cpu_shards[dtype_manifest.dtype] -- buf = shard_bufs[dtype_manifest.dtype] -- for chunk in dtype_manifest.chunks: -- if buf.numel() >= 2 * chunk.local_numel: -- width = buf.numel() // 2 -- input_slot = chunk.chunk_id % 2 -- else: -- width = buf.numel() -- input_slot = 0 -- reuse = self._chunk_reuse_events[input_slot] -- if reuse is not None: -- self.copy_stream.wait_event(reuse) -- local_input = buf[input_slot * width : input_slot * width + chunk.local_numel] -- source = cpu_shard[chunk.cpu_offset : chunk.cpu_offset + chunk.local_numel] -- with current_omni_platform.stream(self.copy_stream): -- local_input.copy_(source, non_blocking=non_blocking) -- self._h2d_done_events[input_slot].record(self.copy_stream) -- self.comm_stream.wait_event(self._h2d_done_events[input_slot]) -- with current_omni_platform.stream(self.comm_stream): -- full_out = gpu_weights[dtype_manifest.dtype][ -- chunk.full_offset : chunk.full_offset + chunk.padded_numel -- ] -- torch.distributed.all_gather_into_tensor( -- full_out, -- local_input, -- group=self.dp_group, -- ) -- self._ag_done_events[input_slot].record(self.comm_stream) -- self._chunk_reuse_events[input_slot] = self._ag_done_events[input_slot] -- with current_omni_platform.stream(self.comm_stream): -- evt.record(self.comm_stream) -+ self._issue_chunked_allgather(slot, gpu_weights, evt, non_blocking=non_blocking) - - self.ready_events[slot] = evt - self._prefetch_done = evt -@@ -719,6 +709,131 @@ class DistributedLayerwiseOffloadHook(ModelHook): - ), - ) - -+ def _issue_chunked_allgather(self, slot, gpu_weights, evt, *, gated=False, non_blocking=True) -> None: -+ """Submit chunk copies and gathers together after the foreground gate. -+ -+ Chunk staging slots are bounded and reused. Deferring only gathers -+ would overwrite an input slot before its collective consumes it. -+ """ -+ shard_bufs = self.gpu_shard_buffers[slot] -+ assert shard_bufs is not None and self.manifest is not None -+ if gated: -+ self.comm_stream.wait_event(self._a2a_event) -+ for dtype_manifest in self.manifest.dtypes: -+ cpu_shard = self.cpu_shards[dtype_manifest.dtype] -+ buf = shard_bufs[dtype_manifest.dtype] -+ for chunk in dtype_manifest.chunks: -+ if buf.numel() >= 2 * chunk.local_numel: -+ width = buf.numel() // 2 -+ input_slot = chunk.chunk_id % 2 -+ else: -+ width = buf.numel() -+ input_slot = 0 -+ reuse = self._chunk_reuse_events[input_slot] -+ if reuse is not None: -+ self.copy_stream.wait_event(reuse) -+ local_input = buf[input_slot * width : input_slot * width + chunk.local_numel] -+ source = cpu_shard[chunk.cpu_offset : chunk.cpu_offset + chunk.local_numel] -+ with current_omni_platform.stream(self.copy_stream): -+ local_input.copy_(source, non_blocking=non_blocking) -+ self._h2d_done_events[input_slot].record(self.copy_stream) -+ self.comm_stream.wait_event(self._h2d_done_events[input_slot]) -+ with current_omni_platform.stream(self.comm_stream): -+ full_out = gpu_weights[dtype_manifest.dtype][ -+ chunk.full_offset : chunk.full_offset + chunk.padded_numel -+ ] -+ torch.distributed.all_gather_into_tensor( -+ full_out, -+ local_input, -+ group=self.dp_group, -+ ) -+ self._ag_done_events[input_slot].record(self.comm_stream) -+ self._chunk_reuse_events[input_slot] = self._ag_done_events[input_slot] -+ with current_omni_platform.stream(self.comm_stream): -+ evt.record(self.comm_stream) -+ -+ def _issue_allgather( -+ self, -+ slot: int, -+ gpu_shards: dict[torch.dtype, torch.Tensor], -+ gpu_weights: dict[torch.dtype, torch.Tensor], -+ evt: Any, -+ *, -+ gated: bool = False, -+ ) -> None: -+ """Submit this block's weight AllGather on ``comm_stream``. -+ -+ ``gated`` additionally waits on the QKV AllToAll barrier event recorded -+ by the block's attention, so the foreground collective gets the links -+ first and this gather queues behind it. -+ """ -+ self.comm_stream.wait_stream(self.copy_stream) -+ if gated: -+ self.comm_stream.wait_event(self._a2a_event) -+ with current_omni_platform.stream(self.comm_stream): -+ for dtype, local_shard in gpu_shards.items(): -+ # Slice the shared (max-sized) output buffer down to this -+ # block's actual AllGather output size. The buffers are -+ # sized to the *largest* block across all groups, so for any -+ # smaller block the full buffer would violate the -+ # all_gather_into_tensor contract -+ # (output.numel() == world_size * input.numel()). -+ # Repoint offsets are relative to the block's flattened -+ # buffer, so a prefix slice is safe. -+ gw = gpu_weights[dtype][: self._ag_output_sizes[dtype]] -+ torch.distributed.all_gather_into_tensor( -+ gw, -+ local_shard, -+ group=self.dp_group, -+ ) -+ evt.record(self.comm_stream) -+ -+ def _allgather_after_attn_a2a(self) -> None: -+ """Barrier callback: the block's QKV AllToAll is now enqueued. -+ -+ Recording on the compute stream here (rather than at block entry) is -+ what makes the gate effective: an event wait snapshots the records -+ enqueued before it, so the wait must be enqueued *after* the all-to-all. -+ """ -+ chunked = self._deferred_chunked_allgather -+ if chunked is not None: -+ self._deferred_chunked_allgather = None -+ assert self._a2a_event is not None -+ self._a2a_event.record(current_omni_platform.current_stream()) -+ self._issue_chunked_allgather(*chunked, gated=True) -+ return -+ deferred = self._deferred_allgather -+ if deferred is None: -+ return -+ self._deferred_allgather = None -+ assert self._a2a_event is not None -+ self._a2a_event.record(current_omni_platform.current_stream()) -+ slot, gpu_shards, gpu_weights, evt = deferred -+ self._issue_allgather(slot, gpu_shards, gpu_weights, evt, gated=True) -+ -+ def _flush_deferred_allgather(self) -> None: -+ """Issue a deferred gather whose attention never signalled. -+ -+ Keeps the double buffer live: a block whose attention is skipped -+ (cache-dit, an all-resident stage) still needs its weights, so the -+ gather is submitted ungated at the end of that block's forward. -+ """ -+ chunked = self._deferred_chunked_allgather -+ if chunked is not None: -+ self._deferred_chunked_allgather = None -+ if self._a2a_barrier is not None: -+ self._a2a_barrier.disarm() -+ self._issue_chunked_allgather(*chunked) -+ return -+ deferred = self._deferred_allgather -+ if deferred is None: -+ return -+ self._deferred_allgather = None -+ if self._a2a_barrier is not None: -+ self._a2a_barrier.disarm() -+ slot, gpu_shards, gpu_weights, evt = deferred -+ self._issue_allgather(slot, gpu_shards, gpu_weights, evt) -+ - def _wait_for_weights(self, slot: int) -> None: - """Wait until the slot holding this block is ready for compute. - -@@ -840,6 +955,10 @@ class DistributedLayerwiseOffloadHook(ModelHook): - return args, kwargs - - def post_forward(self, module: nn.Module, output: Any) -> Any: -+ # Pair with pre_forward: if the block's attention never signalled the -+ # gate, the deferred gather must still go out before this block's -+ # storage is released (offload_layer waits on _prefetch_done). -+ self._flush_deferred_allgather() - self.offload_layer() - return output - -@@ -864,6 +983,7 @@ def apply_distributed_block_hook( - tensor_transforms: dict[int, Any] | None = None, - materialization_probe_tensor: torch.Tensor | None = None, - chunk_size_bytes: int = 64 * 1024 * 1024, -+ order_allgather_after_attn_a2a: bool = False, - ) -> DistributedLayerwiseOffloadHook: - """Register a DistributedLayerwiseOffloadHook on *module*.""" - registry = HookRegistry.get_or_create(module) -@@ -881,11 +1001,32 @@ def apply_distributed_block_hook( - tensor_transforms=tensor_transforms, - materialization_probe_tensor=materialization_probe_tensor, - chunk_size_bytes=chunk_size_bytes, -+ order_allgather_after_attn_a2a=order_allgather_after_attn_a2a, - ) - registry.register_hook(DistributedLayerwiseOffloadHook._HOOK_NAME, hook) - return hook - - -+def attach_collective_gate(block: nn.Module, hook: DistributedLayerwiseOffloadHook) -> int: -+ """Hand ``hook``'s barrier to every Attention layer inside *block*. -+ -+ The attention signals the barrier right after its QKV AllToAll. The import -+ is local to keep this module independent of the attention package at import -+ time. -+ -+ Returns the number of attention layers wired (0 means the hook will rely on -+ the ``post_forward`` flush instead). -+ """ -+ from vllm_omni.diffusion.attention.layer import Attention -+ -+ attached = 0 -+ for module in block.modules(): -+ if isinstance(module, Attention): -+ module._collective_gate = hook._a2a_barrier -+ attached += 1 -+ return attached -+ -+ - def remove_distributed_block_hook(module: nn.Module) -> None: - """Remove the distributed layerwise offload hook from *module*.""" - registry: HookRegistry | None = getattr(module, "_hook_registry", None) -@@ -1566,29 +1707,45 @@ class DistributedLayerwiseOffloadBackend(OffloadBackend): - self._all_hook_groups.append(hooks) - self._blocks.append(block_list) - probes = {id(block): module_materialization_probe(block) for block in block_list} -+ # Only the DiT blocks issue a QKV AllToAll the gather can queue behind. -+ order_gate = bool(getattr(self.config, "dlo_order_allgather_after_attn_a2a", False)) and ( -+ component == DIT_COMPONENT -+ ) -+ if order_gate and not getattr(self, "_collective_gate_logged", False): -+ self._collective_gate_logged = True -+ logger.info( -+ "Distributed layerwise offload will hold each DiT block's weight AllGather " -+ "until that block's QKV AllToAll is on the compute stream" -+ ) - for block, next_block in zip( - chain((block_list[-1],), block_list[:-1]), - block_list, - strict=True, - ): -- hooks.append( -- apply_distributed_block_hook( -- block, -- next_block, -- self.device, -- group, -- group_size, -- group_rank, -- self.copy_stream, -- self.comm_stream, -- self.config.pin_cpu_memory, -- shared_buffers=[None, None], -- rank_local_mmap=self._using_rank_local_mmap if use_dit_mmap else False, -- tensor_transforms=self._mmap_transforms_by_tensor_id if use_dit_mmap else None, -- materialization_probe_tensor=probes[id(block)], -- chunk_size_bytes=self.config.chunk_size_bytes, -- ) -+ hook = apply_distributed_block_hook( -+ block, -+ next_block, -+ self.device, -+ group, -+ group_size, -+ group_rank, -+ self.copy_stream, -+ self.comm_stream, -+ self.config.pin_cpu_memory, -+ shared_buffers=[None, None], -+ rank_local_mmap=self._using_rank_local_mmap if use_dit_mmap else False, -+ tensor_transforms=self._mmap_transforms_by_tensor_id if use_dit_mmap else None, -+ materialization_probe_tensor=probes[id(block)], -+ chunk_size_bytes=self.config.chunk_size_bytes, -+ order_allgather_after_attn_a2a=order_gate, - ) -+ if hook._a2a_barrier is not None and attach_collective_gate(block, hook) == 0: -+ logger.warning( -+ "DLO AllGather ordering is enabled but block %s exposes no Attention layer; " -+ "its gather will fall back to the ungated path", -+ block.__class__.__name__, -+ ) -+ hooks.append(hook) - - # hooks = [last -> first, block0 -> block1, ...]. Alternating slots - # keep a prefetch from overwriting the current block for any ring size. -diff --git a/vllm_omni/engine/arg_utils.py b/vllm_omni/engine/arg_utils.py -index 0a876b9..bf8dcdc 100644 ---- a/vllm_omni/engine/arg_utils.py -+++ b/vllm_omni/engine/arg_utils.py -@@ -807,6 +807,7 @@ class OrchestratorArgs: - dlo_use_allgather: bool = True - dlo_chunk_size_mb: int = 64 - dlo_resident_layers: int = 0 -+ dlo_order_allgather_after_attn_a2a: bool = False - host_weight_runtime_mode: str = "disabled" - host_weight_runtime_root: str | None = None - dlo_host_registration_limit_gib: float = 0.0 -diff --git a/vllm_omni/entrypoints/cli/serve.py b/vllm_omni/entrypoints/cli/serve.py -index f2b55e7..aff71b5 100644 ---- a/vllm_omni/entrypoints/cli/serve.py -+++ b/vllm_omni/entrypoints/cli/serve.py -@@ -1041,6 +1041,17 @@ class OmniServeCommand(CLISubcommand): - help="Full-chunk target size in MiB for distributed layerwise offload " - "(each chunk is one host-to-device copy + AllGather unit).", - ) -+ omni_config_group.add_argument( -+ "--dlo-order-allgather-after-attn-a2a", -+ action="store_true", -+ default=False, -+ help=( -+ "Hold each DiT block's weight AllGather until that block's QKV AllToAll " -+ "has been enqueued, so the background gather does not take the collective " -+ "links ahead of the critical-path all-to-all. Requires distributed " -+ "layerwise offload with AllGather." -+ ), -+ ) - omni_config_group.add_argument( - "--host-weight-runtime-mode", - choices=("disabled", "preferred", "required"), diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c21-source-identity.json b/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c21-source-identity.json deleted file mode 100644 index 160c55a95..000000000 --- a/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c21-source-identity.json +++ /dev/null @@ -1,28 +0,0 @@ -{ - "head": "d48fa51d03dd748568cbcf57ed40f533c691c9bd", - "tree": "637647dbe51ccbd092f076c25e0ced0fe0404207", - "base": "d027b4d0c53bf40876215d4e3856bc17450a4c39", - "branch": "rsi/h3-297-video-pre-gate", - "files": { - "tests/diffusion/distributed/test_collective_gate.py": "6a02f32882d85eca1e3bb9a023ce3adb3d5b59c213845e11e85c96a89432f477", - "tests/diffusion/models/minimax_h3/test_minimax_h3_pre_gate_cache.py": "a4924d0b3dd58ae413259edba6cf2487b3e41d391f943fb74358934be72eb53b", - "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "e09b872f6900e821d42632a0ca09595f2affe6f9794049dbc74d69ba6a52b2b0", - "vllm_omni/config/omni_config.py": "6894120a330474f718d43bf2bf7928892eb996152839ce6469c34d18a7df09d1", - "vllm_omni/config/stage_config.py": "0752e8cc1c6f1021f737830d08663cfc118923a83fd909c607f9ec3ab2a3d2fb", - "vllm_omni/diffusion/attention/layer.py": "76c5fce3dabd23e39a8890316e1691c03bc203066cb60af5611ec4790eefc632", - "vllm_omni/diffusion/data.py": "5db83d5c71cb27cc7810d35fcfc891da80f0b003adbb16bdecc3816a63305e11", - "vllm_omni/diffusion/distributed/collective_gate.py": "01c6317cfa615840681b8bb8b8513fa3736d26e845a24ad9e1ab7f87489b962f", - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ca87df1936fd2694fcc768079e9e8f926b6d45969ddc61ebf353f5e726d70b8b", - "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "c5fab405ee82bc553e9a902fa27e6dfbc5213ec18311e17b6415396a69b50f43", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "8933b536317652d0dbc7a974ea7e0bca928e7b3bdf438988999ff5a498a6ec9d", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "019bf71106ba3263b55ad7b682064e3a176eecb81725cb48bbf178387356d5d9", - "vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py": "e8e2e74fb8edb05e02d3e2328ea1f3ad5ddcfd7a10ed09c5cf3911b1d063e849", - "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "04a9212994b5c8593f8d196820300ded077fed9af4274b64d87d9e6bf71eae07", - "vllm_omni/diffusion/offloader/base.py": "23922136deaf54e2d98e827b007b34c7e3ad36171b71bfe9060d390efc0b679f", - "vllm_omni/diffusion/offloader/distributed_layerwise_backend.py": "19fbce0f1036e4784d68cab798c499526a9099101909e41ce96dd0caa033b6a4", - "vllm_omni/engine/arg_utils.py": "52569e3688cc708729a48ca0a2c875c28193733c2f934f6ff89f6c815e4009c7", - "vllm_omni/entrypoints/cli/serve.py": "f8284ad6b751ba4a9e4f7254d78132cc2cfd49871a0f9af7fde49c6789d79cb0" - }, - "published": false, - "patch_base": "be319692b96413918973b5bd399cdda3f320c574" -} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c22-candidate.patch b/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c22-candidate.patch deleted file mode 100644 index bcf1323e4..000000000 --- a/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c22-candidate.patch +++ /dev/null @@ -1,2187 +0,0 @@ -diff --git a/tests/diffusion/distributed/test_collective_gate.py b/tests/diffusion/distributed/test_collective_gate.py -new file mode 100644 -index 0000000..e66fdae ---- /dev/null -+++ b/tests/diffusion/distributed/test_collective_gate.py -@@ -0,0 +1,205 @@ -+# SPDX-License-Identifier: Apache-2.0 -+# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project -+ -+"""Unit tests for the DLO/QKV-AllToAll ordering gate. -+ -+Covers the barrier rendezvous and the hook's deferred-AllGather state machine -+that keeps the background weight gather from taking the links ahead of the -+block's foreground AllToAll. -+""" -+ -+from unittest.mock import Mock -+ -+import pytest -+import torch -+from torch import nn -+ -+import vllm_omni.diffusion.offloader.distributed_layerwise_backend as dist_backend_module -+from tests.diffusion.offloader.helpers import DummyStream, patch_offload_runtime -+from vllm_omni.diffusion.distributed.collective_gate import QKVAllToAllBarrier -+from vllm_omni.diffusion.offloader.distributed_layerwise_backend import ( -+ DistributedLayerwiseOffloadHook, -+ attach_collective_gate, -+) -+ -+pytestmark = [pytest.mark.diffusion, pytest.mark.cpu, pytest.mark.core_model] -+ -+ -+@pytest.fixture -+def patched_offload_runtime(monkeypatch): -+ patch_offload_runtime(monkeypatch, dist_backend_module.current_omni_platform, synchronize=True) -+ -+ -+class TestQKVAllToAllBarrier: -+ def test_signal_without_arm_is_a_noop(self): -+ on_signal = Mock() -+ barrier = QKVAllToAllBarrier(on_signal) -+ -+ barrier.signal() -+ -+ on_signal.assert_not_called() -+ assert not barrier.armed -+ -+ def test_arm_then_signal_fires_once(self): -+ on_signal = Mock() -+ barrier = QKVAllToAllBarrier(on_signal) -+ -+ barrier.arm() -+ assert barrier.armed -+ barrier.signal() -+ -+ on_signal.assert_called_once_with() -+ assert not barrier.armed -+ -+ # A second signal in the same forward must not re-issue the gather. -+ barrier.signal() -+ on_signal.assert_called_once_with() -+ -+ def test_disarm_suppresses_a_pending_signal(self): -+ on_signal = Mock() -+ barrier = QKVAllToAllBarrier(on_signal) -+ -+ barrier.arm() -+ barrier.disarm() -+ barrier.signal() -+ -+ on_signal.assert_not_called() -+ -+ -+def _gated_hook() -> DistributedLayerwiseOffloadHook: -+ """A hook wired for the gated path, with hand-set shard buffers.""" -+ hook = DistributedLayerwiseOffloadHook( -+ next_block=nn.Linear(2, 2, bias=False), -+ device=torch.device("cpu"), -+ dp_group=object(), -+ dp_size=2, -+ rank=0, -+ copy_stream=DummyStream(), -+ comm_stream=DummyStream(), -+ pin_memory=False, -+ order_allgather_after_attn_a2a=True, -+ ) -+ device_buf = {torch.float32: torch.zeros(4)} -+ shard_buf = {torch.float32: torch.zeros(2)} -+ hook.gpu_buffers = [dict(device_buf), dict(device_buf)] -+ hook.gpu_shard_buffers = [dict(shard_buf), dict(shard_buf)] -+ hook.cpu_shards = {torch.float32: torch.zeros(2)} -+ hook._ag_output_sizes = {torch.float32: 4} -+ hook.metadata = {} -+ hook._cached_repoint = () -+ hook.rank_local_mmap = False -+ return hook -+ -+ -+class TestDeferredAllGather: -+ def test_prefetch_defers_the_gather_until_the_barrier_signals(self, patched_offload_runtime, monkeypatch): -+ hook = _gated_hook() -+ gather = Mock() -+ monkeypatch.setattr(torch.distributed, "all_gather_into_tensor", gather) -+ -+ hook.prefetch_layer(0, non_blocking=True) -+ -+ # H2D ran; the collective is held back for the block's AllToAll. -+ gather.assert_not_called() -+ assert hook._a2a_barrier.armed -+ assert hook._deferred_allgather is not None -+ # The ready event must still be published for the consumer. -+ assert hook.ready_events[0] is hook._prefetch_done -+ -+ hook._a2a_barrier.signal() -+ -+ gather.assert_called_once() -+ assert hook._deferred_allgather is None -+ assert not hook._a2a_barrier.armed -+ -+ def test_post_forward_flush_issues_an_unsignalled_gather(self, patched_offload_runtime, monkeypatch): -+ """A block whose attention never ran must still get its weights.""" -+ hook = _gated_hook() -+ gather = Mock() -+ monkeypatch.setattr(torch.distributed, "all_gather_into_tensor", gather) -+ -+ hook.prefetch_layer(0, non_blocking=True) -+ gather.assert_not_called() -+ -+ # pre_forward ran, attention did not: the pairing flush must fire. -+ hook.post_forward(hook.next_block, None) -+ -+ gather.assert_called_once() -+ assert hook._deferred_allgather is None -+ assert not hook._a2a_barrier.armed -+ -+ def test_sync_prefetch_issues_immediately_without_arming(self, patched_offload_runtime, monkeypatch): -+ """The cache-dit sync fallback has no attention signal to wait for.""" -+ hook = _gated_hook() -+ gather = Mock() -+ monkeypatch.setattr(torch.distributed, "all_gather_into_tensor", gather) -+ -+ hook.prefetch_layer(0, non_blocking=False) -+ -+ gather.assert_called_once() -+ assert not hook._a2a_barrier.armed -+ assert hook._deferred_allgather is None -+ -+ def test_disabled_by_default(self, patched_offload_runtime, monkeypatch): -+ hook = DistributedLayerwiseOffloadHook( -+ next_block=nn.Linear(2, 2, bias=False), -+ device=torch.device("cpu"), -+ dp_group=object(), -+ dp_size=2, -+ rank=0, -+ copy_stream=DummyStream(), -+ comm_stream=DummyStream(), -+ pin_memory=False, -+ ) -+ hook.gpu_buffers = [{torch.float32: torch.zeros(4)}] * 2 -+ hook.gpu_shard_buffers = [{torch.float32: torch.zeros(2)}] * 2 -+ hook.cpu_shards = {torch.float32: torch.zeros(2)} -+ hook._ag_output_sizes = {torch.float32: 4} -+ hook.metadata = {} -+ hook._cached_repoint = () -+ gather = Mock() -+ monkeypatch.setattr(torch.distributed, "all_gather_into_tensor", gather) -+ -+ hook.prefetch_layer(0, non_blocking=True) -+ -+ assert hook._a2a_barrier is None -+ gather.assert_called_once() -+ -+ -+class TestAttachCollectiveGate: -+ def test_block_without_attention_reports_zero(self): -+ hook = _gated_hook() -+ -+ assert attach_collective_gate(nn.Linear(2, 2), hook) == 0 -+ -+ -+@pytest.mark.parametrize("flush", [False, True]) -+def test_chunked_gate_defers_copies_and_reuses_bounded_slots(patched_offload_runtime, monkeypatch, flush): -+ from types import SimpleNamespace -+ -+ hook = _gated_hook() -+ chunks = tuple( -+ SimpleNamespace(chunk_id=i, local_numel=1, cpu_offset=i, full_offset=2 * i, padded_numel=2) for i in range(3) -+ ) -+ hook.manifest = SimpleNamespace(dtypes=(SimpleNamespace(dtype=torch.float32, chunks=chunks),)) -+ hook.cpu_shards = {torch.float32: torch.tensor([11.0, 22.0, 33.0])} -+ hook.gpu_buffers = [{torch.float32: torch.zeros(6)} for _ in range(2)] -+ calls = [] -+ -+ def gather(output, local, **kwargs): -+ calls.append(local.clone()) -+ output.copy_(local.repeat(2)) -+ -+ monkeypatch.setattr(torch.distributed, "all_gather_into_tensor", gather) -+ hook.prefetch_layer(0, non_blocking=True) -+ assert not calls and hook._deferred_chunked_allgather is not None -+ assert torch.equal(hook.gpu_shard_buffers[0][torch.float32], torch.zeros(2)) -+ if flush: -+ hook._flush_deferred_allgather() -+ else: -+ hook._a2a_barrier.signal() -+ assert len(calls) == 3 and hook._deferred_chunked_allgather is None -+ assert not hook._a2a_barrier.armed -+ assert torch.equal(hook.gpu_buffers[0][torch.float32], torch.tensor([11.0, 11.0, 22.0, 22.0, 33.0, 33.0])) -+ hook._a2a_barrier.signal() -+ assert len(calls) == 3 -diff --git a/tests/diffusion/models/minimax_h3/test_minimax_h3_pre_gate_cache.py b/tests/diffusion/models/minimax_h3/test_minimax_h3_pre_gate_cache.py -new file mode 100644 -index 0000000..19b8246 ---- /dev/null -+++ b/tests/diffusion/models/minimax_h3/test_minimax_h3_pre_gate_cache.py -@@ -0,0 +1,199 @@ -+# SPDX-License-Identifier: Apache-2.0 -+# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project -+ -+from datetime import timedelta -+ -+import pytest -+import torch -+import torch.distributed as dist -+import torch.nn as nn -+ -+from tests.diffusion.models.minimax_h3.test_minimax_h3_video_row_cache import tiny_block -+from tests.helpers.mark import hardware_marks -+from vllm_omni.diffusion.layers.indexed_modulation import indexed_gate -+from vllm_omni.diffusion.models.minimax_h3.attention.flash_h3 import _prefix_bnsd, _subset_prefix_bnsd -+from vllm_omni.diffusion.models.minimax_h3.video_row_cache import MiniMaxH3VideoRowCache -+from vllm_omni.platforms import current_omni_platform -+ -+pytestmark = [pytest.mark.core_model, pytest.mark.diffusion, *hardware_marks(res={"npu": "A3"}, num_cards=4)] -+ -+ -+def make_state(start=0, length=4): -+ state = MiniMaxH3VideoRowCache(torch.tensor([1, 2]), seq_len=4, used_len=3, num_layers=1, pre_gate=True) -+ state.prepare_local_span(start, length) -+ return state -+ -+ -+def test_fixed_pre_gate_schedule_storage_and_request_cleanup(): -+ state = make_state() -+ x = torch.ones(4, 2, dtype=torch.bfloat16) -+ full = [] -+ for step in range(50): -+ state.begin_step(step) -+ if not state.partial: -+ full.append(step) -+ for stage in ("attn", "mlp"): -+ out = state.pre_gate_output(0, x, stage) -+ assert out.dtype == torch.bfloat16 -+ if state.partial: -+ state.record_fresh_gate(stage) -+ state.end_step() -+ assert full == [0, 1, 4, 9, 19, 29, 39, 49] -+ assert state.full_blocks == 8 and state.partial_blocks == 42 -+ assert state.fresh_attention_gates == state.fresh_mlp_gates == 42 -+ assert all(t.dtype == torch.bfloat16 for t in [*state._attention_outputs.values(), *state._mlp_outputs.values()]) -+ assert state.stats()["live_cache_bytes"] == 16 -+ state.close() -+ assert state.stats()["live_cache_bytes"] == 0 and state.closed -+ -+ -+@pytest.mark.parametrize("failure", ["missing_mlp", "mlp_first", "gated_capture", "gated_restore"]) -+def test_pre_gate_stage_order_cannot_silently_use_block_residual(failure): -+ state = make_state() -+ x = torch.ones(4, 2, dtype=torch.bfloat16) -+ state.begin_step(0) -+ with pytest.raises(RuntimeError): -+ if failure == "missing_mlp": -+ state.pre_gate_output(0, x, "attn") -+ state.end_step() -+ elif failure == "mlp_first": -+ state.pre_gate_output(0, x, "mlp") -+ elif failure == "gated_capture": -+ state.capture(0, x, x) -+ else: -+ state.restore(0, x) -+ state.close() -+ -+ -+class ConstantMLP(nn.Module): -+ def __init__(self): -+ super().__init__() -+ self.rows = [] -+ -+ def forward(self, x): -+ self.rows.append(x.shape[0]) -+ return torch.full_like(x, 3) -+ -+ -+@pytest.mark.parametrize("start,length", [(0, 4), (1, 2)]) -+def test_real_block_reapplies_changed_gates_and_preserves_empty_dense_shards(start, length): -+ state = make_state(start, length) -+ block = tiny_block() -+ gates = [1.0, 1.0] -+ block.adaln_proj = lambda t: ( -+ torch.zeros(3, 2), -+ torch.zeros(3, 2), -+ torch.full((3, 2), gates[0]), -+ torch.zeros(3, 2), -+ torch.zeros(3, 2), -+ torch.full((3, 2), gates[1]), -+ ) -+ block.attn = lambda x, **kwargs: torch.full_like(x, 2) -+ block.mlp = ConstantMLP() -+ x = torch.ones(length, 2, dtype=torch.bfloat16) -+ kwargs = dict( -+ t_emb=torch.zeros(1), -+ combined_indices=torch.zeros(length, dtype=torch.long), -+ rope_table=None, -+ cu_seqlens=torch.tensor([0, 3, 4], dtype=torch.int32), -+ max_seqlen=3, -+ packed_total=4, -+ ) -+ for step in (0, 1): -+ state.begin_step(step) -+ expected = block(x, **kwargs) -+ actual = block(x, video_row_cache=state, **kwargs) -+ assert torch.equal(actual.view(torch.uint8), expected.view(torch.uint8)) -+ state.end_step() -+ gates[:] = [0.5, -0.25] -+ current = x * 2 -+ saved = current.clone() -+ expected = block(current, **kwargs) -+ block.mlp.rows.clear() -+ state.begin_step(2) -+ actual = block(current, video_row_cache=state, **kwargs) -+ state.end_step() -+ rows = state.local_dense_positions -+ assert block.mlp.rows == ([rows.numel()] if rows.numel() else []) -+ assert torch.equal( -+ actual[: min(length, 3 - start)].view(torch.uint8), expected[: min(length, 3 - start)].view(torch.uint8) -+ ) -+ assert torch.equal( -+ actual.index_select(0, state.local_video_positions), torch.full((state.local_video_positions.numel(), 2), 2.25) -+ ) -+ assert not torch.equal( -+ actual.index_select(0, state.local_video_positions), current.index_select(0, state.local_video_positions) + 5 -+ ) -+ assert torch.equal(current, saved) -+ assert state.fresh_attention_gates == state.fresh_mlp_gates == 1 -+ state.close() -+ -+ -+def _native_pre_gate_worker(rank, secant=False): -+ import torch_npu # noqa: F401 -+ -+ torch.npu.set_device(rank) -+ dist.init_process_group( -+ "hccl", init_method="tcp://127.0.0.1:29597", rank=rank, world_size=4, timeout=timedelta(seconds=120) -+ ) -+ try: -+ torch.manual_seed(219) -+ full = [torch.randn(32, 4, 128, dtype=torch.bfloat16).npu() for _ in range(3)] -+ exchanged = [] -+ for tensor in full: -+ local = tensor[rank * 8 : (rank + 1) * 8] -+ send = local.reshape(8, 4, 1, 128).permute(1, 0, 2, 3).contiguous() -+ receive = torch.empty_like(send) -+ dist.all_to_all_single(receive, send) -+ exchanged.append(receive.reshape(1, 32, 1, 128)) -+ state = MiniMaxH3VideoRowCache( -+ torch.arange(8, 24, device="npu"), seq_len=32, used_len=30, num_layers=1, pre_gate=True, secant=secant -+ ) -+ state.prepare_local_span(rank * 8, 8) -+ x = torch.full((8, 512), 2, dtype=torch.bfloat16, device="npu") -+ for step in (0, 1): -+ state.begin_step(step, sigma=1 - step / 10) -+ state.pre_gate_output(0, torch.full_like(x, 0.5 + (step * 0.25 if secant else 0)), "attn") -+ state.pre_gate_output(0, torch.full_like(x, 0.25 + (step * 0.125 if secant else 0)), "mlp") -+ state.end_step() -+ state.begin_step(2, sigma=0.8) -+ out = _subset_prefix_bnsd(exchanged[0], exchanged[1][:, :30], exchanged[2][:, :30], 128**-0.5, state) -+ send, receive = out.reshape(4, 8, 1, 128).contiguous(), torch.empty_like(out.reshape(4, 8, 1, 128)) -+ dist.all_to_all_single(receive, send) -+ local_out = receive.permute(1, 0, 2, 3).reshape(8, 512) -+ expected = _prefix_bnsd(full[0].unsqueeze(0), full[1][:30].unsqueeze(0), full[2][:30].unsqueeze(0), 128**-0.5)[ -+ 0 -+ ].reshape(32, 512) -+ dense = state.local_dense_positions -+ torch.testing.assert_close( -+ local_out.index_select(0, dense).float(), -+ expected[rank * 8 : (rank + 1) * 8].index_select(0, dense).float(), -+ atol=0.03125, -+ rtol=0.02, -+ ) -+ attn = state.pre_gate_output(0, local_out, "attn") -+ indices = torch.zeros(8, dtype=torch.long, device="npu") -+ gate = torch.full((1, 512), 0.5, dtype=torch.float32, device="npu") -+ after_attn = indexed_gate(x, gate, attn, indices) -+ state.record_fresh_gate("attn") -+ mlp = state.pre_gate_output(0, torch.zeros_like(x), "mlp") -+ result = indexed_gate(after_attn, gate, mlp, indices) -+ state.record_fresh_gate("mlp") -+ video = state.local_video_positions -+ assert torch.equal( -+ result.index_select(0, video), -+ torch.full((video.numel(), 512), 2.75 if secant else 2.375, dtype=torch.bfloat16, device="npu"), -+ ) -+ assert dense.numel() == [8, 0, 0, 6][rank] -+ state.end_step() -+ state.close() -+ assert state.stats()["live_cache_bytes"] == 0 -+ torch.npu.synchronize() -+ dist.barrier() -+ finally: -+ dist.destroy_process_group() -+ -+ -+@pytest.mark.skipif(not current_omni_platform.is_npu(), reason="Requires four native Ascend devices") -+def test_native_four_rank_pre_gate_subset_attention_and_current_gates(): -+ torch.multiprocessing.spawn(_native_pre_gate_worker, nprocs=4, join=True) -diff --git a/tests/diffusion/models/minimax_h3/test_minimax_h3_pre_gate_secant.py b/tests/diffusion/models/minimax_h3/test_minimax_h3_pre_gate_secant.py -new file mode 100644 -index 0000000..307723f ---- /dev/null -+++ b/tests/diffusion/models/minimax_h3/test_minimax_h3_pre_gate_secant.py -@@ -0,0 +1,83 @@ -+# SPDX-License-Identifier: Apache-2.0 -+# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project -+ -+import pytest -+import torch -+ -+from tests.diffusion.models.minimax_h3.test_minimax_h3_pre_gate_cache import _native_pre_gate_worker -+from tests.helpers.mark import hardware_marks -+from vllm_omni.diffusion.models.minimax_h3.video_row_cache import MiniMaxH3VideoRowCache -+from vllm_omni.platforms import current_omni_platform -+ -+pytestmark = [pytest.mark.core_model, pytest.mark.diffusion, *hardware_marks(res={"npu": "A3"}, num_cards=4)] -+ -+ -+def state(): -+ cache = MiniMaxH3VideoRowCache( -+ torch.tensor([1, 2]), seq_len=4, used_len=3, num_layers=1, pre_gate=True, secant=True -+ ) -+ cache.prepare_local_span(0, 4) -+ return cache -+ -+ -+@pytest.mark.parametrize("sigma", [None, float("nan"), float("inf"), -0.1, 1.1]) -+def test_secant_requires_explicit_finite_solver_sigma(sigma): -+ cache = state() -+ with pytest.raises(ValueError, match="finite solver sigma"): -+ cache.begin_step(0, sigma=sigma) -+ cache.close() -+ -+ -+def test_secant_cannot_select_gated_block_residual(): -+ with pytest.raises(ValueError, match="pre-gate"): -+ MiniMaxH3VideoRowCache(torch.tensor([1]), seq_len=4, used_len=3, num_layers=1, secant=True) -+ -+ -+def test_two_anchor_forecast_follows_sigma_and_preserves_dense_rows(): -+ cache = state() -+ for step, sigma, values in [(0, 1.0, [4, -8]), (1, 0.5, [2, -4])]: -+ cache.begin_step(step, sigma=sigma) -+ x = torch.tensor([values] * 4, dtype=torch.bfloat16) -+ for stage in ("attn", "mlp"): -+ cache.pre_gate_output(0, x, stage) -+ cache.end_step() -+ assert all(t.dtype == torch.int8 for t in cache._slopes.values()) -+ assert all(t.dtype == torch.float32 for t in cache._slope_scales.values()) -+ assert cache.stats()["live_cache_bytes"] == 40 -+ cache.begin_step(2, sigma=0.25) -+ current = torch.full((4, 2), 100, dtype=torch.bfloat16) -+ saved = current.clone() -+ for stage in ("attn", "mlp"): -+ out = cache.pre_gate_output(0, current, stage) -+ assert torch.equal(out[1:3], torch.tensor([[1, -2], [1, -2]], dtype=torch.bfloat16)) -+ assert torch.equal(out[[0, 3]], current[[0, 3]]) -+ out.zero_() -+ cache.end_step() -+ assert torch.equal(current, saved) -+ assert cache.secant_forecasts == 2 -+ with pytest.raises(ValueError, match="strictly decreasing"): -+ cache.begin_step(3, sigma=0.25) -+ cache.close() -+ assert cache.stats()["live_cache_bytes"] == 0 -+ assert not cache._slopes and not cache._anchor_sigmas -+ -+ -+def test_linear_trajectory_and_zero_slope_cover_fixed_fifty_steps(): -+ cache = state() -+ for step in range(50): -+ cache.begin_step(step, sigma=1 - step / 50) -+ # The first channel drifts; the second channel's exact zero difference -+ # exercises finite scale handling without adding an artificial epsilon. -+ actual = torch.tensor([[1 + step / 8, 2]] * 4, dtype=torch.bfloat16) -+ for stage in ("attn", "mlp"): -+ out = cache.pre_gate_output(0, actual, stage) -+ torch.testing.assert_close(out[1:3].float(), actual[1:3].float(), atol=0.03125, rtol=0) -+ cache.end_step() -+ assert cache.secant_forecasts == 84 -+ assert cache.full_steps == 8 and cache.partial_steps == 42 -+ cache.close() -+ -+ -+@pytest.mark.skipif(not current_omni_platform.is_npu(), reason="Requires four native Ascend devices") -+def test_native_four_rank_sigma_secant_subset_attention_and_current_gates(): -+ torch.multiprocessing.spawn(_native_pre_gate_worker, args=(True,), nprocs=4, join=True) -diff --git a/tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py b/tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py -new file mode 100644 -index 0000000..94ff958 ---- /dev/null -+++ b/tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py -@@ -0,0 +1,406 @@ -+# SPDX-License-Identifier: Apache-2.0 -+# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project -+ -+from datetime import timedelta -+from types import SimpleNamespace -+ -+import pytest -+import torch -+import torch.distributed as dist -+import torch.nn as nn -+ -+from tests.helpers.mark import hardware_marks -+from vllm_omni.diffusion.models.minimax_h3 import minimax_h3_blocks as blocks -+from vllm_omni.diffusion.models.minimax_h3.attention.flash_h3 import _prefix_bnsd, _subset_prefix_bnsd -+from vllm_omni.diffusion.models.minimax_h3.video_row_cache import MiniMaxH3VideoRowCache -+from vllm_omni.platforms import current_omni_platform -+ -+pytestmark = [pytest.mark.core_model, pytest.mark.diffusion, *hardware_marks(res={"npu": "A3"}, num_cards=4)] -+ -+ -+def state_for_rows(device="cpu", start=0, length=4): -+ state = MiniMaxH3VideoRowCache(torch.tensor([1, 2], device=device), seq_len=4, used_len=3, num_layers=1) -+ state.prepare_local_span(start, length) -+ return state -+ -+ -+def anchor(state, x): -+ for step in range(5): -+ state.begin_step(step) -+ state.capture(0, x, x + 0.125) -+ state.end_step() -+ state.begin_step(5) -+ -+ -+def test_fixed_schedule_and_row_ownership_excludes_padding(): -+ assert [i for i in range(50) if MiniMaxH3VideoRowCache.is_full_step(i)] == [ -+ 0, -+ 1, -+ 2, -+ 3, -+ 4, -+ 9, -+ 14, -+ 19, -+ 24, -+ 29, -+ 34, -+ 39, -+ 44, -+ 46, -+ 47, -+ 48, -+ 49, -+ ] -+ state = state_for_rows() -+ assert state.global_dense_positions.tolist() == [0] -+ assert state.local_dense_positions.tolist() == [0] -+ assert state.local_video_positions.tolist() == [1, 2] -+ with pytest.raises(RuntimeError, match="ownership"): -+ state.prepare_local_span(0, 3) -+ -+ -+@pytest.mark.parametrize("positions", [[1, 1], [2, 1], [-1], [3]]) -+def test_invalid_target_positions_are_rejected(positions): -+ with pytest.raises(ValueError): -+ MiniMaxH3VideoRowCache(torch.tensor(positions), seq_len=4, used_len=3, num_layers=1) -+ -+ -+def test_delta_is_fp32_and_restore_owns_output(): -+ state = state_for_rows() -+ x = torch.tensor([[1, 2], [3, 4], [5, 6], [7, 8]], dtype=torch.bfloat16) -+ anchor(state, x) -+ assert state._deltas[0].dtype == torch.float32 -+ before = x.clone() -+ output = state.restore(0, x) -+ assert torch.equal(output[1:3], x[1:3] + 0.125) -+ assert torch.equal(output[[0, 3]], x[[0, 3]]) -+ output.zero_() -+ assert torch.equal(x, before) -+ state.end_step() -+ state.close() -+ assert state.closed and not state._deltas and state.global_dense_positions.numel() == 0 -+ -+ -+def large_delta_roundtrip(device="cpu"): -+ state = state_for_rows(device=device) -+ x = torch.full((4, 2), 100000, dtype=torch.bfloat16, device=device) -+ expected = -x -+ delta = expected.float() - x.float() -+ assert torch.isfinite(delta).all() -+ assert not torch.isfinite(delta.to(torch.float16)).all() -+ for step in range(5): -+ state.begin_step(step) -+ state.capture(0, x, expected) -+ state.end_step() -+ state.begin_step(5) -+ actual = state.restore(0, x) -+ assert torch.isfinite(actual).all() -+ assert torch.equal(actual[1:3].cpu().view(torch.uint8), expected[1:3].cpu().view(torch.uint8)) -+ assert torch.equal(actual[[0, 3]], x[[0, 3]]) -+ state.end_step() -+ state.close() -+ assert state.stats()["live_cache_bytes"] == 0 -+ -+ -+def test_large_finite_bf16_delta_roundtrip_exceeds_fp16_storage_range(): -+ large_delta_roundtrip() -+ -+ -+class CountMLP(nn.Module): -+ def __init__(self): -+ super().__init__() -+ self.rows = [] -+ -+ def forward(self, x): -+ self.rows.append(x.shape[0]) -+ return x + 0.25 -+ -+ -+def tiny_block(): -+ block = object.__new__(blocks.MiniMaxH3DiTBlock) -+ nn.Module.__init__(block) -+ block.layer_index = 0 -+ block.norm1 = SimpleNamespace(weight=torch.ones(2), variance_epsilon=1e-6) -+ block.norm2 = SimpleNamespace(weight=torch.ones(2), variance_epsilon=1e-6) -+ block.adaln_proj = lambda t: ( -+ torch.zeros(3, 2), -+ torch.zeros(3, 2), -+ torch.ones(3, 2), -+ torch.zeros(3, 2), -+ torch.zeros(3, 2), -+ torch.ones(3, 2), -+ ) -+ block.attn = lambda x, **kwargs: x * 0.5 -+ block.mlp = CountMLP() -+ return block -+ -+ -+@pytest.mark.parametrize("start,length", [(0, 4), (1, 2)]) -+def test_real_block_dense_path_and_partial_mlp_empty_rows(start, length): -+ state = state_for_rows(start=start, length=length) -+ block = tiny_block() -+ x = torch.arange(length * 2).reshape(length, 2).to(torch.bfloat16) -+ before = x.clone() -+ kwargs = dict( -+ t_emb=torch.zeros(1), -+ combined_indices=torch.zeros(length, dtype=torch.long), -+ rope_table=None, -+ cu_seqlens=torch.tensor([0, 3, 4], dtype=torch.int32), -+ max_seqlen=3, -+ packed_total=4, -+ ) -+ expected = block(x, **kwargs) -+ for step in range(5): -+ state.begin_step(step) -+ actual = block(x, video_row_cache=state, **kwargs) -+ assert torch.equal(actual.view(torch.uint8), expected.view(torch.uint8)) -+ state.end_step() -+ block.mlp.rows.clear() -+ state.begin_step(5) -+ actual = block(x, video_row_cache=state, **kwargs) -+ state.end_step() -+ rows = state.local_dense_positions -+ assert block.mlp.rows == ([rows.numel()] if rows.numel() else []) -+ assert torch.equal(actual.index_select(0, rows), expected.index_select(0, rows)) -+ assert torch.equal(x, before) -+ -+ -+@pytest.mark.parametrize("failure", ["success", "model", "cancel"]) -+def test_request_state_closed_and_removed_on_every_exit(monkeypatch, failure): -+ from vllm_omni.diffusion.models.minimax_h3 import denoise_loop as loop -+ -+ monkeypatch.setenv("H3_VIDEO_ROW_CACHE", "1") -+ monkeypatch.setattr(loop.current_omni_platform, "is_npu", lambda: False) -+ monkeypatch.setattr(loop, "minimax_h3_publish_denoise_progress", lambda *args: None) -+ state = MiniMaxH3VideoRowCache(torch.tensor([1]), seq_len=4, used_len=3, num_layers=1) -+ -+ class Model: -+ def create_video_row_cache(self, *args, **kwargs): -+ return state -+ -+ def __call__(self, **kwargs): -+ assert kwargs["video_row_cache"] is state -+ if failure == "model": -+ raise RuntimeError("model failed") -+ state.prepare_local_span(0, 4) -+ x = torch.ones(4, 2, dtype=torch.bfloat16) -+ if state.partial: -+ state.restore(0, x) -+ else: -+ state.capture(0, x, x + 0.125) -+ return torch.zeros(2, 96), torch.zeros(1, 32) -+ -+ class Branch: -+ task = "ref2va" -+ seq_len = 4 -+ used_len = 3 -+ img_pos = torch.tensor([0, 1]) -+ update_mask = torch.tensor([False, True]) -+ update_mask_dev = update_mask -+ audio_update_mask_dev = torch.tensor([True]) -+ locked_audio_rows = None -+ static_kwargs = {"video_token_layout": SimpleNamespace(video_spans=[SimpleNamespace(role="target", start=1)])} -+ -+ def forward_kwargs(self, **kwargs): -+ return dict(self.static_kwargs) -+ -+ monkeypatch.setattr( -+ loop, -+ "minimax_h3_prepare_denoise_rows", -+ lambda **kwargs: (kwargs["initial_video_rows"], kwargs["initial_audio_rows"], torch.zeros(1, 96), None), -+ ) -+ -+ def cancellation(**kwargs): -+ if failure == "cancel": -+ raise RuntimeError("request cancelled") -+ -+ monkeypatch.setattr(loop, "check_request_cancellation", cancellation) -+ branch, model = Branch(), Model() -+ kwargs = dict( -+ model=model, -+ positive=branch, -+ initial_video_rows=torch.zeros(2, 96), -+ initial_audio_rows=torch.zeros(1, 32), -+ keyframe_cond_rows=torch.zeros(1, 96), -+ sigmas_video=[1 - i / 50 for i in range(51)], -+ sigmas_audio=[1 - i / 50 for i in range(51)], -+ device=torch.device("cpu"), -+ ) -+ if failure == "success": -+ loop.minimax_h3_denoise_loop(**kwargs) -+ assert state.full_steps == 17 and state.partial_steps == 33 -+ else: -+ with pytest.raises(RuntimeError): -+ loop.minimax_h3_denoise_loop(**kwargs) -+ assert state.closed and not state._deltas and "video_row_cache" not in branch.static_kwargs -+ assert model._video_row_cache_stats["closed"] -+ -+ -+def test_default_factory_does_not_create_state(monkeypatch): -+ from vllm_omni.diffusion.models.minimax_h3.minimax_h3_transformer import MiniMaxH3DiTModel -+ -+ monkeypatch.delenv("H3_VIDEO_ROW_CACHE", raising=False) -+ assert ( -+ MiniMaxH3DiTModel.create_video_row_cache(None, torch.tensor([1]), seq_len=4, used_len=3, num_steps=50) is None -+ ) -+ -+ -+@pytest.mark.parametrize("ref2va", [False, True]) -+def test_real_packing_task_selects_ref2va_and_rejects_fl2va(monkeypatch, ref2va): -+ from vllm_omni.diffusion.models.minimax_h3 import denoise_loop as loop -+ from vllm_omni.diffusion.models.minimax_h3.packed_sequence import ( -+ minimax_h3_packed_sequence, -+ minimax_h3_packed_sequence_ref2va_blocks, -+ ) -+ -+ common = dict(text_len=2, latent_t=2, latent_h=4, latent_w=4, audio_t=2) -+ packed = ( -+ minimax_h3_packed_sequence_ref2va_blocks(**common, ref_blocks=[{"kind": "image", "latent_h": 4, "latent_w": 4}]) -+ if ref2va -+ else minimax_h3_packed_sequence(**common, include_keyframe_cond=True, keyframe_frame_indices=[0], frame_count=5) -+ ) -+ branch = loop.MiniMaxH3DenoiseBranch( -+ packed=packed, -+ text_embeddings=torch.zeros(2, 5120), -+ token_tags=packed["token_tags"], -+ device=torch.device("cpu"), -+ task="ref2va" if ref2va else "fl2va", -+ ) -+ layout = branch.static_kwargs["video_token_layout"] -+ assert not any(span.role == "reference" for span in layout.video_spans) -+ monkeypatch.setenv("H3_VIDEO_ROW_CACHE", "1") -+ monkeypatch.setattr(loop.current_omni_platform, "is_npu", lambda: False) -+ monkeypatch.setattr(loop, "minimax_h3_publish_denoise_progress", lambda *args: None) -+ monkeypatch.setattr(loop, "check_request_cancellation", lambda **kwargs: None) -+ states = [] -+ -+ class Model: -+ def create_video_row_cache(self, positions, **kwargs): -+ state = MiniMaxH3VideoRowCache( -+ positions, seq_len=kwargs["seq_len"], used_len=kwargs["used_len"], num_layers=1 -+ ) -+ state.prepare_local_span(0, branch.seq_len) -+ states.append(state) -+ return state -+ -+ def __call__(self, **kwargs): -+ state = kwargs["video_row_cache"] -+ x = torch.ones(branch.seq_len, 2, dtype=torch.bfloat16) -+ if state.partial: -+ state.restore(0, x) -+ else: -+ state.capture(0, x, x + 0.125) -+ return torch.zeros(branch.img_pos.numel(), 96), torch.zeros(branch.audio_pos.numel(), 32) -+ -+ kwargs = dict( -+ model=Model(), -+ positive=branch, -+ initial_video_rows=torch.zeros(branch.img_pos.numel(), 96), -+ initial_audio_rows=torch.zeros(branch.audio_pos.numel(), 32), -+ keyframe_cond_rows=torch.zeros(int((~branch.update_mask).sum()), 96), -+ sigmas_video=[1 - i / 50 for i in range(51)], -+ sigmas_audio=[1 - i / 50 for i in range(51)], -+ device=torch.device("cpu"), -+ ) -+ if ref2va: -+ loop.minimax_h3_denoise_loop(**kwargs) -+ assert len(states) == 1 and states[0].closed -+ assert states[0].full_steps == 17 and states[0].partial_steps == 33 -+ else: -+ with pytest.raises(ValueError, match="requires Ref2VA"): -+ loop.minimax_h3_denoise_loop(**kwargs) -+ assert not states -+ -+ -+@pytest.mark.parametrize("start,length", [(0, 4), (1, 2)]) -+def test_attention_keeps_collective_path_and_projects_only_dense_local_rows(start, length): -+ state = state_for_rows(start=start, length=length) -+ x = torch.ones(length, 2, dtype=torch.bfloat16) -+ anchor(state, x) -+ attention = object.__new__(blocks.MiniMaxH3Attention) -+ nn.Module.__init__(attention) -+ attention.num_heads = 1 -+ attention.head_dim = 2 -+ attention.vdn = None -+ attention.to_gate_compress = None -+ calls, projected_rows = [], [] -+ attention._project_qkv = lambda *args: (x.reshape(length, 1, 2),) * 5 -+ -+ def packed(q, k, v, **kwargs): -+ calls.append(q.shape[0]) -+ assert kwargs["video_row_cache"] is state -+ return q -+ -+ def project(rows): -+ projected_rows.append(rows.shape[0]) -+ return rows * 2, None -+ -+ attention._run_packed_attention = packed -+ attention.out_proj = project -+ output = attention( -+ x, -+ rope_table=None, -+ cu_seqlens=torch.tensor([0, 3, 4], dtype=torch.int32), -+ max_seqlen=3, -+ packed_total=4, -+ video_row_cache=state, -+ ) -+ rows = state.local_dense_positions -+ assert calls == [length] -+ assert projected_rows == ([rows.numel()] if rows.numel() else []) -+ assert torch.equal(output.index_select(0, rows), (x * 2).index_select(0, rows)) -+ assert torch.count_nonzero(output.index_select(0, state.local_video_positions)) == 0 -+ -+ -+def _native_worker(rank): -+ import torch_npu # noqa: F401 -+ -+ torch.npu.set_device(rank) -+ dist.init_process_group( -+ "hccl", init_method="tcp://127.0.0.1:29597", rank=rank, world_size=4, timeout=timedelta(seconds=120) -+ ) -+ try: -+ large_delta_roundtrip(device=f"npu:{rank}") -+ torch.manual_seed(19) -+ full = [torch.randn(32, 4, 128, dtype=torch.bfloat16).npu() for _ in range(3)] -+ exchanged = [] -+ for tensor in full: -+ local = tensor[rank * 8 : (rank + 1) * 8] -+ send = local.reshape(8, 4, 1, 128).permute(1, 0, 2, 3).contiguous() -+ receive = torch.empty_like(send) -+ dist.all_to_all_single(receive, send) -+ exchanged.append(receive.reshape(1, 32, 1, 128)) -+ state = MiniMaxH3VideoRowCache(torch.arange(8, 24, device="npu"), seq_len=32, used_len=30, num_layers=1) -+ state.prepare_local_span(rank * 8, 8) -+ x = torch.ones(8, 2, dtype=torch.bfloat16, device="npu") -+ anchor(state, x) -+ output = _subset_prefix_bnsd(exchanged[0], exchanged[1][:, :30], exchanged[2][:, :30], 128**-0.5, state) -+ send = output.reshape(4, 8, 1, 128).contiguous() -+ receive = torch.empty_like(send) -+ dist.all_to_all_single(receive, send) -+ local_output = receive.permute(1, 0, 2, 3).reshape(8, 4, 128) -+ expected = _prefix_bnsd(full[0].unsqueeze(0), full[1][:30].unsqueeze(0), full[2][:30].unsqueeze(0), 128**-0.5)[ -+ 0 -+ ] -+ rows = state.local_dense_positions -+ torch.testing.assert_close( -+ local_output.index_select(0, rows).float(), -+ expected[rank * 8 : (rank + 1) * 8].index_select(0, rows).float(), -+ atol=0.03125, -+ rtol=0.02, -+ ) -+ assert state.subset_attention_calls == 1 and state.subset_query_rows == 14 -+ assert rows.numel() == [8, 0, 0, 6][rank] -+ state.restore(0, x) -+ state.end_step() -+ state.close() -+ torch.npu.synchronize() -+ dist.barrier() -+ finally: -+ dist.destroy_process_group() -+ -+ -+@pytest.mark.skipif(not current_omni_platform.is_npu(), reason="Requires four native Ascend devices") -+def test_native_four_rank_subset_attention_with_empty_local_dense_rows(): -+ torch.multiprocessing.spawn(_native_worker, nprocs=4, join=True) -diff --git a/vllm_omni/config/omni_config.py b/vllm_omni/config/omni_config.py -index 0a803cf..44fc48c 100644 ---- a/vllm_omni/config/omni_config.py -+++ b/vllm_omni/config/omni_config.py -@@ -863,6 +863,7 @@ class _DiffusionConfigProjection(_TrackExplicitConfigFields): - dlo_use_allgather: bool = True - dlo_chunk_size_mb: int = Field(default=64, gt=0, strict=True) - dlo_resident_layers: int = Field(default=0, ge=0) -+ dlo_order_allgather_after_attn_a2a: bool = False - host_weight_runtime_mode: Literal["disabled", "preferred", "required"] = "disabled" - host_weight_runtime_root: str | None = None - dlo_host_registration_limit_gib: float = Field(default=0.0, ge=0) -diff --git a/vllm_omni/config/stage_config.py b/vllm_omni/config/stage_config.py -index 76a58a9..3c69986 100644 ---- a/vllm_omni/config/stage_config.py -+++ b/vllm_omni/config/stage_config.py -@@ -672,6 +672,7 @@ class StageDeployConfig: - dlo_use_allgather: bool | None = None - dlo_chunk_size_mb: int | None = None - dlo_resident_layers: int | None = None -+ dlo_order_allgather_after_attn_a2a: bool | None = None - host_weight_runtime_mode: str | None = None - host_weight_runtime_root: str | None = None - dlo_host_registration_limit_gib: float | None = None -diff --git a/vllm_omni/diffusion/attention/layer.py b/vllm_omni/diffusion/attention/layer.py -index 5520390..574eafa 100644 ---- a/vllm_omni/diffusion/attention/layer.py -+++ b/vllm_omni/diffusion/attention/layer.py -@@ -9,7 +9,7 @@ - - from collections.abc import Mapping - from dataclasses import replace --from typing import cast -+from typing import Any, cast - - import torch - import torch.nn as nn -@@ -333,6 +333,11 @@ class Attention(nn.Module): - # Local strategy when SP is intentionally inactive outside sharded regions. - self._no_parallel_strategy = NoParallelAttention() - -+ # Optional cross-module rendezvous (see distributed/collective_gate.py). -+ # Installed by the DLO backend when it must keep its background weight -+ # AllGather from taking the links ahead of this layer's QKV AllToAll. -+ self._collective_gate: Any | None = None -+ - self.layer_idx: int | None = _try_extract_layer_index(prefix) - - self._kv_cache_dtype: str | None = None -@@ -506,10 +511,12 @@ class Attention(nn.Module): - reason = self._cross_backend_limitation(attn_metadata) - if reason is not None: - blocked = self._cross_fallback_blocker(attn_metadata) -- result = self.sdpa_fallback.resolve_execution_path(resolved_context, query, key, value, attn_metadata) -+ result = cast(SDPAImpl, self.sdpa_fallback).resolve_execution_path( -+ resolved_context, query, key, value, attn_metadata -+ ) - return replace( - result, -- backend=self.attn_backend.get_name() if blocked else "SDPA", -+ backend=cast(type[AttentionBackend], self.attn_backend).get_name() if blocked else "SDPA", - path="cross_unsupported" if blocked else "cross_sdpa_fallback", - support=CapabilityResult.unsupported(blocked) if blocked else CapabilityResult.unmigrated(reason), - ) -@@ -591,6 +598,12 @@ class Attention(nn.Module): - # For Ring: Concat joint_q - query, key, value, attn_metadata, ctx = strategy.pre_attention(query, key, value, attn_metadata) - -+ # The QKV AllToAll is now enqueued on this stream. Release any gather -+ # that was deferred to give this collective the links first. -+ gate = self._collective_gate -+ if gate is not None: -+ gate.signal() -+ - # MiniMax-H3 Tier2 stores reference K/V in the balanced layout produced - # by Ulysses. Keeping this hook at the generic post-parallel boundary - # avoids a model import in the parallel strategy and leaves other models -@@ -760,7 +773,7 @@ class Attention(nn.Module): - def _cross_fallback_blocker(self, attn_metadata: AttentionMetadata | None) -> str | None: - if self.backend_explicit: - return ( -- f"Explicit backend {self.attn_backend.get_name()} does not support " -+ f"Explicit backend {cast(type[AttentionBackend], self.attn_backend).get_name()} does not support " - f"{self._cross_backend_limitation(attn_metadata)}" - ) - if attn_metadata is not None and attn_metadata.extra.get("kv_cache_dtype") not in (None, "float"): -@@ -793,7 +806,7 @@ class Attention(nn.Module): - blocked = self._cross_fallback_blocker(attn_metadata) - if blocked is not None: - raise NotImplementedError(blocked) -- return self.sdpa_fallback.forward(query, key, value, attn_metadata) -+ return cast(SDPAImpl, self.sdpa_fallback).forward(query, key, value, attn_metadata) - dense_mask = attn_metadata.attn_mask if attn_metadata is not None else None - # Piecewise backends honor full_attn_spans ahead of attn_mask and never read the - # dense mask (e.g. HunyuanImage3, which sends both), so there is nothing to divert. -diff --git a/vllm_omni/diffusion/data.py b/vllm_omni/diffusion/data.py -index 8528574..fcbeec0 100644 ---- a/vllm_omni/diffusion/data.py -+++ b/vllm_omni/diffusion/data.py -@@ -962,6 +962,10 @@ class OmniDiffusionConfig: - dlo_chunk_size_mb: int = 64 - # Leading main-DiT blocks kept resident by distributed layerwise offload. - dlo_resident_layers: int = 0 -+ # Hold each DiT block's weight AllGather until that block's QKV AllToAll has -+ # been enqueued, so the background collective does not take the links ahead -+ # of the critical-path all-to-all. -+ dlo_order_allgather_after_attn_a2a: bool = False - # Final-layout Host Weight Runtime policy. The loader only activates this - # for eligible no-AllGather DLO; all other configurations preserve their - # existing loader/storage path. -diff --git a/vllm_omni/diffusion/distributed/collective_gate.py b/vllm_omni/diffusion/distributed/collective_gate.py -new file mode 100644 -index 0000000..1386d89 ---- /dev/null -+++ b/vllm_omni/diffusion/distributed/collective_gate.py -@@ -0,0 +1,72 @@ -+# SPDX-License-Identifier: Apache-2.0 -+# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project -+"""Cross-module rendezvous that orders background collectives after the -+foreground all-to-all of a transformer block. -+ -+Distributed layerwise offload (DLO) issues the *next* block's weight AllGather -+on a dedicated ``comm_stream`` while the *current* block computes. Both that -+AllGather and the block's QKV sequence-parallel AllToAll are collectives over -+the same ranks, so whichever reaches the links first wins: the AllGather is -+submitted at block entry, well before the block reaches its attention, and the -+critical-path AllToAll ends up queued behind it. -+ -+A plain event wait cannot fix this by itself. ``wait_event`` snapshots the -+records enqueued *before* the call, so "enqueue the wait at block entry, record -+the event later in the block" is a no-op -- the wait resolves against the -+previous iteration's record. The gather therefore has to be *issued* after the -+AllToAll: the hook arms this barrier at block entry and defers the gather, the -+attention signals it right after ``strategy.pre_attention``, and the signal -+records the event and lets the hook submit the gather gated on it. -+ -+The gather still overlaps the rest of the block (attention output, MLP, the -+next block's projections); it simply cannot start before the AllToAll finished. -+""" -+ -+from __future__ import annotations -+ -+from collections.abc import Callable -+ -+__all__ = ["QKVAllToAllBarrier"] -+ -+ -+class QKVAllToAllBarrier: -+ """One-block rendezvous between a DLO hook and the block's attention. -+ -+ Owned by a single :class:`DistributedLayerwiseOffloadHook` (one per -+ streaming block) and shared with the attention layer inside that block. -+ Both live on the same block, so the arming and the signalling always come -+ from the same module forward. -+ -+ Args: -+ on_signal: Invoked at most once per :meth:`arm` when the block's -+ attention reports its QKV AllToAll. It must submit the deferred -+ weight AllGather. -+ """ -+ -+ def __init__(self, on_signal: Callable[[], None]) -> None: -+ self._on_signal = on_signal -+ self._armed = False -+ -+ @property -+ def armed(self) -> bool: -+ """True between :meth:`arm` and the matching :meth:`signal`.""" -+ return self._armed -+ -+ def arm(self) -> None: -+ """Mark the next attention QKV AllToAll as the gather's gate.""" -+ self._armed = True -+ -+ def disarm(self) -> None: -+ """Drop a pending arm without firing it (the caller flushes instead).""" -+ self._armed = False -+ -+ def signal(self) -> None: -+ """Report that the QKV AllToAll is now enqueued on the compute stream. -+ -+ A no-op unless armed, so it is cheap to call unconditionally from the -+ attention hot path. -+ """ -+ if not self._armed: -+ return -+ self._armed = False -+ self._on_signal() -diff --git a/vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py b/vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py -index 6ebfb10..044c1f0 100644 ---- a/vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py -+++ b/vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py -@@ -36,6 +36,16 @@ def _prefix_bnsd(query: torch.Tensor, key: torch.Tensor, value: torch.Tensor, sc - return output.reshape(batch, heads, rows, dim).transpose(1, 2).contiguous() - - -+def _subset_prefix_bnsd(query, key, value, scale, state): -+ """Compute only fresh conditioning queries; preserve the shared SP shape.""" -+ rows = state.global_dense_positions -+ dense = _prefix_bnsd(query.index_select(1, rows), key, value, scale) -+ output = torch.zeros_like(query) -+ output.index_copy_(1, rows, dense) -+ state.record_subset_attention(rows.numel()) -+ return output -+ -+ - class MiniMaxH3FlashAttentionImpl(FlashAttentionImpl): - """Use independent head batches for explicit long BF16 prefix attention. - -@@ -51,7 +61,10 @@ class MiniMaxH3FlashAttentionImpl(FlashAttentionImpl): - value: torch.Tensor, - attn_metadata: AttentionMetadata | None = None, - ) -> torch.Tensor: -+ state = attn_metadata.extra.get("minimax_h3_video_row_cache") if attn_metadata is not None else None - if os.environ.get("MINDIE_SD_FA_TYPE") != "fused_attn_score" or attn_metadata is None: -+ if state is not None: -+ raise RuntimeError("H3 video-row cache requires the explicit fused attention path") - return super().forward_fa_npu(query, key, value, attn_metadata) - extra = attn_metadata.extra - if ( -@@ -68,9 +81,17 @@ class MiniMaxH3FlashAttentionImpl(FlashAttentionImpl): - or query.shape[2:] != value.shape[2:] - or key.shape[1] != value.shape[1] - ): -+ if state is not None: -+ raise RuntimeError("H3 video-row cache received unsupported fused attention inputs") - return super().forward_fa_npu(query, key, value, attn_metadata) - resolved = self._resolve_packed_seq_npu(query, key, extra) - if resolved is None: -+ if state is not None: -+ raise RuntimeError("H3 video-row cache requires a resolved packed prefix") - return super().forward_fa_npu(query, key, value, attn_metadata) - used_k = resolved[1][0] -+ if state is not None: -+ if not state.partial or used_k != state.used_len or query.shape[1] != state.seq_len: -+ raise RuntimeError("H3 video-row cache attention layout changed") -+ return _subset_prefix_bnsd(query, key[:, :used_k], value[:, :used_k], self.softmax_scale, state) - return _prefix_bnsd(query, key[:, :used_k], value[:, :used_k], self.softmax_scale) -diff --git a/vllm_omni/diffusion/models/minimax_h3/denoise_loop.py b/vllm_omni/diffusion/models/minimax_h3/denoise_loop.py -index 8cca219..eef175b 100644 ---- a/vllm_omni/diffusion/models/minimax_h3/denoise_loop.py -+++ b/vllm_omni/diffusion/models/minimax_h3/denoise_loop.py -@@ -9,6 +9,7 @@ rows stay pinned to their noised step-0 anchors. - - from __future__ import annotations - -+import os - from collections.abc import Callable - from contextlib import AbstractContextManager, nullcontext - from typing import Any -@@ -69,7 +70,9 @@ class MiniMaxH3DenoiseBranch: - text_embeddings: torch.Tensor, - token_tags: torch.Tensor, - device: torch.device, -+ task: str | None = None, - ) -> None: -+ self.task = task - seq_len = int(packed["seq_len"]) - self.seq_len = seq_len - self.img_pos = packed["img_pos"].view(-1).to(torch.long) -@@ -380,6 +383,24 @@ def minimax_h3_denoise_loop( - positive.static_kwargs["reference_kv_state"] = reference_kv_state - - try: -+ video_row_cache = None -+ create_video_cache = getattr(model, "create_video_row_cache", None) -+ if os.environ.get("H3_VIDEO_ROW_CACHE") == "1": -+ if ( -+ not callable(create_video_cache) -+ or video_edit is not None -+ or audio_edit is not None -+ or keyframe_cond_rows is None -+ or positive.task != "ref2va" -+ ): -+ raise ValueError("H3 video-row cache requires Ref2VA image references without editing") -+ video_row_cache = create_video_cache( -+ positive.img_pos[positive.update_mask].to(device), -+ seq_len=positive.seq_len, -+ used_len=positive.used_len, -+ num_steps=num_steps, -+ ) -+ positive.static_kwargs["video_row_cache"] = video_row_cache - for step in range(num_steps): - check_request_cancellation() - step_cm = step_profiler(step) if step_profiler is not None else nullcontext() -@@ -388,6 +409,8 @@ def minimax_h3_denoise_loop( - s_a = sigmas_audio[step] - if reference_kv_state is not None: - reference_kv_state.begin_step(step, sigma=float(s_v)) -+ if video_row_cache is not None: -+ video_row_cache.begin_step(step, sigma=float(s_v)) - # Publish where we are so step-gated attention features (the dense - # warmup of RAINFUSION_ATTN, the timestep gate of TRTLLM_ATTN) can - # see it. Gates use the scheduler-style descending timestep, which -@@ -430,6 +453,8 @@ def minimax_h3_denoise_loop( - v_video, v_audio = model(**fk) - if reference_kv_state is not None: - reference_kv_state.end_step() -+ if video_row_cache is not None: -+ video_row_cache.end_step() - mv_video_t = v_video.float()[update] - mv_audio_t = v_audio.float()[audio_update] - -@@ -476,6 +501,10 @@ def minimax_h3_denoise_loop( - - finally: - minimax_h3_publish_denoise_progress(None, None, None) -+ if video_row_cache is not None: -+ positive.static_kwargs.pop("video_row_cache", None) -+ video_row_cache.close() -+ model._video_row_cache_stats = video_row_cache.stats() - if reference_kv_state is not None: - positive.static_kwargs.pop("reference_kv_state", None) - reference_kv_state.close() -diff --git a/vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py b/vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py -index 09f2e7d..601e2ae 100644 ---- a/vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py -+++ b/vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py -@@ -45,6 +45,7 @@ if TYPE_CHECKING: - from vllm.model_executor.layers.quantization.base_config import QuantizationConfig - - from .reference_kv_cache import MiniMaxH3ReferenceKVTier2State -+ from .video_row_cache import MiniMaxH3VideoRowCache - - _BF16_DTYPE = torch.bfloat16 - _FP32_DTYPE = torch.float32 -@@ -291,6 +292,7 @@ class MiniMaxH3Attention(nn.Module): - reference_kv_layer_index: int | None = None, - reference_kv_compact: bool = False, - vdn_window: VDNLayout | None = None, -+ video_row_cache: MiniMaxH3VideoRowCache | None = None, - ) -> torch.Tensor: - """Run packed attention as a small eager island. - -@@ -368,6 +370,8 @@ class MiniMaxH3Attention(nn.Module): - "vsa_h3_ref_policy": self.vsa_ref_policy, - "vsa_h3_ref_keep_rate": self.vsa_ref_keep_rate, - } -+ if video_row_cache is not None and video_row_cache.partial: -+ extra["minimax_h3_video_row_cache"] = video_row_cache - if gate_compress is not None: - extra["gate_compress"] = gate_compress.unsqueeze(0) - if video_layout is not None and video_layout.video_spans: -@@ -455,6 +459,7 @@ class MiniMaxH3Attention(nn.Module): - reference_kv_layer_index: int | None = None, - reference_kv_compact: bool = False, - vdn_window: VDNLayout | None = None, -+ video_row_cache: MiniMaxH3VideoRowCache | None = None, - ) -> torch.Tensor: - """x: [T, hidden] packed thd rows -> [T, hidden]. - -@@ -508,10 +513,19 @@ class MiniMaxH3Attention(nn.Module): - reference_kv_layer_index=reference_kv_layer_index, - reference_kv_compact=reference_kv_compact, - vdn_window=vdn_window, -+ **({"video_row_cache": video_row_cache} if video_row_cache is not None else {}), - ) - if self.vdn is not None: - out = self.vdn.gate_softmax(out, x) - out = out.reshape(total, self.num_heads * self.head_dim) -+ if video_row_cache is not None and video_row_cache.partial: -+ rows = video_row_cache.local_dense_positions -+ assert rows is not None -+ projected = x.new_zeros((total, x.shape[-1])) -+ if rows.numel(): -+ dense, _ = self.out_proj(out.index_select(0, rows)) -+ projected.index_copy_(0, rows, dense) -+ return projected - out, _ = self.out_proj(out) - if self.vdn is not None: - out = self.vdn.add_linear(out, x, q_raw, k_raw, v, vdn_window) -@@ -742,6 +756,7 @@ class MiniMaxH3DiTBlock(nn.Module): - reference_kv_state: MiniMaxH3ReferenceKVTier2State | None = None, - reference_kv_compact: bool = False, - vdn_window: VDNLayout | None = None, -+ video_row_cache: MiniMaxH3VideoRowCache | None = None, - ) -> torch.Tensor: - """x: [T, H]; t_emb: [M, t_dim]; combined_indices: [T] - (= inverse_indices * modality_num + token_tags.clamp(min=0)). -@@ -759,6 +774,7 @@ class MiniMaxH3DiTBlock(nn.Module): - gate_mlp, - ) = self.adaln_proj(t_emb) - -+ block_input = x - residual = x - h = rms_norm_indexed_scale_shift( - x, -@@ -782,7 +798,10 @@ class MiniMaxH3DiTBlock(nn.Module): - reference_kv_layer_index=self.layer_index, - reference_kv_compact=reference_kv_compact, - vdn_window=vdn_window, -+ **({"video_row_cache": video_row_cache} if video_row_cache is not None else {}), - ) -+ if video_row_cache is not None and video_row_cache.pre_gate: -+ h = video_row_cache.pre_gate_output(self.layer_index, h, "attn") - x, h = indexed_gate_rms_norm_scale_shift( - residual, - gate_msa, -@@ -793,6 +812,36 @@ class MiniMaxH3DiTBlock(nn.Module): - combined_indices, - self.norm2.variance_epsilon, - ) -+ if video_row_cache is not None and video_row_cache.pre_gate and video_row_cache.partial: -+ video_row_cache.record_fresh_gate("attn") - residual = x -+ if video_row_cache is not None and video_row_cache.partial: -+ rows = video_row_cache.local_dense_positions -+ assert rows is not None -+ if video_row_cache.pre_gate: -+ mlp_output = torch.zeros_like(h) -+ if rows.numel(): -+ mlp_output.index_copy_(0, rows, self.mlp(h.index_select(0, rows))) -+ mlp_output = video_row_cache.pre_gate_output(self.layer_index, mlp_output, "mlp") -+ output = indexed_gate(residual, gate_mlp, mlp_output, combined_indices) -+ video_row_cache.record_fresh_gate("mlp") -+ video_row_cache.partial_mlp_rows += rows.numel() -+ return output -+ output = video_row_cache.restore(self.layer_index, block_input) -+ if rows.numel(): -+ dense = indexed_gate( -+ residual.index_select(0, rows), -+ gate_mlp, -+ self.mlp(h.index_select(0, rows)), -+ combined_indices.index_select(0, rows), -+ ) -+ output.index_copy_(0, rows, dense) -+ video_row_cache.partial_mlp_rows += rows.numel() -+ return output - h = self.mlp(h) -- return indexed_gate(residual, gate_mlp, h, combined_indices) -+ if video_row_cache is not None and video_row_cache.pre_gate: -+ h = video_row_cache.pre_gate_output(self.layer_index, h, "mlp") -+ output = indexed_gate(residual, gate_mlp, h, combined_indices) -+ if video_row_cache is not None and not video_row_cache.pre_gate: -+ video_row_cache.capture(self.layer_index, block_input, output) -+ return output -diff --git a/vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py b/vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py -index 9518828..3953f86 100644 ---- a/vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py -+++ b/vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py -@@ -10,6 +10,7 @@ layout. - from __future__ import annotations - - import math -+import os - from bisect import bisect_left - from collections.abc import Iterable, Mapping - from dataclasses import replace -@@ -66,6 +67,8 @@ if TYPE_CHECKING: - from vllm_omni.diffusion.data import OmniDiffusionConfig - from vllm_omni.diffusion.models.minimax_h3.reference_kv_cache import MiniMaxH3ReferenceKVTier2State - -+ from .video_row_cache import MiniMaxH3VideoRowCache -+ - logger = init_logger(__name__) - - -@@ -245,6 +248,7 @@ _FORWARD_SUPPORTED_KWARGS = frozenset( - "control_rows", - "control_context_scale", - "reference_kv_state", -+ "video_row_cache", - } - ) - -@@ -1232,6 +1236,42 @@ class MiniMaxH3DiTModel(nn.Module): - t_emb = self.time_embedder(unique_timesteps) - return embeddings, t_emb - -+ def create_video_row_cache( -+ self, target_positions: torch.Tensor, *, seq_len: int, used_len: int, num_steps: int -+ ) -> MiniMaxH3VideoRowCache | None: -+ if os.environ.get("H3_VIDEO_ROW_CACHE") != "1": -+ return None -+ p = self.parallel_config -+ if ( -+ not current_omni_platform.is_npu() -+ or num_steps != 50 -+ or p.tensor_parallel_size != 1 -+ or p.data_parallel_size not in (None, 1) -+ or p.cfg_parallel_size != 1 -+ or p.pipeline_parallel_size != 1 -+ or p.ulysses_degree != 4 -+ or p.ring_degree != 1 -+ or p.allgather_degree != 1 -+ or p.ulysses_mode != "strict" -+ or self.od_config.cache_backend not in (None, "none") -+ or self.od_config.kv_offload_config is not None -+ or self.vdn_config is not None -+ or os.environ.get("MINDIE_SD_FA_TYPE") != "fused_attn_score" -+ or any(block.attn.attention.attn_backend.get_name() != "FLASH_ATTN" for block in self.blocks) -+ ): -+ raise ValueError("H3 video-row cache requires dense NPU BF16 Ref2VA with strict Ulysses4, TP/DP/CFG/PP1") -+ from .video_row_cache import MiniMaxH3VideoRowCache -+ -+ self._video_row_cache_stats = None -+ return MiniMaxH3VideoRowCache( -+ target_positions, -+ seq_len=seq_len, -+ used_len=used_len, -+ num_layers=len(self.blocks), -+ pre_gate=os.environ.get("H3_VIDEO_PRE_GATE_CACHE") == "1", -+ secant=os.environ.get("H3_VIDEO_PRE_GATE_SECANT") == "1", -+ ) -+ - def create_reference_kv_state( - self, - *, -@@ -1308,6 +1348,14 @@ class MiniMaxH3DiTModel(nn.Module): - refiner_max = int(self._psp_field(refiner_psp, "refiner_packed_seq_params", "max_seqlen_q")) - video_layout = kwargs.get("video_token_layout") - reference_kv_state = kwargs.get("reference_kv_state") -+ video_row_cache = kwargs.get("video_row_cache") -+ if video_row_cache is not None and ( -+ num_requests != 1 -+ or reference_kv_state is not None -+ or self.vdn_config is not None -+ or kwargs.get("control_rows") is not None -+ ): -+ raise ValueError("H3 video-row cache cannot combine with packed requests, reference KV, VDN or control") - vdn_window = None - if self.vdn_config is not None: - # Text rows lead the packed document. -@@ -1442,6 +1490,11 @@ class MiniMaxH3DiTModel(nn.Module): - if reference_kv_state is not None: - reference_kv_state.set_local_sequence_rows(hidden.shape[0]) - -+ if video_row_cache is not None: -+ if hidden.shape[0] != local_len: -+ raise RuntimeError("H3 video-row cache requires the strict local sequence shard") -+ video_row_cache.prepare_local_span(local_start, local_len) -+ - block_video_layout = video_layout - if compact_reference: - block_video_layout = None -@@ -1505,6 +1558,7 @@ class MiniMaxH3DiTModel(nn.Module): - reference_kv_state=reference_kv_state, - reference_kv_compact=compact_reference, - vdn_window=vdn_window, -+ **({"video_row_cache": video_row_cache} if video_row_cache is not None else {}), - ) - if block_index in hints: - hidden = hidden + hints.pop(block_index) * control_scale -diff --git a/vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py b/vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py -index 476c3cc..ebce8fc 100644 ---- a/vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py -+++ b/vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py -@@ -2115,6 +2115,7 @@ class MiniMaxH3Pipeline( - text_embeddings=text_embeddings, - token_tags=tags, - device=self.device, -+ task=task, - ) - - if control_rows is not None: -diff --git a/vllm_omni/diffusion/models/minimax_h3/video_row_cache.py b/vllm_omni/diffusion/models/minimax_h3/video_row_cache.py -new file mode 100644 -index 0000000..19a550d ---- /dev/null -+++ b/vllm_omni/diffusion/models/minimax_h3/video_row_cache.py -@@ -0,0 +1,291 @@ -+# SPDX-License-Identifier: Apache-2.0 -+# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project -+"""Request-owned, experimental target-video block residuals for H3 Ref2VA.""" -+ -+from __future__ import annotations -+ -+import math -+ -+import torch -+ -+ -+class MiniMaxH3VideoRowCache: -+ """Keep non-video computation dense while reusing video block updates. -+ -+ The first experimental profile is deliberately fixed. No state is shared -+ across requests, and every rank retains all attention collectives. -+ """ -+ -+ def __init__( -+ self, -+ target_positions: torch.Tensor, -+ *, -+ seq_len: int, -+ used_len: int, -+ num_layers: int, -+ pre_gate: bool = False, -+ secant: bool = False, -+ ) -> None: -+ if secant and not pre_gate: -+ raise ValueError("sigma secants require pre-gate outputs") -+ positions = target_positions.detach().cpu().tolist() -+ if not 0 < used_len <= seq_len or not positions or num_layers < 1: -+ raise ValueError("video-row cache requires a nonempty valid packed document") -+ if positions != sorted(set(positions)) or positions[0] < 0 or positions[-1] >= used_len: -+ raise ValueError("target-video positions must be sorted unique valid rows") -+ target_set = set(positions) -+ dense = [i for i in range(used_len) if i not in target_set] -+ if not dense: -+ raise ValueError("video-row cache requires non-video conditioning rows") -+ self.seq_len = seq_len -+ self.used_len = used_len -+ self.num_layers = num_layers -+ self._target_positions = positions -+ self.global_dense_positions = torch.tensor(dense, dtype=torch.long, device=target_positions.device) -+ self.local_dense_positions: torch.Tensor | None = None -+ self.local_video_positions: torch.Tensor | None = None -+ self._local_span: tuple[int, int] | None = None -+ self._deltas: dict[int, torch.Tensor] = {} -+ self.pre_gate = pre_gate -+ self.secant = secant -+ self._sigma: float | None = None -+ self._anchor_sigmas: dict[tuple[str, int], float] = {} -+ self._slope_intervals: dict[tuple[str, int], float] = {} -+ self._slopes: dict[tuple[str, int], torch.Tensor] = {} -+ self._slope_scales: dict[tuple[str, int], torch.Tensor] = {} -+ self.secant_forecasts = 0 -+ self._attention_outputs: dict[int, torch.Tensor] = {} -+ self._mlp_outputs: dict[int, torch.Tensor] = {} -+ self._awaiting_mlp_layer: int | None = None -+ self.fresh_attention_gates = 0 -+ self.fresh_mlp_gates = 0 -+ self._step = -1 -+ self._blocks_this_step = 0 -+ self.partial = False -+ self.closed = False -+ self.full_steps = 0 -+ self.partial_steps = 0 -+ self.full_blocks = 0 -+ self.partial_blocks = 0 -+ self.subset_attention_calls = 0 -+ self.subset_query_rows = 0 -+ self.omitted_query_rows = 0 -+ self.partial_mlp_rows = 0 -+ self.peak_cache_bytes = 0 -+ -+ @staticmethod -+ def is_full_step(step: int) -> bool: -+ if not 0 <= step < 50: -+ raise ValueError("the video-row cache profile requires 50 steps") -+ return step < 4 or step >= 46 or (step - 4) % 5 == 0 -+ -+ def begin_step(self, step: int, *, sigma: float | None = None) -> None: -+ if self.closed or step != self._step + 1: -+ raise RuntimeError("video-row cache step order changed") -+ if self._step >= 0 and self._blocks_this_step != self.num_layers: -+ raise RuntimeError("previous video-row cache step is incomplete") -+ full = self.is_full_step(step) -+ if self.pre_gate: -+ full = step in (0, 1, 4, 9, 19, 29, 39, 49) -+ self.partial = not full -+ if self._awaiting_mlp_layer is not None: -+ raise RuntimeError("previous video pre-gate sublayer is incomplete") -+ complete = ( -+ len(self._attention_outputs) == len(self._mlp_outputs) == self.num_layers -+ if self.pre_gate -+ else len(self._deltas) == self.num_layers -+ ) -+ if self.partial and not complete: -+ raise RuntimeError("partial video step has no complete dense anchor") -+ if self.secant: -+ if sigma is None or not math.isfinite(sigma) or not 0 <= sigma <= 1: -+ raise ValueError("video secant requires a finite solver sigma") -+ if self._sigma is not None and sigma >= self._sigma: -+ raise ValueError("video secant requires strictly decreasing solver sigma") -+ if self.partial and len(self._slopes) != 2 * self.num_layers: -+ raise RuntimeError("video secant requires two complete dense anchors") -+ self._sigma = float(sigma) -+ self._step = step -+ self._blocks_this_step = 0 -+ if self.partial: -+ self.partial_steps += 1 -+ else: -+ self.full_steps += 1 -+ -+ def prepare_local_span(self, start: int, length: int) -> None: -+ if self.closed or start < 0 or length < 1 or start + length > self.seq_len: -+ raise ValueError("invalid local packed span") -+ if self._local_span is not None: -+ if self._local_span != (start, length): -+ raise RuntimeError("video-row cache local ownership changed") -+ return -+ device = self.global_dense_positions.device -+ target = set(self._target_positions) -+ self.local_video_positions = torch.tensor( -+ [i - start for i in self._target_positions if start <= i < start + length], dtype=torch.long, device=device -+ ) -+ self.local_dense_positions = torch.tensor( -+ [i - start for i in range(start, min(start + length, self.used_len)) if i not in target], -+ dtype=torch.long, -+ device=device, -+ ) -+ self._local_span = (start, length) -+ -+ def _visit(self, layer: int, x: torch.Tensor) -> None: -+ if self.closed or self._step < 0 or layer != self._blocks_this_step: -+ raise RuntimeError("video-row cache block order changed") -+ if self._local_span is None or x.shape[0] != self._local_span[1] or x.dtype != torch.bfloat16: -+ raise ValueError("video-row cache requires the prepared BF16 row shard") -+ self._blocks_this_step += 1 -+ -+ def capture(self, layer: int, x: torch.Tensor, output: torch.Tensor) -> None: -+ if self.pre_gate: -+ raise RuntimeError("pre-gate state cannot capture a gated block residual") -+ if self.partial: -+ raise RuntimeError("cannot replace the dense anchor during a partial step") -+ self._visit(layer, x) -+ rows = self.local_video_positions -+ assert rows is not None -+ # BF16 endpoints may have a finite difference outside FP16's range. -+ # Keep that difference in FP32; only the reconstructed block output is -+ # rounded to the original BF16 activation dtype. -+ self._deltas[layer] = output.index_select(0, rows).float() - x.index_select(0, rows).float() -+ self.peak_cache_bytes = max( -+ self.peak_cache_bytes, sum(t.numel() * t.element_size() for t in self._deltas.values()) -+ ) -+ self.full_blocks += 1 -+ -+ def restore(self, layer: int, x: torch.Tensor) -> torch.Tensor: -+ if self.pre_gate: -+ raise RuntimeError("pre-gate state cannot restore a gated block residual") -+ if not self.partial: -+ raise RuntimeError("cannot reuse a video residual on a full step") -+ self._visit(layer, x) -+ rows = self.local_video_positions -+ assert rows is not None -+ # This copy owns the returned full shard, preserving caller input and -+ # unused padding while fresh non-video rows are filled by the block. -+ output = x.clone() -+ if rows.numel(): -+ output.index_copy_(0, rows, (x.index_select(0, rows).float() + self._deltas[layer].float()).to(x.dtype)) -+ self.partial_blocks += 1 -+ return output -+ -+ def _cache_bytes(self) -> int: -+ return sum( -+ tensor.numel() * tensor.element_size() -+ for cache in (self._deltas, self._attention_outputs, self._mlp_outputs, self._slopes, self._slope_scales) -+ for tensor in cache.values() -+ ) -+ -+ def pre_gate_output(self, layer: int, output: torch.Tensor, stage: str) -> torch.Tensor: -+ """Capture or fill only target rows before applying the current gate.""" -+ if not self.pre_gate or stage not in ("attn", "mlp"): -+ raise ValueError("pre-gate output requires the selected attention or MLP stage") -+ if stage == "attn": -+ if self._awaiting_mlp_layer is not None: -+ raise RuntimeError("previous video pre-gate sublayer is incomplete") -+ self._visit(layer, output) -+ self._awaiting_mlp_layer = layer -+ if self.partial: -+ self.partial_blocks += 1 -+ else: -+ self.full_blocks += 1 -+ else: -+ if self.closed or self._awaiting_mlp_layer != layer: -+ raise RuntimeError("video pre-gate MLP order changed") -+ if output.dtype != torch.bfloat16 or output.shape[0] != self._local_span[1]: -+ raise ValueError("video pre-gate output must retain the BF16 local shard") -+ self._awaiting_mlp_layer = None -+ rows = self.local_video_positions -+ assert rows is not None -+ cache = self._attention_outputs if stage == "attn" else self._mlp_outputs -+ key = (stage, layer) -+ if self.partial: -+ restored = output.clone() -+ if self.secant: -+ ratio = (self._sigma - self._anchor_sigmas[key]) / self._slope_intervals[key] -+ predicted = cache[layer].float() + ratio * self._slopes[key].float() * self._slope_scales[key] -+ restored.index_copy_(0, rows, predicted.to(output.dtype)) -+ self.secant_forecasts += 1 -+ else: -+ restored.index_copy_(0, rows, cache[layer]) -+ return restored -+ anchor = output.index_select(0, rows) -+ if self.secant: -+ if layer in cache: -+ delta = anchor.float() - cache[layer].float() -+ if delta.shape[0]: -+ maximum = delta.abs().amax(dim=0, keepdim=True) -+ scale = torch.where(maximum > 0, maximum / 127, torch.ones_like(maximum)) -+ else: -+ scale = torch.ones((1, anchor.shape[1]), dtype=torch.float32, device=anchor.device) -+ self._slopes[key] = (delta / scale).round().clamp(-127, 127).to(torch.int8) -+ self._slope_scales[key] = scale -+ self._slope_intervals[key] = self._sigma - self._anchor_sigmas[key] -+ self._anchor_sigmas[key] = self._sigma -+ cache[layer] = anchor -+ self.peak_cache_bytes = max(self.peak_cache_bytes, self._cache_bytes()) -+ return output -+ -+ def record_fresh_gate(self, stage: str) -> None: -+ if not self.pre_gate or not self.partial or stage not in ("attn", "mlp"): -+ raise RuntimeError("unexpected fresh gate record") -+ if stage == "attn": -+ self.fresh_attention_gates += 1 -+ else: -+ self.fresh_mlp_gates += 1 -+ -+ def record_subset_attention(self, rows: int) -> None: -+ if not self.partial or rows != self.global_dense_positions.numel(): -+ raise RuntimeError("unexpected subset attention query count") -+ self.subset_attention_calls += 1 -+ self.subset_query_rows += rows -+ self.omitted_query_rows += len(self._target_positions) -+ -+ def end_step(self) -> None: -+ if self._blocks_this_step != self.num_layers or self._awaiting_mlp_layer is not None: -+ raise RuntimeError("video-row cache step did not visit every main block") -+ -+ def stats(self) -> dict[str, int | bool | str]: -+ counters: dict[str, int | bool | str] = { -+ name: getattr(self, name) -+ for name in [ -+ "full_steps", -+ "partial_steps", -+ "full_blocks", -+ "partial_blocks", -+ "subset_attention_calls", -+ "subset_query_rows", -+ "omitted_query_rows", -+ "partial_mlp_rows", -+ "peak_cache_bytes", -+ "closed", -+ ] -+ } -+ counters["storage_dtype"] = "torch.bfloat16" if self.pre_gate else "torch.float32" -+ counters["cache_stage"] = ( -+ "pre_gate_secant" if self.secant else "pre_gate" if self.pre_gate else "block_residual" -+ ) -+ counters["secant_forecasts"] = self.secant_forecasts -+ counters["slope_codec"] = "int8_per_channel" if self.secant else "none" -+ counters["fresh_attention_gates"] = self.fresh_attention_gates -+ counters["fresh_mlp_gates"] = self.fresh_mlp_gates -+ counters["live_cache_bytes"] = self._cache_bytes() -+ return counters -+ -+ def close(self) -> None: -+ self._deltas.clear() -+ self._attention_outputs.clear() -+ self._mlp_outputs.clear() -+ self._slopes.clear() -+ self._slope_scales.clear() -+ self._anchor_sigmas.clear() -+ self._slope_intervals.clear() -+ self._sigma = None -+ self._awaiting_mlp_layer = None -+ self._target_positions.clear() -+ self.global_dense_positions = torch.empty(0, dtype=torch.long) -+ self.local_dense_positions = None -+ self.local_video_positions = None -+ self.closed = True -diff --git a/vllm_omni/diffusion/offloader/base.py b/vllm_omni/diffusion/offloader/base.py -index 2505d48..50b73e5 100644 ---- a/vllm_omni/diffusion/offloader/base.py -+++ b/vllm_omni/diffusion/offloader/base.py -@@ -71,6 +71,10 @@ class OffloadConfig: - dlo_use_allgather: bool = True - dlo_resident_layers: int = 0 # leading DiT layers kept on device - chunk_size_bytes: int = 64 * 1024 * 1024 -+ # Hold each DiT block's weight AllGather until that block's QKV AllToAll has -+ # been enqueued, so the background collective does not take the links ahead -+ # of the critical-path all-to-all. See ``collective_gate.py``. -+ dlo_order_allgather_after_attn_a2a: bool = False - # Optional per-worker ceiling for registering an HWR mmap. Zero means no - # additional ceiling; pin_cpu_memory controls whether registration is tried. - dlo_host_registration_limit_gib: float = 0.0 -@@ -216,6 +220,7 @@ class OffloadConfig: - dlo_use_allgather=dit_uses_allgather, - dlo_resident_layers=dlo_resident_layers, - chunk_size_bytes=chunk_size_bytes, -+ dlo_order_allgather_after_attn_a2a=bool(getattr(od_config, "dlo_order_allgather_after_attn_a2a", False)), - dlo_host_registration_limit_gib=dlo_host_registration_limit_gib, - components=components, - dlo_transfers=dlo_transfers, -diff --git a/vllm_omni/diffusion/offloader/distributed_layerwise_backend.py b/vllm_omni/diffusion/offloader/distributed_layerwise_backend.py -index 7f0f081..8c4f87f 100644 ---- a/vllm_omni/diffusion/offloader/distributed_layerwise_backend.py -+++ b/vllm_omni/diffusion/offloader/distributed_layerwise_backend.py -@@ -31,6 +31,7 @@ import torch.distributed - from torch import nn - from vllm.logger import init_logger - -+from vllm_omni.diffusion.distributed.collective_gate import QKVAllToAllBarrier - from vllm_omni.diffusion.hooks import HookRegistry, ModelHook - from vllm_omni.diffusion.model_loader.host_weight_plan import ( - HostWeightPlan, -@@ -142,6 +143,7 @@ class DistributedLayerwiseOffloadHook(ModelHook): - tensor_transforms: dict[int, Any] | None = None, - materialization_probe_tensor: torch.Tensor | None = None, - chunk_size_bytes: int = 64 * 1024 * 1024, -+ order_allgather_after_attn_a2a: bool = False, - ): - assert isinstance(next_block, nn.Module), "transformer block must be type `torch.nn.Module`" - if type(dp_size) is not int or dp_size < 1: -@@ -167,6 +169,7 @@ class DistributedLayerwiseOffloadHook(ModelHook): - - self.chunk_size_bytes = chunk_size_bytes - self.manifest: PartManifest | None = None -+ self._deferred_chunked_allgather = None - self._h2d_done_events = [current_omni_platform.Event(), current_omni_platform.Event()] - self._ag_done_events = [current_omni_platform.Event(), current_omni_platform.Event()] - # Hook-local events protect chunk-input reuse within this hook. Across -@@ -239,6 +242,19 @@ class DistributedLayerwiseOffloadHook(ModelHook): - # Per-block synchronization primitive: set after H2D copy completes. - self._prefetch_done: Any | None = None - -+ # Optional ordering: hold the background weight AllGather until the -+ # block's QKV AllToAll is on the compute stream. See -+ # distributed/collective_gate.py. Off by default; the attention layer -+ # inside this block is handed the same barrier at install time. -+ self._a2a_barrier: QKVAllToAllBarrier | None = None -+ self._a2a_event: Any | None = None -+ # ``(slot, gpu_shards, gpu_weights, evt)`` of a gather armed but not yet -+ # issued, so ``post_forward`` can flush it if the attention never ran. -+ self._deferred_allgather: tuple[int, dict, dict, Any] | None = None -+ if order_allgather_after_attn_a2a: -+ self._a2a_event = current_omni_platform.Event() -+ self._a2a_barrier = QKVAllToAllBarrier(self._allgather_after_attn_a2a) -+ - # Shared shard (AllGather input) buffers — assigned by backend. - self.gpu_shard_buffers: list[dict[torch.dtype, torch.Tensor] | None] = [None, None] - -@@ -653,49 +669,23 @@ class DistributedLayerwiseOffloadHook(ModelHook): - gpu_shard = shard_bufs[dtype][: cpu_shard.numel()] - gpu_shard.copy_(cpu_shard, non_blocking=non_blocking) - gpu_shards[dtype] = gpu_shard -- self.comm_stream.wait_stream(self.copy_stream) -- with current_omni_platform.stream(self.comm_stream): -- for dtype, local_shard in gpu_shards.items(): -- gw = gpu_weights[dtype][: self._ag_output_sizes[dtype]] -- torch.distributed.all_gather_into_tensor( -- gw, -- local_shard, -- group=self.dp_group, -- ) -- evt.record(self.comm_stream) -+ -+ if self._a2a_barrier is not None and non_blocking and getattr(self, "_nz_cast_fn", None) is None: -+ # Hold the collective until the block's attention reports its -+ # QKV AllToAll, so the background gather cannot take the links -+ # ahead of the critical-path all-to-all. The H2D above has -+ # already been enqueued; only the gather moves, so it still -+ # overlaps the rest of the block. Deferral needs a repoint that -+ # emits no stream work, hence the NZ-cast guard. -+ self._deferred_allgather = (slot, gpu_shards, gpu_weights, evt) -+ self._a2a_barrier.arm() -+ else: -+ self._issue_allgather(slot, gpu_shards, gpu_weights, evt) -+ elif self._a2a_barrier is not None and non_blocking: -+ self._deferred_chunked_allgather = (slot, gpu_weights, evt) -+ self._a2a_barrier.arm() - else: -- for dtype_manifest in self.manifest.dtypes: -- cpu_shard = self.cpu_shards[dtype_manifest.dtype] -- buf = shard_bufs[dtype_manifest.dtype] -- for chunk in dtype_manifest.chunks: -- if buf.numel() >= 2 * chunk.local_numel: -- width = buf.numel() // 2 -- input_slot = chunk.chunk_id % 2 -- else: -- width = buf.numel() -- input_slot = 0 -- reuse = self._chunk_reuse_events[input_slot] -- if reuse is not None: -- self.copy_stream.wait_event(reuse) -- local_input = buf[input_slot * width : input_slot * width + chunk.local_numel] -- source = cpu_shard[chunk.cpu_offset : chunk.cpu_offset + chunk.local_numel] -- with current_omni_platform.stream(self.copy_stream): -- local_input.copy_(source, non_blocking=non_blocking) -- self._h2d_done_events[input_slot].record(self.copy_stream) -- self.comm_stream.wait_event(self._h2d_done_events[input_slot]) -- with current_omni_platform.stream(self.comm_stream): -- full_out = gpu_weights[dtype_manifest.dtype][ -- chunk.full_offset : chunk.full_offset + chunk.padded_numel -- ] -- torch.distributed.all_gather_into_tensor( -- full_out, -- local_input, -- group=self.dp_group, -- ) -- self._ag_done_events[input_slot].record(self.comm_stream) -- self._chunk_reuse_events[input_slot] = self._ag_done_events[input_slot] -- with current_omni_platform.stream(self.comm_stream): -- evt.record(self.comm_stream) -+ self._issue_chunked_allgather(slot, gpu_weights, evt, non_blocking=non_blocking) - - self.ready_events[slot] = evt - self._prefetch_done = evt -@@ -719,6 +709,131 @@ class DistributedLayerwiseOffloadHook(ModelHook): - ), - ) - -+ def _issue_chunked_allgather(self, slot, gpu_weights, evt, *, gated=False, non_blocking=True) -> None: -+ """Submit chunk copies and gathers together after the foreground gate. -+ -+ Chunk staging slots are bounded and reused. Deferring only gathers -+ would overwrite an input slot before its collective consumes it. -+ """ -+ shard_bufs = self.gpu_shard_buffers[slot] -+ assert shard_bufs is not None and self.manifest is not None -+ if gated: -+ self.comm_stream.wait_event(self._a2a_event) -+ for dtype_manifest in self.manifest.dtypes: -+ cpu_shard = self.cpu_shards[dtype_manifest.dtype] -+ buf = shard_bufs[dtype_manifest.dtype] -+ for chunk in dtype_manifest.chunks: -+ if buf.numel() >= 2 * chunk.local_numel: -+ width = buf.numel() // 2 -+ input_slot = chunk.chunk_id % 2 -+ else: -+ width = buf.numel() -+ input_slot = 0 -+ reuse = self._chunk_reuse_events[input_slot] -+ if reuse is not None: -+ self.copy_stream.wait_event(reuse) -+ local_input = buf[input_slot * width : input_slot * width + chunk.local_numel] -+ source = cpu_shard[chunk.cpu_offset : chunk.cpu_offset + chunk.local_numel] -+ with current_omni_platform.stream(self.copy_stream): -+ local_input.copy_(source, non_blocking=non_blocking) -+ self._h2d_done_events[input_slot].record(self.copy_stream) -+ self.comm_stream.wait_event(self._h2d_done_events[input_slot]) -+ with current_omni_platform.stream(self.comm_stream): -+ full_out = gpu_weights[dtype_manifest.dtype][ -+ chunk.full_offset : chunk.full_offset + chunk.padded_numel -+ ] -+ torch.distributed.all_gather_into_tensor( -+ full_out, -+ local_input, -+ group=self.dp_group, -+ ) -+ self._ag_done_events[input_slot].record(self.comm_stream) -+ self._chunk_reuse_events[input_slot] = self._ag_done_events[input_slot] -+ with current_omni_platform.stream(self.comm_stream): -+ evt.record(self.comm_stream) -+ -+ def _issue_allgather( -+ self, -+ slot: int, -+ gpu_shards: dict[torch.dtype, torch.Tensor], -+ gpu_weights: dict[torch.dtype, torch.Tensor], -+ evt: Any, -+ *, -+ gated: bool = False, -+ ) -> None: -+ """Submit this block's weight AllGather on ``comm_stream``. -+ -+ ``gated`` additionally waits on the QKV AllToAll barrier event recorded -+ by the block's attention, so the foreground collective gets the links -+ first and this gather queues behind it. -+ """ -+ self.comm_stream.wait_stream(self.copy_stream) -+ if gated: -+ self.comm_stream.wait_event(self._a2a_event) -+ with current_omni_platform.stream(self.comm_stream): -+ for dtype, local_shard in gpu_shards.items(): -+ # Slice the shared (max-sized) output buffer down to this -+ # block's actual AllGather output size. The buffers are -+ # sized to the *largest* block across all groups, so for any -+ # smaller block the full buffer would violate the -+ # all_gather_into_tensor contract -+ # (output.numel() == world_size * input.numel()). -+ # Repoint offsets are relative to the block's flattened -+ # buffer, so a prefix slice is safe. -+ gw = gpu_weights[dtype][: self._ag_output_sizes[dtype]] -+ torch.distributed.all_gather_into_tensor( -+ gw, -+ local_shard, -+ group=self.dp_group, -+ ) -+ evt.record(self.comm_stream) -+ -+ def _allgather_after_attn_a2a(self) -> None: -+ """Barrier callback: the block's QKV AllToAll is now enqueued. -+ -+ Recording on the compute stream here (rather than at block entry) is -+ what makes the gate effective: an event wait snapshots the records -+ enqueued before it, so the wait must be enqueued *after* the all-to-all. -+ """ -+ chunked = self._deferred_chunked_allgather -+ if chunked is not None: -+ self._deferred_chunked_allgather = None -+ assert self._a2a_event is not None -+ self._a2a_event.record(current_omni_platform.current_stream()) -+ self._issue_chunked_allgather(*chunked, gated=True) -+ return -+ deferred = self._deferred_allgather -+ if deferred is None: -+ return -+ self._deferred_allgather = None -+ assert self._a2a_event is not None -+ self._a2a_event.record(current_omni_platform.current_stream()) -+ slot, gpu_shards, gpu_weights, evt = deferred -+ self._issue_allgather(slot, gpu_shards, gpu_weights, evt, gated=True) -+ -+ def _flush_deferred_allgather(self) -> None: -+ """Issue a deferred gather whose attention never signalled. -+ -+ Keeps the double buffer live: a block whose attention is skipped -+ (cache-dit, an all-resident stage) still needs its weights, so the -+ gather is submitted ungated at the end of that block's forward. -+ """ -+ chunked = self._deferred_chunked_allgather -+ if chunked is not None: -+ self._deferred_chunked_allgather = None -+ if self._a2a_barrier is not None: -+ self._a2a_barrier.disarm() -+ self._issue_chunked_allgather(*chunked) -+ return -+ deferred = self._deferred_allgather -+ if deferred is None: -+ return -+ self._deferred_allgather = None -+ if self._a2a_barrier is not None: -+ self._a2a_barrier.disarm() -+ slot, gpu_shards, gpu_weights, evt = deferred -+ self._issue_allgather(slot, gpu_shards, gpu_weights, evt) -+ - def _wait_for_weights(self, slot: int) -> None: - """Wait until the slot holding this block is ready for compute. - -@@ -840,6 +955,10 @@ class DistributedLayerwiseOffloadHook(ModelHook): - return args, kwargs - - def post_forward(self, module: nn.Module, output: Any) -> Any: -+ # Pair with pre_forward: if the block's attention never signalled the -+ # gate, the deferred gather must still go out before this block's -+ # storage is released (offload_layer waits on _prefetch_done). -+ self._flush_deferred_allgather() - self.offload_layer() - return output - -@@ -864,6 +983,7 @@ def apply_distributed_block_hook( - tensor_transforms: dict[int, Any] | None = None, - materialization_probe_tensor: torch.Tensor | None = None, - chunk_size_bytes: int = 64 * 1024 * 1024, -+ order_allgather_after_attn_a2a: bool = False, - ) -> DistributedLayerwiseOffloadHook: - """Register a DistributedLayerwiseOffloadHook on *module*.""" - registry = HookRegistry.get_or_create(module) -@@ -881,11 +1001,32 @@ def apply_distributed_block_hook( - tensor_transforms=tensor_transforms, - materialization_probe_tensor=materialization_probe_tensor, - chunk_size_bytes=chunk_size_bytes, -+ order_allgather_after_attn_a2a=order_allgather_after_attn_a2a, - ) - registry.register_hook(DistributedLayerwiseOffloadHook._HOOK_NAME, hook) - return hook - - -+def attach_collective_gate(block: nn.Module, hook: DistributedLayerwiseOffloadHook) -> int: -+ """Hand ``hook``'s barrier to every Attention layer inside *block*. -+ -+ The attention signals the barrier right after its QKV AllToAll. The import -+ is local to keep this module independent of the attention package at import -+ time. -+ -+ Returns the number of attention layers wired (0 means the hook will rely on -+ the ``post_forward`` flush instead). -+ """ -+ from vllm_omni.diffusion.attention.layer import Attention -+ -+ attached = 0 -+ for module in block.modules(): -+ if isinstance(module, Attention): -+ module._collective_gate = hook._a2a_barrier -+ attached += 1 -+ return attached -+ -+ - def remove_distributed_block_hook(module: nn.Module) -> None: - """Remove the distributed layerwise offload hook from *module*.""" - registry: HookRegistry | None = getattr(module, "_hook_registry", None) -@@ -1566,29 +1707,45 @@ class DistributedLayerwiseOffloadBackend(OffloadBackend): - self._all_hook_groups.append(hooks) - self._blocks.append(block_list) - probes = {id(block): module_materialization_probe(block) for block in block_list} -+ # Only the DiT blocks issue a QKV AllToAll the gather can queue behind. -+ order_gate = bool(getattr(self.config, "dlo_order_allgather_after_attn_a2a", False)) and ( -+ component == DIT_COMPONENT -+ ) -+ if order_gate and not getattr(self, "_collective_gate_logged", False): -+ self._collective_gate_logged = True -+ logger.info( -+ "Distributed layerwise offload will hold each DiT block's weight AllGather " -+ "until that block's QKV AllToAll is on the compute stream" -+ ) - for block, next_block in zip( - chain((block_list[-1],), block_list[:-1]), - block_list, - strict=True, - ): -- hooks.append( -- apply_distributed_block_hook( -- block, -- next_block, -- self.device, -- group, -- group_size, -- group_rank, -- self.copy_stream, -- self.comm_stream, -- self.config.pin_cpu_memory, -- shared_buffers=[None, None], -- rank_local_mmap=self._using_rank_local_mmap if use_dit_mmap else False, -- tensor_transforms=self._mmap_transforms_by_tensor_id if use_dit_mmap else None, -- materialization_probe_tensor=probes[id(block)], -- chunk_size_bytes=self.config.chunk_size_bytes, -- ) -+ hook = apply_distributed_block_hook( -+ block, -+ next_block, -+ self.device, -+ group, -+ group_size, -+ group_rank, -+ self.copy_stream, -+ self.comm_stream, -+ self.config.pin_cpu_memory, -+ shared_buffers=[None, None], -+ rank_local_mmap=self._using_rank_local_mmap if use_dit_mmap else False, -+ tensor_transforms=self._mmap_transforms_by_tensor_id if use_dit_mmap else None, -+ materialization_probe_tensor=probes[id(block)], -+ chunk_size_bytes=self.config.chunk_size_bytes, -+ order_allgather_after_attn_a2a=order_gate, - ) -+ if hook._a2a_barrier is not None and attach_collective_gate(block, hook) == 0: -+ logger.warning( -+ "DLO AllGather ordering is enabled but block %s exposes no Attention layer; " -+ "its gather will fall back to the ungated path", -+ block.__class__.__name__, -+ ) -+ hooks.append(hook) - - # hooks = [last -> first, block0 -> block1, ...]. Alternating slots - # keep a prefetch from overwriting the current block for any ring size. -diff --git a/vllm_omni/engine/arg_utils.py b/vllm_omni/engine/arg_utils.py -index 0a876b9..bf8dcdc 100644 ---- a/vllm_omni/engine/arg_utils.py -+++ b/vllm_omni/engine/arg_utils.py -@@ -807,6 +807,7 @@ class OrchestratorArgs: - dlo_use_allgather: bool = True - dlo_chunk_size_mb: int = 64 - dlo_resident_layers: int = 0 -+ dlo_order_allgather_after_attn_a2a: bool = False - host_weight_runtime_mode: str = "disabled" - host_weight_runtime_root: str | None = None - dlo_host_registration_limit_gib: float = 0.0 -diff --git a/vllm_omni/entrypoints/cli/serve.py b/vllm_omni/entrypoints/cli/serve.py -index f2b55e7..aff71b5 100644 ---- a/vllm_omni/entrypoints/cli/serve.py -+++ b/vllm_omni/entrypoints/cli/serve.py -@@ -1041,6 +1041,17 @@ class OmniServeCommand(CLISubcommand): - help="Full-chunk target size in MiB for distributed layerwise offload " - "(each chunk is one host-to-device copy + AllGather unit).", - ) -+ omni_config_group.add_argument( -+ "--dlo-order-allgather-after-attn-a2a", -+ action="store_true", -+ default=False, -+ help=( -+ "Hold each DiT block's weight AllGather until that block's QKV AllToAll " -+ "has been enqueued, so the background gather does not take the collective " -+ "links ahead of the critical-path all-to-all. Requires distributed " -+ "layerwise offload with AllGather." -+ ), -+ ) - omni_config_group.add_argument( - "--host-weight-runtime-mode", - choices=("disabled", "preferred", "required"), diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c22-source-identity.json b/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c22-source-identity.json deleted file mode 100644 index bce9b1703..000000000 --- a/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c22-source-identity.json +++ /dev/null @@ -1,29 +0,0 @@ -{ - "head": "7b0d12bfefdf05868e93e74bc91f04c70ee34128", - "tree": "1390788d5fcee97e48ba874be718db25ae89b064", - "base": "d48fa51d03dd748568cbcf57ed40f533c691c9bd", - "branch": "rsi/h3-297-video-pre-gate-secant", - "files": { - "tests/diffusion/distributed/test_collective_gate.py": "6a02f32882d85eca1e3bb9a023ce3adb3d5b59c213845e11e85c96a89432f477", - "tests/diffusion/models/minimax_h3/test_minimax_h3_pre_gate_cache.py": "b8394d22b054d60c1ab8e069e9600e1947cc73b5826e1f206e66c548123f632d", - "tests/diffusion/models/minimax_h3/test_minimax_h3_pre_gate_secant.py": "70a215fbb0dc63b90652d96f8f6d106a5785bd7d64dae75a2c4f6ec5c627ac1c", - "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "e09b872f6900e821d42632a0ca09595f2affe6f9794049dbc74d69ba6a52b2b0", - "vllm_omni/config/omni_config.py": "6894120a330474f718d43bf2bf7928892eb996152839ce6469c34d18a7df09d1", - "vllm_omni/config/stage_config.py": "0752e8cc1c6f1021f737830d08663cfc118923a83fd909c607f9ec3ab2a3d2fb", - "vllm_omni/diffusion/attention/layer.py": "76c5fce3dabd23e39a8890316e1691c03bc203066cb60af5611ec4790eefc632", - "vllm_omni/diffusion/data.py": "5db83d5c71cb27cc7810d35fcfc891da80f0b003adbb16bdecc3816a63305e11", - "vllm_omni/diffusion/distributed/collective_gate.py": "01c6317cfa615840681b8bb8b8513fa3736d26e845a24ad9e1ab7f87489b962f", - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ca87df1936fd2694fcc768079e9e8f926b6d45969ddc61ebf353f5e726d70b8b", - "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "b991ebb5d43242357d5ab8cfb476808a76fc0658d6048db6f3b7131122c8a993", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "8933b536317652d0dbc7a974ea7e0bca928e7b3bdf438988999ff5a498a6ec9d", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "3904bc8ac6679bc51614b95f5510789331a41bf20a43a517f3a99d33505201a6", - "vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py": "e8e2e74fb8edb05e02d3e2328ea1f3ad5ddcfd7a10ed09c5cf3911b1d063e849", - "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "e21e0afe8f8948d805d374428a34f3b2c5c0b5041d63ea791f047af8f455cf16", - "vllm_omni/diffusion/offloader/base.py": "23922136deaf54e2d98e827b007b34c7e3ad36171b71bfe9060d390efc0b679f", - "vllm_omni/diffusion/offloader/distributed_layerwise_backend.py": "19fbce0f1036e4784d68cab798c499526a9099101909e41ce96dd0caa033b6a4", - "vllm_omni/engine/arg_utils.py": "52569e3688cc708729a48ca0a2c875c28193733c2f934f6ff89f6c815e4009c7", - "vllm_omni/entrypoints/cli/serve.py": "f8284ad6b751ba4a9e4f7254d78132cc2cfd49871a0f9af7fde49c6789d79cb0" - }, - "published": false, - "patch_base": "be319692b96413918973b5bd399cdda3f320c574" -} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c23-candidate.patch b/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c23-candidate.patch deleted file mode 100644 index f863e0140..000000000 --- a/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c23-candidate.patch +++ /dev/null @@ -1,2430 +0,0 @@ -diff --git a/tests/diffusion/distributed/test_collective_gate.py b/tests/diffusion/distributed/test_collective_gate.py -new file mode 100644 -index 0000000..e66fdae ---- /dev/null -+++ b/tests/diffusion/distributed/test_collective_gate.py -@@ -0,0 +1,205 @@ -+# SPDX-License-Identifier: Apache-2.0 -+# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project -+ -+"""Unit tests for the DLO/QKV-AllToAll ordering gate. -+ -+Covers the barrier rendezvous and the hook's deferred-AllGather state machine -+that keeps the background weight gather from taking the links ahead of the -+block's foreground AllToAll. -+""" -+ -+from unittest.mock import Mock -+ -+import pytest -+import torch -+from torch import nn -+ -+import vllm_omni.diffusion.offloader.distributed_layerwise_backend as dist_backend_module -+from tests.diffusion.offloader.helpers import DummyStream, patch_offload_runtime -+from vllm_omni.diffusion.distributed.collective_gate import QKVAllToAllBarrier -+from vllm_omni.diffusion.offloader.distributed_layerwise_backend import ( -+ DistributedLayerwiseOffloadHook, -+ attach_collective_gate, -+) -+ -+pytestmark = [pytest.mark.diffusion, pytest.mark.cpu, pytest.mark.core_model] -+ -+ -+@pytest.fixture -+def patched_offload_runtime(monkeypatch): -+ patch_offload_runtime(monkeypatch, dist_backend_module.current_omni_platform, synchronize=True) -+ -+ -+class TestQKVAllToAllBarrier: -+ def test_signal_without_arm_is_a_noop(self): -+ on_signal = Mock() -+ barrier = QKVAllToAllBarrier(on_signal) -+ -+ barrier.signal() -+ -+ on_signal.assert_not_called() -+ assert not barrier.armed -+ -+ def test_arm_then_signal_fires_once(self): -+ on_signal = Mock() -+ barrier = QKVAllToAllBarrier(on_signal) -+ -+ barrier.arm() -+ assert barrier.armed -+ barrier.signal() -+ -+ on_signal.assert_called_once_with() -+ assert not barrier.armed -+ -+ # A second signal in the same forward must not re-issue the gather. -+ barrier.signal() -+ on_signal.assert_called_once_with() -+ -+ def test_disarm_suppresses_a_pending_signal(self): -+ on_signal = Mock() -+ barrier = QKVAllToAllBarrier(on_signal) -+ -+ barrier.arm() -+ barrier.disarm() -+ barrier.signal() -+ -+ on_signal.assert_not_called() -+ -+ -+def _gated_hook() -> DistributedLayerwiseOffloadHook: -+ """A hook wired for the gated path, with hand-set shard buffers.""" -+ hook = DistributedLayerwiseOffloadHook( -+ next_block=nn.Linear(2, 2, bias=False), -+ device=torch.device("cpu"), -+ dp_group=object(), -+ dp_size=2, -+ rank=0, -+ copy_stream=DummyStream(), -+ comm_stream=DummyStream(), -+ pin_memory=False, -+ order_allgather_after_attn_a2a=True, -+ ) -+ device_buf = {torch.float32: torch.zeros(4)} -+ shard_buf = {torch.float32: torch.zeros(2)} -+ hook.gpu_buffers = [dict(device_buf), dict(device_buf)] -+ hook.gpu_shard_buffers = [dict(shard_buf), dict(shard_buf)] -+ hook.cpu_shards = {torch.float32: torch.zeros(2)} -+ hook._ag_output_sizes = {torch.float32: 4} -+ hook.metadata = {} -+ hook._cached_repoint = () -+ hook.rank_local_mmap = False -+ return hook -+ -+ -+class TestDeferredAllGather: -+ def test_prefetch_defers_the_gather_until_the_barrier_signals(self, patched_offload_runtime, monkeypatch): -+ hook = _gated_hook() -+ gather = Mock() -+ monkeypatch.setattr(torch.distributed, "all_gather_into_tensor", gather) -+ -+ hook.prefetch_layer(0, non_blocking=True) -+ -+ # H2D ran; the collective is held back for the block's AllToAll. -+ gather.assert_not_called() -+ assert hook._a2a_barrier.armed -+ assert hook._deferred_allgather is not None -+ # The ready event must still be published for the consumer. -+ assert hook.ready_events[0] is hook._prefetch_done -+ -+ hook._a2a_barrier.signal() -+ -+ gather.assert_called_once() -+ assert hook._deferred_allgather is None -+ assert not hook._a2a_barrier.armed -+ -+ def test_post_forward_flush_issues_an_unsignalled_gather(self, patched_offload_runtime, monkeypatch): -+ """A block whose attention never ran must still get its weights.""" -+ hook = _gated_hook() -+ gather = Mock() -+ monkeypatch.setattr(torch.distributed, "all_gather_into_tensor", gather) -+ -+ hook.prefetch_layer(0, non_blocking=True) -+ gather.assert_not_called() -+ -+ # pre_forward ran, attention did not: the pairing flush must fire. -+ hook.post_forward(hook.next_block, None) -+ -+ gather.assert_called_once() -+ assert hook._deferred_allgather is None -+ assert not hook._a2a_barrier.armed -+ -+ def test_sync_prefetch_issues_immediately_without_arming(self, patched_offload_runtime, monkeypatch): -+ """The cache-dit sync fallback has no attention signal to wait for.""" -+ hook = _gated_hook() -+ gather = Mock() -+ monkeypatch.setattr(torch.distributed, "all_gather_into_tensor", gather) -+ -+ hook.prefetch_layer(0, non_blocking=False) -+ -+ gather.assert_called_once() -+ assert not hook._a2a_barrier.armed -+ assert hook._deferred_allgather is None -+ -+ def test_disabled_by_default(self, patched_offload_runtime, monkeypatch): -+ hook = DistributedLayerwiseOffloadHook( -+ next_block=nn.Linear(2, 2, bias=False), -+ device=torch.device("cpu"), -+ dp_group=object(), -+ dp_size=2, -+ rank=0, -+ copy_stream=DummyStream(), -+ comm_stream=DummyStream(), -+ pin_memory=False, -+ ) -+ hook.gpu_buffers = [{torch.float32: torch.zeros(4)}] * 2 -+ hook.gpu_shard_buffers = [{torch.float32: torch.zeros(2)}] * 2 -+ hook.cpu_shards = {torch.float32: torch.zeros(2)} -+ hook._ag_output_sizes = {torch.float32: 4} -+ hook.metadata = {} -+ hook._cached_repoint = () -+ gather = Mock() -+ monkeypatch.setattr(torch.distributed, "all_gather_into_tensor", gather) -+ -+ hook.prefetch_layer(0, non_blocking=True) -+ -+ assert hook._a2a_barrier is None -+ gather.assert_called_once() -+ -+ -+class TestAttachCollectiveGate: -+ def test_block_without_attention_reports_zero(self): -+ hook = _gated_hook() -+ -+ assert attach_collective_gate(nn.Linear(2, 2), hook) == 0 -+ -+ -+@pytest.mark.parametrize("flush", [False, True]) -+def test_chunked_gate_defers_copies_and_reuses_bounded_slots(patched_offload_runtime, monkeypatch, flush): -+ from types import SimpleNamespace -+ -+ hook = _gated_hook() -+ chunks = tuple( -+ SimpleNamespace(chunk_id=i, local_numel=1, cpu_offset=i, full_offset=2 * i, padded_numel=2) for i in range(3) -+ ) -+ hook.manifest = SimpleNamespace(dtypes=(SimpleNamespace(dtype=torch.float32, chunks=chunks),)) -+ hook.cpu_shards = {torch.float32: torch.tensor([11.0, 22.0, 33.0])} -+ hook.gpu_buffers = [{torch.float32: torch.zeros(6)} for _ in range(2)] -+ calls = [] -+ -+ def gather(output, local, **kwargs): -+ calls.append(local.clone()) -+ output.copy_(local.repeat(2)) -+ -+ monkeypatch.setattr(torch.distributed, "all_gather_into_tensor", gather) -+ hook.prefetch_layer(0, non_blocking=True) -+ assert not calls and hook._deferred_chunked_allgather is not None -+ assert torch.equal(hook.gpu_shard_buffers[0][torch.float32], torch.zeros(2)) -+ if flush: -+ hook._flush_deferred_allgather() -+ else: -+ hook._a2a_barrier.signal() -+ assert len(calls) == 3 and hook._deferred_chunked_allgather is None -+ assert not hook._a2a_barrier.armed -+ assert torch.equal(hook.gpu_buffers[0][torch.float32], torch.tensor([11.0, 11.0, 22.0, 22.0, 33.0, 33.0])) -+ hook._a2a_barrier.signal() -+ assert len(calls) == 3 -diff --git a/tests/diffusion/models/minimax_h3/test_minimax_h3_pre_gate_cache.py b/tests/diffusion/models/minimax_h3/test_minimax_h3_pre_gate_cache.py -new file mode 100644 -index 0000000..4e9979e ---- /dev/null -+++ b/tests/diffusion/models/minimax_h3/test_minimax_h3_pre_gate_cache.py -@@ -0,0 +1,198 @@ -+# SPDX-License-Identifier: Apache-2.0 -+# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project -+ -+from datetime import timedelta -+ -+import pytest -+import torch -+import torch.distributed as dist -+import torch.nn as nn -+ -+from tests.diffusion.models.minimax_h3.test_minimax_h3_video_row_cache import tiny_block -+from tests.helpers.mark import hardware_marks -+from vllm_omni.diffusion.layers.indexed_modulation import indexed_gate -+from vllm_omni.diffusion.models.minimax_h3.attention.flash_h3 import _prefix_bnsd, _subset_prefix_bnsd -+from vllm_omni.diffusion.models.minimax_h3.video_row_cache import MiniMaxH3VideoRowCache -+from vllm_omni.platforms import current_omni_platform -+ -+pytestmark = [pytest.mark.core_model, pytest.mark.diffusion, *hardware_marks(res={"npu": "A3"}, num_cards=4)] -+ -+ -+def make_state(start=0, length=4): -+ state = MiniMaxH3VideoRowCache(torch.tensor([1, 2]), seq_len=4, used_len=3, num_layers=1, pre_gate=True) -+ state.prepare_local_span(start, length) -+ return state -+ -+ -+def test_fixed_pre_gate_schedule_storage_and_request_cleanup(): -+ state = make_state() -+ x = torch.ones(4, 2, dtype=torch.bfloat16) -+ full = [] -+ for step in range(50): -+ state.begin_step(step) -+ if not state.partial: -+ full.append(step) -+ for stage in ("attn", "mlp"): -+ out = state.pre_gate_output(0, x, stage) -+ assert out.dtype == torch.bfloat16 -+ if state.partial: -+ state.record_fresh_gate(stage) -+ state.end_step() -+ assert full == [0, 1, 4, 9, 19, 29, 39, 49] -+ assert state.full_blocks == 8 and state.partial_blocks == 42 -+ assert state.fresh_attention_gates == state.fresh_mlp_gates == 42 -+ assert all(t.dtype == torch.bfloat16 for t in [*state._attention_outputs.values(), *state._mlp_outputs.values()]) -+ assert state.stats()["live_cache_bytes"] == 16 -+ state.close() -+ assert state.stats()["live_cache_bytes"] == 0 and state.closed -+ -+ -+@pytest.mark.parametrize("failure", ["missing_mlp", "mlp_first", "gated_capture", "gated_restore"]) -+def test_pre_gate_stage_order_cannot_silently_use_block_residual(failure): -+ state = make_state() -+ x = torch.ones(4, 2, dtype=torch.bfloat16) -+ state.begin_step(0) -+ with pytest.raises(RuntimeError): -+ if failure == "missing_mlp": -+ state.pre_gate_output(0, x, "attn") -+ state.end_step() -+ elif failure == "mlp_first": -+ state.pre_gate_output(0, x, "mlp") -+ elif failure == "gated_capture": -+ state.capture(0, x, x) -+ else: -+ state.restore(0, x) -+ state.close() -+ -+ -+class ConstantMLP(nn.Module): -+ def __init__(self): -+ super().__init__() -+ self.rows = [] -+ -+ def forward(self, x): -+ self.rows.append(x.shape[0]) -+ return torch.full_like(x, 3) -+ -+ -+@pytest.mark.parametrize("start,length", [(0, 4), (1, 2)]) -+def test_real_block_reapplies_changed_gates_and_preserves_empty_dense_shards(start, length): -+ state = make_state(start, length) -+ block = tiny_block() -+ gates = [1.0, 1.0] -+ block.adaln_proj = lambda t: ( -+ torch.zeros(3, 2), -+ torch.zeros(3, 2), -+ torch.full((3, 2), gates[0]), -+ torch.zeros(3, 2), -+ torch.zeros(3, 2), -+ torch.full((3, 2), gates[1]), -+ ) -+ block.attn = lambda x, **kwargs: torch.full_like(x, 2) -+ block.mlp = ConstantMLP() -+ x = torch.ones(length, 2, dtype=torch.bfloat16) -+ kwargs = dict( -+ t_emb=torch.zeros(1), -+ combined_indices=torch.zeros(length, dtype=torch.long), -+ rope_table=None, -+ cu_seqlens=torch.tensor([0, 3, 4], dtype=torch.int32), -+ max_seqlen=3, -+ packed_total=4, -+ ) -+ for step in (0, 1): -+ state.begin_step(step) -+ expected = block(x, **kwargs) -+ actual = block(x, video_row_cache=state, **kwargs) -+ assert torch.equal(actual.view(torch.uint8), expected.view(torch.uint8)) -+ state.end_step() -+ gates[:] = [0.5, -0.25] -+ current = x * 2 -+ saved = current.clone() -+ expected = block(current, **kwargs) -+ block.mlp.rows.clear() -+ state.begin_step(2) -+ actual = block(current, video_row_cache=state, **kwargs) -+ state.end_step() -+ rows = state.local_dense_positions -+ assert block.mlp.rows == ([rows.numel()] if rows.numel() else []) -+ assert torch.equal( -+ actual[: min(length, 3 - start)].view(torch.uint8), expected[: min(length, 3 - start)].view(torch.uint8) -+ ) -+ assert torch.equal( -+ actual.index_select(0, state.local_video_positions), torch.full((state.local_video_positions.numel(), 2), 2.25) -+ ) -+ assert not torch.equal( -+ actual.index_select(0, state.local_video_positions), current.index_select(0, state.local_video_positions) + 5 -+ ) -+ assert torch.equal(current, saved) -+ assert state.fresh_attention_gates == state.fresh_mlp_gates == 1 -+ state.close() -+ -+ -+def _native_pre_gate_worker(rank): -+ import torch_npu # noqa: F401 -+ -+ torch.npu.set_device(rank) -+ dist.init_process_group( -+ "hccl", init_method="tcp://127.0.0.1:29597", rank=rank, world_size=4, timeout=timedelta(seconds=120) -+ ) -+ try: -+ torch.manual_seed(219) -+ full = [torch.randn(32, 4, 128, dtype=torch.bfloat16).npu() for _ in range(3)] -+ exchanged = [] -+ for tensor in full: -+ local = tensor[rank * 8 : (rank + 1) * 8] -+ send = local.reshape(8, 4, 1, 128).permute(1, 0, 2, 3).contiguous() -+ receive = torch.empty_like(send) -+ dist.all_to_all_single(receive, send) -+ exchanged.append(receive.reshape(1, 32, 1, 128)) -+ state = MiniMaxH3VideoRowCache( -+ torch.arange(8, 24, device="npu"), seq_len=32, used_len=30, num_layers=1, pre_gate=True -+ ) -+ state.prepare_local_span(rank * 8, 8) -+ x = torch.full((8, 512), 2, dtype=torch.bfloat16, device="npu") -+ for step in (0, 1): -+ state.begin_step(step) -+ state.pre_gate_output(0, torch.full_like(x, 0.5), "attn") -+ state.pre_gate_output(0, torch.full_like(x, 0.25), "mlp") -+ state.end_step() -+ state.begin_step(2) -+ out = _subset_prefix_bnsd(exchanged[0], exchanged[1][:, :30], exchanged[2][:, :30], 128**-0.5, state) -+ send, receive = out.reshape(4, 8, 1, 128).contiguous(), torch.empty_like(out.reshape(4, 8, 1, 128)) -+ dist.all_to_all_single(receive, send) -+ local_out = receive.permute(1, 0, 2, 3).reshape(8, 512) -+ expected = _prefix_bnsd(full[0].unsqueeze(0), full[1][:30].unsqueeze(0), full[2][:30].unsqueeze(0), 128**-0.5)[ -+ 0 -+ ].reshape(32, 512) -+ dense = state.local_dense_positions -+ torch.testing.assert_close( -+ local_out.index_select(0, dense).float(), -+ expected[rank * 8 : (rank + 1) * 8].index_select(0, dense).float(), -+ atol=0.03125, -+ rtol=0.02, -+ ) -+ attn = state.pre_gate_output(0, local_out, "attn") -+ indices = torch.zeros(8, dtype=torch.long, device="npu") -+ gate = torch.full((1, 512), 0.5, dtype=torch.float32, device="npu") -+ after_attn = indexed_gate(x, gate, attn, indices) -+ state.record_fresh_gate("attn") -+ mlp = state.pre_gate_output(0, torch.zeros_like(x), "mlp") -+ result = indexed_gate(after_attn, gate, mlp, indices) -+ state.record_fresh_gate("mlp") -+ video = state.local_video_positions -+ assert torch.equal( -+ result.index_select(0, video), torch.full((video.numel(), 512), 2.375, dtype=torch.bfloat16, device="npu") -+ ) -+ assert dense.numel() == [8, 0, 0, 6][rank] -+ state.end_step() -+ state.close() -+ assert state.stats()["live_cache_bytes"] == 0 -+ torch.npu.synchronize() -+ dist.barrier() -+ finally: -+ dist.destroy_process_group() -+ -+ -+@pytest.mark.skipif(not current_omni_platform.is_npu(), reason="Requires four native Ascend devices") -+def test_native_four_rank_pre_gate_subset_attention_and_current_gates(): -+ torch.multiprocessing.spawn(_native_pre_gate_worker, nprocs=4, join=True) -diff --git a/tests/diffusion/models/minimax_h3/test_minimax_h3_video_context.py b/tests/diffusion/models/minimax_h3/test_minimax_h3_video_context.py -new file mode 100644 -index 0000000..71c6dd5 ---- /dev/null -+++ b/tests/diffusion/models/minimax_h3/test_minimax_h3_video_context.py -@@ -0,0 +1,225 @@ -+# SPDX-License-Identifier: Apache-2.0 -+# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project -+ -+from datetime import timedelta -+ -+import pytest -+import torch -+import torch.distributed as dist -+ -+from tests.diffusion.models.minimax_h3.test_minimax_h3_video_row_cache import tiny_block -+from tests.helpers.mark import hardware_marks -+from vllm_omni.diffusion.models.minimax_h3.attention import video_context as context -+from vllm_omni.diffusion.models.minimax_h3.attention.flash_h3 import _prefix_bnsd -+from vllm_omni.diffusion.models.minimax_h3.video_row_cache import MiniMaxH3VideoRowCache -+from vllm_omni.platforms import current_omni_platform -+ -+pytestmark = [pytest.mark.core_model, pytest.mark.diffusion, *hardware_marks(res={"npu": "A3"}, num_cards=4)] -+ -+ -+def oracle(query, key, value, scale): -+ q, k, v = [x.transpose(1, 2).float() for x in (query, key, value)] -+ logits = q @ k.transpose(-2, -1) * scale -+ output = (torch.softmax(logits, -1) @ v).transpose(1, 2).to(query.dtype) -+ return output.contiguous(), torch.logsumexp(logits, -1).transpose(1, 2).unsqueeze(-1).contiguous() -+ -+ -+def reference_mix(a, b, lse_a, lse_b): -+ maximum = torch.maximum(lse_a, lse_b) -+ wa, wb = (lse_a - maximum).exp(), (lse_b - maximum).exp() -+ return ((wa * a.float() + wb * b.float()) / (wa + wb)).to(a.dtype) -+ -+ -+def make_state(seq=12, used=10, prefix=3, device="cpu", start=0, length=None): -+ state = MiniMaxH3VideoRowCache( -+ torch.arange(prefix, used, device=device), seq_len=seq, used_len=used, num_layers=1, context_only=True -+ ) -+ state.prepare_local_span(start, seq if length is None else length) -+ return state -+ -+ -+@pytest.mark.parametrize("positions,pregate", [([1], False), ([1, 2], True)]) -+def test_context_mode_rejects_nonfinal_video_or_pre_gate_ffn(positions, pregate): -+ with pytest.raises(ValueError, match="final target span"): -+ MiniMaxH3VideoRowCache( -+ torch.tensor(positions), seq_len=4, used_len=3, num_layers=1, context_only=True, pre_gate=pregate -+ ) -+ -+ -+def test_partition_matches_dense_and_refreshes_prefix_without_mutating_inputs(monkeypatch): -+ monkeypatch.setattr(context, "_prefix_bnsd", lambda *args: oracle(*args)[0]) -+ monkeypatch.setattr(context, "prefix_attention_lse", oracle) -+ monkeypatch.setattr(context, "mix_context", reference_mix) -+ torch.manual_seed(23) -+ q = torch.randn(1, 12, 2, 8, dtype=torch.bfloat16) -+ k, v = [torch.randn(1, 10, 2, 8, dtype=torch.bfloat16) for _ in range(2)] -+ state = make_state() -+ dummy = torch.ones(12, 2, dtype=torch.bfloat16) -+ for step in (0, 1): -+ state.begin_step(step) -+ output = context.video_context_attention(q, k, v, 0.25, state) -+ expected = oracle(q[:, :10], k, v, 0.25)[0] -+ torch.testing.assert_close(output[:, :10].float(), expected.float(), atol=0.015625, rtol=0.02) -+ assert torch.count_nonzero(output[:, 10:]) == 0 -+ state.finish_context_block(0, dummy) -+ state.end_step() -+ previous = output.clone() -+ anchors = [tensor.clone() for tensor in state._video_contexts[0]] -+ v = v.clone() -+ v[:, :3] += 1 -+ saved = [t.clone() for t in (q, k, v)] -+ state.begin_step(2) -+ output = context.video_context_attention(q, k, v, 0.25, state) -+ expected = oracle(q[:, :10], k, v, 0.25)[0] -+ torch.testing.assert_close(output[:, :10].float(), expected.float(), atol=0.015625, rtol=0.02) -+ assert not torch.equal(output[:, 3:10], previous[:, 3:10]) -+ assert all(torch.equal(a, b) for a, b in zip(saved, (q, k, v))) -+ assert all(torch.equal(a, b) for a, b in zip(anchors, state._video_contexts[0])) -+ state.finish_context_block(0, dummy) -+ state.end_step() -+ assert state.video_context_refreshes == 2 and state.video_context_reuses == 1 -+ assert state.fresh_video_prefix_calls == 3 -+ state.close() -+ assert state.stats()["live_cache_bytes"] == 0 -+ -+ -+@pytest.mark.parametrize("start,length", [(0, 4), (1, 2)]) -+def test_real_block_keeps_all_ffn_rows_fresh_on_context_reuse(start, length): -+ state = make_state(seq=4, used=3, prefix=1, start=start, length=length) -+ block = tiny_block() -+ -+ def attention(x, **kwargs): -+ cache = kwargs.get("video_row_cache") -+ if cache is not None: -+ if cache.partial: -+ cache.read_video_context() -+ else: -+ cache.store_video_context(torch.zeros(1, 2, 1, 2, dtype=torch.bfloat16), torch.zeros(1, 2, 1, 1)) -+ cache.record_context_attention() -+ return x * 0.5 -+ -+ block.attn = attention -+ x = torch.ones(length, 2, dtype=torch.bfloat16) -+ kwargs = dict( -+ t_emb=torch.zeros(1), -+ combined_indices=torch.zeros(length, dtype=torch.long), -+ rope_table=None, -+ cu_seqlens=torch.tensor([0, 3, 4], dtype=torch.int32), -+ max_seqlen=3, -+ packed_total=4, -+ ) -+ expected = block(x, **kwargs) -+ block.mlp.rows.clear() -+ for step in (0, 1, 2): -+ state.begin_step(step) -+ actual = block(x, video_row_cache=state, **kwargs) -+ assert torch.equal(actual.view(torch.uint8), expected.view(torch.uint8)) -+ state.end_step() -+ assert block.mlp.rows == [length] * 3 -+ assert state.full_row_mlp_blocks == 3 -+ state.close() -+ -+ -+def test_context_lifecycle_requires_actual_attention(): -+ state = make_state() -+ state.begin_step(0) -+ dummy = torch.ones(12, 2, dtype=torch.bfloat16) -+ with pytest.raises(RuntimeError, match="not executed"): -+ state.finish_context_block(0, dummy) -+ state.close() -+ -+ -+def test_long_forward_selects_context_on_full_and_partial_steps(monkeypatch): -+ from vllm_omni.diffusion.attention.backends.abstract import AttentionMetadata -+ from vllm_omni.diffusion.models.minimax_h3.attention.flash_h3 import MiniMaxH3FlashAttentionImpl -+ -+ monkeypatch.setenv("MINDIE_SD_FA_TYPE", "fused_attn_score") -+ state = make_state(seq=4192, used=4160, prefix=64) -+ impl = object.__new__(MiniMaxH3FlashAttentionImpl) -+ impl.causal, impl.softmax_scale = False, 0.25 -+ impl._resolve_packed_seq_npu = lambda *args: ([4160], [4160]) -+ q = torch.ones(1, 4192, 1, 128, dtype=torch.bfloat16) -+ metadata = AttentionMetadata(extra={"npu_attn_varlen": True, "minimax_h3_video_row_cache": state}) -+ calls = [] -+ -+ def selected(query, key, value, scale, cache): -+ assert query is q and key.shape[1] == value.shape[1] == 4160 -+ calls.append(cache.partial) -+ if cache.partial: -+ cache.read_video_context() -+ else: -+ cache.store_video_context(torch.zeros(1, 4096, 1, 128, dtype=torch.bfloat16), torch.zeros(1, 4096, 1, 1)) -+ cache.record_context_attention() -+ return q -+ -+ monkeypatch.setattr(context, "video_context_attention", selected) -+ dummy = torch.ones(4192, 2, dtype=torch.bfloat16) -+ for step in (0, 1, 2): -+ state.begin_step(step) -+ assert impl.forward_fa_npu(q, q, q, metadata) is q -+ state.finish_context_block(0, dummy) -+ state.end_step() -+ assert calls == [False, False, True] -+ state.close() -+ -+ -+def _native_worker(rank): -+ import torch_npu # noqa: F401 -+ -+ torch.npu.set_device(rank) -+ dist.init_process_group( -+ "hccl", init_method="tcp://127.0.0.1:29597", rank=rank, world_size=4, timeout=timedelta(seconds=120) -+ ) -+ try: -+ torch.manual_seed(223) -+ full = [torch.randn(32, 4, 128, dtype=torch.bfloat16).npu() for _ in range(3)] -+ state = make_state(seq=32, used=30, prefix=8, device="npu", start=rank * 8, length=8) -+ dummy = torch.ones(8, 2, dtype=torch.bfloat16, device="npu") -+ for step in (0, 1, 2): -+ state.begin_step(step) -+ if step == 2: -+ full[2][:8] += 0.125 -+ exchanged = [] -+ for tensor in full: -+ send = tensor[rank * 8 : (rank + 1) * 8].reshape(8, 4, 1, 128).permute(1, 0, 2, 3).contiguous() -+ receive = torch.empty_like(send) -+ dist.all_to_all_single(receive, send) -+ exchanged.append(receive.reshape(1, 32, 1, 128)) -+ output = context.video_context_attention( -+ exchanged[0], exchanged[1][:, :30], exchanged[2][:, :30], 128**-0.5, state -+ ) -+ send = output.reshape(4, 8, 1, 128).contiguous() -+ receive = torch.empty_like(send) -+ dist.all_to_all_single(receive, send) -+ local = receive.permute(1, 0, 2, 3).reshape(8, 4, 128) -+ expected = _prefix_bnsd( -+ full[0].unsqueeze(0), full[1][:30].unsqueeze(0), full[2][:30].unsqueeze(0), 128**-0.5 -+ )[0] -+ valid = min(8, max(30 - rank * 8, 0)) -+ torch.testing.assert_close( -+ local[:valid].float(), expected[rank * 8 : rank * 8 + valid].float(), atol=0.03125, rtol=0.02 -+ ) -+ state.finish_context_block(0, dummy) -+ state.end_step() -+ assert state.video_context_refreshes == 2 and state.video_context_reuses == 1 -+ assert state.local_dense_positions.numel() == [8, 0, 0, 0][rank] -+ state.close() -+ assert state.stats()["live_cache_bytes"] == 0 -+ if rank == 0: -+ # Regression shape exceeded the first manual merge grid. Native -+ # tensor operations must cover the complete output and its tail. -+ shape = (1, 65536 * 8 + 1, 1, 128) -+ a = torch.zeros(shape, dtype=torch.bfloat16, device="npu") -+ b = torch.ones_like(a) -+ lse = torch.zeros(shape[:-1] + (1,), dtype=torch.float32, device="npu") -+ mixed = context.mix_context(a, b, lse, lse) -+ assert torch.equal(mixed, torch.full_like(a, 0.5)) -+ torch.npu.synchronize() -+ dist.barrier() -+ finally: -+ dist.destroy_process_group() -+ -+ -+@pytest.mark.skipif(not current_omni_platform.is_npu(), reason="Requires four native Ascend devices") -+def test_native_four_rank_video_context_lse_merge_and_prefix_updates(): -+ torch.multiprocessing.spawn(_native_worker, nprocs=4, join=True) -diff --git a/tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py b/tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py -new file mode 100644 -index 0000000..94ff958 ---- /dev/null -+++ b/tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py -@@ -0,0 +1,406 @@ -+# SPDX-License-Identifier: Apache-2.0 -+# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project -+ -+from datetime import timedelta -+from types import SimpleNamespace -+ -+import pytest -+import torch -+import torch.distributed as dist -+import torch.nn as nn -+ -+from tests.helpers.mark import hardware_marks -+from vllm_omni.diffusion.models.minimax_h3 import minimax_h3_blocks as blocks -+from vllm_omni.diffusion.models.minimax_h3.attention.flash_h3 import _prefix_bnsd, _subset_prefix_bnsd -+from vllm_omni.diffusion.models.minimax_h3.video_row_cache import MiniMaxH3VideoRowCache -+from vllm_omni.platforms import current_omni_platform -+ -+pytestmark = [pytest.mark.core_model, pytest.mark.diffusion, *hardware_marks(res={"npu": "A3"}, num_cards=4)] -+ -+ -+def state_for_rows(device="cpu", start=0, length=4): -+ state = MiniMaxH3VideoRowCache(torch.tensor([1, 2], device=device), seq_len=4, used_len=3, num_layers=1) -+ state.prepare_local_span(start, length) -+ return state -+ -+ -+def anchor(state, x): -+ for step in range(5): -+ state.begin_step(step) -+ state.capture(0, x, x + 0.125) -+ state.end_step() -+ state.begin_step(5) -+ -+ -+def test_fixed_schedule_and_row_ownership_excludes_padding(): -+ assert [i for i in range(50) if MiniMaxH3VideoRowCache.is_full_step(i)] == [ -+ 0, -+ 1, -+ 2, -+ 3, -+ 4, -+ 9, -+ 14, -+ 19, -+ 24, -+ 29, -+ 34, -+ 39, -+ 44, -+ 46, -+ 47, -+ 48, -+ 49, -+ ] -+ state = state_for_rows() -+ assert state.global_dense_positions.tolist() == [0] -+ assert state.local_dense_positions.tolist() == [0] -+ assert state.local_video_positions.tolist() == [1, 2] -+ with pytest.raises(RuntimeError, match="ownership"): -+ state.prepare_local_span(0, 3) -+ -+ -+@pytest.mark.parametrize("positions", [[1, 1], [2, 1], [-1], [3]]) -+def test_invalid_target_positions_are_rejected(positions): -+ with pytest.raises(ValueError): -+ MiniMaxH3VideoRowCache(torch.tensor(positions), seq_len=4, used_len=3, num_layers=1) -+ -+ -+def test_delta_is_fp32_and_restore_owns_output(): -+ state = state_for_rows() -+ x = torch.tensor([[1, 2], [3, 4], [5, 6], [7, 8]], dtype=torch.bfloat16) -+ anchor(state, x) -+ assert state._deltas[0].dtype == torch.float32 -+ before = x.clone() -+ output = state.restore(0, x) -+ assert torch.equal(output[1:3], x[1:3] + 0.125) -+ assert torch.equal(output[[0, 3]], x[[0, 3]]) -+ output.zero_() -+ assert torch.equal(x, before) -+ state.end_step() -+ state.close() -+ assert state.closed and not state._deltas and state.global_dense_positions.numel() == 0 -+ -+ -+def large_delta_roundtrip(device="cpu"): -+ state = state_for_rows(device=device) -+ x = torch.full((4, 2), 100000, dtype=torch.bfloat16, device=device) -+ expected = -x -+ delta = expected.float() - x.float() -+ assert torch.isfinite(delta).all() -+ assert not torch.isfinite(delta.to(torch.float16)).all() -+ for step in range(5): -+ state.begin_step(step) -+ state.capture(0, x, expected) -+ state.end_step() -+ state.begin_step(5) -+ actual = state.restore(0, x) -+ assert torch.isfinite(actual).all() -+ assert torch.equal(actual[1:3].cpu().view(torch.uint8), expected[1:3].cpu().view(torch.uint8)) -+ assert torch.equal(actual[[0, 3]], x[[0, 3]]) -+ state.end_step() -+ state.close() -+ assert state.stats()["live_cache_bytes"] == 0 -+ -+ -+def test_large_finite_bf16_delta_roundtrip_exceeds_fp16_storage_range(): -+ large_delta_roundtrip() -+ -+ -+class CountMLP(nn.Module): -+ def __init__(self): -+ super().__init__() -+ self.rows = [] -+ -+ def forward(self, x): -+ self.rows.append(x.shape[0]) -+ return x + 0.25 -+ -+ -+def tiny_block(): -+ block = object.__new__(blocks.MiniMaxH3DiTBlock) -+ nn.Module.__init__(block) -+ block.layer_index = 0 -+ block.norm1 = SimpleNamespace(weight=torch.ones(2), variance_epsilon=1e-6) -+ block.norm2 = SimpleNamespace(weight=torch.ones(2), variance_epsilon=1e-6) -+ block.adaln_proj = lambda t: ( -+ torch.zeros(3, 2), -+ torch.zeros(3, 2), -+ torch.ones(3, 2), -+ torch.zeros(3, 2), -+ torch.zeros(3, 2), -+ torch.ones(3, 2), -+ ) -+ block.attn = lambda x, **kwargs: x * 0.5 -+ block.mlp = CountMLP() -+ return block -+ -+ -+@pytest.mark.parametrize("start,length", [(0, 4), (1, 2)]) -+def test_real_block_dense_path_and_partial_mlp_empty_rows(start, length): -+ state = state_for_rows(start=start, length=length) -+ block = tiny_block() -+ x = torch.arange(length * 2).reshape(length, 2).to(torch.bfloat16) -+ before = x.clone() -+ kwargs = dict( -+ t_emb=torch.zeros(1), -+ combined_indices=torch.zeros(length, dtype=torch.long), -+ rope_table=None, -+ cu_seqlens=torch.tensor([0, 3, 4], dtype=torch.int32), -+ max_seqlen=3, -+ packed_total=4, -+ ) -+ expected = block(x, **kwargs) -+ for step in range(5): -+ state.begin_step(step) -+ actual = block(x, video_row_cache=state, **kwargs) -+ assert torch.equal(actual.view(torch.uint8), expected.view(torch.uint8)) -+ state.end_step() -+ block.mlp.rows.clear() -+ state.begin_step(5) -+ actual = block(x, video_row_cache=state, **kwargs) -+ state.end_step() -+ rows = state.local_dense_positions -+ assert block.mlp.rows == ([rows.numel()] if rows.numel() else []) -+ assert torch.equal(actual.index_select(0, rows), expected.index_select(0, rows)) -+ assert torch.equal(x, before) -+ -+ -+@pytest.mark.parametrize("failure", ["success", "model", "cancel"]) -+def test_request_state_closed_and_removed_on_every_exit(monkeypatch, failure): -+ from vllm_omni.diffusion.models.minimax_h3 import denoise_loop as loop -+ -+ monkeypatch.setenv("H3_VIDEO_ROW_CACHE", "1") -+ monkeypatch.setattr(loop.current_omni_platform, "is_npu", lambda: False) -+ monkeypatch.setattr(loop, "minimax_h3_publish_denoise_progress", lambda *args: None) -+ state = MiniMaxH3VideoRowCache(torch.tensor([1]), seq_len=4, used_len=3, num_layers=1) -+ -+ class Model: -+ def create_video_row_cache(self, *args, **kwargs): -+ return state -+ -+ def __call__(self, **kwargs): -+ assert kwargs["video_row_cache"] is state -+ if failure == "model": -+ raise RuntimeError("model failed") -+ state.prepare_local_span(0, 4) -+ x = torch.ones(4, 2, dtype=torch.bfloat16) -+ if state.partial: -+ state.restore(0, x) -+ else: -+ state.capture(0, x, x + 0.125) -+ return torch.zeros(2, 96), torch.zeros(1, 32) -+ -+ class Branch: -+ task = "ref2va" -+ seq_len = 4 -+ used_len = 3 -+ img_pos = torch.tensor([0, 1]) -+ update_mask = torch.tensor([False, True]) -+ update_mask_dev = update_mask -+ audio_update_mask_dev = torch.tensor([True]) -+ locked_audio_rows = None -+ static_kwargs = {"video_token_layout": SimpleNamespace(video_spans=[SimpleNamespace(role="target", start=1)])} -+ -+ def forward_kwargs(self, **kwargs): -+ return dict(self.static_kwargs) -+ -+ monkeypatch.setattr( -+ loop, -+ "minimax_h3_prepare_denoise_rows", -+ lambda **kwargs: (kwargs["initial_video_rows"], kwargs["initial_audio_rows"], torch.zeros(1, 96), None), -+ ) -+ -+ def cancellation(**kwargs): -+ if failure == "cancel": -+ raise RuntimeError("request cancelled") -+ -+ monkeypatch.setattr(loop, "check_request_cancellation", cancellation) -+ branch, model = Branch(), Model() -+ kwargs = dict( -+ model=model, -+ positive=branch, -+ initial_video_rows=torch.zeros(2, 96), -+ initial_audio_rows=torch.zeros(1, 32), -+ keyframe_cond_rows=torch.zeros(1, 96), -+ sigmas_video=[1 - i / 50 for i in range(51)], -+ sigmas_audio=[1 - i / 50 for i in range(51)], -+ device=torch.device("cpu"), -+ ) -+ if failure == "success": -+ loop.minimax_h3_denoise_loop(**kwargs) -+ assert state.full_steps == 17 and state.partial_steps == 33 -+ else: -+ with pytest.raises(RuntimeError): -+ loop.minimax_h3_denoise_loop(**kwargs) -+ assert state.closed and not state._deltas and "video_row_cache" not in branch.static_kwargs -+ assert model._video_row_cache_stats["closed"] -+ -+ -+def test_default_factory_does_not_create_state(monkeypatch): -+ from vllm_omni.diffusion.models.minimax_h3.minimax_h3_transformer import MiniMaxH3DiTModel -+ -+ monkeypatch.delenv("H3_VIDEO_ROW_CACHE", raising=False) -+ assert ( -+ MiniMaxH3DiTModel.create_video_row_cache(None, torch.tensor([1]), seq_len=4, used_len=3, num_steps=50) is None -+ ) -+ -+ -+@pytest.mark.parametrize("ref2va", [False, True]) -+def test_real_packing_task_selects_ref2va_and_rejects_fl2va(monkeypatch, ref2va): -+ from vllm_omni.diffusion.models.minimax_h3 import denoise_loop as loop -+ from vllm_omni.diffusion.models.minimax_h3.packed_sequence import ( -+ minimax_h3_packed_sequence, -+ minimax_h3_packed_sequence_ref2va_blocks, -+ ) -+ -+ common = dict(text_len=2, latent_t=2, latent_h=4, latent_w=4, audio_t=2) -+ packed = ( -+ minimax_h3_packed_sequence_ref2va_blocks(**common, ref_blocks=[{"kind": "image", "latent_h": 4, "latent_w": 4}]) -+ if ref2va -+ else minimax_h3_packed_sequence(**common, include_keyframe_cond=True, keyframe_frame_indices=[0], frame_count=5) -+ ) -+ branch = loop.MiniMaxH3DenoiseBranch( -+ packed=packed, -+ text_embeddings=torch.zeros(2, 5120), -+ token_tags=packed["token_tags"], -+ device=torch.device("cpu"), -+ task="ref2va" if ref2va else "fl2va", -+ ) -+ layout = branch.static_kwargs["video_token_layout"] -+ assert not any(span.role == "reference" for span in layout.video_spans) -+ monkeypatch.setenv("H3_VIDEO_ROW_CACHE", "1") -+ monkeypatch.setattr(loop.current_omni_platform, "is_npu", lambda: False) -+ monkeypatch.setattr(loop, "minimax_h3_publish_denoise_progress", lambda *args: None) -+ monkeypatch.setattr(loop, "check_request_cancellation", lambda **kwargs: None) -+ states = [] -+ -+ class Model: -+ def create_video_row_cache(self, positions, **kwargs): -+ state = MiniMaxH3VideoRowCache( -+ positions, seq_len=kwargs["seq_len"], used_len=kwargs["used_len"], num_layers=1 -+ ) -+ state.prepare_local_span(0, branch.seq_len) -+ states.append(state) -+ return state -+ -+ def __call__(self, **kwargs): -+ state = kwargs["video_row_cache"] -+ x = torch.ones(branch.seq_len, 2, dtype=torch.bfloat16) -+ if state.partial: -+ state.restore(0, x) -+ else: -+ state.capture(0, x, x + 0.125) -+ return torch.zeros(branch.img_pos.numel(), 96), torch.zeros(branch.audio_pos.numel(), 32) -+ -+ kwargs = dict( -+ model=Model(), -+ positive=branch, -+ initial_video_rows=torch.zeros(branch.img_pos.numel(), 96), -+ initial_audio_rows=torch.zeros(branch.audio_pos.numel(), 32), -+ keyframe_cond_rows=torch.zeros(int((~branch.update_mask).sum()), 96), -+ sigmas_video=[1 - i / 50 for i in range(51)], -+ sigmas_audio=[1 - i / 50 for i in range(51)], -+ device=torch.device("cpu"), -+ ) -+ if ref2va: -+ loop.minimax_h3_denoise_loop(**kwargs) -+ assert len(states) == 1 and states[0].closed -+ assert states[0].full_steps == 17 and states[0].partial_steps == 33 -+ else: -+ with pytest.raises(ValueError, match="requires Ref2VA"): -+ loop.minimax_h3_denoise_loop(**kwargs) -+ assert not states -+ -+ -+@pytest.mark.parametrize("start,length", [(0, 4), (1, 2)]) -+def test_attention_keeps_collective_path_and_projects_only_dense_local_rows(start, length): -+ state = state_for_rows(start=start, length=length) -+ x = torch.ones(length, 2, dtype=torch.bfloat16) -+ anchor(state, x) -+ attention = object.__new__(blocks.MiniMaxH3Attention) -+ nn.Module.__init__(attention) -+ attention.num_heads = 1 -+ attention.head_dim = 2 -+ attention.vdn = None -+ attention.to_gate_compress = None -+ calls, projected_rows = [], [] -+ attention._project_qkv = lambda *args: (x.reshape(length, 1, 2),) * 5 -+ -+ def packed(q, k, v, **kwargs): -+ calls.append(q.shape[0]) -+ assert kwargs["video_row_cache"] is state -+ return q -+ -+ def project(rows): -+ projected_rows.append(rows.shape[0]) -+ return rows * 2, None -+ -+ attention._run_packed_attention = packed -+ attention.out_proj = project -+ output = attention( -+ x, -+ rope_table=None, -+ cu_seqlens=torch.tensor([0, 3, 4], dtype=torch.int32), -+ max_seqlen=3, -+ packed_total=4, -+ video_row_cache=state, -+ ) -+ rows = state.local_dense_positions -+ assert calls == [length] -+ assert projected_rows == ([rows.numel()] if rows.numel() else []) -+ assert torch.equal(output.index_select(0, rows), (x * 2).index_select(0, rows)) -+ assert torch.count_nonzero(output.index_select(0, state.local_video_positions)) == 0 -+ -+ -+def _native_worker(rank): -+ import torch_npu # noqa: F401 -+ -+ torch.npu.set_device(rank) -+ dist.init_process_group( -+ "hccl", init_method="tcp://127.0.0.1:29597", rank=rank, world_size=4, timeout=timedelta(seconds=120) -+ ) -+ try: -+ large_delta_roundtrip(device=f"npu:{rank}") -+ torch.manual_seed(19) -+ full = [torch.randn(32, 4, 128, dtype=torch.bfloat16).npu() for _ in range(3)] -+ exchanged = [] -+ for tensor in full: -+ local = tensor[rank * 8 : (rank + 1) * 8] -+ send = local.reshape(8, 4, 1, 128).permute(1, 0, 2, 3).contiguous() -+ receive = torch.empty_like(send) -+ dist.all_to_all_single(receive, send) -+ exchanged.append(receive.reshape(1, 32, 1, 128)) -+ state = MiniMaxH3VideoRowCache(torch.arange(8, 24, device="npu"), seq_len=32, used_len=30, num_layers=1) -+ state.prepare_local_span(rank * 8, 8) -+ x = torch.ones(8, 2, dtype=torch.bfloat16, device="npu") -+ anchor(state, x) -+ output = _subset_prefix_bnsd(exchanged[0], exchanged[1][:, :30], exchanged[2][:, :30], 128**-0.5, state) -+ send = output.reshape(4, 8, 1, 128).contiguous() -+ receive = torch.empty_like(send) -+ dist.all_to_all_single(receive, send) -+ local_output = receive.permute(1, 0, 2, 3).reshape(8, 4, 128) -+ expected = _prefix_bnsd(full[0].unsqueeze(0), full[1][:30].unsqueeze(0), full[2][:30].unsqueeze(0), 128**-0.5)[ -+ 0 -+ ] -+ rows = state.local_dense_positions -+ torch.testing.assert_close( -+ local_output.index_select(0, rows).float(), -+ expected[rank * 8 : (rank + 1) * 8].index_select(0, rows).float(), -+ atol=0.03125, -+ rtol=0.02, -+ ) -+ assert state.subset_attention_calls == 1 and state.subset_query_rows == 14 -+ assert rows.numel() == [8, 0, 0, 6][rank] -+ state.restore(0, x) -+ state.end_step() -+ state.close() -+ torch.npu.synchronize() -+ dist.barrier() -+ finally: -+ dist.destroy_process_group() -+ -+ -+@pytest.mark.skipif(not current_omni_platform.is_npu(), reason="Requires four native Ascend devices") -+def test_native_four_rank_subset_attention_with_empty_local_dense_rows(): -+ torch.multiprocessing.spawn(_native_worker, nprocs=4, join=True) -diff --git a/vllm_omni/config/omni_config.py b/vllm_omni/config/omni_config.py -index 0a803cf..44fc48c 100644 ---- a/vllm_omni/config/omni_config.py -+++ b/vllm_omni/config/omni_config.py -@@ -863,6 +863,7 @@ class _DiffusionConfigProjection(_TrackExplicitConfigFields): - dlo_use_allgather: bool = True - dlo_chunk_size_mb: int = Field(default=64, gt=0, strict=True) - dlo_resident_layers: int = Field(default=0, ge=0) -+ dlo_order_allgather_after_attn_a2a: bool = False - host_weight_runtime_mode: Literal["disabled", "preferred", "required"] = "disabled" - host_weight_runtime_root: str | None = None - dlo_host_registration_limit_gib: float = Field(default=0.0, ge=0) -diff --git a/vllm_omni/config/stage_config.py b/vllm_omni/config/stage_config.py -index 76a58a9..3c69986 100644 ---- a/vllm_omni/config/stage_config.py -+++ b/vllm_omni/config/stage_config.py -@@ -672,6 +672,7 @@ class StageDeployConfig: - dlo_use_allgather: bool | None = None - dlo_chunk_size_mb: int | None = None - dlo_resident_layers: int | None = None -+ dlo_order_allgather_after_attn_a2a: bool | None = None - host_weight_runtime_mode: str | None = None - host_weight_runtime_root: str | None = None - dlo_host_registration_limit_gib: float | None = None -diff --git a/vllm_omni/diffusion/attention/layer.py b/vllm_omni/diffusion/attention/layer.py -index 5520390..574eafa 100644 ---- a/vllm_omni/diffusion/attention/layer.py -+++ b/vllm_omni/diffusion/attention/layer.py -@@ -9,7 +9,7 @@ - - from collections.abc import Mapping - from dataclasses import replace --from typing import cast -+from typing import Any, cast - - import torch - import torch.nn as nn -@@ -333,6 +333,11 @@ class Attention(nn.Module): - # Local strategy when SP is intentionally inactive outside sharded regions. - self._no_parallel_strategy = NoParallelAttention() - -+ # Optional cross-module rendezvous (see distributed/collective_gate.py). -+ # Installed by the DLO backend when it must keep its background weight -+ # AllGather from taking the links ahead of this layer's QKV AllToAll. -+ self._collective_gate: Any | None = None -+ - self.layer_idx: int | None = _try_extract_layer_index(prefix) - - self._kv_cache_dtype: str | None = None -@@ -506,10 +511,12 @@ class Attention(nn.Module): - reason = self._cross_backend_limitation(attn_metadata) - if reason is not None: - blocked = self._cross_fallback_blocker(attn_metadata) -- result = self.sdpa_fallback.resolve_execution_path(resolved_context, query, key, value, attn_metadata) -+ result = cast(SDPAImpl, self.sdpa_fallback).resolve_execution_path( -+ resolved_context, query, key, value, attn_metadata -+ ) - return replace( - result, -- backend=self.attn_backend.get_name() if blocked else "SDPA", -+ backend=cast(type[AttentionBackend], self.attn_backend).get_name() if blocked else "SDPA", - path="cross_unsupported" if blocked else "cross_sdpa_fallback", - support=CapabilityResult.unsupported(blocked) if blocked else CapabilityResult.unmigrated(reason), - ) -@@ -591,6 +598,12 @@ class Attention(nn.Module): - # For Ring: Concat joint_q - query, key, value, attn_metadata, ctx = strategy.pre_attention(query, key, value, attn_metadata) - -+ # The QKV AllToAll is now enqueued on this stream. Release any gather -+ # that was deferred to give this collective the links first. -+ gate = self._collective_gate -+ if gate is not None: -+ gate.signal() -+ - # MiniMax-H3 Tier2 stores reference K/V in the balanced layout produced - # by Ulysses. Keeping this hook at the generic post-parallel boundary - # avoids a model import in the parallel strategy and leaves other models -@@ -760,7 +773,7 @@ class Attention(nn.Module): - def _cross_fallback_blocker(self, attn_metadata: AttentionMetadata | None) -> str | None: - if self.backend_explicit: - return ( -- f"Explicit backend {self.attn_backend.get_name()} does not support " -+ f"Explicit backend {cast(type[AttentionBackend], self.attn_backend).get_name()} does not support " - f"{self._cross_backend_limitation(attn_metadata)}" - ) - if attn_metadata is not None and attn_metadata.extra.get("kv_cache_dtype") not in (None, "float"): -@@ -793,7 +806,7 @@ class Attention(nn.Module): - blocked = self._cross_fallback_blocker(attn_metadata) - if blocked is not None: - raise NotImplementedError(blocked) -- return self.sdpa_fallback.forward(query, key, value, attn_metadata) -+ return cast(SDPAImpl, self.sdpa_fallback).forward(query, key, value, attn_metadata) - dense_mask = attn_metadata.attn_mask if attn_metadata is not None else None - # Piecewise backends honor full_attn_spans ahead of attn_mask and never read the - # dense mask (e.g. HunyuanImage3, which sends both), so there is nothing to divert. -diff --git a/vllm_omni/diffusion/data.py b/vllm_omni/diffusion/data.py -index 8528574..fcbeec0 100644 ---- a/vllm_omni/diffusion/data.py -+++ b/vllm_omni/diffusion/data.py -@@ -962,6 +962,10 @@ class OmniDiffusionConfig: - dlo_chunk_size_mb: int = 64 - # Leading main-DiT blocks kept resident by distributed layerwise offload. - dlo_resident_layers: int = 0 -+ # Hold each DiT block's weight AllGather until that block's QKV AllToAll has -+ # been enqueued, so the background collective does not take the links ahead -+ # of the critical-path all-to-all. -+ dlo_order_allgather_after_attn_a2a: bool = False - # Final-layout Host Weight Runtime policy. The loader only activates this - # for eligible no-AllGather DLO; all other configurations preserve their - # existing loader/storage path. -diff --git a/vllm_omni/diffusion/distributed/collective_gate.py b/vllm_omni/diffusion/distributed/collective_gate.py -new file mode 100644 -index 0000000..1386d89 ---- /dev/null -+++ b/vllm_omni/diffusion/distributed/collective_gate.py -@@ -0,0 +1,72 @@ -+# SPDX-License-Identifier: Apache-2.0 -+# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project -+"""Cross-module rendezvous that orders background collectives after the -+foreground all-to-all of a transformer block. -+ -+Distributed layerwise offload (DLO) issues the *next* block's weight AllGather -+on a dedicated ``comm_stream`` while the *current* block computes. Both that -+AllGather and the block's QKV sequence-parallel AllToAll are collectives over -+the same ranks, so whichever reaches the links first wins: the AllGather is -+submitted at block entry, well before the block reaches its attention, and the -+critical-path AllToAll ends up queued behind it. -+ -+A plain event wait cannot fix this by itself. ``wait_event`` snapshots the -+records enqueued *before* the call, so "enqueue the wait at block entry, record -+the event later in the block" is a no-op -- the wait resolves against the -+previous iteration's record. The gather therefore has to be *issued* after the -+AllToAll: the hook arms this barrier at block entry and defers the gather, the -+attention signals it right after ``strategy.pre_attention``, and the signal -+records the event and lets the hook submit the gather gated on it. -+ -+The gather still overlaps the rest of the block (attention output, MLP, the -+next block's projections); it simply cannot start before the AllToAll finished. -+""" -+ -+from __future__ import annotations -+ -+from collections.abc import Callable -+ -+__all__ = ["QKVAllToAllBarrier"] -+ -+ -+class QKVAllToAllBarrier: -+ """One-block rendezvous between a DLO hook and the block's attention. -+ -+ Owned by a single :class:`DistributedLayerwiseOffloadHook` (one per -+ streaming block) and shared with the attention layer inside that block. -+ Both live on the same block, so the arming and the signalling always come -+ from the same module forward. -+ -+ Args: -+ on_signal: Invoked at most once per :meth:`arm` when the block's -+ attention reports its QKV AllToAll. It must submit the deferred -+ weight AllGather. -+ """ -+ -+ def __init__(self, on_signal: Callable[[], None]) -> None: -+ self._on_signal = on_signal -+ self._armed = False -+ -+ @property -+ def armed(self) -> bool: -+ """True between :meth:`arm` and the matching :meth:`signal`.""" -+ return self._armed -+ -+ def arm(self) -> None: -+ """Mark the next attention QKV AllToAll as the gather's gate.""" -+ self._armed = True -+ -+ def disarm(self) -> None: -+ """Drop a pending arm without firing it (the caller flushes instead).""" -+ self._armed = False -+ -+ def signal(self) -> None: -+ """Report that the QKV AllToAll is now enqueued on the compute stream. -+ -+ A no-op unless armed, so it is cheap to call unconditionally from the -+ attention hot path. -+ """ -+ if not self._armed: -+ return -+ self._armed = False -+ self._on_signal() -diff --git a/vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py b/vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py -index 6ebfb10..da008bf 100644 ---- a/vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py -+++ b/vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py -@@ -36,6 +36,16 @@ def _prefix_bnsd(query: torch.Tensor, key: torch.Tensor, value: torch.Tensor, sc - return output.reshape(batch, heads, rows, dim).transpose(1, 2).contiguous() - - -+def _subset_prefix_bnsd(query, key, value, scale, state): -+ """Compute only fresh conditioning queries; preserve the shared SP shape.""" -+ rows = state.global_dense_positions -+ dense = _prefix_bnsd(query.index_select(1, rows), key, value, scale) -+ output = torch.zeros_like(query) -+ output.index_copy_(1, rows, dense) -+ state.record_subset_attention(rows.numel()) -+ return output -+ -+ - class MiniMaxH3FlashAttentionImpl(FlashAttentionImpl): - """Use independent head batches for explicit long BF16 prefix attention. - -@@ -51,7 +61,10 @@ class MiniMaxH3FlashAttentionImpl(FlashAttentionImpl): - value: torch.Tensor, - attn_metadata: AttentionMetadata | None = None, - ) -> torch.Tensor: -+ state = attn_metadata.extra.get("minimax_h3_video_row_cache") if attn_metadata is not None else None - if os.environ.get("MINDIE_SD_FA_TYPE") != "fused_attn_score" or attn_metadata is None: -+ if state is not None: -+ raise RuntimeError("H3 video-row cache requires the explicit fused attention path") - return super().forward_fa_npu(query, key, value, attn_metadata) - extra = attn_metadata.extra - if ( -@@ -68,9 +81,25 @@ class MiniMaxH3FlashAttentionImpl(FlashAttentionImpl): - or query.shape[2:] != value.shape[2:] - or key.shape[1] != value.shape[1] - ): -+ if state is not None: -+ raise RuntimeError("H3 video-row cache received unsupported fused attention inputs") - return super().forward_fa_npu(query, key, value, attn_metadata) - resolved = self._resolve_packed_seq_npu(query, key, extra) - if resolved is None: -+ if state is not None: -+ raise RuntimeError("H3 video-row cache requires a resolved packed prefix") - return super().forward_fa_npu(query, key, value, attn_metadata) - used_k = resolved[1][0] -+ if state is not None: -+ if ( -+ (not state.partial and not state.context_only) -+ or used_k != state.used_len -+ or query.shape[1] != state.seq_len -+ ): -+ raise RuntimeError("H3 video-row cache attention layout changed") -+ if state.context_only: -+ from .video_context import video_context_attention -+ -+ return video_context_attention(query, key[:, :used_k], value[:, :used_k], self.softmax_scale, state) -+ return _subset_prefix_bnsd(query, key[:, :used_k], value[:, :used_k], self.softmax_scale, state) - return _prefix_bnsd(query, key[:, :used_k], value[:, :used_k], self.softmax_scale) -diff --git a/vllm_omni/diffusion/models/minimax_h3/attention/video_context.py b/vllm_omni/diffusion/models/minimax_h3/attention/video_context.py -new file mode 100644 -index 0000000..4603233 ---- /dev/null -+++ b/vllm_omni/diffusion/models/minimax_h3/attention/video_context.py -@@ -0,0 +1,77 @@ -+# SPDX-License-Identifier: Apache-2.0 -+# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project -+"""Request-owned video context with fresh global conditioning attention.""" -+ -+from __future__ import annotations -+ -+import torch -+ -+from .flash_h3 import _prefix_bnsd -+ -+ -+def prefix_attention_lse(query, key, value, scale): -+ """Use native FAS output and its FP32 softmax statistics in BSND order.""" -+ import torch_npu -+ -+ batch, rows, heads, dim = query.shape -+ -+ def fold(tensor): -+ return tensor.transpose(1, 2).contiguous().reshape(batch * heads, 1, tensor.shape[1], dim) -+ -+ output, maximum, total, *_ = torch_npu.npu_fusion_attention( -+ fold(query), -+ fold(key), -+ fold(value), -+ head_num=1, -+ input_layout="BNSD", -+ scale=scale, -+ pre_tockens=2147483647, -+ next_tockens=2147483647, -+ sync=False, -+ ) -+ output = output.reshape(batch, heads, rows, dim).transpose(1, 2).contiguous() -+ lse = (maximum[..., 0].float() + total[..., 0].float().log()).reshape(batch, heads, rows) -+ return output, lse.transpose(1, 2).unsqueeze(-1).contiguous() -+ -+ -+def mix_context(prefix, video, prefix_lse, video_lse): -+ if ( -+ prefix.shape != video.shape -+ or prefix.dtype != torch.bfloat16 -+ or video.dtype != prefix.dtype -+ or prefix_lse.shape != prefix.shape[:-1] + (1,) -+ or video_lse.shape != prefix_lse.shape -+ or prefix_lse.dtype != torch.float32 -+ or video_lse.dtype != torch.float32 -+ or not all(t.is_contiguous() for t in (prefix, video, prefix_lse, video_lse)) -+ ): -+ raise ValueError("H3 context merge requires contiguous BF16 outputs and FP32 row/head LSE") -+ maximum = torch.maximum(prefix_lse, video_lse) -+ weight_a, weight_b = (prefix_lse - maximum).exp(), (video_lse - maximum).exp() -+ return ((weight_a * prefix.float() + weight_b * video.float()) / (weight_a + weight_b)).to(prefix.dtype) -+ -+ -+def video_context_attention(query, key, value, scale, state): -+ prefix = state.target_start -+ if ( -+ not state.context_only -+ or query.shape[0] != 1 -+ or query.shape[1] != state.seq_len -+ or key.shape[1] != state.used_len -+ or value.shape != key.shape -+ or query.shape[2:] != key.shape[2:] -+ or not 0 < prefix < state.used_len -+ ): -+ raise ValueError("H3 video context requires the final target span and original packed query order") -+ output = torch.zeros_like(query) -+ output[:, :prefix] = _prefix_bnsd(query[:, :prefix], key, value, scale) -+ video_query = query[:, prefix : state.used_len] -+ fresh, fresh_lse = prefix_attention_lse(video_query, key[:, :prefix], value[:, :prefix], scale) -+ if state.partial: -+ video, video_lse = state.read_video_context() -+ else: -+ video, video_lse = prefix_attention_lse(video_query, key[:, prefix:], value[:, prefix:], scale) -+ state.store_video_context(video, video_lse) -+ output[:, prefix : state.used_len] = mix_context(fresh, video, fresh_lse, video_lse) -+ state.record_context_attention() -+ return output -diff --git a/vllm_omni/diffusion/models/minimax_h3/denoise_loop.py b/vllm_omni/diffusion/models/minimax_h3/denoise_loop.py -index 8cca219..07fb372 100644 ---- a/vllm_omni/diffusion/models/minimax_h3/denoise_loop.py -+++ b/vllm_omni/diffusion/models/minimax_h3/denoise_loop.py -@@ -9,6 +9,7 @@ rows stay pinned to their noised step-0 anchors. - - from __future__ import annotations - -+import os - from collections.abc import Callable - from contextlib import AbstractContextManager, nullcontext - from typing import Any -@@ -69,7 +70,9 @@ class MiniMaxH3DenoiseBranch: - text_embeddings: torch.Tensor, - token_tags: torch.Tensor, - device: torch.device, -+ task: str | None = None, - ) -> None: -+ self.task = task - seq_len = int(packed["seq_len"]) - self.seq_len = seq_len - self.img_pos = packed["img_pos"].view(-1).to(torch.long) -@@ -380,6 +383,24 @@ def minimax_h3_denoise_loop( - positive.static_kwargs["reference_kv_state"] = reference_kv_state - - try: -+ video_row_cache = None -+ create_video_cache = getattr(model, "create_video_row_cache", None) -+ if os.environ.get("H3_VIDEO_ROW_CACHE") == "1": -+ if ( -+ not callable(create_video_cache) -+ or video_edit is not None -+ or audio_edit is not None -+ or keyframe_cond_rows is None -+ or positive.task != "ref2va" -+ ): -+ raise ValueError("H3 video-row cache requires Ref2VA image references without editing") -+ video_row_cache = create_video_cache( -+ positive.img_pos[positive.update_mask].to(device), -+ seq_len=positive.seq_len, -+ used_len=positive.used_len, -+ num_steps=num_steps, -+ ) -+ positive.static_kwargs["video_row_cache"] = video_row_cache - for step in range(num_steps): - check_request_cancellation() - step_cm = step_profiler(step) if step_profiler is not None else nullcontext() -@@ -388,6 +409,8 @@ def minimax_h3_denoise_loop( - s_a = sigmas_audio[step] - if reference_kv_state is not None: - reference_kv_state.begin_step(step, sigma=float(s_v)) -+ if video_row_cache is not None: -+ video_row_cache.begin_step(step) - # Publish where we are so step-gated attention features (the dense - # warmup of RAINFUSION_ATTN, the timestep gate of TRTLLM_ATTN) can - # see it. Gates use the scheduler-style descending timestep, which -@@ -430,6 +453,8 @@ def minimax_h3_denoise_loop( - v_video, v_audio = model(**fk) - if reference_kv_state is not None: - reference_kv_state.end_step() -+ if video_row_cache is not None: -+ video_row_cache.end_step() - mv_video_t = v_video.float()[update] - mv_audio_t = v_audio.float()[audio_update] - -@@ -476,6 +501,10 @@ def minimax_h3_denoise_loop( - - finally: - minimax_h3_publish_denoise_progress(None, None, None) -+ if video_row_cache is not None: -+ positive.static_kwargs.pop("video_row_cache", None) -+ video_row_cache.close() -+ model._video_row_cache_stats = video_row_cache.stats() - if reference_kv_state is not None: - positive.static_kwargs.pop("reference_kv_state", None) - reference_kv_state.close() -diff --git a/vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py b/vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py -index 09f2e7d..ae870ed 100644 ---- a/vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py -+++ b/vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py -@@ -45,6 +45,7 @@ if TYPE_CHECKING: - from vllm.model_executor.layers.quantization.base_config import QuantizationConfig - - from .reference_kv_cache import MiniMaxH3ReferenceKVTier2State -+ from .video_row_cache import MiniMaxH3VideoRowCache - - _BF16_DTYPE = torch.bfloat16 - _FP32_DTYPE = torch.float32 -@@ -291,6 +292,7 @@ class MiniMaxH3Attention(nn.Module): - reference_kv_layer_index: int | None = None, - reference_kv_compact: bool = False, - vdn_window: VDNLayout | None = None, -+ video_row_cache: MiniMaxH3VideoRowCache | None = None, - ) -> torch.Tensor: - """Run packed attention as a small eager island. - -@@ -368,6 +370,8 @@ class MiniMaxH3Attention(nn.Module): - "vsa_h3_ref_policy": self.vsa_ref_policy, - "vsa_h3_ref_keep_rate": self.vsa_ref_keep_rate, - } -+ if video_row_cache is not None and (video_row_cache.partial or video_row_cache.context_only): -+ extra["minimax_h3_video_row_cache"] = video_row_cache - if gate_compress is not None: - extra["gate_compress"] = gate_compress.unsqueeze(0) - if video_layout is not None and video_layout.video_spans: -@@ -455,6 +459,7 @@ class MiniMaxH3Attention(nn.Module): - reference_kv_layer_index: int | None = None, - reference_kv_compact: bool = False, - vdn_window: VDNLayout | None = None, -+ video_row_cache: MiniMaxH3VideoRowCache | None = None, - ) -> torch.Tensor: - """x: [T, hidden] packed thd rows -> [T, hidden]. - -@@ -508,10 +513,19 @@ class MiniMaxH3Attention(nn.Module): - reference_kv_layer_index=reference_kv_layer_index, - reference_kv_compact=reference_kv_compact, - vdn_window=vdn_window, -+ **({"video_row_cache": video_row_cache} if video_row_cache is not None else {}), - ) - if self.vdn is not None: - out = self.vdn.gate_softmax(out, x) - out = out.reshape(total, self.num_heads * self.head_dim) -+ if video_row_cache is not None and video_row_cache.partial and not video_row_cache.context_only: -+ rows = video_row_cache.local_dense_positions -+ assert rows is not None -+ projected = x.new_zeros((total, x.shape[-1])) -+ if rows.numel(): -+ dense, _ = self.out_proj(out.index_select(0, rows)) -+ projected.index_copy_(0, rows, dense) -+ return projected - out, _ = self.out_proj(out) - if self.vdn is not None: - out = self.vdn.add_linear(out, x, q_raw, k_raw, v, vdn_window) -@@ -742,6 +756,7 @@ class MiniMaxH3DiTBlock(nn.Module): - reference_kv_state: MiniMaxH3ReferenceKVTier2State | None = None, - reference_kv_compact: bool = False, - vdn_window: VDNLayout | None = None, -+ video_row_cache: MiniMaxH3VideoRowCache | None = None, - ) -> torch.Tensor: - """x: [T, H]; t_emb: [M, t_dim]; combined_indices: [T] - (= inverse_indices * modality_num + token_tags.clamp(min=0)). -@@ -759,6 +774,7 @@ class MiniMaxH3DiTBlock(nn.Module): - gate_mlp, - ) = self.adaln_proj(t_emb) - -+ block_input = x - residual = x - h = rms_norm_indexed_scale_shift( - x, -@@ -782,7 +798,10 @@ class MiniMaxH3DiTBlock(nn.Module): - reference_kv_layer_index=self.layer_index, - reference_kv_compact=reference_kv_compact, - vdn_window=vdn_window, -+ **({"video_row_cache": video_row_cache} if video_row_cache is not None else {}), - ) -+ if video_row_cache is not None and video_row_cache.pre_gate: -+ h = video_row_cache.pre_gate_output(self.layer_index, h, "attn") - x, h = indexed_gate_rms_norm_scale_shift( - residual, - gate_msa, -@@ -793,6 +812,39 @@ class MiniMaxH3DiTBlock(nn.Module): - combined_indices, - self.norm2.variance_epsilon, - ) -+ if video_row_cache is not None and video_row_cache.pre_gate and video_row_cache.partial: -+ video_row_cache.record_fresh_gate("attn") - residual = x -+ if video_row_cache is not None and video_row_cache.partial and not video_row_cache.context_only: -+ rows = video_row_cache.local_dense_positions -+ assert rows is not None -+ if video_row_cache.pre_gate: -+ mlp_output = torch.zeros_like(h) -+ if rows.numel(): -+ mlp_output.index_copy_(0, rows, self.mlp(h.index_select(0, rows))) -+ mlp_output = video_row_cache.pre_gate_output(self.layer_index, mlp_output, "mlp") -+ output = indexed_gate(residual, gate_mlp, mlp_output, combined_indices) -+ video_row_cache.record_fresh_gate("mlp") -+ video_row_cache.partial_mlp_rows += rows.numel() -+ return output -+ output = video_row_cache.restore(self.layer_index, block_input) -+ if rows.numel(): -+ dense = indexed_gate( -+ residual.index_select(0, rows), -+ gate_mlp, -+ self.mlp(h.index_select(0, rows)), -+ combined_indices.index_select(0, rows), -+ ) -+ output.index_copy_(0, rows, dense) -+ video_row_cache.partial_mlp_rows += rows.numel() -+ return output - h = self.mlp(h) -- return indexed_gate(residual, gate_mlp, h, combined_indices) -+ if video_row_cache is not None and video_row_cache.pre_gate: -+ h = video_row_cache.pre_gate_output(self.layer_index, h, "mlp") -+ output = indexed_gate(residual, gate_mlp, h, combined_indices) -+ if video_row_cache is not None and not video_row_cache.pre_gate: -+ if video_row_cache.context_only: -+ video_row_cache.finish_context_block(self.layer_index, block_input) -+ else: -+ video_row_cache.capture(self.layer_index, block_input, output) -+ return output -diff --git a/vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py b/vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py -index 9518828..189aa31 100644 ---- a/vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py -+++ b/vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py -@@ -10,6 +10,7 @@ layout. - from __future__ import annotations - - import math -+import os - from bisect import bisect_left - from collections.abc import Iterable, Mapping - from dataclasses import replace -@@ -66,6 +67,8 @@ if TYPE_CHECKING: - from vllm_omni.diffusion.data import OmniDiffusionConfig - from vllm_omni.diffusion.models.minimax_h3.reference_kv_cache import MiniMaxH3ReferenceKVTier2State - -+ from .video_row_cache import MiniMaxH3VideoRowCache -+ - logger = init_logger(__name__) - - -@@ -245,6 +248,7 @@ _FORWARD_SUPPORTED_KWARGS = frozenset( - "control_rows", - "control_context_scale", - "reference_kv_state", -+ "video_row_cache", - } - ) - -@@ -1232,6 +1236,42 @@ class MiniMaxH3DiTModel(nn.Module): - t_emb = self.time_embedder(unique_timesteps) - return embeddings, t_emb - -+ def create_video_row_cache( -+ self, target_positions: torch.Tensor, *, seq_len: int, used_len: int, num_steps: int -+ ) -> MiniMaxH3VideoRowCache | None: -+ if os.environ.get("H3_VIDEO_ROW_CACHE") != "1": -+ return None -+ p = self.parallel_config -+ if ( -+ not current_omni_platform.is_npu() -+ or num_steps != 50 -+ or p.tensor_parallel_size != 1 -+ or p.data_parallel_size not in (None, 1) -+ or p.cfg_parallel_size != 1 -+ or p.pipeline_parallel_size != 1 -+ or p.ulysses_degree != 4 -+ or p.ring_degree != 1 -+ or p.allgather_degree != 1 -+ or p.ulysses_mode != "strict" -+ or self.od_config.cache_backend not in (None, "none") -+ or self.od_config.kv_offload_config is not None -+ or self.vdn_config is not None -+ or os.environ.get("MINDIE_SD_FA_TYPE") != "fused_attn_score" -+ or any(block.attn.attention.attn_backend.get_name() != "FLASH_ATTN" for block in self.blocks) -+ ): -+ raise ValueError("H3 video-row cache requires dense NPU BF16 Ref2VA with strict Ulysses4, TP/DP/CFG/PP1") -+ from .video_row_cache import MiniMaxH3VideoRowCache -+ -+ self._video_row_cache_stats = None -+ return MiniMaxH3VideoRowCache( -+ target_positions, -+ seq_len=seq_len, -+ used_len=used_len, -+ num_layers=len(self.blocks), -+ pre_gate=os.environ.get("H3_VIDEO_PRE_GATE_CACHE") == "1", -+ context_only=os.environ.get("H3_VIDEO_CONTEXT_CACHE") == "1", -+ ) -+ - def create_reference_kv_state( - self, - *, -@@ -1308,6 +1348,14 @@ class MiniMaxH3DiTModel(nn.Module): - refiner_max = int(self._psp_field(refiner_psp, "refiner_packed_seq_params", "max_seqlen_q")) - video_layout = kwargs.get("video_token_layout") - reference_kv_state = kwargs.get("reference_kv_state") -+ video_row_cache = kwargs.get("video_row_cache") -+ if video_row_cache is not None and ( -+ num_requests != 1 -+ or reference_kv_state is not None -+ or self.vdn_config is not None -+ or kwargs.get("control_rows") is not None -+ ): -+ raise ValueError("H3 video-row cache cannot combine with packed requests, reference KV, VDN or control") - vdn_window = None - if self.vdn_config is not None: - # Text rows lead the packed document. -@@ -1442,6 +1490,11 @@ class MiniMaxH3DiTModel(nn.Module): - if reference_kv_state is not None: - reference_kv_state.set_local_sequence_rows(hidden.shape[0]) - -+ if video_row_cache is not None: -+ if hidden.shape[0] != local_len: -+ raise RuntimeError("H3 video-row cache requires the strict local sequence shard") -+ video_row_cache.prepare_local_span(local_start, local_len) -+ - block_video_layout = video_layout - if compact_reference: - block_video_layout = None -@@ -1505,6 +1558,7 @@ class MiniMaxH3DiTModel(nn.Module): - reference_kv_state=reference_kv_state, - reference_kv_compact=compact_reference, - vdn_window=vdn_window, -+ **({"video_row_cache": video_row_cache} if video_row_cache is not None else {}), - ) - if block_index in hints: - hidden = hidden + hints.pop(block_index) * control_scale -diff --git a/vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py b/vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py -index 476c3cc..ebce8fc 100644 ---- a/vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py -+++ b/vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py -@@ -2115,6 +2115,7 @@ class MiniMaxH3Pipeline( - text_embeddings=text_embeddings, - token_tags=tags, - device=self.device, -+ task=task, - ) - - if control_rows is not None: -diff --git a/vllm_omni/diffusion/models/minimax_h3/video_row_cache.py b/vllm_omni/diffusion/models/minimax_h3/video_row_cache.py -new file mode 100644 -index 0000000..0670cf1 ---- /dev/null -+++ b/vllm_omni/diffusion/models/minimax_h3/video_row_cache.py -@@ -0,0 +1,299 @@ -+# SPDX-License-Identifier: Apache-2.0 -+# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project -+"""Request-owned, experimental target-video block residuals for H3 Ref2VA.""" -+ -+from __future__ import annotations -+ -+import torch -+ -+ -+class MiniMaxH3VideoRowCache: -+ """Keep non-video computation dense while reusing video block updates. -+ -+ The first experimental profile is deliberately fixed. No state is shared -+ across requests, and every rank retains all attention collectives. -+ """ -+ -+ def __init__( -+ self, -+ target_positions: torch.Tensor, -+ *, -+ seq_len: int, -+ used_len: int, -+ num_layers: int, -+ pre_gate: bool = False, -+ context_only: bool = False, -+ ) -> None: -+ positions = target_positions.detach().cpu().tolist() -+ if not 0 < used_len <= seq_len or not positions or num_layers < 1: -+ raise ValueError("video-row cache requires a nonempty valid packed document") -+ if positions != sorted(set(positions)) or positions[0] < 0 or positions[-1] >= used_len: -+ raise ValueError("target-video positions must be sorted unique valid rows") -+ target_set = set(positions) -+ dense = [i for i in range(used_len) if i not in target_set] -+ if not dense: -+ raise ValueError("video-row cache requires non-video conditioning rows") -+ if context_only and (pre_gate or positions != list(range(positions[0], used_len))): -+ raise ValueError("video context cache requires one final target span and fresh FFN") -+ self.seq_len = seq_len -+ self.used_len = used_len -+ self.num_layers = num_layers -+ self._target_positions = positions -+ self.global_dense_positions = torch.tensor(dense, dtype=torch.long, device=target_positions.device) -+ self.local_dense_positions: torch.Tensor | None = None -+ self.local_video_positions: torch.Tensor | None = None -+ self._local_span: tuple[int, int] | None = None -+ self._deltas: dict[int, torch.Tensor] = {} -+ self.pre_gate = pre_gate -+ self.context_only = context_only -+ self.target_start = positions[0] -+ self._video_contexts: dict[int, tuple[torch.Tensor, torch.Tensor]] = {} -+ self._context_calls_this_step = 0 -+ self.video_context_refreshes = 0 -+ self.video_context_reuses = 0 -+ self.fresh_video_prefix_calls = 0 -+ self.full_row_mlp_blocks = 0 -+ self._attention_outputs: dict[int, torch.Tensor] = {} -+ self._mlp_outputs: dict[int, torch.Tensor] = {} -+ self._awaiting_mlp_layer: int | None = None -+ self.fresh_attention_gates = 0 -+ self.fresh_mlp_gates = 0 -+ self._step = -1 -+ self._blocks_this_step = 0 -+ self.partial = False -+ self.closed = False -+ self.full_steps = 0 -+ self.partial_steps = 0 -+ self.full_blocks = 0 -+ self.partial_blocks = 0 -+ self.subset_attention_calls = 0 -+ self.subset_query_rows = 0 -+ self.omitted_query_rows = 0 -+ self.partial_mlp_rows = 0 -+ self.peak_cache_bytes = 0 -+ -+ @staticmethod -+ def is_full_step(step: int) -> bool: -+ if not 0 <= step < 50: -+ raise ValueError("the video-row cache profile requires 50 steps") -+ return step < 4 or step >= 46 or (step - 4) % 5 == 0 -+ -+ def begin_step(self, step: int) -> None: -+ if self.closed or step != self._step + 1: -+ raise RuntimeError("video-row cache step order changed") -+ if self._step >= 0 and self._blocks_this_step != self.num_layers: -+ raise RuntimeError("previous video-row cache step is incomplete") -+ full = self.is_full_step(step) -+ if self.pre_gate or self.context_only: -+ full = step in (0, 1, 4, 9, 19, 29, 39, 49) -+ self.partial = not full -+ if self._awaiting_mlp_layer is not None: -+ raise RuntimeError("previous video pre-gate sublayer is incomplete") -+ complete = ( -+ len(self._attention_outputs) == len(self._mlp_outputs) == self.num_layers -+ if self.pre_gate -+ else len(self._video_contexts) == self.num_layers -+ if self.context_only -+ else len(self._deltas) == self.num_layers -+ ) -+ if self.partial and not complete: -+ raise RuntimeError("partial video step has no complete dense anchor") -+ self._step = step -+ self._blocks_this_step = 0 -+ self._context_calls_this_step = 0 -+ if self.partial: -+ self.partial_steps += 1 -+ else: -+ self.full_steps += 1 -+ -+ def prepare_local_span(self, start: int, length: int) -> None: -+ if self.closed or start < 0 or length < 1 or start + length > self.seq_len: -+ raise ValueError("invalid local packed span") -+ if self._local_span is not None: -+ if self._local_span != (start, length): -+ raise RuntimeError("video-row cache local ownership changed") -+ return -+ device = self.global_dense_positions.device -+ target = set(self._target_positions) -+ self.local_video_positions = torch.tensor( -+ [i - start for i in self._target_positions if start <= i < start + length], dtype=torch.long, device=device -+ ) -+ self.local_dense_positions = torch.tensor( -+ [i - start for i in range(start, min(start + length, self.used_len)) if i not in target], -+ dtype=torch.long, -+ device=device, -+ ) -+ self._local_span = (start, length) -+ -+ def _visit(self, layer: int, x: torch.Tensor) -> None: -+ if self.closed or self._step < 0 or layer != self._blocks_this_step: -+ raise RuntimeError("video-row cache block order changed") -+ if self._local_span is None or x.shape[0] != self._local_span[1] or x.dtype != torch.bfloat16: -+ raise ValueError("video-row cache requires the prepared BF16 row shard") -+ self._blocks_this_step += 1 -+ -+ def capture(self, layer: int, x: torch.Tensor, output: torch.Tensor) -> None: -+ if self.context_only: -+ raise RuntimeError("video context uses its attention/FFN completion point, not a block residual") -+ if self.pre_gate: -+ raise RuntimeError("pre-gate state cannot capture a gated block residual") -+ if self.partial: -+ raise RuntimeError("cannot replace the dense anchor during a partial step") -+ self._visit(layer, x) -+ rows = self.local_video_positions -+ assert rows is not None -+ # BF16 endpoints may have a finite difference outside FP16's range. -+ # Keep that difference in FP32; only the reconstructed block output is -+ # rounded to the original BF16 activation dtype. -+ self._deltas[layer] = output.index_select(0, rows).float() - x.index_select(0, rows).float() -+ self.peak_cache_bytes = max( -+ self.peak_cache_bytes, sum(t.numel() * t.element_size() for t in self._deltas.values()) -+ ) -+ self.full_blocks += 1 -+ -+ def restore(self, layer: int, x: torch.Tensor) -> torch.Tensor: -+ if self.pre_gate: -+ raise RuntimeError("pre-gate state cannot restore a gated block residual") -+ if not self.partial: -+ raise RuntimeError("cannot reuse a video residual on a full step") -+ self._visit(layer, x) -+ rows = self.local_video_positions -+ assert rows is not None -+ # This copy owns the returned full shard, preserving caller input and -+ # unused padding while fresh non-video rows are filled by the block. -+ output = x.clone() -+ if rows.numel(): -+ output.index_copy_(0, rows, (x.index_select(0, rows).float() + self._deltas[layer].float()).to(x.dtype)) -+ self.partial_blocks += 1 -+ return output -+ -+ def _cache_bytes(self) -> int: -+ return sum( -+ tensor.numel() * tensor.element_size() -+ for cache in (self._deltas, self._attention_outputs, self._mlp_outputs) -+ for tensor in cache.values() -+ ) + sum(t.numel() * t.element_size() for pair in self._video_contexts.values() for t in pair) -+ -+ def store_video_context(self, output: torch.Tensor, lse: torch.Tensor) -> None: -+ if not self.context_only or self.partial or self.closed or self._step < 0: -+ raise RuntimeError("cannot refresh video context on this step") -+ layer = self._blocks_this_step -+ if layer >= self.num_layers or output.dtype != torch.bfloat16 or lse.dtype != torch.float32: -+ raise ValueError("video context requires native BF16 output and FP32 LSE") -+ self._video_contexts[layer] = (output.detach(), lse.detach()) -+ self.video_context_refreshes += 1 -+ self.peak_cache_bytes = max(self.peak_cache_bytes, self._cache_bytes()) -+ -+ def finish_context_block(self, layer: int, x: torch.Tensor) -> None: -+ if not self.context_only: -+ raise RuntimeError("video context completion requires the selected mode") -+ self._visit(layer, x) -+ if self._context_calls_this_step != self._blocks_this_step or layer not in self._video_contexts: -+ raise RuntimeError("video context attention was not executed before the fresh FFN") -+ self.full_row_mlp_blocks += 1 -+ if self.partial: -+ self.partial_blocks += 1 -+ else: -+ self.full_blocks += 1 -+ -+ def read_video_context(self) -> tuple[torch.Tensor, torch.Tensor]: -+ if not self.context_only or not self.partial or self.closed: -+ raise RuntimeError("cannot reuse video context on this step") -+ self.video_context_reuses += 1 -+ return self._video_contexts[self._blocks_this_step] -+ -+ def record_context_attention(self) -> None: -+ if not self.context_only or self.closed or self._context_calls_this_step != self._blocks_this_step: -+ raise RuntimeError("video context attention order changed") -+ self._context_calls_this_step += 1 -+ self.fresh_video_prefix_calls += 1 -+ -+ def pre_gate_output(self, layer: int, output: torch.Tensor, stage: str) -> torch.Tensor: -+ """Capture or fill only target rows before applying the current gate.""" -+ if not self.pre_gate or stage not in ("attn", "mlp"): -+ raise ValueError("pre-gate output requires the selected attention or MLP stage") -+ if stage == "attn": -+ if self._awaiting_mlp_layer is not None: -+ raise RuntimeError("previous video pre-gate sublayer is incomplete") -+ self._visit(layer, output) -+ self._awaiting_mlp_layer = layer -+ if self.partial: -+ self.partial_blocks += 1 -+ else: -+ self.full_blocks += 1 -+ else: -+ if self.closed or self._awaiting_mlp_layer != layer: -+ raise RuntimeError("video pre-gate MLP order changed") -+ if output.dtype != torch.bfloat16 or output.shape[0] != self._local_span[1]: -+ raise ValueError("video pre-gate output must retain the BF16 local shard") -+ self._awaiting_mlp_layer = None -+ rows = self.local_video_positions -+ assert rows is not None -+ cache = self._attention_outputs if stage == "attn" else self._mlp_outputs -+ if self.partial: -+ restored = output.clone() -+ restored.index_copy_(0, rows, cache[layer]) -+ return restored -+ cache[layer] = output.index_select(0, rows) -+ self.peak_cache_bytes = max(self.peak_cache_bytes, self._cache_bytes()) -+ return output -+ -+ def record_fresh_gate(self, stage: str) -> None: -+ if not self.pre_gate or not self.partial or stage not in ("attn", "mlp"): -+ raise RuntimeError("unexpected fresh gate record") -+ if stage == "attn": -+ self.fresh_attention_gates += 1 -+ else: -+ self.fresh_mlp_gates += 1 -+ -+ def record_subset_attention(self, rows: int) -> None: -+ if not self.partial or rows != self.global_dense_positions.numel(): -+ raise RuntimeError("unexpected subset attention query count") -+ self.subset_attention_calls += 1 -+ self.subset_query_rows += rows -+ self.omitted_query_rows += len(self._target_positions) -+ -+ def end_step(self) -> None: -+ if self._blocks_this_step != self.num_layers or self._awaiting_mlp_layer is not None: -+ raise RuntimeError("video-row cache step did not visit every main block") -+ -+ def stats(self) -> dict[str, int | bool | str]: -+ counters: dict[str, int | bool | str] = { -+ name: getattr(self, name) -+ for name in [ -+ "full_steps", -+ "partial_steps", -+ "full_blocks", -+ "partial_blocks", -+ "subset_attention_calls", -+ "subset_query_rows", -+ "omitted_query_rows", -+ "partial_mlp_rows", -+ "peak_cache_bytes", -+ "closed", -+ ] -+ } -+ counters["storage_dtype"] = "torch.bfloat16" if self.pre_gate or self.context_only else "torch.float32" -+ counters["cache_stage"] = ( -+ "video_context_lse" if self.context_only else "pre_gate" if self.pre_gate else "block_residual" -+ ) -+ counters["video_context_refreshes"] = self.video_context_refreshes -+ counters["video_context_reuses"] = self.video_context_reuses -+ counters["fresh_video_prefix_calls"] = self.fresh_video_prefix_calls -+ counters["full_row_mlp_blocks"] = self.full_row_mlp_blocks -+ counters["fresh_attention_gates"] = self.fresh_attention_gates -+ counters["fresh_mlp_gates"] = self.fresh_mlp_gates -+ counters["live_cache_bytes"] = self._cache_bytes() -+ return counters -+ -+ def close(self) -> None: -+ self._deltas.clear() -+ self._attention_outputs.clear() -+ self._mlp_outputs.clear() -+ self._video_contexts.clear() -+ self._awaiting_mlp_layer = None -+ self._target_positions.clear() -+ self.global_dense_positions = torch.empty(0, dtype=torch.long) -+ self.local_dense_positions = None -+ self.local_video_positions = None -+ self.closed = True -diff --git a/vllm_omni/diffusion/offloader/base.py b/vllm_omni/diffusion/offloader/base.py -index 2505d48..50b73e5 100644 ---- a/vllm_omni/diffusion/offloader/base.py -+++ b/vllm_omni/diffusion/offloader/base.py -@@ -71,6 +71,10 @@ class OffloadConfig: - dlo_use_allgather: bool = True - dlo_resident_layers: int = 0 # leading DiT layers kept on device - chunk_size_bytes: int = 64 * 1024 * 1024 -+ # Hold each DiT block's weight AllGather until that block's QKV AllToAll has -+ # been enqueued, so the background collective does not take the links ahead -+ # of the critical-path all-to-all. See ``collective_gate.py``. -+ dlo_order_allgather_after_attn_a2a: bool = False - # Optional per-worker ceiling for registering an HWR mmap. Zero means no - # additional ceiling; pin_cpu_memory controls whether registration is tried. - dlo_host_registration_limit_gib: float = 0.0 -@@ -216,6 +220,7 @@ class OffloadConfig: - dlo_use_allgather=dit_uses_allgather, - dlo_resident_layers=dlo_resident_layers, - chunk_size_bytes=chunk_size_bytes, -+ dlo_order_allgather_after_attn_a2a=bool(getattr(od_config, "dlo_order_allgather_after_attn_a2a", False)), - dlo_host_registration_limit_gib=dlo_host_registration_limit_gib, - components=components, - dlo_transfers=dlo_transfers, -diff --git a/vllm_omni/diffusion/offloader/distributed_layerwise_backend.py b/vllm_omni/diffusion/offloader/distributed_layerwise_backend.py -index 7f0f081..8c4f87f 100644 ---- a/vllm_omni/diffusion/offloader/distributed_layerwise_backend.py -+++ b/vllm_omni/diffusion/offloader/distributed_layerwise_backend.py -@@ -31,6 +31,7 @@ import torch.distributed - from torch import nn - from vllm.logger import init_logger - -+from vllm_omni.diffusion.distributed.collective_gate import QKVAllToAllBarrier - from vllm_omni.diffusion.hooks import HookRegistry, ModelHook - from vllm_omni.diffusion.model_loader.host_weight_plan import ( - HostWeightPlan, -@@ -142,6 +143,7 @@ class DistributedLayerwiseOffloadHook(ModelHook): - tensor_transforms: dict[int, Any] | None = None, - materialization_probe_tensor: torch.Tensor | None = None, - chunk_size_bytes: int = 64 * 1024 * 1024, -+ order_allgather_after_attn_a2a: bool = False, - ): - assert isinstance(next_block, nn.Module), "transformer block must be type `torch.nn.Module`" - if type(dp_size) is not int or dp_size < 1: -@@ -167,6 +169,7 @@ class DistributedLayerwiseOffloadHook(ModelHook): - - self.chunk_size_bytes = chunk_size_bytes - self.manifest: PartManifest | None = None -+ self._deferred_chunked_allgather = None - self._h2d_done_events = [current_omni_platform.Event(), current_omni_platform.Event()] - self._ag_done_events = [current_omni_platform.Event(), current_omni_platform.Event()] - # Hook-local events protect chunk-input reuse within this hook. Across -@@ -239,6 +242,19 @@ class DistributedLayerwiseOffloadHook(ModelHook): - # Per-block synchronization primitive: set after H2D copy completes. - self._prefetch_done: Any | None = None - -+ # Optional ordering: hold the background weight AllGather until the -+ # block's QKV AllToAll is on the compute stream. See -+ # distributed/collective_gate.py. Off by default; the attention layer -+ # inside this block is handed the same barrier at install time. -+ self._a2a_barrier: QKVAllToAllBarrier | None = None -+ self._a2a_event: Any | None = None -+ # ``(slot, gpu_shards, gpu_weights, evt)`` of a gather armed but not yet -+ # issued, so ``post_forward`` can flush it if the attention never ran. -+ self._deferred_allgather: tuple[int, dict, dict, Any] | None = None -+ if order_allgather_after_attn_a2a: -+ self._a2a_event = current_omni_platform.Event() -+ self._a2a_barrier = QKVAllToAllBarrier(self._allgather_after_attn_a2a) -+ - # Shared shard (AllGather input) buffers — assigned by backend. - self.gpu_shard_buffers: list[dict[torch.dtype, torch.Tensor] | None] = [None, None] - -@@ -653,49 +669,23 @@ class DistributedLayerwiseOffloadHook(ModelHook): - gpu_shard = shard_bufs[dtype][: cpu_shard.numel()] - gpu_shard.copy_(cpu_shard, non_blocking=non_blocking) - gpu_shards[dtype] = gpu_shard -- self.comm_stream.wait_stream(self.copy_stream) -- with current_omni_platform.stream(self.comm_stream): -- for dtype, local_shard in gpu_shards.items(): -- gw = gpu_weights[dtype][: self._ag_output_sizes[dtype]] -- torch.distributed.all_gather_into_tensor( -- gw, -- local_shard, -- group=self.dp_group, -- ) -- evt.record(self.comm_stream) -+ -+ if self._a2a_barrier is not None and non_blocking and getattr(self, "_nz_cast_fn", None) is None: -+ # Hold the collective until the block's attention reports its -+ # QKV AllToAll, so the background gather cannot take the links -+ # ahead of the critical-path all-to-all. The H2D above has -+ # already been enqueued; only the gather moves, so it still -+ # overlaps the rest of the block. Deferral needs a repoint that -+ # emits no stream work, hence the NZ-cast guard. -+ self._deferred_allgather = (slot, gpu_shards, gpu_weights, evt) -+ self._a2a_barrier.arm() -+ else: -+ self._issue_allgather(slot, gpu_shards, gpu_weights, evt) -+ elif self._a2a_barrier is not None and non_blocking: -+ self._deferred_chunked_allgather = (slot, gpu_weights, evt) -+ self._a2a_barrier.arm() - else: -- for dtype_manifest in self.manifest.dtypes: -- cpu_shard = self.cpu_shards[dtype_manifest.dtype] -- buf = shard_bufs[dtype_manifest.dtype] -- for chunk in dtype_manifest.chunks: -- if buf.numel() >= 2 * chunk.local_numel: -- width = buf.numel() // 2 -- input_slot = chunk.chunk_id % 2 -- else: -- width = buf.numel() -- input_slot = 0 -- reuse = self._chunk_reuse_events[input_slot] -- if reuse is not None: -- self.copy_stream.wait_event(reuse) -- local_input = buf[input_slot * width : input_slot * width + chunk.local_numel] -- source = cpu_shard[chunk.cpu_offset : chunk.cpu_offset + chunk.local_numel] -- with current_omni_platform.stream(self.copy_stream): -- local_input.copy_(source, non_blocking=non_blocking) -- self._h2d_done_events[input_slot].record(self.copy_stream) -- self.comm_stream.wait_event(self._h2d_done_events[input_slot]) -- with current_omni_platform.stream(self.comm_stream): -- full_out = gpu_weights[dtype_manifest.dtype][ -- chunk.full_offset : chunk.full_offset + chunk.padded_numel -- ] -- torch.distributed.all_gather_into_tensor( -- full_out, -- local_input, -- group=self.dp_group, -- ) -- self._ag_done_events[input_slot].record(self.comm_stream) -- self._chunk_reuse_events[input_slot] = self._ag_done_events[input_slot] -- with current_omni_platform.stream(self.comm_stream): -- evt.record(self.comm_stream) -+ self._issue_chunked_allgather(slot, gpu_weights, evt, non_blocking=non_blocking) - - self.ready_events[slot] = evt - self._prefetch_done = evt -@@ -719,6 +709,131 @@ class DistributedLayerwiseOffloadHook(ModelHook): - ), - ) - -+ def _issue_chunked_allgather(self, slot, gpu_weights, evt, *, gated=False, non_blocking=True) -> None: -+ """Submit chunk copies and gathers together after the foreground gate. -+ -+ Chunk staging slots are bounded and reused. Deferring only gathers -+ would overwrite an input slot before its collective consumes it. -+ """ -+ shard_bufs = self.gpu_shard_buffers[slot] -+ assert shard_bufs is not None and self.manifest is not None -+ if gated: -+ self.comm_stream.wait_event(self._a2a_event) -+ for dtype_manifest in self.manifest.dtypes: -+ cpu_shard = self.cpu_shards[dtype_manifest.dtype] -+ buf = shard_bufs[dtype_manifest.dtype] -+ for chunk in dtype_manifest.chunks: -+ if buf.numel() >= 2 * chunk.local_numel: -+ width = buf.numel() // 2 -+ input_slot = chunk.chunk_id % 2 -+ else: -+ width = buf.numel() -+ input_slot = 0 -+ reuse = self._chunk_reuse_events[input_slot] -+ if reuse is not None: -+ self.copy_stream.wait_event(reuse) -+ local_input = buf[input_slot * width : input_slot * width + chunk.local_numel] -+ source = cpu_shard[chunk.cpu_offset : chunk.cpu_offset + chunk.local_numel] -+ with current_omni_platform.stream(self.copy_stream): -+ local_input.copy_(source, non_blocking=non_blocking) -+ self._h2d_done_events[input_slot].record(self.copy_stream) -+ self.comm_stream.wait_event(self._h2d_done_events[input_slot]) -+ with current_omni_platform.stream(self.comm_stream): -+ full_out = gpu_weights[dtype_manifest.dtype][ -+ chunk.full_offset : chunk.full_offset + chunk.padded_numel -+ ] -+ torch.distributed.all_gather_into_tensor( -+ full_out, -+ local_input, -+ group=self.dp_group, -+ ) -+ self._ag_done_events[input_slot].record(self.comm_stream) -+ self._chunk_reuse_events[input_slot] = self._ag_done_events[input_slot] -+ with current_omni_platform.stream(self.comm_stream): -+ evt.record(self.comm_stream) -+ -+ def _issue_allgather( -+ self, -+ slot: int, -+ gpu_shards: dict[torch.dtype, torch.Tensor], -+ gpu_weights: dict[torch.dtype, torch.Tensor], -+ evt: Any, -+ *, -+ gated: bool = False, -+ ) -> None: -+ """Submit this block's weight AllGather on ``comm_stream``. -+ -+ ``gated`` additionally waits on the QKV AllToAll barrier event recorded -+ by the block's attention, so the foreground collective gets the links -+ first and this gather queues behind it. -+ """ -+ self.comm_stream.wait_stream(self.copy_stream) -+ if gated: -+ self.comm_stream.wait_event(self._a2a_event) -+ with current_omni_platform.stream(self.comm_stream): -+ for dtype, local_shard in gpu_shards.items(): -+ # Slice the shared (max-sized) output buffer down to this -+ # block's actual AllGather output size. The buffers are -+ # sized to the *largest* block across all groups, so for any -+ # smaller block the full buffer would violate the -+ # all_gather_into_tensor contract -+ # (output.numel() == world_size * input.numel()). -+ # Repoint offsets are relative to the block's flattened -+ # buffer, so a prefix slice is safe. -+ gw = gpu_weights[dtype][: self._ag_output_sizes[dtype]] -+ torch.distributed.all_gather_into_tensor( -+ gw, -+ local_shard, -+ group=self.dp_group, -+ ) -+ evt.record(self.comm_stream) -+ -+ def _allgather_after_attn_a2a(self) -> None: -+ """Barrier callback: the block's QKV AllToAll is now enqueued. -+ -+ Recording on the compute stream here (rather than at block entry) is -+ what makes the gate effective: an event wait snapshots the records -+ enqueued before it, so the wait must be enqueued *after* the all-to-all. -+ """ -+ chunked = self._deferred_chunked_allgather -+ if chunked is not None: -+ self._deferred_chunked_allgather = None -+ assert self._a2a_event is not None -+ self._a2a_event.record(current_omni_platform.current_stream()) -+ self._issue_chunked_allgather(*chunked, gated=True) -+ return -+ deferred = self._deferred_allgather -+ if deferred is None: -+ return -+ self._deferred_allgather = None -+ assert self._a2a_event is not None -+ self._a2a_event.record(current_omni_platform.current_stream()) -+ slot, gpu_shards, gpu_weights, evt = deferred -+ self._issue_allgather(slot, gpu_shards, gpu_weights, evt, gated=True) -+ -+ def _flush_deferred_allgather(self) -> None: -+ """Issue a deferred gather whose attention never signalled. -+ -+ Keeps the double buffer live: a block whose attention is skipped -+ (cache-dit, an all-resident stage) still needs its weights, so the -+ gather is submitted ungated at the end of that block's forward. -+ """ -+ chunked = self._deferred_chunked_allgather -+ if chunked is not None: -+ self._deferred_chunked_allgather = None -+ if self._a2a_barrier is not None: -+ self._a2a_barrier.disarm() -+ self._issue_chunked_allgather(*chunked) -+ return -+ deferred = self._deferred_allgather -+ if deferred is None: -+ return -+ self._deferred_allgather = None -+ if self._a2a_barrier is not None: -+ self._a2a_barrier.disarm() -+ slot, gpu_shards, gpu_weights, evt = deferred -+ self._issue_allgather(slot, gpu_shards, gpu_weights, evt) -+ - def _wait_for_weights(self, slot: int) -> None: - """Wait until the slot holding this block is ready for compute. - -@@ -840,6 +955,10 @@ class DistributedLayerwiseOffloadHook(ModelHook): - return args, kwargs - - def post_forward(self, module: nn.Module, output: Any) -> Any: -+ # Pair with pre_forward: if the block's attention never signalled the -+ # gate, the deferred gather must still go out before this block's -+ # storage is released (offload_layer waits on _prefetch_done). -+ self._flush_deferred_allgather() - self.offload_layer() - return output - -@@ -864,6 +983,7 @@ def apply_distributed_block_hook( - tensor_transforms: dict[int, Any] | None = None, - materialization_probe_tensor: torch.Tensor | None = None, - chunk_size_bytes: int = 64 * 1024 * 1024, -+ order_allgather_after_attn_a2a: bool = False, - ) -> DistributedLayerwiseOffloadHook: - """Register a DistributedLayerwiseOffloadHook on *module*.""" - registry = HookRegistry.get_or_create(module) -@@ -881,11 +1001,32 @@ def apply_distributed_block_hook( - tensor_transforms=tensor_transforms, - materialization_probe_tensor=materialization_probe_tensor, - chunk_size_bytes=chunk_size_bytes, -+ order_allgather_after_attn_a2a=order_allgather_after_attn_a2a, - ) - registry.register_hook(DistributedLayerwiseOffloadHook._HOOK_NAME, hook) - return hook - - -+def attach_collective_gate(block: nn.Module, hook: DistributedLayerwiseOffloadHook) -> int: -+ """Hand ``hook``'s barrier to every Attention layer inside *block*. -+ -+ The attention signals the barrier right after its QKV AllToAll. The import -+ is local to keep this module independent of the attention package at import -+ time. -+ -+ Returns the number of attention layers wired (0 means the hook will rely on -+ the ``post_forward`` flush instead). -+ """ -+ from vllm_omni.diffusion.attention.layer import Attention -+ -+ attached = 0 -+ for module in block.modules(): -+ if isinstance(module, Attention): -+ module._collective_gate = hook._a2a_barrier -+ attached += 1 -+ return attached -+ -+ - def remove_distributed_block_hook(module: nn.Module) -> None: - """Remove the distributed layerwise offload hook from *module*.""" - registry: HookRegistry | None = getattr(module, "_hook_registry", None) -@@ -1566,29 +1707,45 @@ class DistributedLayerwiseOffloadBackend(OffloadBackend): - self._all_hook_groups.append(hooks) - self._blocks.append(block_list) - probes = {id(block): module_materialization_probe(block) for block in block_list} -+ # Only the DiT blocks issue a QKV AllToAll the gather can queue behind. -+ order_gate = bool(getattr(self.config, "dlo_order_allgather_after_attn_a2a", False)) and ( -+ component == DIT_COMPONENT -+ ) -+ if order_gate and not getattr(self, "_collective_gate_logged", False): -+ self._collective_gate_logged = True -+ logger.info( -+ "Distributed layerwise offload will hold each DiT block's weight AllGather " -+ "until that block's QKV AllToAll is on the compute stream" -+ ) - for block, next_block in zip( - chain((block_list[-1],), block_list[:-1]), - block_list, - strict=True, - ): -- hooks.append( -- apply_distributed_block_hook( -- block, -- next_block, -- self.device, -- group, -- group_size, -- group_rank, -- self.copy_stream, -- self.comm_stream, -- self.config.pin_cpu_memory, -- shared_buffers=[None, None], -- rank_local_mmap=self._using_rank_local_mmap if use_dit_mmap else False, -- tensor_transforms=self._mmap_transforms_by_tensor_id if use_dit_mmap else None, -- materialization_probe_tensor=probes[id(block)], -- chunk_size_bytes=self.config.chunk_size_bytes, -- ) -+ hook = apply_distributed_block_hook( -+ block, -+ next_block, -+ self.device, -+ group, -+ group_size, -+ group_rank, -+ self.copy_stream, -+ self.comm_stream, -+ self.config.pin_cpu_memory, -+ shared_buffers=[None, None], -+ rank_local_mmap=self._using_rank_local_mmap if use_dit_mmap else False, -+ tensor_transforms=self._mmap_transforms_by_tensor_id if use_dit_mmap else None, -+ materialization_probe_tensor=probes[id(block)], -+ chunk_size_bytes=self.config.chunk_size_bytes, -+ order_allgather_after_attn_a2a=order_gate, - ) -+ if hook._a2a_barrier is not None and attach_collective_gate(block, hook) == 0: -+ logger.warning( -+ "DLO AllGather ordering is enabled but block %s exposes no Attention layer; " -+ "its gather will fall back to the ungated path", -+ block.__class__.__name__, -+ ) -+ hooks.append(hook) - - # hooks = [last -> first, block0 -> block1, ...]. Alternating slots - # keep a prefetch from overwriting the current block for any ring size. -diff --git a/vllm_omni/engine/arg_utils.py b/vllm_omni/engine/arg_utils.py -index 0a876b9..bf8dcdc 100644 ---- a/vllm_omni/engine/arg_utils.py -+++ b/vllm_omni/engine/arg_utils.py -@@ -807,6 +807,7 @@ class OrchestratorArgs: - dlo_use_allgather: bool = True - dlo_chunk_size_mb: int = 64 - dlo_resident_layers: int = 0 -+ dlo_order_allgather_after_attn_a2a: bool = False - host_weight_runtime_mode: str = "disabled" - host_weight_runtime_root: str | None = None - dlo_host_registration_limit_gib: float = 0.0 -diff --git a/vllm_omni/entrypoints/cli/serve.py b/vllm_omni/entrypoints/cli/serve.py -index f2b55e7..aff71b5 100644 ---- a/vllm_omni/entrypoints/cli/serve.py -+++ b/vllm_omni/entrypoints/cli/serve.py -@@ -1041,6 +1041,17 @@ class OmniServeCommand(CLISubcommand): - help="Full-chunk target size in MiB for distributed layerwise offload " - "(each chunk is one host-to-device copy + AllGather unit).", - ) -+ omni_config_group.add_argument( -+ "--dlo-order-allgather-after-attn-a2a", -+ action="store_true", -+ default=False, -+ help=( -+ "Hold each DiT block's weight AllGather until that block's QKV AllToAll " -+ "has been enqueued, so the background gather does not take the collective " -+ "links ahead of the critical-path all-to-all. Requires distributed " -+ "layerwise offload with AllGather." -+ ), -+ ) - omni_config_group.add_argument( - "--host-weight-runtime-mode", - choices=("disabled", "preferred", "required"), diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c23-source-identity.json b/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c23-source-identity.json deleted file mode 100644 index 1eb4e74f5..000000000 --- a/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c23-source-identity.json +++ /dev/null @@ -1,30 +0,0 @@ -{ - "head": "6eb7bb094effb40853431208d1631effaf452a3f", - "tree": "685c971a476b9df16cb5f79b3db41e12930306ab", - "base": "d48fa51d03dd748568cbcf57ed40f533c691c9bd", - "branch": "rsi/h3-297-video-context-lse", - "files": { - "tests/diffusion/distributed/test_collective_gate.py": "6a02f32882d85eca1e3bb9a023ce3adb3d5b59c213845e11e85c96a89432f477", - "tests/diffusion/models/minimax_h3/test_minimax_h3_pre_gate_cache.py": "a4924d0b3dd58ae413259edba6cf2487b3e41d391f943fb74358934be72eb53b", - "tests/diffusion/models/minimax_h3/test_minimax_h3_video_context.py": "1e76074b71f0ea4e070c435db623add702d1414a351427fd86ebb40e8dfd67fd", - "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "e09b872f6900e821d42632a0ca09595f2affe6f9794049dbc74d69ba6a52b2b0", - "vllm_omni/config/omni_config.py": "6894120a330474f718d43bf2bf7928892eb996152839ce6469c34d18a7df09d1", - "vllm_omni/config/stage_config.py": "0752e8cc1c6f1021f737830d08663cfc118923a83fd909c607f9ec3ab2a3d2fb", - "vllm_omni/diffusion/attention/layer.py": "76c5fce3dabd23e39a8890316e1691c03bc203066cb60af5611ec4790eefc632", - "vllm_omni/diffusion/data.py": "5db83d5c71cb27cc7810d35fcfc891da80f0b003adbb16bdecc3816a63305e11", - "vllm_omni/diffusion/distributed/collective_gate.py": "01c6317cfa615840681b8bb8b8513fa3736d26e845a24ad9e1ab7f87489b962f", - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "48533c941a50719995206d18fa9198330824b940ecba9399b46ba7615b969ccb", - "vllm_omni/diffusion/models/minimax_h3/attention/video_context.py": "f4cfd2225514b6f99d36567879f06fb4a42f9d235ac0114fc643001b345cb22e", - "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "c5fab405ee82bc553e9a902fa27e6dfbc5213ec18311e17b6415396a69b50f43", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "73c96090e6fb0e13ce734bc96b23472fd5adb83bceb71f3eba2815d6343a33e4", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "577e290c2d92fc02822cd516cc499c3c4d1f465145ad2a837edfdfeaa09ed9fb", - "vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py": "e8e2e74fb8edb05e02d3e2328ea1f3ad5ddcfd7a10ed09c5cf3911b1d063e849", - "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "cff17568fbd856c28394d104d393a806931613b7b057f288006ad8c6fb9aa05a", - "vllm_omni/diffusion/offloader/base.py": "23922136deaf54e2d98e827b007b34c7e3ad36171b71bfe9060d390efc0b679f", - "vllm_omni/diffusion/offloader/distributed_layerwise_backend.py": "19fbce0f1036e4784d68cab798c499526a9099101909e41ce96dd0caa033b6a4", - "vllm_omni/engine/arg_utils.py": "52569e3688cc708729a48ca0a2c875c28193733c2f934f6ff89f6c815e4009c7", - "vllm_omni/entrypoints/cli/serve.py": "f8284ad6b751ba4a9e4f7254d78132cc2cfd49871a0f9af7fde49c6789d79cb0" - }, - "published": false, - "patch_base": "be319692b96413918973b5bd399cdda3f320c574" -} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/plan/README.md b/evidence/h3-ref2va-910c-4logical-20261101-297/plan/README.md index d9cd612b2..485a2948e 100644 --- a/evidence/h3-ref2va-910c-4logical-20261101-297/plan/README.md +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/plan/README.md @@ -1,45 +1,15 @@ -# Frozen campaign contracts and execution checkpoint +# Frozen plan and projection -The governing issue is [#297](https://github.com/InferMatrixLab/InferRSI/issues/297). -This tracking snapshot records actual execution; it does not authorize a new -budget, replace original B0 or establish campaign acceptance. +Campaign [#297](https://github.com/InferMatrixLab/InferRSI/issues/297) +owns the acceptance criteria and source revisions. Retained original files keep +their exact bytes; the complete plan is also preserved in the external archive. -- [Execution manifest](execution-checkpoint.json): source/model identities, hardware, - private-stack versions, workload, original deadline and outstanding scope. -- [Lossless contract](frozen-lossless-contract-v1.json): strict latent/schedule and - decoded RGB24/PCM checks, qualified original source, and frozen timing protocol. -- [Lossy contract](frozen-lossy-contract-v1.json): original pre-search scorer bytes, - revisions/weights, ten fixed samples, aggregation and automatic quality gates. -- [Requester decisions](requester-decisions.json): user-selected tracks, no human - output checks, target and shared ceiling; later execution does not reset them. +Configuration entry points: +- `plan/execution-checkpoint.json` +- `plan/frozen-lossless-contract-v1.json` +- `plan/frozen-lossy-contract-v1.json` -Original B0 repeatability passed at 5 s and 15 s. Migration numerical comparison -passed at 5 s; default dense migration 15 s remains unqualified. C1 subsequently -passed full strict 5 s and15 s comparison,439 checks each. These are numerical conclusions, not latency -acceptance. The model's 81 files / 144051182613 bytes were fully SHA/LFS verified -in private preparation; the raw private manifest is not bundled. - -Every actual diagnostic/native/operator run used the pinned InferRSI external -executor. The first native CLI clean 5 s AB pair completed: strict quality PASS35 -plus the separate PASS439 audit, source integrity and cleanup passed. Performance -FAIL: 1.5729% reduction versus the required50%. Physical NPU discovery IDs1,2 map to the -four execution logical devices2,3,4,5; native inventory remains8physical/16logical. -The case-local driver owns fresh servers, readiness, warmups, measured requests, -media identities and native cleanup; it returns complete HTTP latency in ms. - -Clean screening requires 2 warmups and 3 measured requests per fresh server. -Final confirmation requires 3 fresh pairs in AB/BA/AB order, the same inner -request counts, and each pair plus the aggregate reaching the 50% reduction bar. -Keep profiler, audits and CPU scoring out of clean timing. The completed -[5 s screening](../results/c1-clean-5s-screening.json) is not final confirmation; -15 s clean timing and complete four-track acceptance remain outstanding. - -The current requester-authorized ceiling is 24 directions / 32 wall hours / -128 logical-device-hours, including preparation, failures, retries, scoring and -cleanup. It started at 2026-11-01 17:19 +08 and ends at 2026-11-03 01:19 +08. -The original 12-direction / 16-hour / 64-device-hour allowance is retained as -decision history in requester-decisions.json. The original start never changed. -All 24 families are charged. No extra final-confirmation amount is authorized. - -[Replay and availability](reproduction.md) separates local byte checks from -device replay requiring the external artifacts and original private environment. +`archive-projection.json` declares which original source records supply the +compact summary, sample arrays and configuration references. The identical +projection and its digest are pinned in `results/artifacts.json`. It is a +read-only storage projection, not a new controller or retrospective freeze. diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/plan/archive-projection.json b/evidence/h3-ref2va-910c-4logical-20261101-297/plan/archive-projection.json new file mode 100644 index 000000000..2e7d7d210 --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/plan/archive-projection.json @@ -0,0 +1,67 @@ +{ + "schema_version": 1, + "campaign": "h3-ref2va-910c-4logical-20261101-297", + "configuration": [ + "plan/execution-checkpoint.json", + "plan/frozen-lossless-contract-v1.json", + "plan/frozen-lossy-contract-v1.json" + ], + "integrity_manifests": [ + "results/publication-integrity.json", + "validator/frozen-snapshots.json" + ], + "verdict": { + "path": "plan/execution-checkpoint.json", + "pointer": "/campaign_verdict" + }, + "metrics": [ + { + "name": "c1-clean-5s-baseline", + "path": "results/c1-clean-5s-screening.json", + "pointer": "/arms/baseline/samples_ms", + "unit": "ms", + "scope": "One fresh-server screening pair, 2 warmups and 3 timed requests per arm; not final" + }, + { + "name": "c1-clean-5s-candidate", + "path": "results/c1-clean-5s-screening.json", + "pointer": "/arms/candidate/samples_ms", + "unit": "ms", + "scope": "One fresh-server screening pair, 2 warmups and 3 timed requests per arm; not final" + } + ], + "recorded_results": [ + { + "path": "plan/execution-checkpoint.json", + "pointer": "/acceptance" + }, + { + "path": "results/c1-clean-5s-screening.json", + "pointer": "/native_verdict" + }, + { + "path": "results/c1-clean-5s-screening.json", + "pointer": "/gain_attribution_limit" + }, + { + "path": "results/c23-5s-00-execution-proof.json", + "pointer": "/quality_verdict" + }, + { + "path": "results/c23-5s-00-execution-proof.json", + "pointer": "/failures" + }, + { + "path": "results/c12-through-c23-execution-checkpoint.json", + "pointer": "/decisions" + } + ], + "limitations": [ + "Campaign acceptance remains INCONCLUSIVE; this is a storage-only projection.", + "The index covers all published result JSON documents and their named/nested attempt observations, not unpublished controller history.", + "Repeated identical observations of one named attempt are merged with every source pointer retained. Changed observations and anonymous records remain distinct.", + "Record/index entry counts are not independent trial counts. Text-only history and per-request traces remain in the byte-preserved original archive.", + "Only declared sample distributions are recomputed. Other retained results are explicitly reported frozen values, not independently requalified quality or statistics.", + "Original private models, media, raw receipts and runtime records were not published; archiving does not restore them or complete outstanding acceptance." + ] +} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/plan/reproduction.md b/evidence/h3-ref2va-910c-4logical-20261101-297/plan/reproduction.md deleted file mode 100644 index 8518e5f10..000000000 --- a/evidence/h3-ref2va-910c-4logical-20261101-297/plan/reproduction.md +++ /dev/null @@ -1,76 +0,0 @@ -# Replay and artifact availability - -This snapshot records device experiments performed with InferRSI -`50c19da43e0b358e6f990c00719891ad308a8ac1`. The PR's repository base may be newer; -publishing evidence does not rerun or migrate those experiments. - -## Checks available from this checkout - -With the repository's development dependencies installed: - -```sh -python tools/validate_evidence.py -python -m pytest tests/test_evidence_inventory.py tests/test_smoke.py tests/test_claim.py -``` - -The inventory checks the publication digest map and eight byte-preserved frozen -snapshots. Its external/unavailable list is expected: it does not download model -weights or raw media, replay NPU execution, certify privacy, or change a verdict. - -Read [the experiment report](../results/report.md), then -[the execution snapshot](execution-checkpoint.json), -[the frozen lossless contract](frozen-lossless-contract-v1.json) and -[the frozen lossy contract](frozen-lossy-contract-v1.json). The lossy contract -includes fixed requests and loopback image URLs. Supply the original reference -image with the recorded SHA256; a replacement image changes the workload. -Held-out requests have not been generated or scored and remain reserved for -selection under that contract. - -## Engine source reconstruction - -The engine changes are patch artifacts under [patch/](../patch/README.md), not -changes to the installed InferRSI runtime. Each identity states the tested -private engine revision and patch base. The private commit object itself may -be unavailable publicly; the diff is supplied for source inspection. - -For C1, use an isolated vLLM-Omni checkout at -`922f1e9f1cb85b9f2b66d86e1eb500e6872b2dcb`, then apply -`patch/c1-h3-bf16-prefix.patch`. Each C12/C13/C15–C23 patch is cumulative over -the `patch_base` recorded in its identity: - -- C12, C13, C15, C16, C21, C22 and C23 use C1 - `be319692b96413918973b5bd399cdda3f320c574`. Chain: `922f1e9f` → C1 → one - selected patch. -- C17, C18, C19 and C20 use C13 `e89b1d40d37f7f72767892a56fa7980ea572ba8e`. - Chain: `922f1e9f` → C1 → C13 → one selected patch. Applying one of these - directly over C1 omits C13's ten DLO/collective paths, although the patch - still applies. - -Candidates that share a base are alternative experiments, not a sequence to -stack. C10/C11 patch bases are stated separately in their identities. -Use `git apply --check` before applying a patch. Do not overwrite a shared checkout. - -The identity's raw file hashes describe the tested Windows/Linux working bytes; -Git can normalize CRLF/LF while preserving the Git tree. A reconstructed checkout -does not recreate a private signed commit or prove numerical equivalence. - -## Additional requirements for device replay - -An authorized replay needs four idle logical 910C devices with 64 GiB each, the -exact original model and reference image, the recorded dependency stack, -isolated reference/candidate/evaluator environments and an independently frozen -resource/timing plan. The private Ubuntu namespace, CANN installation, request -driver, diagnostics and bootstrap scripts are not bundled. Full device replay -is therefore not self-contained from this PR. - -The [strict validator](../validator/strict_compare.py) compares original audit -layouts and raw tensors/decoded media; the archived scoring scripts require the -dependency weights and directory layout recorded in the frozen contract. Those -scripts are preserved evidence, not a newly generalized public runner. Retained -source/weight hashes identify missing inputs but do not make them available. - -Never substitute diagnostic request latency for a clean pair. Original B0 remains -the comparator; the migration branch and candidate use separate source identities. -Final confirmation still requires all four tracks and the prospective three-pair -protocol. Known quality failures remain FAIL and should not be rerun to select -favorable samples. No device replay is launched by this tracking publication. diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/results/README.md b/evidence/h3-ref2va-910c-4logical-20261101-297/results/README.md deleted file mode 100644 index 77b62dc27..000000000 --- a/evidence/h3-ref2va-910c-4logical-20261101-297/results/README.md +++ /dev/null @@ -1,104 +0,0 @@ -# Results status - -Campaign verdict: **INCONCLUSIVE**. No accepted complete-HTTP gain exists. - -- [C1 strict 5 s comparison](c1-5s-strict-comparison.json): 439 checks pass, - zero failures/missing, two complete 50-step requests compared with the original - B0; selected video/audio latent bits, schedule and every decoded RGB24/PCM sample. - This qualifies this fixed 5 s request, not performance. -- [C1 strict 15 s comparison](c1-15s-strict-comparison.json): 439 checks pass, - zero failures/missing, two complete 50-step requests compared directly with - original B0. [Execution projection](c1-15s-execution.json) records actual - prefix-attention calls: 2,500 per rank per request. Its HTTP values are audited - diagnostics, not clean performance qualification. -- [Automatic quality checkpoint](automatic-quality-checkpoint.json): observed - cache, Laser and RainFusion failures are retained with missing sample coverage. - In particular, the Taylor2 .08/max3 profile failed both 15s-00 and 5s-01 despite - its first 5s-00 observed gates passing. No human output checks were used. -- [Query-folding operator study](c1-queryfold-operator.json): 2/4/8 query tiles - remain bit-exact on one real 5 s fixture but are slower by approximately - 2.0%/2.4%/5.2%, including repeated K/V and output-layout restoration. Do not integrate. - -- Original C10 Fn1/Bn1 failed 5s-00 CLAP and audio adherence. Its Bn0 - variant passed the first clip's observed checks but failed independent 5s-01 - CLAP. Both variants are rejected for 5 s; 15 s remains independently untested. -- Taylor3 Fn1/Bn1 failed 5s-01 CLAP and a CLAP segment. Diagnostic HTTP - 176.088 s does not qualify this rejected configuration. -- [C11 native execution](c11-native-execution.json): 20 tests passed, including - two executed Ascend tests covering two-clock forecasting and the actual - Cache-DiT wrapper's compute/forecast path. Source and cleanup checks passed. - The private five-file patch is bound to 61b8efa. Both Fn1/Bn0 and Fn1/Bn1 - failed different frozen 5s-00 audio guards; neither is promoted. -- [FP32 residual native revision](c11-fp32-native-execution.json): 23 tests - passed, including three executed NPU tests. Two test-reference failures are - retained. Production bytes stayed unchanged across those native attempts. - The new private patch is bound to d4d25b4. It failed5s-00 adherence - and five independent15s-00 visual/audio guards; neither duration is promoted. - [5s audit](c11-fp32-5s-00-audit.json) and - [15s audit](c11-fp32-15s-00-audit.json) retain actual counts and violations. - -Original private executor receipts, native inventory and raw captures remain -outside the repository. These files are selected field projections, not complete -portable trial archives. - -- [First clean 5 s AB screening](c1-clean-5s-screening.json): original B0 - median291.247763 s, migration+C1 median286.666760 s;1.572888689% reduction. - Native performance FAIL at the unchanged50% bar; strict pair quality PASS35 - with separate audit PASS439; source integrity and cleanup passed. One fresh - pair, two warmups plus three measured requests per arm; no significance or - final acceptance claim. The delta includes migration+C1, without an isolated - migration-only timing arm. - -15 s clean timing, final three-pair confirmation, full lossy quality coverage -and all four final track decisions remain outstanding. - -## Earlier C12–C17 execution checkpoint - -[Earlier projection](c12-through-c17-execution-checkpoint.json) retains failures -and actual counters. C13 direct-B0 strict5s comparison PASS439; clean performance -and15s remain unqualified. C14 block-FP8/INT8 probes failed before FIA2; public -MindIE dev already has quant_attention, while the installed3.1.0 wheel lacks it. -C15 FP16 residual failed finite velocity. [C16 actual model proof](c16-5s-00-execution-proof.json) -shows finite full50steps and134input checks but three audio quality failures. -C17 passed15native tests and a real5s operator gate:69.603→25.266ms,63.7004% -reduction, intentionally nonbit output. Its full model failed11frozen quality checks. -No newly accepted complete-HTTP gain exists. All four final tracks remain open. - -## Earlier checkpoint through C20 - -[Earlier C12–C20 projection](c12-through-c20-execution-checkpoint.json) and -[C17 model proof](c17-5s-00-execution-proof.json) retain the completed failures. -C18/C19/C20 each passed17native tests, then failed the prospective operator -screen:7.25%/52.85%/48.42%operation reduction,relL2 .218013/.258805/.263361. -The unchanged primitive bounds require>=50%and relL2<.258052; no full models -were run for these three. All source branches and bundles are preserved. -At that checkpoint selected devices/ports were idle and 21/24 directions were -charged. Runtime diagnostic remains v17. No accepted complete-HTTP gain exists. - -## Current checkpoint through C23 - -[Current C12–C23 projection](c12-through-c23-execution-checkpoint.json) retains -the source-bound native, operator and full-model attempts. C21 passed 36 native -tests but failed seven frozen quality checks on its complete 5s-00 request. -C22 passed 45 native tests but failed nine frozen quality checks. Neither fixed -profile proceeds to 15 s or clean timing. The audited controller-alias repair -preserves the raw budget snapshot, failed attempts and all time charges. - -C23 V1 passed 44 native tests but reduced its static real-input 50-call attention -cycle by only 38.26%, below the unchanged 50% operator gate. Component diagnosis -found a bit-equal Torch merge faster than the manual Triton implementation. -C23 V2 passed 44 native tests, three executed four-rank suites, exact real-input -merge replay and the unchanged operator gate: 3477.473 to 1166.387 ms per cycle, -66.46% reduction, finite outputs and maximum relative L2 0.00201662 < 0.005. -This excludes FFN, changing-input quality, full-model timing and HTTP. -[Full 5s-00 evidence](c23-5s-00-execution-proof.json) records completed 50 steps, -134 input checks, expected cache/FFN/DLO counters and cleanup. Diagnostic HTTP -was 199.065 s; 39 frozen quality checks produced eight failures in visual/audio -similarity, reference/audio adherence, audio level and envelope. The fixed -profile is rejected; no 15 s or clean timing extension follows. -[Retirement proof](c23-retired-proof.json) records idle devices, free ports, -unchanged tested source, runtime v17 and frozen validators at 15:48:59 +08. - -All 24 direction families are charged. The original 32 h / 128 logical-device- -hours and 2026-11-03 01:19 +08 deadline remain unchanged. All four final tracks -remain open; heldout requests remain sealed and no public publication occurred. diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/results/artifacts.json b/evidence/h3-ref2va-910c-4logical-20261101-297/results/artifacts.json new file mode 100644 index 000000000..4ae1c186a --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/results/artifacts.json @@ -0,0 +1,84 @@ +{ + "schema_version": 2, + "archive": { + "id": "h3-ref2va-910c-4logical-20261101-297-6c7e237-v1", + "file": "h3-ref2va-910c-4logical-20261101-297-6c7e237-v1.tar.gz", + "url": "https://github.com/InferMatrixLab/InferRSI/releases/download/evidence-model-cleanup-v1/h3-ref2va-910c-4logical-20261101-297-6c7e237-v1.tar.gz", + "sha256": "2d4a2ef1eaabb8f3efe1b4fe8337fc2792c735c7e235e354ca6e962553684377", + "bytes": 244406, + "format": "tar+gzip", + "retention_owner": "amy-why-3459", + "source_revision": "6c7e237935a22cc5e90e954d4f397376e753e7d8", + "root": "h3-ref2va-910c-4logical-20261101-297", + "files": 96, + "uncompressed_bytes": 1259483, + "key_files": { + "historical_narrative": "h3-ref2va-910c-4logical-20261101-297/README.md", + "snapshot": "snapshot.json", + "projection": "replay/projection.json", + "control": "h3-ref2va-910c-4logical-20261101-297/results/c12-through-c23-execution-checkpoint.json", + "validator": "h3-ref2va-910c-4logical-20261101-297/validator/strict_compare.py" + } + }, + "verification": { + "summary": "results/summary.json", + "trials": "results/trials.jsonl", + "command": [ + "python", + "tools/rebuild_model_evidence.py", + "evidence/h3-ref2va-910c-4logical-20261101-297/results/artifacts.json", + "--archive", + "", + "--output", + "" + ], + "inputs": [ + { + "role": "control", + "path": "h3-ref2va-910c-4logical-20261101-297/results/c12-through-c23-execution-checkpoint.json", + "sha256": "f87cc234efc10adb703d01124c76d2076b46d5251d142ad178b3dc6590079f6b" + }, + { + "role": "configuration", + "path": "h3-ref2va-910c-4logical-20261101-297/plan/execution-checkpoint.json", + "sha256": "8decf6192d1e99d6665efff3af375909ac8f223ac6dc81a0d4077ca43c920416" + }, + { + "role": "validator", + "path": "h3-ref2va-910c-4logical-20261101-297/validator/strict_compare.py", + "sha256": "b83f2bff553ff6ca2970a16e25525cac5d9523612e9c05ea563852c814019028" + }, + { + "role": "configuration", + "path": "replay/projection.json", + "sha256": "a461ab6386cfea8db2e197a23c1237b38087e33111758fcc57e6363354e9749a" + }, + { + "role": "control", + "path": "snapshot.json", + "sha256": "2695c4b1f345999214c5098a444b62756f47b57c57c8fc51d95de12de9cde673" + }, + { + "role": "validator", + "path": "replay/rebuild_model_evidence.py", + "sha256": "bf9472493aff02d52ed341eba46e71f3f620cad48fd6eddc86cd9fe34036f607" + }, + { + "role": "validator", + "path": "replay/evidence_archives.py", + "sha256": "0c141cf6d925627ffd3ddafc18f927ae19449bbf8ff4c26175d975801311e936" + } + ], + "absolute_tolerance": 1e-12, + "relative_tolerance": 1e-12 + }, + "availability_limits": [ + "Campaign acceptance remains INCONCLUSIVE; this is a storage-only projection.", + "The index covers all published result JSON documents and their named/nested attempt observations, not unpublished controller history.", + "Repeated identical observations of one named attempt are merged with every source pointer retained. Changed observations and anonymous records remain distinct.", + "Record/index entry counts are not independent trial counts. Text-only history and per-request traces remain in the byte-preserved original archive.", + "Only declared sample distributions are recomputed. Other retained results are explicitly reported frozen values, not independently requalified quality or statistics.", + "Original private models, media, raw receipts and runtime records were not published; archiving does not restore them or complete outstanding acceptance." + ], + "retention_policy": "Keep this exact asset retrievable while referenced; changed bytes require a new version." +} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/results/automatic-quality-checkpoint.json b/evidence/h3-ref2va-910c-4logical-20261101-297/results/automatic-quality-checkpoint.json deleted file mode 100644 index 1e5707182..000000000 --- a/evidence/h3-ref2va-910c-4logical-20261101-297/results/automatic-quality-checkpoint.json +++ /dev/null @@ -1,1847 +0,0 @@ -{ - "campaign_verdict": "INCONCLUSIVE", - "comparisons": [ - { - "configuration_key": "audio-cache-guard", - "sample_id": "5s-00", - "comparison_sha256": "7e1ae17e18f57e1fa59c9e4eab199ea73ccdfcbc450be8d765f29b954dd98f6a", - "verdict": "FAIL", - "checks": 76, - "failures": [ - "5s-00:clap: similarity", - "5s-00:audio_caption_clap_cosine: adherence" - ], - "missing": [ - "5s-02", - "5s-03", - "5s-04", - "15s-00", - "15s-01", - "15s-02", - "15s-03", - "15s-04" - ], - "observed": { - "5s-00": { - "seconds": "5", - "baseline": { - "subject_consistency": 0.9653875539457895, - "background_consistency": 0.9668180954650165 - }, - "candidate": { - "subject_consistency": 0.9661745814288535, - "background_consistency": 0.9675477634115917 - }, - "cosines": { - "dino": { - "mean": 0.9991652965545654, - "min": 0.9982653260231018, - "p05": 0.9986943453550339 - }, - "clip": { - "mean": 0.9960272908210754, - "min": 0.9868044853210449, - "p05": 0.9926694452762603 - }, - "clap": { - "mean": 0.985465407371521, - "min": 0.985465407371521, - "p05": 0.985465407371521 - } - }, - "reduced_relative_error": 0.0007849554886155881, - "audio_rms_db": -0.30719158571507965, - "audio_envelope_relative_l2": 0.12341521943906826, - "motion_ratio": 0.9977740956441081 - }, - "5s-01": { - "seconds": "5", - "baseline": { - "subject_consistency": 0.9213417344461612, - "background_consistency": 0.9521126235888256 - }, - "candidate": { - "subject_consistency": 0.9199168895318256, - "background_consistency": 0.962323812692146 - }, - "cosines": { - "dino": { - "mean": 0.9927577376365662, - "min": 0.978660523891449, - "p05": 0.9818145453929901 - }, - "clip": { - "mean": 0.9899253249168396, - "min": 0.9589802026748657, - "p05": 0.9826266527175903 - }, - "clap": { - "mean": 0.9935711026191711, - "min": 0.9935711026191711, - "p05": 0.9935711026191711 - } - }, - "reduced_relative_error": 0.004689916330921762, - "audio_rms_db": 0.08751117168316971, - "audio_envelope_relative_l2": 0.01751661659815182, - "motion_ratio": 1.001235231687485 - } - }, - "scope": "Fixed Ref2VA sunset-image corpus: relative BF16 VBench subject/background consistency plus reference, content, audio-feature, motion and media-timestamp guards. No human check. These proxies do not certify absolute audiovisual semantic synchrony or general model quality.", - "performance_acceptance": "separate" - }, - { - "configuration_key": "audio-cache-guard", - "sample_id": "5s-01", - "comparison_sha256": "e9c8225f5437078f47ecbcbdd62bd6c5dc1c831dafb25cffa0b92fdda4e7bcfa", - "verdict": "INCONCLUSIVE", - "checks": 39, - "failures": [], - "missing": [ - "5s-00", - "5s-02", - "5s-03", - "5s-04", - "15s-00", - "15s-01", - "15s-02", - "15s-03", - "15s-04" - ], - "observed": { - "5s-01": { - "seconds": "5", - "baseline": { - "subject_consistency": 0.9213417344461612, - "background_consistency": 0.9521126235888256 - }, - "candidate": { - "subject_consistency": 0.9199168895318256, - "background_consistency": 0.962323812692146 - }, - "cosines": { - "dino": { - "mean": 0.9927577376365662, - "min": 0.978660523891449, - "p05": 0.9818145453929901 - }, - "clip": { - "mean": 0.9899253249168396, - "min": 0.9589802026748657, - "p05": 0.9826266527175903 - }, - "clap": { - "mean": 0.9935711026191711, - "min": 0.9935711026191711, - "p05": 0.9935711026191711 - } - }, - "reduced_relative_error": 0.004689916330921762, - "audio_rms_db": 0.08751117168316971, - "audio_envelope_relative_l2": 0.01751661659815182, - "motion_ratio": 1.001235231687485 - } - }, - "scope": "Fixed Ref2VA sunset-image corpus: relative BF16 VBench subject/background consistency plus reference, content, audio-feature, motion and media-timestamp guards. No human check. These proxies do not certify absolute audiovisual semantic synchrony or general model quality.", - "performance_acceptance": "separate" - }, - { - "configuration_key": "audio-guard-bn0", - "sample_id": "5s-00", - "comparison_sha256": "1c06eb16613344654fd0d2d3e5f53faaebc8ae521e5a2fc6fde710db02f1a10e", - "verdict": "INCONCLUSIVE", - "checks": 39, - "failures": [], - "missing": [ - "5s-01", - "5s-02", - "5s-03", - "5s-04", - "15s-00", - "15s-01", - "15s-02", - "15s-03", - "15s-04" - ], - "observed": { - "5s-00": { - "seconds": "5", - "baseline": { - "subject_consistency": 0.9653875539457895, - "background_consistency": 0.9668180954650165 - }, - "candidate": { - "subject_consistency": 0.9660068995099726, - "background_consistency": 0.9688905959691458 - }, - "cosines": { - "dino": { - "mean": 0.9992567300796509, - "min": 0.9986443519592285, - "p05": 0.9990008890628814 - }, - "clip": { - "mean": 0.9961938261985779, - "min": 0.9860672950744629, - "p05": 0.9928072810173034 - }, - "clap": { - "mean": 0.9955079555511475, - "min": 0.9955079555511475, - "p05": 0.9955079555511475 - } - }, - "reduced_relative_error": 0.0013931467745854867, - "audio_rms_db": -0.0815295231710133, - "audio_envelope_relative_l2": 0.09144958691574608, - "motion_ratio": 0.9973141825132419 - } - }, - "scope": "Fixed Ref2VA sunset-image corpus: relative BF16 VBench subject/background consistency plus reference, content, audio-feature, motion and media-timestamp guards. No human check. These proxies do not certify absolute audiovisual semantic synchrony or general model quality.", - "performance_acceptance": "separate" - }, - { - "configuration_key": "audio-guard-bn0", - "sample_id": "5s-01", - "comparison_sha256": "95c023ee20e9e7a32ac2d763b6dda759fa42bf0f21b819fab5a4d855a75ae694", - "verdict": "FAIL", - "checks": 76, - "failures": [ - "5s-01:clap: similarity" - ], - "missing": [ - "5s-02", - "5s-03", - "5s-04", - "15s-00", - "15s-01", - "15s-02", - "15s-03", - "15s-04" - ], - "observed": { - "5s-00": { - "seconds": "5", - "baseline": { - "subject_consistency": 0.9653875539457895, - "background_consistency": 0.9668180954650165 - }, - "candidate": { - "subject_consistency": 0.9660068995099726, - "background_consistency": 0.9688905959691458 - }, - "cosines": { - "dino": { - "mean": 0.9992567300796509, - "min": 0.9986443519592285, - "p05": 0.9990008890628814 - }, - "clip": { - "mean": 0.9961938261985779, - "min": 0.9860672950744629, - "p05": 0.9928072810173034 - }, - "clap": { - "mean": 0.9955079555511475, - "min": 0.9955079555511475, - "p05": 0.9955079555511475 - } - }, - "reduced_relative_error": 0.0013931467745854867, - "audio_rms_db": -0.0815295231710133, - "audio_envelope_relative_l2": 0.09144958691574608, - "motion_ratio": 0.9973141825132419 - }, - "5s-01": { - "seconds": "5", - "baseline": { - "subject_consistency": 0.9213417344461612, - "background_consistency": 0.9521126235888256 - }, - "candidate": { - "subject_consistency": 0.9195573150627012, - "background_consistency": 0.9624702191449762 - }, - "cosines": { - "dino": { - "mean": 0.9923730492591858, - "min": 0.9756415486335754, - "p05": 0.9809045672416687 - }, - "clip": { - "mean": 0.9897363781929016, - "min": 0.9703266620635986, - "p05": 0.979900699853897 - }, - "clap": { - "mean": 0.983326256275177, - "min": 0.983326256275177, - "p05": 0.983326256275177 - } - }, - "reduced_relative_error": 0.004576132925748447, - "audio_rms_db": 0.12675221378818413, - "audio_envelope_relative_l2": 0.028032476073448085, - "motion_ratio": 0.9837732598446114 - } - }, - "scope": "Fixed Ref2VA sunset-image corpus: relative BF16 VBench subject/background consistency plus reference, content, audio-feature, motion and media-timestamp guards. No human check. These proxies do not certify absolute audiovisual semantic synchrony or general model quality.", - "performance_acceptance": "separate" - }, - { - "configuration_key": "cache-balanced-008-max2", - "sample_id": "5s-00", - "comparison_sha256": "e760ac0f421d8188407b5e057720a938d984366e299dc324bbc3bb2c71259b14", - "verdict": "FAIL", - "checks": 39, - "failures": [ - "5s-00:clap: similarity", - "5s-00: clap segment", - "5s-00: audio temporal envelope" - ], - "missing": [ - "5s-01", - "5s-02", - "5s-03", - "5s-04", - "15s-00", - "15s-01", - "15s-02", - "15s-03", - "15s-04" - ], - "observed": { - "5s-00": { - "seconds": "5", - "baseline": { - "subject_consistency": 0.9653875539457895, - "background_consistency": 0.9668180954650165 - }, - "candidate": { - "subject_consistency": 0.9651354716076115, - "background_consistency": 0.966441892996067 - }, - "cosines": { - "dino": { - "mean": 0.9980329871177673, - "min": 0.9964174628257751, - "p05": 0.997240173816681 - }, - "clip": { - "mean": 0.9923331141471863, - "min": 0.9756749868392944, - "p05": 0.9842084020376205 - }, - "clap": { - "mean": 0.9760740995407104, - "min": 0.9760740995407104, - "p05": 0.9760740995407104 - } - }, - "reduced_relative_error": 0.0003251645637818601, - "audio_rms_db": -0.3659990076999208, - "audio_envelope_relative_l2": 0.26497643494616335, - "motion_ratio": 0.9831136213720726 - } - }, - "scope": "Fixed Ref2VA sunset-image corpus: relative BF16 VBench subject/background consistency plus reference, content, audio-feature, motion and media-timestamp guards. No human check. These proxies do not certify absolute audiovisual semantic synchrony or general model quality.", - "performance_acceptance": "separate" - }, - { - "configuration_key": "cache-taylor2-004-max1-bn1", - "sample_id": "5s-01", - "comparison_sha256": "423b5e479f3519755109661cd48298f2e6d71ce803fbfcc4b1a5360696a7fc2a", - "verdict": "FAIL", - "checks": 39, - "failures": [ - "5s-01:clap: similarity", - "5s-01:audio_caption_clap_cosine: adherence" - ], - "missing": [ - "5s-00", - "5s-02", - "5s-03", - "5s-04", - "15s-00", - "15s-01", - "15s-02", - "15s-03", - "15s-04" - ], - "observed": { - "5s-01": { - "seconds": "5", - "baseline": { - "subject_consistency": 0.9213417344461612, - "background_consistency": 0.9521126235888256 - }, - "candidate": { - "subject_consistency": 0.9194389666483654, - "background_consistency": 0.9604502952680355 - }, - "cosines": { - "dino": { - "mean": 0.9951377511024475, - "min": 0.9648282527923584, - "p05": 0.9873370617628098 - }, - "clip": { - "mean": 0.9926162362098694, - "min": 0.9726531505584717, - "p05": 0.9865877836942673 - }, - "clap": { - "mean": 0.9898052215576172, - "min": 0.9898052215576172, - "p05": 0.9898052215576172 - } - }, - "reduced_relative_error": 0.0034347801716203, - "audio_rms_db": -0.04559134981289249, - "audio_envelope_relative_l2": 0.01794616595761191, - "motion_ratio": 0.992853542121268 - } - }, - "scope": "Fixed Ref2VA sunset-image corpus: relative BF16 VBench subject/background consistency plus reference, content, audio-feature, motion and media-timestamp guards. No human check. These proxies do not certify absolute audiovisual semantic synchrony or general model quality.", - "performance_acceptance": "separate" - }, - { - "configuration_key": "cache-taylor2-008-max3", - "sample_id": "15s-00", - "comparison_sha256": "101ba21fcd0245a72b9003dadcae4a6f59525c044f1f117f51d73e5b826073f0", - "verdict": "FAIL", - "checks": 76, - "failures": [ - "15s-00:dino: similarity", - "15s-00:clip: similarity", - "15s-00:clap: similarity", - "15s-00: dino lower tail", - "15s-00: clap segment" - ], - "missing": [ - "5s-01", - "5s-02", - "5s-03", - "5s-04", - "15s-01", - "15s-02", - "15s-03", - "15s-04" - ], - "observed": { - "5s-00": { - "seconds": "5", - "baseline": { - "subject_consistency": 0.9653875539457895, - "background_consistency": 0.9668180954650165 - }, - "candidate": { - "subject_consistency": 0.9654010900152408, - "background_consistency": 0.9671699547670721 - }, - "cosines": { - "dino": { - "mean": 0.9983525276184082, - "min": 0.9966764450073242, - "p05": 0.9975502401590347 - }, - "clip": { - "mean": 0.9948052167892456, - "min": 0.9867727160453796, - "p05": 0.9891456693410874 - }, - "clap": { - "mean": 0.9920889735221863, - "min": 0.9920889735221863, - "p05": 0.9920889735221863 - } - }, - "reduced_relative_error": 0.00018910790971873886, - "audio_rms_db": -0.2371983786171961, - "audio_envelope_relative_l2": 0.18912491681824536, - "motion_ratio": 0.9983713680460938 - }, - "15s-00": { - "seconds": "15", - "baseline": { - "subject_consistency": 0.8167167416157155, - "background_consistency": 0.8849122500320551 - }, - "candidate": { - "subject_consistency": 0.814778186318947, - "background_consistency": 0.9055140411424505 - }, - "cosines": { - "dino": { - "mean": 0.9708892107009888, - "min": 0.8723620176315308, - "p05": 0.9347653150558471 - }, - "clip": { - "mean": 0.9714338779449463, - "min": 0.8665174245834351, - "p05": 0.9213008940219879 - }, - "clap": { - "mean": 0.979804515838623, - "min": 0.9607230424880981, - "p05": 0.963468861579895 - } - }, - "reduced_relative_error": 0.010967864267259823, - "audio_rms_db": 0.0547138050738972, - "audio_envelope_relative_l2": 0.07447108624639824, - "motion_ratio": 0.983092625811758 - } - }, - "scope": "Fixed Ref2VA sunset-image corpus: relative BF16 VBench subject/background consistency plus reference, content, audio-feature, motion and media-timestamp guards. No human check. These proxies do not certify absolute audiovisual semantic synchrony or general model quality.", - "performance_acceptance": "separate" - }, - { - "configuration_key": "cache-taylor2-008-max3", - "sample_id": "5s-00", - "comparison_sha256": "8a436513633dcc897d1f28607054928d04011cf0263abbbd41948ff29af9b232", - "verdict": "INCONCLUSIVE", - "checks": 39, - "failures": [], - "missing": [ - "5s-01", - "5s-02", - "5s-03", - "5s-04", - "15s-00", - "15s-01", - "15s-02", - "15s-03", - "15s-04" - ], - "observed": { - "5s-00": { - "seconds": "5", - "baseline": { - "subject_consistency": 0.9653875539457895, - "background_consistency": 0.9668180954650165 - }, - "candidate": { - "subject_consistency": 0.9654010900152408, - "background_consistency": 0.9671699547670721 - }, - "cosines": { - "dino": { - "mean": 0.9983525276184082, - "min": 0.9966764450073242, - "p05": 0.9975502401590347 - }, - "clip": { - "mean": 0.9948052167892456, - "min": 0.9867727160453796, - "p05": 0.9891456693410874 - }, - "clap": { - "mean": 0.9920889735221863, - "min": 0.9920889735221863, - "p05": 0.9920889735221863 - } - }, - "reduced_relative_error": 0.00018910790971873886, - "audio_rms_db": -0.2371983786171961, - "audio_envelope_relative_l2": 0.18912491681824536, - "motion_ratio": 0.9983713680460938 - } - }, - "scope": "Fixed Ref2VA sunset-image corpus: relative BF16 VBench subject/background consistency plus reference, content, audio-feature, motion and media-timestamp guards. No human check. These proxies do not certify absolute audiovisual semantic synchrony or general model quality.", - "performance_acceptance": "separate" - }, - { - "configuration_key": "cache-taylor2-008-max3", - "sample_id": "5s-01", - "comparison_sha256": "4fcb44d3efcf482968a4ba83da7de10dd96a722da392373e11f6d03415043069", - "verdict": "FAIL", - "checks": 113, - "failures": [ - "5s-01:clap: similarity", - "5s-01: clap segment", - "5s-01:audio_caption_clap_cosine: adherence", - "15s-00:dino: similarity", - "15s-00:clip: similarity", - "15s-00:clap: similarity", - "15s-00: dino lower tail", - "15s-00: clap segment" - ], - "missing": [ - "5s-02", - "5s-03", - "5s-04", - "15s-01", - "15s-02", - "15s-03", - "15s-04" - ], - "observed": { - "5s-00": { - "seconds": "5", - "baseline": { - "subject_consistency": 0.9653875539457895, - "background_consistency": 0.9668180954650165 - }, - "candidate": { - "subject_consistency": 0.9654010900152408, - "background_consistency": 0.9671699547670721 - }, - "cosines": { - "dino": { - "mean": 0.9983525276184082, - "min": 0.9966764450073242, - "p05": 0.9975502401590347 - }, - "clip": { - "mean": 0.9948052167892456, - "min": 0.9867727160453796, - "p05": 0.9891456693410874 - }, - "clap": { - "mean": 0.9920889735221863, - "min": 0.9920889735221863, - "p05": 0.9920889735221863 - } - }, - "reduced_relative_error": 0.00018910790971873886, - "audio_rms_db": -0.2371983786171961, - "audio_envelope_relative_l2": 0.18912491681824536, - "motion_ratio": 0.9983713680460938 - }, - "5s-01": { - "seconds": "5", - "baseline": { - "subject_consistency": 0.9213417344461612, - "background_consistency": 0.9521126235888256 - }, - "candidate": { - "subject_consistency": 0.9207581187166819, - "background_consistency": 0.9634070793787638 - }, - "cosines": { - "dino": { - "mean": 0.9845563769340515, - "min": 0.9531147480010986, - "p05": 0.9630601137876511 - }, - "clip": { - "mean": 0.9853087663650513, - "min": 0.9452791810035706, - "p05": 0.9642424941062927 - }, - "clap": { - "mean": 0.9720799326896667, - "min": 0.9720799326896667, - "p05": 0.9720799326896667 - } - }, - "reduced_relative_error": 0.005717160930300574, - "audio_rms_db": 0.4509269168547554, - "audio_envelope_relative_l2": 0.09309591765214825, - "motion_ratio": 1.018571144526001 - }, - "15s-00": { - "seconds": "15", - "baseline": { - "subject_consistency": 0.8167167416157155, - "background_consistency": 0.8849122500320551 - }, - "candidate": { - "subject_consistency": 0.814778186318947, - "background_consistency": 0.9055140411424505 - }, - "cosines": { - "dino": { - "mean": 0.9708892107009888, - "min": 0.8723620176315308, - "p05": 0.9347653150558471 - }, - "clip": { - "mean": 0.9714338779449463, - "min": 0.8665174245834351, - "p05": 0.9213008940219879 - }, - "clap": { - "mean": 0.979804515838623, - "min": 0.9607230424880981, - "p05": 0.963468861579895 - } - }, - "reduced_relative_error": 0.010967864267259823, - "audio_rms_db": 0.0547138050738972, - "audio_envelope_relative_l2": 0.07447108624639824, - "motion_ratio": 0.983092625811758 - } - }, - "scope": "Fixed Ref2VA sunset-image corpus: relative BF16 VBench subject/background consistency plus reference, content, audio-feature, motion and media-timestamp guards. No human check. These proxies do not certify absolute audiovisual semantic synchrony or general model quality.", - "performance_acceptance": "separate" - }, - { - "configuration_key": "cache-taylor2-008-max3-bn1", - "sample_id": "5s-01", - "comparison_sha256": "0ced9712635d0ffbd609b436a77f1652aae2e6d981c098b8d0477dc0dc9f3f24", - "verdict": "FAIL", - "checks": 39, - "failures": [ - "5s-01:clap: similarity", - "5s-01: clap segment" - ], - "missing": [ - "5s-00", - "5s-02", - "5s-03", - "5s-04", - "15s-00", - "15s-01", - "15s-02", - "15s-03", - "15s-04" - ], - "observed": { - "5s-01": { - "seconds": "5", - "baseline": { - "subject_consistency": 0.9213417344461612, - "background_consistency": 0.9521126235888256 - }, - "candidate": { - "subject_consistency": 0.921592558787121, - "background_consistency": 0.9582489375176468 - }, - "cosines": { - "dino": { - "mean": 0.9852516651153564, - "min": 0.9538165330886841, - "p05": 0.9698864668607712 - }, - "clip": { - "mean": 0.9861429929733276, - "min": 0.9640627503395081, - "p05": 0.9699478477239609 - }, - "clap": { - "mean": 0.973999559879303, - "min": 0.973999559879303, - "p05": 0.973999559879303 - } - }, - "reduced_relative_error": 0.003409284161307452, - "audio_rms_db": 0.20808666707388404, - "audio_envelope_relative_l2": 0.03845678935899298, - "motion_ratio": 1.0336753638251845 - } - }, - "scope": "Fixed Ref2VA sunset-image corpus: relative BF16 VBench subject/background consistency plus reference, content, audio-feature, motion and media-timestamp guards. No human check. These proxies do not certify absolute audiovisual semantic synchrony or general model quality.", - "performance_acceptance": "separate" - }, - { - "configuration_key": "cache-taylor2-008-max3-bn4", - "sample_id": "5s-01", - "comparison_sha256": "d2bf98312b742f1dada1e0acdb453bdc1759567902a878737b9464ad4dfc08f6", - "verdict": "FAIL", - "checks": 39, - "failures": [ - "5s-01:dino: similarity", - "5s-01:clip: similarity", - "5s-01:clap: similarity", - "5s-01: dino lower tail", - "5s-01: clap segment", - "5s-01:reference_dino_cosine: adherence", - "5s-01:audio_caption_clap_cosine: adherence" - ], - "missing": [ - "5s-00", - "5s-02", - "5s-03", - "5s-04", - "15s-00", - "15s-01", - "15s-02", - "15s-03", - "15s-04" - ], - "observed": { - "5s-01": { - "seconds": "5", - "baseline": { - "subject_consistency": 0.9213417344461612, - "background_consistency": 0.9521126235888256 - }, - "candidate": { - "subject_consistency": 0.9193919410065907, - "background_consistency": 0.9695822967261802 - }, - "cosines": { - "dino": { - "mean": 0.9559677839279175, - "min": 0.917838454246521, - "p05": 0.9317350178956986 - }, - "clip": { - "mean": 0.9704718589782715, - "min": 0.937095046043396, - "p05": 0.9475325375795365 - }, - "clap": { - "mean": 0.9627834558486938, - "min": 0.9627834558486938, - "p05": 0.9627834558486938 - } - }, - "reduced_relative_error": 0.00828409810531098, - "audio_rms_db": 0.04268223342783043, - "audio_envelope_relative_l2": 0.06482673389999478, - "motion_ratio": 1.2091018487554375 - } - }, - "scope": "Fixed Ref2VA sunset-image corpus: relative BF16 VBench subject/background consistency plus reference, content, audio-feature, motion and media-timestamp guards. No human check. These proxies do not certify absolute audiovisual semantic synchrony or general model quality.", - "performance_acceptance": "separate" - }, - { - "configuration_key": "cache-taylor3-008-max3-bn1", - "sample_id": "5s-01", - "comparison_sha256": "a20e4bd5b4c919bb5a65cd11d2bb0cc7dc6aa045ec2f3b1bd5a9c6e28489411b", - "verdict": "FAIL", - "checks": 39, - "failures": [ - "5s-01:clap: similarity", - "5s-01: clap segment" - ], - "missing": [ - "5s-00", - "5s-02", - "5s-03", - "5s-04", - "15s-00", - "15s-01", - "15s-02", - "15s-03", - "15s-04" - ], - "observed": { - "5s-01": { - "seconds": "5", - "baseline": { - "subject_consistency": 0.9213417344461612, - "background_consistency": 0.9521126235888256 - }, - "candidate": { - "subject_consistency": 0.9205945810166801, - "background_consistency": 0.961419354124767 - }, - "cosines": { - "dino": { - "mean": 0.9861611127853394, - "min": 0.9520018696784973, - "p05": 0.9690358877182007 - }, - "clip": { - "mean": 0.9868983626365662, - "min": 0.94464111328125, - "p05": 0.9714847385883332 - }, - "clap": { - "mean": 0.973813533782959, - "min": 0.973813533782959, - "p05": 0.973813533782959 - } - }, - "reduced_relative_error": 0.004568874106673343, - "audio_rms_db": 0.32023464803440704, - "audio_envelope_relative_l2": 0.048263250142338246, - "motion_ratio": 1.037731901309175 - } - }, - "scope": "Fixed Ref2VA sunset-image corpus: relative BF16 VBench subject/background consistency plus reference, content, audio-feature, motion and media-timestamp guards. No human check. These proxies do not certify absolute audiovisual semantic synchrony or general model quality.", - "performance_acceptance": "separate" - }, - { - "configuration_key": "hybrid-laser-video-bf16-context", - "sample_id": "5s-00", - "comparison_sha256": "60c4356e41b34ff791043bf34179a1fa693780b70c273405e0011e19269017d6", - "verdict": "FAIL", - "checks": 39, - "failures": [ - "5s-00:audio_caption_clap_cosine: adherence" - ], - "missing": [ - "5s-01", - "5s-02", - "5s-03", - "5s-04", - "15s-00", - "15s-01", - "15s-02", - "15s-03", - "15s-04" - ], - "observed": { - "5s-00": { - "seconds": "5", - "baseline": { - "subject_consistency": 0.9653875539457895, - "background_consistency": 0.9668180954650165 - }, - "candidate": { - "subject_consistency": 0.9661054570015853, - "background_consistency": 0.9678300546436776 - }, - "cosines": { - "dino": { - "mean": 0.9992612600326538, - "min": 0.9987664222717285, - "p05": 0.9990039587020874 - }, - "clip": { - "mean": 0.9964061379432678, - "min": 0.986167848110199, - "p05": 0.9930137008428573 - }, - "clap": { - "mean": 0.9925766587257385, - "min": 0.9925766587257385, - "p05": 0.9925766587257385 - } - }, - "reduced_relative_error": 0.0008952785305148697, - "audio_rms_db": -0.04112309268137339, - "audio_envelope_relative_l2": 0.03829713389408406, - "motion_ratio": 0.9988179931035557 - } - }, - "scope": "Fixed Ref2VA sunset-image corpus: relative BF16 VBench subject/background consistency plus reference, content, audio-feature, motion and media-timestamp guards. No human check. These proxies do not certify absolute audiovisual semantic synchrony or general model quality.", - "performance_acceptance": "separate" - }, - { - "configuration_key": "laser-scale256", - "sample_id": "5s-00", - "comparison_sha256": "91c2419327f5333f59b8d3f74cfc47d84634ad6e7d4f026e3abd61d422b9aad8", - "verdict": "FAIL", - "checks": 39, - "failures": [ - "5s-00:audio_caption_clap_cosine: adherence" - ], - "missing": [ - "5s-01", - "5s-02", - "5s-03", - "5s-04", - "15s-00", - "15s-01", - "15s-02", - "15s-03", - "15s-04" - ], - "observed": { - "5s-00": { - "seconds": "5", - "baseline": { - "subject_consistency": 0.9653875539457895, - "background_consistency": 0.9668180954650165 - }, - "candidate": { - "subject_consistency": 0.9667792068264349, - "background_consistency": 0.9680149632740797 - }, - "cosines": { - "dino": { - "mean": 0.999154806137085, - "min": 0.9985394477844238, - "p05": 0.9986860930919648 - }, - "clip": { - "mean": 0.9963837265968323, - "min": 0.9901332855224609, - "p05": 0.9924647331237793 - }, - "clap": { - "mean": 0.9945911169052124, - "min": 0.9945911169052124, - "p05": 0.9945911169052124 - } - }, - "reduced_relative_error": 0.0013396714218789517, - "audio_rms_db": -0.11301796107390463, - "audio_envelope_relative_l2": 0.12640093834958974, - "motion_ratio": 1.0010309184980533 - } - }, - "scope": "Fixed Ref2VA sunset-image corpus: relative BF16 VBench subject/background consistency plus reference, content, audio-feature, motion and media-timestamp guards. No human check. These proxies do not certify absolute audiovisual semantic synchrony or general model quality.", - "performance_acceptance": "separate" - }, - { - "configuration_key": "modality-sigma-fp32-anchor", - "sample_id": "15s-00", - "comparison_sha256": "1b75e2ea84298165d6c19e86b2a8e7ebd08835924c6e71832eed9b79d920f452", - "verdict": "FAIL", - "checks": 76, - "failures": [ - "5s-00:audio_caption_clap_cosine: adherence", - "15s-00:dino: similarity", - "15s-00:clip: similarity", - "15s-00:clap: similarity", - "15s-00: dino lower tail", - "15s-00: clap segment" - ], - "missing": [ - "5s-01", - "5s-02", - "5s-03", - "5s-04", - "15s-01", - "15s-02", - "15s-03", - "15s-04" - ], - "observed": { - "5s-00": { - "seconds": "5", - "baseline": { - "subject_consistency": 0.9653875539457895, - "background_consistency": 0.9668180954650165 - }, - "candidate": { - "subject_consistency": 0.9634841426116664, - "background_consistency": 0.9637262268764216 - }, - "cosines": { - "dino": { - "mean": 0.9979137182235718, - "min": 0.9962100982666016, - "p05": 0.9968585044145584 - }, - "clip": { - "mean": 0.993086576461792, - "min": 0.9763539433479309, - "p05": 0.9824884951114654 - }, - "clap": { - "mean": 0.9918106198310852, - "min": 0.9918106198310852, - "p05": 0.9918106198310852 - } - }, - "reduced_relative_error": 0.002585273427929995, - "audio_rms_db": -0.1567548220318038, - "audio_envelope_relative_l2": 0.08416217762953182, - "motion_ratio": 1.0089539814707082 - }, - "15s-00": { - "seconds": "15", - "baseline": { - "subject_consistency": 0.8167167416157155, - "background_consistency": 0.8849122500320551 - }, - "candidate": { - "subject_consistency": 0.8073931144875502, - "background_consistency": 0.8748147027809534 - }, - "cosines": { - "dino": { - "mean": 0.9541988372802734, - "min": 0.8350636959075928, - "p05": 0.9045321136713028 - }, - "clip": { - "mean": 0.9663479924201965, - "min": 0.8494424819946289, - "p05": 0.9202105909585953 - }, - "clap": { - "mean": 0.9531505703926086, - "min": 0.9011195302009583, - "p05": 0.9078725516796112 - } - }, - "reduced_relative_error": 0.011413283667940134, - "audio_rms_db": 0.1285537754516634, - "audio_envelope_relative_l2": 0.13562143033577656, - "motion_ratio": 1.0165335096317705 - } - }, - "scope": "Fixed Ref2VA sunset-image corpus: relative BF16 VBench subject/background consistency plus reference, content, audio-feature, motion and media-timestamp guards. No human check. These proxies do not certify absolute audiovisual semantic synchrony or general model quality.", - "performance_acceptance": "separate" - }, - { - "configuration_key": "modality-sigma-fp32-anchor", - "sample_id": "5s-00", - "comparison_sha256": "5ea1cf7b5bdc342a7b298e0469dd4042cee7c981ba533df3510052538b65e8c0", - "verdict": "FAIL", - "checks": 39, - "failures": [ - "5s-00:audio_caption_clap_cosine: adherence" - ], - "missing": [ - "5s-01", - "5s-02", - "5s-03", - "5s-04", - "15s-00", - "15s-01", - "15s-02", - "15s-03", - "15s-04" - ], - "observed": { - "5s-00": { - "seconds": "5", - "baseline": { - "subject_consistency": 0.9653875539457895, - "background_consistency": 0.9668180954650165 - }, - "candidate": { - "subject_consistency": 0.9634841426116664, - "background_consistency": 0.9637262268764216 - }, - "cosines": { - "dino": { - "mean": 0.9979137182235718, - "min": 0.9962100982666016, - "p05": 0.9968585044145584 - }, - "clip": { - "mean": 0.993086576461792, - "min": 0.9763539433479309, - "p05": 0.9824884951114654 - }, - "clap": { - "mean": 0.9918106198310852, - "min": 0.9918106198310852, - "p05": 0.9918106198310852 - } - }, - "reduced_relative_error": 0.002585273427929995, - "audio_rms_db": -0.1567548220318038, - "audio_envelope_relative_l2": 0.08416217762953182, - "motion_ratio": 1.0089539814707082 - } - }, - "scope": "Fixed Ref2VA sunset-image corpus: relative BF16 VBench subject/background consistency plus reference, content, audio-feature, motion and media-timestamp guards. No human check. These proxies do not certify absolute audiovisual semantic synchrony or general model quality.", - "performance_acceptance": "separate" - }, - { - "configuration_key": "modality-sigma-quadratic", - "sample_id": "5s-00", - "comparison_sha256": "4b795a65ab2165390844e117813b8efd24aec3d5b7b75cf5652f73576b0a8099", - "verdict": "FAIL", - "checks": 39, - "failures": [ - "5s-00:audio_caption_clap_cosine: adherence" - ], - "missing": [ - "5s-01", - "5s-02", - "5s-03", - "5s-04", - "15s-00", - "15s-01", - "15s-02", - "15s-03", - "15s-04" - ], - "observed": { - "5s-00": { - "seconds": "5", - "baseline": { - "subject_consistency": 0.9653875539457895, - "background_consistency": 0.9668180954650165 - }, - "candidate": { - "subject_consistency": 0.9642229327341405, - "background_consistency": 0.9690693287345452 - }, - "cosines": { - "dino": { - "mean": 0.9978399872779846, - "min": 0.9946956634521484, - "p05": 0.9965563297271729 - }, - "clip": { - "mean": 0.9933723211288452, - "min": 0.9708013534545898, - "p05": 0.9835974663496018 - }, - "clap": { - "mean": 0.9960051774978638, - "min": 0.9960051774978638, - "p05": 0.9960051774978638 - } - }, - "reduced_relative_error": 0.0005623687407243513, - "audio_rms_db": -0.141298705336359, - "audio_envelope_relative_l2": 0.08778629602289918, - "motion_ratio": 1.0088439516314678 - } - }, - "scope": "Fixed Ref2VA sunset-image corpus: relative BF16 VBench subject/background consistency plus reference, content, audio-feature, motion and media-timestamp guards. No human check. These proxies do not certify absolute audiovisual semantic synchrony or general model quality.", - "performance_acceptance": "separate" - }, - { - "configuration_key": "modality-sigma-quadratic-bn1", - "sample_id": "5s-00", - "comparison_sha256": "6fde89564e6572398df194e82f7634af92020ff9a84d5f2d1383df18809354f7", - "verdict": "FAIL", - "checks": 39, - "failures": [ - "5s-00: audio level" - ], - "missing": [ - "5s-01", - "5s-02", - "5s-03", - "5s-04", - "15s-00", - "15s-01", - "15s-02", - "15s-03", - "15s-04" - ], - "observed": { - "5s-00": { - "seconds": "5", - "baseline": { - "subject_consistency": 0.9653875539457895, - "background_consistency": 0.9668180954650165 - }, - "candidate": { - "subject_consistency": 0.9667953994215989, - "background_consistency": 0.9636606847852226 - }, - "cosines": { - "dino": { - "mean": 0.993774950504303, - "min": 0.9827978014945984, - "p05": 0.9883693248033524 - }, - "clip": { - "mean": 0.9904270172119141, - "min": 0.972300112247467, - "p05": 0.9838350206613541 - }, - "clap": { - "mean": 0.9939356446266174, - "min": 0.9939356446266174, - "p05": 0.9939356446266174 - } - }, - "reduced_relative_error": 0.0009054756694858051, - "audio_rms_db": -1.0007078553545243, - "audio_envelope_relative_l2": 0.19737799824495614, - "motion_ratio": 1.0067671166204435 - } - }, - "scope": "Fixed Ref2VA sunset-image corpus: relative BF16 VBench subject/background consistency plus reference, content, audio-feature, motion and media-timestamp guards. No human check. These proxies do not certify absolute audiovisual semantic synchrony or general model quality.", - "performance_acceptance": "separate" - }, - { - "configuration_key": "rainfusion50-bf16-w8-tail8", - "sample_id": "5s-00", - "comparison_sha256": "7f64efe80fcca7b976e82a2cdd9f9876832c85b226de7f26992c7fe70d020864", - "verdict": "FAIL", - "checks": 39, - "failures": [ - "5s-00:clap: similarity", - "5s-00: clap segment", - "5s-00: audio temporal envelope" - ], - "missing": [ - "5s-01", - "5s-02", - "5s-03", - "5s-04", - "15s-00", - "15s-01", - "15s-02", - "15s-03", - "15s-04" - ], - "observed": { - "5s-00": { - "seconds": "5", - "baseline": { - "subject_consistency": 0.9653875539457895, - "background_consistency": 0.9668180954650165 - }, - "candidate": { - "subject_consistency": 0.9667071133125119, - "background_consistency": 0.9689083639683762 - }, - "cosines": { - "dino": { - "mean": 0.9946473836898804, - "min": 0.9871225357055664, - "p05": 0.9900005757808685 - }, - "clip": { - "mean": 0.9907172918319702, - "min": 0.9721568822860718, - "p05": 0.9800667852163315 - }, - "clap": { - "mean": 0.9769790172576904, - "min": 0.9769790172576904, - "p05": 0.9769790172576904 - } - }, - "reduced_relative_error": 0.001764733412885843, - "audio_rms_db": -0.14111524718973395, - "audio_envelope_relative_l2": 0.2797538746282675, - "motion_ratio": 1.026263875056369 - } - }, - "scope": "Fixed Ref2VA sunset-image corpus: relative BF16 VBench subject/background consistency plus reference, content, audio-feature, motion and media-timestamp guards. No human check. These proxies do not certify absolute audiovisual semantic synchrony or general model quality.", - "performance_acceptance": "separate" - }, - { - "configuration_key": "rainfusion80-bf16-w4-tail4", - "sample_id": "5s-00", - "comparison_sha256": "c739c5b0f2db605991f40a5d2c0adc7ced4d025717a581bbeed157f5f80a717e", - "verdict": "FAIL", - "checks": 39, - "failures": [ - "5s-00:dino: similarity", - "5s-00:clip: similarity", - "5s-00:clap: similarity", - "5s-00: clap segment", - "5s-00: audio level", - "5s-00: audio temporal envelope" - ], - "missing": [ - "5s-01", - "5s-02", - "5s-03", - "5s-04", - "15s-00", - "15s-01", - "15s-02", - "15s-03", - "15s-04" - ], - "observed": { - "5s-00": { - "seconds": "5", - "baseline": { - "subject_consistency": 0.9653875539457895, - "background_consistency": 0.9668180954650165 - }, - "candidate": { - "subject_consistency": 0.9632140490097728, - "background_consistency": 0.9603373892908174 - }, - "cosines": { - "dino": { - "mean": 0.9794362187385559, - "min": 0.9407444000244141, - "p05": 0.9508843153715134 - }, - "clip": { - "mean": 0.9764617681503296, - "min": 0.9348478317260742, - "p05": 0.9474407732486725 - }, - "clap": { - "mean": 0.9637646079063416, - "min": 0.9637646079063416, - "p05": 0.9637646079063416 - } - }, - "reduced_relative_error": 0.00447892858239735, - "audio_rms_db": -2.173295241375743, - "audio_envelope_relative_l2": 0.36216840555897617, - "motion_ratio": 1.0477099329740793 - } - }, - "scope": "Fixed Ref2VA sunset-image corpus: relative BF16 VBench subject/background consistency plus reference, content, audio-feature, motion and media-timestamp guards. No human check. These proxies do not certify absolute audiovisual semantic synchrony or general model quality.", - "performance_acceptance": "separate" - }, - { - "configuration_key": "refkv-device-refresh2", - "sample_id": "5s-00", - "comparison_sha256": "6682812c58757690b5114492834e68c54e17988e22834e45dd93856a3e4cfd27", - "verdict": "FAIL", - "checks": 39, - "failures": [ - "5s-00:clap: similarity" - ], - "missing": [ - "5s-01", - "5s-02", - "5s-03", - "5s-04", - "15s-00", - "15s-01", - "15s-02", - "15s-03", - "15s-04" - ], - "observed": { - "5s-00": { - "seconds": "5", - "baseline": { - "subject_consistency": 0.9653875539457895, - "background_consistency": 0.9668180954650165 - }, - "candidate": { - "subject_consistency": 0.9673074055008772, - "background_consistency": 0.9747018482142348 - }, - "cosines": { - "dino": { - "mean": 0.9847110509872437, - "min": 0.937708854675293, - "p05": 0.9595070838928222 - }, - "clip": { - "mean": 0.9860638380050659, - "min": 0.9606876373291016, - "p05": 0.9736308753490448 - }, - "clap": { - "mean": 0.9891108274459839, - "min": 0.9891108274459839, - "p05": 0.9891108274459839 - } - }, - "reduced_relative_error": 0.005073789276672285, - "audio_rms_db": -0.11514590616150458, - "audio_envelope_relative_l2": 0.1555914434459509, - "motion_ratio": 0.8933582476698538 - } - }, - "scope": "Fixed Ref2VA sunset-image corpus: relative BF16 VBench subject/background consistency plus reference, content, audio-feature, motion and media-timestamp guards. No human check. These proxies do not certify absolute audiovisual semantic synchrony or general model quality.", - "performance_acceptance": "separate" - }, - { - "configuration_key": "state-aligned-residual-v1", - "sample_id": "5s-00", - "comparison_sha256": "f4b14d6eba264c0c348cc01862c45ad62ed71409cadd5cd66465b85d63c89d60", - "verdict": "INCONCLUSIVE", - "checks": 39, - "failures": [], - "missing": [ - "5s-01", - "5s-02", - "5s-03", - "5s-04", - "15s-00", - "15s-01", - "15s-02", - "15s-03", - "15s-04" - ], - "observed": { - "5s-00": { - "seconds": "5", - "baseline": { - "subject_consistency": 0.9653875539457895, - "background_consistency": 0.9668180954650165 - }, - "candidate": { - "subject_consistency": 0.9653875539457895, - "background_consistency": 0.9668180954650165 - }, - "cosines": { - "dino": { - "mean": 1.0, - "min": 0.9999998211860657, - "p05": 0.9999998807907104 - }, - "clip": { - "mean": 1.0, - "min": 0.9999997615814209, - "p05": 0.9999998807907104 - }, - "clap": { - "mean": 1.0, - "min": 1.0, - "p05": 1.0 - } - }, - "reduced_relative_error": 0.0, - "audio_rms_db": 0.0, - "audio_envelope_relative_l2": 0.0, - "motion_ratio": 1.0 - } - }, - "scope": "Fixed Ref2VA sunset-image corpus: relative BF16 VBench subject/background consistency plus reference, content, audio-feature, motion and media-timestamp guards. No human check. These proxies do not certify absolute audiovisual semantic synchrony or general model quality.", - "performance_acceptance": "separate" - }, - { - "configuration_key": "tp4-all-resident", - "sample_id": "5s-00", - "comparison_sha256": "7b9e4ba13d0b6c098c7c529e6d3f23a885559681114850ae551f7a6cdeee8684", - "verdict": "FAIL", - "checks": 39, - "failures": [ - "5s-00:audio_caption_clap_cosine: adherence" - ], - "missing": [ - "5s-01", - "5s-02", - "5s-03", - "5s-04", - "15s-00", - "15s-01", - "15s-02", - "15s-03", - "15s-04" - ], - "observed": { - "5s-00": { - "seconds": "5", - "baseline": { - "subject_consistency": 0.9653875539457895, - "background_consistency": 0.9668180954650165 - }, - "candidate": { - "subject_consistency": 0.966968424678818, - "background_consistency": 0.9685012627907885 - }, - "cosines": { - "dino": { - "mean": 0.9991623163223267, - "min": 0.9984581470489502, - "p05": 0.9986790925264358 - }, - "clip": { - "mean": 0.9960500597953796, - "min": 0.9835643768310547, - "p05": 0.991750293970108 - }, - "clap": { - "mean": 0.9936883449554443, - "min": 0.9936883449554443, - "p05": 0.9936883449554443 - } - }, - "reduced_relative_error": 0.001689280879494273, - "audio_rms_db": -0.5914884173619972, - "audio_envelope_relative_l2": 0.14660693414345288, - "motion_ratio": 0.9986264182059804 - } - }, - "scope": "Fixed Ref2VA sunset-image corpus: relative BF16 VBench subject/background consistency plus reference, content, audio-feature, motion and media-timestamp guards. No human check. These proxies do not certify absolute audiovisual semantic synchrony or general model quality.", - "performance_acceptance": "separate" - }, - { - "configuration_key": "video-context-lse", - "sample_id": "5s-00", - "comparison_sha256": "2b3d31ae933c50c78571a71deee42a3a23ec6f4e6986b3fe4b1e76945544aa13", - "verdict": "FAIL", - "checks": 39, - "failures": [ - "5s-00:dino: similarity", - "5s-00:clip: similarity", - "5s-00:clap: similarity", - "5s-00: clap segment", - "5s-00:reference_dino_cosine: adherence", - "5s-00:audio_caption_clap_cosine: adherence", - "5s-00: audio level", - "5s-00: audio temporal envelope" - ], - "missing": [ - "5s-01", - "5s-02", - "5s-03", - "5s-04", - "15s-00", - "15s-01", - "15s-02", - "15s-03", - "15s-04" - ], - "observed": { - "5s-00": { - "seconds": "5", - "baseline": { - "subject_consistency": 0.9653875539457895, - "background_consistency": 0.9668180954650165 - }, - "candidate": { - "subject_consistency": 0.9650656626961096, - "background_consistency": 0.9708255100540999 - }, - "cosines": { - "dino": { - "mean": 0.97896808385849, - "min": 0.9462954998016357, - "p05": 0.9537913143634796 - }, - "clip": { - "mean": 0.9749853014945984, - "min": 0.9158563017845154, - "p05": 0.9535898625850677 - }, - "clap": { - "mean": 0.9371471405029297, - "min": 0.9371471405029297, - "p05": 0.9371471405029297 - } - }, - "reduced_relative_error": 0.001907417743306655, - "audio_rms_db": -6.375729760723826, - "audio_envelope_relative_l2": 0.5422772289154131, - "motion_ratio": 0.9732356030412213 - } - }, - "scope": "Fixed Ref2VA sunset-image corpus: relative BF16 VBench subject/background consistency plus reference, content, audio-feature, motion and media-timestamp guards. No human check. These proxies do not certify absolute audiovisual semantic synchrony or general model quality.", - "performance_acceptance": "separate" - }, - { - "configuration_key": "video-pre-gate", - "sample_id": "5s-00", - "comparison_sha256": "f821a089e7bacbbd51d2debe40169d23db13cc5ff744b2237acc3474765bf68c", - "verdict": "FAIL", - "checks": 39, - "failures": [ - "5s-00:dino: similarity", - "5s-00:clip: similarity", - "5s-00:clap: similarity", - "5s-00: dino lower tail", - "5s-00: clap segment", - "5s-00:reference_dino_cosine: adherence", - "5s-00: audio temporal envelope" - ], - "missing": [ - "5s-01", - "5s-02", - "5s-03", - "5s-04", - "15s-00", - "15s-01", - "15s-02", - "15s-03", - "15s-04" - ], - "observed": { - "5s-00": { - "seconds": "5", - "baseline": { - "subject_consistency": 0.9653875539457895, - "background_consistency": 0.9668180954650165 - }, - "candidate": { - "subject_consistency": 0.9604354704298624, - "background_consistency": 0.9678790501462735 - }, - "cosines": { - "dino": { - "mean": 0.9699685573577881, - "min": 0.9310290217399597, - "p05": 0.9455805897712708 - }, - "clip": { - "mean": 0.9705709218978882, - "min": 0.9431105852127075, - "p05": 0.9498869240283966 - }, - "clap": { - "mean": 0.9706100225448608, - "min": 0.9706100225448608, - "p05": 0.9706100225448608 - } - }, - "reduced_relative_error": 0.0020138274804530603, - "audio_rms_db": -0.20747782268393988, - "audio_envelope_relative_l2": 0.2799833627923924, - "motion_ratio": 1.121115241596866 - } - }, - "scope": "Fixed Ref2VA sunset-image corpus: relative BF16 VBench subject/background consistency plus reference, content, audio-feature, motion and media-timestamp guards. No human check. These proxies do not certify absolute audiovisual semantic synchrony or general model quality.", - "performance_acceptance": "separate" - }, - { - "configuration_key": "video-pre-gate-secant", - "sample_id": "5s-00", - "comparison_sha256": "6a809a10611ec2231cc9da15514924d8c1a61ecb3a6abdcf8baad1425a0d357c", - "verdict": "FAIL", - "checks": 39, - "failures": [ - "5s-00:dino: similarity", - "5s-00:clip: similarity", - "5s-00:clap: similarity", - "5s-00: dino lower tail", - "5s-00: clap segment", - "5s-00:reference_clip_cosine: adherence", - "5s-00:reference_dino_cosine: adherence", - "5s-00:audio_caption_clap_cosine: adherence", - "5s-00: motion energy" - ], - "missing": [ - "5s-01", - "5s-02", - "5s-03", - "5s-04", - "15s-00", - "15s-01", - "15s-02", - "15s-03", - "15s-04" - ], - "observed": { - "5s-00": { - "seconds": "5", - "baseline": { - "subject_consistency": 0.9653875539457895, - "background_consistency": 0.9668180954650165 - }, - "candidate": { - "subject_consistency": 0.9611704327711245, - "background_consistency": 0.9549353379544204 - }, - "cosines": { - "dino": { - "mean": 0.9575454592704773, - "min": 0.9346060752868652, - "p05": 0.9436570793390274 - }, - "clip": { - "mean": 0.9614511132240295, - "min": 0.919788658618927, - "p05": 0.9337257981300354 - }, - "clap": { - "mean": 0.9732030630111694, - "min": 0.9732030630111694, - "p05": 0.9732030630111694 - } - }, - "reduced_relative_error": 0.008332383610497398, - "audio_rms_db": 0.5065753171403786, - "audio_envelope_relative_l2": 0.24859357669358687, - "motion_ratio": 1.2605876841404495 - } - }, - "scope": "Fixed Ref2VA sunset-image corpus: relative BF16 VBench subject/background consistency plus reference, content, audio-feature, motion and media-timestamp guards. No human check. These proxies do not certify absolute audiovisual semantic synchrony or general model quality.", - "performance_acceptance": "separate" - }, - { - "configuration_key": "video-row-fp32", - "sample_id": "5s-00", - "comparison_sha256": "9226ab5741347e5928911c4e01a136c64f85bfc7406f812ff4a936b18a41c272", - "verdict": "FAIL", - "checks": 39, - "failures": [ - "5s-00:clap: similarity", - "5s-00: clap segment", - "5s-00: audio temporal envelope" - ], - "missing": [ - "5s-01", - "5s-02", - "5s-03", - "5s-04", - "15s-00", - "15s-01", - "15s-02", - "15s-03", - "15s-04" - ], - "observed": { - "5s-00": { - "seconds": "5", - "baseline": { - "subject_consistency": 0.9653875539457895, - "background_consistency": 0.9668180954650165 - }, - "candidate": { - "subject_consistency": 0.9644393346658567, - "background_consistency": 0.9664301145367507 - }, - "cosines": { - "dino": { - "mean": 0.997343897819519, - "min": 0.9954850673675537, - "p05": 0.9960428774356842 - }, - "clip": { - "mean": 0.9929363131523132, - "min": 0.9782494902610779, - "p05": 0.9856640666723251 - }, - "clap": { - "mean": 0.9739290475845337, - "min": 0.9739290475845337, - "p05": 0.9739290475845337 - } - }, - "reduced_relative_error": 0.0006915414043044125, - "audio_rms_db": -0.10083169173992401, - "audio_envelope_relative_l2": 0.2743734098838323, - "motion_ratio": 1.0172588417564994 - } - }, - "scope": "Fixed Ref2VA sunset-image corpus: relative BF16 VBench subject/background consistency plus reference, content, audio-feature, motion and media-timestamp guards. No human check. These proxies do not certify absolute audiovisual semantic synchrony or general model quality.", - "performance_acceptance": "separate" - }, - { - "configuration_key": "video-temporal-window", - "sample_id": "5s-00", - "comparison_sha256": "f7f23452cea62d3ff53d6289a1b988e5279de9be52856c5ea7a4af849025ed40", - "verdict": "FAIL", - "checks": 39, - "failures": [ - "5s-00:dino: similarity", - "5s-00:clip: similarity", - "5s-00:clap: similarity", - "5s-00: dino lower tail", - "5s-00: clap segment", - "5s-00:reference_clip_cosine: adherence", - "5s-00:reference_dino_cosine: adherence", - "5s-00:prompt_clip_cosine: adherence", - "5s-00: audio level", - "5s-00: audio temporal envelope", - "5s-00: motion energy" - ], - "missing": [ - "5s-01", - "5s-02", - "5s-03", - "5s-04", - "15s-00", - "15s-01", - "15s-02", - "15s-03", - "15s-04" - ], - "observed": { - "5s-00": { - "seconds": "5", - "baseline": { - "subject_consistency": 0.9653875539457895, - "background_consistency": 0.9668180954650165 - }, - "candidate": { - "subject_consistency": 0.9351117109380117, - "background_consistency": 0.9415582151916938 - }, - "cosines": { - "dino": { - "mean": 0.8847789168357849, - "min": 0.7981574535369873, - "p05": 0.8222497254610062 - }, - "clip": { - "mean": 0.9208264350891113, - "min": 0.843837559223175, - "p05": 0.870898300409317 - }, - "clap": { - "mean": 0.06382762640714645, - "min": 0.06382762640714645, - "p05": 0.06382762640714645 - } - }, - "reduced_relative_error": 0.02874213896333196, - "audio_rms_db": -36.23827660321877, - "audio_envelope_relative_l2": 0.9866005095726371, - "motion_ratio": 5.481376339509837 - } - }, - "scope": "Fixed Ref2VA sunset-image corpus: relative BF16 VBench subject/background consistency plus reference, content, audio-feature, motion and media-timestamp guards. No human check. These proxies do not certify absolute audiovisual semantic synchrony or general model quality.", - "performance_acceptance": "separate" - } - ] -} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c1-15s-execution.json b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c1-15s-execution.json deleted file mode 100644 index 76d33fdf2..000000000 --- a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c1-15s-execution.json +++ /dev/null @@ -1,47 +0,0 @@ -{ - "source_commit": "be319692b96413918973b5bd399cdda3f320c574", - "scope": "http_complete_mp4", - "samples_ms": [ - 1707035.2496140404, - 1677409.5148360357 - ], - "latency_median_ms": 1692222.382225038, - "warmup": 0, - "repetitions": 2, - "diagnostic": "audit", - "clean_timing": false, - "cleanup_passed": true, - "actual_attention_calls": { - "0": { - "0": { - "_prefix_bnsd": 2500 - }, - "1": { - "_prefix_bnsd": 2500 - }, - "2": { - "_prefix_bnsd": 2500 - }, - "3": { - "_prefix_bnsd": 2500 - } - }, - "1": { - "0": { - "_prefix_bnsd": 2500 - }, - "1": { - "_prefix_bnsd": 2500 - }, - "2": { - "_prefix_bnsd": 2500 - }, - "3": { - "_prefix_bnsd": 2500 - } - } - }, - "numerical_result": "c1-15s-strict-comparison.json", - "performance_acceptance": "NOT_EVALUATED", - "campaign_verdict": "INCONCLUSIVE" -} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c1-clean-5s-screening.json b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c1-clean-5s-screening.json deleted file mode 100644 index cbb3c90af..000000000 --- a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c1-clean-5s-screening.json +++ /dev/null @@ -1,85 +0,0 @@ -{ - "native_run_id": "20261101T181531Z-164cfec13c", - "inferrsi_revision": "50c19da43e0b358e6f990c00719891ad308a8ac1", - "candidate_patch_base": "922f1e9f1cb85b9f2b66d86e1eb500e6872b2dcb", - "gain_attribution_limit": "Original B0 versus migration+C1; no clean migration-only timing arm in this pair", - "duration_s": 5, - "frames": 124, - "solver_steps": 50, - "order": "AB", - "fresh_pairs": 1, - "kind": "screening", - "native_cli_discovery_physical_ids": [ - 1, - 2 - ], - "runtime_logical_devices": [ - 2, - 3, - 4, - 5 - ], - "arms": { - "baseline": { - "source_head": "df1c1b05936381dbaa3891c1ae4c4b21ab375a5d", - "samples_ms": [ - 291247.76331102476, - 291434.0972539503, - 291051.2047139928 - ], - "server_median_ms": 291247.76331102476, - "inner_range_pct_of_median": 0.13146625938158557, - "warmup": 2, - "repetitions": 3, - "clean_timing": true, - "diagnostic": "off", - "scope": "http_complete_mp4", - "cleanup_passed": true, - "helpers_unchanged": true - }, - "candidate": { - "source_head": "be319692b96413918973b5bd399cdda3f320c574", - "samples_ms": [ - 286812.4624299817, - 286666.7601839872, - 286197.25764304167 - ], - "server_median_ms": 286666.7601839872, - "inner_range_pct_of_median": 0.21460625101604527, - "warmup": 2, - "repetitions": 3, - "clean_timing": true, - "diagnostic": "off", - "scope": "http_complete_mp4", - "cleanup_passed": true, - "helpers_unchanged": true - } - }, - "http_reduction_pct": 1.5728886893271898, - "required_reduction_pct": 50.0, - "native_verdict": "FAIL", - "native_reasons": [ - "measured gain 1.57289% must be positive and meet 50% threshold" - ], - "run_state": "completed", - "source_integrity": { - "verified": true, - "reason": "Git source snapshots unchanged", - "command_checks": 3 - }, - "strict_quality": { - "verdict": "PASS", - "checks": 35, - "failures": [], - "scope": "matched strict qualification plus every measured clean RGB24/PCM response", - "final_protocol": false - }, - "strict_validation_execution": { - "status": "completed", - "returncode": 0 - }, - "statistical_significance": "not established", - "campaign_verdict": "INCONCLUSIVE: four tracks and final protocol are incomplete", - "raw_media_included": false, - "raw_records_included": false -} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c1-queryfold-operator.json b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c1-queryfold-operator.json deleted file mode 100644 index 0354f9809..000000000 --- a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c1-queryfold-operator.json +++ /dev/null @@ -1,97 +0,0 @@ -{ - "scope": "Single real5s first-long-attention fixture,logical device count1; not whole-model performance", - "cases": [ - { - "name": "bnsd_query_tiles2", - "bit_exact": true, - "unequal_elements": 0, - "compared_elements": 71269632, - "finite": true, - "reference_sha256": "7cabc1030b41d2e42c77aac352c20a2b676e194d1f9883b54d5c9b8f72b38c21", - "candidate_sha256": "7cabc1030b41d2e42c77aac352c20a2b676e194d1f9883b54d5c9b8f72b38c21", - "baseline_ms": [ - 71.68554002419114, - 71.42315799137577, - 71.29112700931728, - 71.63919898448512, - 71.81268103886396, - 71.30170601885766, - 71.39379699947312 - ], - "candidate_ms": [ - 73.04394000675529, - 72.85228901309893, - 72.96156999655068, - 73.32086301175877, - 72.36289500724524, - 72.75056897196919, - 72.87441898370162 - ], - "baseline_median_ms": 71.42315799137577, - "candidate_median_ms": 72.87441898370162, - "reduction_pct": -2.0319193846078276 - }, - { - "name": "bnsd_query_tiles4", - "bit_exact": true, - "unequal_elements": 0, - "compared_elements": 71269632, - "finite": true, - "reference_sha256": "7cabc1030b41d2e42c77aac352c20a2b676e194d1f9883b54d5c9b8f72b38c21", - "candidate_sha256": "7cabc1030b41d2e42c77aac352c20a2b676e194d1f9883b54d5c9b8f72b38c21", - "baseline_ms": [ - 71.867641992867, - 71.62905996665359, - 71.80321001214907, - 71.19845598936081, - 71.26071699894965, - 71.53511798242107, - 71.6239099856466 - ], - "candidate_ms": [ - 73.54129396844655, - 72.99410999985412, - 73.14522104570642, - 73.51523399120197, - 73.63296498078853, - 73.1829310534522, - 73.33417201880366 - ], - "baseline_median_ms": 71.6239099856466, - "candidate_median_ms": 73.33417201880366, - "reduction_pct": -2.387836734269033 - }, - { - "name": "bnsd_query_tiles8", - "bit_exact": true, - "unequal_elements": 0, - "compared_elements": 71269632, - "finite": true, - "reference_sha256": "7cabc1030b41d2e42c77aac352c20a2b676e194d1f9883b54d5c9b8f72b38c21", - "candidate_sha256": "7cabc1030b41d2e42c77aac352c20a2b676e194d1f9883b54d5c9b8f72b38c21", - "baseline_ms": [ - 71.52353797573596, - 71.5064779506065, - 71.6628700029105, - 71.23498595319688, - 71.53764902614057, - 71.93902198923752, - 72.01983203412965 - ], - "candidate_ms": [ - 74.3553499924019, - 75.21554798586294, - 76.08375401468948, - 75.20787802059203, - 75.23201801814139, - 75.40346903260797, - 75.78394201118499 - ], - "baseline_median_ms": 71.53764902614057, - "candidate_median_ms": 75.23201801814139, - "reduction_pct": -5.164230363023048 - } - ], - "campaign_verdict": "INCONCLUSIVE", - "decision": "All three variants are bit-exact but slower; do not integrate." -} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c1-upstream-main-native.json b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c1-upstream-main-native.json deleted file mode 100644 index 1412027bb..000000000 --- a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c1-upstream-main-native.json +++ /dev/null @@ -1,74 +0,0 @@ -{ - "upstream_pr": "https://github.com/vllm-project/vllm-omni/pull/9745", - "draft": true, - "base": "b84bade9b9be8fdad0a1a9b3e8dd3ab4273dc4d9", - "head": "7a20cf9b987b22fe3ce00221cb30cc3c6fb90e63", - "source_bound_files": { - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": { - "tested_working_sha256": "25989bba0e0f07b65c33d69e1c217655e65bf62722c982032f9c749bf560553c", - "published_blob_sha256": "25989bba0e0f07b65c33d69e1c217655e65bf62722c982032f9c749bf560553c", - "difference": "none" - }, - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": { - "tested_working_sha256": "e7e26ad0b2a9120b99281bf59a67eced7f7207f5679a8f02f2ede5859695be1a", - "published_blob_sha256": "e7e26ad0b2a9120b99281bf59a67eced7f7207f5679a8f02f2ede5859695be1a", - "difference": "none" - }, - "tests/diffusion/models/minimax_h3/test_minimax_h3_flash_attention.py": { - "tested_working_sha256": "94c17d91c4dfe1f611655002f7a4b10a6ea9e49cf21d37469c26daff58f1e766", - "published_blob_sha256": "94c17d91c4dfe1f611655002f7a4b10a6ea9e49cf21d37469c26daff58f1e766", - "difference": "none" - } - }, - "source_snapshot_sha256": "925d80d2c26f4e37e3cbf287b2d942d474744236200e12f130d2003cbcddc9cf", - "all_source_support_files_bound": 2285, - "native_run": "c1-upstream-pr-v2-native-a3c17706", - "hardware": { - "model": "Ascend910C", - "logical_device_count": 1, - "memory_gib": 64 - }, - "stack": { - "torch": "2.10.0+cpu", - "torch_npu": "2.10.0.post4", - "cann": "9.1", - "mindie_sd": "3.1.0", - "mindie_sd_source_revision": "UNKNOWN", - "vllm": "0.32.0+empty" - }, - "execution_status": "completed", - "tests_passed": 12, - "tests_failed": 0, - "tests_skipped": 0, - "required_native_executed": true, - "source_unchanged": true, - "cleanup_passed": true, - "executor_elapsed_seconds": 50.47770592797315, - "test_cases": [ - "test_h3_prefix_dispatch_preserves_unselected_paths[fused]", - "test_h3_prefix_dispatch_preserves_unselected_paths[default]", - "test_h3_prefix_dispatch_preserves_unselected_paths[laser]", - "test_h3_prefix_dispatch_preserves_unselected_paths[causal]", - "test_h3_prefix_dispatch_preserves_unselected_paths[masked]", - "test_h3_prefix_dispatch_preserves_unselected_paths[compact]", - "test_h3_prefix_dispatch_preserves_unselected_paths[gqa]", - "test_h3_prefix_dispatch_preserves_unselected_paths[unresolved]", - "test_h3_prefix_dispatch_preserves_unselected_paths[short]", - "test_h3_prefix_dispatch_preserves_unselected_paths[piecewise]", - "test_h3_prefix_dispatch_preserves_unselected_paths[ranges]", - "test_native_prefix_bnsd_excludes_padding_and_preserves_inputs" - ], - "prior_attempt": { - "run": "c1-upstream-pr-native-80d5d51f", - "status": "failed", - "phase": "collection", - "missing_existing_upstream_helper": "tests.model_executor.helpers", - "tests_executed": 0 - }, - "scope": "Main-based C1 dispatch/native operation checks; not full-model/HTTP requalification.", - "historical_c1_strict_results_apply_to": "be319692b96413918973b5bd399cdda3f320c574", - "full_model_5s_15s_on_new_base": "NOT_RUN", - "clean_http_on_new_base": "NOT_RUN", - "new_search_direction_charged": false, - "campaign_verdict": "INCONCLUSIVE" -} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c10-5s-00-audit.json b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c10-5s-00-audit.json deleted file mode 100644 index 328cadaf3..000000000 --- a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c10-5s-00-audit.json +++ /dev/null @@ -1,73 +0,0 @@ -{ - "source_commit": "0af99184bc860cffa8d1e5343c2bff4664f667c5", - "sample_id": "5s-00", - "scope": "http_complete_mp4", - "diagnostic": "audit", - "clean_timing": false, - "samples_ms": [ - 243038.95941301016 - ], - "cleanup_passed": true, - "actual_execution": { - "0": { - "calls": { - "MiniMaxH3AudioGuardCachedBlocks.forward": 50 - }, - "guard": { - "checks": 42, - "vetoes": 27, - "audio_reuse_limit": 0.04 - } - }, - "1": { - "calls": { - "MiniMaxH3AudioGuardCachedBlocks.forward": 50 - }, - "guard": { - "checks": 42, - "vetoes": 27, - "audio_reuse_limit": 0.04 - } - }, - "2": { - "calls": { - "MiniMaxH3AudioGuardCachedBlocks.forward": 50 - }, - "guard": { - "checks": 42, - "vetoes": 27, - "audio_reuse_limit": 0.04 - } - }, - "3": { - "calls": { - "MiniMaxH3AudioGuardCachedBlocks.forward": 50 - }, - "guard": { - "checks": 42, - "vetoes": 27, - "audio_reuse_limit": 0.04 - } - } - }, - "quality_contract_sha256": "b8fd9e634e6db5ebfbce3842f3360f1b2241a08119953ee71bfc26553dbbafed", - "cumulative_observed_checks": 76, - "observed_failures": [ - "5s-00:clap: similarity", - "5s-00:audio_caption_clap_cosine: adherence" - ], - "quality_verdict": "FAIL", - "missing_samples": [ - "5s-02", - "5s-03", - "5s-04", - "15s-00", - "15s-01", - "15s-02", - "15s-03", - "15s-04" - ], - "decision": "Reject original C10 for5s; its first5s-01 observation did not generalize.", - "performance_acceptance": "NOT_EVALUATED", - "campaign_verdict": "INCONCLUSIVE" -} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c10-5s-01-audit.json b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c10-5s-01-audit.json deleted file mode 100644 index 4b1162530..000000000 --- a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c10-5s-01-audit.json +++ /dev/null @@ -1,31 +0,0 @@ -{ - "source_commit": "0af99184bc860cffa8d1e5343c2bff4664f667c5", - "sample_id": "5s-01", - "scope": "http_complete_mp4", - "diagnostic": "audit", - "clean_timing": false, - "warmups": 0, - "repetitions": 1, - "samples_ms": [ - 236856.30676796427 - ], - "cleanup_passed": true, - "quality_contract_sha256": "b8fd9e634e6db5ebfbce3842f3360f1b2241a08119953ee71bfc26553dbbafed", - "observed_quality_checks": 39, - "observed_failures": [], - "quality_verdict": "INCONCLUSIVE", - "missing_samples": [ - "5s-00", - "5s-02", - "5s-03", - "5s-04", - "15s-00", - "15s-01", - "15s-02", - "15s-03", - "15s-04" - ], - "execution_proof_limit": "The actual log selects the H3 wrapper and reports16cachehits. Post-denoise model-tree traversal missed its closure-held wrapper; direct guard check/veto counters remain unobserved for this run.", - "campaign_verdict": "INCONCLUSIVE", - "performance_acceptance": "NOT_EVALUATED" -} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c10-native-tests.json b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c10-native-tests.json deleted file mode 100644 index 6618ec113..000000000 --- a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c10-native-tests.json +++ /dev/null @@ -1,18 +0,0 @@ -{ - "scope": "focused tests only; no full-model quality or performance acceptance", - "returncode": 0, - "status": "completed", - "cleanup_passed": true, - "source_unchanged": true, - "native_test_executed": true, - "tested_source_files": { - "vllm_omni/diffusion/models/minimax_h3/cache_audio_guard.py": "2844e749c9c876e52bffeef565f2c69904693e2d62c359da6abe66844930daed", - "vllm_omni/diffusion/models/minimax_h3/reference_kv_cache_dit.py": "d3381079ff012f32176a917bb0825d12288c058d3a058f5169ecc97d1e2c675c", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "1d5e806489de94cec2f9224f7d31ff96d9ace0a3fb80c23d8f8ba25f262d275d", - "tests/diffusion/models/minimax_h3/test_minimax_h3_audio_cache_guard.py": "c13e1ec677531457e92853d3a97e1e3fdcccb9547883ef0088f75852473d523d" - }, - "source_base_head": "922f1e9f1cb85b9f2b66d86e1eb500e6872b2dcb", - "source_commit": "0af99184bc860cffa8d1e5343c2bff4664f667c5", - "full_model_quality": "configuration-specific; original Fn1Bn1 failed5s; see quality records", - "performance_acceptance": "PENDING" -} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c11-fp32-15s-00-audit.json b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c11-fp32-15s-00-audit.json deleted file mode 100644 index 717956262..000000000 --- a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c11-fp32-15s-00-audit.json +++ /dev/null @@ -1,72 +0,0 @@ -{ - "trial": "c11-sigma-fp32-quality-15s-00-aaa2b0d6", - "source_commit": "d4d25b420889965ab42459c96ab4a82389c9fa64", - "cache_configuration": { - "Fn_compute_blocks": 1, - "Bn_compute_blocks": 0, - "max_warmup_steps": 4, - "scm_steps_mask_policy": null, - "residual_diff_threshold": 0.08, - "max_continuous_cached_steps": 3, - "enable_taylorseer": false, - "minimax_h3_sigma_cache": true - }, - "latency_diagnostic_ms": 845953.8551190053, - "clean_timing": false, - "input_checks_passed": 134, - "cleanup_passed": true, - "actual_middle_counts_by_rank": [ - { - "computed": 23, - "predicted": 27, - "history_vetoes": 0 - }, - { - "computed": 23, - "predicted": 27, - "history_vetoes": 0 - }, - { - "computed": 23, - "predicted": 27, - "history_vetoes": 0 - }, - { - "computed": 23, - "predicted": 27, - "history_vetoes": 0 - } - ], - "quality_contract_sha256": "b8fd9e634e6db5ebfbce3842f3360f1b2241a08119953ee71bfc26553dbbafed", - "duration_verdict": "FAIL", - "duration_failures": [ - "15s-00:dino: similarity", - "15s-00:clip: similarity", - "15s-00:clap: similarity", - "15s-00: dino lower tail", - "15s-00: clap segment" - ], - "all_comparison_failures": [ - "5s-00:audio_caption_clap_cosine: adherence", - "15s-00:dino: similarity", - "15s-00:clip: similarity", - "15s-00:clap: similarity", - "15s-00: dino lower tail", - "15s-00: clap segment" - ], - "global_verdict": "FAIL", - "missing_samples": [ - "5s-01", - "5s-02", - "5s-03", - "5s-04", - "15s-01", - "15s-02", - "15s-03", - "15s-04" - ], - "media_sha256": "afe42452f0852eabb49d123a8e6e328891b56876cf265fc71b7fdacff1f6bbbd", - "features_sha256": "561381462449683f8fb0ca3536fbff580789352aaf36b74efc9ece5c23b9fabf", - "decision": "Reject15s from its own five observed violations. Known5s failure is retained separately.", - "performance_acceptance": "PENDING; diagnostic HTTP cannot qualify a rejected candidate" -} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c11-fp32-15s-first-step.json b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c11-fp32-15s-first-step.json deleted file mode 100644 index 2968f3182..000000000 --- a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c11-fp32-15s-first-step.json +++ /dev/null @@ -1,102 +0,0 @@ -{ - "checks": [ - { - "rank": 0, - "point": "initial_noise", - "stream": "video", - "equal": true - }, - { - "rank": 0, - "point": "initial_noise", - "stream": "audio", - "equal": true - }, - { - "rank": 0, - "point": "post_step_000", - "stream": "video", - "equal": true - }, - { - "rank": 0, - "point": "post_step_000", - "stream": "audio", - "equal": true - }, - { - "rank": 1, - "point": "initial_noise", - "stream": "video", - "equal": true - }, - { - "rank": 1, - "point": "initial_noise", - "stream": "audio", - "equal": true - }, - { - "rank": 1, - "point": "post_step_000", - "stream": "video", - "equal": true - }, - { - "rank": 1, - "point": "post_step_000", - "stream": "audio", - "equal": true - }, - { - "rank": 2, - "point": "initial_noise", - "stream": "video", - "equal": true - }, - { - "rank": 2, - "point": "initial_noise", - "stream": "audio", - "equal": true - }, - { - "rank": 2, - "point": "post_step_000", - "stream": "video", - "equal": true - }, - { - "rank": 2, - "point": "post_step_000", - "stream": "audio", - "equal": true - }, - { - "rank": 3, - "point": "initial_noise", - "stream": "video", - "equal": true - }, - { - "rank": 3, - "point": "initial_noise", - "stream": "audio", - "equal": true - }, - { - "rank": 3, - "point": "post_step_000", - "stream": "video", - "equal": true - }, - { - "rank": 3, - "point": "post_step_000", - "stream": "audio", - "equal": true - } - ], - "all_equal": true, - "scope": "15s initialnoise and full-compute poststep0 only; not later cachedstates, fullquality or timing acceptance" -} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c11-fp32-5s-00-audit.json b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c11-fp32-5s-00-audit.json deleted file mode 100644 index abc73241f..000000000 --- a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c11-fp32-5s-00-audit.json +++ /dev/null @@ -1,62 +0,0 @@ -{ - "trial": "c11-sigma-fp32-quality-5s-00-7c7b2032", - "source_commit": "d4d25b420889965ab42459c96ab4a82389c9fa64", - "cache_configuration": { - "Fn_compute_blocks": 1, - "Bn_compute_blocks": 0, - "max_warmup_steps": 4, - "scm_steps_mask_policy": null, - "residual_diff_threshold": 0.08, - "max_continuous_cached_steps": 3, - "enable_taylorseer": false, - "minimax_h3_sigma_cache": true - }, - "latency_diagnostic_ms": 169512.50694599003, - "clean_timing": false, - "input_checks_passed": 134, - "cleanup_passed": true, - "actual_middle_counts_by_rank": [ - { - "computed": 22, - "predicted": 28, - "history_vetoes": 0 - }, - { - "computed": 22, - "predicted": 28, - "history_vetoes": 0 - }, - { - "computed": 22, - "predicted": 28, - "history_vetoes": 0 - }, - { - "computed": 22, - "predicted": 28, - "history_vetoes": 0 - } - ], - "quality_contract_sha256": "b8fd9e634e6db5ebfbce3842f3360f1b2241a08119953ee71bfc26553dbbafed", - "quality_verdict": "FAIL", - "observed_checks": 39, - "observed_failures": [ - "5s-00:audio_caption_clap_cosine: adherence" - ], - "missing_samples": [ - "5s-01", - "5s-02", - "5s-03", - "5s-04", - "15s-00", - "15s-01", - "15s-02", - "15s-03", - "15s-04" - ], - "audio_caption_clap_cosine": 0.0498393215239048, - "media_sha256": "0d48934b0fec90326ae505deae54ce26a5f03733eaf0bca2e2fb5d13202c518b", - "features_sha256": "ff0b426c3e791ea782bf1d00cf2859f71678fd8c42289533f8f6f2842369133a", - "decision": "Reject5s-00 for frozenadherencefailure. No further predictorgrid. 15s requires independent qualification.", - "performance_acceptance": "PENDING; no clean pairedgain" -} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c11-fp32-5s-first-step.json b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c11-fp32-5s-first-step.json deleted file mode 100644 index a1ab69b0f..000000000 --- a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c11-fp32-5s-first-step.json +++ /dev/null @@ -1,102 +0,0 @@ -{ - "checks": [ - { - "rank": 0, - "point": "initial_noise", - "stream": "video", - "equal": true - }, - { - "rank": 0, - "point": "initial_noise", - "stream": "audio", - "equal": true - }, - { - "rank": 0, - "point": "post_step_000", - "stream": "video", - "equal": true - }, - { - "rank": 0, - "point": "post_step_000", - "stream": "audio", - "equal": true - }, - { - "rank": 1, - "point": "initial_noise", - "stream": "video", - "equal": true - }, - { - "rank": 1, - "point": "initial_noise", - "stream": "audio", - "equal": true - }, - { - "rank": 1, - "point": "post_step_000", - "stream": "video", - "equal": true - }, - { - "rank": 1, - "point": "post_step_000", - "stream": "audio", - "equal": true - }, - { - "rank": 2, - "point": "initial_noise", - "stream": "video", - "equal": true - }, - { - "rank": 2, - "point": "initial_noise", - "stream": "audio", - "equal": true - }, - { - "rank": 2, - "point": "post_step_000", - "stream": "video", - "equal": true - }, - { - "rank": 2, - "point": "post_step_000", - "stream": "audio", - "equal": true - }, - { - "rank": 3, - "point": "initial_noise", - "stream": "video", - "equal": true - }, - { - "rank": 3, - "point": "initial_noise", - "stream": "audio", - "equal": true - }, - { - "rank": 3, - "point": "post_step_000", - "stream": "video", - "equal": true - }, - { - "rank": 3, - "point": "post_step_000", - "stream": "audio", - "equal": true - } - ], - "all_equal": true, - "scope": "initialnoise and fullcompute poststep0 only; not later cached states, fullquality or timing acceptance" -} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c11-fp32-native-execution.json b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c11-fp32-native-execution.json deleted file mode 100644 index 7ec4c89a2..000000000 --- a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c11-fp32-native-execution.json +++ /dev/null @@ -1,76 +0,0 @@ -{ - "attempts": [ - { - "trial": "c11-sigma-fp32-native-tests-b873db4a", - "status": "failed", - "cleanup_passed": true, - "source_unchanged": true, - "files_before": { - "vllm_omni/diffusion/models/minimax_h3/cache_sigma_prediction.py": "fb27fdebdf0cb45e678243c0180d56cf3e7a7470ff424bdf2d3bf144c8e2aee3", - "vllm_omni/diffusion/models/minimax_h3/reference_kv_cache_dit.py": "d3381079ff012f32176a917bb0825d12288c058d3a058f5169ecc97d1e2c675c", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "a649a09e5b59256b5887c12fe508d8e14e3dedd9019bdb82c65db2d59f11de91", - "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "a9524734a4f3410a0c6f766fa64fa9e5bd5a4d41bd7481738e9024cac54ce7f9", - "tests/diffusion/models/minimax_h3/test_minimax_h3_sigma_cache.py": "f48d3180de2b25b8ef03c7ca37eee5ff2adb0243268d18adae17142d5eedcd13" - }, - "native_test_names": [ - "test_native_two_clock_forecast_and_input_immutability", - "test_native_real_cache_manager_computes_then_forecasts" - ], - "elapsed_seconds": 51.38206260098377 - }, - { - "trial": "c11-sigma-fp32-native-tests-cc806d17", - "status": "failed", - "cleanup_passed": true, - "source_unchanged": true, - "files_before": { - "vllm_omni/diffusion/models/minimax_h3/cache_sigma_prediction.py": "fb27fdebdf0cb45e678243c0180d56cf3e7a7470ff424bdf2d3bf144c8e2aee3", - "vllm_omni/diffusion/models/minimax_h3/reference_kv_cache_dit.py": "d3381079ff012f32176a917bb0825d12288c058d3a058f5169ecc97d1e2c675c", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "a649a09e5b59256b5887c12fe508d8e14e3dedd9019bdb82c65db2d59f11de91", - "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "a9524734a4f3410a0c6f766fa64fa9e5bd5a4d41bd7481738e9024cac54ce7f9", - "tests/diffusion/models/minimax_h3/test_minimax_h3_sigma_cache.py": "f52b3050a8dab6141eb9d84abf6f62f9cb1b78c9233588117fe47817c62cc72c" - }, - "native_test_names": [ - "test_native_two_clock_forecast_and_input_immutability" - ], - "elapsed_seconds": 49.97501417802414 - }, - { - "trial": "c11-sigma-fp32-native-tests-9542a13d", - "status": "completed", - "cleanup_passed": true, - "source_unchanged": true, - "files_before": { - "vllm_omni/diffusion/models/minimax_h3/cache_sigma_prediction.py": "fb27fdebdf0cb45e678243c0180d56cf3e7a7470ff424bdf2d3bf144c8e2aee3", - "vllm_omni/diffusion/models/minimax_h3/reference_kv_cache_dit.py": "d3381079ff012f32176a917bb0825d12288c058d3a058f5169ecc97d1e2c675c", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "a649a09e5b59256b5887c12fe508d8e14e3dedd9019bdb82c65db2d59f11de91", - "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "a9524734a4f3410a0c6f766fa64fa9e5bd5a4d41bd7481738e9024cac54ce7f9", - "tests/diffusion/models/minimax_h3/test_minimax_h3_sigma_cache.py": "16d1e9329c0088eabf4bb17214fc364ec0c6176d92f871786456a2b92aaa418f" - }, - "native_test_names": [ - "test_native_two_clock_forecast_and_input_immutability", - "test_native_real_cache_manager_computes_then_forecasts", - "test_native_bf16_blocks_use_fp32_anchors_and_keep_output_dtype" - ], - "elapsed_seconds": 50.17931097600376 - } - ], - "tests_passed_after_fixture_correction": 23, - "corrections": [ - { - "failed_run": "c11-sigma-fp32-native-tests-b873db4a", - "old_test_sha256": "f48d3180de2b25b8ef03c7ca37eee5ff2adb0243268d18adae17142d5eedcd13", - "new_test_sha256": "f52b3050a8dab6141eb9d84abf6f62f9cb1b78c9233588117fe47817c62cc72c", - "production_unchanged": true, - "reason": "Old fixture ignored BF16 anchor quantization. Replace analytic unquantized reference with independent FP64 Vandermonde solve through actual anchors; exact full-compute checks and stricter forecast bound. Frozen quality evaluator unchanged." - }, - { - "failed_run": "c11-sigma-fp32-native-tests-cc806d17", - "old_test_sha256": "f52b3050a8dab6141eb9d84abf6f62f9cb1b78c9233588117fe47817c62cc72c", - "new_test_sha256": "16d1e9329c0088eabf4bb17214fc364ec0c6176d92f871786456a2b92aaa418f", - "production_unchanged": true, - "reason": "Move actual NPU anchor values to CPU before .double(): Ascend warns and downgrades double operations on device. CPU FP64 solve now receives matching Double A/B. No numerical/quality threshold change." - } - ], - "scope": "Two failed test-reference attempts retained, then23passed incl3executed NPUtests. Productionbytes unchanged across attempts. Frozen modelquality thresholds/scorers unchanged." -} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c11-full-compute-first-step.json b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c11-full-compute-first-step.json deleted file mode 100644 index e3be0bcb7..000000000 --- a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c11-full-compute-first-step.json +++ /dev/null @@ -1,102 +0,0 @@ -{ - "checks": [ - { - "rank": 0, - "point": "initial_noise", - "stream": "video", - "equal": true - }, - { - "rank": 0, - "point": "initial_noise", - "stream": "audio", - "equal": true - }, - { - "rank": 0, - "point": "post_step_000", - "stream": "video", - "equal": true - }, - { - "rank": 0, - "point": "post_step_000", - "stream": "audio", - "equal": true - }, - { - "rank": 1, - "point": "initial_noise", - "stream": "video", - "equal": true - }, - { - "rank": 1, - "point": "initial_noise", - "stream": "audio", - "equal": true - }, - { - "rank": 1, - "point": "post_step_000", - "stream": "video", - "equal": true - }, - { - "rank": 1, - "point": "post_step_000", - "stream": "audio", - "equal": true - }, - { - "rank": 2, - "point": "initial_noise", - "stream": "video", - "equal": true - }, - { - "rank": 2, - "point": "initial_noise", - "stream": "audio", - "equal": true - }, - { - "rank": 2, - "point": "post_step_000", - "stream": "video", - "equal": true - }, - { - "rank": 2, - "point": "post_step_000", - "stream": "audio", - "equal": true - }, - { - "rank": 3, - "point": "initial_noise", - "stream": "video", - "equal": true - }, - { - "rank": 3, - "point": "initial_noise", - "stream": "audio", - "equal": true - }, - { - "rank": 3, - "point": "post_step_000", - "stream": "video", - "equal": true - }, - { - "rank": 3, - "point": "post_step_000", - "stream": "audio", - "equal": true - } - ], - "all_equal": true, - "scope": "initialnoise and full-compute poststep0 only; not bit-exact later cached steps or quality acceptance" -} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c11-native-execution.json b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c11-native-execution.json deleted file mode 100644 index f2088a524..000000000 --- a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c11-native-execution.json +++ /dev/null @@ -1,21 +0,0 @@ -{ - "trial": "c11-sigma-native-tests-e4a95751", - "status": "completed", - "source_base_head": "922f1e9f1cb85b9f2b66d86e1eb500e6872b2dcb", - "source_commit": "61b8efa06e88372bc5ab87b9e2668a133eaac282", - "files_before": { - "vllm_omni/diffusion/models/minimax_h3/cache_sigma_prediction.py": "ac29ef338d569529e40f59210c89a57dd2e07c510b5e2952c0da451f183da52a", - "vllm_omni/diffusion/models/minimax_h3/reference_kv_cache_dit.py": "d3381079ff012f32176a917bb0825d12288c058d3a058f5169ecc97d1e2c675c", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "a649a09e5b59256b5887c12fe508d8e14e3dedd9019bdb82c65db2d59f11de91", - "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "a9524734a4f3410a0c6f766fa64fa9e5bd5a4d41bd7481738e9024cac54ce7f9", - "tests/diffusion/models/minimax_h3/test_minimax_h3_sigma_cache.py": "9ba094c0bc0686e8d044ebe064beaa2606a6cad496449e63ec282411bb2e727a" - }, - "native_test_names": [ - "test_native_two_clock_forecast_and_input_immutability", - "test_native_real_cache_manager_computes_then_forecasts" - ], - "cleanup_passed": true, - "source_unchanged": true, - "elapsed_seconds": 49.4796573020285, - "scope": "20 focused tests including two executed native Ascend tests; not full-model quality or speed qualification" -} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c11-original-5s-00-audit.json b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c11-original-5s-00-audit.json deleted file mode 100644 index 44f81660d..000000000 --- a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c11-original-5s-00-audit.json +++ /dev/null @@ -1,58 +0,0 @@ -{ - "trial": "c11-modality-sigma-quality-5s-00-eb726462", - "source_commit": "61b8efa06e88372bc5ab87b9e2668a133eaac282", - "cache_configuration": { - "Fn_compute_blocks": 1, - "Bn_compute_blocks": 0, - "max_warmup_steps": 4, - "scm_steps_mask_policy": null, - "residual_diff_threshold": 0.08, - "max_continuous_cached_steps": 3, - "enable_taylorseer": false, - "minimax_h3_sigma_cache": true - }, - "latency_diagnostic_ms": 171179.77748898556, - "clean_timing": false, - "input_checks_passed": 134, - "cleanup_passed": true, - "actual_middle_counts_by_rank": [ - { - "computed": 22, - "predicted": 28, - "history_vetoes": 0 - }, - { - "computed": 22, - "predicted": 28, - "history_vetoes": 0 - }, - { - "computed": 22, - "predicted": 28, - "history_vetoes": 0 - }, - { - "computed": 22, - "predicted": 28, - "history_vetoes": 0 - } - ], - "observed_quality_checks": 39, - "observed_failures": [ - "5s-00:audio_caption_clap_cosine: adherence" - ], - "quality_verdict": "FAIL", - "missing_samples": [ - "5s-01", - "5s-02", - "5s-03", - "5s-04", - "15s-00", - "15s-01", - "15s-02", - "15s-03", - "15s-04" - ], - "decision": "Reject original Fn1Bn0 for5s-00; frozen adherence failure is retained; Bn1 is a separate configuration.", - "performance_acceptance": "PENDING; diagnostic HTTP is not a clean paired gain" -} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c12-through-c15-execution-checkpoint.json b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c12-through-c15-execution-checkpoint.json deleted file mode 100644 index 4b1b3752b..000000000 --- a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c12-through-c15-execution-checkpoint.json +++ /dev/null @@ -1,509 +0,0 @@ -{ - "campaign_verdict": "INCONCLUSIVE", - "performance_acceptance": "No newly accepted gain", - "runs": [ - { - "trial": "c12-alignment-quality-5s-00-9bf5c373", - "model_controller": { - "status": "completed", - "returncode": 0, - "elapsed_seconds": 471.80311120999977 - }, - "model_diagnostic": { - "source_head": "a29648e5acd3077b7b22739bc9522017782fe1ac", - "samples_ms": [ - 318280.59680899605 - ], - "latency_median_ms": 318280.59680899605, - "scope": "http_complete_mp4", - "warmup": 0, - "repetitions": 1, - "kind": "quality", - "diagnostic": "audit", - "clean_timing": false, - "cleanup_passed": true - }, - "actual_counters": { - "request-000\\rank-0\\alignment_cache_stats.json": { - "request_counters": { - "computed": 50, - "predicted": 0, - "vetoes": 45 - }, - "scope": "actual state-aligned compute/forecast/veto counts; not quality acceptance" - }, - "request-000\\rank-1\\alignment_cache_stats.json": { - "request_counters": { - "computed": 50, - "predicted": 0, - "vetoes": 45 - }, - "scope": "actual state-aligned compute/forecast/veto counts; not quality acceptance" - }, - "request-000\\rank-2\\alignment_cache_stats.json": { - "request_counters": { - "computed": 50, - "predicted": 0, - "vetoes": 45 - }, - "scope": "actual state-aligned compute/forecast/veto counts; not quality acceptance" - }, - "request-000\\rank-3\\alignment_cache_stats.json": { - "request_counters": { - "computed": 50, - "predicted": 0, - "vetoes": 45 - }, - "scope": "actual state-aligned compute/forecast/veto counts; not quality acceptance" - }, - "request-000\\rank-0\\attention_path_counts.json": { - "_prefix_bnsd": 2500 - }, - "request-000\\rank-1\\attention_path_counts.json": { - "_prefix_bnsd": 2500 - }, - "request-000\\rank-2\\attention_path_counts.json": { - "_prefix_bnsd": 2500 - }, - "request-000\\rank-3\\attention_path_counts.json": { - "_prefix_bnsd": 2500 - } - } - }, - { - "trial": "c12-state-alignment-native-tests-0dc5859b", - "cleanup_passed": true, - "source_unchanged": true, - "native_test_executed": true, - "files_before": { - "vllm_omni/diffusion/models/minimax_h3/cache_state_alignment.py": "3340ec21a77aef78531c601be241bcfb1e44107c795e8c8649907d1a18ed377f", - "vllm_omni/diffusion/models/minimax_h3/reference_kv_cache_dit.py": "f4cd0e9aa9e824c92ec1acd02c57a8f7d8684c35d4edf92ffdfae3345ff44e99", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "436ec52cd6329cbc64e91d94b450bfa6f773a9a953b8f974db73a5eb986a98bb", - "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "4dec8f02a04bfe7e8d2ada24fc6d5ae40a0058671d8515c8a266ecf88824b7b0", - "tests/diffusion/models/minimax_h3/test_minimax_h3_state_cache.py": "89cf57482b35d8d8ad8d4978aec87e9ee3ef45330e5b911161adef4379950a65" - }, - "scope": "focused tests only; no full-model quality or performance acceptance", - "status": "completed", - "returncode": 0, - "elapsed_seconds": 82.06077561300481 - }, - { - "trial": "c13-baseline-cuda-staging-e22d8651", - "cleanup_passed": true, - "source_unchanged": true, - "native_test_executed": false, - "files_before": { - "vllm_omni/diffusion/offloader/sequential_backend.py": "d542935b656a0d9dced78de8fb086b8e1c5aade26fac6cee5316b32a49f0d569", - "tests/diffusion/offloader/test_sequential_backend.py": "dcf1d64e65bed572d65b095c1350950f55d7e8d9ddf8c7dfd72d0ea6e721685a" - }, - "scope": "Reproduce stock CUDA-graph staging expectation on unchanged migration owning source in the same Ascend test environment", - "status": "failed", - "returncode": 1, - "elapsed_seconds": 50.234599059971515 - }, - { - "trial": "c13-c1-dlo-5s-audit-cead10d1", - "model_controller": { - "status": "failed", - "returncode": 2, - "elapsed_seconds": 0.11969929095357656 - } - }, - { - "trial": "c13-c1-dlo-5s-audit-v2-b0b86c6d", - "model_controller": { - "status": "completed", - "returncode": 0, - "elapsed_seconds": 749.505351218977 - }, - "model_diagnostic": { - "source_head": "e89b1d40d37f7f72767892a56fa7980ea572ba8e", - "samples_ms": [ - 315593.86175096733, - 284456.94282796467 - ], - "latency_median_ms": 300025.402289466, - "scope": "http_complete_mp4", - "warmup": 0, - "repetitions": 2, - "kind": "diagnostic", - "diagnostic": "audit", - "clean_timing": false, - "cleanup_passed": true - }, - "actual_counters": { - "request-000\\rank-0\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-000\\rank-1\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-000\\rank-2\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-000\\rank-3\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-001\\rank-0\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-001\\rank-1\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-001\\rank-2\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-001\\rank-3\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - } - } - }, - { - "trial": "c13-c1-dlo-5s-calibration-c98129d8", - "model_controller": { - "status": "completed", - "returncode": 0, - "elapsed_seconds": 768.8735517460154 - }, - "model_diagnostic": { - "source_head": "e89b1d40d37f7f72767892a56fa7980ea572ba8e", - "samples_ms": [ - 317177.1984040388, - 285534.089674009 - ], - "latency_median_ms": 301355.6440390239, - "scope": "http_complete_mp4", - "warmup": 0, - "repetitions": 2, - "kind": "calibration", - "diagnostic": "audit", - "clean_timing": false, - "cleanup_passed": true - }, - "actual_counters": { - "request-000\\rank-0\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-000\\rank-1\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-000\\rank-2\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-000\\rank-3\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-001\\rank-0\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-001\\rank-1\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-001\\rank-2\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-001\\rank-3\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - } - } - }, - { - "trial": "c13-dlo-gate-tests-bc92a71d", - "cleanup_passed": true, - "source_unchanged": true, - "native_test_executed": false, - "files_before": { - "tests/diffusion/distributed/test_collective_gate.py": "2b33f73dff9c79118cc2b1bd4d50e3542ebc45b7bf029b38c17abbef37e4b917", - "vllm_omni/config/omni_config.py": "6894120a330474f718d43bf2bf7928892eb996152839ce6469c34d18a7df09d1", - "vllm_omni/config/stage_config.py": "0752e8cc1c6f1021f737830d08663cfc118923a83fd909c607f9ec3ab2a3d2fb", - "vllm_omni/diffusion/attention/layer.py": "76c5fce3dabd23e39a8890316e1691c03bc203066cb60af5611ec4790eefc632", - "vllm_omni/diffusion/data.py": "5db83d5c71cb27cc7810d35fcfc891da80f0b003adbb16bdecc3816a63305e11", - "vllm_omni/diffusion/distributed/collective_gate.py": "01c6317cfa615840681b8bb8b8513fa3736d26e845a24ad9e1ab7f87489b962f", - "vllm_omni/diffusion/offloader/base.py": "533c793119e4cdc14a2ea3c6cb5bf97ffede673befeab3f0fb8dbbf51e655379", - "vllm_omni/diffusion/offloader/distributed_layerwise_backend.py": "d441481513aba9e90a91265f4bba3f53f8766197be8c1302c3c714e690d5ea76", - "vllm_omni/engine/arg_utils.py": "52569e3688cc708729a48ca0a2c875c28193733c2f934f6ff89f6c815e4009c7", - "vllm_omni/entrypoints/cli/serve.py": "350461dfca565bc0b07087ac7af1e27b0e247f5eac717e350b637e37f62dd088", - "tests/diffusion/offloader/helpers.py": "729c73abba3b9a12911282d4bcd3cf8bfa148fbcbfdc2e2b7c86000e387fa687" - }, - "scope": "CPU mock barrier/chunk tests; no actual NPU collective or full-model quality/performance acceptance", - "status": "completed", - "returncode": 0, - "elapsed_seconds": 47.3752507569734 - }, - { - "trial": "c13-dlo-regression-tests-95e9ed2a", - "cleanup_passed": true, - "source_unchanged": true, - "native_test_executed": false, - "files_before": { - "tests/diffusion/distributed/test_collective_gate.py": "2b33f73dff9c79118cc2b1bd4d50e3542ebc45b7bf029b38c17abbef37e4b917", - "vllm_omni/config/omni_config.py": "6894120a330474f718d43bf2bf7928892eb996152839ce6469c34d18a7df09d1", - "vllm_omni/config/stage_config.py": "0752e8cc1c6f1021f737830d08663cfc118923a83fd909c607f9ec3ab2a3d2fb", - "vllm_omni/diffusion/attention/layer.py": "76c5fce3dabd23e39a8890316e1691c03bc203066cb60af5611ec4790eefc632", - "vllm_omni/diffusion/data.py": "5db83d5c71cb27cc7810d35fcfc891da80f0b003adbb16bdecc3816a63305e11", - "vllm_omni/diffusion/distributed/collective_gate.py": "01c6317cfa615840681b8bb8b8513fa3736d26e845a24ad9e1ab7f87489b962f", - "vllm_omni/diffusion/offloader/base.py": "533c793119e4cdc14a2ea3c6cb5bf97ffede673befeab3f0fb8dbbf51e655379", - "vllm_omni/diffusion/offloader/distributed_layerwise_backend.py": "d441481513aba9e90a91265f4bba3f53f8766197be8c1302c3c714e690d5ea76", - "vllm_omni/engine/arg_utils.py": "52569e3688cc708729a48ca0a2c875c28193733c2f934f6ff89f6c815e4009c7", - "vllm_omni/entrypoints/cli/serve.py": "350461dfca565bc0b07087ac7af1e27b0e247f5eac717e350b637e37f62dd088", - "tests/diffusion/offloader/helpers.py": "729c73abba3b9a12911282d4bcd3cf8bfa148fbcbfdc2e2b7c86000e387fa687" - }, - "scope": "CPU mock barrier/chunk tests; no actual NPU collective or full-model quality/performance acceptance", - "status": "failed", - "returncode": 1, - "elapsed_seconds": 62.34616496198578 - }, - { - "trial": "c13-dlo-regression-v2-12c217d5", - "cleanup_passed": true, - "source_unchanged": true, - "native_test_executed": false, - "files_before": { - "tests/diffusion/distributed/test_collective_gate.py": "2b33f73dff9c79118cc2b1bd4d50e3542ebc45b7bf029b38c17abbef37e4b917", - "vllm_omni/config/omni_config.py": "6894120a330474f718d43bf2bf7928892eb996152839ce6469c34d18a7df09d1", - "vllm_omni/config/stage_config.py": "0752e8cc1c6f1021f737830d08663cfc118923a83fd909c607f9ec3ab2a3d2fb", - "vllm_omni/diffusion/attention/layer.py": "76c5fce3dabd23e39a8890316e1691c03bc203066cb60af5611ec4790eefc632", - "vllm_omni/diffusion/data.py": "5db83d5c71cb27cc7810d35fcfc891da80f0b003adbb16bdecc3816a63305e11", - "vllm_omni/diffusion/distributed/collective_gate.py": "01c6317cfa615840681b8bb8b8513fa3736d26e845a24ad9e1ab7f87489b962f", - "vllm_omni/diffusion/offloader/base.py": "533c793119e4cdc14a2ea3c6cb5bf97ffede673befeab3f0fb8dbbf51e655379", - "vllm_omni/diffusion/offloader/distributed_layerwise_backend.py": "d441481513aba9e90a91265f4bba3f53f8766197be8c1302c3c714e690d5ea76", - "vllm_omni/engine/arg_utils.py": "52569e3688cc708729a48ca0a2c875c28193733c2f934f6ff89f6c815e4009c7", - "vllm_omni/entrypoints/cli/serve.py": "350461dfca565bc0b07087ac7af1e27b0e247f5eac717e350b637e37f62dd088", - "tests/diffusion/offloader/helpers.py": "729c73abba3b9a12911282d4bcd3cf8bfa148fbcbfdc2e2b7c86000e387fa687" - }, - "scope": "Expanded offloader tests in Ascend environment; some stock accelerator fixtures use NPU, not a full model test", - "status": "failed", - "returncode": 1, - "elapsed_seconds": 65.14432259398745 - }, - { - "trial": "c14-int8-prerequisites-fdd6633b", - "cleanup_passed": true, - "source_unchanged": true, - "helpers_unchanged": true, - "scope": "Bounded native prerequisites only; a failed probe is retained as negative evidence.", - "status": "failed", - "returncode": 1, - "elapsed_seconds": 20.795623040001374, - "native_prerequisite": { - "verdict": "FAIL", - "scope": "TorchNPU block-INT8 attention prerequisites only", - "phase": "dynamic_block_int8_quant_query", - "versions": { - "torch": "2.10.0+cpu", - "torch_npu": "2.10.0.post4", - "mindiesd": "3.1.0" - }, - "apis": { - "float8_e4m3fn": true, - "npu_dynamic_block_quant": true, - "npu_fused_infer_attention_score_v2": true - }, - "device_name": "Ascend910_9392", - "input": { - "shape": [ - 1, - 14, - 256, - 128 - ], - "dtype": "torch.bfloat16", - "visible_devices": "2" - }, - "error_type": "RuntimeError", - "probe_sha256": "fab18252f860231f9b7fcaa54587afbaa96fce15987fd551922246c8964f7217", - "error": "npu_dynamic_block_quant:../third_party/op-plugin/op_plugin/ops/opapi/DynamicBlockQuantNpuOpApi.cpp:61 NPU function error: call aclnnDynamicBlockQuantV2 failed, error code is 561002\n[ERROR] 2026-11-02-02:00:57 (PID:2662830, Device:0, RankID:-1) ERR00100 PTA call acl api failed.\n[PID: 2662830] 2026-11-02-02:00:57.039.986 Invalid_Argument(EZ0024): Parameter row_block_size of DynamicBlockQuant has incorrect value 128. It should be 1.\n Solution: Check whether the parameter value is correct.\nTraceBack (most recent call last):\n The attr get failed.[FUNC:CheckParams][FILE:dynamic_block_quant_i8_tiling.cpp][LINE:78]\n The params check failed.[FUNC:Tiling4DynamicBlockQuant][FILE:dynamic_block_quant_tiling.cpp][LINE:564]\n Failed to execute operator DynamicBlockQuant_0. Reason: Failed to execute tiling function.\n Check NnopbaseExecutorDoTiling(executor) failed\n Check NnopbaseExecutorTilingAndUpdateBinInfo(executor) failed\n Check NnopbaseExecutorMatchCache(executor) failed\n Check NnopbaseRunForWorkspace(*executor, workspaceSize) failed\n" - } - }, - { - "trial": "c14-quant-prerequisites-358b9b28", - "cleanup_passed": true, - "source_unchanged": true, - "helpers_unchanged": true, - "scope": "Bounded native prerequisites only; a failed probe is retained as negative evidence.", - "status": "failed", - "returncode": 1, - "elapsed_seconds": 20.143024684977718, - "native_prerequisite": { - "verdict": "FAIL", - "scope": "TorchNPU FP8 prerequisites only", - "phase": "dynamic_block_quant_query", - "versions": { - "torch": "2.10.0+cpu", - "torch_npu": "2.10.0.post4", - "mindiesd": "3.1.0" - }, - "apis": { - "float8_e4m3fn": true, - "npu_dynamic_block_quant": true, - "npu_fused_infer_attention_score_v2": true - }, - "device_name": "Ascend910_9392", - "input": { - "shape": [ - 1, - 14, - 256, - 128 - ], - "dtype": "torch.bfloat16", - "visible_devices": "2" - }, - "error_type": "RuntimeError", - "probe_sha256": "df6465956e6f73bbca140fb9880057e30869fd4b7976b2dc091f71a575dd1096", - "error": "npu_dynamic_block_quant:../third_party/op-plugin/op_plugin/ops/opapi/DynamicBlockQuantNpuOpApi.cpp:61 NPU function error: call aclnnDynamicBlockQuantV2 failed, error code is 161002\n[ERROR] 2026-11-02-01:45:24 (PID:2491012, Device:0, RankID:-1) ERR00100 PTA call acl api failed.\n[PID: 2491012] 2026-11-02-01:45:24.732.576 Execution_Error(EZ1009): Failed to execute operator DynamicBlockQuant_0. Reason: The dtype or format of the actual input or output parameter of the operator is inconsistent with that defined in the operator prototype OpDef, details of parameter are x(DT_BFLOAT16, ND), yOut(DT_FLOAT8_E4M3FN, ND), scaleOut(DT_FLOAT, ND).\nTraceBack (most recent call last):\n Supported dtype and format list is:\n SupportInfo[0]: x(DT_FLOAT16, ND), yOut(DT_INT8, ND), scaleOut(DT_FLOAT, ND)\n SupportInfo[1]: x(DT_BFLOAT16, ND), yOut(DT_INT8, ND), scaleOut(DT_FLOAT, ND)\n Check NnopbaseExecutorTilingAndUpdateBinInfo(executor) failed\n Check NnopbaseExecutorMatchCache(executor) failed\n Check NnopbaseRunForWorkspace(*executor, workspaceSize) failed\n" - } - }, - { - "trial": "c15-video-row-native-066a9a40", - "cleanup_passed": true, - "source_unchanged": true, - "required_native_test_executed": true, - "files_before": { - "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "e35b551b8fc0ec3b0ba1384b4df96a8daa0e5179c5ed39d641125310b2ee957f", - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ca87df1936fd2694fcc768079e9e8f926b6d45969ddc61ebf353f5e726d70b8b", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "1258655fcdca4692c2306100846761a2b0b149bbcaa5cde48a9abaa559913bc3", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "a8257f98c2e30c7bc7978bf645966d2eaa98cd49a42a8a923c062b5808dcdcdf", - "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "6f890fcf84047bc5bc084f7b756a2b02e72573888a3bcc12ba7d0211e1bb62ef", - "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "d2a5bed10620a337ff9aaf99cf4a3740d94a8962ef903bf2da75d65e3b50e21c" - }, - "scope": "Focused state/block tests and native four-rank subset attention; no full-model quality/performance acceptance", - "status": "failed", - "returncode": 1, - "elapsed_seconds": 82.27795292000519 - }, - { - "trial": "c15-video-row-native-3a9721a6", - "cleanup_passed": true, - "source_unchanged": true, - "required_native_test_executed": true, - "files_before": { - "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "cf9fb13c5f24c7595d62d13cae6e283cafb2178612b20028d35357f0cafaac83", - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ca87df1936fd2694fcc768079e9e8f926b6d45969ddc61ebf353f5e726d70b8b", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "1258655fcdca4692c2306100846761a2b0b149bbcaa5cde48a9abaa559913bc3", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "a8257f98c2e30c7bc7978bf645966d2eaa98cd49a42a8a923c062b5808dcdcdf", - "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "c5fab405ee82bc553e9a902fa27e6dfbc5213ec18311e17b6415396a69b50f43", - "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "8ec1b3ae4cd252f5d95ab23e0232b983900939e8cb15d1e64f4740f7204abf39", - "vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py": "e8e2e74fb8edb05e02d3e2328ea1f3ad5ddcfd7a10ed09c5cf3911b1d063e849" - }, - "scope": "Focused state/block tests and native four-rank subset attention; no full-model quality/performance acceptance", - "status": "completed", - "returncode": 0, - "elapsed_seconds": 81.91829407500336 - }, - { - "trial": "c15-video-row-native-a2a224de", - "cleanup_passed": true, - "source_unchanged": true, - "required_native_test_executed": true, - "files_before": { - "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "cf9fb13c5f24c7595d62d13cae6e283cafb2178612b20028d35357f0cafaac83", - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ca87df1936fd2694fcc768079e9e8f926b6d45969ddc61ebf353f5e726d70b8b", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "1258655fcdca4692c2306100846761a2b0b149bbcaa5cde48a9abaa559913bc3", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "a8257f98c2e30c7bc7978bf645966d2eaa98cd49a42a8a923c062b5808dcdcdf", - "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "118afcc28379dd73feec763fbb9cfc9f17c6a208afe5969a85b3c969b8059552", - "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "875d0e70621fc76906be130519d7bb1242de5dd58159e31025e26da37e60d98c" - }, - "scope": "Focused state/block tests and native four-rank subset attention; no full-model quality/performance acceptance", - "status": "failed", - "returncode": 1, - "elapsed_seconds": 82.86056435096543 - }, - { - "trial": "c15-video-row-native-bb5084cc", - "cleanup_passed": true, - "source_unchanged": true, - "required_native_test_executed": true, - "files_before": { - "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "cf9fb13c5f24c7595d62d13cae6e283cafb2178612b20028d35357f0cafaac83", - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ca87df1936fd2694fcc768079e9e8f926b6d45969ddc61ebf353f5e726d70b8b", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "1258655fcdca4692c2306100846761a2b0b149bbcaa5cde48a9abaa559913bc3", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "a8257f98c2e30c7bc7978bf645966d2eaa98cd49a42a8a923c062b5808dcdcdf", - "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "118afcc28379dd73feec763fbb9cfc9f17c6a208afe5969a85b3c969b8059552", - "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "fa1ac8fd375d754503b3507fae006381138d8875afdad2418f1ea5ae338b9c26" - }, - "scope": "Focused state/block tests and native four-rank subset attention; no full-model quality/performance acceptance", - "status": "completed", - "returncode": 0, - "elapsed_seconds": 81.50971999001922 - }, - { - "trial": "c15-video-row-quality-5s-00-710315e0", - "model_controller": { - "status": "failed", - "returncode": 1, - "elapsed_seconds": 153.44603093498154 - } - } - ], - "decisions": [ - "C12 fixed profile stopped: zero actual predictions.", - "C14 pinned block-FP8 and block-INT8 prerequisites failed; per-token INT8 is not excluded.", - "C13 numerical qualification and clean performance are separate.", - "C15 is a separate lossy fixed profile; native fixtures do not establish model quality." - ] -} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c12-through-c17-execution-checkpoint.json b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c12-through-c17-execution-checkpoint.json deleted file mode 100644 index 5ed1d87ab..000000000 --- a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c12-through-c17-execution-checkpoint.json +++ /dev/null @@ -1,931 +0,0 @@ -{ - "campaign_verdict": "INCONCLUSIVE", - "performance_acceptance": "No newly accepted gain", - "runs": [ - { - "trial": "c12-alignment-quality-5s-00-9bf5c373", - "model_controller": { - "status": "completed", - "returncode": 0, - "elapsed_seconds": 471.80311120999977 - }, - "model_diagnostic": { - "source_head": "a29648e5acd3077b7b22739bc9522017782fe1ac", - "samples_ms": [ - 318280.59680899605 - ], - "latency_median_ms": 318280.59680899605, - "scope": "http_complete_mp4", - "warmup": 0, - "repetitions": 1, - "kind": "quality", - "diagnostic": "audit", - "clean_timing": false, - "cleanup_passed": true - }, - "actual_counters": { - "request-000\\rank-0\\alignment_cache_stats.json": { - "request_counters": { - "computed": 50, - "predicted": 0, - "vetoes": 45 - }, - "scope": "actual state-aligned compute/forecast/veto counts; not quality acceptance" - }, - "request-000\\rank-1\\alignment_cache_stats.json": { - "request_counters": { - "computed": 50, - "predicted": 0, - "vetoes": 45 - }, - "scope": "actual state-aligned compute/forecast/veto counts; not quality acceptance" - }, - "request-000\\rank-2\\alignment_cache_stats.json": { - "request_counters": { - "computed": 50, - "predicted": 0, - "vetoes": 45 - }, - "scope": "actual state-aligned compute/forecast/veto counts; not quality acceptance" - }, - "request-000\\rank-3\\alignment_cache_stats.json": { - "request_counters": { - "computed": 50, - "predicted": 0, - "vetoes": 45 - }, - "scope": "actual state-aligned compute/forecast/veto counts; not quality acceptance" - }, - "request-000\\rank-0\\attention_path_counts.json": { - "_prefix_bnsd": 2500 - }, - "request-000\\rank-1\\attention_path_counts.json": { - "_prefix_bnsd": 2500 - }, - "request-000\\rank-2\\attention_path_counts.json": { - "_prefix_bnsd": 2500 - }, - "request-000\\rank-3\\attention_path_counts.json": { - "_prefix_bnsd": 2500 - } - } - }, - { - "trial": "c12-state-alignment-native-tests-0dc5859b", - "cleanup_passed": true, - "source_unchanged": true, - "native_test_executed": true, - "files_before": { - "vllm_omni/diffusion/models/minimax_h3/cache_state_alignment.py": "3340ec21a77aef78531c601be241bcfb1e44107c795e8c8649907d1a18ed377f", - "vllm_omni/diffusion/models/minimax_h3/reference_kv_cache_dit.py": "f4cd0e9aa9e824c92ec1acd02c57a8f7d8684c35d4edf92ffdfae3345ff44e99", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "436ec52cd6329cbc64e91d94b450bfa6f773a9a953b8f974db73a5eb986a98bb", - "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "4dec8f02a04bfe7e8d2ada24fc6d5ae40a0058671d8515c8a266ecf88824b7b0", - "tests/diffusion/models/minimax_h3/test_minimax_h3_state_cache.py": "89cf57482b35d8d8ad8d4978aec87e9ee3ef45330e5b911161adef4379950a65" - }, - "scope": "focused tests only; no full-model quality or performance acceptance", - "status": "completed", - "returncode": 0, - "elapsed_seconds": 82.06077561300481 - }, - { - "trial": "c13-baseline-cuda-staging-e22d8651", - "cleanup_passed": true, - "source_unchanged": true, - "native_test_executed": false, - "files_before": { - "vllm_omni/diffusion/offloader/sequential_backend.py": "d542935b656a0d9dced78de8fb086b8e1c5aade26fac6cee5316b32a49f0d569", - "tests/diffusion/offloader/test_sequential_backend.py": "dcf1d64e65bed572d65b095c1350950f55d7e8d9ddf8c7dfd72d0ea6e721685a" - }, - "scope": "Reproduce stock CUDA-graph staging expectation on unchanged migration owning source in the same Ascend test environment", - "status": "failed", - "returncode": 1, - "elapsed_seconds": 50.234599059971515 - }, - { - "trial": "c13-c1-dlo-5s-audit-cead10d1", - "model_controller": { - "status": "failed", - "returncode": 2, - "elapsed_seconds": 0.11969929095357656 - } - }, - { - "trial": "c13-c1-dlo-5s-audit-v2-b0b86c6d", - "model_controller": { - "status": "completed", - "returncode": 0, - "elapsed_seconds": 749.505351218977 - }, - "model_diagnostic": { - "source_head": "e89b1d40d37f7f72767892a56fa7980ea572ba8e", - "samples_ms": [ - 315593.86175096733, - 284456.94282796467 - ], - "latency_median_ms": 300025.402289466, - "scope": "http_complete_mp4", - "warmup": 0, - "repetitions": 2, - "kind": "diagnostic", - "diagnostic": "audit", - "clean_timing": false, - "cleanup_passed": true - }, - "actual_counters": { - "request-000\\rank-0\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-000\\rank-1\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-000\\rank-2\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-000\\rank-3\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-001\\rank-0\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-001\\rank-1\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-001\\rank-2\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-001\\rank-3\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - } - } - }, - { - "trial": "c13-c1-dlo-5s-calibration-c98129d8", - "model_controller": { - "status": "completed", - "returncode": 0, - "elapsed_seconds": 768.8735517460154 - }, - "model_diagnostic": { - "source_head": "e89b1d40d37f7f72767892a56fa7980ea572ba8e", - "samples_ms": [ - 317177.1984040388, - 285534.089674009 - ], - "latency_median_ms": 301355.6440390239, - "scope": "http_complete_mp4", - "warmup": 0, - "repetitions": 2, - "kind": "calibration", - "diagnostic": "audit", - "clean_timing": false, - "cleanup_passed": true - }, - "actual_counters": { - "request-000\\rank-0\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-000\\rank-1\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-000\\rank-2\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-000\\rank-3\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-001\\rank-0\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-001\\rank-1\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-001\\rank-2\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-001\\rank-3\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - } - } - }, - { - "trial": "c13-dlo-gate-tests-bc92a71d", - "cleanup_passed": true, - "source_unchanged": true, - "native_test_executed": false, - "files_before": { - "tests/diffusion/distributed/test_collective_gate.py": "2b33f73dff9c79118cc2b1bd4d50e3542ebc45b7bf029b38c17abbef37e4b917", - "vllm_omni/config/omni_config.py": "6894120a330474f718d43bf2bf7928892eb996152839ce6469c34d18a7df09d1", - "vllm_omni/config/stage_config.py": "0752e8cc1c6f1021f737830d08663cfc118923a83fd909c607f9ec3ab2a3d2fb", - "vllm_omni/diffusion/attention/layer.py": "76c5fce3dabd23e39a8890316e1691c03bc203066cb60af5611ec4790eefc632", - "vllm_omni/diffusion/data.py": "5db83d5c71cb27cc7810d35fcfc891da80f0b003adbb16bdecc3816a63305e11", - "vllm_omni/diffusion/distributed/collective_gate.py": "01c6317cfa615840681b8bb8b8513fa3736d26e845a24ad9e1ab7f87489b962f", - "vllm_omni/diffusion/offloader/base.py": "533c793119e4cdc14a2ea3c6cb5bf97ffede673befeab3f0fb8dbbf51e655379", - "vllm_omni/diffusion/offloader/distributed_layerwise_backend.py": "d441481513aba9e90a91265f4bba3f53f8766197be8c1302c3c714e690d5ea76", - "vllm_omni/engine/arg_utils.py": "52569e3688cc708729a48ca0a2c875c28193733c2f934f6ff89f6c815e4009c7", - "vllm_omni/entrypoints/cli/serve.py": "350461dfca565bc0b07087ac7af1e27b0e247f5eac717e350b637e37f62dd088", - "tests/diffusion/offloader/helpers.py": "729c73abba3b9a12911282d4bcd3cf8bfa148fbcbfdc2e2b7c86000e387fa687" - }, - "scope": "CPU mock barrier/chunk tests; no actual NPU collective or full-model quality/performance acceptance", - "status": "completed", - "returncode": 0, - "elapsed_seconds": 47.3752507569734 - }, - { - "trial": "c13-dlo-regression-tests-95e9ed2a", - "cleanup_passed": true, - "source_unchanged": true, - "native_test_executed": false, - "files_before": { - "tests/diffusion/distributed/test_collective_gate.py": "2b33f73dff9c79118cc2b1bd4d50e3542ebc45b7bf029b38c17abbef37e4b917", - "vllm_omni/config/omni_config.py": "6894120a330474f718d43bf2bf7928892eb996152839ce6469c34d18a7df09d1", - "vllm_omni/config/stage_config.py": "0752e8cc1c6f1021f737830d08663cfc118923a83fd909c607f9ec3ab2a3d2fb", - "vllm_omni/diffusion/attention/layer.py": "76c5fce3dabd23e39a8890316e1691c03bc203066cb60af5611ec4790eefc632", - "vllm_omni/diffusion/data.py": "5db83d5c71cb27cc7810d35fcfc891da80f0b003adbb16bdecc3816a63305e11", - "vllm_omni/diffusion/distributed/collective_gate.py": "01c6317cfa615840681b8bb8b8513fa3736d26e845a24ad9e1ab7f87489b962f", - "vllm_omni/diffusion/offloader/base.py": "533c793119e4cdc14a2ea3c6cb5bf97ffede673befeab3f0fb8dbbf51e655379", - "vllm_omni/diffusion/offloader/distributed_layerwise_backend.py": "d441481513aba9e90a91265f4bba3f53f8766197be8c1302c3c714e690d5ea76", - "vllm_omni/engine/arg_utils.py": "52569e3688cc708729a48ca0a2c875c28193733c2f934f6ff89f6c815e4009c7", - "vllm_omni/entrypoints/cli/serve.py": "350461dfca565bc0b07087ac7af1e27b0e247f5eac717e350b637e37f62dd088", - "tests/diffusion/offloader/helpers.py": "729c73abba3b9a12911282d4bcd3cf8bfa148fbcbfdc2e2b7c86000e387fa687" - }, - "scope": "CPU mock barrier/chunk tests; no actual NPU collective or full-model quality/performance acceptance", - "status": "failed", - "returncode": 1, - "elapsed_seconds": 62.34616496198578 - }, - { - "trial": "c13-dlo-regression-v2-12c217d5", - "cleanup_passed": true, - "source_unchanged": true, - "native_test_executed": false, - "files_before": { - "tests/diffusion/distributed/test_collective_gate.py": "2b33f73dff9c79118cc2b1bd4d50e3542ebc45b7bf029b38c17abbef37e4b917", - "vllm_omni/config/omni_config.py": "6894120a330474f718d43bf2bf7928892eb996152839ce6469c34d18a7df09d1", - "vllm_omni/config/stage_config.py": "0752e8cc1c6f1021f737830d08663cfc118923a83fd909c607f9ec3ab2a3d2fb", - "vllm_omni/diffusion/attention/layer.py": "76c5fce3dabd23e39a8890316e1691c03bc203066cb60af5611ec4790eefc632", - "vllm_omni/diffusion/data.py": "5db83d5c71cb27cc7810d35fcfc891da80f0b003adbb16bdecc3816a63305e11", - "vllm_omni/diffusion/distributed/collective_gate.py": "01c6317cfa615840681b8bb8b8513fa3736d26e845a24ad9e1ab7f87489b962f", - "vllm_omni/diffusion/offloader/base.py": "533c793119e4cdc14a2ea3c6cb5bf97ffede673befeab3f0fb8dbbf51e655379", - "vllm_omni/diffusion/offloader/distributed_layerwise_backend.py": "d441481513aba9e90a91265f4bba3f53f8766197be8c1302c3c714e690d5ea76", - "vllm_omni/engine/arg_utils.py": "52569e3688cc708729a48ca0a2c875c28193733c2f934f6ff89f6c815e4009c7", - "vllm_omni/entrypoints/cli/serve.py": "350461dfca565bc0b07087ac7af1e27b0e247f5eac717e350b637e37f62dd088", - "tests/diffusion/offloader/helpers.py": "729c73abba3b9a12911282d4bcd3cf8bfa148fbcbfdc2e2b7c86000e387fa687" - }, - "scope": "Expanded offloader tests in Ascend environment; some stock accelerator fixtures use NPU, not a full model test", - "status": "failed", - "returncode": 1, - "elapsed_seconds": 65.14432259398745 - }, - { - "trial": "c14-int8-prerequisites-fdd6633b", - "cleanup_passed": true, - "source_unchanged": true, - "helpers_unchanged": true, - "scope": "Bounded native prerequisites only; a failed probe is retained as negative evidence.", - "status": "failed", - "returncode": 1, - "elapsed_seconds": 20.795623040001374, - "native_prerequisite": { - "verdict": "FAIL", - "scope": "TorchNPU block-INT8 attention prerequisites only", - "phase": "dynamic_block_int8_quant_query", - "versions": { - "torch": "2.10.0+cpu", - "torch_npu": "2.10.0.post4", - "mindiesd": "3.1.0" - }, - "apis": { - "float8_e4m3fn": true, - "npu_dynamic_block_quant": true, - "npu_fused_infer_attention_score_v2": true - }, - "device_name": "Ascend910_9392", - "input": { - "shape": [ - 1, - 14, - 256, - 128 - ], - "dtype": "torch.bfloat16", - "visible_devices": "2" - }, - "error_type": "RuntimeError", - "probe_sha256": "fab18252f860231f9b7fcaa54587afbaa96fce15987fd551922246c8964f7217", - "error": "npu_dynamic_block_quant:../third_party/op-plugin/op_plugin/ops/opapi/DynamicBlockQuantNpuOpApi.cpp:61 NPU function error: call aclnnDynamicBlockQuantV2 failed, error code is 561002\n[ERROR] 2026-11-02-02:00:57 (PID:2662830, Device:0, RankID:-1) ERR00100 PTA call acl api failed.\n[PID: 2662830] 2026-11-02-02:00:57.039.986 Invalid_Argument(EZ0024): Parameter row_block_size of DynamicBlockQuant has incorrect value 128. It should be 1.\n Solution: Check whether the parameter value is correct.\nTraceBack (most recent call last):\n The attr get failed.[FUNC:CheckParams][FILE:dynamic_block_quant_i8_tiling.cpp][LINE:78]\n The params check failed.[FUNC:Tiling4DynamicBlockQuant][FILE:dynamic_block_quant_tiling.cpp][LINE:564]\n Failed to execute operator DynamicBlockQuant_0. Reason: Failed to execute tiling function.\n Check NnopbaseExecutorDoTiling(executor) failed\n Check NnopbaseExecutorTilingAndUpdateBinInfo(executor) failed\n Check NnopbaseExecutorMatchCache(executor) failed\n Check NnopbaseRunForWorkspace(*executor, workspaceSize) failed\n" - } - }, - { - "trial": "c14-quant-prerequisites-358b9b28", - "cleanup_passed": true, - "source_unchanged": true, - "helpers_unchanged": true, - "scope": "Bounded native prerequisites only; a failed probe is retained as negative evidence.", - "status": "failed", - "returncode": 1, - "elapsed_seconds": 20.143024684977718, - "native_prerequisite": { - "verdict": "FAIL", - "scope": "TorchNPU FP8 prerequisites only", - "phase": "dynamic_block_quant_query", - "versions": { - "torch": "2.10.0+cpu", - "torch_npu": "2.10.0.post4", - "mindiesd": "3.1.0" - }, - "apis": { - "float8_e4m3fn": true, - "npu_dynamic_block_quant": true, - "npu_fused_infer_attention_score_v2": true - }, - "device_name": "Ascend910_9392", - "input": { - "shape": [ - 1, - 14, - 256, - 128 - ], - "dtype": "torch.bfloat16", - "visible_devices": "2" - }, - "error_type": "RuntimeError", - "probe_sha256": "df6465956e6f73bbca140fb9880057e30869fd4b7976b2dc091f71a575dd1096", - "error": "npu_dynamic_block_quant:../third_party/op-plugin/op_plugin/ops/opapi/DynamicBlockQuantNpuOpApi.cpp:61 NPU function error: call aclnnDynamicBlockQuantV2 failed, error code is 161002\n[ERROR] 2026-11-02-01:45:24 (PID:2491012, Device:0, RankID:-1) ERR00100 PTA call acl api failed.\n[PID: 2491012] 2026-11-02-01:45:24.732.576 Execution_Error(EZ1009): Failed to execute operator DynamicBlockQuant_0. Reason: The dtype or format of the actual input or output parameter of the operator is inconsistent with that defined in the operator prototype OpDef, details of parameter are x(DT_BFLOAT16, ND), yOut(DT_FLOAT8_E4M3FN, ND), scaleOut(DT_FLOAT, ND).\nTraceBack (most recent call last):\n Supported dtype and format list is:\n SupportInfo[0]: x(DT_FLOAT16, ND), yOut(DT_INT8, ND), scaleOut(DT_FLOAT, ND)\n SupportInfo[1]: x(DT_BFLOAT16, ND), yOut(DT_INT8, ND), scaleOut(DT_FLOAT, ND)\n Check NnopbaseExecutorTilingAndUpdateBinInfo(executor) failed\n Check NnopbaseExecutorMatchCache(executor) failed\n Check NnopbaseRunForWorkspace(*executor, workspaceSize) failed\n" - } - }, - { - "trial": "c15-video-row-native-066a9a40", - "cleanup_passed": true, - "source_unchanged": true, - "required_native_test_executed": true, - "files_before": { - "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "e35b551b8fc0ec3b0ba1384b4df96a8daa0e5179c5ed39d641125310b2ee957f", - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ca87df1936fd2694fcc768079e9e8f926b6d45969ddc61ebf353f5e726d70b8b", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "1258655fcdca4692c2306100846761a2b0b149bbcaa5cde48a9abaa559913bc3", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "a8257f98c2e30c7bc7978bf645966d2eaa98cd49a42a8a923c062b5808dcdcdf", - "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "6f890fcf84047bc5bc084f7b756a2b02e72573888a3bcc12ba7d0211e1bb62ef", - "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "d2a5bed10620a337ff9aaf99cf4a3740d94a8962ef903bf2da75d65e3b50e21c" - }, - "scope": "Focused state/block tests and native four-rank subset attention; no full-model quality/performance acceptance", - "status": "failed", - "returncode": 1, - "elapsed_seconds": 82.27795292000519 - }, - { - "trial": "c15-video-row-native-3a9721a6", - "cleanup_passed": true, - "source_unchanged": true, - "required_native_test_executed": true, - "files_before": { - "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "cf9fb13c5f24c7595d62d13cae6e283cafb2178612b20028d35357f0cafaac83", - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ca87df1936fd2694fcc768079e9e8f926b6d45969ddc61ebf353f5e726d70b8b", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "1258655fcdca4692c2306100846761a2b0b149bbcaa5cde48a9abaa559913bc3", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "a8257f98c2e30c7bc7978bf645966d2eaa98cd49a42a8a923c062b5808dcdcdf", - "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "c5fab405ee82bc553e9a902fa27e6dfbc5213ec18311e17b6415396a69b50f43", - "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "8ec1b3ae4cd252f5d95ab23e0232b983900939e8cb15d1e64f4740f7204abf39", - "vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py": "e8e2e74fb8edb05e02d3e2328ea1f3ad5ddcfd7a10ed09c5cf3911b1d063e849" - }, - "scope": "Focused state/block tests and native four-rank subset attention; no full-model quality/performance acceptance", - "status": "completed", - "returncode": 0, - "elapsed_seconds": 81.91829407500336 - }, - { - "trial": "c15-video-row-native-a2a224de", - "cleanup_passed": true, - "source_unchanged": true, - "required_native_test_executed": true, - "files_before": { - "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "cf9fb13c5f24c7595d62d13cae6e283cafb2178612b20028d35357f0cafaac83", - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ca87df1936fd2694fcc768079e9e8f926b6d45969ddc61ebf353f5e726d70b8b", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "1258655fcdca4692c2306100846761a2b0b149bbcaa5cde48a9abaa559913bc3", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "a8257f98c2e30c7bc7978bf645966d2eaa98cd49a42a8a923c062b5808dcdcdf", - "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "118afcc28379dd73feec763fbb9cfc9f17c6a208afe5969a85b3c969b8059552", - "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "875d0e70621fc76906be130519d7bb1242de5dd58159e31025e26da37e60d98c" - }, - "scope": "Focused state/block tests and native four-rank subset attention; no full-model quality/performance acceptance", - "status": "failed", - "returncode": 1, - "elapsed_seconds": 82.86056435096543 - }, - { - "trial": "c15-video-row-native-bb5084cc", - "cleanup_passed": true, - "source_unchanged": true, - "required_native_test_executed": true, - "files_before": { - "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "cf9fb13c5f24c7595d62d13cae6e283cafb2178612b20028d35357f0cafaac83", - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ca87df1936fd2694fcc768079e9e8f926b6d45969ddc61ebf353f5e726d70b8b", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "1258655fcdca4692c2306100846761a2b0b149bbcaa5cde48a9abaa559913bc3", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "a8257f98c2e30c7bc7978bf645966d2eaa98cd49a42a8a923c062b5808dcdcdf", - "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "118afcc28379dd73feec763fbb9cfc9f17c6a208afe5969a85b3c969b8059552", - "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "fa1ac8fd375d754503b3507fae006381138d8875afdad2418f1ea5ae338b9c26" - }, - "scope": "Focused state/block tests and native four-rank subset attention; no full-model quality/performance acceptance", - "status": "completed", - "returncode": 0, - "elapsed_seconds": 81.50971999001922 - }, - { - "trial": "c15-video-row-quality-5s-00-710315e0", - "model_controller": { - "status": "failed", - "returncode": 1, - "elapsed_seconds": 153.44603093498154 - }, - "actual_counters": { - "request-000\\rank-0\\attention_path_counts.json": {}, - "request-000\\rank-1\\attention_path_counts.json": {}, - "request-000\\rank-2\\attention_path_counts.json": {}, - "request-000\\rank-3\\attention_path_counts.json": {} - } - }, - { - "trial": "c15-video-row-v2-quality-5s-00-d6b921b9", - "model_controller": { - "status": "failed", - "returncode": 1, - "elapsed_seconds": 199.99670764501207 - }, - "actual_counters": { - "request-000\\rank-0\\video_row_cache_stats.json": { - "request_counters": { - "full_steps": 5, - "partial_steps": 1, - "full_blocks": 250, - "partial_blocks": 1, - "subset_attention_calls": 1, - "subset_query_rows": 2475, - "omitted_query_rows": 37296, - "partial_mlp_rows": 2475, - "cache_bytes": 4019635200, - "closed": true - }, - "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" - }, - "request-000\\rank-1\\video_row_cache_stats.json": { - "request_counters": { - "full_steps": 5, - "partial_steps": 1, - "full_blocks": 250, - "partial_blocks": 1, - "subset_attention_calls": 1, - "subset_query_rows": 2475, - "omitted_query_rows": 37296, - "partial_mlp_rows": 0, - "cache_bytes": 5350195200, - "closed": true - }, - "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" - }, - "request-000\\rank-2\\video_row_cache_stats.json": { - "request_counters": { - "full_steps": 5, - "partial_steps": 1, - "full_blocks": 250, - "partial_blocks": 1, - "subset_attention_calls": 1, - "subset_query_rows": 2475, - "omitted_query_rows": 37296, - "partial_mlp_rows": 0, - "cache_bytes": 5350195200, - "closed": true - }, - "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" - }, - "request-000\\rank-3\\video_row_cache_stats.json": { - "request_counters": { - "full_steps": 5, - "partial_steps": 1, - "full_blocks": 250, - "partial_blocks": 1, - "subset_attention_calls": 1, - "subset_query_rows": 2475, - "omitted_query_rows": 37296, - "partial_mlp_rows": 0, - "cache_bytes": 5330304000, - "closed": true - }, - "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" - }, - "request-000\\rank-0\\attention_path_counts.json": { - "_prefix_bnsd": 251 - }, - "request-000\\rank-1\\attention_path_counts.json": { - "_prefix_bnsd": 251 - }, - "request-000\\rank-2\\attention_path_counts.json": { - "_prefix_bnsd": 251 - }, - "request-000\\rank-3\\attention_path_counts.json": { - "_prefix_bnsd": 251 - } - } - }, - { - "trial": "c15-video-row-v3-quality-5s-00-70703586", - "model_controller": { - "status": "failed", - "returncode": 1, - "elapsed_seconds": 191.09544755500974 - }, - "actual_counters": { - "request-000\\rank-0\\video_row_cache_stats.json": { - "request_counters": { - "full_steps": 5, - "partial_steps": 1, - "full_blocks": 250, - "partial_blocks": 50, - "subset_attention_calls": 50, - "subset_query_rows": 123750, - "omitted_query_rows": 1864800, - "partial_mlp_rows": 123750, - "cache_bytes": 4019635200, - "closed": true - }, - "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" - }, - "request-000\\rank-1\\video_row_cache_stats.json": { - "request_counters": { - "full_steps": 5, - "partial_steps": 1, - "full_blocks": 250, - "partial_blocks": 50, - "subset_attention_calls": 50, - "subset_query_rows": 123750, - "omitted_query_rows": 1864800, - "partial_mlp_rows": 0, - "cache_bytes": 5350195200, - "closed": true - }, - "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" - }, - "request-000\\rank-2\\video_row_cache_stats.json": { - "request_counters": { - "full_steps": 5, - "partial_steps": 1, - "full_blocks": 250, - "partial_blocks": 50, - "subset_attention_calls": 50, - "subset_query_rows": 123750, - "omitted_query_rows": 1864800, - "partial_mlp_rows": 0, - "cache_bytes": 5350195200, - "closed": true - }, - "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" - }, - "request-000\\rank-3\\video_row_cache_stats.json": { - "request_counters": { - "full_steps": 5, - "partial_steps": 1, - "full_blocks": 250, - "partial_blocks": 50, - "subset_attention_calls": 50, - "subset_query_rows": 123750, - "omitted_query_rows": 1864800, - "partial_mlp_rows": 0, - "cache_bytes": 5330304000, - "closed": true - }, - "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" - }, - "request-000\\rank-0\\attention_path_counts.json": { - "_prefix_bnsd": 300 - }, - "request-000\\rank-1\\attention_path_counts.json": { - "_prefix_bnsd": 300 - }, - "request-000\\rank-2\\attention_path_counts.json": { - "_prefix_bnsd": 300 - }, - "request-000\\rank-3\\attention_path_counts.json": { - "_prefix_bnsd": 300 - } - } - }, - { - "trial": "c16-video-row-fp32-native-d3fe320c", - "cleanup_passed": true, - "source_unchanged": true, - "required_native_test_executed": true, - "files_before": { - "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "c555d76f7bd6457a81238fca56100d11cb7ab043a4a649d55e23b6203d09bde2", - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ca87df1936fd2694fcc768079e9e8f926b6d45969ddc61ebf353f5e726d70b8b", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "1258655fcdca4692c2306100846761a2b0b149bbcaa5cde48a9abaa559913bc3", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "a8257f98c2e30c7bc7978bf645966d2eaa98cd49a42a8a923c062b5808dcdcdf", - "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "c5fab405ee82bc553e9a902fa27e6dfbc5213ec18311e17b6415396a69b50f43", - "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "e09b872f6900e821d42632a0ca09595f2affe6f9794049dbc74d69ba6a52b2b0", - "vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py": "e8e2e74fb8edb05e02d3e2328ea1f3ad5ddcfd7a10ed09c5cf3911b1d063e849" - }, - "scope": "Focused state/block tests and native four-rank subset attention; no full-model quality/performance acceptance", - "status": "completed", - "returncode": 0, - "elapsed_seconds": 78.8189791989862 - }, - { - "trial": "c16-video-row-fp32-quality-5s-00-8600757a", - "model_controller": { - "status": "completed", - "returncode": 0, - "elapsed_seconds": 355.51310362201184 - }, - "model_diagnostic": { - "source_head": "e0a3462fcc44de698b5dedcf8ff2c99f9802f757", - "samples_ms": [ - 197716.36760496767 - ], - "latency_median_ms": 197716.36760496767, - "scope": "http_complete_mp4", - "warmup": 0, - "repetitions": 1, - "kind": "quality", - "diagnostic": "audit", - "clean_timing": false, - "cleanup_passed": true - }, - "actual_counters": { - "request-000\\rank-0\\video_row_cache_stats.json": { - "request_counters": { - "full_steps": 17, - "partial_steps": 33, - "full_blocks": 850, - "partial_blocks": 1650, - "subset_attention_calls": 1650, - "subset_query_rows": 4083750, - "omitted_query_rows": 61538400, - "partial_mlp_rows": 4083750, - "peak_cache_bytes": 8039270400, - "closed": true, - "storage_dtype": "torch.float32", - "live_cache_bytes": 0 - }, - "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" - }, - "request-000\\rank-1\\video_row_cache_stats.json": { - "request_counters": { - "full_steps": 17, - "partial_steps": 33, - "full_blocks": 850, - "partial_blocks": 1650, - "subset_attention_calls": 1650, - "subset_query_rows": 4083750, - "omitted_query_rows": 61538400, - "partial_mlp_rows": 0, - "peak_cache_bytes": 10700390400, - "closed": true, - "storage_dtype": "torch.float32", - "live_cache_bytes": 0 - }, - "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" - }, - "request-000\\rank-2\\video_row_cache_stats.json": { - "request_counters": { - "full_steps": 17, - "partial_steps": 33, - "full_blocks": 850, - "partial_blocks": 1650, - "subset_attention_calls": 1650, - "subset_query_rows": 4083750, - "omitted_query_rows": 61538400, - "partial_mlp_rows": 0, - "peak_cache_bytes": 10700390400, - "closed": true, - "storage_dtype": "torch.float32", - "live_cache_bytes": 0 - }, - "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" - }, - "request-000\\rank-3\\video_row_cache_stats.json": { - "request_counters": { - "full_steps": 17, - "partial_steps": 33, - "full_blocks": 850, - "partial_blocks": 1650, - "subset_attention_calls": 1650, - "subset_query_rows": 4083750, - "omitted_query_rows": 61538400, - "partial_mlp_rows": 0, - "peak_cache_bytes": 10660608000, - "closed": true, - "storage_dtype": "torch.float32", - "live_cache_bytes": 0 - }, - "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" - }, - "request-000\\rank-0\\attention_path_counts.json": { - "_prefix_bnsd": 2500 - }, - "request-000\\rank-1\\attention_path_counts.json": { - "_prefix_bnsd": 2500 - }, - "request-000\\rank-2\\attention_path_counts.json": { - "_prefix_bnsd": 2500 - }, - "request-000\\rank-3\\attention_path_counts.json": { - "_prefix_bnsd": 2500 - } - } - }, - { - "trial": "c17-video-window-native-9b48cb56", - "cleanup_passed": true, - "source_unchanged": true, - "required_native_test_executed": true, - "files_before": { - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "66ca27991ce42ce302e5de078be6395c9d1ab53752378babef25089f3f75c3aa", - "tests/diffusion/models/minimax_h3/test_minimax_h3_video_window.py": "0bc59f35eff41164e125a5070e034c94ea60bf0112733ebe59eb6378eccbfed8" - }, - "scope": "CPU window reference/routing and native batched BF16 FAS; no full-model quality/performance acceptance", - "status": "completed", - "returncode": 0, - "elapsed_seconds": 51.878022204036824 - }, - { - "trial": "c17-video-window-operator-9bf7de3e", - "cleanup_passed": true, - "source_unchanged": true, - "files_before": { - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "66ca27991ce42ce302e5de078be6395c9d1ab53752378babef25089f3f75c3aa", - "tests/diffusion/models/minimax_h3/test_minimax_h3_video_window.py": "0bc59f35eff41164e125a5070e034c94ea60bf0112733ebe59eb6378eccbfed8" - }, - "scope": "Real captured H3 attention operator benchmark; full-model quality and HTTP are separate; no full-model quality/performance acceptance", - "status": "completed", - "returncode": 0, - "elapsed_seconds": 51.778123575029895, - "operator_benchmark": { - "scope": "real captured 5s attention operation on logical2; no model quality or HTTP acceptance", - "fixture_sha256": "00767e6afb2ba777bb2677eb9b60dc2f00fb649f5006ae8c1d77191214fbf912", - "fixture_tensor_sha256": { - "query": "313095dd143adf0636d57c90d77b689405dee510df3ebd6385a49d071c3323e9", - "key": "6638aee85f0039ee814206e7c976a2d1a9a2b185723c7bbb74c69b1e68c5bad4", - "value": "101eede996c8b13d61d487019d47f0b5fd2193664d8c178665063675cb66fdca" - }, - "query_shape": [ - 1, - 39808, - 14, - 128 - ], - "valid_kv": 39771, - "frames": 37, - "rows_per_frame": 1008, - "target_start": 2475, - "window": 3, - "finite": true, - "relative_l2_vs_dense": 0.5161040425300598, - "baseline_samples_ms": [ - 69.0619180095382, - 69.60304302629083, - 69.27597103640437, - 69.81903495034203, - 69.96654602698982 - ], - "candidate_samples_ms": [ - 25.191069988068193, - 25.301200977992266, - 25.325032009277493, - 25.26562102138996, - 25.219489994924515 - ], - "baseline_median_ms": 69.60304302629083, - "candidate_median_ms": 25.26562102138996, - "reduction_pct": 63.70040744936038, - "operator_gate_passed": true, - "warmups_per_operation": 2, - "measured_per_operation": 5, - "order": "AB/BA/AB/BA/AB", - "probe_sha256": "2465cf08173950a241ac3cc4713a4227d4c4b5a87f35358692a7ef6b60821824" - } - }, - { - "trial": "c17-video-window-quality-5s-00-a8c689e4", - "model_controller": { - "status": "completed", - "returncode": 0, - "elapsed_seconds": 358.6752056070254 - }, - "model_diagnostic": { - "source_head": "c1589a2b5e4ada617e7cca38ef7c300fe9495c58", - "samples_ms": [ - 210397.12271699682 - ], - "latency_median_ms": 210397.12271699682, - "scope": "http_complete_mp4", - "warmup": 0, - "repetitions": 1, - "kind": "quality", - "diagnostic": "audit", - "clean_timing": false, - "cleanup_passed": true - }, - "actual_counters": { - "request-000\\rank-0\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 4900, - "_video_temporal_window_bnsd": 2400 - }, - "request-000\\rank-1\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 4900, - "_video_temporal_window_bnsd": 2400 - }, - "request-000\\rank-2\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 4900, - "_video_temporal_window_bnsd": 2400 - }, - "request-000\\rank-3\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 4900, - "_video_temporal_window_bnsd": 2400 - } - } - } - ], - "decisions": [ - "C12 fixed profile stopped: zero actual predictions.", - "C14 pinned block-FP8 and block-INT8 prerequisites failed; per-token INT8 is not excluded.", - "C13 numerical qualification and clean performance are separate.", - "C15 fixed FP16 video residual profile failed finite velocity at first reuse.", - "C16 FP32 residual completed but failed three frozen audio quality guards; fixed profile stopped.", - "C17 operator reduction is separate from full-model quality and complete HTTP acceptance." - ] -} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c12-through-c19-execution-checkpoint.json b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c12-through-c19-execution-checkpoint.json deleted file mode 100644 index 02c18c9cc..000000000 --- a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c12-through-c19-execution-checkpoint.json +++ /dev/null @@ -1,1018 +0,0 @@ -{ - "campaign_verdict": "INCONCLUSIVE", - "performance_acceptance": "No newly accepted gain", - "runs": [ - { - "trial": "c12-alignment-quality-5s-00-9bf5c373", - "model_controller": { - "status": "completed", - "returncode": 0, - "elapsed_seconds": 471.80311120999977 - }, - "model_diagnostic": { - "source_head": "a29648e5acd3077b7b22739bc9522017782fe1ac", - "samples_ms": [ - 318280.59680899605 - ], - "latency_median_ms": 318280.59680899605, - "scope": "http_complete_mp4", - "warmup": 0, - "repetitions": 1, - "kind": "quality", - "diagnostic": "audit", - "clean_timing": false, - "cleanup_passed": true - }, - "actual_counters": { - "request-000\\rank-0\\alignment_cache_stats.json": { - "request_counters": { - "computed": 50, - "predicted": 0, - "vetoes": 45 - }, - "scope": "actual state-aligned compute/forecast/veto counts; not quality acceptance" - }, - "request-000\\rank-1\\alignment_cache_stats.json": { - "request_counters": { - "computed": 50, - "predicted": 0, - "vetoes": 45 - }, - "scope": "actual state-aligned compute/forecast/veto counts; not quality acceptance" - }, - "request-000\\rank-2\\alignment_cache_stats.json": { - "request_counters": { - "computed": 50, - "predicted": 0, - "vetoes": 45 - }, - "scope": "actual state-aligned compute/forecast/veto counts; not quality acceptance" - }, - "request-000\\rank-3\\alignment_cache_stats.json": { - "request_counters": { - "computed": 50, - "predicted": 0, - "vetoes": 45 - }, - "scope": "actual state-aligned compute/forecast/veto counts; not quality acceptance" - }, - "request-000\\rank-0\\attention_path_counts.json": { - "_prefix_bnsd": 2500 - }, - "request-000\\rank-1\\attention_path_counts.json": { - "_prefix_bnsd": 2500 - }, - "request-000\\rank-2\\attention_path_counts.json": { - "_prefix_bnsd": 2500 - }, - "request-000\\rank-3\\attention_path_counts.json": { - "_prefix_bnsd": 2500 - } - } - }, - { - "trial": "c12-state-alignment-native-tests-0dc5859b", - "cleanup_passed": true, - "source_unchanged": true, - "native_test_executed": true, - "files_before": { - "vllm_omni/diffusion/models/minimax_h3/cache_state_alignment.py": "3340ec21a77aef78531c601be241bcfb1e44107c795e8c8649907d1a18ed377f", - "vllm_omni/diffusion/models/minimax_h3/reference_kv_cache_dit.py": "f4cd0e9aa9e824c92ec1acd02c57a8f7d8684c35d4edf92ffdfae3345ff44e99", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "436ec52cd6329cbc64e91d94b450bfa6f773a9a953b8f974db73a5eb986a98bb", - "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "4dec8f02a04bfe7e8d2ada24fc6d5ae40a0058671d8515c8a266ecf88824b7b0", - "tests/diffusion/models/minimax_h3/test_minimax_h3_state_cache.py": "89cf57482b35d8d8ad8d4978aec87e9ee3ef45330e5b911161adef4379950a65" - }, - "scope": "focused tests only; no full-model quality or performance acceptance", - "status": "completed", - "returncode": 0, - "elapsed_seconds": 82.06077561300481 - }, - { - "trial": "c13-baseline-cuda-staging-e22d8651", - "cleanup_passed": true, - "source_unchanged": true, - "native_test_executed": false, - "files_before": { - "vllm_omni/diffusion/offloader/sequential_backend.py": "d542935b656a0d9dced78de8fb086b8e1c5aade26fac6cee5316b32a49f0d569", - "tests/diffusion/offloader/test_sequential_backend.py": "dcf1d64e65bed572d65b095c1350950f55d7e8d9ddf8c7dfd72d0ea6e721685a" - }, - "scope": "Reproduce stock CUDA-graph staging expectation on unchanged migration owning source in the same Ascend test environment", - "status": "failed", - "returncode": 1, - "elapsed_seconds": 50.234599059971515 - }, - { - "trial": "c13-c1-dlo-5s-audit-cead10d1", - "model_controller": { - "status": "failed", - "returncode": 2, - "elapsed_seconds": 0.11969929095357656 - } - }, - { - "trial": "c13-c1-dlo-5s-audit-v2-b0b86c6d", - "model_controller": { - "status": "completed", - "returncode": 0, - "elapsed_seconds": 749.505351218977 - }, - "model_diagnostic": { - "source_head": "e89b1d40d37f7f72767892a56fa7980ea572ba8e", - "samples_ms": [ - 315593.86175096733, - 284456.94282796467 - ], - "latency_median_ms": 300025.402289466, - "scope": "http_complete_mp4", - "warmup": 0, - "repetitions": 2, - "kind": "diagnostic", - "diagnostic": "audit", - "clean_timing": false, - "cleanup_passed": true - }, - "actual_counters": { - "request-000\\rank-0\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-000\\rank-1\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-000\\rank-2\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-000\\rank-3\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-001\\rank-0\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-001\\rank-1\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-001\\rank-2\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-001\\rank-3\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - } - } - }, - { - "trial": "c13-c1-dlo-5s-calibration-c98129d8", - "model_controller": { - "status": "completed", - "returncode": 0, - "elapsed_seconds": 768.8735517460154 - }, - "model_diagnostic": { - "source_head": "e89b1d40d37f7f72767892a56fa7980ea572ba8e", - "samples_ms": [ - 317177.1984040388, - 285534.089674009 - ], - "latency_median_ms": 301355.6440390239, - "scope": "http_complete_mp4", - "warmup": 0, - "repetitions": 2, - "kind": "calibration", - "diagnostic": "audit", - "clean_timing": false, - "cleanup_passed": true - }, - "actual_counters": { - "request-000\\rank-0\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-000\\rank-1\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-000\\rank-2\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-000\\rank-3\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-001\\rank-0\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-001\\rank-1\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-001\\rank-2\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-001\\rank-3\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - } - } - }, - { - "trial": "c13-dlo-gate-tests-bc92a71d", - "cleanup_passed": true, - "source_unchanged": true, - "native_test_executed": false, - "files_before": { - "tests/diffusion/distributed/test_collective_gate.py": "2b33f73dff9c79118cc2b1bd4d50e3542ebc45b7bf029b38c17abbef37e4b917", - "vllm_omni/config/omni_config.py": "6894120a330474f718d43bf2bf7928892eb996152839ce6469c34d18a7df09d1", - "vllm_omni/config/stage_config.py": "0752e8cc1c6f1021f737830d08663cfc118923a83fd909c607f9ec3ab2a3d2fb", - "vllm_omni/diffusion/attention/layer.py": "76c5fce3dabd23e39a8890316e1691c03bc203066cb60af5611ec4790eefc632", - "vllm_omni/diffusion/data.py": "5db83d5c71cb27cc7810d35fcfc891da80f0b003adbb16bdecc3816a63305e11", - "vllm_omni/diffusion/distributed/collective_gate.py": "01c6317cfa615840681b8bb8b8513fa3736d26e845a24ad9e1ab7f87489b962f", - "vllm_omni/diffusion/offloader/base.py": "533c793119e4cdc14a2ea3c6cb5bf97ffede673befeab3f0fb8dbbf51e655379", - "vllm_omni/diffusion/offloader/distributed_layerwise_backend.py": "d441481513aba9e90a91265f4bba3f53f8766197be8c1302c3c714e690d5ea76", - "vllm_omni/engine/arg_utils.py": "52569e3688cc708729a48ca0a2c875c28193733c2f934f6ff89f6c815e4009c7", - "vllm_omni/entrypoints/cli/serve.py": "350461dfca565bc0b07087ac7af1e27b0e247f5eac717e350b637e37f62dd088", - "tests/diffusion/offloader/helpers.py": "729c73abba3b9a12911282d4bcd3cf8bfa148fbcbfdc2e2b7c86000e387fa687" - }, - "scope": "CPU mock barrier/chunk tests; no actual NPU collective or full-model quality/performance acceptance", - "status": "completed", - "returncode": 0, - "elapsed_seconds": 47.3752507569734 - }, - { - "trial": "c13-dlo-regression-tests-95e9ed2a", - "cleanup_passed": true, - "source_unchanged": true, - "native_test_executed": false, - "files_before": { - "tests/diffusion/distributed/test_collective_gate.py": "2b33f73dff9c79118cc2b1bd4d50e3542ebc45b7bf029b38c17abbef37e4b917", - "vllm_omni/config/omni_config.py": "6894120a330474f718d43bf2bf7928892eb996152839ce6469c34d18a7df09d1", - "vllm_omni/config/stage_config.py": "0752e8cc1c6f1021f737830d08663cfc118923a83fd909c607f9ec3ab2a3d2fb", - "vllm_omni/diffusion/attention/layer.py": "76c5fce3dabd23e39a8890316e1691c03bc203066cb60af5611ec4790eefc632", - "vllm_omni/diffusion/data.py": "5db83d5c71cb27cc7810d35fcfc891da80f0b003adbb16bdecc3816a63305e11", - "vllm_omni/diffusion/distributed/collective_gate.py": "01c6317cfa615840681b8bb8b8513fa3736d26e845a24ad9e1ab7f87489b962f", - "vllm_omni/diffusion/offloader/base.py": "533c793119e4cdc14a2ea3c6cb5bf97ffede673befeab3f0fb8dbbf51e655379", - "vllm_omni/diffusion/offloader/distributed_layerwise_backend.py": "d441481513aba9e90a91265f4bba3f53f8766197be8c1302c3c714e690d5ea76", - "vllm_omni/engine/arg_utils.py": "52569e3688cc708729a48ca0a2c875c28193733c2f934f6ff89f6c815e4009c7", - "vllm_omni/entrypoints/cli/serve.py": "350461dfca565bc0b07087ac7af1e27b0e247f5eac717e350b637e37f62dd088", - "tests/diffusion/offloader/helpers.py": "729c73abba3b9a12911282d4bcd3cf8bfa148fbcbfdc2e2b7c86000e387fa687" - }, - "scope": "CPU mock barrier/chunk tests; no actual NPU collective or full-model quality/performance acceptance", - "status": "failed", - "returncode": 1, - "elapsed_seconds": 62.34616496198578 - }, - { - "trial": "c13-dlo-regression-v2-12c217d5", - "cleanup_passed": true, - "source_unchanged": true, - "native_test_executed": false, - "files_before": { - "tests/diffusion/distributed/test_collective_gate.py": "2b33f73dff9c79118cc2b1bd4d50e3542ebc45b7bf029b38c17abbef37e4b917", - "vllm_omni/config/omni_config.py": "6894120a330474f718d43bf2bf7928892eb996152839ce6469c34d18a7df09d1", - "vllm_omni/config/stage_config.py": "0752e8cc1c6f1021f737830d08663cfc118923a83fd909c607f9ec3ab2a3d2fb", - "vllm_omni/diffusion/attention/layer.py": "76c5fce3dabd23e39a8890316e1691c03bc203066cb60af5611ec4790eefc632", - "vllm_omni/diffusion/data.py": "5db83d5c71cb27cc7810d35fcfc891da80f0b003adbb16bdecc3816a63305e11", - "vllm_omni/diffusion/distributed/collective_gate.py": "01c6317cfa615840681b8bb8b8513fa3736d26e845a24ad9e1ab7f87489b962f", - "vllm_omni/diffusion/offloader/base.py": "533c793119e4cdc14a2ea3c6cb5bf97ffede673befeab3f0fb8dbbf51e655379", - "vllm_omni/diffusion/offloader/distributed_layerwise_backend.py": "d441481513aba9e90a91265f4bba3f53f8766197be8c1302c3c714e690d5ea76", - "vllm_omni/engine/arg_utils.py": "52569e3688cc708729a48ca0a2c875c28193733c2f934f6ff89f6c815e4009c7", - "vllm_omni/entrypoints/cli/serve.py": "350461dfca565bc0b07087ac7af1e27b0e247f5eac717e350b637e37f62dd088", - "tests/diffusion/offloader/helpers.py": "729c73abba3b9a12911282d4bcd3cf8bfa148fbcbfdc2e2b7c86000e387fa687" - }, - "scope": "Expanded offloader tests in Ascend environment; some stock accelerator fixtures use NPU, not a full model test", - "status": "failed", - "returncode": 1, - "elapsed_seconds": 65.14432259398745 - }, - { - "trial": "c14-int8-prerequisites-fdd6633b", - "cleanup_passed": true, - "source_unchanged": true, - "helpers_unchanged": true, - "scope": "Bounded native prerequisites only; a failed probe is retained as negative evidence.", - "status": "failed", - "returncode": 1, - "elapsed_seconds": 20.795623040001374, - "native_prerequisite": { - "verdict": "FAIL", - "scope": "TorchNPU block-INT8 attention prerequisites only", - "phase": "dynamic_block_int8_quant_query", - "versions": { - "torch": "2.10.0+cpu", - "torch_npu": "2.10.0.post4", - "mindiesd": "3.1.0" - }, - "apis": { - "float8_e4m3fn": true, - "npu_dynamic_block_quant": true, - "npu_fused_infer_attention_score_v2": true - }, - "device_name": "Ascend910_9392", - "input": { - "shape": [ - 1, - 14, - 256, - 128 - ], - "dtype": "torch.bfloat16", - "visible_devices": "2" - }, - "error_type": "RuntimeError", - "probe_sha256": "fab18252f860231f9b7fcaa54587afbaa96fce15987fd551922246c8964f7217", - "error": "npu_dynamic_block_quant:../third_party/op-plugin/op_plugin/ops/opapi/DynamicBlockQuantNpuOpApi.cpp:61 NPU function error: call aclnnDynamicBlockQuantV2 failed, error code is 561002\n[ERROR] 2026-11-02-02:00:57 (PID:2662830, Device:0, RankID:-1) ERR00100 PTA call acl api failed.\n[PID: 2662830] 2026-11-02-02:00:57.039.986 Invalid_Argument(EZ0024): Parameter row_block_size of DynamicBlockQuant has incorrect value 128. It should be 1.\n Solution: Check whether the parameter value is correct.\nTraceBack (most recent call last):\n The attr get failed.[FUNC:CheckParams][FILE:dynamic_block_quant_i8_tiling.cpp][LINE:78]\n The params check failed.[FUNC:Tiling4DynamicBlockQuant][FILE:dynamic_block_quant_tiling.cpp][LINE:564]\n Failed to execute operator DynamicBlockQuant_0. Reason: Failed to execute tiling function.\n Check NnopbaseExecutorDoTiling(executor) failed\n Check NnopbaseExecutorTilingAndUpdateBinInfo(executor) failed\n Check NnopbaseExecutorMatchCache(executor) failed\n Check NnopbaseRunForWorkspace(*executor, workspaceSize) failed\n" - } - }, - { - "trial": "c14-quant-prerequisites-358b9b28", - "cleanup_passed": true, - "source_unchanged": true, - "helpers_unchanged": true, - "scope": "Bounded native prerequisites only; a failed probe is retained as negative evidence.", - "status": "failed", - "returncode": 1, - "elapsed_seconds": 20.143024684977718, - "native_prerequisite": { - "verdict": "FAIL", - "scope": "TorchNPU FP8 prerequisites only", - "phase": "dynamic_block_quant_query", - "versions": { - "torch": "2.10.0+cpu", - "torch_npu": "2.10.0.post4", - "mindiesd": "3.1.0" - }, - "apis": { - "float8_e4m3fn": true, - "npu_dynamic_block_quant": true, - "npu_fused_infer_attention_score_v2": true - }, - "device_name": "Ascend910_9392", - "input": { - "shape": [ - 1, - 14, - 256, - 128 - ], - "dtype": "torch.bfloat16", - "visible_devices": "2" - }, - "error_type": "RuntimeError", - "probe_sha256": "df6465956e6f73bbca140fb9880057e30869fd4b7976b2dc091f71a575dd1096", - "error": "npu_dynamic_block_quant:../third_party/op-plugin/op_plugin/ops/opapi/DynamicBlockQuantNpuOpApi.cpp:61 NPU function error: call aclnnDynamicBlockQuantV2 failed, error code is 161002\n[ERROR] 2026-11-02-01:45:24 (PID:2491012, Device:0, RankID:-1) ERR00100 PTA call acl api failed.\n[PID: 2491012] 2026-11-02-01:45:24.732.576 Execution_Error(EZ1009): Failed to execute operator DynamicBlockQuant_0. Reason: The dtype or format of the actual input or output parameter of the operator is inconsistent with that defined in the operator prototype OpDef, details of parameter are x(DT_BFLOAT16, ND), yOut(DT_FLOAT8_E4M3FN, ND), scaleOut(DT_FLOAT, ND).\nTraceBack (most recent call last):\n Supported dtype and format list is:\n SupportInfo[0]: x(DT_FLOAT16, ND), yOut(DT_INT8, ND), scaleOut(DT_FLOAT, ND)\n SupportInfo[1]: x(DT_BFLOAT16, ND), yOut(DT_INT8, ND), scaleOut(DT_FLOAT, ND)\n Check NnopbaseExecutorTilingAndUpdateBinInfo(executor) failed\n Check NnopbaseExecutorMatchCache(executor) failed\n Check NnopbaseRunForWorkspace(*executor, workspaceSize) failed\n" - } - }, - { - "trial": "c15-video-row-native-066a9a40", - "cleanup_passed": true, - "source_unchanged": true, - "required_native_test_executed": true, - "files_before": { - "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "e35b551b8fc0ec3b0ba1384b4df96a8daa0e5179c5ed39d641125310b2ee957f", - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ca87df1936fd2694fcc768079e9e8f926b6d45969ddc61ebf353f5e726d70b8b", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "1258655fcdca4692c2306100846761a2b0b149bbcaa5cde48a9abaa559913bc3", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "a8257f98c2e30c7bc7978bf645966d2eaa98cd49a42a8a923c062b5808dcdcdf", - "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "6f890fcf84047bc5bc084f7b756a2b02e72573888a3bcc12ba7d0211e1bb62ef", - "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "d2a5bed10620a337ff9aaf99cf4a3740d94a8962ef903bf2da75d65e3b50e21c" - }, - "scope": "Focused state/block tests and native four-rank subset attention; no full-model quality/performance acceptance", - "status": "failed", - "returncode": 1, - "elapsed_seconds": 82.27795292000519 - }, - { - "trial": "c15-video-row-native-3a9721a6", - "cleanup_passed": true, - "source_unchanged": true, - "required_native_test_executed": true, - "files_before": { - "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "cf9fb13c5f24c7595d62d13cae6e283cafb2178612b20028d35357f0cafaac83", - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ca87df1936fd2694fcc768079e9e8f926b6d45969ddc61ebf353f5e726d70b8b", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "1258655fcdca4692c2306100846761a2b0b149bbcaa5cde48a9abaa559913bc3", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "a8257f98c2e30c7bc7978bf645966d2eaa98cd49a42a8a923c062b5808dcdcdf", - "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "c5fab405ee82bc553e9a902fa27e6dfbc5213ec18311e17b6415396a69b50f43", - "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "8ec1b3ae4cd252f5d95ab23e0232b983900939e8cb15d1e64f4740f7204abf39", - "vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py": "e8e2e74fb8edb05e02d3e2328ea1f3ad5ddcfd7a10ed09c5cf3911b1d063e849" - }, - "scope": "Focused state/block tests and native four-rank subset attention; no full-model quality/performance acceptance", - "status": "completed", - "returncode": 0, - "elapsed_seconds": 81.91829407500336 - }, - { - "trial": "c15-video-row-native-a2a224de", - "cleanup_passed": true, - "source_unchanged": true, - "required_native_test_executed": true, - "files_before": { - "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "cf9fb13c5f24c7595d62d13cae6e283cafb2178612b20028d35357f0cafaac83", - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ca87df1936fd2694fcc768079e9e8f926b6d45969ddc61ebf353f5e726d70b8b", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "1258655fcdca4692c2306100846761a2b0b149bbcaa5cde48a9abaa559913bc3", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "a8257f98c2e30c7bc7978bf645966d2eaa98cd49a42a8a923c062b5808dcdcdf", - "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "118afcc28379dd73feec763fbb9cfc9f17c6a208afe5969a85b3c969b8059552", - "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "875d0e70621fc76906be130519d7bb1242de5dd58159e31025e26da37e60d98c" - }, - "scope": "Focused state/block tests and native four-rank subset attention; no full-model quality/performance acceptance", - "status": "failed", - "returncode": 1, - "elapsed_seconds": 82.86056435096543 - }, - { - "trial": "c15-video-row-native-bb5084cc", - "cleanup_passed": true, - "source_unchanged": true, - "required_native_test_executed": true, - "files_before": { - "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "cf9fb13c5f24c7595d62d13cae6e283cafb2178612b20028d35357f0cafaac83", - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ca87df1936fd2694fcc768079e9e8f926b6d45969ddc61ebf353f5e726d70b8b", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "1258655fcdca4692c2306100846761a2b0b149bbcaa5cde48a9abaa559913bc3", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "a8257f98c2e30c7bc7978bf645966d2eaa98cd49a42a8a923c062b5808dcdcdf", - "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "118afcc28379dd73feec763fbb9cfc9f17c6a208afe5969a85b3c969b8059552", - "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "fa1ac8fd375d754503b3507fae006381138d8875afdad2418f1ea5ae338b9c26" - }, - "scope": "Focused state/block tests and native four-rank subset attention; no full-model quality/performance acceptance", - "status": "completed", - "returncode": 0, - "elapsed_seconds": 81.50971999001922 - }, - { - "trial": "c15-video-row-quality-5s-00-710315e0", - "model_controller": { - "status": "failed", - "returncode": 1, - "elapsed_seconds": 153.44603093498154 - }, - "actual_counters": { - "request-000\\rank-0\\attention_path_counts.json": {}, - "request-000\\rank-1\\attention_path_counts.json": {}, - "request-000\\rank-2\\attention_path_counts.json": {}, - "request-000\\rank-3\\attention_path_counts.json": {} - } - }, - { - "trial": "c15-video-row-v2-quality-5s-00-d6b921b9", - "model_controller": { - "status": "failed", - "returncode": 1, - "elapsed_seconds": 199.99670764501207 - }, - "actual_counters": { - "request-000\\rank-0\\video_row_cache_stats.json": { - "request_counters": { - "full_steps": 5, - "partial_steps": 1, - "full_blocks": 250, - "partial_blocks": 1, - "subset_attention_calls": 1, - "subset_query_rows": 2475, - "omitted_query_rows": 37296, - "partial_mlp_rows": 2475, - "cache_bytes": 4019635200, - "closed": true - }, - "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" - }, - "request-000\\rank-1\\video_row_cache_stats.json": { - "request_counters": { - "full_steps": 5, - "partial_steps": 1, - "full_blocks": 250, - "partial_blocks": 1, - "subset_attention_calls": 1, - "subset_query_rows": 2475, - "omitted_query_rows": 37296, - "partial_mlp_rows": 0, - "cache_bytes": 5350195200, - "closed": true - }, - "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" - }, - "request-000\\rank-2\\video_row_cache_stats.json": { - "request_counters": { - "full_steps": 5, - "partial_steps": 1, - "full_blocks": 250, - "partial_blocks": 1, - "subset_attention_calls": 1, - "subset_query_rows": 2475, - "omitted_query_rows": 37296, - "partial_mlp_rows": 0, - "cache_bytes": 5350195200, - "closed": true - }, - "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" - }, - "request-000\\rank-3\\video_row_cache_stats.json": { - "request_counters": { - "full_steps": 5, - "partial_steps": 1, - "full_blocks": 250, - "partial_blocks": 1, - "subset_attention_calls": 1, - "subset_query_rows": 2475, - "omitted_query_rows": 37296, - "partial_mlp_rows": 0, - "cache_bytes": 5330304000, - "closed": true - }, - "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" - }, - "request-000\\rank-0\\attention_path_counts.json": { - "_prefix_bnsd": 251 - }, - "request-000\\rank-1\\attention_path_counts.json": { - "_prefix_bnsd": 251 - }, - "request-000\\rank-2\\attention_path_counts.json": { - "_prefix_bnsd": 251 - }, - "request-000\\rank-3\\attention_path_counts.json": { - "_prefix_bnsd": 251 - } - } - }, - { - "trial": "c15-video-row-v3-quality-5s-00-70703586", - "model_controller": { - "status": "failed", - "returncode": 1, - "elapsed_seconds": 191.09544755500974 - }, - "actual_counters": { - "request-000\\rank-0\\video_row_cache_stats.json": { - "request_counters": { - "full_steps": 5, - "partial_steps": 1, - "full_blocks": 250, - "partial_blocks": 50, - "subset_attention_calls": 50, - "subset_query_rows": 123750, - "omitted_query_rows": 1864800, - "partial_mlp_rows": 123750, - "cache_bytes": 4019635200, - "closed": true - }, - "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" - }, - "request-000\\rank-1\\video_row_cache_stats.json": { - "request_counters": { - "full_steps": 5, - "partial_steps": 1, - "full_blocks": 250, - "partial_blocks": 50, - "subset_attention_calls": 50, - "subset_query_rows": 123750, - "omitted_query_rows": 1864800, - "partial_mlp_rows": 0, - "cache_bytes": 5350195200, - "closed": true - }, - "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" - }, - "request-000\\rank-2\\video_row_cache_stats.json": { - "request_counters": { - "full_steps": 5, - "partial_steps": 1, - "full_blocks": 250, - "partial_blocks": 50, - "subset_attention_calls": 50, - "subset_query_rows": 123750, - "omitted_query_rows": 1864800, - "partial_mlp_rows": 0, - "cache_bytes": 5350195200, - "closed": true - }, - "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" - }, - "request-000\\rank-3\\video_row_cache_stats.json": { - "request_counters": { - "full_steps": 5, - "partial_steps": 1, - "full_blocks": 250, - "partial_blocks": 50, - "subset_attention_calls": 50, - "subset_query_rows": 123750, - "omitted_query_rows": 1864800, - "partial_mlp_rows": 0, - "cache_bytes": 5330304000, - "closed": true - }, - "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" - }, - "request-000\\rank-0\\attention_path_counts.json": { - "_prefix_bnsd": 300 - }, - "request-000\\rank-1\\attention_path_counts.json": { - "_prefix_bnsd": 300 - }, - "request-000\\rank-2\\attention_path_counts.json": { - "_prefix_bnsd": 300 - }, - "request-000\\rank-3\\attention_path_counts.json": { - "_prefix_bnsd": 300 - } - } - }, - { - "trial": "c16-video-row-fp32-native-d3fe320c", - "cleanup_passed": true, - "source_unchanged": true, - "required_native_test_executed": true, - "files_before": { - "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "c555d76f7bd6457a81238fca56100d11cb7ab043a4a649d55e23b6203d09bde2", - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ca87df1936fd2694fcc768079e9e8f926b6d45969ddc61ebf353f5e726d70b8b", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "1258655fcdca4692c2306100846761a2b0b149bbcaa5cde48a9abaa559913bc3", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "a8257f98c2e30c7bc7978bf645966d2eaa98cd49a42a8a923c062b5808dcdcdf", - "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "c5fab405ee82bc553e9a902fa27e6dfbc5213ec18311e17b6415396a69b50f43", - "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "e09b872f6900e821d42632a0ca09595f2affe6f9794049dbc74d69ba6a52b2b0", - "vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py": "e8e2e74fb8edb05e02d3e2328ea1f3ad5ddcfd7a10ed09c5cf3911b1d063e849" - }, - "scope": "Focused state/block tests and native four-rank subset attention; no full-model quality/performance acceptance", - "status": "completed", - "returncode": 0, - "elapsed_seconds": 78.8189791989862 - }, - { - "trial": "c16-video-row-fp32-quality-5s-00-8600757a", - "model_controller": { - "status": "completed", - "returncode": 0, - "elapsed_seconds": 355.51310362201184 - }, - "model_diagnostic": { - "source_head": "e0a3462fcc44de698b5dedcf8ff2c99f9802f757", - "samples_ms": [ - 197716.36760496767 - ], - "latency_median_ms": 197716.36760496767, - "scope": "http_complete_mp4", - "warmup": 0, - "repetitions": 1, - "kind": "quality", - "diagnostic": "audit", - "clean_timing": false, - "cleanup_passed": true - }, - "actual_counters": { - "request-000\\rank-0\\video_row_cache_stats.json": { - "request_counters": { - "full_steps": 17, - "partial_steps": 33, - "full_blocks": 850, - "partial_blocks": 1650, - "subset_attention_calls": 1650, - "subset_query_rows": 4083750, - "omitted_query_rows": 61538400, - "partial_mlp_rows": 4083750, - "peak_cache_bytes": 8039270400, - "closed": true, - "storage_dtype": "torch.float32", - "live_cache_bytes": 0 - }, - "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" - }, - "request-000\\rank-1\\video_row_cache_stats.json": { - "request_counters": { - "full_steps": 17, - "partial_steps": 33, - "full_blocks": 850, - "partial_blocks": 1650, - "subset_attention_calls": 1650, - "subset_query_rows": 4083750, - "omitted_query_rows": 61538400, - "partial_mlp_rows": 0, - "peak_cache_bytes": 10700390400, - "closed": true, - "storage_dtype": "torch.float32", - "live_cache_bytes": 0 - }, - "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" - }, - "request-000\\rank-2\\video_row_cache_stats.json": { - "request_counters": { - "full_steps": 17, - "partial_steps": 33, - "full_blocks": 850, - "partial_blocks": 1650, - "subset_attention_calls": 1650, - "subset_query_rows": 4083750, - "omitted_query_rows": 61538400, - "partial_mlp_rows": 0, - "peak_cache_bytes": 10700390400, - "closed": true, - "storage_dtype": "torch.float32", - "live_cache_bytes": 0 - }, - "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" - }, - "request-000\\rank-3\\video_row_cache_stats.json": { - "request_counters": { - "full_steps": 17, - "partial_steps": 33, - "full_blocks": 850, - "partial_blocks": 1650, - "subset_attention_calls": 1650, - "subset_query_rows": 4083750, - "omitted_query_rows": 61538400, - "partial_mlp_rows": 0, - "peak_cache_bytes": 10660608000, - "closed": true, - "storage_dtype": "torch.float32", - "live_cache_bytes": 0 - }, - "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" - }, - "request-000\\rank-0\\attention_path_counts.json": { - "_prefix_bnsd": 2500 - }, - "request-000\\rank-1\\attention_path_counts.json": { - "_prefix_bnsd": 2500 - }, - "request-000\\rank-2\\attention_path_counts.json": { - "_prefix_bnsd": 2500 - }, - "request-000\\rank-3\\attention_path_counts.json": { - "_prefix_bnsd": 2500 - } - } - }, - { - "trial": "c17-video-window-native-9b48cb56", - "cleanup_passed": true, - "source_unchanged": true, - "required_native_test_executed": true, - "files_before": { - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "66ca27991ce42ce302e5de078be6395c9d1ab53752378babef25089f3f75c3aa", - "tests/diffusion/models/minimax_h3/test_minimax_h3_video_window.py": "0bc59f35eff41164e125a5070e034c94ea60bf0112733ebe59eb6378eccbfed8" - }, - "scope": "CPU window reference/routing and native batched BF16 FAS; no full-model quality/performance acceptance", - "status": "completed", - "returncode": 0, - "elapsed_seconds": 51.878022204036824 - }, - { - "trial": "c17-video-window-operator-9bf7de3e", - "cleanup_passed": true, - "source_unchanged": true, - "files_before": { - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "66ca27991ce42ce302e5de078be6395c9d1ab53752378babef25089f3f75c3aa", - "tests/diffusion/models/minimax_h3/test_minimax_h3_video_window.py": "0bc59f35eff41164e125a5070e034c94ea60bf0112733ebe59eb6378eccbfed8" - }, - "scope": "Real captured H3 attention operator benchmark; full-model quality and HTTP are separate; no full-model quality/performance acceptance", - "status": "completed", - "returncode": 0, - "elapsed_seconds": 51.778123575029895, - "operator_benchmark": { - "scope": "real captured 5s attention operation on logical2; no model quality or HTTP acceptance", - "fixture_sha256": "00767e6afb2ba777bb2677eb9b60dc2f00fb649f5006ae8c1d77191214fbf912", - "fixture_tensor_sha256": { - "query": "313095dd143adf0636d57c90d77b689405dee510df3ebd6385a49d071c3323e9", - "key": "6638aee85f0039ee814206e7c976a2d1a9a2b185723c7bbb74c69b1e68c5bad4", - "value": "101eede996c8b13d61d487019d47f0b5fd2193664d8c178665063675cb66fdca" - }, - "query_shape": [ - 1, - 39808, - 14, - 128 - ], - "valid_kv": 39771, - "frames": 37, - "rows_per_frame": 1008, - "target_start": 2475, - "window": 3, - "finite": true, - "relative_l2_vs_dense": 0.5161040425300598, - "baseline_samples_ms": [ - 69.0619180095382, - 69.60304302629083, - 69.27597103640437, - 69.81903495034203, - 69.96654602698982 - ], - "candidate_samples_ms": [ - 25.191069988068193, - 25.301200977992266, - 25.325032009277493, - 25.26562102138996, - 25.219489994924515 - ], - "baseline_median_ms": 69.60304302629083, - "candidate_median_ms": 25.26562102138996, - "reduction_pct": 63.70040744936038, - "operator_gate_passed": true, - "warmups_per_operation": 2, - "measured_per_operation": 5, - "order": "AB/BA/AB/BA/AB", - "probe_sha256": "2465cf08173950a241ac3cc4713a4227d4c4b5a87f35358692a7ef6b60821824" - } - }, - { - "trial": "c17-video-window-quality-5s-00-a8c689e4", - "model_controller": { - "status": "completed", - "returncode": 0, - "elapsed_seconds": 358.6752056070254 - }, - "model_diagnostic": { - "source_head": "c1589a2b5e4ada617e7cca38ef7c300fe9495c58", - "samples_ms": [ - 210397.12271699682 - ], - "latency_median_ms": 210397.12271699682, - "scope": "http_complete_mp4", - "warmup": 0, - "repetitions": 1, - "kind": "quality", - "diagnostic": "audit", - "clean_timing": false, - "cleanup_passed": true - }, - "actual_counters": { - "request-000\\rank-0\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 4900, - "_video_temporal_window_bnsd": 2400 - }, - "request-000\\rank-1\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 4900, - "_video_temporal_window_bnsd": 2400 - }, - "request-000\\rank-2\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 4900, - "_video_temporal_window_bnsd": 2400 - }, - "request-000\\rank-3\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 4900, - "_video_temporal_window_bnsd": 2400 - } - } - }, - { - "trial": "c18-video-pooled-context-native-1673db72", - "cleanup_passed": true, - "source_unchanged": true, - "required_native_test_executed": true, - "files_before": { - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "1febc030a7917b6e10fe287014fe2aacef76ade7bf58ebb604fdedc36fb7c21d", - "tests/diffusion/models/minimax_h3/test_minimax_h3_pooled_context.py": "45a9ae5ba56858daee6a1088a8c8b1ed612ea9afd3c229c90a9b6db3c80a06b2" - }, - "scope": "Independent weighted global-context reference and native padded-head BF16 FAS; no full-model quality/performance acceptance", - "status": "completed", - "returncode": 0, - "elapsed_seconds": 49.97671151399845 - }, - { - "trial": "c18-video-pooled-context-operator-315db0e5", - "cleanup_passed": true, - "source_unchanged": true, - "files_before": { - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "1febc030a7917b6e10fe287014fe2aacef76ade7bf58ebb604fdedc36fb7c21d", - "tests/diffusion/models/minimax_h3/test_minimax_h3_pooled_context.py": "45a9ae5ba56858daee6a1088a8c8b1ed612ea9afd3c229c90a9b6db3c80a06b2" - }, - "scope": "Real captured H3 attention operator benchmark; full-model quality and HTTP are separate; no full-model quality/performance acceptance", - "status": "completed", - "returncode": 0, - "elapsed_seconds": 51.37963652302278, - "operator_benchmark": { - "scope": "real captured 5s attention operation on logical2; no model quality or HTTP acceptance", - "fixture_sha256": "00767e6afb2ba777bb2677eb9b60dc2f00fb649f5006ae8c1d77191214fbf912", - "fixture_tensor_sha256": { - "query": "313095dd143adf0636d57c90d77b689405dee510df3ebd6385a49d071c3323e9", - "key": "6638aee85f0039ee814206e7c976a2d1a9a2b185723c7bbb74c69b1e68c5bad4", - "value": "101eede996c8b13d61d487019d47f0b5fd2193664d8c178665063675cb66fdca" - }, - "query_shape": [ - 1, - 39808, - 14, - 128 - ], - "valid_kv": 39771, - "frames": 37, - "rows_per_frame": 1008, - "target_start": 2475, - "profile": "3x3-local1", - "finite": true, - "relative_l2_vs_dense": 0.21801254153251648, - "baseline_samples_ms": [ - 69.19049099087715, - 69.70348599134013, - 69.43478400353342, - 69.81050700414926, - 69.24652203451842 - ], - "candidate_samples_ms": [ - 68.29233502503484, - 64.26435202592984, - 71.05128600960597, - 64.39994304673746, - 64.39412303734571 - ], - "baseline_median_ms": 69.43478400353342, - "candidate_median_ms": 64.39994304673746, - "reduction_pct": 7.251179691924648, - "operator_gate_passed": false, - "warmups_per_operation": 2, - "measured_per_operation": 5, - "order": "AB/BA/AB/BA/AB", - "probe_sha256": "42852c0e4e693c5b3685981db617de00648feefe651f7060ad83c359b08dd81d" - } - }, - { - "trial": "c19-video-global-pool-native-f0f4c502", - "cleanup_passed": true, - "source_unchanged": true, - "required_native_test_executed": true, - "files_before": { - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ddfd6d6822cc6a5528fc95539f7f98dfdeed25d5489da3ed5cf76e6076a5ec94", - "tests/diffusion/models/minimax_h3/test_minimax_h3_global_pool.py": "5457db550b59e3373be8f5f21873c1b78ac357a6ac170c2603ee531b50e91467" - }, - "scope": "Independent weighted global-context reference and native padded-head BF16 FAS; no full-model quality/performance acceptance", - "status": "completed", - "returncode": 0, - "elapsed_seconds": 51.528875903983135 - } - ], - "decisions": [ - "C12 fixed profile stopped: zero actual predictions.", - "C14 pinned block-FP8 and block-INT8 prerequisites failed; per-token INT8 is not excluded.", - "C13 numerical qualification and clean performance are separate.", - "C15 fixed FP16 video residual profile failed finite velocity at first reuse.", - "C16 FP32 residual completed but failed three frozen audio quality guards; fixed profile stopped.", - "C17 completed5s but failed11frozen quality checks; fixed window profile stopped.", - "C18 native17PASS but operation reduced only7.25%; no full model run.", - "C19 shares global pooledKV; native/operator/full model qualification are separate." - ] -} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c12-through-c20-execution-checkpoint.json b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c12-through-c20-execution-checkpoint.json deleted file mode 100644 index cdaaac095..000000000 --- a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c12-through-c20-execution-checkpoint.json +++ /dev/null @@ -1,1147 +0,0 @@ -{ - "campaign_verdict": "INCONCLUSIVE", - "performance_acceptance": "No newly accepted gain", - "runs": [ - { - "trial": "c12-alignment-quality-5s-00-9bf5c373", - "model_controller": { - "status": "completed", - "returncode": 0, - "elapsed_seconds": 471.80311120999977 - }, - "model_diagnostic": { - "source_head": "a29648e5acd3077b7b22739bc9522017782fe1ac", - "samples_ms": [ - 318280.59680899605 - ], - "latency_median_ms": 318280.59680899605, - "scope": "http_complete_mp4", - "warmup": 0, - "repetitions": 1, - "kind": "quality", - "diagnostic": "audit", - "clean_timing": false, - "cleanup_passed": true - }, - "actual_counters": { - "request-000\\rank-0\\alignment_cache_stats.json": { - "request_counters": { - "computed": 50, - "predicted": 0, - "vetoes": 45 - }, - "scope": "actual state-aligned compute/forecast/veto counts; not quality acceptance" - }, - "request-000\\rank-1\\alignment_cache_stats.json": { - "request_counters": { - "computed": 50, - "predicted": 0, - "vetoes": 45 - }, - "scope": "actual state-aligned compute/forecast/veto counts; not quality acceptance" - }, - "request-000\\rank-2\\alignment_cache_stats.json": { - "request_counters": { - "computed": 50, - "predicted": 0, - "vetoes": 45 - }, - "scope": "actual state-aligned compute/forecast/veto counts; not quality acceptance" - }, - "request-000\\rank-3\\alignment_cache_stats.json": { - "request_counters": { - "computed": 50, - "predicted": 0, - "vetoes": 45 - }, - "scope": "actual state-aligned compute/forecast/veto counts; not quality acceptance" - }, - "request-000\\rank-0\\attention_path_counts.json": { - "_prefix_bnsd": 2500 - }, - "request-000\\rank-1\\attention_path_counts.json": { - "_prefix_bnsd": 2500 - }, - "request-000\\rank-2\\attention_path_counts.json": { - "_prefix_bnsd": 2500 - }, - "request-000\\rank-3\\attention_path_counts.json": { - "_prefix_bnsd": 2500 - } - } - }, - { - "trial": "c12-state-alignment-native-tests-0dc5859b", - "cleanup_passed": true, - "source_unchanged": true, - "native_test_executed": true, - "files_before": { - "vllm_omni/diffusion/models/minimax_h3/cache_state_alignment.py": "3340ec21a77aef78531c601be241bcfb1e44107c795e8c8649907d1a18ed377f", - "vllm_omni/diffusion/models/minimax_h3/reference_kv_cache_dit.py": "f4cd0e9aa9e824c92ec1acd02c57a8f7d8684c35d4edf92ffdfae3345ff44e99", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "436ec52cd6329cbc64e91d94b450bfa6f773a9a953b8f974db73a5eb986a98bb", - "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "4dec8f02a04bfe7e8d2ada24fc6d5ae40a0058671d8515c8a266ecf88824b7b0", - "tests/diffusion/models/minimax_h3/test_minimax_h3_state_cache.py": "89cf57482b35d8d8ad8d4978aec87e9ee3ef45330e5b911161adef4379950a65" - }, - "scope": "focused tests only; no full-model quality or performance acceptance", - "status": "completed", - "returncode": 0, - "elapsed_seconds": 82.06077561300481 - }, - { - "trial": "c13-baseline-cuda-staging-e22d8651", - "cleanup_passed": true, - "source_unchanged": true, - "native_test_executed": false, - "files_before": { - "vllm_omni/diffusion/offloader/sequential_backend.py": "d542935b656a0d9dced78de8fb086b8e1c5aade26fac6cee5316b32a49f0d569", - "tests/diffusion/offloader/test_sequential_backend.py": "dcf1d64e65bed572d65b095c1350950f55d7e8d9ddf8c7dfd72d0ea6e721685a" - }, - "scope": "Reproduce stock CUDA-graph staging expectation on unchanged migration owning source in the same Ascend test environment", - "status": "failed", - "returncode": 1, - "elapsed_seconds": 50.234599059971515 - }, - { - "trial": "c13-c1-dlo-5s-audit-cead10d1", - "model_controller": { - "status": "failed", - "returncode": 2, - "elapsed_seconds": 0.11969929095357656 - } - }, - { - "trial": "c13-c1-dlo-5s-audit-v2-b0b86c6d", - "model_controller": { - "status": "completed", - "returncode": 0, - "elapsed_seconds": 749.505351218977 - }, - "model_diagnostic": { - "source_head": "e89b1d40d37f7f72767892a56fa7980ea572ba8e", - "samples_ms": [ - 315593.86175096733, - 284456.94282796467 - ], - "latency_median_ms": 300025.402289466, - "scope": "http_complete_mp4", - "warmup": 0, - "repetitions": 2, - "kind": "diagnostic", - "diagnostic": "audit", - "clean_timing": false, - "cleanup_passed": true - }, - "actual_counters": { - "request-000\\rank-0\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-000\\rank-1\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-000\\rank-2\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-000\\rank-3\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-001\\rank-0\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-001\\rank-1\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-001\\rank-2\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-001\\rank-3\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - } - } - }, - { - "trial": "c13-c1-dlo-5s-calibration-c98129d8", - "model_controller": { - "status": "completed", - "returncode": 0, - "elapsed_seconds": 768.8735517460154 - }, - "model_diagnostic": { - "source_head": "e89b1d40d37f7f72767892a56fa7980ea572ba8e", - "samples_ms": [ - 317177.1984040388, - 285534.089674009 - ], - "latency_median_ms": 301355.6440390239, - "scope": "http_complete_mp4", - "warmup": 0, - "repetitions": 2, - "kind": "calibration", - "diagnostic": "audit", - "clean_timing": false, - "cleanup_passed": true - }, - "actual_counters": { - "request-000\\rank-0\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-000\\rank-1\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-000\\rank-2\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-000\\rank-3\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-001\\rank-0\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-001\\rank-1\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-001\\rank-2\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-001\\rank-3\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - } - } - }, - { - "trial": "c13-dlo-gate-tests-bc92a71d", - "cleanup_passed": true, - "source_unchanged": true, - "native_test_executed": false, - "files_before": { - "tests/diffusion/distributed/test_collective_gate.py": "2b33f73dff9c79118cc2b1bd4d50e3542ebc45b7bf029b38c17abbef37e4b917", - "vllm_omni/config/omni_config.py": "6894120a330474f718d43bf2bf7928892eb996152839ce6469c34d18a7df09d1", - "vllm_omni/config/stage_config.py": "0752e8cc1c6f1021f737830d08663cfc118923a83fd909c607f9ec3ab2a3d2fb", - "vllm_omni/diffusion/attention/layer.py": "76c5fce3dabd23e39a8890316e1691c03bc203066cb60af5611ec4790eefc632", - "vllm_omni/diffusion/data.py": "5db83d5c71cb27cc7810d35fcfc891da80f0b003adbb16bdecc3816a63305e11", - "vllm_omni/diffusion/distributed/collective_gate.py": "01c6317cfa615840681b8bb8b8513fa3736d26e845a24ad9e1ab7f87489b962f", - "vllm_omni/diffusion/offloader/base.py": "533c793119e4cdc14a2ea3c6cb5bf97ffede673befeab3f0fb8dbbf51e655379", - "vllm_omni/diffusion/offloader/distributed_layerwise_backend.py": "d441481513aba9e90a91265f4bba3f53f8766197be8c1302c3c714e690d5ea76", - "vllm_omni/engine/arg_utils.py": "52569e3688cc708729a48ca0a2c875c28193733c2f934f6ff89f6c815e4009c7", - "vllm_omni/entrypoints/cli/serve.py": "350461dfca565bc0b07087ac7af1e27b0e247f5eac717e350b637e37f62dd088", - "tests/diffusion/offloader/helpers.py": "729c73abba3b9a12911282d4bcd3cf8bfa148fbcbfdc2e2b7c86000e387fa687" - }, - "scope": "CPU mock barrier/chunk tests; no actual NPU collective or full-model quality/performance acceptance", - "status": "completed", - "returncode": 0, - "elapsed_seconds": 47.3752507569734 - }, - { - "trial": "c13-dlo-regression-tests-95e9ed2a", - "cleanup_passed": true, - "source_unchanged": true, - "native_test_executed": false, - "files_before": { - "tests/diffusion/distributed/test_collective_gate.py": "2b33f73dff9c79118cc2b1bd4d50e3542ebc45b7bf029b38c17abbef37e4b917", - "vllm_omni/config/omni_config.py": "6894120a330474f718d43bf2bf7928892eb996152839ce6469c34d18a7df09d1", - "vllm_omni/config/stage_config.py": "0752e8cc1c6f1021f737830d08663cfc118923a83fd909c607f9ec3ab2a3d2fb", - "vllm_omni/diffusion/attention/layer.py": "76c5fce3dabd23e39a8890316e1691c03bc203066cb60af5611ec4790eefc632", - "vllm_omni/diffusion/data.py": "5db83d5c71cb27cc7810d35fcfc891da80f0b003adbb16bdecc3816a63305e11", - "vllm_omni/diffusion/distributed/collective_gate.py": "01c6317cfa615840681b8bb8b8513fa3736d26e845a24ad9e1ab7f87489b962f", - "vllm_omni/diffusion/offloader/base.py": "533c793119e4cdc14a2ea3c6cb5bf97ffede673befeab3f0fb8dbbf51e655379", - "vllm_omni/diffusion/offloader/distributed_layerwise_backend.py": "d441481513aba9e90a91265f4bba3f53f8766197be8c1302c3c714e690d5ea76", - "vllm_omni/engine/arg_utils.py": "52569e3688cc708729a48ca0a2c875c28193733c2f934f6ff89f6c815e4009c7", - "vllm_omni/entrypoints/cli/serve.py": "350461dfca565bc0b07087ac7af1e27b0e247f5eac717e350b637e37f62dd088", - "tests/diffusion/offloader/helpers.py": "729c73abba3b9a12911282d4bcd3cf8bfa148fbcbfdc2e2b7c86000e387fa687" - }, - "scope": "CPU mock barrier/chunk tests; no actual NPU collective or full-model quality/performance acceptance", - "status": "failed", - "returncode": 1, - "elapsed_seconds": 62.34616496198578 - }, - { - "trial": "c13-dlo-regression-v2-12c217d5", - "cleanup_passed": true, - "source_unchanged": true, - "native_test_executed": false, - "files_before": { - "tests/diffusion/distributed/test_collective_gate.py": "2b33f73dff9c79118cc2b1bd4d50e3542ebc45b7bf029b38c17abbef37e4b917", - "vllm_omni/config/omni_config.py": "6894120a330474f718d43bf2bf7928892eb996152839ce6469c34d18a7df09d1", - "vllm_omni/config/stage_config.py": "0752e8cc1c6f1021f737830d08663cfc118923a83fd909c607f9ec3ab2a3d2fb", - "vllm_omni/diffusion/attention/layer.py": "76c5fce3dabd23e39a8890316e1691c03bc203066cb60af5611ec4790eefc632", - "vllm_omni/diffusion/data.py": "5db83d5c71cb27cc7810d35fcfc891da80f0b003adbb16bdecc3816a63305e11", - "vllm_omni/diffusion/distributed/collective_gate.py": "01c6317cfa615840681b8bb8b8513fa3736d26e845a24ad9e1ab7f87489b962f", - "vllm_omni/diffusion/offloader/base.py": "533c793119e4cdc14a2ea3c6cb5bf97ffede673befeab3f0fb8dbbf51e655379", - "vllm_omni/diffusion/offloader/distributed_layerwise_backend.py": "d441481513aba9e90a91265f4bba3f53f8766197be8c1302c3c714e690d5ea76", - "vllm_omni/engine/arg_utils.py": "52569e3688cc708729a48ca0a2c875c28193733c2f934f6ff89f6c815e4009c7", - "vllm_omni/entrypoints/cli/serve.py": "350461dfca565bc0b07087ac7af1e27b0e247f5eac717e350b637e37f62dd088", - "tests/diffusion/offloader/helpers.py": "729c73abba3b9a12911282d4bcd3cf8bfa148fbcbfdc2e2b7c86000e387fa687" - }, - "scope": "Expanded offloader tests in Ascend environment; some stock accelerator fixtures use NPU, not a full model test", - "status": "failed", - "returncode": 1, - "elapsed_seconds": 65.14432259398745 - }, - { - "trial": "c14-int8-prerequisites-fdd6633b", - "cleanup_passed": true, - "source_unchanged": true, - "helpers_unchanged": true, - "scope": "Bounded native prerequisites only; a failed probe is retained as negative evidence.", - "status": "failed", - "returncode": 1, - "elapsed_seconds": 20.795623040001374, - "native_prerequisite": { - "verdict": "FAIL", - "scope": "TorchNPU block-INT8 attention prerequisites only", - "phase": "dynamic_block_int8_quant_query", - "versions": { - "torch": "2.10.0+cpu", - "torch_npu": "2.10.0.post4", - "mindiesd": "3.1.0" - }, - "apis": { - "float8_e4m3fn": true, - "npu_dynamic_block_quant": true, - "npu_fused_infer_attention_score_v2": true - }, - "device_name": "Ascend910_9392", - "input": { - "shape": [ - 1, - 14, - 256, - 128 - ], - "dtype": "torch.bfloat16", - "visible_devices": "2" - }, - "error_type": "RuntimeError", - "probe_sha256": "fab18252f860231f9b7fcaa54587afbaa96fce15987fd551922246c8964f7217", - "error": "npu_dynamic_block_quant:../third_party/op-plugin/op_plugin/ops/opapi/DynamicBlockQuantNpuOpApi.cpp:61 NPU function error: call aclnnDynamicBlockQuantV2 failed, error code is 561002\n[ERROR] 2026-11-02-02:00:57 (PID:2662830, Device:0, RankID:-1) ERR00100 PTA call acl api failed.\n[PID: 2662830] 2026-11-02-02:00:57.039.986 Invalid_Argument(EZ0024): Parameter row_block_size of DynamicBlockQuant has incorrect value 128. It should be 1.\n Solution: Check whether the parameter value is correct.\nTraceBack (most recent call last):\n The attr get failed.[FUNC:CheckParams][FILE:dynamic_block_quant_i8_tiling.cpp][LINE:78]\n The params check failed.[FUNC:Tiling4DynamicBlockQuant][FILE:dynamic_block_quant_tiling.cpp][LINE:564]\n Failed to execute operator DynamicBlockQuant_0. Reason: Failed to execute tiling function.\n Check NnopbaseExecutorDoTiling(executor) failed\n Check NnopbaseExecutorTilingAndUpdateBinInfo(executor) failed\n Check NnopbaseExecutorMatchCache(executor) failed\n Check NnopbaseRunForWorkspace(*executor, workspaceSize) failed\n" - } - }, - { - "trial": "c14-quant-prerequisites-358b9b28", - "cleanup_passed": true, - "source_unchanged": true, - "helpers_unchanged": true, - "scope": "Bounded native prerequisites only; a failed probe is retained as negative evidence.", - "status": "failed", - "returncode": 1, - "elapsed_seconds": 20.143024684977718, - "native_prerequisite": { - "verdict": "FAIL", - "scope": "TorchNPU FP8 prerequisites only", - "phase": "dynamic_block_quant_query", - "versions": { - "torch": "2.10.0+cpu", - "torch_npu": "2.10.0.post4", - "mindiesd": "3.1.0" - }, - "apis": { - "float8_e4m3fn": true, - "npu_dynamic_block_quant": true, - "npu_fused_infer_attention_score_v2": true - }, - "device_name": "Ascend910_9392", - "input": { - "shape": [ - 1, - 14, - 256, - 128 - ], - "dtype": "torch.bfloat16", - "visible_devices": "2" - }, - "error_type": "RuntimeError", - "probe_sha256": "df6465956e6f73bbca140fb9880057e30869fd4b7976b2dc091f71a575dd1096", - "error": "npu_dynamic_block_quant:../third_party/op-plugin/op_plugin/ops/opapi/DynamicBlockQuantNpuOpApi.cpp:61 NPU function error: call aclnnDynamicBlockQuantV2 failed, error code is 161002\n[ERROR] 2026-11-02-01:45:24 (PID:2491012, Device:0, RankID:-1) ERR00100 PTA call acl api failed.\n[PID: 2491012] 2026-11-02-01:45:24.732.576 Execution_Error(EZ1009): Failed to execute operator DynamicBlockQuant_0. Reason: The dtype or format of the actual input or output parameter of the operator is inconsistent with that defined in the operator prototype OpDef, details of parameter are x(DT_BFLOAT16, ND), yOut(DT_FLOAT8_E4M3FN, ND), scaleOut(DT_FLOAT, ND).\nTraceBack (most recent call last):\n Supported dtype and format list is:\n SupportInfo[0]: x(DT_FLOAT16, ND), yOut(DT_INT8, ND), scaleOut(DT_FLOAT, ND)\n SupportInfo[1]: x(DT_BFLOAT16, ND), yOut(DT_INT8, ND), scaleOut(DT_FLOAT, ND)\n Check NnopbaseExecutorTilingAndUpdateBinInfo(executor) failed\n Check NnopbaseExecutorMatchCache(executor) failed\n Check NnopbaseRunForWorkspace(*executor, workspaceSize) failed\n" - } - }, - { - "trial": "c15-video-row-native-066a9a40", - "cleanup_passed": true, - "source_unchanged": true, - "required_native_test_executed": true, - "files_before": { - "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "e35b551b8fc0ec3b0ba1384b4df96a8daa0e5179c5ed39d641125310b2ee957f", - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ca87df1936fd2694fcc768079e9e8f926b6d45969ddc61ebf353f5e726d70b8b", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "1258655fcdca4692c2306100846761a2b0b149bbcaa5cde48a9abaa559913bc3", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "a8257f98c2e30c7bc7978bf645966d2eaa98cd49a42a8a923c062b5808dcdcdf", - "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "6f890fcf84047bc5bc084f7b756a2b02e72573888a3bcc12ba7d0211e1bb62ef", - "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "d2a5bed10620a337ff9aaf99cf4a3740d94a8962ef903bf2da75d65e3b50e21c" - }, - "scope": "Focused state/block tests and native four-rank subset attention; no full-model quality/performance acceptance", - "status": "failed", - "returncode": 1, - "elapsed_seconds": 82.27795292000519 - }, - { - "trial": "c15-video-row-native-3a9721a6", - "cleanup_passed": true, - "source_unchanged": true, - "required_native_test_executed": true, - "files_before": { - "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "cf9fb13c5f24c7595d62d13cae6e283cafb2178612b20028d35357f0cafaac83", - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ca87df1936fd2694fcc768079e9e8f926b6d45969ddc61ebf353f5e726d70b8b", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "1258655fcdca4692c2306100846761a2b0b149bbcaa5cde48a9abaa559913bc3", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "a8257f98c2e30c7bc7978bf645966d2eaa98cd49a42a8a923c062b5808dcdcdf", - "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "c5fab405ee82bc553e9a902fa27e6dfbc5213ec18311e17b6415396a69b50f43", - "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "8ec1b3ae4cd252f5d95ab23e0232b983900939e8cb15d1e64f4740f7204abf39", - "vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py": "e8e2e74fb8edb05e02d3e2328ea1f3ad5ddcfd7a10ed09c5cf3911b1d063e849" - }, - "scope": "Focused state/block tests and native four-rank subset attention; no full-model quality/performance acceptance", - "status": "completed", - "returncode": 0, - "elapsed_seconds": 81.91829407500336 - }, - { - "trial": "c15-video-row-native-a2a224de", - "cleanup_passed": true, - "source_unchanged": true, - "required_native_test_executed": true, - "files_before": { - "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "cf9fb13c5f24c7595d62d13cae6e283cafb2178612b20028d35357f0cafaac83", - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ca87df1936fd2694fcc768079e9e8f926b6d45969ddc61ebf353f5e726d70b8b", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "1258655fcdca4692c2306100846761a2b0b149bbcaa5cde48a9abaa559913bc3", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "a8257f98c2e30c7bc7978bf645966d2eaa98cd49a42a8a923c062b5808dcdcdf", - "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "118afcc28379dd73feec763fbb9cfc9f17c6a208afe5969a85b3c969b8059552", - "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "875d0e70621fc76906be130519d7bb1242de5dd58159e31025e26da37e60d98c" - }, - "scope": "Focused state/block tests and native four-rank subset attention; no full-model quality/performance acceptance", - "status": "failed", - "returncode": 1, - "elapsed_seconds": 82.86056435096543 - }, - { - "trial": "c15-video-row-native-bb5084cc", - "cleanup_passed": true, - "source_unchanged": true, - "required_native_test_executed": true, - "files_before": { - "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "cf9fb13c5f24c7595d62d13cae6e283cafb2178612b20028d35357f0cafaac83", - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ca87df1936fd2694fcc768079e9e8f926b6d45969ddc61ebf353f5e726d70b8b", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "1258655fcdca4692c2306100846761a2b0b149bbcaa5cde48a9abaa559913bc3", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "a8257f98c2e30c7bc7978bf645966d2eaa98cd49a42a8a923c062b5808dcdcdf", - "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "118afcc28379dd73feec763fbb9cfc9f17c6a208afe5969a85b3c969b8059552", - "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "fa1ac8fd375d754503b3507fae006381138d8875afdad2418f1ea5ae338b9c26" - }, - "scope": "Focused state/block tests and native four-rank subset attention; no full-model quality/performance acceptance", - "status": "completed", - "returncode": 0, - "elapsed_seconds": 81.50971999001922 - }, - { - "trial": "c15-video-row-quality-5s-00-710315e0", - "model_controller": { - "status": "failed", - "returncode": 1, - "elapsed_seconds": 153.44603093498154 - }, - "actual_counters": { - "request-000\\rank-0\\attention_path_counts.json": {}, - "request-000\\rank-1\\attention_path_counts.json": {}, - "request-000\\rank-2\\attention_path_counts.json": {}, - "request-000\\rank-3\\attention_path_counts.json": {} - } - }, - { - "trial": "c15-video-row-v2-quality-5s-00-d6b921b9", - "model_controller": { - "status": "failed", - "returncode": 1, - "elapsed_seconds": 199.99670764501207 - }, - "actual_counters": { - "request-000\\rank-0\\video_row_cache_stats.json": { - "request_counters": { - "full_steps": 5, - "partial_steps": 1, - "full_blocks": 250, - "partial_blocks": 1, - "subset_attention_calls": 1, - "subset_query_rows": 2475, - "omitted_query_rows": 37296, - "partial_mlp_rows": 2475, - "cache_bytes": 4019635200, - "closed": true - }, - "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" - }, - "request-000\\rank-1\\video_row_cache_stats.json": { - "request_counters": { - "full_steps": 5, - "partial_steps": 1, - "full_blocks": 250, - "partial_blocks": 1, - "subset_attention_calls": 1, - "subset_query_rows": 2475, - "omitted_query_rows": 37296, - "partial_mlp_rows": 0, - "cache_bytes": 5350195200, - "closed": true - }, - "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" - }, - "request-000\\rank-2\\video_row_cache_stats.json": { - "request_counters": { - "full_steps": 5, - "partial_steps": 1, - "full_blocks": 250, - "partial_blocks": 1, - "subset_attention_calls": 1, - "subset_query_rows": 2475, - "omitted_query_rows": 37296, - "partial_mlp_rows": 0, - "cache_bytes": 5350195200, - "closed": true - }, - "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" - }, - "request-000\\rank-3\\video_row_cache_stats.json": { - "request_counters": { - "full_steps": 5, - "partial_steps": 1, - "full_blocks": 250, - "partial_blocks": 1, - "subset_attention_calls": 1, - "subset_query_rows": 2475, - "omitted_query_rows": 37296, - "partial_mlp_rows": 0, - "cache_bytes": 5330304000, - "closed": true - }, - "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" - }, - "request-000\\rank-0\\attention_path_counts.json": { - "_prefix_bnsd": 251 - }, - "request-000\\rank-1\\attention_path_counts.json": { - "_prefix_bnsd": 251 - }, - "request-000\\rank-2\\attention_path_counts.json": { - "_prefix_bnsd": 251 - }, - "request-000\\rank-3\\attention_path_counts.json": { - "_prefix_bnsd": 251 - } - } - }, - { - "trial": "c15-video-row-v3-quality-5s-00-70703586", - "model_controller": { - "status": "failed", - "returncode": 1, - "elapsed_seconds": 191.09544755500974 - }, - "actual_counters": { - "request-000\\rank-0\\video_row_cache_stats.json": { - "request_counters": { - "full_steps": 5, - "partial_steps": 1, - "full_blocks": 250, - "partial_blocks": 50, - "subset_attention_calls": 50, - "subset_query_rows": 123750, - "omitted_query_rows": 1864800, - "partial_mlp_rows": 123750, - "cache_bytes": 4019635200, - "closed": true - }, - "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" - }, - "request-000\\rank-1\\video_row_cache_stats.json": { - "request_counters": { - "full_steps": 5, - "partial_steps": 1, - "full_blocks": 250, - "partial_blocks": 50, - "subset_attention_calls": 50, - "subset_query_rows": 123750, - "omitted_query_rows": 1864800, - "partial_mlp_rows": 0, - "cache_bytes": 5350195200, - "closed": true - }, - "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" - }, - "request-000\\rank-2\\video_row_cache_stats.json": { - "request_counters": { - "full_steps": 5, - "partial_steps": 1, - "full_blocks": 250, - "partial_blocks": 50, - "subset_attention_calls": 50, - "subset_query_rows": 123750, - "omitted_query_rows": 1864800, - "partial_mlp_rows": 0, - "cache_bytes": 5350195200, - "closed": true - }, - "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" - }, - "request-000\\rank-3\\video_row_cache_stats.json": { - "request_counters": { - "full_steps": 5, - "partial_steps": 1, - "full_blocks": 250, - "partial_blocks": 50, - "subset_attention_calls": 50, - "subset_query_rows": 123750, - "omitted_query_rows": 1864800, - "partial_mlp_rows": 0, - "cache_bytes": 5330304000, - "closed": true - }, - "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" - }, - "request-000\\rank-0\\attention_path_counts.json": { - "_prefix_bnsd": 300 - }, - "request-000\\rank-1\\attention_path_counts.json": { - "_prefix_bnsd": 300 - }, - "request-000\\rank-2\\attention_path_counts.json": { - "_prefix_bnsd": 300 - }, - "request-000\\rank-3\\attention_path_counts.json": { - "_prefix_bnsd": 300 - } - } - }, - { - "trial": "c16-video-row-fp32-native-d3fe320c", - "cleanup_passed": true, - "source_unchanged": true, - "required_native_test_executed": true, - "files_before": { - "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "c555d76f7bd6457a81238fca56100d11cb7ab043a4a649d55e23b6203d09bde2", - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ca87df1936fd2694fcc768079e9e8f926b6d45969ddc61ebf353f5e726d70b8b", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "1258655fcdca4692c2306100846761a2b0b149bbcaa5cde48a9abaa559913bc3", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "a8257f98c2e30c7bc7978bf645966d2eaa98cd49a42a8a923c062b5808dcdcdf", - "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "c5fab405ee82bc553e9a902fa27e6dfbc5213ec18311e17b6415396a69b50f43", - "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "e09b872f6900e821d42632a0ca09595f2affe6f9794049dbc74d69ba6a52b2b0", - "vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py": "e8e2e74fb8edb05e02d3e2328ea1f3ad5ddcfd7a10ed09c5cf3911b1d063e849" - }, - "scope": "Focused state/block tests and native four-rank subset attention; no full-model quality/performance acceptance", - "status": "completed", - "returncode": 0, - "elapsed_seconds": 78.8189791989862 - }, - { - "trial": "c16-video-row-fp32-quality-5s-00-8600757a", - "model_controller": { - "status": "completed", - "returncode": 0, - "elapsed_seconds": 355.51310362201184 - }, - "model_diagnostic": { - "source_head": "e0a3462fcc44de698b5dedcf8ff2c99f9802f757", - "samples_ms": [ - 197716.36760496767 - ], - "latency_median_ms": 197716.36760496767, - "scope": "http_complete_mp4", - "warmup": 0, - "repetitions": 1, - "kind": "quality", - "diagnostic": "audit", - "clean_timing": false, - "cleanup_passed": true - }, - "actual_counters": { - "request-000\\rank-0\\video_row_cache_stats.json": { - "request_counters": { - "full_steps": 17, - "partial_steps": 33, - "full_blocks": 850, - "partial_blocks": 1650, - "subset_attention_calls": 1650, - "subset_query_rows": 4083750, - "omitted_query_rows": 61538400, - "partial_mlp_rows": 4083750, - "peak_cache_bytes": 8039270400, - "closed": true, - "storage_dtype": "torch.float32", - "live_cache_bytes": 0 - }, - "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" - }, - "request-000\\rank-1\\video_row_cache_stats.json": { - "request_counters": { - "full_steps": 17, - "partial_steps": 33, - "full_blocks": 850, - "partial_blocks": 1650, - "subset_attention_calls": 1650, - "subset_query_rows": 4083750, - "omitted_query_rows": 61538400, - "partial_mlp_rows": 0, - "peak_cache_bytes": 10700390400, - "closed": true, - "storage_dtype": "torch.float32", - "live_cache_bytes": 0 - }, - "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" - }, - "request-000\\rank-2\\video_row_cache_stats.json": { - "request_counters": { - "full_steps": 17, - "partial_steps": 33, - "full_blocks": 850, - "partial_blocks": 1650, - "subset_attention_calls": 1650, - "subset_query_rows": 4083750, - "omitted_query_rows": 61538400, - "partial_mlp_rows": 0, - "peak_cache_bytes": 10700390400, - "closed": true, - "storage_dtype": "torch.float32", - "live_cache_bytes": 0 - }, - "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" - }, - "request-000\\rank-3\\video_row_cache_stats.json": { - "request_counters": { - "full_steps": 17, - "partial_steps": 33, - "full_blocks": 850, - "partial_blocks": 1650, - "subset_attention_calls": 1650, - "subset_query_rows": 4083750, - "omitted_query_rows": 61538400, - "partial_mlp_rows": 0, - "peak_cache_bytes": 10660608000, - "closed": true, - "storage_dtype": "torch.float32", - "live_cache_bytes": 0 - }, - "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" - }, - "request-000\\rank-0\\attention_path_counts.json": { - "_prefix_bnsd": 2500 - }, - "request-000\\rank-1\\attention_path_counts.json": { - "_prefix_bnsd": 2500 - }, - "request-000\\rank-2\\attention_path_counts.json": { - "_prefix_bnsd": 2500 - }, - "request-000\\rank-3\\attention_path_counts.json": { - "_prefix_bnsd": 2500 - } - } - }, - { - "trial": "c17-video-window-native-9b48cb56", - "cleanup_passed": true, - "source_unchanged": true, - "required_native_test_executed": true, - "files_before": { - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "66ca27991ce42ce302e5de078be6395c9d1ab53752378babef25089f3f75c3aa", - "tests/diffusion/models/minimax_h3/test_minimax_h3_video_window.py": "0bc59f35eff41164e125a5070e034c94ea60bf0112733ebe59eb6378eccbfed8" - }, - "scope": "CPU window reference/routing and native batched BF16 FAS; no full-model quality/performance acceptance", - "status": "completed", - "returncode": 0, - "elapsed_seconds": 51.878022204036824 - }, - { - "trial": "c17-video-window-operator-9bf7de3e", - "cleanup_passed": true, - "source_unchanged": true, - "files_before": { - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "66ca27991ce42ce302e5de078be6395c9d1ab53752378babef25089f3f75c3aa", - "tests/diffusion/models/minimax_h3/test_minimax_h3_video_window.py": "0bc59f35eff41164e125a5070e034c94ea60bf0112733ebe59eb6378eccbfed8" - }, - "scope": "Real captured H3 attention operator benchmark; full-model quality and HTTP are separate; no full-model quality/performance acceptance", - "status": "completed", - "returncode": 0, - "elapsed_seconds": 51.778123575029895, - "operator_benchmark": { - "scope": "real captured 5s attention operation on logical2; no model quality or HTTP acceptance", - "fixture_sha256": "00767e6afb2ba777bb2677eb9b60dc2f00fb649f5006ae8c1d77191214fbf912", - "fixture_tensor_sha256": { - "query": "313095dd143adf0636d57c90d77b689405dee510df3ebd6385a49d071c3323e9", - "key": "6638aee85f0039ee814206e7c976a2d1a9a2b185723c7bbb74c69b1e68c5bad4", - "value": "101eede996c8b13d61d487019d47f0b5fd2193664d8c178665063675cb66fdca" - }, - "query_shape": [ - 1, - 39808, - 14, - 128 - ], - "valid_kv": 39771, - "frames": 37, - "rows_per_frame": 1008, - "target_start": 2475, - "window": 3, - "finite": true, - "relative_l2_vs_dense": 0.5161040425300598, - "baseline_samples_ms": [ - 69.0619180095382, - 69.60304302629083, - 69.27597103640437, - 69.81903495034203, - 69.96654602698982 - ], - "candidate_samples_ms": [ - 25.191069988068193, - 25.301200977992266, - 25.325032009277493, - 25.26562102138996, - 25.219489994924515 - ], - "baseline_median_ms": 69.60304302629083, - "candidate_median_ms": 25.26562102138996, - "reduction_pct": 63.70040744936038, - "operator_gate_passed": true, - "warmups_per_operation": 2, - "measured_per_operation": 5, - "order": "AB/BA/AB/BA/AB", - "probe_sha256": "2465cf08173950a241ac3cc4713a4227d4c4b5a87f35358692a7ef6b60821824" - } - }, - { - "trial": "c17-video-window-quality-5s-00-a8c689e4", - "model_controller": { - "status": "completed", - "returncode": 0, - "elapsed_seconds": 358.6752056070254 - }, - "model_diagnostic": { - "source_head": "c1589a2b5e4ada617e7cca38ef7c300fe9495c58", - "samples_ms": [ - 210397.12271699682 - ], - "latency_median_ms": 210397.12271699682, - "scope": "http_complete_mp4", - "warmup": 0, - "repetitions": 1, - "kind": "quality", - "diagnostic": "audit", - "clean_timing": false, - "cleanup_passed": true - }, - "actual_counters": { - "request-000\\rank-0\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 4900, - "_video_temporal_window_bnsd": 2400 - }, - "request-000\\rank-1\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 4900, - "_video_temporal_window_bnsd": 2400 - }, - "request-000\\rank-2\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 4900, - "_video_temporal_window_bnsd": 2400 - }, - "request-000\\rank-3\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 4900, - "_video_temporal_window_bnsd": 2400 - } - } - }, - { - "trial": "c18-video-pooled-context-native-1673db72", - "cleanup_passed": true, - "source_unchanged": true, - "required_native_test_executed": true, - "files_before": { - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "1febc030a7917b6e10fe287014fe2aacef76ade7bf58ebb604fdedc36fb7c21d", - "tests/diffusion/models/minimax_h3/test_minimax_h3_pooled_context.py": "45a9ae5ba56858daee6a1088a8c8b1ed612ea9afd3c229c90a9b6db3c80a06b2" - }, - "scope": "Independent weighted global-context reference and native padded-head BF16 FAS; no full-model quality/performance acceptance", - "status": "completed", - "returncode": 0, - "elapsed_seconds": 49.97671151399845 - }, - { - "trial": "c18-video-pooled-context-operator-315db0e5", - "cleanup_passed": true, - "source_unchanged": true, - "files_before": { - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "1febc030a7917b6e10fe287014fe2aacef76ade7bf58ebb604fdedc36fb7c21d", - "tests/diffusion/models/minimax_h3/test_minimax_h3_pooled_context.py": "45a9ae5ba56858daee6a1088a8c8b1ed612ea9afd3c229c90a9b6db3c80a06b2" - }, - "scope": "Real captured H3 attention operator benchmark; full-model quality and HTTP are separate; no full-model quality/performance acceptance", - "status": "completed", - "returncode": 0, - "elapsed_seconds": 51.37963652302278, - "operator_benchmark": { - "scope": "real captured 5s attention operation on logical2; no model quality or HTTP acceptance", - "fixture_sha256": "00767e6afb2ba777bb2677eb9b60dc2f00fb649f5006ae8c1d77191214fbf912", - "fixture_tensor_sha256": { - "query": "313095dd143adf0636d57c90d77b689405dee510df3ebd6385a49d071c3323e9", - "key": "6638aee85f0039ee814206e7c976a2d1a9a2b185723c7bbb74c69b1e68c5bad4", - "value": "101eede996c8b13d61d487019d47f0b5fd2193664d8c178665063675cb66fdca" - }, - "query_shape": [ - 1, - 39808, - 14, - 128 - ], - "valid_kv": 39771, - "frames": 37, - "rows_per_frame": 1008, - "target_start": 2475, - "profile": "3x3-local1", - "finite": true, - "relative_l2_vs_dense": 0.21801254153251648, - "baseline_samples_ms": [ - 69.19049099087715, - 69.70348599134013, - 69.43478400353342, - 69.81050700414926, - 69.24652203451842 - ], - "candidate_samples_ms": [ - 68.29233502503484, - 64.26435202592984, - 71.05128600960597, - 64.39994304673746, - 64.39412303734571 - ], - "baseline_median_ms": 69.43478400353342, - "candidate_median_ms": 64.39994304673746, - "reduction_pct": 7.251179691924648, - "operator_gate_passed": false, - "warmups_per_operation": 2, - "measured_per_operation": 5, - "order": "AB/BA/AB/BA/AB", - "probe_sha256": "42852c0e4e693c5b3685981db617de00648feefe651f7060ad83c359b08dd81d" - } - }, - { - "trial": "c19-video-global-pool-native-f0f4c502", - "cleanup_passed": true, - "source_unchanged": true, - "required_native_test_executed": true, - "files_before": { - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ddfd6d6822cc6a5528fc95539f7f98dfdeed25d5489da3ed5cf76e6076a5ec94", - "tests/diffusion/models/minimax_h3/test_minimax_h3_global_pool.py": "5457db550b59e3373be8f5f21873c1b78ac357a6ac170c2603ee531b50e91467" - }, - "scope": "Independent weighted global-context reference and native padded-head BF16 FAS; no full-model quality/performance acceptance", - "status": "completed", - "returncode": 0, - "elapsed_seconds": 51.528875903983135 - }, - { - "trial": "c19-video-global-pool-operator-933511c9", - "cleanup_passed": true, - "source_unchanged": true, - "files_before": { - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ddfd6d6822cc6a5528fc95539f7f98dfdeed25d5489da3ed5cf76e6076a5ec94", - "tests/diffusion/models/minimax_h3/test_minimax_h3_global_pool.py": "5457db550b59e3373be8f5f21873c1b78ac357a6ac170c2603ee531b50e91467" - }, - "scope": "Real captured H3 attention operator benchmark; full-model quality and HTTP are separate; no full-model quality/performance acceptance", - "status": "completed", - "returncode": 0, - "elapsed_seconds": 51.02866222098237, - "operator_benchmark": { - "scope": "real captured 5s attention operation on logical2; no model quality or HTTP acceptance", - "fixture_sha256": "00767e6afb2ba777bb2677eb9b60dc2f00fb649f5006ae8c1d77191214fbf912", - "fixture_tensor_sha256": { - "query": "313095dd143adf0636d57c90d77b689405dee510df3ebd6385a49d071c3323e9", - "key": "6638aee85f0039ee814206e7c976a2d1a9a2b185723c7bbb74c69b1e68c5bad4", - "value": "101eede996c8b13d61d487019d47f0b5fd2193664d8c178665063675cb66fdca" - }, - "query_shape": [ - 1, - 39808, - 14, - 128 - ], - "valid_kv": 39771, - "frames": 37, - "rows_per_frame": 1008, - "target_start": 2475, - "profile": "3x3-global", - "finite": true, - "relative_l2_vs_dense": 0.2588050067424774, - "baseline_samples_ms": [ - 69.95450798422098, - 69.79658600175753, - 69.58012498216704, - 70.22821099963039, - 69.11802099784836 - ], - "candidate_samples_ms": [ - 32.91054297005758, - 32.899541954975575, - 32.924533006735146, - 32.779561006464064, - 32.96928299823776 - ], - "baseline_median_ms": 69.79658600175753, - "candidate_median_ms": 32.91054297005758, - "reduction_pct": 52.847918708761966, - "operator_gate_passed": false, - "warmups_per_operation": 2, - "measured_per_operation": 5, - "order": "AB/BA/AB/BA/AB", - "probe_sha256": "a27906d8bb0b65e2f6ffb41ddeb4b513163b7bf23b9d2dbd33a5015fc93b2a08" - } - }, - { - "trial": "c20-video-variance-pool-native-50101063", - "cleanup_passed": true, - "source_unchanged": true, - "required_native_test_executed": true, - "files_before": { - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "4390450bc5324f7028125101cd4bb32e9de237b04bc12fa879b48110d7122e3c", - "tests/diffusion/models/minimax_h3/test_minimax_h3_variance_pool.py": "e871fb4b222d61d10b6c442a333a52c976cceaa0d406233502ecbcaddfeba8c9" - }, - "scope": "Independent weighted global-context reference and native padded-head BF16 FAS; no full-model quality/performance acceptance", - "status": "completed", - "returncode": 0, - "elapsed_seconds": 50.230675689992495 - }, - { - "trial": "c20-video-variance-pool-operator-209f263d", - "cleanup_passed": true, - "source_unchanged": true, - "files_before": { - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "4390450bc5324f7028125101cd4bb32e9de237b04bc12fa879b48110d7122e3c", - "tests/diffusion/models/minimax_h3/test_minimax_h3_variance_pool.py": "e871fb4b222d61d10b6c442a333a52c976cceaa0d406233502ecbcaddfeba8c9" - }, - "scope": "Real captured H3 attention operator benchmark; full-model quality and HTTP are separate; no full-model quality/performance acceptance", - "status": "completed", - "returncode": 0, - "elapsed_seconds": 49.880220040970016, - "operator_benchmark": { - "scope": "real captured 5s attention operation on logical2; no model quality or HTTP acceptance", - "fixture_sha256": "00767e6afb2ba777bb2677eb9b60dc2f00fb649f5006ae8c1d77191214fbf912", - "fixture_tensor_sha256": { - "query": "313095dd143adf0636d57c90d77b689405dee510df3ebd6385a49d071c3323e9", - "key": "6638aee85f0039ee814206e7c976a2d1a9a2b185723c7bbb74c69b1e68c5bad4", - "value": "101eede996c8b13d61d487019d47f0b5fd2193664d8c178665063675cb66fdca" - }, - "query_shape": [ - 1, - 39808, - 14, - 128 - ], - "valid_kv": 39771, - "frames": 37, - "rows_per_frame": 1008, - "target_start": 2475, - "profile": "3x3-variance", - "finite": true, - "relative_l2_vs_dense": 0.26336145401000977, - "baseline_samples_ms": [ - 69.74446203093976, - 69.06059599714354, - 69.64961998164654, - 69.69038100214675, - 69.62426099926233 - ], - "candidate_samples_ms": [ - 35.81449302146211, - 36.30960697773844, - 35.925014002714306, - 35.99672397831455, - 35.80518299713731 - ], - "baseline_median_ms": 69.64961998164654, - "candidate_median_ms": 35.925014002714306, - "reduction_pct": 48.42037327385138, - "operator_gate_passed": false, - "warmups_per_operation": 2, - "measured_per_operation": 5, - "order": "AB/BA/AB/BA/AB", - "probe_sha256": "785e7a0c13719e37d5cc44fb66af05649e51e1f7e5d0b212e92ec44f79cb0184" - } - } - ], - "decisions": [ - "C12 fixed profile stopped: zero actual predictions.", - "C14 pinned block-FP8 and block-INT8 prerequisites failed; per-token INT8 is not excluded.", - "C13 numerical qualification and clean performance are separate.", - "C15 fixed FP16 video residual profile failed finite velocity at first reuse.", - "C16 FP32 residual completed but failed three frozen audio quality guards; fixed profile stopped.", - "C17 completed5s but failed11frozen quality checks; fixed window profile stopped.", - "C18 native17PASS but operation reduced only7.25%; no full model run.", - "C19 native17PASS; operation52.85%faster but relL2 .258805 exceeded fixed .258052screen; no model run.", - "C20 native17PASS; operation48.42%faster and relL2 .263361 failed both fixed primitive gates; no model run." - ] -} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c12-through-c21-execution-checkpoint.json b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c12-through-c21-execution-checkpoint.json deleted file mode 100644 index 353fac2a7..000000000 --- a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c12-through-c21-execution-checkpoint.json +++ /dev/null @@ -1,1310 +0,0 @@ -{ - "campaign_verdict": "INCONCLUSIVE", - "performance_acceptance": "No newly accepted gain", - "runs": [ - { - "trial": "c12-alignment-quality-5s-00-9bf5c373", - "model_controller": { - "status": "completed", - "returncode": 0, - "elapsed_seconds": 471.80311120999977 - }, - "model_diagnostic": { - "source_head": "a29648e5acd3077b7b22739bc9522017782fe1ac", - "samples_ms": [ - 318280.59680899605 - ], - "latency_median_ms": 318280.59680899605, - "scope": "http_complete_mp4", - "warmup": 0, - "repetitions": 1, - "kind": "quality", - "diagnostic": "audit", - "clean_timing": false, - "cleanup_passed": true - }, - "actual_counters": { - "request-000\\rank-0\\alignment_cache_stats.json": { - "request_counters": { - "computed": 50, - "predicted": 0, - "vetoes": 45 - }, - "scope": "actual state-aligned compute/forecast/veto counts; not quality acceptance" - }, - "request-000\\rank-1\\alignment_cache_stats.json": { - "request_counters": { - "computed": 50, - "predicted": 0, - "vetoes": 45 - }, - "scope": "actual state-aligned compute/forecast/veto counts; not quality acceptance" - }, - "request-000\\rank-2\\alignment_cache_stats.json": { - "request_counters": { - "computed": 50, - "predicted": 0, - "vetoes": 45 - }, - "scope": "actual state-aligned compute/forecast/veto counts; not quality acceptance" - }, - "request-000\\rank-3\\alignment_cache_stats.json": { - "request_counters": { - "computed": 50, - "predicted": 0, - "vetoes": 45 - }, - "scope": "actual state-aligned compute/forecast/veto counts; not quality acceptance" - }, - "request-000\\rank-0\\attention_path_counts.json": { - "_prefix_bnsd": 2500 - }, - "request-000\\rank-1\\attention_path_counts.json": { - "_prefix_bnsd": 2500 - }, - "request-000\\rank-2\\attention_path_counts.json": { - "_prefix_bnsd": 2500 - }, - "request-000\\rank-3\\attention_path_counts.json": { - "_prefix_bnsd": 2500 - } - } - }, - { - "trial": "c12-state-alignment-native-tests-0dc5859b", - "cleanup_passed": true, - "source_unchanged": true, - "native_test_executed": true, - "files_before": { - "vllm_omni/diffusion/models/minimax_h3/cache_state_alignment.py": "3340ec21a77aef78531c601be241bcfb1e44107c795e8c8649907d1a18ed377f", - "vllm_omni/diffusion/models/minimax_h3/reference_kv_cache_dit.py": "f4cd0e9aa9e824c92ec1acd02c57a8f7d8684c35d4edf92ffdfae3345ff44e99", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "436ec52cd6329cbc64e91d94b450bfa6f773a9a953b8f974db73a5eb986a98bb", - "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "4dec8f02a04bfe7e8d2ada24fc6d5ae40a0058671d8515c8a266ecf88824b7b0", - "tests/diffusion/models/minimax_h3/test_minimax_h3_state_cache.py": "89cf57482b35d8d8ad8d4978aec87e9ee3ef45330e5b911161adef4379950a65" - }, - "scope": "focused tests only; no full-model quality or performance acceptance", - "status": "completed", - "returncode": 0, - "elapsed_seconds": 82.06077561300481 - }, - { - "trial": "c13-baseline-cuda-staging-e22d8651", - "cleanup_passed": true, - "source_unchanged": true, - "native_test_executed": false, - "files_before": { - "vllm_omni/diffusion/offloader/sequential_backend.py": "d542935b656a0d9dced78de8fb086b8e1c5aade26fac6cee5316b32a49f0d569", - "tests/diffusion/offloader/test_sequential_backend.py": "dcf1d64e65bed572d65b095c1350950f55d7e8d9ddf8c7dfd72d0ea6e721685a" - }, - "scope": "Reproduce stock CUDA-graph staging expectation on unchanged migration owning source in the same Ascend test environment", - "status": "failed", - "returncode": 1, - "elapsed_seconds": 50.234599059971515 - }, - { - "trial": "c13-c1-dlo-5s-audit-cead10d1", - "model_controller": { - "status": "failed", - "returncode": 2, - "elapsed_seconds": 0.11969929095357656 - } - }, - { - "trial": "c13-c1-dlo-5s-audit-v2-b0b86c6d", - "model_controller": { - "status": "completed", - "returncode": 0, - "elapsed_seconds": 749.505351218977 - }, - "model_diagnostic": { - "source_head": "e89b1d40d37f7f72767892a56fa7980ea572ba8e", - "samples_ms": [ - 315593.86175096733, - 284456.94282796467 - ], - "latency_median_ms": 300025.402289466, - "scope": "http_complete_mp4", - "warmup": 0, - "repetitions": 2, - "kind": "diagnostic", - "diagnostic": "audit", - "clean_timing": false, - "cleanup_passed": true - }, - "actual_counters": { - "request-000\\rank-0\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-000\\rank-1\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-000\\rank-2\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-000\\rank-3\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-001\\rank-0\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-001\\rank-1\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-001\\rank-2\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-001\\rank-3\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - } - } - }, - { - "trial": "c13-c1-dlo-5s-calibration-c98129d8", - "model_controller": { - "status": "completed", - "returncode": 0, - "elapsed_seconds": 768.8735517460154 - }, - "model_diagnostic": { - "source_head": "e89b1d40d37f7f72767892a56fa7980ea572ba8e", - "samples_ms": [ - 317177.1984040388, - 285534.089674009 - ], - "latency_median_ms": 301355.6440390239, - "scope": "http_complete_mp4", - "warmup": 0, - "repetitions": 2, - "kind": "calibration", - "diagnostic": "audit", - "clean_timing": false, - "cleanup_passed": true - }, - "actual_counters": { - "request-000\\rank-0\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-000\\rank-1\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-000\\rank-2\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-000\\rank-3\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-001\\rank-0\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-001\\rank-1\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-001\\rank-2\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-001\\rank-3\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - } - } - }, - { - "trial": "c13-dlo-gate-tests-bc92a71d", - "cleanup_passed": true, - "source_unchanged": true, - "native_test_executed": false, - "files_before": { - "tests/diffusion/distributed/test_collective_gate.py": "2b33f73dff9c79118cc2b1bd4d50e3542ebc45b7bf029b38c17abbef37e4b917", - "vllm_omni/config/omni_config.py": "6894120a330474f718d43bf2bf7928892eb996152839ce6469c34d18a7df09d1", - "vllm_omni/config/stage_config.py": "0752e8cc1c6f1021f737830d08663cfc118923a83fd909c607f9ec3ab2a3d2fb", - "vllm_omni/diffusion/attention/layer.py": "76c5fce3dabd23e39a8890316e1691c03bc203066cb60af5611ec4790eefc632", - "vllm_omni/diffusion/data.py": "5db83d5c71cb27cc7810d35fcfc891da80f0b003adbb16bdecc3816a63305e11", - "vllm_omni/diffusion/distributed/collective_gate.py": "01c6317cfa615840681b8bb8b8513fa3736d26e845a24ad9e1ab7f87489b962f", - "vllm_omni/diffusion/offloader/base.py": "533c793119e4cdc14a2ea3c6cb5bf97ffede673befeab3f0fb8dbbf51e655379", - "vllm_omni/diffusion/offloader/distributed_layerwise_backend.py": "d441481513aba9e90a91265f4bba3f53f8766197be8c1302c3c714e690d5ea76", - "vllm_omni/engine/arg_utils.py": "52569e3688cc708729a48ca0a2c875c28193733c2f934f6ff89f6c815e4009c7", - "vllm_omni/entrypoints/cli/serve.py": "350461dfca565bc0b07087ac7af1e27b0e247f5eac717e350b637e37f62dd088", - "tests/diffusion/offloader/helpers.py": "729c73abba3b9a12911282d4bcd3cf8bfa148fbcbfdc2e2b7c86000e387fa687" - }, - "scope": "CPU mock barrier/chunk tests; no actual NPU collective or full-model quality/performance acceptance", - "status": "completed", - "returncode": 0, - "elapsed_seconds": 47.3752507569734 - }, - { - "trial": "c13-dlo-regression-tests-95e9ed2a", - "cleanup_passed": true, - "source_unchanged": true, - "native_test_executed": false, - "files_before": { - "tests/diffusion/distributed/test_collective_gate.py": "2b33f73dff9c79118cc2b1bd4d50e3542ebc45b7bf029b38c17abbef37e4b917", - "vllm_omni/config/omni_config.py": "6894120a330474f718d43bf2bf7928892eb996152839ce6469c34d18a7df09d1", - "vllm_omni/config/stage_config.py": "0752e8cc1c6f1021f737830d08663cfc118923a83fd909c607f9ec3ab2a3d2fb", - "vllm_omni/diffusion/attention/layer.py": "76c5fce3dabd23e39a8890316e1691c03bc203066cb60af5611ec4790eefc632", - "vllm_omni/diffusion/data.py": "5db83d5c71cb27cc7810d35fcfc891da80f0b003adbb16bdecc3816a63305e11", - "vllm_omni/diffusion/distributed/collective_gate.py": "01c6317cfa615840681b8bb8b8513fa3736d26e845a24ad9e1ab7f87489b962f", - "vllm_omni/diffusion/offloader/base.py": "533c793119e4cdc14a2ea3c6cb5bf97ffede673befeab3f0fb8dbbf51e655379", - "vllm_omni/diffusion/offloader/distributed_layerwise_backend.py": "d441481513aba9e90a91265f4bba3f53f8766197be8c1302c3c714e690d5ea76", - "vllm_omni/engine/arg_utils.py": "52569e3688cc708729a48ca0a2c875c28193733c2f934f6ff89f6c815e4009c7", - "vllm_omni/entrypoints/cli/serve.py": "350461dfca565bc0b07087ac7af1e27b0e247f5eac717e350b637e37f62dd088", - "tests/diffusion/offloader/helpers.py": "729c73abba3b9a12911282d4bcd3cf8bfa148fbcbfdc2e2b7c86000e387fa687" - }, - "scope": "CPU mock barrier/chunk tests; no actual NPU collective or full-model quality/performance acceptance", - "status": "failed", - "returncode": 1, - "elapsed_seconds": 62.34616496198578 - }, - { - "trial": "c13-dlo-regression-v2-12c217d5", - "cleanup_passed": true, - "source_unchanged": true, - "native_test_executed": false, - "files_before": { - "tests/diffusion/distributed/test_collective_gate.py": "2b33f73dff9c79118cc2b1bd4d50e3542ebc45b7bf029b38c17abbef37e4b917", - "vllm_omni/config/omni_config.py": "6894120a330474f718d43bf2bf7928892eb996152839ce6469c34d18a7df09d1", - "vllm_omni/config/stage_config.py": "0752e8cc1c6f1021f737830d08663cfc118923a83fd909c607f9ec3ab2a3d2fb", - "vllm_omni/diffusion/attention/layer.py": "76c5fce3dabd23e39a8890316e1691c03bc203066cb60af5611ec4790eefc632", - "vllm_omni/diffusion/data.py": "5db83d5c71cb27cc7810d35fcfc891da80f0b003adbb16bdecc3816a63305e11", - "vllm_omni/diffusion/distributed/collective_gate.py": "01c6317cfa615840681b8bb8b8513fa3736d26e845a24ad9e1ab7f87489b962f", - "vllm_omni/diffusion/offloader/base.py": "533c793119e4cdc14a2ea3c6cb5bf97ffede673befeab3f0fb8dbbf51e655379", - "vllm_omni/diffusion/offloader/distributed_layerwise_backend.py": "d441481513aba9e90a91265f4bba3f53f8766197be8c1302c3c714e690d5ea76", - "vllm_omni/engine/arg_utils.py": "52569e3688cc708729a48ca0a2c875c28193733c2f934f6ff89f6c815e4009c7", - "vllm_omni/entrypoints/cli/serve.py": "350461dfca565bc0b07087ac7af1e27b0e247f5eac717e350b637e37f62dd088", - "tests/diffusion/offloader/helpers.py": "729c73abba3b9a12911282d4bcd3cf8bfa148fbcbfdc2e2b7c86000e387fa687" - }, - "scope": "Expanded offloader tests in Ascend environment; some stock accelerator fixtures use NPU, not a full model test", - "status": "failed", - "returncode": 1, - "elapsed_seconds": 65.14432259398745 - }, - { - "trial": "c14-int8-prerequisites-fdd6633b", - "cleanup_passed": true, - "source_unchanged": true, - "helpers_unchanged": true, - "scope": "Bounded native prerequisites only; a failed probe is retained as negative evidence.", - "status": "failed", - "returncode": 1, - "elapsed_seconds": 20.795623040001374, - "native_prerequisite": { - "verdict": "FAIL", - "scope": "TorchNPU block-INT8 attention prerequisites only", - "phase": "dynamic_block_int8_quant_query", - "versions": { - "torch": "2.10.0+cpu", - "torch_npu": "2.10.0.post4", - "mindiesd": "3.1.0" - }, - "apis": { - "float8_e4m3fn": true, - "npu_dynamic_block_quant": true, - "npu_fused_infer_attention_score_v2": true - }, - "device_name": "Ascend910_9392", - "input": { - "shape": [ - 1, - 14, - 256, - 128 - ], - "dtype": "torch.bfloat16", - "visible_devices": "2" - }, - "error_type": "RuntimeError", - "probe_sha256": "fab18252f860231f9b7fcaa54587afbaa96fce15987fd551922246c8964f7217", - "error": "npu_dynamic_block_quant:../third_party/op-plugin/op_plugin/ops/opapi/DynamicBlockQuantNpuOpApi.cpp:61 NPU function error: call aclnnDynamicBlockQuantV2 failed, error code is 561002\n[ERROR] 2026-11-02-02:00:57 (PID:2662830, Device:0, RankID:-1) ERR00100 PTA call acl api failed.\n[PID: 2662830] 2026-11-02-02:00:57.039.986 Invalid_Argument(EZ0024): Parameter row_block_size of DynamicBlockQuant has incorrect value 128. It should be 1.\n Solution: Check whether the parameter value is correct.\nTraceBack (most recent call last):\n The attr get failed.[FUNC:CheckParams][FILE:dynamic_block_quant_i8_tiling.cpp][LINE:78]\n The params check failed.[FUNC:Tiling4DynamicBlockQuant][FILE:dynamic_block_quant_tiling.cpp][LINE:564]\n Failed to execute operator DynamicBlockQuant_0. Reason: Failed to execute tiling function.\n Check NnopbaseExecutorDoTiling(executor) failed\n Check NnopbaseExecutorTilingAndUpdateBinInfo(executor) failed\n Check NnopbaseExecutorMatchCache(executor) failed\n Check NnopbaseRunForWorkspace(*executor, workspaceSize) failed\n" - } - }, - { - "trial": "c14-quant-prerequisites-358b9b28", - "cleanup_passed": true, - "source_unchanged": true, - "helpers_unchanged": true, - "scope": "Bounded native prerequisites only; a failed probe is retained as negative evidence.", - "status": "failed", - "returncode": 1, - "elapsed_seconds": 20.143024684977718, - "native_prerequisite": { - "verdict": "FAIL", - "scope": "TorchNPU FP8 prerequisites only", - "phase": "dynamic_block_quant_query", - "versions": { - "torch": "2.10.0+cpu", - "torch_npu": "2.10.0.post4", - "mindiesd": "3.1.0" - }, - "apis": { - "float8_e4m3fn": true, - "npu_dynamic_block_quant": true, - "npu_fused_infer_attention_score_v2": true - }, - "device_name": "Ascend910_9392", - "input": { - "shape": [ - 1, - 14, - 256, - 128 - ], - "dtype": "torch.bfloat16", - "visible_devices": "2" - }, - "error_type": "RuntimeError", - "probe_sha256": "df6465956e6f73bbca140fb9880057e30869fd4b7976b2dc091f71a575dd1096", - "error": "npu_dynamic_block_quant:../third_party/op-plugin/op_plugin/ops/opapi/DynamicBlockQuantNpuOpApi.cpp:61 NPU function error: call aclnnDynamicBlockQuantV2 failed, error code is 161002\n[ERROR] 2026-11-02-01:45:24 (PID:2491012, Device:0, RankID:-1) ERR00100 PTA call acl api failed.\n[PID: 2491012] 2026-11-02-01:45:24.732.576 Execution_Error(EZ1009): Failed to execute operator DynamicBlockQuant_0. Reason: The dtype or format of the actual input or output parameter of the operator is inconsistent with that defined in the operator prototype OpDef, details of parameter are x(DT_BFLOAT16, ND), yOut(DT_FLOAT8_E4M3FN, ND), scaleOut(DT_FLOAT, ND).\nTraceBack (most recent call last):\n Supported dtype and format list is:\n SupportInfo[0]: x(DT_FLOAT16, ND), yOut(DT_INT8, ND), scaleOut(DT_FLOAT, ND)\n SupportInfo[1]: x(DT_BFLOAT16, ND), yOut(DT_INT8, ND), scaleOut(DT_FLOAT, ND)\n Check NnopbaseExecutorTilingAndUpdateBinInfo(executor) failed\n Check NnopbaseExecutorMatchCache(executor) failed\n Check NnopbaseRunForWorkspace(*executor, workspaceSize) failed\n" - } - }, - { - "trial": "c15-video-row-native-066a9a40", - "cleanup_passed": true, - "source_unchanged": true, - "required_native_test_executed": true, - "files_before": { - "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "e35b551b8fc0ec3b0ba1384b4df96a8daa0e5179c5ed39d641125310b2ee957f", - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ca87df1936fd2694fcc768079e9e8f926b6d45969ddc61ebf353f5e726d70b8b", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "1258655fcdca4692c2306100846761a2b0b149bbcaa5cde48a9abaa559913bc3", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "a8257f98c2e30c7bc7978bf645966d2eaa98cd49a42a8a923c062b5808dcdcdf", - "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "6f890fcf84047bc5bc084f7b756a2b02e72573888a3bcc12ba7d0211e1bb62ef", - "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "d2a5bed10620a337ff9aaf99cf4a3740d94a8962ef903bf2da75d65e3b50e21c" - }, - "scope": "Focused state/block tests and native four-rank subset attention; no full-model quality/performance acceptance", - "status": "failed", - "returncode": 1, - "elapsed_seconds": 82.27795292000519 - }, - { - "trial": "c15-video-row-native-3a9721a6", - "cleanup_passed": true, - "source_unchanged": true, - "required_native_test_executed": true, - "files_before": { - "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "cf9fb13c5f24c7595d62d13cae6e283cafb2178612b20028d35357f0cafaac83", - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ca87df1936fd2694fcc768079e9e8f926b6d45969ddc61ebf353f5e726d70b8b", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "1258655fcdca4692c2306100846761a2b0b149bbcaa5cde48a9abaa559913bc3", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "a8257f98c2e30c7bc7978bf645966d2eaa98cd49a42a8a923c062b5808dcdcdf", - "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "c5fab405ee82bc553e9a902fa27e6dfbc5213ec18311e17b6415396a69b50f43", - "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "8ec1b3ae4cd252f5d95ab23e0232b983900939e8cb15d1e64f4740f7204abf39", - "vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py": "e8e2e74fb8edb05e02d3e2328ea1f3ad5ddcfd7a10ed09c5cf3911b1d063e849" - }, - "scope": "Focused state/block tests and native four-rank subset attention; no full-model quality/performance acceptance", - "status": "completed", - "returncode": 0, - "elapsed_seconds": 81.91829407500336 - }, - { - "trial": "c15-video-row-native-a2a224de", - "cleanup_passed": true, - "source_unchanged": true, - "required_native_test_executed": true, - "files_before": { - "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "cf9fb13c5f24c7595d62d13cae6e283cafb2178612b20028d35357f0cafaac83", - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ca87df1936fd2694fcc768079e9e8f926b6d45969ddc61ebf353f5e726d70b8b", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "1258655fcdca4692c2306100846761a2b0b149bbcaa5cde48a9abaa559913bc3", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "a8257f98c2e30c7bc7978bf645966d2eaa98cd49a42a8a923c062b5808dcdcdf", - "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "118afcc28379dd73feec763fbb9cfc9f17c6a208afe5969a85b3c969b8059552", - "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "875d0e70621fc76906be130519d7bb1242de5dd58159e31025e26da37e60d98c" - }, - "scope": "Focused state/block tests and native four-rank subset attention; no full-model quality/performance acceptance", - "status": "failed", - "returncode": 1, - "elapsed_seconds": 82.86056435096543 - }, - { - "trial": "c15-video-row-native-bb5084cc", - "cleanup_passed": true, - "source_unchanged": true, - "required_native_test_executed": true, - "files_before": { - "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "cf9fb13c5f24c7595d62d13cae6e283cafb2178612b20028d35357f0cafaac83", - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ca87df1936fd2694fcc768079e9e8f926b6d45969ddc61ebf353f5e726d70b8b", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "1258655fcdca4692c2306100846761a2b0b149bbcaa5cde48a9abaa559913bc3", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "a8257f98c2e30c7bc7978bf645966d2eaa98cd49a42a8a923c062b5808dcdcdf", - "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "118afcc28379dd73feec763fbb9cfc9f17c6a208afe5969a85b3c969b8059552", - "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "fa1ac8fd375d754503b3507fae006381138d8875afdad2418f1ea5ae338b9c26" - }, - "scope": "Focused state/block tests and native four-rank subset attention; no full-model quality/performance acceptance", - "status": "completed", - "returncode": 0, - "elapsed_seconds": 81.50971999001922 - }, - { - "trial": "c15-video-row-quality-5s-00-710315e0", - "model_controller": { - "status": "failed", - "returncode": 1, - "elapsed_seconds": 153.44603093498154 - }, - "actual_counters": { - "request-000\\rank-0\\attention_path_counts.json": {}, - "request-000\\rank-1\\attention_path_counts.json": {}, - "request-000\\rank-2\\attention_path_counts.json": {}, - "request-000\\rank-3\\attention_path_counts.json": {} - } - }, - { - "trial": "c15-video-row-v2-quality-5s-00-d6b921b9", - "model_controller": { - "status": "failed", - "returncode": 1, - "elapsed_seconds": 199.99670764501207 - }, - "actual_counters": { - "request-000\\rank-0\\video_row_cache_stats.json": { - "request_counters": { - "full_steps": 5, - "partial_steps": 1, - "full_blocks": 250, - "partial_blocks": 1, - "subset_attention_calls": 1, - "subset_query_rows": 2475, - "omitted_query_rows": 37296, - "partial_mlp_rows": 2475, - "cache_bytes": 4019635200, - "closed": true - }, - "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" - }, - "request-000\\rank-1\\video_row_cache_stats.json": { - "request_counters": { - "full_steps": 5, - "partial_steps": 1, - "full_blocks": 250, - "partial_blocks": 1, - "subset_attention_calls": 1, - "subset_query_rows": 2475, - "omitted_query_rows": 37296, - "partial_mlp_rows": 0, - "cache_bytes": 5350195200, - "closed": true - }, - "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" - }, - "request-000\\rank-2\\video_row_cache_stats.json": { - "request_counters": { - "full_steps": 5, - "partial_steps": 1, - "full_blocks": 250, - "partial_blocks": 1, - "subset_attention_calls": 1, - "subset_query_rows": 2475, - "omitted_query_rows": 37296, - "partial_mlp_rows": 0, - "cache_bytes": 5350195200, - "closed": true - }, - "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" - }, - "request-000\\rank-3\\video_row_cache_stats.json": { - "request_counters": { - "full_steps": 5, - "partial_steps": 1, - "full_blocks": 250, - "partial_blocks": 1, - "subset_attention_calls": 1, - "subset_query_rows": 2475, - "omitted_query_rows": 37296, - "partial_mlp_rows": 0, - "cache_bytes": 5330304000, - "closed": true - }, - "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" - }, - "request-000\\rank-0\\attention_path_counts.json": { - "_prefix_bnsd": 251 - }, - "request-000\\rank-1\\attention_path_counts.json": { - "_prefix_bnsd": 251 - }, - "request-000\\rank-2\\attention_path_counts.json": { - "_prefix_bnsd": 251 - }, - "request-000\\rank-3\\attention_path_counts.json": { - "_prefix_bnsd": 251 - } - } - }, - { - "trial": "c15-video-row-v3-quality-5s-00-70703586", - "model_controller": { - "status": "failed", - "returncode": 1, - "elapsed_seconds": 191.09544755500974 - }, - "actual_counters": { - "request-000\\rank-0\\video_row_cache_stats.json": { - "request_counters": { - "full_steps": 5, - "partial_steps": 1, - "full_blocks": 250, - "partial_blocks": 50, - "subset_attention_calls": 50, - "subset_query_rows": 123750, - "omitted_query_rows": 1864800, - "partial_mlp_rows": 123750, - "cache_bytes": 4019635200, - "closed": true - }, - "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" - }, - "request-000\\rank-1\\video_row_cache_stats.json": { - "request_counters": { - "full_steps": 5, - "partial_steps": 1, - "full_blocks": 250, - "partial_blocks": 50, - "subset_attention_calls": 50, - "subset_query_rows": 123750, - "omitted_query_rows": 1864800, - "partial_mlp_rows": 0, - "cache_bytes": 5350195200, - "closed": true - }, - "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" - }, - "request-000\\rank-2\\video_row_cache_stats.json": { - "request_counters": { - "full_steps": 5, - "partial_steps": 1, - "full_blocks": 250, - "partial_blocks": 50, - "subset_attention_calls": 50, - "subset_query_rows": 123750, - "omitted_query_rows": 1864800, - "partial_mlp_rows": 0, - "cache_bytes": 5350195200, - "closed": true - }, - "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" - }, - "request-000\\rank-3\\video_row_cache_stats.json": { - "request_counters": { - "full_steps": 5, - "partial_steps": 1, - "full_blocks": 250, - "partial_blocks": 50, - "subset_attention_calls": 50, - "subset_query_rows": 123750, - "omitted_query_rows": 1864800, - "partial_mlp_rows": 0, - "cache_bytes": 5330304000, - "closed": true - }, - "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" - }, - "request-000\\rank-0\\attention_path_counts.json": { - "_prefix_bnsd": 300 - }, - "request-000\\rank-1\\attention_path_counts.json": { - "_prefix_bnsd": 300 - }, - "request-000\\rank-2\\attention_path_counts.json": { - "_prefix_bnsd": 300 - }, - "request-000\\rank-3\\attention_path_counts.json": { - "_prefix_bnsd": 300 - } - } - }, - { - "trial": "c16-video-row-fp32-native-d3fe320c", - "cleanup_passed": true, - "source_unchanged": true, - "required_native_test_executed": true, - "files_before": { - "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "c555d76f7bd6457a81238fca56100d11cb7ab043a4a649d55e23b6203d09bde2", - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ca87df1936fd2694fcc768079e9e8f926b6d45969ddc61ebf353f5e726d70b8b", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "1258655fcdca4692c2306100846761a2b0b149bbcaa5cde48a9abaa559913bc3", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "a8257f98c2e30c7bc7978bf645966d2eaa98cd49a42a8a923c062b5808dcdcdf", - "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "c5fab405ee82bc553e9a902fa27e6dfbc5213ec18311e17b6415396a69b50f43", - "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "e09b872f6900e821d42632a0ca09595f2affe6f9794049dbc74d69ba6a52b2b0", - "vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py": "e8e2e74fb8edb05e02d3e2328ea1f3ad5ddcfd7a10ed09c5cf3911b1d063e849" - }, - "scope": "Focused state/block tests and native four-rank subset attention; no full-model quality/performance acceptance", - "status": "completed", - "returncode": 0, - "elapsed_seconds": 78.8189791989862 - }, - { - "trial": "c16-video-row-fp32-quality-5s-00-8600757a", - "model_controller": { - "status": "completed", - "returncode": 0, - "elapsed_seconds": 355.51310362201184 - }, - "model_diagnostic": { - "source_head": "e0a3462fcc44de698b5dedcf8ff2c99f9802f757", - "samples_ms": [ - 197716.36760496767 - ], - "latency_median_ms": 197716.36760496767, - "scope": "http_complete_mp4", - "warmup": 0, - "repetitions": 1, - "kind": "quality", - "diagnostic": "audit", - "clean_timing": false, - "cleanup_passed": true - }, - "actual_counters": { - "request-000\\rank-0\\video_row_cache_stats.json": { - "request_counters": { - "full_steps": 17, - "partial_steps": 33, - "full_blocks": 850, - "partial_blocks": 1650, - "subset_attention_calls": 1650, - "subset_query_rows": 4083750, - "omitted_query_rows": 61538400, - "partial_mlp_rows": 4083750, - "peak_cache_bytes": 8039270400, - "closed": true, - "storage_dtype": "torch.float32", - "live_cache_bytes": 0 - }, - "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" - }, - "request-000\\rank-1\\video_row_cache_stats.json": { - "request_counters": { - "full_steps": 17, - "partial_steps": 33, - "full_blocks": 850, - "partial_blocks": 1650, - "subset_attention_calls": 1650, - "subset_query_rows": 4083750, - "omitted_query_rows": 61538400, - "partial_mlp_rows": 0, - "peak_cache_bytes": 10700390400, - "closed": true, - "storage_dtype": "torch.float32", - "live_cache_bytes": 0 - }, - "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" - }, - "request-000\\rank-2\\video_row_cache_stats.json": { - "request_counters": { - "full_steps": 17, - "partial_steps": 33, - "full_blocks": 850, - "partial_blocks": 1650, - "subset_attention_calls": 1650, - "subset_query_rows": 4083750, - "omitted_query_rows": 61538400, - "partial_mlp_rows": 0, - "peak_cache_bytes": 10700390400, - "closed": true, - "storage_dtype": "torch.float32", - "live_cache_bytes": 0 - }, - "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" - }, - "request-000\\rank-3\\video_row_cache_stats.json": { - "request_counters": { - "full_steps": 17, - "partial_steps": 33, - "full_blocks": 850, - "partial_blocks": 1650, - "subset_attention_calls": 1650, - "subset_query_rows": 4083750, - "omitted_query_rows": 61538400, - "partial_mlp_rows": 0, - "peak_cache_bytes": 10660608000, - "closed": true, - "storage_dtype": "torch.float32", - "live_cache_bytes": 0 - }, - "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" - }, - "request-000\\rank-0\\attention_path_counts.json": { - "_prefix_bnsd": 2500 - }, - "request-000\\rank-1\\attention_path_counts.json": { - "_prefix_bnsd": 2500 - }, - "request-000\\rank-2\\attention_path_counts.json": { - "_prefix_bnsd": 2500 - }, - "request-000\\rank-3\\attention_path_counts.json": { - "_prefix_bnsd": 2500 - } - } - }, - { - "trial": "c17-video-window-native-9b48cb56", - "cleanup_passed": true, - "source_unchanged": true, - "required_native_test_executed": true, - "files_before": { - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "66ca27991ce42ce302e5de078be6395c9d1ab53752378babef25089f3f75c3aa", - "tests/diffusion/models/minimax_h3/test_minimax_h3_video_window.py": "0bc59f35eff41164e125a5070e034c94ea60bf0112733ebe59eb6378eccbfed8" - }, - "scope": "CPU window reference/routing and native batched BF16 FAS; no full-model quality/performance acceptance", - "status": "completed", - "returncode": 0, - "elapsed_seconds": 51.878022204036824 - }, - { - "trial": "c17-video-window-operator-9bf7de3e", - "cleanup_passed": true, - "source_unchanged": true, - "files_before": { - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "66ca27991ce42ce302e5de078be6395c9d1ab53752378babef25089f3f75c3aa", - "tests/diffusion/models/minimax_h3/test_minimax_h3_video_window.py": "0bc59f35eff41164e125a5070e034c94ea60bf0112733ebe59eb6378eccbfed8" - }, - "scope": "Real captured H3 attention operator benchmark; full-model quality and HTTP are separate; no full-model quality/performance acceptance", - "status": "completed", - "returncode": 0, - "elapsed_seconds": 51.778123575029895, - "operator_benchmark": { - "scope": "real captured 5s attention operation on logical2; no model quality or HTTP acceptance", - "fixture_sha256": "00767e6afb2ba777bb2677eb9b60dc2f00fb649f5006ae8c1d77191214fbf912", - "fixture_tensor_sha256": { - "query": "313095dd143adf0636d57c90d77b689405dee510df3ebd6385a49d071c3323e9", - "key": "6638aee85f0039ee814206e7c976a2d1a9a2b185723c7bbb74c69b1e68c5bad4", - "value": "101eede996c8b13d61d487019d47f0b5fd2193664d8c178665063675cb66fdca" - }, - "query_shape": [ - 1, - 39808, - 14, - 128 - ], - "valid_kv": 39771, - "frames": 37, - "rows_per_frame": 1008, - "target_start": 2475, - "window": 3, - "finite": true, - "relative_l2_vs_dense": 0.5161040425300598, - "baseline_samples_ms": [ - 69.0619180095382, - 69.60304302629083, - 69.27597103640437, - 69.81903495034203, - 69.96654602698982 - ], - "candidate_samples_ms": [ - 25.191069988068193, - 25.301200977992266, - 25.325032009277493, - 25.26562102138996, - 25.219489994924515 - ], - "baseline_median_ms": 69.60304302629083, - "candidate_median_ms": 25.26562102138996, - "reduction_pct": 63.70040744936038, - "operator_gate_passed": true, - "warmups_per_operation": 2, - "measured_per_operation": 5, - "order": "AB/BA/AB/BA/AB", - "probe_sha256": "2465cf08173950a241ac3cc4713a4227d4c4b5a87f35358692a7ef6b60821824" - } - }, - { - "trial": "c17-video-window-quality-5s-00-a8c689e4", - "model_controller": { - "status": "completed", - "returncode": 0, - "elapsed_seconds": 358.6752056070254 - }, - "model_diagnostic": { - "source_head": "c1589a2b5e4ada617e7cca38ef7c300fe9495c58", - "samples_ms": [ - 210397.12271699682 - ], - "latency_median_ms": 210397.12271699682, - "scope": "http_complete_mp4", - "warmup": 0, - "repetitions": 1, - "kind": "quality", - "diagnostic": "audit", - "clean_timing": false, - "cleanup_passed": true - }, - "actual_counters": { - "request-000\\rank-0\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 4900, - "_video_temporal_window_bnsd": 2400 - }, - "request-000\\rank-1\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 4900, - "_video_temporal_window_bnsd": 2400 - }, - "request-000\\rank-2\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 4900, - "_video_temporal_window_bnsd": 2400 - }, - "request-000\\rank-3\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 4900, - "_video_temporal_window_bnsd": 2400 - } - } - }, - { - "trial": "c18-video-pooled-context-native-1673db72", - "cleanup_passed": true, - "source_unchanged": true, - "required_native_test_executed": true, - "files_before": { - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "1febc030a7917b6e10fe287014fe2aacef76ade7bf58ebb604fdedc36fb7c21d", - "tests/diffusion/models/minimax_h3/test_minimax_h3_pooled_context.py": "45a9ae5ba56858daee6a1088a8c8b1ed612ea9afd3c229c90a9b6db3c80a06b2" - }, - "scope": "Independent weighted global-context reference and native padded-head BF16 FAS; no full-model quality/performance acceptance", - "status": "completed", - "returncode": 0, - "elapsed_seconds": 49.97671151399845 - }, - { - "trial": "c18-video-pooled-context-operator-315db0e5", - "cleanup_passed": true, - "source_unchanged": true, - "files_before": { - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "1febc030a7917b6e10fe287014fe2aacef76ade7bf58ebb604fdedc36fb7c21d", - "tests/diffusion/models/minimax_h3/test_minimax_h3_pooled_context.py": "45a9ae5ba56858daee6a1088a8c8b1ed612ea9afd3c229c90a9b6db3c80a06b2" - }, - "scope": "Real captured H3 attention operator benchmark; full-model quality and HTTP are separate; no full-model quality/performance acceptance", - "status": "completed", - "returncode": 0, - "elapsed_seconds": 51.37963652302278, - "operator_benchmark": { - "scope": "real captured 5s attention operation on logical2; no model quality or HTTP acceptance", - "fixture_sha256": "00767e6afb2ba777bb2677eb9b60dc2f00fb649f5006ae8c1d77191214fbf912", - "fixture_tensor_sha256": { - "query": "313095dd143adf0636d57c90d77b689405dee510df3ebd6385a49d071c3323e9", - "key": "6638aee85f0039ee814206e7c976a2d1a9a2b185723c7bbb74c69b1e68c5bad4", - "value": "101eede996c8b13d61d487019d47f0b5fd2193664d8c178665063675cb66fdca" - }, - "query_shape": [ - 1, - 39808, - 14, - 128 - ], - "valid_kv": 39771, - "frames": 37, - "rows_per_frame": 1008, - "target_start": 2475, - "profile": "3x3-local1", - "finite": true, - "relative_l2_vs_dense": 0.21801254153251648, - "baseline_samples_ms": [ - 69.19049099087715, - 69.70348599134013, - 69.43478400353342, - 69.81050700414926, - 69.24652203451842 - ], - "candidate_samples_ms": [ - 68.29233502503484, - 64.26435202592984, - 71.05128600960597, - 64.39994304673746, - 64.39412303734571 - ], - "baseline_median_ms": 69.43478400353342, - "candidate_median_ms": 64.39994304673746, - "reduction_pct": 7.251179691924648, - "operator_gate_passed": false, - "warmups_per_operation": 2, - "measured_per_operation": 5, - "order": "AB/BA/AB/BA/AB", - "probe_sha256": "42852c0e4e693c5b3685981db617de00648feefe651f7060ad83c359b08dd81d" - } - }, - { - "trial": "c19-video-global-pool-native-f0f4c502", - "cleanup_passed": true, - "source_unchanged": true, - "required_native_test_executed": true, - "files_before": { - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ddfd6d6822cc6a5528fc95539f7f98dfdeed25d5489da3ed5cf76e6076a5ec94", - "tests/diffusion/models/minimax_h3/test_minimax_h3_global_pool.py": "5457db550b59e3373be8f5f21873c1b78ac357a6ac170c2603ee531b50e91467" - }, - "scope": "Independent weighted global-context reference and native padded-head BF16 FAS; no full-model quality/performance acceptance", - "status": "completed", - "returncode": 0, - "elapsed_seconds": 51.528875903983135 - }, - { - "trial": "c19-video-global-pool-operator-933511c9", - "cleanup_passed": true, - "source_unchanged": true, - "files_before": { - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ddfd6d6822cc6a5528fc95539f7f98dfdeed25d5489da3ed5cf76e6076a5ec94", - "tests/diffusion/models/minimax_h3/test_minimax_h3_global_pool.py": "5457db550b59e3373be8f5f21873c1b78ac357a6ac170c2603ee531b50e91467" - }, - "scope": "Real captured H3 attention operator benchmark; full-model quality and HTTP are separate; no full-model quality/performance acceptance", - "status": "completed", - "returncode": 0, - "elapsed_seconds": 51.02866222098237, - "operator_benchmark": { - "scope": "real captured 5s attention operation on logical2; no model quality or HTTP acceptance", - "fixture_sha256": "00767e6afb2ba777bb2677eb9b60dc2f00fb649f5006ae8c1d77191214fbf912", - "fixture_tensor_sha256": { - "query": "313095dd143adf0636d57c90d77b689405dee510df3ebd6385a49d071c3323e9", - "key": "6638aee85f0039ee814206e7c976a2d1a9a2b185723c7bbb74c69b1e68c5bad4", - "value": "101eede996c8b13d61d487019d47f0b5fd2193664d8c178665063675cb66fdca" - }, - "query_shape": [ - 1, - 39808, - 14, - 128 - ], - "valid_kv": 39771, - "frames": 37, - "rows_per_frame": 1008, - "target_start": 2475, - "profile": "3x3-global", - "finite": true, - "relative_l2_vs_dense": 0.2588050067424774, - "baseline_samples_ms": [ - 69.95450798422098, - 69.79658600175753, - 69.58012498216704, - 70.22821099963039, - 69.11802099784836 - ], - "candidate_samples_ms": [ - 32.91054297005758, - 32.899541954975575, - 32.924533006735146, - 32.779561006464064, - 32.96928299823776 - ], - "baseline_median_ms": 69.79658600175753, - "candidate_median_ms": 32.91054297005758, - "reduction_pct": 52.847918708761966, - "operator_gate_passed": false, - "warmups_per_operation": 2, - "measured_per_operation": 5, - "order": "AB/BA/AB/BA/AB", - "probe_sha256": "a27906d8bb0b65e2f6ffb41ddeb4b513163b7bf23b9d2dbd33a5015fc93b2a08" - } - }, - { - "trial": "c20-video-variance-pool-native-50101063", - "cleanup_passed": true, - "source_unchanged": true, - "required_native_test_executed": true, - "files_before": { - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "4390450bc5324f7028125101cd4bb32e9de237b04bc12fa879b48110d7122e3c", - "tests/diffusion/models/minimax_h3/test_minimax_h3_variance_pool.py": "e871fb4b222d61d10b6c442a333a52c976cceaa0d406233502ecbcaddfeba8c9" - }, - "scope": "Independent weighted global-context reference and native padded-head BF16 FAS; no full-model quality/performance acceptance", - "status": "completed", - "returncode": 0, - "elapsed_seconds": 50.230675689992495 - }, - { - "trial": "c20-video-variance-pool-operator-209f263d", - "cleanup_passed": true, - "source_unchanged": true, - "files_before": { - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "4390450bc5324f7028125101cd4bb32e9de237b04bc12fa879b48110d7122e3c", - "tests/diffusion/models/minimax_h3/test_minimax_h3_variance_pool.py": "e871fb4b222d61d10b6c442a333a52c976cceaa0d406233502ecbcaddfeba8c9" - }, - "scope": "Real captured H3 attention operator benchmark; full-model quality and HTTP are separate; no full-model quality/performance acceptance", - "status": "completed", - "returncode": 0, - "elapsed_seconds": 49.880220040970016, - "operator_benchmark": { - "scope": "real captured 5s attention operation on logical2; no model quality or HTTP acceptance", - "fixture_sha256": "00767e6afb2ba777bb2677eb9b60dc2f00fb649f5006ae8c1d77191214fbf912", - "fixture_tensor_sha256": { - "query": "313095dd143adf0636d57c90d77b689405dee510df3ebd6385a49d071c3323e9", - "key": "6638aee85f0039ee814206e7c976a2d1a9a2b185723c7bbb74c69b1e68c5bad4", - "value": "101eede996c8b13d61d487019d47f0b5fd2193664d8c178665063675cb66fdca" - }, - "query_shape": [ - 1, - 39808, - 14, - 128 - ], - "valid_kv": 39771, - "frames": 37, - "rows_per_frame": 1008, - "target_start": 2475, - "profile": "3x3-variance", - "finite": true, - "relative_l2_vs_dense": 0.26336145401000977, - "baseline_samples_ms": [ - 69.74446203093976, - 69.06059599714354, - 69.64961998164654, - 69.69038100214675, - 69.62426099926233 - ], - "candidate_samples_ms": [ - 35.81449302146211, - 36.30960697773844, - 35.925014002714306, - 35.99672397831455, - 35.80518299713731 - ], - "baseline_median_ms": 69.64961998164654, - "candidate_median_ms": 35.925014002714306, - "reduction_pct": 48.42037327385138, - "operator_gate_passed": false, - "warmups_per_operation": 2, - "measured_per_operation": 5, - "order": "AB/BA/AB/BA/AB", - "probe_sha256": "785e7a0c13719e37d5cc44fb66af05649e51e1f7e5d0b212e92ec44f79cb0184" - } - }, - { - "trial": "c21-video-pre-gate-native-3c5eebbe", - "cleanup_passed": true, - "source_unchanged": true, - "required_native_test_executed": true, - "files_before": { - "tests/diffusion/distributed/test_collective_gate.py": "6a02f32882d85eca1e3bb9a023ce3adb3d5b59c213845e11e85c96a89432f477", - "tests/diffusion/models/minimax_h3/test_minimax_h3_pre_gate_cache.py": "a4924d0b3dd58ae413259edba6cf2487b3e41d391f943fb74358934be72eb53b", - "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "e09b872f6900e821d42632a0ca09595f2affe6f9794049dbc74d69ba6a52b2b0", - "vllm_omni/config/omni_config.py": "6894120a330474f718d43bf2bf7928892eb996152839ce6469c34d18a7df09d1", - "vllm_omni/config/stage_config.py": "0752e8cc1c6f1021f737830d08663cfc118923a83fd909c607f9ec3ab2a3d2fb", - "vllm_omni/diffusion/attention/layer.py": "76c5fce3dabd23e39a8890316e1691c03bc203066cb60af5611ec4790eefc632", - "vllm_omni/diffusion/data.py": "5db83d5c71cb27cc7810d35fcfc891da80f0b003adbb16bdecc3816a63305e11", - "vllm_omni/diffusion/distributed/collective_gate.py": "01c6317cfa615840681b8bb8b8513fa3736d26e845a24ad9e1ab7f87489b962f", - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ca87df1936fd2694fcc768079e9e8f926b6d45969ddc61ebf353f5e726d70b8b", - "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "c5fab405ee82bc553e9a902fa27e6dfbc5213ec18311e17b6415396a69b50f43", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "8933b536317652d0dbc7a974ea7e0bca928e7b3bdf438988999ff5a498a6ec9d", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "019bf71106ba3263b55ad7b682064e3a176eecb81725cb48bbf178387356d5d9", - "vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py": "e8e2e74fb8edb05e02d3e2328ea1f3ad5ddcfd7a10ed09c5cf3911b1d063e849", - "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "04a9212994b5c8593f8d196820300ded077fed9af4274b64d87d9e6bf71eae07", - "vllm_omni/diffusion/offloader/base.py": "23922136deaf54e2d98e827b007b34c7e3ad36171b71bfe9060d390efc0b679f", - "vllm_omni/diffusion/offloader/distributed_layerwise_backend.py": "19fbce0f1036e4784d68cab798c499526a9099101909e41ce96dd0caa033b6a4", - "vllm_omni/engine/arg_utils.py": "52569e3688cc708729a48ca0a2c875c28193733c2f934f6ff89f6c815e4009c7", - "vllm_omni/entrypoints/cli/serve.py": "f8284ad6b751ba4a9e4f7254d78132cc2cfd49871a0f9af7fde49c6789d79cb0" - }, - "scope": "Legacy/pre-gate/DLO tests, two executed native four-rank HCCL/subsetFA suites and fresh current gates; no full-model quality/performance acceptance", - "status": "completed", - "returncode": 0, - "elapsed_seconds": 127.31179964594776 - }, - { - "trial": "c21-video-pre-gate-quality-5s-00-7265c33e", - "model_controller": { - "status": "completed", - "returncode": 0, - "elapsed_seconds": 318.92388185299933 - }, - "model_diagnostic": { - "source_head": "d48fa51d03dd748568cbcf57ed40f533c691c9bd", - "samples_ms": [ - 163805.5620979867 - ], - "latency_median_ms": 163805.5620979867, - "scope": "http_complete_mp4", - "warmup": 0, - "repetitions": 1, - "kind": "quality", - "diagnostic": "audit", - "clean_timing": false, - "cleanup_passed": true - }, - "actual_counters": { - "request-000\\rank-0\\video_row_cache_stats.json": { - "request_counters": { - "full_steps": 8, - "partial_steps": 42, - "full_blocks": 400, - "partial_blocks": 2100, - "subset_attention_calls": 2100, - "subset_query_rows": 5197500, - "omitted_query_rows": 78321600, - "partial_mlp_rows": 5197500, - "peak_cache_bytes": 8039270400, - "closed": true, - "storage_dtype": "torch.bfloat16", - "cache_stage": "pre_gate", - "fresh_attention_gates": 2100, - "fresh_mlp_gates": 2100, - "live_cache_bytes": 0 - }, - "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" - }, - "request-000\\rank-1\\video_row_cache_stats.json": { - "request_counters": { - "full_steps": 8, - "partial_steps": 42, - "full_blocks": 400, - "partial_blocks": 2100, - "subset_attention_calls": 2100, - "subset_query_rows": 5197500, - "omitted_query_rows": 78321600, - "partial_mlp_rows": 0, - "peak_cache_bytes": 10700390400, - "closed": true, - "storage_dtype": "torch.bfloat16", - "cache_stage": "pre_gate", - "fresh_attention_gates": 2100, - "fresh_mlp_gates": 2100, - "live_cache_bytes": 0 - }, - "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" - }, - "request-000\\rank-2\\video_row_cache_stats.json": { - "request_counters": { - "full_steps": 8, - "partial_steps": 42, - "full_blocks": 400, - "partial_blocks": 2100, - "subset_attention_calls": 2100, - "subset_query_rows": 5197500, - "omitted_query_rows": 78321600, - "partial_mlp_rows": 0, - "peak_cache_bytes": 10700390400, - "closed": true, - "storage_dtype": "torch.bfloat16", - "cache_stage": "pre_gate", - "fresh_attention_gates": 2100, - "fresh_mlp_gates": 2100, - "live_cache_bytes": 0 - }, - "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" - }, - "request-000\\rank-3\\video_row_cache_stats.json": { - "request_counters": { - "full_steps": 8, - "partial_steps": 42, - "full_blocks": 400, - "partial_blocks": 2100, - "subset_attention_calls": 2100, - "subset_query_rows": 5197500, - "omitted_query_rows": 78321600, - "partial_mlp_rows": 0, - "peak_cache_bytes": 10660608000, - "closed": true, - "storage_dtype": "torch.bfloat16", - "cache_stage": "pre_gate", - "fresh_attention_gates": 2100, - "fresh_mlp_gates": 2100, - "live_cache_bytes": 0 - }, - "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" - }, - "request-000\\rank-0\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-000\\rank-1\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-000\\rank-2\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-000\\rank-3\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - } - } - } - ], - "decisions": [ - "C12 fixed profile stopped: zero actual predictions.", - "C14 pinned block-FP8 and block-INT8 prerequisites failed; per-token INT8 is not excluded.", - "C13 numerical qualification and clean performance are separate.", - "C15 fixed FP16 video residual profile failed finite velocity at first reuse.", - "C16 FP32 residual completed but failed three frozen audio quality guards; fixed profile stopped.", - "C17 completed5s but failed11frozen quality checks; fixed window profile stopped.", - "C18 native17PASS but operation reduced only7.25%; no full model run.", - "C19 native17PASS; operation52.85%faster but relL2 .258805 exceeded fixed .258052screen; no model run.", - "C20 native17PASS; operation48.42%faster and relL2 .263361 failed both fixed primitive gates; no model run.", - "C21 native36PASS; current gates over cached BF16 video sublayers require separate frozen media and clean HTTP qualification." - ] -} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c12-through-c22-execution-checkpoint.json b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c12-through-c22-execution-checkpoint.json deleted file mode 100644 index ad0f10fcd..000000000 --- a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c12-through-c22-execution-checkpoint.json +++ /dev/null @@ -1,1351 +0,0 @@ -{ - "campaign_verdict": "INCONCLUSIVE", - "performance_acceptance": "No newly accepted gain", - "runs": [ - { - "trial": "c12-alignment-quality-5s-00-9bf5c373", - "model_controller": { - "status": "completed", - "returncode": 0, - "elapsed_seconds": 471.80311120999977 - }, - "model_diagnostic": { - "source_head": "a29648e5acd3077b7b22739bc9522017782fe1ac", - "samples_ms": [ - 318280.59680899605 - ], - "latency_median_ms": 318280.59680899605, - "scope": "http_complete_mp4", - "warmup": 0, - "repetitions": 1, - "kind": "quality", - "diagnostic": "audit", - "clean_timing": false, - "cleanup_passed": true - }, - "actual_counters": { - "request-000\\rank-0\\alignment_cache_stats.json": { - "request_counters": { - "computed": 50, - "predicted": 0, - "vetoes": 45 - }, - "scope": "actual state-aligned compute/forecast/veto counts; not quality acceptance" - }, - "request-000\\rank-1\\alignment_cache_stats.json": { - "request_counters": { - "computed": 50, - "predicted": 0, - "vetoes": 45 - }, - "scope": "actual state-aligned compute/forecast/veto counts; not quality acceptance" - }, - "request-000\\rank-2\\alignment_cache_stats.json": { - "request_counters": { - "computed": 50, - "predicted": 0, - "vetoes": 45 - }, - "scope": "actual state-aligned compute/forecast/veto counts; not quality acceptance" - }, - "request-000\\rank-3\\alignment_cache_stats.json": { - "request_counters": { - "computed": 50, - "predicted": 0, - "vetoes": 45 - }, - "scope": "actual state-aligned compute/forecast/veto counts; not quality acceptance" - }, - "request-000\\rank-0\\attention_path_counts.json": { - "_prefix_bnsd": 2500 - }, - "request-000\\rank-1\\attention_path_counts.json": { - "_prefix_bnsd": 2500 - }, - "request-000\\rank-2\\attention_path_counts.json": { - "_prefix_bnsd": 2500 - }, - "request-000\\rank-3\\attention_path_counts.json": { - "_prefix_bnsd": 2500 - } - } - }, - { - "trial": "c12-state-alignment-native-tests-0dc5859b", - "cleanup_passed": true, - "source_unchanged": true, - "native_test_executed": true, - "files_before": { - "vllm_omni/diffusion/models/minimax_h3/cache_state_alignment.py": "3340ec21a77aef78531c601be241bcfb1e44107c795e8c8649907d1a18ed377f", - "vllm_omni/diffusion/models/minimax_h3/reference_kv_cache_dit.py": "f4cd0e9aa9e824c92ec1acd02c57a8f7d8684c35d4edf92ffdfae3345ff44e99", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "436ec52cd6329cbc64e91d94b450bfa6f773a9a953b8f974db73a5eb986a98bb", - "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "4dec8f02a04bfe7e8d2ada24fc6d5ae40a0058671d8515c8a266ecf88824b7b0", - "tests/diffusion/models/minimax_h3/test_minimax_h3_state_cache.py": "89cf57482b35d8d8ad8d4978aec87e9ee3ef45330e5b911161adef4379950a65" - }, - "scope": "focused tests only; no full-model quality or performance acceptance", - "status": "completed", - "returncode": 0, - "elapsed_seconds": 82.06077561300481 - }, - { - "trial": "c13-baseline-cuda-staging-e22d8651", - "cleanup_passed": true, - "source_unchanged": true, - "native_test_executed": false, - "files_before": { - "vllm_omni/diffusion/offloader/sequential_backend.py": "d542935b656a0d9dced78de8fb086b8e1c5aade26fac6cee5316b32a49f0d569", - "tests/diffusion/offloader/test_sequential_backend.py": "dcf1d64e65bed572d65b095c1350950f55d7e8d9ddf8c7dfd72d0ea6e721685a" - }, - "scope": "Reproduce stock CUDA-graph staging expectation on unchanged migration owning source in the same Ascend test environment", - "status": "failed", - "returncode": 1, - "elapsed_seconds": 50.234599059971515 - }, - { - "trial": "c13-c1-dlo-5s-audit-cead10d1", - "model_controller": { - "status": "failed", - "returncode": 2, - "elapsed_seconds": 0.11969929095357656 - } - }, - { - "trial": "c13-c1-dlo-5s-audit-v2-b0b86c6d", - "model_controller": { - "status": "completed", - "returncode": 0, - "elapsed_seconds": 749.505351218977 - }, - "model_diagnostic": { - "source_head": "e89b1d40d37f7f72767892a56fa7980ea572ba8e", - "samples_ms": [ - 315593.86175096733, - 284456.94282796467 - ], - "latency_median_ms": 300025.402289466, - "scope": "http_complete_mp4", - "warmup": 0, - "repetitions": 2, - "kind": "diagnostic", - "diagnostic": "audit", - "clean_timing": false, - "cleanup_passed": true - }, - "actual_counters": { - "request-000\\rank-0\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-000\\rank-1\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-000\\rank-2\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-000\\rank-3\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-001\\rank-0\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-001\\rank-1\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-001\\rank-2\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-001\\rank-3\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - } - } - }, - { - "trial": "c13-c1-dlo-5s-calibration-c98129d8", - "model_controller": { - "status": "completed", - "returncode": 0, - "elapsed_seconds": 768.8735517460154 - }, - "model_diagnostic": { - "source_head": "e89b1d40d37f7f72767892a56fa7980ea572ba8e", - "samples_ms": [ - 317177.1984040388, - 285534.089674009 - ], - "latency_median_ms": 301355.6440390239, - "scope": "http_complete_mp4", - "warmup": 0, - "repetitions": 2, - "kind": "calibration", - "diagnostic": "audit", - "clean_timing": false, - "cleanup_passed": true - }, - "actual_counters": { - "request-000\\rank-0\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-000\\rank-1\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-000\\rank-2\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-000\\rank-3\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-001\\rank-0\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-001\\rank-1\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-001\\rank-2\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-001\\rank-3\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - } - } - }, - { - "trial": "c13-dlo-gate-tests-bc92a71d", - "cleanup_passed": true, - "source_unchanged": true, - "native_test_executed": false, - "files_before": { - "tests/diffusion/distributed/test_collective_gate.py": "2b33f73dff9c79118cc2b1bd4d50e3542ebc45b7bf029b38c17abbef37e4b917", - "vllm_omni/config/omni_config.py": "6894120a330474f718d43bf2bf7928892eb996152839ce6469c34d18a7df09d1", - "vllm_omni/config/stage_config.py": "0752e8cc1c6f1021f737830d08663cfc118923a83fd909c607f9ec3ab2a3d2fb", - "vllm_omni/diffusion/attention/layer.py": "76c5fce3dabd23e39a8890316e1691c03bc203066cb60af5611ec4790eefc632", - "vllm_omni/diffusion/data.py": "5db83d5c71cb27cc7810d35fcfc891da80f0b003adbb16bdecc3816a63305e11", - "vllm_omni/diffusion/distributed/collective_gate.py": "01c6317cfa615840681b8bb8b8513fa3736d26e845a24ad9e1ab7f87489b962f", - "vllm_omni/diffusion/offloader/base.py": "533c793119e4cdc14a2ea3c6cb5bf97ffede673befeab3f0fb8dbbf51e655379", - "vllm_omni/diffusion/offloader/distributed_layerwise_backend.py": "d441481513aba9e90a91265f4bba3f53f8766197be8c1302c3c714e690d5ea76", - "vllm_omni/engine/arg_utils.py": "52569e3688cc708729a48ca0a2c875c28193733c2f934f6ff89f6c815e4009c7", - "vllm_omni/entrypoints/cli/serve.py": "350461dfca565bc0b07087ac7af1e27b0e247f5eac717e350b637e37f62dd088", - "tests/diffusion/offloader/helpers.py": "729c73abba3b9a12911282d4bcd3cf8bfa148fbcbfdc2e2b7c86000e387fa687" - }, - "scope": "CPU mock barrier/chunk tests; no actual NPU collective or full-model quality/performance acceptance", - "status": "completed", - "returncode": 0, - "elapsed_seconds": 47.3752507569734 - }, - { - "trial": "c13-dlo-regression-tests-95e9ed2a", - "cleanup_passed": true, - "source_unchanged": true, - "native_test_executed": false, - "files_before": { - "tests/diffusion/distributed/test_collective_gate.py": "2b33f73dff9c79118cc2b1bd4d50e3542ebc45b7bf029b38c17abbef37e4b917", - "vllm_omni/config/omni_config.py": "6894120a330474f718d43bf2bf7928892eb996152839ce6469c34d18a7df09d1", - "vllm_omni/config/stage_config.py": "0752e8cc1c6f1021f737830d08663cfc118923a83fd909c607f9ec3ab2a3d2fb", - "vllm_omni/diffusion/attention/layer.py": "76c5fce3dabd23e39a8890316e1691c03bc203066cb60af5611ec4790eefc632", - "vllm_omni/diffusion/data.py": "5db83d5c71cb27cc7810d35fcfc891da80f0b003adbb16bdecc3816a63305e11", - "vllm_omni/diffusion/distributed/collective_gate.py": "01c6317cfa615840681b8bb8b8513fa3736d26e845a24ad9e1ab7f87489b962f", - "vllm_omni/diffusion/offloader/base.py": "533c793119e4cdc14a2ea3c6cb5bf97ffede673befeab3f0fb8dbbf51e655379", - "vllm_omni/diffusion/offloader/distributed_layerwise_backend.py": "d441481513aba9e90a91265f4bba3f53f8766197be8c1302c3c714e690d5ea76", - "vllm_omni/engine/arg_utils.py": "52569e3688cc708729a48ca0a2c875c28193733c2f934f6ff89f6c815e4009c7", - "vllm_omni/entrypoints/cli/serve.py": "350461dfca565bc0b07087ac7af1e27b0e247f5eac717e350b637e37f62dd088", - "tests/diffusion/offloader/helpers.py": "729c73abba3b9a12911282d4bcd3cf8bfa148fbcbfdc2e2b7c86000e387fa687" - }, - "scope": "CPU mock barrier/chunk tests; no actual NPU collective or full-model quality/performance acceptance", - "status": "failed", - "returncode": 1, - "elapsed_seconds": 62.34616496198578 - }, - { - "trial": "c13-dlo-regression-v2-12c217d5", - "cleanup_passed": true, - "source_unchanged": true, - "native_test_executed": false, - "files_before": { - "tests/diffusion/distributed/test_collective_gate.py": "2b33f73dff9c79118cc2b1bd4d50e3542ebc45b7bf029b38c17abbef37e4b917", - "vllm_omni/config/omni_config.py": "6894120a330474f718d43bf2bf7928892eb996152839ce6469c34d18a7df09d1", - "vllm_omni/config/stage_config.py": "0752e8cc1c6f1021f737830d08663cfc118923a83fd909c607f9ec3ab2a3d2fb", - "vllm_omni/diffusion/attention/layer.py": "76c5fce3dabd23e39a8890316e1691c03bc203066cb60af5611ec4790eefc632", - "vllm_omni/diffusion/data.py": "5db83d5c71cb27cc7810d35fcfc891da80f0b003adbb16bdecc3816a63305e11", - "vllm_omni/diffusion/distributed/collective_gate.py": "01c6317cfa615840681b8bb8b8513fa3736d26e845a24ad9e1ab7f87489b962f", - "vllm_omni/diffusion/offloader/base.py": "533c793119e4cdc14a2ea3c6cb5bf97ffede673befeab3f0fb8dbbf51e655379", - "vllm_omni/diffusion/offloader/distributed_layerwise_backend.py": "d441481513aba9e90a91265f4bba3f53f8766197be8c1302c3c714e690d5ea76", - "vllm_omni/engine/arg_utils.py": "52569e3688cc708729a48ca0a2c875c28193733c2f934f6ff89f6c815e4009c7", - "vllm_omni/entrypoints/cli/serve.py": "350461dfca565bc0b07087ac7af1e27b0e247f5eac717e350b637e37f62dd088", - "tests/diffusion/offloader/helpers.py": "729c73abba3b9a12911282d4bcd3cf8bfa148fbcbfdc2e2b7c86000e387fa687" - }, - "scope": "Expanded offloader tests in Ascend environment; some stock accelerator fixtures use NPU, not a full model test", - "status": "failed", - "returncode": 1, - "elapsed_seconds": 65.14432259398745 - }, - { - "trial": "c14-int8-prerequisites-fdd6633b", - "cleanup_passed": true, - "source_unchanged": true, - "helpers_unchanged": true, - "scope": "Bounded native prerequisites only; a failed probe is retained as negative evidence.", - "status": "failed", - "returncode": 1, - "elapsed_seconds": 20.795623040001374, - "native_prerequisite": { - "verdict": "FAIL", - "scope": "TorchNPU block-INT8 attention prerequisites only", - "phase": "dynamic_block_int8_quant_query", - "versions": { - "torch": "2.10.0+cpu", - "torch_npu": "2.10.0.post4", - "mindiesd": "3.1.0" - }, - "apis": { - "float8_e4m3fn": true, - "npu_dynamic_block_quant": true, - "npu_fused_infer_attention_score_v2": true - }, - "device_name": "Ascend910_9392", - "input": { - "shape": [ - 1, - 14, - 256, - 128 - ], - "dtype": "torch.bfloat16", - "visible_devices": "2" - }, - "error_type": "RuntimeError", - "probe_sha256": "fab18252f860231f9b7fcaa54587afbaa96fce15987fd551922246c8964f7217", - "error": "npu_dynamic_block_quant:../third_party/op-plugin/op_plugin/ops/opapi/DynamicBlockQuantNpuOpApi.cpp:61 NPU function error: call aclnnDynamicBlockQuantV2 failed, error code is 561002\n[ERROR] 2026-11-02-02:00:57 (PID:2662830, Device:0, RankID:-1) ERR00100 PTA call acl api failed.\n[PID: 2662830] 2026-11-02-02:00:57.039.986 Invalid_Argument(EZ0024): Parameter row_block_size of DynamicBlockQuant has incorrect value 128. It should be 1.\n Solution: Check whether the parameter value is correct.\nTraceBack (most recent call last):\n The attr get failed.[FUNC:CheckParams][FILE:dynamic_block_quant_i8_tiling.cpp][LINE:78]\n The params check failed.[FUNC:Tiling4DynamicBlockQuant][FILE:dynamic_block_quant_tiling.cpp][LINE:564]\n Failed to execute operator DynamicBlockQuant_0. Reason: Failed to execute tiling function.\n Check NnopbaseExecutorDoTiling(executor) failed\n Check NnopbaseExecutorTilingAndUpdateBinInfo(executor) failed\n Check NnopbaseExecutorMatchCache(executor) failed\n Check NnopbaseRunForWorkspace(*executor, workspaceSize) failed\n" - } - }, - { - "trial": "c14-quant-prerequisites-358b9b28", - "cleanup_passed": true, - "source_unchanged": true, - "helpers_unchanged": true, - "scope": "Bounded native prerequisites only; a failed probe is retained as negative evidence.", - "status": "failed", - "returncode": 1, - "elapsed_seconds": 20.143024684977718, - "native_prerequisite": { - "verdict": "FAIL", - "scope": "TorchNPU FP8 prerequisites only", - "phase": "dynamic_block_quant_query", - "versions": { - "torch": "2.10.0+cpu", - "torch_npu": "2.10.0.post4", - "mindiesd": "3.1.0" - }, - "apis": { - "float8_e4m3fn": true, - "npu_dynamic_block_quant": true, - "npu_fused_infer_attention_score_v2": true - }, - "device_name": "Ascend910_9392", - "input": { - "shape": [ - 1, - 14, - 256, - 128 - ], - "dtype": "torch.bfloat16", - "visible_devices": "2" - }, - "error_type": "RuntimeError", - "probe_sha256": "df6465956e6f73bbca140fb9880057e30869fd4b7976b2dc091f71a575dd1096", - "error": "npu_dynamic_block_quant:../third_party/op-plugin/op_plugin/ops/opapi/DynamicBlockQuantNpuOpApi.cpp:61 NPU function error: call aclnnDynamicBlockQuantV2 failed, error code is 161002\n[ERROR] 2026-11-02-01:45:24 (PID:2491012, Device:0, RankID:-1) ERR00100 PTA call acl api failed.\n[PID: 2491012] 2026-11-02-01:45:24.732.576 Execution_Error(EZ1009): Failed to execute operator DynamicBlockQuant_0. Reason: The dtype or format of the actual input or output parameter of the operator is inconsistent with that defined in the operator prototype OpDef, details of parameter are x(DT_BFLOAT16, ND), yOut(DT_FLOAT8_E4M3FN, ND), scaleOut(DT_FLOAT, ND).\nTraceBack (most recent call last):\n Supported dtype and format list is:\n SupportInfo[0]: x(DT_FLOAT16, ND), yOut(DT_INT8, ND), scaleOut(DT_FLOAT, ND)\n SupportInfo[1]: x(DT_BFLOAT16, ND), yOut(DT_INT8, ND), scaleOut(DT_FLOAT, ND)\n Check NnopbaseExecutorTilingAndUpdateBinInfo(executor) failed\n Check NnopbaseExecutorMatchCache(executor) failed\n Check NnopbaseRunForWorkspace(*executor, workspaceSize) failed\n" - } - }, - { - "trial": "c15-video-row-native-066a9a40", - "cleanup_passed": true, - "source_unchanged": true, - "required_native_test_executed": true, - "files_before": { - "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "e35b551b8fc0ec3b0ba1384b4df96a8daa0e5179c5ed39d641125310b2ee957f", - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ca87df1936fd2694fcc768079e9e8f926b6d45969ddc61ebf353f5e726d70b8b", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "1258655fcdca4692c2306100846761a2b0b149bbcaa5cde48a9abaa559913bc3", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "a8257f98c2e30c7bc7978bf645966d2eaa98cd49a42a8a923c062b5808dcdcdf", - "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "6f890fcf84047bc5bc084f7b756a2b02e72573888a3bcc12ba7d0211e1bb62ef", - "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "d2a5bed10620a337ff9aaf99cf4a3740d94a8962ef903bf2da75d65e3b50e21c" - }, - "scope": "Focused state/block tests and native four-rank subset attention; no full-model quality/performance acceptance", - "status": "failed", - "returncode": 1, - "elapsed_seconds": 82.27795292000519 - }, - { - "trial": "c15-video-row-native-3a9721a6", - "cleanup_passed": true, - "source_unchanged": true, - "required_native_test_executed": true, - "files_before": { - "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "cf9fb13c5f24c7595d62d13cae6e283cafb2178612b20028d35357f0cafaac83", - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ca87df1936fd2694fcc768079e9e8f926b6d45969ddc61ebf353f5e726d70b8b", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "1258655fcdca4692c2306100846761a2b0b149bbcaa5cde48a9abaa559913bc3", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "a8257f98c2e30c7bc7978bf645966d2eaa98cd49a42a8a923c062b5808dcdcdf", - "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "c5fab405ee82bc553e9a902fa27e6dfbc5213ec18311e17b6415396a69b50f43", - "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "8ec1b3ae4cd252f5d95ab23e0232b983900939e8cb15d1e64f4740f7204abf39", - "vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py": "e8e2e74fb8edb05e02d3e2328ea1f3ad5ddcfd7a10ed09c5cf3911b1d063e849" - }, - "scope": "Focused state/block tests and native four-rank subset attention; no full-model quality/performance acceptance", - "status": "completed", - "returncode": 0, - "elapsed_seconds": 81.91829407500336 - }, - { - "trial": "c15-video-row-native-a2a224de", - "cleanup_passed": true, - "source_unchanged": true, - "required_native_test_executed": true, - "files_before": { - "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "cf9fb13c5f24c7595d62d13cae6e283cafb2178612b20028d35357f0cafaac83", - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ca87df1936fd2694fcc768079e9e8f926b6d45969ddc61ebf353f5e726d70b8b", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "1258655fcdca4692c2306100846761a2b0b149bbcaa5cde48a9abaa559913bc3", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "a8257f98c2e30c7bc7978bf645966d2eaa98cd49a42a8a923c062b5808dcdcdf", - "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "118afcc28379dd73feec763fbb9cfc9f17c6a208afe5969a85b3c969b8059552", - "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "875d0e70621fc76906be130519d7bb1242de5dd58159e31025e26da37e60d98c" - }, - "scope": "Focused state/block tests and native four-rank subset attention; no full-model quality/performance acceptance", - "status": "failed", - "returncode": 1, - "elapsed_seconds": 82.86056435096543 - }, - { - "trial": "c15-video-row-native-bb5084cc", - "cleanup_passed": true, - "source_unchanged": true, - "required_native_test_executed": true, - "files_before": { - "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "cf9fb13c5f24c7595d62d13cae6e283cafb2178612b20028d35357f0cafaac83", - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ca87df1936fd2694fcc768079e9e8f926b6d45969ddc61ebf353f5e726d70b8b", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "1258655fcdca4692c2306100846761a2b0b149bbcaa5cde48a9abaa559913bc3", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "a8257f98c2e30c7bc7978bf645966d2eaa98cd49a42a8a923c062b5808dcdcdf", - "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "118afcc28379dd73feec763fbb9cfc9f17c6a208afe5969a85b3c969b8059552", - "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "fa1ac8fd375d754503b3507fae006381138d8875afdad2418f1ea5ae338b9c26" - }, - "scope": "Focused state/block tests and native four-rank subset attention; no full-model quality/performance acceptance", - "status": "completed", - "returncode": 0, - "elapsed_seconds": 81.50971999001922 - }, - { - "trial": "c15-video-row-quality-5s-00-710315e0", - "model_controller": { - "status": "failed", - "returncode": 1, - "elapsed_seconds": 153.44603093498154 - }, - "actual_counters": { - "request-000\\rank-0\\attention_path_counts.json": {}, - "request-000\\rank-1\\attention_path_counts.json": {}, - "request-000\\rank-2\\attention_path_counts.json": {}, - "request-000\\rank-3\\attention_path_counts.json": {} - } - }, - { - "trial": "c15-video-row-v2-quality-5s-00-d6b921b9", - "model_controller": { - "status": "failed", - "returncode": 1, - "elapsed_seconds": 199.99670764501207 - }, - "actual_counters": { - "request-000\\rank-0\\video_row_cache_stats.json": { - "request_counters": { - "full_steps": 5, - "partial_steps": 1, - "full_blocks": 250, - "partial_blocks": 1, - "subset_attention_calls": 1, - "subset_query_rows": 2475, - "omitted_query_rows": 37296, - "partial_mlp_rows": 2475, - "cache_bytes": 4019635200, - "closed": true - }, - "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" - }, - "request-000\\rank-1\\video_row_cache_stats.json": { - "request_counters": { - "full_steps": 5, - "partial_steps": 1, - "full_blocks": 250, - "partial_blocks": 1, - "subset_attention_calls": 1, - "subset_query_rows": 2475, - "omitted_query_rows": 37296, - "partial_mlp_rows": 0, - "cache_bytes": 5350195200, - "closed": true - }, - "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" - }, - "request-000\\rank-2\\video_row_cache_stats.json": { - "request_counters": { - "full_steps": 5, - "partial_steps": 1, - "full_blocks": 250, - "partial_blocks": 1, - "subset_attention_calls": 1, - "subset_query_rows": 2475, - "omitted_query_rows": 37296, - "partial_mlp_rows": 0, - "cache_bytes": 5350195200, - "closed": true - }, - "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" - }, - "request-000\\rank-3\\video_row_cache_stats.json": { - "request_counters": { - "full_steps": 5, - "partial_steps": 1, - "full_blocks": 250, - "partial_blocks": 1, - "subset_attention_calls": 1, - "subset_query_rows": 2475, - "omitted_query_rows": 37296, - "partial_mlp_rows": 0, - "cache_bytes": 5330304000, - "closed": true - }, - "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" - }, - "request-000\\rank-0\\attention_path_counts.json": { - "_prefix_bnsd": 251 - }, - "request-000\\rank-1\\attention_path_counts.json": { - "_prefix_bnsd": 251 - }, - "request-000\\rank-2\\attention_path_counts.json": { - "_prefix_bnsd": 251 - }, - "request-000\\rank-3\\attention_path_counts.json": { - "_prefix_bnsd": 251 - } - } - }, - { - "trial": "c15-video-row-v3-quality-5s-00-70703586", - "model_controller": { - "status": "failed", - "returncode": 1, - "elapsed_seconds": 191.09544755500974 - }, - "actual_counters": { - "request-000\\rank-0\\video_row_cache_stats.json": { - "request_counters": { - "full_steps": 5, - "partial_steps": 1, - "full_blocks": 250, - "partial_blocks": 50, - "subset_attention_calls": 50, - "subset_query_rows": 123750, - "omitted_query_rows": 1864800, - "partial_mlp_rows": 123750, - "cache_bytes": 4019635200, - "closed": true - }, - "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" - }, - "request-000\\rank-1\\video_row_cache_stats.json": { - "request_counters": { - "full_steps": 5, - "partial_steps": 1, - "full_blocks": 250, - "partial_blocks": 50, - "subset_attention_calls": 50, - "subset_query_rows": 123750, - "omitted_query_rows": 1864800, - "partial_mlp_rows": 0, - "cache_bytes": 5350195200, - "closed": true - }, - "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" - }, - "request-000\\rank-2\\video_row_cache_stats.json": { - "request_counters": { - "full_steps": 5, - "partial_steps": 1, - "full_blocks": 250, - "partial_blocks": 50, - "subset_attention_calls": 50, - "subset_query_rows": 123750, - "omitted_query_rows": 1864800, - "partial_mlp_rows": 0, - "cache_bytes": 5350195200, - "closed": true - }, - "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" - }, - "request-000\\rank-3\\video_row_cache_stats.json": { - "request_counters": { - "full_steps": 5, - "partial_steps": 1, - "full_blocks": 250, - "partial_blocks": 50, - "subset_attention_calls": 50, - "subset_query_rows": 123750, - "omitted_query_rows": 1864800, - "partial_mlp_rows": 0, - "cache_bytes": 5330304000, - "closed": true - }, - "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" - }, - "request-000\\rank-0\\attention_path_counts.json": { - "_prefix_bnsd": 300 - }, - "request-000\\rank-1\\attention_path_counts.json": { - "_prefix_bnsd": 300 - }, - "request-000\\rank-2\\attention_path_counts.json": { - "_prefix_bnsd": 300 - }, - "request-000\\rank-3\\attention_path_counts.json": { - "_prefix_bnsd": 300 - } - } - }, - { - "trial": "c16-video-row-fp32-native-d3fe320c", - "cleanup_passed": true, - "source_unchanged": true, - "required_native_test_executed": true, - "files_before": { - "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "c555d76f7bd6457a81238fca56100d11cb7ab043a4a649d55e23b6203d09bde2", - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ca87df1936fd2694fcc768079e9e8f926b6d45969ddc61ebf353f5e726d70b8b", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "1258655fcdca4692c2306100846761a2b0b149bbcaa5cde48a9abaa559913bc3", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "a8257f98c2e30c7bc7978bf645966d2eaa98cd49a42a8a923c062b5808dcdcdf", - "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "c5fab405ee82bc553e9a902fa27e6dfbc5213ec18311e17b6415396a69b50f43", - "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "e09b872f6900e821d42632a0ca09595f2affe6f9794049dbc74d69ba6a52b2b0", - "vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py": "e8e2e74fb8edb05e02d3e2328ea1f3ad5ddcfd7a10ed09c5cf3911b1d063e849" - }, - "scope": "Focused state/block tests and native four-rank subset attention; no full-model quality/performance acceptance", - "status": "completed", - "returncode": 0, - "elapsed_seconds": 78.8189791989862 - }, - { - "trial": "c16-video-row-fp32-quality-5s-00-8600757a", - "model_controller": { - "status": "completed", - "returncode": 0, - "elapsed_seconds": 355.51310362201184 - }, - "model_diagnostic": { - "source_head": "e0a3462fcc44de698b5dedcf8ff2c99f9802f757", - "samples_ms": [ - 197716.36760496767 - ], - "latency_median_ms": 197716.36760496767, - "scope": "http_complete_mp4", - "warmup": 0, - "repetitions": 1, - "kind": "quality", - "diagnostic": "audit", - "clean_timing": false, - "cleanup_passed": true - }, - "actual_counters": { - "request-000\\rank-0\\video_row_cache_stats.json": { - "request_counters": { - "full_steps": 17, - "partial_steps": 33, - "full_blocks": 850, - "partial_blocks": 1650, - "subset_attention_calls": 1650, - "subset_query_rows": 4083750, - "omitted_query_rows": 61538400, - "partial_mlp_rows": 4083750, - "peak_cache_bytes": 8039270400, - "closed": true, - "storage_dtype": "torch.float32", - "live_cache_bytes": 0 - }, - "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" - }, - "request-000\\rank-1\\video_row_cache_stats.json": { - "request_counters": { - "full_steps": 17, - "partial_steps": 33, - "full_blocks": 850, - "partial_blocks": 1650, - "subset_attention_calls": 1650, - "subset_query_rows": 4083750, - "omitted_query_rows": 61538400, - "partial_mlp_rows": 0, - "peak_cache_bytes": 10700390400, - "closed": true, - "storage_dtype": "torch.float32", - "live_cache_bytes": 0 - }, - "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" - }, - "request-000\\rank-2\\video_row_cache_stats.json": { - "request_counters": { - "full_steps": 17, - "partial_steps": 33, - "full_blocks": 850, - "partial_blocks": 1650, - "subset_attention_calls": 1650, - "subset_query_rows": 4083750, - "omitted_query_rows": 61538400, - "partial_mlp_rows": 0, - "peak_cache_bytes": 10700390400, - "closed": true, - "storage_dtype": "torch.float32", - "live_cache_bytes": 0 - }, - "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" - }, - "request-000\\rank-3\\video_row_cache_stats.json": { - "request_counters": { - "full_steps": 17, - "partial_steps": 33, - "full_blocks": 850, - "partial_blocks": 1650, - "subset_attention_calls": 1650, - "subset_query_rows": 4083750, - "omitted_query_rows": 61538400, - "partial_mlp_rows": 0, - "peak_cache_bytes": 10660608000, - "closed": true, - "storage_dtype": "torch.float32", - "live_cache_bytes": 0 - }, - "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" - }, - "request-000\\rank-0\\attention_path_counts.json": { - "_prefix_bnsd": 2500 - }, - "request-000\\rank-1\\attention_path_counts.json": { - "_prefix_bnsd": 2500 - }, - "request-000\\rank-2\\attention_path_counts.json": { - "_prefix_bnsd": 2500 - }, - "request-000\\rank-3\\attention_path_counts.json": { - "_prefix_bnsd": 2500 - } - } - }, - { - "trial": "c17-video-window-native-9b48cb56", - "cleanup_passed": true, - "source_unchanged": true, - "required_native_test_executed": true, - "files_before": { - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "66ca27991ce42ce302e5de078be6395c9d1ab53752378babef25089f3f75c3aa", - "tests/diffusion/models/minimax_h3/test_minimax_h3_video_window.py": "0bc59f35eff41164e125a5070e034c94ea60bf0112733ebe59eb6378eccbfed8" - }, - "scope": "CPU window reference/routing and native batched BF16 FAS; no full-model quality/performance acceptance", - "status": "completed", - "returncode": 0, - "elapsed_seconds": 51.878022204036824 - }, - { - "trial": "c17-video-window-operator-9bf7de3e", - "cleanup_passed": true, - "source_unchanged": true, - "files_before": { - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "66ca27991ce42ce302e5de078be6395c9d1ab53752378babef25089f3f75c3aa", - "tests/diffusion/models/minimax_h3/test_minimax_h3_video_window.py": "0bc59f35eff41164e125a5070e034c94ea60bf0112733ebe59eb6378eccbfed8" - }, - "scope": "Real captured H3 attention operator benchmark; full-model quality and HTTP are separate; no full-model quality/performance acceptance", - "status": "completed", - "returncode": 0, - "elapsed_seconds": 51.778123575029895, - "operator_benchmark": { - "scope": "real captured 5s attention operation on logical2; no model quality or HTTP acceptance", - "fixture_sha256": "00767e6afb2ba777bb2677eb9b60dc2f00fb649f5006ae8c1d77191214fbf912", - "fixture_tensor_sha256": { - "query": "313095dd143adf0636d57c90d77b689405dee510df3ebd6385a49d071c3323e9", - "key": "6638aee85f0039ee814206e7c976a2d1a9a2b185723c7bbb74c69b1e68c5bad4", - "value": "101eede996c8b13d61d487019d47f0b5fd2193664d8c178665063675cb66fdca" - }, - "query_shape": [ - 1, - 39808, - 14, - 128 - ], - "valid_kv": 39771, - "frames": 37, - "rows_per_frame": 1008, - "target_start": 2475, - "window": 3, - "finite": true, - "relative_l2_vs_dense": 0.5161040425300598, - "baseline_samples_ms": [ - 69.0619180095382, - 69.60304302629083, - 69.27597103640437, - 69.81903495034203, - 69.96654602698982 - ], - "candidate_samples_ms": [ - 25.191069988068193, - 25.301200977992266, - 25.325032009277493, - 25.26562102138996, - 25.219489994924515 - ], - "baseline_median_ms": 69.60304302629083, - "candidate_median_ms": 25.26562102138996, - "reduction_pct": 63.70040744936038, - "operator_gate_passed": true, - "warmups_per_operation": 2, - "measured_per_operation": 5, - "order": "AB/BA/AB/BA/AB", - "probe_sha256": "2465cf08173950a241ac3cc4713a4227d4c4b5a87f35358692a7ef6b60821824" - } - }, - { - "trial": "c17-video-window-quality-5s-00-a8c689e4", - "model_controller": { - "status": "completed", - "returncode": 0, - "elapsed_seconds": 358.6752056070254 - }, - "model_diagnostic": { - "source_head": "c1589a2b5e4ada617e7cca38ef7c300fe9495c58", - "samples_ms": [ - 210397.12271699682 - ], - "latency_median_ms": 210397.12271699682, - "scope": "http_complete_mp4", - "warmup": 0, - "repetitions": 1, - "kind": "quality", - "diagnostic": "audit", - "clean_timing": false, - "cleanup_passed": true - }, - "actual_counters": { - "request-000\\rank-0\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 4900, - "_video_temporal_window_bnsd": 2400 - }, - "request-000\\rank-1\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 4900, - "_video_temporal_window_bnsd": 2400 - }, - "request-000\\rank-2\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 4900, - "_video_temporal_window_bnsd": 2400 - }, - "request-000\\rank-3\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 4900, - "_video_temporal_window_bnsd": 2400 - } - } - }, - { - "trial": "c18-video-pooled-context-native-1673db72", - "cleanup_passed": true, - "source_unchanged": true, - "required_native_test_executed": true, - "files_before": { - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "1febc030a7917b6e10fe287014fe2aacef76ade7bf58ebb604fdedc36fb7c21d", - "tests/diffusion/models/minimax_h3/test_minimax_h3_pooled_context.py": "45a9ae5ba56858daee6a1088a8c8b1ed612ea9afd3c229c90a9b6db3c80a06b2" - }, - "scope": "Independent weighted global-context reference and native padded-head BF16 FAS; no full-model quality/performance acceptance", - "status": "completed", - "returncode": 0, - "elapsed_seconds": 49.97671151399845 - }, - { - "trial": "c18-video-pooled-context-operator-315db0e5", - "cleanup_passed": true, - "source_unchanged": true, - "files_before": { - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "1febc030a7917b6e10fe287014fe2aacef76ade7bf58ebb604fdedc36fb7c21d", - "tests/diffusion/models/minimax_h3/test_minimax_h3_pooled_context.py": "45a9ae5ba56858daee6a1088a8c8b1ed612ea9afd3c229c90a9b6db3c80a06b2" - }, - "scope": "Real captured H3 attention operator benchmark; full-model quality and HTTP are separate; no full-model quality/performance acceptance", - "status": "completed", - "returncode": 0, - "elapsed_seconds": 51.37963652302278, - "operator_benchmark": { - "scope": "real captured 5s attention operation on logical2; no model quality or HTTP acceptance", - "fixture_sha256": "00767e6afb2ba777bb2677eb9b60dc2f00fb649f5006ae8c1d77191214fbf912", - "fixture_tensor_sha256": { - "query": "313095dd143adf0636d57c90d77b689405dee510df3ebd6385a49d071c3323e9", - "key": "6638aee85f0039ee814206e7c976a2d1a9a2b185723c7bbb74c69b1e68c5bad4", - "value": "101eede996c8b13d61d487019d47f0b5fd2193664d8c178665063675cb66fdca" - }, - "query_shape": [ - 1, - 39808, - 14, - 128 - ], - "valid_kv": 39771, - "frames": 37, - "rows_per_frame": 1008, - "target_start": 2475, - "profile": "3x3-local1", - "finite": true, - "relative_l2_vs_dense": 0.21801254153251648, - "baseline_samples_ms": [ - 69.19049099087715, - 69.70348599134013, - 69.43478400353342, - 69.81050700414926, - 69.24652203451842 - ], - "candidate_samples_ms": [ - 68.29233502503484, - 64.26435202592984, - 71.05128600960597, - 64.39994304673746, - 64.39412303734571 - ], - "baseline_median_ms": 69.43478400353342, - "candidate_median_ms": 64.39994304673746, - "reduction_pct": 7.251179691924648, - "operator_gate_passed": false, - "warmups_per_operation": 2, - "measured_per_operation": 5, - "order": "AB/BA/AB/BA/AB", - "probe_sha256": "42852c0e4e693c5b3685981db617de00648feefe651f7060ad83c359b08dd81d" - } - }, - { - "trial": "c19-video-global-pool-native-f0f4c502", - "cleanup_passed": true, - "source_unchanged": true, - "required_native_test_executed": true, - "files_before": { - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ddfd6d6822cc6a5528fc95539f7f98dfdeed25d5489da3ed5cf76e6076a5ec94", - "tests/diffusion/models/minimax_h3/test_minimax_h3_global_pool.py": "5457db550b59e3373be8f5f21873c1b78ac357a6ac170c2603ee531b50e91467" - }, - "scope": "Independent weighted global-context reference and native padded-head BF16 FAS; no full-model quality/performance acceptance", - "status": "completed", - "returncode": 0, - "elapsed_seconds": 51.528875903983135 - }, - { - "trial": "c19-video-global-pool-operator-933511c9", - "cleanup_passed": true, - "source_unchanged": true, - "files_before": { - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ddfd6d6822cc6a5528fc95539f7f98dfdeed25d5489da3ed5cf76e6076a5ec94", - "tests/diffusion/models/minimax_h3/test_minimax_h3_global_pool.py": "5457db550b59e3373be8f5f21873c1b78ac357a6ac170c2603ee531b50e91467" - }, - "scope": "Real captured H3 attention operator benchmark; full-model quality and HTTP are separate; no full-model quality/performance acceptance", - "status": "completed", - "returncode": 0, - "elapsed_seconds": 51.02866222098237, - "operator_benchmark": { - "scope": "real captured 5s attention operation on logical2; no model quality or HTTP acceptance", - "fixture_sha256": "00767e6afb2ba777bb2677eb9b60dc2f00fb649f5006ae8c1d77191214fbf912", - "fixture_tensor_sha256": { - "query": "313095dd143adf0636d57c90d77b689405dee510df3ebd6385a49d071c3323e9", - "key": "6638aee85f0039ee814206e7c976a2d1a9a2b185723c7bbb74c69b1e68c5bad4", - "value": "101eede996c8b13d61d487019d47f0b5fd2193664d8c178665063675cb66fdca" - }, - "query_shape": [ - 1, - 39808, - 14, - 128 - ], - "valid_kv": 39771, - "frames": 37, - "rows_per_frame": 1008, - "target_start": 2475, - "profile": "3x3-global", - "finite": true, - "relative_l2_vs_dense": 0.2588050067424774, - "baseline_samples_ms": [ - 69.95450798422098, - 69.79658600175753, - 69.58012498216704, - 70.22821099963039, - 69.11802099784836 - ], - "candidate_samples_ms": [ - 32.91054297005758, - 32.899541954975575, - 32.924533006735146, - 32.779561006464064, - 32.96928299823776 - ], - "baseline_median_ms": 69.79658600175753, - "candidate_median_ms": 32.91054297005758, - "reduction_pct": 52.847918708761966, - "operator_gate_passed": false, - "warmups_per_operation": 2, - "measured_per_operation": 5, - "order": "AB/BA/AB/BA/AB", - "probe_sha256": "a27906d8bb0b65e2f6ffb41ddeb4b513163b7bf23b9d2dbd33a5015fc93b2a08" - } - }, - { - "trial": "c20-video-variance-pool-native-50101063", - "cleanup_passed": true, - "source_unchanged": true, - "required_native_test_executed": true, - "files_before": { - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "4390450bc5324f7028125101cd4bb32e9de237b04bc12fa879b48110d7122e3c", - "tests/diffusion/models/minimax_h3/test_minimax_h3_variance_pool.py": "e871fb4b222d61d10b6c442a333a52c976cceaa0d406233502ecbcaddfeba8c9" - }, - "scope": "Independent weighted global-context reference and native padded-head BF16 FAS; no full-model quality/performance acceptance", - "status": "completed", - "returncode": 0, - "elapsed_seconds": 50.230675689992495 - }, - { - "trial": "c20-video-variance-pool-operator-209f263d", - "cleanup_passed": true, - "source_unchanged": true, - "files_before": { - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "4390450bc5324f7028125101cd4bb32e9de237b04bc12fa879b48110d7122e3c", - "tests/diffusion/models/minimax_h3/test_minimax_h3_variance_pool.py": "e871fb4b222d61d10b6c442a333a52c976cceaa0d406233502ecbcaddfeba8c9" - }, - "scope": "Real captured H3 attention operator benchmark; full-model quality and HTTP are separate; no full-model quality/performance acceptance", - "status": "completed", - "returncode": 0, - "elapsed_seconds": 49.880220040970016, - "operator_benchmark": { - "scope": "real captured 5s attention operation on logical2; no model quality or HTTP acceptance", - "fixture_sha256": "00767e6afb2ba777bb2677eb9b60dc2f00fb649f5006ae8c1d77191214fbf912", - "fixture_tensor_sha256": { - "query": "313095dd143adf0636d57c90d77b689405dee510df3ebd6385a49d071c3323e9", - "key": "6638aee85f0039ee814206e7c976a2d1a9a2b185723c7bbb74c69b1e68c5bad4", - "value": "101eede996c8b13d61d487019d47f0b5fd2193664d8c178665063675cb66fdca" - }, - "query_shape": [ - 1, - 39808, - 14, - 128 - ], - "valid_kv": 39771, - "frames": 37, - "rows_per_frame": 1008, - "target_start": 2475, - "profile": "3x3-variance", - "finite": true, - "relative_l2_vs_dense": 0.26336145401000977, - "baseline_samples_ms": [ - 69.74446203093976, - 69.06059599714354, - 69.64961998164654, - 69.69038100214675, - 69.62426099926233 - ], - "candidate_samples_ms": [ - 35.81449302146211, - 36.30960697773844, - 35.925014002714306, - 35.99672397831455, - 35.80518299713731 - ], - "baseline_median_ms": 69.64961998164654, - "candidate_median_ms": 35.925014002714306, - "reduction_pct": 48.42037327385138, - "operator_gate_passed": false, - "warmups_per_operation": 2, - "measured_per_operation": 5, - "order": "AB/BA/AB/BA/AB", - "probe_sha256": "785e7a0c13719e37d5cc44fb66af05649e51e1f7e5d0b212e92ec44f79cb0184" - } - }, - { - "trial": "c21-video-pre-gate-native-3c5eebbe", - "cleanup_passed": true, - "source_unchanged": true, - "required_native_test_executed": true, - "files_before": { - "tests/diffusion/distributed/test_collective_gate.py": "6a02f32882d85eca1e3bb9a023ce3adb3d5b59c213845e11e85c96a89432f477", - "tests/diffusion/models/minimax_h3/test_minimax_h3_pre_gate_cache.py": "a4924d0b3dd58ae413259edba6cf2487b3e41d391f943fb74358934be72eb53b", - "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "e09b872f6900e821d42632a0ca09595f2affe6f9794049dbc74d69ba6a52b2b0", - "vllm_omni/config/omni_config.py": "6894120a330474f718d43bf2bf7928892eb996152839ce6469c34d18a7df09d1", - "vllm_omni/config/stage_config.py": "0752e8cc1c6f1021f737830d08663cfc118923a83fd909c607f9ec3ab2a3d2fb", - "vllm_omni/diffusion/attention/layer.py": "76c5fce3dabd23e39a8890316e1691c03bc203066cb60af5611ec4790eefc632", - "vllm_omni/diffusion/data.py": "5db83d5c71cb27cc7810d35fcfc891da80f0b003adbb16bdecc3816a63305e11", - "vllm_omni/diffusion/distributed/collective_gate.py": "01c6317cfa615840681b8bb8b8513fa3736d26e845a24ad9e1ab7f87489b962f", - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ca87df1936fd2694fcc768079e9e8f926b6d45969ddc61ebf353f5e726d70b8b", - "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "c5fab405ee82bc553e9a902fa27e6dfbc5213ec18311e17b6415396a69b50f43", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "8933b536317652d0dbc7a974ea7e0bca928e7b3bdf438988999ff5a498a6ec9d", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "019bf71106ba3263b55ad7b682064e3a176eecb81725cb48bbf178387356d5d9", - "vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py": "e8e2e74fb8edb05e02d3e2328ea1f3ad5ddcfd7a10ed09c5cf3911b1d063e849", - "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "04a9212994b5c8593f8d196820300ded077fed9af4274b64d87d9e6bf71eae07", - "vllm_omni/diffusion/offloader/base.py": "23922136deaf54e2d98e827b007b34c7e3ad36171b71bfe9060d390efc0b679f", - "vllm_omni/diffusion/offloader/distributed_layerwise_backend.py": "19fbce0f1036e4784d68cab798c499526a9099101909e41ce96dd0caa033b6a4", - "vllm_omni/engine/arg_utils.py": "52569e3688cc708729a48ca0a2c875c28193733c2f934f6ff89f6c815e4009c7", - "vllm_omni/entrypoints/cli/serve.py": "f8284ad6b751ba4a9e4f7254d78132cc2cfd49871a0f9af7fde49c6789d79cb0" - }, - "scope": "Legacy/pre-gate/DLO tests, two executed native four-rank HCCL/subsetFA suites and fresh current gates; no full-model quality/performance acceptance", - "status": "completed", - "returncode": 0, - "elapsed_seconds": 127.31179964594776 - }, - { - "trial": "c21-video-pre-gate-quality-5s-00-7265c33e", - "model_controller": { - "status": "completed", - "returncode": 0, - "elapsed_seconds": 318.92388185299933 - }, - "model_diagnostic": { - "source_head": "d48fa51d03dd748568cbcf57ed40f533c691c9bd", - "samples_ms": [ - 163805.5620979867 - ], - "latency_median_ms": 163805.5620979867, - "scope": "http_complete_mp4", - "warmup": 0, - "repetitions": 1, - "kind": "quality", - "diagnostic": "audit", - "clean_timing": false, - "cleanup_passed": true - }, - "actual_counters": { - "request-000\\rank-0\\video_row_cache_stats.json": { - "request_counters": { - "full_steps": 8, - "partial_steps": 42, - "full_blocks": 400, - "partial_blocks": 2100, - "subset_attention_calls": 2100, - "subset_query_rows": 5197500, - "omitted_query_rows": 78321600, - "partial_mlp_rows": 5197500, - "peak_cache_bytes": 8039270400, - "closed": true, - "storage_dtype": "torch.bfloat16", - "cache_stage": "pre_gate", - "fresh_attention_gates": 2100, - "fresh_mlp_gates": 2100, - "live_cache_bytes": 0 - }, - "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" - }, - "request-000\\rank-1\\video_row_cache_stats.json": { - "request_counters": { - "full_steps": 8, - "partial_steps": 42, - "full_blocks": 400, - "partial_blocks": 2100, - "subset_attention_calls": 2100, - "subset_query_rows": 5197500, - "omitted_query_rows": 78321600, - "partial_mlp_rows": 0, - "peak_cache_bytes": 10700390400, - "closed": true, - "storage_dtype": "torch.bfloat16", - "cache_stage": "pre_gate", - "fresh_attention_gates": 2100, - "fresh_mlp_gates": 2100, - "live_cache_bytes": 0 - }, - "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" - }, - "request-000\\rank-2\\video_row_cache_stats.json": { - "request_counters": { - "full_steps": 8, - "partial_steps": 42, - "full_blocks": 400, - "partial_blocks": 2100, - "subset_attention_calls": 2100, - "subset_query_rows": 5197500, - "omitted_query_rows": 78321600, - "partial_mlp_rows": 0, - "peak_cache_bytes": 10700390400, - "closed": true, - "storage_dtype": "torch.bfloat16", - "cache_stage": "pre_gate", - "fresh_attention_gates": 2100, - "fresh_mlp_gates": 2100, - "live_cache_bytes": 0 - }, - "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" - }, - "request-000\\rank-3\\video_row_cache_stats.json": { - "request_counters": { - "full_steps": 8, - "partial_steps": 42, - "full_blocks": 400, - "partial_blocks": 2100, - "subset_attention_calls": 2100, - "subset_query_rows": 5197500, - "omitted_query_rows": 78321600, - "partial_mlp_rows": 0, - "peak_cache_bytes": 10660608000, - "closed": true, - "storage_dtype": "torch.bfloat16", - "cache_stage": "pre_gate", - "fresh_attention_gates": 2100, - "fresh_mlp_gates": 2100, - "live_cache_bytes": 0 - }, - "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" - }, - "request-000\\rank-0\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-000\\rank-1\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-000\\rank-2\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-000\\rank-3\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - } - } - }, - { - "trial": "c22-video-pre-gate-secant-native-3521747e", - "cleanup_passed": true, - "source_unchanged": true, - "required_native_test_executed": true, - "files_before": { - "tests/diffusion/distributed/test_collective_gate.py": "6a02f32882d85eca1e3bb9a023ce3adb3d5b59c213845e11e85c96a89432f477", - "tests/diffusion/models/minimax_h3/test_minimax_h3_pre_gate_cache.py": "b8394d22b054d60c1ab8e069e9600e1947cc73b5826e1f206e66c548123f632d", - "tests/diffusion/models/minimax_h3/test_minimax_h3_pre_gate_secant.py": "70a215fbb0dc63b90652d96f8f6d106a5785bd7d64dae75a2c4f6ec5c627ac1c", - "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "e09b872f6900e821d42632a0ca09595f2affe6f9794049dbc74d69ba6a52b2b0", - "vllm_omni/config/omni_config.py": "6894120a330474f718d43bf2bf7928892eb996152839ce6469c34d18a7df09d1", - "vllm_omni/config/stage_config.py": "0752e8cc1c6f1021f737830d08663cfc118923a83fd909c607f9ec3ab2a3d2fb", - "vllm_omni/diffusion/attention/layer.py": "76c5fce3dabd23e39a8890316e1691c03bc203066cb60af5611ec4790eefc632", - "vllm_omni/diffusion/data.py": "5db83d5c71cb27cc7810d35fcfc891da80f0b003adbb16bdecc3816a63305e11", - "vllm_omni/diffusion/distributed/collective_gate.py": "01c6317cfa615840681b8bb8b8513fa3736d26e845a24ad9e1ab7f87489b962f", - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ca87df1936fd2694fcc768079e9e8f926b6d45969ddc61ebf353f5e726d70b8b", - "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "b991ebb5d43242357d5ab8cfb476808a76fc0658d6048db6f3b7131122c8a993", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "8933b536317652d0dbc7a974ea7e0bca928e7b3bdf438988999ff5a498a6ec9d", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "3904bc8ac6679bc51614b95f5510789331a41bf20a43a517f3a99d33505201a6", - "vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py": "e8e2e74fb8edb05e02d3e2328ea1f3ad5ddcfd7a10ed09c5cf3911b1d063e849", - "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "e21e0afe8f8948d805d374428a34f3b2c5c0b5041d63ea791f047af8f455cf16", - "vllm_omni/diffusion/offloader/base.py": "23922136deaf54e2d98e827b007b34c7e3ad36171b71bfe9060d390efc0b679f", - "vllm_omni/diffusion/offloader/distributed_layerwise_backend.py": "19fbce0f1036e4784d68cab798c499526a9099101909e41ce96dd0caa033b6a4", - "vllm_omni/engine/arg_utils.py": "52569e3688cc708729a48ca0a2c875c28193733c2f934f6ff89f6c815e4009c7", - "vllm_omni/entrypoints/cli/serve.py": "f8284ad6b751ba4a9e4f7254d78132cc2cfd49871a0f9af7fde49c6789d79cb0" - }, - "scope": "Legacy/pre-gate/DLO tests, three executed native four-rank HCCL/subsetFA suites and fresh current gates; no full-model quality/performance acceptance", - "status": "completed", - "returncode": 0, - "elapsed_seconds": 154.04585235595005 - }, - { - "trial": "c22-video-pre-gate-secant-quality-5s-00-72202558", - "model_controller": { - "status": "failed", - "returncode": 1, - "elapsed_seconds": 5.625923136016354 - } - } - ], - "decisions": [ - "C12 fixed profile stopped: zero actual predictions.", - "C14 pinned block-FP8 and block-INT8 prerequisites failed; per-token INT8 is not excluded.", - "C13 numerical qualification and clean performance are separate.", - "C15 fixed FP16 video residual profile failed finite velocity at first reuse.", - "C16 FP32 residual completed but failed three frozen audio quality guards; fixed profile stopped.", - "C17 completed5s but failed11frozen quality checks; fixed window profile stopped.", - "C18 native17PASS but operation reduced only7.25%; no full model run.", - "C19 native17PASS; operation52.85%faster but relL2 .258805 exceeded fixed .258052screen; no model run.", - "C20 native17PASS; operation48.42%faster and relL2 .263361 failed both fixed primitive gates; no model run.", - "C21 native36PASS; full5s completed but failed7frozen quality checks; fixed zero-order profile stopped.", - "C22 native45PASS; same eight-anchor schedule with sigma secants requires independent media qualification.", - "First C22 model stopped before startup on duplicate controller aliases; audited canonicalization retains all attempts and time charges." - ] -} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c12-through-c23-execution-checkpoint.json b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c12-through-c23-execution-checkpoint.json deleted file mode 100644 index 459b9a3d3..000000000 --- a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c12-through-c23-execution-checkpoint.json +++ /dev/null @@ -1,2080 +0,0 @@ -{ - "campaign_verdict": "INCONCLUSIVE", - "performance_acceptance": "No newly accepted gain", - "runs": [ - { - "trial": "c12-alignment-quality-5s-00-9bf5c373", - "model_controller": { - "status": "completed", - "returncode": 0, - "elapsed_seconds": 471.80311120999977 - }, - "model_diagnostic": { - "source_head": "a29648e5acd3077b7b22739bc9522017782fe1ac", - "samples_ms": [ - 318280.59680899605 - ], - "latency_median_ms": 318280.59680899605, - "scope": "http_complete_mp4", - "warmup": 0, - "repetitions": 1, - "kind": "quality", - "diagnostic": "audit", - "clean_timing": false, - "cleanup_passed": true - }, - "actual_counters": { - "request-000\\rank-0\\alignment_cache_stats.json": { - "request_counters": { - "computed": 50, - "predicted": 0, - "vetoes": 45 - }, - "scope": "actual state-aligned compute/forecast/veto counts; not quality acceptance" - }, - "request-000\\rank-1\\alignment_cache_stats.json": { - "request_counters": { - "computed": 50, - "predicted": 0, - "vetoes": 45 - }, - "scope": "actual state-aligned compute/forecast/veto counts; not quality acceptance" - }, - "request-000\\rank-2\\alignment_cache_stats.json": { - "request_counters": { - "computed": 50, - "predicted": 0, - "vetoes": 45 - }, - "scope": "actual state-aligned compute/forecast/veto counts; not quality acceptance" - }, - "request-000\\rank-3\\alignment_cache_stats.json": { - "request_counters": { - "computed": 50, - "predicted": 0, - "vetoes": 45 - }, - "scope": "actual state-aligned compute/forecast/veto counts; not quality acceptance" - }, - "request-000\\rank-0\\attention_path_counts.json": { - "_prefix_bnsd": 2500 - }, - "request-000\\rank-1\\attention_path_counts.json": { - "_prefix_bnsd": 2500 - }, - "request-000\\rank-2\\attention_path_counts.json": { - "_prefix_bnsd": 2500 - }, - "request-000\\rank-3\\attention_path_counts.json": { - "_prefix_bnsd": 2500 - } - } - }, - { - "trial": "c12-state-alignment-native-tests-0dc5859b", - "cleanup_passed": true, - "source_unchanged": true, - "native_test_executed": true, - "files_before": { - "vllm_omni/diffusion/models/minimax_h3/cache_state_alignment.py": "3340ec21a77aef78531c601be241bcfb1e44107c795e8c8649907d1a18ed377f", - "vllm_omni/diffusion/models/minimax_h3/reference_kv_cache_dit.py": "f4cd0e9aa9e824c92ec1acd02c57a8f7d8684c35d4edf92ffdfae3345ff44e99", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "436ec52cd6329cbc64e91d94b450bfa6f773a9a953b8f974db73a5eb986a98bb", - "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "4dec8f02a04bfe7e8d2ada24fc6d5ae40a0058671d8515c8a266ecf88824b7b0", - "tests/diffusion/models/minimax_h3/test_minimax_h3_state_cache.py": "89cf57482b35d8d8ad8d4978aec87e9ee3ef45330e5b911161adef4379950a65" - }, - "scope": "focused tests only; no full-model quality or performance acceptance", - "status": "completed", - "returncode": 0, - "elapsed_seconds": 82.06077561300481 - }, - { - "trial": "c13-baseline-cuda-staging-e22d8651", - "cleanup_passed": true, - "source_unchanged": true, - "native_test_executed": false, - "files_before": { - "vllm_omni/diffusion/offloader/sequential_backend.py": "d542935b656a0d9dced78de8fb086b8e1c5aade26fac6cee5316b32a49f0d569", - "tests/diffusion/offloader/test_sequential_backend.py": "dcf1d64e65bed572d65b095c1350950f55d7e8d9ddf8c7dfd72d0ea6e721685a" - }, - "scope": "Reproduce stock CUDA-graph staging expectation on unchanged migration owning source in the same Ascend test environment", - "status": "failed", - "returncode": 1, - "elapsed_seconds": 50.234599059971515 - }, - { - "trial": "c13-c1-dlo-5s-audit-cead10d1", - "model_controller": { - "status": "failed", - "returncode": 2, - "elapsed_seconds": 0.11969929095357656 - } - }, - { - "trial": "c13-c1-dlo-5s-audit-v2-b0b86c6d", - "model_controller": { - "status": "completed", - "returncode": 0, - "elapsed_seconds": 749.505351218977 - }, - "model_diagnostic": { - "source_head": "e89b1d40d37f7f72767892a56fa7980ea572ba8e", - "samples_ms": [ - 315593.86175096733, - 284456.94282796467 - ], - "latency_median_ms": 300025.402289466, - "scope": "http_complete_mp4", - "warmup": 0, - "repetitions": 2, - "kind": "diagnostic", - "diagnostic": "audit", - "clean_timing": false, - "cleanup_passed": true - }, - "actual_counters": { - "request-000\\rank-0\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-000\\rank-1\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-000\\rank-2\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-000\\rank-3\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-001\\rank-0\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-001\\rank-1\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-001\\rank-2\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-001\\rank-3\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - } - } - }, - { - "trial": "c13-c1-dlo-5s-calibration-c98129d8", - "model_controller": { - "status": "completed", - "returncode": 0, - "elapsed_seconds": 768.8735517460154 - }, - "model_diagnostic": { - "source_head": "e89b1d40d37f7f72767892a56fa7980ea572ba8e", - "samples_ms": [ - 317177.1984040388, - 285534.089674009 - ], - "latency_median_ms": 301355.6440390239, - "scope": "http_complete_mp4", - "warmup": 0, - "repetitions": 2, - "kind": "calibration", - "diagnostic": "audit", - "clean_timing": false, - "cleanup_passed": true - }, - "actual_counters": { - "request-000\\rank-0\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-000\\rank-1\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-000\\rank-2\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-000\\rank-3\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-001\\rank-0\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-001\\rank-1\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-001\\rank-2\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-001\\rank-3\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - } - } - }, - { - "trial": "c13-dlo-gate-tests-bc92a71d", - "cleanup_passed": true, - "source_unchanged": true, - "native_test_executed": false, - "files_before": { - "tests/diffusion/distributed/test_collective_gate.py": "2b33f73dff9c79118cc2b1bd4d50e3542ebc45b7bf029b38c17abbef37e4b917", - "vllm_omni/config/omni_config.py": "6894120a330474f718d43bf2bf7928892eb996152839ce6469c34d18a7df09d1", - "vllm_omni/config/stage_config.py": "0752e8cc1c6f1021f737830d08663cfc118923a83fd909c607f9ec3ab2a3d2fb", - "vllm_omni/diffusion/attention/layer.py": "76c5fce3dabd23e39a8890316e1691c03bc203066cb60af5611ec4790eefc632", - "vllm_omni/diffusion/data.py": "5db83d5c71cb27cc7810d35fcfc891da80f0b003adbb16bdecc3816a63305e11", - "vllm_omni/diffusion/distributed/collective_gate.py": "01c6317cfa615840681b8bb8b8513fa3736d26e845a24ad9e1ab7f87489b962f", - "vllm_omni/diffusion/offloader/base.py": "533c793119e4cdc14a2ea3c6cb5bf97ffede673befeab3f0fb8dbbf51e655379", - "vllm_omni/diffusion/offloader/distributed_layerwise_backend.py": "d441481513aba9e90a91265f4bba3f53f8766197be8c1302c3c714e690d5ea76", - "vllm_omni/engine/arg_utils.py": "52569e3688cc708729a48ca0a2c875c28193733c2f934f6ff89f6c815e4009c7", - "vllm_omni/entrypoints/cli/serve.py": "350461dfca565bc0b07087ac7af1e27b0e247f5eac717e350b637e37f62dd088", - "tests/diffusion/offloader/helpers.py": "729c73abba3b9a12911282d4bcd3cf8bfa148fbcbfdc2e2b7c86000e387fa687" - }, - "scope": "CPU mock barrier/chunk tests; no actual NPU collective or full-model quality/performance acceptance", - "status": "completed", - "returncode": 0, - "elapsed_seconds": 47.3752507569734 - }, - { - "trial": "c13-dlo-regression-tests-95e9ed2a", - "cleanup_passed": true, - "source_unchanged": true, - "native_test_executed": false, - "files_before": { - "tests/diffusion/distributed/test_collective_gate.py": "2b33f73dff9c79118cc2b1bd4d50e3542ebc45b7bf029b38c17abbef37e4b917", - "vllm_omni/config/omni_config.py": "6894120a330474f718d43bf2bf7928892eb996152839ce6469c34d18a7df09d1", - "vllm_omni/config/stage_config.py": "0752e8cc1c6f1021f737830d08663cfc118923a83fd909c607f9ec3ab2a3d2fb", - "vllm_omni/diffusion/attention/layer.py": "76c5fce3dabd23e39a8890316e1691c03bc203066cb60af5611ec4790eefc632", - "vllm_omni/diffusion/data.py": "5db83d5c71cb27cc7810d35fcfc891da80f0b003adbb16bdecc3816a63305e11", - "vllm_omni/diffusion/distributed/collective_gate.py": "01c6317cfa615840681b8bb8b8513fa3736d26e845a24ad9e1ab7f87489b962f", - "vllm_omni/diffusion/offloader/base.py": "533c793119e4cdc14a2ea3c6cb5bf97ffede673befeab3f0fb8dbbf51e655379", - "vllm_omni/diffusion/offloader/distributed_layerwise_backend.py": "d441481513aba9e90a91265f4bba3f53f8766197be8c1302c3c714e690d5ea76", - "vllm_omni/engine/arg_utils.py": "52569e3688cc708729a48ca0a2c875c28193733c2f934f6ff89f6c815e4009c7", - "vllm_omni/entrypoints/cli/serve.py": "350461dfca565bc0b07087ac7af1e27b0e247f5eac717e350b637e37f62dd088", - "tests/diffusion/offloader/helpers.py": "729c73abba3b9a12911282d4bcd3cf8bfa148fbcbfdc2e2b7c86000e387fa687" - }, - "scope": "CPU mock barrier/chunk tests; no actual NPU collective or full-model quality/performance acceptance", - "status": "failed", - "returncode": 1, - "elapsed_seconds": 62.34616496198578 - }, - { - "trial": "c13-dlo-regression-v2-12c217d5", - "cleanup_passed": true, - "source_unchanged": true, - "native_test_executed": false, - "files_before": { - "tests/diffusion/distributed/test_collective_gate.py": "2b33f73dff9c79118cc2b1bd4d50e3542ebc45b7bf029b38c17abbef37e4b917", - "vllm_omni/config/omni_config.py": "6894120a330474f718d43bf2bf7928892eb996152839ce6469c34d18a7df09d1", - "vllm_omni/config/stage_config.py": "0752e8cc1c6f1021f737830d08663cfc118923a83fd909c607f9ec3ab2a3d2fb", - "vllm_omni/diffusion/attention/layer.py": "76c5fce3dabd23e39a8890316e1691c03bc203066cb60af5611ec4790eefc632", - "vllm_omni/diffusion/data.py": "5db83d5c71cb27cc7810d35fcfc891da80f0b003adbb16bdecc3816a63305e11", - "vllm_omni/diffusion/distributed/collective_gate.py": "01c6317cfa615840681b8bb8b8513fa3736d26e845a24ad9e1ab7f87489b962f", - "vllm_omni/diffusion/offloader/base.py": "533c793119e4cdc14a2ea3c6cb5bf97ffede673befeab3f0fb8dbbf51e655379", - "vllm_omni/diffusion/offloader/distributed_layerwise_backend.py": "d441481513aba9e90a91265f4bba3f53f8766197be8c1302c3c714e690d5ea76", - "vllm_omni/engine/arg_utils.py": "52569e3688cc708729a48ca0a2c875c28193733c2f934f6ff89f6c815e4009c7", - "vllm_omni/entrypoints/cli/serve.py": "350461dfca565bc0b07087ac7af1e27b0e247f5eac717e350b637e37f62dd088", - "tests/diffusion/offloader/helpers.py": "729c73abba3b9a12911282d4bcd3cf8bfa148fbcbfdc2e2b7c86000e387fa687" - }, - "scope": "Expanded offloader tests in Ascend environment; some stock accelerator fixtures use NPU, not a full model test", - "status": "failed", - "returncode": 1, - "elapsed_seconds": 65.14432259398745 - }, - { - "trial": "c14-int8-prerequisites-fdd6633b", - "cleanup_passed": true, - "source_unchanged": true, - "helpers_unchanged": true, - "scope": "Bounded native prerequisites only; a failed probe is retained as negative evidence.", - "status": "failed", - "returncode": 1, - "elapsed_seconds": 20.795623040001374, - "native_prerequisite": { - "verdict": "FAIL", - "scope": "TorchNPU block-INT8 attention prerequisites only", - "phase": "dynamic_block_int8_quant_query", - "versions": { - "torch": "2.10.0+cpu", - "torch_npu": "2.10.0.post4", - "mindiesd": "3.1.0" - }, - "apis": { - "float8_e4m3fn": true, - "npu_dynamic_block_quant": true, - "npu_fused_infer_attention_score_v2": true - }, - "device_name": "Ascend910_9392", - "input": { - "shape": [ - 1, - 14, - 256, - 128 - ], - "dtype": "torch.bfloat16", - "visible_devices": "2" - }, - "error_type": "RuntimeError", - "probe_sha256": "fab18252f860231f9b7fcaa54587afbaa96fce15987fd551922246c8964f7217", - "error": "npu_dynamic_block_quant:../third_party/op-plugin/op_plugin/ops/opapi/DynamicBlockQuantNpuOpApi.cpp:61 NPU function error: call aclnnDynamicBlockQuantV2 failed, error code is 561002\n[ERROR] 2026-11-02-02:00:57 (PID:2662830, Device:0, RankID:-1) ERR00100 PTA call acl api failed.\n[PID: 2662830] 2026-11-02-02:00:57.039.986 Invalid_Argument(EZ0024): Parameter row_block_size of DynamicBlockQuant has incorrect value 128. It should be 1.\n Solution: Check whether the parameter value is correct.\nTraceBack (most recent call last):\n The attr get failed.[FUNC:CheckParams][FILE:dynamic_block_quant_i8_tiling.cpp][LINE:78]\n The params check failed.[FUNC:Tiling4DynamicBlockQuant][FILE:dynamic_block_quant_tiling.cpp][LINE:564]\n Failed to execute operator DynamicBlockQuant_0. Reason: Failed to execute tiling function.\n Check NnopbaseExecutorDoTiling(executor) failed\n Check NnopbaseExecutorTilingAndUpdateBinInfo(executor) failed\n Check NnopbaseExecutorMatchCache(executor) failed\n Check NnopbaseRunForWorkspace(*executor, workspaceSize) failed\n" - } - }, - { - "trial": "c14-quant-prerequisites-358b9b28", - "cleanup_passed": true, - "source_unchanged": true, - "helpers_unchanged": true, - "scope": "Bounded native prerequisites only; a failed probe is retained as negative evidence.", - "status": "failed", - "returncode": 1, - "elapsed_seconds": 20.143024684977718, - "native_prerequisite": { - "verdict": "FAIL", - "scope": "TorchNPU FP8 prerequisites only", - "phase": "dynamic_block_quant_query", - "versions": { - "torch": "2.10.0+cpu", - "torch_npu": "2.10.0.post4", - "mindiesd": "3.1.0" - }, - "apis": { - "float8_e4m3fn": true, - "npu_dynamic_block_quant": true, - "npu_fused_infer_attention_score_v2": true - }, - "device_name": "Ascend910_9392", - "input": { - "shape": [ - 1, - 14, - 256, - 128 - ], - "dtype": "torch.bfloat16", - "visible_devices": "2" - }, - "error_type": "RuntimeError", - "probe_sha256": "df6465956e6f73bbca140fb9880057e30869fd4b7976b2dc091f71a575dd1096", - "error": "npu_dynamic_block_quant:../third_party/op-plugin/op_plugin/ops/opapi/DynamicBlockQuantNpuOpApi.cpp:61 NPU function error: call aclnnDynamicBlockQuantV2 failed, error code is 161002\n[ERROR] 2026-11-02-01:45:24 (PID:2491012, Device:0, RankID:-1) ERR00100 PTA call acl api failed.\n[PID: 2491012] 2026-11-02-01:45:24.732.576 Execution_Error(EZ1009): Failed to execute operator DynamicBlockQuant_0. Reason: The dtype or format of the actual input or output parameter of the operator is inconsistent with that defined in the operator prototype OpDef, details of parameter are x(DT_BFLOAT16, ND), yOut(DT_FLOAT8_E4M3FN, ND), scaleOut(DT_FLOAT, ND).\nTraceBack (most recent call last):\n Supported dtype and format list is:\n SupportInfo[0]: x(DT_FLOAT16, ND), yOut(DT_INT8, ND), scaleOut(DT_FLOAT, ND)\n SupportInfo[1]: x(DT_BFLOAT16, ND), yOut(DT_INT8, ND), scaleOut(DT_FLOAT, ND)\n Check NnopbaseExecutorTilingAndUpdateBinInfo(executor) failed\n Check NnopbaseExecutorMatchCache(executor) failed\n Check NnopbaseRunForWorkspace(*executor, workspaceSize) failed\n" - } - }, - { - "trial": "c15-video-row-native-066a9a40", - "cleanup_passed": true, - "source_unchanged": true, - "required_native_test_executed": true, - "files_before": { - "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "e35b551b8fc0ec3b0ba1384b4df96a8daa0e5179c5ed39d641125310b2ee957f", - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ca87df1936fd2694fcc768079e9e8f926b6d45969ddc61ebf353f5e726d70b8b", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "1258655fcdca4692c2306100846761a2b0b149bbcaa5cde48a9abaa559913bc3", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "a8257f98c2e30c7bc7978bf645966d2eaa98cd49a42a8a923c062b5808dcdcdf", - "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "6f890fcf84047bc5bc084f7b756a2b02e72573888a3bcc12ba7d0211e1bb62ef", - "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "d2a5bed10620a337ff9aaf99cf4a3740d94a8962ef903bf2da75d65e3b50e21c" - }, - "scope": "Focused state/block tests and native four-rank subset attention; no full-model quality/performance acceptance", - "status": "failed", - "returncode": 1, - "elapsed_seconds": 82.27795292000519 - }, - { - "trial": "c15-video-row-native-3a9721a6", - "cleanup_passed": true, - "source_unchanged": true, - "required_native_test_executed": true, - "files_before": { - "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "cf9fb13c5f24c7595d62d13cae6e283cafb2178612b20028d35357f0cafaac83", - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ca87df1936fd2694fcc768079e9e8f926b6d45969ddc61ebf353f5e726d70b8b", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "1258655fcdca4692c2306100846761a2b0b149bbcaa5cde48a9abaa559913bc3", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "a8257f98c2e30c7bc7978bf645966d2eaa98cd49a42a8a923c062b5808dcdcdf", - "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "c5fab405ee82bc553e9a902fa27e6dfbc5213ec18311e17b6415396a69b50f43", - "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "8ec1b3ae4cd252f5d95ab23e0232b983900939e8cb15d1e64f4740f7204abf39", - "vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py": "e8e2e74fb8edb05e02d3e2328ea1f3ad5ddcfd7a10ed09c5cf3911b1d063e849" - }, - "scope": "Focused state/block tests and native four-rank subset attention; no full-model quality/performance acceptance", - "status": "completed", - "returncode": 0, - "elapsed_seconds": 81.91829407500336 - }, - { - "trial": "c15-video-row-native-a2a224de", - "cleanup_passed": true, - "source_unchanged": true, - "required_native_test_executed": true, - "files_before": { - "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "cf9fb13c5f24c7595d62d13cae6e283cafb2178612b20028d35357f0cafaac83", - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ca87df1936fd2694fcc768079e9e8f926b6d45969ddc61ebf353f5e726d70b8b", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "1258655fcdca4692c2306100846761a2b0b149bbcaa5cde48a9abaa559913bc3", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "a8257f98c2e30c7bc7978bf645966d2eaa98cd49a42a8a923c062b5808dcdcdf", - "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "118afcc28379dd73feec763fbb9cfc9f17c6a208afe5969a85b3c969b8059552", - "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "875d0e70621fc76906be130519d7bb1242de5dd58159e31025e26da37e60d98c" - }, - "scope": "Focused state/block tests and native four-rank subset attention; no full-model quality/performance acceptance", - "status": "failed", - "returncode": 1, - "elapsed_seconds": 82.86056435096543 - }, - { - "trial": "c15-video-row-native-bb5084cc", - "cleanup_passed": true, - "source_unchanged": true, - "required_native_test_executed": true, - "files_before": { - "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "cf9fb13c5f24c7595d62d13cae6e283cafb2178612b20028d35357f0cafaac83", - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ca87df1936fd2694fcc768079e9e8f926b6d45969ddc61ebf353f5e726d70b8b", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "1258655fcdca4692c2306100846761a2b0b149bbcaa5cde48a9abaa559913bc3", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "a8257f98c2e30c7bc7978bf645966d2eaa98cd49a42a8a923c062b5808dcdcdf", - "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "118afcc28379dd73feec763fbb9cfc9f17c6a208afe5969a85b3c969b8059552", - "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "fa1ac8fd375d754503b3507fae006381138d8875afdad2418f1ea5ae338b9c26" - }, - "scope": "Focused state/block tests and native four-rank subset attention; no full-model quality/performance acceptance", - "status": "completed", - "returncode": 0, - "elapsed_seconds": 81.50971999001922 - }, - { - "trial": "c15-video-row-quality-5s-00-710315e0", - "model_controller": { - "status": "failed", - "returncode": 1, - "elapsed_seconds": 153.44603093498154 - }, - "actual_counters": { - "request-000\\rank-0\\attention_path_counts.json": {}, - "request-000\\rank-1\\attention_path_counts.json": {}, - "request-000\\rank-2\\attention_path_counts.json": {}, - "request-000\\rank-3\\attention_path_counts.json": {} - } - }, - { - "trial": "c15-video-row-v2-quality-5s-00-d6b921b9", - "model_controller": { - "status": "failed", - "returncode": 1, - "elapsed_seconds": 199.99670764501207 - }, - "actual_counters": { - "request-000\\rank-0\\video_row_cache_stats.json": { - "request_counters": { - "full_steps": 5, - "partial_steps": 1, - "full_blocks": 250, - "partial_blocks": 1, - "subset_attention_calls": 1, - "subset_query_rows": 2475, - "omitted_query_rows": 37296, - "partial_mlp_rows": 2475, - "cache_bytes": 4019635200, - "closed": true - }, - "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" - }, - "request-000\\rank-1\\video_row_cache_stats.json": { - "request_counters": { - "full_steps": 5, - "partial_steps": 1, - "full_blocks": 250, - "partial_blocks": 1, - "subset_attention_calls": 1, - "subset_query_rows": 2475, - "omitted_query_rows": 37296, - "partial_mlp_rows": 0, - "cache_bytes": 5350195200, - "closed": true - }, - "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" - }, - "request-000\\rank-2\\video_row_cache_stats.json": { - "request_counters": { - "full_steps": 5, - "partial_steps": 1, - "full_blocks": 250, - "partial_blocks": 1, - "subset_attention_calls": 1, - "subset_query_rows": 2475, - "omitted_query_rows": 37296, - "partial_mlp_rows": 0, - "cache_bytes": 5350195200, - "closed": true - }, - "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" - }, - "request-000\\rank-3\\video_row_cache_stats.json": { - "request_counters": { - "full_steps": 5, - "partial_steps": 1, - "full_blocks": 250, - "partial_blocks": 1, - "subset_attention_calls": 1, - "subset_query_rows": 2475, - "omitted_query_rows": 37296, - "partial_mlp_rows": 0, - "cache_bytes": 5330304000, - "closed": true - }, - "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" - }, - "request-000\\rank-0\\attention_path_counts.json": { - "_prefix_bnsd": 251 - }, - "request-000\\rank-1\\attention_path_counts.json": { - "_prefix_bnsd": 251 - }, - "request-000\\rank-2\\attention_path_counts.json": { - "_prefix_bnsd": 251 - }, - "request-000\\rank-3\\attention_path_counts.json": { - "_prefix_bnsd": 251 - } - } - }, - { - "trial": "c15-video-row-v3-quality-5s-00-70703586", - "model_controller": { - "status": "failed", - "returncode": 1, - "elapsed_seconds": 191.09544755500974 - }, - "actual_counters": { - "request-000\\rank-0\\video_row_cache_stats.json": { - "request_counters": { - "full_steps": 5, - "partial_steps": 1, - "full_blocks": 250, - "partial_blocks": 50, - "subset_attention_calls": 50, - "subset_query_rows": 123750, - "omitted_query_rows": 1864800, - "partial_mlp_rows": 123750, - "cache_bytes": 4019635200, - "closed": true - }, - "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" - }, - "request-000\\rank-1\\video_row_cache_stats.json": { - "request_counters": { - "full_steps": 5, - "partial_steps": 1, - "full_blocks": 250, - "partial_blocks": 50, - "subset_attention_calls": 50, - "subset_query_rows": 123750, - "omitted_query_rows": 1864800, - "partial_mlp_rows": 0, - "cache_bytes": 5350195200, - "closed": true - }, - "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" - }, - "request-000\\rank-2\\video_row_cache_stats.json": { - "request_counters": { - "full_steps": 5, - "partial_steps": 1, - "full_blocks": 250, - "partial_blocks": 50, - "subset_attention_calls": 50, - "subset_query_rows": 123750, - "omitted_query_rows": 1864800, - "partial_mlp_rows": 0, - "cache_bytes": 5350195200, - "closed": true - }, - "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" - }, - "request-000\\rank-3\\video_row_cache_stats.json": { - "request_counters": { - "full_steps": 5, - "partial_steps": 1, - "full_blocks": 250, - "partial_blocks": 50, - "subset_attention_calls": 50, - "subset_query_rows": 123750, - "omitted_query_rows": 1864800, - "partial_mlp_rows": 0, - "cache_bytes": 5330304000, - "closed": true - }, - "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" - }, - "request-000\\rank-0\\attention_path_counts.json": { - "_prefix_bnsd": 300 - }, - "request-000\\rank-1\\attention_path_counts.json": { - "_prefix_bnsd": 300 - }, - "request-000\\rank-2\\attention_path_counts.json": { - "_prefix_bnsd": 300 - }, - "request-000\\rank-3\\attention_path_counts.json": { - "_prefix_bnsd": 300 - } - } - }, - { - "trial": "c16-video-row-fp32-native-d3fe320c", - "cleanup_passed": true, - "source_unchanged": true, - "required_native_test_executed": true, - "files_before": { - "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "c555d76f7bd6457a81238fca56100d11cb7ab043a4a649d55e23b6203d09bde2", - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ca87df1936fd2694fcc768079e9e8f926b6d45969ddc61ebf353f5e726d70b8b", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "1258655fcdca4692c2306100846761a2b0b149bbcaa5cde48a9abaa559913bc3", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "a8257f98c2e30c7bc7978bf645966d2eaa98cd49a42a8a923c062b5808dcdcdf", - "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "c5fab405ee82bc553e9a902fa27e6dfbc5213ec18311e17b6415396a69b50f43", - "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "e09b872f6900e821d42632a0ca09595f2affe6f9794049dbc74d69ba6a52b2b0", - "vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py": "e8e2e74fb8edb05e02d3e2328ea1f3ad5ddcfd7a10ed09c5cf3911b1d063e849" - }, - "scope": "Focused state/block tests and native four-rank subset attention; no full-model quality/performance acceptance", - "status": "completed", - "returncode": 0, - "elapsed_seconds": 78.8189791989862 - }, - { - "trial": "c16-video-row-fp32-quality-5s-00-8600757a", - "model_controller": { - "status": "completed", - "returncode": 0, - "elapsed_seconds": 355.51310362201184 - }, - "model_diagnostic": { - "source_head": "e0a3462fcc44de698b5dedcf8ff2c99f9802f757", - "samples_ms": [ - 197716.36760496767 - ], - "latency_median_ms": 197716.36760496767, - "scope": "http_complete_mp4", - "warmup": 0, - "repetitions": 1, - "kind": "quality", - "diagnostic": "audit", - "clean_timing": false, - "cleanup_passed": true - }, - "actual_counters": { - "request-000\\rank-0\\video_row_cache_stats.json": { - "request_counters": { - "full_steps": 17, - "partial_steps": 33, - "full_blocks": 850, - "partial_blocks": 1650, - "subset_attention_calls": 1650, - "subset_query_rows": 4083750, - "omitted_query_rows": 61538400, - "partial_mlp_rows": 4083750, - "peak_cache_bytes": 8039270400, - "closed": true, - "storage_dtype": "torch.float32", - "live_cache_bytes": 0 - }, - "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" - }, - "request-000\\rank-1\\video_row_cache_stats.json": { - "request_counters": { - "full_steps": 17, - "partial_steps": 33, - "full_blocks": 850, - "partial_blocks": 1650, - "subset_attention_calls": 1650, - "subset_query_rows": 4083750, - "omitted_query_rows": 61538400, - "partial_mlp_rows": 0, - "peak_cache_bytes": 10700390400, - "closed": true, - "storage_dtype": "torch.float32", - "live_cache_bytes": 0 - }, - "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" - }, - "request-000\\rank-2\\video_row_cache_stats.json": { - "request_counters": { - "full_steps": 17, - "partial_steps": 33, - "full_blocks": 850, - "partial_blocks": 1650, - "subset_attention_calls": 1650, - "subset_query_rows": 4083750, - "omitted_query_rows": 61538400, - "partial_mlp_rows": 0, - "peak_cache_bytes": 10700390400, - "closed": true, - "storage_dtype": "torch.float32", - "live_cache_bytes": 0 - }, - "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" - }, - "request-000\\rank-3\\video_row_cache_stats.json": { - "request_counters": { - "full_steps": 17, - "partial_steps": 33, - "full_blocks": 850, - "partial_blocks": 1650, - "subset_attention_calls": 1650, - "subset_query_rows": 4083750, - "omitted_query_rows": 61538400, - "partial_mlp_rows": 0, - "peak_cache_bytes": 10660608000, - "closed": true, - "storage_dtype": "torch.float32", - "live_cache_bytes": 0 - }, - "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" - }, - "request-000\\rank-0\\attention_path_counts.json": { - "_prefix_bnsd": 2500 - }, - "request-000\\rank-1\\attention_path_counts.json": { - "_prefix_bnsd": 2500 - }, - "request-000\\rank-2\\attention_path_counts.json": { - "_prefix_bnsd": 2500 - }, - "request-000\\rank-3\\attention_path_counts.json": { - "_prefix_bnsd": 2500 - } - } - }, - { - "trial": "c17-video-window-native-9b48cb56", - "cleanup_passed": true, - "source_unchanged": true, - "required_native_test_executed": true, - "files_before": { - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "66ca27991ce42ce302e5de078be6395c9d1ab53752378babef25089f3f75c3aa", - "tests/diffusion/models/minimax_h3/test_minimax_h3_video_window.py": "0bc59f35eff41164e125a5070e034c94ea60bf0112733ebe59eb6378eccbfed8" - }, - "scope": "CPU window reference/routing and native batched BF16 FAS; no full-model quality/performance acceptance", - "status": "completed", - "returncode": 0, - "elapsed_seconds": 51.878022204036824 - }, - { - "trial": "c17-video-window-operator-9bf7de3e", - "cleanup_passed": true, - "source_unchanged": true, - "operator_gate_passed": true, - "files_before": { - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "66ca27991ce42ce302e5de078be6395c9d1ab53752378babef25089f3f75c3aa", - "tests/diffusion/models/minimax_h3/test_minimax_h3_video_window.py": "0bc59f35eff41164e125a5070e034c94ea60bf0112733ebe59eb6378eccbfed8" - }, - "scope": "Real captured H3 attention operator benchmark; full-model quality and HTTP are separate; no full-model quality/performance acceptance", - "status": "completed", - "returncode": 0, - "elapsed_seconds": 51.778123575029895, - "operator_benchmark": { - "scope": "real captured 5s attention operation on logical2; no model quality or HTTP acceptance", - "fixture_sha256": "00767e6afb2ba777bb2677eb9b60dc2f00fb649f5006ae8c1d77191214fbf912", - "fixture_tensor_sha256": { - "query": "313095dd143adf0636d57c90d77b689405dee510df3ebd6385a49d071c3323e9", - "key": "6638aee85f0039ee814206e7c976a2d1a9a2b185723c7bbb74c69b1e68c5bad4", - "value": "101eede996c8b13d61d487019d47f0b5fd2193664d8c178665063675cb66fdca" - }, - "query_shape": [ - 1, - 39808, - 14, - 128 - ], - "valid_kv": 39771, - "frames": 37, - "rows_per_frame": 1008, - "target_start": 2475, - "window": 3, - "finite": true, - "relative_l2_vs_dense": 0.5161040425300598, - "baseline_samples_ms": [ - 69.0619180095382, - 69.60304302629083, - 69.27597103640437, - 69.81903495034203, - 69.96654602698982 - ], - "candidate_samples_ms": [ - 25.191069988068193, - 25.301200977992266, - 25.325032009277493, - 25.26562102138996, - 25.219489994924515 - ], - "baseline_median_ms": 69.60304302629083, - "candidate_median_ms": 25.26562102138996, - "reduction_pct": 63.70040744936038, - "operator_gate_passed": true, - "warmups_per_operation": 2, - "measured_per_operation": 5, - "order": "AB/BA/AB/BA/AB", - "probe_sha256": "2465cf08173950a241ac3cc4713a4227d4c4b5a87f35358692a7ef6b60821824" - } - }, - { - "trial": "c17-video-window-quality-5s-00-a8c689e4", - "model_controller": { - "status": "completed", - "returncode": 0, - "elapsed_seconds": 358.6752056070254 - }, - "model_diagnostic": { - "source_head": "c1589a2b5e4ada617e7cca38ef7c300fe9495c58", - "samples_ms": [ - 210397.12271699682 - ], - "latency_median_ms": 210397.12271699682, - "scope": "http_complete_mp4", - "warmup": 0, - "repetitions": 1, - "kind": "quality", - "diagnostic": "audit", - "clean_timing": false, - "cleanup_passed": true - }, - "actual_counters": { - "request-000\\rank-0\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 4900, - "_video_temporal_window_bnsd": 2400 - }, - "request-000\\rank-1\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 4900, - "_video_temporal_window_bnsd": 2400 - }, - "request-000\\rank-2\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 4900, - "_video_temporal_window_bnsd": 2400 - }, - "request-000\\rank-3\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 4900, - "_video_temporal_window_bnsd": 2400 - } - } - }, - { - "trial": "c18-video-pooled-context-native-1673db72", - "cleanup_passed": true, - "source_unchanged": true, - "required_native_test_executed": true, - "files_before": { - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "1febc030a7917b6e10fe287014fe2aacef76ade7bf58ebb604fdedc36fb7c21d", - "tests/diffusion/models/minimax_h3/test_minimax_h3_pooled_context.py": "45a9ae5ba56858daee6a1088a8c8b1ed612ea9afd3c229c90a9b6db3c80a06b2" - }, - "scope": "Independent weighted global-context reference and native padded-head BF16 FAS; no full-model quality/performance acceptance", - "status": "completed", - "returncode": 0, - "elapsed_seconds": 49.97671151399845 - }, - { - "trial": "c18-video-pooled-context-operator-315db0e5", - "cleanup_passed": true, - "source_unchanged": true, - "operator_gate_passed": false, - "files_before": { - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "1febc030a7917b6e10fe287014fe2aacef76ade7bf58ebb604fdedc36fb7c21d", - "tests/diffusion/models/minimax_h3/test_minimax_h3_pooled_context.py": "45a9ae5ba56858daee6a1088a8c8b1ed612ea9afd3c229c90a9b6db3c80a06b2" - }, - "scope": "Real captured H3 attention operator benchmark; full-model quality and HTTP are separate; no full-model quality/performance acceptance", - "status": "completed", - "returncode": 0, - "elapsed_seconds": 51.37963652302278, - "operator_benchmark": { - "scope": "real captured 5s attention operation on logical2; no model quality or HTTP acceptance", - "fixture_sha256": "00767e6afb2ba777bb2677eb9b60dc2f00fb649f5006ae8c1d77191214fbf912", - "fixture_tensor_sha256": { - "query": "313095dd143adf0636d57c90d77b689405dee510df3ebd6385a49d071c3323e9", - "key": "6638aee85f0039ee814206e7c976a2d1a9a2b185723c7bbb74c69b1e68c5bad4", - "value": "101eede996c8b13d61d487019d47f0b5fd2193664d8c178665063675cb66fdca" - }, - "query_shape": [ - 1, - 39808, - 14, - 128 - ], - "valid_kv": 39771, - "frames": 37, - "rows_per_frame": 1008, - "target_start": 2475, - "profile": "3x3-local1", - "finite": true, - "relative_l2_vs_dense": 0.21801254153251648, - "baseline_samples_ms": [ - 69.19049099087715, - 69.70348599134013, - 69.43478400353342, - 69.81050700414926, - 69.24652203451842 - ], - "candidate_samples_ms": [ - 68.29233502503484, - 64.26435202592984, - 71.05128600960597, - 64.39994304673746, - 64.39412303734571 - ], - "baseline_median_ms": 69.43478400353342, - "candidate_median_ms": 64.39994304673746, - "reduction_pct": 7.251179691924648, - "operator_gate_passed": false, - "warmups_per_operation": 2, - "measured_per_operation": 5, - "order": "AB/BA/AB/BA/AB", - "probe_sha256": "42852c0e4e693c5b3685981db617de00648feefe651f7060ad83c359b08dd81d" - } - }, - { - "trial": "c19-video-global-pool-native-f0f4c502", - "cleanup_passed": true, - "source_unchanged": true, - "required_native_test_executed": true, - "files_before": { - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ddfd6d6822cc6a5528fc95539f7f98dfdeed25d5489da3ed5cf76e6076a5ec94", - "tests/diffusion/models/minimax_h3/test_minimax_h3_global_pool.py": "5457db550b59e3373be8f5f21873c1b78ac357a6ac170c2603ee531b50e91467" - }, - "scope": "Independent weighted global-context reference and native padded-head BF16 FAS; no full-model quality/performance acceptance", - "status": "completed", - "returncode": 0, - "elapsed_seconds": 51.528875903983135 - }, - { - "trial": "c19-video-global-pool-operator-933511c9", - "cleanup_passed": true, - "source_unchanged": true, - "operator_gate_passed": false, - "files_before": { - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ddfd6d6822cc6a5528fc95539f7f98dfdeed25d5489da3ed5cf76e6076a5ec94", - "tests/diffusion/models/minimax_h3/test_minimax_h3_global_pool.py": "5457db550b59e3373be8f5f21873c1b78ac357a6ac170c2603ee531b50e91467" - }, - "scope": "Real captured H3 attention operator benchmark; full-model quality and HTTP are separate; no full-model quality/performance acceptance", - "status": "completed", - "returncode": 0, - "elapsed_seconds": 51.02866222098237, - "operator_benchmark": { - "scope": "real captured 5s attention operation on logical2; no model quality or HTTP acceptance", - "fixture_sha256": "00767e6afb2ba777bb2677eb9b60dc2f00fb649f5006ae8c1d77191214fbf912", - "fixture_tensor_sha256": { - "query": "313095dd143adf0636d57c90d77b689405dee510df3ebd6385a49d071c3323e9", - "key": "6638aee85f0039ee814206e7c976a2d1a9a2b185723c7bbb74c69b1e68c5bad4", - "value": "101eede996c8b13d61d487019d47f0b5fd2193664d8c178665063675cb66fdca" - }, - "query_shape": [ - 1, - 39808, - 14, - 128 - ], - "valid_kv": 39771, - "frames": 37, - "rows_per_frame": 1008, - "target_start": 2475, - "profile": "3x3-global", - "finite": true, - "relative_l2_vs_dense": 0.2588050067424774, - "baseline_samples_ms": [ - 69.95450798422098, - 69.79658600175753, - 69.58012498216704, - 70.22821099963039, - 69.11802099784836 - ], - "candidate_samples_ms": [ - 32.91054297005758, - 32.899541954975575, - 32.924533006735146, - 32.779561006464064, - 32.96928299823776 - ], - "baseline_median_ms": 69.79658600175753, - "candidate_median_ms": 32.91054297005758, - "reduction_pct": 52.847918708761966, - "operator_gate_passed": false, - "warmups_per_operation": 2, - "measured_per_operation": 5, - "order": "AB/BA/AB/BA/AB", - "probe_sha256": "a27906d8bb0b65e2f6ffb41ddeb4b513163b7bf23b9d2dbd33a5015fc93b2a08" - } - }, - { - "trial": "c20-video-variance-pool-native-50101063", - "cleanup_passed": true, - "source_unchanged": true, - "required_native_test_executed": true, - "files_before": { - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "4390450bc5324f7028125101cd4bb32e9de237b04bc12fa879b48110d7122e3c", - "tests/diffusion/models/minimax_h3/test_minimax_h3_variance_pool.py": "e871fb4b222d61d10b6c442a333a52c976cceaa0d406233502ecbcaddfeba8c9" - }, - "scope": "Independent weighted global-context reference and native padded-head BF16 FAS; no full-model quality/performance acceptance", - "status": "completed", - "returncode": 0, - "elapsed_seconds": 50.230675689992495 - }, - { - "trial": "c20-video-variance-pool-operator-209f263d", - "cleanup_passed": true, - "source_unchanged": true, - "operator_gate_passed": false, - "files_before": { - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "4390450bc5324f7028125101cd4bb32e9de237b04bc12fa879b48110d7122e3c", - "tests/diffusion/models/minimax_h3/test_minimax_h3_variance_pool.py": "e871fb4b222d61d10b6c442a333a52c976cceaa0d406233502ecbcaddfeba8c9" - }, - "scope": "Real captured H3 attention operator benchmark; full-model quality and HTTP are separate; no full-model quality/performance acceptance", - "status": "completed", - "returncode": 0, - "elapsed_seconds": 49.880220040970016, - "operator_benchmark": { - "scope": "real captured 5s attention operation on logical2; no model quality or HTTP acceptance", - "fixture_sha256": "00767e6afb2ba777bb2677eb9b60dc2f00fb649f5006ae8c1d77191214fbf912", - "fixture_tensor_sha256": { - "query": "313095dd143adf0636d57c90d77b689405dee510df3ebd6385a49d071c3323e9", - "key": "6638aee85f0039ee814206e7c976a2d1a9a2b185723c7bbb74c69b1e68c5bad4", - "value": "101eede996c8b13d61d487019d47f0b5fd2193664d8c178665063675cb66fdca" - }, - "query_shape": [ - 1, - 39808, - 14, - 128 - ], - "valid_kv": 39771, - "frames": 37, - "rows_per_frame": 1008, - "target_start": 2475, - "profile": "3x3-variance", - "finite": true, - "relative_l2_vs_dense": 0.26336145401000977, - "baseline_samples_ms": [ - 69.74446203093976, - 69.06059599714354, - 69.64961998164654, - 69.69038100214675, - 69.62426099926233 - ], - "candidate_samples_ms": [ - 35.81449302146211, - 36.30960697773844, - 35.925014002714306, - 35.99672397831455, - 35.80518299713731 - ], - "baseline_median_ms": 69.64961998164654, - "candidate_median_ms": 35.925014002714306, - "reduction_pct": 48.42037327385138, - "operator_gate_passed": false, - "warmups_per_operation": 2, - "measured_per_operation": 5, - "order": "AB/BA/AB/BA/AB", - "probe_sha256": "785e7a0c13719e37d5cc44fb66af05649e51e1f7e5d0b212e92ec44f79cb0184" - } - }, - { - "trial": "c21-video-pre-gate-native-3c5eebbe", - "cleanup_passed": true, - "source_unchanged": true, - "required_native_test_executed": true, - "files_before": { - "tests/diffusion/distributed/test_collective_gate.py": "6a02f32882d85eca1e3bb9a023ce3adb3d5b59c213845e11e85c96a89432f477", - "tests/diffusion/models/minimax_h3/test_minimax_h3_pre_gate_cache.py": "a4924d0b3dd58ae413259edba6cf2487b3e41d391f943fb74358934be72eb53b", - "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "e09b872f6900e821d42632a0ca09595f2affe6f9794049dbc74d69ba6a52b2b0", - "vllm_omni/config/omni_config.py": "6894120a330474f718d43bf2bf7928892eb996152839ce6469c34d18a7df09d1", - "vllm_omni/config/stage_config.py": "0752e8cc1c6f1021f737830d08663cfc118923a83fd909c607f9ec3ab2a3d2fb", - "vllm_omni/diffusion/attention/layer.py": "76c5fce3dabd23e39a8890316e1691c03bc203066cb60af5611ec4790eefc632", - "vllm_omni/diffusion/data.py": "5db83d5c71cb27cc7810d35fcfc891da80f0b003adbb16bdecc3816a63305e11", - "vllm_omni/diffusion/distributed/collective_gate.py": "01c6317cfa615840681b8bb8b8513fa3736d26e845a24ad9e1ab7f87489b962f", - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ca87df1936fd2694fcc768079e9e8f926b6d45969ddc61ebf353f5e726d70b8b", - "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "c5fab405ee82bc553e9a902fa27e6dfbc5213ec18311e17b6415396a69b50f43", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "8933b536317652d0dbc7a974ea7e0bca928e7b3bdf438988999ff5a498a6ec9d", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "019bf71106ba3263b55ad7b682064e3a176eecb81725cb48bbf178387356d5d9", - "vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py": "e8e2e74fb8edb05e02d3e2328ea1f3ad5ddcfd7a10ed09c5cf3911b1d063e849", - "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "04a9212994b5c8593f8d196820300ded077fed9af4274b64d87d9e6bf71eae07", - "vllm_omni/diffusion/offloader/base.py": "23922136deaf54e2d98e827b007b34c7e3ad36171b71bfe9060d390efc0b679f", - "vllm_omni/diffusion/offloader/distributed_layerwise_backend.py": "19fbce0f1036e4784d68cab798c499526a9099101909e41ce96dd0caa033b6a4", - "vllm_omni/engine/arg_utils.py": "52569e3688cc708729a48ca0a2c875c28193733c2f934f6ff89f6c815e4009c7", - "vllm_omni/entrypoints/cli/serve.py": "f8284ad6b751ba4a9e4f7254d78132cc2cfd49871a0f9af7fde49c6789d79cb0" - }, - "scope": "Legacy/pre-gate/DLO tests, two executed native four-rank HCCL/subsetFA suites and fresh current gates; no full-model quality/performance acceptance", - "status": "completed", - "returncode": 0, - "elapsed_seconds": 127.31179964594776 - }, - { - "trial": "c21-video-pre-gate-quality-5s-00-7265c33e", - "model_controller": { - "status": "completed", - "returncode": 0, - "elapsed_seconds": 318.92388185299933 - }, - "model_diagnostic": { - "source_head": "d48fa51d03dd748568cbcf57ed40f533c691c9bd", - "samples_ms": [ - 163805.5620979867 - ], - "latency_median_ms": 163805.5620979867, - "scope": "http_complete_mp4", - "warmup": 0, - "repetitions": 1, - "kind": "quality", - "diagnostic": "audit", - "clean_timing": false, - "cleanup_passed": true - }, - "actual_counters": { - "request-000\\rank-0\\video_row_cache_stats.json": { - "request_counters": { - "full_steps": 8, - "partial_steps": 42, - "full_blocks": 400, - "partial_blocks": 2100, - "subset_attention_calls": 2100, - "subset_query_rows": 5197500, - "omitted_query_rows": 78321600, - "partial_mlp_rows": 5197500, - "peak_cache_bytes": 8039270400, - "closed": true, - "storage_dtype": "torch.bfloat16", - "cache_stage": "pre_gate", - "fresh_attention_gates": 2100, - "fresh_mlp_gates": 2100, - "live_cache_bytes": 0 - }, - "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" - }, - "request-000\\rank-1\\video_row_cache_stats.json": { - "request_counters": { - "full_steps": 8, - "partial_steps": 42, - "full_blocks": 400, - "partial_blocks": 2100, - "subset_attention_calls": 2100, - "subset_query_rows": 5197500, - "omitted_query_rows": 78321600, - "partial_mlp_rows": 0, - "peak_cache_bytes": 10700390400, - "closed": true, - "storage_dtype": "torch.bfloat16", - "cache_stage": "pre_gate", - "fresh_attention_gates": 2100, - "fresh_mlp_gates": 2100, - "live_cache_bytes": 0 - }, - "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" - }, - "request-000\\rank-2\\video_row_cache_stats.json": { - "request_counters": { - "full_steps": 8, - "partial_steps": 42, - "full_blocks": 400, - "partial_blocks": 2100, - "subset_attention_calls": 2100, - "subset_query_rows": 5197500, - "omitted_query_rows": 78321600, - "partial_mlp_rows": 0, - "peak_cache_bytes": 10700390400, - "closed": true, - "storage_dtype": "torch.bfloat16", - "cache_stage": "pre_gate", - "fresh_attention_gates": 2100, - "fresh_mlp_gates": 2100, - "live_cache_bytes": 0 - }, - "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" - }, - "request-000\\rank-3\\video_row_cache_stats.json": { - "request_counters": { - "full_steps": 8, - "partial_steps": 42, - "full_blocks": 400, - "partial_blocks": 2100, - "subset_attention_calls": 2100, - "subset_query_rows": 5197500, - "omitted_query_rows": 78321600, - "partial_mlp_rows": 0, - "peak_cache_bytes": 10660608000, - "closed": true, - "storage_dtype": "torch.bfloat16", - "cache_stage": "pre_gate", - "fresh_attention_gates": 2100, - "fresh_mlp_gates": 2100, - "live_cache_bytes": 0 - }, - "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" - }, - "request-000\\rank-0\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-000\\rank-1\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-000\\rank-2\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-000\\rank-3\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - } - } - }, - { - "trial": "c22-video-pre-gate-secant-native-3521747e", - "cleanup_passed": true, - "source_unchanged": true, - "required_native_test_executed": true, - "files_before": { - "tests/diffusion/distributed/test_collective_gate.py": "6a02f32882d85eca1e3bb9a023ce3adb3d5b59c213845e11e85c96a89432f477", - "tests/diffusion/models/minimax_h3/test_minimax_h3_pre_gate_cache.py": "b8394d22b054d60c1ab8e069e9600e1947cc73b5826e1f206e66c548123f632d", - "tests/diffusion/models/minimax_h3/test_minimax_h3_pre_gate_secant.py": "70a215fbb0dc63b90652d96f8f6d106a5785bd7d64dae75a2c4f6ec5c627ac1c", - "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "e09b872f6900e821d42632a0ca09595f2affe6f9794049dbc74d69ba6a52b2b0", - "vllm_omni/config/omni_config.py": "6894120a330474f718d43bf2bf7928892eb996152839ce6469c34d18a7df09d1", - "vllm_omni/config/stage_config.py": "0752e8cc1c6f1021f737830d08663cfc118923a83fd909c607f9ec3ab2a3d2fb", - "vllm_omni/diffusion/attention/layer.py": "76c5fce3dabd23e39a8890316e1691c03bc203066cb60af5611ec4790eefc632", - "vllm_omni/diffusion/data.py": "5db83d5c71cb27cc7810d35fcfc891da80f0b003adbb16bdecc3816a63305e11", - "vllm_omni/diffusion/distributed/collective_gate.py": "01c6317cfa615840681b8bb8b8513fa3736d26e845a24ad9e1ab7f87489b962f", - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ca87df1936fd2694fcc768079e9e8f926b6d45969ddc61ebf353f5e726d70b8b", - "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "b991ebb5d43242357d5ab8cfb476808a76fc0658d6048db6f3b7131122c8a993", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "8933b536317652d0dbc7a974ea7e0bca928e7b3bdf438988999ff5a498a6ec9d", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "3904bc8ac6679bc51614b95f5510789331a41bf20a43a517f3a99d33505201a6", - "vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py": "e8e2e74fb8edb05e02d3e2328ea1f3ad5ddcfd7a10ed09c5cf3911b1d063e849", - "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "e21e0afe8f8948d805d374428a34f3b2c5c0b5041d63ea791f047af8f455cf16", - "vllm_omni/diffusion/offloader/base.py": "23922136deaf54e2d98e827b007b34c7e3ad36171b71bfe9060d390efc0b679f", - "vllm_omni/diffusion/offloader/distributed_layerwise_backend.py": "19fbce0f1036e4784d68cab798c499526a9099101909e41ce96dd0caa033b6a4", - "vllm_omni/engine/arg_utils.py": "52569e3688cc708729a48ca0a2c875c28193733c2f934f6ff89f6c815e4009c7", - "vllm_omni/entrypoints/cli/serve.py": "f8284ad6b751ba4a9e4f7254d78132cc2cfd49871a0f9af7fde49c6789d79cb0" - }, - "scope": "Legacy/pre-gate/DLO tests, three executed native four-rank HCCL/subsetFA suites and fresh current gates; no full-model quality/performance acceptance", - "status": "completed", - "returncode": 0, - "elapsed_seconds": 154.04585235595005 - }, - { - "trial": "c22-video-pre-gate-secant-quality-5s-00-72202558", - "model_controller": { - "status": "failed", - "returncode": 1, - "elapsed_seconds": 5.625923136016354 - } - }, - { - "trial": "c22-video-pre-gate-secant-v2-quality-5s-00-777147cb", - "model_controller": { - "status": "completed", - "returncode": 0, - "elapsed_seconds": 317.8191881469684 - }, - "model_diagnostic": { - "source_head": "7b0d12bfefdf05868e93e74bc91f04c70ee34128", - "samples_ms": [ - 166858.99333201814 - ], - "latency_median_ms": 166858.99333201814, - "scope": "http_complete_mp4", - "warmup": 0, - "repetitions": 1, - "kind": "quality", - "diagnostic": "audit", - "clean_timing": false, - "cleanup_passed": true - }, - "actual_counters": { - "request-000\\rank-0\\video_row_cache_stats.json": { - "request_counters": { - "full_steps": 8, - "partial_steps": 42, - "full_blocks": 400, - "partial_blocks": 2100, - "subset_attention_calls": 2100, - "subset_query_rows": 5197500, - "omitted_query_rows": 78321600, - "partial_mlp_rows": 5197500, - "peak_cache_bytes": 12061056000, - "closed": true, - "storage_dtype": "torch.bfloat16", - "cache_stage": "pre_gate_secant", - "secant_forecasts": 4200, - "slope_codec": "int8_per_channel", - "fresh_attention_gates": 2100, - "fresh_mlp_gates": 2100, - "live_cache_bytes": 0 - }, - "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" - }, - "request-000\\rank-1\\video_row_cache_stats.json": { - "request_counters": { - "full_steps": 8, - "partial_steps": 42, - "full_blocks": 400, - "partial_blocks": 2100, - "subset_attention_calls": 2100, - "subset_query_rows": 5197500, - "omitted_query_rows": 78321600, - "partial_mlp_rows": 0, - "peak_cache_bytes": 16052736000, - "closed": true, - "storage_dtype": "torch.bfloat16", - "cache_stage": "pre_gate_secant", - "secant_forecasts": 4200, - "slope_codec": "int8_per_channel", - "fresh_attention_gates": 2100, - "fresh_mlp_gates": 2100, - "live_cache_bytes": 0 - }, - "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" - }, - "request-000\\rank-2\\video_row_cache_stats.json": { - "request_counters": { - "full_steps": 8, - "partial_steps": 42, - "full_blocks": 400, - "partial_blocks": 2100, - "subset_attention_calls": 2100, - "subset_query_rows": 5197500, - "omitted_query_rows": 78321600, - "partial_mlp_rows": 0, - "peak_cache_bytes": 16052736000, - "closed": true, - "storage_dtype": "torch.bfloat16", - "cache_stage": "pre_gate_secant", - "secant_forecasts": 4200, - "slope_codec": "int8_per_channel", - "fresh_attention_gates": 2100, - "fresh_mlp_gates": 2100, - "live_cache_bytes": 0 - }, - "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" - }, - "request-000\\rank-3\\video_row_cache_stats.json": { - "request_counters": { - "full_steps": 8, - "partial_steps": 42, - "full_blocks": 400, - "partial_blocks": 2100, - "subset_attention_calls": 2100, - "subset_query_rows": 5197500, - "omitted_query_rows": 78321600, - "partial_mlp_rows": 0, - "peak_cache_bytes": 15993062400, - "closed": true, - "storage_dtype": "torch.bfloat16", - "cache_stage": "pre_gate_secant", - "secant_forecasts": 4200, - "slope_codec": "int8_per_channel", - "fresh_attention_gates": 2100, - "fresh_mlp_gates": 2100, - "live_cache_bytes": 0 - }, - "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" - }, - "request-000\\rank-0\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-000\\rank-1\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-000\\rank-2\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-000\\rank-3\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - } - } - }, - { - "trial": "c23-video-context-lse-components-783670a5", - "cleanup_passed": true, - "source_unchanged": true, - "files_before": { - "tests/diffusion/distributed/test_collective_gate.py": "6a02f32882d85eca1e3bb9a023ce3adb3d5b59c213845e11e85c96a89432f477", - "tests/diffusion/models/minimax_h3/test_minimax_h3_pre_gate_cache.py": "a4924d0b3dd58ae413259edba6cf2487b3e41d391f943fb74358934be72eb53b", - "tests/diffusion/models/minimax_h3/test_minimax_h3_video_context.py": "ac7ff16b789b7304add457cea82859d6ae21548ac355344a390125f2e2c36fbe", - "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "e09b872f6900e821d42632a0ca09595f2affe6f9794049dbc74d69ba6a52b2b0", - "vllm_omni/config/omni_config.py": "6894120a330474f718d43bf2bf7928892eb996152839ce6469c34d18a7df09d1", - "vllm_omni/config/stage_config.py": "0752e8cc1c6f1021f737830d08663cfc118923a83fd909c607f9ec3ab2a3d2fb", - "vllm_omni/diffusion/attention/layer.py": "76c5fce3dabd23e39a8890316e1691c03bc203066cb60af5611ec4790eefc632", - "vllm_omni/diffusion/data.py": "5db83d5c71cb27cc7810d35fcfc891da80f0b003adbb16bdecc3816a63305e11", - "vllm_omni/diffusion/distributed/collective_gate.py": "01c6317cfa615840681b8bb8b8513fa3736d26e845a24ad9e1ab7f87489b962f", - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "48533c941a50719995206d18fa9198330824b940ecba9399b46ba7615b969ccb", - "vllm_omni/diffusion/models/minimax_h3/attention/video_context.py": "f53c94e2ac9a8df8f4d7abd753393b2907170a086e92ad7b23818810868e02e1", - "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "c5fab405ee82bc553e9a902fa27e6dfbc5213ec18311e17b6415396a69b50f43", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "73c96090e6fb0e13ce734bc96b23472fd5adb83bceb71f3eba2815d6343a33e4", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "577e290c2d92fc02822cd516cc499c3c4d1f465145ad2a837edfdfeaa09ed9fb", - "vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py": "e8e2e74fb8edb05e02d3e2328ea1f3ad5ddcfd7a10ed09c5cf3911b1d063e849", - "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "cff17568fbd856c28394d104d393a806931613b7b057f288006ad8c6fb9aa05a", - "vllm_omni/diffusion/offloader/base.py": "23922136deaf54e2d98e827b007b34c7e3ad36171b71bfe9060d390efc0b679f", - "vllm_omni/diffusion/offloader/distributed_layerwise_backend.py": "19fbce0f1036e4784d68cab798c499526a9099101909e41ce96dd0caa033b6a4", - "vllm_omni/engine/arg_utils.py": "52569e3688cc708729a48ca0a2c875c28193733c2f934f6ff89f6c815e4009c7", - "vllm_omni/entrypoints/cli/serve.py": "f8284ad6b751ba4a9e4f7254d78132cc2cfd49871a0f9af7fde49c6789d79cb0" - }, - "scope": "Real captured H3 attention operator benchmark; full-model quality and HTTP are separate; no full-model quality/performance acceptance", - "status": "completed", - "returncode": 0, - "elapsed_seconds": 52.930839737004135, - "component_diagnosis": { - "scope": "Synchronized stage diagnosis on real captured5s QKV; not clean model timing. Raw derived merge inputs/old output retained privately.", - "measurements": { - "nonvideo_query_full_kv": { - "samples_ms": [ - 5.2369519835338, - 5.142281006556004, - 5.150350974872708, - 5.160611995961517, - 5.198111000936478 - ], - "median_ms": 5.160611995961517 - }, - "video_query_prefix_kv_lse": { - "samples_ms": [ - 5.12795097893104, - 5.113500985316932, - 5.11934095993638, - 5.1073210197500885, - 5.115851003210992 - ], - "median_ms": 5.115851003210992 - }, - "video_query_video_kv_lse": { - "samples_ms": [ - 62.014786002691835, - 61.919544998090714, - 61.41817104071379, - 61.652702977880836, - 61.9437750428915 - ], - "median_ms": 61.919544998090714 - }, - "triton_lse_merge": { - "samples_ms": [ - 22.78909203596413, - 22.761480999179184, - 22.750200994778425, - 22.737870982382447, - 22.741902037523687 - ], - "median_ms": 22.750200994778425 - }, - "torch_lse_merge": { - "samples_ms": [ - 2.7590019744820893, - 2.740311960224062, - 2.7479419950395823, - 2.737232018262148, - 2.7427320019342005 - ], - "median_ms": 2.7427320019342005 - } - }, - "merge_fixture": { - "prefix_output": { - "shape": [ - 1, - 37296, - 14, - 128 - ], - "stride": [ - 66834432, - 1792, - 128, - 1 - ], - "dtype": "torch.bfloat16", - "bytes": 133668864, - "sha256": "27ca510cb9270c976fe10741d505e1e9542315efa0c09f3947e0ae109be6667f", - "file": "prefix_output.bin" - }, - "video_output": { - "shape": [ - 1, - 37296, - 14, - 128 - ], - "stride": [ - 66834432, - 1792, - 128, - 1 - ], - "dtype": "torch.bfloat16", - "bytes": 133668864, - "sha256": "61da35ea1aea5cfea779f630ea287675541d52adb28e60dcaaa2697d70d7d5cd", - "file": "video_output.bin" - }, - "prefix_lse": { - "shape": [ - 1, - 37296, - 14, - 1 - ], - "stride": [ - 522144, - 14, - 1, - 1 - ], - "dtype": "torch.float32", - "bytes": 2088576, - "sha256": "731027f3ec32dfe68302dc4198c46d06661362977106717425fbd751b4df2450", - "file": "prefix_lse.bin" - }, - "video_lse": { - "shape": [ - 1, - 37296, - 14, - 1 - ], - "stride": [ - 522144, - 14, - 1, - 1 - ], - "dtype": "torch.float32", - "bytes": 2088576, - "sha256": "eacaded11a805bf19f94a18b25fdbd90243f71760b7de9071b3b3bbd8ae6e987", - "file": "video_lse.bin" - }, - "original_merge": { - "shape": [ - 1, - 37296, - 14, - 128 - ], - "stride": [ - 66834432, - 1792, - 128, - 1 - ], - "dtype": "torch.bfloat16", - "bytes": 133668864, - "sha256": "b1fa215f72bdfbddf11431ddec28cd2a06049e4bbbbb9b0857b6312713139919", - "file": "original_merge.bin" - }, - "torch_merge": { - "shape": [ - 1, - 37296, - 14, - 128 - ], - "stride": [ - 66834432, - 1792, - 128, - 1 - ], - "dtype": "torch.bfloat16", - "bytes": 133668864, - "sha256": "b1fa215f72bdfbddf11431ddec28cd2a06049e4bbbbb9b0857b6312713139919", - "file": "torch_merge.bin" - } - }, - "original_merge_bit_equal_torch": true, - "probe_sha256": "5e53eff1cb66ec478d1b04eb39510f253927614679499065d7deb0d1a80950ea" - } - }, - { - "trial": "c23-video-context-lse-native-ef09a725", - "cleanup_passed": true, - "source_unchanged": true, - "required_native_test_executed": true, - "files_before": { - "tests/diffusion/distributed/test_collective_gate.py": "6a02f32882d85eca1e3bb9a023ce3adb3d5b59c213845e11e85c96a89432f477", - "tests/diffusion/models/minimax_h3/test_minimax_h3_pre_gate_cache.py": "a4924d0b3dd58ae413259edba6cf2487b3e41d391f943fb74358934be72eb53b", - "tests/diffusion/models/minimax_h3/test_minimax_h3_video_context.py": "ac7ff16b789b7304add457cea82859d6ae21548ac355344a390125f2e2c36fbe", - "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "e09b872f6900e821d42632a0ca09595f2affe6f9794049dbc74d69ba6a52b2b0", - "vllm_omni/config/omni_config.py": "6894120a330474f718d43bf2bf7928892eb996152839ce6469c34d18a7df09d1", - "vllm_omni/config/stage_config.py": "0752e8cc1c6f1021f737830d08663cfc118923a83fd909c607f9ec3ab2a3d2fb", - "vllm_omni/diffusion/attention/layer.py": "76c5fce3dabd23e39a8890316e1691c03bc203066cb60af5611ec4790eefc632", - "vllm_omni/diffusion/data.py": "5db83d5c71cb27cc7810d35fcfc891da80f0b003adbb16bdecc3816a63305e11", - "vllm_omni/diffusion/distributed/collective_gate.py": "01c6317cfa615840681b8bb8b8513fa3736d26e845a24ad9e1ab7f87489b962f", - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "48533c941a50719995206d18fa9198330824b940ecba9399b46ba7615b969ccb", - "vllm_omni/diffusion/models/minimax_h3/attention/video_context.py": "f53c94e2ac9a8df8f4d7abd753393b2907170a086e92ad7b23818810868e02e1", - "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "c5fab405ee82bc553e9a902fa27e6dfbc5213ec18311e17b6415396a69b50f43", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "73c96090e6fb0e13ce734bc96b23472fd5adb83bceb71f3eba2815d6343a33e4", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "577e290c2d92fc02822cd516cc499c3c4d1f465145ad2a837edfdfeaa09ed9fb", - "vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py": "e8e2e74fb8edb05e02d3e2328ea1f3ad5ddcfd7a10ed09c5cf3911b1d063e849", - "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "cff17568fbd856c28394d104d393a806931613b7b057f288006ad8c6fb9aa05a", - "vllm_omni/diffusion/offloader/base.py": "23922136deaf54e2d98e827b007b34c7e3ad36171b71bfe9060d390efc0b679f", - "vllm_omni/diffusion/offloader/distributed_layerwise_backend.py": "19fbce0f1036e4784d68cab798c499526a9099101909e41ce96dd0caa033b6a4", - "vllm_omni/engine/arg_utils.py": "52569e3688cc708729a48ca0a2c875c28193733c2f934f6ff89f6c815e4009c7", - "vllm_omni/entrypoints/cli/serve.py": "f8284ad6b751ba4a9e4f7254d78132cc2cfd49871a0f9af7fde49c6789d79cb0" - }, - "scope": "Legacy/pre-gate/DLO tests, three executed native four-rank HCCL/subsetFA suites and fresh current gates; no full-model quality/performance acceptance", - "status": "completed", - "returncode": 0, - "elapsed_seconds": 158.8542353429948 - }, - { - "trial": "c23-video-context-lse-operator-66d6ce78", - "cleanup_passed": true, - "source_unchanged": true, - "operator_gate_passed": false, - "files_before": { - "tests/diffusion/distributed/test_collective_gate.py": "6a02f32882d85eca1e3bb9a023ce3adb3d5b59c213845e11e85c96a89432f477", - "tests/diffusion/models/minimax_h3/test_minimax_h3_pre_gate_cache.py": "a4924d0b3dd58ae413259edba6cf2487b3e41d391f943fb74358934be72eb53b", - "tests/diffusion/models/minimax_h3/test_minimax_h3_video_context.py": "ac7ff16b789b7304add457cea82859d6ae21548ac355344a390125f2e2c36fbe", - "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "e09b872f6900e821d42632a0ca09595f2affe6f9794049dbc74d69ba6a52b2b0", - "vllm_omni/config/omni_config.py": "6894120a330474f718d43bf2bf7928892eb996152839ce6469c34d18a7df09d1", - "vllm_omni/config/stage_config.py": "0752e8cc1c6f1021f737830d08663cfc118923a83fd909c607f9ec3ab2a3d2fb", - "vllm_omni/diffusion/attention/layer.py": "76c5fce3dabd23e39a8890316e1691c03bc203066cb60af5611ec4790eefc632", - "vllm_omni/diffusion/data.py": "5db83d5c71cb27cc7810d35fcfc891da80f0b003adbb16bdecc3816a63305e11", - "vllm_omni/diffusion/distributed/collective_gate.py": "01c6317cfa615840681b8bb8b8513fa3736d26e845a24ad9e1ab7f87489b962f", - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "48533c941a50719995206d18fa9198330824b940ecba9399b46ba7615b969ccb", - "vllm_omni/diffusion/models/minimax_h3/attention/video_context.py": "f53c94e2ac9a8df8f4d7abd753393b2907170a086e92ad7b23818810868e02e1", - "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "c5fab405ee82bc553e9a902fa27e6dfbc5213ec18311e17b6415396a69b50f43", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "73c96090e6fb0e13ce734bc96b23472fd5adb83bceb71f3eba2815d6343a33e4", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "577e290c2d92fc02822cd516cc499c3c4d1f465145ad2a837edfdfeaa09ed9fb", - "vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py": "e8e2e74fb8edb05e02d3e2328ea1f3ad5ddcfd7a10ed09c5cf3911b1d063e849", - "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "cff17568fbd856c28394d104d393a806931613b7b057f288006ad8c6fb9aa05a", - "vllm_omni/diffusion/offloader/base.py": "23922136deaf54e2d98e827b007b34c7e3ad36171b71bfe9060d390efc0b679f", - "vllm_omni/diffusion/offloader/distributed_layerwise_backend.py": "19fbce0f1036e4784d68cab798c499526a9099101909e41ce96dd0caa033b6a4", - "vllm_omni/engine/arg_utils.py": "52569e3688cc708729a48ca0a2c875c28193733c2f934f6ff89f6c815e4009c7", - "vllm_omni/entrypoints/cli/serve.py": "f8284ad6b751ba4a9e4f7254d78132cc2cfd49871a0f9af7fde49c6789d79cb0" - }, - "scope": "Real captured H3 attention operator benchmark; full-model quality and HTTP are separate; no full-model quality/performance acceptance", - "status": "completed", - "returncode": 0, - "elapsed_seconds": 94.22810779500287, - "operator_benchmark": { - "scope": "One captured static5s attention input over50calls/8refresh42reuse; includes packing/LSE merge/lifecycle, excludes FFN/model/HTTP and changing-input quality.", - "fixture_sha256": "00767e6afb2ba777bb2677eb9b60dc2f00fb649f5006ae8c1d77191214fbf912", - "fixture_tensor_sha256": { - "query": "313095dd143adf0636d57c90d77b689405dee510df3ebd6385a49d071c3323e9", - "key": "6638aee85f0039ee814206e7c976a2d1a9a2b185723c7bbb74c69b1e68c5bad4", - "value": "101eede996c8b13d61d487019d47f0b5fd2193664d8c178665063675cb66fdca" - }, - "query_shape": [ - 1, - 39808, - 14, - 128 - ], - "valid_kv": 39771, - "target_start": 2475, - "finite": true, - "relative_l2_vs_dense": 0.0020166223403066397, - "baseline_cycle_samples_ms": [ - 3497.467301960569, - 3494.0786649822257, - 3493.1092370534316, - 3494.0861849463545, - 3495.386554975994 - ], - "candidate_cycle_samples_ms": [ - 2157.6043039676733, - 2157.4149020016193, - 2156.61749598803, - 2157.2416909621097, - 2159.0202350053005 - ], - "baseline_median_ms": 3494.0861849463545, - "candidate_median_ms": 2157.4149020016193, - "reduction_pct": 38.25524649917178, - "actual_context_calls": { - "video_context_refreshes": 8, - "video_context_reuses": 42, - "fresh_video_prefix_calls": 50, - "peak_cache_bytes": 135757440 - }, - "untimed_validation": { - "finite_all50": true, - "max_relative_l2_all50": 0.0020166223403066397 - }, - "operator_gate_passed": false, - "warmup_cycles_per_operation": 2, - "measured_cycles_per_operation": 5, - "order": "AB/BA/AB/BA/AB", - "probe_sha256": "4c494437eeaeda7721cb7b4e170297dbc49ba595c66e75a4afbae02895bcbf3f" - } - }, - { - "trial": "c23-video-context-lse-quality-5s-00-704b6bb0", - "model_controller": { - "status": "completed", - "returncode": 0, - "elapsed_seconds": 351.4251744679641 - }, - "model_diagnostic": { - "source_head": "6eb7bb094effb40853431208d1631effaf452a3f", - "samples_ms": [ - 199064.918508986 - ], - "latency_median_ms": 199064.918508986, - "scope": "http_complete_mp4", - "warmup": 0, - "repetitions": 1, - "kind": "quality", - "diagnostic": "audit", - "clean_timing": false, - "cleanup_passed": true - }, - "actual_counters": { - "request-000\\rank-0\\video_row_cache_stats.json": { - "request_counters": { - "full_steps": 8, - "partial_steps": 42, - "full_blocks": 400, - "partial_blocks": 2100, - "subset_attention_calls": 0, - "subset_query_rows": 0, - "omitted_query_rows": 0, - "partial_mlp_rows": 0, - "peak_cache_bytes": 6787872000, - "closed": true, - "storage_dtype": "torch.bfloat16", - "cache_stage": "video_context_lse", - "video_context_refreshes": 400, - "video_context_reuses": 2100, - "fresh_video_prefix_calls": 2500, - "full_row_mlp_blocks": 2500, - "fresh_attention_gates": 0, - "fresh_mlp_gates": 0, - "live_cache_bytes": 0 - }, - "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" - }, - "request-000\\rank-1\\video_row_cache_stats.json": { - "request_counters": { - "full_steps": 8, - "partial_steps": 42, - "full_blocks": 400, - "partial_blocks": 2100, - "subset_attention_calls": 0, - "subset_query_rows": 0, - "omitted_query_rows": 0, - "partial_mlp_rows": 0, - "peak_cache_bytes": 6787872000, - "closed": true, - "storage_dtype": "torch.bfloat16", - "cache_stage": "video_context_lse", - "video_context_refreshes": 400, - "video_context_reuses": 2100, - "fresh_video_prefix_calls": 2500, - "full_row_mlp_blocks": 2500, - "fresh_attention_gates": 0, - "fresh_mlp_gates": 0, - "live_cache_bytes": 0 - }, - "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" - }, - "request-000\\rank-2\\video_row_cache_stats.json": { - "request_counters": { - "full_steps": 8, - "partial_steps": 42, - "full_blocks": 400, - "partial_blocks": 2100, - "subset_attention_calls": 0, - "subset_query_rows": 0, - "omitted_query_rows": 0, - "partial_mlp_rows": 0, - "peak_cache_bytes": 6787872000, - "closed": true, - "storage_dtype": "torch.bfloat16", - "cache_stage": "video_context_lse", - "video_context_refreshes": 400, - "video_context_reuses": 2100, - "fresh_video_prefix_calls": 2500, - "full_row_mlp_blocks": 2500, - "fresh_attention_gates": 0, - "fresh_mlp_gates": 0, - "live_cache_bytes": 0 - }, - "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" - }, - "request-000\\rank-3\\video_row_cache_stats.json": { - "request_counters": { - "full_steps": 8, - "partial_steps": 42, - "full_blocks": 400, - "partial_blocks": 2100, - "subset_attention_calls": 0, - "subset_query_rows": 0, - "omitted_query_rows": 0, - "partial_mlp_rows": 0, - "peak_cache_bytes": 6787872000, - "closed": true, - "storage_dtype": "torch.bfloat16", - "cache_stage": "video_context_lse", - "video_context_refreshes": 400, - "video_context_reuses": 2100, - "fresh_video_prefix_calls": 2500, - "full_row_mlp_blocks": 2500, - "fresh_attention_gates": 0, - "fresh_mlp_gates": 0, - "live_cache_bytes": 0 - }, - "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" - }, - "request-000\\rank-0\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-000\\rank-1\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-000\\rank-2\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "request-000\\rank-3\\attention_path_counts.json": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - } - } - }, - { - "trial": "c23-video-context-lse-v2-native-dc375564", - "cleanup_passed": true, - "source_unchanged": true, - "required_native_test_executed": true, - "files_before": { - "tests/diffusion/distributed/test_collective_gate.py": "6a02f32882d85eca1e3bb9a023ce3adb3d5b59c213845e11e85c96a89432f477", - "tests/diffusion/models/minimax_h3/test_minimax_h3_pre_gate_cache.py": "a4924d0b3dd58ae413259edba6cf2487b3e41d391f943fb74358934be72eb53b", - "tests/diffusion/models/minimax_h3/test_minimax_h3_video_context.py": "1e76074b71f0ea4e070c435db623add702d1414a351427fd86ebb40e8dfd67fd", - "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "e09b872f6900e821d42632a0ca09595f2affe6f9794049dbc74d69ba6a52b2b0", - "vllm_omni/config/omni_config.py": "6894120a330474f718d43bf2bf7928892eb996152839ce6469c34d18a7df09d1", - "vllm_omni/config/stage_config.py": "0752e8cc1c6f1021f737830d08663cfc118923a83fd909c607f9ec3ab2a3d2fb", - "vllm_omni/diffusion/attention/layer.py": "76c5fce3dabd23e39a8890316e1691c03bc203066cb60af5611ec4790eefc632", - "vllm_omni/diffusion/data.py": "5db83d5c71cb27cc7810d35fcfc891da80f0b003adbb16bdecc3816a63305e11", - "vllm_omni/diffusion/distributed/collective_gate.py": "01c6317cfa615840681b8bb8b8513fa3736d26e845a24ad9e1ab7f87489b962f", - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "48533c941a50719995206d18fa9198330824b940ecba9399b46ba7615b969ccb", - "vllm_omni/diffusion/models/minimax_h3/attention/video_context.py": "f4cfd2225514b6f99d36567879f06fb4a42f9d235ac0114fc643001b345cb22e", - "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "c5fab405ee82bc553e9a902fa27e6dfbc5213ec18311e17b6415396a69b50f43", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "73c96090e6fb0e13ce734bc96b23472fd5adb83bceb71f3eba2815d6343a33e4", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "577e290c2d92fc02822cd516cc499c3c4d1f465145ad2a837edfdfeaa09ed9fb", - "vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py": "e8e2e74fb8edb05e02d3e2328ea1f3ad5ddcfd7a10ed09c5cf3911b1d063e849", - "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "cff17568fbd856c28394d104d393a806931613b7b057f288006ad8c6fb9aa05a", - "vllm_omni/diffusion/offloader/base.py": "23922136deaf54e2d98e827b007b34c7e3ad36171b71bfe9060d390efc0b679f", - "vllm_omni/diffusion/offloader/distributed_layerwise_backend.py": "19fbce0f1036e4784d68cab798c499526a9099101909e41ce96dd0caa033b6a4", - "vllm_omni/engine/arg_utils.py": "52569e3688cc708729a48ca0a2c875c28193733c2f934f6ff89f6c815e4009c7", - "vllm_omni/entrypoints/cli/serve.py": "f8284ad6b751ba4a9e4f7254d78132cc2cfd49871a0f9af7fde49c6789d79cb0" - }, - "scope": "Legacy/pre-gate/DLO tests, three executed native four-rank HCCL/subsetFA suites and fresh current gates; no full-model quality/performance acceptance", - "status": "completed", - "returncode": 0, - "elapsed_seconds": 157.24341464199824 - }, - { - "trial": "c23-video-context-lse-v2-operator-5614bf46", - "cleanup_passed": true, - "source_unchanged": true, - "operator_gate_passed": true, - "files_before": { - "tests/diffusion/distributed/test_collective_gate.py": "6a02f32882d85eca1e3bb9a023ce3adb3d5b59c213845e11e85c96a89432f477", - "tests/diffusion/models/minimax_h3/test_minimax_h3_pre_gate_cache.py": "a4924d0b3dd58ae413259edba6cf2487b3e41d391f943fb74358934be72eb53b", - "tests/diffusion/models/minimax_h3/test_minimax_h3_video_context.py": "1e76074b71f0ea4e070c435db623add702d1414a351427fd86ebb40e8dfd67fd", - "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "e09b872f6900e821d42632a0ca09595f2affe6f9794049dbc74d69ba6a52b2b0", - "vllm_omni/config/omni_config.py": "6894120a330474f718d43bf2bf7928892eb996152839ce6469c34d18a7df09d1", - "vllm_omni/config/stage_config.py": "0752e8cc1c6f1021f737830d08663cfc118923a83fd909c607f9ec3ab2a3d2fb", - "vllm_omni/diffusion/attention/layer.py": "76c5fce3dabd23e39a8890316e1691c03bc203066cb60af5611ec4790eefc632", - "vllm_omni/diffusion/data.py": "5db83d5c71cb27cc7810d35fcfc891da80f0b003adbb16bdecc3816a63305e11", - "vllm_omni/diffusion/distributed/collective_gate.py": "01c6317cfa615840681b8bb8b8513fa3736d26e845a24ad9e1ab7f87489b962f", - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "48533c941a50719995206d18fa9198330824b940ecba9399b46ba7615b969ccb", - "vllm_omni/diffusion/models/minimax_h3/attention/video_context.py": "f4cfd2225514b6f99d36567879f06fb4a42f9d235ac0114fc643001b345cb22e", - "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "c5fab405ee82bc553e9a902fa27e6dfbc5213ec18311e17b6415396a69b50f43", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "73c96090e6fb0e13ce734bc96b23472fd5adb83bceb71f3eba2815d6343a33e4", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "577e290c2d92fc02822cd516cc499c3c4d1f465145ad2a837edfdfeaa09ed9fb", - "vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py": "e8e2e74fb8edb05e02d3e2328ea1f3ad5ddcfd7a10ed09c5cf3911b1d063e849", - "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "cff17568fbd856c28394d104d393a806931613b7b057f288006ad8c6fb9aa05a", - "vllm_omni/diffusion/offloader/base.py": "23922136deaf54e2d98e827b007b34c7e3ad36171b71bfe9060d390efc0b679f", - "vllm_omni/diffusion/offloader/distributed_layerwise_backend.py": "19fbce0f1036e4784d68cab798c499526a9099101909e41ce96dd0caa033b6a4", - "vllm_omni/engine/arg_utils.py": "52569e3688cc708729a48ca0a2c875c28193733c2f934f6ff89f6c815e4009c7", - "vllm_omni/entrypoints/cli/serve.py": "f8284ad6b751ba4a9e4f7254d78132cc2cfd49871a0f9af7fde49c6789d79cb0" - }, - "scope": "Real captured H3 attention operator benchmark; full-model quality and HTTP are separate; no full-model quality/performance acceptance", - "status": "completed", - "returncode": 0, - "elapsed_seconds": 85.06819563696627, - "operator_benchmark": { - "scope": "One captured static5s attention input over50calls/8refresh42reuse; includes packing/LSE merge/lifecycle, excludes FFN/model/HTTP and changing-input quality.", - "fixture_sha256": "00767e6afb2ba777bb2677eb9b60dc2f00fb649f5006ae8c1d77191214fbf912", - "fixture_tensor_sha256": { - "query": "313095dd143adf0636d57c90d77b689405dee510df3ebd6385a49d071c3323e9", - "key": "6638aee85f0039ee814206e7c976a2d1a9a2b185723c7bbb74c69b1e68c5bad4", - "value": "101eede996c8b13d61d487019d47f0b5fd2193664d8c178665063675cb66fdca" - }, - "query_shape": [ - 1, - 39808, - 14, - 128 - ], - "valid_kv": 39771, - "target_start": 2475, - "finite": true, - "relative_l2_vs_dense": 0.0020166223403066397, - "baseline_cycle_samples_ms": [ - 3477.4728709599003, - 3477.3380900151096, - 3478.938751970418, - 3477.9253639862873, - 3477.4123289971612 - ], - "candidate_cycle_samples_ms": [ - 1166.7012969846837, - 1168.3354200213216, - 1166.2831630092114, - 1165.7003490254283, - 1166.387434990611 - ], - "baseline_median_ms": 3477.4728709599003, - "candidate_median_ms": 1166.387434990611, - "reduction_pct": 66.45876249011114, - "actual_context_calls": { - "video_context_refreshes": 8, - "video_context_reuses": 42, - "fresh_video_prefix_calls": 50, - "peak_cache_bytes": 135757440 - }, - "untimed_validation": { - "finite_all50": true, - "max_relative_l2_all50": 0.0020166223403066397 - }, - "real_merge_replay": { - "bit_equal_v1_real_merge": true, - "output_sha256": "b1fa215f72bdfbddf11431ddec28cd2a06049e4bbbbb9b0857b6312713139919", - "elements": 66834432, - "bytes": 133668864 - }, - "operator_gate_passed": true, - "warmup_cycles_per_operation": 2, - "measured_cycles_per_operation": 5, - "order": "AB/BA/AB/BA/AB", - "probe_sha256": "6f57858263f0e9e0ddd32724b2276224081645cab6149b06e221df2d441cb886" - } - } - ], - "decisions": [ - "C12 fixed profile stopped: zero actual predictions.", - "C14 pinned block-FP8 and block-INT8 prerequisites failed; per-token INT8 is not excluded.", - "C13 numerical qualification and clean performance are separate.", - "C15 fixed FP16 video residual profile failed finite velocity at first reuse.", - "C16 FP32 residual completed but failed three frozen audio quality guards; fixed profile stopped.", - "C17 completed5s but failed11frozen quality checks; fixed window profile stopped.", - "C18 native17PASS but operation reduced only7.25%; no full model run.", - "C19 native17PASS; operation52.85%faster but relL2 .258805 exceeded fixed .258052screen; no model run.", - "C20 native17PASS; operation48.42%faster and relL2 .263361 failed both fixed primitive gates; no model run.", - "C21 native36PASS; full5s completed but failed7frozen quality checks; fixed zero-order profile stopped.", - "C22 native45PASS; full5s completed but failed9frozen quality checks; fixed secant profile stopped.", - "C23 V1 native44PASS; weighted attention cycle reduced38.255%, below unchanged50%operator gate; no model run for V1.", - "C23 component diagnosis found a bit-equal native Torch merge faster than manual Triton on real tensors; V2 native44PASS and weighted operator66.459%reduction passed, full-model quality and HTTP remain separate.", - "C23 full5s completed with134input checks and expected counters but failed8frozen quality checks; fixed profile stopped, no15s or clean timing expansion.", - "First C22 model stopped before startup on duplicate controller aliases; audited canonicalization retains all attempts and time charges." - ] -} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c13-5s-strict-comparison.json b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c13-5s-strict-comparison.json deleted file mode 100644 index 9280a6dad..000000000 --- a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c13-5s-strict-comparison.json +++ /dev/null @@ -1,9 +0,0 @@ -{ - "verdict": "PASS", - "checks": 439, - "failures": [], - "missing_count": 0, - "scope": "specified latent points, schedule, decoded RGB24/PCM; no performance acceptance", - "source_commit": "e89b1d40d37f7f72767892a56fa7980ea572ba8e", - "performance_acceptance": "NOT_EVALUATED" -} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c15-native-qualification.json b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c15-native-qualification.json deleted file mode 100644 index 4e1d3d786..000000000 --- a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c15-native-qualification.json +++ /dev/null @@ -1,19 +0,0 @@ -{ - "trial": "c15-video-row-native-3a9721a6", - "tests_passed": 17, - "source_base_head": "0917a23dac44fee3b4d156de5cfa5addc329cc4a", - "files_before": { - "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "cf9fb13c5f24c7595d62d13cae6e283cafb2178612b20028d35357f0cafaac83", - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ca87df1936fd2694fcc768079e9e8f926b6d45969ddc61ebf353f5e726d70b8b", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "1258655fcdca4692c2306100846761a2b0b149bbcaa5cde48a9abaa559913bc3", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "a8257f98c2e30c7bc7978bf645966d2eaa98cd49a42a8a923c062b5808dcdcdf", - "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "c5fab405ee82bc553e9a902fa27e6dfbc5213ec18311e17b6415396a69b50f43", - "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "8ec1b3ae4cd252f5d95ab23e0232b983900939e8cb15d1e64f4740f7204abf39", - "vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py": "e8e2e74fb8edb05e02d3e2328ea1f3ad5ddcfd7a10ed09c5cf3911b1d063e849" - }, - "cleanup_passed": true, - "source_unchanged": true, - "required_native_test_executed": true, - "scope": "Focused state/block/request tests, real Ref2VA/FL2VA packing and native four-rank subset attention. Full-model quality/performance remain separate.", - "first_model_attempt": "Failed before step0: image references are not reference video spans. Task propagation corrected; original failure retained." -} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c16-5s-00-execution-proof.json b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c16-5s-00-execution-proof.json deleted file mode 100644 index 14308ba8c..000000000 --- a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c16-5s-00-execution-proof.json +++ /dev/null @@ -1,100 +0,0 @@ -{ - "run": "c16-video-row-fp32-quality-5s-00-8600757a", - "source_commit": "e0a3462fcc44de698b5dedcf8ff2c99f9802f757", - "latency_diagnostic_ms": 197716.36760496767, - "clean_timing": false, - "cleanup_passed": true, - "helpers_unchanged": true, - "input_checks": { - "verdict": "PASS", - "checks": 134, - "failures": [], - "scope": "input/schedule/geometry/finite-state checks only; perceptual quality and latency acceptance are separate" - }, - "counters_by_rank": { - "0": { - "full_steps": 17, - "partial_steps": 33, - "full_blocks": 850, - "partial_blocks": 1650, - "subset_attention_calls": 1650, - "subset_query_rows": 4083750, - "omitted_query_rows": 61538400, - "partial_mlp_rows": 4083750, - "peak_cache_bytes": 8039270400, - "closed": true, - "storage_dtype": "torch.float32", - "live_cache_bytes": 0 - }, - "1": { - "full_steps": 17, - "partial_steps": 33, - "full_blocks": 850, - "partial_blocks": 1650, - "subset_attention_calls": 1650, - "subset_query_rows": 4083750, - "omitted_query_rows": 61538400, - "partial_mlp_rows": 0, - "peak_cache_bytes": 10700390400, - "closed": true, - "storage_dtype": "torch.float32", - "live_cache_bytes": 0 - }, - "2": { - "full_steps": 17, - "partial_steps": 33, - "full_blocks": 850, - "partial_blocks": 1650, - "subset_attention_calls": 1650, - "subset_query_rows": 4083750, - "omitted_query_rows": 61538400, - "partial_mlp_rows": 0, - "peak_cache_bytes": 10700390400, - "closed": true, - "storage_dtype": "torch.float32", - "live_cache_bytes": 0 - }, - "3": { - "full_steps": 17, - "partial_steps": 33, - "full_blocks": 850, - "partial_blocks": 1650, - "subset_attention_calls": 1650, - "subset_query_rows": 4083750, - "omitted_query_rows": 61538400, - "partial_mlp_rows": 0, - "peak_cache_bytes": 10660608000, - "closed": true, - "storage_dtype": "torch.float32", - "live_cache_bytes": 0 - } - }, - "step4_range": { - "records": 200, - "elements": 10025164800, - "fp32_nonfinite": 0, - "counterfactual_fp16_nonfinite": 166071, - "max_abs_delta": 2232320.0 - }, - "quality_verdict": "FAIL", - "observed_checks": 39, - "failures": [ - "5s-00:clap: similarity", - "5s-00: clap segment", - "5s-00: audio temporal envelope" - ], - "missing_samples": [ - "5s-01", - "5s-02", - "5s-03", - "5s-04", - "15s-00", - "15s-01", - "15s-02", - "15s-03", - "15s-04" - ], - "media_sha256": "eeb8b2dca5c467f90680986a545d874ae6df8fa558ebd1816974af5e692f2951", - "features_sha256": "170179e1c4916ae9317456206091d9822496bad04b941c42bfe74abd82db0bd2", - "decision": "Reject fixed C16 profile on three observed audio violations; no5s01/15s expansion or clean performance qualification." -} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c16-native-qualification.json b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c16-native-qualification.json deleted file mode 100644 index d69798b07..000000000 --- a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c16-native-qualification.json +++ /dev/null @@ -1,18 +0,0 @@ -{ - "trial": "c16-video-row-fp32-native-d3fe320c", - "tests_passed": 18, - "source_base_head": "81c1beb6e47ecfa7602e4309a7342a38108aabc8", - "files_before": { - "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "c555d76f7bd6457a81238fca56100d11cb7ab043a4a649d55e23b6203d09bde2", - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ca87df1936fd2694fcc768079e9e8f926b6d45969ddc61ebf353f5e726d70b8b", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "1258655fcdca4692c2306100846761a2b0b149bbcaa5cde48a9abaa559913bc3", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "a8257f98c2e30c7bc7978bf645966d2eaa98cd49a42a8a923c062b5808dcdcdf", - "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "c5fab405ee82bc553e9a902fa27e6dfbc5213ec18311e17b6415396a69b50f43", - "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "e09b872f6900e821d42632a0ca09595f2affe6f9794049dbc74d69ba6a52b2b0", - "vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py": "e8e2e74fb8edb05e02d3e2328ea1f3ad5ddcfd7a10ed09c5cf3911b1d063e849" - }, - "cleanup_passed": true, - "source_unchanged": true, - "required_native_test_executed": true, - "scope": "Focused state/block tests and native four-rank subset attention; no full-model quality/performance acceptance" -} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c17-5s-00-execution-proof.json b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c17-5s-00-execution-proof.json deleted file mode 100644 index 81a3fd2c6..000000000 --- a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c17-5s-00-execution-proof.json +++ /dev/null @@ -1,75 +0,0 @@ -{ - "run": "c17-video-window-quality-5s-00-a8c689e4", - "source_commit": "c1589a2b5e4ada617e7cca38ef7c300fe9495c58", - "latency_diagnostic_ms": 210397.12271699682, - "clean_timing": false, - "cleanup_passed": true, - "helpers_unchanged": true, - "input_checks": { - "verdict": "PASS", - "checks": 134, - "failures": [], - "scope": "input/schedule/geometry/finite-state checks only; perceptual quality and latency acceptance are separate" - }, - "counters_by_rank": { - "0": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 4900, - "_video_temporal_window_bnsd": 2400 - }, - "1": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 4900, - "_video_temporal_window_bnsd": 2400 - }, - "2": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 4900, - "_video_temporal_window_bnsd": 2400 - }, - "3": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 4900, - "_video_temporal_window_bnsd": 2400 - } - }, - "quality_verdict": "FAIL", - "observed_checks": 39, - "failures": [ - "5s-00:dino: similarity", - "5s-00:clip: similarity", - "5s-00:clap: similarity", - "5s-00: dino lower tail", - "5s-00: clap segment", - "5s-00:reference_clip_cosine: adherence", - "5s-00:reference_dino_cosine: adherence", - "5s-00:prompt_clip_cosine: adherence", - "5s-00: audio level", - "5s-00: audio temporal envelope", - "5s-00: motion energy" - ], - "missing_samples": [ - "5s-01", - "5s-02", - "5s-03", - "5s-04", - "15s-00", - "15s-01", - "15s-02", - "15s-03", - "15s-04" - ], - "decision": "Reject fixed C17 window profile; no15s or clean performance expansion." -} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c17-native-qualification.json b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c17-native-qualification.json deleted file mode 100644 index 9a5138353..000000000 --- a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c17-native-qualification.json +++ /dev/null @@ -1,13 +0,0 @@ -{ - "trial": "c17-video-window-native-9b48cb56", - "tests_passed": 15, - "source_base_head": "e89b1d40d37f7f72767892a56fa7980ea572ba8e", - "files_before": { - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "66ca27991ce42ce302e5de078be6395c9d1ab53752378babef25089f3f75c3aa", - "tests/diffusion/models/minimax_h3/test_minimax_h3_video_window.py": "0bc59f35eff41164e125a5070e034c94ea60bf0112733ebe59eb6378eccbfed8" - }, - "cleanup_passed": true, - "source_unchanged": true, - "required_native_test_executed": true, - "scope": "CPU window reference/routing and native batched BF16 FAS; no full-model quality/performance acceptance" -} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c18-native-qualification.json b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c18-native-qualification.json deleted file mode 100644 index cc7a0daa9..000000000 --- a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c18-native-qualification.json +++ /dev/null @@ -1,13 +0,0 @@ -{ - "trial": "c18-video-pooled-context-native-1673db72", - "tests_passed": 17, - "source_base_head": "e89b1d40d37f7f72767892a56fa7980ea572ba8e", - "files_before": { - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "1febc030a7917b6e10fe287014fe2aacef76ade7bf58ebb604fdedc36fb7c21d", - "tests/diffusion/models/minimax_h3/test_minimax_h3_pooled_context.py": "45a9ae5ba56858daee6a1088a8c8b1ed612ea9afd3c229c90a9b6db3c80a06b2" - }, - "cleanup_passed": true, - "source_unchanged": true, - "required_native_test_executed": true, - "scope": "Independent weighted global-context reference and native padded-head BF16 FAS; no full-model quality/performance acceptance" -} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c19-native-qualification.json b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c19-native-qualification.json deleted file mode 100644 index d3a307edb..000000000 --- a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c19-native-qualification.json +++ /dev/null @@ -1,13 +0,0 @@ -{ - "trial": "c19-video-global-pool-native-f0f4c502", - "tests_passed": 17, - "source_base_head": "e89b1d40d37f7f72767892a56fa7980ea572ba8e", - "files_before": { - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ddfd6d6822cc6a5528fc95539f7f98dfdeed25d5489da3ed5cf76e6076a5ec94", - "tests/diffusion/models/minimax_h3/test_minimax_h3_global_pool.py": "5457db550b59e3373be8f5f21873c1b78ac357a6ac170c2603ee531b50e91467" - }, - "cleanup_passed": true, - "source_unchanged": true, - "required_native_test_executed": true, - "scope": "Independent weighted global-context reference and native padded-head BF16 FAS; no full-model quality/performance acceptance" -} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c20-native-qualification.json b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c20-native-qualification.json deleted file mode 100644 index fc85c5389..000000000 --- a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c20-native-qualification.json +++ /dev/null @@ -1,13 +0,0 @@ -{ - "trial": "c20-video-variance-pool-native-50101063", - "tests_passed": 17, - "source_base_head": "e89b1d40d37f7f72767892a56fa7980ea572ba8e", - "files_before": { - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "4390450bc5324f7028125101cd4bb32e9de237b04bc12fa879b48110d7122e3c", - "tests/diffusion/models/minimax_h3/test_minimax_h3_variance_pool.py": "e871fb4b222d61d10b6c442a333a52c976cceaa0d406233502ecbcaddfeba8c9" - }, - "cleanup_passed": true, - "source_unchanged": true, - "required_native_test_executed": true, - "scope": "Independent weighted global-context reference and native padded-head BF16 FAS; no full-model quality/performance acceptance" -} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c21-5s-00-execution-proof.json b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c21-5s-00-execution-proof.json deleted file mode 100644 index b657b2f8b..000000000 --- a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c21-5s-00-execution-proof.json +++ /dev/null @@ -1,139 +0,0 @@ -{ - "run": "c21-video-pre-gate-quality-5s-00-7265c33e", - "source_commit": "d48fa51d03dd748568cbcf57ed40f533c691c9bd", - "latency_diagnostic_ms": 163805.5620979867, - "clean_timing": false, - "cleanup_passed": true, - "helpers_unchanged": true, - "input_checks": { - "verdict": "PASS", - "checks": 134, - "failures": [], - "scope": "input/schedule/geometry/finite-state checks only; perceptual quality and latency acceptance are separate" - }, - "counters_by_rank": { - "0": { - "full_steps": 8, - "partial_steps": 42, - "full_blocks": 400, - "partial_blocks": 2100, - "subset_attention_calls": 2100, - "subset_query_rows": 5197500, - "omitted_query_rows": 78321600, - "partial_mlp_rows": 5197500, - "peak_cache_bytes": 8039270400, - "closed": true, - "storage_dtype": "torch.bfloat16", - "cache_stage": "pre_gate", - "fresh_attention_gates": 2100, - "fresh_mlp_gates": 2100, - "live_cache_bytes": 0 - }, - "1": { - "full_steps": 8, - "partial_steps": 42, - "full_blocks": 400, - "partial_blocks": 2100, - "subset_attention_calls": 2100, - "subset_query_rows": 5197500, - "omitted_query_rows": 78321600, - "partial_mlp_rows": 0, - "peak_cache_bytes": 10700390400, - "closed": true, - "storage_dtype": "torch.bfloat16", - "cache_stage": "pre_gate", - "fresh_attention_gates": 2100, - "fresh_mlp_gates": 2100, - "live_cache_bytes": 0 - }, - "2": { - "full_steps": 8, - "partial_steps": 42, - "full_blocks": 400, - "partial_blocks": 2100, - "subset_attention_calls": 2100, - "subset_query_rows": 5197500, - "omitted_query_rows": 78321600, - "partial_mlp_rows": 0, - "peak_cache_bytes": 10700390400, - "closed": true, - "storage_dtype": "torch.bfloat16", - "cache_stage": "pre_gate", - "fresh_attention_gates": 2100, - "fresh_mlp_gates": 2100, - "live_cache_bytes": 0 - }, - "3": { - "full_steps": 8, - "partial_steps": 42, - "full_blocks": 400, - "partial_blocks": 2100, - "subset_attention_calls": 2100, - "subset_query_rows": 5197500, - "omitted_query_rows": 78321600, - "partial_mlp_rows": 0, - "peak_cache_bytes": 10660608000, - "closed": true, - "storage_dtype": "torch.bfloat16", - "cache_stage": "pre_gate", - "fresh_attention_gates": 2100, - "fresh_mlp_gates": 2100, - "live_cache_bytes": 0 - } - }, - "actual_paths_by_rank": { - "0": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "1": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "2": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "3": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - } - }, - "quality_verdict": "FAIL", - "observed_checks": 39, - "failures": [ - "5s-00:dino: similarity", - "5s-00:clip: similarity", - "5s-00:clap: similarity", - "5s-00: dino lower tail", - "5s-00: clap segment", - "5s-00:reference_dino_cosine: adherence", - "5s-00: audio temporal envelope" - ], - "missing_samples": [ - "5s-01", - "5s-02", - "5s-03", - "5s-04", - "15s-00", - "15s-01", - "15s-02", - "15s-03", - "15s-04" - ], - "media_sha256": "c03567a778ed6aae6a8b8e05674e6d83065178871751603ba3ffe2f7a363a4ce", - "features_sha256": "71638d355adff149f27d135fceb8b0b0e2de34557a26344113218adcb2a6ea75", - "decision": "Reject fixed C21 eight-anchor zero-order pre-gate profile on seven observed quality violations; no15s or clean timing expansion." -} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c21-c22-direction-identity-correction.json b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c21-c22-direction-identity-correction.json deleted file mode 100644 index 284057775..000000000 --- a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c21-c22-direction-identity-correction.json +++ /dev/null @@ -1,79 +0,0 @@ -{ - "checked_at": "2026-11-02T06:55:00.775781+00:00", - "before_count": 24, - "after_count": 23, - "before_directions": [ - "C0-upstream-migration", - "C1-BF16-attention-execution", - "C2-HSDP2-replicate2", - "C3-Cache-DiT-residual-reuse", - "C4-dense-BF16-attention-alternatives", - "C5-H3-Laser-video-queries-BF16-context", - "C6-Laser-input-range-scaling", - "C7-native-RainFusion-BF16", - "C8-reference-KV-device-refresh2", - "C9-TP4-resident-DiT", - "C10-H3-audio-cache-veto", - "C11-H3-modality-sigma-residual", - "C12-H3-state-aligned-residual", - "C13-C1-DLO-QKV-order", - "C14-A3-quantized-attention-prerequisites", - "C15-H3-target-video-row-cache", - "C16-H3-video-row-FP32-residual", - "C17-H3-video-temporal-window", - "C18-H3-video-global-pooled-context", - "C19-H3-video-shared-global-pool", - "C20-H3-video-variance-pool", - "C21-H3-video-row-FP32-residual", - "C21-H3-video-pre-gate-cache", - "C22-H3-video-row-FP32-residual" - ], - "canonical_directions": [ - "C0-upstream-migration", - "C1-BF16-attention-execution", - "C2-HSDP2-replicate2", - "C3-Cache-DiT-residual-reuse", - "C4-dense-BF16-attention-alternatives", - "C5-H3-Laser-video-queries-BF16-context", - "C6-Laser-input-range-scaling", - "C7-native-RainFusion-BF16", - "C8-reference-KV-device-refresh2", - "C9-TP4-resident-DiT", - "C10-H3-audio-cache-veto", - "C11-H3-modality-sigma-residual", - "C12-H3-state-aligned-residual", - "C13-C1-DLO-QKV-order", - "C14-A3-quantized-attention-prerequisites", - "C15-H3-target-video-row-cache", - "C16-H3-video-row-FP32-residual", - "C17-H3-video-temporal-window", - "C18-H3-video-global-pooled-context", - "C19-H3-video-shared-global-pool", - "C20-H3-video-variance-pool", - "C21-H3-video-pre-gate-cache", - "C22-H3-video-pre-gate-sigma-secant" - ], - "aliases": { - "C21-H3-video-row-FP32-residual": "C21-H3-video-pre-gate-cache", - "C22-H3-video-row-FP32-residual": "C22-H3-video-pre-gate-sigma-secant" - }, - "scope": "Controller label correction for two existing mechanisms; no new direction/budget approval inferred.", - "cost_policy": "Retain all raw attempts, failures and retries. Original start/deadline/device-hour ceiling unchanged; wall-clock accounting includes this repair.", - "source_binding": { - "C21": "d48fa51d03dd748568cbcf57ed40f533c691c9bd", - "C22": "7b0d12bfefdf05868e93e74bc91f04c70ee34128" - }, - "retained_failed_model": "c22-video-pre-gate-secant-quality-5s-00-72202558", - "controllers": { - "C21": { - "before_sha256": "1ca526f2040b5c53d091068766dac0346c79268c4f5ae0b10674e8d2a97b3a40", - "after_sha256": "a3ab34ba341fad6b1e3041c8b06716cc669a3fe0171bad2b6e4a6693e54d0780", - "original_bytes": "run_c21_native_checks-before-family-alias.py" - }, - "C22": { - "before_sha256": "3e58beffbe688a15cfa73a00ec8fbf20c6e78a25235a002422898f6506cae1d2", - "after_sha256": "4a9175a8f8db782df06f14af6a0329368da27ccb5ea776a02ce97ad86df0dd33", - "original_bytes": "run_c22_native_checks-before-family-alias.py" - } - } -} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c21-native-qualification.json b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c21-native-qualification.json deleted file mode 100644 index f29e129c6..000000000 --- a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c21-native-qualification.json +++ /dev/null @@ -1,29 +0,0 @@ -{ - "trial": "c21-video-pre-gate-native-3c5eebbe", - "tests_passed": 36, - "source_base_head": "d027b4d0c53bf40876215d4e3856bc17450a4c39", - "files_before": { - "tests/diffusion/distributed/test_collective_gate.py": "6a02f32882d85eca1e3bb9a023ce3adb3d5b59c213845e11e85c96a89432f477", - "tests/diffusion/models/minimax_h3/test_minimax_h3_pre_gate_cache.py": "a4924d0b3dd58ae413259edba6cf2487b3e41d391f943fb74358934be72eb53b", - "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "e09b872f6900e821d42632a0ca09595f2affe6f9794049dbc74d69ba6a52b2b0", - "vllm_omni/config/omni_config.py": "6894120a330474f718d43bf2bf7928892eb996152839ce6469c34d18a7df09d1", - "vllm_omni/config/stage_config.py": "0752e8cc1c6f1021f737830d08663cfc118923a83fd909c607f9ec3ab2a3d2fb", - "vllm_omni/diffusion/attention/layer.py": "76c5fce3dabd23e39a8890316e1691c03bc203066cb60af5611ec4790eefc632", - "vllm_omni/diffusion/data.py": "5db83d5c71cb27cc7810d35fcfc891da80f0b003adbb16bdecc3816a63305e11", - "vllm_omni/diffusion/distributed/collective_gate.py": "01c6317cfa615840681b8bb8b8513fa3736d26e845a24ad9e1ab7f87489b962f", - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ca87df1936fd2694fcc768079e9e8f926b6d45969ddc61ebf353f5e726d70b8b", - "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "c5fab405ee82bc553e9a902fa27e6dfbc5213ec18311e17b6415396a69b50f43", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "8933b536317652d0dbc7a974ea7e0bca928e7b3bdf438988999ff5a498a6ec9d", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "019bf71106ba3263b55ad7b682064e3a176eecb81725cb48bbf178387356d5d9", - "vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py": "e8e2e74fb8edb05e02d3e2328ea1f3ad5ddcfd7a10ed09c5cf3911b1d063e849", - "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "04a9212994b5c8593f8d196820300ded077fed9af4274b64d87d9e6bf71eae07", - "vllm_omni/diffusion/offloader/base.py": "23922136deaf54e2d98e827b007b34c7e3ad36171b71bfe9060d390efc0b679f", - "vllm_omni/diffusion/offloader/distributed_layerwise_backend.py": "19fbce0f1036e4784d68cab798c499526a9099101909e41ce96dd0caa033b6a4", - "vllm_omni/engine/arg_utils.py": "52569e3688cc708729a48ca0a2c875c28193733c2f934f6ff89f6c815e4009c7", - "vllm_omni/entrypoints/cli/serve.py": "f8284ad6b751ba4a9e4f7254d78132cc2cfd49871a0f9af7fde49c6789d79cb0" - }, - "cleanup_passed": true, - "source_unchanged": true, - "required_native_test_executed": true, - "scope": "Legacy/pre-gate/DLO tests, two executed native four-rank HCCL/subsetFA suites and fresh current gates; no full-model quality/performance acceptance" -} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c22-5s-00-execution-proof.json b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c22-5s-00-execution-proof.json deleted file mode 100644 index 287869077..000000000 --- a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c22-5s-00-execution-proof.json +++ /dev/null @@ -1,151 +0,0 @@ -{ - "run": "c22-video-pre-gate-secant-v2-quality-5s-00-777147cb", - "source_commit": "7b0d12bfefdf05868e93e74bc91f04c70ee34128", - "latency_diagnostic_ms": 166858.99333201814, - "clean_timing": false, - "cleanup_passed": true, - "helpers_unchanged": true, - "input_checks": { - "verdict": "PASS", - "checks": 134, - "failures": [], - "scope": "input/schedule/geometry/finite-state checks only; perceptual quality and latency acceptance are separate" - }, - "counters_by_rank": { - "0": { - "full_steps": 8, - "partial_steps": 42, - "full_blocks": 400, - "partial_blocks": 2100, - "subset_attention_calls": 2100, - "subset_query_rows": 5197500, - "omitted_query_rows": 78321600, - "partial_mlp_rows": 5197500, - "peak_cache_bytes": 12061056000, - "closed": true, - "storage_dtype": "torch.bfloat16", - "cache_stage": "pre_gate_secant", - "secant_forecasts": 4200, - "slope_codec": "int8_per_channel", - "fresh_attention_gates": 2100, - "fresh_mlp_gates": 2100, - "live_cache_bytes": 0 - }, - "1": { - "full_steps": 8, - "partial_steps": 42, - "full_blocks": 400, - "partial_blocks": 2100, - "subset_attention_calls": 2100, - "subset_query_rows": 5197500, - "omitted_query_rows": 78321600, - "partial_mlp_rows": 0, - "peak_cache_bytes": 16052736000, - "closed": true, - "storage_dtype": "torch.bfloat16", - "cache_stage": "pre_gate_secant", - "secant_forecasts": 4200, - "slope_codec": "int8_per_channel", - "fresh_attention_gates": 2100, - "fresh_mlp_gates": 2100, - "live_cache_bytes": 0 - }, - "2": { - "full_steps": 8, - "partial_steps": 42, - "full_blocks": 400, - "partial_blocks": 2100, - "subset_attention_calls": 2100, - "subset_query_rows": 5197500, - "omitted_query_rows": 78321600, - "partial_mlp_rows": 0, - "peak_cache_bytes": 16052736000, - "closed": true, - "storage_dtype": "torch.bfloat16", - "cache_stage": "pre_gate_secant", - "secant_forecasts": 4200, - "slope_codec": "int8_per_channel", - "fresh_attention_gates": 2100, - "fresh_mlp_gates": 2100, - "live_cache_bytes": 0 - }, - "3": { - "full_steps": 8, - "partial_steps": 42, - "full_blocks": 400, - "partial_blocks": 2100, - "subset_attention_calls": 2100, - "subset_query_rows": 5197500, - "omitted_query_rows": 78321600, - "partial_mlp_rows": 0, - "peak_cache_bytes": 15993062400, - "closed": true, - "storage_dtype": "torch.bfloat16", - "cache_stage": "pre_gate_secant", - "secant_forecasts": 4200, - "slope_codec": "int8_per_channel", - "fresh_attention_gates": 2100, - "fresh_mlp_gates": 2100, - "live_cache_bytes": 0 - } - }, - "actual_paths_by_rank": { - "0": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "1": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "2": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "3": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - } - }, - "quality_verdict": "FAIL", - "observed_checks": 39, - "failures": [ - "5s-00:dino: similarity", - "5s-00:clip: similarity", - "5s-00:clap: similarity", - "5s-00: dino lower tail", - "5s-00: clap segment", - "5s-00:reference_clip_cosine: adherence", - "5s-00:reference_dino_cosine: adherence", - "5s-00:audio_caption_clap_cosine: adherence", - "5s-00: motion energy" - ], - "missing_samples": [ - "5s-01", - "5s-02", - "5s-03", - "5s-04", - "15s-00", - "15s-01", - "15s-02", - "15s-03", - "15s-04" - ], - "media_sha256": "d0f6bb62948680a171e7368ec9d6b3ad916fb856b9e0969fc437b292aae7d1e5", - "features_sha256": "c3121d4fe81f3eadd50c4b720cf161b5ba6fa9f6596828395520ede1d296bcba", - "decision": "Reject fixed C22 sigma-secant pre-gate profile on nine observed violations; no15s or clean timing expansion.", - "prior_failed_attempt": "c22-video-pre-gate-secant-quality-5s-00-72202558", - "direction_identity_correction": "c21-c22-direction-identity-correction.json" -} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c22-native-qualification.json b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c22-native-qualification.json deleted file mode 100644 index b2271d113..000000000 --- a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c22-native-qualification.json +++ /dev/null @@ -1,30 +0,0 @@ -{ - "trial": "c22-video-pre-gate-secant-native-3521747e", - "tests_passed": 45, - "source_base_head": "d48fa51d03dd748568cbcf57ed40f533c691c9bd", - "files_before": { - "tests/diffusion/distributed/test_collective_gate.py": "6a02f32882d85eca1e3bb9a023ce3adb3d5b59c213845e11e85c96a89432f477", - "tests/diffusion/models/minimax_h3/test_minimax_h3_pre_gate_cache.py": "b8394d22b054d60c1ab8e069e9600e1947cc73b5826e1f206e66c548123f632d", - "tests/diffusion/models/minimax_h3/test_minimax_h3_pre_gate_secant.py": "70a215fbb0dc63b90652d96f8f6d106a5785bd7d64dae75a2c4f6ec5c627ac1c", - "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "e09b872f6900e821d42632a0ca09595f2affe6f9794049dbc74d69ba6a52b2b0", - "vllm_omni/config/omni_config.py": "6894120a330474f718d43bf2bf7928892eb996152839ce6469c34d18a7df09d1", - "vllm_omni/config/stage_config.py": "0752e8cc1c6f1021f737830d08663cfc118923a83fd909c607f9ec3ab2a3d2fb", - "vllm_omni/diffusion/attention/layer.py": "76c5fce3dabd23e39a8890316e1691c03bc203066cb60af5611ec4790eefc632", - "vllm_omni/diffusion/data.py": "5db83d5c71cb27cc7810d35fcfc891da80f0b003adbb16bdecc3816a63305e11", - "vllm_omni/diffusion/distributed/collective_gate.py": "01c6317cfa615840681b8bb8b8513fa3736d26e845a24ad9e1ab7f87489b962f", - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ca87df1936fd2694fcc768079e9e8f926b6d45969ddc61ebf353f5e726d70b8b", - "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "b991ebb5d43242357d5ab8cfb476808a76fc0658d6048db6f3b7131122c8a993", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "8933b536317652d0dbc7a974ea7e0bca928e7b3bdf438988999ff5a498a6ec9d", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "3904bc8ac6679bc51614b95f5510789331a41bf20a43a517f3a99d33505201a6", - "vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py": "e8e2e74fb8edb05e02d3e2328ea1f3ad5ddcfd7a10ed09c5cf3911b1d063e849", - "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "e21e0afe8f8948d805d374428a34f3b2c5c0b5041d63ea791f047af8f455cf16", - "vllm_omni/diffusion/offloader/base.py": "23922136deaf54e2d98e827b007b34c7e3ad36171b71bfe9060d390efc0b679f", - "vllm_omni/diffusion/offloader/distributed_layerwise_backend.py": "19fbce0f1036e4784d68cab798c499526a9099101909e41ce96dd0caa033b6a4", - "vllm_omni/engine/arg_utils.py": "52569e3688cc708729a48ca0a2c875c28193733c2f934f6ff89f6c815e4009c7", - "vllm_omni/entrypoints/cli/serve.py": "f8284ad6b751ba4a9e4f7254d78132cc2cfd49871a0f9af7fde49c6789d79cb0" - }, - "cleanup_passed": true, - "source_unchanged": true, - "required_native_test_executed": true, - "scope": "Legacy/pre-gate/DLO tests, three executed native four-rank HCCL/subsetFA suites and fresh current gates; no full-model quality/performance acceptance" -} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c23-5s-00-execution-proof.json b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c23-5s-00-execution-proof.json deleted file mode 100644 index 1858c95cf..000000000 --- a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c23-5s-00-execution-proof.json +++ /dev/null @@ -1,157 +0,0 @@ -{ - "run": "c23-video-context-lse-quality-5s-00-704b6bb0", - "source_commit": "6eb7bb094effb40853431208d1631effaf452a3f", - "latency_diagnostic_ms": 199064.918508986, - "clean_timing": false, - "cleanup_passed": true, - "helpers_unchanged": true, - "source_unchanged": true, - "input_checks": { - "verdict": "PASS", - "checks": 134, - "failures": [], - "scope": "input/schedule/geometry/finite-state checks only; perceptual quality and latency acceptance are separate" - }, - "counters_by_rank": { - "0": { - "full_steps": 8, - "partial_steps": 42, - "full_blocks": 400, - "partial_blocks": 2100, - "subset_attention_calls": 0, - "subset_query_rows": 0, - "omitted_query_rows": 0, - "partial_mlp_rows": 0, - "peak_cache_bytes": 6787872000, - "closed": true, - "storage_dtype": "torch.bfloat16", - "cache_stage": "video_context_lse", - "video_context_refreshes": 400, - "video_context_reuses": 2100, - "fresh_video_prefix_calls": 2500, - "full_row_mlp_blocks": 2500, - "fresh_attention_gates": 0, - "fresh_mlp_gates": 0, - "live_cache_bytes": 0 - }, - "1": { - "full_steps": 8, - "partial_steps": 42, - "full_blocks": 400, - "partial_blocks": 2100, - "subset_attention_calls": 0, - "subset_query_rows": 0, - "omitted_query_rows": 0, - "partial_mlp_rows": 0, - "peak_cache_bytes": 6787872000, - "closed": true, - "storage_dtype": "torch.bfloat16", - "cache_stage": "video_context_lse", - "video_context_refreshes": 400, - "video_context_reuses": 2100, - "fresh_video_prefix_calls": 2500, - "full_row_mlp_blocks": 2500, - "fresh_attention_gates": 0, - "fresh_mlp_gates": 0, - "live_cache_bytes": 0 - }, - "2": { - "full_steps": 8, - "partial_steps": 42, - "full_blocks": 400, - "partial_blocks": 2100, - "subset_attention_calls": 0, - "subset_query_rows": 0, - "omitted_query_rows": 0, - "partial_mlp_rows": 0, - "peak_cache_bytes": 6787872000, - "closed": true, - "storage_dtype": "torch.bfloat16", - "cache_stage": "video_context_lse", - "video_context_refreshes": 400, - "video_context_reuses": 2100, - "fresh_video_prefix_calls": 2500, - "full_row_mlp_blocks": 2500, - "fresh_attention_gates": 0, - "fresh_mlp_gates": 0, - "live_cache_bytes": 0 - }, - "3": { - "full_steps": 8, - "partial_steps": 42, - "full_blocks": 400, - "partial_blocks": 2100, - "subset_attention_calls": 0, - "subset_query_rows": 0, - "omitted_query_rows": 0, - "partial_mlp_rows": 0, - "peak_cache_bytes": 6787872000, - "closed": true, - "storage_dtype": "torch.bfloat16", - "cache_stage": "video_context_lse", - "video_context_refreshes": 400, - "video_context_reuses": 2100, - "fresh_video_prefix_calls": 2500, - "full_row_mlp_blocks": 2500, - "fresh_attention_gates": 0, - "fresh_mlp_gates": 0, - "live_cache_bytes": 0 - } - }, - "actual_attention_counts": { - "0": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "1": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "2": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - }, - "3": { - "dlo._issue_chunked_allgather.ungated": 100, - "dlo._allgather_after_attn_a2a": 2600, - "dlo._issue_chunked_allgather.gated": 2600, - "dlo._flush_deferred_allgather": 2600, - "_prefix_bnsd": 2500 - } - }, - "quality_verdict": "FAIL", - "observed_checks": 39, - "failures": [ - "5s-00:dino: similarity", - "5s-00:clip: similarity", - "5s-00:clap: similarity", - "5s-00: clap segment", - "5s-00:reference_dino_cosine: adherence", - "5s-00:audio_caption_clap_cosine: adherence", - "5s-00: audio level", - "5s-00: audio temporal envelope" - ], - "missing_samples": [ - "5s-01", - "5s-02", - "5s-03", - "5s-04", - "15s-00", - "15s-01", - "15s-02", - "15s-03", - "15s-04" - ], - "media_sha256": "f3a34a7bf46a2049b154edacab0bd0c9ae254c6dcdee02c50133a25c987ab5c5", - "features_sha256": "e10cfd296ebc7d38a8d36c0686b94b5ad7e2732751268006a61517f731c8fbfa", - "decision": "Reject fixed C23 profile on observed frozen quality violations; no15s or clean timing expansion." -} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c23-native-qualification.json b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c23-native-qualification.json deleted file mode 100644 index c389bb8c2..000000000 --- a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c23-native-qualification.json +++ /dev/null @@ -1,31 +0,0 @@ -{ - "trial": "c23-video-context-lse-v2-native-dc375564", - "tests_passed": 44, - "source_base_head": "d48fa51d03dd748568cbcf57ed40f533c691c9bd", - "files_before": { - "tests/diffusion/distributed/test_collective_gate.py": "6a02f32882d85eca1e3bb9a023ce3adb3d5b59c213845e11e85c96a89432f477", - "tests/diffusion/models/minimax_h3/test_minimax_h3_pre_gate_cache.py": "a4924d0b3dd58ae413259edba6cf2487b3e41d391f943fb74358934be72eb53b", - "tests/diffusion/models/minimax_h3/test_minimax_h3_video_context.py": "1e76074b71f0ea4e070c435db623add702d1414a351427fd86ebb40e8dfd67fd", - "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "e09b872f6900e821d42632a0ca09595f2affe6f9794049dbc74d69ba6a52b2b0", - "vllm_omni/config/omni_config.py": "6894120a330474f718d43bf2bf7928892eb996152839ce6469c34d18a7df09d1", - "vllm_omni/config/stage_config.py": "0752e8cc1c6f1021f737830d08663cfc118923a83fd909c607f9ec3ab2a3d2fb", - "vllm_omni/diffusion/attention/layer.py": "76c5fce3dabd23e39a8890316e1691c03bc203066cb60af5611ec4790eefc632", - "vllm_omni/diffusion/data.py": "5db83d5c71cb27cc7810d35fcfc891da80f0b003adbb16bdecc3816a63305e11", - "vllm_omni/diffusion/distributed/collective_gate.py": "01c6317cfa615840681b8bb8b8513fa3736d26e845a24ad9e1ab7f87489b962f", - "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "48533c941a50719995206d18fa9198330824b940ecba9399b46ba7615b969ccb", - "vllm_omni/diffusion/models/minimax_h3/attention/video_context.py": "f4cfd2225514b6f99d36567879f06fb4a42f9d235ac0114fc643001b345cb22e", - "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "c5fab405ee82bc553e9a902fa27e6dfbc5213ec18311e17b6415396a69b50f43", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "73c96090e6fb0e13ce734bc96b23472fd5adb83bceb71f3eba2815d6343a33e4", - "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "577e290c2d92fc02822cd516cc499c3c4d1f465145ad2a837edfdfeaa09ed9fb", - "vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py": "e8e2e74fb8edb05e02d3e2328ea1f3ad5ddcfd7a10ed09c5cf3911b1d063e849", - "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "cff17568fbd856c28394d104d393a806931613b7b057f288006ad8c6fb9aa05a", - "vllm_omni/diffusion/offloader/base.py": "23922136deaf54e2d98e827b007b34c7e3ad36171b71bfe9060d390efc0b679f", - "vllm_omni/diffusion/offloader/distributed_layerwise_backend.py": "19fbce0f1036e4784d68cab798c499526a9099101909e41ce96dd0caa033b6a4", - "vllm_omni/engine/arg_utils.py": "52569e3688cc708729a48ca0a2c875c28193733c2f934f6ff89f6c815e4009c7", - "vllm_omni/entrypoints/cli/serve.py": "f8284ad6b751ba4a9e4f7254d78132cc2cfd49871a0f9af7fde49c6789d79cb0" - }, - "cleanup_passed": true, - "source_unchanged": true, - "required_native_test_executed": true, - "scope": "Legacy/pre-gate/DLO tests, three executed native four-rank HCCL/subsetFA suites and fresh current gates; no full-model quality/performance acceptance" -} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c23-retired-proof.json b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c23-retired-proof.json deleted file mode 100644 index c80ca79fd..000000000 --- a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c23-retired-proof.json +++ /dev/null @@ -1,31 +0,0 @@ -{ - "checked_at": "2026-11-02T15:48:59.537140+08:00", - "selected_logical_devices": [ - 2, - 3, - 4, - 5 - ], - "selected_devices_idle": true, - "ports_free": { - "19193": true, - "19194": true, - "29597": true - }, - "source_head": "6eb7bb094effb40853431208d1631effaf452a3f", - "source_clean": true, - "tested_files_unchanged": true, - "helper_and_validators": { - "scripts/h3_diagnostics.py": "514d1c735f204693be2af7e4f4fbbba064e2655a59b3c59ea62c35755802fae7", - "validator/strict_compare.py": "b83f2bff553ff6ca2970a16e25525cac5d9523612e9c05ea563852c814019028", - "quality-tools/score_media.py": "99f5a76436a3c8d37462fab83f63d24eae74b4458c30e5a00b60c19fde139f3e", - "quality-tools/compare_quality.py": "99e8e23463f8ce83b52a22de9cdc697ef4bbf14d910a0c67e76bfd01e94c4b3d" - }, - "runtime_diagnostic_version": 17, - "charged_direction_families": 24, - "direction_ceiling": 24, - "deadline": "2026-11-03T01:19:00+08:00", - "candidate_quality_verdict": "FAIL", - "accepted_50pct_http_gain": false, - "scope": "Retire rejected C23 fixed profile; preserve evidence and original budget. No new search direction launched." -} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c4-pfa-operator.json b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c4-pfa-operator.json deleted file mode 100644 index 4994b1ceb..000000000 --- a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c4-pfa-operator.json +++ /dev/null @@ -1,71 +0,0 @@ -{ - "scope": "Single real5s first-long-attention fixture,one logical device; not model performance", - "cases": [ - { - "name": "pfa_bnsd_prefix", - "bit_exact": false, - "unequal_elements": 1577651, - "compared_elements": 71269632, - "finite": true, - "reference_sha256": "7cabc1030b41d2e42c77aac352c20a2b676e194d1f9883b54d5c9b8f72b38c21", - "candidate_sha256": "8487659367fd0776592f548aa9142d5a9631ab4b8c2877df692de76f1166c23c", - "relative_l2": 0.0005370235303416848, - "maximum_absolute_error": 0.03125, - "baseline_ms": [ - 72.55231699673459, - 71.770221053157, - 71.50708901463076, - 71.71859999652952, - 71.54928898671642, - 72.0424129976891, - 71.68136001564562 - ], - "candidate_ms": [ - 85.81408398458734, - 85.6127529987134, - 85.21084999665618, - 85.61692299554124, - 85.85116500034928, - 84.94565699947998, - 85.60653298627585 - ], - "baseline_median_ms": 71.71859999652952, - "candidate_median_ms": 85.6127529987134, - "reduction_pct": -19.37315145981129 - }, - { - "name": "pfa_bnsd_head_batches", - "bit_exact": false, - "unequal_elements": 1577651, - "compared_elements": 71269632, - "finite": true, - "reference_sha256": "7cabc1030b41d2e42c77aac352c20a2b676e194d1f9883b54d5c9b8f72b38c21", - "candidate_sha256": "8487659367fd0776592f548aa9142d5a9631ab4b8c2877df692de76f1166c23c", - "relative_l2": 0.0005370235303416848, - "maximum_absolute_error": 0.03125, - "baseline_ms": [ - 72.57743796799332, - 71.2658169795759, - 71.57570996787399, - 71.21048599947244, - 70.94532495830208, - 71.41773804323748, - 71.54709001770243 - ], - "candidate_ms": [ - 86.40253898920491, - 85.96450602635741, - 86.10950701404363, - 86.288328981027, - 85.96854598727077, - 85.93391597969458, - 86.74239204265177 - ], - "baseline_median_ms": 71.41773804323748, - "candidate_median_ms": 86.10950701404363, - "reduction_pct": -20.57159659958918 - } - ], - "campaign_verdict": "INCONCLUSIVE", - "decision": "Both nativePFA layouts are non-bit-exact and slower by19.4/20.6%; reject,do not integrate." -} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/results/core-integrity.json b/evidence/h3-ref2va-910c-4logical-20261101-297/results/core-integrity.json new file mode 100644 index 000000000..58eae77e2 --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/results/core-integrity.json @@ -0,0 +1,26 @@ +{ + "README.md": "d324b1c1038dddd466f70aacc94341f3df6b44242081a4d084e3be6ca08ef75b", + "patch/README.md": "9d6568ff26f174fbc8d2685c2cd5316ad7338ec928187e42cbfe335756a06a4c", + "patch/c1-h3-bf16-prefix.patch": "afecd661c342860a563ad7c45af81560f16d4ccf10ab71f5ca3b33cdb2691d00", + "patch/c1-source-identity.json": "c6b90d9e91040a96c8761af2823110134593c5aa883847ef7c91594cd8025dec", + "plan/README.md": "64367801c7e8681bdd600c0c6e840dddf45a5a47a1404479be97e62747c4d8e9", + "plan/archive-projection.json": "a461ab6386cfea8db2e197a23c1237b38087e33111758fcc57e6363354e9749a", + "plan/c11-modality-sigma-residual.json": "4a52e72c1b48f4bbd09be742148720c64f528bb09068dcb0c5f4b50930621eb2", + "plan/c11-sigma-fp32-residual.json": "7a3104c1b23c3a89d97ec2a398ad703e31f050f41fd80efe4a0f46e209493c0f", + "plan/execution-checkpoint.json": "8decf6192d1e99d6665efff3af375909ac8f223ac6dc81a0d4077ca43c920416", + "plan/frozen-lossless-contract-v1.json": "b3ec4b2f58d1f9de4475977ed4bbdd75e4d38cad3857244fe0cfd842f7469cab", + "plan/frozen-lossy-contract-v1.json": "b8fd9e634e6db5ebfbce3842f3360f1b2241a08119953ee71bfc26553dbbafed", + "plan/native-cli-910c-device-map.json": "bfe453f783c51b747920255113fbc7abbf4f6a1523e10b708e703ca1448e8eb3", + "plan/requester-decisions.json": "e62c5398fb1a5cabf4c51e2609d890d1d68b15a84fb00d08167e7ccefe341ccb", + "results/artifacts.json": "d248b9fb370388bd4851909be43aa4a7646c6a4f9ddb0c680d2325a1899622ce", + "results/c1-15s-strict-comparison.json": "72b196500c0f045eb78c4e994279189c36b59c48f103e0ab4f94ff51968c62be", + "results/c1-5s-strict-comparison.json": "f333a0cf8d65d59e92d67a3882608a344a00f57542ab5a22e832c6800d81a4cb", + "results/summary.json": "e1cce3bc45741c1d3fd7d487973998523927f2dfe38b367fc04dc4abeb295bd8", + "results/trials.jsonl": "188460119baf8899a9f8e0bb4d05f9c7631b6ed50fc7ab002b4790ed11e2209c", + "validator/README.md": "9cad15840f65dbf3053dc0d53ef36a392f1b94b29b7b983c08a5c11aeda89226", + "validator/check_media.py": "617c79d3c55dd74b5ccd2a10ba51f70241e36fe95be28a08e3f714fddcdbfa38", + "validator/frozen-snapshots.json": "f52b54bc0a6eb872991611af1368793d25631797389d09380857b2ef228ca283", + "validator/quality-tools/compare_quality.py": "99e8e23463f8ce83b52a22de9cdc697ef4bbf14d910a0c67e76bfd01e94c4b3d", + "validator/quality-tools/score_media.py": "99f5a76436a3c8d37462fab83f63d24eae74b4458c30e5a00b60c19fde139f3e", + "validator/strict_compare.py": "b83f2bff553ff6ca2970a16e25525cac5d9523612e9c05ea563852c814019028" +} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/results/installed-foca-source-check.json b/evidence/h3-ref2va-910c-4logical-20261101-297/results/installed-foca-source-check.json deleted file mode 100644 index 58af866da..000000000 --- a/evidence/h3-ref2va-910c-4logical-20261101-297/results/installed-foca-source-check.json +++ /dev/null @@ -1,22 +0,0 @@ -{ - "verified_utc": "2026-11-01T18:28:58Z", - "module": "cache_dit/caching/cache_contexts/calibrators/foca.py", - "sha256": "258119cffacf83da942d278a29e9a40a1b1f3ffbbf30359bda41bc4af0b0a804", - "bytes": 792, - "distribution_metadata": [ - { - "distribution_directory": "cache_dit-1.5.0.dist-info", - "fields": [ - "Name: cache_dit", - "Version: 1.5.0" - ] - } - ], - "not_implemented_methods": [ - "reset_cache", - "approximate", - "mark_step_begin", - "update" - ], - "scope": "static source check only; not an available H3 configuration or a device experiment" -} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/results/mindie-quant-api-source-check.json b/evidence/h3-ref2va-910c-4logical-20261101-297/results/mindie-quant-api-source-check.json deleted file mode 100644 index a7320bf69..000000000 --- a/evidence/h3-ref2va-910c-4logical-20261101-297/results/mindie-quant-api-source-check.json +++ /dev/null @@ -1,10 +0,0 @@ -{ - "installed_label": "3.1.0", - "installed_source_revision": "UNKNOWN", - "installed_quant_attention_export_and_file": false, - "private_source_commit": "ee3920f9cd61030b90e5dcb1a0a432e52810251d", - "public_api_introduction_commit": "76f8761794a0837f4fae3b5efbe52ccdd9847816", - "public_api_introduction_date": "2026-09-14", - "source": "https://github.com/Ascend/MindIE-SD/commit/76f8761794a0837f4fae3b5efbe52ccdd9847816", - "decision": "The API exists in dev; a version label alone does not establish wheel contents or 910C native support." -} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/results/native-cli-doctor.json b/evidence/h3-ref2va-910c-4logical-20261101-297/results/native-cli-doctor.json deleted file mode 100644 index 29e669a44..000000000 --- a/evidence/h3-ref2va-910c-4logical-20261101-297/results/native-cli-doctor.json +++ /dev/null @@ -1,15 +0,0 @@ -{ - "doctor_passed": true, - "native_cli_returncode": 0, - "parsed_record_count": 16, - "unique_physical_npu_count": 8, - "requested_physical_npu_ids": [ - "1", - "2" - ], - "execution_visible_logical_ids": "2,3,4,5", - "native_inventory_unchanged": true, - "compatibility": "unverified", - "first_wrapper_failure": "Own check incorrectly expected8rawrecords; nativeCLIitselfreturned0. DuplicatedphysicalIDsare retained in the raw16chiprows.", - "scope": "CLI discovery/targetPythonmetadata only; no pairedbenchmark or backendcompatibility acceptance" -} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/results/publication-integrity.json b/evidence/h3-ref2va-910c-4logical-20261101-297/results/publication-integrity.json deleted file mode 100644 index 337b74bbf..000000000 --- a/evidence/h3-ref2va-910c-4logical-20261101-297/results/publication-integrity.json +++ /dev/null @@ -1,97 +0,0 @@ -{ - "patch/c1-h3-bf16-prefix.patch": "afecd661c342860a563ad7c45af81560f16d4ccf10ab71f5ca3b33cdb2691d00", - "patch/c1-source-identity.json": "c6b90d9e91040a96c8761af2823110134593c5aa883847ef7c91594cd8025dec", - "patch/c10-h3-audio-cache-guard.patch": "626cd6267e7eec35efabb53a67d281303b05dc1f978188085521a35fbeba02fc", - "patch/c10-source-identity.json": "12bc7d5c39482cae653c8db9a43766cb9b63662565172562a84e057a5364425d", - "patch/c11-fp32-source-identity.json": "ea6fbce617f372dae895b08f6bfa35012ba4835c0e747762ba20b693a2a108f7", - "patch/c11-modality-sigma-residual.patch": "b2458c0dad78e479d17887cd86be7800181fbbe4751f4426d17587e41046ddea", - "patch/c11-sigma-fp32-residual.patch": "f8308af2c3f0cccc3b8c48f0a3d584dd6fd68ee4c8c87c0bc3fb8f67e239bd95", - "patch/c11-source-identity.json": "b8475b5e02073e4dfccf893d852d4139efdad10b5505b620e4782ede4b94032d", - "patch/c12-candidate.patch": "cb54ff0e336903290a9af8a48a6f06c0893747c7ba2be763886a9100f8cbe9d8", - "patch/c12-source-identity.json": "da6a05d52f3dfdff204d77e34ef3448f38f21cfe8496fd08c9be2a7e88f2e6fe", - "patch/c13-candidate.patch": "79c836a3919b9bcf7d2d01f3c1d496a193bf72484c01d4284daaa967b622b3e2", - "patch/c13-source-identity.json": "6ecfb0b0f9e5f3279e71b0c28ebdbbd46ab75b45ea965e717328042963cbba0e", - "patch/c15-candidate.patch": "6b98e4dd9938ed387c5d07180470233f5172c87fdd6c5636bd9a7b5141ebf7d5", - "patch/c15-source-identity.json": "8ac49340c964e3925542fc7c192024095d6983ab3508c3006bf6b192bc6b9557", - "patch/c16-candidate.patch": "6d9b2d42d43f47bc0185ffbdd6aca79405664b10f62652733ea74c2b4c07eee7", - "patch/c16-source-identity.json": "4884d8cfc3799651a96d84a315a64040fb90eeb7975168f13119ee84d00ce217", - "patch/c17-candidate.patch": "1fcfc4d1fdae4b6492ccd6556cb61e1f361ebba33b0c7defeab922d0797f6f16", - "patch/c17-source-identity.json": "bf663a65aad387fe6c962284a7d95de063295827981c63e2d1234399448bfa47", - "patch/c18-candidate.patch": "c338d2efe83ab6c6508849a1abe1df0f11d55c7d28753835a45dc996472d7daa", - "patch/c18-source-identity.json": "56c70cc3d09bc7c4478b67e6e629b3bf6987de3ad96941a23f0a110075c5836b", - "patch/c19-candidate.patch": "5a4b0e2083bbbd1a74eb12e480d2d18aff600b7baff7b41f43034522b25cc328", - "patch/c19-source-identity.json": "0d52e081037ab827ddebefe20b1517845edaaa498b2b6308bdd4fe3176dbd71e", - "patch/c20-candidate.patch": "da86f4c6ebbf5005f6daa8c8ecadcb77cbfa4f0c7cf39cb096fea4dd427a2a4d", - "patch/c20-source-identity.json": "85e44ab21bbb463f35e6211b472d8405ed278302ed3cd0c3edb8d17aa712671d", - "patch/c21-candidate.patch": "fe6bce768eb90fcd8f5b5f49327c2ea6ee3f2c6078dcffd2f58d3934ec74a6c6", - "patch/c21-source-identity.json": "0cdca3f7764c05abdcd4b7cf2fb3349fd12a82da927baf479c59bab0b049a25c", - "patch/c22-candidate.patch": "c4e2988f91057f896a0ac297a507dc8fc7853eaf5da112d1a066aa4c344e9967", - "patch/c22-source-identity.json": "b44d3b8d6e285618831b1e25101d49ad16188bb08e86559595c33c70c7eeedea", - "patch/c23-candidate.patch": "cd1db1acacf5fe41afedb775ec618f369d4960d7deda9478eaffd51752a6e7c4", - "patch/c23-source-identity.json": "c85034206b2e544efd455249ab2add7f41ea41bc27241f36f605053fe0493f96", - "patch/README.md": "477741c1a4b68b1d702e96c03768db45ededf2ae4e6e35198edb4f971f78d23d", - "plan/c11-modality-sigma-residual.json": "4a52e72c1b48f4bbd09be742148720c64f528bb09068dcb0c5f4b50930621eb2", - "plan/c11-sigma-fp32-residual.json": "7a3104c1b23c3a89d97ec2a398ad703e31f050f41fd80efe4a0f46e209493c0f", - "plan/execution-checkpoint.json": "8decf6192d1e99d6665efff3af375909ac8f223ac6dc81a0d4077ca43c920416", - "plan/frozen-lossless-contract-v1.json": "b3ec4b2f58d1f9de4475977ed4bbdd75e4d38cad3857244fe0cfd842f7469cab", - "plan/frozen-lossy-contract-v1.json": "b8fd9e634e6db5ebfbce3842f3360f1b2241a08119953ee71bfc26553dbbafed", - "plan/native-cli-910c-device-map.json": "bfe453f783c51b747920255113fbc7abbf4f6a1523e10b708e703ca1448e8eb3", - "plan/README.md": "930282bf6d990c86930d8a11595dcdbb33995fd1c927060215507830ca5ba00e", - "plan/reproduction.md": "02e769c6e053b5fd39bb20571c3c269894783d4a071dd1eaa522cef1b306e28b", - "plan/requester-decisions.json": "e62c5398fb1a5cabf4c51e2609d890d1d68b15a84fb00d08167e7ccefe341ccb", - "README.md": "d32ccc8a727e6587a54c002d8234033f00cd773aff0687009335a100f1b53bb4", - "results/automatic-quality-checkpoint.json": "19f896ca61ad692082ac8c714b8aaa05f85ae96e525c25cfa1696fc59f22596e", - "results/c1-15s-execution.json": "ba790d7d9e352fd27848104e235a82dad25e758b2801578e0576eaf93417c8b0", - "results/c1-15s-strict-comparison.json": "72b196500c0f045eb78c4e994279189c36b59c48f103e0ab4f94ff51968c62be", - "results/c1-5s-strict-comparison.json": "f333a0cf8d65d59e92d67a3882608a344a00f57542ab5a22e832c6800d81a4cb", - "results/c1-clean-5s-screening.json": "11bf3c1c71fc78d1c83c62e5244cf02f14d2d5eebefb18dbf85477211070d277", - "results/c1-queryfold-operator.json": "273cdcab38055d28925c1209f5b3b8ea59264a674b089abee586ba005d8cdb5b", - "results/c1-upstream-main-native.json": "3b9e08044b52b0b7e477225d7ab6fecfc4db50e7411cf974a83eb3b0bf129b09", - "results/c10-5s-00-audit.json": "9248fac9a87d76e01d9eb512a35dce7794496f6baa5598643bc3c8a3f14acde1", - "results/c10-5s-01-audit.json": "e96f01f455f42363deb3921be5332bc60694148e5ae12915f351713c16b7b3fd", - "results/c10-native-tests.json": "023c10a332567bf98507fd9ae91acee1cfb498aa43896f5f58e9edc17ab016cd", - "results/c11-fp32-15s-00-audit.json": "d18b0a6cb1492f815747eb4c8bce664db9062527676f8ac8c999e6d479e95da8", - "results/c11-fp32-15s-first-step.json": "f8d5d696dcbf3b725b132188325644688adc8ff1b0c95267585ef3a31bfe4b11", - "results/c11-fp32-5s-00-audit.json": "c1062b404ad9c82430aecac4fcad42fc7bbffc4e38ca33ca470dfadacb3ff804", - "results/c11-fp32-5s-first-step.json": "706cc16402428315e0d8a3d1312f9d40b02250a56c606321c27374de707a6e0c", - "results/c11-fp32-native-execution.json": "74bd1021b2fcadf00ac5a72503dc5d1ad8e388419faf58b5693376c0d8018567", - "results/c11-full-compute-first-step.json": "10f847c6eb4fc446853708148596445e2340129dd5acfebf8d18b32966f889ab", - "results/c11-native-execution.json": "e226442b31c54acbd8a1dcc1921598ae353b7839fd9e9cee771414d748165562", - "results/c11-original-5s-00-audit.json": "1f8d96141fa286655a3c6224853fd59408a7847fbf1eb16e2b299168f8637973", - "results/c12-through-c15-execution-checkpoint.json": "ceb8db8f9dbb4154db8b3b2ec3ac62c02e7dc8b0e59afdf581b12e3241b58d4d", - "results/c12-through-c17-execution-checkpoint.json": "c9ecb3988ed19ff2abfec58c87b46b69f86eda7571c35ce9c85dfc04505cd3cf", - "results/c12-through-c19-execution-checkpoint.json": "db77306a076f5118df38106c3971d0217f92cf369f7ca8905a5552d47806a71b", - "results/c12-through-c20-execution-checkpoint.json": "124cdd33a46aad0d81a32dc6c0f26d14315ea62d0387af9f045888b68ecf28ca", - "results/c12-through-c21-execution-checkpoint.json": "9f4d5d4a091dfedc24addc36bc0e0f2a6a71486c1ad8bfcd3a7ad84a2ad1f0f3", - "results/c12-through-c22-execution-checkpoint.json": "37edc3c7503a175b2be2d410c199f23e9bbde00fa88b6cc970199fdab8662bf0", - "results/c12-through-c23-execution-checkpoint.json": "f87cc234efc10adb703d01124c76d2076b46d5251d142ad178b3dc6590079f6b", - "results/c13-5s-strict-comparison.json": "563d1ac0411ed3fa78b78556f521f11bfc510fb1fdef67291d09b9b9e2fa3884", - "results/c15-native-qualification.json": "148f26ac90d2fc151f94efc819f4463da1d3afc2cfc48337eb35ebd14d2a59fa", - "results/c16-5s-00-execution-proof.json": "3d113052ae2584fa22cb6a5e1b566715e22c57a89453469e8a96eb4dbd630b40", - "results/c16-native-qualification.json": "70291bdde95ca6edb2bdf5b33e0612c4b6f0605529c02083ff97efe6adccbac0", - "results/c17-5s-00-execution-proof.json": "2fb3367b8baff791c8be3a5ad6bd64a08631074828862f176353b7147a859488", - "results/c17-native-qualification.json": "e109f84ff1c718a339cbc42e2bf3eb6b5a0381be21ac761bf20f7ecd8d4f4ee5", - "results/c18-native-qualification.json": "dfd17be0be798c1586f9aa343a55aa441bde667254ad77571762904476af4216", - "results/c19-native-qualification.json": "946eaec18d2104c1c12cd468d8abec3f0b6a36a7f84635300d8c44db268de262", - "results/c20-native-qualification.json": "23f1ff63ff080dbc11ea44ce5e4abb6e8eeb27aa6555f612fbcf744f5e4b2403", - "results/c21-5s-00-execution-proof.json": "a1295ce47e68ec2abb623dc3ab34b2ceb9c583a00bf79a11d54c10d4c37b1b44", - "results/c21-c22-direction-identity-correction.json": "42fbf61b4cd487fffc55361d53f3399f51ea00d75a4232755bddd5e07f4f0c12", - "results/c21-native-qualification.json": "280d086e2f3d04fc9e86bd92eee9c15df65298349a97260d3d2bde9d920469a4", - "results/c22-5s-00-execution-proof.json": "4f3044a8892d33a5d62f597dc6210149195b1e5416924cd6078d821779066f93", - "results/c22-native-qualification.json": "d259bbde6732f8a3b560eee64af417335afe46a7cf1d78cbd8d4d45b446e5ee0", - "results/c23-5s-00-execution-proof.json": "02ceb72f9c125b842bf89a6654e24816da496c5487498e6115462b37f961c72a", - "results/c23-native-qualification.json": "f5538b528d9b18b3a11f7bdb3a7c8e3d94a7ed2876a35f44c647030149a34850", - "results/c23-retired-proof.json": "be82f9834955b02739cc08f1576345c74ba89431faa4f733ab5d9bbfe4769e8a", - "results/c4-pfa-operator.json": "37c4e36ba947da1054846b07648bcbd11adbabe6bb0c9357928bf6b785befda2", - "results/installed-foca-source-check.json": "bd2ac184c76d9bf9b1122e2ea2179ad6b51f5be4996e16813a0b9a50951cfdc6", - "results/mindie-quant-api-source-check.json": "4b26ec103344405b3fca2ceed6ed2a7d86831abf765ecadf5171b1277b939123", - "results/native-cli-doctor.json": "ac99ef541ce0febd6b018210de333e38f459c48c3e949c45f66f977c11512499", - "results/README.md": "07fbbe5dff05c1bd0499077cbbdebab1516c9ad7ddd7b3ddfb193555769890de", - "results/report.md": "3d46aba9dbf938bb3ae34eee3e972afc952c8e1e7e6bcacaad49516259d784f8", - "validator/check_media.py": "617c79d3c55dd74b5ccd2a10ba51f70241e36fe95be28a08e3f714fddcdbfa38", - "validator/frozen-snapshots.json": "f52b54bc0a6eb872991611af1368793d25631797389d09380857b2ef228ca283", - "validator/quality-tools/compare_quality.py": "99e8e23463f8ce83b52a22de9cdc697ef4bbf14d910a0c67e76bfd01e94c4b3d", - "validator/quality-tools/score_media.py": "99f5a76436a3c8d37462fab83f63d24eae74b4458c30e5a00b60c19fde139f3e", - "validator/README.md": "7588ed18fc68276f0f4f3d7603ea45779fbc3f61b87595d3d35719b084efe135", - "validator/strict_compare.py": "b83f2bff553ff6ca2970a16e25525cac5d9523612e9c05ea563852c814019028" -} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/results/report.md b/evidence/h3-ref2va-910c-4logical-20261101-297/results/report.md deleted file mode 100644 index e5aa46019..000000000 --- a/evidence/h3-ref2va-910c-4logical-20261101-297/results/report.md +++ /dev/null @@ -1,146 +0,0 @@ -# MiniMax-H3 Ref2VA on four logical Ascend 910C devices - -Evidence cutoff: 2026-11-02 15:52 +08. Governing acceptance: [#297](https://github.com/InferMatrixLab/InferRSI/issues/297). -**Campaign verdict: INCONCLUSIVE. No accepted 50% complete-HTTP latency reduction.** -This is a tracking snapshot of completed, failed and incomplete work. - -## Workload and identities - -| Item | Tested scope | -| --- | --- | -| Hardware | Four logical 910C devices, two dual-die physical NPUs, 64 GiB per logical device | -| Model | Original MiniMax-H3 Ref2VA snapshot `6818f6c32d12b210915e44ad56a4228c2608f160` | -| Inputs | 1344×768, 24 FPS, 124/362 frames for 5/15 s, 50 solver steps | -| Reference | Original BF16 dense B0 `df1c1b05936381dbaa3891c1ae4c4b21ab375a5d` | -| Migration source | vLLM-Omni snapshot `922f1e9f1cb85b9f2b66d86e1eb500e6872b2dcb`; separate from original B0 | -| Execution harness | InferRSI `50c19da43e0b358e6f990c00719891ad308a8ac1` | -| Stack | Private Ubuntu 22.04 namespace, GLIBC 2.35, CANN 9.1, Torch 2.10.0+cpu, TorchNPU 2.10.0.post4, MindIE-SD 3.1.0, Cache-DiT 1.5.0 | -| Parallelism | USP4/ring1, TE TP4, tiled VAE parallel4, DLO enabled in the reference; exceptions recorded per candidate | -| Quality | Frozen strict bit-exact contract and automatic lossy gates; no human media inspection | - -The MindIE wheel's source revision is UNKNOWN. Model bytes were SHA/LFS checked -privately; weights, input image, raw media, tensors and host logs are external. -This campaign is separate from the eight-device 910B1 evidence in -[PR #294](https://github.com/InferMatrixLab/InferRSI/pull/294). Its results do not transfer here. - -## Four-track target status - -| Track | Numerical or model-quality evidence | Clean complete-HTTP evidence | Target status | -| --- | --- | --- | --- | -| 5 s lossless | C1 direct-B0 strict comparison: 439 PASS, zero failures/missing; C13 also 439 PASS on the fixed request | One C1 fresh-server screening pair: 291.247763→286.666760 s, 1.572888689% reduction | Performance FAIL at the 50% gate; final confirmation incomplete | -| 15 s lossless | C1 direct-B0 strict comparison: 439 PASS, zero failures/missing on the fixed request | Not run | INCONCLUSIVE | -| 5 s lossy | Tested fast profiles have observed frozen quality failures; no selected complete quality cohort | No qualified candidate's clean pair | INCONCLUSIVE overall; rejected configurations remain FAIL | -| 15 s lossy | Tested Taylor/C11 configurations have independent visual/audio failures at this duration | No qualified candidate's clean pair | INCONCLUSIVE overall; rejected configurations remain FAIL | - -Strict qualification covers fixed requests, not a universal bit-exact guarantee. -The original B0 repeated exactly under the contract at each duration (219 checks -each). The default dense migration's 15 s qualification is separate and incomplete. - -## Clean screening and timing boundaries - -[C1 clean screening](c1-clean-5s-screening.json) used one fresh server per arm, -two same-shape warmups and three measured complete MP4 responses. Audit and -profiling were disabled. Source integrity, cleanup and measured RGB24/PCM checks -passed; the paired strict check had 35 PASS, with the separate audit's 439 PASS. - -The measured delta compares original B0 with migration+C1. There is no isolated -migration-only clean arm. Only one screening pair completed; statistical -significance and final acceptance are not claimed. The frozen final protocol -requires three fresh pairs in AB/BA/AB order for each track/duration, two warmups -and three measurements per service, and at least 50% reduction in every pair -and the aggregate. No final-confirmation pair completed. - -Audited request times below use zero warmups and one request unless stated. -They are diagnostic observations, not accepted speedups against this clean B0. - -## Direction ledger - -This index summarizes carried device evidence and points to the published -projections. Full private executor archives remain external. A native-test PASS -or primitive-gate PASS does not establish model quality or service performance. - -| Direction | Mechanism and origin | Observed disposition | -| --- | --- | --- | -| C0 | Pinned migration and profiling | Fixed 5 s strict PASS; a two-step trace attributed 7148.150 of 11332.154 ms to FA union. This is not the full HTTP fraction. Raw trace is external. | -| C1 | H3-owned BF16 attention layout and valid-KV prefix | Fixed 5/15 s strict PASS; clean 5 s reduction 1.5729%, below target. Query-fold variants preserved fixture bits but were slower. | -| C2 | HSDP2/replicate2 topology | Four-step bit-exact smoke; cold request about 85.649 s. No complete 50-step performance qualification. | -| C3 | Cache-DiT Taylor2/Taylor3 profiles | Frozen audio or visual gates failed across independent samples/durations; a first passing clip did not qualify a profile. | -| C4 | Native dense FAS/FIA/FIA2 alternatives | FIA2 fixture improvement about 5.4%/5.0% at 5/15 s, non-bit-exact; PFA and query/layout alternatives failed their useful-gain tests. | -| C5 | MindIE Laser for video queries, BF16 conditioning queries | Complete 5 s diagnostic 279.751 s; frozen audio-caption adherence failed. | -| C6 | Laser input-range scaling | Tested scales did not materially improve about 0.002713 relative L2; stopped. | -| C7 | Native RainFusion BF16 | Actual leaf execution observed; fixed sparsity profiles failed frozen quality checks. | -| C8 | Reference-KV device residency and refresh2 | Complete request failed CLAP; creation alone did not prove reuse counts in the older diagnostic. | -| C9 | TP4 with resident DiT | Complete 5 s diagnostic 327.206 s; audio-caption adherence failed. An earlier offload attempt failed before step0. | -| C10 | Local audio modality-veto guard | Native 15 PASS; independent 5 s clips exposed different audio failures. Fixed variants rejected. | -| C11 | Local dual-sigma residual forecasting, including FP32 anchors | Native original 20/FP32 revision 23 PASS. Fixed 5 s and independent 15 s profiles failed frozen quality guards. | -| C12 | Local state-aligned residual prediction | Native 14 PASS; full 5 s executed 50 computes/zero predictions/45 vetoes. No acceleration; cohort incomplete. | -| C13 | Adapted upstream vLLM-Omni PR #9351 ordering to current chunked DLO, over C1 | Focused 10 PASS; expanded309 PASS/1 FAIL, reproduced on the migration baseline. Full 5 s strict 439 PASS; clean timing and 15 s incomplete. | -| C14 | Quantized-attention prerequisites on the actual A3 stack | Block-FP8 and block-INT8 probes failed before FIA2. Per-token INT8 and other SDKs are not disproved. | -| C15 | Local target-video block-residual cache, FP16 storage | Corrected native17 PASS; complete-path attempt failed finite video velocity at first reuse. | -| C16 | FP32 storage repair for C15 | Native 18 PASS; complete 5 s input 134 PASS, but three frozen audio checks failed. | -| C17 | Local three-latent-frame attention window | Native 15 PASS; real-input operation 63.7004% lower. Full-model quality failed 11 checks. | -| C18 | Current-frame detail plus global 3×3 pooled KV | Native 17 PASS; operation 7.2512% lower, below the fixed 50% primitive gate. No full model. | -| C19 | Shared global 3×3 pooled KV | Native 17 PASS; operation 52.8479% lower, but relative L2 0.258805 exceeded the fixed 0.258052 bound. No full model. | -| C20 | C19 plus isotropic key-variance correction | Native 17 PASS; operation 48.4204% lower and relative L2 0.263361; both primitive gates failed. No full model. | -| C21 | Local pre-gate video activation cache | Native 36 PASS; full 5 s diagnostic 163.806 s, input 134 PASS; seven frozen quality failures. | -| C22 | Local sigma-secant pre-gate cache, BF16 anchor/INT8 per-channel delta | Native 45 PASS; full 5 s diagnostic 166.859 s, input 134 PASS; nine frozen quality failures. | -| C23 | Local fresh conditioning attention with cached video-to-video output/LSE | V2 native 44 PASS, static attention cycle 66.4588% lower; full 5 s diagnostic 199.065 s, input 134 PASS; eight frozen quality failures. | - -Primary result indexes: [automatic quality comparisons](automatic-quality-checkpoint.json), -[C12–C23 run projections](c12-through-c23-execution-checkpoint.json), -[C1 query folding](c1-queryfold-operator.json), [C4 PFA](c4-pfa-operator.json), -[C11 15 s audit](c11-fp32-15s-00-audit.json), and -[source identities and patches](../patch/README.md). -The ledger summarizes 24 direction families, not 24 successful candidates. - -## C23: primitive improvement rejected by full-model quality - -Private engine commit `6eb7bb094effb40853431208d1631effaf452a3f` retains all -50 solver steps, fresh QKV/SP/projections/FFN/current gates and original weights. -Each step refreshes nonvideo-to-full and video-to-conditioning attention. -Only video-to-video BF16 attention output and FP32 LSE are reused between full -steps 0,1,4,9,19,29,39,49. This is an intentional approximation for changing Q/KV. - -The first manual Triton merge passed 44 native tests but the full 50-call static -attention cycle improved only 38.2552%, below the prospective 50% gate. Component -diagnosis measured 22.750201 ms for that merge and 2.742732 ms for a native Torch -expression, with all 66,834,432 real BF16 output elements bit-equal. V1 failures -and source hashes are retained in the run projection. - -V2 adopted that measured Torch expression, keeping the algorithm and gates fixed. -It passed 44 focused tests, including three executed four-rank suites. On one -captured real 5 s input, 50 attention calls with 8 refresh/42 reuse, 2 warm cycles and -5 measured cycles in AB/BA/AB/BA/AB order, cycle medians were 3477.472871 and -1166.387435 ms: 66.458762% lower. All 50 validation outputs were finite, max relative -L2 was 0.00201662234 < 0.005, and saved real-merge replay was bit-equal to V1. -This excludes FFN, changing inputs, complete-model timing and HTTP. - -[Full 5s-00 execution](c23-5s-00-execution-proof.json) then completed all 50 steps -with 134 input checks and expected per-rank counters: 400 context refreshes, -2100 reuses, 2500 fresh conditioning-attention calls and 2500 full-row FFN blocks. -Persistent cache peaked at 6787872000 bytes/rank and closed to zero live bytes. -Diagnostic complete HTTP was 199064.918509 ms. The frozen scorer rejected 8 of 39 -checks: DINO/CLIP/CLAP similarity, segmented CLAP, reference-DINO adherence, -audio-caption adherence, audio level and audio envelope. The other 9 missing -samples do not erase these violations. No 15 s or clean timing extension follows. - -## Limits and continuation - -All 24 direction families are charged. The requester extended the original budget -to 32 wall hours/128 logical-device-hours, ending 2026-11-03 01:19+08; preparation, -failures and retries remain charged. The controller-alias repair groups two -duplicate labels under their existing source-bound families, preserving the -raw budget and failed attempts: [audit receipt](c21-c22-direction-identity-correction.json). -No additional final-confirmation budget is authorized. - -[Retirement readback](c23-retired-proof.json) verified selected devices idle, -experiment ports free and tested source/helpers/validators unchanged at -2026-11-02 15:48:59+08. No device job is scheduled by this tracking PR. -Further mechanism search requires a requester extension beyond 24 directions. -Four-track final confirmation, full lossy corpus coverage and independent review -remain incomplete. Goal acceptance belongs to the requester. - -Published files are sanitized field projections and patch snapshots. They do -not contain model/scorer weights, media, raw tensors, full executor archives or -the private namespace/bootstrap environment. [Replay and availability notes](../plan/reproduction.md) -describe what can be verified locally and what still needs external artifacts. diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/results/summary.json b/evidence/h3-ref2va-910c-4logical-20261101-297/results/summary.json new file mode 100644 index 000000000..b740207b3 --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/results/summary.json @@ -0,0 +1,186 @@ +{ + "schema_version": 2, + "source_revision": "6c7e237935a22cc5e90e954d4f397376e753e7d8", + "artifact_id": "h3-ref2va-910c-4logical-20261101-297-6c7e237-v1", + "campaign_verdict": "INCONCLUSIVE", + "coverage": { + "original_files": 96, + "original_bytes": 1259483, + "indexed_result_documents": [ + "results/automatic-quality-checkpoint.json", + "results/c1-15s-execution.json", + "results/c1-15s-strict-comparison.json", + "results/c1-5s-strict-comparison.json", + "results/c1-clean-5s-screening.json", + "results/c1-queryfold-operator.json", + "results/c1-upstream-main-native.json", + "results/c10-5s-00-audit.json", + "results/c10-5s-01-audit.json", + "results/c10-native-tests.json", + "results/c11-fp32-15s-00-audit.json", + "results/c11-fp32-15s-first-step.json", + "results/c11-fp32-5s-00-audit.json", + "results/c11-fp32-5s-first-step.json", + "results/c11-fp32-native-execution.json", + "results/c11-full-compute-first-step.json", + "results/c11-native-execution.json", + "results/c11-original-5s-00-audit.json", + "results/c12-through-c15-execution-checkpoint.json", + "results/c12-through-c17-execution-checkpoint.json", + "results/c12-through-c19-execution-checkpoint.json", + "results/c12-through-c20-execution-checkpoint.json", + "results/c12-through-c21-execution-checkpoint.json", + "results/c12-through-c22-execution-checkpoint.json", + "results/c12-through-c23-execution-checkpoint.json", + "results/c13-5s-strict-comparison.json", + "results/c15-native-qualification.json", + "results/c16-5s-00-execution-proof.json", + "results/c16-native-qualification.json", + "results/c17-5s-00-execution-proof.json", + "results/c17-native-qualification.json", + "results/c18-native-qualification.json", + "results/c19-native-qualification.json", + "results/c20-native-qualification.json", + "results/c21-5s-00-execution-proof.json", + "results/c21-c22-direction-identity-correction.json", + "results/c21-native-qualification.json", + "results/c22-5s-00-execution-proof.json", + "results/c22-native-qualification.json", + "results/c23-5s-00-execution-proof.json", + "results/c23-native-qualification.json", + "results/c23-retired-proof.json", + "results/c4-pfa-operator.json", + "results/installed-foca-source-check.json", + "results/mindie-quant-api-source-check.json", + "results/native-cli-doctor.json" + ], + "index_entries": 95, + "named_attempt_identities": 54, + "record_occurrences": 248, + "status_counts": { + "completed": 36, + "failed": 16, + "recorded": 43 + }, + "unit": "published record/attempt observation; not a count of independent trials" + }, + "recomputed_metrics": { + "c1-clean-5s-baseline": { + "source": { + "name": "c1-clean-5s-baseline", + "path": "results/c1-clean-5s-screening.json", + "pointer": "/arms/baseline/samples_ms", + "unit": "ms", + "scope": "One fresh-server screening pair, 2 warmups and 3 timed requests per arm; not final" + }, + "distribution": { + "count": 3, + "min": 291051.2047139928, + "max": 291434.0972539503, + "median": 291247.76331102476, + "mean": 291244.3550929893, + "stdev": 191.46902164950015 + } + }, + "c1-clean-5s-candidate": { + "source": { + "name": "c1-clean-5s-candidate", + "path": "results/c1-clean-5s-screening.json", + "pointer": "/arms/candidate/samples_ms", + "unit": "ms", + "scope": "One fresh-server screening pair, 2 warmups and 3 timed requests per arm; not final" + }, + "distribution": { + "count": 3, + "min": 286197.25764304167, + "max": 286812.4624299817, + "median": 286666.7601839872, + "mean": 286558.82675233687, + "stdev": 321.49098232974296 + } + } + }, + "recorded_results": [ + { + "source": { + "path": "plan/execution-checkpoint.json", + "pointer": "/acceptance" + }, + "value": { + "complete_http_reduction_pct": 50, + "per_duration_and_quality_track": true, + "clean_paired_runs_completed": 1, + "final_confirmation_pairs_completed": 0, + "human_output_checks_in_rsi": false + } + }, + { + "source": { + "path": "results/c1-clean-5s-screening.json", + "pointer": "/native_verdict" + }, + "value": "FAIL" + }, + { + "source": { + "path": "results/c1-clean-5s-screening.json", + "pointer": "/gain_attribution_limit" + }, + "value": "Original B0 versus migration+C1; no clean migration-only timing arm in this pair" + }, + { + "source": { + "path": "results/c23-5s-00-execution-proof.json", + "pointer": "/quality_verdict" + }, + "value": "FAIL" + }, + { + "source": { + "path": "results/c23-5s-00-execution-proof.json", + "pointer": "/failures" + }, + "value": [ + "5s-00:dino: similarity", + "5s-00:clip: similarity", + "5s-00:clap: similarity", + "5s-00: clap segment", + "5s-00:reference_dino_cosine: adherence", + "5s-00:audio_caption_clap_cosine: adherence", + "5s-00: audio level", + "5s-00: audio temporal envelope" + ] + }, + { + "source": { + "path": "results/c12-through-c23-execution-checkpoint.json", + "pointer": "/decisions" + }, + "value": [ + "C12 fixed profile stopped: zero actual predictions.", + "C14 pinned block-FP8 and block-INT8 prerequisites failed; per-token INT8 is not excluded.", + "C13 numerical qualification and clean performance are separate.", + "C15 fixed FP16 video residual profile failed finite velocity at first reuse.", + "C16 FP32 residual completed but failed three frozen audio quality guards; fixed profile stopped.", + "C17 completed5s but failed11frozen quality checks; fixed window profile stopped.", + "C18 native17PASS but operation reduced only7.25%; no full model run.", + "C19 native17PASS; operation52.85%faster but relL2 .258805 exceeded fixed .258052screen; no model run.", + "C20 native17PASS; operation48.42%faster and relL2 .263361 failed both fixed primitive gates; no model run.", + "C21 native36PASS; full5s completed but failed7frozen quality checks; fixed zero-order profile stopped.", + "C22 native45PASS; full5s completed but failed9frozen quality checks; fixed secant profile stopped.", + "C23 V1 native44PASS; weighted attention cycle reduced38.255%, below unchanged50%operator gate; no model run for V1.", + "C23 component diagnosis found a bit-equal native Torch merge faster than manual Triton on real tensors; V2 native44PASS and weighted operator66.459%reduction passed, full-model quality and HTTP remain separate.", + "C23 full5s completed with134input checks and expected counters but failed8frozen quality checks; fixed profile stopped, no15s or clean timing expansion.", + "First C22 model stopped before startup on duplicate controller aliases; audited canonicalization retains all attempts and time charges." + ] + } + ], + "limitations": [ + "Campaign acceptance remains INCONCLUSIVE; this is a storage-only projection.", + "The index covers all published result JSON documents and their named/nested attempt observations, not unpublished controller history.", + "Repeated identical observations of one named attempt are merged with every source pointer retained. Changed observations and anonymous records remain distinct.", + "Record/index entry counts are not independent trial counts. Text-only history and per-request traces remain in the byte-preserved original archive.", + "Only declared sample distributions are recomputed. Other retained results are explicitly reported frozen values, not independently requalified quality or statistics.", + "Original private models, media, raw receipts and runtime records were not published; archiving does not restore them or complete outstanding acceptance." + ] +} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/results/trials.jsonl b/evidence/h3-ref2va-910c-4logical-20261101-297/results/trials.jsonl new file mode 100644 index 000000000..c12bf8694 --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/results/trials.jsonl @@ -0,0 +1,95 @@ +{"artifact_id":"h3-ref2va-910c-4logical-20261101-297-6c7e237-v1","conclusion":"{\"scope\":\"Real captured H3 attention operator benchmark; full-model quality and HTTP are separate; no full-model quality/performance acceptance\"}","configuration":{"campaign_records":["h3-ref2va-910c-4logical-20261101-297/plan/execution-checkpoint.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossless-contract-v1.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossy-contract-v1.json"],"record_sha256":"9e996992042bc4f7fd0dd7c035a0800417ce99892bf64a40a188e0e28b066733"},"id":"01cf2496403f488db522ccdc2bdbda3b655057254d88b426a69ee370015a48e4","identity":"trial:c18-video-pooled-context-operator-315db0e5","kind":"attempt_observation","source_record":"h3-ref2va-910c-4logical-20261101-297/results/c12-through-c19-execution-checkpoint.json#/runs/24","source_records":["h3-ref2va-910c-4logical-20261101-297/results/c12-through-c19-execution-checkpoint.json#/runs/24","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c20-execution-checkpoint.json#/runs/24","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c21-execution-checkpoint.json#/runs/24","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c22-execution-checkpoint.json#/runs/24"],"status":"completed"} +{"artifact_id":"h3-ref2va-910c-4logical-20261101-297-6c7e237-v1","conclusion":"{\"decision\":\"Reject fixed C23 profile on observed frozen quality violations; no15s or clean timing expansion.\",\"quality_verdict\":\"FAIL\"}","configuration":{"campaign_records":["h3-ref2va-910c-4logical-20261101-297/plan/execution-checkpoint.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossless-contract-v1.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossy-contract-v1.json"],"record_sha256":"c922c5c7e9289f47e0b23a43633eb90459459b2365710f56a6deca8c007ceb9d"},"id":"0479e1e4c23e3f9c9b4915c49e7e891f652b94f823fe39054431f2a9dbac6126","identity":"run:c23-video-context-lse-quality-5s-00-704b6bb0","kind":"attempt_observation","source_record":"h3-ref2va-910c-4logical-20261101-297/results/c23-5s-00-execution-proof.json#","source_records":["h3-ref2va-910c-4logical-20261101-297/results/c23-5s-00-execution-proof.json#"],"status":"recorded"} +{"artifact_id":"h3-ref2va-910c-4logical-20261101-297-6c7e237-v1","conclusion":"{\"campaign_verdict\":\"INCONCLUSIVE\",\"scope\":\"Main-based C1 dispatch/native operation checks; not full-model/HTTP requalification.\"}","configuration":{"campaign_records":["h3-ref2va-910c-4logical-20261101-297/plan/execution-checkpoint.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossless-contract-v1.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossy-contract-v1.json"],"record_sha256":"82f19390dde19c0b02e76439727a5cbfcbad1ae3c631ad4cea7df76751196789"},"id":"048a9dfb50d46326487fff1492ba8ec45d5eb95edd90abed1cee5c287fdedea1","identity":null,"kind":"published_record","source_record":"h3-ref2va-910c-4logical-20261101-297/results/c1-upstream-main-native.json#","source_records":["h3-ref2va-910c-4logical-20261101-297/results/c1-upstream-main-native.json#"],"status":"recorded"} +{"artifact_id":"h3-ref2va-910c-4logical-20261101-297-6c7e237-v1","conclusion":"{\"scope\":\"Independent weighted global-context reference and native padded-head BF16 FAS; no full-model quality/performance acceptance\"}","configuration":{"campaign_records":["h3-ref2va-910c-4logical-20261101-297/plan/execution-checkpoint.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossless-contract-v1.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossy-contract-v1.json"],"record_sha256":"b058c3f999f20fab76d8be3ac052851f6be7cbab2169d38f73075a73d6a2e875"},"id":"0c0b3c2824238463a842ebf5faae3bbfc1034919a2ba9d00aa9d64df24f2f0dd","identity":"trial:c19-video-global-pool-native-f0f4c502","kind":"attempt_observation","source_record":"h3-ref2va-910c-4logical-20261101-297/results/c12-through-c19-execution-checkpoint.json#/runs/25","source_records":["h3-ref2va-910c-4logical-20261101-297/results/c12-through-c19-execution-checkpoint.json#/runs/25","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c20-execution-checkpoint.json#/runs/25","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c21-execution-checkpoint.json#/runs/25","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c22-execution-checkpoint.json#/runs/25","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c23-execution-checkpoint.json#/runs/25"],"status":"completed"} +{"artifact_id":"h3-ref2va-910c-4logical-20261101-297-6c7e237-v1","conclusion":"{\"decision\":\"Reject fixed C17 window profile; no15s or clean performance expansion.\",\"quality_verdict\":\"FAIL\"}","configuration":{"campaign_records":["h3-ref2va-910c-4logical-20261101-297/plan/execution-checkpoint.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossless-contract-v1.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossy-contract-v1.json"],"record_sha256":"649fb63f297c1df51b7634968c4a00abffa423c7712adb16984a72435465bc63"},"id":"0d650d446cb0404ece206483cd3621e9dfc26e02dbe1c742b8a40d200562a5bb","identity":"run:c17-video-window-quality-5s-00-a8c689e4","kind":"attempt_observation","source_record":"h3-ref2va-910c-4logical-20261101-297/results/c17-5s-00-execution-proof.json#","source_records":["h3-ref2va-910c-4logical-20261101-297/results/c17-5s-00-execution-proof.json#"],"status":"recorded"} +{"artifact_id":"h3-ref2va-910c-4logical-20261101-297-6c7e237-v1","conclusion":"{\"scope\":\"Independent weighted global-context reference and native padded-head BF16 FAS; no full-model quality/performance acceptance\"}","configuration":{"campaign_records":["h3-ref2va-910c-4logical-20261101-297/plan/execution-checkpoint.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossless-contract-v1.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossy-contract-v1.json"],"record_sha256":"1812d172746c334108712a015cb5956acae33822fcb95ee50b6f6cc9b062d4ca"},"id":"0dbb667206c4031704823a8f77a298c5bc2d7b14b70079ec909bb14b0ca93637","identity":"trial:c18-video-pooled-context-native-1673db72","kind":"attempt_observation","source_record":"h3-ref2va-910c-4logical-20261101-297/results/c12-through-c19-execution-checkpoint.json#/runs/23","source_records":["h3-ref2va-910c-4logical-20261101-297/results/c12-through-c19-execution-checkpoint.json#/runs/23","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c20-execution-checkpoint.json#/runs/23","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c21-execution-checkpoint.json#/runs/23","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c22-execution-checkpoint.json#/runs/23","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c23-execution-checkpoint.json#/runs/23"],"status":"completed"} +{"artifact_id":"h3-ref2va-910c-4logical-20261101-297-6c7e237-v1","conclusion":"{\"scope\":\"Independent weighted global-context reference and native padded-head BF16 FAS; no full-model quality/performance acceptance\"}","configuration":{"campaign_records":["h3-ref2va-910c-4logical-20261101-297/plan/execution-checkpoint.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossless-contract-v1.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossy-contract-v1.json"],"record_sha256":"bda0a8cc52ac3293dd82e650f527acc44cd4c6c7dd4808655cf89dc4105c67c2"},"id":"0dccfbf4c5fe8b01ec841100d63354ea98fcf37c982eca2a6f496fd34bd4fbe3","identity":"trial:c18-video-pooled-context-native-1673db72","kind":"attempt_observation","source_record":"h3-ref2va-910c-4logical-20261101-297/results/c18-native-qualification.json#","source_records":["h3-ref2va-910c-4logical-20261101-297/results/c18-native-qualification.json#"],"status":"recorded"} +{"artifact_id":"h3-ref2va-910c-4logical-20261101-297-6c7e237-v1","conclusion":"{\"decision\":\"Reject original Fn1Bn0 for5s-00; frozen adherence failure is retained; Bn1 is a separate configuration.\",\"quality_verdict\":\"FAIL\"}","configuration":{"campaign_records":["h3-ref2va-910c-4logical-20261101-297/plan/execution-checkpoint.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossless-contract-v1.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossy-contract-v1.json"],"record_sha256":"3c789de2677a63caca255daa18a0abd59eb077b0cddf009d61a671bf65f7e538"},"id":"0f24f82daf4d19cebb52d9f9635d5dd0693ffbbd21ff764635cf98fe81a8c875","identity":"trial:c11-modality-sigma-quality-5s-00-eb726462","kind":"attempt_observation","source_record":"h3-ref2va-910c-4logical-20261101-297/results/c11-original-5s-00-audit.json#","source_records":["h3-ref2va-910c-4logical-20261101-297/results/c11-original-5s-00-audit.json#"],"status":"recorded"} +{"artifact_id":"h3-ref2va-910c-4logical-20261101-297-6c7e237-v1","conclusion":"INCONCLUSIVE: no standalone verdict recorded","configuration":{"campaign_records":["h3-ref2va-910c-4logical-20261101-297/plan/execution-checkpoint.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossless-contract-v1.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossy-contract-v1.json"],"record_sha256":"7122199248bd1b6b35ee1d6e8669e04e5b9add0de3ffea3a77d9522c77bd7d8c"},"id":"0fe7d71b983e81253f57f0d48074029145a8587ceed0a5f609e6adabf9203fc7","identity":"trial:c23-video-context-lse-quality-5s-00-704b6bb0","kind":"attempt_observation","source_record":"h3-ref2va-910c-4logical-20261101-297/results/c12-through-c23-execution-checkpoint.json#/runs/37","source_records":["h3-ref2va-910c-4logical-20261101-297/results/c12-through-c23-execution-checkpoint.json#/runs/37"],"status":"completed"} +{"artifact_id":"h3-ref2va-910c-4logical-20261101-297-6c7e237-v1","conclusion":"{\"decision\":\"Reject fixed C21 eight-anchor zero-order pre-gate profile on seven observed quality violations; no15s or clean timing expansion.\",\"quality_verdict\":\"FAIL\"}","configuration":{"campaign_records":["h3-ref2va-910c-4logical-20261101-297/plan/execution-checkpoint.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossless-contract-v1.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossy-contract-v1.json"],"record_sha256":"fe741296f71398f8b7d5a4f8ceba02c578a8c681862b928770babf1194a7d70f"},"id":"1184ae5f5c466832dbefacda0e11f0f17bcf241b6b546c53c44f6aac54daa70c","identity":"run:c21-video-pre-gate-quality-5s-00-7265c33e","kind":"attempt_observation","source_record":"h3-ref2va-910c-4logical-20261101-297/results/c21-5s-00-execution-proof.json#","source_records":["h3-ref2va-910c-4logical-20261101-297/results/c21-5s-00-execution-proof.json#"],"status":"recorded"} +{"artifact_id":"h3-ref2va-910c-4logical-20261101-297-6c7e237-v1","conclusion":"INCONCLUSIVE: no standalone verdict recorded","configuration":{"campaign_records":["h3-ref2va-910c-4logical-20261101-297/plan/execution-checkpoint.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossless-contract-v1.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossy-contract-v1.json"],"record_sha256":"a47d56f1bf812afed5362e25c00cf447088e42b9e124285dc4ce54914271e781"},"id":"125343913ae6d07873c5235200abb3a4446c3707925a1da7c95c951ed1966130","identity":"run:c1-upstream-pr-native-80d5d51f","kind":"attempt_observation","source_record":"h3-ref2va-910c-4logical-20261101-297/results/c1-upstream-main-native.json#/prior_attempt","source_records":["h3-ref2va-910c-4logical-20261101-297/results/c1-upstream-main-native.json#/prior_attempt"],"status":"failed"} +{"artifact_id":"h3-ref2va-910c-4logical-20261101-297-6c7e237-v1","conclusion":"{\"scope\":\"20 focused tests including two executed native Ascend tests; not full-model quality or speed qualification\"}","configuration":{"campaign_records":["h3-ref2va-910c-4logical-20261101-297/plan/execution-checkpoint.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossless-contract-v1.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossy-contract-v1.json"],"record_sha256":"4d60280ce1f446c1d7383eba758f9f98025521d3dbe73ba686839266ff3e1b1c"},"id":"16ba8f5a9de19579e23f115be74bb3b8be1d6bf95778f2932d07a0cd70f664be","identity":"trial:c11-sigma-native-tests-e4a95751","kind":"attempt_observation","source_record":"h3-ref2va-910c-4logical-20261101-297/results/c11-native-execution.json#","source_records":["h3-ref2va-910c-4logical-20261101-297/results/c11-native-execution.json#"],"status":"completed"} +{"artifact_id":"h3-ref2va-910c-4logical-20261101-297-6c7e237-v1","conclusion":"{\"scope\":\"Focused state/block/request tests, real Ref2VA/FL2VA packing and native four-rank subset attention. Full-model quality/performance remain separate.\"}","configuration":{"campaign_records":["h3-ref2va-910c-4logical-20261101-297/plan/execution-checkpoint.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossless-contract-v1.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossy-contract-v1.json"],"record_sha256":"510527da3bfd8ca22402d75565676f675e37158f37b484618ab22d84a84dfffc"},"id":"176f1c1b5676d2bbfcfe7d2274005a40147e65f4975fb1834a73bb943cf1edca","identity":"trial:c15-video-row-native-3a9721a6","kind":"attempt_observation","source_record":"h3-ref2va-910c-4logical-20261101-297/results/c15-native-qualification.json#","source_records":["h3-ref2va-910c-4logical-20261101-297/results/c15-native-qualification.json#"],"status":"recorded"} +{"artifact_id":"h3-ref2va-910c-4logical-20261101-297-6c7e237-v1","conclusion":"{\"scope\":\"Controller label correction for two existing mechanisms; no new direction/budget approval inferred.\"}","configuration":{"campaign_records":["h3-ref2va-910c-4logical-20261101-297/plan/execution-checkpoint.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossless-contract-v1.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossy-contract-v1.json"],"record_sha256":"e0a7a555f8d70c07bccb8571cfec7abaf3a9adea28f6d832840bc95a42b1cf0b"},"id":"1a7f3f18a463aa11da8349deb420a1b800bbbe1438eb411f7aff6c5af22b125a","identity":null,"kind":"published_record","source_record":"h3-ref2va-910c-4logical-20261101-297/results/c21-c22-direction-identity-correction.json#","source_records":["h3-ref2va-910c-4logical-20261101-297/results/c21-c22-direction-identity-correction.json#"],"status":"recorded"} +{"artifact_id":"h3-ref2va-910c-4logical-20261101-297-6c7e237-v1","conclusion":"INCONCLUSIVE: no standalone verdict recorded","configuration":{"campaign_records":["h3-ref2va-910c-4logical-20261101-297/plan/execution-checkpoint.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossless-contract-v1.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossy-contract-v1.json"],"record_sha256":"f8d6bd8fcb49b3d45cdec45105cbb4892e90ef9ca0cdb7e8c1b08d8772035ed5"},"id":"1b9eb962c45225c3c0a012cdc3ae008ccb3d3a331564abd36e12659d3272b4e2","identity":"trial:c15-video-row-v3-quality-5s-00-70703586","kind":"attempt_observation","source_record":"h3-ref2va-910c-4logical-20261101-297/results/c12-through-c17-execution-checkpoint.json#/runs/17","source_records":["h3-ref2va-910c-4logical-20261101-297/results/c12-through-c17-execution-checkpoint.json#/runs/17","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c19-execution-checkpoint.json#/runs/17","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c20-execution-checkpoint.json#/runs/17","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c21-execution-checkpoint.json#/runs/17","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c22-execution-checkpoint.json#/runs/17","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c23-execution-checkpoint.json#/runs/17"],"status":"failed"} +{"artifact_id":"h3-ref2va-910c-4logical-20261101-297-6c7e237-v1","conclusion":"{\"campaign_verdict\":\"INCONCLUSIVE\"}","configuration":{"campaign_records":["h3-ref2va-910c-4logical-20261101-297/plan/execution-checkpoint.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossless-contract-v1.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossy-contract-v1.json"],"record_sha256":"a45e4e68e3b64a40ffd346a4b09c7ae9a6967ca0e27ec634692d31233fd241aa"},"id":"1caf0f924965042786eef08457e23a90769dd6af74343b18f9b918645eb024f5","identity":null,"kind":"published_record","source_record":"h3-ref2va-910c-4logical-20261101-297/results/c12-through-c17-execution-checkpoint.json#","source_records":["h3-ref2va-910c-4logical-20261101-297/results/c12-through-c17-execution-checkpoint.json#"],"status":"recorded"} +{"artifact_id":"h3-ref2va-910c-4logical-20261101-297-6c7e237-v1","conclusion":"{\"campaign_verdict\":\"INCONCLUSIVE\"}","configuration":{"campaign_records":["h3-ref2va-910c-4logical-20261101-297/plan/execution-checkpoint.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossless-contract-v1.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossy-contract-v1.json"],"record_sha256":"28fb3be451deca868cd84269058acde587ed19881a9a9823064f2785a3dffe20"},"id":"1f006ba8a1109192b436eb71530615d76c45d5fdaa5f59a9cc30bf30b3f058e2","identity":null,"kind":"published_record","source_record":"h3-ref2va-910c-4logical-20261101-297/results/c12-through-c20-execution-checkpoint.json#","source_records":["h3-ref2va-910c-4logical-20261101-297/results/c12-through-c20-execution-checkpoint.json#"],"status":"recorded"} +{"artifact_id":"h3-ref2va-910c-4logical-20261101-297-6c7e237-v1","conclusion":"{\"campaign_verdict\":\"INCONCLUSIVE\"}","configuration":{"campaign_records":["h3-ref2va-910c-4logical-20261101-297/plan/execution-checkpoint.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossless-contract-v1.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossy-contract-v1.json"],"record_sha256":"faed14f36cf2deabf67b2c6901508d4b600e70f39374e109758953ebaeb469bd"},"id":"200ce3cebaf6b011f299052a97091842c836ef91f19eef97d3a270f81484c613","identity":null,"kind":"published_record","source_record":"h3-ref2va-910c-4logical-20261101-297/results/c12-through-c15-execution-checkpoint.json#","source_records":["h3-ref2va-910c-4logical-20261101-297/results/c12-through-c15-execution-checkpoint.json#"],"status":"recorded"} +{"artifact_id":"h3-ref2va-910c-4logical-20261101-297-6c7e237-v1","conclusion":"INCONCLUSIVE: no standalone verdict recorded","configuration":{"campaign_records":["h3-ref2va-910c-4logical-20261101-297/plan/execution-checkpoint.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossless-contract-v1.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossy-contract-v1.json"],"record_sha256":"d76cfe8635ad64948cd126d4b60be01e29a497169bd4970e376b43b82a84861c"},"id":"24091e5a81f6ce5b8d07d7370417c6b9f895b1b9debc4e913065c8b74fed7f5c","identity":"trial:c12-alignment-quality-5s-00-9bf5c373","kind":"attempt_observation","source_record":"h3-ref2va-910c-4logical-20261101-297/results/c12-through-c15-execution-checkpoint.json#/runs/0","source_records":["h3-ref2va-910c-4logical-20261101-297/results/c12-through-c15-execution-checkpoint.json#/runs/0","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c17-execution-checkpoint.json#/runs/0","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c19-execution-checkpoint.json#/runs/0","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c20-execution-checkpoint.json#/runs/0","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c21-execution-checkpoint.json#/runs/0","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c22-execution-checkpoint.json#/runs/0","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c23-execution-checkpoint.json#/runs/0"],"status":"completed"} +{"artifact_id":"h3-ref2va-910c-4logical-20261101-297-6c7e237-v1","conclusion":"{\"scope\":\"CPU mock barrier/chunk tests; no actual NPU collective or full-model quality/performance acceptance\"}","configuration":{"campaign_records":["h3-ref2va-910c-4logical-20261101-297/plan/execution-checkpoint.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossless-contract-v1.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossy-contract-v1.json"],"record_sha256":"8720dafa1629e6f233a669404d39d886755182f82cf219877a1fb1118958ed5f"},"id":"28141bac57576e2227649fc23e8c4a34dcd6e73461256bf023da4fc64becd85d","identity":"trial:c13-dlo-regression-tests-95e9ed2a","kind":"attempt_observation","source_record":"h3-ref2va-910c-4logical-20261101-297/results/c12-through-c15-execution-checkpoint.json#/runs/7","source_records":["h3-ref2va-910c-4logical-20261101-297/results/c12-through-c15-execution-checkpoint.json#/runs/7","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c17-execution-checkpoint.json#/runs/7","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c19-execution-checkpoint.json#/runs/7","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c20-execution-checkpoint.json#/runs/7","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c21-execution-checkpoint.json#/runs/7","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c22-execution-checkpoint.json#/runs/7","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c23-execution-checkpoint.json#/runs/7"],"status":"failed"} +{"artifact_id":"h3-ref2va-910c-4logical-20261101-297-6c7e237-v1","conclusion":"{\"campaign_verdict\":\"INCONCLUSIVE: four tracks and final protocol are incomplete\",\"native_verdict\":\"FAIL\"}","configuration":{"campaign_records":["h3-ref2va-910c-4logical-20261101-297/plan/execution-checkpoint.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossless-contract-v1.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossy-contract-v1.json"],"record_sha256":"32c1829c833384b9aa0bfef643e68ada9a2e152a552f58e9913ed73e39c0c2ef"},"id":"28cd8b648a77b73612ecaf4b4df148f819c8e146cedf7b497ff680d4f4061a6c","identity":"native_run_id:20261101T181531Z-164cfec13c","kind":"attempt_observation","source_record":"h3-ref2va-910c-4logical-20261101-297/results/c1-clean-5s-screening.json#","source_records":["h3-ref2va-910c-4logical-20261101-297/results/c1-clean-5s-screening.json#"],"status":"completed"} +{"artifact_id":"h3-ref2va-910c-4logical-20261101-297-6c7e237-v1","conclusion":"{\"scope\":\"Focused state/block tests and native four-rank subset attention; no full-model quality/performance acceptance\"}","configuration":{"campaign_records":["h3-ref2va-910c-4logical-20261101-297/plan/execution-checkpoint.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossless-contract-v1.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossy-contract-v1.json"],"record_sha256":"cce02306371a36d72830e96218ae83478c67e43400de6bce4ec9e41aae5c1194"},"id":"29a3a862b0f45b029496a0defd3ffca2bea76aa776dbd3d0cd3c7e210cdaeb5e","identity":"trial:c15-video-row-native-bb5084cc","kind":"attempt_observation","source_record":"h3-ref2va-910c-4logical-20261101-297/results/c12-through-c15-execution-checkpoint.json#/runs/14","source_records":["h3-ref2va-910c-4logical-20261101-297/results/c12-through-c15-execution-checkpoint.json#/runs/14","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c17-execution-checkpoint.json#/runs/14","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c19-execution-checkpoint.json#/runs/14","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c20-execution-checkpoint.json#/runs/14","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c21-execution-checkpoint.json#/runs/14","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c22-execution-checkpoint.json#/runs/14","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c23-execution-checkpoint.json#/runs/14"],"status":"completed"} +{"artifact_id":"h3-ref2va-910c-4logical-20261101-297-6c7e237-v1","conclusion":"{\"decision\":\"Reject fixed C22 sigma-secant pre-gate profile on nine observed violations; no15s or clean timing expansion.\",\"quality_verdict\":\"FAIL\"}","configuration":{"campaign_records":["h3-ref2va-910c-4logical-20261101-297/plan/execution-checkpoint.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossless-contract-v1.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossy-contract-v1.json"],"record_sha256":"0bcaa8c682bdde022e1f2653f12742ad53280b748ff1cd88996cfabbf1f20132"},"id":"2b919e70748bd5c9d21768dc9bb2cca08f57d9230b99323112c232a7390c027c","identity":"run:c22-video-pre-gate-secant-v2-quality-5s-00-777147cb","kind":"attempt_observation","source_record":"h3-ref2va-910c-4logical-20261101-297/results/c22-5s-00-execution-proof.json#","source_records":["h3-ref2va-910c-4logical-20261101-297/results/c22-5s-00-execution-proof.json#"],"status":"recorded"} +{"artifact_id":"h3-ref2va-910c-4logical-20261101-297-6c7e237-v1","conclusion":"{\"scope\":\"Real captured H3 attention operator benchmark; full-model quality and HTTP are separate; no full-model quality/performance acceptance\"}","configuration":{"campaign_records":["h3-ref2va-910c-4logical-20261101-297/plan/execution-checkpoint.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossless-contract-v1.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossy-contract-v1.json"],"record_sha256":"06f79b1ff972dd6cebc545a38f27fe35068fc6fe3fb70133ece6572cf7f008d2"},"id":"33468e090e32e3bcda0cd1c3ee33546c9f4073046ab41ce084aa7f660e6d5c58","identity":"trial:c23-video-context-lse-components-783670a5","kind":"attempt_observation","source_record":"h3-ref2va-910c-4logical-20261101-297/results/c12-through-c23-execution-checkpoint.json#/runs/34","source_records":["h3-ref2va-910c-4logical-20261101-297/results/c12-through-c23-execution-checkpoint.json#/runs/34"],"status":"completed"} +{"artifact_id":"h3-ref2va-910c-4logical-20261101-297-6c7e237-v1","conclusion":"{\"campaign_verdict\":\"INCONCLUSIVE\"}","configuration":{"campaign_records":["h3-ref2va-910c-4logical-20261101-297/plan/execution-checkpoint.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossless-contract-v1.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossy-contract-v1.json"],"record_sha256":"0c68be9f7a52b67e29357449086d20fc9780b0e834580fd527500aaab14c0b19"},"id":"34e3d53f6fa4d2d2adaec63bf717cbbe500c8df277e2f64a0180f02e84d53df2","identity":null,"kind":"published_record","source_record":"h3-ref2va-910c-4logical-20261101-297/results/automatic-quality-checkpoint.json#","source_records":["h3-ref2va-910c-4logical-20261101-297/results/automatic-quality-checkpoint.json#"],"status":"recorded"} +{"artifact_id":"h3-ref2va-910c-4logical-20261101-297-6c7e237-v1","conclusion":"{\"campaign_verdict\":\"INCONCLUSIVE\"}","configuration":{"campaign_records":["h3-ref2va-910c-4logical-20261101-297/plan/execution-checkpoint.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossless-contract-v1.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossy-contract-v1.json"],"record_sha256":"5d645ad75b946f2a139264c7c39b180b6ea70d8d91d2019d2f4d93856cde67bc"},"id":"35027304eb1a60d2de651f3f87bfc18e325fabb350fa6ebf92a00a997a806d12","identity":null,"kind":"published_record","source_record":"h3-ref2va-910c-4logical-20261101-297/results/c12-through-c22-execution-checkpoint.json#","source_records":["h3-ref2va-910c-4logical-20261101-297/results/c12-through-c22-execution-checkpoint.json#"],"status":"recorded"} +{"artifact_id":"h3-ref2va-910c-4logical-20261101-297-6c7e237-v1","conclusion":"{\"scope\":\"initialnoise and full-compute poststep0 only; not bit-exact later cached steps or quality acceptance\"}","configuration":{"campaign_records":["h3-ref2va-910c-4logical-20261101-297/plan/execution-checkpoint.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossless-contract-v1.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossy-contract-v1.json"],"record_sha256":"38a9035b441cab54acffe48545d5d2dcfe58f40a64c0eee1abcfc4ba9a9e86ce"},"id":"3878595d4917b09c3e02b18a46a7bcb5bdc9541a1c9d958300c93237a4890f0c","identity":null,"kind":"published_record","source_record":"h3-ref2va-910c-4logical-20261101-297/results/c11-full-compute-first-step.json#","source_records":["h3-ref2va-910c-4logical-20261101-297/results/c11-full-compute-first-step.json#"],"status":"recorded"} +{"artifact_id":"h3-ref2va-910c-4logical-20261101-297-6c7e237-v1","conclusion":"{\"decision\":\"Reject fixed C16 profile on three observed audio violations; no5s01/15s expansion or clean performance qualification.\",\"quality_verdict\":\"FAIL\"}","configuration":{"campaign_records":["h3-ref2va-910c-4logical-20261101-297/plan/execution-checkpoint.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossless-contract-v1.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossy-contract-v1.json"],"record_sha256":"ae80b07b1c0e9cfcb1dfefa3664a7b34163dcda0c0c88d32bac1e5a16269ca31"},"id":"3a03a7ff9514c862a84586ad7978dc1bb6d74f525326bd35e5cc33ece959e65b","identity":"run:c16-video-row-fp32-quality-5s-00-8600757a","kind":"attempt_observation","source_record":"h3-ref2va-910c-4logical-20261101-297/results/c16-5s-00-execution-proof.json#","source_records":["h3-ref2va-910c-4logical-20261101-297/results/c16-5s-00-execution-proof.json#"],"status":"recorded"} +{"artifact_id":"h3-ref2va-910c-4logical-20261101-297-6c7e237-v1","conclusion":"{\"scope\":\"specified latent points, schedule, decoded RGB24/PCM; no performance acceptance\",\"verdict\":\"PASS\"}","configuration":{"campaign_records":["h3-ref2va-910c-4logical-20261101-297/plan/execution-checkpoint.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossless-contract-v1.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossy-contract-v1.json"],"record_sha256":"c1ddefa6a52acb0944cfb0f20f7525576343f7b94378c132dbc5faa5ceee6850"},"id":"414acdc4096df98141cf58dd11e462037f16d7bd2773d2b69fd7861e219148aa","identity":null,"kind":"published_record","source_record":"h3-ref2va-910c-4logical-20261101-297/results/c1-5s-strict-comparison.json#","source_records":["h3-ref2va-910c-4logical-20261101-297/results/c1-5s-strict-comparison.json#"],"status":"recorded"} +{"artifact_id":"h3-ref2va-910c-4logical-20261101-297-6c7e237-v1","conclusion":"{\"scope\":\"Real captured H3 attention operator benchmark; full-model quality and HTTP are separate; no full-model quality/performance acceptance\"}","configuration":{"campaign_records":["h3-ref2va-910c-4logical-20261101-297/plan/execution-checkpoint.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossless-contract-v1.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossy-contract-v1.json"],"record_sha256":"12e3c4b5b278f5a76bb9e7bb73e06cff135d2a599280474a7160c2ddbbe3b59b"},"id":"433339f40d156df038f05c92b6b230471b39747f0a82f9dfa58e78badd424c93","identity":"trial:c20-video-variance-pool-operator-209f263d","kind":"attempt_observation","source_record":"h3-ref2va-910c-4logical-20261101-297/results/c12-through-c23-execution-checkpoint.json#/runs/28","source_records":["h3-ref2va-910c-4logical-20261101-297/results/c12-through-c23-execution-checkpoint.json#/runs/28"],"status":"completed"} +{"artifact_id":"h3-ref2va-910c-4logical-20261101-297-6c7e237-v1","conclusion":"{\"scope\":\"Legacy/pre-gate/DLO tests, three executed native four-rank HCCL/subsetFA suites and fresh current gates; no full-model quality/performance acceptance\"}","configuration":{"campaign_records":["h3-ref2va-910c-4logical-20261101-297/plan/execution-checkpoint.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossless-contract-v1.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossy-contract-v1.json"],"record_sha256":"78119db0dddd01ff358d047c576a9ef816284d45b127359a14131ed2e7001b7d"},"id":"43605d34544eb45b8d2e13a8ea2e5b6133ab628d4a7900af42110390aafeb772","identity":"trial:c22-video-pre-gate-secant-native-3521747e","kind":"attempt_observation","source_record":"h3-ref2va-910c-4logical-20261101-297/results/c12-through-c22-execution-checkpoint.json#/runs/31","source_records":["h3-ref2va-910c-4logical-20261101-297/results/c12-through-c22-execution-checkpoint.json#/runs/31","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c23-execution-checkpoint.json#/runs/31"],"status":"completed"} +{"artifact_id":"h3-ref2va-910c-4logical-20261101-297-6c7e237-v1","conclusion":"INCONCLUSIVE: no standalone verdict recorded","configuration":{"campaign_records":["h3-ref2va-910c-4logical-20261101-297/plan/execution-checkpoint.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossless-contract-v1.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossy-contract-v1.json"],"record_sha256":"2eaa3d2183e277f640da7d521e2087d71591e7778e95c4c8dccfbdb0a1ee5a3b"},"id":"4ac41b59d996cfad636af9a9f199e25216125c08ec00ccc6d0e5fcb79060015d","identity":"trial:c11-sigma-fp32-native-tests-cc806d17","kind":"attempt_observation","source_record":"h3-ref2va-910c-4logical-20261101-297/results/c11-fp32-native-execution.json#/attempts/1","source_records":["h3-ref2va-910c-4logical-20261101-297/results/c11-fp32-native-execution.json#/attempts/1"],"status":"failed"} +{"artifact_id":"h3-ref2va-910c-4logical-20261101-297-6c7e237-v1","conclusion":"{\"scope\":\"Expanded offloader tests in Ascend environment; some stock accelerator fixtures use NPU, not a full model test\"}","configuration":{"campaign_records":["h3-ref2va-910c-4logical-20261101-297/plan/execution-checkpoint.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossless-contract-v1.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossy-contract-v1.json"],"record_sha256":"88f1423b67970fc94d9467cfa5c481230b9b7a621286adde3d5a48230cef1cec"},"id":"4d64cf7e4d09bf7fc9a418cae3e49923cb2e85cfec456d47fca574b617cc1b2e","identity":"trial:c13-dlo-regression-v2-12c217d5","kind":"attempt_observation","source_record":"h3-ref2va-910c-4logical-20261101-297/results/c12-through-c15-execution-checkpoint.json#/runs/8","source_records":["h3-ref2va-910c-4logical-20261101-297/results/c12-through-c15-execution-checkpoint.json#/runs/8","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c17-execution-checkpoint.json#/runs/8","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c19-execution-checkpoint.json#/runs/8","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c20-execution-checkpoint.json#/runs/8","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c21-execution-checkpoint.json#/runs/8","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c22-execution-checkpoint.json#/runs/8","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c23-execution-checkpoint.json#/runs/8"],"status":"failed"} +{"artifact_id":"h3-ref2va-910c-4logical-20261101-297-6c7e237-v1","conclusion":"{\"scope\":\"Legacy/pre-gate/DLO tests, two executed native four-rank HCCL/subsetFA suites and fresh current gates; no full-model quality/performance acceptance\"}","configuration":{"campaign_records":["h3-ref2va-910c-4logical-20261101-297/plan/execution-checkpoint.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossless-contract-v1.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossy-contract-v1.json"],"record_sha256":"888fd2bbb4db486ed58eaa4c5eb47361f479185b00de3f840c309e9a9f4502af"},"id":"4e989e8402eeaf14876651d30b2f4be8e8153c74b71765b134c730b26ab8a3df","identity":"trial:c21-video-pre-gate-native-3c5eebbe","kind":"attempt_observation","source_record":"h3-ref2va-910c-4logical-20261101-297/results/c21-native-qualification.json#","source_records":["h3-ref2va-910c-4logical-20261101-297/results/c21-native-qualification.json#"],"status":"recorded"} +{"artifact_id":"h3-ref2va-910c-4logical-20261101-297-6c7e237-v1","conclusion":"{\"scope\":\"specified latent points, schedule, decoded RGB24/PCM; no performance acceptance\",\"verdict\":\"PASS\"}","configuration":{"campaign_records":["h3-ref2va-910c-4logical-20261101-297/plan/execution-checkpoint.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossless-contract-v1.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossy-contract-v1.json"],"record_sha256":"c1ddefa6a52acb0944cfb0f20f7525576343f7b94378c132dbc5faa5ceee6850"},"id":"53b1a499d555396ae8ae8017b0c3c0c1f976d5adcae8f59c63c4f448d9ea010c","identity":null,"kind":"published_record","source_record":"h3-ref2va-910c-4logical-20261101-297/results/c1-15s-strict-comparison.json#","source_records":["h3-ref2va-910c-4logical-20261101-297/results/c1-15s-strict-comparison.json#"],"status":"recorded"} +{"artifact_id":"h3-ref2va-910c-4logical-20261101-297-6c7e237-v1","conclusion":"{\"scope\":\"Two failed test-reference attempts retained, then23passed incl3executed NPUtests. Productionbytes unchanged across attempts. Frozen modelquality thresholds/scorers unchanged.\"}","configuration":{"campaign_records":["h3-ref2va-910c-4logical-20261101-297/plan/execution-checkpoint.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossless-contract-v1.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossy-contract-v1.json"],"record_sha256":"7e99ee759f022a9c8ff311b7121e4c4b769e41bc573ec692d4b6511bc291aa33"},"id":"55d8806d1a1db29e1177d2528be2a9b856f57c61dcac8d45d42210af0f446690","identity":null,"kind":"published_record","source_record":"h3-ref2va-910c-4logical-20261101-297/results/c11-fp32-native-execution.json#","source_records":["h3-ref2va-910c-4logical-20261101-297/results/c11-fp32-native-execution.json#"],"status":"recorded"} +{"artifact_id":"h3-ref2va-910c-4logical-20261101-297-6c7e237-v1","conclusion":"{\"campaign_verdict\":\"INCONCLUSIVE\"}","configuration":{"campaign_records":["h3-ref2va-910c-4logical-20261101-297/plan/execution-checkpoint.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossless-contract-v1.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossy-contract-v1.json"],"record_sha256":"11589eb89a58cd53a3d5f6e09a4c859b21a4990d47b8d30ac540a54a406e8696"},"id":"59290662702e2be421f4be736d7cdb48600257a17e19b80e6b6f49d0db0cba4d","identity":null,"kind":"published_record","source_record":"h3-ref2va-910c-4logical-20261101-297/results/c12-through-c21-execution-checkpoint.json#","source_records":["h3-ref2va-910c-4logical-20261101-297/results/c12-through-c21-execution-checkpoint.json#"],"status":"recorded"} +{"artifact_id":"h3-ref2va-910c-4logical-20261101-297-6c7e237-v1","conclusion":"{\"campaign_verdict\":\"INCONCLUSIVE\"}","configuration":{"campaign_records":["h3-ref2va-910c-4logical-20261101-297/plan/execution-checkpoint.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossless-contract-v1.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossy-contract-v1.json"],"record_sha256":"50eb21a3888cc8a69291d653a8031a34b456a2a9c9570e84e0e79e3b1f3dd58f"},"id":"5c5f11cb4bcf119e35173dd787c39fe8da940411df4db55de4816b52cea51b78","identity":null,"kind":"published_record","source_record":"h3-ref2va-910c-4logical-20261101-297/results/c12-through-c23-execution-checkpoint.json#","source_records":["h3-ref2va-910c-4logical-20261101-297/results/c12-through-c23-execution-checkpoint.json#"],"status":"recorded"} +{"artifact_id":"h3-ref2va-910c-4logical-20261101-297-6c7e237-v1","conclusion":"{\"scope\":\"Bounded native prerequisites only; a failed probe is retained as negative evidence.\"}","configuration":{"campaign_records":["h3-ref2va-910c-4logical-20261101-297/plan/execution-checkpoint.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossless-contract-v1.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossy-contract-v1.json"],"record_sha256":"8032278b92d4470ad466681af9143e5593b6f06f07844e702de9bb6ce4010478"},"id":"5dd125cad97c3f406d99659e67cb561837b52989b2de26fa92bf2beb144e7307","identity":"trial:c14-int8-prerequisites-fdd6633b","kind":"attempt_observation","source_record":"h3-ref2va-910c-4logical-20261101-297/results/c12-through-c15-execution-checkpoint.json#/runs/9","source_records":["h3-ref2va-910c-4logical-20261101-297/results/c12-through-c15-execution-checkpoint.json#/runs/9","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c17-execution-checkpoint.json#/runs/9","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c19-execution-checkpoint.json#/runs/9","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c20-execution-checkpoint.json#/runs/9","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c21-execution-checkpoint.json#/runs/9","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c22-execution-checkpoint.json#/runs/9","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c23-execution-checkpoint.json#/runs/9"],"status":"failed"} +{"artifact_id":"h3-ref2va-910c-4logical-20261101-297-6c7e237-v1","conclusion":"{\"scope\":\"Real captured H3 attention operator benchmark; full-model quality and HTTP are separate; no full-model quality/performance acceptance\"}","configuration":{"campaign_records":["h3-ref2va-910c-4logical-20261101-297/plan/execution-checkpoint.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossless-contract-v1.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossy-contract-v1.json"],"record_sha256":"48e7cf46100570ed3dcc4ca9432c8c1c10f18feb257d881d7a09eab3d53412ff"},"id":"6574c8caa92a324b9ef1b828582f2d768de7a0ecb7503598e27e29d15febcce7","identity":"trial:c18-video-pooled-context-operator-315db0e5","kind":"attempt_observation","source_record":"h3-ref2va-910c-4logical-20261101-297/results/c12-through-c23-execution-checkpoint.json#/runs/24","source_records":["h3-ref2va-910c-4logical-20261101-297/results/c12-through-c23-execution-checkpoint.json#/runs/24"],"status":"completed"} +{"artifact_id":"h3-ref2va-910c-4logical-20261101-297-6c7e237-v1","conclusion":"{\"scope\":\"Independent weighted global-context reference and native padded-head BF16 FAS; no full-model quality/performance acceptance\"}","configuration":{"campaign_records":["h3-ref2va-910c-4logical-20261101-297/plan/execution-checkpoint.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossless-contract-v1.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossy-contract-v1.json"],"record_sha256":"2f6394936bf77ad1d25494f1c381d6369329c7ab97797e6b49e3738648089912"},"id":"6599c8cd06aa0155d56bdd9a397b1e8199c6ca7cc056337fce5fef1ab67a89f9","identity":"trial:c20-video-variance-pool-native-50101063","kind":"attempt_observation","source_record":"h3-ref2va-910c-4logical-20261101-297/results/c12-through-c20-execution-checkpoint.json#/runs/27","source_records":["h3-ref2va-910c-4logical-20261101-297/results/c12-through-c20-execution-checkpoint.json#/runs/27","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c21-execution-checkpoint.json#/runs/27","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c22-execution-checkpoint.json#/runs/27","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c23-execution-checkpoint.json#/runs/27"],"status":"completed"} +{"artifact_id":"h3-ref2va-910c-4logical-20261101-297-6c7e237-v1","conclusion":"{\"scope\":\"Real captured H3 attention operator benchmark; full-model quality and HTTP are separate; no full-model quality/performance acceptance\"}","configuration":{"campaign_records":["h3-ref2va-910c-4logical-20261101-297/plan/execution-checkpoint.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossless-contract-v1.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossy-contract-v1.json"],"record_sha256":"1df78f25dd29fe6712aef6418fe2ea5394211c94d96ae61dab9865d53ee54890"},"id":"6856307fff85091a77d04939ff9ff0027e1a9f0b1fa40d5f4aa83173e66d8f47","identity":"trial:c23-video-context-lse-operator-66d6ce78","kind":"attempt_observation","source_record":"h3-ref2va-910c-4logical-20261101-297/results/c12-through-c23-execution-checkpoint.json#/runs/36","source_records":["h3-ref2va-910c-4logical-20261101-297/results/c12-through-c23-execution-checkpoint.json#/runs/36"],"status":"completed"} +{"artifact_id":"h3-ref2va-910c-4logical-20261101-297-6c7e237-v1","conclusion":"{\"scope\":\"CLI discovery/targetPythonmetadata only; no pairedbenchmark or backendcompatibility acceptance\"}","configuration":{"campaign_records":["h3-ref2va-910c-4logical-20261101-297/plan/execution-checkpoint.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossless-contract-v1.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossy-contract-v1.json"],"record_sha256":"1d2ff3ba7f4b698f03c0a9ac262414ea7399448f0a79be2249903409446604f6"},"id":"69c49edb6c59b0b64ddca6791c121c2b0551440aff9af3915f46a00979c48f15","identity":null,"kind":"published_record","source_record":"h3-ref2va-910c-4logical-20261101-297/results/native-cli-doctor.json#","source_records":["h3-ref2va-910c-4logical-20261101-297/results/native-cli-doctor.json#"],"status":"recorded"} +{"artifact_id":"h3-ref2va-910c-4logical-20261101-297-6c7e237-v1","conclusion":"INCONCLUSIVE: no standalone verdict recorded","configuration":{"campaign_records":["h3-ref2va-910c-4logical-20261101-297/plan/execution-checkpoint.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossless-contract-v1.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossy-contract-v1.json"],"record_sha256":"bb3ccf8164490d0c407b29c2d4e4b48f215bb148fbc87ff6a45270f71d23b204"},"id":"6e938940dc2683090eb9de3f70882be51b9fc4b289e52c61bda924ee35f91a7d","identity":"trial:c15-video-row-quality-5s-00-710315e0","kind":"attempt_observation","source_record":"h3-ref2va-910c-4logical-20261101-297/results/c12-through-c15-execution-checkpoint.json#/runs/15","source_records":["h3-ref2va-910c-4logical-20261101-297/results/c12-through-c15-execution-checkpoint.json#/runs/15"],"status":"failed"} +{"artifact_id":"h3-ref2va-910c-4logical-20261101-297-6c7e237-v1","conclusion":"{\"decision\":\"The API exists in dev; a version label alone does not establish wheel contents or 910C native support.\"}","configuration":{"campaign_records":["h3-ref2va-910c-4logical-20261101-297/plan/execution-checkpoint.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossless-contract-v1.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossy-contract-v1.json"],"record_sha256":"2f2a2964728601946b1e0f6d9b9e1e8f671eacabd43a5601c1100f5739cf277a"},"id":"70ae9012d23344c35166febfe9e87dfabc432bafbbeba1a37f469a32293f4440","identity":null,"kind":"published_record","source_record":"h3-ref2va-910c-4logical-20261101-297/results/mindie-quant-api-source-check.json#","source_records":["h3-ref2va-910c-4logical-20261101-297/results/mindie-quant-api-source-check.json#"],"status":"recorded"} +{"artifact_id":"h3-ref2va-910c-4logical-20261101-297-6c7e237-v1","conclusion":"INCONCLUSIVE: no standalone verdict recorded","configuration":{"campaign_records":["h3-ref2va-910c-4logical-20261101-297/plan/execution-checkpoint.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossless-contract-v1.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossy-contract-v1.json"],"record_sha256":"c94ea829dfd171828f98383aa99c28fda658ecec15f2b5041e0d4870eac47dad"},"id":"76a925db9dd67c78a05e250269087c97d3ef66128fa053d7674bab13a3457ae3","identity":"trial:c22-video-pre-gate-secant-quality-5s-00-72202558","kind":"attempt_observation","source_record":"h3-ref2va-910c-4logical-20261101-297/results/c12-through-c22-execution-checkpoint.json#/runs/32","source_records":["h3-ref2va-910c-4logical-20261101-297/results/c12-through-c22-execution-checkpoint.json#/runs/32","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c23-execution-checkpoint.json#/runs/32"],"status":"failed"} +{"artifact_id":"h3-ref2va-910c-4logical-20261101-297-6c7e237-v1","conclusion":"{\"scope\":\"focused tests only; no full-model quality or performance acceptance\"}","configuration":{"campaign_records":["h3-ref2va-910c-4logical-20261101-297/plan/execution-checkpoint.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossless-contract-v1.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossy-contract-v1.json"],"record_sha256":"6300acfc1918c6a50b9360d509384f4f638180120735dbb5f9ff73e733c42380"},"id":"76f5ac8f849991924a5ff18deb7e717f64673ddf361046ea5304762d9eea352e","identity":null,"kind":"published_record","source_record":"h3-ref2va-910c-4logical-20261101-297/results/c10-native-tests.json#","source_records":["h3-ref2va-910c-4logical-20261101-297/results/c10-native-tests.json#"],"status":"completed"} +{"artifact_id":"h3-ref2va-910c-4logical-20261101-297-6c7e237-v1","conclusion":"{\"scope\":\"Independent weighted global-context reference and native padded-head BF16 FAS; no full-model quality/performance acceptance\"}","configuration":{"campaign_records":["h3-ref2va-910c-4logical-20261101-297/plan/execution-checkpoint.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossless-contract-v1.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossy-contract-v1.json"],"record_sha256":"1b4ffcecbda5bc60d82c95a6aed4e2879a2fd5a4ebc035b4381fd0119ca5332a"},"id":"77381e61a2c56ec599d02811dbdf73287fc2cb3928af5ffe974905c783089da1","identity":"trial:c20-video-variance-pool-native-50101063","kind":"attempt_observation","source_record":"h3-ref2va-910c-4logical-20261101-297/results/c20-native-qualification.json#","source_records":["h3-ref2va-910c-4logical-20261101-297/results/c20-native-qualification.json#"],"status":"recorded"} +{"artifact_id":"h3-ref2va-910c-4logical-20261101-297-6c7e237-v1","conclusion":"{\"scope\":\"Focused state/block tests and native four-rank subset attention; no full-model quality/performance acceptance\"}","configuration":{"campaign_records":["h3-ref2va-910c-4logical-20261101-297/plan/execution-checkpoint.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossless-contract-v1.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossy-contract-v1.json"],"record_sha256":"a574dc9bd57d88d9cdefe57f589e20586c82ae721a239374c310434a84bffd52"},"id":"78ed31225676bea83b53a8171aead7678b05bae25f8a3c8adc7fb3cceb82fdb3","identity":"trial:c16-video-row-fp32-native-d3fe320c","kind":"attempt_observation","source_record":"h3-ref2va-910c-4logical-20261101-297/results/c12-through-c17-execution-checkpoint.json#/runs/18","source_records":["h3-ref2va-910c-4logical-20261101-297/results/c12-through-c17-execution-checkpoint.json#/runs/18","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c19-execution-checkpoint.json#/runs/18","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c20-execution-checkpoint.json#/runs/18","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c21-execution-checkpoint.json#/runs/18","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c22-execution-checkpoint.json#/runs/18","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c23-execution-checkpoint.json#/runs/18"],"status":"completed"} +{"artifact_id":"h3-ref2va-910c-4logical-20261101-297-6c7e237-v1","conclusion":"{\"scope\":\"Legacy/pre-gate/DLO tests, two executed native four-rank HCCL/subsetFA suites and fresh current gates; no full-model quality/performance acceptance\"}","configuration":{"campaign_records":["h3-ref2va-910c-4logical-20261101-297/plan/execution-checkpoint.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossless-contract-v1.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossy-contract-v1.json"],"record_sha256":"46bfa8b376d308a73a106199f0c1401135953afbeadb9bc9a146393023634d65"},"id":"7ae7eee8902b05b6b6530511ace835775e1bce7c2214c1ca5fdaed2a4850f58f","identity":"trial:c21-video-pre-gate-native-3c5eebbe","kind":"attempt_observation","source_record":"h3-ref2va-910c-4logical-20261101-297/results/c12-through-c21-execution-checkpoint.json#/runs/29","source_records":["h3-ref2va-910c-4logical-20261101-297/results/c12-through-c21-execution-checkpoint.json#/runs/29","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c22-execution-checkpoint.json#/runs/29","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c23-execution-checkpoint.json#/runs/29"],"status":"completed"} +{"artifact_id":"h3-ref2va-910c-4logical-20261101-297-6c7e237-v1","conclusion":"INCONCLUSIVE: no standalone verdict recorded","configuration":{"campaign_records":["h3-ref2va-910c-4logical-20261101-297/plan/execution-checkpoint.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossless-contract-v1.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossy-contract-v1.json"],"record_sha256":"ab5c6560f94b6c5bd11d2831db1d29dd54186ecb18f59574684d69aba287d731"},"id":"80948ab372429ced75ee17bf7b3159c593d31138528b300b36fd4993894f7d22","identity":"trial:c16-video-row-fp32-quality-5s-00-8600757a","kind":"attempt_observation","source_record":"h3-ref2va-910c-4logical-20261101-297/results/c12-through-c17-execution-checkpoint.json#/runs/19","source_records":["h3-ref2va-910c-4logical-20261101-297/results/c12-through-c17-execution-checkpoint.json#/runs/19","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c19-execution-checkpoint.json#/runs/19","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c20-execution-checkpoint.json#/runs/19","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c21-execution-checkpoint.json#/runs/19","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c22-execution-checkpoint.json#/runs/19","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c23-execution-checkpoint.json#/runs/19"],"status":"completed"} +{"artifact_id":"h3-ref2va-910c-4logical-20261101-297-6c7e237-v1","conclusion":"{\"scope\":\"focused tests only; no full-model quality or performance acceptance\"}","configuration":{"campaign_records":["h3-ref2va-910c-4logical-20261101-297/plan/execution-checkpoint.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossless-contract-v1.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossy-contract-v1.json"],"record_sha256":"a07ece96c80cc981929b1de2a527543459ed82225b246914f6313533684bb72f"},"id":"8193f7f989b4c6891afbf0be00cff152e89d3f315fa026f7240ded4683f7d446","identity":"trial:c12-state-alignment-native-tests-0dc5859b","kind":"attempt_observation","source_record":"h3-ref2va-910c-4logical-20261101-297/results/c12-through-c15-execution-checkpoint.json#/runs/1","source_records":["h3-ref2va-910c-4logical-20261101-297/results/c12-through-c15-execution-checkpoint.json#/runs/1","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c17-execution-checkpoint.json#/runs/1","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c19-execution-checkpoint.json#/runs/1","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c20-execution-checkpoint.json#/runs/1","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c21-execution-checkpoint.json#/runs/1","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c22-execution-checkpoint.json#/runs/1","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c23-execution-checkpoint.json#/runs/1"],"status":"completed"} +{"artifact_id":"h3-ref2va-910c-4logical-20261101-297-6c7e237-v1","conclusion":"{\"scope\":\"Focused state/block tests and native four-rank subset attention; no full-model quality/performance acceptance\"}","configuration":{"campaign_records":["h3-ref2va-910c-4logical-20261101-297/plan/execution-checkpoint.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossless-contract-v1.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossy-contract-v1.json"],"record_sha256":"c6e1a9a94af15f6527c7a5684bdd3fc08de29c92fd6d75cdc647fe3c009075fb"},"id":"8829f99ddc5b970d9bafbc9b6a22a74625ef7c5f2903d873f68f5d21cabc115e","identity":"trial:c15-video-row-native-066a9a40","kind":"attempt_observation","source_record":"h3-ref2va-910c-4logical-20261101-297/results/c12-through-c15-execution-checkpoint.json#/runs/11","source_records":["h3-ref2va-910c-4logical-20261101-297/results/c12-through-c15-execution-checkpoint.json#/runs/11","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c17-execution-checkpoint.json#/runs/11","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c19-execution-checkpoint.json#/runs/11","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c20-execution-checkpoint.json#/runs/11","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c21-execution-checkpoint.json#/runs/11","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c22-execution-checkpoint.json#/runs/11","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c23-execution-checkpoint.json#/runs/11"],"status":"failed"} +{"artifact_id":"h3-ref2va-910c-4logical-20261101-297-6c7e237-v1","conclusion":"{\"campaign_verdict\":\"INCONCLUSIVE\",\"decision\":\"Both nativePFA layouts are non-bit-exact and slower by19.4/20.6%; reject,do not integrate.\",\"scope\":\"Single real5s first-long-attention fixture,one logical device; not model performance\"}","configuration":{"campaign_records":["h3-ref2va-910c-4logical-20261101-297/plan/execution-checkpoint.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossless-contract-v1.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossy-contract-v1.json"],"record_sha256":"91c47092f589784245350937343a56ca20cfacbcc2c672357ae9857863fec159"},"id":"8a48d9608fcc01b958bdba9a7f6a7c800bf9c2c2429a7d5cfb79c614bda68571","identity":null,"kind":"published_record","source_record":"h3-ref2va-910c-4logical-20261101-297/results/c4-pfa-operator.json#","source_records":["h3-ref2va-910c-4logical-20261101-297/results/c4-pfa-operator.json#"],"status":"recorded"} +{"artifact_id":"h3-ref2va-910c-4logical-20261101-297-6c7e237-v1","conclusion":"INCONCLUSIVE: no standalone verdict recorded","configuration":{"campaign_records":["h3-ref2va-910c-4logical-20261101-297/plan/execution-checkpoint.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossless-contract-v1.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossy-contract-v1.json"],"record_sha256":"52e037d467cd95ea250000c0798d3d8f81d48d1c8b0bdae00cd2b099cd27a76f"},"id":"8cd394c01c0f3d82427bfe3d1dd87050488e3ceb1689865b841c83217dda28e1","identity":"trial:c13-c1-dlo-5s-audit-v2-b0b86c6d","kind":"attempt_observation","source_record":"h3-ref2va-910c-4logical-20261101-297/results/c12-through-c15-execution-checkpoint.json#/runs/4","source_records":["h3-ref2va-910c-4logical-20261101-297/results/c12-through-c15-execution-checkpoint.json#/runs/4","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c17-execution-checkpoint.json#/runs/4","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c19-execution-checkpoint.json#/runs/4","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c20-execution-checkpoint.json#/runs/4","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c21-execution-checkpoint.json#/runs/4","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c22-execution-checkpoint.json#/runs/4","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c23-execution-checkpoint.json#/runs/4"],"status":"completed"} +{"artifact_id":"h3-ref2va-910c-4logical-20261101-297-6c7e237-v1","conclusion":"INCONCLUSIVE: no standalone verdict recorded","configuration":{"campaign_records":["h3-ref2va-910c-4logical-20261101-297/plan/execution-checkpoint.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossless-contract-v1.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossy-contract-v1.json"],"record_sha256":"9d3465ca68f9517322da904fb13177e0cf0d2b5d1ab63421810c020dd75e2e4b"},"id":"91af9f7f9967c41da1a1f7cec06b1da5cf9974296b837a111ce0d61e92634a37","identity":"trial:c11-sigma-fp32-native-tests-b873db4a","kind":"attempt_observation","source_record":"h3-ref2va-910c-4logical-20261101-297/results/c11-fp32-native-execution.json#/attempts/0","source_records":["h3-ref2va-910c-4logical-20261101-297/results/c11-fp32-native-execution.json#/attempts/0"],"status":"failed"} +{"artifact_id":"h3-ref2va-910c-4logical-20261101-297-6c7e237-v1","conclusion":"{\"scope\":\"Real captured H3 attention operator benchmark; full-model quality and HTTP are separate; no full-model quality/performance acceptance\"}","configuration":{"campaign_records":["h3-ref2va-910c-4logical-20261101-297/plan/execution-checkpoint.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossless-contract-v1.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossy-contract-v1.json"],"record_sha256":"e2e5469c7cc2787bc58ac328308feb151b71d855f2d1c015c9b876b00883073b"},"id":"9228870a025961a61627f17346457784de3ac7303b803a4cf3a395a474c5c23e","identity":"trial:c23-video-context-lse-v2-operator-5614bf46","kind":"attempt_observation","source_record":"h3-ref2va-910c-4logical-20261101-297/results/c12-through-c23-execution-checkpoint.json#/runs/39","source_records":["h3-ref2va-910c-4logical-20261101-297/results/c12-through-c23-execution-checkpoint.json#/runs/39"],"status":"completed"} +{"artifact_id":"h3-ref2va-910c-4logical-20261101-297-6c7e237-v1","conclusion":"INCONCLUSIVE: no standalone verdict recorded","configuration":{"campaign_records":["h3-ref2va-910c-4logical-20261101-297/plan/execution-checkpoint.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossless-contract-v1.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossy-contract-v1.json"],"record_sha256":"da330f8b2c18872e7d88a54294ab84bbc7058103d3aaa7a642e79397703d8600"},"id":"971405a272d41ce69fd95dce9bef9c2a6faa315a37e59f390014adfaea2050f7","identity":"trial:c15-video-row-quality-5s-00-710315e0","kind":"attempt_observation","source_record":"h3-ref2va-910c-4logical-20261101-297/results/c12-through-c17-execution-checkpoint.json#/runs/15","source_records":["h3-ref2va-910c-4logical-20261101-297/results/c12-through-c17-execution-checkpoint.json#/runs/15","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c19-execution-checkpoint.json#/runs/15","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c20-execution-checkpoint.json#/runs/15","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c21-execution-checkpoint.json#/runs/15","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c22-execution-checkpoint.json#/runs/15","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c23-execution-checkpoint.json#/runs/15"],"status":"failed"} +{"artifact_id":"h3-ref2va-910c-4logical-20261101-297-6c7e237-v1","conclusion":"{\"scope\":\"Reproduce stock CUDA-graph staging expectation on unchanged migration owning source in the same Ascend test environment\"}","configuration":{"campaign_records":["h3-ref2va-910c-4logical-20261101-297/plan/execution-checkpoint.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossless-contract-v1.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossy-contract-v1.json"],"record_sha256":"b26122995208876f8a5d29af6ea6b7826c8d95292b6527f969ada15987859f11"},"id":"9ae1fa85af839acec356b0657fdff122a8e54340f3cf14e1dfba8b37b3a1911c","identity":"trial:c13-baseline-cuda-staging-e22d8651","kind":"attempt_observation","source_record":"h3-ref2va-910c-4logical-20261101-297/results/c12-through-c15-execution-checkpoint.json#/runs/2","source_records":["h3-ref2va-910c-4logical-20261101-297/results/c12-through-c15-execution-checkpoint.json#/runs/2","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c17-execution-checkpoint.json#/runs/2","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c19-execution-checkpoint.json#/runs/2","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c20-execution-checkpoint.json#/runs/2","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c21-execution-checkpoint.json#/runs/2","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c22-execution-checkpoint.json#/runs/2","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c23-execution-checkpoint.json#/runs/2"],"status":"failed"} +{"artifact_id":"h3-ref2va-910c-4logical-20261101-297-6c7e237-v1","conclusion":"INCONCLUSIVE: no standalone verdict recorded","configuration":{"campaign_records":["h3-ref2va-910c-4logical-20261101-297/plan/execution-checkpoint.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossless-contract-v1.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossy-contract-v1.json"],"record_sha256":"593f56fd72523596625c7eee8009fd9f4a986908cb6be999efc3661c1a604599"},"id":"9bb35d6b00f7c9253a711525ed958ae095b33db58f33ac219b0028dd12d2e4c1","identity":"trial:c21-video-pre-gate-quality-5s-00-7265c33e","kind":"attempt_observation","source_record":"h3-ref2va-910c-4logical-20261101-297/results/c12-through-c21-execution-checkpoint.json#/runs/30","source_records":["h3-ref2va-910c-4logical-20261101-297/results/c12-through-c21-execution-checkpoint.json#/runs/30","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c22-execution-checkpoint.json#/runs/30","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c23-execution-checkpoint.json#/runs/30"],"status":"completed"} +{"artifact_id":"h3-ref2va-910c-4logical-20261101-297-6c7e237-v1","conclusion":"{\"scope\":\"initialnoise and fullcompute poststep0 only; not later cached states, fullquality or timing acceptance\"}","configuration":{"campaign_records":["h3-ref2va-910c-4logical-20261101-297/plan/execution-checkpoint.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossless-contract-v1.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossy-contract-v1.json"],"record_sha256":"e867abfa4de5e3ccb3f613be26eb44a0c7412e90d23b65c8392d03662f6df36a"},"id":"a0522ee89b1a426fff29b63215f535444b611b87ca789d70b0da37756812814b","identity":null,"kind":"published_record","source_record":"h3-ref2va-910c-4logical-20261101-297/results/c11-fp32-5s-first-step.json#","source_records":["h3-ref2va-910c-4logical-20261101-297/results/c11-fp32-5s-first-step.json#"],"status":"recorded"} +{"artifact_id":"h3-ref2va-910c-4logical-20261101-297-6c7e237-v1","conclusion":"INCONCLUSIVE: no standalone verdict recorded","configuration":{"campaign_records":["h3-ref2va-910c-4logical-20261101-297/plan/execution-checkpoint.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossless-contract-v1.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossy-contract-v1.json"],"record_sha256":"befb971b0328e119eb562a1230efd684f0fe50859abf72777f7c53254f26fcfa"},"id":"a4bbcdeb60c5560c2f1c087647933e3cd3800dbe7f31abed3b5d1125e1c0b7b6","identity":"trial:c11-sigma-fp32-native-tests-9542a13d","kind":"attempt_observation","source_record":"h3-ref2va-910c-4logical-20261101-297/results/c11-fp32-native-execution.json#/attempts/2","source_records":["h3-ref2va-910c-4logical-20261101-297/results/c11-fp32-native-execution.json#/attempts/2"],"status":"completed"} +{"artifact_id":"h3-ref2va-910c-4logical-20261101-297-6c7e237-v1","conclusion":"{\"scope\":\"Legacy/pre-gate/DLO tests, three executed native four-rank HCCL/subsetFA suites and fresh current gates; no full-model quality/performance acceptance\"}","configuration":{"campaign_records":["h3-ref2va-910c-4logical-20261101-297/plan/execution-checkpoint.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossless-contract-v1.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossy-contract-v1.json"],"record_sha256":"f38319c16210ed8e7548e0b9bbe6fb485139abeddb8beafbf84d327af5b7d715"},"id":"aad073cafc71bbe0543f4ac5c7134ee0a810b67f90a41b7aff5699aced4d29d8","identity":"trial:c23-video-context-lse-v2-native-dc375564","kind":"attempt_observation","source_record":"h3-ref2va-910c-4logical-20261101-297/results/c12-through-c23-execution-checkpoint.json#/runs/38","source_records":["h3-ref2va-910c-4logical-20261101-297/results/c12-through-c23-execution-checkpoint.json#/runs/38"],"status":"completed"} +{"artifact_id":"h3-ref2va-910c-4logical-20261101-297-6c7e237-v1","conclusion":"{\"campaign_verdict\":\"INCONCLUSIVE\",\"decision\":\"All three variants are bit-exact but slower; do not integrate.\",\"scope\":\"Single real5s first-long-attention fixture,logical device count1; not whole-model performance\"}","configuration":{"campaign_records":["h3-ref2va-910c-4logical-20261101-297/plan/execution-checkpoint.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossless-contract-v1.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossy-contract-v1.json"],"record_sha256":"a7df4b4273c713c0dbaede4e2124f2be524765bc18558f0a067ebbe429ae43a8"},"id":"abba0a05d9a14a18efd39043a628e57f866a92409461c816e17b6e9479ea8eac","identity":null,"kind":"published_record","source_record":"h3-ref2va-910c-4logical-20261101-297/results/c1-queryfold-operator.json#","source_records":["h3-ref2va-910c-4logical-20261101-297/results/c1-queryfold-operator.json#"],"status":"recorded"} +{"artifact_id":"h3-ref2va-910c-4logical-20261101-297-6c7e237-v1","conclusion":"INCONCLUSIVE: no standalone verdict recorded","configuration":{"campaign_records":["h3-ref2va-910c-4logical-20261101-297/plan/execution-checkpoint.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossless-contract-v1.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossy-contract-v1.json"],"record_sha256":"47c82534c68f02d6404dbc01564dacebcf86c822298601d34807f1ed9cf4d3c6"},"id":"aca719e74a98f5a138af7908414cb890e9dfb7d78f0e3d228d8b987868752904","identity":"trial:c22-video-pre-gate-secant-v2-quality-5s-00-777147cb","kind":"attempt_observation","source_record":"h3-ref2va-910c-4logical-20261101-297/results/c12-through-c23-execution-checkpoint.json#/runs/33","source_records":["h3-ref2va-910c-4logical-20261101-297/results/c12-through-c23-execution-checkpoint.json#/runs/33"],"status":"completed"} +{"artifact_id":"h3-ref2va-910c-4logical-20261101-297-6c7e237-v1","conclusion":"INCONCLUSIVE: no standalone verdict recorded","configuration":{"campaign_records":["h3-ref2va-910c-4logical-20261101-297/plan/execution-checkpoint.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossless-contract-v1.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossy-contract-v1.json"],"record_sha256":"2a212b7cfa8f14e90f88279a5913f28054b6955c9eb0a31a7eb389f696bc7833"},"id":"ad7f0ac492a675abc493d08457cc265681de0c9603c59ed816357fe65f8245f3","identity":"trial:c15-video-row-v2-quality-5s-00-d6b921b9","kind":"attempt_observation","source_record":"h3-ref2va-910c-4logical-20261101-297/results/c12-through-c17-execution-checkpoint.json#/runs/16","source_records":["h3-ref2va-910c-4logical-20261101-297/results/c12-through-c17-execution-checkpoint.json#/runs/16","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c19-execution-checkpoint.json#/runs/16","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c20-execution-checkpoint.json#/runs/16","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c21-execution-checkpoint.json#/runs/16","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c22-execution-checkpoint.json#/runs/16","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c23-execution-checkpoint.json#/runs/16"],"status":"failed"} +{"artifact_id":"h3-ref2va-910c-4logical-20261101-297-6c7e237-v1","conclusion":"{\"scope\":\"CPU mock barrier/chunk tests; no actual NPU collective or full-model quality/performance acceptance\"}","configuration":{"campaign_records":["h3-ref2va-910c-4logical-20261101-297/plan/execution-checkpoint.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossless-contract-v1.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossy-contract-v1.json"],"record_sha256":"c265df029909d749b78f4839d7be322b5ecba9457854520c9f5027abc236f6a6"},"id":"add6d6f2ee363dd58360673a0ed69f3de0375792e0faf02647c48871e8e190bd","identity":"trial:c13-dlo-gate-tests-bc92a71d","kind":"attempt_observation","source_record":"h3-ref2va-910c-4logical-20261101-297/results/c12-through-c15-execution-checkpoint.json#/runs/6","source_records":["h3-ref2va-910c-4logical-20261101-297/results/c12-through-c15-execution-checkpoint.json#/runs/6","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c17-execution-checkpoint.json#/runs/6","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c19-execution-checkpoint.json#/runs/6","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c20-execution-checkpoint.json#/runs/6","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c21-execution-checkpoint.json#/runs/6","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c22-execution-checkpoint.json#/runs/6","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c23-execution-checkpoint.json#/runs/6"],"status":"completed"} +{"artifact_id":"h3-ref2va-910c-4logical-20261101-297-6c7e237-v1","conclusion":"INCONCLUSIVE: no standalone verdict recorded","configuration":{"campaign_records":["h3-ref2va-910c-4logical-20261101-297/plan/execution-checkpoint.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossless-contract-v1.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossy-contract-v1.json"],"record_sha256":"36888cb5ee7fbd886422a665b6549e19ebfa1a4f8e35c3b349366543bfd95384"},"id":"ae0729d011929a37341372d2e664912f713c60e26ea5a368658a67565fff5b9b","identity":"trial:c13-c1-dlo-5s-calibration-c98129d8","kind":"attempt_observation","source_record":"h3-ref2va-910c-4logical-20261101-297/results/c12-through-c15-execution-checkpoint.json#/runs/5","source_records":["h3-ref2va-910c-4logical-20261101-297/results/c12-through-c15-execution-checkpoint.json#/runs/5","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c17-execution-checkpoint.json#/runs/5","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c19-execution-checkpoint.json#/runs/5","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c20-execution-checkpoint.json#/runs/5","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c21-execution-checkpoint.json#/runs/5","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c22-execution-checkpoint.json#/runs/5","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c23-execution-checkpoint.json#/runs/5"],"status":"completed"} +{"artifact_id":"h3-ref2va-910c-4logical-20261101-297-6c7e237-v1","conclusion":"{\"scope\":\"CPU window reference/routing and native batched BF16 FAS; no full-model quality/performance acceptance\"}","configuration":{"campaign_records":["h3-ref2va-910c-4logical-20261101-297/plan/execution-checkpoint.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossless-contract-v1.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossy-contract-v1.json"],"record_sha256":"a96495e199903f7770d1f3e1368b684a754f688caf433e27c2b22eac1e2d01d3"},"id":"b1b8d0849c13af6cb524257187bb86d67dcf1da2972515080438bbd73a92fad2","identity":"trial:c17-video-window-native-9b48cb56","kind":"attempt_observation","source_record":"h3-ref2va-910c-4logical-20261101-297/results/c12-through-c17-execution-checkpoint.json#/runs/20","source_records":["h3-ref2va-910c-4logical-20261101-297/results/c12-through-c17-execution-checkpoint.json#/runs/20","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c19-execution-checkpoint.json#/runs/20","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c20-execution-checkpoint.json#/runs/20","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c21-execution-checkpoint.json#/runs/20","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c22-execution-checkpoint.json#/runs/20","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c23-execution-checkpoint.json#/runs/20"],"status":"completed"} +{"artifact_id":"h3-ref2va-910c-4logical-20261101-297-6c7e237-v1","conclusion":"{\"campaign_verdict\":\"INCONCLUSIVE\",\"quality_verdict\":\"INCONCLUSIVE\",\"scope\":\"http_complete_mp4\"}","configuration":{"campaign_records":["h3-ref2va-910c-4logical-20261101-297/plan/execution-checkpoint.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossless-contract-v1.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossy-contract-v1.json"],"record_sha256":"36733817a326df2463b46ed6ec081936d0937dfbcb2b45f3d0ba959163237747"},"id":"b2a56936053912cf2e6a67a1f2673a82c5a6f71c94b9fcedcbd66ae2a7103fae","identity":null,"kind":"published_record","source_record":"h3-ref2va-910c-4logical-20261101-297/results/c10-5s-01-audit.json#","source_records":["h3-ref2va-910c-4logical-20261101-297/results/c10-5s-01-audit.json#"],"status":"recorded"} +{"artifact_id":"h3-ref2va-910c-4logical-20261101-297-6c7e237-v1","conclusion":"{\"scope\":\"Legacy/pre-gate/DLO tests, three executed native four-rank HCCL/subsetFA suites and fresh current gates; no full-model quality/performance acceptance\"}","configuration":{"campaign_records":["h3-ref2va-910c-4logical-20261101-297/plan/execution-checkpoint.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossless-contract-v1.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossy-contract-v1.json"],"record_sha256":"af7493eca4425e14f790e8997711a7d2aff0e4ca9463f68be372b3d697b3c9fe"},"id":"b4259f9804d3ff3661ec2bb02291910e10cebf32d066155702d213163073c0fe","identity":"trial:c22-video-pre-gate-secant-native-3521747e","kind":"attempt_observation","source_record":"h3-ref2va-910c-4logical-20261101-297/results/c22-native-qualification.json#","source_records":["h3-ref2va-910c-4logical-20261101-297/results/c22-native-qualification.json#"],"status":"recorded"} +{"artifact_id":"h3-ref2va-910c-4logical-20261101-297-6c7e237-v1","conclusion":"{\"scope\":\"specified latent points, schedule, decoded RGB24/PCM; no performance acceptance\",\"verdict\":\"PASS\"}","configuration":{"campaign_records":["h3-ref2va-910c-4logical-20261101-297/plan/execution-checkpoint.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossless-contract-v1.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossy-contract-v1.json"],"record_sha256":"5833621c59968784b613b58e38c509a7bcc87888621c576c065e6b62cdd8a026"},"id":"b929a136713b599a77f2a23bfa6301fbccfded8ce7330b9b992e92944485e5fe","identity":null,"kind":"published_record","source_record":"h3-ref2va-910c-4logical-20261101-297/results/c13-5s-strict-comparison.json#","source_records":["h3-ref2va-910c-4logical-20261101-297/results/c13-5s-strict-comparison.json#"],"status":"recorded"} +{"artifact_id":"h3-ref2va-910c-4logical-20261101-297-6c7e237-v1","conclusion":"{\"scope\":\"Retire rejected C23 fixed profile; preserve evidence and original budget. No new search direction launched.\"}","configuration":{"campaign_records":["h3-ref2va-910c-4logical-20261101-297/plan/execution-checkpoint.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossless-contract-v1.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossy-contract-v1.json"],"record_sha256":"6c91a89ebe85fed258f1a4b5ec9676a9175c252e67511f1830fa4d142c801eb9"},"id":"b97288ca9821ddd768b82feedf952d18e7da8f076a3cd7f1cb61d89e8f2e2c25","identity":null,"kind":"published_record","source_record":"h3-ref2va-910c-4logical-20261101-297/results/c23-retired-proof.json#","source_records":["h3-ref2va-910c-4logical-20261101-297/results/c23-retired-proof.json#"],"status":"recorded"} +{"artifact_id":"h3-ref2va-910c-4logical-20261101-297-6c7e237-v1","conclusion":"{\"scope\":\"Real captured H3 attention operator benchmark; full-model quality and HTTP are separate; no full-model quality/performance acceptance\"}","configuration":{"campaign_records":["h3-ref2va-910c-4logical-20261101-297/plan/execution-checkpoint.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossless-contract-v1.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossy-contract-v1.json"],"record_sha256":"947cacc6502f5c0e4da2d4e8c525d5ffc519216311f6867f16dc669315755568"},"id":"bc0fffb798bd08f1700d0eaf3894ddd728c4850e626fc79f6f61a4cc1e2f89d0","identity":"trial:c17-video-window-operator-9bf7de3e","kind":"attempt_observation","source_record":"h3-ref2va-910c-4logical-20261101-297/results/c12-through-c23-execution-checkpoint.json#/runs/21","source_records":["h3-ref2va-910c-4logical-20261101-297/results/c12-through-c23-execution-checkpoint.json#/runs/21"],"status":"completed"} +{"artifact_id":"h3-ref2va-910c-4logical-20261101-297-6c7e237-v1","conclusion":"{\"campaign_verdict\":\"INCONCLUSIVE\"}","configuration":{"campaign_records":["h3-ref2va-910c-4logical-20261101-297/plan/execution-checkpoint.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossless-contract-v1.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossy-contract-v1.json"],"record_sha256":"f4bfd1a22466ffd4a10d8f820736514ecf06897ac439cc358cbcefeab5f22b1d"},"id":"bdc9c1bf531e7975090d51b89fa51f106a422eccf34790d16eaf769673eb7d51","identity":null,"kind":"published_record","source_record":"h3-ref2va-910c-4logical-20261101-297/results/c12-through-c19-execution-checkpoint.json#","source_records":["h3-ref2va-910c-4logical-20261101-297/results/c12-through-c19-execution-checkpoint.json#"],"status":"recorded"} +{"artifact_id":"h3-ref2va-910c-4logical-20261101-297-6c7e237-v1","conclusion":"{\"scope\":\"Real captured H3 attention operator benchmark; full-model quality and HTTP are separate; no full-model quality/performance acceptance\"}","configuration":{"campaign_records":["h3-ref2va-910c-4logical-20261101-297/plan/execution-checkpoint.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossless-contract-v1.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossy-contract-v1.json"],"record_sha256":"984fd1f5e284d6d0838a0e0c9025bf5d33d1009bdd13018925a69cf2c4977c12"},"id":"bec94cd6c61927e4d1d35bfb163ec936c7a681b921bf52805544d60f0d5bcb77","identity":"trial:c20-video-variance-pool-operator-209f263d","kind":"attempt_observation","source_record":"h3-ref2va-910c-4logical-20261101-297/results/c12-through-c20-execution-checkpoint.json#/runs/28","source_records":["h3-ref2va-910c-4logical-20261101-297/results/c12-through-c20-execution-checkpoint.json#/runs/28","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c21-execution-checkpoint.json#/runs/28","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c22-execution-checkpoint.json#/runs/28"],"status":"completed"} +{"artifact_id":"h3-ref2va-910c-4logical-20261101-297-6c7e237-v1","conclusion":"{\"decision\":\"Reject15s from its own five observed violations. Known5s failure is retained separately.\",\"duration_verdict\":\"FAIL\"}","configuration":{"campaign_records":["h3-ref2va-910c-4logical-20261101-297/plan/execution-checkpoint.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossless-contract-v1.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossy-contract-v1.json"],"record_sha256":"f5a510519467f83f5e7ce4f238671c4a946e8abd3e282dd783ceaa43ffbe7926"},"id":"c02e766d7c6b0837cc4e212088264dae91223e09e7a6e53bb786f3be28342da0","identity":"trial:c11-sigma-fp32-quality-15s-00-aaa2b0d6","kind":"attempt_observation","source_record":"h3-ref2va-910c-4logical-20261101-297/results/c11-fp32-15s-00-audit.json#","source_records":["h3-ref2va-910c-4logical-20261101-297/results/c11-fp32-15s-00-audit.json#"],"status":"recorded"} +{"artifact_id":"h3-ref2va-910c-4logical-20261101-297-6c7e237-v1","conclusion":"{\"scope\":\"Bounded native prerequisites only; a failed probe is retained as negative evidence.\"}","configuration":{"campaign_records":["h3-ref2va-910c-4logical-20261101-297/plan/execution-checkpoint.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossless-contract-v1.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossy-contract-v1.json"],"record_sha256":"7d6a5c95b3c51f3d7c8a72de9b28f8164e1b41734a158080ba08e143dea66a60"},"id":"c0b8999cde4c090e29ffe64d92764e2c991ca0fda5bc65f7283481328dc74f0b","identity":"trial:c14-quant-prerequisites-358b9b28","kind":"attempt_observation","source_record":"h3-ref2va-910c-4logical-20261101-297/results/c12-through-c15-execution-checkpoint.json#/runs/10","source_records":["h3-ref2va-910c-4logical-20261101-297/results/c12-through-c15-execution-checkpoint.json#/runs/10","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c17-execution-checkpoint.json#/runs/10","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c19-execution-checkpoint.json#/runs/10","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c20-execution-checkpoint.json#/runs/10","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c21-execution-checkpoint.json#/runs/10","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c22-execution-checkpoint.json#/runs/10","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c23-execution-checkpoint.json#/runs/10"],"status":"failed"} +{"artifact_id":"h3-ref2va-910c-4logical-20261101-297-6c7e237-v1","conclusion":"{\"scope\":\"Legacy/pre-gate/DLO tests, three executed native four-rank HCCL/subsetFA suites and fresh current gates; no full-model quality/performance acceptance\"}","configuration":{"campaign_records":["h3-ref2va-910c-4logical-20261101-297/plan/execution-checkpoint.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossless-contract-v1.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossy-contract-v1.json"],"record_sha256":"785b252151651d6aca8ab610b330746da7c1ca4b0e079fd1e9da8e2554e18010"},"id":"c9f0630433473dfb55e4cfe0e0673605afaa6fad616979c93ee1f23efc58d50b","identity":"trial:c23-video-context-lse-native-ef09a725","kind":"attempt_observation","source_record":"h3-ref2va-910c-4logical-20261101-297/results/c12-through-c23-execution-checkpoint.json#/runs/35","source_records":["h3-ref2va-910c-4logical-20261101-297/results/c12-through-c23-execution-checkpoint.json#/runs/35"],"status":"completed"} +{"artifact_id":"h3-ref2va-910c-4logical-20261101-297-6c7e237-v1","conclusion":"{\"scope\":\"CPU window reference/routing and native batched BF16 FAS; no full-model quality/performance acceptance\"}","configuration":{"campaign_records":["h3-ref2va-910c-4logical-20261101-297/plan/execution-checkpoint.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossless-contract-v1.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossy-contract-v1.json"],"record_sha256":"dcf98f80d5f3485cac2e05459e72d5cdae9a043a84618b894fbefa827bd86ccb"},"id":"cc048b49645a60e8a9393d7298f6b0abde86b6ac677126f1308b58ae4a734dab","identity":"trial:c17-video-window-native-9b48cb56","kind":"attempt_observation","source_record":"h3-ref2va-910c-4logical-20261101-297/results/c17-native-qualification.json#","source_records":["h3-ref2va-910c-4logical-20261101-297/results/c17-native-qualification.json#"],"status":"recorded"} +{"artifact_id":"h3-ref2va-910c-4logical-20261101-297-6c7e237-v1","conclusion":"{\"scope\":\"Real captured H3 attention operator benchmark; full-model quality and HTTP are separate; no full-model quality/performance acceptance\"}","configuration":{"campaign_records":["h3-ref2va-910c-4logical-20261101-297/plan/execution-checkpoint.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossless-contract-v1.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossy-contract-v1.json"],"record_sha256":"884b28cf6c257b05fbe94f2b3ae15b869ffc90159810ea709998279f27773145"},"id":"d5d4c4dd353e65a7085f47c5c8dc5a73506ef66db40deb41d201af5343bedb8e","identity":"trial:c19-video-global-pool-operator-933511c9","kind":"attempt_observation","source_record":"h3-ref2va-910c-4logical-20261101-297/results/c12-through-c23-execution-checkpoint.json#/runs/26","source_records":["h3-ref2va-910c-4logical-20261101-297/results/c12-through-c23-execution-checkpoint.json#/runs/26"],"status":"completed"} +{"artifact_id":"h3-ref2va-910c-4logical-20261101-297-6c7e237-v1","conclusion":"{\"scope\":\"15s initialnoise and full-compute poststep0 only; not later cachedstates, fullquality or timing acceptance\"}","configuration":{"campaign_records":["h3-ref2va-910c-4logical-20261101-297/plan/execution-checkpoint.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossless-contract-v1.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossy-contract-v1.json"],"record_sha256":"bb0ee3d557b64de1b4cb7ca595b460e206b748f4e5b44aa52afa1bf447c4d6e9"},"id":"d8faf27dfd9049a86b7de2268ee954e5dc76516daf15e0555bb81118ce680b2e","identity":null,"kind":"published_record","source_record":"h3-ref2va-910c-4logical-20261101-297/results/c11-fp32-15s-first-step.json#","source_records":["h3-ref2va-910c-4logical-20261101-297/results/c11-fp32-15s-first-step.json#"],"status":"recorded"} +{"artifact_id":"h3-ref2va-910c-4logical-20261101-297-6c7e237-v1","conclusion":"{\"campaign_verdict\":\"INCONCLUSIVE\",\"scope\":\"http_complete_mp4\"}","configuration":{"campaign_records":["h3-ref2va-910c-4logical-20261101-297/plan/execution-checkpoint.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossless-contract-v1.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossy-contract-v1.json"],"record_sha256":"272f5e9f999976f6652e4632cf823a9dce12f124cecf9399a237644338cfe079"},"id":"dbe5498bd9468563e3887e2ebf053de6e64378900372243575d73896ed42946a","identity":null,"kind":"published_record","source_record":"h3-ref2va-910c-4logical-20261101-297/results/c1-15s-execution.json#","source_records":["h3-ref2va-910c-4logical-20261101-297/results/c1-15s-execution.json#"],"status":"recorded"} +{"artifact_id":"h3-ref2va-910c-4logical-20261101-297-6c7e237-v1","conclusion":"INCONCLUSIVE: no standalone verdict recorded","configuration":{"campaign_records":["h3-ref2va-910c-4logical-20261101-297/plan/execution-checkpoint.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossless-contract-v1.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossy-contract-v1.json"],"record_sha256":"627b5b7568f41977e24b8f81ed178f83af210411238fdebcaff40922a364959a"},"id":"de2282a67798d131727fd652d6bcfd5248338c6b4ef428b1b424dcb4317ef64f","identity":"trial:c17-video-window-quality-5s-00-a8c689e4","kind":"attempt_observation","source_record":"h3-ref2va-910c-4logical-20261101-297/results/c12-through-c17-execution-checkpoint.json#/runs/22","source_records":["h3-ref2va-910c-4logical-20261101-297/results/c12-through-c17-execution-checkpoint.json#/runs/22","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c19-execution-checkpoint.json#/runs/22","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c20-execution-checkpoint.json#/runs/22","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c21-execution-checkpoint.json#/runs/22","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c22-execution-checkpoint.json#/runs/22","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c23-execution-checkpoint.json#/runs/22"],"status":"completed"} +{"artifact_id":"h3-ref2va-910c-4logical-20261101-297-6c7e237-v1","conclusion":"{\"scope\":\"Focused state/block tests and native four-rank subset attention; no full-model quality/performance acceptance\"}","configuration":{"campaign_records":["h3-ref2va-910c-4logical-20261101-297/plan/execution-checkpoint.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossless-contract-v1.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossy-contract-v1.json"],"record_sha256":"569dae5314b74d50e2aa97d5594aaa8c815e39e69e361af09bccde958f927b32"},"id":"e0167fffe73af200c66706b46aac2b0f7fcff6e51fd79851ef3386273b357bcc","identity":"trial:c16-video-row-fp32-native-d3fe320c","kind":"attempt_observation","source_record":"h3-ref2va-910c-4logical-20261101-297/results/c16-native-qualification.json#","source_records":["h3-ref2va-910c-4logical-20261101-297/results/c16-native-qualification.json#"],"status":"recorded"} +{"artifact_id":"h3-ref2va-910c-4logical-20261101-297-6c7e237-v1","conclusion":"INCONCLUSIVE: no standalone verdict recorded","configuration":{"campaign_records":["h3-ref2va-910c-4logical-20261101-297/plan/execution-checkpoint.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossless-contract-v1.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossy-contract-v1.json"],"record_sha256":"1e2adc8ac2c944876f6e447f22eb4b1b04d19235348269509fae9d56d51e3c90"},"id":"e2204b9a3ba69d3efdf3b03fc0aca7418ce95e088443df0dc50f7ddd05b0f6a3","identity":"trial:c13-c1-dlo-5s-audit-cead10d1","kind":"attempt_observation","source_record":"h3-ref2va-910c-4logical-20261101-297/results/c12-through-c15-execution-checkpoint.json#/runs/3","source_records":["h3-ref2va-910c-4logical-20261101-297/results/c12-through-c15-execution-checkpoint.json#/runs/3","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c17-execution-checkpoint.json#/runs/3","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c19-execution-checkpoint.json#/runs/3","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c20-execution-checkpoint.json#/runs/3","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c21-execution-checkpoint.json#/runs/3","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c22-execution-checkpoint.json#/runs/3","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c23-execution-checkpoint.json#/runs/3"],"status":"failed"} +{"artifact_id":"h3-ref2va-910c-4logical-20261101-297-6c7e237-v1","conclusion":"{\"scope\":\"static source check only; not an available H3 configuration or a device experiment\"}","configuration":{"campaign_records":["h3-ref2va-910c-4logical-20261101-297/plan/execution-checkpoint.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossless-contract-v1.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossy-contract-v1.json"],"record_sha256":"94576e854e83aae1942cee5a8fb0e239089d03c02317015b0a57699f5afc8618"},"id":"e4663f8584d8baa45e6966338f7061db6ddfb03b0dfc6bee64d725fa3cc5bf4f","identity":null,"kind":"published_record","source_record":"h3-ref2va-910c-4logical-20261101-297/results/installed-foca-source-check.json#","source_records":["h3-ref2va-910c-4logical-20261101-297/results/installed-foca-source-check.json#"],"status":"recorded"} +{"artifact_id":"h3-ref2va-910c-4logical-20261101-297-6c7e237-v1","conclusion":"{\"decision\":\"Reject5s-00 for frozenadherencefailure. No further predictorgrid. 15s requires independent qualification.\",\"quality_verdict\":\"FAIL\"}","configuration":{"campaign_records":["h3-ref2va-910c-4logical-20261101-297/plan/execution-checkpoint.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossless-contract-v1.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossy-contract-v1.json"],"record_sha256":"ab925e42efb9bad2797eff6da10b6de09335ce0e587726f862dce60bb718a50c"},"id":"eddc97f596574687cdac20e89689de21b140613d53961dab177d9fe7891adb91","identity":"trial:c11-sigma-fp32-quality-5s-00-7c7b2032","kind":"attempt_observation","source_record":"h3-ref2va-910c-4logical-20261101-297/results/c11-fp32-5s-00-audit.json#","source_records":["h3-ref2va-910c-4logical-20261101-297/results/c11-fp32-5s-00-audit.json#"],"status":"recorded"} +{"artifact_id":"h3-ref2va-910c-4logical-20261101-297-6c7e237-v1","conclusion":"{\"scope\":\"Real captured H3 attention operator benchmark; full-model quality and HTTP are separate; no full-model quality/performance acceptance\"}","configuration":{"campaign_records":["h3-ref2va-910c-4logical-20261101-297/plan/execution-checkpoint.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossless-contract-v1.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossy-contract-v1.json"],"record_sha256":"8550d61f969a0a12db2dfd9e215a2f27c840b5ead1e5b28d7b2274acbfa53688"},"id":"f01be55edf26f0e7f1231972d1fe4a6048afa4154d4605e73565659c607befb2","identity":"trial:c19-video-global-pool-operator-933511c9","kind":"attempt_observation","source_record":"h3-ref2va-910c-4logical-20261101-297/results/c12-through-c20-execution-checkpoint.json#/runs/26","source_records":["h3-ref2va-910c-4logical-20261101-297/results/c12-through-c20-execution-checkpoint.json#/runs/26","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c21-execution-checkpoint.json#/runs/26","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c22-execution-checkpoint.json#/runs/26"],"status":"completed"} +{"artifact_id":"h3-ref2va-910c-4logical-20261101-297-6c7e237-v1","conclusion":"{\"campaign_verdict\":\"INCONCLUSIVE\",\"decision\":\"Reject original C10 for5s; its first5s-01 observation did not generalize.\",\"quality_verdict\":\"FAIL\",\"scope\":\"http_complete_mp4\"}","configuration":{"campaign_records":["h3-ref2va-910c-4logical-20261101-297/plan/execution-checkpoint.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossless-contract-v1.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossy-contract-v1.json"],"record_sha256":"20145aa52b29573fb378b11f496be9f3478ed59e450561b0d3691b8804105bd8"},"id":"f080031221bd5bf1808a167e30f22ed3821b798383c4a0edd5f852b086241872","identity":null,"kind":"published_record","source_record":"h3-ref2va-910c-4logical-20261101-297/results/c10-5s-00-audit.json#","source_records":["h3-ref2va-910c-4logical-20261101-297/results/c10-5s-00-audit.json#"],"status":"recorded"} +{"artifact_id":"h3-ref2va-910c-4logical-20261101-297-6c7e237-v1","conclusion":"{\"scope\":\"Legacy/pre-gate/DLO tests, three executed native four-rank HCCL/subsetFA suites and fresh current gates; no full-model quality/performance acceptance\"}","configuration":{"campaign_records":["h3-ref2va-910c-4logical-20261101-297/plan/execution-checkpoint.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossless-contract-v1.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossy-contract-v1.json"],"record_sha256":"a844e659b644d783e55d5b0f9cf2f8b100af1a1001221d4bd03b21fa327b1d52"},"id":"f329501c6a7cb4906e17abe732f2906a240cb485f2ed64ecba0f7012aa00ee28","identity":"trial:c23-video-context-lse-v2-native-dc375564","kind":"attempt_observation","source_record":"h3-ref2va-910c-4logical-20261101-297/results/c23-native-qualification.json#","source_records":["h3-ref2va-910c-4logical-20261101-297/results/c23-native-qualification.json#"],"status":"recorded"} +{"artifact_id":"h3-ref2va-910c-4logical-20261101-297-6c7e237-v1","conclusion":"{\"scope\":\"Focused state/block tests and native four-rank subset attention; no full-model quality/performance acceptance\"}","configuration":{"campaign_records":["h3-ref2va-910c-4logical-20261101-297/plan/execution-checkpoint.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossless-contract-v1.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossy-contract-v1.json"],"record_sha256":"bbc5cb862a8af814562ee4477bb9a69c847f25abe94a355df5ea9aa6b20ce53d"},"id":"f38845a5aa15feb9ae9c3c568c2529d8f05d5eabf042d9534bce8dde1bb2c071","identity":"trial:c15-video-row-native-3a9721a6","kind":"attempt_observation","source_record":"h3-ref2va-910c-4logical-20261101-297/results/c12-through-c15-execution-checkpoint.json#/runs/12","source_records":["h3-ref2va-910c-4logical-20261101-297/results/c12-through-c15-execution-checkpoint.json#/runs/12","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c17-execution-checkpoint.json#/runs/12","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c19-execution-checkpoint.json#/runs/12","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c20-execution-checkpoint.json#/runs/12","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c21-execution-checkpoint.json#/runs/12","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c22-execution-checkpoint.json#/runs/12","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c23-execution-checkpoint.json#/runs/12"],"status":"completed"} +{"artifact_id":"h3-ref2va-910c-4logical-20261101-297-6c7e237-v1","conclusion":"{\"scope\":\"Independent weighted global-context reference and native padded-head BF16 FAS; no full-model quality/performance acceptance\"}","configuration":{"campaign_records":["h3-ref2va-910c-4logical-20261101-297/plan/execution-checkpoint.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossless-contract-v1.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossy-contract-v1.json"],"record_sha256":"10be6e358ee205f88673c6dfca2d6e88824a5ad310fbe88b3b586cfca1a57347"},"id":"f38faa24c93dc2317d35f734b5801047cacbdd2a547eca4b9abf56fc6b33f813","identity":"trial:c19-video-global-pool-native-f0f4c502","kind":"attempt_observation","source_record":"h3-ref2va-910c-4logical-20261101-297/results/c19-native-qualification.json#","source_records":["h3-ref2va-910c-4logical-20261101-297/results/c19-native-qualification.json#"],"status":"recorded"} +{"artifact_id":"h3-ref2va-910c-4logical-20261101-297-6c7e237-v1","conclusion":"{\"scope\":\"Real captured H3 attention operator benchmark; full-model quality and HTTP are separate; no full-model quality/performance acceptance\"}","configuration":{"campaign_records":["h3-ref2va-910c-4logical-20261101-297/plan/execution-checkpoint.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossless-contract-v1.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossy-contract-v1.json"],"record_sha256":"016944ede030791c4dea386193f59dc01fb62f301a887458559083a6413ea355"},"id":"f57153c2e7390a66cf5b43519467a85c14062824a61c66c463e2d2767b002fc4","identity":"trial:c17-video-window-operator-9bf7de3e","kind":"attempt_observation","source_record":"h3-ref2va-910c-4logical-20261101-297/results/c12-through-c17-execution-checkpoint.json#/runs/21","source_records":["h3-ref2va-910c-4logical-20261101-297/results/c12-through-c17-execution-checkpoint.json#/runs/21","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c19-execution-checkpoint.json#/runs/21","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c20-execution-checkpoint.json#/runs/21","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c21-execution-checkpoint.json#/runs/21","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c22-execution-checkpoint.json#/runs/21"],"status":"completed"} +{"artifact_id":"h3-ref2va-910c-4logical-20261101-297-6c7e237-v1","conclusion":"{\"scope\":\"Focused state/block tests and native four-rank subset attention; no full-model quality/performance acceptance\"}","configuration":{"campaign_records":["h3-ref2va-910c-4logical-20261101-297/plan/execution-checkpoint.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossless-contract-v1.json","h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossy-contract-v1.json"],"record_sha256":"33bbdd3d2e2c036449fe9711c682a122bed4aabe1e343c5e9ec61f04af2b53ac"},"id":"fe25967d317c4f3f40fb2962296ad643a45fbb025575ee8410075baab1d4984f","identity":"trial:c15-video-row-native-a2a224de","kind":"attempt_observation","source_record":"h3-ref2va-910c-4logical-20261101-297/results/c12-through-c15-execution-checkpoint.json#/runs/13","source_records":["h3-ref2va-910c-4logical-20261101-297/results/c12-through-c15-execution-checkpoint.json#/runs/13","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c17-execution-checkpoint.json#/runs/13","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c19-execution-checkpoint.json#/runs/13","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c20-execution-checkpoint.json#/runs/13","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c21-execution-checkpoint.json#/runs/13","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c22-execution-checkpoint.json#/runs/13","h3-ref2va-910c-4logical-20261101-297/results/c12-through-c23-execution-checkpoint.json#/runs/13"],"status":"failed"} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/validator/README.md b/evidence/h3-ref2va-910c-4logical-20261101-297/validator/README.md index a35ce8a7e..b466a1418 100644 --- a/evidence/h3-ref2va-910c-4logical-20261101-297/validator/README.md +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/validator/README.md @@ -1,25 +1,28 @@ -# Validator status +# Archive replay and frozen validation -The requester selected **strict bit-exact** for the lossless tracks. The -[lossless contract](../plan/frozen-lossless-contract-v1.json) was frozen before -search. It fixes video/audio FP32 checkpoints, schedule and all decoded RGB24/PCM -bytes. Original B0 repeatability passed separately at both durations. Equal MP4 -hashes alone do not establish this contract. +From the repository root, with Python 3.10+ and the standard library: -The [lossy contract](../plan/frozen-lossy-contract-v1.json) was frozen before real -lossy outputs. It specifies ten prompts, pinned scorers/weights, reduced official -VBench dimensions and paired video/audio/reference/temporal guards. The source -requirement's human inspection is excluded from this RSI loop and is not claimed -as completed. These proxies do not certify absolute quality or semantic AV synchrony. +```bash +python tools/verify_evidence_archive.py evidence/h3-ref2va-910c-4logical-20261101-297/results/artifacts.json --download +# Download the pinned asset separately to ; never run archived code. +python tools/rebuild_model_evidence.py evidence/h3-ref2va-910c-4logical-20261101-297/results/artifacts.json \ + --archive --output +python tools/verify_evidence_archive.py evidence/h3-ref2va-910c-4logical-20261101-297/results/artifacts.json \ + --archive --recomputed +``` -The [snapshot manifest](frozen-snapshots.json) identifies exact original bytes. -Archived scorer paths assume the original execution-tree layout and pinned -dependencies; this evidence folder alone is not a self-contained runtime. No -weights or raw media are bundled. Keep numerical qualification, automatic model -quality and clean timing separate. Missing samples remain INCONCLUSIVE. An observed -gate violation was intended to remain FAIL even when other samples are missing. The -frozen `strict_compare.py` (line 109) does not do that: it returns INCONCLUSIVE -whenever any evidence is missing, even if failures were also recorded. A -qualified successor should report both. These validator bytes stay frozen, so -the limitation is documented here instead of corrected; no recorded verdict in -this bundle combines failures with missing evidence. +Replay authenticates compressed bytes, bounds archive reads, checks every +original file and original integrity manifest, and checks that the local replay +implementation equals the pinned archived implementation. It neither extracts +nor executes archived code, and refuses an existing output directory. It scans +all published result JSON documents, including nested failed/cancelled attempts, +then regenerates the index and summary without reading retained views. +Structure, IDs, source pointers and integers compare exactly. Finite floats use +predeclared absolute and relative tolerances of 1e-12 across Python versions. + +Only the sample distributions declared by `plan/archive-projection.json` are +recomputed. Frozen statistical/quality verdicts are labelled recorded results; +unavailable raw inputs are not requalified. The index is complete for published +structured observations, not unavailable private history or narrative-only +attempts. The original validators remain byte-preserved in the archive and +require their original model/device/input dependencies for model acceptance. diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/README.md b/evidence/qwen38-h20-fp8-single-request-vllm032/README.md index 88644d0a4..63ae41599 100644 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/README.md +++ b/evidence/qwen38-h20-fp8-single-request-vllm032/README.md @@ -1,106 +1,33 @@ -# Qwen3.8 FP8 single-request campaign — diagnostic takeover - -Governing acceptance: https://github.com/InferMatrixLab/InferRSI/issues/67 -Status: **INCONCLUSIVE. The first complete paired search fails both frozen performance parity gates; task quality and retrieval remain unqualified.** - -Current activation: `plan/search-8k-v1/`, recorded before candidate job 2163. -Completed standalone baseline: `results/formal-8k-baseline/`. Earlier historical -entries below retain their original diagnostic scope and status. - -Current completed search evidence: - -- [Merged FP8 paired search](results/fp8-projections-search-v1/observation.md): - job 2163 completed in 2244 GPU-seconds; both TTFT and TPOT fail the frozen - performance gates. This cumulative candidate comparison does not isolate - projection-merge speedup. Warnings and parent statistical review are retained. - These parity failures do not revoke earlier verified engineering observations - under their own scopes; cumulative parity and incremental optimization are - separate questions. -- [Decode profiler diagnosis](results/profile-merged-v1/observation.md): job 2169 - completed in 91 GPU-seconds. Eight instrumented decode steps guide the separate - BF16 b/a fusion hypothesis; they are not formal TPOT or service gain evidence. -- [FlashAttention prefill engineering screen](results/merged-flash-prefill-v1/observation.md): - job 2175 completed in 278 GPU-seconds. The single-case TTFT observation improved, - but the measured output sequences differ. This is ineligible for formal - statistics and establishes no quality-preserving gain. - -At the historical snapshot before candidate-quality job 2180, readiness spent is 7119 / 7200 GPU-seconds, -with the earlier initial 147 GPU-seconds accounted separately. Baseline-phase -completed use is 1221 GPU-seconds (failed job 2139: 7; completed job 2140: 1214). -Baseline-quality job 2157 remains live at the latest 58m46s snapshot, with a -9000-second reservation and no terminal receipt or score yet. Search spent is -2613 / 28800 GPU-seconds (2244 + 91 + 278). The final 108000-GPU-second allocation -is unused. No further GPU job was reserved at that snapshot. Candidate-quality job 2180 -is now running the original 232-item screen plus 10 retrieval cases under a -2700-second cap: search use is 2613 spent plus 2700 reserved within 28800 -GPU-seconds, with no quality result claimed. Live reservations -are not completed usage, and these totals do not qualify pending quality. - -The additive `results/search-observations-2163-2175-integrity.json` binds the -published sanitized observations, reviews, results and KnowledgeStore receipts. -Private raw traces, token streams, scheduler/device identifiers and operational -logs remain unavailable in this checkout. Frozen formal rules and earlier -receipts are unchanged; inventory validation checks published bytes only. - - -This isolated checkout retains the 2026-10-22/23 Codex takeover diagnostics and -prospective plan. Earlier readiness drafts and DSH evidence remain in their -original stores; this directory does not replace or retrospectively freeze them. - -- `plan/diagnostics/fp8-graph-takeover.md`: bounded diagnosis and repair contrast. -- `plan/freeze-proposal.md`: owner-review statistical/retrieval/budget proposal. -- `results/diagnostics/`: original and repaired numerical traces, summaries and - source digests. Trace data contains no service timing claims. -- `patch/diagnostics/`: separate configured-backend and model-prefill repairs. -- `validator/diagnostics/`: diagnostic observer; not the final acceptance validator. - -Publication follow-up: `results/diagnostics/http-impact.md` binds the clean -main-based engine commits to actual HTTP timing and token evidence. The repaired -Graph matches eager for all six short requests; original main differs on two. -The exact publication-head CPU suite has one inherited failure, reproduced on -unmodified main; see the report rather than transferring earlier green totals. -The subsequent accounting adapter and failed long-output boundary check are -recorded in `results/diagnostics/benchmark-http-impact.md`; full model readiness -remains outstanding. - -The HTTP follow-up consumed 0.04 GPU-hours of the owner-approved 44 GPU-hour -phased ceiling. It does not activate the statistical/retrieval freeze proposal. -KnowledgeStore record `store:case.qwen38-publication-http-prefill-cand`, revision -`sha256:83c7df1d6debb6e2797a96fd30d6a8270d842d6cf97428610821b3d78a964c32`, -retains positive, negative and inconclusive observations with verified readback. - -Current local engine implementation has two independently bounded commits: the -configured FP8 backend is validated/reported, and initial prefill preserves its -original K/V operands with reserved Graph storage. GPU repair evidence applies -to the model-only diff. CPU regressions also cover the combined source. - -The prefill discrepancy is repaired for all three diagnostic cases. Replay -matches fixed-capacity eager at all 48 steps, but compact/buffered decode logits -still differ in 13/48 steps despite identical greedy choices. Full service, -long-context, quality and performance qualification remain outstanding. - -Knowledge candidates were recorded using InferRSI KnowledgeStore, with retained -revisions and verified readback. They are not promoted library coverage. Raw -operational logs and private device identities remain in controlled task storage. - -Further bounded diagnostics retain the official-client accounting adapter, -failed long-output request, readiness-file preflight race and separate Graph -demotion fix. See `plan/diagnostics/benchmark-http.md` and -`plan/diagnostics/graph-demotion.md`. Original observers/results are preserved; -versioned successors do not retroactively change their verdicts. - -The completed10-request boundary repair and its remaining numerical differences -are recorded in `results/diagnostics/graph-demotion-impact.md`. Engine PR9 is -merged; full Graph/eager sequences agree on3/5 pairs and campaign acceptance -remains INCONCLUSIVE. Readiness allocation use is839 GPU-seconds. - -Fixed-history follow-up `results/diagnostics/graph-forced96-impact.md` narrows -the case783 discrepancy: all96 capacity-eager/Graph logits and live states -are exact; compact-versus-capacity changes argmax at index81. This remains -a bounded diagnosis, not task-quality acceptance. Readiness total904 seconds. - -Baseline readiness follow-up: `results/diagnostics/baseline-long-impact.md` -retains failed infrastructure1959, CPU transport1960, and observer-rejected1961. -All target lengths returned the requested short outputs;128K coalesced two -tokens into one choice, a supported upstream behavior rejected by the observer. -Original failure remains; no formal baseline or parity PASS. Readiness1509s. +# Qwen3.8 FP8 single-request evidence + +Acceptance: [#67](https://github.com/InferMatrixLab/InferRSI/issues/67). +**Campaign verdict remains INCONCLUSIVE.** Both frozen paired performance parity gates failed; task quality, retrieval and final acceptance remain incomplete. + +This explicitly requested cleanup preserves the 413 previously public +files (5,205,664 bytes) at `6c7e237935a22cc5e90e954d4f397376e753e7d8` unchanged in a +versioned archive. It moves cumulative checkpoints, repeated diagnostic source +copies and generated details out of the current tree. The original README, +failed/interrupted attempts, superseded observations, all patches and frozen +validators remain at their original paths inside the archive. No model run, +historical gate, result or original byte was changed. + +## Retained evidence + +- [Summary](results/summary.json): key recorded conclusions, sample distributions + recomputed from published samples, and explicit coverage/availability limits. +- [Index](results/trials.jsonl): 130 published record/attempt observations; + every row has configuration references, status, conclusion, archive identity + and an archive-relative JSON Pointer. These are not independent trial counts. +- [Archive authority](results/artifacts.json): v2 URL/hash/size/source/retention + and pinned replay inputs. The inventory references this document. +- [Plan](plan/README.md), [patch](patch/README.md) and [validator](validator/README.md). + +Identical observations of the same named attempt collapse to one row with all +`source_records`. Different observations of that attempt, retries and anonymous +records remain separate. Historical revisions are not replaced by the latest +checkpoint. Unpublished attempts cannot be reconstructed; Markdown-only history +and per-request JSONL remain in the original archive. Passing archive checks +does not complete the campaign or turn individual engineering results into PASS. + +The [original tree](https://github.com/InferMatrixLab/InferRSI/tree/6c7e237935a22cc5e90e954d4f397376e753e7d8/evidence/qwen38-h20-fp8-single-request-vllm032/) +is another immutable reference. This change does not rewrite Git history. diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/README.md b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/README.md new file mode 100644 index 000000000..bb0e5a6c3 --- /dev/null +++ b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/README.md @@ -0,0 +1,10 @@ +# Source and patch identities + +All original patch/source-identity records remain under `qwen38-h20-fp8-single-request-vllm032/patch/` in the +archive declared by `../results/artifacts.json`, with exact per-file SHA-256 in +its `snapshot.json`. The original README retains upstream implementation links +and the distinction between tested, rejected and proposed variants. A later +source revision does not inherit an earlier quality or performance verdict. + +The immutable original source tree is +https://github.com/InferMatrixLab/InferRSI/tree/6c7e237935a22cc5e90e954d4f397376e753e7d8/evidence/qwen38-h20-fp8-single-request-vllm032/patch diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/deepgemm-prefill/engine.patch b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/deepgemm-prefill/engine.patch deleted file mode 100644 index 7d06e5e23..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/deepgemm-prefill/engine.patch +++ /dev/null @@ -1,237 +0,0 @@ -diff --git a/docs/qwen38-phase1.md b/docs/qwen38-phase1.md -index 9f78c58..7becfe9 100644 ---- a/docs/qwen38-phase1.md -+++ b/docs/qwen38-phase1.md -@@ -96,7 +96,8 @@ CPU socket tests do not establish that service acceptance. - The model math is derived from the Apache-2.0 Transformers Qwen3.5 source. - See [NOTICE-qwen38](../NOTICE-qwen38) and [Apache license](../LICENSES/Apache-2.0.txt). - The original nano-vllm code retains its [MIT license](../LICENSE). --# Optional small-M FP8 GEMM -+ -+## Optional FP8 GEMM backends - - `NANOVLLM_FP8_GEMM=flashinfer` selects FlashInfer's SM90 block-scaled - FP8 GEMM for fewer than 32 activation rows, including single-token decoding. -@@ -111,3 +112,9 @@ Triton Python call counts. These counts include warmup/capture and exclude CUDA - Graph replay. Selecting the backend does not establish model quality or service - performance; those require the governing acceptance campaign's evidence. - -+`NANOVLLM_FP8_GEMM=flashinfer_deepgemm` additionally uses an independently -+installed `deep_gemm` package for M>=32 prefill. It keeps the same FlashInfer -+decode path. Activation quantization uses the existing block-128 FP8 semantics -+and produces TMA-aligned FP32 scales; the GEMM explicitly disables UE8M0 casting. -+Official checkpoint weight bytes and multipliers remain unchanged. This mode -+does not import the vLLM runtime and fails if DeepGEMM is unavailable. -diff --git a/nanovllm/qwen38/_deepgemm_fp8.py b/nanovllm/qwen38/_deepgemm_fp8.py -new file mode 100644 -index 0000000..ef125a2 ---- /dev/null -+++ b/nanovllm/qwen38/_deepgemm_fp8.py -@@ -0,0 +1,46 @@ -+"""SM90 prefill GEMM with unchanged checkpoint weights and FP32 block scales.""" -+ -+import torch -+import triton -+import triton.language as tl -+ -+ -+@triton.jit -+def _quantize_groups(X, Q, S, K: tl.constexpr, PADDED_M: tl.constexpr): -+ row = tl.program_id(0) -+ group = tl.program_id(1) -+ offsets = row * K + group * 128 + tl.arange(0, 128) -+ values = tl.load(X + offsets).to(tl.float32) -+ # Preserve the existing native kernel's per-token/block-128 quantization. -+ # The scale is not rounded to E8M0 and weights are never requantized. -+ scale = tl.max(tl.abs(values), axis=0) / 448.0 -+ quantized = (values / tl.maximum(scale, 1e-12)).to(tl.float8e4nv) -+ tl.store(Q + offsets, quantized) -+ tl.store(S + group * PADDED_M + row, scale) -+ -+ -+def quantize(inputs: torch.Tensor): -+ m, k = inputs.shape -+ if inputs.dtype != torch.bfloat16 or not inputs.is_cuda or not inputs.is_contiguous() or k % 128: -+ raise ValueError("DeepGEMM quantization requires contiguous CUDA BF16 [M, K128]") -+ padded_m = triton.cdiv(m, 4) * 4 -+ output = torch.empty((m, k), device=inputs.device, dtype=torch.float8_e4m3fn) -+ scales = torch.empty_strided((m, k // 128), (1, padded_m), device=inputs.device, dtype=torch.float32) -+ _quantize_groups[(m, k // 128)](inputs, output, scales, k, padded_m, num_warps=4) -+ return output, scales -+ -+ -+def matmul(inputs, weight, scales): -+ # An external DeepGEMM package is required. Do not import vLLM or silently -+ # fall back to a different implementation when the dependency is absent. -+ from deep_gemm import fp8_gemm_nt -+ -+ m, k = inputs.shape -+ n = weight.shape[0] -+ if k % 128 or n % 64: -+ raise ValueError("DeepGEMM FP8 requires K divisible by 128 and N by 64") -+ quantized, activation_scales = quantize(inputs) -+ output = torch.empty((m, n), device=inputs.device, dtype=torch.bfloat16) -+ fp8_gemm_nt((quantized, activation_scales), (weight, scales), output, -+ recipe=(1, 128, 128), disable_ue8m0_cast=True) -+ return output -diff --git a/nanovllm/qwen38/engine.py b/nanovllm/qwen38/engine.py -index f9f93cd..e6fe55a 100644 ---- a/nanovllm/qwen38/engine.py -+++ b/nanovllm/qwen38/engine.py -@@ -166,7 +166,7 @@ class DedicatedEngine: - origins[name] = {"path": str(Path(path).resolve()), - "sha256": hashlib.sha256(Path(path).read_bytes()).hexdigest()} - versions = {name: str(getattr(sys.modules[name], "__version__", "unknown")) -- for name in ("torch", "transformers", "triton", "flashinfer") if name in sys.modules} -+ for name in ("torch", "transformers", "triton", "flashinfer", "deep_gemm") if name in sys.modules} - device = {"device_type": "synthetic", "device_count": 0, "device_model": "test_only"} - if self.loader_report["dtype"] != "test_only": - import torch -@@ -186,7 +186,7 @@ class DedicatedEngine: - "kernel_name": self.loader_report["kernel_name"], - "executed_modules": sum(count > 0 for count in calls.values()), "per_module_calls": calls, - "kernel_calls": {kernel: sum(module.kernel_calls[kernel] for module in modules.values()) -- for kernel in ("flashinfer", "triton")}, -+ for kernel in ("flashinfer", "triton", "deepgemm")}, - "weight_dtype": "float8_e4m3fn", "scale_dtype": "float32", - "actual_dtypes_valid": all(module.weight.dtype == torch.float8_e4m3fn - and module.weight_scale_inv.dtype == torch.float32 -diff --git a/nanovllm/qwen38/quant.py b/nanovllm/qwen38/quant.py -index 1a06f0a..47979ec 100644 ---- a/nanovllm/qwen38/quant.py -+++ b/nanovllm/qwen38/quant.py -@@ -24,15 +24,19 @@ class FP8Linear(nn.Module): - def __init__(self, in_features: int, out_features: int, *, device=None): - super().__init__() - requested = os.environ.get("NANOVLLM_FP8_GEMM", "triton") -- if requested not in ("triton", self.backend, "flashinfer"): -+ if requested not in ("triton", self.backend, "flashinfer", "flashinfer_deepgemm"): - raise ValueError( - f"Unsupported NANOVLLM_FP8_GEMM={requested!r}; this build implements " -- "triton and flashinfer (small-M only). No fallback was selected." -+ "triton, flashinfer and flashinfer_deepgemm. No fallback was selected." - ) -- self.use_flashinfer = requested == "flashinfer" -+ self.use_flashinfer = requested in ("flashinfer", "flashinfer_deepgemm") -+ self.use_deepgemm = requested == "flashinfer_deepgemm" - if self.use_flashinfer: - self.backend = "flashinfer_small_m_triton_prefill" - self.kernel_name = "fp8_blockscale_gemm_sm90+qwen38_block_fp8" -+ if self.use_deepgemm: -+ self.backend = "flashinfer_small_m_deepgemm_prefill" -+ self.kernel_name = "fp8_blockscale_gemm_sm90+deep_gemm.fp8_gemm_nt" - if in_features <= 0 or out_features <= 0: - raise ValueError("FP8 linear dimensions must be positive") - self.in_features = in_features -@@ -45,7 +49,7 @@ class FP8Linear(nn.Module): - device=device, dtype=torch.float32, - ), requires_grad=False) - self.native_fp8_calls = 0 -- self.kernel_calls = {"flashinfer": 0, "triton": 0} -+ self.kernel_calls = {"flashinfer": 0, "triton": 0, "deepgemm": 0} - - def forward(self, inputs: torch.Tensor) -> torch.Tensor: - if inputs.ndim < 2 or inputs.shape[-1] != self.in_features or inputs.numel() == 0: -@@ -74,6 +78,13 @@ class FP8Linear(nn.Module): - weight_scale=self.weight_scale_inv, out_dtype=torch.bfloat16, - ).reshape(*inputs.shape[:-1], self.out_features) - self.kernel_calls["flashinfer"] += 1 -+ elif self.use_deepgemm: -+ from ._deepgemm_fp8 import matmul -+ -+ result = matmul(inputs.reshape(rows, self.in_features).contiguous(), -+ self.weight, self.weight_scale_inv) -+ result = result.reshape(*inputs.shape[:-1], self.out_features) -+ self.kernel_calls["deepgemm"] += 1 - else: - from ._triton_fp8 import matmul - -diff --git a/tests/test_qwen38_quant.py b/tests/test_qwen38_quant.py -index 2f8f0ce..39036bd 100644 ---- a/tests/test_qwen38_quant.py -+++ b/tests/test_qwen38_quant.py -@@ -31,8 +31,11 @@ def test_supported_backend_aliases_report_implemented_kernel(monkeypatch, backen - - @pytest.mark.parametrize("shape,expected", [((1, 128), "flashinfer"), ((1, 31, 128), "flashinfer"), - ((2, 16, 128), "triton"), ((33, 128), "triton")]) --def test_small_m_dispatch_preserves_shape_and_checkpoint_scales(monkeypatch, shape, expected): -- monkeypatch.setenv("NANOVLLM_FP8_GEMM", "flashinfer") -+@pytest.mark.parametrize("backend", ["flashinfer", "flashinfer_deepgemm"]) -+def test_small_m_dispatch_preserves_shape_and_checkpoint_scales(monkeypatch, shape, expected, backend): -+ monkeypatch.setenv("NANOVLLM_FP8_GEMM", backend) -+ if backend == "flashinfer_deepgemm" and expected == "triton": -+ expected = "deepgemm" - module = FP8Linear(128, 64) - inputs = torch.zeros(shape, dtype=torch.bfloat16) - seen = [] -@@ -48,12 +51,21 @@ def test_small_m_dispatch_preserves_shape_and_checkpoint_scales(monkeypatch, sha - seen.append("triton") - return x.new_zeros(*x.shape[:-1], 64) - -+ def deepgemm(x, weight, scales): -+ assert x.ndim == 2 and x.is_contiguous() -+ assert weight is module.weight and scales is module.weight_scale_inv -+ seen.append("deepgemm") -+ return x.new_zeros(x.shape[0], 64) -+ - fake_flashinfer = ModuleType("flashinfer.gemm") - fake_flashinfer.fp8_blockscale_gemm_sm90 = flashinfer - fake_triton = ModuleType("nanovllm.qwen38._triton_fp8") - fake_triton.matmul = triton -+ fake_deepgemm = ModuleType("nanovllm.qwen38._deepgemm_fp8") -+ fake_deepgemm.matmul = deepgemm - monkeypatch.setitem(sys.modules, "flashinfer.gemm", fake_flashinfer) - monkeypatch.setitem(sys.modules, "nanovllm.qwen38._triton_fp8", fake_triton) -+ monkeypatch.setitem(sys.modules, "nanovllm.qwen38._deepgemm_fp8", fake_deepgemm) - output = module._matmul(inputs) - assert output.shape == (*shape[:-1], 64) and output.dtype == torch.bfloat16 - assert seen == [expected] and module.kernel_calls[expected] == 1 -@@ -79,6 +91,48 @@ def test_report_uses_constructed_backend_not_current_environment(monkeypatch, tm - fp8_report([*modules, FP8Linear(128, 64)]) - - -+@pytest.mark.parametrize("shape", [(32, 128), (1, 33, 128)]) -+def test_deepgemm_prefill_keeps_original_weights_and_float_scales(monkeypatch, shape): -+ from nanovllm.qwen38 import _deepgemm_fp8 -+ -+ monkeypatch.setenv("NANOVLLM_FP8_GEMM", "flashinfer_deepgemm") -+ module = FP8Linear(128, 64) -+ module.weight.data.fill_(0.25) -+ module.weight_scale_inv.data.fill_(0.125) -+ before = module.weight.view(torch.uint8).clone() -+ rows = torch.zeros(shape, dtype=torch.bfloat16).reshape(-1, 128) -+ quantized = torch.zeros_like(rows, dtype=torch.float8_e4m3fn) -+ scales = torch.ones((1, len(rows)), dtype=torch.float32).t() -+ monkeypatch.setattr(_deepgemm_fp8, "quantize", lambda x: (quantized, scales)) -+ called = [] -+ -+ def fp8_gemm_nt(a, b, output, *, recipe, disable_ue8m0_cast): -+ assert a[0] is quantized and a[1] is scales -+ assert b[0] is module.weight and b[1] is module.weight_scale_inv -+ assert recipe == (1, 128, 128) and disable_ue8m0_cast is True -+ assert output.dtype == torch.bfloat16 and output.shape == (len(rows), 64) -+ output.fill_(1) -+ called.append(True) -+ -+ dependency = ModuleType("deep_gemm") -+ dependency.fp8_gemm_nt = fp8_gemm_nt -+ monkeypatch.setitem(sys.modules, "deep_gemm", dependency) -+ result = module._matmul(rows.reshape(shape)) -+ assert result.shape == (*shape[:-1], 64) and torch.all(result == 1) -+ assert called == [True] and module.kernel_calls["deepgemm"] == 1 -+ assert torch.equal(module.weight.view(torch.uint8), before) -+ assert torch.all(module.weight_scale_inv == 0.125) -+ -+ -+def test_deepgemm_missing_dependency_does_not_fallback(monkeypatch): -+ monkeypatch.setenv("NANOVLLM_FP8_GEMM", "flashinfer_deepgemm") -+ module = FP8Linear(128, 64) -+ monkeypatch.setitem(sys.modules, "deep_gemm", None) -+ with pytest.raises(ModuleNotFoundError): -+ module._matmul(torch.zeros(32, 128, dtype=torch.bfloat16)) -+ assert sum(module.kernel_calls.values()) == 0 -+ -+ - def tiny_fp8_model(): - model = Qwen38ForCausalLM(Qwen38TextConfig.for_testing()).to(torch.bfloat16) - exclusions = ("lm_head", "model.language_model.layers.0.linear_attn.in_proj_a", diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/benchmark-http.patch b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/benchmark-http.patch deleted file mode 100644 index 8b74a4837..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/benchmark-http.patch +++ /dev/null @@ -1,224 +0,0 @@ -diff --git a/docs/qwen38-service.md b/docs/qwen38-service.md -index 9251200..107e460 100644 ---- a/docs/qwen38-service.md -+++ b/docs/qwen38-service.md -@@ -23,7 +23,7 @@ The dedicated constructor accepts `max_context_tokens` and - `max_active_requests`. Legacy constructor options are not aliases for these - limits. Generation accepts text or token-list prompts and greedy sampling only. - Each result contains `text`, `token_ids`, and `finish_reason` (`stop` or `length`). --`ignore_eos=True` is available through the Python API. Stochastic sampling and -+`ignore_eos=True` is available through the Python and HTTP APIs. Stochastic sampling and - unsupported constructor or HTTP fields fail explicitly. The unrelated legacy - engine continues to require positive sampling temperature. - -@@ -56,12 +56,23 @@ One HTTP request is served per connection; chunked request bodies are unsupporte - | `temperature` | Optional; only zero is supported | - | `stream` | Optional boolean, default false | - | `request_id` | Optional identifier using letters, digits, `_`, `.`, `:`, `-` | -+| `ignore_eos` | Optional boolean, default false; true generates through EOS up to the requested cap | -+| `return_token_ids` | Optional boolean; true also echoes exact prompt IDs in the first stream choice or non-stream response | -+| `repetition_penalty` | Optional; only the neutral value 1.0 is supported | -+| `logprobs` | Optional; only null is supported | -+| `stream_options` | Streaming only; supports boolean `include_usage`, default false | - - Non-streaming JSON contains `choices[0].text`, `token_ids`, `finish_reason`, and - token usage. Streaming responses use `text/event-stream`: each `data:` JSON chunk - contains the request ID and a choice with cumulative-decoding text delta and new --token IDs. The terminal choice has empty text/token IDs and a finish reason, --followed by `data: [DONE]`. Normal termination uses `stop` or `length`; -+token IDs. Existing generated `token_ids` remain present regardless of the new -+prompt-echo flag. Normal termination puts its finish reason on the final token -+choice, without an additional empty choice that a benchmark could time as a -+token. A cancellation with no pending delta uses an empty terminal choice. -+With `include_usage=true`, choice chunks carry null usage and one final chunk -+has empty `choices` and actual prompt/completion/total token counts. Prompt IDs -+are echoed only once. The stream ends with `data: [DONE]`. -+Normal termination uses `stop` or `length`; - explicit cancellation uses `cancelled`; service drain uses `shutdown`. Runtime - errors emit a sanitized error object and terminal `error` framing when the - connection is still writable. HTTP runtime failures return 500. Validation -diff --git a/nanovllm/qwen38/server.py b/nanovllm/qwen38/server.py -index 3847e23..776962e 100644 ---- a/nanovllm/qwen38/server.py -+++ b/nanovllm/qwen38/server.py -@@ -163,14 +163,29 @@ class CompletionHandler(BaseHTTPRequestHandler): - self.wfile.flush() - - def _chunk(self, request, text, token_ids, finish_reason=None): -- return {"id": request.request_id, "request_id": request.request_id, -- "object": "text_completion", "model": self.server.engine.model_id, -- "choices": [{"index": 0, "text": text, "token_ids": token_ids, -- "finish_reason": finish_reason}]} -+ chunk = {"id": request.request_id, "request_id": request.request_id, -+ "object": "text_completion", "model": self.server.engine.model_id, -+ "choices": [{"index": 0, "text": text, "token_ids": token_ids, -+ "finish_reason": finish_reason}]} -+ if self._include_usage: -+ chunk["usage"] = None -+ if self._prompt_token_ids is not None: -+ chunk["choices"][0]["prompt_token_ids"] = self._prompt_token_ids -+ self._prompt_token_ids = None -+ return chunk -+ -+ @staticmethod -+ def _usage(request): -+ return {"prompt_tokens": len(request.prompt_ids), -+ "completion_tokens": len(request.token_ids), -+ "total_tokens": len(request.prompt_ids) + len(request.token_ids)} - - def do_POST(self): - request = None - streaming = False -+ self._include_usage = False -+ self._prompt_token_ids = None -+ terminal_sent = False - headers_sent = False - try: - if self.path.startswith("/v1/cancel/"): -@@ -182,7 +197,8 @@ class CompletionHandler(BaseHTTPRequestHandler): - self._error(404, "unknown route") - return - body = self._body() -- allowed = {"model", "prompt", "input_ids", "max_tokens", "temperature", "stream", "request_id"} -+ allowed = {"model", "prompt", "input_ids", "max_tokens", "temperature", "stream", "request_id", -+ "repetition_penalty", "logprobs", "stream_options", "ignore_eos", "return_token_ids"} - if set(body) - allowed: - raise ValueError("unsupported completion fields") - if body.get("model", self.server.engine.model_id) != self.server.engine.model_id: -@@ -194,12 +210,33 @@ class CompletionHandler(BaseHTTPRequestHandler): - streaming = body.get("stream", False) - if type(streaming) is not bool: - raise ValueError("stream must be boolean") -+ repetition_penalty = body.get("repetition_penalty", 1.0) -+ if type(repetition_penalty) not in (int, float) or repetition_penalty != 1.0: -+ raise ValueError("only repetition_penalty=1.0 is supported") -+ if body.get("logprobs") is not None: -+ raise ValueError("only logprobs=null is supported") -+ if "stream_options" in body: -+ options = body["stream_options"] -+ if not streaming or not isinstance(options, dict) or set(options) - {"include_usage"}: -+ raise ValueError("stream_options requires streaming and supports only include_usage") -+ self._include_usage = options.get("include_usage", False) -+ if type(self._include_usage) is not bool: -+ raise ValueError("stream_options.include_usage must be boolean") -+ ignore_eos = body.get("ignore_eos", False) -+ if type(ignore_eos) is not bool: -+ raise ValueError("ignore_eos must be boolean") -+ return_token_ids = body.get("return_token_ids", False) -+ if type(return_token_ids) is not bool: -+ raise ValueError("return_token_ids must be boolean") - prompt = body.get("prompt", body.get("input_ids")) - if "input_ids" in body and not isinstance(prompt, list): - raise ValueError("input_ids must be a token list") - if self._disconnected(): - return -- request = self.server.engine.submit(prompt, max_tokens=body.get("max_tokens"), request_id=body.get("request_id")) -+ request = self.server.engine.submit(prompt, max_tokens=body.get("max_tokens"), -+ request_id=body.get("request_id"), ignore_eos=ignore_eos) -+ if return_token_ids: -+ self._prompt_token_ids = list(request.prompt_ids) - with self.server._connections_lock: - self.server._connections[self.connection] = True - if streaming: -@@ -218,15 +255,23 @@ class CompletionHandler(BaseHTTPRequestHandler): - return - event = request.step(disconnect_check=self._disconnected) - if streaming and (event["token_id"] is not None or event["delta"]): -- self._sse(self._chunk(request, event["delta"], [] if event["token_id"] is None else [event["token_id"]])) -+ self._sse(self._chunk(request, event["delta"], -+ [] if event["token_id"] is None else [event["token_id"]], -+ request.finish_reason)) -+ terminal_sent = request.finish_reason is not None - if streaming: -- self._sse(self._chunk(request, "", [], request.finish_reason)) -+ # Match the upstream completion stream: the final token carries -+ # finish_reason, so the benchmark does not time an extra choice. -+ if not terminal_sent: -+ self._sse(self._chunk(request, "", [], request.finish_reason)) -+ if self._include_usage: -+ usage = self._chunk(request, "", []) -+ usage.update(choices=[], usage=self._usage(request)) -+ self._sse(usage) - self._sse("[DONE]") - else: - result = self._chunk(request, request.text, request.token_ids, request.finish_reason) -- result["usage"] = {"prompt_tokens": len(request.prompt_ids), -- "completion_tokens": len(request.token_ids), -- "total_tokens": len(request.prompt_ids) + len(request.token_ids)} -+ result["usage"] = self._usage(request) - self._json(200, result) - except (BrokenPipeError, ConnectionError, TimeoutError, OSError): - if request is not None and request.finish_reason == "error": -diff --git a/tests/test_qwen38_service.py b/tests/test_qwen38_service.py -index f520730..fd2555a 100644 ---- a/tests/test_qwen38_service.py -+++ b/tests/test_qwen38_service.py -@@ -188,6 +188,68 @@ def test_eos_empty_completion_text_and_normal_cleanup(): - assert_clean(model) - - -+@pytest.mark.parametrize("ignore_eos, expected_count, reason", [(False, 1, "stop"), (True, 4, "length")]) -+def test_benchmark_stream_counts_real_tokens_and_emits_one_choice_per_token(ignore_eos, expected_count, reason): -+ with serving(engine(eos=(1,))) as (model, address): -+ connection, response = stream(address, ignore_eos=ignore_eos, return_token_ids=True, -+ repetition_penalty=1.0, logprobs=None, -+ stream_options={"include_usage": True}) -+ chunks = events(response) -+ connection.close() -+ choices = [chunk["choices"][0] for chunk in chunks if chunk["choices"]] -+ ids = [token for choice in choices for token in choice["token_ids"]] -+ assert len(ids) == len(choices) == expected_count -+ assert choices[0]["prompt_token_ids"] == [1] -+ assert all("prompt_token_ids" not in choice for choice in choices[1:]) -+ assert all(choice["finish_reason"] is None for choice in choices[:-1]) -+ assert choices[-1]["finish_reason"] == reason -+ assert all(chunk["usage"] is None for chunk in chunks[:-1]) -+ assert chunks[-1]["choices"] == [] -+ assert chunks[-1]["usage"] == {"prompt_tokens": 1, "completion_tokens": expected_count, -+ "total_tokens": 1 + expected_count} -+ assert_clean(model) -+ -+ -+def test_nonstream_token_echo_and_usage_use_actual_input_and_output(): -+ with serving(engine(eos=(1,))) as (model, address): -+ status, result = complete(address, ignore_eos=True, return_token_ids=True) -+ assert status == 200 -+ assert result["choices"][0]["prompt_token_ids"] == [1] -+ assert len(result["choices"][0]["token_ids"]) == 4 -+ assert result["usage"] == {"prompt_tokens": 1, "completion_tokens": 4, "total_tokens": 5} -+ assert_clean(model) -+ -+ -+@pytest.mark.parametrize("options", [{}, {"include_usage": False}]) -+def test_optional_usage_is_absent_when_disabled(options): -+ with serving() as (model, address): -+ connection, response = stream(address, return_token_ids=False, stream_options=options) -+ chunks = events(response) -+ connection.close() -+ assert len(chunks) == 4 -+ assert all("usage" not in chunk for chunk in chunks) -+ assert all("prompt_token_ids" not in chunk["choices"][0] for chunk in chunks) -+ assert_clean(model) -+ -+ -+@pytest.mark.parametrize("fields", [ -+ {"ignore_eos": 1}, {"return_token_ids": "true"}, -+ {"repetition_penalty": True}, {"repetition_penalty": 1.1}, {"logprobs": 0}, -+ {"stream_options": None}, {"stream_options": {"include_usage": 1}}, -+ {"stream_options": {"continuous_usage_stats": True}}, -+ {"stream": False, "stream_options": {"include_usage": True}}, -+]) -+def test_invalid_benchmark_options_rejected_before_model_work(fields): -+ with serving() as (model, address): -+ connection, response = request(address, body={"input_ids": [1], "max_tokens": 4, -+ "stream": True, **fields}) -+ assert response.status == 400 -+ response.read() -+ connection.close() -+ assert not model.runner.model.calls -+ assert_clean(model) -+ -+ - def test_concurrent_requests_match_serial_and_are_isolated(): - with serving() as (model, address): - baseline = model.generate([[1], [3]], SamplingParams(temperature=0, max_tokens=4)) diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/fp8-observability.patch b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/fp8-observability.patch deleted file mode 100644 index cc35cbbdd..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/fp8-observability.patch +++ /dev/null @@ -1,83 +0,0 @@ -diff --git a/nanovllm/qwen38/engine.py b/nanovllm/qwen38/engine.py -index 3976480..1d770cb 100644 ---- a/nanovllm/qwen38/engine.py -+++ b/nanovllm/qwen38/engine.py -@@ -181,6 +181,9 @@ class DedicatedEngine: - if isinstance(module, FP8Linear)} - calls = {name: module.native_fp8_calls for name, module in modules.items()} - execution = {"fp8_execution": {"modules": len(calls), "total_calls": sum(calls.values()), -+ "call_counter_scope": "python_forward_only_includes_warmup_and_capture_excludes_graph_replay", -+ "backend": self.loader_report["backend"], -+ "kernel_name": self.loader_report["kernel_name"], - "executed_modules": sum(count > 0 for count in calls.values()), "per_module_calls": calls, - "weight_dtype": "float8_e4m3fn", "scale_dtype": "float32", - "actual_dtypes_valid": all(module.weight.dtype == torch.float8_e4m3fn -diff --git a/nanovllm/qwen38/quant.py b/nanovllm/qwen38/quant.py -index 34863c5..6df0144 100644 ---- a/nanovllm/qwen38/quant.py -+++ b/nanovllm/qwen38/quant.py -@@ -2,6 +2,8 @@ - - from __future__ import annotations - -+import os -+ - import torch - from torch import nn - -@@ -21,6 +23,12 @@ class FP8Linear(nn.Module): - - def __init__(self, in_features: int, out_features: int, *, device=None): - super().__init__() -+ requested = os.environ.get("NANOVLLM_FP8_GEMM", "triton") -+ if requested not in ("triton", self.backend): -+ raise ValueError( -+ f"Unsupported NANOVLLM_FP8_GEMM={requested!r}; this build implements " -+ f"only {self.backend}. No fallback was selected." -+ ) - if in_features <= 0 or out_features <= 0: - raise ValueError("FP8 linear dimensions must be positive") - self.in_features = in_features -diff --git a/nanovllm/qwen38/server.py b/nanovllm/qwen38/server.py -index e4adf9e..4fe45d2 100644 ---- a/nanovllm/qwen38/server.py -+++ b/nanovllm/qwen38/server.py -@@ -397,9 +397,9 @@ def main(): - "max_active_requests=%d graph=%s fp8_gemm=%s headroom=%s" - % (server.server_address, args.model, args.max_context_tokens, - args.max_active_requests, -- os.environ.get("NANOVLLM_CUDA_GRAPH", "0"), -- os.environ.get("NANOVLLM_FP8_GEMM", "triton"), -- os.environ.get("NANOVLLM_GRAPH_HEADROOM", "256")), -+ int(engine.runner.graph_enabled), -+ engine.loader_report.get("backend", "unquantized"), -+ engine.runner.graph_headroom), - file=sys.stderr, flush=True) - try: - server.serve_forever(poll_interval=0.05) -diff --git a/tests/test_qwen38_quant.py b/tests/test_qwen38_quant.py -index 52e9016..8dd0767 100644 ---- a/tests/test_qwen38_quant.py -+++ b/tests/test_qwen38_quant.py -@@ -12,6 +12,21 @@ from nanovllm.qwen38.model import Qwen38ForCausalLM - from nanovllm.qwen38.quant import FP8Linear, convert_fp8_linears - - -+@pytest.mark.parametrize("backend", ["flashinfer", "deepgemm", "typo"]) -+def test_unimplemented_backend_is_not_silently_ignored(monkeypatch, backend): -+ monkeypatch.setenv("NANOVLLM_FP8_GEMM", backend) -+ with pytest.raises(ValueError, match="Unsupported NANOVLLM_FP8_GEMM"): -+ FP8Linear(128, 128, device="meta") -+ -+ -+@pytest.mark.parametrize("backend", ["triton", "triton_block_fp8"]) -+def test_supported_backend_aliases_report_implemented_kernel(monkeypatch, backend): -+ monkeypatch.setenv("NANOVLLM_FP8_GEMM", backend) -+ module = FP8Linear(128, 128, device="meta") -+ assert module.backend == "triton_block_fp8" -+ assert module.kernel_name == "qwen38_block_fp8" -+ -+ - def tiny_fp8_model(): - model = Qwen38ForCausalLM(Qwen38TextConfig.for_testing()).to(torch.bfloat16) - exclusions = ("lm_head", "model.language_model.layers.0.linear_attn.in_proj_a", diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/graph-demotion.patch b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/graph-demotion.patch deleted file mode 100644 index 1576ceb86..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/graph-demotion.patch +++ /dev/null @@ -1,81 +0,0 @@ -diff --git a/nanovllm/qwen38/runner.py b/nanovllm/qwen38/runner.py -index 311336e..f9ff561 100644 ---- a/nanovllm/qwen38/runner.py -+++ b/nanovllm/qwen38/runner.py -@@ -195,8 +195,11 @@ class RequestRunner: - length = request.length - for layer in state.layers: - if getattr(layer, "capacity", 0): -- layer.key = layer.key[:, :, :length].clone() -- layer.value = layer.value[:, :, :length].clone() -+ # Linear-attention layers reserve conv/recurrent state, not KV. -+ # Preserve that history while switching every layer to eager. -+ if layer.key is not None: -+ layer.key = layer.key[:, :, :length].clone() -+ layer.value = layer.value[:, :, :length].clone() - layer.capacity = 0 - state.capacity = 0 - state.positions = None -diff --git a/tests/test_qwen38_graph_demotion.py b/tests/test_qwen38_graph_demotion.py -new file mode 100644 -index 0000000..93d74ba ---- /dev/null -+++ b/tests/test_qwen38_graph_demotion.py -@@ -0,0 +1,57 @@ -+"""CPU regression for hybrid state ownership at a decode graph boundary.""" -+ -+from types import SimpleNamespace -+ -+import pytest -+from test_qwen38_runner import Logits, MODULE, RequestRunner, Row -+ -+ -+class Buffer: -+ def __init__(self, values): -+ self.values = values -+ -+ def __getitem__(self, key): -+ return Buffer(self.values[key[2]]) -+ -+ def clone(self): -+ return Buffer(self.values.copy()) -+ -+ -+@pytest.mark.parametrize("linear_first", [True, False]) -+def test_graph_boundary_preserves_hybrid_state_and_invalidates_pool(linear_first): -+ key, value = Buffer([1, 2, 99]), Buffer([3, 4, 99]) -+ conv, recurrent = object(), object() -+ attention = SimpleNamespace(capacity=2, key=key, value=value) -+ linear = SimpleNamespace(capacity=2, key=None, value=None, -+ conv=conv, recurrent=recurrent, fresh=False) -+ layers = [linear, attention] if linear_first else [attention, linear] -+ state = SimpleNamespace(layers=layers, capacity=2, positions=object(), length=1) -+ -+ class Model: -+ def __call__(self, inputs, *, state): -+ assert state.capacity == 0 and state.positions is None -+ assert all(layer.capacity == 0 for layer in state.layers) -+ assert linear.key is linear.value is None -+ assert linear.conv is conv and linear.recurrent is recurrent -+ assert not linear.fresh -+ assert attention.key is not key and attention.value is not value -+ assert attention.key.values == [1, 2] -+ assert attention.value.values == [3, 4] -+ state.length += len(inputs[0]) -+ return Logits([Row([0., 1.])], 2) -+ -+ runner = RequestRunner(Model(), max_context_tokens=4, tensor_factory=lambda x: x) -+ graph = SimpleNamespace(capacity=2) -+ request = MODULE._Request(state, length=2, graph=graph, graph_bucket=2) -+ runner._requests["boundary"] = request -+ runner._graph_pool[2] = graph -+ runner._graph_busy.add(2) -+ runner.append("boundary", [1]) -+ assert request.length == state.length == 3 -+ assert request.graph is None and request.graph_bucket == 0 -+ assert not runner._graph_pool and not runner._graph_busy -+ assert runner._acquire_graph_state(2) is None -+ runner.append("boundary", [1]) -+ assert request.length == state.length == 4 -+ assert runner.release("boundary") -+ assert not runner.active_request_ids diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/graph-prefill.patch b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/graph-prefill.patch deleted file mode 100644 index 7fbba4a8f..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/graph-prefill.patch +++ /dev/null @@ -1,88 +0,0 @@ -diff --git a/nanovllm/qwen38/model.py b/nanovllm/qwen38/model.py -index 83cddbd..e710727 100644 ---- a/nanovllm/qwen38/model.py -+++ b/nanovllm/qwen38/model.py -@@ -159,7 +159,7 @@ class FullAttention(nn.Module): - self.q_norm = RMSNorm(config.head_dim, config.rms_norm_eps) - self.k_norm = RMSNorm(config.head_dim, config.rms_norm_eps) - -- def forward(self, x, state, positions, cos, sin): -+ def forward(self, x, state, positions, cos, sin, *, fresh_prefill=False): - config = self.config - count = x.shape[1] - # Q and gate are interleaved per head, not two contiguous projection halves. -@@ -190,8 +190,13 @@ class FullAttention(nn.Module): - # 14ms 非 GEMM 里最大的一块。改为 batch=4 的等形 bmm 后不再产生该拷贝。 - q_grouped = q.reshape(kv_heads, repeats * count, config.head_dim) - keys, values = ( -- (state.key, state.value) if state.capacity else (k, v) -+ (state.key, state.value) if state.capacity and not fresh_prefill else (k, v) - ) -+ # Initial prefill has no cached prefix. Keep its original K/V operands -+ # and reduction shape even when reserving storage for later graph decode. -+ # Reading the padded cache here changes BF16 rounding before capture and -+ # can change subsequent greedy tokens. The writes above still populate -+ # the fixed addresses consumed by replay. - keys_bt = keys.reshape(kv_heads, -1, config.head_dim).transpose(1, 2) - scores = torch.matmul(q_grouped, keys_bt).reshape( - 1, kv_heads, repeats, count, keys.shape[2] -@@ -427,12 +432,13 @@ class DecoderLayer(nn.Module): - self.post_attention_layernorm = RMSNorm(config.hidden_size, config.rms_norm_eps) - self.mlp = MLP(config) - -- def forward(self, x, state, positions, cos, sin): -+ def forward(self, x, state, positions, cos, sin, *, fresh_prefill=False): - normalized = self.input_layernorm(x) - mixed = ( - self.linear_attn(normalized, state) - if self.kind == "linear_attention" -- else self.self_attn(normalized, state, positions, cos, sin) -+ else self.self_attn(normalized, state, positions, cos, sin, -+ fresh_prefill=fresh_prefill) - ) - # 残差相加与 post-attention norm 折成一个 kernel:x ← x + mixed(原地), - # 同时产出归一化结果,少一次物化。数值语义与分开写相同。 -@@ -571,7 +577,8 @@ class Qwen38ForCausalLM(nn.Module): - cos, sin = rotary_tables(self.config, positions, hidden.dtype) - cos, sin = cos[None, None], sin[None, None] - for layer, layer_state in zip(self.model.layers, pending): -- hidden = layer(hidden, layer_state, positions, cos, sin) -+ hidden = layer(hidden, layer_state, positions, cos, sin, -+ fresh_prefill=state.length == 0 and state.positions is None) - logits = self.lm_head(self.model.norm(hidden)) - state.layers = pending - if state.positions is None: -diff --git a/tests/test_qwen38_graph_prefill.py b/tests/test_qwen38_graph_prefill.py -new file mode 100644 -index 0000000..ebe8702 ---- /dev/null -+++ b/tests/test_qwen38_graph_prefill.py -@@ -0,0 +1,28 @@ -+"""Reserving graph KV storage must not change initial prefill computation.""" -+import pytest -+import torch -+ -+from nanovllm.qwen38.config import Qwen38TextConfig -+from nanovllm.qwen38.model import Qwen38ForCausalLM -+ -+ -+@pytest.mark.parametrize("dtype", [torch.float32, torch.bfloat16]) -+@pytest.mark.parametrize("length", [1, 7, 65]) -+def test_initial_prefill_is_independent_of_reserved_decode_capacity(dtype, length): -+ torch.manual_seed(17) -+ model = Qwen38ForCausalLM(Qwen38TextConfig.for_testing()).to(dtype).eval() -+ tokens = torch.randint(0, model.config.vocab_size, (1, length)) -+ compact, buffered = model.new_state(), model.new_state(capacity=256) -+ a, b = model(tokens, compact), model(tokens, buffered) -+ assert torch.equal(a, b) -+ assert compact.length == buffered.length == length -+ for left, right in zip(compact.layers, buffered.layers): -+ for field in ("key", "value", "conv", "recurrent"): -+ x, y = getattr(left, field), getattr(right, field) -+ if x is None: -+ assert y is None -+ else: -+ if field in ("key", "value"): -+ assert y.shape[2] == 256 -+ y = y[:, :, :length] -+ assert torch.equal(x, y) diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/publication-repair.patch b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/publication-repair.patch deleted file mode 100644 index 5713ab5f4..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/publication-repair.patch +++ /dev/null @@ -1,154 +0,0 @@ -diff --git a/nanovllm/qwen38/engine.py b/nanovllm/qwen38/engine.py -index 3976480..1d770cb 100644 ---- a/nanovllm/qwen38/engine.py -+++ b/nanovllm/qwen38/engine.py -@@ -181,6 +181,9 @@ class DedicatedEngine: - if isinstance(module, FP8Linear)} - calls = {name: module.native_fp8_calls for name, module in modules.items()} - execution = {"fp8_execution": {"modules": len(calls), "total_calls": sum(calls.values()), -+ "call_counter_scope": "python_forward_only_includes_warmup_and_capture_excludes_graph_replay", -+ "backend": self.loader_report["backend"], -+ "kernel_name": self.loader_report["kernel_name"], - "executed_modules": sum(count > 0 for count in calls.values()), "per_module_calls": calls, - "weight_dtype": "float8_e4m3fn", "scale_dtype": "float32", - "actual_dtypes_valid": all(module.weight.dtype == torch.float8_e4m3fn -diff --git a/nanovllm/qwen38/model.py b/nanovllm/qwen38/model.py -index 83cddbd..e710727 100644 ---- a/nanovllm/qwen38/model.py -+++ b/nanovllm/qwen38/model.py -@@ -159,7 +159,7 @@ class FullAttention(nn.Module): - self.q_norm = RMSNorm(config.head_dim, config.rms_norm_eps) - self.k_norm = RMSNorm(config.head_dim, config.rms_norm_eps) - -- def forward(self, x, state, positions, cos, sin): -+ def forward(self, x, state, positions, cos, sin, *, fresh_prefill=False): - config = self.config - count = x.shape[1] - # Q and gate are interleaved per head, not two contiguous projection halves. -@@ -190,8 +190,13 @@ class FullAttention(nn.Module): - # 14ms 非 GEMM 里最大的一块。改为 batch=4 的等形 bmm 后不再产生该拷贝。 - q_grouped = q.reshape(kv_heads, repeats * count, config.head_dim) - keys, values = ( -- (state.key, state.value) if state.capacity else (k, v) -+ (state.key, state.value) if state.capacity and not fresh_prefill else (k, v) - ) -+ # Initial prefill has no cached prefix. Keep its original K/V operands -+ # and reduction shape even when reserving storage for later graph decode. -+ # Reading the padded cache here changes BF16 rounding before capture and -+ # can change subsequent greedy tokens. The writes above still populate -+ # the fixed addresses consumed by replay. - keys_bt = keys.reshape(kv_heads, -1, config.head_dim).transpose(1, 2) - scores = torch.matmul(q_grouped, keys_bt).reshape( - 1, kv_heads, repeats, count, keys.shape[2] -@@ -427,12 +432,13 @@ class DecoderLayer(nn.Module): - self.post_attention_layernorm = RMSNorm(config.hidden_size, config.rms_norm_eps) - self.mlp = MLP(config) - -- def forward(self, x, state, positions, cos, sin): -+ def forward(self, x, state, positions, cos, sin, *, fresh_prefill=False): - normalized = self.input_layernorm(x) - mixed = ( - self.linear_attn(normalized, state) - if self.kind == "linear_attention" -- else self.self_attn(normalized, state, positions, cos, sin) -+ else self.self_attn(normalized, state, positions, cos, sin, -+ fresh_prefill=fresh_prefill) - ) - # 残差相加与 post-attention norm 折成一个 kernel:x ← x + mixed(原地), - # 同时产出归一化结果,少一次物化。数值语义与分开写相同。 -@@ -571,7 +577,8 @@ class Qwen38ForCausalLM(nn.Module): - cos, sin = rotary_tables(self.config, positions, hidden.dtype) - cos, sin = cos[None, None], sin[None, None] - for layer, layer_state in zip(self.model.layers, pending): -- hidden = layer(hidden, layer_state, positions, cos, sin) -+ hidden = layer(hidden, layer_state, positions, cos, sin, -+ fresh_prefill=state.length == 0 and state.positions is None) - logits = self.lm_head(self.model.norm(hidden)) - state.layers = pending - if state.positions is None: -diff --git a/nanovllm/qwen38/quant.py b/nanovllm/qwen38/quant.py -index 34863c5..6df0144 100644 ---- a/nanovllm/qwen38/quant.py -+++ b/nanovllm/qwen38/quant.py -@@ -2,6 +2,8 @@ - - from __future__ import annotations - -+import os -+ - import torch - from torch import nn - -@@ -21,6 +23,12 @@ class FP8Linear(nn.Module): - - def __init__(self, in_features: int, out_features: int, *, device=None): - super().__init__() -+ requested = os.environ.get("NANOVLLM_FP8_GEMM", "triton") -+ if requested not in ("triton", self.backend): -+ raise ValueError( -+ f"Unsupported NANOVLLM_FP8_GEMM={requested!r}; this build implements " -+ f"only {self.backend}. No fallback was selected." -+ ) - if in_features <= 0 or out_features <= 0: - raise ValueError("FP8 linear dimensions must be positive") - self.in_features = in_features -diff --git a/tests/test_qwen38_graph_prefill.py b/tests/test_qwen38_graph_prefill.py -new file mode 100644 -index 0000000..ebe8702 ---- /dev/null -+++ b/tests/test_qwen38_graph_prefill.py -@@ -0,0 +1,28 @@ -+"""Reserving graph KV storage must not change initial prefill computation.""" -+import pytest -+import torch -+ -+from nanovllm.qwen38.config import Qwen38TextConfig -+from nanovllm.qwen38.model import Qwen38ForCausalLM -+ -+ -+@pytest.mark.parametrize("dtype", [torch.float32, torch.bfloat16]) -+@pytest.mark.parametrize("length", [1, 7, 65]) -+def test_initial_prefill_is_independent_of_reserved_decode_capacity(dtype, length): -+ torch.manual_seed(17) -+ model = Qwen38ForCausalLM(Qwen38TextConfig.for_testing()).to(dtype).eval() -+ tokens = torch.randint(0, model.config.vocab_size, (1, length)) -+ compact, buffered = model.new_state(), model.new_state(capacity=256) -+ a, b = model(tokens, compact), model(tokens, buffered) -+ assert torch.equal(a, b) -+ assert compact.length == buffered.length == length -+ for left, right in zip(compact.layers, buffered.layers): -+ for field in ("key", "value", "conv", "recurrent"): -+ x, y = getattr(left, field), getattr(right, field) -+ if x is None: -+ assert y is None -+ else: -+ if field in ("key", "value"): -+ assert y.shape[2] == 256 -+ y = y[:, :, :length] -+ assert torch.equal(x, y) -diff --git a/tests/test_qwen38_quant.py b/tests/test_qwen38_quant.py -index 52e9016..8dd0767 100644 ---- a/tests/test_qwen38_quant.py -+++ b/tests/test_qwen38_quant.py -@@ -12,6 +12,21 @@ from nanovllm.qwen38.model import Qwen38ForCausalLM - from nanovllm.qwen38.quant import FP8Linear, convert_fp8_linears - - -+@pytest.mark.parametrize("backend", ["flashinfer", "deepgemm", "typo"]) -+def test_unimplemented_backend_is_not_silently_ignored(monkeypatch, backend): -+ monkeypatch.setenv("NANOVLLM_FP8_GEMM", backend) -+ with pytest.raises(ValueError, match="Unsupported NANOVLLM_FP8_GEMM"): -+ FP8Linear(128, 128, device="meta") -+ -+ -+@pytest.mark.parametrize("backend", ["triton", "triton_block_fp8"]) -+def test_supported_backend_aliases_report_implemented_kernel(monkeypatch, backend): -+ monkeypatch.setenv("NANOVLLM_FP8_GEMM", backend) -+ module = FP8Linear(128, 128, device="meta") -+ assert module.backend == "triton_block_fp8" -+ assert module.kernel_name == "qwen38_block_fp8" -+ -+ - def tiny_fp8_model(): - model = Qwen38ForCausalLM(Qwen38TextConfig.for_testing()).to(torch.bfloat16) - exclusions = ("lm_head", "model.language_model.layers.0.linear_attn.in_proj_a", diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/quality-check_score_v5.py b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/quality-check_score_v5.py deleted file mode 100644 index ad6870114..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/quality-check_score_v5.py +++ /dev/null @@ -1,240 +0,0 @@ -# ruff: noqa: E402 -"""Exercise full export/scoring binding with one handwritten synthetic response.""" - -import hashlib -import json -import os -import sys -import tempfile -from pathlib import Path - -ROOT = Path(__file__).parent -sys.path.insert(0, os.environ.get("QUALITY_SOURCE_DIR", str(ROOT.parent / "quality-source-v5"))) -from prepare import EVALPLUS_REV -from protocol import canonical, sha, validator_identity -from run import parse_events, termination_policy -from score import score - - -def write(path, value): - path.write_bytes(canonical(value)) - - -def main(): - with tempfile.TemporaryDirectory(prefix="score-fixture-", dir=ROOT) as temporary: - root = Path(temporary) - prepared, tokenized, run = [root / name for name in ("prepared", "tokenized", "run")] - for directory in (prepared, tokenized, run): - directory.mkdir() - config = root / "generation.json" - write(config, {"eos_token_id": [248046]}) - config_sha = sha(config.read_bytes()) - labels = {"gsm8k": {}, "humaneval_plus": {"Synthetic/fixture": "fixture"}, "mmlu": {}} - write(prepared / "labels.json", labels) - (prepared / "humaneval_plus.jsonl").write_bytes( - canonical({"task_id": "Synthetic/fixture", "fixture_only": True}) + b"\n" - ) - write( - prepared / "prepared.json", - { - "files": [ - {"path": name, "sha256": sha((prepared / name).read_bytes())} - for name in ("labels.json", "humaneval_plus.jsonl") - ] - }, - ) - request = { - "id": "Synthetic/fixture", - "task": "humaneval_plus", - "screen": True, - "prompt": [2, 3], - "prompt_sha256": sha(canonical([2, 3])), - } - (tokenized / "tokens.jsonl").write_bytes(canonical(request) + b"\n") - write( - tokenized / "tokenized.json", - { - "prepared_sha256": sha((prepared / "prepared.json").read_bytes()), - "tokens_sha256": sha((tokenized / "tokens.jsonl").read_bytes()), - }, - ) - text = "```python\ndef f(x):\n return x + 1\n```" - choice = { - "index": 0, - "text": text, - "token_ids": [8, 248046], - "prompt_token_ids": [2, 3], - "finish_reason": "stop", - "stop_reason": None, - } - events = [ - {"choices": [choice]}, - { - "choices": [], - "usage": {"prompt_tokens": 2, "completion_tokens": 2, "total_tokens": 4}, - }, - ] - wire = ( - b"".join(b"data: " + canonical(row) + b"\n\n" for row in events) + b"data: [DONE]\n\n" - ) - (run / "0.sse").write_bytes(wire) - actual = parse_events(wire, [2, 3], eos_token_ids=[248046]) - response = { - "id": request["id"], - "task": request["task"], - "status": "completed", - "raw_path": "0.sse", - "raw_sha256": sha(wire), - "prompt_sha256": request["prompt_sha256"], - **actual, - } - (run / "responses.jsonl").write_bytes(canonical(response) + b"\n") - write( - run / "run.json", - { - "phase": "screen", - "arm": "baseline", - "expected_ids": [request["id"]], - "tokenized_manifest_sha256": sha((tokenized / "tokenized.json").read_bytes()), - "validator_files": validator_identity(), - "termination_policy": termination_policy(config, config_sha), - }, - ) - exported = root / "export" - score(prepared, tokenized, run, exported, config, config_sha) - sample = json.loads((exported / "humaneval-solutions.jsonl").read_bytes()) - subset = (exported / "humaneval-subset.jsonl").read_bytes() - human = root / "human-results.json" - write( - human, - { - "hash": hashlib.md5(subset, usedforsecurity=False).hexdigest(), - "eval": { - request["id"]: [ - { - "solution": sample["solution"], - "base_status": "pass", - "plus_status": "pass", - } - ] - }, - }, - ) - identity = { - "schema_version": 1, - "runtime_kind": "readonly-bind-set-v1", - "runtime_digest": "sha256:" + "a" * 64, - "evaluator_revision": EVALPLUS_REV, - "executor_sha256": "b" * 64, - "driver_sha256": "c" * 64, - "sandbox_binary_sha256": "d" * 64, - "policy_sha256": "e" * 64, - } - qualification, protocol_file, record_file = [ - root / name for name in ("qualification.json", "protocol.json", "record.json") - ] - write( - qualification, - { - "scope": "generated-code-executor", - "verdict": "PASS", - "fixture_only": True, - "qualified_generated_code_executor": True, - "identity": identity, - }, - ) - protocol = {**identity, "qualification_sha256": sha(qualification.read_bytes())} - write(protocol_file, protocol) - record = { - "protocol": protocol, - "network": "none", - "exit_code": 0, - "timed_out": False, - "cleanup_complete": True, - "qualified_generated_code_executor": True, - "results_sha256": sha(human.read_bytes()), - "samples_sha256": sha((exported / "humaneval-solutions.jsonl").read_bytes()), - "dataset_sha256": sha(subset), - } - - def call(name): - score( - prepared, - tokenized, - run, - root / name, - config, - config_sha, - human, - record_file, - protocol_file, - sha(protocol_file.read_bytes()), - qualification, - ) - - write(record_file, record) - call("valid") - summary = json.loads((root / "valid/summary.json").read_bytes()) - if summary["sandbox_protocol"] != protocol or summary["sandbox_record_sha256"] != sha( - record_file.read_bytes() - ): - raise RuntimeError("Scorer did not retain bound identities") - checks = ["export-without-evaluation", "full-score-binds-frozen-protocol"] - - class Once: - def __init__(self, path): - self.raw = path.read_bytes() - self.read = False - - def read_bytes(self): - if self.read: - raise RuntimeError("Scoring evidence was read more than once") - self.read = True - return self.raw - - human_once, record_once = Once(human), Once(record_file) - score( - prepared, - tokenized, - run, - root / "single-read", - config, - config_sha, - human_once, - record_once, - Once(protocol_file), - sha(protocol_file.read_bytes()), - Once(qualification), - ) - observed = json.loads((root / "single-read/summary.json").read_bytes()) - if observed["human_results_sha256"] != sha(human_once.raw) or observed[ - "sandbox_record_sha256" - ] != sha(record_once.raw): - raise RuntimeError("Scoring evidence digests differ from parsed snapshots") - checks.append("full-score-reads-provenance-once") - for field, value in ( - ("protocol", {**protocol, "policy_sha256": "0" * 64}), - ("samples_sha256", "0" * 64), - ("dataset_sha256", "0" * 64), - ): - write(record_file, {**record, field: value}) - try: - call("invalid-" + field) - except ValueError: - checks.append("full-score-rejects-" + field) - else: - raise RuntimeError("Scorer accepted mismatched evidence") - report = { - "scope": "handwritten_synthetic_scoring_fixture", - "verdict": "INCONCLUSIVE", - "checks": checks, - "checks_passed": len(checks), - "real_evaluator_executed": False, - "real_quality_result": False, - } - (ROOT / "v5-score-controls.json").write_text(json.dumps(report, indent=2) + "\n") - print(json.dumps(report)) - - -if __name__ == "__main__": - main() diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/quality-check_v5.py b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/quality-check_v5.py deleted file mode 100644 index 2d286a1cd..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/quality-check_v5.py +++ /dev/null @@ -1,250 +0,0 @@ -# ruff: noqa: E402 -"""Fabricated provenance/control fixtures only; no benchmark data or execution.""" - -import copy -import hashlib -import json -import os -import sys -import tempfile -from pathlib import Path - -ROOT = Path(__file__).parent -sys.path.insert(0, os.environ.get("QUALITY_SOURCE_DIR", str(ROOT.parent / "quality-source-v5"))) -from prepare import EVALPLUS_REV -from protocol import FINAL_COUNTS -from sandbox_provenance import execution_record, frozen_protocol -from score import compare - - -def digest(path): - return hashlib.sha256(path.read_bytes()).hexdigest() - - -def dump(path, value): - path.write_text(json.dumps(value, sort_keys=True)) - - -def main(): - passed = [] - - def rejects(label, operation): - try: - operation() - except (ValueError, TypeError, FileNotFoundError): - passed.append(label) - else: - raise RuntimeError(f"Invalid fixture accepted: {label}") - - with tempfile.TemporaryDirectory(prefix="synthetic-v5-", dir=ROOT) as temporary: - root = Path(temporary) - p, q, r, h = [ - root / name - for name in ("protocol.json", "qualification.json", "execution.json", "results.json") - ] - identity = { - "schema_version": 1, - "runtime_kind": "readonly-bind-set-v1", - "runtime_digest": "sha256:" + "a" * 64, - "evaluator_revision": EVALPLUS_REV, - "executor_sha256": "b" * 64, - "driver_sha256": "c" * 64, - "sandbox_binary_sha256": "d" * 64, - "policy_sha256": "e" * 64, - } - qualification = { - "scope": "generated-code-executor", - "verdict": "PASS", - "qualified_generated_code_executor": True, - "identity": identity, - "fixture_only": True, - } - dump(q, qualification) - protocol = {**identity, "qualification_sha256": digest(q)} - dump(p, protocol) - pinned = digest(p) - dump(h, {"fixture_only": True}) - execution = { - "protocol": protocol, - "network": "none", - "exit_code": 0, - "timed_out": False, - "cleanup_complete": True, - "qualified_generated_code_executor": True, - "results_sha256": digest(h), - "samples_sha256": "1" * 64, - "dataset_sha256": "2" * 64, - } - dump(r, execution) - if ( - frozen_protocol(p, pinned, q, EVALPLUS_REV) != protocol - or execution_record(r.read_bytes(), protocol, h.read_bytes()) != execution - ): - raise RuntimeError("Valid synthetic fixture rejected") - passed.append("valid-synthetic-binding") - rejects( - "missing-independent-protocol", lambda: frozen_protocol(None, pinned, q, EVALPLUS_REV) - ) - rejects("wrong-protocol-file-hash", lambda: frozen_protocol(p, "0" * 64, q, EVALPLUS_REV)) - rejects("missing-qualification", lambda: frozen_protocol(p, pinned, None, EVALPLUS_REV)) - dump(q, {**qualification, "verdict": "INCONCLUSIVE"}) - rejects("changed-qualification-bytes", lambda: frozen_protocol(p, pinned, q, EVALPLUS_REV)) - for name, changed in ( - ("benign-scope", {**qualification, "scope": "trusted-benign-probe"}), - ("unqualified", {**qualification, "qualified_generated_code_executor": False}), - ("failed-qualification", {**qualification, "verdict": "FAIL"}), - ( - "wrong-qualified-runtime", - {**qualification, "identity": {**identity, "runtime_digest": "sha256:" + "f" * 64}}, - ), - ): - dump(q, changed) - dump(p, {**protocol, "qualification_sha256": digest(q)}) - rejects(name, lambda: frozen_protocol(p, digest(p), q, EVALPLUS_REV)) - dump(q, qualification) - dump(p, protocol) - for field, value in ( - ("schema_version", True), - ("runtime_kind", "oci"), - ("runtime_digest", "image:placeholder"), - ("evaluator_revision", "wrong"), - ): - dump(p, {**protocol, field: value}) - rejects("bad-protocol-" + field, lambda: frozen_protocol(p, digest(p), q, EVALPLUS_REV)) - p.write_text(json.dumps(protocol)[:-1] + ', "schema_version":1}') - rejects("duplicate-protocol-key", lambda: frozen_protocol(p, digest(p), q, EVALPLUS_REV)) - dump(p, protocol) - for field, value in ( - ("protocol", {**protocol, "schema_version": True}), - ("network", "host"), - ("exit_code", False), - ("exit_code", 1), - ("timed_out", True), - ("cleanup_complete", False), - ("qualified_generated_code_executor", False), - ("results_sha256", "0" * 64), - ("samples_sha256", None), - ): - dump(r, {**execution, field: value}) - rejects( - "bad-execution-" + field + "-" + str(value)[:12], - lambda: execution_record(r.read_bytes(), protocol, h.read_bytes()), - ) - r.write_text(json.dumps(execution)[:-1] + ', "exit_code":0}') - rejects( - "duplicate-execution-key", - lambda: execution_record(r.read_bytes(), protocol, h.read_bytes()), - ) - dump(r, execution) - - def summary(arm): - return { - "phase": "final", - "evaluation_complete": True, - "arm": arm, - "prepared_manifest_sha256": "3" * 64, - "tokenized_manifest_sha256": "4" * 64, - "validator_files": {"fixture": True}, - "termination_policy": {"fixture": True}, - "sandbox_protocol": protocol, - "sandbox_protocol_sha256": pinned, - "run_manifest_sha256": ("5" if arm == "baseline" else "6") * 64, - "responses_sha256": ("7" if arm == "baseline" else "8") * 64, - "sandbox_record_sha256": ("9" if arm == "baseline" else "a") * 64, - "human_results_sha256": ("b" if arm == "baseline" else "c") * 64, - "counts": { - task: {"expected": n, "completed": n, "correct": 0, "infrastructure_errors": 0} - for task, n in FINAL_COUNTS.items() - }, - } - - b, c, out = root / "baseline.json", root / "candidate.json", root / "comparison.json" - - def call_compare(): - compare(b, c, out, p, pinned, q) - - for variant in ("missing", "different", "same-wrong", "wrong-freeze-hash", "bool-version"): - left, right = summary("baseline"), summary("candidate") - if variant == "missing": - left.pop("sandbox_protocol") - right.pop("sandbox_protocol") - elif variant == "wrong-freeze-hash": - left["sandbox_protocol_sha256"] = right["sandbox_protocol_sha256"] = "0" * 64 - else: - right["sandbox_protocol"] = copy.deepcopy(protocol) - field = "schema_version" if variant == "bool-version" else "runtime_digest" - right["sandbox_protocol"][field] = ( - True if variant == "bool-version" else "sha256:" + "f" * 64 - ) - if variant == "same-wrong": - left["sandbox_protocol"] = copy.deepcopy(right["sandbox_protocol"]) - dump(b, left) - dump(c, right) - rejects("comparison-" + variant, call_compare) - for arm, target in (("baseline", b), ("candidate", c)): - for key in ( - "run_manifest_sha256", - "responses_sha256", - "prepared_manifest_sha256", - "tokenized_manifest_sha256", - "sandbox_record_sha256", - "human_results_sha256", - "sandbox_protocol_sha256", - ): - for value in ("missing", None, "", "G" * 64, "a" * 63, 0): - dump(b, summary("baseline")) - dump(c, summary("candidate")) - broken = summary(arm) - if value == "missing": - broken.pop(key) - else: - broken[key] = value - dump(target, broken) - rejects(f"{arm}-{key}-{value}", call_compare) - dump(b, summary("baseline")) - dump(c, summary("candidate")) - b.write_text(b.read_text()[:-1] + ', "evaluation_complete":true}') - rejects("duplicate-summary-key", call_compare) - dump(b, summary("baseline")) - call_compare() - if json.loads(out.read_bytes())["verdict"] != "PASS": - raise RuntimeError("Valid synthetic comparator control failed") - passed.append("comparison-matched-frozen-identity-distinct-arm-digests") - - class Once: - def __init__(self, path): - self.raw = path.read_bytes() - self.read = False - - def read_bytes(self): - if self.read: - raise RuntimeError("Evidence was read more than once") - self.read = True - return self.raw - - if frozen_protocol(Once(p), pinned, Once(q), EVALPLUS_REV) != protocol: - raise RuntimeError("Frozen snapshot control failed") - passed.append("protocol-and-qualification-single-read") - b_once, c_once = Once(b), Once(c) - out = root / "comparison-single-read.json" - compare(b_once, c_once, out, Once(p), pinned, Once(q)) - result = json.loads(out.read_bytes()) - if result["baseline_summary_sha256"] != hashlib.sha256(b_once.raw).hexdigest(): - raise RuntimeError("Baseline summary binding differs") - if result["candidate_summary_sha256"] != hashlib.sha256(c_once.raw).hexdigest(): - raise RuntimeError("Candidate summary binding differs") - passed.append("summary-digests-bind-parsed-snapshot") - report = { - "scope": "fabricated_provenance_fixtures_only", - "verdict": "INCONCLUSIVE", - "checks_passed": len(passed), - "checks": passed, - "real_evaluator_executed": False, - "qualification_established": False, - } - (ROOT / "v5-controls.json").write_text(json.dumps(report, indent=2) + "\n") - print(json.dumps(report)) - - -if __name__ == "__main__": - main() diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/quality-eos-v4.patch b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/quality-eos-v4.patch deleted file mode 100644 index 7e0c89449..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/quality-eos-v4.patch +++ /dev/null @@ -1,219 +0,0 @@ ---- a/run.py -+++ b/run.py -@@ -16,12 +16,36 @@ - pass - - --def parse_events(raw, prompt_input, max_tokens=2048): -+def termination_policy(generation_config, expected_sha256): -+ raw = Path(generation_config).read_bytes() -+ if len(expected_sha256) != 64 or sha(raw) != expected_sha256: -+ raise ValueError("generation config differs from declared source identity") -+ config = json.loads(raw) -+ eos = config.get("eos_token_id") -+ eos = [eos] if type(eos) is int else eos -+ if not isinstance(eos, list) or not eos or any(type(t) is not int or t < 0 for t in eos): -+ raise ValueError("invalid generation EOS token IDs") -+ return { -+ "mode": "natural_eos_no_user_stop", -+ "generation_config_sha256": sha(raw), -+ "eos_token_ids": sorted(set(eos)), -+ "ignore_eos": False, -+ } -+ -+ -+def parse_events(raw, prompt_input, max_tokens=2048, *, eos_token_ids=None): - expected_prompt = prompt_input if isinstance(prompt_input, list) else None - prompt_length = len(prompt_input) if expected_prompt is not None else prompt_input - observed_prompt = None - text, tokens, terminal, usage = [], [], None, None - done, has_tokens = False, False -+ stop_reason = None -+ if eos_token_ids is not None and ( -+ not isinstance(eos_token_ids, (list, tuple)) -+ or not eos_token_ids -+ or any(type(t) is not int or t < 0 for t in eos_token_ids) -+ ): -+ raise ProtocolError("invalid authenticated EOS allowlist") - for message in raw.replace(b"\r\n", b"\n").split(b"\n\n"): - data = [line[5:].lstrip() for line in message.split(b"\n") if line.startswith(b"data:")] - if not data: -@@ -59,6 +83,9 @@ - tokens.extend(ids) - if choice.get("finish_reason") is not None: - terminal = choice["finish_reason"] -+ stop_reason = choice.get("stop_reason") -+ elif choice.get("stop_reason") is not None: -+ raise ProtocolError("stop reason before terminal choice") - event_usage = event.get("usage") - if event_usage is not None: - if usage is not None or choices or terminal is None: -@@ -82,6 +109,18 @@ - raise ProtocolError("output token IDs differ from usage count") - if expected_prompt is not None and (observed_prompt != expected_prompt or not has_tokens): - raise ProtocolError("missing or changed actual input/output token identity") -+ if eos_token_ids is not None: -+ eos = set(eos_token_ids) -+ if not has_tokens or not tokens or any(token in eos for token in tokens[:-1]): -+ raise ProtocolError("missing output identity or continued generation after EOS") -+ if terminal == "stop": -+ if tokens[-1] not in eos or ( -+ stop_reason is not None -+ and (type(stop_reason) is not int or stop_reason != tokens[-1]) -+ ): -+ raise ProtocolError("stop does not identify an authenticated EOS") -+ elif tokens[-1] in eos or stop_reason is not None: -+ raise ProtocolError("length terminal conflicts with natural EOS policy") - return { - "text": "".join(text), - "token_ids": tokens if has_tokens else None, -@@ -91,7 +130,7 @@ - } - - --def complete(endpoint, body, raw_path, timeout): -+def complete(endpoint, body, raw_path, timeout, *, eos_token_ids): - target = urlsplit(endpoint) - if ( - target.scheme != "http" -@@ -128,7 +167,12 @@ - if not response.getheader("Content-Type", "").startswith("text/event-stream"): - raise ProtocolError("expected SSE content type") - return { -- **parse_events(raw_path.read_bytes(), body["prompt"], body["max_tokens"]), -+ **parse_events( -+ raw_path.read_bytes(), -+ body["prompt"], -+ body["max_tokens"], -+ eos_token_ids=eos_token_ids, -+ ), - "request_seconds": time.monotonic() - start, - } - finally: -@@ -153,6 +197,8 @@ - def main(): - parser = argparse.ArgumentParser(description=__doc__) - parser.add_argument("--tokenized", type=Path, required=True) -+ parser.add_argument("--generation-config", type=Path, required=True) -+ parser.add_argument("--generation-config-sha256", required=True) - parser.add_argument("--endpoint", required=True) - parser.add_argument("--served-model", required=True) - parser.add_argument("--phase", choices=("transport_smoke", "screen", "final"), required=True) -@@ -163,6 +209,7 @@ - args = parser.parse_args() - if args.timeout <= 0: - parser.error("positive absolute request timeout required") -+ stopping = termination_policy(args.generation_config, args.generation_config_sha256) - manifest = json.loads((args.tokenized / "tokenized.json").read_bytes()) - token_file = args.tokenized / "tokens.jsonl" - if sha(token_file.read_bytes()) != manifest["tokens_sha256"]: -@@ -188,6 +235,7 @@ - "stream_options": {"include_usage": True}, - }, - "performance_measurement": False, -+ "termination_policy": stopping, - }, - ) - with (args.output / "responses.jsonl").open("xb") as responses: -@@ -211,7 +259,15 @@ - "raw_path": raw.name, - } - try: -- result.update(complete(args.endpoint, payload, raw, args.timeout)) -+ result.update( -+ complete( -+ args.endpoint, -+ payload, -+ raw, -+ args.timeout, -+ eos_token_ids=stopping["eos_token_ids"], -+ ) -+ ) - result["status"] = "completed" - except Exception as error: - # No endpoint, error text or runtime path in the shareable summary. ---- a/score.py -+++ b/score.py -@@ -17,15 +17,27 @@ - unique, - validator_identity, - ) --from run import parse_events, select -- -- --def score(prepared, tokenized, run, output, human_results=None, sandbox_record=None): -+from run import parse_events, select, termination_policy -+ -+ -+def score( -+ prepared, -+ tokenized, -+ run, -+ output, -+ generation_config, -+ generation_config_sha256, -+ human_results=None, -+ sandbox_record=None, -+): - prepared_manifest = json.loads((prepared / "prepared.json").read_bytes()) - for item in prepared_manifest["files"]: - if sha((prepared / item["path"]).read_bytes()) != item["sha256"]: - raise ValueError("prepared inputs changed") - frozen = json.loads((run / "run.json").read_bytes()) -+ stopping = termination_policy(generation_config, generation_config_sha256) -+ if frozen.get("termination_policy") != stopping: -+ raise ValueError("run termination policy differs from authenticated configuration") - token_manifest = json.loads((tokenized / "tokenized.json").read_bytes()) - if ( - sha((tokenized / "tokenized.json").read_bytes()) != frozen["tokenized_manifest_sha256"] -@@ -91,7 +103,9 @@ - request = by_id[sample_id] - if row["prompt_sha256"] != request["prompt_sha256"]: - raise ValueError("response binds another prompt") -- actual = parse_events(raw.read_bytes(), request["prompt"]) -+ actual = parse_events( -+ raw.read_bytes(), request["prompt"], eos_token_ids=stopping["eos_token_ids"] -+ ) - if any(actual[key] != row[key] for key in actual): - raise ValueError("response summary differs from raw SSE") - counts[task]["completed"] += 1 -@@ -177,6 +191,7 @@ - "prepared_manifest_sha256": sha((prepared / "prepared.json").read_bytes()), - "tokenized_manifest_sha256": frozen["tokenized_manifest_sha256"], - "validator_files": validator_identity(), -+ "termination_policy": stopping, - "human_results_sha256": sha(human_results.read_bytes()) if human_results else None, - }, - ) -@@ -190,7 +205,12 @@ - raise ValueError("wrong arm identities") - if any( - b[key] != c[key] -- for key in ("prepared_manifest_sha256", "tokenized_manifest_sha256", "validator_files") -+ for key in ( -+ "prepared_manifest_sha256", -+ "tokenized_manifest_sha256", -+ "validator_files", -+ "termination_policy", -+ ) - ): - raise ValueError("datasets/prompts/scorers differ") - result = compare_counts(b["counts"], c["counts"]) -@@ -205,6 +225,8 @@ - sub = parser.add_subparsers(dest="command", required=True) - p = sub.add_parser("score") - p.add_argument("--prepared", type=Path, required=True) -+ p.add_argument("--generation-config", type=Path, required=True) -+ p.add_argument("--generation-config-sha256", required=True) - p.add_argument("--tokenized", type=Path, required=True) - p.add_argument("--run", type=Path, required=True) - p.add_argument("--human-results", type=Path) -@@ -221,6 +243,8 @@ - args.tokenized, - args.run, - args.output, -+ args.generation_config, -+ args.generation_config_sha256, - args.human_results, - args.sandbox_record, - ) diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/quality-null-echo.patch b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/quality-null-echo.patch deleted file mode 100644 index cee69bfbf..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/quality-null-echo.patch +++ /dev/null @@ -1,11 +0,0 @@ ---- a/run.py -+++ b/run.py -@@ -44,7 +44,7 @@ - choice = choices[0] - if choice.get("index") != 0 or not isinstance(choice.get("text"), str): - raise ProtocolError("invalid choice") -- if "prompt_token_ids" in choice: -+ if choice.get("prompt_token_ids") is not None: - if observed_prompt is not None or text: - raise ProtocolError("prompt IDs must occur once in the first choice") - observed_prompt = choice["prompt_token_ids"] diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/quality-sandbox-binding-v5.patch b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/quality-sandbox-binding-v5.patch deleted file mode 100644 index c54baa645..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/quality-sandbox-binding-v5.patch +++ /dev/null @@ -1,311 +0,0 @@ ---- a/protocol.py -+++ b/protocol.py -@@ -26,7 +26,14 @@ - def validator_identity(): - return { - name: sha((Path(__file__).parent / name).read_bytes()) -- for name in ("protocol.py", "prepare.py", "run.py", "score.py", "source-lock.json") -+ for name in ( -+ "protocol.py", -+ "prepare.py", -+ "run.py", -+ "score.py", -+ "sandbox_provenance.py", -+ "source-lock.json", -+ ) - } - - ---- a/score.py -+++ b/score.py -@@ -3,7 +3,6 @@ - import argparse - import hashlib - import json --import re - from pathlib import Path - - from prepare import jsonl_lines, lines, write -@@ -18,6 +17,13 @@ - validator_identity, - ) - from run import parse_events, select, termination_policy -+from sandbox_provenance import ( -+ digest, -+ document, -+ execution_record, -+ frozen_protocol, -+ same_identity, -+) - - - def score( -@@ -29,6 +35,9 @@ - generation_config_sha256, - human_results=None, - sandbox_record=None, -+ sandbox_protocol=None, -+ sandbox_protocol_sha256=None, -+ sandbox_qualification=None, - ): - prepared_manifest = json.loads((prepared / "prepared.json").read_bytes()) - for item in prepared_manifest["files"]: -@@ -52,24 +61,21 @@ - expected = frozen["expected_ids"] - if expected != [row["id"] for row in requests]: - raise ValueError("run cohort differs from frozen inputs") -- if set(r["id"] for r in records) - set(expected): -+ if {r["id"] for r in records} - set(expected): - raise ValueError("unexpected response IDs") - labels = json.loads((prepared / "labels.json").read_bytes()) -- human = json.loads(human_results.read_bytes()) if human_results else None -+ human_raw = human_results.read_bytes() if human_results else None -+ human = document(human_raw) if human_raw is not None else None -+ sandbox_raw = None -+ evaluator_protocol = None - if human is not None: -- if sandbox_record is None: -- raise ValueError("sandbox execution provenance required") -- sandbox = json.loads(sandbox_record.read_bytes()) - from prepare import EVALPLUS_REV - -- if ( -- sandbox.get("evaluator_revision") != EVALPLUS_REV -- or sandbox.get("network") != "none" -- or sandbox.get("exit_code") != 0 -- or re.fullmatch(r"sha256:[0-9a-f]{64}", str(sandbox.get("image_digest", ""))) is None -- or sandbox.get("results_sha256") != sha(human_results.read_bytes()) -- ): -- raise ValueError("unqualified sandbox evidence") -+ evaluator_protocol = frozen_protocol( -+ sandbox_protocol, sandbox_protocol_sha256, sandbox_qualification, EVALPLUS_REV -+ ) -+ sandbox_raw = sandbox_record.read_bytes() if sandbox_record is not None else None -+ sandbox = execution_record(sandbox_raw, evaluator_protocol, human_raw) - counts = { - task: { - "expected": sum(sample_id in labels[task] for sample_id in expected), -@@ -192,13 +198,35 @@ - "tokenized_manifest_sha256": frozen["tokenized_manifest_sha256"], - "validator_files": validator_identity(), - "termination_policy": stopping, -- "human_results_sha256": sha(human_results.read_bytes()) if human_results else None, -+ "human_results_sha256": sha(human_raw) if human_raw is not None else None, -+ "sandbox_protocol": evaluator_protocol, -+ "sandbox_protocol_sha256": sandbox_protocol_sha256 if human is not None else None, -+ "sandbox_record_sha256": sha(sandbox_raw) if sandbox_raw is not None else None, - }, - ) - - --def compare(baseline, candidate, output): -- b, c = (json.loads(path.read_bytes()) for path in (baseline, candidate)) -+def compare( -+ baseline, candidate, output, sandbox_protocol, sandbox_protocol_sha256, sandbox_qualification -+): -+ from prepare import EVALPLUS_REV -+ -+ expected_protocol = frozen_protocol( -+ sandbox_protocol, sandbox_protocol_sha256, sandbox_qualification, EVALPLUS_REV -+ ) -+ baseline_raw, candidate_raw = baseline.read_bytes(), candidate.read_bytes() -+ b, c = (document(raw) for raw in (baseline_raw, candidate_raw)) -+ for row in (b, c): -+ for name in ( -+ "run_manifest_sha256", -+ "responses_sha256", -+ "prepared_manifest_sha256", -+ "tokenized_manifest_sha256", -+ "human_results_sha256", -+ "sandbox_record_sha256", -+ "sandbox_protocol_sha256", -+ ): -+ digest(row.get(name)) - if any(row["phase"] != "final" or row["evaluation_complete"] is not True for row in (b, c)): - raise ValueError("complete final evaluation required") - if b["arm"] != "baseline" or c["arm"] != "candidate": -@@ -213,9 +241,17 @@ - ) - ): - raise ValueError("datasets/prompts/scorers differ") -+ if any( -+ not same_identity(row.get("sandbox_protocol"), expected_protocol) -+ or row.get("sandbox_protocol_sha256") != sandbox_protocol_sha256 -+ for row in (b, c) -+ ): -+ raise ValueError("quality evaluator differs from independent frozen protocol") - result = compare_counts(b["counts"], c["counts"]) -- result["baseline_summary_sha256"] = sha(baseline.read_bytes()) -- result["candidate_summary_sha256"] = sha(candidate.read_bytes()) -+ result["baseline_summary_sha256"] = sha(baseline_raw) -+ result["candidate_summary_sha256"] = sha(candidate_raw) -+ result["sandbox_protocol"] = expected_protocol -+ result["sandbox_protocol_sha256"] = sandbox_protocol_sha256 - result["limitation"] = "task-quality gate only; does not award campaign PASS" - write(output, result) - -@@ -231,8 +267,14 @@ - p.add_argument("--run", type=Path, required=True) - p.add_argument("--human-results", type=Path) - p.add_argument("--sandbox-record", type=Path) -+ p.add_argument("--sandbox-protocol", type=Path) -+ p.add_argument("--sandbox-protocol-sha256") -+ p.add_argument("--sandbox-qualification", type=Path) - p.add_argument("--output", type=Path, required=True) - p = sub.add_parser("compare") -+ p.add_argument("--sandbox-protocol", type=Path, required=True) -+ p.add_argument("--sandbox-protocol-sha256", required=True) -+ p.add_argument("--sandbox-qualification", type=Path, required=True) - p.add_argument("--baseline", type=Path, required=True) - p.add_argument("--candidate", type=Path, required=True) - p.add_argument("--output", type=Path, required=True) -@@ -247,9 +289,19 @@ - args.generation_config_sha256, - args.human_results, - args.sandbox_record, -+ args.sandbox_protocol, -+ args.sandbox_protocol_sha256, -+ args.sandbox_qualification, - ) - else: -- compare(args.baseline, args.candidate, args.output) -+ compare( -+ args.baseline, -+ args.candidate, -+ args.output, -+ args.sandbox_protocol, -+ args.sandbox_protocol_sha256, -+ args.sandbox_qualification, -+ ) - - - if __name__ == "__main__": ---- /dev/null -+++ b/sandbox_provenance.py -@@ -0,0 +1,116 @@ -+"""Bind quality evidence to a separately frozen evaluator protocol. -+ -+This reader authenticates identities and report scope, not isolation by itself. -+The qualification artifact must contain independently reviewed execution evidence -+before its digest is accepted into the campaign freeze. Test fixtures are not -+qualification artifacts. No OCI identity is inferred from readonly file binds. -+""" -+ -+import hashlib -+import json -+import re -+ -+ -+def sha(raw): -+ return hashlib.sha256(raw).hexdigest() -+ -+ -+def unique_keys(pairs): -+ result = {} -+ for key, value in pairs: -+ if key in result: -+ raise ValueError("duplicate provenance key") -+ result[key] = value -+ return result -+ -+ -+def document(raw): -+ value = json.loads(raw, object_pairs_hook=unique_keys) -+ if not isinstance(value, dict): -+ raise TypeError("provenance must be an object") -+ return value -+ -+ -+def digest(value): -+ if not isinstance(value, str) or re.fullmatch(r"[0-9a-f]{64}", value) is None: -+ raise ValueError("invalid provenance digest") -+ -+ -+def same_identity(actual, expected): -+ return json.dumps(actual, sort_keys=True, allow_nan=False) == json.dumps( -+ expected, sort_keys=True, allow_nan=False -+ ) -+ -+ -+def protocol_identity(protocol, evaluator_revision): -+ hashes = { -+ "executor_sha256", -+ "driver_sha256", -+ "sandbox_binary_sha256", -+ "policy_sha256", -+ "qualification_sha256", -+ } -+ if set(protocol) != hashes | { -+ "schema_version", -+ "runtime_kind", -+ "runtime_digest", -+ "evaluator_revision", -+ }: -+ raise ValueError("unexpected evaluator protocol schema") -+ if type(protocol["schema_version"]) is not int or protocol["schema_version"] != 1: -+ raise ValueError("unsupported evaluator protocol version") -+ if ( -+ protocol["runtime_kind"] != "readonly-bind-set-v1" -+ or protocol["evaluator_revision"] != evaluator_revision -+ ): -+ raise ValueError("evaluator kind or revision differs") -+ value = protocol["runtime_digest"] -+ if not isinstance(value, str) or not value.startswith("sha256:"): -+ raise ValueError("invalid readonly-runtime digest") -+ digest(value[7:]) -+ for name in hashes: -+ digest(protocol[name]) -+ return {name: value for name, value in protocol.items() if name != "qualification_sha256"} -+ -+ -+def frozen_protocol(path, expected_sha256, qualification_path, evaluator_revision): -+ if path is None or qualification_path is None: -+ raise ValueError("independent frozen evaluator protocol and qualification required") -+ digest(expected_sha256) -+ protocol_raw = path.read_bytes() -+ if sha(protocol_raw) != expected_sha256: -+ raise ValueError("frozen evaluator protocol bytes changed") -+ protocol = document(protocol_raw) -+ identity = protocol_identity(protocol, evaluator_revision) -+ qualification_raw = qualification_path.read_bytes() -+ if sha(qualification_raw) != protocol["qualification_sha256"]: -+ raise ValueError("frozen qualification artifact changed") -+ qualification = document(qualification_raw) -+ if ( -+ qualification.get("scope") != "generated-code-executor" -+ or qualification.get("verdict") != "PASS" -+ or qualification.get("qualified_generated_code_executor") is not True -+ or not same_identity(qualification.get("identity"), identity) -+ ): -+ raise ValueError("qualification scope or evaluator identity differs") -+ return protocol -+ -+ -+def execution_record(raw, protocol, results_raw): -+ if raw is None: -+ raise ValueError("sandbox execution provenance required") -+ record = document(raw) -+ if ( -+ not same_identity(record.get("protocol"), protocol) -+ or record.get("network") != "none" -+ or type(record.get("exit_code")) is not int -+ or record["exit_code"] != 0 -+ or record.get("timed_out") is not False -+ or record.get("cleanup_complete") is not True -+ or record.get("qualified_generated_code_executor") is not True -+ or record.get("results_sha256") != sha(results_raw) -+ ): -+ raise ValueError("execution differs from frozen evaluator or did not complete") -+ for name in ("samples_sha256", "dataset_sha256"): -+ digest(record.get(name)) -+ return record ---- a/prepare_guard.py -+++ b/prepare_guard.py -@@ -17,6 +17,7 @@ - "prepare.py", - "run.py", - "score.py", -+ "sandbox_provenance.py", - "retrieval.py", - "source-lock.json", - "prepare_guard.py", diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/quality-test_guard_v5.py b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/quality-test_guard_v5.py deleted file mode 100644 index a85459791..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/quality-test_guard_v5.py +++ /dev/null @@ -1,42 +0,0 @@ -# ruff: noqa: E402 -"""Synthetic source manifest coverage; no real dataset or sandbox execution.""" - -import json -import os -import sys -from pathlib import Path - -ROOT = Path(__file__).parent -sys.path.insert(0, os.environ.get("QUALITY_SOURCE_DIR", str(ROOT.parent / "quality-source-v5"))) -import prepare_guard - -sys.path.append(os.environ["QUALITY_REFERENCE_TEST_DIR"]) -import pytest -from protocol import canonical, sha -from test_prepare_guard import fixture - - -def test_complete_manifest_includes_provenance(tmp_path, monkeypatch): - args = fixture(tmp_path, monkeypatch) - observed = prepare_guard.snapshot(args) - assert len(observed["tools"]) == 9 - assert "sandbox_provenance.py" in observed["tools"] - - -def test_old_eight_file_manifest_rejected(tmp_path, monkeypatch): - args = fixture(tmp_path, monkeypatch) - manifest = json.loads(args.source_manifest.read_bytes()) - manifest["files"] = [ - item for item in manifest["files"] if item["path"] != "sandbox_provenance.py" - ] - args.source_manifest.write_bytes(canonical(manifest)) - args.source_manifest_sha256 = sha(args.source_manifest.read_bytes()) - with pytest.raises(ValueError, match="complete preparation tool set"): - prepare_guard.snapshot(args) - - -def test_provenance_source_drift_rejected(tmp_path, monkeypatch): - args = fixture(tmp_path, monkeypatch) - (args.source_dir / "sandbox_provenance.py").write_bytes(b"changed synthetic source") - with pytest.raises(ValueError, match="preparation source changed"): - prepare_guard.snapshot(args) diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/ready-file-test.patch b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/ready-file-test.patch deleted file mode 100644 index ff7cd26db..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/ready-file-test.patch +++ /dev/null @@ -1,24 +0,0 @@ -diff --git a/tests/test_qwen38_service.py b/tests/test_qwen38_service.py -index fd2555a..ced2243 100644 ---- a/tests/test_qwen38_service.py -+++ b/tests/test_qwen38_service.py -@@ -449,9 +449,16 @@ main() - process = subprocess.Popen([sys.executable, "-c", source, str(ready), str(final)], env=environment) - try: - deadline = time.monotonic() + 3 -- while not ready.exists() and process.poll() is None and time.monotonic() < deadline: -- time.sleep(0.01) -- address = json.loads(ready.read_text())["address"] -+ # The producer opens the file before serializing its health report. -+ # Existence alone can expose an empty/partial JSON document. -+ while process.poll() is None and time.monotonic() < deadline: -+ try: -+ address = json.loads(ready.read_text())["address"] -+ break -+ except (FileNotFoundError, json.JSONDecodeError): -+ time.sleep(0.01) -+ else: -+ raise AssertionError("server did not publish complete readiness JSON") - connection, response = stream(address, request_id="signal-active") - idle = socket.create_connection(address) - idle.sendall(b"POST /v1/completions HTTP/1.1\r\nHost: localhost\r\nContent-Length: 1000\r\n\r\n{") diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-benign-retry-v1/audit_packages.py b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-benign-retry-v1/audit_packages.py deleted file mode 100644 index 946efc625..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-benign-retry-v1/audit_packages.py +++ /dev/null @@ -1,59 +0,0 @@ -"""Verify the complete installed set against authenticated exact dependency pins.""" - -import importlib.metadata -import json -import re -import sys -from pathlib import Path - - -def normalize(name): - return re.sub(r"[-_.]+", "-", name).lower() - - -def locked(paths): - expected = {} - for path in paths: - rows = re.findall(r"^([A-Za-z0-9_.-]+)==([^\s\\]+)", path.read_text(), re.MULTILINE) - if not rows: - raise ValueError("Empty lock") - for name, version in rows: - key = normalize(name) - if key in expected and expected[key] != version: - raise ValueError("Conflicting build/runtime pins") - expected[key] = version - return expected - - -def validate(expected, rows, phase): - actual = {} - for name, version in rows: - key = normalize(name) - if key in actual: - raise ValueError("Duplicate installed distribution") - actual[key] = version - wanted = dict(expected) - if phase == "evaluator": - if "evalplus" not in actual: - raise ValueError("EvalPlus missing") - # Its version is SCM-derived; source bytes are independently authenticated. - wanted["evalplus"] = actual["evalplus"] - elif phase != "dependencies": - raise ValueError("Unknown phase") - if actual != wanted: - raise ValueError("Installed distribution set or versions differ from locks") - return actual - - -def main(): - build, runtime, phase, output = sys.argv[1:] - expected = locked([Path(build), Path(runtime)]) - rows = [(dist.metadata["Name"], dist.version) for dist in importlib.metadata.distributions()] - actual = validate(expected, rows, phase) - with Path(output).open("x") as stream: - json.dump({"phase": phase, "expected": expected, "actual": actual}, stream, sort_keys=True) - stream.write("\n") - - -if __name__ == "__main__": - main() diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-benign-retry-v1/run.sbatch b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-benign-retry-v1/run.sbatch deleted file mode 100644 index e29473a14..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-benign-retry-v1/run.sbatch +++ /dev/null @@ -1,93 +0,0 @@ -#!/usr/bin/env bash -# Reuse the installed evaluator; only inventory and the frozen trusted probe run. -#SBATCH --job-name=qwen38-eval-benign-retry -#SBATCH --nodes=1 -#SBATCH --ntasks=1 -#SBATCH --cpus-per-task=4 -#SBATCH --mem=16G -#SBATCH --time=00:15:00 -set -euo pipefail -umask 077 -: "${TASK_ROOT:?}" "${SANDBOX_BUNDLE:?}" "${SANDBOX_FREEZE_SHA256:?}" -: "${SANDBOX_VENV:?}" "${SOURCE_DIR:?}" "${OUTPUT_DIR:?}" -: "${SCRIPT_SHA256:?}" "${PACKAGE_AUDIT_SHA256:?}" "${BWRAP_BIN:?}" "${BWRAP_SHA256:?}" -: "${PYTHON_SHA256:?}" -[[ -n ${SLURM_JOB_ID:-} && ${SLURM_GPUS_ON_NODE:-0} == 0 ]] -root=$(realpath -e "$TASK_ROOT") -check_path() { case "$(realpath -m "$1")" in "$root"/*) ;; *) exit 2;; esac; } -check_digest() { - [[ $2 =~ ^[a-f0-9]{64}$ ]] - [[ $(sha256sum "$1" | cut -d' ' -f1) == "$2" ]] -} -for path in "$SANDBOX_BUNDLE" "$SANDBOX_VENV" "$SOURCE_DIR" "$OUTPUT_DIR" "$BWRAP_BIN"; do check_path "$path"; done -check_digest "$SOURCE_DIR/run.sbatch" "$SCRIPT_SHA256" -check_digest "$SOURCE_DIR/audit_packages.py" "$PACKAGE_AUDIT_SHA256" -check_digest "$SANDBOX_BUNDLE/freeze.sha256" "$SANDBOX_FREEZE_SHA256" -check_digest "$SANDBOX_VENV/bin/python" "$PYTHON_SHA256" -check_digest "$BWRAP_BIN" "$BWRAP_SHA256" -cmp "${BASH_SOURCE[0]}" "$SOURCE_DIR/run.sbatch" -(cd "$SANDBOX_BUNDLE" && sha256sum --check --strict freeze.sha256) -[[ ! -e $OUTPUT_DIR && ! -L $OUTPUT_DIR ]] -export PATH="$(dirname "$BWRAP_BIN"):/usr/bin:/bin" -[[ $(command -v bwrap) == "$BWRAP_BIN" ]] -export TMPDIR="$root/tmp/qwen38-benign-retry-${SLURM_JOB_ID}" -export XDG_CACHE_HOME="$root/cache/qwen38-benign-retry-${SLURM_JOB_ID}" -export HF_HOME="$XDG_CACHE_HOME/huggingface" TORCH_HOME="$XDG_CACHE_HOME/torch" -export TRITON_CACHE_DIR="$XDG_CACHE_HOME/triton" VLLM_CACHE_ROOT="$XDG_CACHE_HOME/vllm" -export PYTHONNOUSERSITE=1 PYTHONDONTWRITEBYTECODE=1 -unset PYTHONPATH PYTHONHOME PYTHONOPTIMIZE PYTHONINSPECT PYTHONSTARTUP VIRTUAL_ENV -for path in "$TMPDIR" "$XDG_CACHE_HOME" "$HF_HOME" "$TORCH_HOME" "$TRITON_CACHE_DIR" "$VLLM_CACHE_ROOT"; do check_path "$path"; mkdir -p "$path"; done -mkdir "$OUTPUT_DIR" -completed=false -signal_name=none -finish() { - local status=$? - trap - EXIT - [[ $status != 0 || $completed == true ]] || status=1 - [[ $status == 0 ]] || completed=false - printf '%s\n' "$status" > "$OUTPUT_DIR/exit-code" - printf '{"completed":%s,"signal":"%s","exit_code":%s,"job_id":"%s"}\n' \ - "$completed" "$signal_name" "$status" "$SLURM_JOB_ID" > "$OUTPUT_DIR/terminal-status.json" - exit "$status" -} -trap finish EXIT -trap 'signal_name=TERM; exit 143' TERM -trap 'signal_name=INT; exit 130' INT -trap 'signal_name=HUP; exit 129' HUP -cp "$SOURCE_DIR/run.sbatch" "$SOURCE_DIR/audit_packages.py" "$OUTPUT_DIR/" -cd "$root" -python="$SANDBOX_VENV/bin/python" -"$python" -IB "$SOURCE_DIR/audit_packages.py" "$SANDBOX_BUNDLE/build.lock" \ - "$SANDBOX_BUNDLE/dependencies.lock" evaluator "$OUTPUT_DIR/evaluator-packages-before.json" -"$python" -IB "$SANDBOX_BUNDLE/runtime.py" verify-source --bundle "$SANDBOX_BUNDLE" -[[ -z $(git -C "$SANDBOX_BUNDLE/source" ls-files --others --exclude-standard) ]] -"$python" -IB "$SANDBOX_BUNDLE/runtime.py" inventory --bundle "$SANDBOX_BUNDLE" \ - --output "$OUTPUT_DIR/rootfs-before" -probe_status=0 -"$python" -IB "$SANDBOX_BUNDLE/runtime.py" probe --bundle "$SANDBOX_BUNDLE" \ - --inventory "$OUTPUT_DIR/rootfs-before" --output "$OUTPUT_DIR/probe" || probe_status=$? -printf '%s\n' "$probe_status" > "$OUTPUT_DIR/probe-status" -post_status=0 -run_postcheck() { - local name=$1 status=0 - shift - "$@" || status=$? - printf '%s\t%s\n' "$name" "$status" >> "$OUTPUT_DIR/postflight-status.tsv" - if [[ $status != 0 ]]; then post_status=$status; fi -} -check_frozen_files() { (cd "$SANDBOX_BUNDLE" && sha256sum --check --strict freeze.sha256); } -run_postcheck inventory "$python" -IB "$SANDBOX_BUNDLE/runtime.py" inventory \ - --bundle "$SANDBOX_BUNDLE" --output "$OUTPUT_DIR/rootfs-after" -run_postcheck runtime-byte-comparison cmp "$OUTPUT_DIR/rootfs-before/rootfs.json" "$OUTPUT_DIR/rootfs-after/rootfs.json" -run_postcheck source "$python" -IB "$SANDBOX_BUNDLE/runtime.py" verify-source --bundle "$SANDBOX_BUNDLE" -run_postcheck package-set "$python" -IB "$SOURCE_DIR/audit_packages.py" "$SANDBOX_BUNDLE/build.lock" \ - "$SANDBOX_BUNDLE/dependencies.lock" evaluator "$OUTPUT_DIR/evaluator-packages-after.json" -run_postcheck batch-script check_digest "$SOURCE_DIR/run.sbatch" "$SCRIPT_SHA256" -run_postcheck audit-script check_digest "$SOURCE_DIR/audit_packages.py" "$PACKAGE_AUDIT_SHA256" -run_postcheck bwrap-binary check_digest "$BWRAP_BIN" "$BWRAP_SHA256" -run_postcheck python-binary check_digest "$python" "$PYTHON_SHA256" -run_postcheck freeze-manifest check_digest "$SANDBOX_BUNDLE/freeze.sha256" "$SANDBOX_FREEZE_SHA256" -run_postcheck freeze-files check_frozen_files -if [[ $probe_status != 0 ]]; then exit "$probe_status"; fi -if [[ $post_status != 0 ]]; then exit "$post_status"; fi -completed=true diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-benign-retry-v2/audit_packages.py b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-benign-retry-v2/audit_packages.py deleted file mode 100644 index 946efc625..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-benign-retry-v2/audit_packages.py +++ /dev/null @@ -1,59 +0,0 @@ -"""Verify the complete installed set against authenticated exact dependency pins.""" - -import importlib.metadata -import json -import re -import sys -from pathlib import Path - - -def normalize(name): - return re.sub(r"[-_.]+", "-", name).lower() - - -def locked(paths): - expected = {} - for path in paths: - rows = re.findall(r"^([A-Za-z0-9_.-]+)==([^\s\\]+)", path.read_text(), re.MULTILINE) - if not rows: - raise ValueError("Empty lock") - for name, version in rows: - key = normalize(name) - if key in expected and expected[key] != version: - raise ValueError("Conflicting build/runtime pins") - expected[key] = version - return expected - - -def validate(expected, rows, phase): - actual = {} - for name, version in rows: - key = normalize(name) - if key in actual: - raise ValueError("Duplicate installed distribution") - actual[key] = version - wanted = dict(expected) - if phase == "evaluator": - if "evalplus" not in actual: - raise ValueError("EvalPlus missing") - # Its version is SCM-derived; source bytes are independently authenticated. - wanted["evalplus"] = actual["evalplus"] - elif phase != "dependencies": - raise ValueError("Unknown phase") - if actual != wanted: - raise ValueError("Installed distribution set or versions differ from locks") - return actual - - -def main(): - build, runtime, phase, output = sys.argv[1:] - expected = locked([Path(build), Path(runtime)]) - rows = [(dist.metadata["Name"], dist.version) for dist in importlib.metadata.distributions()] - actual = validate(expected, rows, phase) - with Path(output).open("x") as stream: - json.dump({"phase": phase, "expected": expected, "actual": actual}, stream, sort_keys=True) - stream.write("\n") - - -if __name__ == "__main__": - main() diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-benign-retry-v2/benign_probe.py b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-benign-retry-v2/benign_probe.py deleted file mode 100644 index 6d34e59bc..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-benign-retry-v2/benign_probe.py +++ /dev/null @@ -1,100 +0,0 @@ -"""Frozen trusted containment probe. Never loads benchmark data or generated code.""" - -import importlib.metadata -import json -import os -import resource -import socket -import sys -from pathlib import Path - -sys.path.insert(0, "/opt/site-packages") -resource.setrlimit(resource.RLIMIT_NPROC, (64, 64)) - -import evalplus # noqa: E402 - intentionally isolated installed package path. -import numpy # noqa: E402 -import psutil # noqa: E402 - - -def main(): - expected_namespaces = json.loads(os.environ["PROBE_PARENT_NAMESPACES"]) - changed = { - name: os.readlink("/proc/self/ns/" + name) != value - for name, value in expected_namespaces.items() - } - status = dict(line.split(":", 1) for line in Path("/proc/self/status").read_text().splitlines()) - absent = { - name: not Path(name).exists() - for name in ( - "/home", - "/root", - "/etc/passwd", - "/run/user", - "/dev/nvidia0", - os.environ["PROBE_UNMOUNTED_SENTINEL"], - ) - } - readonly = {} - for name in ("/opt/python/.write-probe", "/opt/site-packages/.write-probe", "/.write-probe"): - try: - Path(name).write_text("synthetic") - readonly[name] = False - except OSError: - readonly[name] = True - Path("/work/tmp/benign.txt").write_text("synthetic owned temporary data") - Path("/work/output/benign.txt").write_text("synthetic owned output") - interfaces = [name for _, name in socket.if_nameindex()] - blocked = False - with socket.socket() as channel: - channel.settimeout(0.2) - try: - channel.connect(("198.51.100.1", 9)) # Reserved documentation address. - except OSError: - blocked = True - limits = { - name: list(resource.getrlimit(getattr(resource, "RLIMIT_" + name))) - for name in ("AS", "CPU", "NPROC", "NOFILE", "FSIZE", "CORE") - } - checks = { - "new_namespaces": all(changed.values()), - "host_paths_absent": all(absent.values()), - "readonly_runtime_and_root": all(readonly.values()), - "network_blocked": blocked, - "only_loopback": interfaces == ["lo"], - "no_new_privileges": status["NoNewPrivs"].strip() == "1", - "zero_effective_capabilities": int(status["CapEff"].strip(), 16) == 0, - "nonroot_namespace_uid": os.geteuid() == 65534, - "environment_cleared": "PROBE_INHERITED_SECRET" not in os.environ, - "relocated_python": Path(sys.base_prefix) == Path("/opt/python"), - "limited_cpu_affinity": len(os.sched_getaffinity(0)) == 1, - "fixed_rlimits": limits - == { - "AS": [2147483648] * 2, - "CPU": [30] * 2, - "NPROC": [64] * 2, - "NOFILE": [64] * 2, - "FSIZE": [1048576] * 2, - "CORE": [0, 0], - }, - } - report = { - "checks": checks, - "passed": all(checks.values()), - "namespaces_changed": changed, - "packages": { - name: importlib.metadata.version(name) for name in ("evalplus", "numpy", "psutil") - }, - "numpy_benign_result": int(numpy.array([2, 3]).sum()), - "psutil_imported": bool(psutil.__version__), - "evalplus_imported": evalplus is not None, - "generated_code_executed": False, - "benchmark_data_mounted": False, - "full_evaluator_executed": False, - "qualified_generated_code_executor": False, - } - print(json.dumps(report, sort_keys=True)) - return 0 if report["passed"] else 2 - - -if __name__ == "__main__": - raise SystemExit(main()) diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-benign-retry-v2/check_loader.initial.py b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-benign-retry-v2/check_loader.initial.py deleted file mode 100644 index 8b8f572a4..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-benign-retry-v2/check_loader.initial.py +++ /dev/null @@ -1,61 +0,0 @@ -"""Exercise loader-policy failures without invoking ldd or bubblewrap.""" -import importlib.util -import json -import os -import subprocess -import tempfile -from pathlib import Path -from unittest.mock import patch - -spec = importlib.util.spec_from_file_location('runtime_candidate', Path(__file__).with_name('runtime.py')) -runtime = importlib.util.module_from_spec(spec) -spec.loader.exec_module(runtime) -checks = {} -with tempfile.TemporaryDirectory() as directory: - root = Path(directory) - base, site = root / 'base', root / 'site' - for path in (base / 'lib', site): - path.mkdir(parents=True) - library = base / 'lib/libsynthetic.so' - library.write_bytes(b'\x7fELFsynthetic') - extension = site / 'extension.so' - extension.write_bytes(b'\x7fELFsynthetic') - outside = root / 'outside.so' - outside.write_bytes(b'\x7fELFsynthetic') - calls = [] - - def successful(args, **kwargs): - calls.append((args, kwargs)) - return subprocess.CompletedProcess(args, 0, f'libsynthetic.so => {library}\n'.encode(), b'') - - with patch.dict(os.environ, {'LD_PRELOAD': '/synthetic-untrusted.so', 'LD_LIBRARY_PATH': '/synthetic-host'}): - with patch.object(runtime.shutil, 'which', return_value='/usr/bin/ldd'): - with patch.object(runtime.subprocess, 'run', side_effect=successful): - assert runtime.system_dependencies((base, site)) == {} - assert len(calls) == 2 - assert all(kw['env'] == {'PATH': '/usr/bin:/bin', 'LC_ALL': 'C', 'LD_LIBRARY_PATH': str(base / 'lib')} for _, kw in calls) - checks['cleared_subprocess_environment_and_both_roots_scanned'] = True - for name, code, stdout in ( - ('nonzero_rejected', 1, b''), - ('missing_dependency_rejected_even_with_exit_zero', 0, b'libmissing.so => not found\n'), - ('unexpected_external_dependency_rejected', 0, f'liboutside.so => {outside}\n'.encode()), - ): - result = subprocess.CompletedProcess(['/usr/bin/ldd'], code, stdout, b'') - with patch.object(runtime.subprocess, 'run', return_value=result): - try: - runtime.system_dependencies((base, site)) - except ValueError: - checks[name] = True - else: - raise AssertionError(name) - locators = {'bwrap': '/synthetic/bwrap', 'python_root': str(base), 'site_root': str(site), - 'system_libraries': {}, 'python_relative': 'bin/python3.12'} - args = runtime.command(locators, root / 'probe.py', {}, root / 'sentinel') - index = args.index('LD_LIBRARY_PATH') - assert args[index - 1:index + 2] == ['--setenv', 'LD_LIBRARY_PATH', '/opt/python/lib'] - assert '--clearenv' in args and '--disable-userns' in args and '--remount-ro' in args - checks['sandbox_has_explicit_relocated_loader_directory_and_isolation_flags'] = True -report = {'scope': 'local_mocked_loader_controls', 'checks': checks, 'passed': all(checks.values()), - 'external_command_executed': False, 'runtime_qualified': False} -Path(__file__).with_name('loader-controls.json').write_text(json.dumps(report, indent=2) + '\n') -print(json.dumps(report)) diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-benign-retry-v2/check_loader.py b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-benign-retry-v2/check_loader.py deleted file mode 100644 index 6cfa19408..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-benign-retry-v2/check_loader.py +++ /dev/null @@ -1,61 +0,0 @@ -"""Exercise loader-policy failures without invoking ldd or bubblewrap.""" -import importlib.util -import json -import os -import subprocess -import tempfile -from pathlib import Path -from unittest.mock import patch - -spec = importlib.util.spec_from_file_location('runtime_candidate', Path(__file__).with_name('runtime.py')) -runtime = importlib.util.module_from_spec(spec) -spec.loader.exec_module(runtime) -checks = {} -with tempfile.TemporaryDirectory() as directory: - root = Path(directory).resolve(strict=True) - base, site = root / 'base', root / 'site' - for path in (base / 'lib', site): - path.mkdir(parents=True) - library = base / 'lib/libsynthetic.so' - library.write_bytes(b'\x7fELFsynthetic') - extension = site / 'extension.so' - extension.write_bytes(b'\x7fELFsynthetic') - outside = root / 'outside.so' - outside.write_bytes(b'\x7fELFsynthetic') - calls = [] - - def successful(args, **kwargs): - calls.append((args, kwargs)) - return subprocess.CompletedProcess(args, 0, f'libsynthetic.so => {library}\n'.encode(), b'') - - with patch.dict(os.environ, {'LD_PRELOAD': '/synthetic-untrusted.so', 'LD_LIBRARY_PATH': '/synthetic-host'}): - with patch.object(runtime.shutil, 'which', return_value='/usr/bin/ldd'): - with patch.object(runtime.subprocess, 'run', side_effect=successful): - assert runtime.system_dependencies((base, site)) == {} - assert len(calls) == 2 - assert all(kw['env'] == {'PATH': '/usr/bin:/bin', 'LC_ALL': 'C', 'LD_LIBRARY_PATH': str(base / 'lib')} for _, kw in calls) - checks['cleared_subprocess_environment_and_both_roots_scanned'] = True - for name, code, stdout in ( - ('nonzero_rejected', 1, b''), - ('missing_dependency_rejected_even_with_exit_zero', 0, b'libmissing.so => not found\n'), - ('unexpected_external_dependency_rejected', 0, f'liboutside.so => {outside}\n'.encode()), - ): - result = subprocess.CompletedProcess(['/usr/bin/ldd'], code, stdout, b'') - with patch.object(runtime.subprocess, 'run', return_value=result): - try: - runtime.system_dependencies((base, site)) - except ValueError: - checks[name] = True - else: - raise AssertionError(name) - locators = {'bwrap': '/synthetic/bwrap', 'python_root': str(base), 'site_root': str(site), - 'system_libraries': {}, 'python_relative': 'bin/python3.12'} - args = runtime.command(locators, root / 'probe.py', {}, root / 'sentinel') - index = args.index('LD_LIBRARY_PATH') - assert args[index - 1:index + 2] == ['--setenv', 'LD_LIBRARY_PATH', '/opt/python/lib'] - assert '--clearenv' in args and '--disable-userns' in args and '--remount-ro' in args - checks['sandbox_has_explicit_relocated_loader_directory_and_isolation_flags'] = True -report = {'scope': 'local_mocked_loader_controls', 'checks': checks, 'passed': all(checks.values()), - 'external_command_executed': False, 'runtime_qualified': False} -Path(__file__).with_name('loader-controls.json').write_text(json.dumps(report, indent=2) + '\n') -print(json.dumps(report)) diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-benign-retry-v2/freeze.sha256 b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-benign-retry-v2/freeze.sha256 deleted file mode 100644 index e72ac683c..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-benign-retry-v2/freeze.sha256 +++ /dev/null @@ -1,4 +0,0 @@ -d323597129f32a5c6cbcb944172d05e182a2b7694bb0fb6cde6a85483c722104 run.sbatch -dc315ea40742ec3d27c7466e962864f4023996943e00776460d8be075c3174bf runtime.py -8593cf53ca863b497ccb5ceced2b81d1c846522add2c9acd68b8a03975469d5c benign_probe.py -779c258fb64ee8dcc668dde7d30ee3a7acdc736a695914afee4c91ee823bed63 audit_packages.py diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-benign-retry-v2/loader-controls.json b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-benign-retry-v2/loader-controls.json deleted file mode 100644 index ddf480a52..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-benign-retry-v2/loader-controls.json +++ /dev/null @@ -1,13 +0,0 @@ -{ - "scope": "local_mocked_loader_controls", - "checks": { - "cleared_subprocess_environment_and_both_roots_scanned": true, - "nonzero_rejected": true, - "missing_dependency_rejected_even_with_exit_zero": true, - "unexpected_external_dependency_rejected": true, - "sandbox_has_explicit_relocated_loader_directory_and_isolation_flags": true - }, - "passed": true, - "external_command_executed": false, - "runtime_qualified": false -} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-benign-retry-v2/run.sbatch b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-benign-retry-v2/run.sbatch deleted file mode 100644 index ca9528193..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-benign-retry-v2/run.sbatch +++ /dev/null @@ -1,99 +0,0 @@ -#!/usr/bin/env bash -# Reuse the installed evaluator; only inventory and the frozen trusted probe run. -#SBATCH --job-name=qwen38-eval-benign-retry-v2 -#SBATCH --nodes=1 -#SBATCH --ntasks=1 -#SBATCH --cpus-per-task=4 -#SBATCH --mem=16G -#SBATCH --time=00:15:00 -set -euo pipefail -umask 077 -: "${TASK_ROOT:?}" "${SANDBOX_BUNDLE:?}" "${SANDBOX_FREEZE_SHA256:?}" -: "${SANDBOX_VENV:?}" "${SOURCE_DIR:?}" "${OUTPUT_DIR:?}" -: "${SCRIPT_SHA256:?}" "${PACKAGE_AUDIT_SHA256:?}" "${BWRAP_BIN:?}" "${BWRAP_SHA256:?}" -: "${PYTHON_SHA256:?}" "${SOURCE_FREEZE_SHA256:?}" -[[ -n ${SLURM_JOB_ID:-} && ${SLURM_GPUS_ON_NODE:-0} == 0 ]] -root=$(realpath -e "$TASK_ROOT") -check_path() { case "$(realpath -m "$1")" in "$root"/*) ;; *) exit 2;; esac; } -check_digest() { - [[ $2 =~ ^[a-f0-9]{64}$ ]] - [[ $(sha256sum "$1" | cut -d' ' -f1) == "$2" ]] -} -for path in "$SANDBOX_BUNDLE" "$SANDBOX_VENV" "$SOURCE_DIR" "$OUTPUT_DIR" "$BWRAP_BIN"; do check_path "$path"; done -check_digest "$SOURCE_DIR/run.sbatch" "$SCRIPT_SHA256" -check_digest "$SOURCE_DIR/audit_packages.py" "$PACKAGE_AUDIT_SHA256" -check_digest "$SOURCE_DIR/freeze.sha256" "$SOURCE_FREEZE_SHA256" -(cd "$SOURCE_DIR" && sha256sum --check --strict freeze.sha256) -check_digest "$SANDBOX_BUNDLE/freeze.sha256" "$SANDBOX_FREEZE_SHA256" -check_digest "$SANDBOX_VENV/bin/python" "$PYTHON_SHA256" -check_digest "$BWRAP_BIN" "$BWRAP_SHA256" -cmp "${BASH_SOURCE[0]}" "$SOURCE_DIR/run.sbatch" -(cd "$SANDBOX_BUNDLE" && sha256sum --check --strict freeze.sha256) -[[ ! -e $OUTPUT_DIR && ! -L $OUTPUT_DIR ]] -export PATH="$(dirname "$BWRAP_BIN"):/usr/bin:/bin" -[[ $(command -v bwrap) == "$BWRAP_BIN" ]] -export TMPDIR="$root/tmp/qwen38-benign-retry-v2-${SLURM_JOB_ID}" -export XDG_CACHE_HOME="$root/cache/qwen38-benign-retry-v2-${SLURM_JOB_ID}" -export HF_HOME="$XDG_CACHE_HOME/huggingface" TORCH_HOME="$XDG_CACHE_HOME/torch" -export TRITON_CACHE_DIR="$XDG_CACHE_HOME/triton" VLLM_CACHE_ROOT="$XDG_CACHE_HOME/vllm" -export PYTHONNOUSERSITE=1 PYTHONDONTWRITEBYTECODE=1 -unset PYTHONPATH PYTHONHOME PYTHONOPTIMIZE PYTHONINSPECT PYTHONSTARTUP VIRTUAL_ENV -for path in "$TMPDIR" "$XDG_CACHE_HOME" "$HF_HOME" "$TORCH_HOME" "$TRITON_CACHE_DIR" "$VLLM_CACHE_ROOT"; do check_path "$path"; mkdir -p "$path"; done -mkdir "$OUTPUT_DIR" -completed=false -signal_name=none -finish() { - local status=$? - trap - EXIT - [[ $status != 0 || $completed == true ]] || status=1 - [[ $status == 0 ]] || completed=false - printf '%s\n' "$status" > "$OUTPUT_DIR/exit-code" - printf '{"completed":%s,"signal":"%s","exit_code":%s,"job_id":"%s"}\n' \ - "$completed" "$signal_name" "$status" "$SLURM_JOB_ID" > "$OUTPUT_DIR/terminal-status.json" - exit "$status" -} -trap finish EXIT -trap 'signal_name=TERM; exit 143' TERM -trap 'signal_name=INT; exit 130' INT -trap 'signal_name=HUP; exit 129' HUP -cp "$SOURCE_DIR/run.sbatch" "$SOURCE_DIR/audit_packages.py" "$SOURCE_DIR/runtime.py" \ - "$SOURCE_DIR/benign_probe.py" "$SOURCE_DIR/freeze.sha256" "$OUTPUT_DIR/" -cd "$root" -python="$SANDBOX_VENV/bin/python" -"$python" -IB "$SOURCE_DIR/audit_packages.py" "$SANDBOX_BUNDLE/build.lock" \ - "$SANDBOX_BUNDLE/dependencies.lock" evaluator "$OUTPUT_DIR/evaluator-packages-before.json" -"$python" -IB "$SOURCE_DIR/runtime.py" verify-source --bundle "$SANDBOX_BUNDLE" -[[ -z $(git -C "$SANDBOX_BUNDLE/source" ls-files --others --exclude-standard) ]] -"$python" -IB "$SOURCE_DIR/runtime.py" inventory --bundle "$SANDBOX_BUNDLE" \ - --output "$OUTPUT_DIR/rootfs-before" -probe_status=0 -"$python" -IB "$SOURCE_DIR/runtime.py" probe --bundle "$SANDBOX_BUNDLE" \ - --inventory "$OUTPUT_DIR/rootfs-before" --output "$OUTPUT_DIR/probe" || probe_status=$? -printf '%s\n' "$probe_status" > "$OUTPUT_DIR/probe-status" -post_status=0 -run_postcheck() { - local name=$1 status=0 - shift - "$@" || status=$? - printf '%s\t%s\n' "$name" "$status" >> "$OUTPUT_DIR/postflight-status.tsv" - if [[ $status != 0 ]]; then post_status=$status; fi -} -check_source_files() { (cd "$SOURCE_DIR" && sha256sum --check --strict freeze.sha256); } -check_frozen_files() { (cd "$SANDBOX_BUNDLE" && sha256sum --check --strict freeze.sha256); } -run_postcheck inventory "$python" -IB "$SOURCE_DIR/runtime.py" inventory \ - --bundle "$SANDBOX_BUNDLE" --output "$OUTPUT_DIR/rootfs-after" -run_postcheck runtime-byte-comparison cmp "$OUTPUT_DIR/rootfs-before/rootfs.json" "$OUTPUT_DIR/rootfs-after/rootfs.json" -run_postcheck source "$python" -IB "$SOURCE_DIR/runtime.py" verify-source --bundle "$SANDBOX_BUNDLE" -run_postcheck package-set "$python" -IB "$SOURCE_DIR/audit_packages.py" "$SANDBOX_BUNDLE/build.lock" \ - "$SANDBOX_BUNDLE/dependencies.lock" evaluator "$OUTPUT_DIR/evaluator-packages-after.json" -run_postcheck batch-script check_digest "$SOURCE_DIR/run.sbatch" "$SCRIPT_SHA256" -run_postcheck audit-script check_digest "$SOURCE_DIR/audit_packages.py" "$PACKAGE_AUDIT_SHA256" -run_postcheck bwrap-binary check_digest "$BWRAP_BIN" "$BWRAP_SHA256" -run_postcheck python-binary check_digest "$python" "$PYTHON_SHA256" -run_postcheck freeze-manifest check_digest "$SANDBOX_BUNDLE/freeze.sha256" "$SANDBOX_FREEZE_SHA256" -run_postcheck freeze-files check_frozen_files -run_postcheck source-freeze-manifest check_digest "$SOURCE_DIR/freeze.sha256" "$SOURCE_FREEZE_SHA256" -run_postcheck source-freeze-files check_source_files -if [[ $probe_status != 0 ]]; then exit "$probe_status"; fi -if [[ $post_status != 0 ]]; then exit "$post_status"; fi -completed=true diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-benign-retry-v2/runtime.py b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-benign-retry-v2/runtime.py deleted file mode 100644 index 5d1ab7441..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-benign-retry-v2/runtime.py +++ /dev/null @@ -1,414 +0,0 @@ -"""Resolve pinned EvalPlus metadata, inventory readonly binds and run a trusted probe.""" - -import argparse -import configparser -import hashlib -import importlib.metadata -import json -import os -import re -import resource -import shutil -import signal -import subprocess -import sys -import sysconfig -from pathlib import Path - -import tomllib - -REVISION = "26d6d00bb1fd0fa37f39c99d5290da67891d1c5e" - - -def canonical(value): - return json.dumps(value, sort_keys=True, separators=(",", ":"), allow_nan=False).encode() - - -def sha(raw): - return hashlib.sha256(raw).hexdigest() - - -def write(path, value): - with path.open("xb") as stream: - stream.write(canonical(value) + b"\n") - - -def git(source, *args): - return subprocess.check_output(["git", "-C", str(source), *args]) - - -def source_identity(bundle): - source = bundle / "source" - if git(source, "rev-parse", "HEAD").decode().strip() != REVISION: - raise ValueError("wrong EvalPlus source revision") - if git(source, "diff", "HEAD", "--"): - raise ValueError("tracked EvalPlus source changed") - return { - "evaluator_revision": REVISION, - "archive_sha256": sha(git(source, "archive", "--format=tar", "HEAD")), - } - - -def requirements(bundle): - identity = source_identity(bundle) - config = configparser.ConfigParser() - config.read(bundle / "source/setup.cfg") - dependencies = [ - line.strip() for line in config["options"]["install_requires"].splitlines() if line.strip() - ] - build = tomllib.loads((bundle / "source/pyproject.toml").read_text())["build-system"][ - "requires" - ] - (bundle / "dependencies.in").write_text("\n".join(dependencies) + "\n") - (bundle / "build.in").write_text("\n".join([*build, "wheel"]) + "\n") - write( - bundle / "source.json", - { - **identity, - "official_default_dependencies": dependencies, - "build_dependencies": [*build, "wheel"], - "extras_enabled": [], - "generated_code_executed": False, - }, - ) - - -def verify_source(bundle): - expected = json.loads((bundle / "source.json").read_bytes()) - if any( - source_identity(bundle)[name] != expected[name] - for name in ("evaluator_revision", "archive_sha256") - ): - raise ValueError("EvalPlus source snapshot drift") - - -def tree(root, alias): - result = [] - for path in sorted(root.rglob("*")): - relative = path.relative_to(root).as_posix() - name = alias + "/" + relative - if path.is_symlink(): - target = os.readlink(path) - if Path(target).is_absolute() or not path.resolve(strict=True).is_relative_to(root): - raise ValueError("runtime symlink escapes its readonly root") - result.append({"path": name, "type": "symlink", "target": target}) - elif path.is_dir(): - result.append({"path": name, "type": "directory", "mode": path.stat().st_mode & 0o777}) - elif path.is_file(): - result.append( - { - "path": name, - "type": "file", - "mode": path.stat().st_mode & 0o777, - "bytes": path.stat().st_size, - "sha256": sha(path.read_bytes()), - } - ) - else: - raise ValueError("unsupported runtime filesystem entry") - return result - - -def system_dependencies(roots): - # Mirror the explicit relocated loader directory; inherit no host loader state. - loader_env = { - "PATH": "/usr/bin:/bin", "LC_ALL": "C", - "LD_LIBRARY_PATH": str(roots[0] / "lib"), - } - ldd = shutil.which("ldd") - if ldd is None: - raise ValueError("system ELF dependency scanner unavailable") - dependencies = {} - for root in roots: - for path in sorted(root.rglob("*")): - if not path.is_file() or path.is_symlink(): - continue - with path.open("rb") as stream: - if stream.read(4) != b"\x7fELF": - continue - result = subprocess.run( - [ldd, str(path)], capture_output=True, timeout=10, check=False, env=loader_env - ) - output = result.stdout.decode(errors="replace") - if result.returncode != 0 or "not found" in output: - raise ValueError("ELF runtime dependency missing") - for line in output.splitlines(): - match = re.search(r"(?:=>\s*)?(/[^\s]+)", line) - if not match: - continue - original = Path(match.group(1)) - resolved = original.resolve(strict=True) - if any(resolved.is_relative_to(bound) for bound in roots): - continue - if not str(original).startswith(("/lib/", "/lib64/", "/usr/lib/", "/usr/lib64/")): - raise ValueError("unexpected external runtime library") - dependencies[str(original)] = str(resolved) - return dependencies - - -def inventory(bundle, output): - verify_source(bundle) - task_root = Path(os.environ["TASK_ROOT"]).resolve(strict=True) - base = Path(sys.base_prefix).resolve(strict=True) - site = Path(sysconfig.get_path("purelib")).resolve(strict=True) - if not base.is_relative_to(task_root) or not site.is_relative_to(task_root): - raise ValueError("runtime roots must remain inside TASK_ROOT") - executable = Path(sys._base_executable).resolve(strict=True).relative_to(base).as_posix() - bwrap = shutil.which("bwrap") - if bwrap is None: - raise ValueError("bubblewrap is unavailable on this compute node") - help_text = subprocess.check_output([bwrap, "--help"], stderr=subprocess.STDOUT).decode() - for flag in ( - "--unshare-user", - "--unshare-net", - "--unshare-pid", - "--disable-userns", - "--assert-userns-disabled", - "--clearenv", - "--remount-ro", - "--size", - ): - if flag not in help_text: - raise ValueError("required bubblewrap isolation option unavailable") - dependencies = system_dependencies((base, site)) - entries = tree(base, "/opt/python") + tree(site, "/opt/site-packages") - for destination, source in dependencies.items(): - path = Path(source) - entries.append( - { - "path": destination, - "type": "file", - "bytes": path.stat().st_size, - "sha256": sha(path.read_bytes()), - } - ) - probe = Path(__file__).with_name("benign_probe.py") - entries.append( - { - "path": "/opt/probe.py", - "type": "file", - "bytes": probe.stat().st_size, - "sha256": sha(probe.read_bytes()), - } - ) - entries.sort(key=lambda row: row["path"]) - # Bind-set identity describes actual readonly bytes, not an OCI image. - rootfs_digest = "sha256:" + sha(canonical(entries)) - packages = sorted( - [ - {"name": d.metadata["Name"], "version": d.version} - for d in importlib.metadata.distributions() - ], - key=lambda row: row["name"].lower(), - ) - # Match every pinned evaluator Python source byte to the installed package. - checked = 0 - source = bundle / "source" - for encoded in git(source, "ls-files", "-z", "evalplus").split(b"\0"): - if not encoded or not encoded.endswith(b".py"): - continue - name = encoded.decode() - if (source / name).read_bytes() != (site / name).read_bytes(): - raise ValueError("installed EvalPlus differs from pinned source") - checked += 1 - if checked == 0: - raise ValueError("no pinned evaluator Python sources verified") - output.mkdir(parents=True, exist_ok=False) - write( - output / "rootfs.json", - { - "schema": 1, - "rootfs_kind": "readonly-bind-set-v1", - "loader_policy": { - "scanner_inherits_environment": False, - "base_library_directory": "/opt/python/lib", - "sandbox_ld_library_path": "/opt/python/lib", - }, - "rootfs_digest": rootfs_digest, - "evaluator_revision": REVISION, - "entries": entries, - "python": sys.version.split()[0], - "packages": packages, - "checked_evaluator_sources": checked, - "executor": { - "kind": "bubblewrap", - "binary_sha256": sha(Path(bwrap).read_bytes()), - "version": subprocess.check_output([bwrap, "--version"]).decode().strip(), - }, - "generated_code_executed": False, - "qualified_generated_code_executor": False, - }, - ) - write( - output / "runtime-locators.private.json", - { - "python_root": str(base), - "site_root": str(site), - "python_relative": executable, - "bwrap": bwrap, - "system_libraries": dependencies, - }, - ) - - -def command(locators, probe, namespaces, sentinel): - args = [ - locators["bwrap"], - "--unshare-user", - "--uid", - "65534", - "--gid", - "65534", - "--unshare-net", - "--unshare-pid", - "--unshare-ipc", - "--unshare-uts", - "--hostname", - "sandbox", - "--disable-userns", - "--assert-userns-disabled", - "--cap-drop", - "ALL", - "--die-with-parent", - "--new-session", - "--clearenv", - "--ro-bind", - locators["python_root"], - "/opt/python", - "--ro-bind", - locators["site_root"], - "/opt/site-packages", - "--ro-bind", - str(probe), - "/opt/probe.py", - ] - for destination, source in sorted(locators["system_libraries"].items()): - args.extend(["--ro-bind", source, destination]) - args += [ - "--proc", - "/proc", - "--dev", - "/dev", - "--size", - "33554432", - "--tmpfs", - "/dev/shm", - "--size", - "67108864", - "--tmpfs", - "/work", - "--dir", - "/work/tmp", - "--dir", - "/work/output", - "--chdir", - "/work", - "--remount-ro", - "/", - ] - environment = { - "HOME": "/work", - "TMPDIR": "/work/tmp", - "PATH": "/opt/python/bin", - "LD_LIBRARY_PATH": "/opt/python/lib", - "PYTHONDONTWRITEBYTECODE": "1", - "OMP_NUM_THREADS": "1", - "OPENBLAS_NUM_THREADS": "1", - "LC_ALL": "C.UTF-8", - "PROBE_PARENT_NAMESPACES": json.dumps(namespaces), - "PROBE_UNMOUNTED_SENTINEL": str(sentinel), - } - for name, value in environment.items(): - args += ["--setenv", name, value] - return args + ["/opt/python/" + locators["python_relative"], "-I", "/opt/probe.py"] - - -def limits(): - os.sched_setaffinity(0, {min(os.sched_getaffinity(0))}) - for name, value in ( - ("AS", 2147483648), - ("CPU", 30), - ("NOFILE", 64), - ("FSIZE", 1048576), - ("CORE", 0), - ): - resource.setrlimit(getattr(resource, "RLIMIT_" + name), (value, value)) - - -def probe(inventory_path, output): - rootfs = json.loads((inventory_path / "rootfs.json").read_bytes()) - locators = json.loads((inventory_path / "runtime-locators.private.json").read_bytes()) - if sha(Path(locators["bwrap"]).read_bytes()) != rootfs["executor"]["binary_sha256"]: - raise ValueError("bubblewrap binary changed after inventory") - output.mkdir(parents=True, exist_ok=False) - sentinel = output / "unmounted-sentinel.private" - sentinel.write_text("synthetic host marker, not a credential") - namespaces = { - name: os.readlink("/proc/self/ns/" + name) for name in ("user", "net", "pid", "uts", "ipc") - } - args = command(locators, Path(__file__).with_name("benign_probe.py"), namespaces, sentinel) - timed_out = False - with ( - (output / "stdout.json").open("xb") as stdout, - (output / "stderr.private.log").open("xb") as stderr, - ): - process = subprocess.Popen( - args, - stdin=subprocess.DEVNULL, - stdout=stdout, - stderr=stderr, - start_new_session=True, - preexec_fn=limits, - env={**os.environ, "PROBE_INHERITED_SECRET": "synthetic-must-be-cleared"}, - ) - try: - code = process.wait(timeout=60) - except subprocess.TimeoutExpired: - timed_out = True - if os.getpgid(process.pid) != process.pid: - raise RuntimeError("owned sandbox process group changed unexpectedly") - os.killpg(process.pid, signal.SIGKILL) - code = process.wait(timeout=10) - result = { - "phase": "trusted_benign_probe", - "exit_code": code, - "timed_out": timed_out, - "rootfs_digest": rootfs["rootfs_digest"], - "executor": rootfs["executor"], - "policy_script_sha256": sha(Path(__file__).read_bytes()), - "probe_script_sha256": sha(Path(__file__).with_name("benign_probe.py").read_bytes()), - "stdout_sha256": sha((output / "stdout.json").read_bytes()), - "generated_code_executed": False, - "full_evaluator_executed": False, - "qualified_generated_code_executor": False, - "verdict": "INCONCLUSIVE", - } - write(output / "execution.json", result) - if code != 0 or timed_out: - raise ValueError("benign sandbox probe failed; retain evidence without fallback") - payload = json.loads((output / "stdout.json").read_bytes()) - if payload.get("passed") is not True: - raise ValueError("benign sandbox invariants failed") - - -def main(): - parser = argparse.ArgumentParser(description=__doc__) - parser.add_argument("phase", choices=("requirements", "verify-source", "inventory", "probe")) - parser.add_argument("--bundle", type=Path, required=True) - parser.add_argument("--output", type=Path) - parser.add_argument("--inventory", type=Path) - args = parser.parse_args() - if not os.environ.get("SLURM_JOB_ID") or os.environ.get("SLURM_GPUS_ON_NODE", "0") != "0": - raise ValueError("CPU-only Slurm allocation required") - if args.phase == "requirements": - requirements(args.bundle) - elif args.phase == "verify-source": - verify_source(args.bundle) - elif args.phase == "inventory": - inventory(args.bundle, args.output) - else: - probe(args.inventory, args.output) - - -if __name__ == "__main__": - main() diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v1/benign-reference-controls.capture-conflict.txt b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v1/benign-reference-controls.capture-conflict.txt deleted file mode 100644 index 31d7f576d..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v1/benign-reference-controls.capture-conflict.txt +++ /dev/null @@ -1,5 +0,0 @@ -{"benign_reference_cases": 13, "cases": ["observed_success", "exit_bool", "exit_nonzero", "timeout", "wrong_phase", "wrong_output_digest", "wrong_rootfs", "wrong_executor", "empty_checks", "missing_check", "integer_check", "integer_passed", "duplicate_key"], "optimized_python": true, "evaluator_or_sandbox_executed": false, "passed": true} - true, - "evaluator_or_sandbox_executed": false, - "passed": true -} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v1/benign-reference-controls.json b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v1/benign-reference-controls.json deleted file mode 100644 index 2a5d0061d..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v1/benign-reference-controls.json +++ /dev/null @@ -1,21 +0,0 @@ -{ - "benign_reference_cases": 13, - "cases": [ - "observed_success", - "exit_bool", - "exit_nonzero", - "timeout", - "wrong_phase", - "wrong_output_digest", - "wrong_rootfs", - "wrong_executor", - "empty_checks", - "missing_check", - "integer_check", - "integer_passed", - "duplicate_key" - ], - "optimized_python": true, - "evaluator_or_sandbox_executed": false, - "passed": true -} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v1/bounds_driver.initial.py b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v1/bounds_driver.initial.py deleted file mode 100644 index 7cc3dfc0a..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v1/bounds_driver.initial.py +++ /dev/null @@ -1,291 +0,0 @@ -"""Supervise only fixed trusted probes and retain actual namespace-init exit evidence.""" - -import argparse -import json -import os -import select -import signal -import subprocess -import sys -import time -from pathlib import Path - -import psutil -import runtime - - -def unique(pairs): - result = {} - for key, value in pairs: - if key in result: - raise ValueError("duplicate JSON key") - result[key] = value - return result - - -def read_line(fd, deadline): - raw = bytearray() - while time.monotonic() < deadline and len(raw) <= 1048576: - ready, _, _ = select.select([fd], [], [], max(0, deadline - time.monotonic())) - if not ready: - break - data = os.read(fd, 1) - if not data: - raise ValueError("premature protocol EOF") - raw.extend(data) - if data == b"\n": - return bytes(raw) - raise ValueError("trusted protocol size or time limit exceeded") - - -def exited(fd, milliseconds=0): - poller = select.poll() - poller.register(fd, select.POLLIN) - events = poller.poll(milliseconds) - if any(mask & (select.POLLERR | select.POLLNVAL) for _, mask in events): - raise ValueError("pidfd wait failed") - return any(mask & (select.POLLIN | select.POLLHUP) for _, mask in events) - - -def status(pid): - return dict( - line.split(":", 1) - for line in Path(f"/proc/{pid}/status").read_text().splitlines() - if ":" in line - ) - - -def bind_init(record, launcher_pid): - pid = record.get("child-pid") - if type(pid) is not int or pid <= 1: - raise ValueError("invalid namespace-init PID") - fd = os.pidfd_open(pid) - try: - fields = status(pid) - if exited(fd) or int(fields["PPid"]) != launcher_pid: - raise ValueError("namespace init identity or parent changed") - namespace_pids = [int(value) for value in fields["NSpid"].split()] - if namespace_pids[0] != pid or namespace_pids[-1] != 1: - raise ValueError("reported child is not namespace PID1") - # The child is blocked before payload execution. Only the PID namespace - # is compared: bwrap can create a further user namespace during setup. - pid_namespace = os.readlink(f"/proc/{pid}/ns/pid") - if pid_namespace != f"pid:[{record.get('pid-namespace')}]": - raise ValueError("reported PID namespace identity differs") - if exited(fd): - raise ValueError("namespace init exited during binding") - return fd, {"pid": pid, "namespace_pids": namespace_pids, "pid_namespace": pid_namespace} - except BaseException: - os.close(fd) - raise - - -def bind_children(init_pid, readiness): - wanted = readiness.get("live_descendant_pids") - if not isinstance(wanted, list) or len(wanted) != 2 or len(set(wanted)) != 2: - raise ValueError("expected two registered live trusted children") - if any(type(pid) is not int or pid <= 2 for pid in wanted): - raise ValueError("invalid trusted child PID") - held = [] - records = [] - try: - for child in psutil.Process(init_pid).children(recursive=True): - try: - fields = status(child.pid) - except FileNotFoundError: - continue - namespace_pids = [int(value) for value in fields["NSpid"].split()] - if namespace_pids[-1] not in wanted: - continue - fd = os.pidfd_open(child.pid) - held.append(fd) - current = status(child.pid) - if exited(fd) or current["NSpid"] != fields["NSpid"]: - raise ValueError("trusted child exited during binding") - records.append({"pid": child.pid, "namespace_pids": namespace_pids}) - if sorted(row["namespace_pids"][-1] for row in records) != sorted(wanted): - raise ValueError("registered trusted children not both observed") - return held, records - except BaseException: - for fd in held: - os.close(fd) - raise - - -def run_case(locators, rootfs, mode, output): - output.mkdir() - sentinel = output / "unmounted-sentinel.private" - sentinel.write_text("trusted synthetic marker") - namespaces = { - name: os.readlink("/proc/self/ns/" + name) for name in ("user", "net", "pid", "uts", "ipc") - } - args = runtime.command( - locators, Path(__file__).with_name("synthetic_probe.py"), namespaces, sentinel - ) - status_read, status_write = os.pipe() - block_read, block_write = os.pipe() - # Additional mount policy; the independent /dev/shm remains a bounded tmpfs. - args[1:1] = ["--json-status-fd", str(status_write), "--block-fd", str(block_read)] - payload_index = args.index("/opt/python/" + locators["python_relative"]) - args[payload_index:payload_index] = ["--remount-ro", "/dev"] - args.append(mode) - process = None - init_fd = None - children = [] - report = {"mode": mode, "passed": False, "generated_code_executed": False} - stdout_raw = b"" - status_raw = b"" - try: - with (output / "stderr.private.log").open("xb") as stderr: - process = subprocess.Popen( - [sys.executable, "-IB", str(Path(__file__).resolve()), "_exec", json.dumps(args)], - stdin=subprocess.DEVNULL, - stdout=subprocess.PIPE, - stderr=stderr, - pass_fds=(status_write, block_read), - start_new_session=True, - ) - os.close(status_write) - status_write = -1 - os.close(block_read) - block_read = -1 - status_raw = read_line(status_read, time.monotonic() + 10) - start = json.loads(status_raw, object_pairs_hook=unique) - init_fd, identity = bind_init(start, process.pid) - report["init_identity"] = identity - report["initial_status"] = start - os.write(block_write, b"1") - os.close(block_write) - block_write = -1 - if mode in ("bounds", "synthetic"): - stdout_raw, _ = process.communicate(timeout=60) - payload = json.loads(stdout_raw, object_pairs_hook=unique) - report["probe_passed"] = payload.get("passed") is True - else: - stdout_raw = read_line(process.stdout.fileno(), time.monotonic() + 10) - payload = json.loads(stdout_raw, object_pairs_hook=unique) - if payload.get("ready") is not True or payload.get("mode") != mode: - raise ValueError("trusted cleanup probe did not become ready") - if payload.get("namespace_pid") != identity["pid_namespace"]: - raise ValueError("payload PID namespace differs from bound init") - children, child_records = bind_children(identity["pid"], payload) - report["registered_child_identities"] = child_records - if mode == "timeout": - try: - process.wait(timeout=1) - except subprocess.TimeoutExpired: - report["timeout_observed"] = True - signal.pidfd_send_signal(init_fd, signal.SIGKILL) - else: - raise ValueError("timeout probe returned before deadline") - elif mode == "launcher_kill": - process.kill() - tail, _ = process.communicate(timeout=8) - stdout_raw += tail - report["probe_passed"] = True - report["exit_code"] = process.returncode - report["init_exit_observed"] = exited(init_fd, 5000) - report["registered_child_exit_observed"] = [exited(fd, 1000) for fd in children] - # Parent has exited and bwrap's payload closes the status fd. Read - # the remainder with a finite deadline; EOF is expected. - deadline = time.monotonic() + 2 - status_eof = False - while time.monotonic() < deadline: - ready, _, _ = select.select( - [status_read], [], [], max(0, deadline - time.monotonic()) - ) - if not ready: - break - data = os.read(status_read, 4096) - if not data: - status_eof = True - break - status_raw += data - if len(status_raw) > 65536: - raise ValueError("status stream unexpectedly large") - report["status_eof"] = status_eof - expected_exit = ( - -signal.SIGKILL if mode == "launcher_kill" else 137 if mode == "timeout" else 0 - ) - report["passed"] = ( - report["probe_passed"] - and process.returncode == expected_exit - and report["init_exit_observed"] - and all(report["registered_child_exit_observed"]) - and status_eof - ) - except BaseException as error: - report["error"] = {"type": type(error).__name__, "message": str(error)} - finally: - report["forced_cleanup"] = False - if init_fd is not None: - if not exited(init_fd): - signal.pidfd_send_signal(init_fd, signal.SIGKILL) - report["forced_cleanup"] = True - report["forced_init_exit_observed"] = exited(init_fd, 5000) - os.close(init_fd) - if process is not None and process.poll() is None: - process.kill() - process.wait(timeout=5) - report["forced_cleanup"] = True - for fd in children: - os.close(fd) - for fd in (status_read, status_write, block_read, block_write): - if fd >= 0: - os.close(fd) - if process is not None and process.stdout is not None: - process.stdout.close() - if report["forced_cleanup"]: - report["passed"] = False - (output / "stdout.json").write_bytes(stdout_raw) - (output / "status.private.jsonl").write_bytes(status_raw) - report["stdout_sha256"] = runtime.sha(stdout_raw) - report["status_sha256"] = runtime.sha(status_raw) - report["rootfs_digest"] = rootfs["rootfs_digest"] - runtime.write(output / "report.json", report) - return report - - -def main(): - parser = argparse.ArgumentParser(description=__doc__) - parser.add_argument("phase", choices=("probe",)) - parser.add_argument("--bundle", type=Path) - parser.add_argument("--inventory", type=Path, required=True) - parser.add_argument("--output", type=Path, required=True) - args = parser.parse_args() - rootfs = json.loads((args.inventory / "rootfs.json").read_bytes()) - locators = json.loads((args.inventory / "runtime-locators.private.json").read_bytes()) - if runtime.sha(Path(locators["bwrap"]).read_bytes()) != rootfs["executor"]["binary_sha256"]: - raise ValueError("bwrap binary differs from inventory") - args.output.mkdir() - results = [ - run_case(locators, rootfs, mode, args.output / mode) - for mode in ("bounds", "synthetic", "normal", "timeout", "launcher_kill") - ] - runtime.write( - args.output / "summary.json", - { - "scope": "trusted_kernel_boundary_and_cleanup_probes", - "passed": all(row["passed"] for row in results), - "cases": results, - "generated_code_executed": False, - "qualified_generated_code_executor": False, - "verdict": "INCONCLUSIVE", - "driver_sha256": runtime.sha(Path(__file__).read_bytes()), - "probe_sha256": runtime.sha( - Path(__file__).with_name("synthetic_probe.py").read_bytes() - ), - "runtime_sha256": runtime.sha(Path(runtime.__file__).read_bytes()), - }, - ) - return 0 if all(row["passed"] for row in results) else 2 - - -if __name__ == "__main__": - if not os.environ.get("SLURM_JOB_ID") or os.environ.get("SLURM_GPUS_ON_NODE", "0") != "0": - raise ValueError("CPU-only Slurm allocation required") - if len(sys.argv) == 3 and sys.argv[1] == "_exec": - runtime.limits() - command = json.loads(sys.argv[2]) - os.execv(command[0], command) - raise SystemExit(main()) diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v1/bounds_driver.py b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v1/bounds_driver.py deleted file mode 100644 index 5a65c882a..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v1/bounds_driver.py +++ /dev/null @@ -1,366 +0,0 @@ -"""Supervise only fixed trusted probes and retain actual namespace-init exit evidence.""" - -import argparse -import hashlib -import importlib.util -import json -import os -import select -import signal -import subprocess -import sys -import time -from pathlib import Path - -import psutil - -RUNTIME_PATH = Path(__file__).with_name("runtime.py") -RUNTIME_SHA256 = "997cbfed8398e8ad62d1beb4f6b4bc6e61a0f4572b9156cfd0c6f3fb52b03233" -if hashlib.sha256(RUNTIME_PATH.read_bytes()).hexdigest() != RUNTIME_SHA256: - raise ValueError("unexpected runtime source bytes") -runtime_spec = importlib.util.spec_from_file_location("trusted_bound_runtime", RUNTIME_PATH) -runtime = importlib.util.module_from_spec(runtime_spec) -runtime_spec.loader.exec_module(runtime) - - -def unique(pairs): - result = {} - for key, value in pairs: - if key in result: - raise ValueError("duplicate JSON key") - result[key] = value - return result - - -def read_line(fd, deadline, raw): - while time.monotonic() < deadline and len(raw) <= 1048576: - ready, _, _ = select.select([fd], [], [], max(0, deadline - time.monotonic())) - if not ready: - break - data = os.read(fd, 1) - if not data: - raise ValueError("premature protocol EOF") - raw.extend(data) - if data == b"\n": - return bytes(raw) - raise ValueError("trusted protocol size or time limit exceeded") - - -def read_ready_file(path, deadline): - while time.monotonic() < deadline: - raw = path.read_bytes() - if len(raw) > 1048576: - raise ValueError("trusted output exceeds size limit") - if raw.endswith(b"\n"): - return raw - time.sleep(0.01) - raise ValueError("trusted readiness deadline exceeded") - - -def exited(fd, milliseconds=0): - poller = select.poll() - poller.register(fd, select.POLLIN) - events = poller.poll(milliseconds) - if any(mask & (select.POLLERR | select.POLLNVAL) for _, mask in events): - raise ValueError("pidfd wait failed") - return any(mask & (select.POLLIN | select.POLLHUP) for _, mask in events) - - -def status(pid): - return dict( - line.split(":", 1) - for line in Path(f"/proc/{pid}/status").read_text().splitlines() - if ":" in line - ) - - -def bind_init(record, launcher_pid): - pid = record.get("child-pid") - if type(pid) is not int or pid <= 1: - raise ValueError("invalid namespace-init PID") - fd = os.pidfd_open(pid) - try: - fields = status(pid) - if exited(fd) or int(fields["PPid"]) != launcher_pid: - raise ValueError("namespace init identity or parent changed") - namespace_pids = [int(value) for value in fields["NSpid"].split()] - if namespace_pids[0] != pid or namespace_pids[-1] != 1: - raise ValueError("reported child is not namespace PID1") - # The child is blocked before payload execution. Only the PID namespace - # is compared: bwrap can create a further user namespace during setup. - pid_namespace = os.readlink(f"/proc/{pid}/ns/pid") - if pid_namespace != f"pid:[{record.get('pid-namespace')}]": - raise ValueError("reported PID namespace identity differs") - if exited(fd): - raise ValueError("namespace init exited during binding") - return fd, {"pid": pid, "namespace_pids": namespace_pids, "pid_namespace": pid_namespace} - except BaseException: - os.close(fd) - raise - - -def bind_children(init_pid, readiness): - wanted = readiness.get("live_descendant_pids") - if not isinstance(wanted, list) or len(wanted) != 2 or len(set(wanted)) != 2: - raise ValueError("expected two registered live trusted children") - if any(type(pid) is not int or pid <= 2 for pid in wanted): - raise ValueError("invalid trusted child PID") - held = [] - records = [] - try: - for child in psutil.Process(init_pid).children(recursive=True): - try: - fields = status(child.pid) - except FileNotFoundError: - continue - namespace_pids = [int(value) for value in fields["NSpid"].split()] - if namespace_pids[-1] not in wanted: - continue - fd = os.pidfd_open(child.pid) - held.append(fd) - current = status(child.pid) - if exited(fd) or current["NSpid"] != fields["NSpid"]: - raise ValueError("trusted child exited during binding") - records.append({"pid": child.pid, "namespace_pids": namespace_pids}) - if sorted(row["namespace_pids"][-1] for row in records) != sorted(wanted): - raise ValueError("registered trusted children not both observed") - return held, records - except BaseException: - for fd in held: - os.close(fd) - raise - - -def finalize_case(report, process, init_fd, children, pipes, output, status_raw, rootfs): - errors = [] - report["forced_cleanup"] = False - - def attempt(name, operation): - try: - return operation() - except BaseException as error: - errors.append({"operation": name, "type": type(error).__name__, "message": str(error)}) - return None - - if init_fd is not None: - if attempt("check_init_exit", lambda: exited(init_fd)) is not True: - try: - signal.pidfd_send_signal(init_fd, signal.SIGKILL) - report["forced_cleanup"] = True - except ProcessLookupError: - # Exit between poll and signal is not a failed cleanup if the - # stable pidfd independently confirms that same process exited. - confirmed = attempt("confirm_signal_race", lambda: exited(init_fd)) - report["signal_race_already_exited"] = confirmed is True - if confirmed is not True: - errors.append({"operation": "signal_init", "type": "UnconfirmedExitRace"}) - except BaseException as error: - errors.append( - { - "operation": "signal_init", - "type": type(error).__name__, - "message": str(error), - } - ) - report["forced_init_exit_observed"] = attempt( - "wait_init_exit", lambda: exited(init_fd, 5000) - ) - if report["forced_init_exit_observed"] is not True: - errors.append({"operation": "wait_init_exit", "type": "ExitNotConfirmed"}) - attempt("close_init_fd", lambda: os.close(init_fd)) - if process is not None: - if attempt("poll_launcher", process.poll) is None: - report["forced_cleanup"] = True - attempt("kill_launcher", process.kill) - attempt("wait_launcher", lambda: process.wait(timeout=5)) - report["final_launcher_exit_code"] = process.returncode - if process.returncode is None: - errors.append({"operation": "wait_launcher", "type": "ExitNotConfirmed"}) - report["cleanup_registered_child_exits"] = [] - for fd in children: - observed = attempt("wait_child_exit", lambda fd=fd: exited(fd, 1000)) - report["cleanup_registered_child_exits"].append(observed) - if observed is not True: - errors.append({"operation": "wait_child_exit", "type": "ExitNotConfirmed"}) - attempt("close_child_fd", lambda fd=fd: os.close(fd)) - for fd in pipes: - if fd >= 0: - attempt("close_pipe_fd", lambda fd=fd: os.close(fd)) - stdout_raw = attempt("read_stdout", (output / "stdout.json").read_bytes) - attempt("write_status", lambda: (output / "status.private.jsonl").write_bytes(status_raw)) - report["stdout_sha256"] = runtime.sha(stdout_raw) if stdout_raw is not None else None - report["status_sha256"] = runtime.sha(status_raw) - report["rootfs_digest"] = rootfs["rootfs_digest"] - report["cleanup_or_evidence_errors"] = errors - if report["forced_cleanup"] or errors: - report["passed"] = False - # If even the report destination fails, the wrapper still captures the - # nonzero process status and stderr; no success record is fabricated. - runtime.write(output / "report.json", report) - - -def run_case(locators, rootfs, mode, output): - output.mkdir() - sentinel = output / "unmounted-sentinel.private" - sentinel.write_text("trusted synthetic marker") - namespaces = { - name: os.readlink("/proc/self/ns/" + name) for name in ("user", "net", "pid", "uts", "ipc") - } - args = runtime.command( - locators, Path(__file__).with_name("synthetic_probe.py"), namespaces, sentinel - ) - status_read, status_write = os.pipe() - block_read, block_write = os.pipe() - # Additional mount policy; the independent /dev/shm remains a bounded tmpfs. - args[1:1] = ["--json-status-fd", str(status_write), "--block-fd", str(block_read)] - payload_index = args.index("/opt/python/" + locators["python_relative"]) - args[payload_index:payload_index] = ["--remount-ro", "/dev"] - args.append(mode) - process = None - init_fd = None - children = [] - report = {"mode": mode, "passed": False, "generated_code_executed": False} - stdout_raw = b"" - status_raw = bytearray() - try: - with ( - (output / "stderr.private.log").open("xb") as stderr, - (output / "stdout.json").open("xb") as stdout, - ): - process = subprocess.Popen( - [sys.executable, "-IB", str(Path(__file__).resolve()), "_exec", json.dumps(args)], - stdin=subprocess.DEVNULL, - stdout=stdout, - stderr=stderr, - pass_fds=(status_write, block_read), - start_new_session=True, - ) - os.close(status_write) - status_write = -1 - os.close(block_read) - block_read = -1 - read_line(status_read, time.monotonic() + 10, status_raw) - start = json.loads(status_raw, object_pairs_hook=unique) - init_fd, identity = bind_init(start, process.pid) - report["init_identity"] = identity - report["initial_status"] = start - os.write(block_write, b"1") - os.close(block_write) - block_write = -1 - if mode in ("bounds", "synthetic"): - process.wait(timeout=60) - stdout_raw = (output / "stdout.json").read_bytes() - payload = json.loads(stdout_raw, object_pairs_hook=unique) - report["probe_passed"] = payload.get("passed") is True - else: - stdout_raw = read_ready_file(output / "stdout.json", time.monotonic() + 10) - payload = json.loads(stdout_raw, object_pairs_hook=unique) - if payload.get("ready") is not True or payload.get("mode") != mode: - raise ValueError("trusted cleanup probe did not become ready") - if payload.get("namespace_pid") != identity["pid_namespace"]: - raise ValueError("payload PID namespace differs from bound init") - children, child_records = bind_children(identity["pid"], payload) - report["registered_child_identities"] = child_records - if mode == "timeout": - try: - process.wait(timeout=1) - except subprocess.TimeoutExpired: - report["timeout_observed"] = True - signal.pidfd_send_signal(init_fd, signal.SIGKILL) - else: - raise ValueError("timeout probe returned before deadline") - elif mode == "launcher_kill": - process.kill() - process.wait(timeout=8) - stdout_raw = (output / "stdout.json").read_bytes() - report["probe_passed"] = True - report["exit_code"] = process.returncode - report["init_exit_observed"] = exited(init_fd, 5000) - report["registered_child_exit_observed"] = [exited(fd, 1000) for fd in children] - # Parent has exited and bwrap's payload closes the status fd. Read - # the remainder with a finite deadline; EOF is expected. - deadline = time.monotonic() + 2 - status_eof = False - while time.monotonic() < deadline: - ready, _, _ = select.select( - [status_read], [], [], max(0, deadline - time.monotonic()) - ) - if not ready: - break - data = os.read(status_read, 4096) - if not data: - status_eof = True - break - status_raw += data - if len(status_raw) > 65536: - raise ValueError("status stream unexpectedly large") - report["status_eof"] = status_eof - expected_exit = ( - -signal.SIGKILL if mode == "launcher_kill" else 137 if mode == "timeout" else 0 - ) - report["passed"] = ( - report["probe_passed"] - and process.returncode == expected_exit - and report["init_exit_observed"] - and all(report["registered_child_exit_observed"]) - and status_eof - ) - except BaseException as error: - report["error"] = {"type": type(error).__name__, "message": str(error)} - finally: - finalize_case( - report, - process, - init_fd, - children, - (status_read, status_write, block_read, block_write), - output, - status_raw, - rootfs, - ) - return report - - -def main(): - parser = argparse.ArgumentParser(description=__doc__) - parser.add_argument("phase", choices=("probe",)) - parser.add_argument("--bundle", type=Path) - parser.add_argument("--inventory", type=Path, required=True) - parser.add_argument("--output", type=Path, required=True) - args = parser.parse_args() - rootfs = json.loads((args.inventory / "rootfs.json").read_bytes()) - locators = json.loads((args.inventory / "runtime-locators.private.json").read_bytes()) - if runtime.sha(Path(locators["bwrap"]).read_bytes()) != rootfs["executor"]["binary_sha256"]: - raise ValueError("bwrap binary differs from inventory") - args.output.mkdir() - results = [ - run_case(locators, rootfs, mode, args.output / mode) - for mode in ("bounds", "synthetic", "normal", "timeout", "launcher_kill") - ] - runtime.write( - args.output / "summary.json", - { - "scope": "trusted_kernel_boundary_and_cleanup_probes", - "passed": all(row["passed"] for row in results), - "cases": results, - "generated_code_executed": False, - "qualified_generated_code_executor": False, - "verdict": "INCONCLUSIVE", - "driver_sha256": runtime.sha(Path(__file__).read_bytes()), - "probe_sha256": runtime.sha( - Path(__file__).with_name("synthetic_probe.py").read_bytes() - ), - "runtime_sha256": runtime.sha(Path(runtime.__file__).read_bytes()), - }, - ) - return 0 if all(row["passed"] for row in results) else 2 - - -if __name__ == "__main__": - if not os.environ.get("SLURM_JOB_ID") or os.environ.get("SLURM_GPUS_ON_NODE", "0") != "0": - raise ValueError("CPU-only Slurm allocation required") - if len(sys.argv) == 3 and sys.argv[1] == "_exec": - runtime.limits() - command = json.loads(sys.argv[2]) - os.execv(command[0], command) - raise SystemExit(main()) diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v1/bounds_driver.review-rejected-cleanup.py b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v1/bounds_driver.review-rejected-cleanup.py deleted file mode 100644 index d6e6c093b..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v1/bounds_driver.review-rejected-cleanup.py +++ /dev/null @@ -1,313 +0,0 @@ -"""Supervise only fixed trusted probes and retain actual namespace-init exit evidence.""" - -import argparse -import hashlib -import importlib.util -import json -import os -import select -import signal -import subprocess -import sys -import time -from pathlib import Path - -import psutil - -RUNTIME_PATH = Path(__file__).with_name("runtime.py") -RUNTIME_SHA256 = "997cbfed8398e8ad62d1beb4f6b4bc6e61a0f4572b9156cfd0c6f3fb52b03233" -if hashlib.sha256(RUNTIME_PATH.read_bytes()).hexdigest() != RUNTIME_SHA256: - raise ValueError("unexpected runtime source bytes") -runtime_spec = importlib.util.spec_from_file_location("trusted_bound_runtime", RUNTIME_PATH) -runtime = importlib.util.module_from_spec(runtime_spec) -runtime_spec.loader.exec_module(runtime) - - -def unique(pairs): - result = {} - for key, value in pairs: - if key in result: - raise ValueError("duplicate JSON key") - result[key] = value - return result - - -def read_line(fd, deadline): - raw = bytearray() - while time.monotonic() < deadline and len(raw) <= 1048576: - ready, _, _ = select.select([fd], [], [], max(0, deadline - time.monotonic())) - if not ready: - break - data = os.read(fd, 1) - if not data: - raise ValueError("premature protocol EOF") - raw.extend(data) - if data == b"\n": - return bytes(raw) - raise ValueError("trusted protocol size or time limit exceeded") - - -def read_ready_file(path, deadline): - while time.monotonic() < deadline: - raw = path.read_bytes() - if len(raw) > 1048576: - raise ValueError("trusted output exceeds size limit") - if raw.endswith(b"\n"): - return raw - time.sleep(0.01) - raise ValueError("trusted readiness deadline exceeded") - - -def exited(fd, milliseconds=0): - poller = select.poll() - poller.register(fd, select.POLLIN) - events = poller.poll(milliseconds) - if any(mask & (select.POLLERR | select.POLLNVAL) for _, mask in events): - raise ValueError("pidfd wait failed") - return any(mask & (select.POLLIN | select.POLLHUP) for _, mask in events) - - -def status(pid): - return dict( - line.split(":", 1) - for line in Path(f"/proc/{pid}/status").read_text().splitlines() - if ":" in line - ) - - -def bind_init(record, launcher_pid): - pid = record.get("child-pid") - if type(pid) is not int or pid <= 1: - raise ValueError("invalid namespace-init PID") - fd = os.pidfd_open(pid) - try: - fields = status(pid) - if exited(fd) or int(fields["PPid"]) != launcher_pid: - raise ValueError("namespace init identity or parent changed") - namespace_pids = [int(value) for value in fields["NSpid"].split()] - if namespace_pids[0] != pid or namespace_pids[-1] != 1: - raise ValueError("reported child is not namespace PID1") - # The child is blocked before payload execution. Only the PID namespace - # is compared: bwrap can create a further user namespace during setup. - pid_namespace = os.readlink(f"/proc/{pid}/ns/pid") - if pid_namespace != f"pid:[{record.get('pid-namespace')}]": - raise ValueError("reported PID namespace identity differs") - if exited(fd): - raise ValueError("namespace init exited during binding") - return fd, {"pid": pid, "namespace_pids": namespace_pids, "pid_namespace": pid_namespace} - except BaseException: - os.close(fd) - raise - - -def bind_children(init_pid, readiness): - wanted = readiness.get("live_descendant_pids") - if not isinstance(wanted, list) or len(wanted) != 2 or len(set(wanted)) != 2: - raise ValueError("expected two registered live trusted children") - if any(type(pid) is not int or pid <= 2 for pid in wanted): - raise ValueError("invalid trusted child PID") - held = [] - records = [] - try: - for child in psutil.Process(init_pid).children(recursive=True): - try: - fields = status(child.pid) - except FileNotFoundError: - continue - namespace_pids = [int(value) for value in fields["NSpid"].split()] - if namespace_pids[-1] not in wanted: - continue - fd = os.pidfd_open(child.pid) - held.append(fd) - current = status(child.pid) - if exited(fd) or current["NSpid"] != fields["NSpid"]: - raise ValueError("trusted child exited during binding") - records.append({"pid": child.pid, "namespace_pids": namespace_pids}) - if sorted(row["namespace_pids"][-1] for row in records) != sorted(wanted): - raise ValueError("registered trusted children not both observed") - return held, records - except BaseException: - for fd in held: - os.close(fd) - raise - - -def run_case(locators, rootfs, mode, output): - output.mkdir() - sentinel = output / "unmounted-sentinel.private" - sentinel.write_text("trusted synthetic marker") - namespaces = { - name: os.readlink("/proc/self/ns/" + name) for name in ("user", "net", "pid", "uts", "ipc") - } - args = runtime.command( - locators, Path(__file__).with_name("synthetic_probe.py"), namespaces, sentinel - ) - status_read, status_write = os.pipe() - block_read, block_write = os.pipe() - # Additional mount policy; the independent /dev/shm remains a bounded tmpfs. - args[1:1] = ["--json-status-fd", str(status_write), "--block-fd", str(block_read)] - payload_index = args.index("/opt/python/" + locators["python_relative"]) - args[payload_index:payload_index] = ["--remount-ro", "/dev"] - args.append(mode) - process = None - init_fd = None - children = [] - report = {"mode": mode, "passed": False, "generated_code_executed": False} - stdout_raw = b"" - status_raw = b"" - try: - with ( - (output / "stderr.private.log").open("xb") as stderr, - (output / "stdout.json").open("xb") as stdout, - ): - process = subprocess.Popen( - [sys.executable, "-IB", str(Path(__file__).resolve()), "_exec", json.dumps(args)], - stdin=subprocess.DEVNULL, - stdout=stdout, - stderr=stderr, - pass_fds=(status_write, block_read), - start_new_session=True, - ) - os.close(status_write) - status_write = -1 - os.close(block_read) - block_read = -1 - status_raw = read_line(status_read, time.monotonic() + 10) - start = json.loads(status_raw, object_pairs_hook=unique) - init_fd, identity = bind_init(start, process.pid) - report["init_identity"] = identity - report["initial_status"] = start - os.write(block_write, b"1") - os.close(block_write) - block_write = -1 - if mode in ("bounds", "synthetic"): - process.wait(timeout=60) - stdout_raw = (output / "stdout.json").read_bytes() - payload = json.loads(stdout_raw, object_pairs_hook=unique) - report["probe_passed"] = payload.get("passed") is True - else: - stdout_raw = read_ready_file(output / "stdout.json", time.monotonic() + 10) - payload = json.loads(stdout_raw, object_pairs_hook=unique) - if payload.get("ready") is not True or payload.get("mode") != mode: - raise ValueError("trusted cleanup probe did not become ready") - if payload.get("namespace_pid") != identity["pid_namespace"]: - raise ValueError("payload PID namespace differs from bound init") - children, child_records = bind_children(identity["pid"], payload) - report["registered_child_identities"] = child_records - if mode == "timeout": - try: - process.wait(timeout=1) - except subprocess.TimeoutExpired: - report["timeout_observed"] = True - signal.pidfd_send_signal(init_fd, signal.SIGKILL) - else: - raise ValueError("timeout probe returned before deadline") - elif mode == "launcher_kill": - process.kill() - process.wait(timeout=8) - stdout_raw = (output / "stdout.json").read_bytes() - report["probe_passed"] = True - report["exit_code"] = process.returncode - report["init_exit_observed"] = exited(init_fd, 5000) - report["registered_child_exit_observed"] = [exited(fd, 1000) for fd in children] - # Parent has exited and bwrap's payload closes the status fd. Read - # the remainder with a finite deadline; EOF is expected. - deadline = time.monotonic() + 2 - status_eof = False - while time.monotonic() < deadline: - ready, _, _ = select.select( - [status_read], [], [], max(0, deadline - time.monotonic()) - ) - if not ready: - break - data = os.read(status_read, 4096) - if not data: - status_eof = True - break - status_raw += data - if len(status_raw) > 65536: - raise ValueError("status stream unexpectedly large") - report["status_eof"] = status_eof - expected_exit = ( - -signal.SIGKILL if mode == "launcher_kill" else 137 if mode == "timeout" else 0 - ) - report["passed"] = ( - report["probe_passed"] - and process.returncode == expected_exit - and report["init_exit_observed"] - and all(report["registered_child_exit_observed"]) - and status_eof - ) - except BaseException as error: - report["error"] = {"type": type(error).__name__, "message": str(error)} - finally: - report["forced_cleanup"] = False - if init_fd is not None: - if not exited(init_fd): - signal.pidfd_send_signal(init_fd, signal.SIGKILL) - report["forced_cleanup"] = True - report["forced_init_exit_observed"] = exited(init_fd, 5000) - os.close(init_fd) - if process is not None and process.poll() is None: - process.kill() - process.wait(timeout=5) - report["forced_cleanup"] = True - for fd in children: - os.close(fd) - for fd in (status_read, status_write, block_read, block_write): - if fd >= 0: - os.close(fd) - if report["forced_cleanup"]: - report["passed"] = False - stdout_raw = (output / "stdout.json").read_bytes() - (output / "status.private.jsonl").write_bytes(status_raw) - report["stdout_sha256"] = runtime.sha(stdout_raw) - report["status_sha256"] = runtime.sha(status_raw) - report["rootfs_digest"] = rootfs["rootfs_digest"] - runtime.write(output / "report.json", report) - return report - - -def main(): - parser = argparse.ArgumentParser(description=__doc__) - parser.add_argument("phase", choices=("probe",)) - parser.add_argument("--bundle", type=Path) - parser.add_argument("--inventory", type=Path, required=True) - parser.add_argument("--output", type=Path, required=True) - args = parser.parse_args() - rootfs = json.loads((args.inventory / "rootfs.json").read_bytes()) - locators = json.loads((args.inventory / "runtime-locators.private.json").read_bytes()) - if runtime.sha(Path(locators["bwrap"]).read_bytes()) != rootfs["executor"]["binary_sha256"]: - raise ValueError("bwrap binary differs from inventory") - args.output.mkdir() - results = [ - run_case(locators, rootfs, mode, args.output / mode) - for mode in ("bounds", "synthetic", "normal", "timeout", "launcher_kill") - ] - runtime.write( - args.output / "summary.json", - { - "scope": "trusted_kernel_boundary_and_cleanup_probes", - "passed": all(row["passed"] for row in results), - "cases": results, - "generated_code_executed": False, - "qualified_generated_code_executor": False, - "verdict": "INCONCLUSIVE", - "driver_sha256": runtime.sha(Path(__file__).read_bytes()), - "probe_sha256": runtime.sha( - Path(__file__).with_name("synthetic_probe.py").read_bytes() - ), - "runtime_sha256": runtime.sha(Path(runtime.__file__).read_bytes()), - }, - ) - return 0 if all(row["passed"] for row in results) else 2 - - -if __name__ == "__main__": - if not os.environ.get("SLURM_JOB_ID") or os.environ.get("SLURM_GPUS_ON_NODE", "0") != "0": - raise ValueError("CPU-only Slurm allocation required") - if len(sys.argv) == 3 and sys.argv[1] == "_exec": - runtime.limits() - command = json.loads(sys.argv[2]) - os.execv(command[0], command) - raise SystemExit(main()) diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v1/bounds_driver.review-rejected-import.py b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v1/bounds_driver.review-rejected-import.py deleted file mode 100644 index 33ba54531..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v1/bounds_driver.review-rejected-import.py +++ /dev/null @@ -1,304 +0,0 @@ -"""Supervise only fixed trusted probes and retain actual namespace-init exit evidence.""" - -import argparse -import json -import os -import select -import signal -import subprocess -import sys -import time -from pathlib import Path - -import psutil -import runtime - - -def unique(pairs): - result = {} - for key, value in pairs: - if key in result: - raise ValueError("duplicate JSON key") - result[key] = value - return result - - -def read_line(fd, deadline): - raw = bytearray() - while time.monotonic() < deadline and len(raw) <= 1048576: - ready, _, _ = select.select([fd], [], [], max(0, deadline - time.monotonic())) - if not ready: - break - data = os.read(fd, 1) - if not data: - raise ValueError("premature protocol EOF") - raw.extend(data) - if data == b"\n": - return bytes(raw) - raise ValueError("trusted protocol size or time limit exceeded") - - -def read_ready_file(path, deadline): - while time.monotonic() < deadline: - raw = path.read_bytes() - if len(raw) > 1048576: - raise ValueError("trusted output exceeds size limit") - if raw.endswith(b"\n"): - return raw - time.sleep(0.01) - raise ValueError("trusted readiness deadline exceeded") - - -def exited(fd, milliseconds=0): - poller = select.poll() - poller.register(fd, select.POLLIN) - events = poller.poll(milliseconds) - if any(mask & (select.POLLERR | select.POLLNVAL) for _, mask in events): - raise ValueError("pidfd wait failed") - return any(mask & (select.POLLIN | select.POLLHUP) for _, mask in events) - - -def status(pid): - return dict( - line.split(":", 1) - for line in Path(f"/proc/{pid}/status").read_text().splitlines() - if ":" in line - ) - - -def bind_init(record, launcher_pid): - pid = record.get("child-pid") - if type(pid) is not int or pid <= 1: - raise ValueError("invalid namespace-init PID") - fd = os.pidfd_open(pid) - try: - fields = status(pid) - if exited(fd) or int(fields["PPid"]) != launcher_pid: - raise ValueError("namespace init identity or parent changed") - namespace_pids = [int(value) for value in fields["NSpid"].split()] - if namespace_pids[0] != pid or namespace_pids[-1] != 1: - raise ValueError("reported child is not namespace PID1") - # The child is blocked before payload execution. Only the PID namespace - # is compared: bwrap can create a further user namespace during setup. - pid_namespace = os.readlink(f"/proc/{pid}/ns/pid") - if pid_namespace != f"pid:[{record.get('pid-namespace')}]": - raise ValueError("reported PID namespace identity differs") - if exited(fd): - raise ValueError("namespace init exited during binding") - return fd, {"pid": pid, "namespace_pids": namespace_pids, "pid_namespace": pid_namespace} - except BaseException: - os.close(fd) - raise - - -def bind_children(init_pid, readiness): - wanted = readiness.get("live_descendant_pids") - if not isinstance(wanted, list) or len(wanted) != 2 or len(set(wanted)) != 2: - raise ValueError("expected two registered live trusted children") - if any(type(pid) is not int or pid <= 2 for pid in wanted): - raise ValueError("invalid trusted child PID") - held = [] - records = [] - try: - for child in psutil.Process(init_pid).children(recursive=True): - try: - fields = status(child.pid) - except FileNotFoundError: - continue - namespace_pids = [int(value) for value in fields["NSpid"].split()] - if namespace_pids[-1] not in wanted: - continue - fd = os.pidfd_open(child.pid) - held.append(fd) - current = status(child.pid) - if exited(fd) or current["NSpid"] != fields["NSpid"]: - raise ValueError("trusted child exited during binding") - records.append({"pid": child.pid, "namespace_pids": namespace_pids}) - if sorted(row["namespace_pids"][-1] for row in records) != sorted(wanted): - raise ValueError("registered trusted children not both observed") - return held, records - except BaseException: - for fd in held: - os.close(fd) - raise - - -def run_case(locators, rootfs, mode, output): - output.mkdir() - sentinel = output / "unmounted-sentinel.private" - sentinel.write_text("trusted synthetic marker") - namespaces = { - name: os.readlink("/proc/self/ns/" + name) for name in ("user", "net", "pid", "uts", "ipc") - } - args = runtime.command( - locators, Path(__file__).with_name("synthetic_probe.py"), namespaces, sentinel - ) - status_read, status_write = os.pipe() - block_read, block_write = os.pipe() - # Additional mount policy; the independent /dev/shm remains a bounded tmpfs. - args[1:1] = ["--json-status-fd", str(status_write), "--block-fd", str(block_read)] - payload_index = args.index("/opt/python/" + locators["python_relative"]) - args[payload_index:payload_index] = ["--remount-ro", "/dev"] - args.append(mode) - process = None - init_fd = None - children = [] - report = {"mode": mode, "passed": False, "generated_code_executed": False} - stdout_raw = b"" - status_raw = b"" - try: - with ( - (output / "stderr.private.log").open("xb") as stderr, - (output / "stdout.json").open("xb") as stdout, - ): - process = subprocess.Popen( - [sys.executable, "-IB", str(Path(__file__).resolve()), "_exec", json.dumps(args)], - stdin=subprocess.DEVNULL, - stdout=stdout, - stderr=stderr, - pass_fds=(status_write, block_read), - start_new_session=True, - ) - os.close(status_write) - status_write = -1 - os.close(block_read) - block_read = -1 - status_raw = read_line(status_read, time.monotonic() + 10) - start = json.loads(status_raw, object_pairs_hook=unique) - init_fd, identity = bind_init(start, process.pid) - report["init_identity"] = identity - report["initial_status"] = start - os.write(block_write, b"1") - os.close(block_write) - block_write = -1 - if mode in ("bounds", "synthetic"): - process.wait(timeout=60) - stdout_raw = (output / "stdout.json").read_bytes() - payload = json.loads(stdout_raw, object_pairs_hook=unique) - report["probe_passed"] = payload.get("passed") is True - else: - stdout_raw = read_ready_file(output / "stdout.json", time.monotonic() + 10) - payload = json.loads(stdout_raw, object_pairs_hook=unique) - if payload.get("ready") is not True or payload.get("mode") != mode: - raise ValueError("trusted cleanup probe did not become ready") - if payload.get("namespace_pid") != identity["pid_namespace"]: - raise ValueError("payload PID namespace differs from bound init") - children, child_records = bind_children(identity["pid"], payload) - report["registered_child_identities"] = child_records - if mode == "timeout": - try: - process.wait(timeout=1) - except subprocess.TimeoutExpired: - report["timeout_observed"] = True - signal.pidfd_send_signal(init_fd, signal.SIGKILL) - else: - raise ValueError("timeout probe returned before deadline") - elif mode == "launcher_kill": - process.kill() - process.wait(timeout=8) - stdout_raw = (output / "stdout.json").read_bytes() - report["probe_passed"] = True - report["exit_code"] = process.returncode - report["init_exit_observed"] = exited(init_fd, 5000) - report["registered_child_exit_observed"] = [exited(fd, 1000) for fd in children] - # Parent has exited and bwrap's payload closes the status fd. Read - # the remainder with a finite deadline; EOF is expected. - deadline = time.monotonic() + 2 - status_eof = False - while time.monotonic() < deadline: - ready, _, _ = select.select( - [status_read], [], [], max(0, deadline - time.monotonic()) - ) - if not ready: - break - data = os.read(status_read, 4096) - if not data: - status_eof = True - break - status_raw += data - if len(status_raw) > 65536: - raise ValueError("status stream unexpectedly large") - report["status_eof"] = status_eof - expected_exit = ( - -signal.SIGKILL if mode == "launcher_kill" else 137 if mode == "timeout" else 0 - ) - report["passed"] = ( - report["probe_passed"] - and process.returncode == expected_exit - and report["init_exit_observed"] - and all(report["registered_child_exit_observed"]) - and status_eof - ) - except BaseException as error: - report["error"] = {"type": type(error).__name__, "message": str(error)} - finally: - report["forced_cleanup"] = False - if init_fd is not None: - if not exited(init_fd): - signal.pidfd_send_signal(init_fd, signal.SIGKILL) - report["forced_cleanup"] = True - report["forced_init_exit_observed"] = exited(init_fd, 5000) - os.close(init_fd) - if process is not None and process.poll() is None: - process.kill() - process.wait(timeout=5) - report["forced_cleanup"] = True - for fd in children: - os.close(fd) - for fd in (status_read, status_write, block_read, block_write): - if fd >= 0: - os.close(fd) - if report["forced_cleanup"]: - report["passed"] = False - stdout_raw = (output / "stdout.json").read_bytes() - (output / "status.private.jsonl").write_bytes(status_raw) - report["stdout_sha256"] = runtime.sha(stdout_raw) - report["status_sha256"] = runtime.sha(status_raw) - report["rootfs_digest"] = rootfs["rootfs_digest"] - runtime.write(output / "report.json", report) - return report - - -def main(): - parser = argparse.ArgumentParser(description=__doc__) - parser.add_argument("phase", choices=("probe",)) - parser.add_argument("--bundle", type=Path) - parser.add_argument("--inventory", type=Path, required=True) - parser.add_argument("--output", type=Path, required=True) - args = parser.parse_args() - rootfs = json.loads((args.inventory / "rootfs.json").read_bytes()) - locators = json.loads((args.inventory / "runtime-locators.private.json").read_bytes()) - if runtime.sha(Path(locators["bwrap"]).read_bytes()) != rootfs["executor"]["binary_sha256"]: - raise ValueError("bwrap binary differs from inventory") - args.output.mkdir() - results = [ - run_case(locators, rootfs, mode, args.output / mode) - for mode in ("bounds", "synthetic", "normal", "timeout", "launcher_kill") - ] - runtime.write( - args.output / "summary.json", - { - "scope": "trusted_kernel_boundary_and_cleanup_probes", - "passed": all(row["passed"] for row in results), - "cases": results, - "generated_code_executed": False, - "qualified_generated_code_executor": False, - "verdict": "INCONCLUSIVE", - "driver_sha256": runtime.sha(Path(__file__).read_bytes()), - "probe_sha256": runtime.sha( - Path(__file__).with_name("synthetic_probe.py").read_bytes() - ), - "runtime_sha256": runtime.sha(Path(runtime.__file__).read_bytes()), - }, - ) - return 0 if all(row["passed"] for row in results) else 2 - - -if __name__ == "__main__": - if not os.environ.get("SLURM_JOB_ID") or os.environ.get("SLURM_GPUS_ON_NODE", "0") != "0": - raise ValueError("CPU-only Slurm allocation required") - if len(sys.argv) == 3 and sys.argv[1] == "_exec": - runtime.limits() - command = json.loads(sys.argv[2]) - os.execv(command[0], command) - raise SystemExit(main()) diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v1/controls.json b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v1/controls.json deleted file mode 100644 index 426ed9e82..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v1/controls.json +++ /dev/null @@ -1 +0,0 @@ -{"passed": true, "controls": ["all native syscall decisions 0 through 450", "three other architectures and two x32 calls denied", "duplicate status keys rejected", "valid namespace-init identity accepted", "five invalid init identities rejected", "exit during identity binding rejected and pidfd closed", "partial status retained on EOF and deadline", "cleanup exit race, wait timeout and stdout failure retain original errors and records", "three runtime sources compile", "isolated interpreter loads exact adjacent runtime; psutil import stubbed"], "linux_runtime_executed": false} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v1/plan.json b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v1/plan.json deleted file mode 100644 index c940887af..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v1/plan.json +++ /dev/null @@ -1,70 +0,0 @@ -{ - "scope": "finite_trusted_kernel_boundaries_and_cleanup", - "status": "reviewed_not_submitted", - "plane": "harness", - "resources": { - "cpus": 4, - "requested_memory_gib": 16, - "walltime_seconds": 900, - "gpus": 0 - }, - "modes": [ - "bounds", - "synthetic", - "normal", - "timeout", - "launcher_kill" - ], - "source_sha256": { - "runtime.py": "997cbfed8398e8ad62d1beb4f6b4bc6e61a0f4572b9156cfd0c6f3fb52b03233", - "bounds_driver.py": "86e5706deaec92b3b1a976382856062cb8d4d305def38d05df050af001664428", - "synthetic_probe.py": "e8492c07878e12f17be272a2b5dc873bae8413de1ebf7ab3f2fce1a3d9e9dbed", - "run-synthetic.sbatch": "8d0f4405931e32ca5c9cbc3a602acdde8e71e999dea379d791f58c859870a46b" - }, - "policy": { - "process_hard_limit": 16, - "per_process_address_space_bytes": 2147483648, - "memlock_bytes": 0, - "posix_message_queue_bytes": 0, - "dev_mount_readonly": true, - "sysv_and_mqopen_seccomp_denied": [ - 29, - 30, - 31, - 64, - 65, - 66, - 67, - 68, - 69, - 70, - 71, - 220, - 240 - ], - "architecture": "Linux x86_64 LP64; foreign ABI and x32 rejected", - "shm_tmpfs_bytes": 33554432, - "work_tmpfs_bytes": 67108864 - }, - "scope_limits": [ - "No generated model program or benchmark data", - "Not an aggregate 16GiB hard memory cap", - "Thread RuntimeError cause remains unclassified", - "No formal qualification, gate activation, candidate search or measured E2E effect" - ], - "review": "Independent review of exact source hashes and negative controls cleared bounded trusted execution", - "prior_rejected_versions": [ - "bounds_driver.initial.py", - "bounds_driver.review-rejected-import.py", - "bounds_driver.review-rejected-cleanup.py", - "benign-reference-controls.capture-conflict.txt" - ], - "source_references": [ - "https://github.com/containers/bubblewrap/tree/362936aea0d5c6e70f7af75d2d44b5a914cb4ba2", - "https://raw.githubusercontent.com/torvalds/linux/v6.12/arch/x86/entry/syscalls/syscall_64.tbl", - "https://man7.org/linux/man-pages/man2/getrlimit.2.html", - "https://man7.org/linux/man-pages/man7/pid_namespaces.7.html", - "https://man7.org/linux/man-pages/man2/pidfd_open.2.html", - "https://man7.org/linux/man-pages/man7/ipc_namespaces.7.html" - ] -} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v1/proposal.md b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v1/proposal.md deleted file mode 100644 index c7acaf13f..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v1/proposal.md +++ /dev/null @@ -1,37 +0,0 @@ -# Finite resource and namespace cleanup diagnostic - -Refs [Issue #67](https://github.com/InferMatrixLab/InferRSI/issues/67). -INCONCLUSIVE; reviewed preparation, not yet submitted. - -CPU2009 demonstrated correct evaluation of four trusted handwritten cases, but -its polling did not prove aggregate limits or complete cleanup. Source inspection -also found a writable independent /dev tmpfs and unbounded SysV object creation. -This bounded follow-up makes /dev readonly while preserving its limited /dev/shm, -sets finite task/address-space/file limits, and applies a narrow ABI-checked kernel -filter to SysV calls and mq_open. It leaves POSIX semaphore operations available. -These restrictions require an actual repeat of the four-case EvalPlus API check. - -Five fixed modes cover limit rejections and inheritance, synthetic evaluation, -and normal/timeout/launcher-kill cleanup of two registered live descendants, -including a setsid double-fork. The supervisor blocks payload startup until it -has bound the actual namespace PID1 to a pidfd and parent/NSpid/PID namespace -identity. Early bubblewrap user-namespace metadata is not treated as final payload -identity. The supervisor waits for actual init/known-child exits and retains errors. -No full generated-code qualification is claimed. RLIMIT_NPROC counts UID threads; -RuntimeError from thread creation is retained without assigning a kernel cause. -The 16GiB scheduler request is not an enforced aggregate memory limit. - -Pre-run independent review corrected isolated-interpreter import resolution, -cleanup errors masking evidence, and lost partial status. Rejected sources remain -preserved. A local control capture conflict corrupted a JSON when stdout and the -test both wrote the same file; the damaged record is preserved and the valid -re-run agrees with independent execution. Local tooling has no psutil; isolated -import controls stub only that module, while cluster execution uses the previously -inventoried registered package. New source lint and shell syntax pass. The existing -frozen runtime style findings remain disclosed. New logic controls and the prior -18 completion / 13 benign-reference cases were independently rerun successfully. - -The actual Python/evaluator dependencies and prior reference inventories are -unchanged. No heldout inputs, model-generated program, GPU or E2E run is authorized -by this preparation. Source-level and Linux-manual references are in plan.json; -the tested source identities and the original rejected attempts remain separate. diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v1/reference-controls.json b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v1/reference-controls.json deleted file mode 100644 index 92d128cf2..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v1/reference-controls.json +++ /dev/null @@ -1 +0,0 @@ -{"reference_completion_cases": 18, "optimized_python": true} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v1/run-synthetic.sbatch b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v1/run-synthetic.sbatch deleted file mode 100644 index fca77cb19..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v1/run-synthetic.sbatch +++ /dev/null @@ -1,197 +0,0 @@ -#!/usr/bin/env bash -# Finite trusted resource/cleanup probes and four synthetic evaluator cases. -#SBATCH --job-name=qwen38-sandbox-bounds-v1 -#SBATCH --nodes=1 -#SBATCH --ntasks=1 -#SBATCH --cpus-per-task=4 -#SBATCH --mem=16G -#SBATCH --time=00:15:00 -set -euo pipefail -umask 077 -: "${TASK_ROOT:?}" -: "${SANDBOX_BUNDLE:?}" -: "${SANDBOX_FREEZE_SHA256:?}" -: "${SANDBOX_VENV:?}" -: "${REFERENCE_INSTALLATION:?}" -: "${REFERENCE_ROOTFS_SHA256:?}" -: "${REFERENCE_TERMINAL_SHA256:?}" -: "${REFERENCE_SLURM_SHA256:?}" -: "${REFERENCE_JOB_ID:?}" -: "${REFERENCE_EXECUTION_SHA256:?}" "${REFERENCE_STDOUT_SHA256:?}" -: "${BWRAP_BIN:?}" "${BWRAP_SHA256:?}" "${PYTHON_SHA256:?}" -: "${SYNTHETIC_SOURCE_DIR:?}" -: "${SYNTHETIC_RUNTIME_SHA256:?}" -: "${SYNTHETIC_PROBE_SHA256:?}" -: "${BOUNDS_DRIVER_SHA256:?}" -: "${SYNTHETIC_SCRIPT_SHA256:?}" -: "${SYNTHETIC_OUTPUT:?}" -unset PYTHONPATH PYTHONHOME PYTHONOPTIMIZE PYTHONINSPECT PYTHONSTARTUP VIRTUAL_ENV -[[ -n ${SLURM_JOB_ID:-} && ${SLURM_GPUS_ON_NODE:-0} == 0 ]] -root=$(realpath -e "$TASK_ROOT") -check_path() { - local value - value=$(realpath -m "$1") - case "$value" in "$root"/*) ;; *) echo 'Path escapes task root.' >&2; exit 2;; esac -} -check_digest() { - if [[ ! "$2" =~ ^[a-f0-9]{64}$ ]]; then return 2; fi - [[ $(sha256sum "$1" | cut -d' ' -f1) == "$2" ]] -} -for path in "$SANDBOX_BUNDLE" "$SANDBOX_VENV" "$REFERENCE_INSTALLATION" \ - "$SYNTHETIC_SOURCE_DIR" "$SYNTHETIC_OUTPUT" "$BWRAP_BIN"; do check_path "$path"; done -source_dir=$(cd -- "$SYNTHETIC_SOURCE_DIR" && pwd -P) -check_digest "$SANDBOX_VENV/bin/python" "$PYTHON_SHA256" -check_digest "$BWRAP_BIN" "$BWRAP_SHA256" -export PATH="$(dirname "$BWRAP_BIN"):/usr/bin:/bin" -[[ $(command -v bwrap) == "$BWRAP_BIN" ]] -check_digest "$source_dir/run-synthetic.sbatch" "$SYNTHETIC_SCRIPT_SHA256" -cmp "${BASH_SOURCE[0]}" "$source_dir/run-synthetic.sbatch" -check_digest "$source_dir/runtime.py" "$SYNTHETIC_RUNTIME_SHA256" -check_digest "$source_dir/synthetic_probe.py" "$SYNTHETIC_PROBE_SHA256" -check_digest "$source_dir/bounds_driver.py" "$BOUNDS_DRIVER_SHA256" -check_digest "$SANDBOX_BUNDLE/freeze.sha256" "$SANDBOX_FREEZE_SHA256" -(cd "$SANDBOX_BUNDLE" && sha256sum --check --strict freeze.sha256) -# Bind both the scheduler outcome and the repaired completion record. -check_digest "$REFERENCE_INSTALLATION/terminal-status.json" "$REFERENCE_TERMINAL_SHA256" -check_digest "$REFERENCE_INSTALLATION/slurm-final.private.txt" "$REFERENCE_SLURM_SHA256" -"$SANDBOX_VENV/bin/python" -IB - "$REFERENCE_INSTALLATION" "$REFERENCE_JOB_ID" <<'PYREF' -import json -import re -import sys -from pathlib import Path -reference = Path(sys.argv[1]) -job = sys.argv[2] -if not re.fullmatch(r"[0-9]+", job): - raise ValueError('Invalid reference job ID') -def unique_keys(pairs): - value = {} - for key, item in pairs: - if key in value: - raise ValueError('Duplicate terminal-record key') - value[key] = item - return value -terminal = json.loads((reference / 'terminal-status.json').read_bytes(), object_pairs_hook=unique_keys) -if terminal.get('completed') is not True or type(terminal.get('exit_code')) is not int: - raise ValueError('Reference terminal record has invalid types') -if terminal != {'completed': True, 'signal': 'none', 'exit_code': 0, 'job_id': job}: - raise ValueError('Reference installation did not complete normally') -slurm = (reference / 'slurm-final.private.txt').read_text() -for key, value in {'JobId': job, 'JobState': 'COMPLETED', 'ExitCode': '0:0'}.items(): - actual = re.findall(r'(? "$SYNTHETIC_OUTPUT/exit-code" - printf '{"completed":%s,"signal":"%s","exit_code":%s,"job_id":"%s"}\n' \ - "$task_completed" "$task_signal" "$status" "$SLURM_JOB_ID" > "$SYNTHETIC_OUTPUT/terminal-status.json" - exit "$status" -} -trap finish_synthetic EXIT -trap 'task_completed=false; task_signal=TERM; exit 143' TERM -trap 'task_completed=false; task_signal=INT; exit 130' INT -trap 'task_completed=false; task_signal=HUP; exit 129' HUP -# END TERMINAL RECORD -cp "$source_dir/run-synthetic.sbatch" "$source_dir/runtime.py" "$source_dir/synthetic_probe.py" "$source_dir/bounds_driver.py" "$SYNTHETIC_OUTPUT/" -cd "$root" -"$SANDBOX_VENV/bin/python" -IB "$source_dir/runtime.py" inventory --bundle "$SANDBOX_BUNDLE" \ - --output "$SYNTHETIC_OUTPUT/rootfs-before" -"$SANDBOX_VENV/bin/python" -IB - "$REFERENCE_INSTALLATION/rootfs-before/rootfs.json" \ - "$SYNTHETIC_OUTPUT/rootfs-before/rootfs.json" <<'PY' -import json -import sys -from pathlib import Path -before, current = [json.loads(Path(path).read_bytes()) for path in sys.argv[1:]] -for record in (before, current): - record.pop('rootfs_digest') - record['entries'] = [row for row in record['entries'] if row['path'] != '/opt/probe.py'] -if before != current: - raise ValueError('Installed runtime changed beyond the explicitly bound trusted probe') -PY -probe_status=0 -"$SANDBOX_VENV/bin/python" -IB "$source_dir/bounds_driver.py" probe --bundle "$SANDBOX_BUNDLE" \ - --inventory "$SYNTHETIC_OUTPUT/rootfs-before" --output "$SYNTHETIC_OUTPUT/probe" || probe_status=$? -printf '%s\n' "$probe_status" > "$SYNTHETIC_OUTPUT/probe-status" -post_status=0 -run_postcheck() { - local name=$1 status=0 - shift - "$@" || status=$? - printf '%s\t%s\n' "$name" "$status" >> "$SYNTHETIC_OUTPUT/postflight-status.tsv" - if [[ "$status" != 0 ]]; then post_status=$status; fi -} -check_frozen_files() { (cd "$SANDBOX_BUNDLE" && sha256sum --check --strict freeze.sha256); } -run_postcheck inventory "$SANDBOX_VENV/bin/python" -IB "$source_dir/runtime.py" inventory \ - --bundle "$SANDBOX_BUNDLE" --output "$SYNTHETIC_OUTPUT/rootfs-after" -run_postcheck runtime-byte-comparison cmp "$SYNTHETIC_OUTPUT/rootfs-before/rootfs.json" \ - "$SYNTHETIC_OUTPUT/rootfs-after/rootfs.json" -run_postcheck runtime-script check_digest "$source_dir/runtime.py" "$SYNTHETIC_RUNTIME_SHA256" -run_postcheck bounds-driver check_digest "$source_dir/bounds_driver.py" "$BOUNDS_DRIVER_SHA256" -run_postcheck synthetic-probe check_digest "$source_dir/synthetic_probe.py" "$SYNTHETIC_PROBE_SHA256" -run_postcheck batch-script check_digest "$source_dir/run-synthetic.sbatch" "$SYNTHETIC_SCRIPT_SHA256" -run_postcheck freeze-manifest check_digest "$SANDBOX_BUNDLE/freeze.sha256" "$SANDBOX_FREEZE_SHA256" -run_postcheck freeze-files check_frozen_files -run_postcheck python-binary check_digest "$SANDBOX_VENV/bin/python" "$PYTHON_SHA256" -run_postcheck bwrap-binary check_digest "$BWRAP_BIN" "$BWRAP_SHA256" -if [[ "$probe_status" != 0 ]]; then exit "$probe_status"; fi -if [[ "$post_status" != 0 ]]; then exit "$post_status"; fi -echo 'Trusted resource/cleanup diagnostic completed; generated-code executor remains unqualified.' -task_completed=true diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v1/runtime.py b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v1/runtime.py deleted file mode 100644 index 51f1903f3..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v1/runtime.py +++ /dev/null @@ -1,481 +0,0 @@ -"""Resolve pinned EvalPlus metadata, inventory readonly binds and run a trusted probe.""" - -import argparse -import configparser -import hashlib -import importlib.metadata -import json -import os -import re -import resource -import shutil -import signal -import subprocess -import sys -import sysconfig -import threading -import tomllib -from pathlib import Path - -REVISION = "26d6d00bb1fd0fa37f39c99d5290da67891d1c5e" - - -def canonical(value): - return json.dumps(value, sort_keys=True, separators=(",", ":"), allow_nan=False).encode() - - -def sha(raw): - return hashlib.sha256(raw).hexdigest() - - -def write(path, value): - with path.open("xb") as stream: - stream.write(canonical(value) + b"\n") - - -def git(source, *args): - return subprocess.check_output(["git", "-C", str(source), *args]) - - -def source_identity(bundle): - source = bundle / "source" - if git(source, "rev-parse", "HEAD").decode().strip() != REVISION: - raise ValueError("wrong EvalPlus source revision") - if git(source, "diff", "HEAD", "--"): - raise ValueError("tracked EvalPlus source changed") - return { - "evaluator_revision": REVISION, - "archive_sha256": sha(git(source, "archive", "--format=tar", "HEAD")), - } - - -def requirements(bundle): - identity = source_identity(bundle) - config = configparser.ConfigParser() - config.read(bundle / "source/setup.cfg") - dependencies = [ - line.strip() for line in config["options"]["install_requires"].splitlines() if line.strip() - ] - build = tomllib.loads((bundle / "source/pyproject.toml").read_text())["build-system"][ - "requires" - ] - (bundle / "dependencies.in").write_text("\n".join(dependencies) + "\n") - (bundle / "build.in").write_text("\n".join([*build, "wheel"]) + "\n") - write( - bundle / "source.json", - { - **identity, - "official_default_dependencies": dependencies, - "build_dependencies": [*build, "wheel"], - "extras_enabled": [], - "generated_code_executed": False, - }, - ) - - -def verify_source(bundle): - expected = json.loads((bundle / "source.json").read_bytes()) - if any( - source_identity(bundle)[name] != expected[name] - for name in ("evaluator_revision", "archive_sha256") - ): - raise ValueError("EvalPlus source snapshot drift") - - -def tree(root, alias): - result = [] - for path in sorted(root.rglob("*")): - relative = path.relative_to(root).as_posix() - name = alias + "/" + relative - if path.is_symlink(): - target = os.readlink(path) - if Path(target).is_absolute() or not path.resolve(strict=True).is_relative_to(root): - raise ValueError("runtime symlink escapes its readonly root") - result.append({"path": name, "type": "symlink", "target": target}) - elif path.is_dir(): - result.append({"path": name, "type": "directory", "mode": path.stat().st_mode & 0o777}) - elif path.is_file(): - result.append( - { - "path": name, - "type": "file", - "mode": path.stat().st_mode & 0o777, - "bytes": path.stat().st_size, - "sha256": sha(path.read_bytes()), - } - ) - else: - raise ValueError("unsupported runtime filesystem entry") - return result - - -def system_dependencies(roots): - # Mirror the explicit relocated loader directory; inherit no host loader state. - loader_env = { - "PATH": "/usr/bin:/bin", "LC_ALL": "C", - "LD_LIBRARY_PATH": str(roots[0] / "lib"), - } - ldd = shutil.which("ldd") - if ldd is None: - raise ValueError("system ELF dependency scanner unavailable") - dependencies = {} - for root in roots: - for path in sorted(root.rglob("*")): - if not path.is_file() or path.is_symlink(): - continue - with path.open("rb") as stream: - if stream.read(4) != b"\x7fELF": - continue - result = subprocess.run( - [ldd, str(path)], capture_output=True, timeout=10, check=False, env=loader_env - ) - output = result.stdout.decode(errors="replace") - if result.returncode != 0 or "not found" in output: - raise ValueError("ELF runtime dependency missing") - for line in output.splitlines(): - match = re.search(r"(?:=>\s*)?(/[^\s]+)", line) - if not match: - continue - original = Path(match.group(1)) - resolved = original.resolve(strict=True) - if any(resolved.is_relative_to(bound) for bound in roots): - continue - if not str(original).startswith(("/lib/", "/lib64/", "/usr/lib/", "/usr/lib64/")): - raise ValueError("unexpected external runtime library") - dependencies[str(original)] = str(resolved) - return dependencies - - -def inventory(bundle, output): - verify_source(bundle) - task_root = Path(os.environ["TASK_ROOT"]).resolve(strict=True) - base = Path(sys.base_prefix).resolve(strict=True) - site = Path(sysconfig.get_path("purelib")).resolve(strict=True) - if not base.is_relative_to(task_root) or not site.is_relative_to(task_root): - raise ValueError("runtime roots must remain inside TASK_ROOT") - executable = Path(sys._base_executable).resolve(strict=True).relative_to(base).as_posix() - bwrap = shutil.which("bwrap") - if bwrap is None: - raise ValueError("bubblewrap is unavailable on this compute node") - help_text = subprocess.check_output([bwrap, "--help"], stderr=subprocess.STDOUT).decode() - for flag in ( - "--unshare-user", - "--unshare-net", - "--unshare-pid", - "--disable-userns", - "--assert-userns-disabled", - "--clearenv", - "--remount-ro", - "--size", - ): - if flag not in help_text: - raise ValueError("required bubblewrap isolation option unavailable") - dependencies = system_dependencies((base, site)) - entries = tree(base, "/opt/python") + tree(site, "/opt/site-packages") - for destination, source in dependencies.items(): - path = Path(source) - entries.append( - { - "path": destination, - "type": "file", - "bytes": path.stat().st_size, - "sha256": sha(path.read_bytes()), - } - ) - probe = Path(__file__).with_name("synthetic_probe.py") - entries.append( - { - "path": "/opt/probe.py", - "type": "file", - "bytes": probe.stat().st_size, - "sha256": sha(probe.read_bytes()), - } - ) - entries.sort(key=lambda row: row["path"]) - # Bind-set identity describes actual readonly bytes, not an OCI image. - rootfs_digest = "sha256:" + sha(canonical(entries)) - packages = sorted( - [ - {"name": d.metadata["Name"], "version": d.version} - for d in importlib.metadata.distributions() - ], - key=lambda row: row["name"].lower(), - ) - # Match every pinned evaluator Python source byte to the installed package. - checked = 0 - source = bundle / "source" - for encoded in git(source, "ls-files", "-z", "evalplus").split(b"\0"): - if not encoded or not encoded.endswith(b".py"): - continue - name = encoded.decode() - if (source / name).read_bytes() != (site / name).read_bytes(): - raise ValueError("installed EvalPlus differs from pinned source") - checked += 1 - if checked == 0: - raise ValueError("no pinned evaluator Python sources verified") - output.mkdir(parents=True, exist_ok=False) - write( - output / "rootfs.json", - { - "schema": 1, - "rootfs_kind": "readonly-bind-set-v1", - "loader_policy": { - "scanner_inherits_environment": False, - "base_library_directory": "/opt/python/lib", - "sandbox_ld_library_path": "/opt/python/lib", - }, - "rootfs_digest": rootfs_digest, - "evaluator_revision": REVISION, - "entries": entries, - "python": sys.version.split()[0], - "packages": packages, - "checked_evaluator_sources": checked, - "executor": { - "kind": "bubblewrap", - "binary_sha256": sha(Path(bwrap).read_bytes()), - "version": subprocess.check_output([bwrap, "--version"]).decode().strip(), - }, - "generated_code_executed": False, - "qualified_generated_code_executor": False, - }, - ) - write( - output / "runtime-locators.private.json", - { - "python_root": str(base), - "site_root": str(site), - "python_relative": executable, - "bwrap": bwrap, - "system_libraries": dependencies, - }, - ) - - -def command(locators, probe, namespaces, sentinel): - args = [ - locators["bwrap"], - "--unshare-user", - "--uid", - "65534", - "--gid", - "65534", - "--unshare-net", - "--unshare-pid", - "--unshare-ipc", - "--unshare-uts", - "--hostname", - "sandbox", - "--disable-userns", - "--assert-userns-disabled", - "--cap-drop", - "ALL", - "--die-with-parent", - "--new-session", - "--clearenv", - "--ro-bind", - locators["python_root"], - "/opt/python", - "--ro-bind", - locators["site_root"], - "/opt/site-packages", - "--ro-bind", - str(probe), - "/opt/probe.py", - ] - for destination, source in sorted(locators["system_libraries"].items()): - args.extend(["--ro-bind", source, destination]) - args += [ - "--proc", - "/proc", - "--dev", - "/dev", - "--size", - "33554432", - "--tmpfs", - "/dev/shm", - "--size", - "67108864", - "--tmpfs", - "/work", - "--dir", - "/work/tmp", - "--dir", - "/work/output", - "--chdir", - "/work", - "--remount-ro", - "/", - ] - environment = { - "HOME": "/work", - "TMPDIR": "/work/tmp", - "PATH": "/opt/python/bin", - "LD_LIBRARY_PATH": "/opt/python/lib", - "PYTHONDONTWRITEBYTECODE": "1", - "OMP_NUM_THREADS": "1", - "OPENBLAS_NUM_THREADS": "1", - "LC_ALL": "C.UTF-8", - "PROBE_PARENT_NAMESPACES": json.dumps(namespaces), - "PROBE_UNMOUNTED_SENTINEL": str(sentinel), - } - for name, value in environment.items(): - args += ["--setenv", name, value] - return args + ["/opt/python/" + locators["python_relative"], "-I", "/opt/probe.py"] - - -def limits(): - os.sched_setaffinity(0, {min(os.sched_getaffinity(0))}) - for name, value in ( - ("AS", 2147483648), - ("CPU", 30), - ("NOFILE", 64), - ("FSIZE", 1048576), - ("CORE", 0), - ): - resource.setrlimit(getattr(resource, "RLIMIT_" + name), (value, value)) - - -def probe(inventory_path, output): - rootfs = json.loads((inventory_path / "rootfs.json").read_bytes()) - locators = json.loads((inventory_path / "runtime-locators.private.json").read_bytes()) - if sha(Path(locators["bwrap"]).read_bytes()) != rootfs["executor"]["binary_sha256"]: - raise ValueError("bubblewrap binary changed after inventory") - output.mkdir(parents=True, exist_ok=False) - sentinel = output / "unmounted-sentinel.private" - sentinel.write_text("synthetic host marker, not a credential") - namespaces = { - name: os.readlink("/proc/self/ns/" + name) for name in ("user", "net", "pid", "uts", "ipc") - } - args = command(locators, Path(__file__).with_name("synthetic_probe.py"), namespaces, sentinel) - timed_out = False - # Sample from the supervisor after Popen. The evaluator process remains free - # of an added observer thread before its own default multiprocessing fork. - import psutil - sampled = {"processes": 0, "threads": 0, "sum_vms_bytes": 0, - "sum_rss_bytes": 0, "sum_fds": 0, "samples": 0, "errors": 0} - known = {} - sampling_failures = [] - stop_sampling = threading.Event() - def observe(pid): - try: - parent = psutil.Process(pid) - except psutil.NoSuchProcess: - sampling_failures.append("launcher exited before sampling") - return - while not stop_sampling.is_set(): - totals = {"processes": 0, "threads": 0, "sum_vms_bytes": 0, - "sum_rss_bytes": 0, "sum_fds": 0} - try: - processes = [parent, *parent.children(recursive=True)] - except psutil.NoSuchProcess: - break - except psutil.AccessDenied: - sampling_failures.append("cannot inspect owned launcher descendants") - return - for child in processes: - try: - known[child.pid] = child.create_time() - memory = child.memory_info() - totals["processes"] += 1 - totals["threads"] += child.num_threads() - totals["sum_vms_bytes"] += memory.vms - totals["sum_rss_bytes"] += memory.rss - totals["sum_fds"] += child.num_fds() - except (psutil.NoSuchProcess, psutil.AccessDenied): - sampled["errors"] += 1 - for name, value in totals.items(): - sampled[name] = max(sampled[name], value) - sampled["samples"] += 1 - stop_sampling.wait(0.01) - with ( - (output / "stdout.json").open("xb") as stdout, - (output / "stderr.private.log").open("xb") as stderr, - ): - process = subprocess.Popen( - [sys.executable, str(Path(__file__).resolve()), "_exec", json.dumps(args)], - stdin=subprocess.DEVNULL, - stdout=stdout, - stderr=stderr, - start_new_session=True, - env={**os.environ, "PROBE_INHERITED_SECRET": "synthetic-must-be-cleared"}, - ) - observer = threading.Thread(target=observe, args=(process.pid,), daemon=True) - observer.start() - try: - code = process.wait(timeout=60) - except subprocess.TimeoutExpired: - timed_out = True - if os.getpgid(process.pid) != process.pid: - raise RuntimeError("owned sandbox process group changed unexpectedly") - os.killpg(process.pid, signal.SIGKILL) - code = process.wait(timeout=10) - finally: - stop_sampling.set() - observer.join(timeout=2) - remaining = [] - for pid, created in known.items(): - try: - child = psutil.Process(pid) - if child.create_time() == created and child.status() != psutil.STATUS_ZOMBIE: - remaining.append(pid) - except psutil.NoSuchProcess: - pass - result = { - "sampled_peaks": sampled, - "sampling_thread_joined": not observer.is_alive(), - "sampling_failures": sampling_failures, - "known_live_descendants_after_exit": remaining, - "sampling_limitation": "Polling may miss transient processes; sampled peaks are not hard bounds or exhaustive cleanup proof.", - "phase": "trusted_synthetic_evaluator", - "exit_code": code, - "timed_out": timed_out, - "rootfs_digest": rootfs["rootfs_digest"], - "executor": rootfs["executor"], - "policy_script_sha256": sha(Path(__file__).read_bytes()), - "probe_script_sha256": sha(Path(__file__).with_name("synthetic_probe.py").read_bytes()), - "stdout_sha256": sha((output / "stdout.json").read_bytes()), - "generated_code_executed": False, - "full_evaluator_attempt_requested": True, - "qualified_generated_code_executor": False, - "verdict": "INCONCLUSIVE", - } - write(output / "execution.json", result) - if code != 0 or timed_out or remaining or observer.is_alive() or sampling_failures or not sampled["samples"]: - raise ValueError("trusted synthetic evaluator probe failed; retain evidence without fallback") - payload = json.loads((output / "stdout.json").read_bytes()) - if payload.get("passed") is not True: - raise ValueError("trusted synthetic evaluator assertions failed") - - -def main(): - parser = argparse.ArgumentParser(description=__doc__) - parser.add_argument("phase", choices=("requirements", "verify-source", "inventory", "probe")) - parser.add_argument("--bundle", type=Path, required=True) - parser.add_argument("--output", type=Path) - parser.add_argument("--inventory", type=Path) - args = parser.parse_args() - if not os.environ.get("SLURM_JOB_ID") or os.environ.get("SLURM_GPUS_ON_NODE", "0") != "0": - raise ValueError("CPU-only Slurm allocation required") - if args.phase == "requirements": - requirements(args.bundle) - elif args.phase == "verify-source": - verify_source(args.bundle) - elif args.phase == "inventory": - inventory(args.bundle, args.output) - else: - probe(args.inventory, args.output) - - -if __name__ == "__main__": - if len(sys.argv) == 3 and sys.argv[1] == "_exec": - # A fresh interpreter sets limits before exec; no Python preexec callback - # runs between fork and exec in the monitoring supervisor. - if not os.environ.get("SLURM_JOB_ID") or os.environ.get("SLURM_GPUS_ON_NODE", "0") != "0": - raise ValueError("CPU-only Slurm allocation required") - command_args = json.loads(sys.argv[2]) - if not command_args or any(not isinstance(value, str) for value in command_args): - raise ValueError("invalid internal command") - limits() - os.execv(command_args[0], command_args) - else: - main() diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v1/synthetic_probe.py b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v1/synthetic_probe.py deleted file mode 100644 index 5325345fc..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v1/synthetic_probe.py +++ /dev/null @@ -1,450 +0,0 @@ -"""Finite handwritten kernel-boundary probes; no model programs or task data.""" - -import ctypes -import errno -import json -import mmap -import os -import platform -import resource -import signal -import subprocess -import sys -import threading -import time -from pathlib import Path - -TASK_LIMIT = 16 -MEMORY_LIMIT = 2147483648 -IPC_SYSCALLS = (29, 30, 31, 64, 65, 66, 67, 68, 69, 70, 71, 220, 240) - - -def filter_instructions(): - # Linux v6.12 x86 syscall table; reject other architectures and the x32 ABI. - result = [ - (0x20, 0, 0, 4), - (0x15, 1, 0, 0xC000003E), - (0x06, 0, 0, 0x80000000), - (0x20, 0, 0, 0), - (0x35, 0, 1, 0x40000000), - (0x06, 0, 0, 0x80000000), - ] - for number in IPC_SYSCALLS: - result.extend([(0x15, 0, 1, number), (0x06, 0, 0, 0x00050000 | errno.EPERM)]) - return [*result, (0x06, 0, 0, 0x7FFF0000)] - - -def install_ipc_filter(): - if platform.machine() != "x86_64" or sys.maxsize != 2**63 - 1: - raise ValueError("trusted filter requires Linux x86_64 LP64") - - class Instruction(ctypes.Structure): - _fields_ = [ - ("code", ctypes.c_ushort), - ("jt", ctypes.c_ubyte), - ("jf", ctypes.c_ubyte), - ("k", ctypes.c_uint32), - ] - - class Program(ctypes.Structure): - _fields_ = [("length", ctypes.c_ushort), ("instructions", ctypes.POINTER(Instruction))] - - instructions = filter_instructions() - array = (Instruction * len(instructions))(*(Instruction(*row) for row in instructions)) - program = Program(len(instructions), array) - libc = ctypes.CDLL(None, use_errno=True) - libc.prctl.restype = ctypes.c_int - if libc.prctl(38, 1, 0, 0, 0) != 0 or libc.prctl(22, 2, ctypes.byref(program), 0, 0) != 0: - raise OSError(ctypes.get_errno(), "fixed seccomp policy could not be installed") - - -def ipc_attempts(): - libc = ctypes.CDLL(None, use_errno=True) - libc.syscall.restype = ctypes.c_long - attempts = [] - # Finite minimal allocations only, even if the expected rejection is absent. - for number, arguments in ((29, (0, 4096, 0o600)), (64, (0, 1, 0o600)), (68, (0, 0o600))): - ctypes.set_errno(0) - value = libc.syscall(ctypes.c_long(number), *(ctypes.c_long(v) for v in arguments)) - attempts.append({"syscall": number, "return": value, "errno": ctypes.get_errno()}) - ctypes.set_errno(0) - value = libc.syscall( - ctypes.c_long(240), - ctypes.c_char_p(b"trusted-probe"), - ctypes.c_long(os.O_CREAT | os.O_WRONLY), - ctypes.c_long(0o600), - ctypes.c_void_p(), - ) - attempts.append({"syscall": 240, "return": value, "errno": ctypes.get_errno()}) - return attempts - - -def emit(value): - print(json.dumps(value, sort_keys=True), flush=True) - - -def limits_record(): - return { - name: list(resource.getrlimit(getattr(resource, "RLIMIT_" + name))) - for name in ("AS", "CPU", "NOFILE", "FSIZE", "CORE", "NPROC", "MEMLOCK", "MSGQUEUE") - } - - -def finite_forks(): - children = [] - read_fd, write_fd = os.pipe() - rejected = None - try: - for _ in range(TASK_LIMIT + 1): - try: - child = os.fork() - except OSError as error: - rejected = error.errno - break - if child == 0: - os.close(write_fd) - signal.alarm(8) - os.read(read_fd, 1) - os._exit(0) - children.append(child) - finally: - os.close(write_fd) - os.close(read_fd) - for child in children: - os.waitpid(child, 0) - return {"created": len(children), "rejection_errno": rejected, "attempt_cap": TASK_LIMIT + 1} - - -def finite_threads(): - threads = [] - stop = threading.Event() - rejected = None - try: - for _ in range(TASK_LIMIT + 1): - thread = threading.Thread(target=stop.wait, args=(8,)) - try: - thread.start() - except RuntimeError as error: - rejected = str(error) - break - threads.append(thread) - finally: - stop.set() - for thread in threads: - thread.join(timeout=2) - return { - "created": len(threads), - "rejection": rejected, - "all_joined": all(not thread.is_alive() for thread in threads), - "attempt_cap": TASK_LIMIT + 1, - } - - -def bounds(): - before = limits_record() - raises = {} - for name in ("NPROC", "AS", "CPU", "NOFILE", "FSIZE", "CORE", "MEMLOCK", "MSGQUEUE"): - code = getattr(resource, "RLIMIT_" + name) - soft, hard = resource.getrlimit(code) - try: - resource.setrlimit(code, (soft, hard + 1)) - except (ValueError, PermissionError) as error: - raises[name] = type(error).__name__ - else: - raises[name] = "unexpected_success" - try: - mapping = mmap.mmap(-1, MEMORY_LIMIT + mmap.PAGESIZE) - except OSError as error: - as_errno = error.errno - else: - mapping.close() - as_errno = None - inherited = subprocess.run( - [sys.executable, "-I", "-B", "/opt/probe.py", "inherit"], - capture_output=True, - timeout=5, - check=False, - ) - inherited_payload = json.loads(inherited.stdout) - forks = finite_forks() - threads = finite_threads() - descriptors = [] - fd_errno = None - try: - for _ in range(65): - try: - descriptors.append(os.open("/dev/null", os.O_RDONLY)) - except OSError as error: - fd_errno = error.errno - break - finally: - for fd in descriptors: - os.close(fd) - old_handler = signal.signal(signal.SIGXFSZ, signal.SIG_IGN) - try: - with open("/work/size-check", "wb") as stream: - try: - stream.truncate(1048577) - except OSError as error: - size_errno = error.errno - else: - size_errno = None - finally: - signal.signal(signal.SIGXFSZ, old_handler) - cpu = subprocess.run( - [sys.executable, "-I", "-B", "/opt/probe.py", "cpu"], - capture_output=True, - timeout=5, - check=False, - ) - status = dict( - line.split(":", 1) - for line in Path("/proc/self/status").read_text().splitlines() - if ":" in line - ) - expected = { - "AS": [MEMORY_LIMIT] * 2, - "CPU": [30] * 2, - "NOFILE": [64] * 2, - "FSIZE": [1048576] * 2, - "CORE": [0, 0], - "NPROC": [TASK_LIMIT] * 2, - "MEMLOCK": [0, 0], - "MSGQUEUE": [0, 0], - } - ipc = ipc_attempts() - try: - with open("/dev/creation-check", "xb"): - pass - except OSError as error: - dev_errno = error.errno - else: - dev_errno = None - with open("/dev/zero", "rb") as stream: - zero = stream.read(4) - checks = { - "exact_limits": before == expected and limits_record() == expected, - "nonroot": os.getuid() == os.geteuid() == 65534, - "no_effective_capabilities": int(status["CapEff"].strip(), 16) == 0, - "no_new_privileges": status["NoNewPrivs"].strip() == "1", - "seccomp_filter_present": status["Seccomp"].strip() == "2", - "sysv_creation_denied": all( - row["return"] == -1 and row["errno"] == errno.EPERM for row in ipc - ), - "device_mount_readonly": dev_errno == errno.EROFS, - "device_read_available": zero == b"\x00" * 4, - "raise_hard_limit_denied": all(value != "unexpected_success" for value in raises.values()), - "address_space_denied": as_errno == errno.ENOMEM, - "fork_exec_inheritance": inherited.returncode == 0 - and inherited_payload["limits"] == expected, - "fork_exec_filter_inheritance": inherited.returncode == 0 - and all( - row["return"] == -1 and row["errno"] == errno.EPERM for row in inherited_payload["ipc"] - ), - "bounded_fork_denied": 0 < forks["created"] < TASK_LIMIT - and forks["rejection_errno"] == errno.EAGAIN, - "bounded_thread_denied": 0 < threads["created"] < TASK_LIMIT - and threads["rejection"] is not None - and threads["all_joined"], - "descriptor_limit_denied": fd_errno == errno.EMFILE, - "file_size_denied": size_errno == errno.EFBIG, - "cpu_limit_kills": cpu.returncode == -signal.SIGKILL, - } - emit( - { - "mode": "bounds", - "checks": checks, - "passed": all(checks.values()), - "limits": before, - "hard_limit_raise_results": raises, - "as_errno": as_errno, - "forks": forks, - "threads": threads, - "descriptor_errno": fd_errno, - "opened_descriptors": len(descriptors), - "file_size_errno": size_errno, - "cpu_exit_code": cpu.returncode, - "ipc_attempts": ipc, - "device_creation_errno": dev_errno, - "inherited_exit_code": inherited.returncode, - "generated_code_executed": False, - } - ) - return 0 if all(checks.values()) else 2 - - -def cleanup(mode): - read_fd, write_fd = os.pipe() - direct = os.fork() - if direct == 0: - os.close(read_fd) - os.write(write_fd, f"{os.getpid()}\n".encode()) - os.close(write_fd) - time.sleep(15) - os._exit(0) - intermediate = os.fork() - if intermediate == 0: - os.close(read_fd) - os.setsid() - grandchild = os.fork() - if grandchild == 0: - os.write(write_fd, f"{os.getpid()}\n".encode()) - os.close(write_fd) - time.sleep(15) - os._exit(0) - os._exit(0) - os.close(write_fd) - with os.fdopen(read_fd, "r") as stream: - live = sorted(int(line) for line in stream) - os.waitpid(intermediate, 0) - emit( - { - "mode": mode, - "ready": True, - "probe_pid": os.getpid(), - "live_descendant_pids": live, - "direct_pid": direct, - "namespace_pid": os.readlink("/proc/self/ns/pid"), - "generated_code_executed": False, - } - ) - if mode == "normal": - # Allow the supervisor to open pidfds for both deliberately live children. - time.sleep(2) - return 0 - time.sleep(15) - return 3 - - -def synthetic_evaluator(): - import base64 - import contextlib - import hashlib - import importlib - import io - - sys.path.insert(0, "/opt/site-packages") - import psutil - - def digest(raw): - return hashlib.sha256(raw).hexdigest() - - os.environ["EVALPLUS_MAX_MEMORY_BYTES"] = str(MEMORY_LIMIT) - os.environ.pop("EVALPLUS_TIMEOUT_PER_TASK", None) - os.environ["XDG_CACHE_HOME"] = "/work/cache" - os.environ["HUMANEVAL_OVERRIDE_PATH"] = "/work/synthetic-dataset.jsonl" - work = Path("/work") - samples = work / "synthetic-samples.jsonl" - dataset = work / "synthetic-dataset.jsonl" - output = work / "output/synthetic-results.json" - assert not samples.exists() and not dataset.exists() and not output.exists() - assert not (work / "cache").exists() - solutions = { - "Synthetic/correct": "def f(x):\n return x + 1\n", - "Synthetic/plus_wrong": "def f(x):\n return 1\n", - "Synthetic/exception": "def f(x):\n raise ArithmeticError('synthetic')\n", - "Synthetic/input_timeout": "import time\ndef f(x):\n time.sleep(5)\n return x + 1\n", - } - rows = [ - { - "task_id": task_id, - "prompt": "def f(x):\n", - "canonical_solution": " return x + 1\n", - "contract": "", - "entry_point": "f", - "base_input": [[0]], - "plus_input": [[1]], - "atol": 0, - } - for task_id in solutions - ] - dataset.write_text("".join(json.dumps(row, sort_keys=True) + "\n" for row in rows)) - samples.write_text( - "".join( - json.dumps({"task_id": key, "solution": value}, sort_keys=True) + "\n" - for key, value in solutions.items() - ) - ) - parent = psutil.Process() - captured_out, captured_err = io.StringIO(), io.StringIO() - started = time.monotonic() - with contextlib.redirect_stdout(captured_out), contextlib.redirect_stderr(captured_err): - evaluator = importlib.import_module("evalplus.evaluate") - evaluator.evaluate( - dataset="humaneval", - samples=str(samples), - output_file=str(output), - parallel=1, - base_only=False, - mini=False, - noextreme=False, - test_details=True, - ) - result = json.loads(output.read_bytes()) - statuses = { - task_id: [[row["base_status"], row["plus_status"]] for row in values] - for task_id, values in result["eval"].items() - } - # The upstream per-input exception handler reports time-limit exceptions as - # fail. This is distinct from its outer child-process timeout classification. - expected = { - "Synthetic/correct": [["pass", "pass"]], - "Synthetic/plus_wrong": [["pass", "fail"]], - "Synthetic/exception": [["fail", "fail"]], - "Synthetic/input_timeout": [["fail", "fail"]], - } - checks = { - "four_exact_status_pairs": statuses == expected, - "dataset_hash": result["hash"] == hashlib.md5(dataset.read_bytes()).hexdigest(), - "bounded_output": output.stat().st_size <= 1048576, - "timeout_override_absent": "EVALPLUS_TIMEOUT_PER_TASK" not in os.environ, - } - payload = { - "checks": checks, - "passed": all(checks.values()), - "statuses": statuses, - "elapsed_seconds": time.monotonic() - started, - "dataset_sha256": digest(dataset.read_bytes()), - "samples_sha256": digest(samples.read_bytes()), - "results_sha256": digest(output.read_bytes()), - "results": result, - "results_raw_base64": base64.b64encode(output.read_bytes()).decode("ascii"), - "evaluator_module_sha256": digest(Path(evaluator.__file__).read_bytes()), - "evaluator_stdout": captured_out.getvalue(), - "evaluator_stderr": captured_err.getvalue(), - "live_children_after_evaluate": len(parent.children(recursive=True)), - "generated_code_executed": False, - "real_benchmark_data_mounted": False, - "full_evaluator_executed": True, - "qualified_generated_code_executor": False, - } - print(json.dumps(payload, sort_keys=True)) - return 0 if payload["passed"] else 2 - - -def main(): - mode = sys.argv[1] - if mode == "inherit": - emit({"limits": limits_record(), "ipc": ipc_attempts()}) - return 0 - if mode == "cpu": - resource.setrlimit(resource.RLIMIT_CPU, (1, 1)) - signal.alarm(4) - while True: - pass - resource.setrlimit(resource.RLIMIT_NPROC, (TASK_LIMIT, TASK_LIMIT)) - resource.setrlimit(resource.RLIMIT_MEMLOCK, (0, 0)) - resource.setrlimit(resource.RLIMIT_MSGQUEUE, (0, 0)) - install_ipc_filter() - signal.alarm(40) - if mode == "synthetic": - return synthetic_evaluator() - if mode == "bounds": - return bounds() - if mode in ("normal", "timeout", "launcher_kill"): - return cleanup(mode) - raise ValueError("unknown trusted probe mode") - - -if __name__ == "__main__": - raise SystemExit(main()) diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v1/test_benign_reference.py b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v1/test_benign_reference.py deleted file mode 100644 index cbdc89c08..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v1/test_benign_reference.py +++ /dev/null @@ -1,65 +0,0 @@ -"""Check the exact embedded reference validator with observed and mutated receipts.""" -import copy -import hashlib -import json -import subprocess -import sys -import tempfile -from pathlib import Path - -root = Path(__file__).parent -wrapper = (root / 'run-synthetic.sbatch').read_text() -code = wrapper.split("<<'PYBENIGN'\n", 1)[1].split('\nPYBENIGN', 1)[0] -reference = root / 'reference.private' -original_execution = json.loads((reference / 'probe/execution.json').read_bytes()) -original_stdout = json.loads((reference / 'probe/stdout.json').read_bytes()) -rootfs = json.loads((reference / 'rootfs-before/rootfs.json').read_bytes()) -bwrap = rootfs['executor']['binary_sha256'] -cases = ['observed_success', 'exit_bool', 'exit_nonzero', 'timeout', 'wrong_phase', - 'wrong_output_digest', 'wrong_rootfs', 'wrong_executor', 'empty_checks', - 'missing_check', 'integer_check', 'integer_passed', 'duplicate_key'] -with tempfile.TemporaryDirectory(dir=root) as temporary: - fixture = Path(temporary) - (fixture / 'probe').mkdir() - (fixture / 'rootfs-before').mkdir() - reference_rootfs = (reference / 'rootfs-before/rootfs.json').read_bytes() - (fixture / 'rootfs-before/rootfs.json').write_bytes(reference_rootfs) - for name in cases: - execution, stdout = copy.deepcopy(original_execution), copy.deepcopy(original_stdout) - if name == 'exit_bool': - execution['exit_code'] = False - elif name == 'exit_nonzero': - execution['exit_code'] = 1 - elif name == 'timeout': - execution['timed_out'] = True - elif name == 'wrong_phase': - execution['phase'] = 'unrelated' - elif name == 'wrong_rootfs': - execution['rootfs_digest'] = 'sha256:' + '0' * 64 - elif name == 'wrong_executor': - execution['executor']['binary_sha256'] = '0' * 64 - elif name == 'empty_checks': - stdout['checks'] = {} - elif name == 'missing_check': - del stdout['checks']['new_namespaces'] - elif name == 'integer_check': - stdout['checks']['new_namespaces'] = 1 - elif name == 'integer_passed': - stdout['passed'] = 1 - encoded = json.dumps(stdout).encode() - execution['stdout_sha256'] = hashlib.sha256(encoded).hexdigest() - if name == 'wrong_output_digest': - execution['stdout_sha256'] = '0' * 64 - encoded_execution = json.dumps(execution) - if name == 'duplicate_key': - encoded_execution = '{"exit_code":1,' + encoded_execution[1:] - (fixture / 'probe/execution.json').write_text(encoded_execution) - (fixture / 'probe/stdout.json').write_bytes(encoded) - result = subprocess.run([sys.executable, '-IBO', '-c', code, str(fixture), bwrap], - capture_output=True, check=False, timeout=5) - if (result.returncode == 0) != (name == 'observed_success'): - raise RuntimeError('Unexpected reference-validation outcome: ' + name) -report = {'benign_reference_cases': len(cases), 'cases': cases, 'optimized_python': True, - 'evaluator_or_sandbox_executed': False, 'passed': True} -(root / 'benign-reference-controls.json').write_text(json.dumps(report, indent=2) + '\n') -print(json.dumps(report)) diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v1/test_controls.py b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v1/test_controls.py deleted file mode 100644 index 3cfb03376..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v1/test_controls.py +++ /dev/null @@ -1,247 +0,0 @@ -"""Local controls exercise decision logic only; no Linux sandbox is executed.""" - -import ast -import errno -import json -import os -import subprocess -import sys -import tempfile -import time -import types -from pathlib import Path -from unittest.mock import Mock, patch - -ROOT = Path(__file__).resolve().parent - - -def require(value, message): - if not value: - raise ValueError(message) - - -def main(): - # Load just the filter constructor, never the finite process/IPC probes. - module = ast.parse((ROOT / "synthetic_probe.py").read_text()) - selected = [ - node - for node in module.body - if (isinstance(node, ast.FunctionDef) and node.name == "filter_instructions") - or ( - isinstance(node, ast.Assign) - and any( - isinstance(target, ast.Name) and target.id == "IPC_SYSCALLS" - for target in node.targets - ) - ) - ] - namespace = {"errno": errno} - exec(compile(ast.Module(body=selected, type_ignores=[]), "filter-control", "exec"), namespace) - program = namespace["filter_instructions"]() - - def evaluate(architecture, number): - accumulator = 0 - pc = 0 - while pc < len(program): - opcode, yes, no, value = program[pc] - if opcode == 0x20: - accumulator = {0: number, 4: architecture}[value] - elif opcode == 0x15: - pc += yes if accumulator == value else no - elif opcode == 0x35: - pc += yes if accumulator >= value else no - elif opcode == 0x06: - return value - else: - raise ValueError("unexpected BPF opcode") - pc += 1 - raise ValueError("filter failed to return") - - expected_denied = {29, 30, 31, 64, 65, 66, 67, 68, 69, 70, 71, 220, 240} - controls = [] - for number in range(451): - expected = 0x50000 | errno.EPERM if number in expected_denied else 0x7FFF0000 - require(evaluate(0xC000003E, number) == expected, f"syscall {number}") - controls.append("all native syscall decisions 0 through 450") - for architecture, number in ( - (0x40000003, 117), - (0xC00000B7, 29), - (0, 0), - (0xC000003E, 0x40000000), - (0xC000003E, 0x4000001D), - ): - require(evaluate(architecture, number) == 0x80000000, "ABI mismatch was not killed") - controls.append("three other architectures and two x32 calls denied") - - # psutil is intentionally not installed in this local tooling environment. - # These controls never use it; the registered cluster runtime already has it. - with patch.dict(sys.modules, {"psutil": types.ModuleType("psutil")}): - sys.path.insert(0, str(ROOT)) - import bounds_driver as driver - - for pairs in ((("x", 1), ("x", 2)),): - try: - driver.unique(pairs) - except ValueError: - pass - else: - raise ValueError("duplicate JSON keys accepted") - controls.append("duplicate status keys rejected") - fields = {"PPid": "10", "NSpid": "20 1"} - record = {"child-pid": 20, "pid-namespace": 99} - with ( - patch.object(os, "pidfd_open", create=True, return_value=77), - patch.object(os, "close"), - patch.object(os, "readlink", return_value="pid:[99]"), - patch.object(driver, "status", return_value=fields), - patch.object(driver, "exited", return_value=False), - ): - held, identity = driver.bind_init(record, 10) - require(held == 77 and identity["namespace_pids"] == [20, 1], "valid init rejected") - controls.append("valid namespace-init identity accepted") - cases = ( - ({"child-pid": True, "pid-namespace": 99}, fields, False, "pid:[99]"), - (record, {"PPid": "11", "NSpid": "20 1"}, False, "pid:[99]"), - (record, {"PPid": "10", "NSpid": "20 2"}, False, "pid:[99]"), - (record, fields, True, "pid:[99]"), - (record, fields, False, "pid:[100]"), - ) - for value, observed, dead, link in cases: - with ( - patch.object(os, "pidfd_open", create=True, return_value=77), - patch.object(os, "close"), - patch.object(os, "readlink", return_value=link), - patch.object(driver, "status", return_value=observed), - patch.object(driver, "exited", return_value=dead), - ): - try: - driver.bind_init(value, 10) - except ValueError: - pass - else: - raise ValueError("invalid namespace-init identity accepted") - controls.append("five invalid init identities rejected") - with ( - patch.object(os, "pidfd_open", create=True, return_value=77), - patch.object(os, "close") as closed, - patch.object(os, "readlink", return_value="pid:[99]"), - patch.object(driver, "status", return_value=fields), - patch.object(driver, "exited", side_effect=[False, True]), - ): - try: - driver.bind_init(record, 10) - except ValueError: - closed.assert_called_once_with(77) - else: - raise ValueError("init exit during binding accepted") - controls.append("exit during identity binding rejected and pidfd closed") - for end in ("eof", "timeout"): - read_fd, write_fd = os.pipe() - raw = bytearray() - try: - os.write(write_fd, b'{"child-') - if end == "eof": - os.close(write_fd) - write_fd = -1 - try: - driver.read_line(read_fd, time.monotonic() + 0.05, raw) - except ValueError: - require(raw == b'{"child-', "partial status bytes lost") - else: - raise ValueError("incomplete status line accepted") - finally: - os.close(read_fd) - if write_fd >= 0: - os.close(write_fd) - controls.append("partial status retained on EOF and deadline") - for scenario in ("exit_race", "wait_timeout", "stdout_failure"): - with tempfile.TemporaryDirectory(dir=ROOT) as temporary: - output = Path(temporary) - (output / "stdout.json").write_bytes(b'{"partial":true}\n') - report = {"passed": False, "error": {"type": "OriginalProbeFailure"}} - process = Mock() - process.returncode = None if scenario == "wait_timeout" else 0 - process.poll.return_value = process.returncode - process.wait.side_effect = subprocess.TimeoutExpired("trusted", 5) - with ( - patch.object(driver, "exited", side_effect=[False, True, True]) as observed, - patch.object( - driver.signal, "pidfd_send_signal", create=True, side_effect=ProcessLookupError - ), - patch.object(os, "close") as closed, - ): - if scenario != "exit_race": - observed.side_effect = None - observed.return_value = True - if scenario == "stdout_failure": - with patch.object( - Path, "read_bytes", side_effect=OSError("synthetic read failure") - ): - driver.finalize_case( - report, - process, - 77, - [], - [80, -1], - output, - bytearray(b"partial-status"), - {"rootfs_digest": "bound"}, - ) - else: - driver.finalize_case( - report, - process, - 77, - [], - [80, -1], - output, - bytearray(b"partial-status"), - {"rootfs_digest": "bound"}, - ) - require(closed.call_count == 2, "later descriptors were not closed") - actual = json.loads((output / "report.json").read_text()) - require( - actual["error"] == report["error"] and not actual["passed"], "original failure lost" - ) - require( - (output / "status.private.jsonl").read_bytes() == b"partial-status", "status lost" - ) - errors = actual["cleanup_or_evidence_errors"] - if scenario == "exit_race": - require( - actual["signal_race_already_exited"] and not errors, "exit race not confirmed" - ) - elif scenario == "wait_timeout": - require(any(row["type"] == "TimeoutExpired" for row in errors), "wait failure lost") - else: - require( - actual["stdout_sha256"] is None - and any(row["operation"] == "read_stdout" for row in errors), - "read failure lost", - ) - controls.append( - "cleanup exit race, wait timeout and stdout failure retain original errors and records" - ) - # Syntax compilation checks all sources without executing platform probes. - for name in ("runtime.py", "bounds_driver.py", "synthetic_probe.py"): - compile((ROOT / name).read_text(), name, "exec") - controls.append("three runtime sources compile") - isolated = subprocess.run( - [ - sys.executable, - "-IB", - "-c", - "import runpy,sys,types; sys.modules['psutil']=types.ModuleType('psutil'); " - "runpy.run_path(sys.argv[1], run_name='trusted_import_control')", - str(ROOT / "bounds_driver.py"), - ], - capture_output=True, - timeout=10, - ) - require(isolated.returncode == 0, isolated.stderr.decode()) - controls.append("isolated interpreter loads exact adjacent runtime; psutil import stubbed") - print(json.dumps({"passed": True, "controls": controls, "linux_runtime_executed": False})) - - -if __name__ == "__main__": - main() diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v1/test_reference_completion.py b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v1/test_reference_completion.py deleted file mode 100644 index 8e29fcc2e..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v1/test_reference_completion.py +++ /dev/null @@ -1,54 +0,0 @@ -"""Reference-completion checks only; no evaluator import or runtime execution.""" - -import json -import subprocess -import sys -import tempfile -from pathlib import Path - - -def main(): - root = Path(__file__).parent - script = (root / "run-synthetic.sbatch").read_text() - code = script.split("<<'PYREF'\n", 1)[1].split("\nPYREF", 1)[0] - good = {"completed": True, "signal": "none", "exit_code": 0, "job_id": "42"} - slurm = "JobId=42 JobState=COMPLETED ExitCode=0:0\n" - cases = [ - (good, slurm, "0", True), - ({**good, "completed": False}, slurm, "0", False), - ({**good, "completed": 1}, slurm, "0", False), - ({**good, "signal": "TERM", "exit_code": 143}, slurm, "143", False), - ({**good, "exit_code": False}, slurm, "0", False), - ({**good, "job_id": "41"}, slurm, "0", False), - (good, slurm.replace("COMPLETED", "TIMEOUT"), "0", False), - (good, slurm.replace("0:0", "0:15"), "0", False), - (good, slurm.replace("JobId=42", "JobId=41"), "0", False), - (good, slurm + "JobState=COMPLETED\n", "0", False), - (good, slurm + "JobState=\n", "0", False), - (good, slurm + "JobId=\n", "0", False), - (good, slurm + "ExitCode=\n", "0", False), - (good, slurm, "1", False), - (None, slurm, "0", False), - ('{"completed":false,"completed":true,"signal":"none","exit_code":0,"job_id":"42"}', slurm, "0", False), - ('{"completed":true,"signal":"none","exit_code":143,"exit_code":0,"job_id":"42"}', slurm, "0", False), - ('{"completed":true,"signal":"none","exit_code":0,"job_id":"41","job_id":"42"}', slurm, "0", False), - ] - with tempfile.TemporaryDirectory(prefix="reference-test-", dir=root) as temporary: - reference = Path(temporary) - for terminal, scheduler, marker, expected in cases: - path = reference / "terminal-status.json" - if terminal is None: - path.unlink(missing_ok=True) - else: - path.write_text(terminal if isinstance(terminal, str) else json.dumps(terminal)) - (reference / "slurm-final.private.txt").write_text(scheduler) - (reference / "exit-code").write_text(marker) - result = subprocess.run([sys.executable, "-IBO", "-c", code, str(reference), "42"], - capture_output=True, check=False, timeout=5) - if (result.returncode == 0) != expected: - raise RuntimeError("Reference completion check mismatch") - print(json.dumps({"reference_completion_cases": len(cases), "optimized_python": True})) - - -if __name__ == "__main__": - main() diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v2/benign-reference-controls.json b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v2/benign-reference-controls.json deleted file mode 100644 index 2a5d0061d..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v2/benign-reference-controls.json +++ /dev/null @@ -1,21 +0,0 @@ -{ - "benign_reference_cases": 13, - "cases": [ - "observed_success", - "exit_bool", - "exit_nonzero", - "timeout", - "wrong_phase", - "wrong_output_digest", - "wrong_rootfs", - "wrong_executor", - "empty_checks", - "missing_check", - "integer_check", - "integer_passed", - "duplicate_key" - ], - "optimized_python": true, - "evaluator_or_sandbox_executed": false, - "passed": true -} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v2/bounds_driver.py b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v2/bounds_driver.py deleted file mode 100644 index 66061a0aa..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v2/bounds_driver.py +++ /dev/null @@ -1,613 +0,0 @@ -"""Supervise only fixed trusted probes and retain actual namespace-init exit evidence.""" - -import argparse -import ctypes -import hashlib -import importlib.util -import json -import os -import platform -import select -import signal -import subprocess -import sys -import time -from pathlib import Path - -import psutil - -RUNTIME_PATH = Path(__file__).with_name("runtime.py") -RUNTIME_SHA256 = "997cbfed8398e8ad62d1beb4f6b4bc6e61a0f4572b9156cfd0c6f3fb52b03233" -if hashlib.sha256(RUNTIME_PATH.read_bytes()).hexdigest() != RUNTIME_SHA256: - raise ValueError("unexpected runtime source bytes") -runtime_spec = importlib.util.spec_from_file_location( - "trusted_bound_runtime", RUNTIME_PATH -) -runtime = importlib.util.module_from_spec(runtime_spec) -runtime_spec.loader.exec_module(runtime) -START_TOKEN = b"inferrsi-trusted-start-v2\n" -AUTHORIZED_MARKER = b"PAYLOAD_START_AUTHORIZED\n" -REJECTED_MARKER = b"PAYLOAD_START_REJECTED\n" -MODES = ( - "gate_valid", - "gate_eof", - "gate_short", - "gate_wrong", - "gate_timeout", - "gate_early_launcher_death", - "bounds", - "synthetic", - "normal", - "timeout", - "launcher_kill", -) - - -def pidfd_syscall(number, *arguments): - if ( - sys.platform != "linux" - or platform.machine() != "x86_64" - or ctypes.sizeof(ctypes.c_void_p) != 8 - or ctypes.sizeof(ctypes.c_long) != 8 - ): - raise ValueError("pidfd bindings require Linux x86_64 LP64") - library = ctypes.CDLL(None, use_errno=True) - library.syscall.restype = ctypes.c_long - ctypes.set_errno(0) - result = library.syscall(ctypes.c_long(number), *arguments) - if result == -1: - error = ctypes.get_errno() - raise OSError(error, os.strerror(error)) - return result - - -def pidfd_open(pid): - if type(pid) is not int or not 0 < pid < 2**31: - raise ValueError("invalid positive PID") - return pidfd_syscall(434, ctypes.c_int(pid), ctypes.c_uint(0)) - - -def pidfd_send_signal(fd, value): - if type(fd) is not int or fd < 0 or type(value) is not int or value < 0: - raise ValueError("invalid pidfd or signal") - result = pidfd_syscall( - 424, ctypes.c_int(fd), ctypes.c_int(value), ctypes.c_void_p(), ctypes.c_uint(0) - ) - if result != 0: - raise ValueError("unexpected pidfd_send_signal result") - - -def capability(output): - report = { - "schema": 1, - "passed": False, - "job_id": os.environ["SLURM_JOB_ID"], - "implementation": "linux-x86_64-lp64-raw-syscalls-434-and-424", - "python_exposes_pidfd_open": hasattr(os, "pidfd_open"), - "python_exposes_pidfd_send_signal": hasattr(signal, "pidfd_send_signal"), - "driver_sha256": runtime.sha(Path(__file__).read_bytes()), - "python_binary_sha256": runtime.sha(Path(sys._base_executable).read_bytes()), - "kernel_release": platform.release(), - "generated_code_executed": False, - } - held = [] - child = None - errors = [] - try: - own_fd = pidfd_open(os.getpid()) - held.append(own_fd) - pidfd_send_signal(own_fd, 0) - report["own_fd_not_exited"] = not exited(own_fd) - child = subprocess.Popen( - [sys.executable, "-IB", "-c", "import time; time.sleep(4)"], - stdin=subprocess.DEVNULL, - stdout=subprocess.DEVNULL, - stderr=subprocess.DEVNULL, - ) - child_fd = pidfd_open(child.pid) - held.append(child_fd) - pidfd_send_signal(child_fd, int(signal.SIGKILL)) - report["child_pidfd_exit_observed"] = exited(child_fd, 5000) - report["child_exit_code"] = child.wait(timeout=5) - report["passed"] = ( - report["own_fd_not_exited"] - and report["child_pidfd_exit_observed"] - and report["child_exit_code"] == -signal.SIGKILL - ) - except BaseException as error: - report["error"] = { - "type": type(error).__name__, - "message": str(error), - "errno": getattr(error, "errno", None), - } - finally: - # The trusted child exits by itself if the API is unavailable; no PID kill fallback. - if child is not None: - try: - report["final_child_exit_code"] = child.wait(timeout=6) - except BaseException as error: - errors.append({"phase": "wait_child", "type": type(error).__name__}) - for fd in held: - try: - os.close(fd) - except OSError as error: - errors.append({"phase": "close_fd", "errno": error.errno}) - report["cleanup_errors"] = errors - if errors: - report["passed"] = False - runtime.write(output, report) - return 0 if report["passed"] else 2 - - -def unique(pairs): - result = {} - for key, value in pairs: - if key in result: - raise ValueError("duplicate JSON key") - result[key] = value - return result - - -def read_line(fd, deadline, raw): - while time.monotonic() < deadline and len(raw) <= 1048576: - ready, _, _ = select.select([fd], [], [], max(0, deadline - time.monotonic())) - if not ready: - break - data = os.read(fd, 1) - if not data: - raise ValueError("premature protocol EOF") - raw.extend(data) - if data == b"\n": - return bytes(raw) - raise ValueError("trusted protocol size or time limit exceeded") - - -def read_ready_file(path, deadline): - while time.monotonic() < deadline: - raw = path.read_bytes() - if len(raw) > 1048576: - raise ValueError("trusted output exceeds size limit") - if raw.endswith(b"\n"): - return raw - time.sleep(0.01) - raise ValueError("trusted readiness deadline exceeded") - - -def exited(fd, milliseconds=0): - poller = select.poll() - poller.register(fd, select.POLLIN) - events = poller.poll(milliseconds) - if any(mask & (select.POLLERR | select.POLLNVAL) for _, mask in events): - raise ValueError("pidfd wait failed") - return any(mask & (select.POLLIN | select.POLLHUP) for _, mask in events) - - -def status(pid): - return dict( - line.split(":", 1) - for line in Path(f"/proc/{pid}/status").read_text().splitlines() - if ":" in line - ) - - -def bind_init(record, launcher_pid): - pid = record.get("child-pid") - if type(pid) is not int or pid <= 1: - raise ValueError("invalid namespace-init PID") - fd = pidfd_open(pid) - try: - fields = status(pid) - if exited(fd) or int(fields["PPid"]) != launcher_pid: - raise ValueError("namespace init identity or parent changed") - namespace_pids = [int(value) for value in fields["NSpid"].split()] - if namespace_pids[0] != pid or namespace_pids[-1] != 1: - raise ValueError("reported child is not namespace PID1") - # The child is blocked before payload execution. Only the PID namespace - # is compared: bwrap can create a further user namespace during setup. - pid_namespace = os.readlink(f"/proc/{pid}/ns/pid") - if pid_namespace != f"pid:[{record.get('pid-namespace')}]": - raise ValueError("reported PID namespace identity differs") - if exited(fd): - raise ValueError("namespace init exited during binding") - return fd, { - "pid": pid, - "namespace_pids": namespace_pids, - "pid_namespace": pid_namespace, - } - except BaseException: - os.close(fd) - raise - - -def bind_children(init_pid, readiness): - wanted = readiness.get("live_descendant_pids") - if not isinstance(wanted, list) or len(wanted) != 2 or len(set(wanted)) != 2: - raise ValueError("expected two registered live trusted children") - if any(type(pid) is not int or pid <= 2 for pid in wanted): - raise ValueError("invalid trusted child PID") - held = [] - records = [] - try: - for child in psutil.Process(init_pid).children(recursive=True): - try: - fields = status(child.pid) - except FileNotFoundError: - continue - namespace_pids = [int(value) for value in fields["NSpid"].split()] - if namespace_pids[-1] not in wanted: - continue - fd = pidfd_open(child.pid) - held.append(fd) - current = status(child.pid) - if exited(fd) or current["NSpid"] != fields["NSpid"]: - raise ValueError("trusted child exited during binding") - records.append({"pid": child.pid, "namespace_pids": namespace_pids}) - if sorted(row["namespace_pids"][-1] for row in records) != sorted(wanted): - raise ValueError("registered trusted children not both observed") - return held, records - except BaseException: - for fd in held: - os.close(fd) - raise - - -def finalize_case( - report, process, init_fd, children, pipes, output, status_raw, rootfs -): - errors = [] - report["forced_cleanup"] = False - - def attempt(name, operation): - try: - return operation() - except BaseException as error: - errors.append( - {"operation": name, "type": type(error).__name__, "message": str(error)} - ) - return None - - # EOF cannot authorize the payload. Close this before releasing setup pipes. - if process is not None and process.stdin is not None: - attempt("close_authorization_pipe", process.stdin.close) - report["final_init_exit_observed"] = False - if init_fd is not None: - if attempt("check_init_exit", lambda: exited(init_fd)) is not True: - try: - pidfd_send_signal(init_fd, int(signal.SIGKILL)) - report["forced_cleanup"] = True - except ProcessLookupError: - # Exit between poll and signal is not a failed cleanup if the - # stable pidfd independently confirms that same process exited. - confirmed = attempt("confirm_signal_race", lambda: exited(init_fd)) - report["signal_race_already_exited"] = confirmed is True - if confirmed is not True: - errors.append( - {"operation": "signal_init", "type": "UnconfirmedExitRace"} - ) - except BaseException as error: - errors.append( - { - "operation": "signal_init", - "type": type(error).__name__, - "message": str(error), - } - ) - report["forced_init_exit_observed"] = attempt( - "wait_init_exit", lambda: exited(init_fd, 5000) - ) - if report["forced_init_exit_observed"] is not True: - errors.append( - {"operation": "wait_init_exit", "type": "ExitNotConfirmed"} - ) - report["final_init_exit_observed"] = ( - attempt("final_init_poll", lambda: exited(init_fd)) is True - ) - attempt("close_init_fd", lambda: os.close(init_fd)) - if process is not None: - if attempt("poll_launcher", process.poll) is None: - report["forced_cleanup"] = True - attempt("kill_launcher", process.kill) - attempt("wait_launcher", lambda: process.wait(timeout=5)) - report["final_launcher_exit_code"] = process.returncode - if process.returncode is None: - errors.append({"operation": "wait_launcher", "type": "ExitNotConfirmed"}) - report["cleanup_registered_child_exits"] = [] - for fd in children: - observed = attempt("wait_child_exit", lambda fd=fd: exited(fd, 1000)) - report["cleanup_registered_child_exits"].append(observed) - if observed is not True: - errors.append({"operation": "wait_child_exit", "type": "ExitNotConfirmed"}) - attempt("close_child_fd", lambda fd=fd: os.close(fd)) - for fd in pipes: - if fd >= 0: - attempt("close_pipe_fd", lambda fd=fd: os.close(fd)) - # Both writers must be gone; launcher exit alone is insufficient, and - # namespace exit alone leaves the launcher able to modify these files. - report["outputs_stable"] = ( - report["final_init_exit_observed"] - and process is not None - and process.returncode is not None - ) - report["cleanup_complete"] = ( - report["outputs_stable"] - and all(value is True for value in report["cleanup_registered_child_exits"]) - and not errors - ) - stdout_raw = attempt("read_stdout", (output / "stdout.json").read_bytes) - stderr_raw = attempt("read_stderr", (output / "stderr.private.log").read_bytes) - attempt( - "write_status", - lambda: (output / "status.private.jsonl").write_bytes(status_raw), - ) - for name, raw in (("stdout", stdout_raw), ("stderr", stderr_raw)): - report[name + "_sha256"] = ( - runtime.sha(raw) if raw is not None and report["outputs_stable"] else None - ) - if not report["outputs_stable"] and raw is not None: - report[name + "_partial_snapshot_sha256"] = runtime.sha(raw) - report["status_sha256"] = runtime.sha(status_raw) - report["rootfs_digest"] = rootfs["rootfs_digest"] - report["cleanup_or_evidence_errors"] = errors - if report["forced_cleanup"] or errors or not report["outputs_stable"]: - report["passed"] = False - # If even the report destination fails, the wrapper still captures the - # nonzero process status and stderr; no success record is fabricated. - runtime.write(output / "report.json", report) - - -def run_case(locators, rootfs, mode, output): - output.mkdir() - sentinel = output / "unmounted-sentinel.private" - sentinel.write_text("trusted synthetic marker") - namespaces = { - name: os.readlink("/proc/self/ns/" + name) - for name in ("user", "net", "pid", "uts", "ipc") - } - args = runtime.command( - locators, Path(__file__).with_name("synthetic_probe.py"), namespaces, sentinel - ) - status_read, status_write = os.pipe() - block_read, block_write = os.pipe() - # Additional mount policy; the independent /dev/shm remains a bounded tmpfs. - args[1:1] = ["--json-status-fd", str(status_write), "--block-fd", str(block_read)] - payload_index = args.index("/opt/python/" + locators["python_relative"]) - args[payload_index:payload_index] = ["--remount-ro", "/dev"] - args.append("gate_check" if mode.startswith("gate_") else mode) - process = None - init_fd = None - children = [] - report = {"mode": mode, "passed": False, "generated_code_executed": False} - stdout_raw = b"" - status_raw = bytearray() - try: - with ( - (output / "stderr.private.log").open("xb") as stderr, - (output / "stdout.json").open("xb") as stdout, - ): - process = subprocess.Popen( - [ - sys.executable, - "-IB", - str(Path(__file__).resolve()), - "_exec", - json.dumps(args), - ], - stdin=subprocess.PIPE, - stdout=stdout, - stderr=stderr, - pass_fds=(status_write, block_read), - start_new_session=True, - ) - os.close(status_write) - status_write = -1 - os.close(block_read) - block_read = -1 - read_line(status_read, time.monotonic() + 10, status_raw) - start = json.loads(status_raw, object_pairs_hook=unique) - init_fd, identity = bind_init(start, process.pid) - report["init_identity"] = identity - report["initial_status"] = start - report["authorization_sent"] = False - if mode == "gate_early_launcher_death": - # Reproduce launcher death before setup release, while holding - # the init pidfd. EOF on the setup pipe must not authorize work. - process.kill() - process.wait(timeout=5) - else: - os.write(block_write, b"1") - os.close(block_write) - block_write = -1 - if mode not in ("gate_eof", "gate_timeout", "gate_early_launcher_death"): - token = ( - START_TOKEN[:-1] - if mode == "gate_short" - else b"wrong\n" - if mode == "gate_wrong" - else START_TOKEN - ) - process.stdin.write(token) - process.stdin.flush() - report["authorization_sent"] = token == START_TOKEN - if mode != "gate_timeout": - process.stdin.close() - if mode.startswith("gate_"): - process.wait(timeout=8) - report["probe_passed"] = True # Exact output/exit checks below. - elif mode in ("bounds", "synthetic"): - process.wait(timeout=60) - report["probe_passed"] = True - else: - stdout_raw = read_ready_file( - output / "stdout.json", time.monotonic() + 10 - ) - payload = json.loads(stdout_raw, object_pairs_hook=unique) - if payload.get("ready") is not True or payload.get("mode") != mode: - raise ValueError("trusted cleanup probe did not become ready") - if payload.get("namespace_pid") != identity["pid_namespace"]: - raise ValueError("payload PID namespace differs from bound init") - children, child_records = bind_children(identity["pid"], payload) - report["registered_child_identities"] = child_records - if mode == "timeout": - try: - process.wait(timeout=1) - except subprocess.TimeoutExpired: - report["timeout_observed"] = True - pidfd_send_signal(init_fd, int(signal.SIGKILL)) - else: - raise ValueError("timeout probe returned before deadline") - elif mode == "launcher_kill": - process.kill() - process.wait(timeout=8) - stdout_raw = (output / "stdout.json").read_bytes() - report["probe_passed"] = True - report["exit_code"] = process.returncode - report["init_exit_observed"] = exited(init_fd, 5000) - report["registered_child_exit_observed"] = [ - exited(fd, 1000) for fd in children - ] - if not report["init_exit_observed"]: - raise ValueError("namespace init exit not confirmed") - stdout_raw = (output / "stdout.json").read_bytes() - stderr_raw = (output / "stderr.private.log").read_bytes() - if mode.startswith("gate_") and mode != "gate_valid": - report["probe_passed"] = ( - stdout_raw == b"" and AUTHORIZED_MARKER not in stderr_raw - ) - if mode != "gate_early_launcher_death": - report["probe_passed"] = ( - report["probe_passed"] and stderr_raw == REJECTED_MARKER - ) - else: - report["probe_passed"] = ( - report["probe_passed"] and stderr_raw == AUTHORIZED_MARKER - ) - if mode in ("bounds", "synthetic", "gate_valid"): - payload = json.loads(stdout_raw, object_pairs_hook=unique) - report["probe_passed"] = ( - report["probe_passed"] and payload.get("passed") is True - ) - # Parent has exited and bwrap's payload closes the status fd. Read - # the remainder with a finite deadline; EOF is expected. - deadline = time.monotonic() + 2 - status_eof = False - while time.monotonic() < deadline: - ready, _, _ = select.select( - [status_read], [], [], max(0, deadline - time.monotonic()) - ) - if not ready: - break - data = os.read(status_read, 4096) - if not data: - status_eof = True - break - status_raw += data - if len(status_raw) > 65536: - raise ValueError("status stream unexpectedly large") - report["status_eof"] = status_eof - expected_exit = ( - -signal.SIGKILL - if mode in ("launcher_kill", "gate_early_launcher_death") - else 137 - if mode == "timeout" - else 23 - if mode.startswith("gate_") and mode != "gate_valid" - else 0 - ) - report["passed"] = ( - report["probe_passed"] - and process.returncode == expected_exit - and report["init_exit_observed"] - and all(report["registered_child_exit_observed"]) - and status_eof - ) - except BaseException as error: - report["error"] = {"type": type(error).__name__, "message": str(error)} - finally: - finalize_case( - report, - process, - init_fd, - children, - (status_read, status_write, block_read, block_write), - output, - status_raw, - rootfs, - ) - return report - - -def main(): - parser = argparse.ArgumentParser(description=__doc__) - parser.add_argument("phase", choices=("capability", "probe")) - parser.add_argument("--bundle", type=Path) - parser.add_argument("--inventory", type=Path) - parser.add_argument("--capability", type=Path) - parser.add_argument("--output", type=Path, required=True) - args = parser.parse_args() - if args.phase == "capability": - return capability(args.output) - if args.capability is None or args.inventory is None: - raise ValueError("probe requires capability evidence and inventory") - capability_raw = args.capability.read_bytes() - preflight = json.loads(capability_raw, object_pairs_hook=unique) - if ( - preflight.get("passed") is not True - or preflight.get("cleanup_errors") != [] - or preflight.get("job_id") != os.environ["SLURM_JOB_ID"] - or preflight.get("driver_sha256") != runtime.sha(Path(__file__).read_bytes()) - or preflight.get("python_binary_sha256") - != runtime.sha(Path(sys._base_executable).read_bytes()) - or preflight.get("kernel_release") != platform.release() - ): - raise ValueError( - "actual pidfd capability preflight did not pass for this job/runtime" - ) - rootfs = json.loads((args.inventory / "rootfs.json").read_bytes()) - locators = json.loads( - (args.inventory / "runtime-locators.private.json").read_bytes() - ) - if ( - runtime.sha(Path(locators["bwrap"]).read_bytes()) - != rootfs["executor"]["binary_sha256"] - ): - raise ValueError("bwrap binary differs from inventory") - args.output.mkdir() - results = [] - for mode in MODES: - result = run_case(locators, rootfs, mode, args.output / mode) - results.append(result) - if result.get("cleanup_complete") is not True: - break - passed = len(results) == len(MODES) and all(row["passed"] for row in results) - runtime.write( - args.output / "summary.json", - { - "scope": "trusted_kernel_boundary_and_cleanup_probes", - "passed": passed, - "unexecuted_modes": list(MODES[len(results) :]), - "capability_sha256": runtime.sha(capability_raw), - "cases": results, - "generated_code_executed": False, - "qualified_generated_code_executor": False, - "verdict": "INCONCLUSIVE", - "driver_sha256": runtime.sha(Path(__file__).read_bytes()), - "probe_sha256": runtime.sha( - Path(__file__).with_name("synthetic_probe.py").read_bytes() - ), - "runtime_sha256": runtime.sha(Path(runtime.__file__).read_bytes()), - }, - ) - return 0 if passed else 2 - - -if __name__ == "__main__": - if ( - not os.environ.get("SLURM_JOB_ID") - or os.environ.get("SLURM_GPUS_ON_NODE", "0") != "0" - ): - raise ValueError("CPU-only Slurm allocation required") - if len(sys.argv) == 3 and sys.argv[1] == "_exec": - runtime.limits() - command = json.loads(sys.argv[2]) - os.execv(command[0], command) - raise SystemExit(main()) diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v2/bounds_driver.review-rejected-stability.py b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v2/bounds_driver.review-rejected-stability.py deleted file mode 100644 index 4c8c4ea80..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v2/bounds_driver.review-rejected-stability.py +++ /dev/null @@ -1,602 +0,0 @@ -"""Supervise only fixed trusted probes and retain actual namespace-init exit evidence.""" - -import argparse -import ctypes -import hashlib -import importlib.util -import json -import os -import platform -import select -import signal -import subprocess -import sys -import time -from pathlib import Path - -import psutil - -RUNTIME_PATH = Path(__file__).with_name("runtime.py") -RUNTIME_SHA256 = "997cbfed8398e8ad62d1beb4f6b4bc6e61a0f4572b9156cfd0c6f3fb52b03233" -if hashlib.sha256(RUNTIME_PATH.read_bytes()).hexdigest() != RUNTIME_SHA256: - raise ValueError("unexpected runtime source bytes") -runtime_spec = importlib.util.spec_from_file_location( - "trusted_bound_runtime", RUNTIME_PATH -) -runtime = importlib.util.module_from_spec(runtime_spec) -runtime_spec.loader.exec_module(runtime) -START_TOKEN = b"inferrsi-trusted-start-v2\n" -AUTHORIZED_MARKER = b"PAYLOAD_START_AUTHORIZED\n" -REJECTED_MARKER = b"PAYLOAD_START_REJECTED\n" -MODES = ( - "gate_valid", - "gate_eof", - "gate_short", - "gate_wrong", - "gate_timeout", - "gate_early_launcher_death", - "bounds", - "synthetic", - "normal", - "timeout", - "launcher_kill", -) - - -def pidfd_syscall(number, *arguments): - if ( - sys.platform != "linux" - or platform.machine() != "x86_64" - or ctypes.sizeof(ctypes.c_void_p) != 8 - or ctypes.sizeof(ctypes.c_long) != 8 - ): - raise ValueError("pidfd bindings require Linux x86_64 LP64") - library = ctypes.CDLL(None, use_errno=True) - library.syscall.restype = ctypes.c_long - ctypes.set_errno(0) - result = library.syscall(ctypes.c_long(number), *arguments) - if result == -1: - error = ctypes.get_errno() - raise OSError(error, os.strerror(error)) - return result - - -def pidfd_open(pid): - if type(pid) is not int or not 0 < pid < 2**31: - raise ValueError("invalid positive PID") - return pidfd_syscall(434, ctypes.c_int(pid), ctypes.c_uint(0)) - - -def pidfd_send_signal(fd, value): - if type(fd) is not int or fd < 0 or type(value) is not int or value < 0: - raise ValueError("invalid pidfd or signal") - result = pidfd_syscall( - 424, ctypes.c_int(fd), ctypes.c_int(value), ctypes.c_void_p(), ctypes.c_uint(0) - ) - if result != 0: - raise ValueError("unexpected pidfd_send_signal result") - - -def capability(output): - report = { - "schema": 1, - "passed": False, - "job_id": os.environ["SLURM_JOB_ID"], - "implementation": "linux-x86_64-lp64-raw-syscalls-434-and-424", - "python_exposes_pidfd_open": hasattr(os, "pidfd_open"), - "python_exposes_pidfd_send_signal": hasattr(signal, "pidfd_send_signal"), - "driver_sha256": runtime.sha(Path(__file__).read_bytes()), - "python_binary_sha256": runtime.sha(Path(sys._base_executable).read_bytes()), - "kernel_release": platform.release(), - "generated_code_executed": False, - } - held = [] - child = None - errors = [] - try: - own_fd = pidfd_open(os.getpid()) - held.append(own_fd) - pidfd_send_signal(own_fd, 0) - report["own_fd_not_exited"] = not exited(own_fd) - child = subprocess.Popen( - [sys.executable, "-IB", "-c", "import time; time.sleep(4)"], - stdin=subprocess.DEVNULL, - stdout=subprocess.DEVNULL, - stderr=subprocess.DEVNULL, - ) - child_fd = pidfd_open(child.pid) - held.append(child_fd) - pidfd_send_signal(child_fd, int(signal.SIGKILL)) - report["child_pidfd_exit_observed"] = exited(child_fd, 5000) - report["child_exit_code"] = child.wait(timeout=5) - report["passed"] = ( - report["own_fd_not_exited"] - and report["child_pidfd_exit_observed"] - and report["child_exit_code"] == -signal.SIGKILL - ) - except BaseException as error: - report["error"] = { - "type": type(error).__name__, - "message": str(error), - "errno": getattr(error, "errno", None), - } - finally: - # The trusted child exits by itself if the API is unavailable; no PID kill fallback. - if child is not None: - try: - report["final_child_exit_code"] = child.wait(timeout=6) - except BaseException as error: - errors.append({"phase": "wait_child", "type": type(error).__name__}) - for fd in held: - try: - os.close(fd) - except OSError as error: - errors.append({"phase": "close_fd", "errno": error.errno}) - report["cleanup_errors"] = errors - if errors: - report["passed"] = False - runtime.write(output, report) - return 0 if report["passed"] else 2 - - -def unique(pairs): - result = {} - for key, value in pairs: - if key in result: - raise ValueError("duplicate JSON key") - result[key] = value - return result - - -def read_line(fd, deadline, raw): - while time.monotonic() < deadline and len(raw) <= 1048576: - ready, _, _ = select.select([fd], [], [], max(0, deadline - time.monotonic())) - if not ready: - break - data = os.read(fd, 1) - if not data: - raise ValueError("premature protocol EOF") - raw.extend(data) - if data == b"\n": - return bytes(raw) - raise ValueError("trusted protocol size or time limit exceeded") - - -def read_ready_file(path, deadline): - while time.monotonic() < deadline: - raw = path.read_bytes() - if len(raw) > 1048576: - raise ValueError("trusted output exceeds size limit") - if raw.endswith(b"\n"): - return raw - time.sleep(0.01) - raise ValueError("trusted readiness deadline exceeded") - - -def exited(fd, milliseconds=0): - poller = select.poll() - poller.register(fd, select.POLLIN) - events = poller.poll(milliseconds) - if any(mask & (select.POLLERR | select.POLLNVAL) for _, mask in events): - raise ValueError("pidfd wait failed") - return any(mask & (select.POLLIN | select.POLLHUP) for _, mask in events) - - -def status(pid): - return dict( - line.split(":", 1) - for line in Path(f"/proc/{pid}/status").read_text().splitlines() - if ":" in line - ) - - -def bind_init(record, launcher_pid): - pid = record.get("child-pid") - if type(pid) is not int or pid <= 1: - raise ValueError("invalid namespace-init PID") - fd = pidfd_open(pid) - try: - fields = status(pid) - if exited(fd) or int(fields["PPid"]) != launcher_pid: - raise ValueError("namespace init identity or parent changed") - namespace_pids = [int(value) for value in fields["NSpid"].split()] - if namespace_pids[0] != pid or namespace_pids[-1] != 1: - raise ValueError("reported child is not namespace PID1") - # The child is blocked before payload execution. Only the PID namespace - # is compared: bwrap can create a further user namespace during setup. - pid_namespace = os.readlink(f"/proc/{pid}/ns/pid") - if pid_namespace != f"pid:[{record.get('pid-namespace')}]": - raise ValueError("reported PID namespace identity differs") - if exited(fd): - raise ValueError("namespace init exited during binding") - return fd, { - "pid": pid, - "namespace_pids": namespace_pids, - "pid_namespace": pid_namespace, - } - except BaseException: - os.close(fd) - raise - - -def bind_children(init_pid, readiness): - wanted = readiness.get("live_descendant_pids") - if not isinstance(wanted, list) or len(wanted) != 2 or len(set(wanted)) != 2: - raise ValueError("expected two registered live trusted children") - if any(type(pid) is not int or pid <= 2 for pid in wanted): - raise ValueError("invalid trusted child PID") - held = [] - records = [] - try: - for child in psutil.Process(init_pid).children(recursive=True): - try: - fields = status(child.pid) - except FileNotFoundError: - continue - namespace_pids = [int(value) for value in fields["NSpid"].split()] - if namespace_pids[-1] not in wanted: - continue - fd = pidfd_open(child.pid) - held.append(fd) - current = status(child.pid) - if exited(fd) or current["NSpid"] != fields["NSpid"]: - raise ValueError("trusted child exited during binding") - records.append({"pid": child.pid, "namespace_pids": namespace_pids}) - if sorted(row["namespace_pids"][-1] for row in records) != sorted(wanted): - raise ValueError("registered trusted children not both observed") - return held, records - except BaseException: - for fd in held: - os.close(fd) - raise - - -def finalize_case( - report, process, init_fd, children, pipes, output, status_raw, rootfs -): - errors = [] - report["forced_cleanup"] = False - - def attempt(name, operation): - try: - return operation() - except BaseException as error: - errors.append( - {"operation": name, "type": type(error).__name__, "message": str(error)} - ) - return None - - # EOF cannot authorize the payload. Close this before releasing setup pipes. - if process is not None and process.stdin is not None: - attempt("close_authorization_pipe", process.stdin.close) - report["final_init_exit_observed"] = False - if init_fd is not None: - if attempt("check_init_exit", lambda: exited(init_fd)) is not True: - try: - pidfd_send_signal(init_fd, int(signal.SIGKILL)) - report["forced_cleanup"] = True - except ProcessLookupError: - # Exit between poll and signal is not a failed cleanup if the - # stable pidfd independently confirms that same process exited. - confirmed = attempt("confirm_signal_race", lambda: exited(init_fd)) - report["signal_race_already_exited"] = confirmed is True - if confirmed is not True: - errors.append( - {"operation": "signal_init", "type": "UnconfirmedExitRace"} - ) - except BaseException as error: - errors.append( - { - "operation": "signal_init", - "type": type(error).__name__, - "message": str(error), - } - ) - report["forced_init_exit_observed"] = attempt( - "wait_init_exit", lambda: exited(init_fd, 5000) - ) - if report["forced_init_exit_observed"] is not True: - errors.append( - {"operation": "wait_init_exit", "type": "ExitNotConfirmed"} - ) - report["final_init_exit_observed"] = ( - attempt("final_init_poll", lambda: exited(init_fd)) is True - ) - attempt("close_init_fd", lambda: os.close(init_fd)) - if process is not None: - if attempt("poll_launcher", process.poll) is None: - report["forced_cleanup"] = True - attempt("kill_launcher", process.kill) - attempt("wait_launcher", lambda: process.wait(timeout=5)) - report["final_launcher_exit_code"] = process.returncode - if process.returncode is None: - errors.append({"operation": "wait_launcher", "type": "ExitNotConfirmed"}) - report["cleanup_registered_child_exits"] = [] - for fd in children: - observed = attempt("wait_child_exit", lambda fd=fd: exited(fd, 1000)) - report["cleanup_registered_child_exits"].append(observed) - if observed is not True: - errors.append({"operation": "wait_child_exit", "type": "ExitNotConfirmed"}) - attempt("close_child_fd", lambda fd=fd: os.close(fd)) - for fd in pipes: - if fd >= 0: - attempt("close_pipe_fd", lambda fd=fd: os.close(fd)) - stdout_raw = attempt("read_stdout", (output / "stdout.json").read_bytes) - stderr_raw = attempt("read_stderr", (output / "stderr.private.log").read_bytes) - attempt( - "write_status", - lambda: (output / "status.private.jsonl").write_bytes(status_raw), - ) - report["outputs_stable"] = report["final_init_exit_observed"] - for name, raw in (("stdout", stdout_raw), ("stderr", stderr_raw)): - report[name + "_sha256"] = ( - runtime.sha(raw) if raw is not None and report["outputs_stable"] else None - ) - if not report["outputs_stable"] and raw is not None: - report[name + "_partial_snapshot_sha256"] = runtime.sha(raw) - report["status_sha256"] = runtime.sha(status_raw) - report["rootfs_digest"] = rootfs["rootfs_digest"] - report["cleanup_or_evidence_errors"] = errors - if report["forced_cleanup"] or errors or not report["outputs_stable"]: - report["passed"] = False - # If even the report destination fails, the wrapper still captures the - # nonzero process status and stderr; no success record is fabricated. - runtime.write(output / "report.json", report) - - -def run_case(locators, rootfs, mode, output): - output.mkdir() - sentinel = output / "unmounted-sentinel.private" - sentinel.write_text("trusted synthetic marker") - namespaces = { - name: os.readlink("/proc/self/ns/" + name) - for name in ("user", "net", "pid", "uts", "ipc") - } - args = runtime.command( - locators, Path(__file__).with_name("synthetic_probe.py"), namespaces, sentinel - ) - status_read, status_write = os.pipe() - block_read, block_write = os.pipe() - # Additional mount policy; the independent /dev/shm remains a bounded tmpfs. - args[1:1] = ["--json-status-fd", str(status_write), "--block-fd", str(block_read)] - payload_index = args.index("/opt/python/" + locators["python_relative"]) - args[payload_index:payload_index] = ["--remount-ro", "/dev"] - args.append("gate_check" if mode.startswith("gate_") else mode) - process = None - init_fd = None - children = [] - report = {"mode": mode, "passed": False, "generated_code_executed": False} - stdout_raw = b"" - status_raw = bytearray() - try: - with ( - (output / "stderr.private.log").open("xb") as stderr, - (output / "stdout.json").open("xb") as stdout, - ): - process = subprocess.Popen( - [ - sys.executable, - "-IB", - str(Path(__file__).resolve()), - "_exec", - json.dumps(args), - ], - stdin=subprocess.PIPE, - stdout=stdout, - stderr=stderr, - pass_fds=(status_write, block_read), - start_new_session=True, - ) - os.close(status_write) - status_write = -1 - os.close(block_read) - block_read = -1 - read_line(status_read, time.monotonic() + 10, status_raw) - start = json.loads(status_raw, object_pairs_hook=unique) - init_fd, identity = bind_init(start, process.pid) - report["init_identity"] = identity - report["initial_status"] = start - report["authorization_sent"] = False - if mode == "gate_early_launcher_death": - # Reproduce launcher death before setup release, while holding - # the init pidfd. EOF on the setup pipe must not authorize work. - process.kill() - process.wait(timeout=5) - else: - os.write(block_write, b"1") - os.close(block_write) - block_write = -1 - if mode not in ("gate_eof", "gate_timeout", "gate_early_launcher_death"): - token = ( - START_TOKEN[:-1] - if mode == "gate_short" - else b"wrong\n" - if mode == "gate_wrong" - else START_TOKEN - ) - process.stdin.write(token) - process.stdin.flush() - report["authorization_sent"] = token == START_TOKEN - if mode != "gate_timeout": - process.stdin.close() - if mode.startswith("gate_"): - process.wait(timeout=8) - report["probe_passed"] = True # Exact output/exit checks below. - elif mode in ("bounds", "synthetic"): - process.wait(timeout=60) - report["probe_passed"] = True - else: - stdout_raw = read_ready_file( - output / "stdout.json", time.monotonic() + 10 - ) - payload = json.loads(stdout_raw, object_pairs_hook=unique) - if payload.get("ready") is not True or payload.get("mode") != mode: - raise ValueError("trusted cleanup probe did not become ready") - if payload.get("namespace_pid") != identity["pid_namespace"]: - raise ValueError("payload PID namespace differs from bound init") - children, child_records = bind_children(identity["pid"], payload) - report["registered_child_identities"] = child_records - if mode == "timeout": - try: - process.wait(timeout=1) - except subprocess.TimeoutExpired: - report["timeout_observed"] = True - pidfd_send_signal(init_fd, int(signal.SIGKILL)) - else: - raise ValueError("timeout probe returned before deadline") - elif mode == "launcher_kill": - process.kill() - process.wait(timeout=8) - stdout_raw = (output / "stdout.json").read_bytes() - report["probe_passed"] = True - report["exit_code"] = process.returncode - report["init_exit_observed"] = exited(init_fd, 5000) - report["registered_child_exit_observed"] = [ - exited(fd, 1000) for fd in children - ] - if not report["init_exit_observed"]: - raise ValueError("namespace init exit not confirmed") - stdout_raw = (output / "stdout.json").read_bytes() - stderr_raw = (output / "stderr.private.log").read_bytes() - if mode.startswith("gate_") and mode != "gate_valid": - report["probe_passed"] = ( - stdout_raw == b"" and AUTHORIZED_MARKER not in stderr_raw - ) - if mode != "gate_early_launcher_death": - report["probe_passed"] = ( - report["probe_passed"] and stderr_raw == REJECTED_MARKER - ) - else: - report["probe_passed"] = ( - report["probe_passed"] and stderr_raw == AUTHORIZED_MARKER - ) - if mode in ("bounds", "synthetic", "gate_valid"): - payload = json.loads(stdout_raw, object_pairs_hook=unique) - report["probe_passed"] = ( - report["probe_passed"] and payload.get("passed") is True - ) - # Parent has exited and bwrap's payload closes the status fd. Read - # the remainder with a finite deadline; EOF is expected. - deadline = time.monotonic() + 2 - status_eof = False - while time.monotonic() < deadline: - ready, _, _ = select.select( - [status_read], [], [], max(0, deadline - time.monotonic()) - ) - if not ready: - break - data = os.read(status_read, 4096) - if not data: - status_eof = True - break - status_raw += data - if len(status_raw) > 65536: - raise ValueError("status stream unexpectedly large") - report["status_eof"] = status_eof - expected_exit = ( - -signal.SIGKILL - if mode in ("launcher_kill", "gate_early_launcher_death") - else 137 - if mode == "timeout" - else 23 - if mode.startswith("gate_") and mode != "gate_valid" - else 0 - ) - report["passed"] = ( - report["probe_passed"] - and process.returncode == expected_exit - and report["init_exit_observed"] - and all(report["registered_child_exit_observed"]) - and status_eof - ) - except BaseException as error: - report["error"] = {"type": type(error).__name__, "message": str(error)} - finally: - finalize_case( - report, - process, - init_fd, - children, - (status_read, status_write, block_read, block_write), - output, - status_raw, - rootfs, - ) - return report - - -def main(): - parser = argparse.ArgumentParser(description=__doc__) - parser.add_argument("phase", choices=("capability", "probe")) - parser.add_argument("--bundle", type=Path) - parser.add_argument("--inventory", type=Path) - parser.add_argument("--capability", type=Path) - parser.add_argument("--output", type=Path, required=True) - args = parser.parse_args() - if args.phase == "capability": - return capability(args.output) - if args.capability is None or args.inventory is None: - raise ValueError("probe requires capability evidence and inventory") - capability_raw = args.capability.read_bytes() - preflight = json.loads(capability_raw, object_pairs_hook=unique) - if ( - preflight.get("passed") is not True - or preflight.get("cleanup_errors") != [] - or preflight.get("job_id") != os.environ["SLURM_JOB_ID"] - or preflight.get("driver_sha256") != runtime.sha(Path(__file__).read_bytes()) - or preflight.get("python_binary_sha256") - != runtime.sha(Path(sys._base_executable).read_bytes()) - or preflight.get("kernel_release") != platform.release() - ): - raise ValueError( - "actual pidfd capability preflight did not pass for this job/runtime" - ) - rootfs = json.loads((args.inventory / "rootfs.json").read_bytes()) - locators = json.loads( - (args.inventory / "runtime-locators.private.json").read_bytes() - ) - if ( - runtime.sha(Path(locators["bwrap"]).read_bytes()) - != rootfs["executor"]["binary_sha256"] - ): - raise ValueError("bwrap binary differs from inventory") - args.output.mkdir() - results = [] - for mode in MODES: - result = run_case(locators, rootfs, mode, args.output / mode) - results.append(result) - if result.get("final_init_exit_observed") is not True: - break - passed = len(results) == len(MODES) and all(row["passed"] for row in results) - runtime.write( - args.output / "summary.json", - { - "scope": "trusted_kernel_boundary_and_cleanup_probes", - "passed": passed, - "unexecuted_modes": list(MODES[len(results) :]), - "capability_sha256": runtime.sha(capability_raw), - "cases": results, - "generated_code_executed": False, - "qualified_generated_code_executor": False, - "verdict": "INCONCLUSIVE", - "driver_sha256": runtime.sha(Path(__file__).read_bytes()), - "probe_sha256": runtime.sha( - Path(__file__).with_name("synthetic_probe.py").read_bytes() - ), - "runtime_sha256": runtime.sha(Path(runtime.__file__).read_bytes()), - }, - ) - return 0 if passed else 2 - - -if __name__ == "__main__": - if ( - not os.environ.get("SLURM_JOB_ID") - or os.environ.get("SLURM_GPUS_ON_NODE", "0") != "0" - ): - raise ValueError("CPU-only Slurm allocation required") - if len(sys.argv) == 3 and sys.argv[1] == "_exec": - runtime.limits() - command = json.loads(sys.argv[2]) - os.execv(command[0], command) - raise SystemExit(main()) diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v2/controls.json b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v2/controls.json deleted file mode 100644 index 426ed9e82..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v2/controls.json +++ /dev/null @@ -1 +0,0 @@ -{"passed": true, "controls": ["all native syscall decisions 0 through 450", "three other architectures and two x32 calls denied", "duplicate status keys rejected", "valid namespace-init identity accepted", "five invalid init identities rejected", "exit during identity binding rejected and pidfd closed", "partial status retained on EOF and deadline", "cleanup exit race, wait timeout and stdout failure retain original errors and records", "three runtime sources compile", "isolated interpreter loads exact adjacent runtime; psutil import stubbed"], "linux_runtime_executed": false} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v2/plan.json b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v2/plan.json deleted file mode 100644 index 485277fe8..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v2/plan.json +++ /dev/null @@ -1,87 +0,0 @@ -{ - "scope": "trusted_startup_authorization_pidfd_and_cleanup_retry", - "status": "reviewed_not_submitted", - "plane": "harness", - "resources": { - "cpus": 4, - "requested_memory_gib": 16, - "walltime_seconds": 900, - "gpus": 0 - }, - "modes": [ - "gate_valid", - "gate_eof", - "gate_short", - "gate_wrong", - "gate_timeout", - "gate_early_launcher_death", - "bounds", - "synthetic", - "normal", - "timeout", - "launcher_kill" - ], - "source_sha256": { - "runtime.py": "997cbfed8398e8ad62d1beb4f6b4bc6e61a0f4572b9156cfd0c6f3fb52b03233", - "bounds_driver.py": "bfeeb6ea794c0ebf24e72a0127458ff74951253471daee080b3a3b1d82a062ba", - "synthetic_probe.py": "236920f27573447b9f3833afb72739033c965e22a9e406b09f28da815529ca64", - "run-synthetic.sbatch": "8c0a3b5699578fe854a3b8f3c9e225d999f5f313a78151310e409bd31791c0f5" - }, - "policy": { - "per_process_address_space_bytes": 2147483648, - "memlock_bytes": 0, - "posix_message_queue_bytes": 0, - "dev_mount_readonly": true, - "sysv_and_mqopen_seccomp_denied": [ - 29, - 30, - 31, - 64, - 65, - 66, - 67, - 68, - 69, - 70, - 71, - 220, - 240 - ], - "architecture": "Linux x86_64 LP64; foreign ABI and x32 rejected", - "shm_tmpfs_bytes": 33554432, - "work_tmpfs_bytes": 67108864, - "uid_task_hard_limit": 16 - }, - "scope_limits": [ - "No generated model program or benchmark data", - "Not an aggregate 16GiB hard memory cap", - "Thread RuntimeError cause remains unclassified", - "No formal qualification, gate activation, candidate search or measured E2E effect" - ], - "review": "Independent exact-source review cleared this bounded trusted CPU diagnostic; actual Linux behavior remains untested.", - "prior_rejected_versions": [ - "test_startup.initial-exclusive-fixture.py", - "startup-controls.initial-failure.txt", - "bounds_driver.review-rejected-stability.py", - "test_controls.review-before-stability.py", - "test_startup.review-before-stability.py" - ], - "source_references": [ - "https://github.com/containers/bubblewrap/tree/362936aea0d5c6e70f7af75d2d44b5a914cb4ba2", - "https://raw.githubusercontent.com/torvalds/linux/v6.12/arch/x86/entry/syscalls/syscall_64.tbl", - "https://man7.org/linux/man-pages/man2/getrlimit.2.html", - "https://man7.org/linux/man-pages/man7/pid_namespaces.7.html", - "https://man7.org/linux/man-pages/man2/pidfd_open.2.html", - "https://man7.org/linux/man-pages/man7/ipc_namespaces.7.html" - ], - "protocol": { - "actual_pidfd_capability_before_bwrap": true, - "raw_syscalls": [ - 434, - 424 - ], - "explicit_token_and_eof_before_payload": true, - "final_output_hash_requires_observed_init_and_launcher_exit": true, - "stop_remaining_cases_on_incomplete_cleanup": true - } -} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v2/proposal.md b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v2/proposal.md deleted file mode 100644 index 455a40a50..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v2/proposal.md +++ /dev/null @@ -1,44 +0,0 @@ -# Startup authorization and actual pidfd capability retry - -Refs [Issue #67](https://github.com/InferMatrixLab/InferRSI/issues/67). -INCONCLUSIVE; reviewed preparation, not yet submitted. - -CPU2012 exposed a missing Python pidfd API and a fail-open setup pipe: EOF -released payloads, and four output hashes were captured before later writes. -This version first exercises raw Linux x86_64 LP64 pidfd syscalls with the -registered interpreter and a finite trusted child. A capability failure stops -before bubblewrap. The probe phase binds that evidence to its job, source, -Python binary and kernel. No PID-based fallback replaces the stable pidfd. - -After binding namespace PID1, the supervisor must send the exact authorization -token and close its pipe. The payload rejects absent, short, wrong, oversized, -or non-terminated tokens before resource tests or evaluator work. Six startup -cases include launcher death before setup release. Five existing trusted modes -then exercise limits, four full EvalPlus synthetic cases and cleanup on normal -exit, timeout and launcher death. Final output hashes require observed namespace -init and launcher exit; incomplete cleanup stops remaining cases and marks output -digest as an unstable partial snapshot. - -The policy and registered packages remain unchanged, including UID task limit16. -Earlier fork EAGAIN may have reflected overlapping late payloads; it is not a -basis for increasing that limit. A16GiB Slurm request is not an enforced aggregate -memory cap. Thread-creation RuntimeError still has no attributed kernel cause. -No heldout task, model-generated code, GPU, performance search or E2E measurement -is part of this diagnostic. Formal generated-code qualification remains pending. - -Local startup controls use the real gate entry and mock only Linux-specific -supervision. They also verify wrong-job rejection, preserved errno, missing-exit -hash suppression and stopping further cases. Existing filter/identity/cleanup -and reference controls pass. The first startup-control fixture failed because it -tried to overwrite an exclusive JSON output; its original source and failure are -preserved, and the corrected separate-file fixture passes. Repository-configured -Ruff and shell syntax checks pass. Ambient global Ruff additionally flags broad -error-retention catches and AST-based test loading; these are not Linux results. - -Independent review found that a live launcher could still write after init exit. -The rejected source is preserved; the repair requires both writers to exit and -stops further cases on incomplete cleanup. The launcher-timeout negative control -now rejects final output hashes. Independent review is recorded against the exact -four source hashes. The actual -runtime result must be retained regardless of outcome. No acceptance gate is -activated by this preparation, and GPU readiness accounting remains unchanged. diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v2/reference-controls.json b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v2/reference-controls.json deleted file mode 100644 index 92d128cf2..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v2/reference-controls.json +++ /dev/null @@ -1 +0,0 @@ -{"reference_completion_cases": 18, "optimized_python": true} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v2/review.json b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v2/review.json deleted file mode 100644 index a4e1edaec..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v2/review.json +++ /dev/null @@ -1,33 +0,0 @@ -{ - "reviewer": "Codex independent review agent /root/review_fp8_graph", - "source_sha256": { - "runtime.py": "997cbfed8398e8ad62d1beb4f6b4bc6e61a0f4572b9156cfd0c6f3fb52b03233", - "bounds_driver.py": "bfeeb6ea794c0ebf24e72a0127458ff74951253471daee080b3a3b1d82a062ba", - "synthetic_probe.py": "236920f27573447b9f3833afb72739033c965e22a9e406b09f28da815529ca64", - "run-synthetic.sbatch": "8c0a3b5699578fe854a3b8f3c9e225d999f5f313a78151310e409bd31791c0f5" - }, - "blocking_findings": [], - "recommendation": "Proceed only with the bounded trusted CPU diagnostic; no runtime qualification or campaign acceptance.", - "closed_finding": { - "priority": "P2", - "original_driver_sha256": "3ca22611381cbffb0e29dfddf57f8deb711ac4cd4a8128b02b06eb8625483135", - "problem": "Final output hashes marked stable with namespace init exited but launcher still live after wait timeout.", - "resolution": "Require both writers exited; stop subsequent modes if cleanup incomplete. Independent timeout counterexample now returns unstable, incomplete and null final hashes." - }, - "independent_controls": [ - "test_controls.py under -IBO", - "test_startup.py under -IBO", - "18 completion controls", - "13 benign-reference controls", - "shell syntax", - "independent live-launcher timeout counterexample" - ], - "actual_linux_execution": false, - "generated_code_executor_qualified": false, - "resource_scope": { - "cpus": 4, - "requested_memory_gib": 16, - "walltime_seconds": 900, - "gpus": 0 - } -} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v2/run-synthetic.sbatch b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v2/run-synthetic.sbatch deleted file mode 100644 index 61e2161ae..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v2/run-synthetic.sbatch +++ /dev/null @@ -1,200 +0,0 @@ -#!/usr/bin/env bash -# Finite trusted resource/cleanup probes and four synthetic evaluator cases. -#SBATCH --job-name=qwen38-sandbox-bounds-v2 -#SBATCH --nodes=1 -#SBATCH --ntasks=1 -#SBATCH --cpus-per-task=4 -#SBATCH --mem=16G -#SBATCH --time=00:15:00 -set -euo pipefail -umask 077 -: "${TASK_ROOT:?}" -: "${SANDBOX_BUNDLE:?}" -: "${SANDBOX_FREEZE_SHA256:?}" -: "${SANDBOX_VENV:?}" -: "${REFERENCE_INSTALLATION:?}" -: "${REFERENCE_ROOTFS_SHA256:?}" -: "${REFERENCE_TERMINAL_SHA256:?}" -: "${REFERENCE_SLURM_SHA256:?}" -: "${REFERENCE_JOB_ID:?}" -: "${REFERENCE_EXECUTION_SHA256:?}" "${REFERENCE_STDOUT_SHA256:?}" -: "${BWRAP_BIN:?}" "${BWRAP_SHA256:?}" "${PYTHON_SHA256:?}" -: "${SYNTHETIC_SOURCE_DIR:?}" -: "${SYNTHETIC_RUNTIME_SHA256:?}" -: "${SYNTHETIC_PROBE_SHA256:?}" -: "${BOUNDS_DRIVER_SHA256:?}" -: "${SYNTHETIC_SCRIPT_SHA256:?}" -: "${SYNTHETIC_OUTPUT:?}" -unset PYTHONPATH PYTHONHOME PYTHONOPTIMIZE PYTHONINSPECT PYTHONSTARTUP VIRTUAL_ENV -[[ -n ${SLURM_JOB_ID:-} && ${SLURM_GPUS_ON_NODE:-0} == 0 ]] -root=$(realpath -e "$TASK_ROOT") -check_path() { - local value - value=$(realpath -m "$1") - case "$value" in "$root"/*) ;; *) echo 'Path escapes task root.' >&2; exit 2;; esac -} -check_digest() { - if [[ ! "$2" =~ ^[a-f0-9]{64}$ ]]; then return 2; fi - [[ $(sha256sum "$1" | cut -d' ' -f1) == "$2" ]] -} -for path in "$SANDBOX_BUNDLE" "$SANDBOX_VENV" "$REFERENCE_INSTALLATION" \ - "$SYNTHETIC_SOURCE_DIR" "$SYNTHETIC_OUTPUT" "$BWRAP_BIN"; do check_path "$path"; done -source_dir=$(cd -- "$SYNTHETIC_SOURCE_DIR" && pwd -P) -check_digest "$SANDBOX_VENV/bin/python" "$PYTHON_SHA256" -check_digest "$BWRAP_BIN" "$BWRAP_SHA256" -export PATH="$(dirname "$BWRAP_BIN"):/usr/bin:/bin" -[[ $(command -v bwrap) == "$BWRAP_BIN" ]] -check_digest "$source_dir/run-synthetic.sbatch" "$SYNTHETIC_SCRIPT_SHA256" -cmp "${BASH_SOURCE[0]}" "$source_dir/run-synthetic.sbatch" -check_digest "$source_dir/runtime.py" "$SYNTHETIC_RUNTIME_SHA256" -check_digest "$source_dir/synthetic_probe.py" "$SYNTHETIC_PROBE_SHA256" -check_digest "$source_dir/bounds_driver.py" "$BOUNDS_DRIVER_SHA256" -check_digest "$SANDBOX_BUNDLE/freeze.sha256" "$SANDBOX_FREEZE_SHA256" -(cd "$SANDBOX_BUNDLE" && sha256sum --check --strict freeze.sha256) -# Bind both the scheduler outcome and the repaired completion record. -check_digest "$REFERENCE_INSTALLATION/terminal-status.json" "$REFERENCE_TERMINAL_SHA256" -check_digest "$REFERENCE_INSTALLATION/slurm-final.private.txt" "$REFERENCE_SLURM_SHA256" -"$SANDBOX_VENV/bin/python" -IB - "$REFERENCE_INSTALLATION" "$REFERENCE_JOB_ID" <<'PYREF' -import json -import re -import sys -from pathlib import Path -reference = Path(sys.argv[1]) -job = sys.argv[2] -if not re.fullmatch(r"[0-9]+", job): - raise ValueError('Invalid reference job ID') -def unique_keys(pairs): - value = {} - for key, item in pairs: - if key in value: - raise ValueError('Duplicate terminal-record key') - value[key] = item - return value -terminal = json.loads((reference / 'terminal-status.json').read_bytes(), object_pairs_hook=unique_keys) -if terminal.get('completed') is not True or type(terminal.get('exit_code')) is not int: - raise ValueError('Reference terminal record has invalid types') -if terminal != {'completed': True, 'signal': 'none', 'exit_code': 0, 'job_id': job}: - raise ValueError('Reference installation did not complete normally') -slurm = (reference / 'slurm-final.private.txt').read_text() -for key, value in {'JobId': job, 'JobState': 'COMPLETED', 'ExitCode': '0:0'}.items(): - actual = re.findall(r'(? "$SYNTHETIC_OUTPUT/exit-code" - printf '{"completed":%s,"signal":"%s","exit_code":%s,"job_id":"%s"}\n' \ - "$task_completed" "$task_signal" "$status" "$SLURM_JOB_ID" > "$SYNTHETIC_OUTPUT/terminal-status.json" - exit "$status" -} -trap finish_synthetic EXIT -trap 'task_completed=false; task_signal=TERM; exit 143' TERM -trap 'task_completed=false; task_signal=INT; exit 130' INT -trap 'task_completed=false; task_signal=HUP; exit 129' HUP -# END TERMINAL RECORD -cp "$source_dir/run-synthetic.sbatch" "$source_dir/runtime.py" "$source_dir/synthetic_probe.py" "$source_dir/bounds_driver.py" "$SYNTHETIC_OUTPUT/" -cd "$root" -"$SANDBOX_VENV/bin/python" -IB "$source_dir/bounds_driver.py" capability \ - --output "$SYNTHETIC_OUTPUT/pidfd-capability.json" -"$SANDBOX_VENV/bin/python" -IB "$source_dir/runtime.py" inventory --bundle "$SANDBOX_BUNDLE" \ - --output "$SYNTHETIC_OUTPUT/rootfs-before" -"$SANDBOX_VENV/bin/python" -IB - "$REFERENCE_INSTALLATION/rootfs-before/rootfs.json" \ - "$SYNTHETIC_OUTPUT/rootfs-before/rootfs.json" <<'PY' -import json -import sys -from pathlib import Path -before, current = [json.loads(Path(path).read_bytes()) for path in sys.argv[1:]] -for record in (before, current): - record.pop('rootfs_digest') - record['entries'] = [row for row in record['entries'] if row['path'] != '/opt/probe.py'] -if before != current: - raise ValueError('Installed runtime changed beyond the explicitly bound trusted probe') -PY -probe_status=0 -"$SANDBOX_VENV/bin/python" -IB "$source_dir/bounds_driver.py" probe --bundle "$SANDBOX_BUNDLE" \ - --inventory "$SYNTHETIC_OUTPUT/rootfs-before" --capability "$SYNTHETIC_OUTPUT/pidfd-capability.json" \ - --output "$SYNTHETIC_OUTPUT/probe" || probe_status=$? -printf '%s\n' "$probe_status" > "$SYNTHETIC_OUTPUT/probe-status" -post_status=0 -run_postcheck() { - local name=$1 status=0 - shift - "$@" || status=$? - printf '%s\t%s\n' "$name" "$status" >> "$SYNTHETIC_OUTPUT/postflight-status.tsv" - if [[ "$status" != 0 ]]; then post_status=$status; fi -} -check_frozen_files() { (cd "$SANDBOX_BUNDLE" && sha256sum --check --strict freeze.sha256); } -run_postcheck inventory "$SANDBOX_VENV/bin/python" -IB "$source_dir/runtime.py" inventory \ - --bundle "$SANDBOX_BUNDLE" --output "$SYNTHETIC_OUTPUT/rootfs-after" -run_postcheck runtime-byte-comparison cmp "$SYNTHETIC_OUTPUT/rootfs-before/rootfs.json" \ - "$SYNTHETIC_OUTPUT/rootfs-after/rootfs.json" -run_postcheck runtime-script check_digest "$source_dir/runtime.py" "$SYNTHETIC_RUNTIME_SHA256" -run_postcheck bounds-driver check_digest "$source_dir/bounds_driver.py" "$BOUNDS_DRIVER_SHA256" -run_postcheck synthetic-probe check_digest "$source_dir/synthetic_probe.py" "$SYNTHETIC_PROBE_SHA256" -run_postcheck batch-script check_digest "$source_dir/run-synthetic.sbatch" "$SYNTHETIC_SCRIPT_SHA256" -run_postcheck freeze-manifest check_digest "$SANDBOX_BUNDLE/freeze.sha256" "$SANDBOX_FREEZE_SHA256" -run_postcheck freeze-files check_frozen_files -run_postcheck python-binary check_digest "$SANDBOX_VENV/bin/python" "$PYTHON_SHA256" -run_postcheck bwrap-binary check_digest "$BWRAP_BIN" "$BWRAP_SHA256" -if [[ "$probe_status" != 0 ]]; then exit "$probe_status"; fi -if [[ "$post_status" != 0 ]]; then exit "$post_status"; fi -echo 'Trusted resource/cleanup diagnostic completed; generated-code executor remains unqualified.' -task_completed=true diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v2/runtime.py b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v2/runtime.py deleted file mode 100644 index 51f1903f3..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v2/runtime.py +++ /dev/null @@ -1,481 +0,0 @@ -"""Resolve pinned EvalPlus metadata, inventory readonly binds and run a trusted probe.""" - -import argparse -import configparser -import hashlib -import importlib.metadata -import json -import os -import re -import resource -import shutil -import signal -import subprocess -import sys -import sysconfig -import threading -import tomllib -from pathlib import Path - -REVISION = "26d6d00bb1fd0fa37f39c99d5290da67891d1c5e" - - -def canonical(value): - return json.dumps(value, sort_keys=True, separators=(",", ":"), allow_nan=False).encode() - - -def sha(raw): - return hashlib.sha256(raw).hexdigest() - - -def write(path, value): - with path.open("xb") as stream: - stream.write(canonical(value) + b"\n") - - -def git(source, *args): - return subprocess.check_output(["git", "-C", str(source), *args]) - - -def source_identity(bundle): - source = bundle / "source" - if git(source, "rev-parse", "HEAD").decode().strip() != REVISION: - raise ValueError("wrong EvalPlus source revision") - if git(source, "diff", "HEAD", "--"): - raise ValueError("tracked EvalPlus source changed") - return { - "evaluator_revision": REVISION, - "archive_sha256": sha(git(source, "archive", "--format=tar", "HEAD")), - } - - -def requirements(bundle): - identity = source_identity(bundle) - config = configparser.ConfigParser() - config.read(bundle / "source/setup.cfg") - dependencies = [ - line.strip() for line in config["options"]["install_requires"].splitlines() if line.strip() - ] - build = tomllib.loads((bundle / "source/pyproject.toml").read_text())["build-system"][ - "requires" - ] - (bundle / "dependencies.in").write_text("\n".join(dependencies) + "\n") - (bundle / "build.in").write_text("\n".join([*build, "wheel"]) + "\n") - write( - bundle / "source.json", - { - **identity, - "official_default_dependencies": dependencies, - "build_dependencies": [*build, "wheel"], - "extras_enabled": [], - "generated_code_executed": False, - }, - ) - - -def verify_source(bundle): - expected = json.loads((bundle / "source.json").read_bytes()) - if any( - source_identity(bundle)[name] != expected[name] - for name in ("evaluator_revision", "archive_sha256") - ): - raise ValueError("EvalPlus source snapshot drift") - - -def tree(root, alias): - result = [] - for path in sorted(root.rglob("*")): - relative = path.relative_to(root).as_posix() - name = alias + "/" + relative - if path.is_symlink(): - target = os.readlink(path) - if Path(target).is_absolute() or not path.resolve(strict=True).is_relative_to(root): - raise ValueError("runtime symlink escapes its readonly root") - result.append({"path": name, "type": "symlink", "target": target}) - elif path.is_dir(): - result.append({"path": name, "type": "directory", "mode": path.stat().st_mode & 0o777}) - elif path.is_file(): - result.append( - { - "path": name, - "type": "file", - "mode": path.stat().st_mode & 0o777, - "bytes": path.stat().st_size, - "sha256": sha(path.read_bytes()), - } - ) - else: - raise ValueError("unsupported runtime filesystem entry") - return result - - -def system_dependencies(roots): - # Mirror the explicit relocated loader directory; inherit no host loader state. - loader_env = { - "PATH": "/usr/bin:/bin", "LC_ALL": "C", - "LD_LIBRARY_PATH": str(roots[0] / "lib"), - } - ldd = shutil.which("ldd") - if ldd is None: - raise ValueError("system ELF dependency scanner unavailable") - dependencies = {} - for root in roots: - for path in sorted(root.rglob("*")): - if not path.is_file() or path.is_symlink(): - continue - with path.open("rb") as stream: - if stream.read(4) != b"\x7fELF": - continue - result = subprocess.run( - [ldd, str(path)], capture_output=True, timeout=10, check=False, env=loader_env - ) - output = result.stdout.decode(errors="replace") - if result.returncode != 0 or "not found" in output: - raise ValueError("ELF runtime dependency missing") - for line in output.splitlines(): - match = re.search(r"(?:=>\s*)?(/[^\s]+)", line) - if not match: - continue - original = Path(match.group(1)) - resolved = original.resolve(strict=True) - if any(resolved.is_relative_to(bound) for bound in roots): - continue - if not str(original).startswith(("/lib/", "/lib64/", "/usr/lib/", "/usr/lib64/")): - raise ValueError("unexpected external runtime library") - dependencies[str(original)] = str(resolved) - return dependencies - - -def inventory(bundle, output): - verify_source(bundle) - task_root = Path(os.environ["TASK_ROOT"]).resolve(strict=True) - base = Path(sys.base_prefix).resolve(strict=True) - site = Path(sysconfig.get_path("purelib")).resolve(strict=True) - if not base.is_relative_to(task_root) or not site.is_relative_to(task_root): - raise ValueError("runtime roots must remain inside TASK_ROOT") - executable = Path(sys._base_executable).resolve(strict=True).relative_to(base).as_posix() - bwrap = shutil.which("bwrap") - if bwrap is None: - raise ValueError("bubblewrap is unavailable on this compute node") - help_text = subprocess.check_output([bwrap, "--help"], stderr=subprocess.STDOUT).decode() - for flag in ( - "--unshare-user", - "--unshare-net", - "--unshare-pid", - "--disable-userns", - "--assert-userns-disabled", - "--clearenv", - "--remount-ro", - "--size", - ): - if flag not in help_text: - raise ValueError("required bubblewrap isolation option unavailable") - dependencies = system_dependencies((base, site)) - entries = tree(base, "/opt/python") + tree(site, "/opt/site-packages") - for destination, source in dependencies.items(): - path = Path(source) - entries.append( - { - "path": destination, - "type": "file", - "bytes": path.stat().st_size, - "sha256": sha(path.read_bytes()), - } - ) - probe = Path(__file__).with_name("synthetic_probe.py") - entries.append( - { - "path": "/opt/probe.py", - "type": "file", - "bytes": probe.stat().st_size, - "sha256": sha(probe.read_bytes()), - } - ) - entries.sort(key=lambda row: row["path"]) - # Bind-set identity describes actual readonly bytes, not an OCI image. - rootfs_digest = "sha256:" + sha(canonical(entries)) - packages = sorted( - [ - {"name": d.metadata["Name"], "version": d.version} - for d in importlib.metadata.distributions() - ], - key=lambda row: row["name"].lower(), - ) - # Match every pinned evaluator Python source byte to the installed package. - checked = 0 - source = bundle / "source" - for encoded in git(source, "ls-files", "-z", "evalplus").split(b"\0"): - if not encoded or not encoded.endswith(b".py"): - continue - name = encoded.decode() - if (source / name).read_bytes() != (site / name).read_bytes(): - raise ValueError("installed EvalPlus differs from pinned source") - checked += 1 - if checked == 0: - raise ValueError("no pinned evaluator Python sources verified") - output.mkdir(parents=True, exist_ok=False) - write( - output / "rootfs.json", - { - "schema": 1, - "rootfs_kind": "readonly-bind-set-v1", - "loader_policy": { - "scanner_inherits_environment": False, - "base_library_directory": "/opt/python/lib", - "sandbox_ld_library_path": "/opt/python/lib", - }, - "rootfs_digest": rootfs_digest, - "evaluator_revision": REVISION, - "entries": entries, - "python": sys.version.split()[0], - "packages": packages, - "checked_evaluator_sources": checked, - "executor": { - "kind": "bubblewrap", - "binary_sha256": sha(Path(bwrap).read_bytes()), - "version": subprocess.check_output([bwrap, "--version"]).decode().strip(), - }, - "generated_code_executed": False, - "qualified_generated_code_executor": False, - }, - ) - write( - output / "runtime-locators.private.json", - { - "python_root": str(base), - "site_root": str(site), - "python_relative": executable, - "bwrap": bwrap, - "system_libraries": dependencies, - }, - ) - - -def command(locators, probe, namespaces, sentinel): - args = [ - locators["bwrap"], - "--unshare-user", - "--uid", - "65534", - "--gid", - "65534", - "--unshare-net", - "--unshare-pid", - "--unshare-ipc", - "--unshare-uts", - "--hostname", - "sandbox", - "--disable-userns", - "--assert-userns-disabled", - "--cap-drop", - "ALL", - "--die-with-parent", - "--new-session", - "--clearenv", - "--ro-bind", - locators["python_root"], - "/opt/python", - "--ro-bind", - locators["site_root"], - "/opt/site-packages", - "--ro-bind", - str(probe), - "/opt/probe.py", - ] - for destination, source in sorted(locators["system_libraries"].items()): - args.extend(["--ro-bind", source, destination]) - args += [ - "--proc", - "/proc", - "--dev", - "/dev", - "--size", - "33554432", - "--tmpfs", - "/dev/shm", - "--size", - "67108864", - "--tmpfs", - "/work", - "--dir", - "/work/tmp", - "--dir", - "/work/output", - "--chdir", - "/work", - "--remount-ro", - "/", - ] - environment = { - "HOME": "/work", - "TMPDIR": "/work/tmp", - "PATH": "/opt/python/bin", - "LD_LIBRARY_PATH": "/opt/python/lib", - "PYTHONDONTWRITEBYTECODE": "1", - "OMP_NUM_THREADS": "1", - "OPENBLAS_NUM_THREADS": "1", - "LC_ALL": "C.UTF-8", - "PROBE_PARENT_NAMESPACES": json.dumps(namespaces), - "PROBE_UNMOUNTED_SENTINEL": str(sentinel), - } - for name, value in environment.items(): - args += ["--setenv", name, value] - return args + ["/opt/python/" + locators["python_relative"], "-I", "/opt/probe.py"] - - -def limits(): - os.sched_setaffinity(0, {min(os.sched_getaffinity(0))}) - for name, value in ( - ("AS", 2147483648), - ("CPU", 30), - ("NOFILE", 64), - ("FSIZE", 1048576), - ("CORE", 0), - ): - resource.setrlimit(getattr(resource, "RLIMIT_" + name), (value, value)) - - -def probe(inventory_path, output): - rootfs = json.loads((inventory_path / "rootfs.json").read_bytes()) - locators = json.loads((inventory_path / "runtime-locators.private.json").read_bytes()) - if sha(Path(locators["bwrap"]).read_bytes()) != rootfs["executor"]["binary_sha256"]: - raise ValueError("bubblewrap binary changed after inventory") - output.mkdir(parents=True, exist_ok=False) - sentinel = output / "unmounted-sentinel.private" - sentinel.write_text("synthetic host marker, not a credential") - namespaces = { - name: os.readlink("/proc/self/ns/" + name) for name in ("user", "net", "pid", "uts", "ipc") - } - args = command(locators, Path(__file__).with_name("synthetic_probe.py"), namespaces, sentinel) - timed_out = False - # Sample from the supervisor after Popen. The evaluator process remains free - # of an added observer thread before its own default multiprocessing fork. - import psutil - sampled = {"processes": 0, "threads": 0, "sum_vms_bytes": 0, - "sum_rss_bytes": 0, "sum_fds": 0, "samples": 0, "errors": 0} - known = {} - sampling_failures = [] - stop_sampling = threading.Event() - def observe(pid): - try: - parent = psutil.Process(pid) - except psutil.NoSuchProcess: - sampling_failures.append("launcher exited before sampling") - return - while not stop_sampling.is_set(): - totals = {"processes": 0, "threads": 0, "sum_vms_bytes": 0, - "sum_rss_bytes": 0, "sum_fds": 0} - try: - processes = [parent, *parent.children(recursive=True)] - except psutil.NoSuchProcess: - break - except psutil.AccessDenied: - sampling_failures.append("cannot inspect owned launcher descendants") - return - for child in processes: - try: - known[child.pid] = child.create_time() - memory = child.memory_info() - totals["processes"] += 1 - totals["threads"] += child.num_threads() - totals["sum_vms_bytes"] += memory.vms - totals["sum_rss_bytes"] += memory.rss - totals["sum_fds"] += child.num_fds() - except (psutil.NoSuchProcess, psutil.AccessDenied): - sampled["errors"] += 1 - for name, value in totals.items(): - sampled[name] = max(sampled[name], value) - sampled["samples"] += 1 - stop_sampling.wait(0.01) - with ( - (output / "stdout.json").open("xb") as stdout, - (output / "stderr.private.log").open("xb") as stderr, - ): - process = subprocess.Popen( - [sys.executable, str(Path(__file__).resolve()), "_exec", json.dumps(args)], - stdin=subprocess.DEVNULL, - stdout=stdout, - stderr=stderr, - start_new_session=True, - env={**os.environ, "PROBE_INHERITED_SECRET": "synthetic-must-be-cleared"}, - ) - observer = threading.Thread(target=observe, args=(process.pid,), daemon=True) - observer.start() - try: - code = process.wait(timeout=60) - except subprocess.TimeoutExpired: - timed_out = True - if os.getpgid(process.pid) != process.pid: - raise RuntimeError("owned sandbox process group changed unexpectedly") - os.killpg(process.pid, signal.SIGKILL) - code = process.wait(timeout=10) - finally: - stop_sampling.set() - observer.join(timeout=2) - remaining = [] - for pid, created in known.items(): - try: - child = psutil.Process(pid) - if child.create_time() == created and child.status() != psutil.STATUS_ZOMBIE: - remaining.append(pid) - except psutil.NoSuchProcess: - pass - result = { - "sampled_peaks": sampled, - "sampling_thread_joined": not observer.is_alive(), - "sampling_failures": sampling_failures, - "known_live_descendants_after_exit": remaining, - "sampling_limitation": "Polling may miss transient processes; sampled peaks are not hard bounds or exhaustive cleanup proof.", - "phase": "trusted_synthetic_evaluator", - "exit_code": code, - "timed_out": timed_out, - "rootfs_digest": rootfs["rootfs_digest"], - "executor": rootfs["executor"], - "policy_script_sha256": sha(Path(__file__).read_bytes()), - "probe_script_sha256": sha(Path(__file__).with_name("synthetic_probe.py").read_bytes()), - "stdout_sha256": sha((output / "stdout.json").read_bytes()), - "generated_code_executed": False, - "full_evaluator_attempt_requested": True, - "qualified_generated_code_executor": False, - "verdict": "INCONCLUSIVE", - } - write(output / "execution.json", result) - if code != 0 or timed_out or remaining or observer.is_alive() or sampling_failures or not sampled["samples"]: - raise ValueError("trusted synthetic evaluator probe failed; retain evidence without fallback") - payload = json.loads((output / "stdout.json").read_bytes()) - if payload.get("passed") is not True: - raise ValueError("trusted synthetic evaluator assertions failed") - - -def main(): - parser = argparse.ArgumentParser(description=__doc__) - parser.add_argument("phase", choices=("requirements", "verify-source", "inventory", "probe")) - parser.add_argument("--bundle", type=Path, required=True) - parser.add_argument("--output", type=Path) - parser.add_argument("--inventory", type=Path) - args = parser.parse_args() - if not os.environ.get("SLURM_JOB_ID") or os.environ.get("SLURM_GPUS_ON_NODE", "0") != "0": - raise ValueError("CPU-only Slurm allocation required") - if args.phase == "requirements": - requirements(args.bundle) - elif args.phase == "verify-source": - verify_source(args.bundle) - elif args.phase == "inventory": - inventory(args.bundle, args.output) - else: - probe(args.inventory, args.output) - - -if __name__ == "__main__": - if len(sys.argv) == 3 and sys.argv[1] == "_exec": - # A fresh interpreter sets limits before exec; no Python preexec callback - # runs between fork and exec in the monitoring supervisor. - if not os.environ.get("SLURM_JOB_ID") or os.environ.get("SLURM_GPUS_ON_NODE", "0") != "0": - raise ValueError("CPU-only Slurm allocation required") - command_args = json.loads(sys.argv[2]) - if not command_args or any(not isinstance(value, str) for value in command_args): - raise ValueError("invalid internal command") - limits() - os.execv(command_args[0], command_args) - else: - main() diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v2/startup-controls.initial-failure.txt b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v2/startup-controls.initial-failure.txt deleted file mode 100644 index 2ab29cd33..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v2/startup-controls.initial-failure.txt +++ /dev/null @@ -1 +0,0 @@ -Local control setup failed after earlier controls completed: runtime.write uses exclusive creation, but the wrong-job fixture attempted to overwrite cap.json. FileExistsError; no Linux execution, no sandbox launch. The test must use a separate capability file. The initial source is preserved. diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v2/startup-controls.json b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v2/startup-controls.json deleted file mode 100644 index d314c5f95..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v2/startup-controls.json +++ /dev/null @@ -1 +0,0 @@ -{"passed": true, "controls": ["actual gate entry: exact token accepted; EOF, wrong, short and extra byte rejected", "held-open empty and full-token pipes time out without authorization", "foreign ABI rejects before any syscall", "fixed raw syscall binding preserves ESRCH and actual descriptor results (mocked)", "capability failure retained before launching a child", "missing or live init never gets final output hash; authorization pipe closed on cleanup", "no next case after unconfirmed cleanup; wrong-job capability rejected before bwrap"], "linux_runtime_executed": false} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v2/synthetic_probe.py b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v2/synthetic_probe.py deleted file mode 100644 index 51cb12f67..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v2/synthetic_probe.py +++ /dev/null @@ -1,529 +0,0 @@ -"""Finite handwritten kernel-boundary probes; no model programs or task data.""" - -import ctypes -import errno -import json -import mmap -import os -import platform -import resource -import select -import signal -import subprocess -import sys -import threading -import time -from pathlib import Path - -TASK_LIMIT = 16 -MEMORY_LIMIT = 2147483648 -START_TOKEN = b"inferrsi-trusted-start-v2\n" -IPC_SYSCALLS = (29, 30, 31, 64, 65, 66, 67, 68, 69, 70, 71, 220, 240) - - -def authorize_start(fd=0, timeout=5): - """Require the supervisor's complete token and EOF before any dispatch.""" - deadline = time.monotonic() + timeout - received = bytearray() - while len(received) <= len(START_TOKEN): - remaining = deadline - time.monotonic() - if remaining <= 0 or not select.select([fd], [], [], remaining)[0]: - raise TimeoutError("startup token deadline exceeded") - part = os.read(fd, len(START_TOKEN) + 1 - len(received)) - if not part: - if bytes(received) != START_TOKEN: - raise ValueError("startup token absent or invalid") - return - received.extend(part) - raise ValueError("startup token too long") - - -def filter_instructions(): - # Linux v6.12 x86 syscall table; reject other architectures and the x32 ABI. - result = [ - (0x20, 0, 0, 4), - (0x15, 1, 0, 0xC000003E), - (0x06, 0, 0, 0x80000000), - (0x20, 0, 0, 0), - (0x35, 0, 1, 0x40000000), - (0x06, 0, 0, 0x80000000), - ] - for number in IPC_SYSCALLS: - result.extend([(0x15, 0, 1, number), (0x06, 0, 0, 0x00050000 | errno.EPERM)]) - return [*result, (0x06, 0, 0, 0x7FFF0000)] - - -def install_ipc_filter(): - if platform.machine() != "x86_64" or sys.maxsize != 2**63 - 1: - raise ValueError("trusted filter requires Linux x86_64 LP64") - - class Instruction(ctypes.Structure): - _fields_ = [ - ("code", ctypes.c_ushort), - ("jt", ctypes.c_ubyte), - ("jf", ctypes.c_ubyte), - ("k", ctypes.c_uint32), - ] - - class Program(ctypes.Structure): - _fields_ = [ - ("length", ctypes.c_ushort), - ("instructions", ctypes.POINTER(Instruction)), - ] - - instructions = filter_instructions() - array = (Instruction * len(instructions))( - *(Instruction(*row) for row in instructions) - ) - program = Program(len(instructions), array) - libc = ctypes.CDLL(None, use_errno=True) - libc.prctl.restype = ctypes.c_int - if ( - libc.prctl(38, 1, 0, 0, 0) != 0 - or libc.prctl(22, 2, ctypes.byref(program), 0, 0) != 0 - ): - raise OSError(ctypes.get_errno(), "fixed seccomp policy could not be installed") - - -def ipc_attempts(): - libc = ctypes.CDLL(None, use_errno=True) - libc.syscall.restype = ctypes.c_long - attempts = [] - # Finite minimal allocations only, even if the expected rejection is absent. - for number, arguments in ( - (29, (0, 4096, 0o600)), - (64, (0, 1, 0o600)), - (68, (0, 0o600)), - ): - ctypes.set_errno(0) - value = libc.syscall( - ctypes.c_long(number), *(ctypes.c_long(v) for v in arguments) - ) - attempts.append( - {"syscall": number, "return": value, "errno": ctypes.get_errno()} - ) - ctypes.set_errno(0) - value = libc.syscall( - ctypes.c_long(240), - ctypes.c_char_p(b"trusted-probe"), - ctypes.c_long(os.O_CREAT | os.O_WRONLY), - ctypes.c_long(0o600), - ctypes.c_void_p(), - ) - attempts.append({"syscall": 240, "return": value, "errno": ctypes.get_errno()}) - return attempts - - -def emit(value): - print(json.dumps(value, sort_keys=True), flush=True) - - -def limits_record(): - return { - name: list(resource.getrlimit(getattr(resource, "RLIMIT_" + name))) - for name in ( - "AS", - "CPU", - "NOFILE", - "FSIZE", - "CORE", - "NPROC", - "MEMLOCK", - "MSGQUEUE", - ) - } - - -def finite_forks(): - children = [] - read_fd, write_fd = os.pipe() - rejected = None - try: - for _ in range(TASK_LIMIT + 1): - try: - child = os.fork() - except OSError as error: - rejected = error.errno - break - if child == 0: - os.close(write_fd) - signal.alarm(8) - os.read(read_fd, 1) - os._exit(0) - children.append(child) - finally: - os.close(write_fd) - os.close(read_fd) - for child in children: - os.waitpid(child, 0) - return { - "created": len(children), - "rejection_errno": rejected, - "attempt_cap": TASK_LIMIT + 1, - } - - -def finite_threads(): - threads = [] - stop = threading.Event() - rejected = None - try: - for _ in range(TASK_LIMIT + 1): - thread = threading.Thread(target=stop.wait, args=(8,)) - try: - thread.start() - except RuntimeError as error: - rejected = str(error) - break - threads.append(thread) - finally: - stop.set() - for thread in threads: - thread.join(timeout=2) - return { - "created": len(threads), - "rejection": rejected, - "all_joined": all(not thread.is_alive() for thread in threads), - "attempt_cap": TASK_LIMIT + 1, - } - - -def bounds(): - before = limits_record() - raises = {} - for name in ( - "NPROC", - "AS", - "CPU", - "NOFILE", - "FSIZE", - "CORE", - "MEMLOCK", - "MSGQUEUE", - ): - code = getattr(resource, "RLIMIT_" + name) - soft, hard = resource.getrlimit(code) - try: - resource.setrlimit(code, (soft, hard + 1)) - except (ValueError, PermissionError) as error: - raises[name] = type(error).__name__ - else: - raises[name] = "unexpected_success" - try: - mapping = mmap.mmap(-1, MEMORY_LIMIT + mmap.PAGESIZE) - except OSError as error: - as_errno = error.errno - else: - mapping.close() - as_errno = None - inherited = subprocess.run( - [sys.executable, "-I", "-B", "/opt/probe.py", "inherit"], - input=START_TOKEN, - capture_output=True, - timeout=5, - check=False, - ) - inherited_payload = json.loads(inherited.stdout) - forks = finite_forks() - threads = finite_threads() - descriptors = [] - fd_errno = None - try: - for _ in range(65): - try: - descriptors.append(os.open("/dev/null", os.O_RDONLY)) - except OSError as error: - fd_errno = error.errno - break - finally: - for fd in descriptors: - os.close(fd) - old_handler = signal.signal(signal.SIGXFSZ, signal.SIG_IGN) - try: - with open("/work/size-check", "wb") as stream: - try: - stream.truncate(1048577) - except OSError as error: - size_errno = error.errno - else: - size_errno = None - finally: - signal.signal(signal.SIGXFSZ, old_handler) - cpu = subprocess.run( - [sys.executable, "-I", "-B", "/opt/probe.py", "cpu"], - input=START_TOKEN, - capture_output=True, - timeout=5, - check=False, - ) - status = dict( - line.split(":", 1) - for line in Path("/proc/self/status").read_text().splitlines() - if ":" in line - ) - expected = { - "AS": [MEMORY_LIMIT] * 2, - "CPU": [30] * 2, - "NOFILE": [64] * 2, - "FSIZE": [1048576] * 2, - "CORE": [0, 0], - "NPROC": [TASK_LIMIT] * 2, - "MEMLOCK": [0, 0], - "MSGQUEUE": [0, 0], - } - ipc = ipc_attempts() - try: - with open("/dev/creation-check", "xb"): - pass - except OSError as error: - dev_errno = error.errno - else: - dev_errno = None - with open("/dev/zero", "rb") as stream: - zero = stream.read(4) - checks = { - "exact_limits": before == expected and limits_record() == expected, - "nonroot": os.getuid() == os.geteuid() == 65534, - "no_effective_capabilities": int(status["CapEff"].strip(), 16) == 0, - "no_new_privileges": status["NoNewPrivs"].strip() == "1", - "seccomp_filter_present": status["Seccomp"].strip() == "2", - "sysv_creation_denied": all( - row["return"] == -1 and row["errno"] == errno.EPERM for row in ipc - ), - "device_mount_readonly": dev_errno == errno.EROFS, - "device_read_available": zero == b"\x00" * 4, - "raise_hard_limit_denied": all( - value != "unexpected_success" for value in raises.values() - ), - "address_space_denied": as_errno == errno.ENOMEM, - "fork_exec_inheritance": inherited.returncode == 0 - and inherited_payload["limits"] == expected, - "fork_exec_filter_inheritance": inherited.returncode == 0 - and all( - row["return"] == -1 and row["errno"] == errno.EPERM - for row in inherited_payload["ipc"] - ), - "bounded_fork_denied": 0 < forks["created"] < TASK_LIMIT - and forks["rejection_errno"] == errno.EAGAIN, - "bounded_thread_denied": 0 < threads["created"] < TASK_LIMIT - and threads["rejection"] is not None - and threads["all_joined"], - "descriptor_limit_denied": fd_errno == errno.EMFILE, - "file_size_denied": size_errno == errno.EFBIG, - "cpu_limit_kills": cpu.returncode == -signal.SIGKILL, - } - emit( - { - "mode": "bounds", - "checks": checks, - "passed": all(checks.values()), - "limits": before, - "hard_limit_raise_results": raises, - "as_errno": as_errno, - "forks": forks, - "threads": threads, - "descriptor_errno": fd_errno, - "opened_descriptors": len(descriptors), - "file_size_errno": size_errno, - "cpu_exit_code": cpu.returncode, - "ipc_attempts": ipc, - "device_creation_errno": dev_errno, - "inherited_exit_code": inherited.returncode, - "generated_code_executed": False, - } - ) - return 0 if all(checks.values()) else 2 - - -def cleanup(mode): - read_fd, write_fd = os.pipe() - direct = os.fork() - if direct == 0: - os.close(read_fd) - os.write(write_fd, f"{os.getpid()}\n".encode()) - os.close(write_fd) - time.sleep(15) - os._exit(0) - intermediate = os.fork() - if intermediate == 0: - os.close(read_fd) - os.setsid() - grandchild = os.fork() - if grandchild == 0: - os.write(write_fd, f"{os.getpid()}\n".encode()) - os.close(write_fd) - time.sleep(15) - os._exit(0) - os._exit(0) - os.close(write_fd) - with os.fdopen(read_fd, "r") as stream: - live = sorted(int(line) for line in stream) - os.waitpid(intermediate, 0) - emit( - { - "mode": mode, - "ready": True, - "probe_pid": os.getpid(), - "live_descendant_pids": live, - "direct_pid": direct, - "namespace_pid": os.readlink("/proc/self/ns/pid"), - "generated_code_executed": False, - } - ) - if mode == "normal": - # Allow the supervisor to open pidfds for both deliberately live children. - time.sleep(2) - return 0 - time.sleep(15) - return 3 - - -def synthetic_evaluator(): - import base64 - import contextlib - import hashlib - import importlib - import io - - sys.path.insert(0, "/opt/site-packages") - import psutil - - def digest(raw): - return hashlib.sha256(raw).hexdigest() - - os.environ["EVALPLUS_MAX_MEMORY_BYTES"] = str(MEMORY_LIMIT) - os.environ.pop("EVALPLUS_TIMEOUT_PER_TASK", None) - os.environ["XDG_CACHE_HOME"] = "/work/cache" - os.environ["HUMANEVAL_OVERRIDE_PATH"] = "/work/synthetic-dataset.jsonl" - work = Path("/work") - samples = work / "synthetic-samples.jsonl" - dataset = work / "synthetic-dataset.jsonl" - output = work / "output/synthetic-results.json" - assert not samples.exists() and not dataset.exists() and not output.exists() - assert not (work / "cache").exists() - solutions = { - "Synthetic/correct": "def f(x):\n return x + 1\n", - "Synthetic/plus_wrong": "def f(x):\n return 1\n", - "Synthetic/exception": "def f(x):\n raise ArithmeticError('synthetic')\n", - "Synthetic/input_timeout": "import time\ndef f(x):\n time.sleep(5)\n return x + 1\n", - } - rows = [ - { - "task_id": task_id, - "prompt": "def f(x):\n", - "canonical_solution": " return x + 1\n", - "contract": "", - "entry_point": "f", - "base_input": [[0]], - "plus_input": [[1]], - "atol": 0, - } - for task_id in solutions - ] - dataset.write_text("".join(json.dumps(row, sort_keys=True) + "\n" for row in rows)) - samples.write_text( - "".join( - json.dumps({"task_id": key, "solution": value}, sort_keys=True) + "\n" - for key, value in solutions.items() - ) - ) - parent = psutil.Process() - captured_out, captured_err = io.StringIO(), io.StringIO() - started = time.monotonic() - with ( - contextlib.redirect_stdout(captured_out), - contextlib.redirect_stderr(captured_err), - ): - evaluator = importlib.import_module("evalplus.evaluate") - evaluator.evaluate( - dataset="humaneval", - samples=str(samples), - output_file=str(output), - parallel=1, - base_only=False, - mini=False, - noextreme=False, - test_details=True, - ) - result = json.loads(output.read_bytes()) - statuses = { - task_id: [[row["base_status"], row["plus_status"]] for row in values] - for task_id, values in result["eval"].items() - } - # The upstream per-input exception handler reports time-limit exceptions as - # fail. This is distinct from its outer child-process timeout classification. - expected = { - "Synthetic/correct": [["pass", "pass"]], - "Synthetic/plus_wrong": [["pass", "fail"]], - "Synthetic/exception": [["fail", "fail"]], - "Synthetic/input_timeout": [["fail", "fail"]], - } - checks = { - "four_exact_status_pairs": statuses == expected, - "dataset_hash": result["hash"] == hashlib.md5(dataset.read_bytes()).hexdigest(), - "bounded_output": output.stat().st_size <= 1048576, - "timeout_override_absent": "EVALPLUS_TIMEOUT_PER_TASK" not in os.environ, - } - payload = { - "checks": checks, - "passed": all(checks.values()), - "statuses": statuses, - "elapsed_seconds": time.monotonic() - started, - "dataset_sha256": digest(dataset.read_bytes()), - "samples_sha256": digest(samples.read_bytes()), - "results_sha256": digest(output.read_bytes()), - "results": result, - "results_raw_base64": base64.b64encode(output.read_bytes()).decode("ascii"), - "evaluator_module_sha256": digest(Path(evaluator.__file__).read_bytes()), - "evaluator_stdout": captured_out.getvalue(), - "evaluator_stderr": captured_err.getvalue(), - "live_children_after_evaluate": len(parent.children(recursive=True)), - "generated_code_executed": False, - "real_benchmark_data_mounted": False, - "full_evaluator_executed": True, - "qualified_generated_code_executor": False, - } - print(json.dumps(payload, sort_keys=True)) - return 0 if payload["passed"] else 2 - - -def main(): - try: - authorize_start() - except (ValueError, OSError): - os.write(2, b"PAYLOAD_START_REJECTED\n") - return 23 - # The authorization pipe is never available to evaluator code or children. - devnull = os.open("/dev/null", os.O_RDONLY) - os.dup2(devnull, 0) - if devnull != 0: - os.close(devnull) - os.write(2, b"PAYLOAD_START_AUTHORIZED\n") - mode = sys.argv[1] - if mode == "gate_check": - emit({"passed": True, "mode": "gate_check"}) - return 0 - if mode == "inherit": - emit({"limits": limits_record(), "ipc": ipc_attempts()}) - return 0 - if mode == "cpu": - resource.setrlimit(resource.RLIMIT_CPU, (1, 1)) - signal.alarm(4) - while True: - pass - resource.setrlimit(resource.RLIMIT_NPROC, (TASK_LIMIT, TASK_LIMIT)) - resource.setrlimit(resource.RLIMIT_MEMLOCK, (0, 0)) - resource.setrlimit(resource.RLIMIT_MSGQUEUE, (0, 0)) - install_ipc_filter() - signal.alarm(40) - if mode == "synthetic": - return synthetic_evaluator() - if mode == "bounds": - return bounds() - if mode in ("normal", "timeout", "launcher_kill"): - return cleanup(mode) - raise ValueError("unknown trusted probe mode") - - -if __name__ == "__main__": - raise SystemExit(main()) diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v2/test_benign_reference.py b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v2/test_benign_reference.py deleted file mode 100644 index cbdc89c08..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v2/test_benign_reference.py +++ /dev/null @@ -1,65 +0,0 @@ -"""Check the exact embedded reference validator with observed and mutated receipts.""" -import copy -import hashlib -import json -import subprocess -import sys -import tempfile -from pathlib import Path - -root = Path(__file__).parent -wrapper = (root / 'run-synthetic.sbatch').read_text() -code = wrapper.split("<<'PYBENIGN'\n", 1)[1].split('\nPYBENIGN', 1)[0] -reference = root / 'reference.private' -original_execution = json.loads((reference / 'probe/execution.json').read_bytes()) -original_stdout = json.loads((reference / 'probe/stdout.json').read_bytes()) -rootfs = json.loads((reference / 'rootfs-before/rootfs.json').read_bytes()) -bwrap = rootfs['executor']['binary_sha256'] -cases = ['observed_success', 'exit_bool', 'exit_nonzero', 'timeout', 'wrong_phase', - 'wrong_output_digest', 'wrong_rootfs', 'wrong_executor', 'empty_checks', - 'missing_check', 'integer_check', 'integer_passed', 'duplicate_key'] -with tempfile.TemporaryDirectory(dir=root) as temporary: - fixture = Path(temporary) - (fixture / 'probe').mkdir() - (fixture / 'rootfs-before').mkdir() - reference_rootfs = (reference / 'rootfs-before/rootfs.json').read_bytes() - (fixture / 'rootfs-before/rootfs.json').write_bytes(reference_rootfs) - for name in cases: - execution, stdout = copy.deepcopy(original_execution), copy.deepcopy(original_stdout) - if name == 'exit_bool': - execution['exit_code'] = False - elif name == 'exit_nonzero': - execution['exit_code'] = 1 - elif name == 'timeout': - execution['timed_out'] = True - elif name == 'wrong_phase': - execution['phase'] = 'unrelated' - elif name == 'wrong_rootfs': - execution['rootfs_digest'] = 'sha256:' + '0' * 64 - elif name == 'wrong_executor': - execution['executor']['binary_sha256'] = '0' * 64 - elif name == 'empty_checks': - stdout['checks'] = {} - elif name == 'missing_check': - del stdout['checks']['new_namespaces'] - elif name == 'integer_check': - stdout['checks']['new_namespaces'] = 1 - elif name == 'integer_passed': - stdout['passed'] = 1 - encoded = json.dumps(stdout).encode() - execution['stdout_sha256'] = hashlib.sha256(encoded).hexdigest() - if name == 'wrong_output_digest': - execution['stdout_sha256'] = '0' * 64 - encoded_execution = json.dumps(execution) - if name == 'duplicate_key': - encoded_execution = '{"exit_code":1,' + encoded_execution[1:] - (fixture / 'probe/execution.json').write_text(encoded_execution) - (fixture / 'probe/stdout.json').write_bytes(encoded) - result = subprocess.run([sys.executable, '-IBO', '-c', code, str(fixture), bwrap], - capture_output=True, check=False, timeout=5) - if (result.returncode == 0) != (name == 'observed_success'): - raise RuntimeError('Unexpected reference-validation outcome: ' + name) -report = {'benign_reference_cases': len(cases), 'cases': cases, 'optimized_python': True, - 'evaluator_or_sandbox_executed': False, 'passed': True} -(root / 'benign-reference-controls.json').write_text(json.dumps(report, indent=2) + '\n') -print(json.dumps(report)) diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v2/test_controls.py b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v2/test_controls.py deleted file mode 100644 index e365074b7..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v2/test_controls.py +++ /dev/null @@ -1,278 +0,0 @@ -"""Local controls exercise decision logic only; no Linux sandbox is executed.""" - -import ast -import errno -import json -import os -import subprocess -import sys -import tempfile -import time -import types -from pathlib import Path -from unittest.mock import Mock, patch - -ROOT = Path(__file__).resolve().parent - - -def require(value, message): - if not value: - raise ValueError(message) - - -def main(): - # Load just the filter constructor, never the finite process/IPC probes. - module = ast.parse((ROOT / "synthetic_probe.py").read_text()) - selected = [ - node - for node in module.body - if (isinstance(node, ast.FunctionDef) and node.name == "filter_instructions") - or ( - isinstance(node, ast.Assign) - and any( - isinstance(target, ast.Name) and target.id == "IPC_SYSCALLS" - for target in node.targets - ) - ) - ] - namespace = {"errno": errno} - exec( - compile(ast.Module(body=selected, type_ignores=[]), "filter-control", "exec"), - namespace, - ) - program = namespace["filter_instructions"]() - - def evaluate(architecture, number): - accumulator = 0 - pc = 0 - while pc < len(program): - opcode, yes, no, value = program[pc] - if opcode == 0x20: - accumulator = {0: number, 4: architecture}[value] - elif opcode == 0x15: - pc += yes if accumulator == value else no - elif opcode == 0x35: - pc += yes if accumulator >= value else no - elif opcode == 0x06: - return value - else: - raise ValueError("unexpected BPF opcode") - pc += 1 - raise ValueError("filter failed to return") - - expected_denied = {29, 30, 31, 64, 65, 66, 67, 68, 69, 70, 71, 220, 240} - controls = [] - for number in range(451): - expected = 0x50000 | errno.EPERM if number in expected_denied else 0x7FFF0000 - require(evaluate(0xC000003E, number) == expected, f"syscall {number}") - controls.append("all native syscall decisions 0 through 450") - for architecture, number in ( - (0x40000003, 117), - (0xC00000B7, 29), - (0, 0), - (0xC000003E, 0x40000000), - (0xC000003E, 0x4000001D), - ): - require( - evaluate(architecture, number) == 0x80000000, "ABI mismatch was not killed" - ) - controls.append("three other architectures and two x32 calls denied") - - # psutil is intentionally not installed in this local tooling environment. - # These controls never use it; the registered cluster runtime already has it. - with patch.dict(sys.modules, {"psutil": types.ModuleType("psutil")}): - sys.path.insert(0, str(ROOT)) - import bounds_driver as driver - - for pairs in ((("x", 1), ("x", 2)),): - try: - driver.unique(pairs) - except ValueError: - pass - else: - raise ValueError("duplicate JSON keys accepted") - controls.append("duplicate status keys rejected") - fields = {"PPid": "10", "NSpid": "20 1"} - record = {"child-pid": 20, "pid-namespace": 99} - with ( - patch.object(driver, "pidfd_open", return_value=77), - patch.object(os, "close"), - patch.object(os, "readlink", return_value="pid:[99]"), - patch.object(driver, "status", return_value=fields), - patch.object(driver, "exited", return_value=False), - ): - held, identity = driver.bind_init(record, 10) - require( - held == 77 and identity["namespace_pids"] == [20, 1], "valid init rejected" - ) - controls.append("valid namespace-init identity accepted") - cases = ( - ({"child-pid": True, "pid-namespace": 99}, fields, False, "pid:[99]"), - (record, {"PPid": "11", "NSpid": "20 1"}, False, "pid:[99]"), - (record, {"PPid": "10", "NSpid": "20 2"}, False, "pid:[99]"), - (record, fields, True, "pid:[99]"), - (record, fields, False, "pid:[100]"), - ) - for value, observed, dead, link in cases: - with ( - patch.object(driver, "pidfd_open", return_value=77), - patch.object(os, "close"), - patch.object(os, "readlink", return_value=link), - patch.object(driver, "status", return_value=observed), - patch.object(driver, "exited", return_value=dead), - ): - try: - driver.bind_init(value, 10) - except ValueError: - pass - else: - raise ValueError("invalid namespace-init identity accepted") - controls.append("five invalid init identities rejected") - with ( - patch.object(driver, "pidfd_open", return_value=77), - patch.object(os, "close") as closed, - patch.object(os, "readlink", return_value="pid:[99]"), - patch.object(driver, "status", return_value=fields), - patch.object(driver, "exited", side_effect=[False, True]), - ): - try: - driver.bind_init(record, 10) - except ValueError: - closed.assert_called_once_with(77) - else: - raise ValueError("init exit during binding accepted") - controls.append("exit during identity binding rejected and pidfd closed") - for end in ("eof", "timeout"): - read_fd, write_fd = os.pipe() - raw = bytearray() - try: - os.write(write_fd, b'{"child-') - if end == "eof": - os.close(write_fd) - write_fd = -1 - try: - driver.read_line(read_fd, time.monotonic() + 0.05, raw) - except ValueError: - require(raw == b'{"child-', "partial status bytes lost") - else: - raise ValueError("incomplete status line accepted") - finally: - os.close(read_fd) - if write_fd >= 0: - os.close(write_fd) - controls.append("partial status retained on EOF and deadline") - for scenario in ("exit_race", "wait_timeout", "stdout_failure"): - with tempfile.TemporaryDirectory(dir=ROOT) as temporary: - output = Path(temporary) - (output / "stdout.json").write_bytes(b'{"partial":true}\n') - (output / "stderr.private.log").write_bytes(b"") - report = {"passed": False, "error": {"type": "OriginalProbeFailure"}} - process = Mock() - process.returncode = None if scenario == "wait_timeout" else 0 - process.poll.return_value = process.returncode - process.wait.side_effect = subprocess.TimeoutExpired("trusted", 5) - with ( - patch.object( - driver, "exited", side_effect=[False, True, True, True] - ) as observed, - patch.object( - driver, "pidfd_send_signal", side_effect=ProcessLookupError - ), - patch.object(os, "close") as closed, - ): - if scenario != "exit_race": - observed.side_effect = None - observed.return_value = True - if scenario == "stdout_failure": - with patch.object( - Path, - "read_bytes", - side_effect=OSError("synthetic read failure"), - ): - driver.finalize_case( - report, - process, - 77, - [], - [80, -1], - output, - bytearray(b"partial-status"), - {"rootfs_digest": "bound"}, - ) - else: - driver.finalize_case( - report, - process, - 77, - [], - [80, -1], - output, - bytearray(b"partial-status"), - {"rootfs_digest": "bound"}, - ) - require(closed.call_count == 2, "later descriptors were not closed") - actual = json.loads((output / "report.json").read_text()) - require( - actual["error"] == report["error"] and not actual["passed"], - "original failure lost", - ) - require( - (output / "status.private.jsonl").read_bytes() == b"partial-status", - "status lost", - ) - errors = actual["cleanup_or_evidence_errors"] - if scenario == "exit_race": - require( - actual["signal_race_already_exited"] and not errors, - "exit race not confirmed", - ) - elif scenario == "wait_timeout": - require( - any(row["type"] == "TimeoutExpired" for row in errors), - "wait failure lost", - ) - require( - actual["outputs_stable"] is False - and actual["cleanup_complete"] is False - and actual["stdout_sha256"] is None - and actual["stderr_sha256"] is None, - "live launcher incorrectly got stable final hashes", - ) - else: - require( - actual["stdout_sha256"] is None - and any(row["operation"] == "read_stdout" for row in errors), - "read failure lost", - ) - controls.append( - "cleanup exit race, wait timeout and stdout failure retain original errors and records" - ) - # Syntax compilation checks all sources without executing platform probes. - for name in ("runtime.py", "bounds_driver.py", "synthetic_probe.py"): - compile((ROOT / name).read_text(), name, "exec") - controls.append("three runtime sources compile") - isolated = subprocess.run( - [ - sys.executable, - "-IB", - "-c", - "import runpy,sys,types; sys.modules['psutil']=types.ModuleType('psutil'); " - "runpy.run_path(sys.argv[1], run_name='trusted_import_control')", - str(ROOT / "bounds_driver.py"), - ], - capture_output=True, - timeout=10, - ) - require(isolated.returncode == 0, isolated.stderr.decode()) - controls.append( - "isolated interpreter loads exact adjacent runtime; psutil import stubbed" - ) - print( - json.dumps( - {"passed": True, "controls": controls, "linux_runtime_executed": False} - ) - ) - - -if __name__ == "__main__": - main() diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v2/test_controls.review-before-stability.py b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v2/test_controls.review-before-stability.py deleted file mode 100644 index 72f95b9a8..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v2/test_controls.review-before-stability.py +++ /dev/null @@ -1,271 +0,0 @@ -"""Local controls exercise decision logic only; no Linux sandbox is executed.""" - -import ast -import errno -import json -import os -import subprocess -import sys -import tempfile -import time -import types -from pathlib import Path -from unittest.mock import Mock, patch - -ROOT = Path(__file__).resolve().parent - - -def require(value, message): - if not value: - raise ValueError(message) - - -def main(): - # Load just the filter constructor, never the finite process/IPC probes. - module = ast.parse((ROOT / "synthetic_probe.py").read_text()) - selected = [ - node - for node in module.body - if (isinstance(node, ast.FunctionDef) and node.name == "filter_instructions") - or ( - isinstance(node, ast.Assign) - and any( - isinstance(target, ast.Name) and target.id == "IPC_SYSCALLS" - for target in node.targets - ) - ) - ] - namespace = {"errno": errno} - exec( - compile(ast.Module(body=selected, type_ignores=[]), "filter-control", "exec"), - namespace, - ) - program = namespace["filter_instructions"]() - - def evaluate(architecture, number): - accumulator = 0 - pc = 0 - while pc < len(program): - opcode, yes, no, value = program[pc] - if opcode == 0x20: - accumulator = {0: number, 4: architecture}[value] - elif opcode == 0x15: - pc += yes if accumulator == value else no - elif opcode == 0x35: - pc += yes if accumulator >= value else no - elif opcode == 0x06: - return value - else: - raise ValueError("unexpected BPF opcode") - pc += 1 - raise ValueError("filter failed to return") - - expected_denied = {29, 30, 31, 64, 65, 66, 67, 68, 69, 70, 71, 220, 240} - controls = [] - for number in range(451): - expected = 0x50000 | errno.EPERM if number in expected_denied else 0x7FFF0000 - require(evaluate(0xC000003E, number) == expected, f"syscall {number}") - controls.append("all native syscall decisions 0 through 450") - for architecture, number in ( - (0x40000003, 117), - (0xC00000B7, 29), - (0, 0), - (0xC000003E, 0x40000000), - (0xC000003E, 0x4000001D), - ): - require( - evaluate(architecture, number) == 0x80000000, "ABI mismatch was not killed" - ) - controls.append("three other architectures and two x32 calls denied") - - # psutil is intentionally not installed in this local tooling environment. - # These controls never use it; the registered cluster runtime already has it. - with patch.dict(sys.modules, {"psutil": types.ModuleType("psutil")}): - sys.path.insert(0, str(ROOT)) - import bounds_driver as driver - - for pairs in ((("x", 1), ("x", 2)),): - try: - driver.unique(pairs) - except ValueError: - pass - else: - raise ValueError("duplicate JSON keys accepted") - controls.append("duplicate status keys rejected") - fields = {"PPid": "10", "NSpid": "20 1"} - record = {"child-pid": 20, "pid-namespace": 99} - with ( - patch.object(driver, "pidfd_open", return_value=77), - patch.object(os, "close"), - patch.object(os, "readlink", return_value="pid:[99]"), - patch.object(driver, "status", return_value=fields), - patch.object(driver, "exited", return_value=False), - ): - held, identity = driver.bind_init(record, 10) - require( - held == 77 and identity["namespace_pids"] == [20, 1], "valid init rejected" - ) - controls.append("valid namespace-init identity accepted") - cases = ( - ({"child-pid": True, "pid-namespace": 99}, fields, False, "pid:[99]"), - (record, {"PPid": "11", "NSpid": "20 1"}, False, "pid:[99]"), - (record, {"PPid": "10", "NSpid": "20 2"}, False, "pid:[99]"), - (record, fields, True, "pid:[99]"), - (record, fields, False, "pid:[100]"), - ) - for value, observed, dead, link in cases: - with ( - patch.object(driver, "pidfd_open", return_value=77), - patch.object(os, "close"), - patch.object(os, "readlink", return_value=link), - patch.object(driver, "status", return_value=observed), - patch.object(driver, "exited", return_value=dead), - ): - try: - driver.bind_init(value, 10) - except ValueError: - pass - else: - raise ValueError("invalid namespace-init identity accepted") - controls.append("five invalid init identities rejected") - with ( - patch.object(driver, "pidfd_open", return_value=77), - patch.object(os, "close") as closed, - patch.object(os, "readlink", return_value="pid:[99]"), - patch.object(driver, "status", return_value=fields), - patch.object(driver, "exited", side_effect=[False, True]), - ): - try: - driver.bind_init(record, 10) - except ValueError: - closed.assert_called_once_with(77) - else: - raise ValueError("init exit during binding accepted") - controls.append("exit during identity binding rejected and pidfd closed") - for end in ("eof", "timeout"): - read_fd, write_fd = os.pipe() - raw = bytearray() - try: - os.write(write_fd, b'{"child-') - if end == "eof": - os.close(write_fd) - write_fd = -1 - try: - driver.read_line(read_fd, time.monotonic() + 0.05, raw) - except ValueError: - require(raw == b'{"child-', "partial status bytes lost") - else: - raise ValueError("incomplete status line accepted") - finally: - os.close(read_fd) - if write_fd >= 0: - os.close(write_fd) - controls.append("partial status retained on EOF and deadline") - for scenario in ("exit_race", "wait_timeout", "stdout_failure"): - with tempfile.TemporaryDirectory(dir=ROOT) as temporary: - output = Path(temporary) - (output / "stdout.json").write_bytes(b'{"partial":true}\n') - (output / "stderr.private.log").write_bytes(b"") - report = {"passed": False, "error": {"type": "OriginalProbeFailure"}} - process = Mock() - process.returncode = None if scenario == "wait_timeout" else 0 - process.poll.return_value = process.returncode - process.wait.side_effect = subprocess.TimeoutExpired("trusted", 5) - with ( - patch.object( - driver, "exited", side_effect=[False, True, True, True] - ) as observed, - patch.object( - driver, "pidfd_send_signal", side_effect=ProcessLookupError - ), - patch.object(os, "close") as closed, - ): - if scenario != "exit_race": - observed.side_effect = None - observed.return_value = True - if scenario == "stdout_failure": - with patch.object( - Path, - "read_bytes", - side_effect=OSError("synthetic read failure"), - ): - driver.finalize_case( - report, - process, - 77, - [], - [80, -1], - output, - bytearray(b"partial-status"), - {"rootfs_digest": "bound"}, - ) - else: - driver.finalize_case( - report, - process, - 77, - [], - [80, -1], - output, - bytearray(b"partial-status"), - {"rootfs_digest": "bound"}, - ) - require(closed.call_count == 2, "later descriptors were not closed") - actual = json.loads((output / "report.json").read_text()) - require( - actual["error"] == report["error"] and not actual["passed"], - "original failure lost", - ) - require( - (output / "status.private.jsonl").read_bytes() == b"partial-status", - "status lost", - ) - errors = actual["cleanup_or_evidence_errors"] - if scenario == "exit_race": - require( - actual["signal_race_already_exited"] and not errors, - "exit race not confirmed", - ) - elif scenario == "wait_timeout": - require( - any(row["type"] == "TimeoutExpired" for row in errors), - "wait failure lost", - ) - else: - require( - actual["stdout_sha256"] is None - and any(row["operation"] == "read_stdout" for row in errors), - "read failure lost", - ) - controls.append( - "cleanup exit race, wait timeout and stdout failure retain original errors and records" - ) - # Syntax compilation checks all sources without executing platform probes. - for name in ("runtime.py", "bounds_driver.py", "synthetic_probe.py"): - compile((ROOT / name).read_text(), name, "exec") - controls.append("three runtime sources compile") - isolated = subprocess.run( - [ - sys.executable, - "-IB", - "-c", - "import runpy,sys,types; sys.modules['psutil']=types.ModuleType('psutil'); " - "runpy.run_path(sys.argv[1], run_name='trusted_import_control')", - str(ROOT / "bounds_driver.py"), - ], - capture_output=True, - timeout=10, - ) - require(isolated.returncode == 0, isolated.stderr.decode()) - controls.append( - "isolated interpreter loads exact adjacent runtime; psutil import stubbed" - ) - print( - json.dumps( - {"passed": True, "controls": controls, "linux_runtime_executed": False} - ) - ) - - -if __name__ == "__main__": - main() diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v2/test_reference_completion.py b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v2/test_reference_completion.py deleted file mode 100644 index 8e29fcc2e..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v2/test_reference_completion.py +++ /dev/null @@ -1,54 +0,0 @@ -"""Reference-completion checks only; no evaluator import or runtime execution.""" - -import json -import subprocess -import sys -import tempfile -from pathlib import Path - - -def main(): - root = Path(__file__).parent - script = (root / "run-synthetic.sbatch").read_text() - code = script.split("<<'PYREF'\n", 1)[1].split("\nPYREF", 1)[0] - good = {"completed": True, "signal": "none", "exit_code": 0, "job_id": "42"} - slurm = "JobId=42 JobState=COMPLETED ExitCode=0:0\n" - cases = [ - (good, slurm, "0", True), - ({**good, "completed": False}, slurm, "0", False), - ({**good, "completed": 1}, slurm, "0", False), - ({**good, "signal": "TERM", "exit_code": 143}, slurm, "143", False), - ({**good, "exit_code": False}, slurm, "0", False), - ({**good, "job_id": "41"}, slurm, "0", False), - (good, slurm.replace("COMPLETED", "TIMEOUT"), "0", False), - (good, slurm.replace("0:0", "0:15"), "0", False), - (good, slurm.replace("JobId=42", "JobId=41"), "0", False), - (good, slurm + "JobState=COMPLETED\n", "0", False), - (good, slurm + "JobState=\n", "0", False), - (good, slurm + "JobId=\n", "0", False), - (good, slurm + "ExitCode=\n", "0", False), - (good, slurm, "1", False), - (None, slurm, "0", False), - ('{"completed":false,"completed":true,"signal":"none","exit_code":0,"job_id":"42"}', slurm, "0", False), - ('{"completed":true,"signal":"none","exit_code":143,"exit_code":0,"job_id":"42"}', slurm, "0", False), - ('{"completed":true,"signal":"none","exit_code":0,"job_id":"41","job_id":"42"}', slurm, "0", False), - ] - with tempfile.TemporaryDirectory(prefix="reference-test-", dir=root) as temporary: - reference = Path(temporary) - for terminal, scheduler, marker, expected in cases: - path = reference / "terminal-status.json" - if terminal is None: - path.unlink(missing_ok=True) - else: - path.write_text(terminal if isinstance(terminal, str) else json.dumps(terminal)) - (reference / "slurm-final.private.txt").write_text(scheduler) - (reference / "exit-code").write_text(marker) - result = subprocess.run([sys.executable, "-IBO", "-c", code, str(reference), "42"], - capture_output=True, check=False, timeout=5) - if (result.returncode == 0) != expected: - raise RuntimeError("Reference completion check mismatch") - print(json.dumps({"reference_completion_cases": len(cases), "optimized_python": True})) - - -if __name__ == "__main__": - main() diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v2/test_startup.initial-exclusive-fixture.py b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v2/test_startup.initial-exclusive-fixture.py deleted file mode 100644 index 88fde6b88..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v2/test_startup.initial-exclusive-fixture.py +++ /dev/null @@ -1,269 +0,0 @@ -"""Local startup/cleanup controls; actual Linux capability remains a separate probe.""" - -import ast -import ctypes -import errno -import importlib.util -import json -import os -import select -import subprocess -import sys -import tempfile -import time -import types -from pathlib import Path -from unittest.mock import Mock, patch - -ROOT = Path(__file__).resolve().parent - - -def require(value, message): - if not value: - raise ValueError(message) - - -def main(): - with patch.dict(sys.modules, {"psutil": types.ModuleType("psutil")}): - spec = importlib.util.spec_from_file_location( - "startup_driver", ROOT / "bounds_driver.py" - ) - driver = importlib.util.module_from_spec(spec) - spec.loader.exec_module(driver) - controls = [] - for data, expected in ( - (driver.START_TOKEN, 0), - (b"", 23), - (b"wrong\n", 23), - (driver.START_TOKEN[:-1], 23), - (driver.START_TOKEN + b"x", 23), - ): - run = subprocess.run( - [sys.executable, "-IB", str(ROOT / "synthetic_probe.py"), "gate_check"], - input=data, - capture_output=True, - timeout=8, - check=False, - ) - require(run.returncode == expected, "incorrect startup decision") - require( - run.stderr - == (driver.AUTHORIZED_MARKER if expected == 0 else driver.REJECTED_MARKER), - "incorrect startup marker", - ) - require( - json.loads(run.stdout) == {"mode": "gate_check", "passed": True} - if expected == 0 - else run.stdout == b"", - "unexpected payload output", - ) - controls.append( - "actual gate entry: exact token accepted; EOF, wrong, short and extra byte rejected" - ) - - source = ast.parse((ROOT / "synthetic_probe.py").read_text()) - node = next( - node - for node in source.body - if isinstance(node, ast.FunctionDef) and node.name == "authorize_start" - ) - namespace = { - "time": time, - "select": select, - "os": os, - "START_TOKEN": driver.START_TOKEN, - } - exec(compile(ast.Module(body=[node], type_ignores=[]), "gate", "exec"), namespace) - for data in (b"", driver.START_TOKEN): - read_fd, write_fd = os.pipe() - try: - if data: - os.write(write_fd, data) - try: - namespace["authorize_start"](read_fd, timeout=0.03) - except TimeoutError: - pass - else: - raise ValueError("open startup writer authorized without EOF") - finally: - os.close(read_fd) - os.close(write_fd) - controls.append( - "held-open empty and full-token pipes time out without authorization" - ) - - with patch.object(driver.ctypes, "CDLL") as library: - with patch.object(driver.sys, "platform", "darwin"): - try: - driver.pidfd_open(1) - except ValueError: - pass - else: - raise ValueError("foreign platform accepted") - library.assert_not_called() - controls.append("foreign ABI rejects before any syscall") - library = Mock() - - def failed_call(*args): - ctypes.set_errno(errno.ESRCH) - return -1 - - library.syscall.side_effect = failed_call - with ( - patch.object(driver.sys, "platform", "linux"), - patch.object(driver.platform, "machine", return_value="x86_64"), - patch.object(driver.ctypes, "CDLL", return_value=library), - ): - try: - driver.pidfd_send_signal(10, 9) - except ProcessLookupError as error: - require(error.errno == errno.ESRCH, "errno lost") - else: - raise ValueError("syscall error accepted") - require(library.syscall.call_args.args[0].value == 424, "wrong syscall number") - library.syscall.side_effect = None - library.syscall.return_value = 55 - require(driver.pidfd_open(123) == 55, "pidfd result lost") - require( - library.syscall.call_args.args[0].value == 434, "wrong open syscall number" - ) - controls.append( - "fixed raw syscall binding preserves ESRCH and actual descriptor results (mocked)" - ) - - with tempfile.TemporaryDirectory(dir=ROOT) as temporary: - output = Path(temporary) - with ( - patch.dict(os.environ, {"SLURM_JOB_ID": "control"}), - patch.object( - driver, "pidfd_open", side_effect=OSError(errno.ENOSYS, "unavailable") - ), - patch.object(driver.subprocess, "Popen") as launch, - ): - require( - driver.capability(output / "capability.json") == 2, - "failed API accepted", - ) - launch.assert_not_called() - record = json.loads((output / "capability.json").read_bytes()) - require( - record["passed"] is False and record["error"]["errno"] == errno.ENOSYS, - "capability failure lost", - ) - controls.append("capability failure retained before launching a child") - - for bound, observed in ((False, False), (True, False), (True, True)): - with tempfile.TemporaryDirectory(dir=ROOT) as temporary: - output = Path(temporary) - (output / "stdout.json").write_bytes(b"late-output") - (output / "stderr.private.log").write_bytes(b"") - process = Mock() - process.returncode = 0 - process.poll.return_value = 0 - report = {"passed": True} - with ( - patch.object(driver, "exited", return_value=observed), - patch.object(driver, "pidfd_send_signal"), - patch.object(os, "close"), - ): - driver.finalize_case( - report, - process, - 77 if bound else None, - [], - [], - output, - bytearray(), - {"rootfs_digest": "control"}, - ) - require( - report["outputs_stable"] is observed, - "unobserved namespace exit marked stable", - ) - require( - (report["stdout_sha256"] is not None) is observed, - "unstable output got final hash", - ) - require(report["passed"] is observed, "unconfirmed cleanup accepted") - process.stdin.close.assert_called_once() - controls.append( - "missing or live init never gets final output hash; authorization pipe closed on cleanup" - ) - - with ( - tempfile.TemporaryDirectory(dir=ROOT) as temporary, - patch.dict(os.environ, {"SLURM_JOB_ID": "control"}), - ): - root = Path(temporary) - binary = root / "bwrap" - binary.write_bytes(b"control") - driver.runtime.write( - root / "rootfs.json", - {"executor": {"binary_sha256": driver.runtime.sha(b"control")}}, - ) - driver.runtime.write( - root / "runtime-locators.private.json", {"bwrap": str(binary)} - ) - preflight = { - "passed": True, - "cleanup_errors": [], - "job_id": "control", - "driver_sha256": driver.runtime.sha( - (ROOT / "bounds_driver.py").read_bytes() - ), - "python_binary_sha256": driver.runtime.sha( - Path(sys._base_executable).read_bytes() - ), - "kernel_release": driver.platform.release(), - } - cap = root / "cap.json" - driver.runtime.write(cap, preflight) - argv = [ - "driver", - "probe", - "--inventory", - str(root), - "--capability", - str(cap), - "--output", - str(root / "probe"), - ] - with ( - patch.object(sys, "argv", argv), - patch.object( - driver, - "run_case", - return_value={"passed": False, "final_init_exit_observed": False}, - ) as run, - ): - require(driver.main() == 2, "incomplete suite accepted") - require( - run.call_count == 1, "next sandbox launched after unconfirmed cleanup" - ) - summary = json.loads((root / "probe/summary.json").read_bytes()) - require( - summary["unexecuted_modes"] == list(driver.MODES[1:]), - "unexecuted cases omitted", - ) - preflight["job_id"] = "different" - driver.runtime.write(cap, preflight) - with patch.object(sys, "argv", argv), patch.object(driver, "run_case") as run: - try: - driver.main() - except ValueError: - pass - else: - raise ValueError("foreign job capability accepted") - run.assert_not_called() - controls.append( - "no next case after unconfirmed cleanup; wrong-job capability rejected before bwrap" - ) - print( - json.dumps( - {"passed": True, "controls": controls, "linux_runtime_executed": False} - ) - ) - - -if __name__ == "__main__": - main() diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v2/test_startup.py b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v2/test_startup.py deleted file mode 100644 index bf9375238..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v2/test_startup.py +++ /dev/null @@ -1,275 +0,0 @@ -"""Local startup/cleanup controls; actual Linux capability remains a separate probe.""" - -import ast -import ctypes -import errno -import importlib.util -import json -import os -import select -import subprocess -import sys -import tempfile -import time -import types -from pathlib import Path -from unittest.mock import Mock, patch - -ROOT = Path(__file__).resolve().parent - - -def require(value, message): - if not value: - raise ValueError(message) - - -def main(): - with patch.dict(sys.modules, {"psutil": types.ModuleType("psutil")}): - spec = importlib.util.spec_from_file_location( - "startup_driver", ROOT / "bounds_driver.py" - ) - driver = importlib.util.module_from_spec(spec) - spec.loader.exec_module(driver) - controls = [] - for data, expected in ( - (driver.START_TOKEN, 0), - (b"", 23), - (b"wrong\n", 23), - (driver.START_TOKEN[:-1], 23), - (driver.START_TOKEN + b"x", 23), - ): - run = subprocess.run( - [sys.executable, "-IB", str(ROOT / "synthetic_probe.py"), "gate_check"], - input=data, - capture_output=True, - timeout=8, - check=False, - ) - require(run.returncode == expected, "incorrect startup decision") - require( - run.stderr - == (driver.AUTHORIZED_MARKER if expected == 0 else driver.REJECTED_MARKER), - "incorrect startup marker", - ) - require( - json.loads(run.stdout) == {"mode": "gate_check", "passed": True} - if expected == 0 - else run.stdout == b"", - "unexpected payload output", - ) - controls.append( - "actual gate entry: exact token accepted; EOF, wrong, short and extra byte rejected" - ) - - source = ast.parse((ROOT / "synthetic_probe.py").read_text()) - node = next( - node - for node in source.body - if isinstance(node, ast.FunctionDef) and node.name == "authorize_start" - ) - namespace = { - "time": time, - "select": select, - "os": os, - "START_TOKEN": driver.START_TOKEN, - } - exec(compile(ast.Module(body=[node], type_ignores=[]), "gate", "exec"), namespace) - for data in (b"", driver.START_TOKEN): - read_fd, write_fd = os.pipe() - try: - if data: - os.write(write_fd, data) - try: - namespace["authorize_start"](read_fd, timeout=0.03) - except TimeoutError: - pass - else: - raise ValueError("open startup writer authorized without EOF") - finally: - os.close(read_fd) - os.close(write_fd) - controls.append( - "held-open empty and full-token pipes time out without authorization" - ) - - with patch.object(driver.ctypes, "CDLL") as library: - with patch.object(driver.sys, "platform", "darwin"): - try: - driver.pidfd_open(1) - except ValueError: - pass - else: - raise ValueError("foreign platform accepted") - library.assert_not_called() - controls.append("foreign ABI rejects before any syscall") - library = Mock() - - def failed_call(*args): - ctypes.set_errno(errno.ESRCH) - return -1 - - library.syscall.side_effect = failed_call - with ( - patch.object(driver.sys, "platform", "linux"), - patch.object(driver.platform, "machine", return_value="x86_64"), - patch.object(driver.ctypes, "CDLL", return_value=library), - ): - try: - driver.pidfd_send_signal(10, 9) - except ProcessLookupError as error: - require(error.errno == errno.ESRCH, "errno lost") - else: - raise ValueError("syscall error accepted") - require(library.syscall.call_args.args[0].value == 424, "wrong syscall number") - library.syscall.side_effect = None - library.syscall.return_value = 55 - require(driver.pidfd_open(123) == 55, "pidfd result lost") - require( - library.syscall.call_args.args[0].value == 434, "wrong open syscall number" - ) - controls.append( - "fixed raw syscall binding preserves ESRCH and actual descriptor results (mocked)" - ) - - with tempfile.TemporaryDirectory(dir=ROOT) as temporary: - output = Path(temporary) - with ( - patch.dict(os.environ, {"SLURM_JOB_ID": "control"}), - patch.object( - driver, "pidfd_open", side_effect=OSError(errno.ENOSYS, "unavailable") - ), - patch.object(driver.subprocess, "Popen") as launch, - ): - require( - driver.capability(output / "capability.json") == 2, - "failed API accepted", - ) - launch.assert_not_called() - record = json.loads((output / "capability.json").read_bytes()) - require( - record["passed"] is False and record["error"]["errno"] == errno.ENOSYS, - "capability failure lost", - ) - controls.append("capability failure retained before launching a child") - - for bound, observed in ((False, False), (True, False), (True, True)): - with tempfile.TemporaryDirectory(dir=ROOT) as temporary: - output = Path(temporary) - (output / "stdout.json").write_bytes(b"late-output") - (output / "stderr.private.log").write_bytes(b"") - process = Mock() - process.returncode = 0 - process.poll.return_value = 0 - report = {"passed": True} - with ( - patch.object(driver, "exited", return_value=observed), - patch.object(driver, "pidfd_send_signal"), - patch.object(os, "close"), - ): - driver.finalize_case( - report, - process, - 77 if bound else None, - [], - [], - output, - bytearray(), - {"rootfs_digest": "control"}, - ) - require( - report["outputs_stable"] is observed, - "unobserved namespace exit marked stable", - ) - require( - (report["stdout_sha256"] is not None) is observed, - "unstable output got final hash", - ) - require(report["passed"] is observed, "unconfirmed cleanup accepted") - process.stdin.close.assert_called_once() - controls.append( - "missing or live init never gets final output hash; authorization pipe closed on cleanup" - ) - - with ( - tempfile.TemporaryDirectory(dir=ROOT) as temporary, - patch.dict(os.environ, {"SLURM_JOB_ID": "control"}), - ): - root = Path(temporary) - binary = root / "bwrap" - binary.write_bytes(b"control") - driver.runtime.write( - root / "rootfs.json", - {"executor": {"binary_sha256": driver.runtime.sha(b"control")}}, - ) - driver.runtime.write( - root / "runtime-locators.private.json", {"bwrap": str(binary)} - ) - preflight = { - "passed": True, - "cleanup_errors": [], - "job_id": "control", - "driver_sha256": driver.runtime.sha( - (ROOT / "bounds_driver.py").read_bytes() - ), - "python_binary_sha256": driver.runtime.sha( - Path(sys._base_executable).read_bytes() - ), - "kernel_release": driver.platform.release(), - } - cap = root / "cap.json" - driver.runtime.write(cap, preflight) - argv = [ - "driver", - "probe", - "--inventory", - str(root), - "--capability", - str(cap), - "--output", - str(root / "probe"), - ] - with ( - patch.object(sys, "argv", argv), - patch.object( - driver, - "run_case", - return_value={ - "passed": False, - "final_init_exit_observed": True, - "cleanup_complete": False, - }, - ) as run, - ): - require(driver.main() == 2, "incomplete suite accepted") - require( - run.call_count == 1, "next sandbox launched after unconfirmed cleanup" - ) - summary = json.loads((root / "probe/summary.json").read_bytes()) - require( - summary["unexecuted_modes"] == list(driver.MODES[1:]), - "unexecuted cases omitted", - ) - preflight["job_id"] = "different" - cap = root / "wrong-job-cap.json" - argv[argv.index("--capability") + 1] = str(cap) - driver.runtime.write(cap, preflight) - with patch.object(sys, "argv", argv), patch.object(driver, "run_case") as run: - try: - driver.main() - except ValueError: - pass - else: - raise ValueError("foreign job capability accepted") - run.assert_not_called() - controls.append( - "no next case after unconfirmed cleanup; wrong-job capability rejected before bwrap" - ) - print( - json.dumps( - {"passed": True, "controls": controls, "linux_runtime_executed": False} - ) - ) - - -if __name__ == "__main__": - main() diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v2/test_startup.review-before-stability.py b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v2/test_startup.review-before-stability.py deleted file mode 100644 index 11b1cf628..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bounds-v2/test_startup.review-before-stability.py +++ /dev/null @@ -1,271 +0,0 @@ -"""Local startup/cleanup controls; actual Linux capability remains a separate probe.""" - -import ast -import ctypes -import errno -import importlib.util -import json -import os -import select -import subprocess -import sys -import tempfile -import time -import types -from pathlib import Path -from unittest.mock import Mock, patch - -ROOT = Path(__file__).resolve().parent - - -def require(value, message): - if not value: - raise ValueError(message) - - -def main(): - with patch.dict(sys.modules, {"psutil": types.ModuleType("psutil")}): - spec = importlib.util.spec_from_file_location( - "startup_driver", ROOT / "bounds_driver.py" - ) - driver = importlib.util.module_from_spec(spec) - spec.loader.exec_module(driver) - controls = [] - for data, expected in ( - (driver.START_TOKEN, 0), - (b"", 23), - (b"wrong\n", 23), - (driver.START_TOKEN[:-1], 23), - (driver.START_TOKEN + b"x", 23), - ): - run = subprocess.run( - [sys.executable, "-IB", str(ROOT / "synthetic_probe.py"), "gate_check"], - input=data, - capture_output=True, - timeout=8, - check=False, - ) - require(run.returncode == expected, "incorrect startup decision") - require( - run.stderr - == (driver.AUTHORIZED_MARKER if expected == 0 else driver.REJECTED_MARKER), - "incorrect startup marker", - ) - require( - json.loads(run.stdout) == {"mode": "gate_check", "passed": True} - if expected == 0 - else run.stdout == b"", - "unexpected payload output", - ) - controls.append( - "actual gate entry: exact token accepted; EOF, wrong, short and extra byte rejected" - ) - - source = ast.parse((ROOT / "synthetic_probe.py").read_text()) - node = next( - node - for node in source.body - if isinstance(node, ast.FunctionDef) and node.name == "authorize_start" - ) - namespace = { - "time": time, - "select": select, - "os": os, - "START_TOKEN": driver.START_TOKEN, - } - exec(compile(ast.Module(body=[node], type_ignores=[]), "gate", "exec"), namespace) - for data in (b"", driver.START_TOKEN): - read_fd, write_fd = os.pipe() - try: - if data: - os.write(write_fd, data) - try: - namespace["authorize_start"](read_fd, timeout=0.03) - except TimeoutError: - pass - else: - raise ValueError("open startup writer authorized without EOF") - finally: - os.close(read_fd) - os.close(write_fd) - controls.append( - "held-open empty and full-token pipes time out without authorization" - ) - - with patch.object(driver.ctypes, "CDLL") as library: - with patch.object(driver.sys, "platform", "darwin"): - try: - driver.pidfd_open(1) - except ValueError: - pass - else: - raise ValueError("foreign platform accepted") - library.assert_not_called() - controls.append("foreign ABI rejects before any syscall") - library = Mock() - - def failed_call(*args): - ctypes.set_errno(errno.ESRCH) - return -1 - - library.syscall.side_effect = failed_call - with ( - patch.object(driver.sys, "platform", "linux"), - patch.object(driver.platform, "machine", return_value="x86_64"), - patch.object(driver.ctypes, "CDLL", return_value=library), - ): - try: - driver.pidfd_send_signal(10, 9) - except ProcessLookupError as error: - require(error.errno == errno.ESRCH, "errno lost") - else: - raise ValueError("syscall error accepted") - require(library.syscall.call_args.args[0].value == 424, "wrong syscall number") - library.syscall.side_effect = None - library.syscall.return_value = 55 - require(driver.pidfd_open(123) == 55, "pidfd result lost") - require( - library.syscall.call_args.args[0].value == 434, "wrong open syscall number" - ) - controls.append( - "fixed raw syscall binding preserves ESRCH and actual descriptor results (mocked)" - ) - - with tempfile.TemporaryDirectory(dir=ROOT) as temporary: - output = Path(temporary) - with ( - patch.dict(os.environ, {"SLURM_JOB_ID": "control"}), - patch.object( - driver, "pidfd_open", side_effect=OSError(errno.ENOSYS, "unavailable") - ), - patch.object(driver.subprocess, "Popen") as launch, - ): - require( - driver.capability(output / "capability.json") == 2, - "failed API accepted", - ) - launch.assert_not_called() - record = json.loads((output / "capability.json").read_bytes()) - require( - record["passed"] is False and record["error"]["errno"] == errno.ENOSYS, - "capability failure lost", - ) - controls.append("capability failure retained before launching a child") - - for bound, observed in ((False, False), (True, False), (True, True)): - with tempfile.TemporaryDirectory(dir=ROOT) as temporary: - output = Path(temporary) - (output / "stdout.json").write_bytes(b"late-output") - (output / "stderr.private.log").write_bytes(b"") - process = Mock() - process.returncode = 0 - process.poll.return_value = 0 - report = {"passed": True} - with ( - patch.object(driver, "exited", return_value=observed), - patch.object(driver, "pidfd_send_signal"), - patch.object(os, "close"), - ): - driver.finalize_case( - report, - process, - 77 if bound else None, - [], - [], - output, - bytearray(), - {"rootfs_digest": "control"}, - ) - require( - report["outputs_stable"] is observed, - "unobserved namespace exit marked stable", - ) - require( - (report["stdout_sha256"] is not None) is observed, - "unstable output got final hash", - ) - require(report["passed"] is observed, "unconfirmed cleanup accepted") - process.stdin.close.assert_called_once() - controls.append( - "missing or live init never gets final output hash; authorization pipe closed on cleanup" - ) - - with ( - tempfile.TemporaryDirectory(dir=ROOT) as temporary, - patch.dict(os.environ, {"SLURM_JOB_ID": "control"}), - ): - root = Path(temporary) - binary = root / "bwrap" - binary.write_bytes(b"control") - driver.runtime.write( - root / "rootfs.json", - {"executor": {"binary_sha256": driver.runtime.sha(b"control")}}, - ) - driver.runtime.write( - root / "runtime-locators.private.json", {"bwrap": str(binary)} - ) - preflight = { - "passed": True, - "cleanup_errors": [], - "job_id": "control", - "driver_sha256": driver.runtime.sha( - (ROOT / "bounds_driver.py").read_bytes() - ), - "python_binary_sha256": driver.runtime.sha( - Path(sys._base_executable).read_bytes() - ), - "kernel_release": driver.platform.release(), - } - cap = root / "cap.json" - driver.runtime.write(cap, preflight) - argv = [ - "driver", - "probe", - "--inventory", - str(root), - "--capability", - str(cap), - "--output", - str(root / "probe"), - ] - with ( - patch.object(sys, "argv", argv), - patch.object( - driver, - "run_case", - return_value={"passed": False, "final_init_exit_observed": False}, - ) as run, - ): - require(driver.main() == 2, "incomplete suite accepted") - require( - run.call_count == 1, "next sandbox launched after unconfirmed cleanup" - ) - summary = json.loads((root / "probe/summary.json").read_bytes()) - require( - summary["unexecuted_modes"] == list(driver.MODES[1:]), - "unexecuted cases omitted", - ) - preflight["job_id"] = "different" - cap = root / "wrong-job-cap.json" - argv[argv.index("--capability") + 1] = str(cap) - driver.runtime.write(cap, preflight) - with patch.object(sys, "argv", argv), patch.object(driver, "run_case") as run: - try: - driver.main() - except ValueError: - pass - else: - raise ValueError("foreign job capability accepted") - run.assert_not_called() - controls.append( - "no next case after unconfirmed cleanup; wrong-job capability rejected before bwrap" - ) - print( - json.dumps( - {"passed": True, "controls": controls, "linux_runtime_executed": False} - ) - ) - - -if __name__ == "__main__": - main() diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-build-reviewed.sbatch b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-build-reviewed.sbatch deleted file mode 100644 index 24b9366ba..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-build-reviewed.sbatch +++ /dev/null @@ -1,79 +0,0 @@ -#!/usr/bin/env bash -# Build only two reviewed source archives; no evaluator or generated-code execution. -#SBATCH --job-name=qwen38-eval-reviewed-wheels -#SBATCH --nodes=1 -#SBATCH --ntasks=1 -#SBATCH --cpus-per-task=4 -#SBATCH --mem=16G -#SBATCH --time=00:20:00 -set -euo pipefail -umask 077 -: "${TASK_ROOT:?}" -: "${BASE_PYTHON:?}" -: "${UV_BIN:?}" -: "${BUILD_SOURCE_DIR:?}" -: "${BUILD_BUNDLE:?}" -: "${BUILD_VENV:?}" -: "${BUILD_INPUTS_SHA256:?}" -: "${BUILD_SCRIPT_SHA256:?}" -task_uv_executable=$UV_BIN -[[ -n ${SLURM_JOB_ID:-} && ${SLURM_GPUS_ON_NODE:-0} == 0 ]] -root=$(realpath -e "$TASK_ROOT") -check_path() { - local value - value=$(realpath -m "$1") - case "$value" in "$root"/*) ;; *) echo 'Path escapes task root.' >&2; exit 2;; esac -} -for path in "$BASE_PYTHON" "$task_uv_executable" "$BUILD_SOURCE_DIR" "$BUILD_BUNDLE" "$BUILD_VENV"; do - check_path "$path" -done -source_dir=$(cd -- "$BUILD_SOURCE_DIR" && pwd -P) -check_digest() { - [[ "$2" =~ ^[a-f0-9]{64}$ ]] - [[ $(sha256sum "$1" | cut -d' ' -f1) == "$2" ]] -} -check_digest "$source_dir/build-reviewed.sbatch" "$BUILD_SCRIPT_SHA256" -cmp "${BASH_SOURCE[0]}" "$source_dir/build-reviewed.sbatch" -check_digest "$source_dir/inputs.sha256" "$BUILD_INPUTS_SHA256" -(cd "$source_dir" && sha256sum --check --strict inputs.sha256) -[[ ! -e "$BUILD_BUNDLE" && ! -L "$BUILD_BUNDLE" && ! -e "$BUILD_VENV" && ! -L "$BUILD_VENV" ]] -for task_resolver_variable in "${!UV_@}" "${!PIP_@}"; do - if [[ -n "$task_resolver_variable" ]]; then unset "$task_resolver_variable"; fi -done -export TMPDIR="$root/tmp/qwen38-reviewed-wheels-${SLURM_JOB_ID}" -export XDG_CACHE_HOME="$root/cache/qwen38-reviewed-wheels" -export UV_CACHE_DIR="$XDG_CACHE_HOME/uv" -export UV_PYTHON_INSTALL_DIR="$root/software/python" -export UV_PYTHON_DOWNLOADS=never -export UV_HTTP_TIMEOUT=60 -export PYTHONNOUSERSITE=1 -export PYTHONDONTWRITEBYTECODE=1 -unset PYTHONPATH VIRTUAL_ENV -for path in "$TMPDIR" "$XDG_CACHE_HOME" "$UV_CACHE_DIR" "$UV_PYTHON_INSTALL_DIR"; do check_path "$path"; done -mkdir -p "$TMPDIR" "$UV_CACHE_DIR" -cd "$root" -"$BASE_PYTHON" -c 'import sys; assert sys.version_info[:2] == (3, 12)' -mkdir "$BUILD_BUNDLE" -trap 'status=$?; printf "%s\n" "$status" > "$BUILD_BUNDLE/exit-code"; exit "$status"' EXIT -cp "$source_dir/build-reviewed.sbatch" "$source_dir/inputs.sha256" "$source_dir/build.lock" \ - "$source_dir/wget-3.2.zip" "$source_dir/tempdir-0.7.1.tar.gz" "$BUILD_BUNDLE/" -"$task_uv_executable" --no-config venv --python "$BASE_PYTHON" "$BUILD_VENV" -"$task_uv_executable" --no-config pip install --python "$BUILD_VENV/bin/python" \ - --no-build --require-hashes --default-index https://pypi.org/simple -r "$BUILD_BUNDLE/build.lock" -"$task_uv_executable" --no-config pip check --python "$BUILD_VENV/bin/python" -"$task_uv_executable" --no-config pip freeze --python "$BUILD_VENV/bin/python" > "$BUILD_BUNDLE/build-installed.txt" -mkdir "$BUILD_BUNDLE/wheelhouse" -for archive in wget-3.2.zip tempdir-0.7.1.tar.gz; do - "$task_uv_executable" --no-config --offline build "$BUILD_BUNDLE/$archive" \ - --python "$BUILD_VENV/bin/python" --no-build-isolation --wheel --no-create-gitignore \ - --out-dir "$BUILD_BUNDLE/wheelhouse" -done -(cd "$source_dir" && sha256sum --check --strict inputs.sha256) -check_digest "$source_dir/build-reviewed.sbatch" "$BUILD_SCRIPT_SHA256" -"$task_uv_executable" --version > "$BUILD_BUNDLE/uv-version.txt" -sha256sum "$task_uv_executable" | cut -d' ' -f1 > "$BUILD_BUNDLE/uv-sha256.txt" -sha256sum "$BASE_PYTHON" | cut -d' ' -f1 > "$BUILD_BUNDLE/python-sha256.txt" -(cd "$BUILD_BUNDLE" && sha256sum build-reviewed.sbatch inputs.sha256 build.lock \ - wget-3.2.zip tempdir-0.7.1.tar.gz build-installed.txt uv-version.txt uv-sha256.txt \ - python-sha256.txt wheelhouse/*.whl > output.sha256) -echo 'Reviewed source-wheel build completed; output review required before resolution or installation.' diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bwrap-build-v1/build.lock b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bwrap-build-v1/build.lock deleted file mode 100644 index dd0e18917..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bwrap-build-v1/build.lock +++ /dev/null @@ -1,2 +0,0 @@ -meson==1.12.1 --hash=sha256:930bc7542cbd9f57009e182fd014eba48cf1a0180a7b9006d2d32d8f168d8b02 -ninja==1.13.2 --hash=sha256:65a24341b5ac09fcadcc37082660be40a94174e51a937fabf6e2cae26225fa2c diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bwrap-build-v1/extract_source.py b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bwrap-build-v1/extract_source.py deleted file mode 100644 index d2bd3a4be..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bwrap-build-v1/extract_source.py +++ /dev/null @@ -1,27 +0,0 @@ -"""Extract the authenticated release with only its two local license symlinks.""" -import argparse -import tarfile -from pathlib import Path - - -def extract(archive, destination): - with tarfile.open(archive) as source: - for member in source.getmembers(): - root = member.isdir() and member.name == "bubblewrap-0.13.1" - license_link = member.issym() and member.name in ( - "bubblewrap-0.13.1/COPYING.LIB", "bubblewrap-0.13.1/LICENSE" - ) and member.linkname == "COPYING" - if not (member.isfile() or member.isdir() or license_link): - raise ValueError("Unexpected archive member type") - if not root and not member.name.startswith("bubblewrap-0.13.1/"): - raise ValueError("Unexpected archive root") - destination.mkdir(exist_ok=False) - source.extractall(destination, filter="data") - - -if __name__ == "__main__": - parser = argparse.ArgumentParser(description=__doc__) - parser.add_argument("archive", type=Path) - parser.add_argument("destination", type=Path) - args = parser.parse_args() - extract(args.archive, args.destination) diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bwrap-build-v1/inputs.json b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bwrap-build-v1/inputs.json deleted file mode 100644 index 988d8b096..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bwrap-build-v1/inputs.json +++ /dev/null @@ -1,33 +0,0 @@ -{ - "scope": "raw_build_input_preparation", - "verdict": "INCONCLUSIVE", - "downloaded_content_executed": false, - "source_archive_sha256": "0e85c8702248b8dd64144f594147553650cdd0e4c951690345faace79c217c4d", - "build_lock_sha256": "b28a5f4618b1890bb9b796206e14a56517cef73cf7811efb7340414ef7a77aab", - "files": [ - { - "name": "meson", - "version": "1.12.1", - "filename": "meson-1.12.1-py3-none-any.whl", - "sha256": "930bc7542cbd9f57009e182fd014eba48cf1a0180a7b9006d2d32d8f168d8b02", - "size": 1108668, - "url": "https://files.pythonhosted.org/packages/57/99/36cd25f9598eef70db97577440a70a360af505f6f01c2338a18d1556079c/meson-1.12.1-py3-none-any.whl", - "requires_python": ">=3.10", - "target_tag_rank": 866, - "metadata_url": "https://pypi.org/pypi/meson/json", - "metadata_sha256": "34801ae0bdba0e23dd7261e35a3868fdbac647d7006739dc086a8310a654c612" - }, - { - "name": "ninja", - "version": "1.13.2", - "filename": "ninja-1.13.2-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", - "sha256": "65a24341b5ac09fcadcc37082660be40a94174e51a937fabf6e2cae26225fa2c", - "size": 183365, - "url": "https://files.pythonhosted.org/packages/6e/53/ebfed7b689c338dd8ebeec9c0730c8d56821292f14e2536e5f3ef1a05744/ninja-1.13.2-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", - "requires_python": ">=3.8", - "target_tag_rank": 464, - "metadata_url": "https://pypi.org/pypi/ninja/1.13.2/json", - "metadata_sha256": "4093daaae326c46316289eb9ae9c758d4cd06ec75f75c59633e5a2cb0ff91dc3" - } - ] -} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bwrap-build-v1/inputs.sha256 b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bwrap-build-v1/inputs.sha256 deleted file mode 100644 index 71aefbd1a..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bwrap-build-v1/inputs.sha256 +++ /dev/null @@ -1,6 +0,0 @@ -0e85c8702248b8dd64144f594147553650cdd0e4c951690345faace79c217c4d bubblewrap-0.13.1.tar.xz -b28a5f4618b1890bb9b796206e14a56517cef73cf7811efb7340414ef7a77aab build.lock -e4037153d4c6d1f4a8d2f6e165fb9dba67aec3ec9e4d7b2a85a91c1a31017ad1 inputs.json -98c7382ae519c9a34e3ada3fa4ebe585c7872b855b6f240fd3bcab783e189362 readset.json -930bc7542cbd9f57009e182fd014eba48cf1a0180a7b9006d2d32d8f168d8b02 wheelhouse/meson-1.12.1-py3-none-any.whl -65a24341b5ac09fcadcc37082660be40a94174e51a937fabf6e2cae26225fa2c wheelhouse/ninja-1.13.2-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bwrap-build-v1/readset.json b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bwrap-build-v1/readset.json deleted file mode 100644 index 329ba1d7a..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bwrap-build-v1/readset.json +++ /dev/null @@ -1,16 +0,0 @@ -{ - "source": "https://github.com/containers/bubblewrap", - "tag": "v0.13.1", - "commit": "362936aea0d5c6e70f7af75d2d44b5a914cb4ba2", - "release_url": "https://github.com/containers/bubblewrap/releases/tag/v0.13.1", - "release_date": "2026-10-14", - "published_tar_filename": "bubblewrap-0.13.1.tar.xz", - "published_tar_sha256": "0e85c8702248b8dd64144f594147553650cdd0e4c951690345faace79c217c4d", - "source_git_blobs": { - "meson.build": "08d28c97e64932dd24811c41623a12506fa84ce1", - "meson_options.txt": "f4f4ac708911764eebbbecb0dee84391366f3a8d", - "README.md": "69addd6f4cbda5ccd1c47e7022a022d305ad4965" - }, - "observed_build_requirements": ["C compiler", "Meson >=0.49.0", "libcap development dependency required", "optional libselinux", "explicit registered Python via -Dpython"], - "state": "source read only; archive not downloaded, source not built, no isolation capability validated" -} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bwrap-build-v1/review-rejected-v1/run.sbatch b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bwrap-build-v1/review-rejected-v1/run.sbatch deleted file mode 100644 index b0bc8725f..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bwrap-build-v1/review-rejected-v1/run.sbatch +++ /dev/null @@ -1,118 +0,0 @@ -#!/usr/bin/env bash -# Build a task-local tool; only its pure utility test and version/help are run. -#SBATCH --job-name=qwen38-bwrap-build -#SBATCH --nodes=1 -#SBATCH --ntasks=1 -#SBATCH --cpus-per-task=4 -#SBATCH --mem=8G -#SBATCH --time=00:10:00 -set -euo pipefail -umask 077 -: "${TASK_ROOT:?}" "${BASE_PYTHON:?}" "${UV_BIN:?}" "${SOURCE_DIR:?}" -: "${OUTPUT_DIR:?}" "${BUILD_VENV:?}" "${INSTALL_PREFIX:?}" -: "${SCRIPT_SHA256:?}" "${EXTRACTOR_SHA256:?}" "${INPUTS_SHA256:?}" -: "${PYTHON_SHA256:?}" "${UV_SHA256:?}" -[[ -n ${SLURM_JOB_ID:-} && ${SLURM_GPUS_ON_NODE:-0} == 0 ]] -root=$(realpath -e "$TASK_ROOT") -inside_root() { case "$(realpath -m "$1")" in "$root"/*) ;; *) exit 2;; esac; } -for path in "$BASE_PYTHON" "$UV_BIN" "$SOURCE_DIR" "$OUTPUT_DIR" "$BUILD_VENV" "$INSTALL_PREFIX"; do inside_root "$path"; done -check_digest() { - [[ $2 =~ ^[a-f0-9]{64}$ ]] - [[ $(sha256sum "$1" | cut -d' ' -f1) == "$2" ]] -} -check_digest "$SOURCE_DIR/run.sbatch" "$SCRIPT_SHA256" -check_digest "$SOURCE_DIR/extract_source.py" "$EXTRACTOR_SHA256" -check_digest "$SOURCE_DIR/inputs.sha256" "$INPUTS_SHA256" -check_digest "$BASE_PYTHON" "$PYTHON_SHA256" -check_digest "$UV_BIN" "$UV_SHA256" -cmp "${BASH_SOURCE[0]}" "$SOURCE_DIR/run.sbatch" -(cd "$SOURCE_DIR" && sha256sum --check --strict inputs.sha256) -for path in "$OUTPUT_DIR" "$BUILD_VENV" "$INSTALL_PREFIX"; do [[ ! -e $path && ! -L $path ]]; done -task_uv=$UV_BIN -for variable in "${!UV_@}" "${!PIP_@}"; do [[ -z $variable ]] || unset "$variable"; done -export UV_OFFLINE=true UV_PYTHON_DOWNLOADS=never -export TMPDIR="$root/tmp/qwen38-bwrap-build-${SLURM_JOB_ID}" -export XDG_CACHE_HOME="$root/cache/qwen38-bwrap-build" -export UV_CACHE_DIR="$XDG_CACHE_HOME/uv" HF_HOME="$XDG_CACHE_HOME/huggingface" -export TORCH_HOME="$XDG_CACHE_HOME/torch" TRITON_CACHE_DIR="$XDG_CACHE_HOME/triton" -export VLLM_CACHE_ROOT="$XDG_CACHE_HOME/vllm" UV_PYTHON_INSTALL_DIR="$root/software/python" -export PYTHONNOUSERSITE=1 PYTHONDONTWRITEBYTECODE=1 -unset PYTHONPATH PYTHONHOME PYTHONOPTIMIZE PYTHONINSPECT PYTHONSTARTUP VIRTUAL_ENV -unset CC CXX CFLAGS CXXFLAGS CPPFLAGS LDFLAGS PKG_CONFIG_PATH PKG_CONFIG_LIBDIR PKG_CONFIG_SYSROOT_DIR -for path in "$TMPDIR" "$XDG_CACHE_HOME" "$UV_CACHE_DIR" "$HF_HOME" "$TORCH_HOME" "$TRITON_CACHE_DIR" "$VLLM_CACHE_ROOT" "$UV_PYTHON_INSTALL_DIR"; do inside_root "$path"; mkdir -p "$path"; done -mkdir "$OUTPUT_DIR" -completed=false -phase=dependency_preflight -signal_name=none -finish() { - local status=$? - trap - EXIT - [[ $status != 0 || $completed == true ]] || status=1 - [[ $status == 0 ]] || completed=false - printf '%s\n' "$status" > "$OUTPUT_DIR/exit-code" - printf '{"completed":%s,"exit_code":%s,"phase":"%s","signal":"%s","job_id":"%s"}\n' \ - "$completed" "$status" "$phase" "$signal_name" "$SLURM_JOB_ID" > "$OUTPUT_DIR/terminal-status.json" - exit "$status" -} -trap finish EXIT -trap 'signal_name=TERM; exit 143' TERM -trap 'signal_name=INT; exit 130' INT -trap 'signal_name=HUP; exit 129' HUP -cp "$SOURCE_DIR/run.sbatch" "$SOURCE_DIR/extract_source.py" "$SOURCE_DIR/inputs.sha256" "$OUTPUT_DIR/" -cd "$root" -export CC -CC=$(command -v cc) -printf '%s\n' "$CC" > "$OUTPUT_DIR/compiler-path.private.txt" -sha256sum "$(realpath -e "$CC")" > "$OUTPUT_DIR/compiler-sha256.private.txt" -"$CC" --version > "$OUTPUT_DIR/compiler-version.private.txt" -pkg-config --modversion libcap > "$OUTPUT_DIR/libcap-version.txt" -pkg-config --cflags --libs libcap > "$OUTPUT_DIR/libcap-flags.private.txt" -phase=build_environment -"$task_uv" --no-config --offline venv --python "$BASE_PYTHON" "$BUILD_VENV" -"$task_uv" --no-config --offline pip install --python "$BUILD_VENV/bin/python" \ - --no-index --find-links "$SOURCE_DIR/wheelhouse" --no-deps --no-build \ - --require-hashes -r "$SOURCE_DIR/build.lock" -"$BUILD_VENV/bin/python" -IB - "$OUTPUT_DIR/build-packages.json" <<'PY' -import importlib.metadata -import json -import sys -packages = {p.metadata['Name'].lower(): p.version for p in importlib.metadata.distributions()} -if packages != {'meson': '1.12.1', 'ninja': '1.13.2'}: - raise ValueError('Unexpected build package set') -with open(sys.argv[1], 'x') as stream: - json.dump(packages, stream, sort_keys=True) - stream.write('\n') -PY -export PATH="$BUILD_VENV/bin:$PATH" -phase=source_extraction -"$BASE_PYTHON" -IB "$SOURCE_DIR/extract_source.py" \ - "$SOURCE_DIR/bubblewrap-0.13.1.tar.xz" "$OUTPUT_DIR/source" -phase=configure_compile -"$BUILD_VENV/bin/meson" setup "$OUTPUT_DIR/build" "$OUTPUT_DIR/source/bubblewrap-0.13.1" \ - --prefix="$INSTALL_PREFIX" --wrap-mode=nodownload \ - -Dpython="$BUILD_VENV/bin/python" -Dman=disabled \ - -Dbash_completion=disabled -Dzsh_completion=disabled -Dtests=true -"$BUILD_VENV/bin/meson" compile -C "$OUTPUT_DIR/build" -j 4 bwrap test-utils -phase=utility_test -"$BUILD_VENV/bin/meson" test -C "$OUTPUT_DIR/build" --no-rebuild --print-errorlogs test-utils -phase=task_local_install -"$BUILD_VENV/bin/meson" install -C "$OUTPUT_DIR/build" --no-rebuild -inside_root "$INSTALL_PREFIX/bin/bwrap" -[[ ! -u "$INSTALL_PREFIX/bin/bwrap" && ! -g "$INSTALL_PREFIX/bin/bwrap" ]] -phase=version_help_checks -"$INSTALL_PREFIX/bin/bwrap" --version > "$OUTPUT_DIR/version.txt" -[[ $(cat "$OUTPUT_DIR/version.txt") == 'bubblewrap 0.13.1' ]] -"$INSTALL_PREFIX/bin/bwrap" --help > "$OUTPUT_DIR/help.txt" 2>&1 -for flag in --unshare-user --unshare-net --unshare-pid --disable-userns \ - --assert-userns-disabled --clearenv --remount-ro --size; do - grep -Fq -- "$flag" "$OUTPUT_DIR/help.txt" -done -sha256sum "$INSTALL_PREFIX/bin/bwrap" > "$OUTPUT_DIR/binary-sha256.private.txt" -ldd "$INSTALL_PREFIX/bin/bwrap" > "$OUTPUT_DIR/linked-libraries.private.txt" -phase=postflight -check_digest "$SOURCE_DIR/run.sbatch" "$SCRIPT_SHA256" -check_digest "$SOURCE_DIR/extract_source.py" "$EXTRACTOR_SHA256" -check_digest "$SOURCE_DIR/inputs.sha256" "$INPUTS_SHA256" -(cd "$SOURCE_DIR" && sha256sum --check --strict inputs.sha256) -phase=complete -completed=true diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bwrap-build-v1/review-rejected-v2/run.sbatch b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bwrap-build-v1/review-rejected-v2/run.sbatch deleted file mode 100644 index 19933b291..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bwrap-build-v1/review-rejected-v2/run.sbatch +++ /dev/null @@ -1,119 +0,0 @@ -#!/usr/bin/env bash -# Build a task-local tool; only its pure utility test and version/help are run. -#SBATCH --job-name=qwen38-bwrap-build -#SBATCH --nodes=1 -#SBATCH --ntasks=1 -#SBATCH --cpus-per-task=4 -#SBATCH --mem=8G -#SBATCH --time=00:10:00 -set -euo pipefail -umask 077 -: "${TASK_ROOT:?}" "${BASE_PYTHON:?}" "${UV_BIN:?}" "${SOURCE_DIR:?}" -: "${OUTPUT_DIR:?}" "${BUILD_VENV:?}" "${INSTALL_PREFIX:?}" -: "${SCRIPT_SHA256:?}" "${EXTRACTOR_SHA256:?}" "${INPUTS_SHA256:?}" -: "${PYTHON_SHA256:?}" "${UV_SHA256:?}" -[[ -n ${SLURM_JOB_ID:-} && ${SLURM_GPUS_ON_NODE:-0} == 0 ]] -root=$(realpath -e "$TASK_ROOT") -inside_root() { case "$(realpath -m "$1")" in "$root"/*) ;; *) exit 2;; esac; } -for path in "$BASE_PYTHON" "$UV_BIN" "$SOURCE_DIR" "$OUTPUT_DIR" "$BUILD_VENV" "$INSTALL_PREFIX"; do inside_root "$path"; done -check_digest() { - [[ $2 =~ ^[a-f0-9]{64}$ ]] - [[ $(sha256sum "$1" | cut -d' ' -f1) == "$2" ]] -} -check_digest "$SOURCE_DIR/run.sbatch" "$SCRIPT_SHA256" -check_digest "$SOURCE_DIR/extract_source.py" "$EXTRACTOR_SHA256" -check_digest "$SOURCE_DIR/inputs.sha256" "$INPUTS_SHA256" -check_digest "$BASE_PYTHON" "$PYTHON_SHA256" -check_digest "$UV_BIN" "$UV_SHA256" -cmp "${BASH_SOURCE[0]}" "$SOURCE_DIR/run.sbatch" -(cd "$SOURCE_DIR" && sha256sum --check --strict inputs.sha256) -for path in "$OUTPUT_DIR" "$BUILD_VENV" "$INSTALL_PREFIX"; do [[ ! -e $path && ! -L $path ]]; done -task_uv=$UV_BIN -for variable in "${!UV_@}" "${!PIP_@}"; do [[ -z $variable ]] || unset "$variable"; done -export UV_OFFLINE=true UV_PYTHON_DOWNLOADS=never -export TMPDIR="$root/tmp/qwen38-bwrap-build-${SLURM_JOB_ID}" -export XDG_CACHE_HOME="$root/cache/qwen38-bwrap-build" -export UV_CACHE_DIR="$XDG_CACHE_HOME/uv" HF_HOME="$XDG_CACHE_HOME/huggingface" -export TORCH_HOME="$XDG_CACHE_HOME/torch" TRITON_CACHE_DIR="$XDG_CACHE_HOME/triton" -export VLLM_CACHE_ROOT="$XDG_CACHE_HOME/vllm" UV_PYTHON_INSTALL_DIR="$root/software/python" -export PYTHONNOUSERSITE=1 PYTHONDONTWRITEBYTECODE=1 -unset PYTHONPATH PYTHONHOME PYTHONOPTIMIZE PYTHONINSPECT PYTHONSTARTUP VIRTUAL_ENV -unset DESTDIR -unset CC CXX CFLAGS CXXFLAGS CPPFLAGS LDFLAGS PKG_CONFIG_PATH PKG_CONFIG_LIBDIR PKG_CONFIG_SYSROOT_DIR -for path in "$TMPDIR" "$XDG_CACHE_HOME" "$UV_CACHE_DIR" "$HF_HOME" "$TORCH_HOME" "$TRITON_CACHE_DIR" "$VLLM_CACHE_ROOT" "$UV_PYTHON_INSTALL_DIR"; do inside_root "$path"; mkdir -p "$path"; done -mkdir "$OUTPUT_DIR" -completed=false -phase=dependency_preflight -signal_name=none -finish() { - local status=$? - trap - EXIT - [[ $status != 0 || $completed == true ]] || status=1 - [[ $status == 0 ]] || completed=false - printf '%s\n' "$status" > "$OUTPUT_DIR/exit-code" - printf '{"completed":%s,"exit_code":%s,"phase":"%s","signal":"%s","job_id":"%s"}\n' \ - "$completed" "$status" "$phase" "$signal_name" "$SLURM_JOB_ID" > "$OUTPUT_DIR/terminal-status.json" - exit "$status" -} -trap finish EXIT -trap 'signal_name=TERM; exit 143' TERM -trap 'signal_name=INT; exit 130' INT -trap 'signal_name=HUP; exit 129' HUP -cp "$SOURCE_DIR/run.sbatch" "$SOURCE_DIR/extract_source.py" "$SOURCE_DIR/inputs.sha256" "$OUTPUT_DIR/" -cd "$root" -export CC -CC=$(command -v cc) -printf '%s\n' "$CC" > "$OUTPUT_DIR/compiler-path.private.txt" -sha256sum "$(realpath -e "$CC")" > "$OUTPUT_DIR/compiler-sha256.private.txt" -"$CC" --version > "$OUTPUT_DIR/compiler-version.private.txt" -pkg-config --modversion libcap > "$OUTPUT_DIR/libcap-version.txt" -pkg-config --cflags --libs libcap > "$OUTPUT_DIR/libcap-flags.private.txt" -phase=build_environment -"$task_uv" --no-config --offline venv --python "$BASE_PYTHON" "$BUILD_VENV" -"$task_uv" --no-config --offline pip install --python "$BUILD_VENV/bin/python" \ - --no-index --find-links "$SOURCE_DIR/wheelhouse" --no-deps --no-build \ - --require-hashes -r "$SOURCE_DIR/build.lock" -"$BUILD_VENV/bin/python" -IB - "$OUTPUT_DIR/build-packages.json" <<'PY' -import importlib.metadata -import json -import sys -packages = {p.metadata['Name'].lower(): p.version for p in importlib.metadata.distributions()} -if packages != {'meson': '1.12.1', 'ninja': '1.13.2'}: - raise ValueError('Unexpected build package set') -with open(sys.argv[1], 'x') as stream: - json.dump(packages, stream, sort_keys=True) - stream.write('\n') -PY -export PATH="$BUILD_VENV/bin:$PATH" -phase=source_extraction -"$BASE_PYTHON" -IB "$SOURCE_DIR/extract_source.py" \ - "$SOURCE_DIR/bubblewrap-0.13.1.tar.xz" "$OUTPUT_DIR/source" -phase=configure_compile -"$BUILD_VENV/bin/meson" setup "$OUTPUT_DIR/build" "$OUTPUT_DIR/source/bubblewrap-0.13.1" \ - --prefix="$INSTALL_PREFIX" --wrap-mode=nodownload \ - -Dpython="$BUILD_VENV/bin/python" -Dman=disabled \ - -Dbash_completion=disabled -Dzsh_completion=disabled -Dtests=true -"$BUILD_VENV/bin/meson" compile -C "$OUTPUT_DIR/build" -j 4 bwrap test-utils -phase=utility_test -"$BUILD_VENV/bin/meson" test -C "$OUTPUT_DIR/build" --no-rebuild --print-errorlogs test-utils -phase=task_local_install -"$BUILD_VENV/bin/meson" install -C "$OUTPUT_DIR/build" --no-rebuild --destdir='' -inside_root "$INSTALL_PREFIX/bin/bwrap" -[[ ! -u "$INSTALL_PREFIX/bin/bwrap" && ! -g "$INSTALL_PREFIX/bin/bwrap" ]] -phase=version_help_checks -"$INSTALL_PREFIX/bin/bwrap" --version > "$OUTPUT_DIR/version.txt" -[[ $(cat "$OUTPUT_DIR/version.txt") == 'bubblewrap 0.13.1' ]] -"$INSTALL_PREFIX/bin/bwrap" --help > "$OUTPUT_DIR/help.txt" 2>&1 -for flag in --unshare-user --unshare-net --unshare-pid --disable-userns \ - --assert-userns-disabled --clearenv --remount-ro --size; do - grep -Fq -- "$flag" "$OUTPUT_DIR/help.txt" -done -sha256sum "$INSTALL_PREFIX/bin/bwrap" > "$OUTPUT_DIR/binary-sha256.private.txt" -ldd "$INSTALL_PREFIX/bin/bwrap" > "$OUTPUT_DIR/linked-libraries.private.txt" -phase=postflight -check_digest "$SOURCE_DIR/run.sbatch" "$SCRIPT_SHA256" -check_digest "$SOURCE_DIR/extract_source.py" "$EXTRACTOR_SHA256" -check_digest "$SOURCE_DIR/inputs.sha256" "$INPUTS_SHA256" -(cd "$SOURCE_DIR" && sha256sum --check --strict inputs.sha256) -phase=complete -completed=true diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bwrap-build-v1/run.sbatch b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bwrap-build-v1/run.sbatch deleted file mode 100644 index f927a01e2..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bwrap-build-v1/run.sbatch +++ /dev/null @@ -1,120 +0,0 @@ -#!/usr/bin/env bash -# Build a task-local tool; only its pure utility test and version/help are run. -#SBATCH --job-name=qwen38-bwrap-build -#SBATCH --nodes=1 -#SBATCH --ntasks=1 -#SBATCH --cpus-per-task=4 -#SBATCH --mem=8G -#SBATCH --time=00:10:00 -set -euo pipefail -umask 077 -: "${TASK_ROOT:?}" "${BASE_PYTHON:?}" "${UV_BIN:?}" "${SOURCE_DIR:?}" -: "${OUTPUT_DIR:?}" "${BUILD_VENV:?}" "${INSTALL_PREFIX:?}" -: "${SCRIPT_SHA256:?}" "${EXTRACTOR_SHA256:?}" "${INPUTS_SHA256:?}" -: "${PYTHON_SHA256:?}" "${UV_SHA256:?}" -[[ -n ${SLURM_JOB_ID:-} && ${SLURM_GPUS_ON_NODE:-0} == 0 ]] -root=$(realpath -e "$TASK_ROOT") -inside_root() { case "$(realpath -m "$1")" in "$root"/*) ;; *) exit 2;; esac; } -for path in "$BASE_PYTHON" "$UV_BIN" "$SOURCE_DIR" "$OUTPUT_DIR" "$BUILD_VENV" "$INSTALL_PREFIX"; do inside_root "$path"; done -check_digest() { - [[ $2 =~ ^[a-f0-9]{64}$ ]] - [[ $(sha256sum "$1" | cut -d' ' -f1) == "$2" ]] -} -check_digest "$SOURCE_DIR/run.sbatch" "$SCRIPT_SHA256" -check_digest "$SOURCE_DIR/extract_source.py" "$EXTRACTOR_SHA256" -check_digest "$SOURCE_DIR/inputs.sha256" "$INPUTS_SHA256" -check_digest "$BASE_PYTHON" "$PYTHON_SHA256" -check_digest "$UV_BIN" "$UV_SHA256" -cmp "${BASH_SOURCE[0]}" "$SOURCE_DIR/run.sbatch" -(cd "$SOURCE_DIR" && sha256sum --check --strict inputs.sha256) -for path in "$OUTPUT_DIR" "$BUILD_VENV" "$INSTALL_PREFIX"; do [[ ! -e $path && ! -L $path ]]; done -task_uv=$UV_BIN -for variable in "${!UV_@}" "${!PIP_@}"; do [[ -z $variable ]] || unset "$variable"; done -export UV_OFFLINE=true UV_PYTHON_DOWNLOADS=never -export TMPDIR="$root/tmp/qwen38-bwrap-build-${SLURM_JOB_ID}" -export XDG_CACHE_HOME="$root/cache/qwen38-bwrap-build" -export UV_CACHE_DIR="$XDG_CACHE_HOME/uv" HF_HOME="$XDG_CACHE_HOME/huggingface" -export TORCH_HOME="$XDG_CACHE_HOME/torch" TRITON_CACHE_DIR="$XDG_CACHE_HOME/triton" -export VLLM_CACHE_ROOT="$XDG_CACHE_HOME/vllm" UV_PYTHON_INSTALL_DIR="$root/software/python" -export PYTHONNOUSERSITE=1 PYTHONDONTWRITEBYTECODE=1 -unset PYTHONPATH PYTHONHOME PYTHONOPTIMIZE PYTHONINSPECT PYTHONSTARTUP VIRTUAL_ENV -unset DESTDIR -unset CC CXX CFLAGS CXXFLAGS CPPFLAGS LDFLAGS PKG_CONFIG_PATH PKG_CONFIG_LIBDIR PKG_CONFIG_SYSROOT_DIR -for path in "$TMPDIR" "$XDG_CACHE_HOME" "$UV_CACHE_DIR" "$HF_HOME" "$TORCH_HOME" "$TRITON_CACHE_DIR" "$VLLM_CACHE_ROOT" "$UV_PYTHON_INSTALL_DIR"; do inside_root "$path"; mkdir -p "$path"; done -mkdir "$OUTPUT_DIR" -completed=false -phase=dependency_preflight -signal_name=none -finish() { - local status=$? - trap - EXIT - [[ $status != 0 || $completed == true ]] || status=1 - [[ $status == 0 ]] || completed=false - printf '%s\n' "$status" > "$OUTPUT_DIR/exit-code" - printf '{"completed":%s,"exit_code":%s,"phase":"%s","signal":"%s","job_id":"%s"}\n' \ - "$completed" "$status" "$phase" "$signal_name" "$SLURM_JOB_ID" > "$OUTPUT_DIR/terminal-status.json" - exit "$status" -} -trap finish EXIT -trap 'signal_name=TERM; exit 143' TERM -trap 'signal_name=INT; exit 130' INT -trap 'signal_name=HUP; exit 129' HUP -cp "$SOURCE_DIR/run.sbatch" "$SOURCE_DIR/extract_source.py" "$SOURCE_DIR/inputs.sha256" "$OUTPUT_DIR/" -cd "$root" -export CC -CC=$(command -v cc) -printf '%s\n' "$CC" > "$OUTPUT_DIR/compiler-path.private.txt" -sha256sum "$(realpath -e "$CC")" > "$OUTPUT_DIR/compiler-sha256.private.txt" -"$CC" --version > "$OUTPUT_DIR/compiler-version.private.txt" -pkg-config --modversion libcap > "$OUTPUT_DIR/libcap-version.txt" -pkg-config --cflags --libs libcap > "$OUTPUT_DIR/libcap-flags.private.txt" -phase=build_environment -"$task_uv" --no-config --offline venv --python "$BASE_PYTHON" "$BUILD_VENV" -"$task_uv" --no-config --offline pip install --python "$BUILD_VENV/bin/python" \ - --no-index --find-links "$SOURCE_DIR/wheelhouse" --no-deps --no-build \ - --require-hashes -r "$SOURCE_DIR/build.lock" -"$BUILD_VENV/bin/python" -IB - "$OUTPUT_DIR/build-packages.json" <<'PY' -import importlib.metadata -import json -import sys -packages = {p.metadata['Name'].lower(): p.version for p in importlib.metadata.distributions()} -if packages != {'meson': '1.12.1', 'ninja': '1.13.2'}: - raise ValueError('Unexpected build package set') -with open(sys.argv[1], 'x') as stream: - json.dump(packages, stream, sort_keys=True) - stream.write('\n') -PY -export PATH="$BUILD_VENV/bin:$PATH" -export NINJA="$BUILD_VENV/bin/ninja" -phase=source_extraction -"$BASE_PYTHON" -IB "$SOURCE_DIR/extract_source.py" \ - "$SOURCE_DIR/bubblewrap-0.13.1.tar.xz" "$OUTPUT_DIR/source" -phase=configure_compile -"$BUILD_VENV/bin/meson" setup "$OUTPUT_DIR/build" "$OUTPUT_DIR/source/bubblewrap-0.13.1" \ - --prefix="$INSTALL_PREFIX" --wrap-mode=nodownload \ - -Dpython="$BUILD_VENV/bin/python" -Dman=disabled \ - -Dbash_completion=disabled -Dzsh_completion=disabled -Dtests=true -"$BUILD_VENV/bin/meson" compile -C "$OUTPUT_DIR/build" -j 4 bwrap test-utils -phase=utility_test -"$BUILD_VENV/bin/meson" test -C "$OUTPUT_DIR/build" --no-rebuild --print-errorlogs test-utils -phase=task_local_install -"$BUILD_VENV/bin/meson" install -C "$OUTPUT_DIR/build" --no-rebuild --destdir='' -inside_root "$INSTALL_PREFIX/bin/bwrap" -[[ ! -u "$INSTALL_PREFIX/bin/bwrap" && ! -g "$INSTALL_PREFIX/bin/bwrap" ]] -phase=version_help_checks -"$INSTALL_PREFIX/bin/bwrap" --version > "$OUTPUT_DIR/version.txt" -[[ $(cat "$OUTPUT_DIR/version.txt") == 'bubblewrap 0.13.1' ]] -"$INSTALL_PREFIX/bin/bwrap" --help > "$OUTPUT_DIR/help.txt" 2>&1 -for flag in --unshare-user --unshare-net --unshare-pid --disable-userns \ - --assert-userns-disabled --clearenv --remount-ro --size; do - grep -Fq -- "$flag" "$OUTPUT_DIR/help.txt" -done -sha256sum "$INSTALL_PREFIX/bin/bwrap" > "$OUTPUT_DIR/binary-sha256.private.txt" -ldd "$INSTALL_PREFIX/bin/bwrap" > "$OUTPUT_DIR/linked-libraries.private.txt" -phase=postflight -check_digest "$SOURCE_DIR/run.sbatch" "$SCRIPT_SHA256" -check_digest "$SOURCE_DIR/extract_source.py" "$EXTRACTOR_SHA256" -check_digest "$SOURCE_DIR/inputs.sha256" "$INPUTS_SHA256" -(cd "$SOURCE_DIR" && sha256sum --check --strict inputs.sha256) -phase=complete -completed=true diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bwrap-build-v2/build.lock b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bwrap-build-v2/build.lock deleted file mode 100644 index dd0e18917..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bwrap-build-v2/build.lock +++ /dev/null @@ -1,2 +0,0 @@ -meson==1.12.1 --hash=sha256:930bc7542cbd9f57009e182fd014eba48cf1a0180a7b9006d2d32d8f168d8b02 -ninja==1.13.2 --hash=sha256:65a24341b5ac09fcadcc37082660be40a94174e51a937fabf6e2cae26225fa2c diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bwrap-build-v2/extract_source.py b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bwrap-build-v2/extract_source.py deleted file mode 100644 index 4341723b1..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bwrap-build-v2/extract_source.py +++ /dev/null @@ -1,28 +0,0 @@ -"""Extract the authenticated release with only its two local license symlinks.""" -import argparse -import tarfile -from pathlib import Path - - -def extract(archive, destination, archive_root): - with tarfile.open(archive) as source: - for member in source.getmembers(): - root = member.isdir() and member.name == archive_root - license_link = archive_root == "bubblewrap-0.13.1" and member.issym() and member.name in ( - "bubblewrap-0.13.1/COPYING.LIB", "bubblewrap-0.13.1/LICENSE" - ) and member.linkname == "COPYING" - if not (member.isfile() or member.isdir() or license_link): - raise ValueError("Unexpected archive member type") - if not root and not member.name.startswith(archive_root + "/"): - raise ValueError("Unexpected archive root") - destination.mkdir(exist_ok=False) - source.extractall(destination, filter="data") - - -if __name__ == "__main__": - parser = argparse.ArgumentParser(description=__doc__) - parser.add_argument("archive", type=Path) - parser.add_argument("destination", type=Path) - parser.add_argument("--root", required=True, choices=("bubblewrap-0.13.1", "libcap-2.78")) - args = parser.parse_args() - extract(args.archive, args.destination, args.root) diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bwrap-build-v2/inputs.json b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bwrap-build-v2/inputs.json deleted file mode 100644 index 988d8b096..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bwrap-build-v2/inputs.json +++ /dev/null @@ -1,33 +0,0 @@ -{ - "scope": "raw_build_input_preparation", - "verdict": "INCONCLUSIVE", - "downloaded_content_executed": false, - "source_archive_sha256": "0e85c8702248b8dd64144f594147553650cdd0e4c951690345faace79c217c4d", - "build_lock_sha256": "b28a5f4618b1890bb9b796206e14a56517cef73cf7811efb7340414ef7a77aab", - "files": [ - { - "name": "meson", - "version": "1.12.1", - "filename": "meson-1.12.1-py3-none-any.whl", - "sha256": "930bc7542cbd9f57009e182fd014eba48cf1a0180a7b9006d2d32d8f168d8b02", - "size": 1108668, - "url": "https://files.pythonhosted.org/packages/57/99/36cd25f9598eef70db97577440a70a360af505f6f01c2338a18d1556079c/meson-1.12.1-py3-none-any.whl", - "requires_python": ">=3.10", - "target_tag_rank": 866, - "metadata_url": "https://pypi.org/pypi/meson/json", - "metadata_sha256": "34801ae0bdba0e23dd7261e35a3868fdbac647d7006739dc086a8310a654c612" - }, - { - "name": "ninja", - "version": "1.13.2", - "filename": "ninja-1.13.2-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", - "sha256": "65a24341b5ac09fcadcc37082660be40a94174e51a937fabf6e2cae26225fa2c", - "size": 183365, - "url": "https://files.pythonhosted.org/packages/6e/53/ebfed7b689c338dd8ebeec9c0730c8d56821292f14e2536e5f3ef1a05744/ninja-1.13.2-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", - "requires_python": ">=3.8", - "target_tag_rank": 464, - "metadata_url": "https://pypi.org/pypi/ninja/1.13.2/json", - "metadata_sha256": "4093daaae326c46316289eb9ae9c758d4cd06ec75f75c59633e5a2cb0ff91dc3" - } - ] -} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bwrap-build-v2/inputs.sha256 b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bwrap-build-v2/inputs.sha256 deleted file mode 100644 index 85520ca5f..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bwrap-build-v2/inputs.sha256 +++ /dev/null @@ -1,9 +0,0 @@ -0e85c8702248b8dd64144f594147553650cdd0e4c951690345faace79c217c4d bubblewrap-0.13.1.tar.xz -b28a5f4618b1890bb9b796206e14a56517cef73cf7811efb7340414ef7a77aab build.lock -e4037153d4c6d1f4a8d2f6e165fb9dba67aec3ec9e4d7b2a85a91c1a31017ad1 inputs.json -98c7382ae519c9a34e3ada3fa4ebe585c7872b855b6f240fd3bcab783e189362 readset.json -0d621e562fd932ccf67b9660fb018e468a683d7b827541df27813228c996bb11 libcap-2.78.tar.xz -fa84c9906cc850a744733a7a57427c011f9a9ba1c7e924d4a4a73dc8a45275f6 sha256sums.asc -1cbac37787a24dbeee6812c46a6c87c56b37d0859deb4f94b4e5d9d1ad5d83a0 source-provenance.json -930bc7542cbd9f57009e182fd014eba48cf1a0180a7b9006d2d32d8f168d8b02 wheelhouse/meson-1.12.1-py3-none-any.whl -65a24341b5ac09fcadcc37082660be40a94174e51a937fabf6e2cae26225fa2c wheelhouse/ninja-1.13.2-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bwrap-build-v2/readset.json b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bwrap-build-v2/readset.json deleted file mode 100644 index 329ba1d7a..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bwrap-build-v2/readset.json +++ /dev/null @@ -1,16 +0,0 @@ -{ - "source": "https://github.com/containers/bubblewrap", - "tag": "v0.13.1", - "commit": "362936aea0d5c6e70f7af75d2d44b5a914cb4ba2", - "release_url": "https://github.com/containers/bubblewrap/releases/tag/v0.13.1", - "release_date": "2026-10-14", - "published_tar_filename": "bubblewrap-0.13.1.tar.xz", - "published_tar_sha256": "0e85c8702248b8dd64144f594147553650cdd0e4c951690345faace79c217c4d", - "source_git_blobs": { - "meson.build": "08d28c97e64932dd24811c41623a12506fa84ce1", - "meson_options.txt": "f4f4ac708911764eebbbecb0dee84391366f3a8d", - "README.md": "69addd6f4cbda5ccd1c47e7022a022d305ad4965" - }, - "observed_build_requirements": ["C compiler", "Meson >=0.49.0", "libcap development dependency required", "optional libselinux", "explicit registered Python via -Dpython"], - "state": "source read only; archive not downloaded, source not built, no isolation capability validated" -} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bwrap-build-v2/run.sbatch b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bwrap-build-v2/run.sbatch deleted file mode 100644 index 910449f54..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bwrap-build-v2/run.sbatch +++ /dev/null @@ -1,148 +0,0 @@ -#!/usr/bin/env bash -# Build a task-local tool; only its pure utility test and version/help are run. -#SBATCH --job-name=qwen38-bwrap-build-v2 -#SBATCH --nodes=1 -#SBATCH --ntasks=1 -#SBATCH --cpus-per-task=4 -#SBATCH --mem=8G -#SBATCH --time=00:10:00 -set -euo pipefail -umask 077 -: "${TASK_ROOT:?}" "${BASE_PYTHON:?}" "${UV_BIN:?}" "${SOURCE_DIR:?}" -: "${OUTPUT_DIR:?}" "${BUILD_VENV:?}" "${INSTALL_PREFIX:?}" "${LIBCAP_PREFIX:?}" -: "${SCRIPT_SHA256:?}" "${EXTRACTOR_SHA256:?}" "${INPUTS_SHA256:?}" -: "${PYTHON_SHA256:?}" "${UV_SHA256:?}" -[[ -n ${SLURM_JOB_ID:-} && ${SLURM_GPUS_ON_NODE:-0} == 0 ]] -root=$(realpath -e "$TASK_ROOT") -inside_root() { case "$(realpath -m "$1")" in "$root"/*) ;; *) exit 2;; esac; } -for path in "$BASE_PYTHON" "$UV_BIN" "$SOURCE_DIR" "$OUTPUT_DIR" "$BUILD_VENV" "$INSTALL_PREFIX" "$LIBCAP_PREFIX"; do inside_root "$path"; done -check_digest() { - [[ $2 =~ ^[a-f0-9]{64}$ ]] - [[ $(sha256sum "$1" | cut -d' ' -f1) == "$2" ]] -} -check_digest "$SOURCE_DIR/run.sbatch" "$SCRIPT_SHA256" -check_digest "$SOURCE_DIR/extract_source.py" "$EXTRACTOR_SHA256" -check_digest "$SOURCE_DIR/inputs.sha256" "$INPUTS_SHA256" -check_digest "$BASE_PYTHON" "$PYTHON_SHA256" -check_digest "$UV_BIN" "$UV_SHA256" -cmp "${BASH_SOURCE[0]}" "$SOURCE_DIR/run.sbatch" -(cd "$SOURCE_DIR" && sha256sum --check --strict inputs.sha256) -for path in "$OUTPUT_DIR" "$BUILD_VENV" "$INSTALL_PREFIX" "$LIBCAP_PREFIX"; do [[ ! -e $path && ! -L $path ]]; done -task_uv=$UV_BIN -for variable in "${!UV_@}" "${!PIP_@}"; do [[ -z $variable ]] || unset "$variable"; done -export UV_OFFLINE=true UV_PYTHON_DOWNLOADS=never -export TMPDIR="$root/tmp/qwen38-bwrap-build-v2-${SLURM_JOB_ID}" -export XDG_CACHE_HOME="$root/cache/qwen38-bwrap-build-v2" -export UV_CACHE_DIR="$XDG_CACHE_HOME/uv" HF_HOME="$XDG_CACHE_HOME/huggingface" -export TORCH_HOME="$XDG_CACHE_HOME/torch" TRITON_CACHE_DIR="$XDG_CACHE_HOME/triton" -export VLLM_CACHE_ROOT="$XDG_CACHE_HOME/vllm" UV_PYTHON_INSTALL_DIR="$root/software/python" -export PYTHONNOUSERSITE=1 PYTHONDONTWRITEBYTECODE=1 -unset PYTHONPATH PYTHONHOME PYTHONOPTIMIZE PYTHONINSPECT PYTHONSTARTUP VIRTUAL_ENV -unset DESTDIR MAKEFLAGS MFLAGS GNUMAKEFLAGS MAKEFILES MAKEOVERRIDES -unset CC CXX CFLAGS CXXFLAGS CPPFLAGS LDFLAGS PKG_CONFIG_PATH PKG_CONFIG_LIBDIR PKG_CONFIG_SYSROOT_DIR -for path in "$TMPDIR" "$XDG_CACHE_HOME" "$UV_CACHE_DIR" "$HF_HOME" "$TORCH_HOME" "$TRITON_CACHE_DIR" "$VLLM_CACHE_ROOT" "$UV_PYTHON_INSTALL_DIR"; do inside_root "$path"; mkdir -p "$path"; done -mkdir "$OUTPUT_DIR" -completed=false -phase=dependency_preflight -signal_name=none -finish() { - local status=$? - trap - EXIT - [[ $status != 0 || $completed == true ]] || status=1 - [[ $status == 0 ]] || completed=false - printf '%s\n' "$status" > "$OUTPUT_DIR/exit-code" - printf '{"completed":%s,"exit_code":%s,"phase":"%s","signal":"%s","job_id":"%s"}\n' \ - "$completed" "$status" "$phase" "$signal_name" "$SLURM_JOB_ID" > "$OUTPUT_DIR/terminal-status.json" - exit "$status" -} -trap finish EXIT -trap 'signal_name=TERM; exit 143' TERM -trap 'signal_name=INT; exit 130' INT -trap 'signal_name=HUP; exit 129' HUP -cp "$SOURCE_DIR/run.sbatch" "$SOURCE_DIR/extract_source.py" "$SOURCE_DIR/inputs.sha256" "$OUTPUT_DIR/" -cd "$root" -export PATH=/usr/bin:/bin -export PKG_CONFIG=/usr/bin/pkg-config -export CC -CC=$(command -v cc) -printf '%s\n' "$CC" > "$OUTPUT_DIR/compiler-path.private.txt" -sha256sum "$(realpath -e "$CC")" > "$OUTPUT_DIR/compiler-sha256.private.txt" -"$CC" --version > "$OUTPUT_DIR/compiler-version.private.txt" -# Build only the reviewed static libcap dependency; env -i removes Make injection. -phase=libcap_static_build -"$BASE_PYTHON" -IB "$SOURCE_DIR/extract_source.py" \ - "$SOURCE_DIR/libcap-2.78.tar.xz" "$OUTPUT_DIR/libcap-source" --root libcap-2.78 -libcap_source="$OUTPUT_DIR/libcap-source/libcap-2.78/libcap" -libcap_args=( -C "$libcap_source" -j 4 "prefix=$LIBCAP_PREFIX" lib=lib - FAKEROOT= DESTDIR= DEPS= SHARED=no PTHREADS=no GOLANG=no PAM_CAP=no USE_GPERF=no - "CC=$CC" "BUILD_CC=$CC" AR=/usr/bin/ar RANLIB=/usr/bin/ranlib - BUILD_SED=/usr/bin/sed BUILD_GREP=/usr/bin/grep SHELL=/bin/sh ) -env -i PATH=/usr/bin:/bin TMPDIR="$TMPDIR" LANG=C LC_ALL=C \ - /usr/bin/make "${libcap_args[@]}" libcap.a libcap.pc -env -i PATH=/usr/bin:/bin TMPDIR="$TMPDIR" LANG=C LC_ALL=C \ - /usr/bin/make "${libcap_args[@]}" install-static-cap -inside_root "$LIBCAP_PREFIX/lib/libcap.a" -inside_root "$LIBCAP_PREFIX/include/sys/capability.h" -inside_root "$LIBCAP_PREFIX/lib/pkgconfig/libcap.pc" -sha256sum "$LIBCAP_PREFIX/lib/libcap.a" "$LIBCAP_PREFIX/include/sys/capability.h" \ - "$LIBCAP_PREFIX/lib/pkgconfig/libcap.pc" > "$OUTPUT_DIR/libcap-installed.private.sha256" -export PKG_CONFIG_PATH="$LIBCAP_PREFIX/lib/pkgconfig" -[[ $(pkg-config --variable=libdir libcap) == "$LIBCAP_PREFIX/lib" ]] -[[ $(pkg-config --modversion libcap) == '2.78' ]] -pkg-config --modversion libcap > "$OUTPUT_DIR/libcap-version.txt" -pkg-config --cflags --libs libcap > "$OUTPUT_DIR/libcap-flags.private.txt" -phase=build_environment -"$task_uv" --no-config --offline venv --python "$BASE_PYTHON" "$BUILD_VENV" -"$task_uv" --no-config --offline pip install --python "$BUILD_VENV/bin/python" \ - --no-index --find-links "$SOURCE_DIR/wheelhouse" --no-deps --no-build \ - --require-hashes -r "$SOURCE_DIR/build.lock" -"$BUILD_VENV/bin/python" -IB - "$OUTPUT_DIR/build-packages.json" <<'PY' -import importlib.metadata -import json -import sys -packages = {p.metadata['Name'].lower(): p.version for p in importlib.metadata.distributions()} -if packages != {'meson': '1.12.1', 'ninja': '1.13.2'}: - raise ValueError('Unexpected build package set') -with open(sys.argv[1], 'x') as stream: - json.dump(packages, stream, sort_keys=True) - stream.write('\n') -PY -export PATH="$BUILD_VENV/bin:$PATH" -export NINJA="$BUILD_VENV/bin/ninja" -phase=source_extraction -"$BASE_PYTHON" -IB "$SOURCE_DIR/extract_source.py" \ - "$SOURCE_DIR/bubblewrap-0.13.1.tar.xz" "$OUTPUT_DIR/source" --root bubblewrap-0.13.1 -phase=configure_compile -"$BUILD_VENV/bin/meson" setup "$OUTPUT_DIR/build" "$OUTPUT_DIR/source/bubblewrap-0.13.1" \ - --prefix="$INSTALL_PREFIX" --wrap-mode=nodownload \ - -Dpython="$BUILD_VENV/bin/python" -Dman=disabled \ - -Dbash_completion=disabled -Dzsh_completion=disabled -Dtests=true -"$BUILD_VENV/bin/meson" compile -C "$OUTPUT_DIR/build" -j 4 bwrap test-utils -# Capture the actual link command and require the reviewed task-local archive. -"$NINJA" -C "$OUTPUT_DIR/build" -t commands bwrap > "$OUTPUT_DIR/link-commands.private.txt" -grep -Fq -- "$LIBCAP_PREFIX/lib/libcap.a" "$OUTPUT_DIR/link-commands.private.txt" -phase=utility_test -"$BUILD_VENV/bin/meson" test -C "$OUTPUT_DIR/build" --no-rebuild --print-errorlogs test-utils -phase=task_local_install -"$BUILD_VENV/bin/meson" install -C "$OUTPUT_DIR/build" --no-rebuild --destdir='' -inside_root "$INSTALL_PREFIX/bin/bwrap" -[[ ! -u "$INSTALL_PREFIX/bin/bwrap" && ! -g "$INSTALL_PREFIX/bin/bwrap" ]] -phase=version_help_checks -"$INSTALL_PREFIX/bin/bwrap" --version > "$OUTPUT_DIR/version.txt" -[[ $(cat "$OUTPUT_DIR/version.txt") == 'bubblewrap 0.13.1' ]] -"$INSTALL_PREFIX/bin/bwrap" --help > "$OUTPUT_DIR/help.txt" 2>&1 -for flag in --unshare-user --unshare-net --unshare-pid --disable-userns \ - --assert-userns-disabled --clearenv --remount-ro --size; do - grep -Fq -- "$flag" "$OUTPUT_DIR/help.txt" -done -sha256sum "$INSTALL_PREFIX/bin/bwrap" > "$OUTPUT_DIR/binary-sha256.private.txt" -ldd "$INSTALL_PREFIX/bin/bwrap" > "$OUTPUT_DIR/linked-libraries.private.txt" -/usr/bin/readelf -d "$INSTALL_PREFIX/bin/bwrap" > "$OUTPUT_DIR/dynamic-section.txt" -if grep -E 'NEEDED.*libcap\.so' "$OUTPUT_DIR/dynamic-section.txt"; then exit 1; fi -phase=postflight -check_digest "$SOURCE_DIR/run.sbatch" "$SCRIPT_SHA256" -check_digest "$SOURCE_DIR/extract_source.py" "$EXTRACTOR_SHA256" -check_digest "$SOURCE_DIR/inputs.sha256" "$INPUTS_SHA256" -(cd "$SOURCE_DIR" && sha256sum --check --strict inputs.sha256) -phase=complete -completed=true diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bwrap-build-v2/source-provenance.json b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bwrap-build-v2/source-provenance.json deleted file mode 100644 index 646504f38..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bwrap-build-v2/source-provenance.json +++ /dev/null @@ -1,8 +0,0 @@ -{ - "source_url": "https://www.kernel.org/pub/linux/libs/security/linux-privs/libcap2/libcap-2.78.tar.xz", - "checksum_url": "https://www.kernel.org/pub/linux/libs/security/linux-privs/libcap2/sha256sums.asc", - "source_archive_sha256": "0d621e562fd932ccf67b9660fb018e468a683d7b827541df27813228c996bb11", - "published_checksum_match": true, - "signature_verified": false, - "downloaded_code_executed": false -} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bwrap-capability-v1/run.sbatch b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bwrap-capability-v1/run.sbatch deleted file mode 100644 index c23e0d354..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-bwrap-capability-v1/run.sbatch +++ /dev/null @@ -1,41 +0,0 @@ -#!/usr/bin/env bash -# Read only the selected system isolation tool's version/help/bytes in Slurm. -#SBATCH --job-name=qwen38-bwrap-capability -#SBATCH --nodes=1 -#SBATCH --ntasks=1 -#SBATCH --cpus-per-task=1 -#SBATCH --mem=2G -#SBATCH --time=00:01:00 -set -euo pipefail -umask 077 -: "${TASK_ROOT:?}" "${SOURCE_DIR:?}" "${OUTPUT_DIR:?}" "${SCRIPT_SHA256:?}" -[[ -n ${SLURM_JOB_ID:-} && ${SLURM_GPUS_ON_NODE:-0} == 0 ]] -root=$(realpath -e "$TASK_ROOT") -inside_root() { case "$(realpath -m "$1")" in "$root"/*) ;; *) exit 2;; esac; } -for path in "$SOURCE_DIR" "$OUTPUT_DIR"; do inside_root "$path"; done -[[ $SCRIPT_SHA256 =~ ^[a-f0-9]{64}$ ]] -[[ $(sha256sum "$SOURCE_DIR/run.sbatch" | cut -d' ' -f1) == "$SCRIPT_SHA256" ]] -cmp "${BASH_SOURCE[0]}" "$SOURCE_DIR/run.sbatch" -[[ ! -e "$OUTPUT_DIR" && ! -L "$OUTPUT_DIR" ]] -export TMPDIR="$root/tmp/qwen38-bwrap-capability-${SLURM_JOB_ID}" -export XDG_CACHE_HOME="$root/cache/qwen38-bwrap-capability" -for path in "$TMPDIR" "$XDG_CACHE_HOME"; do inside_root "$path"; mkdir -p "$path"; done -mkdir "$OUTPUT_DIR" -cd "$root" -binary=$(command -v bwrap) -printf '%s\n' "$binary" > "$OUTPUT_DIR/binary-path.private.txt" -sha256sum "$binary" > "$OUTPUT_DIR/binary-sha256.private.txt" -"$binary" --version > "$OUTPUT_DIR/version.txt" -"$binary" --help > "$OUTPUT_DIR/help.txt" 2>&1 -for flag in --unshare-user --unshare-net --unshare-pid --disable-userns \ - --assert-userns-disabled --clearenv --remount-ro --size; do - present=false - if grep -Fq -- "$flag" "$OUTPUT_DIR/help.txt"; then present=true; fi - printf '%s\t%s\n' "$flag" "$present" >> "$OUTPUT_DIR/required-flags.tsv" -done -for tool in cc gcc meson ninja pkg-config; do - command -v "$tool" >> "$OUTPUT_DIR/build-tool-paths.private.txt" || true -done -cp "$SOURCE_DIR/run.sbatch" "$OUTPUT_DIR/" -[[ $(sha256sum "$SOURCE_DIR/run.sbatch" | cut -d' ' -f1) == "$SCRIPT_SHA256" ]] -printf '0\n' > "$OUTPUT_DIR/exit-code" diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-elf-context-v1/diagnose.py b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-elf-context-v1/diagnose.py deleted file mode 100644 index d5c5c1b15..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-elf-context-v1/diagnose.py +++ /dev/null @@ -1,133 +0,0 @@ -"""Compare explicit ELF lookup contexts without importing evaluator packages.""" -import hashlib -import json -import os -import re -import subprocess -import sys -import sysconfig -import time -from pathlib import Path - -root = Path(os.environ['TASK_ROOT']).resolve(strict=True) -output = Path(sys.argv[1]).resolve(strict=True) -base = Path(sys.base_prefix).resolve(strict=True) -site = Path(sysconfig.get_path('purelib')).resolve(strict=True) -if not all(p.is_relative_to(root) for p in (output, base, site)): - raise ValueError('Runtime/output root escapes task root') -deadline = time.monotonic() + 240 -clean_env = {'PATH': '/usr/bin:/bin', 'LC_ALL': 'C'} -scoped_env = {**clean_env, 'LD_LIBRARY_PATH': str(base / 'lib')} -observations = { - 'scope': 'controlled_elf_loader_context_diagnostic', 'verdict': 'INCONCLUSIVE', - 'runtime_changed': False, 'package_imports_performed': False, - 'cooperative_deadline_seconds': 240, 'slurm_walltime_limit_seconds': 300, - 'cooperative_deadline_excludes_blocking_file_enumeration_and_output_io': True, - 'inherited_loader_variable_presence': { - name: name in os.environ for name in ('LD_LIBRARY_PATH', 'LD_PRELOAD', 'LD_AUDIT') - }, - 'controls': {'clean': clean_env, 'base_lib': {**clean_env, 'LD_LIBRARY_PATH': '/opt/python/lib'}}, - 'selected_elf': [], 'target_comparison': {}, 'full_scan': [], 'command_attempts': [], - 'full_scan_complete': False, 'error_type': None, -} - - -def logical(path): - for alias, directory in (('/opt/python', base), ('/opt/site-packages', site)): - if path.is_relative_to(directory): - return alias + '/' + path.relative_to(directory).as_posix() - raise ValueError('Path is outside registered roots') - - -def run(args, env, prefix): - remaining = deadline - time.monotonic() - if remaining <= 0: - raise TimeoutError('Diagnostic deadline reached') - attempt = {'command': Path(args[0]).name, 'artifact_prefix': prefix, - 'completed': False, 'exception_type': None, 'exit_code': None} - observations['command_attempts'].append(attempt) - stdout = stderr = b'' - try: - result = subprocess.run(args, env=env, capture_output=True, - timeout=min(10, remaining), check=False) - stdout, stderr = result.stdout, result.stderr - attempt.update(completed=True, exit_code=result.returncode) - return result - except subprocess.TimeoutExpired as error: - stdout, stderr = error.stdout or b'', error.stderr or b'' - attempt['exception_type'] = type(error).__name__ - raise - except OSError as error: - attempt['exception_type'] = type(error).__name__ - raise - finally: - attempt['stdout_sha256'] = hashlib.sha256(stdout).hexdigest() - attempt['stderr_sha256'] = hashlib.sha256(stderr).hexdigest() - (output / (prefix + '.stdout.private.txt')).write_bytes(stdout) - (output / (prefix + '.stderr.private.txt')).write_bytes(stderr) - - -def ldd_record(path, env, prefix): - result = run(['/usr/bin/ldd', str(path)], env, prefix) - stdout = result.stdout.decode(errors='replace') - return { - 'path': logical(path), 'exit_code': result.returncode, - 'contains_not_found': 'not found' in stdout, - 'unresolved': re.findall(r'^\s*(\S+)\s+=>\s+not found\s*$', stdout, re.MULTILINE), - 'stdout_sha256': hashlib.sha256(result.stdout).hexdigest(), - 'stderr_sha256': hashlib.sha256(result.stderr).hexdigest(), - 'artifact_prefix': prefix, - } - - -status = 1 -try: - target = base / 'lib/python3.12/lib-dynload/_tkinter.cpython-312-x86_64-linux-gnu.so' - selected = [Path(sys._base_executable), target, base / 'lib/libtcl9.0.so', - base / 'lib/libtcl9tk9.0.so'] - for index, path in enumerate(selected): - resolved = path.resolve(strict=True) - if not resolved.is_relative_to(base) or not resolved.is_file(): - raise ValueError('Selected ELF escapes base runtime or is not a file') - raw = resolved.read_bytes() - if raw[:4] != b'\x7fELF': - raise ValueError('Selected file is not ELF') - result = run(['/usr/bin/readelf', '-d', str(resolved)], clean_env, f'selected-{index}') - if result.returncode != 0: - raise ValueError('readelf failed') - dynamic = result.stdout.decode(errors='replace') - # Full dynamic tags stay in the private readelf transcript, including paths. - tags = re.findall(r'\((NEEDED|RPATH|RUNPATH)\).*?\[(.*?)\]', dynamic) - needed = [value if '/' not in value else '' - for name, value in tags if name == 'NEEDED'] - observations['selected_elf'].append({ - 'path': logical(path), 'resolved_path': logical(resolved), 'bytes': len(raw), - 'sha256': hashlib.sha256(raw).hexdigest(), 'needed_library_names': needed, - 'rpath_present': any(name == 'RPATH' for name, _ in tags), - 'runpath_present': any(name == 'RUNPATH' for name, _ in tags), - 'readelf_stdout_sha256': hashlib.sha256(result.stdout).hexdigest(), - }) - for label, env in (('clean', clean_env), ('base_lib', scoped_env)): - observations['target_comparison'][label] = ldd_record(target, env, 'target-' + label) - for directory in (base, site): - for path in sorted(directory.rglob('*')): - if time.monotonic() >= deadline: - raise TimeoutError('Diagnostic deadline reached') - if not path.is_file() or path.is_symlink(): - continue - with path.open('rb') as stream: - if stream.read(4) != b'\x7fELF': - continue - index = len(observations['full_scan']) - observations['full_scan'].append(ldd_record(path, scoped_env, f'scan-{index:04d}')) - observations['full_scan_complete'] = True - status = 0 -except Exception as error: - observations['error_type'] = type(error).__name__ - raise -finally: - (output / 'observations.json').write_text(json.dumps(observations, indent=2) + '\n') - (output / 'context.private.json').write_text(json.dumps({ - 'base': str(base), 'site': str(site), 'scoped_environment': scoped_env, - }, indent=2) + '\n') -sys.exit(status) diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-elf-context-v1/exception-controls.json b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-elf-context-v1/exception-controls.json deleted file mode 100644 index b92df9bfd..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-elf-context-v1/exception-controls.json +++ /dev/null @@ -1,19 +0,0 @@ -{ - "scope": "actual_run_function_with_mock_subprocess", - "helper_sha256": "8d87ef1fcec257ea62ead056f8cd809187d3dc44768fcdc199a43274ccefd434", - "controls": [ - { - "case": "timeout", - "retained_partial_output": true, - "exception_preserved": true, - "attempt_recorded": true - }, - { - "case": "missing_tool", - "retained_partial_output": true, - "exception_preserved": true, - "attempt_recorded": true - } - ], - "external_commands_executed": false -} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-elf-context-v1/review-rejected-v1/diagnose.py b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-elf-context-v1/review-rejected-v1/diagnose.py deleted file mode 100644 index 26d8d77eb..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-elf-context-v1/review-rejected-v1/diagnose.py +++ /dev/null @@ -1,112 +0,0 @@ -"""Compare explicit ELF lookup contexts without importing evaluator packages.""" -import hashlib -import json -import os -import re -import subprocess -import sys -import sysconfig -import time -from pathlib import Path - -root = Path(os.environ['TASK_ROOT']).resolve(strict=True) -output = Path(sys.argv[1]).resolve(strict=True) -base = Path(sys.base_prefix).resolve(strict=True) -site = Path(sysconfig.get_path('purelib')).resolve(strict=True) -if not all(p.is_relative_to(root) for p in (output, base, site)): - raise ValueError('Runtime/output root escapes task root') -deadline = time.monotonic() + 240 -clean_env = {'PATH': '/usr/bin:/bin', 'LC_ALL': 'C'} -scoped_env = {**clean_env, 'LD_LIBRARY_PATH': str(base / 'lib')} -observations = { - 'scope': 'controlled_elf_loader_context_diagnostic', 'verdict': 'INCONCLUSIVE', - 'runtime_changed': False, 'package_imports_performed': False, - 'inherited_loader_variable_presence': { - name: name in os.environ for name in ('LD_LIBRARY_PATH', 'LD_PRELOAD', 'LD_AUDIT') - }, - 'controls': {'clean': clean_env, 'base_lib': {**clean_env, 'LD_LIBRARY_PATH': '/opt/python/lib'}}, - 'selected_elf': [], 'target_comparison': {}, 'full_scan': [], - 'full_scan_complete': False, 'error_type': None, -} - - -def logical(path): - for alias, directory in (('/opt/python', base), ('/opt/site-packages', site)): - if path.is_relative_to(directory): - return alias + '/' + path.relative_to(directory).as_posix() - raise ValueError('Path is outside registered roots') - - -def run(args, env, prefix): - remaining = deadline - time.monotonic() - if remaining <= 0: - raise TimeoutError('Diagnostic deadline reached') - result = subprocess.run(args, env=env, capture_output=True, - timeout=min(10, remaining), check=False) - (output / (prefix + '.stdout.private.txt')).write_bytes(result.stdout) - (output / (prefix + '.stderr.private.txt')).write_bytes(result.stderr) - return result - - -def ldd_record(path, env, prefix): - result = run(['/usr/bin/ldd', str(path)], env, prefix) - stdout = result.stdout.decode(errors='replace') - return { - 'path': logical(path), 'exit_code': result.returncode, - 'contains_not_found': 'not found' in stdout, - 'unresolved': re.findall(r'^\s*(\S+)\s+=>\s+not found\s*$', stdout, re.MULTILINE), - 'stdout_sha256': hashlib.sha256(result.stdout).hexdigest(), - 'stderr_sha256': hashlib.sha256(result.stderr).hexdigest(), - 'artifact_prefix': prefix, - } - - -status = 1 -try: - target = base / 'lib/python3.12/lib-dynload/_tkinter.cpython-312-x86_64-linux-gnu.so' - selected = [Path(sys._base_executable), target, base / 'lib/libtcl9.0.so', - base / 'lib/libtcl9tk9.0.so'] - for index, path in enumerate(selected): - resolved = path.resolve(strict=True) - if not resolved.is_relative_to(base) or not resolved.is_file(): - raise ValueError('Selected ELF escapes base runtime or is not a file') - raw = resolved.read_bytes() - if raw[:4] != b'\x7fELF': - raise ValueError('Selected file is not ELF') - result = run(['/usr/bin/readelf', '-d', str(resolved)], clean_env, f'selected-{index}') - if result.returncode != 0: - raise ValueError('readelf failed') - dynamic = result.stdout.decode(errors='replace') - # Paths stay private; the public facts retain hashes and literal origin-relative tags. - tags = re.findall(r'\((NEEDED|RPATH|RUNPATH)\).*?\[(.*?)\]', dynamic) - sanitized_tags = [(name, value.replace(str(base), '/opt/python').replace(str(site), '/opt/site-packages')) - for name, value in tags] - observations['selected_elf'].append({ - 'path': logical(path), 'resolved_path': logical(resolved), 'bytes': len(raw), - 'sha256': hashlib.sha256(raw).hexdigest(), 'dynamic_tags': sanitized_tags, - 'readelf_stdout_sha256': hashlib.sha256(result.stdout).hexdigest(), - }) - for label, env in (('clean', clean_env), ('base_lib', scoped_env)): - observations['target_comparison'][label] = ldd_record(target, env, 'target-' + label) - for directory in (base, site): - for path in sorted(directory.rglob('*')): - if time.monotonic() >= deadline: - raise TimeoutError('Diagnostic deadline reached') - if not path.is_file() or path.is_symlink(): - continue - with path.open('rb') as stream: - if stream.read(4) != b'\x7fELF': - continue - index = len(observations['full_scan']) - observations['full_scan'].append(ldd_record(path, scoped_env, f'scan-{index:04d}')) - observations['full_scan_complete'] = True - status = 0 -except Exception as error: - observations['error_type'] = type(error).__name__ - raise -finally: - (output / 'observations.json').write_text(json.dumps(observations, indent=2) + '\n') - (output / 'context.private.json').write_text(json.dumps({ - 'base': str(base), 'site': str(site), 'scoped_environment': scoped_env, - }, indent=2) + '\n') -sys.exit(status) diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-elf-context-v1/review-rejected-v1/run.sbatch b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-elf-context-v1/review-rejected-v1/run.sbatch deleted file mode 100644 index 5cfb03a5a..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-elf-context-v1/review-rejected-v1/run.sbatch +++ /dev/null @@ -1,53 +0,0 @@ -#!/usr/bin/env bash -#SBATCH --job-name=qwen38-elf-context -#SBATCH --nodes=1 -#SBATCH --ntasks=1 -#SBATCH --cpus-per-task=2 -#SBATCH --mem=4G -#SBATCH --time=00:05:00 -set -euo pipefail -umask 077 -: "${TASK_ROOT:?}" "${PYTHON_BIN:?}" "${SOURCE_DIR:?}" "${OUTPUT_DIR:?}" -: "${SCRIPT_SHA256:?}" "${DIAGNOSTIC_SHA256:?}" "${PYTHON_SHA256:?}" -[[ -n ${SLURM_JOB_ID:-} && ${SLURM_GPUS_ON_NODE:-0} == 0 ]] -root=$(realpath -e "$TASK_ROOT") -inside_root() { case "$(realpath -m "$1")" in "$root"/*) ;; *) exit 2;; esac; } -for path in "$PYTHON_BIN" "$SOURCE_DIR" "$OUTPUT_DIR"; do inside_root "$path"; done -check_digest() { [[ $2 =~ ^[a-f0-9]{64}$ ]] && [[ $(sha256sum "$1" | cut -d' ' -f1) == "$2" ]]; } -check_digest "$SOURCE_DIR/run.sbatch" "$SCRIPT_SHA256" -check_digest "$SOURCE_DIR/diagnose.py" "$DIAGNOSTIC_SHA256" -check_digest "$PYTHON_BIN" "$PYTHON_SHA256" -cmp "${BASH_SOURCE[0]}" "$SOURCE_DIR/run.sbatch" -[[ ! -e $OUTPUT_DIR && ! -L $OUTPUT_DIR ]] -export PATH=/usr/bin:/bin -export TMPDIR="$root/tmp/qwen38-elf-context-${SLURM_JOB_ID}" -export XDG_CACHE_HOME="$root/cache/qwen38-elf-context-${SLURM_JOB_ID}" -export PYTHONNOUSERSITE=1 PYTHONDONTWRITEBYTECODE=1 -unset PYTHONPATH PYTHONHOME PYTHONOPTIMIZE PYTHONINSPECT PYTHONSTARTUP VIRTUAL_ENV -for path in "$TMPDIR" "$XDG_CACHE_HOME"; do inside_root "$path"; mkdir -p "$path"; done -mkdir "$OUTPUT_DIR" -completed=false -phase=diagnostic -signal_name=none -finish() { - local status=$? - trap - EXIT - [[ $status != 0 || $completed == true ]] || status=1 - [[ $status == 0 ]] || completed=false - printf '%s\n' "$status" > "$OUTPUT_DIR/exit-code" - printf '{"completed":%s,"exit_code":%s,"phase":"%s","signal":"%s","job_id":"%s"}\n' \ - "$completed" "$status" "$phase" "$signal_name" "$SLURM_JOB_ID" > "$OUTPUT_DIR/terminal-status.json" - exit "$status" -} -trap finish EXIT -trap 'signal_name=TERM; exit 143' TERM -trap 'signal_name=INT; exit 130' INT -trap 'signal_name=HUP; exit 129' HUP -cp "$SOURCE_DIR/run.sbatch" "$SOURCE_DIR/diagnose.py" "$OUTPUT_DIR/" -cd "$root" -"$PYTHON_BIN" -IB "$SOURCE_DIR/diagnose.py" "$OUTPUT_DIR" -check_digest "$SOURCE_DIR/run.sbatch" "$SCRIPT_SHA256" -check_digest "$SOURCE_DIR/diagnose.py" "$DIAGNOSTIC_SHA256" -check_digest "$PYTHON_BIN" "$PYTHON_SHA256" -phase=complete -completed=true diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-elf-context-v1/run.sbatch b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-elf-context-v1/run.sbatch deleted file mode 100644 index 5cfb03a5a..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-elf-context-v1/run.sbatch +++ /dev/null @@ -1,53 +0,0 @@ -#!/usr/bin/env bash -#SBATCH --job-name=qwen38-elf-context -#SBATCH --nodes=1 -#SBATCH --ntasks=1 -#SBATCH --cpus-per-task=2 -#SBATCH --mem=4G -#SBATCH --time=00:05:00 -set -euo pipefail -umask 077 -: "${TASK_ROOT:?}" "${PYTHON_BIN:?}" "${SOURCE_DIR:?}" "${OUTPUT_DIR:?}" -: "${SCRIPT_SHA256:?}" "${DIAGNOSTIC_SHA256:?}" "${PYTHON_SHA256:?}" -[[ -n ${SLURM_JOB_ID:-} && ${SLURM_GPUS_ON_NODE:-0} == 0 ]] -root=$(realpath -e "$TASK_ROOT") -inside_root() { case "$(realpath -m "$1")" in "$root"/*) ;; *) exit 2;; esac; } -for path in "$PYTHON_BIN" "$SOURCE_DIR" "$OUTPUT_DIR"; do inside_root "$path"; done -check_digest() { [[ $2 =~ ^[a-f0-9]{64}$ ]] && [[ $(sha256sum "$1" | cut -d' ' -f1) == "$2" ]]; } -check_digest "$SOURCE_DIR/run.sbatch" "$SCRIPT_SHA256" -check_digest "$SOURCE_DIR/diagnose.py" "$DIAGNOSTIC_SHA256" -check_digest "$PYTHON_BIN" "$PYTHON_SHA256" -cmp "${BASH_SOURCE[0]}" "$SOURCE_DIR/run.sbatch" -[[ ! -e $OUTPUT_DIR && ! -L $OUTPUT_DIR ]] -export PATH=/usr/bin:/bin -export TMPDIR="$root/tmp/qwen38-elf-context-${SLURM_JOB_ID}" -export XDG_CACHE_HOME="$root/cache/qwen38-elf-context-${SLURM_JOB_ID}" -export PYTHONNOUSERSITE=1 PYTHONDONTWRITEBYTECODE=1 -unset PYTHONPATH PYTHONHOME PYTHONOPTIMIZE PYTHONINSPECT PYTHONSTARTUP VIRTUAL_ENV -for path in "$TMPDIR" "$XDG_CACHE_HOME"; do inside_root "$path"; mkdir -p "$path"; done -mkdir "$OUTPUT_DIR" -completed=false -phase=diagnostic -signal_name=none -finish() { - local status=$? - trap - EXIT - [[ $status != 0 || $completed == true ]] || status=1 - [[ $status == 0 ]] || completed=false - printf '%s\n' "$status" > "$OUTPUT_DIR/exit-code" - printf '{"completed":%s,"exit_code":%s,"phase":"%s","signal":"%s","job_id":"%s"}\n' \ - "$completed" "$status" "$phase" "$signal_name" "$SLURM_JOB_ID" > "$OUTPUT_DIR/terminal-status.json" - exit "$status" -} -trap finish EXIT -trap 'signal_name=TERM; exit 143' TERM -trap 'signal_name=INT; exit 130' INT -trap 'signal_name=HUP; exit 129' HUP -cp "$SOURCE_DIR/run.sbatch" "$SOURCE_DIR/diagnose.py" "$OUTPUT_DIR/" -cd "$root" -"$PYTHON_BIN" -IB "$SOURCE_DIR/diagnose.py" "$OUTPUT_DIR" -check_digest "$SOURCE_DIR/run.sbatch" "$SCRIPT_SHA256" -check_digest "$SOURCE_DIR/diagnose.py" "$DIAGNOSTIC_SHA256" -check_digest "$PYTHON_BIN" "$PYTHON_SHA256" -phase=complete -completed=true diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-elf-context-v2/alias-control.json b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-elf-context-v2/alias-control.json deleted file mode 100644 index 3137bc400..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-elf-context-v2/alias-control.json +++ /dev/null @@ -1,6 +0,0 @@ -{ - "scope": "lexical_vs_resolved_interpreter_alias_control", - "original_lexical_alias_rejected": true, - "resolved_alias_valid": true, - "model_or_interpreter_execution": false -} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-elf-context-v2/diagnose.py b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-elf-context-v2/diagnose.py deleted file mode 100644 index 7b9789264..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-elf-context-v2/diagnose.py +++ /dev/null @@ -1,133 +0,0 @@ -"""Compare explicit ELF lookup contexts without importing evaluator packages.""" -import hashlib -import json -import os -import re -import subprocess -import sys -import sysconfig -import time -from pathlib import Path - -root = Path(os.environ['TASK_ROOT']).resolve(strict=True) -output = Path(sys.argv[1]).resolve(strict=True) -base = Path(sys.base_prefix).resolve(strict=True) -site = Path(sysconfig.get_path('purelib')).resolve(strict=True) -if not all(p.is_relative_to(root) for p in (output, base, site)): - raise ValueError('Runtime/output root escapes task root') -deadline = time.monotonic() + 240 -clean_env = {'PATH': '/usr/bin:/bin', 'LC_ALL': 'C'} -scoped_env = {**clean_env, 'LD_LIBRARY_PATH': str(base / 'lib')} -observations = { - 'scope': 'controlled_elf_loader_context_diagnostic', 'verdict': 'INCONCLUSIVE', - 'runtime_changed': False, 'package_imports_performed': False, - 'cooperative_deadline_seconds': 240, 'slurm_walltime_limit_seconds': 300, - 'cooperative_deadline_excludes_blocking_file_enumeration_and_output_io': True, - 'inherited_loader_variable_presence': { - name: name in os.environ for name in ('LD_LIBRARY_PATH', 'LD_PRELOAD', 'LD_AUDIT') - }, - 'controls': {'clean': clean_env, 'base_lib': {**clean_env, 'LD_LIBRARY_PATH': '/opt/python/lib'}}, - 'selected_elf': [], 'target_comparison': {}, 'full_scan': [], 'command_attempts': [], - 'full_scan_complete': False, 'error_type': None, -} - - -def logical(path): - for alias, directory in (('/opt/python', base), ('/opt/site-packages', site)): - if path.is_relative_to(directory): - return alias + '/' + path.relative_to(directory).as_posix() - raise ValueError('Path is outside registered roots') - - -def run(args, env, prefix): - remaining = deadline - time.monotonic() - if remaining <= 0: - raise TimeoutError('Diagnostic deadline reached') - attempt = {'command': Path(args[0]).name, 'artifact_prefix': prefix, - 'completed': False, 'exception_type': None, 'exit_code': None} - observations['command_attempts'].append(attempt) - stdout = stderr = b'' - try: - result = subprocess.run(args, env=env, capture_output=True, - timeout=min(10, remaining), check=False) - stdout, stderr = result.stdout, result.stderr - attempt.update(completed=True, exit_code=result.returncode) - return result - except subprocess.TimeoutExpired as error: - stdout, stderr = error.stdout or b'', error.stderr or b'' - attempt['exception_type'] = type(error).__name__ - raise - except OSError as error: - attempt['exception_type'] = type(error).__name__ - raise - finally: - attempt['stdout_sha256'] = hashlib.sha256(stdout).hexdigest() - attempt['stderr_sha256'] = hashlib.sha256(stderr).hexdigest() - (output / (prefix + '.stdout.private.txt')).write_bytes(stdout) - (output / (prefix + '.stderr.private.txt')).write_bytes(stderr) - - -def ldd_record(path, env, prefix): - result = run(['/usr/bin/ldd', str(path)], env, prefix) - stdout = result.stdout.decode(errors='replace') - return { - 'path': logical(path), 'exit_code': result.returncode, - 'contains_not_found': 'not found' in stdout, - 'unresolved': re.findall(r'^\s*(\S+)\s+=>\s+not found\s*$', stdout, re.MULTILINE), - 'stdout_sha256': hashlib.sha256(result.stdout).hexdigest(), - 'stderr_sha256': hashlib.sha256(result.stderr).hexdigest(), - 'artifact_prefix': prefix, - } - - -status = 1 -try: - target = base / 'lib/python3.12/lib-dynload/_tkinter.cpython-312-x86_64-linux-gnu.so' - selected = [Path(sys._base_executable).resolve(strict=True), target, base / 'lib/libtcl9.0.so', - base / 'lib/libtcl9tk9.0.so'] - for index, path in enumerate(selected): - resolved = path.resolve(strict=True) - if not resolved.is_relative_to(base) or not resolved.is_file(): - raise ValueError('Selected ELF escapes base runtime or is not a file') - raw = resolved.read_bytes() - if raw[:4] != b'\x7fELF': - raise ValueError('Selected file is not ELF') - result = run(['/usr/bin/readelf', '-d', str(resolved)], clean_env, f'selected-{index}') - if result.returncode != 0: - raise ValueError('readelf failed') - dynamic = result.stdout.decode(errors='replace') - # Full dynamic tags stay in the private readelf transcript, including paths. - tags = re.findall(r'\((NEEDED|RPATH|RUNPATH)\).*?\[(.*?)\]', dynamic) - needed = [value if '/' not in value else '' - for name, value in tags if name == 'NEEDED'] - observations['selected_elf'].append({ - 'path': logical(path), 'resolved_path': logical(resolved), 'bytes': len(raw), - 'sha256': hashlib.sha256(raw).hexdigest(), 'needed_library_names': needed, - 'rpath_present': any(name == 'RPATH' for name, _ in tags), - 'runpath_present': any(name == 'RUNPATH' for name, _ in tags), - 'readelf_stdout_sha256': hashlib.sha256(result.stdout).hexdigest(), - }) - for label, env in (('clean', clean_env), ('base_lib', scoped_env)): - observations['target_comparison'][label] = ldd_record(target, env, 'target-' + label) - for directory in (base, site): - for path in sorted(directory.rglob('*')): - if time.monotonic() >= deadline: - raise TimeoutError('Diagnostic deadline reached') - if not path.is_file() or path.is_symlink(): - continue - with path.open('rb') as stream: - if stream.read(4) != b'\x7fELF': - continue - index = len(observations['full_scan']) - observations['full_scan'].append(ldd_record(path, scoped_env, f'scan-{index:04d}')) - observations['full_scan_complete'] = True - status = 0 -except Exception as error: - observations['error_type'] = type(error).__name__ - raise -finally: - (output / 'observations.json').write_text(json.dumps(observations, indent=2) + '\n') - (output / 'context.private.json').write_text(json.dumps({ - 'base': str(base), 'site': str(site), 'scoped_environment': scoped_env, - }, indent=2) + '\n') -sys.exit(status) diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-elf-context-v2/run.sbatch b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-elf-context-v2/run.sbatch deleted file mode 100644 index 556714382..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-elf-context-v2/run.sbatch +++ /dev/null @@ -1,53 +0,0 @@ -#!/usr/bin/env bash -#SBATCH --job-name=qwen38-elf-context-v2 -#SBATCH --nodes=1 -#SBATCH --ntasks=1 -#SBATCH --cpus-per-task=2 -#SBATCH --mem=4G -#SBATCH --time=00:05:00 -set -euo pipefail -umask 077 -: "${TASK_ROOT:?}" "${PYTHON_BIN:?}" "${SOURCE_DIR:?}" "${OUTPUT_DIR:?}" -: "${SCRIPT_SHA256:?}" "${DIAGNOSTIC_SHA256:?}" "${PYTHON_SHA256:?}" -[[ -n ${SLURM_JOB_ID:-} && ${SLURM_GPUS_ON_NODE:-0} == 0 ]] -root=$(realpath -e "$TASK_ROOT") -inside_root() { case "$(realpath -m "$1")" in "$root"/*) ;; *) exit 2;; esac; } -for path in "$PYTHON_BIN" "$SOURCE_DIR" "$OUTPUT_DIR"; do inside_root "$path"; done -check_digest() { [[ $2 =~ ^[a-f0-9]{64}$ ]] && [[ $(sha256sum "$1" | cut -d' ' -f1) == "$2" ]]; } -check_digest "$SOURCE_DIR/run.sbatch" "$SCRIPT_SHA256" -check_digest "$SOURCE_DIR/diagnose.py" "$DIAGNOSTIC_SHA256" -check_digest "$PYTHON_BIN" "$PYTHON_SHA256" -cmp "${BASH_SOURCE[0]}" "$SOURCE_DIR/run.sbatch" -[[ ! -e $OUTPUT_DIR && ! -L $OUTPUT_DIR ]] -export PATH=/usr/bin:/bin -export TMPDIR="$root/tmp/qwen38-elf-context-v2-${SLURM_JOB_ID}" -export XDG_CACHE_HOME="$root/cache/qwen38-elf-context-v2-${SLURM_JOB_ID}" -export PYTHONNOUSERSITE=1 PYTHONDONTWRITEBYTECODE=1 -unset PYTHONPATH PYTHONHOME PYTHONOPTIMIZE PYTHONINSPECT PYTHONSTARTUP VIRTUAL_ENV -for path in "$TMPDIR" "$XDG_CACHE_HOME"; do inside_root "$path"; mkdir -p "$path"; done -mkdir "$OUTPUT_DIR" -completed=false -phase=diagnostic -signal_name=none -finish() { - local status=$? - trap - EXIT - [[ $status != 0 || $completed == true ]] || status=1 - [[ $status == 0 ]] || completed=false - printf '%s\n' "$status" > "$OUTPUT_DIR/exit-code" - printf '{"completed":%s,"exit_code":%s,"phase":"%s","signal":"%s","job_id":"%s"}\n' \ - "$completed" "$status" "$phase" "$signal_name" "$SLURM_JOB_ID" > "$OUTPUT_DIR/terminal-status.json" - exit "$status" -} -trap finish EXIT -trap 'signal_name=TERM; exit 143' TERM -trap 'signal_name=INT; exit 130' INT -trap 'signal_name=HUP; exit 129' HUP -cp "$SOURCE_DIR/run.sbatch" "$SOURCE_DIR/diagnose.py" "$OUTPUT_DIR/" -cd "$root" -"$PYTHON_BIN" -IB "$SOURCE_DIR/diagnose.py" "$OUTPUT_DIR" -check_digest "$SOURCE_DIR/run.sbatch" "$SCRIPT_SHA256" -check_digest "$SOURCE_DIR/diagnose.py" "$DIAGNOSTIC_SHA256" -check_digest "$PYTHON_BIN" "$PYTHON_SHA256" -phase=complete -completed=true diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-elf-dependency-v1/diagnose.py b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-elf-dependency-v1/diagnose.py deleted file mode 100644 index 27a437cb4..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-elf-dependency-v1/diagnose.py +++ /dev/null @@ -1,47 +0,0 @@ -"""Read-only reproduction of the first unresolved ELF in frozen inventory order.""" -import json -import os -import re -import shutil -import subprocess -import sys -import sysconfig -from pathlib import Path - -root = Path(os.environ['TASK_ROOT']).resolve(strict=True) -output = Path(sys.argv[1]).resolve(strict=True) -if not output.is_relative_to(root): - raise ValueError('Output escapes task root') -base = Path(sys.base_prefix).resolve(strict=True) -site = Path(sysconfig.get_path('purelib')).resolve(strict=True) -if not all(p.is_relative_to(root) for p in (base, site)): - raise ValueError('Interpreter roots escape task root') -ldd = shutil.which('ldd') -if ldd is None: - raise ValueError('ldd unavailable') -scanned = 0 -found = None -for alias, directory in (('/opt/python', base), ('/opt/site-packages', site)): - for path in sorted(directory.rglob('*')): - if not path.is_file() or path.is_symlink(): - continue - with path.open('rb') as stream: - if stream.read(4) != b'\x7fELF': - continue - scanned += 1 - result = subprocess.run([ldd, str(path)], capture_output=True, timeout=10, check=False) - stdout = result.stdout.decode(errors='replace') - if 'not found' in stdout: - found = {'path': alias + '/' + path.relative_to(directory).as_posix(), - 'ldd_exit_code': result.returncode, - 'unresolved': re.findall(r'^\s*(\S+)\s+=>\s+not found\s*$', stdout, re.MULTILINE)} - (output / 'ldd-stdout.private.txt').write_bytes(result.stdout) - (output / 'ldd-stderr.private.txt').write_bytes(result.stderr) - (output / 'selected-elf.private.txt').write_text(str(path) + '\n') - break - if found is not None: - break -report = {'scope': 'first_missing_elf_inventory_diagnostic', 'scanned_elf_count': scanned, - 'first_missing': found, 'runtime_changed': False, 'evaluator_or_model_executed': False, - 'verdict': 'INCONCLUSIVE'} -(output / 'observations.json').write_text(json.dumps(report, indent=2) + '\n') diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-elf-dependency-v1/run.sbatch b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-elf-dependency-v1/run.sbatch deleted file mode 100644 index 3aa3b9c1c..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-elf-dependency-v1/run.sbatch +++ /dev/null @@ -1,53 +0,0 @@ -#!/usr/bin/env bash -#SBATCH --job-name=qwen38-elf-dependency -#SBATCH --nodes=1 -#SBATCH --ntasks=1 -#SBATCH --cpus-per-task=2 -#SBATCH --mem=4G -#SBATCH --time=00:03:00 -set -euo pipefail -umask 077 -: "${TASK_ROOT:?}" "${PYTHON_BIN:?}" "${SOURCE_DIR:?}" "${OUTPUT_DIR:?}" -: "${SCRIPT_SHA256:?}" "${DIAGNOSTIC_SHA256:?}" "${PYTHON_SHA256:?}" -[[ -n ${SLURM_JOB_ID:-} && ${SLURM_GPUS_ON_NODE:-0} == 0 ]] -root=$(realpath -e "$TASK_ROOT") -inside_root() { case "$(realpath -m "$1")" in "$root"/*) ;; *) exit 2;; esac; } -for path in "$PYTHON_BIN" "$SOURCE_DIR" "$OUTPUT_DIR"; do inside_root "$path"; done -check_digest() { [[ $2 =~ ^[a-f0-9]{64}$ ]] && [[ $(sha256sum "$1" | cut -d' ' -f1) == "$2" ]]; } -check_digest "$SOURCE_DIR/run.sbatch" "$SCRIPT_SHA256" -check_digest "$SOURCE_DIR/diagnose.py" "$DIAGNOSTIC_SHA256" -check_digest "$PYTHON_BIN" "$PYTHON_SHA256" -cmp "${BASH_SOURCE[0]}" "$SOURCE_DIR/run.sbatch" -[[ ! -e $OUTPUT_DIR && ! -L $OUTPUT_DIR ]] -export PATH=/usr/bin:/bin -export TMPDIR="$root/tmp/qwen38-elf-${SLURM_JOB_ID}" -export XDG_CACHE_HOME="$root/cache/qwen38-elf-${SLURM_JOB_ID}" -export PYTHONNOUSERSITE=1 PYTHONDONTWRITEBYTECODE=1 -unset PYTHONPATH PYTHONHOME PYTHONOPTIMIZE PYTHONINSPECT PYTHONSTARTUP VIRTUAL_ENV -for path in "$TMPDIR" "$XDG_CACHE_HOME"; do inside_root "$path"; mkdir -p "$path"; done -mkdir "$OUTPUT_DIR" -completed=false -phase=diagnostic -signal_name=none -finish() { - local status=$? - trap - EXIT - [[ $status != 0 || $completed == true ]] || status=1 - [[ $status == 0 ]] || completed=false - printf '%s\n' "$status" > "$OUTPUT_DIR/exit-code" - printf '{"completed":%s,"exit_code":%s,"phase":"%s","signal":"%s","job_id":"%s"}\n' \ - "$completed" "$status" "$phase" "$signal_name" "$SLURM_JOB_ID" > "$OUTPUT_DIR/terminal-status.json" - exit "$status" -} -trap finish EXIT -trap 'signal_name=TERM; exit 143' TERM -trap 'signal_name=INT; exit 130' INT -trap 'signal_name=HUP; exit 129' HUP -cp "$SOURCE_DIR/run.sbatch" "$SOURCE_DIR/diagnose.py" "$OUTPUT_DIR/" -cd "$root" -"$PYTHON_BIN" -IB "$SOURCE_DIR/diagnose.py" "$OUTPUT_DIR" -check_digest "$SOURCE_DIR/run.sbatch" "$SCRIPT_SHA256" -check_digest "$SOURCE_DIR/diagnose.py" "$DIAGNOSTIC_SHA256" -check_digest "$PYTHON_BIN" "$PYTHON_SHA256" -phase=complete -completed=true diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-evaluator-synthetic-v3/benign-reference-controls.json b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-evaluator-synthetic-v3/benign-reference-controls.json deleted file mode 100644 index 2a5d0061d..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-evaluator-synthetic-v3/benign-reference-controls.json +++ /dev/null @@ -1,21 +0,0 @@ -{ - "benign_reference_cases": 13, - "cases": [ - "observed_success", - "exit_bool", - "exit_nonzero", - "timeout", - "wrong_phase", - "wrong_output_digest", - "wrong_rootfs", - "wrong_executor", - "empty_checks", - "missing_check", - "integer_check", - "integer_passed", - "duplicate_key" - ], - "optimized_python": true, - "evaluator_or_sandbox_executed": false, - "passed": true -} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-evaluator-synthetic-v3/plan.json b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-evaluator-synthetic-v3/plan.json deleted file mode 100644 index bf0cb3bfa..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-evaluator-synthetic-v3/plan.json +++ /dev/null @@ -1,38 +0,0 @@ -{ - "scope": "trusted_four_case_synthetic_evaluator", - "verdict": "INCONCLUSIVE", - "status": "reviewed_not_submitted", - "requested_cpus": 4, - "requested_memory_gib": 16, - "requested_gpus": 0, - "slurm_walltime_seconds": 900, - "probe_walltime_seconds": 60, - "evaluator_parallel": 1, - "evaluator_memory_override_bytes": 2147483648, - "test_details": true, - "reference_job": 2006, - "source_sha256": { - "runtime.py": "997cbfed8398e8ad62d1beb4f6b4bc6e61a0f4572b9156cfd0c6f3fb52b03233", - "run-synthetic.sbatch": "7fc63a32d51e81ad67b7e18f444e749e145d962ee51d0de095dec943330ee984", - "synthetic_probe.py": "04f2f5f7aba97fec1d4bab54c818cbe46f73356d46477afefcd815c33fe5a7ef", - "test_reference_completion.py": "b72b00324fab0abfdda3ca17be9256f5568b308670239da457f01a7b02bb7456", - "test_benign_reference.py": "d0972d47d9c99da6876d3aa8bbe1f031710fcc8289c263c8a196e42093ba634e", - "reference-controls.json": "970fc4e1f5e8b2ca17bf7c85f2ea64068137fa74f0158874d5b7a5e72630a4fb", - "benign-reference-controls.json": "d47899b42d66a13add50bf99675c54a34af1400fe68601f40037d17824e19a96" - }, - "reference_identities": { - "job_id": "2006", - "reference_rootfs_sha256": "4bd529089780175cc234db209a71ec12dc6cc9b842dfd0cbf089a7b6088e4f89", - "reference_terminal_sha256": "356403f7048f578d4c9f8d6f80fe33a2ba61b667b53bdc9a66644c3c4e4cee5b", - "reference_slurm_sha256": "5cc3aa2b872d63a74efb15af3d0717d61a9f2172509e894faed6e959f999165f", - "reference_probe_execution_sha256": "8fdda7600618f49b2e9d020d7b720ede439c11e4cbd4f0750028aa4047654d89", - "reference_probe_stdout_sha256": "95b04d5b1776fe8359c6dd4e2597aafa9050c9454c48c7167a2e34a4933c0806", - "scope": "observed CPU2006 receipt identities only; no successor activated" - }, - "initial_snapshot_lint_findings": 11, - "new_control_style_findings_fixed": 2, - "retained_snapshot_lint_findings": 9, - "evaluator_executed": false, - "generated_code_executed": false, - "measured_e2e_effect": null -} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-evaluator-synthetic-v3/reference-controls.json b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-evaluator-synthetic-v3/reference-controls.json deleted file mode 100644 index 92d128cf2..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-evaluator-synthetic-v3/reference-controls.json +++ /dev/null @@ -1 +0,0 @@ -{"reference_completion_cases": 18, "optimized_python": true} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-evaluator-synthetic-v3/run-synthetic.sbatch b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-evaluator-synthetic-v3/run-synthetic.sbatch deleted file mode 100644 index c200bb834..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-evaluator-synthetic-v3/run-synthetic.sbatch +++ /dev/null @@ -1,194 +0,0 @@ -#!/usr/bin/env bash -# Trusted handwritten evaluator diagnostic; no real task data or model output. -#SBATCH --job-name=qwen38-eval-synthetic-v3 -#SBATCH --nodes=1 -#SBATCH --ntasks=1 -#SBATCH --cpus-per-task=4 -#SBATCH --mem=16G -#SBATCH --time=00:15:00 -set -euo pipefail -umask 077 -: "${TASK_ROOT:?}" -: "${SANDBOX_BUNDLE:?}" -: "${SANDBOX_FREEZE_SHA256:?}" -: "${SANDBOX_VENV:?}" -: "${REFERENCE_INSTALLATION:?}" -: "${REFERENCE_ROOTFS_SHA256:?}" -: "${REFERENCE_TERMINAL_SHA256:?}" -: "${REFERENCE_SLURM_SHA256:?}" -: "${REFERENCE_JOB_ID:?}" -: "${REFERENCE_EXECUTION_SHA256:?}" "${REFERENCE_STDOUT_SHA256:?}" -: "${BWRAP_BIN:?}" "${BWRAP_SHA256:?}" "${PYTHON_SHA256:?}" -: "${SYNTHETIC_SOURCE_DIR:?}" -: "${SYNTHETIC_RUNTIME_SHA256:?}" -: "${SYNTHETIC_PROBE_SHA256:?}" -: "${SYNTHETIC_SCRIPT_SHA256:?}" -: "${SYNTHETIC_OUTPUT:?}" -unset PYTHONPATH PYTHONHOME PYTHONOPTIMIZE PYTHONINSPECT PYTHONSTARTUP VIRTUAL_ENV -[[ -n ${SLURM_JOB_ID:-} && ${SLURM_GPUS_ON_NODE:-0} == 0 ]] -root=$(realpath -e "$TASK_ROOT") -check_path() { - local value - value=$(realpath -m "$1") - case "$value" in "$root"/*) ;; *) echo 'Path escapes task root.' >&2; exit 2;; esac -} -check_digest() { - if [[ ! "$2" =~ ^[a-f0-9]{64}$ ]]; then return 2; fi - [[ $(sha256sum "$1" | cut -d' ' -f1) == "$2" ]] -} -for path in "$SANDBOX_BUNDLE" "$SANDBOX_VENV" "$REFERENCE_INSTALLATION" \ - "$SYNTHETIC_SOURCE_DIR" "$SYNTHETIC_OUTPUT" "$BWRAP_BIN"; do check_path "$path"; done -source_dir=$(cd -- "$SYNTHETIC_SOURCE_DIR" && pwd -P) -check_digest "$SANDBOX_VENV/bin/python" "$PYTHON_SHA256" -check_digest "$BWRAP_BIN" "$BWRAP_SHA256" -export PATH="$(dirname "$BWRAP_BIN"):/usr/bin:/bin" -[[ $(command -v bwrap) == "$BWRAP_BIN" ]] -check_digest "$source_dir/run-synthetic.sbatch" "$SYNTHETIC_SCRIPT_SHA256" -cmp "${BASH_SOURCE[0]}" "$source_dir/run-synthetic.sbatch" -check_digest "$source_dir/runtime.py" "$SYNTHETIC_RUNTIME_SHA256" -check_digest "$source_dir/synthetic_probe.py" "$SYNTHETIC_PROBE_SHA256" -check_digest "$SANDBOX_BUNDLE/freeze.sha256" "$SANDBOX_FREEZE_SHA256" -(cd "$SANDBOX_BUNDLE" && sha256sum --check --strict freeze.sha256) -# Bind both the scheduler outcome and the repaired completion record. -check_digest "$REFERENCE_INSTALLATION/terminal-status.json" "$REFERENCE_TERMINAL_SHA256" -check_digest "$REFERENCE_INSTALLATION/slurm-final.private.txt" "$REFERENCE_SLURM_SHA256" -"$SANDBOX_VENV/bin/python" -IB - "$REFERENCE_INSTALLATION" "$REFERENCE_JOB_ID" <<'PYREF' -import json -import re -import sys -from pathlib import Path -reference = Path(sys.argv[1]) -job = sys.argv[2] -if not re.fullmatch(r"[0-9]+", job): - raise ValueError('Invalid reference job ID') -def unique_keys(pairs): - value = {} - for key, item in pairs: - if key in value: - raise ValueError('Duplicate terminal-record key') - value[key] = item - return value -terminal = json.loads((reference / 'terminal-status.json').read_bytes(), object_pairs_hook=unique_keys) -if terminal.get('completed') is not True or type(terminal.get('exit_code')) is not int: - raise ValueError('Reference terminal record has invalid types') -if terminal != {'completed': True, 'signal': 'none', 'exit_code': 0, 'job_id': job}: - raise ValueError('Reference installation did not complete normally') -slurm = (reference / 'slurm-final.private.txt').read_text() -for key, value in {'JobId': job, 'JobState': 'COMPLETED', 'ExitCode': '0:0'}.items(): - actual = re.findall(r'(? "$SYNTHETIC_OUTPUT/exit-code" - printf '{"completed":%s,"signal":"%s","exit_code":%s,"job_id":"%s"}\n' \ - "$task_completed" "$task_signal" "$status" "$SLURM_JOB_ID" > "$SYNTHETIC_OUTPUT/terminal-status.json" - exit "$status" -} -trap finish_synthetic EXIT -trap 'task_completed=false; task_signal=TERM; exit 143' TERM -trap 'task_completed=false; task_signal=INT; exit 130' INT -trap 'task_completed=false; task_signal=HUP; exit 129' HUP -# END TERMINAL RECORD -cp "$source_dir/run-synthetic.sbatch" "$source_dir/runtime.py" "$source_dir/synthetic_probe.py" "$SYNTHETIC_OUTPUT/" -cd "$root" -"$SANDBOX_VENV/bin/python" -IB "$source_dir/runtime.py" inventory --bundle "$SANDBOX_BUNDLE" \ - --output "$SYNTHETIC_OUTPUT/rootfs-before" -"$SANDBOX_VENV/bin/python" -IB - "$REFERENCE_INSTALLATION/rootfs-before/rootfs.json" \ - "$SYNTHETIC_OUTPUT/rootfs-before/rootfs.json" <<'PY' -import json -import sys -from pathlib import Path -before, current = [json.loads(Path(path).read_bytes()) for path in sys.argv[1:]] -for record in (before, current): - record.pop('rootfs_digest') - record['entries'] = [row for row in record['entries'] if row['path'] != '/opt/probe.py'] -if before != current: - raise ValueError('Installed runtime changed beyond the explicitly bound trusted probe') -PY -probe_status=0 -"$SANDBOX_VENV/bin/python" -IB "$source_dir/runtime.py" probe --bundle "$SANDBOX_BUNDLE" \ - --inventory "$SYNTHETIC_OUTPUT/rootfs-before" --output "$SYNTHETIC_OUTPUT/probe" || probe_status=$? -printf '%s\n' "$probe_status" > "$SYNTHETIC_OUTPUT/probe-status" -post_status=0 -run_postcheck() { - local name=$1 status=0 - shift - "$@" || status=$? - printf '%s\t%s\n' "$name" "$status" >> "$SYNTHETIC_OUTPUT/postflight-status.tsv" - if [[ "$status" != 0 ]]; then post_status=$status; fi -} -check_frozen_files() { (cd "$SANDBOX_BUNDLE" && sha256sum --check --strict freeze.sha256); } -run_postcheck inventory "$SANDBOX_VENV/bin/python" -IB "$source_dir/runtime.py" inventory \ - --bundle "$SANDBOX_BUNDLE" --output "$SYNTHETIC_OUTPUT/rootfs-after" -run_postcheck runtime-byte-comparison cmp "$SYNTHETIC_OUTPUT/rootfs-before/rootfs.json" \ - "$SYNTHETIC_OUTPUT/rootfs-after/rootfs.json" -run_postcheck runtime-script check_digest "$source_dir/runtime.py" "$SYNTHETIC_RUNTIME_SHA256" -run_postcheck synthetic-probe check_digest "$source_dir/synthetic_probe.py" "$SYNTHETIC_PROBE_SHA256" -run_postcheck batch-script check_digest "$source_dir/run-synthetic.sbatch" "$SYNTHETIC_SCRIPT_SHA256" -run_postcheck freeze-manifest check_digest "$SANDBOX_BUNDLE/freeze.sha256" "$SANDBOX_FREEZE_SHA256" -run_postcheck freeze-files check_frozen_files -run_postcheck python-binary check_digest "$SANDBOX_VENV/bin/python" "$PYTHON_SHA256" -run_postcheck bwrap-binary check_digest "$BWRAP_BIN" "$BWRAP_SHA256" -if [[ "$probe_status" != 0 ]]; then exit "$probe_status"; fi -if [[ "$post_status" != 0 ]]; then exit "$post_status"; fi -echo 'Trusted synthetic evaluator diagnostic completed; generated-code executor remains unqualified.' -task_completed=true diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-evaluator-synthetic-v3/runtime.py b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-evaluator-synthetic-v3/runtime.py deleted file mode 100644 index 51f1903f3..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-evaluator-synthetic-v3/runtime.py +++ /dev/null @@ -1,481 +0,0 @@ -"""Resolve pinned EvalPlus metadata, inventory readonly binds and run a trusted probe.""" - -import argparse -import configparser -import hashlib -import importlib.metadata -import json -import os -import re -import resource -import shutil -import signal -import subprocess -import sys -import sysconfig -import threading -import tomllib -from pathlib import Path - -REVISION = "26d6d00bb1fd0fa37f39c99d5290da67891d1c5e" - - -def canonical(value): - return json.dumps(value, sort_keys=True, separators=(",", ":"), allow_nan=False).encode() - - -def sha(raw): - return hashlib.sha256(raw).hexdigest() - - -def write(path, value): - with path.open("xb") as stream: - stream.write(canonical(value) + b"\n") - - -def git(source, *args): - return subprocess.check_output(["git", "-C", str(source), *args]) - - -def source_identity(bundle): - source = bundle / "source" - if git(source, "rev-parse", "HEAD").decode().strip() != REVISION: - raise ValueError("wrong EvalPlus source revision") - if git(source, "diff", "HEAD", "--"): - raise ValueError("tracked EvalPlus source changed") - return { - "evaluator_revision": REVISION, - "archive_sha256": sha(git(source, "archive", "--format=tar", "HEAD")), - } - - -def requirements(bundle): - identity = source_identity(bundle) - config = configparser.ConfigParser() - config.read(bundle / "source/setup.cfg") - dependencies = [ - line.strip() for line in config["options"]["install_requires"].splitlines() if line.strip() - ] - build = tomllib.loads((bundle / "source/pyproject.toml").read_text())["build-system"][ - "requires" - ] - (bundle / "dependencies.in").write_text("\n".join(dependencies) + "\n") - (bundle / "build.in").write_text("\n".join([*build, "wheel"]) + "\n") - write( - bundle / "source.json", - { - **identity, - "official_default_dependencies": dependencies, - "build_dependencies": [*build, "wheel"], - "extras_enabled": [], - "generated_code_executed": False, - }, - ) - - -def verify_source(bundle): - expected = json.loads((bundle / "source.json").read_bytes()) - if any( - source_identity(bundle)[name] != expected[name] - for name in ("evaluator_revision", "archive_sha256") - ): - raise ValueError("EvalPlus source snapshot drift") - - -def tree(root, alias): - result = [] - for path in sorted(root.rglob("*")): - relative = path.relative_to(root).as_posix() - name = alias + "/" + relative - if path.is_symlink(): - target = os.readlink(path) - if Path(target).is_absolute() or not path.resolve(strict=True).is_relative_to(root): - raise ValueError("runtime symlink escapes its readonly root") - result.append({"path": name, "type": "symlink", "target": target}) - elif path.is_dir(): - result.append({"path": name, "type": "directory", "mode": path.stat().st_mode & 0o777}) - elif path.is_file(): - result.append( - { - "path": name, - "type": "file", - "mode": path.stat().st_mode & 0o777, - "bytes": path.stat().st_size, - "sha256": sha(path.read_bytes()), - } - ) - else: - raise ValueError("unsupported runtime filesystem entry") - return result - - -def system_dependencies(roots): - # Mirror the explicit relocated loader directory; inherit no host loader state. - loader_env = { - "PATH": "/usr/bin:/bin", "LC_ALL": "C", - "LD_LIBRARY_PATH": str(roots[0] / "lib"), - } - ldd = shutil.which("ldd") - if ldd is None: - raise ValueError("system ELF dependency scanner unavailable") - dependencies = {} - for root in roots: - for path in sorted(root.rglob("*")): - if not path.is_file() or path.is_symlink(): - continue - with path.open("rb") as stream: - if stream.read(4) != b"\x7fELF": - continue - result = subprocess.run( - [ldd, str(path)], capture_output=True, timeout=10, check=False, env=loader_env - ) - output = result.stdout.decode(errors="replace") - if result.returncode != 0 or "not found" in output: - raise ValueError("ELF runtime dependency missing") - for line in output.splitlines(): - match = re.search(r"(?:=>\s*)?(/[^\s]+)", line) - if not match: - continue - original = Path(match.group(1)) - resolved = original.resolve(strict=True) - if any(resolved.is_relative_to(bound) for bound in roots): - continue - if not str(original).startswith(("/lib/", "/lib64/", "/usr/lib/", "/usr/lib64/")): - raise ValueError("unexpected external runtime library") - dependencies[str(original)] = str(resolved) - return dependencies - - -def inventory(bundle, output): - verify_source(bundle) - task_root = Path(os.environ["TASK_ROOT"]).resolve(strict=True) - base = Path(sys.base_prefix).resolve(strict=True) - site = Path(sysconfig.get_path("purelib")).resolve(strict=True) - if not base.is_relative_to(task_root) or not site.is_relative_to(task_root): - raise ValueError("runtime roots must remain inside TASK_ROOT") - executable = Path(sys._base_executable).resolve(strict=True).relative_to(base).as_posix() - bwrap = shutil.which("bwrap") - if bwrap is None: - raise ValueError("bubblewrap is unavailable on this compute node") - help_text = subprocess.check_output([bwrap, "--help"], stderr=subprocess.STDOUT).decode() - for flag in ( - "--unshare-user", - "--unshare-net", - "--unshare-pid", - "--disable-userns", - "--assert-userns-disabled", - "--clearenv", - "--remount-ro", - "--size", - ): - if flag not in help_text: - raise ValueError("required bubblewrap isolation option unavailable") - dependencies = system_dependencies((base, site)) - entries = tree(base, "/opt/python") + tree(site, "/opt/site-packages") - for destination, source in dependencies.items(): - path = Path(source) - entries.append( - { - "path": destination, - "type": "file", - "bytes": path.stat().st_size, - "sha256": sha(path.read_bytes()), - } - ) - probe = Path(__file__).with_name("synthetic_probe.py") - entries.append( - { - "path": "/opt/probe.py", - "type": "file", - "bytes": probe.stat().st_size, - "sha256": sha(probe.read_bytes()), - } - ) - entries.sort(key=lambda row: row["path"]) - # Bind-set identity describes actual readonly bytes, not an OCI image. - rootfs_digest = "sha256:" + sha(canonical(entries)) - packages = sorted( - [ - {"name": d.metadata["Name"], "version": d.version} - for d in importlib.metadata.distributions() - ], - key=lambda row: row["name"].lower(), - ) - # Match every pinned evaluator Python source byte to the installed package. - checked = 0 - source = bundle / "source" - for encoded in git(source, "ls-files", "-z", "evalplus").split(b"\0"): - if not encoded or not encoded.endswith(b".py"): - continue - name = encoded.decode() - if (source / name).read_bytes() != (site / name).read_bytes(): - raise ValueError("installed EvalPlus differs from pinned source") - checked += 1 - if checked == 0: - raise ValueError("no pinned evaluator Python sources verified") - output.mkdir(parents=True, exist_ok=False) - write( - output / "rootfs.json", - { - "schema": 1, - "rootfs_kind": "readonly-bind-set-v1", - "loader_policy": { - "scanner_inherits_environment": False, - "base_library_directory": "/opt/python/lib", - "sandbox_ld_library_path": "/opt/python/lib", - }, - "rootfs_digest": rootfs_digest, - "evaluator_revision": REVISION, - "entries": entries, - "python": sys.version.split()[0], - "packages": packages, - "checked_evaluator_sources": checked, - "executor": { - "kind": "bubblewrap", - "binary_sha256": sha(Path(bwrap).read_bytes()), - "version": subprocess.check_output([bwrap, "--version"]).decode().strip(), - }, - "generated_code_executed": False, - "qualified_generated_code_executor": False, - }, - ) - write( - output / "runtime-locators.private.json", - { - "python_root": str(base), - "site_root": str(site), - "python_relative": executable, - "bwrap": bwrap, - "system_libraries": dependencies, - }, - ) - - -def command(locators, probe, namespaces, sentinel): - args = [ - locators["bwrap"], - "--unshare-user", - "--uid", - "65534", - "--gid", - "65534", - "--unshare-net", - "--unshare-pid", - "--unshare-ipc", - "--unshare-uts", - "--hostname", - "sandbox", - "--disable-userns", - "--assert-userns-disabled", - "--cap-drop", - "ALL", - "--die-with-parent", - "--new-session", - "--clearenv", - "--ro-bind", - locators["python_root"], - "/opt/python", - "--ro-bind", - locators["site_root"], - "/opt/site-packages", - "--ro-bind", - str(probe), - "/opt/probe.py", - ] - for destination, source in sorted(locators["system_libraries"].items()): - args.extend(["--ro-bind", source, destination]) - args += [ - "--proc", - "/proc", - "--dev", - "/dev", - "--size", - "33554432", - "--tmpfs", - "/dev/shm", - "--size", - "67108864", - "--tmpfs", - "/work", - "--dir", - "/work/tmp", - "--dir", - "/work/output", - "--chdir", - "/work", - "--remount-ro", - "/", - ] - environment = { - "HOME": "/work", - "TMPDIR": "/work/tmp", - "PATH": "/opt/python/bin", - "LD_LIBRARY_PATH": "/opt/python/lib", - "PYTHONDONTWRITEBYTECODE": "1", - "OMP_NUM_THREADS": "1", - "OPENBLAS_NUM_THREADS": "1", - "LC_ALL": "C.UTF-8", - "PROBE_PARENT_NAMESPACES": json.dumps(namespaces), - "PROBE_UNMOUNTED_SENTINEL": str(sentinel), - } - for name, value in environment.items(): - args += ["--setenv", name, value] - return args + ["/opt/python/" + locators["python_relative"], "-I", "/opt/probe.py"] - - -def limits(): - os.sched_setaffinity(0, {min(os.sched_getaffinity(0))}) - for name, value in ( - ("AS", 2147483648), - ("CPU", 30), - ("NOFILE", 64), - ("FSIZE", 1048576), - ("CORE", 0), - ): - resource.setrlimit(getattr(resource, "RLIMIT_" + name), (value, value)) - - -def probe(inventory_path, output): - rootfs = json.loads((inventory_path / "rootfs.json").read_bytes()) - locators = json.loads((inventory_path / "runtime-locators.private.json").read_bytes()) - if sha(Path(locators["bwrap"]).read_bytes()) != rootfs["executor"]["binary_sha256"]: - raise ValueError("bubblewrap binary changed after inventory") - output.mkdir(parents=True, exist_ok=False) - sentinel = output / "unmounted-sentinel.private" - sentinel.write_text("synthetic host marker, not a credential") - namespaces = { - name: os.readlink("/proc/self/ns/" + name) for name in ("user", "net", "pid", "uts", "ipc") - } - args = command(locators, Path(__file__).with_name("synthetic_probe.py"), namespaces, sentinel) - timed_out = False - # Sample from the supervisor after Popen. The evaluator process remains free - # of an added observer thread before its own default multiprocessing fork. - import psutil - sampled = {"processes": 0, "threads": 0, "sum_vms_bytes": 0, - "sum_rss_bytes": 0, "sum_fds": 0, "samples": 0, "errors": 0} - known = {} - sampling_failures = [] - stop_sampling = threading.Event() - def observe(pid): - try: - parent = psutil.Process(pid) - except psutil.NoSuchProcess: - sampling_failures.append("launcher exited before sampling") - return - while not stop_sampling.is_set(): - totals = {"processes": 0, "threads": 0, "sum_vms_bytes": 0, - "sum_rss_bytes": 0, "sum_fds": 0} - try: - processes = [parent, *parent.children(recursive=True)] - except psutil.NoSuchProcess: - break - except psutil.AccessDenied: - sampling_failures.append("cannot inspect owned launcher descendants") - return - for child in processes: - try: - known[child.pid] = child.create_time() - memory = child.memory_info() - totals["processes"] += 1 - totals["threads"] += child.num_threads() - totals["sum_vms_bytes"] += memory.vms - totals["sum_rss_bytes"] += memory.rss - totals["sum_fds"] += child.num_fds() - except (psutil.NoSuchProcess, psutil.AccessDenied): - sampled["errors"] += 1 - for name, value in totals.items(): - sampled[name] = max(sampled[name], value) - sampled["samples"] += 1 - stop_sampling.wait(0.01) - with ( - (output / "stdout.json").open("xb") as stdout, - (output / "stderr.private.log").open("xb") as stderr, - ): - process = subprocess.Popen( - [sys.executable, str(Path(__file__).resolve()), "_exec", json.dumps(args)], - stdin=subprocess.DEVNULL, - stdout=stdout, - stderr=stderr, - start_new_session=True, - env={**os.environ, "PROBE_INHERITED_SECRET": "synthetic-must-be-cleared"}, - ) - observer = threading.Thread(target=observe, args=(process.pid,), daemon=True) - observer.start() - try: - code = process.wait(timeout=60) - except subprocess.TimeoutExpired: - timed_out = True - if os.getpgid(process.pid) != process.pid: - raise RuntimeError("owned sandbox process group changed unexpectedly") - os.killpg(process.pid, signal.SIGKILL) - code = process.wait(timeout=10) - finally: - stop_sampling.set() - observer.join(timeout=2) - remaining = [] - for pid, created in known.items(): - try: - child = psutil.Process(pid) - if child.create_time() == created and child.status() != psutil.STATUS_ZOMBIE: - remaining.append(pid) - except psutil.NoSuchProcess: - pass - result = { - "sampled_peaks": sampled, - "sampling_thread_joined": not observer.is_alive(), - "sampling_failures": sampling_failures, - "known_live_descendants_after_exit": remaining, - "sampling_limitation": "Polling may miss transient processes; sampled peaks are not hard bounds or exhaustive cleanup proof.", - "phase": "trusted_synthetic_evaluator", - "exit_code": code, - "timed_out": timed_out, - "rootfs_digest": rootfs["rootfs_digest"], - "executor": rootfs["executor"], - "policy_script_sha256": sha(Path(__file__).read_bytes()), - "probe_script_sha256": sha(Path(__file__).with_name("synthetic_probe.py").read_bytes()), - "stdout_sha256": sha((output / "stdout.json").read_bytes()), - "generated_code_executed": False, - "full_evaluator_attempt_requested": True, - "qualified_generated_code_executor": False, - "verdict": "INCONCLUSIVE", - } - write(output / "execution.json", result) - if code != 0 or timed_out or remaining or observer.is_alive() or sampling_failures or not sampled["samples"]: - raise ValueError("trusted synthetic evaluator probe failed; retain evidence without fallback") - payload = json.loads((output / "stdout.json").read_bytes()) - if payload.get("passed") is not True: - raise ValueError("trusted synthetic evaluator assertions failed") - - -def main(): - parser = argparse.ArgumentParser(description=__doc__) - parser.add_argument("phase", choices=("requirements", "verify-source", "inventory", "probe")) - parser.add_argument("--bundle", type=Path, required=True) - parser.add_argument("--output", type=Path) - parser.add_argument("--inventory", type=Path) - args = parser.parse_args() - if not os.environ.get("SLURM_JOB_ID") or os.environ.get("SLURM_GPUS_ON_NODE", "0") != "0": - raise ValueError("CPU-only Slurm allocation required") - if args.phase == "requirements": - requirements(args.bundle) - elif args.phase == "verify-source": - verify_source(args.bundle) - elif args.phase == "inventory": - inventory(args.bundle, args.output) - else: - probe(args.inventory, args.output) - - -if __name__ == "__main__": - if len(sys.argv) == 3 and sys.argv[1] == "_exec": - # A fresh interpreter sets limits before exec; no Python preexec callback - # runs between fork and exec in the monitoring supervisor. - if not os.environ.get("SLURM_JOB_ID") or os.environ.get("SLURM_GPUS_ON_NODE", "0") != "0": - raise ValueError("CPU-only Slurm allocation required") - command_args = json.loads(sys.argv[2]) - if not command_args or any(not isinstance(value, str) for value in command_args): - raise ValueError("invalid internal command") - limits() - os.execv(command_args[0], command_args) - else: - main() diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-evaluator-synthetic-v3/synthetic_probe.py b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-evaluator-synthetic-v3/synthetic_probe.py deleted file mode 100644 index 58b906eef..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-evaluator-synthetic-v3/synthetic_probe.py +++ /dev/null @@ -1,107 +0,0 @@ -"""Unexecuted trusted synthetic EvalPlus diagnostic; never accepts real task data.""" - -import contextlib -import hashlib -import importlib -import io -import json -import os -import resource -import sys -import time -from pathlib import Path - -sys.path.insert(0, "/opt/site-packages") -# This proposal measures a known, bounded handwritten process tree. It does not -# qualify these broad benign limits for model-generated code. -resource.setrlimit(resource.RLIMIT_NPROC, (64, 64)) -assert resource.getrlimit(resource.RLIMIT_AS) == (2147483648, 2147483648) -os.environ["EVALPLUS_MAX_MEMORY_BYTES"] = str(resource.getrlimit(resource.RLIMIT_AS)[0]) -os.environ.pop("EVALPLUS_TIMEOUT_PER_TASK", None) -os.environ["XDG_CACHE_HOME"] = "/work/cache" -os.environ["HUMANEVAL_OVERRIDE_PATH"] = "/work/synthetic-dataset.jsonl" - -import psutil - - -def digest(raw): - return hashlib.sha256(raw).hexdigest() - - -def main(): - work = Path("/work") - samples = work / "synthetic-samples.jsonl" - dataset = work / "synthetic-dataset.jsonl" - output = work / "output/synthetic-results.json" - assert not samples.exists() and not dataset.exists() and not output.exists() - assert not (work / "cache").exists() - solutions = { - "Synthetic/correct": "def f(x):\n return x + 1\n", - "Synthetic/plus_wrong": "def f(x):\n return 1\n", - "Synthetic/exception": "def f(x):\n raise ArithmeticError('synthetic')\n", - "Synthetic/input_timeout": "import time\ndef f(x):\n time.sleep(5)\n return x + 1\n", - } - rows = [ - { - "task_id": task_id, - "prompt": "def f(x):\n", - "canonical_solution": " return x + 1\n", - "contract": "", - "entry_point": "f", - "base_input": [[0]], - "plus_input": [[1]], - "atol": 0, - } - for task_id in solutions - ] - dataset.write_text("".join(json.dumps(row, sort_keys=True) + "\n" for row in rows)) - samples.write_text("".join( - json.dumps({"task_id": key, "solution": value}, sort_keys=True) + "\n" - for key, value in solutions.items() - )) - parent = psutil.Process() - captured_out, captured_err = io.StringIO(), io.StringIO() - started = time.monotonic() - with contextlib.redirect_stdout(captured_out), contextlib.redirect_stderr(captured_err): - evaluator = importlib.import_module("evalplus.evaluate") - evaluator.evaluate( - dataset="humaneval", samples=str(samples), output_file=str(output), - parallel=1, base_only=False, mini=False, noextreme=False, - test_details=True, - ) - result = json.loads(output.read_bytes()) - statuses = { - task_id: [[row["base_status"], row["plus_status"]] for row in values] - for task_id, values in result["eval"].items() - } - # The upstream per-input exception handler reports time-limit exceptions as - # fail. This is distinct from its outer child-process timeout classification. - expected = { - "Synthetic/correct": [["pass", "pass"]], - "Synthetic/plus_wrong": [["pass", "fail"]], - "Synthetic/exception": [["fail", "fail"]], - "Synthetic/input_timeout": [["fail", "fail"]], - } - checks = { - "four_exact_status_pairs": statuses == expected, - "dataset_hash": result["hash"] == hashlib.md5(dataset.read_bytes()).hexdigest(), - "bounded_output": output.stat().st_size <= 1048576, - "timeout_override_absent": "EVALPLUS_TIMEOUT_PER_TASK" not in os.environ, - } - payload = { - "checks": checks, "passed": all(checks.values()), "statuses": statuses, - "elapsed_seconds": time.monotonic() - started, - "dataset_sha256": digest(dataset.read_bytes()), "samples_sha256": digest(samples.read_bytes()), - "results_sha256": digest(output.read_bytes()), "results": result, - "evaluator_module_sha256": digest(Path(evaluator.__file__).read_bytes()), - "evaluator_stdout": captured_out.getvalue(), "evaluator_stderr": captured_err.getvalue(), - "live_children_after_evaluate": len(parent.children(recursive=True)), - "generated_code_executed": False, "real_benchmark_data_mounted": False, - "full_evaluator_executed": True, "qualified_generated_code_executor": False, - } - print(json.dumps(payload, sort_keys=True)) - return 0 if payload["passed"] else 2 - - -if __name__ == "__main__": - raise SystemExit(main()) diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-evaluator-synthetic-v3/test_benign_reference.py b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-evaluator-synthetic-v3/test_benign_reference.py deleted file mode 100644 index cbdc89c08..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-evaluator-synthetic-v3/test_benign_reference.py +++ /dev/null @@ -1,65 +0,0 @@ -"""Check the exact embedded reference validator with observed and mutated receipts.""" -import copy -import hashlib -import json -import subprocess -import sys -import tempfile -from pathlib import Path - -root = Path(__file__).parent -wrapper = (root / 'run-synthetic.sbatch').read_text() -code = wrapper.split("<<'PYBENIGN'\n", 1)[1].split('\nPYBENIGN', 1)[0] -reference = root / 'reference.private' -original_execution = json.loads((reference / 'probe/execution.json').read_bytes()) -original_stdout = json.loads((reference / 'probe/stdout.json').read_bytes()) -rootfs = json.loads((reference / 'rootfs-before/rootfs.json').read_bytes()) -bwrap = rootfs['executor']['binary_sha256'] -cases = ['observed_success', 'exit_bool', 'exit_nonzero', 'timeout', 'wrong_phase', - 'wrong_output_digest', 'wrong_rootfs', 'wrong_executor', 'empty_checks', - 'missing_check', 'integer_check', 'integer_passed', 'duplicate_key'] -with tempfile.TemporaryDirectory(dir=root) as temporary: - fixture = Path(temporary) - (fixture / 'probe').mkdir() - (fixture / 'rootfs-before').mkdir() - reference_rootfs = (reference / 'rootfs-before/rootfs.json').read_bytes() - (fixture / 'rootfs-before/rootfs.json').write_bytes(reference_rootfs) - for name in cases: - execution, stdout = copy.deepcopy(original_execution), copy.deepcopy(original_stdout) - if name == 'exit_bool': - execution['exit_code'] = False - elif name == 'exit_nonzero': - execution['exit_code'] = 1 - elif name == 'timeout': - execution['timed_out'] = True - elif name == 'wrong_phase': - execution['phase'] = 'unrelated' - elif name == 'wrong_rootfs': - execution['rootfs_digest'] = 'sha256:' + '0' * 64 - elif name == 'wrong_executor': - execution['executor']['binary_sha256'] = '0' * 64 - elif name == 'empty_checks': - stdout['checks'] = {} - elif name == 'missing_check': - del stdout['checks']['new_namespaces'] - elif name == 'integer_check': - stdout['checks']['new_namespaces'] = 1 - elif name == 'integer_passed': - stdout['passed'] = 1 - encoded = json.dumps(stdout).encode() - execution['stdout_sha256'] = hashlib.sha256(encoded).hexdigest() - if name == 'wrong_output_digest': - execution['stdout_sha256'] = '0' * 64 - encoded_execution = json.dumps(execution) - if name == 'duplicate_key': - encoded_execution = '{"exit_code":1,' + encoded_execution[1:] - (fixture / 'probe/execution.json').write_text(encoded_execution) - (fixture / 'probe/stdout.json').write_bytes(encoded) - result = subprocess.run([sys.executable, '-IBO', '-c', code, str(fixture), bwrap], - capture_output=True, check=False, timeout=5) - if (result.returncode == 0) != (name == 'observed_success'): - raise RuntimeError('Unexpected reference-validation outcome: ' + name) -report = {'benign_reference_cases': len(cases), 'cases': cases, 'optimized_python': True, - 'evaluator_or_sandbox_executed': False, 'passed': True} -(root / 'benign-reference-controls.json').write_text(json.dumps(report, indent=2) + '\n') -print(json.dumps(report)) diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-evaluator-synthetic-v3/test_reference_completion.py b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-evaluator-synthetic-v3/test_reference_completion.py deleted file mode 100644 index 8e29fcc2e..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-evaluator-synthetic-v3/test_reference_completion.py +++ /dev/null @@ -1,54 +0,0 @@ -"""Reference-completion checks only; no evaluator import or runtime execution.""" - -import json -import subprocess -import sys -import tempfile -from pathlib import Path - - -def main(): - root = Path(__file__).parent - script = (root / "run-synthetic.sbatch").read_text() - code = script.split("<<'PYREF'\n", 1)[1].split("\nPYREF", 1)[0] - good = {"completed": True, "signal": "none", "exit_code": 0, "job_id": "42"} - slurm = "JobId=42 JobState=COMPLETED ExitCode=0:0\n" - cases = [ - (good, slurm, "0", True), - ({**good, "completed": False}, slurm, "0", False), - ({**good, "completed": 1}, slurm, "0", False), - ({**good, "signal": "TERM", "exit_code": 143}, slurm, "143", False), - ({**good, "exit_code": False}, slurm, "0", False), - ({**good, "job_id": "41"}, slurm, "0", False), - (good, slurm.replace("COMPLETED", "TIMEOUT"), "0", False), - (good, slurm.replace("0:0", "0:15"), "0", False), - (good, slurm.replace("JobId=42", "JobId=41"), "0", False), - (good, slurm + "JobState=COMPLETED\n", "0", False), - (good, slurm + "JobState=\n", "0", False), - (good, slurm + "JobId=\n", "0", False), - (good, slurm + "ExitCode=\n", "0", False), - (good, slurm, "1", False), - (None, slurm, "0", False), - ('{"completed":false,"completed":true,"signal":"none","exit_code":0,"job_id":"42"}', slurm, "0", False), - ('{"completed":true,"signal":"none","exit_code":143,"exit_code":0,"job_id":"42"}', slurm, "0", False), - ('{"completed":true,"signal":"none","exit_code":0,"job_id":"41","job_id":"42"}', slurm, "0", False), - ] - with tempfile.TemporaryDirectory(prefix="reference-test-", dir=root) as temporary: - reference = Path(temporary) - for terminal, scheduler, marker, expected in cases: - path = reference / "terminal-status.json" - if terminal is None: - path.unlink(missing_ok=True) - else: - path.write_text(terminal if isinstance(terminal, str) else json.dumps(terminal)) - (reference / "slurm-final.private.txt").write_text(scheduler) - (reference / "exit-code").write_text(marker) - result = subprocess.run([sys.executable, "-IBO", "-c", code, str(reference), "42"], - capture_output=True, check=False, timeout=5) - if (result.returncode == 0) != expected: - raise RuntimeError("Reference completion check mismatch") - print(json.dumps({"reference_completion_cases": len(cases), "optimized_python": True})) - - -if __name__ == "__main__": - main() diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-executor-v1/bounds_driver.py b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-executor-v1/bounds_driver.py deleted file mode 100644 index 66061a0aa..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-executor-v1/bounds_driver.py +++ /dev/null @@ -1,613 +0,0 @@ -"""Supervise only fixed trusted probes and retain actual namespace-init exit evidence.""" - -import argparse -import ctypes -import hashlib -import importlib.util -import json -import os -import platform -import select -import signal -import subprocess -import sys -import time -from pathlib import Path - -import psutil - -RUNTIME_PATH = Path(__file__).with_name("runtime.py") -RUNTIME_SHA256 = "997cbfed8398e8ad62d1beb4f6b4bc6e61a0f4572b9156cfd0c6f3fb52b03233" -if hashlib.sha256(RUNTIME_PATH.read_bytes()).hexdigest() != RUNTIME_SHA256: - raise ValueError("unexpected runtime source bytes") -runtime_spec = importlib.util.spec_from_file_location( - "trusted_bound_runtime", RUNTIME_PATH -) -runtime = importlib.util.module_from_spec(runtime_spec) -runtime_spec.loader.exec_module(runtime) -START_TOKEN = b"inferrsi-trusted-start-v2\n" -AUTHORIZED_MARKER = b"PAYLOAD_START_AUTHORIZED\n" -REJECTED_MARKER = b"PAYLOAD_START_REJECTED\n" -MODES = ( - "gate_valid", - "gate_eof", - "gate_short", - "gate_wrong", - "gate_timeout", - "gate_early_launcher_death", - "bounds", - "synthetic", - "normal", - "timeout", - "launcher_kill", -) - - -def pidfd_syscall(number, *arguments): - if ( - sys.platform != "linux" - or platform.machine() != "x86_64" - or ctypes.sizeof(ctypes.c_void_p) != 8 - or ctypes.sizeof(ctypes.c_long) != 8 - ): - raise ValueError("pidfd bindings require Linux x86_64 LP64") - library = ctypes.CDLL(None, use_errno=True) - library.syscall.restype = ctypes.c_long - ctypes.set_errno(0) - result = library.syscall(ctypes.c_long(number), *arguments) - if result == -1: - error = ctypes.get_errno() - raise OSError(error, os.strerror(error)) - return result - - -def pidfd_open(pid): - if type(pid) is not int or not 0 < pid < 2**31: - raise ValueError("invalid positive PID") - return pidfd_syscall(434, ctypes.c_int(pid), ctypes.c_uint(0)) - - -def pidfd_send_signal(fd, value): - if type(fd) is not int or fd < 0 or type(value) is not int or value < 0: - raise ValueError("invalid pidfd or signal") - result = pidfd_syscall( - 424, ctypes.c_int(fd), ctypes.c_int(value), ctypes.c_void_p(), ctypes.c_uint(0) - ) - if result != 0: - raise ValueError("unexpected pidfd_send_signal result") - - -def capability(output): - report = { - "schema": 1, - "passed": False, - "job_id": os.environ["SLURM_JOB_ID"], - "implementation": "linux-x86_64-lp64-raw-syscalls-434-and-424", - "python_exposes_pidfd_open": hasattr(os, "pidfd_open"), - "python_exposes_pidfd_send_signal": hasattr(signal, "pidfd_send_signal"), - "driver_sha256": runtime.sha(Path(__file__).read_bytes()), - "python_binary_sha256": runtime.sha(Path(sys._base_executable).read_bytes()), - "kernel_release": platform.release(), - "generated_code_executed": False, - } - held = [] - child = None - errors = [] - try: - own_fd = pidfd_open(os.getpid()) - held.append(own_fd) - pidfd_send_signal(own_fd, 0) - report["own_fd_not_exited"] = not exited(own_fd) - child = subprocess.Popen( - [sys.executable, "-IB", "-c", "import time; time.sleep(4)"], - stdin=subprocess.DEVNULL, - stdout=subprocess.DEVNULL, - stderr=subprocess.DEVNULL, - ) - child_fd = pidfd_open(child.pid) - held.append(child_fd) - pidfd_send_signal(child_fd, int(signal.SIGKILL)) - report["child_pidfd_exit_observed"] = exited(child_fd, 5000) - report["child_exit_code"] = child.wait(timeout=5) - report["passed"] = ( - report["own_fd_not_exited"] - and report["child_pidfd_exit_observed"] - and report["child_exit_code"] == -signal.SIGKILL - ) - except BaseException as error: - report["error"] = { - "type": type(error).__name__, - "message": str(error), - "errno": getattr(error, "errno", None), - } - finally: - # The trusted child exits by itself if the API is unavailable; no PID kill fallback. - if child is not None: - try: - report["final_child_exit_code"] = child.wait(timeout=6) - except BaseException as error: - errors.append({"phase": "wait_child", "type": type(error).__name__}) - for fd in held: - try: - os.close(fd) - except OSError as error: - errors.append({"phase": "close_fd", "errno": error.errno}) - report["cleanup_errors"] = errors - if errors: - report["passed"] = False - runtime.write(output, report) - return 0 if report["passed"] else 2 - - -def unique(pairs): - result = {} - for key, value in pairs: - if key in result: - raise ValueError("duplicate JSON key") - result[key] = value - return result - - -def read_line(fd, deadline, raw): - while time.monotonic() < deadline and len(raw) <= 1048576: - ready, _, _ = select.select([fd], [], [], max(0, deadline - time.monotonic())) - if not ready: - break - data = os.read(fd, 1) - if not data: - raise ValueError("premature protocol EOF") - raw.extend(data) - if data == b"\n": - return bytes(raw) - raise ValueError("trusted protocol size or time limit exceeded") - - -def read_ready_file(path, deadline): - while time.monotonic() < deadline: - raw = path.read_bytes() - if len(raw) > 1048576: - raise ValueError("trusted output exceeds size limit") - if raw.endswith(b"\n"): - return raw - time.sleep(0.01) - raise ValueError("trusted readiness deadline exceeded") - - -def exited(fd, milliseconds=0): - poller = select.poll() - poller.register(fd, select.POLLIN) - events = poller.poll(milliseconds) - if any(mask & (select.POLLERR | select.POLLNVAL) for _, mask in events): - raise ValueError("pidfd wait failed") - return any(mask & (select.POLLIN | select.POLLHUP) for _, mask in events) - - -def status(pid): - return dict( - line.split(":", 1) - for line in Path(f"/proc/{pid}/status").read_text().splitlines() - if ":" in line - ) - - -def bind_init(record, launcher_pid): - pid = record.get("child-pid") - if type(pid) is not int or pid <= 1: - raise ValueError("invalid namespace-init PID") - fd = pidfd_open(pid) - try: - fields = status(pid) - if exited(fd) or int(fields["PPid"]) != launcher_pid: - raise ValueError("namespace init identity or parent changed") - namespace_pids = [int(value) for value in fields["NSpid"].split()] - if namespace_pids[0] != pid or namespace_pids[-1] != 1: - raise ValueError("reported child is not namespace PID1") - # The child is blocked before payload execution. Only the PID namespace - # is compared: bwrap can create a further user namespace during setup. - pid_namespace = os.readlink(f"/proc/{pid}/ns/pid") - if pid_namespace != f"pid:[{record.get('pid-namespace')}]": - raise ValueError("reported PID namespace identity differs") - if exited(fd): - raise ValueError("namespace init exited during binding") - return fd, { - "pid": pid, - "namespace_pids": namespace_pids, - "pid_namespace": pid_namespace, - } - except BaseException: - os.close(fd) - raise - - -def bind_children(init_pid, readiness): - wanted = readiness.get("live_descendant_pids") - if not isinstance(wanted, list) or len(wanted) != 2 or len(set(wanted)) != 2: - raise ValueError("expected two registered live trusted children") - if any(type(pid) is not int or pid <= 2 for pid in wanted): - raise ValueError("invalid trusted child PID") - held = [] - records = [] - try: - for child in psutil.Process(init_pid).children(recursive=True): - try: - fields = status(child.pid) - except FileNotFoundError: - continue - namespace_pids = [int(value) for value in fields["NSpid"].split()] - if namespace_pids[-1] not in wanted: - continue - fd = pidfd_open(child.pid) - held.append(fd) - current = status(child.pid) - if exited(fd) or current["NSpid"] != fields["NSpid"]: - raise ValueError("trusted child exited during binding") - records.append({"pid": child.pid, "namespace_pids": namespace_pids}) - if sorted(row["namespace_pids"][-1] for row in records) != sorted(wanted): - raise ValueError("registered trusted children not both observed") - return held, records - except BaseException: - for fd in held: - os.close(fd) - raise - - -def finalize_case( - report, process, init_fd, children, pipes, output, status_raw, rootfs -): - errors = [] - report["forced_cleanup"] = False - - def attempt(name, operation): - try: - return operation() - except BaseException as error: - errors.append( - {"operation": name, "type": type(error).__name__, "message": str(error)} - ) - return None - - # EOF cannot authorize the payload. Close this before releasing setup pipes. - if process is not None and process.stdin is not None: - attempt("close_authorization_pipe", process.stdin.close) - report["final_init_exit_observed"] = False - if init_fd is not None: - if attempt("check_init_exit", lambda: exited(init_fd)) is not True: - try: - pidfd_send_signal(init_fd, int(signal.SIGKILL)) - report["forced_cleanup"] = True - except ProcessLookupError: - # Exit between poll and signal is not a failed cleanup if the - # stable pidfd independently confirms that same process exited. - confirmed = attempt("confirm_signal_race", lambda: exited(init_fd)) - report["signal_race_already_exited"] = confirmed is True - if confirmed is not True: - errors.append( - {"operation": "signal_init", "type": "UnconfirmedExitRace"} - ) - except BaseException as error: - errors.append( - { - "operation": "signal_init", - "type": type(error).__name__, - "message": str(error), - } - ) - report["forced_init_exit_observed"] = attempt( - "wait_init_exit", lambda: exited(init_fd, 5000) - ) - if report["forced_init_exit_observed"] is not True: - errors.append( - {"operation": "wait_init_exit", "type": "ExitNotConfirmed"} - ) - report["final_init_exit_observed"] = ( - attempt("final_init_poll", lambda: exited(init_fd)) is True - ) - attempt("close_init_fd", lambda: os.close(init_fd)) - if process is not None: - if attempt("poll_launcher", process.poll) is None: - report["forced_cleanup"] = True - attempt("kill_launcher", process.kill) - attempt("wait_launcher", lambda: process.wait(timeout=5)) - report["final_launcher_exit_code"] = process.returncode - if process.returncode is None: - errors.append({"operation": "wait_launcher", "type": "ExitNotConfirmed"}) - report["cleanup_registered_child_exits"] = [] - for fd in children: - observed = attempt("wait_child_exit", lambda fd=fd: exited(fd, 1000)) - report["cleanup_registered_child_exits"].append(observed) - if observed is not True: - errors.append({"operation": "wait_child_exit", "type": "ExitNotConfirmed"}) - attempt("close_child_fd", lambda fd=fd: os.close(fd)) - for fd in pipes: - if fd >= 0: - attempt("close_pipe_fd", lambda fd=fd: os.close(fd)) - # Both writers must be gone; launcher exit alone is insufficient, and - # namespace exit alone leaves the launcher able to modify these files. - report["outputs_stable"] = ( - report["final_init_exit_observed"] - and process is not None - and process.returncode is not None - ) - report["cleanup_complete"] = ( - report["outputs_stable"] - and all(value is True for value in report["cleanup_registered_child_exits"]) - and not errors - ) - stdout_raw = attempt("read_stdout", (output / "stdout.json").read_bytes) - stderr_raw = attempt("read_stderr", (output / "stderr.private.log").read_bytes) - attempt( - "write_status", - lambda: (output / "status.private.jsonl").write_bytes(status_raw), - ) - for name, raw in (("stdout", stdout_raw), ("stderr", stderr_raw)): - report[name + "_sha256"] = ( - runtime.sha(raw) if raw is not None and report["outputs_stable"] else None - ) - if not report["outputs_stable"] and raw is not None: - report[name + "_partial_snapshot_sha256"] = runtime.sha(raw) - report["status_sha256"] = runtime.sha(status_raw) - report["rootfs_digest"] = rootfs["rootfs_digest"] - report["cleanup_or_evidence_errors"] = errors - if report["forced_cleanup"] or errors or not report["outputs_stable"]: - report["passed"] = False - # If even the report destination fails, the wrapper still captures the - # nonzero process status and stderr; no success record is fabricated. - runtime.write(output / "report.json", report) - - -def run_case(locators, rootfs, mode, output): - output.mkdir() - sentinel = output / "unmounted-sentinel.private" - sentinel.write_text("trusted synthetic marker") - namespaces = { - name: os.readlink("/proc/self/ns/" + name) - for name in ("user", "net", "pid", "uts", "ipc") - } - args = runtime.command( - locators, Path(__file__).with_name("synthetic_probe.py"), namespaces, sentinel - ) - status_read, status_write = os.pipe() - block_read, block_write = os.pipe() - # Additional mount policy; the independent /dev/shm remains a bounded tmpfs. - args[1:1] = ["--json-status-fd", str(status_write), "--block-fd", str(block_read)] - payload_index = args.index("/opt/python/" + locators["python_relative"]) - args[payload_index:payload_index] = ["--remount-ro", "/dev"] - args.append("gate_check" if mode.startswith("gate_") else mode) - process = None - init_fd = None - children = [] - report = {"mode": mode, "passed": False, "generated_code_executed": False} - stdout_raw = b"" - status_raw = bytearray() - try: - with ( - (output / "stderr.private.log").open("xb") as stderr, - (output / "stdout.json").open("xb") as stdout, - ): - process = subprocess.Popen( - [ - sys.executable, - "-IB", - str(Path(__file__).resolve()), - "_exec", - json.dumps(args), - ], - stdin=subprocess.PIPE, - stdout=stdout, - stderr=stderr, - pass_fds=(status_write, block_read), - start_new_session=True, - ) - os.close(status_write) - status_write = -1 - os.close(block_read) - block_read = -1 - read_line(status_read, time.monotonic() + 10, status_raw) - start = json.loads(status_raw, object_pairs_hook=unique) - init_fd, identity = bind_init(start, process.pid) - report["init_identity"] = identity - report["initial_status"] = start - report["authorization_sent"] = False - if mode == "gate_early_launcher_death": - # Reproduce launcher death before setup release, while holding - # the init pidfd. EOF on the setup pipe must not authorize work. - process.kill() - process.wait(timeout=5) - else: - os.write(block_write, b"1") - os.close(block_write) - block_write = -1 - if mode not in ("gate_eof", "gate_timeout", "gate_early_launcher_death"): - token = ( - START_TOKEN[:-1] - if mode == "gate_short" - else b"wrong\n" - if mode == "gate_wrong" - else START_TOKEN - ) - process.stdin.write(token) - process.stdin.flush() - report["authorization_sent"] = token == START_TOKEN - if mode != "gate_timeout": - process.stdin.close() - if mode.startswith("gate_"): - process.wait(timeout=8) - report["probe_passed"] = True # Exact output/exit checks below. - elif mode in ("bounds", "synthetic"): - process.wait(timeout=60) - report["probe_passed"] = True - else: - stdout_raw = read_ready_file( - output / "stdout.json", time.monotonic() + 10 - ) - payload = json.loads(stdout_raw, object_pairs_hook=unique) - if payload.get("ready") is not True or payload.get("mode") != mode: - raise ValueError("trusted cleanup probe did not become ready") - if payload.get("namespace_pid") != identity["pid_namespace"]: - raise ValueError("payload PID namespace differs from bound init") - children, child_records = bind_children(identity["pid"], payload) - report["registered_child_identities"] = child_records - if mode == "timeout": - try: - process.wait(timeout=1) - except subprocess.TimeoutExpired: - report["timeout_observed"] = True - pidfd_send_signal(init_fd, int(signal.SIGKILL)) - else: - raise ValueError("timeout probe returned before deadline") - elif mode == "launcher_kill": - process.kill() - process.wait(timeout=8) - stdout_raw = (output / "stdout.json").read_bytes() - report["probe_passed"] = True - report["exit_code"] = process.returncode - report["init_exit_observed"] = exited(init_fd, 5000) - report["registered_child_exit_observed"] = [ - exited(fd, 1000) for fd in children - ] - if not report["init_exit_observed"]: - raise ValueError("namespace init exit not confirmed") - stdout_raw = (output / "stdout.json").read_bytes() - stderr_raw = (output / "stderr.private.log").read_bytes() - if mode.startswith("gate_") and mode != "gate_valid": - report["probe_passed"] = ( - stdout_raw == b"" and AUTHORIZED_MARKER not in stderr_raw - ) - if mode != "gate_early_launcher_death": - report["probe_passed"] = ( - report["probe_passed"] and stderr_raw == REJECTED_MARKER - ) - else: - report["probe_passed"] = ( - report["probe_passed"] and stderr_raw == AUTHORIZED_MARKER - ) - if mode in ("bounds", "synthetic", "gate_valid"): - payload = json.loads(stdout_raw, object_pairs_hook=unique) - report["probe_passed"] = ( - report["probe_passed"] and payload.get("passed") is True - ) - # Parent has exited and bwrap's payload closes the status fd. Read - # the remainder with a finite deadline; EOF is expected. - deadline = time.monotonic() + 2 - status_eof = False - while time.monotonic() < deadline: - ready, _, _ = select.select( - [status_read], [], [], max(0, deadline - time.monotonic()) - ) - if not ready: - break - data = os.read(status_read, 4096) - if not data: - status_eof = True - break - status_raw += data - if len(status_raw) > 65536: - raise ValueError("status stream unexpectedly large") - report["status_eof"] = status_eof - expected_exit = ( - -signal.SIGKILL - if mode in ("launcher_kill", "gate_early_launcher_death") - else 137 - if mode == "timeout" - else 23 - if mode.startswith("gate_") and mode != "gate_valid" - else 0 - ) - report["passed"] = ( - report["probe_passed"] - and process.returncode == expected_exit - and report["init_exit_observed"] - and all(report["registered_child_exit_observed"]) - and status_eof - ) - except BaseException as error: - report["error"] = {"type": type(error).__name__, "message": str(error)} - finally: - finalize_case( - report, - process, - init_fd, - children, - (status_read, status_write, block_read, block_write), - output, - status_raw, - rootfs, - ) - return report - - -def main(): - parser = argparse.ArgumentParser(description=__doc__) - parser.add_argument("phase", choices=("capability", "probe")) - parser.add_argument("--bundle", type=Path) - parser.add_argument("--inventory", type=Path) - parser.add_argument("--capability", type=Path) - parser.add_argument("--output", type=Path, required=True) - args = parser.parse_args() - if args.phase == "capability": - return capability(args.output) - if args.capability is None or args.inventory is None: - raise ValueError("probe requires capability evidence and inventory") - capability_raw = args.capability.read_bytes() - preflight = json.loads(capability_raw, object_pairs_hook=unique) - if ( - preflight.get("passed") is not True - or preflight.get("cleanup_errors") != [] - or preflight.get("job_id") != os.environ["SLURM_JOB_ID"] - or preflight.get("driver_sha256") != runtime.sha(Path(__file__).read_bytes()) - or preflight.get("python_binary_sha256") - != runtime.sha(Path(sys._base_executable).read_bytes()) - or preflight.get("kernel_release") != platform.release() - ): - raise ValueError( - "actual pidfd capability preflight did not pass for this job/runtime" - ) - rootfs = json.loads((args.inventory / "rootfs.json").read_bytes()) - locators = json.loads( - (args.inventory / "runtime-locators.private.json").read_bytes() - ) - if ( - runtime.sha(Path(locators["bwrap"]).read_bytes()) - != rootfs["executor"]["binary_sha256"] - ): - raise ValueError("bwrap binary differs from inventory") - args.output.mkdir() - results = [] - for mode in MODES: - result = run_case(locators, rootfs, mode, args.output / mode) - results.append(result) - if result.get("cleanup_complete") is not True: - break - passed = len(results) == len(MODES) and all(row["passed"] for row in results) - runtime.write( - args.output / "summary.json", - { - "scope": "trusted_kernel_boundary_and_cleanup_probes", - "passed": passed, - "unexecuted_modes": list(MODES[len(results) :]), - "capability_sha256": runtime.sha(capability_raw), - "cases": results, - "generated_code_executed": False, - "qualified_generated_code_executor": False, - "verdict": "INCONCLUSIVE", - "driver_sha256": runtime.sha(Path(__file__).read_bytes()), - "probe_sha256": runtime.sha( - Path(__file__).with_name("synthetic_probe.py").read_bytes() - ), - "runtime_sha256": runtime.sha(Path(runtime.__file__).read_bytes()), - }, - ) - return 0 if passed else 2 - - -if __name__ == "__main__": - if ( - not os.environ.get("SLURM_JOB_ID") - or os.environ.get("SLURM_GPUS_ON_NODE", "0") != "0" - ): - raise ValueError("CPU-only Slurm allocation required") - if len(sys.argv) == 3 and sys.argv[1] == "_exec": - runtime.limits() - command = json.loads(sys.argv[2]) - os.execv(command[0], command) - raise SystemExit(main()) diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-executor-v1/contract.py b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-executor-v1/contract.py deleted file mode 100644 index 04f8f2767..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-executor-v1/contract.py +++ /dev/null @@ -1,160 +0,0 @@ -"""Byte-bound input and result checks; never execute solutions on the host.""" - -import hashlib -import json -from pathlib import Path - -INPUT_NAMES = ("humaneval-solutions.jsonl", "humaneval-subset.jsonl", "human-export.json") - - -def sha(raw): - return hashlib.sha256(raw).hexdigest() - - -def unique(pairs): - value = {} - for key, item in pairs: - if key in value: - raise ValueError("duplicate JSON key") - value[key] = item - return value - - -def decode(raw): - return json.loads( - raw, - object_pairs_hook=unique, - parse_constant=lambda value: (_ for _ in ()).throw(ValueError(value)), - ) - - -def canonical(value): - return json.dumps(value, sort_keys=True, separators=(",", ":"), allow_nan=False).encode() - - -def read(path, maximum): - with Path(path).open("rb") as stream: - raw = stream.read(maximum + 1) - if len(raw) > maximum: - raise ValueError("artifact exceeds declared byte cap") - return raw - - -def rows(raw, samples=False): - result = {} - for line in raw.splitlines(): - if not line: - raise ValueError("empty JSONL record") - row = decode(line) - if not isinstance(row, dict): - raise ValueError("JSONL record must be an object") - identity = row.get("task_id") - if not isinstance(identity, str) or not identity or identity in result: - raise ValueError("missing or duplicate task identity") - if samples and ( - set(row) != {"task_id", "solution"} or not isinstance(row["solution"], str) - ): - raise ValueError("expected exactly one complete solution per task") - result[identity] = row - if not result: - raise ValueError("empty input cohort") - return result - - -def inputs(directory, export_sha256, cohort_sha256, maximum): - raw = {name: read(Path(directory) / name, maximum) for name in INPUT_NAMES} - if sha(raw["human-export.json"]) != export_sha256: - raise ValueError("export digest mismatch") - export = decode(raw["human-export.json"]) - if not isinstance(export, dict) or set(export) != { - "samples_sha256", - "dataset_sha256", - "evalplus_dataset_md5", - "task_ids", - "generated_code_executed", - }: - raise ValueError("unsupported export manifest") - if export["generated_code_executed"] is not False: - raise ValueError("export must originate from the nonexecuting score path") - samples = rows(raw["humaneval-solutions.jsonl"], samples=True) - dataset = rows(raw["humaneval-subset.jsonl"]) - identities = sorted(samples) - if sorted(dataset) != identities or export["task_ids"] != identities: - raise ValueError("sample/dataset/cohort coverage differs") - if sha(canonical(identities)) != cohort_sha256: - raise ValueError("declared cohort digest mismatch") - if export["samples_sha256"] != sha(raw["humaneval-solutions.jsonl"]): - raise ValueError("sample digest mismatch") - if export["dataset_sha256"] != sha(raw["humaneval-subset.jsonl"]): - raise ValueError("dataset digest mismatch") - if export["evalplus_dataset_md5"] != hashlib.md5(raw["humaneval-subset.jsonl"]).hexdigest(): - raise ValueError("official evaluator dataset identity differs") - return raw, export, samples - - -def results(raw, export, samples, maximum): - if not raw or len(raw) > maximum: - raise ValueError("empty or oversized evaluator result") - value = decode(raw) - if not isinstance(value, dict) or value.get("hash") != export["evalplus_dataset_md5"]: - raise ValueError("result dataset identity differs") - evaluated = value.get("eval") - if not isinstance(evaluated, dict) or set(evaluated) != set(samples): - raise ValueError("result cohort incomplete or unexpected") - statuses = {} - for identity, answers in evaluated.items(): - if not isinstance(answers, list) or len(answers) != 1: - raise ValueError("expected one result per task") - answer = answers[0] - if not isinstance(answer, dict) or answer.get("task_id") != identity: - raise ValueError("result row task identity differs from its enclosing key") - if not isinstance(answer, dict) or answer.get("solution") != samples[identity]["solution"]: - raise ValueError("result solution differs from executed sample") - pair = [answer.get("base_status"), answer.get("plus_status")] - if any(status not in ("pass", "fail", "timeout") for status in pair): - raise ValueError("unknown or missing evaluator status") - statuses[identity] = pair - return statuses - - -def policy(path, expected_sha256): - raw = read(path, 16384) - if sha(raw) != expected_sha256: - raise ValueError("policy digest mismatch") - value = decode(raw) - expected = { - "schema", - "scope", - "qualified", - "as_bytes", - "cpu_seconds", - "wall_seconds", - "file_bytes", - "work_bytes", - "shm_bytes", - "tasks", - "nofile", - "input_bytes", - "result_bytes", - "evalplus", - } - if not isinstance(value, dict) or set(value) != expected or value["schema"] != 1: - raise ValueError("invalid policy schema") - if value["qualified"] is not False or value["scope"] != "four-handwritten-fixtures-readiness": - raise ValueError("this prototype has no production qualification") - for key in expected - {"schema", "scope", "qualified", "evalplus"}: - if type(value[key]) is not int or value[key] <= 0: - raise ValueError("resource policy must contain finite positive integers") - if value["as_bytes"] < 4 * 1024**3 or value["file_bytes"] < value["result_bytes"]: - raise ValueError("policy cannot preserve evaluator memory/result requirements") - if value["evalplus"] != { - "parallel": 1, - "base_only": False, - "mini": False, - "noextreme": False, - "test_details": True, - "memory_override": None, - "timeout_override": None, - }: - raise ValueError("unsupported evaluator policy") - return value diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-executor-v1/contract.review-before-task-identity.py b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-executor-v1/contract.review-before-task-identity.py deleted file mode 100644 index 253709355..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-executor-v1/contract.review-before-task-identity.py +++ /dev/null @@ -1,158 +0,0 @@ -"""Byte-bound input and result checks; never execute solutions on the host.""" - -import hashlib -import json -from pathlib import Path - -INPUT_NAMES = ("humaneval-solutions.jsonl", "humaneval-subset.jsonl", "human-export.json") - - -def sha(raw): - return hashlib.sha256(raw).hexdigest() - - -def unique(pairs): - value = {} - for key, item in pairs: - if key in value: - raise ValueError("duplicate JSON key") - value[key] = item - return value - - -def decode(raw): - return json.loads( - raw, - object_pairs_hook=unique, - parse_constant=lambda value: (_ for _ in ()).throw(ValueError(value)), - ) - - -def canonical(value): - return json.dumps(value, sort_keys=True, separators=(",", ":"), allow_nan=False).encode() - - -def read(path, maximum): - with Path(path).open("rb") as stream: - raw = stream.read(maximum + 1) - if len(raw) > maximum: - raise ValueError("artifact exceeds declared byte cap") - return raw - - -def rows(raw, samples=False): - result = {} - for line in raw.splitlines(): - if not line: - raise ValueError("empty JSONL record") - row = decode(line) - if not isinstance(row, dict): - raise ValueError("JSONL record must be an object") - identity = row.get("task_id") - if not isinstance(identity, str) or not identity or identity in result: - raise ValueError("missing or duplicate task identity") - if samples and ( - set(row) != {"task_id", "solution"} or not isinstance(row["solution"], str) - ): - raise ValueError("expected exactly one complete solution per task") - result[identity] = row - if not result: - raise ValueError("empty input cohort") - return result - - -def inputs(directory, export_sha256, cohort_sha256, maximum): - raw = {name: read(Path(directory) / name, maximum) for name in INPUT_NAMES} - if sha(raw["human-export.json"]) != export_sha256: - raise ValueError("export digest mismatch") - export = decode(raw["human-export.json"]) - if not isinstance(export, dict) or set(export) != { - "samples_sha256", - "dataset_sha256", - "evalplus_dataset_md5", - "task_ids", - "generated_code_executed", - }: - raise ValueError("unsupported export manifest") - if export["generated_code_executed"] is not False: - raise ValueError("export must originate from the nonexecuting score path") - samples = rows(raw["humaneval-solutions.jsonl"], samples=True) - dataset = rows(raw["humaneval-subset.jsonl"]) - identities = sorted(samples) - if sorted(dataset) != identities or export["task_ids"] != identities: - raise ValueError("sample/dataset/cohort coverage differs") - if sha(canonical(identities)) != cohort_sha256: - raise ValueError("declared cohort digest mismatch") - if export["samples_sha256"] != sha(raw["humaneval-solutions.jsonl"]): - raise ValueError("sample digest mismatch") - if export["dataset_sha256"] != sha(raw["humaneval-subset.jsonl"]): - raise ValueError("dataset digest mismatch") - if export["evalplus_dataset_md5"] != hashlib.md5(raw["humaneval-subset.jsonl"]).hexdigest(): - raise ValueError("official evaluator dataset identity differs") - return raw, export, samples - - -def results(raw, export, samples, maximum): - if not raw or len(raw) > maximum: - raise ValueError("empty or oversized evaluator result") - value = decode(raw) - if not isinstance(value, dict) or value.get("hash") != export["evalplus_dataset_md5"]: - raise ValueError("result dataset identity differs") - evaluated = value.get("eval") - if not isinstance(evaluated, dict) or set(evaluated) != set(samples): - raise ValueError("result cohort incomplete or unexpected") - statuses = {} - for identity, answers in evaluated.items(): - if not isinstance(answers, list) or len(answers) != 1: - raise ValueError("expected one result per task") - answer = answers[0] - if not isinstance(answer, dict) or answer.get("solution") != samples[identity]["solution"]: - raise ValueError("result solution differs from executed sample") - pair = [answer.get("base_status"), answer.get("plus_status")] - if any(status not in ("pass", "fail", "timeout") for status in pair): - raise ValueError("unknown or missing evaluator status") - statuses[identity] = pair - return statuses - - -def policy(path, expected_sha256): - raw = read(path, 16384) - if sha(raw) != expected_sha256: - raise ValueError("policy digest mismatch") - value = decode(raw) - expected = { - "schema", - "scope", - "qualified", - "as_bytes", - "cpu_seconds", - "wall_seconds", - "file_bytes", - "work_bytes", - "shm_bytes", - "tasks", - "nofile", - "input_bytes", - "result_bytes", - "evalplus", - } - if not isinstance(value, dict) or set(value) != expected or value["schema"] != 1: - raise ValueError("invalid policy schema") - if value["qualified"] is not False or value["scope"] != "four-handwritten-fixtures-readiness": - raise ValueError("this prototype has no production qualification") - for key in expected - {"schema", "scope", "qualified", "evalplus"}: - if type(value[key]) is not int or value[key] <= 0: - raise ValueError("resource policy must contain finite positive integers") - if value["as_bytes"] < 4 * 1024**3 or value["file_bytes"] < value["result_bytes"]: - raise ValueError("policy cannot preserve evaluator memory/result requirements") - if value["evalplus"] != { - "parallel": 1, - "base_only": False, - "mini": False, - "noextreme": False, - "test_details": True, - "memory_override": None, - "timeout_override": None, - }: - raise ValueError("unsupported evaluator policy") - return value diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-executor-v1/contract.review-rejected-four-key-export.py b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-executor-v1/contract.review-rejected-four-key-export.py deleted file mode 100644 index e03c5c302..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-executor-v1/contract.review-rejected-four-key-export.py +++ /dev/null @@ -1,155 +0,0 @@ -"""Byte-bound input and result checks; never execute solutions on the host.""" - -import hashlib -import json -from pathlib import Path - -INPUT_NAMES = ("humaneval-solutions.jsonl", "humaneval-subset.jsonl", "human-export.json") - - -def sha(raw): - return hashlib.sha256(raw).hexdigest() - - -def unique(pairs): - value = {} - for key, item in pairs: - if key in value: - raise ValueError("duplicate JSON key") - value[key] = item - return value - - -def decode(raw): - return json.loads( - raw, - object_pairs_hook=unique, - parse_constant=lambda value: (_ for _ in ()).throw(ValueError(value)), - ) - - -def canonical(value): - return json.dumps(value, sort_keys=True, separators=(",", ":"), allow_nan=False).encode() - - -def read(path, maximum): - with Path(path).open("rb") as stream: - raw = stream.read(maximum + 1) - if len(raw) > maximum: - raise ValueError("artifact exceeds declared byte cap") - return raw - - -def rows(raw, samples=False): - result = {} - for line in raw.splitlines(): - if not line: - raise ValueError("empty JSONL record") - row = decode(line) - if not isinstance(row, dict): - raise ValueError("JSONL record must be an object") - identity = row.get("task_id") - if not isinstance(identity, str) or not identity or identity in result: - raise ValueError("missing or duplicate task identity") - if samples and ( - set(row) != {"task_id", "solution"} or not isinstance(row["solution"], str) - ): - raise ValueError("expected exactly one complete solution per task") - result[identity] = row - if not result: - raise ValueError("empty input cohort") - return result - - -def inputs(directory, export_sha256, cohort_sha256, maximum): - raw = {name: read(Path(directory) / name, maximum) for name in INPUT_NAMES} - if sha(raw["human-export.json"]) != export_sha256: - raise ValueError("export digest mismatch") - export = decode(raw["human-export.json"]) - if not isinstance(export, dict) or set(export) != { - "samples_sha256", - "dataset_sha256", - "evalplus_dataset_md5", - "task_ids", - }: - raise ValueError("unsupported export manifest") - samples = rows(raw["humaneval-solutions.jsonl"], samples=True) - dataset = rows(raw["humaneval-subset.jsonl"]) - identities = sorted(samples) - if sorted(dataset) != identities or export["task_ids"] != identities: - raise ValueError("sample/dataset/cohort coverage differs") - if sha(canonical(identities)) != cohort_sha256: - raise ValueError("declared cohort digest mismatch") - if export["samples_sha256"] != sha(raw["humaneval-solutions.jsonl"]): - raise ValueError("sample digest mismatch") - if export["dataset_sha256"] != sha(raw["humaneval-subset.jsonl"]): - raise ValueError("dataset digest mismatch") - if export["evalplus_dataset_md5"] != hashlib.md5(raw["humaneval-subset.jsonl"]).hexdigest(): - raise ValueError("official evaluator dataset identity differs") - return raw, export, samples - - -def results(raw, export, samples, maximum): - if not raw or len(raw) > maximum: - raise ValueError("empty or oversized evaluator result") - value = decode(raw) - if not isinstance(value, dict) or value.get("hash") != export["evalplus_dataset_md5"]: - raise ValueError("result dataset identity differs") - evaluated = value.get("eval") - if not isinstance(evaluated, dict) or set(evaluated) != set(samples): - raise ValueError("result cohort incomplete or unexpected") - statuses = {} - for identity, answers in evaluated.items(): - if not isinstance(answers, list) or len(answers) != 1: - raise ValueError("expected one result per task") - answer = answers[0] - if not isinstance(answer, dict) or answer.get("solution") != samples[identity]["solution"]: - raise ValueError("result solution differs from executed sample") - pair = [answer.get("base_status"), answer.get("plus_status")] - if any(status not in ("pass", "fail", "timeout") for status in pair): - raise ValueError("unknown or missing evaluator status") - statuses[identity] = pair - return statuses - - -def policy(path, expected_sha256): - raw = read(path, 16384) - if sha(raw) != expected_sha256: - raise ValueError("policy digest mismatch") - value = decode(raw) - expected = { - "schema", - "scope", - "qualified", - "as_bytes", - "cpu_seconds", - "wall_seconds", - "file_bytes", - "work_bytes", - "shm_bytes", - "tasks", - "nofile", - "input_bytes", - "result_bytes", - "evalplus", - } - if not isinstance(value, dict) or set(value) != expected or value["schema"] != 1: - raise ValueError("invalid policy schema") - if value["qualified"] is not False or value["scope"] != "four-handwritten-fixtures-readiness": - raise ValueError("this prototype has no production qualification") - for key in expected - {"schema", "scope", "qualified", "evalplus"}: - if type(value[key]) is not int or value[key] <= 0: - raise ValueError("resource policy must contain finite positive integers") - if value["as_bytes"] < 4 * 1024**3 or value["file_bytes"] < value["result_bytes"]: - raise ValueError("policy cannot preserve evaluator memory/result requirements") - if value["evalplus"] != { - "parallel": 1, - "base_only": False, - "mini": False, - "noextreme": False, - "test_details": True, - "memory_override": None, - "timeout_override": None, - }: - raise ValueError("unsupported evaluator policy") - return value diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-executor-v1/controls.json b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-executor-v1/controls.json deleted file mode 100644 index d861edee2..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-executor-v1/controls.json +++ /dev/null @@ -1,28 +0,0 @@ -{ - "source_sha256": { - "runtime.py": "997cbfed8398e8ad62d1beb4f6b4bc6e61a0f4572b9156cfd0c6f3fb52b03233", - "bounds_driver.py": "bfeeb6ea794c0ebf24e72a0127458ff74951253471daee080b3a3b1d82a062ba", - "synthetic_probe.py": "236920f27573447b9f3833afb72739033c965e22a9e406b09f28da815529ca64", - "contract.py": "bcd6a4a4d69f163075b89ae2e55c8fd475a9cc78ac64d39eadda1a54b4557dec", - "inner.py": "5a635ae5773d08a3cbcda07551450b38d3fd6af64f8ca251cc61a7a94c388342", - "executor.py": "15613932aec108c27b2de2390a965aaba4074535f0e02295f4e7846e405f282c", - "fixtures.py": "166dc7c82c9b0847144a56f46a30f97e93f65967a017e79cb9f8b916219364cc", - "exercise.py": "cc4f31657a89f0ca1a3bacd86e9060f2fbf633557ee05b6695f6ccba91d6e92f", - "test_contract.py": "94b32dfc8bc80d4f46f211fc6d0d36065d6043a47bff04a4856d3eb942e7cf1e", - "policy.json": "222326a1b27e48fd55988a2c10a7065fa44ad8cd22e71348bff355aa1c15996d", - "run-executor.sbatch": "99626cbef13f865de7fee59746fe228ebae885aac76bcc6029955c1a36cca16f" - }, - "python_mode": "-IB", - "tests": 18, - "passed": true, - "scope": "host-only controls; Linux-specific actions mocked; no actual evaluator", - "ruff_repository_config": "passed on six new Python sources", - "shell_syntax": "passed", - "retained_preparation_failures": [ - "initial lint:10 findings corrected", - "initial timeout-control mock had no pipe reader, raised BrokenPipeError; fixed mock write, no Linux outcome", - "review:four-key export incompatible with actual five-key v5 schema; fixed", - "review:stale supervisor output digest accepted; fixed", - "review:row task_id not checked; fixed" - ] -} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-executor-v1/executor.py b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-executor-v1/executor.py deleted file mode 100644 index 38fc0767e..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-executor-v1/executor.py +++ /dev/null @@ -1,298 +0,0 @@ -"""Run an authenticated export through a fresh, unqualified evaluator prototype.""" - -import argparse -import json -import os -import resource -import select -import subprocess -import sys -import time -from pathlib import Path - -SOURCE = Path(__file__).resolve().parent -sys.path.insert(0, str(SOURCE)) -# The fixed source directory is deliberately restored under isolated Python (-I). -import bounds_driver as supervisor # noqa: E402 -import contract # noqa: E402 -import fixtures # noqa: E402 - -runtime = supervisor.runtime -STATIC_FILES = ( - "runtime.py", - "bounds_driver.py", - "synthetic_probe.py", - "contract.py", - "inner.py", - "executor.py", - "policy.json", - "fixtures.py", -) - - -def write(path, value): - with Path(path).open("xb") as stream: - stream.write(contract.canonical(value) + b"\n") - - -def inventory(bundle, output): - runtime.inventory(bundle, output) - original = contract.decode((output / "rootfs.json").read_bytes()) - entries = list(original["entries"]) - for name in STATIC_FILES: - raw = (SOURCE / name).read_bytes() - entries.append( - { - "path": "/opt/executor/" + name, - "type": "file", - "bytes": len(raw), - "sha256": contract.sha(raw), - } - ) - entries.sort(key=lambda row: row["path"]) - value = { - **original, - "entries": entries, - "rootfs_digest": "sha256:" + contract.sha(contract.canonical(entries)), - } - write(output / "executor-rootfs.json", value) - return value, contract.decode((output / "runtime-locators.private.json").read_bytes()) - - -def command(locators, snapshot, policy, policy_sha256, export_sha256, cohort_sha256): - args = runtime.command(locators, SOURCE / "synthetic_probe.py", {}, "/unmounted") - # Retain the reviewed mount base, replace the concrete work/shm sizes, and - # replace the legacy diagnostic dispatch without calling its limits(). - for mount, capacity in (("/work", policy["work_bytes"]), ("/dev/shm", policy["shm_bytes"])): - index = args.index(mount) - if args[index - 3 : index] != ["--size", args[index - 2], "--tmpfs"]: - raise ValueError("legacy mount command shape changed") - args[index - 2] = str(capacity) - payload_index = args.index("/opt/python/" + locators["python_relative"]) - mounts = [] - for name in STATIC_FILES: - mounts += ["--ro-bind", str(SOURCE / name), "/opt/executor/" + name] - mounts += ["--ro-bind", str(snapshot), "/input", "--remount-ro", "/dev"] - mount_index = args.index("--remount-ro") - return ( - args[:mount_index] - + mounts - + args[mount_index:payload_index] - + [ - args[payload_index], - "-IB", - "/opt/executor/inner.py", - policy_sha256, - export_sha256, - cohort_sha256, - ] - ) - - -def limits(policy): - os.sched_setaffinity(0, {min(os.sched_getaffinity(0))}) - for name, value in ( - ("AS", policy["as_bytes"]), - ("CPU", policy["cpu_seconds"]), - ("NOFILE", policy["nofile"]), - ("FSIZE", policy["file_bytes"]), - ("CORE", 0), - ): - resource.setrlimit(getattr(resource, "RLIMIT_" + name), (value, value)) - - -def status_end(fd, status_raw): - deadline = time.monotonic() + 2 - while time.monotonic() < deadline: - if not select.select([fd], [], [], max(0, deadline - time.monotonic()))[0]: - break - data = os.read(fd, 4096) - if not data: - return True - status_raw.extend(data) - if len(status_raw) > 65536: - raise ValueError("unexpected status stream size") - return False - - -def execute(args, rootfs, output, policy): - output.mkdir() - read_fd, write_fd = os.pipe() - block_read, block_write = os.pipe() - args[1:1] = ["--json-status-fd", str(write_fd), "--block-fd", str(block_read)] - process, init_fd = None, None - status_raw = bytearray() - report = { - "mode": "external-export", - "passed": False, - "timed_out": False, - "qualified": False, - "generated_code_executed": False, - } - write(output / "command.private.json", args) - try: - with ( - (output / "stdout.json").open("xb") as stdout, - (output / "stderr.private.log").open("xb") as stderr, - ): - process = subprocess.Popen( - [ - sys.executable, - "-IB", - str(SOURCE / "executor.py"), - "_exec", - contract.sha((SOURCE / "policy.json").read_bytes()), - json.dumps(args), - ], - stdin=subprocess.PIPE, - stdout=stdout, - stderr=stderr, - pass_fds=(write_fd, block_read), - start_new_session=True, - ) - os.close(write_fd) - write_fd = -1 - os.close(block_read) - block_read = -1 - supervisor.read_line(read_fd, time.monotonic() + 10, status_raw) - start = contract.decode(status_raw) - init_fd, identity = supervisor.bind_init(start, process.pid) - report["init_identity"] = identity - report["initial_status"] = start - os.write(block_write, b"1") - os.close(block_write) - block_write = -1 - process.stdin.write(supervisor.START_TOKEN) - process.stdin.flush() - process.stdin.close() - report["authorization_sent"] = True - started = time.monotonic() - try: - process.wait(timeout=policy["wall_seconds"]) - except subprocess.TimeoutExpired: - report["timed_out"] = True - raise - report["elapsed_seconds"] = time.monotonic() - started - report["exit_code"] = process.returncode - report["init_exit_observed"] = supervisor.exited(init_fd, 5000) - report["status_eof"] = status_end(read_fd, status_raw) - report["passed"] = ( - process.returncode == 0 and report["init_exit_observed"] and report["status_eof"] - ) - except BaseException as error: - report["error"] = {"type": type(error).__name__, "message": str(error)} - finally: - supervisor.finalize_case( - report, - process, - init_fd, - [], - (read_fd, write_fd, block_read, block_write), - output, - status_raw, - rootfs, - ) - return report - - -def evaluate_export(args): - policy = contract.policy(SOURCE / "policy.json", args.policy_sha256) - raw, export, samples = contract.inputs( - args.inputs, args.export_sha256, args.cohort_sha256, policy["input_bytes"] - ) - # This readiness entry never accepts model output or benchmark questions. - if raw != fixtures.payloads(): - raise ValueError("prototype only permits the exact declared handwritten fixtures") - args.output.mkdir(parents=False, exist_ok=False) - snapshot = args.output / "input-snapshot" - snapshot.mkdir() - for name, content in raw.items(): - with (snapshot / name).open("xb") as stream: - stream.write(content) - if supervisor.capability(args.output / "pidfd-capability.json") != 0: - raise ValueError("actual pidfd capability failed before sandbox creation") - before, locators = inventory(args.bundle, args.output / "inventory-before") - launch = command( - locators, snapshot, policy, args.policy_sha256, args.export_sha256, args.cohort_sha256 - ) - report = execute(launch, before, args.output / "execution", policy) - after, _ = inventory(args.bundle, args.output / "inventory-after") - unchanged = before == after - statuses = None - result_sha256 = None - error = None - success = ( - report.get("passed") is True - and report.get("cleanup_complete") is True - and report.get("outputs_stable") is True - and unchanged - ) - if success: - try: - result_raw = contract.read( - args.output / "execution/stdout.json", policy["result_bytes"] - ) - result_sha256 = contract.sha(result_raw) - if result_sha256 != report["stdout_sha256"]: - raise ValueError("parsed result bytes differ from the final supervisor digest") - statuses = contract.results(result_raw, export, samples, policy["result_bytes"]) - except (ValueError, OSError) as failure: - error = {"type": type(failure).__name__, "message": str(failure)} - success = False - identity = { - "schema_version": 1, - "runtime_kind": "readonly-bind-set-v1", - "runtime_digest": before["rootfs_digest"], - "evaluator_revision": runtime.REVISION, - "executor_sha256": contract.sha((SOURCE / "inner.py").read_bytes()), - "driver_sha256": contract.sha((SOURCE / "executor.py").read_bytes()), - "sandbox_binary_sha256": before["executor"]["binary_sha256"], - "policy_sha256": args.policy_sha256, - } - write( - args.output / "execution-record.json", - { - "scope": policy["scope"], - "protocol": None, - "observed_identity": identity, - "qualified": False, - "verdict": "INCONCLUSIVE", - "execution_completed": success, - "network": "none", - "exit_code": report.get("final_launcher_exit_code"), - "timed_out": report["timed_out"], - "cleanup_complete": report["cleanup_complete"], - "runtime_unchanged": unchanged, - "export_sha256": args.export_sha256, - "cohort_sha256": args.cohort_sha256, - "samples_sha256": export["samples_sha256"], - "dataset_sha256": export["dataset_sha256"], - "results_sha256": result_sha256 if success else None, - "supervisor_stdout_sha256": report["stdout_sha256"], - "statuses": statuses, - "validation_error": error, - }, - ) - return 0 if success else 2 - - -def main(): - parser = argparse.ArgumentParser(description=__doc__) - parser.add_argument("--bundle", type=Path, required=True) - parser.add_argument("--inputs", type=Path, required=True) - parser.add_argument("--output", type=Path, required=True) - parser.add_argument("--policy-sha256", required=True) - parser.add_argument("--export-sha256", required=True) - parser.add_argument("--cohort-sha256", required=True) - return evaluate_export(parser.parse_args()) - - -if __name__ == "__main__": - if not os.environ.get("SLURM_JOB_ID") or os.environ.get("SLURM_GPUS_ON_NODE", "0") != "0": - raise ValueError("CPU-only Slurm allocation required") - if len(sys.argv) == 4 and sys.argv[1] == "_exec": - selected = contract.policy(SOURCE / "policy.json", sys.argv[2]) - limits(selected) - launch = contract.decode(sys.argv[3]) - os.execv(launch[0], launch) - raise SystemExit(main()) diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-executor-v1/executor.review-rejected-stale-result-hash.py b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-executor-v1/executor.review-rejected-stale-result-hash.py deleted file mode 100644 index 53deda080..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-executor-v1/executor.review-rejected-stale-result-hash.py +++ /dev/null @@ -1,293 +0,0 @@ -"""Run an authenticated export through a fresh, unqualified evaluator prototype.""" - -import argparse -import json -import os -import resource -import select -import subprocess -import sys -import time -from pathlib import Path - -SOURCE = Path(__file__).resolve().parent -sys.path.insert(0, str(SOURCE)) -# The fixed source directory is deliberately restored under isolated Python (-I). -import bounds_driver as supervisor # noqa: E402 -import contract # noqa: E402 -import fixtures # noqa: E402 - -runtime = supervisor.runtime -STATIC_FILES = ( - "runtime.py", - "bounds_driver.py", - "synthetic_probe.py", - "contract.py", - "inner.py", - "executor.py", - "policy.json", - "fixtures.py", -) - - -def write(path, value): - with Path(path).open("xb") as stream: - stream.write(contract.canonical(value) + b"\n") - - -def inventory(bundle, output): - runtime.inventory(bundle, output) - original = contract.decode((output / "rootfs.json").read_bytes()) - entries = list(original["entries"]) - for name in STATIC_FILES: - raw = (SOURCE / name).read_bytes() - entries.append( - { - "path": "/opt/executor/" + name, - "type": "file", - "bytes": len(raw), - "sha256": contract.sha(raw), - } - ) - entries.sort(key=lambda row: row["path"]) - value = { - **original, - "entries": entries, - "rootfs_digest": "sha256:" + contract.sha(contract.canonical(entries)), - } - write(output / "executor-rootfs.json", value) - return value, contract.decode((output / "runtime-locators.private.json").read_bytes()) - - -def command(locators, snapshot, policy, policy_sha256, export_sha256, cohort_sha256): - args = runtime.command(locators, SOURCE / "synthetic_probe.py", {}, "/unmounted") - # Retain the reviewed mount base, replace the concrete work/shm sizes, and - # replace the legacy diagnostic dispatch without calling its limits(). - for mount, capacity in (("/work", policy["work_bytes"]), ("/dev/shm", policy["shm_bytes"])): - index = args.index(mount) - if args[index - 3 : index] != ["--size", args[index - 2], "--tmpfs"]: - raise ValueError("legacy mount command shape changed") - args[index - 2] = str(capacity) - payload_index = args.index("/opt/python/" + locators["python_relative"]) - mounts = [] - for name in STATIC_FILES: - mounts += ["--ro-bind", str(SOURCE / name), "/opt/executor/" + name] - mounts += ["--ro-bind", str(snapshot), "/input", "--remount-ro", "/dev"] - mount_index = args.index("--remount-ro") - return ( - args[:mount_index] - + mounts - + args[mount_index:payload_index] - + [ - args[payload_index], - "-IB", - "/opt/executor/inner.py", - policy_sha256, - export_sha256, - cohort_sha256, - ] - ) - - -def limits(policy): - os.sched_setaffinity(0, {min(os.sched_getaffinity(0))}) - for name, value in ( - ("AS", policy["as_bytes"]), - ("CPU", policy["cpu_seconds"]), - ("NOFILE", policy["nofile"]), - ("FSIZE", policy["file_bytes"]), - ("CORE", 0), - ): - resource.setrlimit(getattr(resource, "RLIMIT_" + name), (value, value)) - - -def status_end(fd, status_raw): - deadline = time.monotonic() + 2 - while time.monotonic() < deadline: - if not select.select([fd], [], [], max(0, deadline - time.monotonic()))[0]: - break - data = os.read(fd, 4096) - if not data: - return True - status_raw.extend(data) - if len(status_raw) > 65536: - raise ValueError("unexpected status stream size") - return False - - -def execute(args, rootfs, output, policy): - output.mkdir() - read_fd, write_fd = os.pipe() - block_read, block_write = os.pipe() - args[1:1] = ["--json-status-fd", str(write_fd), "--block-fd", str(block_read)] - process, init_fd = None, None - status_raw = bytearray() - report = { - "mode": "external-export", - "passed": False, - "timed_out": False, - "qualified": False, - "generated_code_executed": False, - } - write(output / "command.private.json", args) - try: - with ( - (output / "stdout.json").open("xb") as stdout, - (output / "stderr.private.log").open("xb") as stderr, - ): - process = subprocess.Popen( - [ - sys.executable, - "-IB", - str(SOURCE / "executor.py"), - "_exec", - contract.sha((SOURCE / "policy.json").read_bytes()), - json.dumps(args), - ], - stdin=subprocess.PIPE, - stdout=stdout, - stderr=stderr, - pass_fds=(write_fd, block_read), - start_new_session=True, - ) - os.close(write_fd) - write_fd = -1 - os.close(block_read) - block_read = -1 - supervisor.read_line(read_fd, time.monotonic() + 10, status_raw) - start = contract.decode(status_raw) - init_fd, identity = supervisor.bind_init(start, process.pid) - report["init_identity"] = identity - report["initial_status"] = start - os.write(block_write, b"1") - os.close(block_write) - block_write = -1 - process.stdin.write(supervisor.START_TOKEN) - process.stdin.flush() - process.stdin.close() - report["authorization_sent"] = True - started = time.monotonic() - try: - process.wait(timeout=policy["wall_seconds"]) - except subprocess.TimeoutExpired: - report["timed_out"] = True - raise - report["elapsed_seconds"] = time.monotonic() - started - report["exit_code"] = process.returncode - report["init_exit_observed"] = supervisor.exited(init_fd, 5000) - report["status_eof"] = status_end(read_fd, status_raw) - report["passed"] = ( - process.returncode == 0 and report["init_exit_observed"] and report["status_eof"] - ) - except BaseException as error: - report["error"] = {"type": type(error).__name__, "message": str(error)} - finally: - supervisor.finalize_case( - report, - process, - init_fd, - [], - (read_fd, write_fd, block_read, block_write), - output, - status_raw, - rootfs, - ) - return report - - -def evaluate_export(args): - policy = contract.policy(SOURCE / "policy.json", args.policy_sha256) - raw, export, samples = contract.inputs( - args.inputs, args.export_sha256, args.cohort_sha256, policy["input_bytes"] - ) - # This readiness entry never accepts model output or benchmark questions. - if raw != fixtures.payloads(): - raise ValueError("prototype only permits the exact declared handwritten fixtures") - args.output.mkdir(parents=False, exist_ok=False) - snapshot = args.output / "input-snapshot" - snapshot.mkdir() - for name, content in raw.items(): - with (snapshot / name).open("xb") as stream: - stream.write(content) - if supervisor.capability(args.output / "pidfd-capability.json") != 0: - raise ValueError("actual pidfd capability failed before sandbox creation") - before, locators = inventory(args.bundle, args.output / "inventory-before") - launch = command( - locators, snapshot, policy, args.policy_sha256, args.export_sha256, args.cohort_sha256 - ) - report = execute(launch, before, args.output / "execution", policy) - after, _ = inventory(args.bundle, args.output / "inventory-after") - unchanged = before == after - statuses = None - error = None - success = ( - report.get("passed") is True - and report.get("cleanup_complete") is True - and report.get("outputs_stable") is True - and unchanged - ) - if success: - try: - result_raw = contract.read( - args.output / "execution/stdout.json", policy["result_bytes"] - ) - statuses = contract.results(result_raw, export, samples, policy["result_bytes"]) - except (ValueError, OSError) as failure: - error = {"type": type(failure).__name__, "message": str(failure)} - success = False - identity = { - "schema_version": 1, - "runtime_kind": "readonly-bind-set-v1", - "runtime_digest": before["rootfs_digest"], - "evaluator_revision": runtime.REVISION, - "executor_sha256": contract.sha((SOURCE / "inner.py").read_bytes()), - "driver_sha256": contract.sha((SOURCE / "executor.py").read_bytes()), - "sandbox_binary_sha256": before["executor"]["binary_sha256"], - "policy_sha256": args.policy_sha256, - } - write( - args.output / "execution-record.json", - { - "scope": policy["scope"], - "protocol": None, - "observed_identity": identity, - "qualified": False, - "verdict": "INCONCLUSIVE", - "execution_completed": success, - "network": "none", - "exit_code": report.get("final_launcher_exit_code"), - "timed_out": report["timed_out"], - "cleanup_complete": report["cleanup_complete"], - "runtime_unchanged": unchanged, - "export_sha256": args.export_sha256, - "cohort_sha256": args.cohort_sha256, - "samples_sha256": export["samples_sha256"], - "dataset_sha256": export["dataset_sha256"], - "results_sha256": report["stdout_sha256"], - "statuses": statuses, - "validation_error": error, - }, - ) - return 0 if success else 2 - - -def main(): - parser = argparse.ArgumentParser(description=__doc__) - parser.add_argument("--bundle", type=Path, required=True) - parser.add_argument("--inputs", type=Path, required=True) - parser.add_argument("--output", type=Path, required=True) - parser.add_argument("--policy-sha256", required=True) - parser.add_argument("--export-sha256", required=True) - parser.add_argument("--cohort-sha256", required=True) - return evaluate_export(parser.parse_args()) - - -if __name__ == "__main__": - if not os.environ.get("SLURM_JOB_ID") or os.environ.get("SLURM_GPUS_ON_NODE", "0") != "0": - raise ValueError("CPU-only Slurm allocation required") - if len(sys.argv) == 4 and sys.argv[1] == "_exec": - selected = contract.policy(SOURCE / "policy.json", sys.argv[2]) - limits(selected) - launch = contract.decode(sys.argv[3]) - os.execv(launch[0], launch) - raise SystemExit(main()) diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-executor-v1/exercise.py b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-executor-v1/exercise.py deleted file mode 100644 index 5b3eeaf9a..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-executor-v1/exercise.py +++ /dev/null @@ -1,112 +0,0 @@ -"""Bounded CPU readiness exercise; all programs come from the fixed fixtures.""" - -import argparse -import sys -from pathlib import Path -from types import SimpleNamespace - -sys.path.insert(0, str(Path(__file__).resolve().parent)) -import contract -import executor -import fixtures - - -def main(): - parser = argparse.ArgumentParser(description=__doc__) - parser.add_argument("--bundle", type=Path, required=True) - parser.add_argument("--output", type=Path, required=True) - arguments = parser.parse_args() - arguments.output.mkdir() - identity = fixtures.create(arguments.output / "fixtures") - policy_raw = (executor.SOURCE / "policy.json").read_bytes() - policy_sha256 = contract.sha(policy_raw) - policy = contract.policy(executor.SOURCE / "policy.json", policy_sha256) - args = SimpleNamespace( - bundle=arguments.bundle, - inputs=arguments.output / "fixtures", - output=arguments.output / "normal", - policy_sha256=policy_sha256, - **identity, - ) - if executor.evaluate_export(args) != 0: - raise ValueError("generic evaluator path did not complete") - record = contract.decode((args.output / "execution-record.json").read_bytes()) - expected = { - "Synthetic/correct": ["pass", "pass"], - "Synthetic/plus_wrong": ["pass", "fail"], - "Synthetic/exception": ["fail", "fail"], - "Synthetic/input_timeout": ["fail", "fail"], - } - result = (args.output / "execution/stdout.json").read_bytes() - checks = { - "four_official_status_pairs": record["statuses"] == expected, - "complete_raw_result_larger_than_old_cap": len(result) > 1048576, - "raw_result_digest_bound": contract.sha(result) == record["results_sha256"], - "not_qualified": record["qualified"] is False and record["protocol"] is None, - "runtime_stable": record["runtime_unchanged"] is True, - } - try: - executor.evaluate_export(args) - except FileExistsError: - checks["old_output_rejected"] = True - else: - checks["old_output_rejected"] = False - inventory = args.output / "inventory-before" - rootfs = contract.decode((inventory / "executor-rootfs.json").read_bytes()) - locators = contract.decode((inventory / "runtime-locators.private.json").read_bytes()) - faults = [] - for name, index in (("wrong_policy", -3), ("wrong_export", -2)): - command = executor.command( - locators, args.output / "input-snapshot", policy, policy_sha256, **identity - ) - command[index] = "0" * 64 - observed = executor.execute(command, rootfs, arguments.output / name, policy) - stderr = (arguments.output / name / "stderr.private.log").read_bytes() - expected_error = ( - b"policy digest mismatch" if name == "wrong_policy" else b"export digest mismatch" - ) - passed = ( - observed["passed"] is False - and observed["final_launcher_exit_code"] != 0 - and observed["timed_out"] is False - and observed["cleanup_complete"] is True - and observed["outputs_stable"] is True - and (arguments.output / name / "stdout.json").read_bytes() == b"" - and stderr.startswith(b"PAYLOAD_START_AUTHORIZED\n") - and stderr.endswith(b"ValueError: " + expected_error + b"\n") - ) - checks[name + "_rejected_without_score"] = passed - faults.append( - { - "name": name, - "passed": passed, - "exit_code": observed["final_launcher_exit_code"], - "cleanup_complete": observed["cleanup_complete"], - } - ) - if observed["cleanup_complete"] is not True: - break - final, _ = executor.inventory(arguments.bundle, arguments.output / "inventory-final") - checks["runtime_stable_after_negative_controls"] = final == rootfs - executor.write( - arguments.output / "summary.json", - { - "scope": "four-handwritten-fixtures-through-external-export-interface", - "passed": all(checks.values()) and len(faults) == 2, - "checks": checks, - "negative_controls": faults, - "statuses": record["statuses"], - "results_bytes": len(result), - "results_sha256": contract.sha(result), - "observed_identity": record["observed_identity"], - "qualified": False, - "generated_code_executed": False, - "real_benchmark_data_mounted": False, - "verdict": "INCONCLUSIVE", - }, - ) - return 0 if all(checks.values()) and len(faults) == 2 else 2 - - -if __name__ == "__main__": - raise SystemExit(main()) diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-executor-v1/fixtures.py b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-executor-v1/fixtures.py deleted file mode 100644 index b8c86f8fe..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-executor-v1/fixtures.py +++ /dev/null @@ -1,64 +0,0 @@ -"""The four reviewed handwritten cases, with deterministic output-size padding.""" - -import hashlib -from pathlib import Path - -import contract - - -def payloads(): - # A comment-only prefix makes the retained result exceed the old 1 MiB cap. - padding = "#" + "fixed-fixture-padding " * 52000 + "\n" - solutions = { - "Synthetic/correct": padding + "def f(x):\n return x + 1\n", - "Synthetic/plus_wrong": "def f(x):\n return 1\n", - "Synthetic/exception": "def f(x):\n raise ArithmeticError('synthetic')\n", - "Synthetic/input_timeout": "import time\ndef f(x):\n time.sleep(5)\n return x + 1\n", - } - dataset = b"".join( - contract.canonical( - { - "task_id": identity, - "prompt": "def f(x):\n", - "canonical_solution": " return x + 1\n", - "contract": "", - "entry_point": "f", - "base_input": [[0]], - "plus_input": [[1]], - "atol": 0, - } - ) - + b"\n" - for identity in solutions - ) - samples = b"".join( - contract.canonical({"task_id": identity, "solution": solution}) + b"\n" - for identity, solution in solutions.items() - ) - export = ( - contract.canonical( - { - "samples_sha256": contract.sha(samples), - "dataset_sha256": contract.sha(dataset), - "evalplus_dataset_md5": hashlib.md5(dataset).hexdigest(), - "task_ids": sorted(solutions), - "generated_code_executed": False, - } - ) - + b"\n" - ) - return dict(zip(contract.INPUT_NAMES, (samples, dataset, export), strict=True)) - - -def create(directory): - Path(directory).mkdir() - values = payloads() - for name, raw in values.items(): - with (Path(directory) / name).open("xb") as stream: - stream.write(raw) - return { - "export_sha256": contract.sha(values["human-export.json"]), - "cohort_sha256": contract.sha( - contract.canonical(contract.decode(values["human-export.json"])["task_ids"]) - ), - } diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-executor-v1/fixtures.review-rejected-four-key-export.py b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-executor-v1/fixtures.review-rejected-four-key-export.py deleted file mode 100644 index a55af6b44..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-executor-v1/fixtures.review-rejected-four-key-export.py +++ /dev/null @@ -1,63 +0,0 @@ -"""The four reviewed handwritten cases, with deterministic output-size padding.""" - -import hashlib -from pathlib import Path - -import contract - - -def payloads(): - # A comment-only prefix makes the retained result exceed the old 1 MiB cap. - padding = "#" + "fixed-fixture-padding " * 52000 + "\n" - solutions = { - "Synthetic/correct": padding + "def f(x):\n return x + 1\n", - "Synthetic/plus_wrong": "def f(x):\n return 1\n", - "Synthetic/exception": "def f(x):\n raise ArithmeticError('synthetic')\n", - "Synthetic/input_timeout": "import time\ndef f(x):\n time.sleep(5)\n return x + 1\n", - } - dataset = b"".join( - contract.canonical( - { - "task_id": identity, - "prompt": "def f(x):\n", - "canonical_solution": " return x + 1\n", - "contract": "", - "entry_point": "f", - "base_input": [[0]], - "plus_input": [[1]], - "atol": 0, - } - ) - + b"\n" - for identity in solutions - ) - samples = b"".join( - contract.canonical({"task_id": identity, "solution": solution}) + b"\n" - for identity, solution in solutions.items() - ) - export = ( - contract.canonical( - { - "samples_sha256": contract.sha(samples), - "dataset_sha256": contract.sha(dataset), - "evalplus_dataset_md5": hashlib.md5(dataset).hexdigest(), - "task_ids": sorted(solutions), - } - ) - + b"\n" - ) - return dict(zip(contract.INPUT_NAMES, (samples, dataset, export), strict=True)) - - -def create(directory): - Path(directory).mkdir() - values = payloads() - for name, raw in values.items(): - with (Path(directory) / name).open("xb") as stream: - stream.write(raw) - return { - "export_sha256": contract.sha(values["human-export.json"]), - "cohort_sha256": contract.sha( - contract.canonical(contract.decode(values["human-export.json"])["task_ids"]) - ), - } diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-executor-v1/inner.py b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-executor-v1/inner.py deleted file mode 100644 index 111705bca..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-executor-v1/inner.py +++ /dev/null @@ -1,65 +0,0 @@ -"""Generic official evaluator entry, available only inside the readonly sandbox.""" - -import contextlib -import importlib -import os -import resource -import sys -from pathlib import Path - -# -I deliberately excludes script paths; only the explicitly readonly helper root is added. -sys.path.insert(0, "/opt/executor") -import contract -import synthetic_probe as boundary - - -def main(): - try: - boundary.authorize_start() - except (ValueError, OSError): - os.write(2, b"PAYLOAD_START_REJECTED\n") - return 23 - descriptor = os.open("/dev/null", os.O_RDONLY) - os.dup2(descriptor, 0) - if descriptor != 0: - os.close(descriptor) - os.write(2, b"PAYLOAD_START_AUTHORIZED\n") - policy = contract.policy("/opt/executor/policy.json", sys.argv[1]) - if sys.flags.optimize: - raise ValueError("EvalPlus correctness assertions must remain enabled") - for name, value in (("NPROC", policy["tasks"]), ("MEMLOCK", 0), ("MSGQUEUE", 0)): - resource.setrlimit(getattr(resource, "RLIMIT_" + name), (value, value)) - boundary.install_ipc_filter() - _, export, samples = contract.inputs("/input", sys.argv[2], sys.argv[3], policy["input_bytes"]) - work = Path("/work") - cache, output = work / "cache", work / "output/results.json" - if cache.exists() or output.exists(): - raise ValueError("evaluator output/cache reuse forbidden") - # All module-captured environment settings precede the first EvalPlus import. - os.environ["XDG_CACHE_HOME"] = str(cache) - os.environ["HUMANEVAL_OVERRIDE_PATH"] = "/input/humaneval-subset.jsonl" - os.environ.pop("EVALPLUS_MAX_MEMORY_BYTES", None) - os.environ.pop("EVALPLUS_TIMEOUT_PER_TASK", None) - sys.path.insert(0, "/opt/site-packages") - with contextlib.redirect_stdout(sys.stderr): - evaluator = importlib.import_module("evalplus.evaluate") - evaluator.evaluate( - dataset="humaneval", - samples="/input/humaneval-solutions.jsonl", - output_file=str(output), - parallel=1, - base_only=False, - mini=False, - noextreme=False, - test_details=True, - ) - raw = contract.read(output, policy["result_bytes"]) - contract.results(raw, export, samples, policy["result_bytes"]) - # Preserve exactly the upstream bytes once, including its original formatting. - sys.stdout.buffer.write(raw) - sys.stdout.buffer.flush() - return 0 - - -if __name__ == "__main__": - raise SystemExit(main()) diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-executor-v1/plan.json b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-executor-v1/plan.json deleted file mode 100644 index 324c9771a..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-executor-v1/plan.json +++ /dev/null @@ -1,57 +0,0 @@ -{ - "scope": "four-handwritten-fixtures-readiness", - "status": "reviewed_not_submitted", - "plane": "harness", - "verdict": "INCONCLUSIVE", - "qualified": false, - "generated_code_executed": false, - "source_sha256": { - "runtime.py": "997cbfed8398e8ad62d1beb4f6b4bc6e61a0f4572b9156cfd0c6f3fb52b03233", - "bounds_driver.py": "bfeeb6ea794c0ebf24e72a0127458ff74951253471daee080b3a3b1d82a062ba", - "synthetic_probe.py": "236920f27573447b9f3833afb72739033c965e22a9e406b09f28da815529ca64", - "contract.py": "bcd6a4a4d69f163075b89ae2e55c8fd475a9cc78ac64d39eadda1a54b4557dec", - "inner.py": "5a635ae5773d08a3cbcda07551450b38d3fd6af64f8ca251cc61a7a94c388342", - "executor.py": "15613932aec108c27b2de2390a965aaba4074535f0e02295f4e7846e405f282c", - "fixtures.py": "166dc7c82c9b0847144a56f46a30f97e93f65967a017e79cb9f8b916219364cc", - "exercise.py": "cc4f31657a89f0ca1a3bacd86e9060f2fbf633557ee05b6695f6ccba91d6e92f", - "test_contract.py": "94b32dfc8bc80d4f46f211fc6d0d36065d6043a47bff04a4856d3eb942e7cf1e", - "policy.json": "222326a1b27e48fd55988a2c10a7065fa44ad8cd22e71348bff355aa1c15996d", - "run-executor.sbatch": "99626cbef13f865de7fee59746fe228ebae885aac76bcc6029955c1a36cca16f" - }, - "source_manifest_sha256": "2d37ee54c8d5c54edcdaa08c5ff1ac0fc1d28ed530e5680be2f0b582a55fa2d6", - "allocation": { - "cpus": 4, - "memory_request_gib": 16, - "wall_minutes": 15, - "gpus": 0 - }, - "policy": { - "schema": 1, - "scope": "four-handwritten-fixtures-readiness", - "qualified": false, - "as_bytes": 4294967296, - "cpu_seconds": 120, - "wall_seconds": 180, - "file_bytes": 67108864, - "work_bytes": 268435456, - "shm_bytes": 33554432, - "tasks": 16, - "nofile": 64, - "input_bytes": 67108864, - "result_bytes": 67108864, - "evalplus": { - "parallel": 1, - "base_only": false, - "mini": false, - "noextreme": false, - "test_details": true, - "memory_override": null, - "timeout_override": null - } - }, - "evaluator_revision": "26d6d00bb1fd0fa37f39c99d5290da67891d1c5e", - "readiness_gpu_seconds_used": 2388, - "readiness_gpu_seconds_cap": 7200, - "initial_gpu_seconds_separate": 147, - "e2e_effect": null -} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-executor-v1/policy.json b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-executor-v1/policy.json deleted file mode 100644 index 6d6cc9dcc..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-executor-v1/policy.json +++ /dev/null @@ -1 +0,0 @@ -{"schema":1,"scope":"four-handwritten-fixtures-readiness","qualified":false,"as_bytes":4294967296,"cpu_seconds":120,"wall_seconds":180,"file_bytes":67108864,"work_bytes":268435456,"shm_bytes":33554432,"tasks":16,"nofile":64,"input_bytes":67108864,"result_bytes":67108864,"evalplus":{"parallel":1,"base_only":false,"mini":false,"noextreme":false,"test_details":true,"memory_override":null,"timeout_override":null}} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-executor-v1/proposal.md b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-executor-v1/proposal.md deleted file mode 100644 index 21b6a7c4e..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-executor-v1/proposal.md +++ /dev/null @@ -1,59 +0,0 @@ -# External-export evaluator readiness - -Refs [Issue #67](https://github.com/InferMatrixLab/InferRSI/issues/67). -Plane: harness. Campaign verdict: INCONCLUSIVE. E2E effect: not measured. - -The prior successful CPU2014 diagnostic wrote its own four inputs inside the -sandbox. This version exercises a parameterized, byte-authenticated external -export: samples, dataset, five-field export manifest and declared cohort identity. -The prototype only permits the exact four handwritten fixture byte streams. A -deterministic comment on the correct fixture makes the raw result exceed the -previous 1 MiB file cap without adding model-generated code or benchmark data. - -Pinned EvalPlus uses a default per-process memory guard of 4 GiB. The old diagnostic -overrode it to 2 GiB and imposed a 30-second whole-process CPU limit. Neither is -an unchanged official default or a qualified complete-164-task configuration. -This readiness policy preserves the official memory/timeout defaults with outer -AS 4 GiB, CPU 120 seconds, wall 180 seconds, 64 MiB per-file/output caps, 256 MiB -work tmpfs and 32 MiB shared tmpfs. UID task limit 16, nofile 64, zero core/locked -memory/message queues, readonly runtime/device mounts, network isolation and the -reviewed IPC filter remain. These are finite prototype limits, not accepted -quality gates or a 16 GiB aggregate memory guarantee. Full-dataset resource -qualification remains required. - -The actual pidfd capability, startup token and final-output cleanup helpers are -reused unchanged. A fresh child applies the new limits directly; it never first -lowers the old hard limits. Runtime identity covers every executed helper and -policy byte. Per-execution sample/dataset/export/cohort hashes remain separate -from the shared readonly runtime identity. Each run gets a fresh input snapshot, -namespace, work directory, output path and cache. EvalPlus environment settings -are applied before import; its existing-result and ground-truth-cache reload -paths cannot silently reuse a previous invocation's files. - -The result validator requires the exact expected cohort, one result per solution, -matching solution bytes, the dataset MD5 used by EvalPlus and known base/plus -statuses. The exact official JSON is retained once after confirmed namespace and -launcher exit. Exit/timeout/cleanup failures yield no accepted score. No protocol -qualification is fabricated: execution records retain `qualified: false`, a -null frozen protocol and an observed runtime identity, so the existing v5 final -score gate remains closed. - -One bounded CPU allocation is proposed: 4 CPUs, 16 GiB request, 15 minutes, -zero GPUs, unchanged registered dependencies. Exercise the four official -status pairs, a result larger than 1 MiB, old-output rejection, and actual -wrong-policy/wrong-export failures with empty score output and complete cleanup. -Compare readonly inventories and all source/dependency/binary hashes afterward. -Any incomplete cleanup stops further cases. Keep raw outputs and terminal state -whether the attempt succeeds or fails. - -Independent review identified a P2 interface mismatch in preparation: the first -validator and fixture used four export fields, while the actual v5 score.py also -exports `generated_code_executed: false`. The rejected source is retained, and -the revised validator requires that fifth field with the exact boolean type. -Eighteen host-only controls cover this compatibility, input/result tampering, -duplicate/missing IDs, truncation, caps, readonly-command policy and output reuse. -They do not establish Linux behavior or generated-code qualification. Initial -lint findings were corrected before source lock. Formal task quality, full freeze -and same-H20 paired performance comparison remain pending. - -Review also rejected a stale-result-digest path: the original driver parsed the final file but retained the earlier supervisor digest without comparing it. The fix hashes the same parsed bytes, requires equality and emits no accepted result digest on failure. Result row task IDs must match their enclosing IDs. Rejected sources remain retained. Negative runtime controls now require the authorization marker and exact digest-mismatch cause, so unrelated startup errors cannot satisfy them. Local bind-failure/timeout controls verify finalization; the initial mock lacked a block-pipe reader and hit BrokenPipeError, and is retained before correction. diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-executor-v1/review.json b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-executor-v1/review.json deleted file mode 100644 index 041b7c417..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-executor-v1/review.json +++ /dev/null @@ -1,30 +0,0 @@ -{ - "reviewer": "independent Codex subagent", - "source_sha256": { - "runtime.py": "997cbfed8398e8ad62d1beb4f6b4bc6e61a0f4572b9156cfd0c6f3fb52b03233", - "bounds_driver.py": "bfeeb6ea794c0ebf24e72a0127458ff74951253471daee080b3a3b1d82a062ba", - "synthetic_probe.py": "236920f27573447b9f3833afb72739033c965e22a9e406b09f28da815529ca64", - "contract.py": "bcd6a4a4d69f163075b89ae2e55c8fd475a9cc78ac64d39eadda1a54b4557dec", - "inner.py": "5a635ae5773d08a3cbcda07551450b38d3fd6af64f8ca251cc61a7a94c388342", - "executor.py": "15613932aec108c27b2de2390a965aaba4074535f0e02295f4e7846e405f282c", - "fixtures.py": "166dc7c82c9b0847144a56f46a30f97e93f65967a017e79cb9f8b916219364cc", - "exercise.py": "cc4f31657a89f0ca1a3bacd86e9060f2fbf633557ee05b6695f6ccba91d6e92f", - "test_contract.py": "94b32dfc8bc80d4f46f211fc6d0d36065d6043a47bff04a4856d3eb942e7cf1e", - "policy.json": "222326a1b27e48fd55988a2c10a7065fa44ad8cd22e71348bff355aa1c15996d", - "run-executor.sbatch": "99626cbef13f865de7fee59746fe228ebae885aac76bcc6029955c1a36cca16f" - }, - "manifest_sha256": "2d37ee54c8d5c54edcdaa08c5ff1ac0fc1d28ed530e5680be2f0b582a55fa2d6", - "blocking_findings": [], - "resolved_findings": [ - "v5 five-key export schema and exact false marker", - "result row task identity", - "same parsed bytes bound to final supervisor digest", - "exact negative-control failure cause" - ], - "checks": [ - "18 host-only controls with -IBO", - "bash -n", - "source inventory and plan/control digest mappings" - ], - "scope": "One4CPU16GiBrequest15min0GPU trusted readiness exercise; no full164task qualification or E2E claim." -} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-executor-v1/run-executor.sbatch b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-executor-v1/run-executor.sbatch deleted file mode 100644 index 535a811fa..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-executor-v1/run-executor.sbatch +++ /dev/null @@ -1,203 +0,0 @@ -#!/usr/bin/env bash -# Four fixed trusted fixtures through the external-export evaluator interface. -#SBATCH --job-name=qwen38-sandbox-executor-v1 -#SBATCH --nodes=1 -#SBATCH --ntasks=1 -#SBATCH --cpus-per-task=4 -#SBATCH --mem=16G -#SBATCH --time=00:15:00 -set -euo pipefail -umask 077 -: "${TASK_ROOT:?}" -: "${SANDBOX_BUNDLE:?}" -: "${SANDBOX_FREEZE_SHA256:?}" -: "${SANDBOX_VENV:?}" -: "${REFERENCE_INSTALLATION:?}" -: "${REFERENCE_ROOTFS_SHA256:?}" -: "${REFERENCE_TERMINAL_SHA256:?}" -: "${REFERENCE_SLURM_SHA256:?}" -: "${REFERENCE_JOB_ID:?}" -: "${REFERENCE_EXECUTION_SHA256:?}" "${REFERENCE_STDOUT_SHA256:?}" -: "${BWRAP_BIN:?}" "${BWRAP_SHA256:?}" "${PYTHON_SHA256:?}" -: "${SYNTHETIC_SOURCE_DIR:?}" -: "${SYNTHETIC_RUNTIME_SHA256:?}" -: "${SYNTHETIC_PROBE_SHA256:?}" -: "${BOUNDS_DRIVER_SHA256:?}" -: "${SYNTHETIC_SCRIPT_SHA256:?}" -: "${SYNTHETIC_OUTPUT:?}" "${EXECUTOR_SOURCES_SHA256:?}" -unset PYTHONPATH PYTHONHOME PYTHONOPTIMIZE PYTHONINSPECT PYTHONSTARTUP VIRTUAL_ENV -[[ -n ${SLURM_JOB_ID:-} && ${SLURM_GPUS_ON_NODE:-0} == 0 ]] -root=$(realpath -e "$TASK_ROOT") -check_path() { - local value - value=$(realpath -m "$1") - case "$value" in "$root"/*) ;; *) echo 'Path escapes task root.' >&2; exit 2;; esac -} -check_digest() { - if [[ ! "$2" =~ ^[a-f0-9]{64}$ ]]; then return 2; fi - [[ $(sha256sum "$1" | cut -d' ' -f1) == "$2" ]] -} -for path in "$SANDBOX_BUNDLE" "$SANDBOX_VENV" "$REFERENCE_INSTALLATION" \ - "$SYNTHETIC_SOURCE_DIR" "$SYNTHETIC_OUTPUT" "$BWRAP_BIN"; do check_path "$path"; done -source_dir=$(cd -- "$SYNTHETIC_SOURCE_DIR" && pwd -P) -check_digest "$SANDBOX_VENV/bin/python" "$PYTHON_SHA256" -check_digest "$BWRAP_BIN" "$BWRAP_SHA256" -export PATH="$(dirname "$BWRAP_BIN"):/usr/bin:/bin" -[[ $(command -v bwrap) == "$BWRAP_BIN" ]] -check_digest "$source_dir/run-executor.sbatch" "$SYNTHETIC_SCRIPT_SHA256" -cmp "${BASH_SOURCE[0]}" "$source_dir/run-executor.sbatch" -check_digest "$source_dir/runtime.py" "$SYNTHETIC_RUNTIME_SHA256" -check_digest "$source_dir/synthetic_probe.py" "$SYNTHETIC_PROBE_SHA256" -check_digest "$source_dir/bounds_driver.py" "$BOUNDS_DRIVER_SHA256" -check_digest "$source_dir/sources.sha256" "$EXECUTOR_SOURCES_SHA256" -(cd "$source_dir" && sha256sum --check --strict sources.sha256) -check_digest "$SANDBOX_BUNDLE/freeze.sha256" "$SANDBOX_FREEZE_SHA256" -(cd "$SANDBOX_BUNDLE" && sha256sum --check --strict freeze.sha256) -# Bind both the scheduler outcome and the repaired completion record. -check_digest "$REFERENCE_INSTALLATION/terminal-status.json" "$REFERENCE_TERMINAL_SHA256" -check_digest "$REFERENCE_INSTALLATION/slurm-final.private.txt" "$REFERENCE_SLURM_SHA256" -"$SANDBOX_VENV/bin/python" -IB - "$REFERENCE_INSTALLATION" "$REFERENCE_JOB_ID" <<'PYREF' -import json -import re -import sys -from pathlib import Path -reference = Path(sys.argv[1]) -job = sys.argv[2] -if not re.fullmatch(r"[0-9]+", job): - raise ValueError('Invalid reference job ID') -def unique_keys(pairs): - value = {} - for key, item in pairs: - if key in value: - raise ValueError('Duplicate terminal-record key') - value[key] = item - return value -terminal = json.loads((reference / 'terminal-status.json').read_bytes(), object_pairs_hook=unique_keys) -if terminal.get('completed') is not True or type(terminal.get('exit_code')) is not int: - raise ValueError('Reference terminal record has invalid types') -if terminal != {'completed': True, 'signal': 'none', 'exit_code': 0, 'job_id': job}: - raise ValueError('Reference installation did not complete normally') -slurm = (reference / 'slurm-final.private.txt').read_text() -for key, value in {'JobId': job, 'JobState': 'COMPLETED', 'ExitCode': '0:0'}.items(): - actual = re.findall(r'(? "$SYNTHETIC_OUTPUT/exit-code" - printf '{"completed":%s,"signal":"%s","exit_code":%s,"job_id":"%s"}\n' \ - "$task_completed" "$task_signal" "$status" "$SLURM_JOB_ID" > "$SYNTHETIC_OUTPUT/terminal-status.json" - exit "$status" -} -trap finish_synthetic EXIT -trap 'task_completed=false; task_signal=TERM; exit 143' TERM -trap 'task_completed=false; task_signal=INT; exit 130' INT -trap 'task_completed=false; task_signal=HUP; exit 129' HUP -# END TERMINAL RECORD -cp "$source_dir/run-executor.sbatch" "$source_dir/runtime.py" "$source_dir/bounds_driver.py" "$source_dir/synthetic_probe.py" "$source_dir/executor.py" "$source_dir/inner.py" "$source_dir/contract.py" "$source_dir/fixtures.py" "$source_dir/exercise.py" "$source_dir/test_contract.py" "$source_dir/policy.json" "$source_dir/sources.sha256" "$SYNTHETIC_OUTPUT/" -cd "$root" -"$SANDBOX_VENV/bin/python" -IB "$source_dir/bounds_driver.py" capability \ - --output "$SYNTHETIC_OUTPUT/pidfd-capability.json" -"$SANDBOX_VENV/bin/python" -IB "$source_dir/runtime.py" inventory --bundle "$SANDBOX_BUNDLE" \ - --output "$SYNTHETIC_OUTPUT/rootfs-before" -"$SANDBOX_VENV/bin/python" -IB - "$REFERENCE_INSTALLATION/rootfs-before/rootfs.json" \ - "$SYNTHETIC_OUTPUT/rootfs-before/rootfs.json" <<'PY' -import json -import sys -from pathlib import Path -before, current = [json.loads(Path(path).read_bytes()) for path in sys.argv[1:]] -for record in (before, current): - record.pop('rootfs_digest') - record['entries'] = [row for row in record['entries'] if row['path'] != '/opt/probe.py'] -if before != current: - raise ValueError('Installed runtime changed beyond the explicitly bound trusted probe') -PY -probe_status=0 -"$SANDBOX_VENV/bin/python" -IB "$source_dir/exercise.py" --bundle "$SANDBOX_BUNDLE" --output "$SYNTHETIC_OUTPUT/exercise" || probe_status=$? -printf '%s\n' "$probe_status" > "$SYNTHETIC_OUTPUT/probe-status" -post_status=0 -run_postcheck() { - local name=$1 status=0 - shift - "$@" || status=$? - printf '%s\t%s\n' "$name" "$status" >> "$SYNTHETIC_OUTPUT/postflight-status.tsv" - if [[ "$status" != 0 ]]; then post_status=$status; fi -} -check_executor_sources() { (cd "$source_dir" && sha256sum --check --strict sources.sha256); } -check_frozen_files() { (cd "$SANDBOX_BUNDLE" && sha256sum --check --strict freeze.sha256); } -run_postcheck inventory "$SANDBOX_VENV/bin/python" -IB "$source_dir/runtime.py" inventory \ - --bundle "$SANDBOX_BUNDLE" --output "$SYNTHETIC_OUTPUT/rootfs-after" -run_postcheck runtime-byte-comparison cmp "$SYNTHETIC_OUTPUT/rootfs-before/rootfs.json" \ - "$SYNTHETIC_OUTPUT/rootfs-after/rootfs.json" -run_postcheck runtime-script check_digest "$source_dir/runtime.py" "$SYNTHETIC_RUNTIME_SHA256" -run_postcheck bounds-driver check_digest "$source_dir/bounds_driver.py" "$BOUNDS_DRIVER_SHA256" -run_postcheck synthetic-probe check_digest "$source_dir/synthetic_probe.py" "$SYNTHETIC_PROBE_SHA256" -run_postcheck batch-script check_digest "$source_dir/run-executor.sbatch" "$SYNTHETIC_SCRIPT_SHA256" -run_postcheck executor-manifest check_digest "$source_dir/sources.sha256" "$EXECUTOR_SOURCES_SHA256" -run_postcheck executor-sources check_executor_sources -run_postcheck freeze-manifest check_digest "$SANDBOX_BUNDLE/freeze.sha256" "$SANDBOX_FREEZE_SHA256" -run_postcheck freeze-files check_frozen_files -run_postcheck python-binary check_digest "$SANDBOX_VENV/bin/python" "$PYTHON_SHA256" -run_postcheck bwrap-binary check_digest "$BWRAP_BIN" "$BWRAP_SHA256" -if [[ "$probe_status" != 0 ]]; then exit "$probe_status"; fi -if [[ "$post_status" != 0 ]]; then exit "$post_status"; fi -echo 'Trusted export-interface exercise completed; generated-code executor remains unqualified.' -task_completed=true diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-executor-v1/runtime.py b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-executor-v1/runtime.py deleted file mode 100644 index 51f1903f3..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-executor-v1/runtime.py +++ /dev/null @@ -1,481 +0,0 @@ -"""Resolve pinned EvalPlus metadata, inventory readonly binds and run a trusted probe.""" - -import argparse -import configparser -import hashlib -import importlib.metadata -import json -import os -import re -import resource -import shutil -import signal -import subprocess -import sys -import sysconfig -import threading -import tomllib -from pathlib import Path - -REVISION = "26d6d00bb1fd0fa37f39c99d5290da67891d1c5e" - - -def canonical(value): - return json.dumps(value, sort_keys=True, separators=(",", ":"), allow_nan=False).encode() - - -def sha(raw): - return hashlib.sha256(raw).hexdigest() - - -def write(path, value): - with path.open("xb") as stream: - stream.write(canonical(value) + b"\n") - - -def git(source, *args): - return subprocess.check_output(["git", "-C", str(source), *args]) - - -def source_identity(bundle): - source = bundle / "source" - if git(source, "rev-parse", "HEAD").decode().strip() != REVISION: - raise ValueError("wrong EvalPlus source revision") - if git(source, "diff", "HEAD", "--"): - raise ValueError("tracked EvalPlus source changed") - return { - "evaluator_revision": REVISION, - "archive_sha256": sha(git(source, "archive", "--format=tar", "HEAD")), - } - - -def requirements(bundle): - identity = source_identity(bundle) - config = configparser.ConfigParser() - config.read(bundle / "source/setup.cfg") - dependencies = [ - line.strip() for line in config["options"]["install_requires"].splitlines() if line.strip() - ] - build = tomllib.loads((bundle / "source/pyproject.toml").read_text())["build-system"][ - "requires" - ] - (bundle / "dependencies.in").write_text("\n".join(dependencies) + "\n") - (bundle / "build.in").write_text("\n".join([*build, "wheel"]) + "\n") - write( - bundle / "source.json", - { - **identity, - "official_default_dependencies": dependencies, - "build_dependencies": [*build, "wheel"], - "extras_enabled": [], - "generated_code_executed": False, - }, - ) - - -def verify_source(bundle): - expected = json.loads((bundle / "source.json").read_bytes()) - if any( - source_identity(bundle)[name] != expected[name] - for name in ("evaluator_revision", "archive_sha256") - ): - raise ValueError("EvalPlus source snapshot drift") - - -def tree(root, alias): - result = [] - for path in sorted(root.rglob("*")): - relative = path.relative_to(root).as_posix() - name = alias + "/" + relative - if path.is_symlink(): - target = os.readlink(path) - if Path(target).is_absolute() or not path.resolve(strict=True).is_relative_to(root): - raise ValueError("runtime symlink escapes its readonly root") - result.append({"path": name, "type": "symlink", "target": target}) - elif path.is_dir(): - result.append({"path": name, "type": "directory", "mode": path.stat().st_mode & 0o777}) - elif path.is_file(): - result.append( - { - "path": name, - "type": "file", - "mode": path.stat().st_mode & 0o777, - "bytes": path.stat().st_size, - "sha256": sha(path.read_bytes()), - } - ) - else: - raise ValueError("unsupported runtime filesystem entry") - return result - - -def system_dependencies(roots): - # Mirror the explicit relocated loader directory; inherit no host loader state. - loader_env = { - "PATH": "/usr/bin:/bin", "LC_ALL": "C", - "LD_LIBRARY_PATH": str(roots[0] / "lib"), - } - ldd = shutil.which("ldd") - if ldd is None: - raise ValueError("system ELF dependency scanner unavailable") - dependencies = {} - for root in roots: - for path in sorted(root.rglob("*")): - if not path.is_file() or path.is_symlink(): - continue - with path.open("rb") as stream: - if stream.read(4) != b"\x7fELF": - continue - result = subprocess.run( - [ldd, str(path)], capture_output=True, timeout=10, check=False, env=loader_env - ) - output = result.stdout.decode(errors="replace") - if result.returncode != 0 or "not found" in output: - raise ValueError("ELF runtime dependency missing") - for line in output.splitlines(): - match = re.search(r"(?:=>\s*)?(/[^\s]+)", line) - if not match: - continue - original = Path(match.group(1)) - resolved = original.resolve(strict=True) - if any(resolved.is_relative_to(bound) for bound in roots): - continue - if not str(original).startswith(("/lib/", "/lib64/", "/usr/lib/", "/usr/lib64/")): - raise ValueError("unexpected external runtime library") - dependencies[str(original)] = str(resolved) - return dependencies - - -def inventory(bundle, output): - verify_source(bundle) - task_root = Path(os.environ["TASK_ROOT"]).resolve(strict=True) - base = Path(sys.base_prefix).resolve(strict=True) - site = Path(sysconfig.get_path("purelib")).resolve(strict=True) - if not base.is_relative_to(task_root) or not site.is_relative_to(task_root): - raise ValueError("runtime roots must remain inside TASK_ROOT") - executable = Path(sys._base_executable).resolve(strict=True).relative_to(base).as_posix() - bwrap = shutil.which("bwrap") - if bwrap is None: - raise ValueError("bubblewrap is unavailable on this compute node") - help_text = subprocess.check_output([bwrap, "--help"], stderr=subprocess.STDOUT).decode() - for flag in ( - "--unshare-user", - "--unshare-net", - "--unshare-pid", - "--disable-userns", - "--assert-userns-disabled", - "--clearenv", - "--remount-ro", - "--size", - ): - if flag not in help_text: - raise ValueError("required bubblewrap isolation option unavailable") - dependencies = system_dependencies((base, site)) - entries = tree(base, "/opt/python") + tree(site, "/opt/site-packages") - for destination, source in dependencies.items(): - path = Path(source) - entries.append( - { - "path": destination, - "type": "file", - "bytes": path.stat().st_size, - "sha256": sha(path.read_bytes()), - } - ) - probe = Path(__file__).with_name("synthetic_probe.py") - entries.append( - { - "path": "/opt/probe.py", - "type": "file", - "bytes": probe.stat().st_size, - "sha256": sha(probe.read_bytes()), - } - ) - entries.sort(key=lambda row: row["path"]) - # Bind-set identity describes actual readonly bytes, not an OCI image. - rootfs_digest = "sha256:" + sha(canonical(entries)) - packages = sorted( - [ - {"name": d.metadata["Name"], "version": d.version} - for d in importlib.metadata.distributions() - ], - key=lambda row: row["name"].lower(), - ) - # Match every pinned evaluator Python source byte to the installed package. - checked = 0 - source = bundle / "source" - for encoded in git(source, "ls-files", "-z", "evalplus").split(b"\0"): - if not encoded or not encoded.endswith(b".py"): - continue - name = encoded.decode() - if (source / name).read_bytes() != (site / name).read_bytes(): - raise ValueError("installed EvalPlus differs from pinned source") - checked += 1 - if checked == 0: - raise ValueError("no pinned evaluator Python sources verified") - output.mkdir(parents=True, exist_ok=False) - write( - output / "rootfs.json", - { - "schema": 1, - "rootfs_kind": "readonly-bind-set-v1", - "loader_policy": { - "scanner_inherits_environment": False, - "base_library_directory": "/opt/python/lib", - "sandbox_ld_library_path": "/opt/python/lib", - }, - "rootfs_digest": rootfs_digest, - "evaluator_revision": REVISION, - "entries": entries, - "python": sys.version.split()[0], - "packages": packages, - "checked_evaluator_sources": checked, - "executor": { - "kind": "bubblewrap", - "binary_sha256": sha(Path(bwrap).read_bytes()), - "version": subprocess.check_output([bwrap, "--version"]).decode().strip(), - }, - "generated_code_executed": False, - "qualified_generated_code_executor": False, - }, - ) - write( - output / "runtime-locators.private.json", - { - "python_root": str(base), - "site_root": str(site), - "python_relative": executable, - "bwrap": bwrap, - "system_libraries": dependencies, - }, - ) - - -def command(locators, probe, namespaces, sentinel): - args = [ - locators["bwrap"], - "--unshare-user", - "--uid", - "65534", - "--gid", - "65534", - "--unshare-net", - "--unshare-pid", - "--unshare-ipc", - "--unshare-uts", - "--hostname", - "sandbox", - "--disable-userns", - "--assert-userns-disabled", - "--cap-drop", - "ALL", - "--die-with-parent", - "--new-session", - "--clearenv", - "--ro-bind", - locators["python_root"], - "/opt/python", - "--ro-bind", - locators["site_root"], - "/opt/site-packages", - "--ro-bind", - str(probe), - "/opt/probe.py", - ] - for destination, source in sorted(locators["system_libraries"].items()): - args.extend(["--ro-bind", source, destination]) - args += [ - "--proc", - "/proc", - "--dev", - "/dev", - "--size", - "33554432", - "--tmpfs", - "/dev/shm", - "--size", - "67108864", - "--tmpfs", - "/work", - "--dir", - "/work/tmp", - "--dir", - "/work/output", - "--chdir", - "/work", - "--remount-ro", - "/", - ] - environment = { - "HOME": "/work", - "TMPDIR": "/work/tmp", - "PATH": "/opt/python/bin", - "LD_LIBRARY_PATH": "/opt/python/lib", - "PYTHONDONTWRITEBYTECODE": "1", - "OMP_NUM_THREADS": "1", - "OPENBLAS_NUM_THREADS": "1", - "LC_ALL": "C.UTF-8", - "PROBE_PARENT_NAMESPACES": json.dumps(namespaces), - "PROBE_UNMOUNTED_SENTINEL": str(sentinel), - } - for name, value in environment.items(): - args += ["--setenv", name, value] - return args + ["/opt/python/" + locators["python_relative"], "-I", "/opt/probe.py"] - - -def limits(): - os.sched_setaffinity(0, {min(os.sched_getaffinity(0))}) - for name, value in ( - ("AS", 2147483648), - ("CPU", 30), - ("NOFILE", 64), - ("FSIZE", 1048576), - ("CORE", 0), - ): - resource.setrlimit(getattr(resource, "RLIMIT_" + name), (value, value)) - - -def probe(inventory_path, output): - rootfs = json.loads((inventory_path / "rootfs.json").read_bytes()) - locators = json.loads((inventory_path / "runtime-locators.private.json").read_bytes()) - if sha(Path(locators["bwrap"]).read_bytes()) != rootfs["executor"]["binary_sha256"]: - raise ValueError("bubblewrap binary changed after inventory") - output.mkdir(parents=True, exist_ok=False) - sentinel = output / "unmounted-sentinel.private" - sentinel.write_text("synthetic host marker, not a credential") - namespaces = { - name: os.readlink("/proc/self/ns/" + name) for name in ("user", "net", "pid", "uts", "ipc") - } - args = command(locators, Path(__file__).with_name("synthetic_probe.py"), namespaces, sentinel) - timed_out = False - # Sample from the supervisor after Popen. The evaluator process remains free - # of an added observer thread before its own default multiprocessing fork. - import psutil - sampled = {"processes": 0, "threads": 0, "sum_vms_bytes": 0, - "sum_rss_bytes": 0, "sum_fds": 0, "samples": 0, "errors": 0} - known = {} - sampling_failures = [] - stop_sampling = threading.Event() - def observe(pid): - try: - parent = psutil.Process(pid) - except psutil.NoSuchProcess: - sampling_failures.append("launcher exited before sampling") - return - while not stop_sampling.is_set(): - totals = {"processes": 0, "threads": 0, "sum_vms_bytes": 0, - "sum_rss_bytes": 0, "sum_fds": 0} - try: - processes = [parent, *parent.children(recursive=True)] - except psutil.NoSuchProcess: - break - except psutil.AccessDenied: - sampling_failures.append("cannot inspect owned launcher descendants") - return - for child in processes: - try: - known[child.pid] = child.create_time() - memory = child.memory_info() - totals["processes"] += 1 - totals["threads"] += child.num_threads() - totals["sum_vms_bytes"] += memory.vms - totals["sum_rss_bytes"] += memory.rss - totals["sum_fds"] += child.num_fds() - except (psutil.NoSuchProcess, psutil.AccessDenied): - sampled["errors"] += 1 - for name, value in totals.items(): - sampled[name] = max(sampled[name], value) - sampled["samples"] += 1 - stop_sampling.wait(0.01) - with ( - (output / "stdout.json").open("xb") as stdout, - (output / "stderr.private.log").open("xb") as stderr, - ): - process = subprocess.Popen( - [sys.executable, str(Path(__file__).resolve()), "_exec", json.dumps(args)], - stdin=subprocess.DEVNULL, - stdout=stdout, - stderr=stderr, - start_new_session=True, - env={**os.environ, "PROBE_INHERITED_SECRET": "synthetic-must-be-cleared"}, - ) - observer = threading.Thread(target=observe, args=(process.pid,), daemon=True) - observer.start() - try: - code = process.wait(timeout=60) - except subprocess.TimeoutExpired: - timed_out = True - if os.getpgid(process.pid) != process.pid: - raise RuntimeError("owned sandbox process group changed unexpectedly") - os.killpg(process.pid, signal.SIGKILL) - code = process.wait(timeout=10) - finally: - stop_sampling.set() - observer.join(timeout=2) - remaining = [] - for pid, created in known.items(): - try: - child = psutil.Process(pid) - if child.create_time() == created and child.status() != psutil.STATUS_ZOMBIE: - remaining.append(pid) - except psutil.NoSuchProcess: - pass - result = { - "sampled_peaks": sampled, - "sampling_thread_joined": not observer.is_alive(), - "sampling_failures": sampling_failures, - "known_live_descendants_after_exit": remaining, - "sampling_limitation": "Polling may miss transient processes; sampled peaks are not hard bounds or exhaustive cleanup proof.", - "phase": "trusted_synthetic_evaluator", - "exit_code": code, - "timed_out": timed_out, - "rootfs_digest": rootfs["rootfs_digest"], - "executor": rootfs["executor"], - "policy_script_sha256": sha(Path(__file__).read_bytes()), - "probe_script_sha256": sha(Path(__file__).with_name("synthetic_probe.py").read_bytes()), - "stdout_sha256": sha((output / "stdout.json").read_bytes()), - "generated_code_executed": False, - "full_evaluator_attempt_requested": True, - "qualified_generated_code_executor": False, - "verdict": "INCONCLUSIVE", - } - write(output / "execution.json", result) - if code != 0 or timed_out or remaining or observer.is_alive() or sampling_failures or not sampled["samples"]: - raise ValueError("trusted synthetic evaluator probe failed; retain evidence without fallback") - payload = json.loads((output / "stdout.json").read_bytes()) - if payload.get("passed") is not True: - raise ValueError("trusted synthetic evaluator assertions failed") - - -def main(): - parser = argparse.ArgumentParser(description=__doc__) - parser.add_argument("phase", choices=("requirements", "verify-source", "inventory", "probe")) - parser.add_argument("--bundle", type=Path, required=True) - parser.add_argument("--output", type=Path) - parser.add_argument("--inventory", type=Path) - args = parser.parse_args() - if not os.environ.get("SLURM_JOB_ID") or os.environ.get("SLURM_GPUS_ON_NODE", "0") != "0": - raise ValueError("CPU-only Slurm allocation required") - if args.phase == "requirements": - requirements(args.bundle) - elif args.phase == "verify-source": - verify_source(args.bundle) - elif args.phase == "inventory": - inventory(args.bundle, args.output) - else: - probe(args.inventory, args.output) - - -if __name__ == "__main__": - if len(sys.argv) == 3 and sys.argv[1] == "_exec": - # A fresh interpreter sets limits before exec; no Python preexec callback - # runs between fork and exec in the monitoring supervisor. - if not os.environ.get("SLURM_JOB_ID") or os.environ.get("SLURM_GPUS_ON_NODE", "0") != "0": - raise ValueError("CPU-only Slurm allocation required") - command_args = json.loads(sys.argv[2]) - if not command_args or any(not isinstance(value, str) for value in command_args): - raise ValueError("invalid internal command") - limits() - os.execv(command_args[0], command_args) - else: - main() diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-executor-v1/sources.sha256 b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-executor-v1/sources.sha256 deleted file mode 100644 index 37b6c890f..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-executor-v1/sources.sha256 +++ /dev/null @@ -1,11 +0,0 @@ -997cbfed8398e8ad62d1beb4f6b4bc6e61a0f4572b9156cfd0c6f3fb52b03233 runtime.py -bfeeb6ea794c0ebf24e72a0127458ff74951253471daee080b3a3b1d82a062ba bounds_driver.py -236920f27573447b9f3833afb72739033c965e22a9e406b09f28da815529ca64 synthetic_probe.py -bcd6a4a4d69f163075b89ae2e55c8fd475a9cc78ac64d39eadda1a54b4557dec contract.py -5a635ae5773d08a3cbcda07551450b38d3fd6af64f8ca251cc61a7a94c388342 inner.py -15613932aec108c27b2de2390a965aaba4074535f0e02295f4e7846e405f282c executor.py -166dc7c82c9b0847144a56f46a30f97e93f65967a017e79cb9f8b916219364cc fixtures.py -cc4f31657a89f0ca1a3bacd86e9060f2fbf633557ee05b6695f6ccba91d6e92f exercise.py -94b32dfc8bc80d4f46f211fc6d0d36065d6043a47bff04a4856d3eb942e7cf1e test_contract.py -222326a1b27e48fd55988a2c10a7065fa44ad8cd22e71348bff355aa1c15996d policy.json -99626cbef13f865de7fee59746fe228ebae885aac76bcc6029955c1a36cca16f run-executor.sbatch diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-executor-v1/synthetic_probe.py b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-executor-v1/synthetic_probe.py deleted file mode 100644 index 51cb12f67..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-executor-v1/synthetic_probe.py +++ /dev/null @@ -1,529 +0,0 @@ -"""Finite handwritten kernel-boundary probes; no model programs or task data.""" - -import ctypes -import errno -import json -import mmap -import os -import platform -import resource -import select -import signal -import subprocess -import sys -import threading -import time -from pathlib import Path - -TASK_LIMIT = 16 -MEMORY_LIMIT = 2147483648 -START_TOKEN = b"inferrsi-trusted-start-v2\n" -IPC_SYSCALLS = (29, 30, 31, 64, 65, 66, 67, 68, 69, 70, 71, 220, 240) - - -def authorize_start(fd=0, timeout=5): - """Require the supervisor's complete token and EOF before any dispatch.""" - deadline = time.monotonic() + timeout - received = bytearray() - while len(received) <= len(START_TOKEN): - remaining = deadline - time.monotonic() - if remaining <= 0 or not select.select([fd], [], [], remaining)[0]: - raise TimeoutError("startup token deadline exceeded") - part = os.read(fd, len(START_TOKEN) + 1 - len(received)) - if not part: - if bytes(received) != START_TOKEN: - raise ValueError("startup token absent or invalid") - return - received.extend(part) - raise ValueError("startup token too long") - - -def filter_instructions(): - # Linux v6.12 x86 syscall table; reject other architectures and the x32 ABI. - result = [ - (0x20, 0, 0, 4), - (0x15, 1, 0, 0xC000003E), - (0x06, 0, 0, 0x80000000), - (0x20, 0, 0, 0), - (0x35, 0, 1, 0x40000000), - (0x06, 0, 0, 0x80000000), - ] - for number in IPC_SYSCALLS: - result.extend([(0x15, 0, 1, number), (0x06, 0, 0, 0x00050000 | errno.EPERM)]) - return [*result, (0x06, 0, 0, 0x7FFF0000)] - - -def install_ipc_filter(): - if platform.machine() != "x86_64" or sys.maxsize != 2**63 - 1: - raise ValueError("trusted filter requires Linux x86_64 LP64") - - class Instruction(ctypes.Structure): - _fields_ = [ - ("code", ctypes.c_ushort), - ("jt", ctypes.c_ubyte), - ("jf", ctypes.c_ubyte), - ("k", ctypes.c_uint32), - ] - - class Program(ctypes.Structure): - _fields_ = [ - ("length", ctypes.c_ushort), - ("instructions", ctypes.POINTER(Instruction)), - ] - - instructions = filter_instructions() - array = (Instruction * len(instructions))( - *(Instruction(*row) for row in instructions) - ) - program = Program(len(instructions), array) - libc = ctypes.CDLL(None, use_errno=True) - libc.prctl.restype = ctypes.c_int - if ( - libc.prctl(38, 1, 0, 0, 0) != 0 - or libc.prctl(22, 2, ctypes.byref(program), 0, 0) != 0 - ): - raise OSError(ctypes.get_errno(), "fixed seccomp policy could not be installed") - - -def ipc_attempts(): - libc = ctypes.CDLL(None, use_errno=True) - libc.syscall.restype = ctypes.c_long - attempts = [] - # Finite minimal allocations only, even if the expected rejection is absent. - for number, arguments in ( - (29, (0, 4096, 0o600)), - (64, (0, 1, 0o600)), - (68, (0, 0o600)), - ): - ctypes.set_errno(0) - value = libc.syscall( - ctypes.c_long(number), *(ctypes.c_long(v) for v in arguments) - ) - attempts.append( - {"syscall": number, "return": value, "errno": ctypes.get_errno()} - ) - ctypes.set_errno(0) - value = libc.syscall( - ctypes.c_long(240), - ctypes.c_char_p(b"trusted-probe"), - ctypes.c_long(os.O_CREAT | os.O_WRONLY), - ctypes.c_long(0o600), - ctypes.c_void_p(), - ) - attempts.append({"syscall": 240, "return": value, "errno": ctypes.get_errno()}) - return attempts - - -def emit(value): - print(json.dumps(value, sort_keys=True), flush=True) - - -def limits_record(): - return { - name: list(resource.getrlimit(getattr(resource, "RLIMIT_" + name))) - for name in ( - "AS", - "CPU", - "NOFILE", - "FSIZE", - "CORE", - "NPROC", - "MEMLOCK", - "MSGQUEUE", - ) - } - - -def finite_forks(): - children = [] - read_fd, write_fd = os.pipe() - rejected = None - try: - for _ in range(TASK_LIMIT + 1): - try: - child = os.fork() - except OSError as error: - rejected = error.errno - break - if child == 0: - os.close(write_fd) - signal.alarm(8) - os.read(read_fd, 1) - os._exit(0) - children.append(child) - finally: - os.close(write_fd) - os.close(read_fd) - for child in children: - os.waitpid(child, 0) - return { - "created": len(children), - "rejection_errno": rejected, - "attempt_cap": TASK_LIMIT + 1, - } - - -def finite_threads(): - threads = [] - stop = threading.Event() - rejected = None - try: - for _ in range(TASK_LIMIT + 1): - thread = threading.Thread(target=stop.wait, args=(8,)) - try: - thread.start() - except RuntimeError as error: - rejected = str(error) - break - threads.append(thread) - finally: - stop.set() - for thread in threads: - thread.join(timeout=2) - return { - "created": len(threads), - "rejection": rejected, - "all_joined": all(not thread.is_alive() for thread in threads), - "attempt_cap": TASK_LIMIT + 1, - } - - -def bounds(): - before = limits_record() - raises = {} - for name in ( - "NPROC", - "AS", - "CPU", - "NOFILE", - "FSIZE", - "CORE", - "MEMLOCK", - "MSGQUEUE", - ): - code = getattr(resource, "RLIMIT_" + name) - soft, hard = resource.getrlimit(code) - try: - resource.setrlimit(code, (soft, hard + 1)) - except (ValueError, PermissionError) as error: - raises[name] = type(error).__name__ - else: - raises[name] = "unexpected_success" - try: - mapping = mmap.mmap(-1, MEMORY_LIMIT + mmap.PAGESIZE) - except OSError as error: - as_errno = error.errno - else: - mapping.close() - as_errno = None - inherited = subprocess.run( - [sys.executable, "-I", "-B", "/opt/probe.py", "inherit"], - input=START_TOKEN, - capture_output=True, - timeout=5, - check=False, - ) - inherited_payload = json.loads(inherited.stdout) - forks = finite_forks() - threads = finite_threads() - descriptors = [] - fd_errno = None - try: - for _ in range(65): - try: - descriptors.append(os.open("/dev/null", os.O_RDONLY)) - except OSError as error: - fd_errno = error.errno - break - finally: - for fd in descriptors: - os.close(fd) - old_handler = signal.signal(signal.SIGXFSZ, signal.SIG_IGN) - try: - with open("/work/size-check", "wb") as stream: - try: - stream.truncate(1048577) - except OSError as error: - size_errno = error.errno - else: - size_errno = None - finally: - signal.signal(signal.SIGXFSZ, old_handler) - cpu = subprocess.run( - [sys.executable, "-I", "-B", "/opt/probe.py", "cpu"], - input=START_TOKEN, - capture_output=True, - timeout=5, - check=False, - ) - status = dict( - line.split(":", 1) - for line in Path("/proc/self/status").read_text().splitlines() - if ":" in line - ) - expected = { - "AS": [MEMORY_LIMIT] * 2, - "CPU": [30] * 2, - "NOFILE": [64] * 2, - "FSIZE": [1048576] * 2, - "CORE": [0, 0], - "NPROC": [TASK_LIMIT] * 2, - "MEMLOCK": [0, 0], - "MSGQUEUE": [0, 0], - } - ipc = ipc_attempts() - try: - with open("/dev/creation-check", "xb"): - pass - except OSError as error: - dev_errno = error.errno - else: - dev_errno = None - with open("/dev/zero", "rb") as stream: - zero = stream.read(4) - checks = { - "exact_limits": before == expected and limits_record() == expected, - "nonroot": os.getuid() == os.geteuid() == 65534, - "no_effective_capabilities": int(status["CapEff"].strip(), 16) == 0, - "no_new_privileges": status["NoNewPrivs"].strip() == "1", - "seccomp_filter_present": status["Seccomp"].strip() == "2", - "sysv_creation_denied": all( - row["return"] == -1 and row["errno"] == errno.EPERM for row in ipc - ), - "device_mount_readonly": dev_errno == errno.EROFS, - "device_read_available": zero == b"\x00" * 4, - "raise_hard_limit_denied": all( - value != "unexpected_success" for value in raises.values() - ), - "address_space_denied": as_errno == errno.ENOMEM, - "fork_exec_inheritance": inherited.returncode == 0 - and inherited_payload["limits"] == expected, - "fork_exec_filter_inheritance": inherited.returncode == 0 - and all( - row["return"] == -1 and row["errno"] == errno.EPERM - for row in inherited_payload["ipc"] - ), - "bounded_fork_denied": 0 < forks["created"] < TASK_LIMIT - and forks["rejection_errno"] == errno.EAGAIN, - "bounded_thread_denied": 0 < threads["created"] < TASK_LIMIT - and threads["rejection"] is not None - and threads["all_joined"], - "descriptor_limit_denied": fd_errno == errno.EMFILE, - "file_size_denied": size_errno == errno.EFBIG, - "cpu_limit_kills": cpu.returncode == -signal.SIGKILL, - } - emit( - { - "mode": "bounds", - "checks": checks, - "passed": all(checks.values()), - "limits": before, - "hard_limit_raise_results": raises, - "as_errno": as_errno, - "forks": forks, - "threads": threads, - "descriptor_errno": fd_errno, - "opened_descriptors": len(descriptors), - "file_size_errno": size_errno, - "cpu_exit_code": cpu.returncode, - "ipc_attempts": ipc, - "device_creation_errno": dev_errno, - "inherited_exit_code": inherited.returncode, - "generated_code_executed": False, - } - ) - return 0 if all(checks.values()) else 2 - - -def cleanup(mode): - read_fd, write_fd = os.pipe() - direct = os.fork() - if direct == 0: - os.close(read_fd) - os.write(write_fd, f"{os.getpid()}\n".encode()) - os.close(write_fd) - time.sleep(15) - os._exit(0) - intermediate = os.fork() - if intermediate == 0: - os.close(read_fd) - os.setsid() - grandchild = os.fork() - if grandchild == 0: - os.write(write_fd, f"{os.getpid()}\n".encode()) - os.close(write_fd) - time.sleep(15) - os._exit(0) - os._exit(0) - os.close(write_fd) - with os.fdopen(read_fd, "r") as stream: - live = sorted(int(line) for line in stream) - os.waitpid(intermediate, 0) - emit( - { - "mode": mode, - "ready": True, - "probe_pid": os.getpid(), - "live_descendant_pids": live, - "direct_pid": direct, - "namespace_pid": os.readlink("/proc/self/ns/pid"), - "generated_code_executed": False, - } - ) - if mode == "normal": - # Allow the supervisor to open pidfds for both deliberately live children. - time.sleep(2) - return 0 - time.sleep(15) - return 3 - - -def synthetic_evaluator(): - import base64 - import contextlib - import hashlib - import importlib - import io - - sys.path.insert(0, "/opt/site-packages") - import psutil - - def digest(raw): - return hashlib.sha256(raw).hexdigest() - - os.environ["EVALPLUS_MAX_MEMORY_BYTES"] = str(MEMORY_LIMIT) - os.environ.pop("EVALPLUS_TIMEOUT_PER_TASK", None) - os.environ["XDG_CACHE_HOME"] = "/work/cache" - os.environ["HUMANEVAL_OVERRIDE_PATH"] = "/work/synthetic-dataset.jsonl" - work = Path("/work") - samples = work / "synthetic-samples.jsonl" - dataset = work / "synthetic-dataset.jsonl" - output = work / "output/synthetic-results.json" - assert not samples.exists() and not dataset.exists() and not output.exists() - assert not (work / "cache").exists() - solutions = { - "Synthetic/correct": "def f(x):\n return x + 1\n", - "Synthetic/plus_wrong": "def f(x):\n return 1\n", - "Synthetic/exception": "def f(x):\n raise ArithmeticError('synthetic')\n", - "Synthetic/input_timeout": "import time\ndef f(x):\n time.sleep(5)\n return x + 1\n", - } - rows = [ - { - "task_id": task_id, - "prompt": "def f(x):\n", - "canonical_solution": " return x + 1\n", - "contract": "", - "entry_point": "f", - "base_input": [[0]], - "plus_input": [[1]], - "atol": 0, - } - for task_id in solutions - ] - dataset.write_text("".join(json.dumps(row, sort_keys=True) + "\n" for row in rows)) - samples.write_text( - "".join( - json.dumps({"task_id": key, "solution": value}, sort_keys=True) + "\n" - for key, value in solutions.items() - ) - ) - parent = psutil.Process() - captured_out, captured_err = io.StringIO(), io.StringIO() - started = time.monotonic() - with ( - contextlib.redirect_stdout(captured_out), - contextlib.redirect_stderr(captured_err), - ): - evaluator = importlib.import_module("evalplus.evaluate") - evaluator.evaluate( - dataset="humaneval", - samples=str(samples), - output_file=str(output), - parallel=1, - base_only=False, - mini=False, - noextreme=False, - test_details=True, - ) - result = json.loads(output.read_bytes()) - statuses = { - task_id: [[row["base_status"], row["plus_status"]] for row in values] - for task_id, values in result["eval"].items() - } - # The upstream per-input exception handler reports time-limit exceptions as - # fail. This is distinct from its outer child-process timeout classification. - expected = { - "Synthetic/correct": [["pass", "pass"]], - "Synthetic/plus_wrong": [["pass", "fail"]], - "Synthetic/exception": [["fail", "fail"]], - "Synthetic/input_timeout": [["fail", "fail"]], - } - checks = { - "four_exact_status_pairs": statuses == expected, - "dataset_hash": result["hash"] == hashlib.md5(dataset.read_bytes()).hexdigest(), - "bounded_output": output.stat().st_size <= 1048576, - "timeout_override_absent": "EVALPLUS_TIMEOUT_PER_TASK" not in os.environ, - } - payload = { - "checks": checks, - "passed": all(checks.values()), - "statuses": statuses, - "elapsed_seconds": time.monotonic() - started, - "dataset_sha256": digest(dataset.read_bytes()), - "samples_sha256": digest(samples.read_bytes()), - "results_sha256": digest(output.read_bytes()), - "results": result, - "results_raw_base64": base64.b64encode(output.read_bytes()).decode("ascii"), - "evaluator_module_sha256": digest(Path(evaluator.__file__).read_bytes()), - "evaluator_stdout": captured_out.getvalue(), - "evaluator_stderr": captured_err.getvalue(), - "live_children_after_evaluate": len(parent.children(recursive=True)), - "generated_code_executed": False, - "real_benchmark_data_mounted": False, - "full_evaluator_executed": True, - "qualified_generated_code_executor": False, - } - print(json.dumps(payload, sort_keys=True)) - return 0 if payload["passed"] else 2 - - -def main(): - try: - authorize_start() - except (ValueError, OSError): - os.write(2, b"PAYLOAD_START_REJECTED\n") - return 23 - # The authorization pipe is never available to evaluator code or children. - devnull = os.open("/dev/null", os.O_RDONLY) - os.dup2(devnull, 0) - if devnull != 0: - os.close(devnull) - os.write(2, b"PAYLOAD_START_AUTHORIZED\n") - mode = sys.argv[1] - if mode == "gate_check": - emit({"passed": True, "mode": "gate_check"}) - return 0 - if mode == "inherit": - emit({"limits": limits_record(), "ipc": ipc_attempts()}) - return 0 - if mode == "cpu": - resource.setrlimit(resource.RLIMIT_CPU, (1, 1)) - signal.alarm(4) - while True: - pass - resource.setrlimit(resource.RLIMIT_NPROC, (TASK_LIMIT, TASK_LIMIT)) - resource.setrlimit(resource.RLIMIT_MEMLOCK, (0, 0)) - resource.setrlimit(resource.RLIMIT_MSGQUEUE, (0, 0)) - install_ipc_filter() - signal.alarm(40) - if mode == "synthetic": - return synthetic_evaluator() - if mode == "bounds": - return bounds() - if mode in ("normal", "timeout", "launcher_kill"): - return cleanup(mode) - raise ValueError("unknown trusted probe mode") - - -if __name__ == "__main__": - raise SystemExit(main()) diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-executor-v1/test_contract.initial-broken-pipe-fixture.py b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-executor-v1/test_contract.initial-broken-pipe-fixture.py deleted file mode 100644 index fba273ae5..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-executor-v1/test_contract.initial-broken-pipe-fixture.py +++ /dev/null @@ -1,319 +0,0 @@ -"""Host-only acceptance/rejection controls. No model/evaluator code is executed.""" - -import copy -import hashlib -import io -import os -import subprocess -import sys -import tempfile -import types -import unittest -from pathlib import Path -from unittest import mock - -sys.path.insert(0, str(Path(__file__).resolve().parent)) -import contract -import fixtures - -# Only this host-only test stubs an unused legacy helper dependency if absent. -try: - import psutil # noqa: F401 -except ImportError: - sys.modules["psutil"] = types.ModuleType("psutil") -import executor - - -class Contracts(unittest.TestCase): - def setUp(self): - self.temp = tempfile.TemporaryDirectory() - self.addCleanup(self.temp.cleanup) - self.root = Path(self.temp.name) - self.inputs = self.root / "inputs" - self.identity = fixtures.create(self.inputs) - self.maximum = 67108864 - _, self.export, self.samples = self.validate() - self.value = { - "hash": self.export["evalplus_dataset_md5"], - "eval": { - identity: [ - { - "task_id": identity, - "solution": row["solution"], - "base_status": "pass", - "plus_status": "pass", - } - ] - for identity, row in self.samples.items() - }, - } - - def validate(self, **changes): - return contract.inputs(self.inputs, **{**self.identity, **changes}, maximum=self.maximum) - - def reject_result(self, value): - with self.assertRaises(ValueError): - contract.results(contract.canonical(value), self.export, self.samples, self.maximum) - - def rebind(self): - samples = (self.inputs / contract.INPUT_NAMES[0]).read_bytes() - dataset = (self.inputs / contract.INPUT_NAMES[1]).read_bytes() - raw = contract.canonical( - { - **self.export, - "samples_sha256": contract.sha(samples), - "dataset_sha256": contract.sha(dataset), - "evalplus_dataset_md5": hashlib.md5(dataset).hexdigest(), - } - ) - (self.inputs / "human-export.json").write_bytes(raw) - self.identity["export_sha256"] = contract.sha(raw) - - def test_valid_and_large_result(self): - raw = contract.canonical(self.value) - self.assertGreater(len(raw), 1048576) - self.assertEqual( - set(contract.results(raw, self.export, self.samples, self.maximum)), set(self.samples) - ) - - def test_wrong_export_hash(self): - with self.assertRaises(ValueError): - self.validate(export_sha256="0" * 64) - - def test_real_v5_export_marker_and_wrong_types(self): - self.assertEqual( - set(self.export), - { - "samples_sha256", - "dataset_sha256", - "evalplus_dataset_md5", - "task_ids", - "generated_code_executed", - }, - ) - self.assertIs(self.export["generated_code_executed"], False) - for marker in (True, 0, "false", None): - value = {**self.export, "generated_code_executed": marker} - raw = contract.canonical(value) - (self.inputs / "human-export.json").write_bytes(raw) - with self.assertRaises(ValueError): - self.validate(export_sha256=contract.sha(raw)) - value.pop("generated_code_executed") - raw = contract.canonical(value) - (self.inputs / "human-export.json").write_bytes(raw) - with self.assertRaises(ValueError): - self.validate(export_sha256=contract.sha(raw)) - - def test_wrong_cohort_hash(self): - with self.assertRaises(ValueError): - self.validate(cohort_sha256="0" * 64) - - def test_tampered_samples(self): - path = self.inputs / contract.INPUT_NAMES[0] - path.write_bytes(path.read_bytes().replace(b"ArithmeticError", b"ValueError")) - with self.assertRaises(ValueError): - self.validate() - - def test_duplicate_samples_even_with_rebound_digest(self): - path = self.inputs / contract.INPUT_NAMES[0] - path.write_bytes(path.read_bytes() + path.read_bytes().splitlines()[0] + b"\n") - self.rebind() - with self.assertRaises(ValueError): - self.validate() - - def test_missing_sample_even_with_rebound_digest(self): - path = self.inputs / contract.INPUT_NAMES[0] - path.write_bytes(b"\n".join(path.read_bytes().splitlines()[1:]) + b"\n") - self.rebind() - with self.assertRaises(ValueError): - self.validate() - - def test_duplicate_dataset_even_with_rebound_digest(self): - path = self.inputs / contract.INPUT_NAMES[1] - path.write_bytes(path.read_bytes() + path.read_bytes().splitlines()[0] + b"\n") - self.rebind() - with self.assertRaises(ValueError): - self.validate() - - def test_input_byte_cap(self): - self.maximum = 1048576 - with self.assertRaises(ValueError): - self.validate() - - def test_duplicate_json_and_nonfinite(self): - for raw in (b'{"x":1,"x":2}', b'{"x":NaN}', b'{"x":Infinity}'): - with self.assertRaises(ValueError): - contract.decode(raw) - - def test_result_truncated_or_oversized(self): - raw = contract.canonical(self.value) - for data, cap in ((raw[:-1], self.maximum), (raw, 1048576), (b"", self.maximum)): - with self.assertRaises(ValueError): - contract.results(data, self.export, self.samples, cap) - - def test_result_wrong_task_and_duplicate(self): - value = copy.deepcopy(self.value) - value["eval"].pop("Synthetic/correct") - self.reject_result(value) - value = copy.deepcopy(self.value) - value["eval"]["Synthetic/correct"] *= 2 - self.reject_result(value) - - def test_result_wrong_solution_status_dataset(self): - for key, replacement in ( - ("solution", "different"), - ("plus_status", "unknown"), - ("task_id", "wrong"), - ): - value = copy.deepcopy(self.value) - value["eval"]["Synthetic/correct"][0][key] = replacement - self.reject_result(value) - self.reject_result({**self.value, "hash": "0" * 32}) - - def test_policy_digest_and_unqualified_scope(self): - path = executor.SOURCE / "policy.json" - raw = path.read_bytes() - policy = contract.policy(path, contract.sha(raw)) - self.assertFalse(policy["qualified"]) - self.assertEqual(policy["as_bytes"], 4294967296) - with self.assertRaises(ValueError): - contract.policy(path, "0" * 64) - changed = self.root / "policy.json" - value = {**policy, "qualified": True} - changed.write_bytes(contract.canonical(value)) - with self.assertRaises(ValueError): - contract.policy(changed, contract.sha(changed.read_bytes())) - - def test_actual_command_and_new_limits(self): - path = executor.SOURCE / "policy.json" - digest = contract.sha(path.read_bytes()) - policy = contract.policy(path, digest) - locators = { - "bwrap": "/tool/bwrap", - "python_root": "/runtime/python", - "site_root": "/runtime/site", - "python_relative": "bin/python3.12", - "system_libraries": {}, - } - args = executor.command(locators, self.inputs, policy, digest, **self.identity) - for target, size in (("/work", "268435456"), ("/dev/shm", "33554432")): - self.assertEqual(args[args.index(target) - 2], size) - self.assertIn("/opt/executor/inner.py", args) - self.assertIn("/input", args) - with ( - mock.patch.object(executor.os, "sched_getaffinity", return_value={3, 4}, create=True), - mock.patch.object(executor.os, "sched_setaffinity", create=True), - mock.patch.object(executor.resource, "setrlimit") as setlimit, - ): - executor.limits(policy) - self.assertIn( - mock.call(executor.resource.RLIMIT_AS, (4294967296, 4294967296)), - setlimit.call_args_list, - ) - self.assertIn(mock.call(executor.resource.RLIMIT_CPU, (120, 120)), setlimit.call_args_list) - - def test_existing_output_rejected_before_capability(self): - output = self.root / "old-output" - output.mkdir() - args = types.SimpleNamespace( - inputs=self.inputs, - output=output, - bundle=self.root, - policy_sha256=contract.sha((executor.SOURCE / "policy.json").read_bytes()), - **self.identity, - ) - with mock.patch.object(executor.supervisor, "capability") as capability: - with self.assertRaises(FileExistsError): - executor.evaluate_export(args) - capability.assert_not_called() - - def test_stale_supervisor_hash_and_failed_execution_never_score(self): - raw = contract.canonical(self.value) - for name, failed, digest in ( - ("stale", False, "0" * 64), - ("exit", True, contract.sha(raw)), - ("matching", False, contract.sha(raw)), - ("unstable", False, contract.sha(raw)), - ): - output = self.root / name - args = types.SimpleNamespace( - inputs=self.inputs, - output=output, - bundle=self.root, - policy_sha256=contract.sha((executor.SOURCE / "policy.json").read_bytes()), - **self.identity, - ) - inventory = { - "rootfs_digest": "sha256:" + "1" * 64, - "executor": {"binary_sha256": "2" * 64}, - } - - def mock_execute(*unused): - (output / "execution").mkdir() - (output / "execution/stdout.json").write_bytes(raw) - return { - "passed": not failed, - "cleanup_complete": True, - "outputs_stable": name != "unstable", - "stdout_sha256": digest, - "final_launcher_exit_code": 1 if failed else 0, - "timed_out": False, - } - - with ( - mock.patch.object(executor.supervisor, "capability", return_value=0), - mock.patch.object(executor, "inventory", return_value=(inventory, {})), - mock.patch.object(executor, "command", return_value=[]), - mock.patch.object(executor, "execute", side_effect=mock_execute), - ): - status = executor.evaluate_export(args) - observed = contract.decode((output / "execution-record.json").read_bytes()) - self.assertEqual(status, 0 if name == "matching" else 2) - self.assertEqual(observed["execution_completed"], name == "matching") - self.assertEqual(observed["results_sha256"], digest if name == "matching" else None) - self.assertIsNone(observed["protocol"]) - self.assertFalse(observed["qualified"]) - - def test_binding_failure_and_timeout_always_finalize(self): - for name in ("binding", "timeout"): - process = mock.Mock(pid=123, stdin=io.BytesIO(), returncode=None) - process.wait.side_effect = subprocess.TimeoutExpired("trusted-fake", 1) - policy = {"wall_seconds": 1} - captured = [] - - def bind(*unused): - if name == "binding": - raise ValueError("synthetic binding failure") - return 99999, {"pid": 124} - - def finalize(report, proc, init, children, pipes, output, status, rootfs): - captured.append({"error": report.get("error"), "init": init}) - report.update(passed=False, cleanup_complete=False, outputs_stable=False) - proc.stdin.close() - for fd in pipes: - if fd >= 0: - os.close(fd) - - with ( - mock.patch.object(executor.subprocess, "Popen", return_value=process), - mock.patch.object( - executor.supervisor, - "read_line", - side_effect=lambda fd, deadline, raw: raw.extend(b"{}\n"), - ), - mock.patch.object(executor.supervisor, "bind_init", side_effect=bind), - mock.patch.object(executor.supervisor, "finalize_case", side_effect=finalize), - ): - report = executor.execute(["fixed-fake-bwrap"], {}, self.root / name, policy) - self.assertEqual(len(captured), 1) - self.assertEqual( - captured[0]["error"]["type"], - "ValueError" if name == "binding" else "TimeoutExpired", - ) - self.assertIs(report["timed_out"], name == "timeout") - self.assertFalse(report["passed"]) - self.assertFalse(report["outputs_stable"]) - - -if __name__ == "__main__": - unittest.main() diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-executor-v1/test_contract.py b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-executor-v1/test_contract.py deleted file mode 100644 index e47d08b6a..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-executor-v1/test_contract.py +++ /dev/null @@ -1,320 +0,0 @@ -"""Host-only acceptance/rejection controls. No model/evaluator code is executed.""" - -import copy -import hashlib -import io -import os -import subprocess -import sys -import tempfile -import types -import unittest -from pathlib import Path -from unittest import mock - -sys.path.insert(0, str(Path(__file__).resolve().parent)) -import contract -import fixtures - -# Only this host-only test stubs an unused legacy helper dependency if absent. -try: - import psutil # noqa: F401 -except ImportError: - sys.modules["psutil"] = types.ModuleType("psutil") -import executor - - -class Contracts(unittest.TestCase): - def setUp(self): - self.temp = tempfile.TemporaryDirectory() - self.addCleanup(self.temp.cleanup) - self.root = Path(self.temp.name) - self.inputs = self.root / "inputs" - self.identity = fixtures.create(self.inputs) - self.maximum = 67108864 - _, self.export, self.samples = self.validate() - self.value = { - "hash": self.export["evalplus_dataset_md5"], - "eval": { - identity: [ - { - "task_id": identity, - "solution": row["solution"], - "base_status": "pass", - "plus_status": "pass", - } - ] - for identity, row in self.samples.items() - }, - } - - def validate(self, **changes): - return contract.inputs(self.inputs, **{**self.identity, **changes}, maximum=self.maximum) - - def reject_result(self, value): - with self.assertRaises(ValueError): - contract.results(contract.canonical(value), self.export, self.samples, self.maximum) - - def rebind(self): - samples = (self.inputs / contract.INPUT_NAMES[0]).read_bytes() - dataset = (self.inputs / contract.INPUT_NAMES[1]).read_bytes() - raw = contract.canonical( - { - **self.export, - "samples_sha256": contract.sha(samples), - "dataset_sha256": contract.sha(dataset), - "evalplus_dataset_md5": hashlib.md5(dataset).hexdigest(), - } - ) - (self.inputs / "human-export.json").write_bytes(raw) - self.identity["export_sha256"] = contract.sha(raw) - - def test_valid_and_large_result(self): - raw = contract.canonical(self.value) - self.assertGreater(len(raw), 1048576) - self.assertEqual( - set(contract.results(raw, self.export, self.samples, self.maximum)), set(self.samples) - ) - - def test_wrong_export_hash(self): - with self.assertRaises(ValueError): - self.validate(export_sha256="0" * 64) - - def test_real_v5_export_marker_and_wrong_types(self): - self.assertEqual( - set(self.export), - { - "samples_sha256", - "dataset_sha256", - "evalplus_dataset_md5", - "task_ids", - "generated_code_executed", - }, - ) - self.assertIs(self.export["generated_code_executed"], False) - for marker in (True, 0, "false", None): - value = {**self.export, "generated_code_executed": marker} - raw = contract.canonical(value) - (self.inputs / "human-export.json").write_bytes(raw) - with self.assertRaises(ValueError): - self.validate(export_sha256=contract.sha(raw)) - value.pop("generated_code_executed") - raw = contract.canonical(value) - (self.inputs / "human-export.json").write_bytes(raw) - with self.assertRaises(ValueError): - self.validate(export_sha256=contract.sha(raw)) - - def test_wrong_cohort_hash(self): - with self.assertRaises(ValueError): - self.validate(cohort_sha256="0" * 64) - - def test_tampered_samples(self): - path = self.inputs / contract.INPUT_NAMES[0] - path.write_bytes(path.read_bytes().replace(b"ArithmeticError", b"ValueError")) - with self.assertRaises(ValueError): - self.validate() - - def test_duplicate_samples_even_with_rebound_digest(self): - path = self.inputs / contract.INPUT_NAMES[0] - path.write_bytes(path.read_bytes() + path.read_bytes().splitlines()[0] + b"\n") - self.rebind() - with self.assertRaises(ValueError): - self.validate() - - def test_missing_sample_even_with_rebound_digest(self): - path = self.inputs / contract.INPUT_NAMES[0] - path.write_bytes(b"\n".join(path.read_bytes().splitlines()[1:]) + b"\n") - self.rebind() - with self.assertRaises(ValueError): - self.validate() - - def test_duplicate_dataset_even_with_rebound_digest(self): - path = self.inputs / contract.INPUT_NAMES[1] - path.write_bytes(path.read_bytes() + path.read_bytes().splitlines()[0] + b"\n") - self.rebind() - with self.assertRaises(ValueError): - self.validate() - - def test_input_byte_cap(self): - self.maximum = 1048576 - with self.assertRaises(ValueError): - self.validate() - - def test_duplicate_json_and_nonfinite(self): - for raw in (b'{"x":1,"x":2}', b'{"x":NaN}', b'{"x":Infinity}'): - with self.assertRaises(ValueError): - contract.decode(raw) - - def test_result_truncated_or_oversized(self): - raw = contract.canonical(self.value) - for data, cap in ((raw[:-1], self.maximum), (raw, 1048576), (b"", self.maximum)): - with self.assertRaises(ValueError): - contract.results(data, self.export, self.samples, cap) - - def test_result_wrong_task_and_duplicate(self): - value = copy.deepcopy(self.value) - value["eval"].pop("Synthetic/correct") - self.reject_result(value) - value = copy.deepcopy(self.value) - value["eval"]["Synthetic/correct"] *= 2 - self.reject_result(value) - - def test_result_wrong_solution_status_dataset(self): - for key, replacement in ( - ("solution", "different"), - ("plus_status", "unknown"), - ("task_id", "wrong"), - ): - value = copy.deepcopy(self.value) - value["eval"]["Synthetic/correct"][0][key] = replacement - self.reject_result(value) - self.reject_result({**self.value, "hash": "0" * 32}) - - def test_policy_digest_and_unqualified_scope(self): - path = executor.SOURCE / "policy.json" - raw = path.read_bytes() - policy = contract.policy(path, contract.sha(raw)) - self.assertFalse(policy["qualified"]) - self.assertEqual(policy["as_bytes"], 4294967296) - with self.assertRaises(ValueError): - contract.policy(path, "0" * 64) - changed = self.root / "policy.json" - value = {**policy, "qualified": True} - changed.write_bytes(contract.canonical(value)) - with self.assertRaises(ValueError): - contract.policy(changed, contract.sha(changed.read_bytes())) - - def test_actual_command_and_new_limits(self): - path = executor.SOURCE / "policy.json" - digest = contract.sha(path.read_bytes()) - policy = contract.policy(path, digest) - locators = { - "bwrap": "/tool/bwrap", - "python_root": "/runtime/python", - "site_root": "/runtime/site", - "python_relative": "bin/python3.12", - "system_libraries": {}, - } - args = executor.command(locators, self.inputs, policy, digest, **self.identity) - for target, size in (("/work", "268435456"), ("/dev/shm", "33554432")): - self.assertEqual(args[args.index(target) - 2], size) - self.assertIn("/opt/executor/inner.py", args) - self.assertIn("/input", args) - with ( - mock.patch.object(executor.os, "sched_getaffinity", return_value={3, 4}, create=True), - mock.patch.object(executor.os, "sched_setaffinity", create=True), - mock.patch.object(executor.resource, "setrlimit") as setlimit, - ): - executor.limits(policy) - self.assertIn( - mock.call(executor.resource.RLIMIT_AS, (4294967296, 4294967296)), - setlimit.call_args_list, - ) - self.assertIn(mock.call(executor.resource.RLIMIT_CPU, (120, 120)), setlimit.call_args_list) - - def test_existing_output_rejected_before_capability(self): - output = self.root / "old-output" - output.mkdir() - args = types.SimpleNamespace( - inputs=self.inputs, - output=output, - bundle=self.root, - policy_sha256=contract.sha((executor.SOURCE / "policy.json").read_bytes()), - **self.identity, - ) - with mock.patch.object(executor.supervisor, "capability") as capability: - with self.assertRaises(FileExistsError): - executor.evaluate_export(args) - capability.assert_not_called() - - def test_stale_supervisor_hash_and_failed_execution_never_score(self): - raw = contract.canonical(self.value) - for name, failed, digest in ( - ("stale", False, "0" * 64), - ("exit", True, contract.sha(raw)), - ("matching", False, contract.sha(raw)), - ("unstable", False, contract.sha(raw)), - ): - output = self.root / name - args = types.SimpleNamespace( - inputs=self.inputs, - output=output, - bundle=self.root, - policy_sha256=contract.sha((executor.SOURCE / "policy.json").read_bytes()), - **self.identity, - ) - inventory = { - "rootfs_digest": "sha256:" + "1" * 64, - "executor": {"binary_sha256": "2" * 64}, - } - - def mock_execute(*unused): - (output / "execution").mkdir() - (output / "execution/stdout.json").write_bytes(raw) - return { - "passed": not failed, - "cleanup_complete": True, - "outputs_stable": name != "unstable", - "stdout_sha256": digest, - "final_launcher_exit_code": 1 if failed else 0, - "timed_out": False, - } - - with ( - mock.patch.object(executor.supervisor, "capability", return_value=0), - mock.patch.object(executor, "inventory", return_value=(inventory, {})), - mock.patch.object(executor, "command", return_value=[]), - mock.patch.object(executor, "execute", side_effect=mock_execute), - ): - status = executor.evaluate_export(args) - observed = contract.decode((output / "execution-record.json").read_bytes()) - self.assertEqual(status, 0 if name == "matching" else 2) - self.assertEqual(observed["execution_completed"], name == "matching") - self.assertEqual(observed["results_sha256"], digest if name == "matching" else None) - self.assertIsNone(observed["protocol"]) - self.assertFalse(observed["qualified"]) - - def test_binding_failure_and_timeout_always_finalize(self): - for name in ("binding", "timeout"): - process = mock.Mock(pid=123, stdin=io.BytesIO(), returncode=None) - process.wait.side_effect = subprocess.TimeoutExpired("trusted-fake", 1) - policy = {"wall_seconds": 1} - captured = [] - - def bind(*unused): - if name == "binding": - raise ValueError("synthetic binding failure") - return 99999, {"pid": 124} - - def finalize(report, proc, init, children, pipes, output, status, rootfs): - captured.append({"error": report.get("error"), "init": init}) - report.update(passed=False, cleanup_complete=False, outputs_stable=False) - proc.stdin.close() - for fd in pipes: - if fd >= 0: - os.close(fd) - - with ( - mock.patch.object(executor.subprocess, "Popen", return_value=process), - mock.patch.object(executor.os, "write", return_value=1), - mock.patch.object( - executor.supervisor, - "read_line", - side_effect=lambda fd, deadline, raw: raw.extend(b"{}\n"), - ), - mock.patch.object(executor.supervisor, "bind_init", side_effect=bind), - mock.patch.object(executor.supervisor, "finalize_case", side_effect=finalize), - ): - report = executor.execute(["fixed-fake-bwrap"], {}, self.root / name, policy) - self.assertEqual(len(captured), 1) - self.assertEqual( - captured[0]["error"]["type"], - "ValueError" if name == "binding" else "TimeoutExpired", - ) - self.assertIs(report["timed_out"], name == "timeout") - self.assertFalse(report["passed"]) - self.assertFalse(report["outputs_stable"]) - - -if __name__ == "__main__": - unittest.main() diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-executor-v1/test_contract.review-before-export-fix.py b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-executor-v1/test_contract.review-before-export-fix.py deleted file mode 100644 index 1f899fbbc..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-executor-v1/test_contract.review-before-export-fix.py +++ /dev/null @@ -1,196 +0,0 @@ -"""Host-only acceptance/rejection controls. No model/evaluator code is executed.""" - -import copy -import hashlib -import sys -import tempfile -import types -import unittest -from pathlib import Path -from unittest import mock - -sys.path.insert(0, str(Path(__file__).resolve().parent)) -import contract -import fixtures - -# Only this host-only test stubs an unused legacy helper dependency if absent. -try: - import psutil # noqa: F401 -except ImportError: - sys.modules["psutil"] = types.ModuleType("psutil") -import executor - - -class Contracts(unittest.TestCase): - def setUp(self): - self.temp = tempfile.TemporaryDirectory() - self.addCleanup(self.temp.cleanup) - self.root = Path(self.temp.name) - self.inputs = self.root / "inputs" - self.identity = fixtures.create(self.inputs) - self.maximum = 67108864 - _, self.export, self.samples = self.validate() - self.value = { - "hash": self.export["evalplus_dataset_md5"], - "eval": { - identity: [ - {"solution": row["solution"], "base_status": "pass", "plus_status": "pass"} - ] - for identity, row in self.samples.items() - }, - } - - def validate(self, **changes): - return contract.inputs(self.inputs, **{**self.identity, **changes}, maximum=self.maximum) - - def reject_result(self, value): - with self.assertRaises(ValueError): - contract.results(contract.canonical(value), self.export, self.samples, self.maximum) - - def rebind(self): - samples = (self.inputs / contract.INPUT_NAMES[0]).read_bytes() - dataset = (self.inputs / contract.INPUT_NAMES[1]).read_bytes() - raw = contract.canonical( - { - **self.export, - "samples_sha256": contract.sha(samples), - "dataset_sha256": contract.sha(dataset), - "evalplus_dataset_md5": hashlib.md5(dataset).hexdigest(), - } - ) - (self.inputs / "human-export.json").write_bytes(raw) - self.identity["export_sha256"] = contract.sha(raw) - - def test_valid_and_large_result(self): - raw = contract.canonical(self.value) - self.assertGreater(len(raw), 1048576) - self.assertEqual( - set(contract.results(raw, self.export, self.samples, self.maximum)), set(self.samples) - ) - - def test_wrong_export_hash(self): - with self.assertRaises(ValueError): - self.validate(export_sha256="0" * 64) - - def test_wrong_cohort_hash(self): - with self.assertRaises(ValueError): - self.validate(cohort_sha256="0" * 64) - - def test_tampered_samples(self): - path = self.inputs / contract.INPUT_NAMES[0] - path.write_bytes(path.read_bytes().replace(b"ArithmeticError", b"ValueError")) - with self.assertRaises(ValueError): - self.validate() - - def test_duplicate_samples_even_with_rebound_digest(self): - path = self.inputs / contract.INPUT_NAMES[0] - path.write_bytes(path.read_bytes() + path.read_bytes().splitlines()[0] + b"\n") - self.rebind() - with self.assertRaises(ValueError): - self.validate() - - def test_missing_sample_even_with_rebound_digest(self): - path = self.inputs / contract.INPUT_NAMES[0] - path.write_bytes(b"\n".join(path.read_bytes().splitlines()[1:]) + b"\n") - self.rebind() - with self.assertRaises(ValueError): - self.validate() - - def test_duplicate_dataset_even_with_rebound_digest(self): - path = self.inputs / contract.INPUT_NAMES[1] - path.write_bytes(path.read_bytes() + path.read_bytes().splitlines()[0] + b"\n") - self.rebind() - with self.assertRaises(ValueError): - self.validate() - - def test_input_byte_cap(self): - self.maximum = 1048576 - with self.assertRaises(ValueError): - self.validate() - - def test_duplicate_json_and_nonfinite(self): - for raw in (b'{"x":1,"x":2}', b'{"x":NaN}', b'{"x":Infinity}'): - with self.assertRaises(ValueError): - contract.decode(raw) - - def test_result_truncated_or_oversized(self): - raw = contract.canonical(self.value) - for data, cap in ((raw[:-1], self.maximum), (raw, 1048576), (b"", self.maximum)): - with self.assertRaises(ValueError): - contract.results(data, self.export, self.samples, cap) - - def test_result_wrong_task_and_duplicate(self): - value = copy.deepcopy(self.value) - value["eval"].pop("Synthetic/correct") - self.reject_result(value) - value = copy.deepcopy(self.value) - value["eval"]["Synthetic/correct"] *= 2 - self.reject_result(value) - - def test_result_wrong_solution_status_dataset(self): - for key, replacement in (("solution", "different"), ("plus_status", "unknown")): - value = copy.deepcopy(self.value) - value["eval"]["Synthetic/correct"][0][key] = replacement - self.reject_result(value) - self.reject_result({**self.value, "hash": "0" * 32}) - - def test_policy_digest_and_unqualified_scope(self): - path = executor.SOURCE / "policy.json" - raw = path.read_bytes() - policy = contract.policy(path, contract.sha(raw)) - self.assertFalse(policy["qualified"]) - self.assertEqual(policy["as_bytes"], 4294967296) - with self.assertRaises(ValueError): - contract.policy(path, "0" * 64) - changed = self.root / "policy.json" - value = {**policy, "qualified": True} - changed.write_bytes(contract.canonical(value)) - with self.assertRaises(ValueError): - contract.policy(changed, contract.sha(changed.read_bytes())) - - def test_actual_command_and_new_limits(self): - path = executor.SOURCE / "policy.json" - digest = contract.sha(path.read_bytes()) - policy = contract.policy(path, digest) - locators = { - "bwrap": "/tool/bwrap", - "python_root": "/runtime/python", - "site_root": "/runtime/site", - "python_relative": "bin/python3.12", - "system_libraries": {}, - } - args = executor.command(locators, self.inputs, policy, digest, **self.identity) - for target, size in (("/work", "268435456"), ("/dev/shm", "33554432")): - self.assertEqual(args[args.index(target) - 2], size) - self.assertIn("/opt/executor/inner.py", args) - self.assertIn("/input", args) - with ( - mock.patch.object(executor.os, "sched_getaffinity", return_value={3, 4}, create=True), - mock.patch.object(executor.os, "sched_setaffinity", create=True), - mock.patch.object(executor.resource, "setrlimit") as setlimit, - ): - executor.limits(policy) - self.assertIn( - mock.call(executor.resource.RLIMIT_AS, (4294967296, 4294967296)), - setlimit.call_args_list, - ) - self.assertIn(mock.call(executor.resource.RLIMIT_CPU, (120, 120)), setlimit.call_args_list) - - def test_existing_output_rejected_before_capability(self): - output = self.root / "old-output" - output.mkdir() - args = types.SimpleNamespace( - inputs=self.inputs, - output=output, - bundle=self.root, - policy_sha256=contract.sha((executor.SOURCE / "policy.json").read_bytes()), - **self.identity, - ) - with mock.patch.object(executor.supervisor, "capability") as capability: - with self.assertRaises(FileExistsError): - executor.evaluate_export(args) - capability.assert_not_called() - - -if __name__ == "__main__": - unittest.main() diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-executor-v1/test_contract.review-before-result-binding.py b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-executor-v1/test_contract.review-before-result-binding.py deleted file mode 100644 index f084ba4dc..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-executor-v1/test_contract.review-before-result-binding.py +++ /dev/null @@ -1,220 +0,0 @@ -"""Host-only acceptance/rejection controls. No model/evaluator code is executed.""" - -import copy -import hashlib -import sys -import tempfile -import types -import unittest -from pathlib import Path -from unittest import mock - -sys.path.insert(0, str(Path(__file__).resolve().parent)) -import contract -import fixtures - -# Only this host-only test stubs an unused legacy helper dependency if absent. -try: - import psutil # noqa: F401 -except ImportError: - sys.modules["psutil"] = types.ModuleType("psutil") -import executor - - -class Contracts(unittest.TestCase): - def setUp(self): - self.temp = tempfile.TemporaryDirectory() - self.addCleanup(self.temp.cleanup) - self.root = Path(self.temp.name) - self.inputs = self.root / "inputs" - self.identity = fixtures.create(self.inputs) - self.maximum = 67108864 - _, self.export, self.samples = self.validate() - self.value = { - "hash": self.export["evalplus_dataset_md5"], - "eval": { - identity: [ - {"solution": row["solution"], "base_status": "pass", "plus_status": "pass"} - ] - for identity, row in self.samples.items() - }, - } - - def validate(self, **changes): - return contract.inputs(self.inputs, **{**self.identity, **changes}, maximum=self.maximum) - - def reject_result(self, value): - with self.assertRaises(ValueError): - contract.results(contract.canonical(value), self.export, self.samples, self.maximum) - - def rebind(self): - samples = (self.inputs / contract.INPUT_NAMES[0]).read_bytes() - dataset = (self.inputs / contract.INPUT_NAMES[1]).read_bytes() - raw = contract.canonical( - { - **self.export, - "samples_sha256": contract.sha(samples), - "dataset_sha256": contract.sha(dataset), - "evalplus_dataset_md5": hashlib.md5(dataset).hexdigest(), - } - ) - (self.inputs / "human-export.json").write_bytes(raw) - self.identity["export_sha256"] = contract.sha(raw) - - def test_valid_and_large_result(self): - raw = contract.canonical(self.value) - self.assertGreater(len(raw), 1048576) - self.assertEqual( - set(contract.results(raw, self.export, self.samples, self.maximum)), set(self.samples) - ) - - def test_wrong_export_hash(self): - with self.assertRaises(ValueError): - self.validate(export_sha256="0" * 64) - - def test_real_v5_export_marker_and_wrong_types(self): - self.assertEqual( - set(self.export), - { - "samples_sha256", - "dataset_sha256", - "evalplus_dataset_md5", - "task_ids", - "generated_code_executed", - }, - ) - self.assertIs(self.export["generated_code_executed"], False) - for marker in (True, 0, "false", None): - value = {**self.export, "generated_code_executed": marker} - raw = contract.canonical(value) - (self.inputs / "human-export.json").write_bytes(raw) - with self.assertRaises(ValueError): - self.validate(export_sha256=contract.sha(raw)) - value.pop("generated_code_executed") - raw = contract.canonical(value) - (self.inputs / "human-export.json").write_bytes(raw) - with self.assertRaises(ValueError): - self.validate(export_sha256=contract.sha(raw)) - - def test_wrong_cohort_hash(self): - with self.assertRaises(ValueError): - self.validate(cohort_sha256="0" * 64) - - def test_tampered_samples(self): - path = self.inputs / contract.INPUT_NAMES[0] - path.write_bytes(path.read_bytes().replace(b"ArithmeticError", b"ValueError")) - with self.assertRaises(ValueError): - self.validate() - - def test_duplicate_samples_even_with_rebound_digest(self): - path = self.inputs / contract.INPUT_NAMES[0] - path.write_bytes(path.read_bytes() + path.read_bytes().splitlines()[0] + b"\n") - self.rebind() - with self.assertRaises(ValueError): - self.validate() - - def test_missing_sample_even_with_rebound_digest(self): - path = self.inputs / contract.INPUT_NAMES[0] - path.write_bytes(b"\n".join(path.read_bytes().splitlines()[1:]) + b"\n") - self.rebind() - with self.assertRaises(ValueError): - self.validate() - - def test_duplicate_dataset_even_with_rebound_digest(self): - path = self.inputs / contract.INPUT_NAMES[1] - path.write_bytes(path.read_bytes() + path.read_bytes().splitlines()[0] + b"\n") - self.rebind() - with self.assertRaises(ValueError): - self.validate() - - def test_input_byte_cap(self): - self.maximum = 1048576 - with self.assertRaises(ValueError): - self.validate() - - def test_duplicate_json_and_nonfinite(self): - for raw in (b'{"x":1,"x":2}', b'{"x":NaN}', b'{"x":Infinity}'): - with self.assertRaises(ValueError): - contract.decode(raw) - - def test_result_truncated_or_oversized(self): - raw = contract.canonical(self.value) - for data, cap in ((raw[:-1], self.maximum), (raw, 1048576), (b"", self.maximum)): - with self.assertRaises(ValueError): - contract.results(data, self.export, self.samples, cap) - - def test_result_wrong_task_and_duplicate(self): - value = copy.deepcopy(self.value) - value["eval"].pop("Synthetic/correct") - self.reject_result(value) - value = copy.deepcopy(self.value) - value["eval"]["Synthetic/correct"] *= 2 - self.reject_result(value) - - def test_result_wrong_solution_status_dataset(self): - for key, replacement in (("solution", "different"), ("plus_status", "unknown")): - value = copy.deepcopy(self.value) - value["eval"]["Synthetic/correct"][0][key] = replacement - self.reject_result(value) - self.reject_result({**self.value, "hash": "0" * 32}) - - def test_policy_digest_and_unqualified_scope(self): - path = executor.SOURCE / "policy.json" - raw = path.read_bytes() - policy = contract.policy(path, contract.sha(raw)) - self.assertFalse(policy["qualified"]) - self.assertEqual(policy["as_bytes"], 4294967296) - with self.assertRaises(ValueError): - contract.policy(path, "0" * 64) - changed = self.root / "policy.json" - value = {**policy, "qualified": True} - changed.write_bytes(contract.canonical(value)) - with self.assertRaises(ValueError): - contract.policy(changed, contract.sha(changed.read_bytes())) - - def test_actual_command_and_new_limits(self): - path = executor.SOURCE / "policy.json" - digest = contract.sha(path.read_bytes()) - policy = contract.policy(path, digest) - locators = { - "bwrap": "/tool/bwrap", - "python_root": "/runtime/python", - "site_root": "/runtime/site", - "python_relative": "bin/python3.12", - "system_libraries": {}, - } - args = executor.command(locators, self.inputs, policy, digest, **self.identity) - for target, size in (("/work", "268435456"), ("/dev/shm", "33554432")): - self.assertEqual(args[args.index(target) - 2], size) - self.assertIn("/opt/executor/inner.py", args) - self.assertIn("/input", args) - with ( - mock.patch.object(executor.os, "sched_getaffinity", return_value={3, 4}, create=True), - mock.patch.object(executor.os, "sched_setaffinity", create=True), - mock.patch.object(executor.resource, "setrlimit") as setlimit, - ): - executor.limits(policy) - self.assertIn( - mock.call(executor.resource.RLIMIT_AS, (4294967296, 4294967296)), - setlimit.call_args_list, - ) - self.assertIn(mock.call(executor.resource.RLIMIT_CPU, (120, 120)), setlimit.call_args_list) - - def test_existing_output_rejected_before_capability(self): - output = self.root / "old-output" - output.mkdir() - args = types.SimpleNamespace( - inputs=self.inputs, - output=output, - bundle=self.root, - policy_sha256=contract.sha((executor.SOURCE / "policy.json").read_bytes()), - **self.identity, - ) - with mock.patch.object(executor.supervisor, "capability") as capability: - with self.assertRaises(FileExistsError): - executor.evaluate_export(args) - capability.assert_not_called() - - -if __name__ == "__main__": - unittest.main() diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-full164-v1/bounds_driver.py b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-full164-v1/bounds_driver.py deleted file mode 100644 index 66061a0aa..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-full164-v1/bounds_driver.py +++ /dev/null @@ -1,613 +0,0 @@ -"""Supervise only fixed trusted probes and retain actual namespace-init exit evidence.""" - -import argparse -import ctypes -import hashlib -import importlib.util -import json -import os -import platform -import select -import signal -import subprocess -import sys -import time -from pathlib import Path - -import psutil - -RUNTIME_PATH = Path(__file__).with_name("runtime.py") -RUNTIME_SHA256 = "997cbfed8398e8ad62d1beb4f6b4bc6e61a0f4572b9156cfd0c6f3fb52b03233" -if hashlib.sha256(RUNTIME_PATH.read_bytes()).hexdigest() != RUNTIME_SHA256: - raise ValueError("unexpected runtime source bytes") -runtime_spec = importlib.util.spec_from_file_location( - "trusted_bound_runtime", RUNTIME_PATH -) -runtime = importlib.util.module_from_spec(runtime_spec) -runtime_spec.loader.exec_module(runtime) -START_TOKEN = b"inferrsi-trusted-start-v2\n" -AUTHORIZED_MARKER = b"PAYLOAD_START_AUTHORIZED\n" -REJECTED_MARKER = b"PAYLOAD_START_REJECTED\n" -MODES = ( - "gate_valid", - "gate_eof", - "gate_short", - "gate_wrong", - "gate_timeout", - "gate_early_launcher_death", - "bounds", - "synthetic", - "normal", - "timeout", - "launcher_kill", -) - - -def pidfd_syscall(number, *arguments): - if ( - sys.platform != "linux" - or platform.machine() != "x86_64" - or ctypes.sizeof(ctypes.c_void_p) != 8 - or ctypes.sizeof(ctypes.c_long) != 8 - ): - raise ValueError("pidfd bindings require Linux x86_64 LP64") - library = ctypes.CDLL(None, use_errno=True) - library.syscall.restype = ctypes.c_long - ctypes.set_errno(0) - result = library.syscall(ctypes.c_long(number), *arguments) - if result == -1: - error = ctypes.get_errno() - raise OSError(error, os.strerror(error)) - return result - - -def pidfd_open(pid): - if type(pid) is not int or not 0 < pid < 2**31: - raise ValueError("invalid positive PID") - return pidfd_syscall(434, ctypes.c_int(pid), ctypes.c_uint(0)) - - -def pidfd_send_signal(fd, value): - if type(fd) is not int or fd < 0 or type(value) is not int or value < 0: - raise ValueError("invalid pidfd or signal") - result = pidfd_syscall( - 424, ctypes.c_int(fd), ctypes.c_int(value), ctypes.c_void_p(), ctypes.c_uint(0) - ) - if result != 0: - raise ValueError("unexpected pidfd_send_signal result") - - -def capability(output): - report = { - "schema": 1, - "passed": False, - "job_id": os.environ["SLURM_JOB_ID"], - "implementation": "linux-x86_64-lp64-raw-syscalls-434-and-424", - "python_exposes_pidfd_open": hasattr(os, "pidfd_open"), - "python_exposes_pidfd_send_signal": hasattr(signal, "pidfd_send_signal"), - "driver_sha256": runtime.sha(Path(__file__).read_bytes()), - "python_binary_sha256": runtime.sha(Path(sys._base_executable).read_bytes()), - "kernel_release": platform.release(), - "generated_code_executed": False, - } - held = [] - child = None - errors = [] - try: - own_fd = pidfd_open(os.getpid()) - held.append(own_fd) - pidfd_send_signal(own_fd, 0) - report["own_fd_not_exited"] = not exited(own_fd) - child = subprocess.Popen( - [sys.executable, "-IB", "-c", "import time; time.sleep(4)"], - stdin=subprocess.DEVNULL, - stdout=subprocess.DEVNULL, - stderr=subprocess.DEVNULL, - ) - child_fd = pidfd_open(child.pid) - held.append(child_fd) - pidfd_send_signal(child_fd, int(signal.SIGKILL)) - report["child_pidfd_exit_observed"] = exited(child_fd, 5000) - report["child_exit_code"] = child.wait(timeout=5) - report["passed"] = ( - report["own_fd_not_exited"] - and report["child_pidfd_exit_observed"] - and report["child_exit_code"] == -signal.SIGKILL - ) - except BaseException as error: - report["error"] = { - "type": type(error).__name__, - "message": str(error), - "errno": getattr(error, "errno", None), - } - finally: - # The trusted child exits by itself if the API is unavailable; no PID kill fallback. - if child is not None: - try: - report["final_child_exit_code"] = child.wait(timeout=6) - except BaseException as error: - errors.append({"phase": "wait_child", "type": type(error).__name__}) - for fd in held: - try: - os.close(fd) - except OSError as error: - errors.append({"phase": "close_fd", "errno": error.errno}) - report["cleanup_errors"] = errors - if errors: - report["passed"] = False - runtime.write(output, report) - return 0 if report["passed"] else 2 - - -def unique(pairs): - result = {} - for key, value in pairs: - if key in result: - raise ValueError("duplicate JSON key") - result[key] = value - return result - - -def read_line(fd, deadline, raw): - while time.monotonic() < deadline and len(raw) <= 1048576: - ready, _, _ = select.select([fd], [], [], max(0, deadline - time.monotonic())) - if not ready: - break - data = os.read(fd, 1) - if not data: - raise ValueError("premature protocol EOF") - raw.extend(data) - if data == b"\n": - return bytes(raw) - raise ValueError("trusted protocol size or time limit exceeded") - - -def read_ready_file(path, deadline): - while time.monotonic() < deadline: - raw = path.read_bytes() - if len(raw) > 1048576: - raise ValueError("trusted output exceeds size limit") - if raw.endswith(b"\n"): - return raw - time.sleep(0.01) - raise ValueError("trusted readiness deadline exceeded") - - -def exited(fd, milliseconds=0): - poller = select.poll() - poller.register(fd, select.POLLIN) - events = poller.poll(milliseconds) - if any(mask & (select.POLLERR | select.POLLNVAL) for _, mask in events): - raise ValueError("pidfd wait failed") - return any(mask & (select.POLLIN | select.POLLHUP) for _, mask in events) - - -def status(pid): - return dict( - line.split(":", 1) - for line in Path(f"/proc/{pid}/status").read_text().splitlines() - if ":" in line - ) - - -def bind_init(record, launcher_pid): - pid = record.get("child-pid") - if type(pid) is not int or pid <= 1: - raise ValueError("invalid namespace-init PID") - fd = pidfd_open(pid) - try: - fields = status(pid) - if exited(fd) or int(fields["PPid"]) != launcher_pid: - raise ValueError("namespace init identity or parent changed") - namespace_pids = [int(value) for value in fields["NSpid"].split()] - if namespace_pids[0] != pid or namespace_pids[-1] != 1: - raise ValueError("reported child is not namespace PID1") - # The child is blocked before payload execution. Only the PID namespace - # is compared: bwrap can create a further user namespace during setup. - pid_namespace = os.readlink(f"/proc/{pid}/ns/pid") - if pid_namespace != f"pid:[{record.get('pid-namespace')}]": - raise ValueError("reported PID namespace identity differs") - if exited(fd): - raise ValueError("namespace init exited during binding") - return fd, { - "pid": pid, - "namespace_pids": namespace_pids, - "pid_namespace": pid_namespace, - } - except BaseException: - os.close(fd) - raise - - -def bind_children(init_pid, readiness): - wanted = readiness.get("live_descendant_pids") - if not isinstance(wanted, list) or len(wanted) != 2 or len(set(wanted)) != 2: - raise ValueError("expected two registered live trusted children") - if any(type(pid) is not int or pid <= 2 for pid in wanted): - raise ValueError("invalid trusted child PID") - held = [] - records = [] - try: - for child in psutil.Process(init_pid).children(recursive=True): - try: - fields = status(child.pid) - except FileNotFoundError: - continue - namespace_pids = [int(value) for value in fields["NSpid"].split()] - if namespace_pids[-1] not in wanted: - continue - fd = pidfd_open(child.pid) - held.append(fd) - current = status(child.pid) - if exited(fd) or current["NSpid"] != fields["NSpid"]: - raise ValueError("trusted child exited during binding") - records.append({"pid": child.pid, "namespace_pids": namespace_pids}) - if sorted(row["namespace_pids"][-1] for row in records) != sorted(wanted): - raise ValueError("registered trusted children not both observed") - return held, records - except BaseException: - for fd in held: - os.close(fd) - raise - - -def finalize_case( - report, process, init_fd, children, pipes, output, status_raw, rootfs -): - errors = [] - report["forced_cleanup"] = False - - def attempt(name, operation): - try: - return operation() - except BaseException as error: - errors.append( - {"operation": name, "type": type(error).__name__, "message": str(error)} - ) - return None - - # EOF cannot authorize the payload. Close this before releasing setup pipes. - if process is not None and process.stdin is not None: - attempt("close_authorization_pipe", process.stdin.close) - report["final_init_exit_observed"] = False - if init_fd is not None: - if attempt("check_init_exit", lambda: exited(init_fd)) is not True: - try: - pidfd_send_signal(init_fd, int(signal.SIGKILL)) - report["forced_cleanup"] = True - except ProcessLookupError: - # Exit between poll and signal is not a failed cleanup if the - # stable pidfd independently confirms that same process exited. - confirmed = attempt("confirm_signal_race", lambda: exited(init_fd)) - report["signal_race_already_exited"] = confirmed is True - if confirmed is not True: - errors.append( - {"operation": "signal_init", "type": "UnconfirmedExitRace"} - ) - except BaseException as error: - errors.append( - { - "operation": "signal_init", - "type": type(error).__name__, - "message": str(error), - } - ) - report["forced_init_exit_observed"] = attempt( - "wait_init_exit", lambda: exited(init_fd, 5000) - ) - if report["forced_init_exit_observed"] is not True: - errors.append( - {"operation": "wait_init_exit", "type": "ExitNotConfirmed"} - ) - report["final_init_exit_observed"] = ( - attempt("final_init_poll", lambda: exited(init_fd)) is True - ) - attempt("close_init_fd", lambda: os.close(init_fd)) - if process is not None: - if attempt("poll_launcher", process.poll) is None: - report["forced_cleanup"] = True - attempt("kill_launcher", process.kill) - attempt("wait_launcher", lambda: process.wait(timeout=5)) - report["final_launcher_exit_code"] = process.returncode - if process.returncode is None: - errors.append({"operation": "wait_launcher", "type": "ExitNotConfirmed"}) - report["cleanup_registered_child_exits"] = [] - for fd in children: - observed = attempt("wait_child_exit", lambda fd=fd: exited(fd, 1000)) - report["cleanup_registered_child_exits"].append(observed) - if observed is not True: - errors.append({"operation": "wait_child_exit", "type": "ExitNotConfirmed"}) - attempt("close_child_fd", lambda fd=fd: os.close(fd)) - for fd in pipes: - if fd >= 0: - attempt("close_pipe_fd", lambda fd=fd: os.close(fd)) - # Both writers must be gone; launcher exit alone is insufficient, and - # namespace exit alone leaves the launcher able to modify these files. - report["outputs_stable"] = ( - report["final_init_exit_observed"] - and process is not None - and process.returncode is not None - ) - report["cleanup_complete"] = ( - report["outputs_stable"] - and all(value is True for value in report["cleanup_registered_child_exits"]) - and not errors - ) - stdout_raw = attempt("read_stdout", (output / "stdout.json").read_bytes) - stderr_raw = attempt("read_stderr", (output / "stderr.private.log").read_bytes) - attempt( - "write_status", - lambda: (output / "status.private.jsonl").write_bytes(status_raw), - ) - for name, raw in (("stdout", stdout_raw), ("stderr", stderr_raw)): - report[name + "_sha256"] = ( - runtime.sha(raw) if raw is not None and report["outputs_stable"] else None - ) - if not report["outputs_stable"] and raw is not None: - report[name + "_partial_snapshot_sha256"] = runtime.sha(raw) - report["status_sha256"] = runtime.sha(status_raw) - report["rootfs_digest"] = rootfs["rootfs_digest"] - report["cleanup_or_evidence_errors"] = errors - if report["forced_cleanup"] or errors or not report["outputs_stable"]: - report["passed"] = False - # If even the report destination fails, the wrapper still captures the - # nonzero process status and stderr; no success record is fabricated. - runtime.write(output / "report.json", report) - - -def run_case(locators, rootfs, mode, output): - output.mkdir() - sentinel = output / "unmounted-sentinel.private" - sentinel.write_text("trusted synthetic marker") - namespaces = { - name: os.readlink("/proc/self/ns/" + name) - for name in ("user", "net", "pid", "uts", "ipc") - } - args = runtime.command( - locators, Path(__file__).with_name("synthetic_probe.py"), namespaces, sentinel - ) - status_read, status_write = os.pipe() - block_read, block_write = os.pipe() - # Additional mount policy; the independent /dev/shm remains a bounded tmpfs. - args[1:1] = ["--json-status-fd", str(status_write), "--block-fd", str(block_read)] - payload_index = args.index("/opt/python/" + locators["python_relative"]) - args[payload_index:payload_index] = ["--remount-ro", "/dev"] - args.append("gate_check" if mode.startswith("gate_") else mode) - process = None - init_fd = None - children = [] - report = {"mode": mode, "passed": False, "generated_code_executed": False} - stdout_raw = b"" - status_raw = bytearray() - try: - with ( - (output / "stderr.private.log").open("xb") as stderr, - (output / "stdout.json").open("xb") as stdout, - ): - process = subprocess.Popen( - [ - sys.executable, - "-IB", - str(Path(__file__).resolve()), - "_exec", - json.dumps(args), - ], - stdin=subprocess.PIPE, - stdout=stdout, - stderr=stderr, - pass_fds=(status_write, block_read), - start_new_session=True, - ) - os.close(status_write) - status_write = -1 - os.close(block_read) - block_read = -1 - read_line(status_read, time.monotonic() + 10, status_raw) - start = json.loads(status_raw, object_pairs_hook=unique) - init_fd, identity = bind_init(start, process.pid) - report["init_identity"] = identity - report["initial_status"] = start - report["authorization_sent"] = False - if mode == "gate_early_launcher_death": - # Reproduce launcher death before setup release, while holding - # the init pidfd. EOF on the setup pipe must not authorize work. - process.kill() - process.wait(timeout=5) - else: - os.write(block_write, b"1") - os.close(block_write) - block_write = -1 - if mode not in ("gate_eof", "gate_timeout", "gate_early_launcher_death"): - token = ( - START_TOKEN[:-1] - if mode == "gate_short" - else b"wrong\n" - if mode == "gate_wrong" - else START_TOKEN - ) - process.stdin.write(token) - process.stdin.flush() - report["authorization_sent"] = token == START_TOKEN - if mode != "gate_timeout": - process.stdin.close() - if mode.startswith("gate_"): - process.wait(timeout=8) - report["probe_passed"] = True # Exact output/exit checks below. - elif mode in ("bounds", "synthetic"): - process.wait(timeout=60) - report["probe_passed"] = True - else: - stdout_raw = read_ready_file( - output / "stdout.json", time.monotonic() + 10 - ) - payload = json.loads(stdout_raw, object_pairs_hook=unique) - if payload.get("ready") is not True or payload.get("mode") != mode: - raise ValueError("trusted cleanup probe did not become ready") - if payload.get("namespace_pid") != identity["pid_namespace"]: - raise ValueError("payload PID namespace differs from bound init") - children, child_records = bind_children(identity["pid"], payload) - report["registered_child_identities"] = child_records - if mode == "timeout": - try: - process.wait(timeout=1) - except subprocess.TimeoutExpired: - report["timeout_observed"] = True - pidfd_send_signal(init_fd, int(signal.SIGKILL)) - else: - raise ValueError("timeout probe returned before deadline") - elif mode == "launcher_kill": - process.kill() - process.wait(timeout=8) - stdout_raw = (output / "stdout.json").read_bytes() - report["probe_passed"] = True - report["exit_code"] = process.returncode - report["init_exit_observed"] = exited(init_fd, 5000) - report["registered_child_exit_observed"] = [ - exited(fd, 1000) for fd in children - ] - if not report["init_exit_observed"]: - raise ValueError("namespace init exit not confirmed") - stdout_raw = (output / "stdout.json").read_bytes() - stderr_raw = (output / "stderr.private.log").read_bytes() - if mode.startswith("gate_") and mode != "gate_valid": - report["probe_passed"] = ( - stdout_raw == b"" and AUTHORIZED_MARKER not in stderr_raw - ) - if mode != "gate_early_launcher_death": - report["probe_passed"] = ( - report["probe_passed"] and stderr_raw == REJECTED_MARKER - ) - else: - report["probe_passed"] = ( - report["probe_passed"] and stderr_raw == AUTHORIZED_MARKER - ) - if mode in ("bounds", "synthetic", "gate_valid"): - payload = json.loads(stdout_raw, object_pairs_hook=unique) - report["probe_passed"] = ( - report["probe_passed"] and payload.get("passed") is True - ) - # Parent has exited and bwrap's payload closes the status fd. Read - # the remainder with a finite deadline; EOF is expected. - deadline = time.monotonic() + 2 - status_eof = False - while time.monotonic() < deadline: - ready, _, _ = select.select( - [status_read], [], [], max(0, deadline - time.monotonic()) - ) - if not ready: - break - data = os.read(status_read, 4096) - if not data: - status_eof = True - break - status_raw += data - if len(status_raw) > 65536: - raise ValueError("status stream unexpectedly large") - report["status_eof"] = status_eof - expected_exit = ( - -signal.SIGKILL - if mode in ("launcher_kill", "gate_early_launcher_death") - else 137 - if mode == "timeout" - else 23 - if mode.startswith("gate_") and mode != "gate_valid" - else 0 - ) - report["passed"] = ( - report["probe_passed"] - and process.returncode == expected_exit - and report["init_exit_observed"] - and all(report["registered_child_exit_observed"]) - and status_eof - ) - except BaseException as error: - report["error"] = {"type": type(error).__name__, "message": str(error)} - finally: - finalize_case( - report, - process, - init_fd, - children, - (status_read, status_write, block_read, block_write), - output, - status_raw, - rootfs, - ) - return report - - -def main(): - parser = argparse.ArgumentParser(description=__doc__) - parser.add_argument("phase", choices=("capability", "probe")) - parser.add_argument("--bundle", type=Path) - parser.add_argument("--inventory", type=Path) - parser.add_argument("--capability", type=Path) - parser.add_argument("--output", type=Path, required=True) - args = parser.parse_args() - if args.phase == "capability": - return capability(args.output) - if args.capability is None or args.inventory is None: - raise ValueError("probe requires capability evidence and inventory") - capability_raw = args.capability.read_bytes() - preflight = json.loads(capability_raw, object_pairs_hook=unique) - if ( - preflight.get("passed") is not True - or preflight.get("cleanup_errors") != [] - or preflight.get("job_id") != os.environ["SLURM_JOB_ID"] - or preflight.get("driver_sha256") != runtime.sha(Path(__file__).read_bytes()) - or preflight.get("python_binary_sha256") - != runtime.sha(Path(sys._base_executable).read_bytes()) - or preflight.get("kernel_release") != platform.release() - ): - raise ValueError( - "actual pidfd capability preflight did not pass for this job/runtime" - ) - rootfs = json.loads((args.inventory / "rootfs.json").read_bytes()) - locators = json.loads( - (args.inventory / "runtime-locators.private.json").read_bytes() - ) - if ( - runtime.sha(Path(locators["bwrap"]).read_bytes()) - != rootfs["executor"]["binary_sha256"] - ): - raise ValueError("bwrap binary differs from inventory") - args.output.mkdir() - results = [] - for mode in MODES: - result = run_case(locators, rootfs, mode, args.output / mode) - results.append(result) - if result.get("cleanup_complete") is not True: - break - passed = len(results) == len(MODES) and all(row["passed"] for row in results) - runtime.write( - args.output / "summary.json", - { - "scope": "trusted_kernel_boundary_and_cleanup_probes", - "passed": passed, - "unexecuted_modes": list(MODES[len(results) :]), - "capability_sha256": runtime.sha(capability_raw), - "cases": results, - "generated_code_executed": False, - "qualified_generated_code_executor": False, - "verdict": "INCONCLUSIVE", - "driver_sha256": runtime.sha(Path(__file__).read_bytes()), - "probe_sha256": runtime.sha( - Path(__file__).with_name("synthetic_probe.py").read_bytes() - ), - "runtime_sha256": runtime.sha(Path(runtime.__file__).read_bytes()), - }, - ) - return 0 if passed else 2 - - -if __name__ == "__main__": - if ( - not os.environ.get("SLURM_JOB_ID") - or os.environ.get("SLURM_GPUS_ON_NODE", "0") != "0" - ): - raise ValueError("CPU-only Slurm allocation required") - if len(sys.argv) == 3 and sys.argv[1] == "_exec": - runtime.limits() - command = json.loads(sys.argv[2]) - os.execv(command[0], command) - raise SystemExit(main()) diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-full164-v1/capacity-precheck.json b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-full164-v1/capacity-precheck.json deleted file mode 100644 index cb51a6609..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-full164-v1/capacity-precheck.json +++ /dev/null @@ -1,12 +0,0 @@ -{ - "all_failure_details_with_empty_solutions_bytes": 7491894, - "all_failure_details_sha256": "95db8433152b97d1f65475431c6611104c232993d97c6f6131d57562c3b63736", - "samples_file_byte_cap": 67108864, - "solution_serialization_bound_bytes": 402653184, - "metadata_slack_bytes": 4096, - "result_upper_bound_bytes": 410149174, - "dataset_bytes": 7714666, - "task_count": 164, - "official_json_dump_defaults_preserved": true, - "reference_or_model_code_executed": false -} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-full164-v1/contract.py b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-full164-v1/contract.py deleted file mode 100644 index dde503b52..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-full164-v1/contract.py +++ /dev/null @@ -1,178 +0,0 @@ -"""Byte-bound input and result checks; never execute solutions on the host.""" - -import hashlib -import json -from pathlib import Path - -INPUT_NAMES = ( - "humaneval-solutions.jsonl", - "humaneval-subset.jsonl", - "human-export.json", -) - - -def sha(raw): - return hashlib.sha256(raw).hexdigest() - - -def unique(pairs): - value = {} - for key, item in pairs: - if key in value: - raise ValueError("duplicate JSON key") - value[key] = item - return value - - -def decode(raw): - return json.loads( - raw, - object_pairs_hook=unique, - parse_constant=lambda value: (_ for _ in ()).throw(ValueError(value)), - ) - - -def canonical(value): - return json.dumps( - value, sort_keys=True, separators=(",", ":"), allow_nan=False - ).encode() - - -def read(path, maximum): - with Path(path).open("rb") as stream: - raw = stream.read(maximum + 1) - if len(raw) > maximum: - raise ValueError("artifact exceeds declared byte cap") - return raw - - -def rows(raw, samples=False): - result = {} - for line in raw.splitlines(): - if not line: - raise ValueError("empty JSONL record") - row = decode(line) - if not isinstance(row, dict): - raise ValueError("JSONL record must be an object") - identity = row.get("task_id") - if not isinstance(identity, str) or not identity or identity in result: - raise ValueError("missing or duplicate task identity") - if samples and ( - set(row) != {"task_id", "solution"} or not isinstance(row["solution"], str) - ): - raise ValueError("expected exactly one complete solution per task") - result[identity] = row - if not result: - raise ValueError("empty input cohort") - return result - - -def inputs(directory, export_sha256, cohort_sha256, maximum): - raw = {name: read(Path(directory) / name, maximum) for name in INPUT_NAMES} - if sha(raw["human-export.json"]) != export_sha256: - raise ValueError("export digest mismatch") - export = decode(raw["human-export.json"]) - if not isinstance(export, dict) or set(export) != { - "samples_sha256", - "dataset_sha256", - "evalplus_dataset_md5", - "task_ids", - "generated_code_executed", - }: - raise ValueError("unsupported export manifest") - if export["generated_code_executed"] is not False: - raise ValueError("export must originate from the nonexecuting score path") - samples = rows(raw["humaneval-solutions.jsonl"], samples=True) - dataset = rows(raw["humaneval-subset.jsonl"]) - identities = sorted(samples) - if sorted(dataset) != identities or export["task_ids"] != identities: - raise ValueError("sample/dataset/cohort coverage differs") - if sha(canonical(identities)) != cohort_sha256: - raise ValueError("declared cohort digest mismatch") - if export["samples_sha256"] != sha(raw["humaneval-solutions.jsonl"]): - raise ValueError("sample digest mismatch") - if export["dataset_sha256"] != sha(raw["humaneval-subset.jsonl"]): - raise ValueError("dataset digest mismatch") - if ( - export["evalplus_dataset_md5"] - != hashlib.md5(raw["humaneval-subset.jsonl"]).hexdigest() - ): - raise ValueError("official evaluator dataset identity differs") - return raw, export, samples - - -def results(raw, export, samples, maximum): - if not raw or len(raw) > maximum: - raise ValueError("empty or oversized evaluator result") - value = decode(raw) - if ( - not isinstance(value, dict) - or value.get("hash") != export["evalplus_dataset_md5"] - ): - raise ValueError("result dataset identity differs") - evaluated = value.get("eval") - if not isinstance(evaluated, dict) or set(evaluated) != set(samples): - raise ValueError("result cohort incomplete or unexpected") - statuses = {} - for identity, answers in evaluated.items(): - if not isinstance(answers, list) or len(answers) != 1: - raise ValueError("expected one result per task") - answer = answers[0] - if not isinstance(answer, dict) or answer.get("task_id") != identity: - raise ValueError("result row task identity differs from its enclosing key") - if ( - not isinstance(answer, dict) - or answer.get("solution") != samples[identity]["solution"] - ): - raise ValueError("result solution differs from executed sample") - pair = [answer.get("base_status"), answer.get("plus_status")] - if any(status not in ("pass", "fail", "timeout") for status in pair): - raise ValueError("unknown or missing evaluator status") - statuses[identity] = pair - return statuses - - -def policy(path, expected_sha256): - raw = read(path, 16384) - if sha(raw) != expected_sha256: - raise ValueError("policy digest mismatch") - value = decode(raw) - expected = { - "schema", - "scope", - "qualified", - "as_bytes", - "cpu_seconds", - "wall_seconds", - "file_bytes", - "work_bytes", - "shm_bytes", - "tasks", - "nofile", - "input_bytes", - "result_bytes", - "evalplus", - } - if not isinstance(value, dict) or set(value) != expected or value["schema"] != 1: - raise ValueError("invalid policy schema") - if ( - value["qualified"] is not False - or value["scope"] != "full164-canonical-resource-pilot" - ): - raise ValueError("this prototype has no production qualification") - for key in expected - {"schema", "scope", "qualified", "evalplus"}: - if type(value[key]) is not int or value[key] <= 0: - raise ValueError("resource policy must contain finite positive integers") - if value["as_bytes"] < 4 * 1024**3 or value["file_bytes"] < value["result_bytes"]: - raise ValueError("policy cannot preserve evaluator memory/result requirements") - if value["evalplus"] != { - "parallel": 1, - "base_only": False, - "mini": False, - "noextreme": False, - "test_details": True, - "memory_override": None, - "timeout_override": None, - }: - raise ValueError("unsupported evaluator policy") - return value diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-full164-v1/controls.txt b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-full164-v1/controls.txt deleted file mode 100644 index 56a3e1caa..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-full164-v1/controls.txt +++ /dev/null @@ -1,5 +0,0 @@ -........ ----------------------------------------------------------------------- -Ran 8 tests in 0.009s - -OK diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-full164-v1/data-lock.json b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-full164-v1/data-lock.json deleted file mode 100644 index 3aae9242d..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-full164-v1/data-lock.json +++ /dev/null @@ -1,8 +0,0 @@ -{ - "cohort_sha256": "8c4a9413be6b6b793de94b702ab733ca734db2f5d6bca361605f1d7f71dd9ebe", - "dataset_bytes": 7714666, - "dataset_sha256": "42526ec0e7d5f3ee0b06d6ced98f8c8bae3d76519151bfb3d36f79010645bd7f", - "gzip_sha256": "272720b90ac375502c8ed23cd791c2a93dfb22a911641a494da74a426c09f101", - "prepared_manifest_sha256": "7a969339239edd8596baffb3977f9394249afa1b9266b880739470a834249a36", - "task_count": 164 -} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-full164-v1/data_binding.py b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-full164-v1/data_binding.py deleted file mode 100644 index d9fb11f53..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-full164-v1/data_binding.py +++ /dev/null @@ -1,129 +0,0 @@ -"""Authenticate full reference inputs without exposing problem or solution text.""" - -import gzip -import hashlib -import json -from pathlib import Path - -import contract - -ROOT = Path(__file__).resolve().parent - - -def lock(): - return contract.decode((ROOT / "data-lock.json").read_bytes()) - - -def references(dataset): - tasks = contract.rows(dataset) - return b"".join( - json.dumps( - { - "task_id": identity, - "solution": tasks[identity]["prompt"] - + tasks[identity]["canonical_solution"], - }, - ensure_ascii=False, - sort_keys=True, - separators=(",", ":"), - allow_nan=False, - ).encode() - + b"\n" - for identity in sorted(tasks) - ) - - -def validate(raw, export, samples): - expected = lock() - dataset = raw["humaneval-subset.jsonl"] - if ( - len(dataset) != expected["dataset_bytes"] - or contract.sha(dataset) != expected["dataset_sha256"] - ): - raise ValueError("pinned full dataset mismatch") - if ( - len(samples) != expected["task_count"] - or contract.sha(contract.canonical(sorted(samples))) - != expected["cohort_sha256"] - ): - raise ValueError("pinned full cohort mismatch") - if raw["humaneval-solutions.jsonl"] != references(dataset): - raise ValueError("pilot accepts only complete canonical references") - - -def create(gzip_path, output): - compressed = contract.read(gzip_path, 2 * 1024**2) - expected = lock() - if contract.sha(compressed) != expected["gzip_sha256"]: - raise ValueError("pinned compressed dataset mismatch") - dataset = gzip.decompress(compressed) - samples_raw = references(dataset) - samples = contract.rows(samples_raw, samples=True) - export = { - "samples_sha256": contract.sha(samples_raw), - "dataset_sha256": contract.sha(dataset), - "evalplus_dataset_md5": hashlib.md5(dataset).hexdigest(), - "task_ids": sorted(samples), - "generated_code_executed": False, - } - raw = dict( - zip( - contract.INPUT_NAMES, - (samples_raw, dataset, contract.canonical(export) + b"\n"), - ) - ) - validate(raw, export, samples) - output.mkdir() - for name, value in raw.items(): - with (output / name).open("xb") as stream: - stream.write(value) - return { - "export_sha256": contract.sha(raw["human-export.json"]), - "cohort_sha256": expected["cohort_sha256"], - } - - -def capacity(dataset, sample_cap): - # Official HumanEval evaluator uses json.dump(results, f) with defaults. - # Enumerate every input as failed, but leave solution strings empty; this - # serializes no reference answers and bounds any subset of failed tests. - tasks = contract.rows(dataset) - result = { - "date": "2000-01-01 00:00", - "hash": hashlib.md5(dataset).hexdigest(), - "eval": { - identity: [ - { - "task_id": identity, - "solution": "", - "base_status": "timeout", - "plus_status": "timeout", - "base_fail_tests": row["base_input"], - "plus_fail_tests": row["plus_input"], - } - ] - for identity, row in tasks.items() - }, - "pass_at_k": { - "base": {"pass@1": 0.12345678901234567}, - "plus": {"pass@1": 0.12345678901234567}, - }, - } - details = json.dumps(result).encode() - # A decoded character's default JSON representation needs at most six - # bytes per original UTF-8 byte (astral chars: 12/4). Input JSONL includes - # solution text plus additional syntax, so 6*whole sample cap is conservative. - # 4096 additional bytes cover fixed scalar number/date formatting slack. - bound = len(details) + 6 * sample_cap + 4096 - return { - "all_failure_details_with_empty_solutions_bytes": len(details), - "all_failure_details_sha256": contract.sha(details), - "samples_file_byte_cap": sample_cap, - "solution_serialization_bound_bytes": 6 * sample_cap, - "metadata_slack_bytes": 4096, - "result_upper_bound_bytes": bound, - "dataset_bytes": len(dataset), - "task_count": len(tasks), - "official_json_dump_defaults_preserved": True, - "reference_or_model_code_executed": False, - } diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-full164-v1/executor.py b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-full164-v1/executor.py deleted file mode 100644 index 91689b665..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-full164-v1/executor.py +++ /dev/null @@ -1,317 +0,0 @@ -"""Run an authenticated export through a fresh, unqualified evaluator prototype.""" - -import argparse -import json -import os -import resource -import select -import subprocess -import sys -import time -from pathlib import Path - -SOURCE = Path(__file__).resolve().parent -sys.path.insert(0, str(SOURCE)) -# The fixed source directory is deliberately restored under isolated Python (-I). -import bounds_driver as supervisor # noqa: E402 -import contract # noqa: E402 -import data_binding # noqa: E402 - -runtime = supervisor.runtime -STATIC_FILES = ( - "runtime.py", - "bounds_driver.py", - "synthetic_probe.py", - "contract.py", - "inner.py", - "executor.py", - "policy.json", - "data_binding.py", - "data-lock.json", -) - - -def write(path, value): - with Path(path).open("xb") as stream: - stream.write(contract.canonical(value) + b"\n") - - -def inventory(bundle, output): - runtime.inventory(bundle, output) - original = contract.decode((output / "rootfs.json").read_bytes()) - entries = list(original["entries"]) - for name in STATIC_FILES: - raw = (SOURCE / name).read_bytes() - entries.append( - { - "path": "/opt/executor/" + name, - "type": "file", - "bytes": len(raw), - "sha256": contract.sha(raw), - } - ) - entries.sort(key=lambda row: row["path"]) - value = { - **original, - "entries": entries, - "rootfs_digest": "sha256:" + contract.sha(contract.canonical(entries)), - } - write(output / "executor-rootfs.json", value) - return value, contract.decode( - (output / "runtime-locators.private.json").read_bytes() - ) - - -def command(locators, snapshot, policy, policy_sha256, export_sha256, cohort_sha256): - args = runtime.command(locators, SOURCE / "synthetic_probe.py", {}, "/unmounted") - # Retain the reviewed mount base, replace the concrete work/shm sizes, and - # replace the legacy diagnostic dispatch without calling its limits(). - for mount, capacity in ( - ("/work", policy["work_bytes"]), - ("/dev/shm", policy["shm_bytes"]), - ): - index = args.index(mount) - if args[index - 3 : index] != ["--size", args[index - 2], "--tmpfs"]: - raise ValueError("legacy mount command shape changed") - args[index - 2] = str(capacity) - payload_index = args.index("/opt/python/" + locators["python_relative"]) - mounts = [] - for name in STATIC_FILES: - mounts += ["--ro-bind", str(SOURCE / name), "/opt/executor/" + name] - mounts += ["--ro-bind", str(snapshot), "/input", "--remount-ro", "/dev"] - mount_index = args.index("--remount-ro") - return ( - args[:mount_index] - + mounts - + args[mount_index:payload_index] - + [ - args[payload_index], - "-IB", - "/opt/executor/inner.py", - policy_sha256, - export_sha256, - cohort_sha256, - ] - ) - - -def limits(policy): - os.sched_setaffinity(0, {min(os.sched_getaffinity(0))}) - for name, value in ( - ("AS", policy["as_bytes"]), - ("CPU", policy["cpu_seconds"]), - ("NOFILE", policy["nofile"]), - ("FSIZE", policy["file_bytes"]), - ("CORE", 0), - ): - resource.setrlimit(getattr(resource, "RLIMIT_" + name), (value, value)) - - -def status_end(fd, status_raw): - deadline = time.monotonic() + 2 - while time.monotonic() < deadline: - if not select.select([fd], [], [], max(0, deadline - time.monotonic()))[0]: - break - data = os.read(fd, 4096) - if not data: - return True - status_raw.extend(data) - if len(status_raw) > 65536: - raise ValueError("unexpected status stream size") - return False - - -def execute(args, rootfs, output, policy): - output.mkdir() - read_fd, write_fd = os.pipe() - block_read, block_write = os.pipe() - args[1:1] = ["--json-status-fd", str(write_fd), "--block-fd", str(block_read)] - process, init_fd = None, None - status_raw = bytearray() - report = { - "mode": "external-export", - "passed": False, - "timed_out": False, - "qualified": False, - "generated_code_executed": False, - } - write(output / "command.private.json", args) - try: - with ( - (output / "stdout.json").open("xb") as stdout, - (output / "stderr.private.log").open("xb") as stderr, - ): - process = subprocess.Popen( - [ - sys.executable, - "-IB", - str(SOURCE / "executor.py"), - "_exec", - contract.sha((SOURCE / "policy.json").read_bytes()), - json.dumps(args), - ], - stdin=subprocess.PIPE, - stdout=stdout, - stderr=stderr, - pass_fds=(write_fd, block_read), - start_new_session=True, - ) - os.close(write_fd) - write_fd = -1 - os.close(block_read) - block_read = -1 - supervisor.read_line(read_fd, time.monotonic() + 10, status_raw) - start = contract.decode(status_raw) - init_fd, identity = supervisor.bind_init(start, process.pid) - report["init_identity"] = identity - report["initial_status"] = start - os.write(block_write, b"1") - os.close(block_write) - block_write = -1 - process.stdin.write(supervisor.START_TOKEN) - process.stdin.flush() - process.stdin.close() - report["authorization_sent"] = True - started = time.monotonic() - try: - process.wait(timeout=policy["wall_seconds"]) - except subprocess.TimeoutExpired: - report["timed_out"] = True - raise - report["elapsed_seconds"] = time.monotonic() - started - report["exit_code"] = process.returncode - report["init_exit_observed"] = supervisor.exited(init_fd, 5000) - report["status_eof"] = status_end(read_fd, status_raw) - report["passed"] = ( - process.returncode == 0 - and report["init_exit_observed"] - and report["status_eof"] - ) - except BaseException as error: - report["error"] = {"type": type(error).__name__, "message": str(error)} - finally: - supervisor.finalize_case( - report, - process, - init_fd, - [], - (read_fd, write_fd, block_read, block_write), - output, - status_raw, - rootfs, - ) - return report - - -def evaluate_export(args): - policy = contract.policy(SOURCE / "policy.json", args.policy_sha256) - raw, export, samples = contract.inputs( - args.inputs, args.export_sha256, args.cohort_sha256, policy["input_bytes"] - ) - # The pilot permits only authenticated canonical references, never model output. - data_binding.validate(raw, export, samples) - args.output.mkdir(parents=False, exist_ok=False) - snapshot = args.output / "input-snapshot" - snapshot.mkdir() - for name, content in raw.items(): - with (snapshot / name).open("xb") as stream: - stream.write(content) - if supervisor.capability(args.output / "pidfd-capability.json") != 0: - raise ValueError("actual pidfd capability failed before sandbox creation") - before, locators = inventory(args.bundle, args.output / "inventory-before") - launch = command( - locators, - snapshot, - policy, - args.policy_sha256, - args.export_sha256, - args.cohort_sha256, - ) - report = execute(launch, before, args.output / "execution", policy) - after, _ = inventory(args.bundle, args.output / "inventory-after") - unchanged = before == after - statuses = None - result_sha256 = None - error = None - success = ( - report.get("passed") is True - and report.get("cleanup_complete") is True - and report.get("outputs_stable") is True - and unchanged - ) - if success: - try: - result_raw = contract.read( - args.output / "execution/stdout.json", policy["result_bytes"] - ) - result_sha256 = contract.sha(result_raw) - if result_sha256 != report["stdout_sha256"]: - raise ValueError( - "parsed result bytes differ from the final supervisor digest" - ) - statuses = contract.results( - result_raw, export, samples, policy["result_bytes"] - ) - except (ValueError, OSError) as failure: - error = {"type": type(failure).__name__, "message": str(failure)} - success = False - identity = { - "schema_version": 1, - "runtime_kind": "readonly-bind-set-v1", - "runtime_digest": before["rootfs_digest"], - "evaluator_revision": runtime.REVISION, - "executor_sha256": contract.sha((SOURCE / "inner.py").read_bytes()), - "driver_sha256": contract.sha((SOURCE / "executor.py").read_bytes()), - "sandbox_binary_sha256": before["executor"]["binary_sha256"], - "policy_sha256": args.policy_sha256, - } - write( - args.output / "execution-record.json", - { - "scope": policy["scope"], - "protocol": None, - "observed_identity": identity, - "qualified": False, - "verdict": "INCONCLUSIVE", - "execution_completed": success, - "network": "none", - "exit_code": report.get("final_launcher_exit_code"), - "timed_out": report["timed_out"], - "cleanup_complete": report["cleanup_complete"], - "runtime_unchanged": unchanged, - "export_sha256": args.export_sha256, - "cohort_sha256": args.cohort_sha256, - "samples_sha256": export["samples_sha256"], - "dataset_sha256": export["dataset_sha256"], - "results_sha256": result_sha256 if success else None, - "supervisor_stdout_sha256": report["stdout_sha256"], - "statuses": statuses, - "validation_error": error, - }, - ) - return 0 if success else 2 - - -def main(): - parser = argparse.ArgumentParser(description=__doc__) - parser.add_argument("--bundle", type=Path, required=True) - parser.add_argument("--inputs", type=Path, required=True) - parser.add_argument("--output", type=Path, required=True) - parser.add_argument("--policy-sha256", required=True) - parser.add_argument("--export-sha256", required=True) - parser.add_argument("--cohort-sha256", required=True) - return evaluate_export(parser.parse_args()) - - -if __name__ == "__main__": - if ( - not os.environ.get("SLURM_JOB_ID") - or os.environ.get("SLURM_GPUS_ON_NODE", "0") != "0" - ): - raise ValueError("CPU-only Slurm allocation required") - if len(sys.argv) == 4 and sys.argv[1] == "_exec": - selected = contract.policy(SOURCE / "policy.json", sys.argv[2]) - limits(selected) - launch = contract.decode(sys.argv[3]) - os.execv(launch[0], launch) - raise SystemExit(main()) diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-full164-v1/exercise.py b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-full164-v1/exercise.py deleted file mode 100644 index 4e1b61b33..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-full164-v1/exercise.py +++ /dev/null @@ -1,177 +0,0 @@ -"""Complete canonical pilot and limited controls; no model-generated programs.""" - -import argparse -import sys -from pathlib import Path -from types import SimpleNamespace - -sys.path.insert(0, str(Path(__file__).resolve().parent)) -import contract -import data_binding -import executor - - -def main(): - parser = argparse.ArgumentParser(description=__doc__) - parser.add_argument("--bundle", type=Path, required=True) - parser.add_argument("--dataset-gzip", type=Path, required=True) - parser.add_argument("--output", type=Path, required=True) - arguments = parser.parse_args() - arguments.output.mkdir() - identity = data_binding.create(arguments.dataset_gzip, arguments.output / "inputs") - policy_sha256 = contract.sha((executor.SOURCE / "policy.json").read_bytes()) - policy = contract.policy(executor.SOURCE / "policy.json", policy_sha256) - capacity = data_binding.capacity( - (arguments.output / "inputs/humaneval-subset.jsonl").read_bytes(), - policy["input_bytes"], - ) - executor.write(arguments.output / "capacity.json", capacity) - if capacity["result_upper_bound_bytes"] > min( - policy["file_bytes"], policy["result_bytes"] - ): - raise ValueError("official result can exceed file/result capacity") - args = SimpleNamespace( - bundle=arguments.bundle, - inputs=arguments.output / "inputs", - output=arguments.output / "normal", - policy_sha256=policy_sha256, - **identity, - ) - normal_exit = executor.evaluate_export(args) - record = contract.decode((args.output / "execution-record.json").read_bytes()) - checks = { - "full_reference_execution_completed": normal_exit == 0, - "all_164_base_and_plus_pass": len(record["statuses"] or {}) == 164 - and all( - pair == ["pass", "pass"] for pair in (record["statuses"] or {}).values() - ), - "not_qualified": record["qualified"] is False and record["protocol"] is None, - "runtime_stable": record["runtime_unchanged"] is True, - "full_failure_result_capacity": capacity["result_upper_bound_bytes"] - < policy["result_bytes"], - } - try: - executor.evaluate_export(args) - except FileExistsError: - checks["old_output_rejected"] = True - else: - checks["old_output_rejected"] = False - stderr = (args.output / "execution/stderr.private.log").read_bytes() - observations = [ - line.removeprefix(b"FULL164_OBSERVATIONS ") - for line in stderr.splitlines() - if line.startswith(b"FULL164_OBSERVATIONS ") - ] - measured = contract.decode(observations[0]) if len(observations) == 1 else None - checks["observed_actual_resource_limits"] = ( - bool(measured) - and all( - measured["limits"][key] == [value, value] - for key, value in { - "AS": policy["as_bytes"], - "CPU": policy["cpu_seconds"], - "FSIZE": policy["file_bytes"], - "NPROC": policy["tasks"], - "NOFILE": policy["nofile"], - }.items() - ) - and measured["work_capacity_bytes"] == policy["work_bytes"] - ) - checks["cache_and_worst_result_fit_work"] = bool(measured) and ( - measured["cache_file_bytes"] + capacity["result_upper_bound_bytes"] + 1024**2 - < policy["work_bytes"] - ) - rootfs = contract.decode( - (args.output / "inventory-before/executor-rootfs.json").read_bytes() - ) - locators = contract.decode( - (args.output / "inventory-before/runtime-locators.private.json").read_bytes() - ) - faults = [] - if record["cleanup_complete"] is True: - for name in ( - "wrong_policy", - "wrong_export", - "preexisting_cache", - "batch_timeout", - ): - command = executor.command( - locators, - args.output / "input-snapshot", - policy, - policy_sha256, - **identity, - ) - control_policy = dict(policy) - if name in ("wrong_policy", "wrong_export"): - command[-3 if name == "wrong_policy" else -2] = "0" * 64 - elif name == "preexisting_cache": - index = command.index("--remount-ro") - command[index:index] = ["--dir", "/work/cache"] - else: - control_policy["wall_seconds"] = 1 - observed = executor.execute( - command, rootfs, arguments.output / name, control_policy - ) - error_raw = (arguments.output / name / "stderr.private.log").read_bytes() - no_score = (arguments.output / name / "stdout.json").read_bytes() == b"" - passed = ( - observed["passed"] is False - and observed["cleanup_complete"] is True - and observed["outputs_stable"] is True - and no_score - ) - if name == "batch_timeout": - passed = passed and observed["timed_out"] is True - else: - reason = { - "wrong_policy": b"policy digest mismatch", - "wrong_export": b"export digest mismatch", - "preexisting_cache": b"evaluator output/cache reuse forbidden", - }[name] - passed = ( - passed - and observed["timed_out"] is False - and observed["final_launcher_exit_code"] != 0 - and error_raw.startswith(b"PAYLOAD_START_AUTHORIZED\n") - and error_raw.endswith(b"ValueError: " + reason + b"\n") - ) - checks[name + "_rejected_without_score"] = passed - faults.append( - { - "name": name, - "passed": passed, - "timed_out": observed["timed_out"], - "cleanup_complete": observed["cleanup_complete"], - } - ) - if observed["cleanup_complete"] is not True: - break - final, _ = executor.inventory( - arguments.bundle, arguments.output / "inventory-final" - ) - checks["runtime_stable_after_controls"] = final == rootfs - executor.write( - arguments.output / "summary.json", - { - "scope": "full164-canonical-resource-pilot", - "passed": all(checks.values()) and len(faults) == 4, - "checks": checks, - "negative_controls": faults, - "capacity": capacity, - "observations": measured, - "observed_identity": record["observed_identity"], - "result_sha256": record["results_sha256"], - "task_count": len(record["statuses"] or {}), - "qualified": False, - "verdict": "INCONCLUSIVE", - "model_generated_code_executed": False, - "trusted_canonical_reference_evaluation_completed": record["execution_completed"], - "formal_worst_case_wall_qualified": False, - }, - ) - return 0 if all(checks.values()) and len(faults) == 4 else 2 - - -if __name__ == "__main__": - raise SystemExit(main()) diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-full164-v1/exercise.review-rejected-execution-claim.py b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-full164-v1/exercise.review-rejected-execution-claim.py deleted file mode 100644 index cd1362837..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-full164-v1/exercise.review-rejected-execution-claim.py +++ /dev/null @@ -1,177 +0,0 @@ -"""Complete canonical pilot and limited controls; no model-generated programs.""" - -import argparse -import sys -from pathlib import Path -from types import SimpleNamespace - -sys.path.insert(0, str(Path(__file__).resolve().parent)) -import contract -import data_binding -import executor - - -def main(): - parser = argparse.ArgumentParser(description=__doc__) - parser.add_argument("--bundle", type=Path, required=True) - parser.add_argument("--dataset-gzip", type=Path, required=True) - parser.add_argument("--output", type=Path, required=True) - arguments = parser.parse_args() - arguments.output.mkdir() - identity = data_binding.create(arguments.dataset_gzip, arguments.output / "inputs") - policy_sha256 = contract.sha((executor.SOURCE / "policy.json").read_bytes()) - policy = contract.policy(executor.SOURCE / "policy.json", policy_sha256) - capacity = data_binding.capacity( - (arguments.output / "inputs/humaneval-subset.jsonl").read_bytes(), - policy["input_bytes"], - ) - executor.write(arguments.output / "capacity.json", capacity) - if capacity["result_upper_bound_bytes"] > min( - policy["file_bytes"], policy["result_bytes"] - ): - raise ValueError("official result can exceed file/result capacity") - args = SimpleNamespace( - bundle=arguments.bundle, - inputs=arguments.output / "inputs", - output=arguments.output / "normal", - policy_sha256=policy_sha256, - **identity, - ) - normal_exit = executor.evaluate_export(args) - record = contract.decode((args.output / "execution-record.json").read_bytes()) - checks = { - "full_reference_execution_completed": normal_exit == 0, - "all_164_base_and_plus_pass": len(record["statuses"] or {}) == 164 - and all( - pair == ["pass", "pass"] for pair in (record["statuses"] or {}).values() - ), - "not_qualified": record["qualified"] is False and record["protocol"] is None, - "runtime_stable": record["runtime_unchanged"] is True, - "full_failure_result_capacity": capacity["result_upper_bound_bytes"] - < policy["result_bytes"], - } - try: - executor.evaluate_export(args) - except FileExistsError: - checks["old_output_rejected"] = True - else: - checks["old_output_rejected"] = False - stderr = (args.output / "execution/stderr.private.log").read_bytes() - observations = [ - line.removeprefix(b"FULL164_OBSERVATIONS ") - for line in stderr.splitlines() - if line.startswith(b"FULL164_OBSERVATIONS ") - ] - measured = contract.decode(observations[0]) if len(observations) == 1 else None - checks["observed_actual_resource_limits"] = ( - bool(measured) - and all( - measured["limits"][key] == [value, value] - for key, value in { - "AS": policy["as_bytes"], - "CPU": policy["cpu_seconds"], - "FSIZE": policy["file_bytes"], - "NPROC": policy["tasks"], - "NOFILE": policy["nofile"], - }.items() - ) - and measured["work_capacity_bytes"] == policy["work_bytes"] - ) - checks["cache_and_worst_result_fit_work"] = bool(measured) and ( - measured["cache_file_bytes"] + capacity["result_upper_bound_bytes"] + 1024**2 - < policy["work_bytes"] - ) - rootfs = contract.decode( - (args.output / "inventory-before/executor-rootfs.json").read_bytes() - ) - locators = contract.decode( - (args.output / "inventory-before/runtime-locators.private.json").read_bytes() - ) - faults = [] - if record["cleanup_complete"] is True: - for name in ( - "wrong_policy", - "wrong_export", - "preexisting_cache", - "batch_timeout", - ): - command = executor.command( - locators, - args.output / "input-snapshot", - policy, - policy_sha256, - **identity, - ) - control_policy = dict(policy) - if name in ("wrong_policy", "wrong_export"): - command[-3 if name == "wrong_policy" else -2] = "0" * 64 - elif name == "preexisting_cache": - index = command.index("--remount-ro") - command[index:index] = ["--dir", "/work/cache"] - else: - control_policy["wall_seconds"] = 1 - observed = executor.execute( - command, rootfs, arguments.output / name, control_policy - ) - error_raw = (arguments.output / name / "stderr.private.log").read_bytes() - no_score = (arguments.output / name / "stdout.json").read_bytes() == b"" - passed = ( - observed["passed"] is False - and observed["cleanup_complete"] is True - and observed["outputs_stable"] is True - and no_score - ) - if name == "batch_timeout": - passed = passed and observed["timed_out"] is True - else: - reason = { - "wrong_policy": b"policy digest mismatch", - "wrong_export": b"export digest mismatch", - "preexisting_cache": b"evaluator output/cache reuse forbidden", - }[name] - passed = ( - passed - and observed["timed_out"] is False - and observed["final_launcher_exit_code"] != 0 - and error_raw.startswith(b"PAYLOAD_START_AUTHORIZED\n") - and error_raw.endswith(b"ValueError: " + reason + b"\n") - ) - checks[name + "_rejected_without_score"] = passed - faults.append( - { - "name": name, - "passed": passed, - "timed_out": observed["timed_out"], - "cleanup_complete": observed["cleanup_complete"], - } - ) - if observed["cleanup_complete"] is not True: - break - final, _ = executor.inventory( - arguments.bundle, arguments.output / "inventory-final" - ) - checks["runtime_stable_after_controls"] = final == rootfs - executor.write( - arguments.output / "summary.json", - { - "scope": "full164-canonical-resource-pilot", - "passed": all(checks.values()) and len(faults) == 4, - "checks": checks, - "negative_controls": faults, - "capacity": capacity, - "observations": measured, - "observed_identity": record["observed_identity"], - "result_sha256": record["results_sha256"], - "task_count": len(record["statuses"] or {}), - "qualified": False, - "verdict": "INCONCLUSIVE", - "model_generated_code_executed": False, - "trusted_canonical_references_executed": True, - "formal_worst_case_wall_qualified": False, - }, - ) - return 0 if all(checks.values()) and len(faults) == 4 else 2 - - -if __name__ == "__main__": - raise SystemExit(main()) diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-full164-v1/inner.py b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-full164-v1/inner.py deleted file mode 100644 index 271f72431..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-full164-v1/inner.py +++ /dev/null @@ -1,110 +0,0 @@ -"""Generic official evaluator entry, available only inside the readonly sandbox.""" - -import contextlib -import importlib -import json -import os -import resource -import sys -import time -from pathlib import Path - -# -I deliberately excludes script paths; only the explicitly readonly helper root is added. -sys.path.insert(0, "/opt/executor") -import contract -import data_binding -import synthetic_probe as boundary - - -def main(): - try: - boundary.authorize_start() - except (ValueError, OSError): - os.write(2, b"PAYLOAD_START_REJECTED\n") - return 23 - descriptor = os.open("/dev/null", os.O_RDONLY) - os.dup2(descriptor, 0) - if descriptor != 0: - os.close(descriptor) - os.write(2, b"PAYLOAD_START_AUTHORIZED\n") - policy = contract.policy("/opt/executor/policy.json", sys.argv[1]) - if sys.flags.optimize: - raise ValueError("EvalPlus correctness assertions must remain enabled") - for name, value in (("NPROC", policy["tasks"]), ("MEMLOCK", 0), ("MSGQUEUE", 0)): - resource.setrlimit(getattr(resource, "RLIMIT_" + name), (value, value)) - boundary.install_ipc_filter() - raw_inputs, export, samples = contract.inputs( - "/input", sys.argv[2], sys.argv[3], policy["input_bytes"] - ) - data_binding.validate(raw_inputs, export, samples) - work = Path("/work") - cache, output = work / "cache", work / "output/results.json" - if cache.exists() or output.exists(): - raise ValueError("evaluator output/cache reuse forbidden") - # All module-captured environment settings precede the first EvalPlus import. - os.environ["XDG_CACHE_HOME"] = str(cache) - os.environ["HUMANEVAL_OVERRIDE_PATH"] = "/input/humaneval-subset.jsonl" - os.environ.pop("EVALPLUS_MAX_MEMORY_BYTES", None) - os.environ.pop("EVALPLUS_TIMEOUT_PER_TASK", None) - sys.path.insert(0, "/opt/site-packages") - observations = { - "scope": "full164-canonical-resource-pilot", - "limits": { - name: list(resource.getrlimit(getattr(resource, "RLIMIT_" + name))) - for name in ("AS", "CPU", "FSIZE", "NPROC", "NOFILE") - }, - "work_capacity_bytes": os.statvfs(work).f_frsize * os.statvfs(work).f_blocks, - } - started = time.monotonic() - with contextlib.redirect_stdout(sys.stderr): - evaluator = importlib.import_module("evalplus.evaluate") - original_groundtruth = evaluator.get_groundtruth - - def measured_groundtruth(*args, **kwargs): - started_gt = time.monotonic() - try: - return original_groundtruth(*args, **kwargs) - finally: - observations["groundtruth_seconds"] = time.monotonic() - started_gt - - evaluator.get_groundtruth = measured_groundtruth - evaluator.evaluate( - dataset="humaneval", - samples="/input/humaneval-solutions.jsonl", - output_file=str(output), - parallel=1, - base_only=False, - mini=False, - noextreme=False, - test_details=True, - ) - raw = contract.read(output, policy["result_bytes"]) - contract.results(raw, export, samples, policy["result_bytes"]) - observations.update( - evaluator_seconds=time.monotonic() - started, - result_bytes=len(raw), - cache_file_bytes=sum( - path.stat().st_size for path in cache.rglob("*") if path.is_file() - ), - work_file_bytes=sum( - path.stat().st_size for path in work.rglob("*") if path.is_file() - ), - work_allocated_bytes=(os.statvfs(work).f_blocks - os.statvfs(work).f_bfree) - * os.statvfs(work).f_frsize, - coordinator_maxrss_kib=resource.getrusage(resource.RUSAGE_SELF).ru_maxrss, - groundtruth_wrapper_preserves_arguments_return_and_exceptions=True, - ) - os.write( - 2, - b"FULL164_OBSERVATIONS " - + json.dumps(observations, sort_keys=True).encode() - + b"\n", - ) - # Preserve exactly the upstream bytes once, including its original formatting. - sys.stdout.buffer.write(raw) - sys.stdout.buffer.flush() - return 0 - - -if __name__ == "__main__": - raise SystemExit(main()) diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-full164-v1/intake-metadata.json b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-full164-v1/intake-metadata.json deleted file mode 100644 index 95e504386..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-full164-v1/intake-metadata.json +++ /dev/null @@ -1,15 +0,0 @@ -{ - "scope": "metadata-only capacity intake; no solution or tests inspected in optimization context or executed", - "gzip_sha256": "272720b90ac375502c8ed23cd791c2a93dfb22a911641a494da74a426c09f101", - "gzip_bytes": 925932, - "dataset_sha256": "42526ec0e7d5f3ee0b06d6ced98f8c8bae3d76519151bfb3d36f79010645bd7f", - "dataset_bytes": 7714666, - "task_count": 164, - "cohort_sha256": "8c4a9413be6b6b793de94b702ab733ca734db2f5d6bca361605f1d7f71dd9ebe", - "reference_solution_utf8_bytes": 101353, - "base_input_count": 1570, - "plus_input_count": 122683, - "prepared_manifest_sha256": "7a969339239edd8596baffb3977f9394249afa1b9266b880739470a834249a36", - "prepared_dataset_match": true, - "prepared_cohort_match": true -} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-full164-v1/plan.json b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-full164-v1/plan.json deleted file mode 100644 index c08889f8b..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-full164-v1/plan.json +++ /dev/null @@ -1,70 +0,0 @@ -{ - "scope": "full164-canonical-resource-pilot", - "status": "reviewed_not_submitted", - "plane": "harness", - "verdict": "INCONCLUSIVE", - "qualified": false, - "model_generated_code_executed": false, - "source_sha256": { - "runtime.py": "997cbfed8398e8ad62d1beb4f6b4bc6e61a0f4572b9156cfd0c6f3fb52b03233", - "bounds_driver.py": "bfeeb6ea794c0ebf24e72a0127458ff74951253471daee080b3a3b1d82a062ba", - "synthetic_probe.py": "236920f27573447b9f3833afb72739033c965e22a9e406b09f28da815529ca64", - "contract.py": "080788af20e7132e7602570ec6eba620439e000a934cc6c15adcac19e31c0c12", - "inner.py": "dd69193feb2b824b5bdc4d037ec63ede64c9fe809faaf78f974542b23935a19f", - "executor.py": "05e59d9928f06c47c235bc5e878c97d9c49260631ae020fd1b6baf851ea2d5b3", - "data_binding.py": "3527f2fe807c1399df245d5a17b84bef0bdd16f21edcc7cf69b00ac895b3b902", - "data-lock.json": "b5949144fd33dafd78a1ed89d23ba1e8ce1bc41775b2765d5fe5991930aae74f", - "exercise.py": "ad0bb37a1898afee20acf1e828e62f32828cc5841cab43f884353d01738e0f8f", - "test_binding.py": "415bf4636f9b717ed22a589fe6a08d88370aca6998f23ac5fb66251797f7d9e6", - "policy.json": "0445dc9a7c281911da4701d4e6ecdd59f083a306aca6509fbb1817cd220af46a", - "run-executor.sbatch": "078961e8190a9903b07e70af96f469f2429884fe9a882fdcdb559e90771b4391" - }, - "source_manifest_sha256": "943c3f46efac0255255a690f1302a30534bd1dcc5b22ead3915b963aeb48599d", - "allocation": { - "cpus": 4, - "memory_request_gib": 16, - "wall_minutes": 60, - "gpus": 0 - }, - "policy": { - "schema": 1, - "scope": "full164-canonical-resource-pilot", - "qualified": false, - "as_bytes": 4294967296, - "cpu_seconds": 3600, - "wall_seconds": 3300, - "file_bytes": 536870912, - "work_bytes": 1073741824, - "shm_bytes": 33554432, - "tasks": 16, - "nofile": 64, - "input_bytes": 67108864, - "result_bytes": 536870912, - "evalplus": { - "parallel": 1, - "base_only": false, - "mini": false, - "noextreme": false, - "test_details": true, - "memory_override": null, - "timeout_override": null - } - }, - "capacity": { - "all_failure_details_with_empty_solutions_bytes": 7491894, - "all_failure_details_sha256": "95db8433152b97d1f65475431c6611104c232993d97c6f6131d57562c3b63736", - "samples_file_byte_cap": 67108864, - "solution_serialization_bound_bytes": 402653184, - "metadata_slack_bytes": 4096, - "result_upper_bound_bytes": 410149174, - "dataset_bytes": 7714666, - "task_count": 164, - "official_json_dump_defaults_preserved": true, - "reference_or_model_code_executed": false - }, - "evaluator_revision": "26d6d00bb1fd0fa37f39c99d5290da67891d1c5e", - "readiness_gpu_seconds_used": 2388, - "readiness_gpu_seconds_cap": 7200, - "initial_gpu_seconds_separate": 147, - "e2e_effect": null -} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-full164-v1/policy.json b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-full164-v1/policy.json deleted file mode 100644 index db7e3bd81..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-full164-v1/policy.json +++ /dev/null @@ -1 +0,0 @@ -{"schema":1,"scope":"full164-canonical-resource-pilot","qualified":false,"as_bytes":4294967296,"cpu_seconds":3600,"wall_seconds":3300,"file_bytes":536870912,"work_bytes":1073741824,"shm_bytes":33554432,"tasks":16,"nofile":64,"input_bytes":67108864,"result_bytes":536870912,"evalplus":{"parallel":1,"base_only":false,"mini":false,"noextreme":false,"test_details":true,"memory_override":null,"timeout_override":null}} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-full164-v1/proposal.md b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-full164-v1/proposal.md deleted file mode 100644 index 3ba9d0de4..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-full164-v1/proposal.md +++ /dev/null @@ -1,62 +0,0 @@ -# Complete HumanEval+ reference resource pilot - -Refs [Issue #67](https://github.com/InferMatrixLab/InferRSI/issues/67). -Campaign INCONCLUSIVE; no model quality or performance claim. - -The next bounded CPU run executes all 164 authenticated canonical references -through the external-export evaluator interface. It permits only exact reference -sample bytes derived from the pinned dataset; it rejects model output. Full task -and solution content remain in controlled storage and are not exposed to the -optimization context. Dataset intake matches the previously prepared final set. -Only aggregate counts, hashes, status counts, timing and resource sizes are shared. - -The pinned official EvalPlus revision and dependency installation are unchanged. -The new source keeps official base/Plus tests, test_details=True, parallel=1, -4 GiB default per-process memory and default per-input timeout calculation. It -uses the previously verified startup token, raw pidfd and finalization helpers. -No dependency rebuild or repetition of unrelated isolation modes is needed. - -A 4-CPU, 16 GiB request, zero-GPU allocation is limited to one hour. The single -CPU-affinity payload has a 3300-second whole-batch wall bound and 3600-second CPU -limit. Timeout means infrastructure INCONCLUSIVE, never a model-wrong result. -The 16 GiB request is not represented as an enforced aggregate memory cap. - -Correction to earlier budgeting interpretation: the ProcessPool coordinator and -worker accumulate their own CPU usage, but each untrusted_check starts a fresh -solution child with reset CPU accounting. Solution CPU does not accumulate in -the reused worker's RLIMIT_CPU counter. Official per-input behavior is unchanged. -The approximate 164 x 2 x 63.2-second worst-case branch waiting sum alone is -20,729.6 seconds, before trusted groundtruth and serialization. This one-hour -canonical pilot does not qualify a worst-case generated-code batch wall budget. - -Capacity precheck serializes the exact official result shape with all fixed -base/Plus inputs as failure details and empty solution strings: 7,491,894 bytes. -Six times the separate 64 MiB samples cap bounds ensure_ascii expansion; 4096 -bytes add scalar-formatting slack. Total upper bound: 410,149,174 bytes. New -result/FSIZE limits are 512 MiB and work tmpfs is 1 GiB. The actual run measures -reference cache and result bytes, observed rlimits/work capacity, groundtruth and -whole-evaluator wall time. Final work occupancy is not called peak usage. Cache -size plus worst-case output must fit; failure stops qualification progression. - -Groundtruth timing wraps the pinned function with unchanged arguments, return -value and exception propagation. The result bytes are emitted unchanged and -bound to the final supervisor hash. Negative controls cover wrong policy/export, -preexisting cache and a deliberate one-second supervisor deadline, with cleanup -and no acceptable score required. Existing-output reuse must fail. Newly observed -resource values must match policy; the program does not fabricate qualification. - -Eight host-only input-binding/capacity tests, repository-configured lint and shell -syntax checks pass. The initial source formatting diagnostics were corrected -before the source lock; they were not an execution or optimization result. -Actual Linux outcome remains pending. Success and failure will both be retained -in the evidence tree and revision-bound KnowledgeStore candidate. - -Full generated-code qualification/provenance, final finite worst-case budget, -owner-confirmed statistics/retrieval decisions and complete command/validator -freeze remain pending. No performance search is activated. E2E effect: not -measured. GPU readiness accounting remains 2388/7200 seconds, initial147 separate. - -Independent review rejected an unconditional reference-executed flag in the -summary, which could misreport a failed startup. The rejected source is retained; -the replacement records only authenticated evaluation completion. A one-second -negative deadline proves supervisor cleanup, not that solution execution began. diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-full164-v1/review.json b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-full164-v1/review.json deleted file mode 100644 index 48ca3e2ac..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-full164-v1/review.json +++ /dev/null @@ -1,42 +0,0 @@ -{ - "reviewer": "Codex /root/review_fp8_graph", - "role": "independent read-only reviewer", - "base_commit": "4e77b8fdd40cc1d8a8844ba3b69230d5e31d3355", - "source_manifest_sha256": "943c3f46efac0255255a690f1302a30534bd1dcc5b22ead3915b963aeb48599d", - "source_sha256": { - "runtime.py": "997cbfed8398e8ad62d1beb4f6b4bc6e61a0f4572b9156cfd0c6f3fb52b03233", - "bounds_driver.py": "bfeeb6ea794c0ebf24e72a0127458ff74951253471daee080b3a3b1d82a062ba", - "synthetic_probe.py": "236920f27573447b9f3833afb72739033c965e22a9e406b09f28da815529ca64", - "contract.py": "080788af20e7132e7602570ec6eba620439e000a934cc6c15adcac19e31c0c12", - "inner.py": "dd69193feb2b824b5bdc4d037ec63ede64c9fe809faaf78f974542b23935a19f", - "executor.py": "05e59d9928f06c47c235bc5e878c97d9c49260631ae020fd1b6baf851ea2d5b3", - "data_binding.py": "3527f2fe807c1399df245d5a17b84bef0bdd16f21edcc7cf69b00ac895b3b902", - "data-lock.json": "b5949144fd33dafd78a1ed89d23ba1e8ce1bc41775b2765d5fe5991930aae74f", - "exercise.py": "ad0bb37a1898afee20acf1e828e62f32828cc5841cab43f884353d01738e0f8f", - "test_binding.py": "415bf4636f9b717ed22a589fe6a08d88370aca6998f23ac5fb66251797f7d9e6", - "policy.json": "0445dc9a7c281911da4701d4e6ecdd59f083a306aca6509fbb1817cd220af46a", - "run-executor.sbatch": "078961e8190a9903b07e70af96f469f2429884fe9a882fdcdb559e90771b4391" - }, - "blocking_findings": [], - "resolved_findings": [ - { - "priority": "P2", - "original": "Unconditional trusted references executed flag could misrepresent failed startup.", - "repair": "Completion is bound to actual execution-record completion; rejected original retained." - } - ], - "checks": { - "source_hashes": 12, - "binding_tests": 8, - "groundtruth_wrapper_mock_controls": 2, - "completion_field_positive_negative_controls": 2, - "bash_syntax": true - }, - "recommendation": "Submit one bounded full164 canonical resource pilot: 1h4CPU16GiBrequest0GPU.", - "limits": [ - "One-second control only proves supervisor deadline behavior, not evaluation-in-progress timeout.", - "Final work occupancy is not peak.", - "16GiB request is not aggregate enforced memory.", - "No model output or final qualification/performance PASS." - ] -} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-full164-v1/run-executor.sbatch b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-full164-v1/run-executor.sbatch deleted file mode 100644 index 6268c6d83..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-full164-v1/run-executor.sbatch +++ /dev/null @@ -1,203 +0,0 @@ -#!/usr/bin/env bash -# Complete pinned 164-task canonical resource pilot; no model-generated code. -#SBATCH --job-name=qwen38-sandbox-full164-v1 -#SBATCH --nodes=1 -#SBATCH --ntasks=1 -#SBATCH --cpus-per-task=4 -#SBATCH --mem=16G -#SBATCH --time=01:00:00 -set -euo pipefail -umask 077 -: "${TASK_ROOT:?}" "${DATASET_GZIP:?}" -: "${SANDBOX_BUNDLE:?}" -: "${SANDBOX_FREEZE_SHA256:?}" -: "${SANDBOX_VENV:?}" -: "${REFERENCE_INSTALLATION:?}" -: "${REFERENCE_ROOTFS_SHA256:?}" -: "${REFERENCE_TERMINAL_SHA256:?}" -: "${REFERENCE_SLURM_SHA256:?}" -: "${REFERENCE_JOB_ID:?}" -: "${REFERENCE_EXECUTION_SHA256:?}" "${REFERENCE_STDOUT_SHA256:?}" -: "${BWRAP_BIN:?}" "${BWRAP_SHA256:?}" "${PYTHON_SHA256:?}" -: "${SYNTHETIC_SOURCE_DIR:?}" -: "${SYNTHETIC_RUNTIME_SHA256:?}" -: "${SYNTHETIC_PROBE_SHA256:?}" -: "${BOUNDS_DRIVER_SHA256:?}" -: "${SYNTHETIC_SCRIPT_SHA256:?}" -: "${SYNTHETIC_OUTPUT:?}" "${EXECUTOR_SOURCES_SHA256:?}" -unset PYTHONPATH PYTHONHOME PYTHONOPTIMIZE PYTHONINSPECT PYTHONSTARTUP VIRTUAL_ENV -[[ -n ${SLURM_JOB_ID:-} && ${SLURM_GPUS_ON_NODE:-0} == 0 ]] -root=$(realpath -e "$TASK_ROOT") -check_path() { - local value - value=$(realpath -m "$1") - case "$value" in "$root"/*) ;; *) echo 'Path escapes task root.' >&2; exit 2;; esac -} -check_digest() { - if [[ ! "$2" =~ ^[a-f0-9]{64}$ ]]; then return 2; fi - [[ $(sha256sum "$1" | cut -d' ' -f1) == "$2" ]] -} -for path in "$SANDBOX_BUNDLE" "$SANDBOX_VENV" "$REFERENCE_INSTALLATION" \ - "$SYNTHETIC_SOURCE_DIR" "$SYNTHETIC_OUTPUT" "$BWRAP_BIN" "$DATASET_GZIP"; do check_path "$path"; done -source_dir=$(cd -- "$SYNTHETIC_SOURCE_DIR" && pwd -P) -check_digest "$SANDBOX_VENV/bin/python" "$PYTHON_SHA256" -check_digest "$BWRAP_BIN" "$BWRAP_SHA256" -export PATH="$(dirname "$BWRAP_BIN"):/usr/bin:/bin" -[[ $(command -v bwrap) == "$BWRAP_BIN" ]] -check_digest "$source_dir/run-executor.sbatch" "$SYNTHETIC_SCRIPT_SHA256" -cmp "${BASH_SOURCE[0]}" "$source_dir/run-executor.sbatch" -check_digest "$source_dir/runtime.py" "$SYNTHETIC_RUNTIME_SHA256" -check_digest "$source_dir/synthetic_probe.py" "$SYNTHETIC_PROBE_SHA256" -check_digest "$source_dir/bounds_driver.py" "$BOUNDS_DRIVER_SHA256" -check_digest "$source_dir/sources.sha256" "$EXECUTOR_SOURCES_SHA256" -(cd "$source_dir" && sha256sum --check --strict sources.sha256) -check_digest "$SANDBOX_BUNDLE/freeze.sha256" "$SANDBOX_FREEZE_SHA256" -(cd "$SANDBOX_BUNDLE" && sha256sum --check --strict freeze.sha256) -# Bind both the scheduler outcome and the repaired completion record. -check_digest "$REFERENCE_INSTALLATION/terminal-status.json" "$REFERENCE_TERMINAL_SHA256" -check_digest "$REFERENCE_INSTALLATION/slurm-final.private.txt" "$REFERENCE_SLURM_SHA256" -"$SANDBOX_VENV/bin/python" -IB - "$REFERENCE_INSTALLATION" "$REFERENCE_JOB_ID" <<'PYREF' -import json -import re -import sys -from pathlib import Path -reference = Path(sys.argv[1]) -job = sys.argv[2] -if not re.fullmatch(r"[0-9]+", job): - raise ValueError('Invalid reference job ID') -def unique_keys(pairs): - value = {} - for key, item in pairs: - if key in value: - raise ValueError('Duplicate terminal-record key') - value[key] = item - return value -terminal = json.loads((reference / 'terminal-status.json').read_bytes(), object_pairs_hook=unique_keys) -if terminal.get('completed') is not True or type(terminal.get('exit_code')) is not int: - raise ValueError('Reference terminal record has invalid types') -if terminal != {'completed': True, 'signal': 'none', 'exit_code': 0, 'job_id': job}: - raise ValueError('Reference installation did not complete normally') -slurm = (reference / 'slurm-final.private.txt').read_text() -for key, value in {'JobId': job, 'JobState': 'COMPLETED', 'ExitCode': '0:0'}.items(): - actual = re.findall(r'(? "$SYNTHETIC_OUTPUT/exit-code" - printf '{"completed":%s,"signal":"%s","exit_code":%s,"job_id":"%s"}\n' \ - "$task_completed" "$task_signal" "$status" "$SLURM_JOB_ID" > "$SYNTHETIC_OUTPUT/terminal-status.json" - exit "$status" -} -trap finish_synthetic EXIT -trap 'task_completed=false; task_signal=TERM; exit 143' TERM -trap 'task_completed=false; task_signal=INT; exit 130' INT -trap 'task_completed=false; task_signal=HUP; exit 129' HUP -# END TERMINAL RECORD -cp "$source_dir/run-executor.sbatch" "$source_dir/runtime.py" "$source_dir/bounds_driver.py" "$source_dir/synthetic_probe.py" "$source_dir/executor.py" "$source_dir/inner.py" "$source_dir/contract.py" "$source_dir/data_binding.py" "$source_dir/data-lock.json" "$source_dir/exercise.py" "$source_dir/test_binding.py" "$source_dir/policy.json" "$source_dir/sources.sha256" "$SYNTHETIC_OUTPUT/" -cd "$root" -"$SANDBOX_VENV/bin/python" -IB "$source_dir/bounds_driver.py" capability \ - --output "$SYNTHETIC_OUTPUT/pidfd-capability.json" -"$SANDBOX_VENV/bin/python" -IB "$source_dir/runtime.py" inventory --bundle "$SANDBOX_BUNDLE" \ - --output "$SYNTHETIC_OUTPUT/rootfs-before" -"$SANDBOX_VENV/bin/python" -IB - "$REFERENCE_INSTALLATION/rootfs-before/rootfs.json" \ - "$SYNTHETIC_OUTPUT/rootfs-before/rootfs.json" <<'PY' -import json -import sys -from pathlib import Path -before, current = [json.loads(Path(path).read_bytes()) for path in sys.argv[1:]] -for record in (before, current): - record.pop('rootfs_digest') - record['entries'] = [row for row in record['entries'] if row['path'] != '/opt/probe.py'] -if before != current: - raise ValueError('Installed runtime changed beyond the explicitly bound trusted probe') -PY -probe_status=0 -"$SANDBOX_VENV/bin/python" -IB "$source_dir/exercise.py" --bundle "$SANDBOX_BUNDLE" --dataset-gzip "$DATASET_GZIP" --output "$SYNTHETIC_OUTPUT/exercise" || probe_status=$? -printf '%s\n' "$probe_status" > "$SYNTHETIC_OUTPUT/probe-status" -post_status=0 -run_postcheck() { - local name=$1 status=0 - shift - "$@" || status=$? - printf '%s\t%s\n' "$name" "$status" >> "$SYNTHETIC_OUTPUT/postflight-status.tsv" - if [[ "$status" != 0 ]]; then post_status=$status; fi -} -check_executor_sources() { (cd "$source_dir" && sha256sum --check --strict sources.sha256); } -check_frozen_files() { (cd "$SANDBOX_BUNDLE" && sha256sum --check --strict freeze.sha256); } -run_postcheck inventory "$SANDBOX_VENV/bin/python" -IB "$source_dir/runtime.py" inventory \ - --bundle "$SANDBOX_BUNDLE" --output "$SYNTHETIC_OUTPUT/rootfs-after" -run_postcheck runtime-byte-comparison cmp "$SYNTHETIC_OUTPUT/rootfs-before/rootfs.json" \ - "$SYNTHETIC_OUTPUT/rootfs-after/rootfs.json" -run_postcheck runtime-script check_digest "$source_dir/runtime.py" "$SYNTHETIC_RUNTIME_SHA256" -run_postcheck bounds-driver check_digest "$source_dir/bounds_driver.py" "$BOUNDS_DRIVER_SHA256" -run_postcheck synthetic-probe check_digest "$source_dir/synthetic_probe.py" "$SYNTHETIC_PROBE_SHA256" -run_postcheck batch-script check_digest "$source_dir/run-executor.sbatch" "$SYNTHETIC_SCRIPT_SHA256" -run_postcheck executor-manifest check_digest "$source_dir/sources.sha256" "$EXECUTOR_SOURCES_SHA256" -run_postcheck executor-sources check_executor_sources -run_postcheck freeze-manifest check_digest "$SANDBOX_BUNDLE/freeze.sha256" "$SANDBOX_FREEZE_SHA256" -run_postcheck freeze-files check_frozen_files -run_postcheck python-binary check_digest "$SANDBOX_VENV/bin/python" "$PYTHON_SHA256" -run_postcheck bwrap-binary check_digest "$BWRAP_BIN" "$BWRAP_SHA256" -if [[ "$probe_status" != 0 ]]; then exit "$probe_status"; fi -if [[ "$post_status" != 0 ]]; then exit "$post_status"; fi -echo 'Full164 canonical resource pilot completed; generated-code executor remains unqualified.' -task_completed=true diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-full164-v1/runtime.py b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-full164-v1/runtime.py deleted file mode 100644 index 51f1903f3..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-full164-v1/runtime.py +++ /dev/null @@ -1,481 +0,0 @@ -"""Resolve pinned EvalPlus metadata, inventory readonly binds and run a trusted probe.""" - -import argparse -import configparser -import hashlib -import importlib.metadata -import json -import os -import re -import resource -import shutil -import signal -import subprocess -import sys -import sysconfig -import threading -import tomllib -from pathlib import Path - -REVISION = "26d6d00bb1fd0fa37f39c99d5290da67891d1c5e" - - -def canonical(value): - return json.dumps(value, sort_keys=True, separators=(",", ":"), allow_nan=False).encode() - - -def sha(raw): - return hashlib.sha256(raw).hexdigest() - - -def write(path, value): - with path.open("xb") as stream: - stream.write(canonical(value) + b"\n") - - -def git(source, *args): - return subprocess.check_output(["git", "-C", str(source), *args]) - - -def source_identity(bundle): - source = bundle / "source" - if git(source, "rev-parse", "HEAD").decode().strip() != REVISION: - raise ValueError("wrong EvalPlus source revision") - if git(source, "diff", "HEAD", "--"): - raise ValueError("tracked EvalPlus source changed") - return { - "evaluator_revision": REVISION, - "archive_sha256": sha(git(source, "archive", "--format=tar", "HEAD")), - } - - -def requirements(bundle): - identity = source_identity(bundle) - config = configparser.ConfigParser() - config.read(bundle / "source/setup.cfg") - dependencies = [ - line.strip() for line in config["options"]["install_requires"].splitlines() if line.strip() - ] - build = tomllib.loads((bundle / "source/pyproject.toml").read_text())["build-system"][ - "requires" - ] - (bundle / "dependencies.in").write_text("\n".join(dependencies) + "\n") - (bundle / "build.in").write_text("\n".join([*build, "wheel"]) + "\n") - write( - bundle / "source.json", - { - **identity, - "official_default_dependencies": dependencies, - "build_dependencies": [*build, "wheel"], - "extras_enabled": [], - "generated_code_executed": False, - }, - ) - - -def verify_source(bundle): - expected = json.loads((bundle / "source.json").read_bytes()) - if any( - source_identity(bundle)[name] != expected[name] - for name in ("evaluator_revision", "archive_sha256") - ): - raise ValueError("EvalPlus source snapshot drift") - - -def tree(root, alias): - result = [] - for path in sorted(root.rglob("*")): - relative = path.relative_to(root).as_posix() - name = alias + "/" + relative - if path.is_symlink(): - target = os.readlink(path) - if Path(target).is_absolute() or not path.resolve(strict=True).is_relative_to(root): - raise ValueError("runtime symlink escapes its readonly root") - result.append({"path": name, "type": "symlink", "target": target}) - elif path.is_dir(): - result.append({"path": name, "type": "directory", "mode": path.stat().st_mode & 0o777}) - elif path.is_file(): - result.append( - { - "path": name, - "type": "file", - "mode": path.stat().st_mode & 0o777, - "bytes": path.stat().st_size, - "sha256": sha(path.read_bytes()), - } - ) - else: - raise ValueError("unsupported runtime filesystem entry") - return result - - -def system_dependencies(roots): - # Mirror the explicit relocated loader directory; inherit no host loader state. - loader_env = { - "PATH": "/usr/bin:/bin", "LC_ALL": "C", - "LD_LIBRARY_PATH": str(roots[0] / "lib"), - } - ldd = shutil.which("ldd") - if ldd is None: - raise ValueError("system ELF dependency scanner unavailable") - dependencies = {} - for root in roots: - for path in sorted(root.rglob("*")): - if not path.is_file() or path.is_symlink(): - continue - with path.open("rb") as stream: - if stream.read(4) != b"\x7fELF": - continue - result = subprocess.run( - [ldd, str(path)], capture_output=True, timeout=10, check=False, env=loader_env - ) - output = result.stdout.decode(errors="replace") - if result.returncode != 0 or "not found" in output: - raise ValueError("ELF runtime dependency missing") - for line in output.splitlines(): - match = re.search(r"(?:=>\s*)?(/[^\s]+)", line) - if not match: - continue - original = Path(match.group(1)) - resolved = original.resolve(strict=True) - if any(resolved.is_relative_to(bound) for bound in roots): - continue - if not str(original).startswith(("/lib/", "/lib64/", "/usr/lib/", "/usr/lib64/")): - raise ValueError("unexpected external runtime library") - dependencies[str(original)] = str(resolved) - return dependencies - - -def inventory(bundle, output): - verify_source(bundle) - task_root = Path(os.environ["TASK_ROOT"]).resolve(strict=True) - base = Path(sys.base_prefix).resolve(strict=True) - site = Path(sysconfig.get_path("purelib")).resolve(strict=True) - if not base.is_relative_to(task_root) or not site.is_relative_to(task_root): - raise ValueError("runtime roots must remain inside TASK_ROOT") - executable = Path(sys._base_executable).resolve(strict=True).relative_to(base).as_posix() - bwrap = shutil.which("bwrap") - if bwrap is None: - raise ValueError("bubblewrap is unavailable on this compute node") - help_text = subprocess.check_output([bwrap, "--help"], stderr=subprocess.STDOUT).decode() - for flag in ( - "--unshare-user", - "--unshare-net", - "--unshare-pid", - "--disable-userns", - "--assert-userns-disabled", - "--clearenv", - "--remount-ro", - "--size", - ): - if flag not in help_text: - raise ValueError("required bubblewrap isolation option unavailable") - dependencies = system_dependencies((base, site)) - entries = tree(base, "/opt/python") + tree(site, "/opt/site-packages") - for destination, source in dependencies.items(): - path = Path(source) - entries.append( - { - "path": destination, - "type": "file", - "bytes": path.stat().st_size, - "sha256": sha(path.read_bytes()), - } - ) - probe = Path(__file__).with_name("synthetic_probe.py") - entries.append( - { - "path": "/opt/probe.py", - "type": "file", - "bytes": probe.stat().st_size, - "sha256": sha(probe.read_bytes()), - } - ) - entries.sort(key=lambda row: row["path"]) - # Bind-set identity describes actual readonly bytes, not an OCI image. - rootfs_digest = "sha256:" + sha(canonical(entries)) - packages = sorted( - [ - {"name": d.metadata["Name"], "version": d.version} - for d in importlib.metadata.distributions() - ], - key=lambda row: row["name"].lower(), - ) - # Match every pinned evaluator Python source byte to the installed package. - checked = 0 - source = bundle / "source" - for encoded in git(source, "ls-files", "-z", "evalplus").split(b"\0"): - if not encoded or not encoded.endswith(b".py"): - continue - name = encoded.decode() - if (source / name).read_bytes() != (site / name).read_bytes(): - raise ValueError("installed EvalPlus differs from pinned source") - checked += 1 - if checked == 0: - raise ValueError("no pinned evaluator Python sources verified") - output.mkdir(parents=True, exist_ok=False) - write( - output / "rootfs.json", - { - "schema": 1, - "rootfs_kind": "readonly-bind-set-v1", - "loader_policy": { - "scanner_inherits_environment": False, - "base_library_directory": "/opt/python/lib", - "sandbox_ld_library_path": "/opt/python/lib", - }, - "rootfs_digest": rootfs_digest, - "evaluator_revision": REVISION, - "entries": entries, - "python": sys.version.split()[0], - "packages": packages, - "checked_evaluator_sources": checked, - "executor": { - "kind": "bubblewrap", - "binary_sha256": sha(Path(bwrap).read_bytes()), - "version": subprocess.check_output([bwrap, "--version"]).decode().strip(), - }, - "generated_code_executed": False, - "qualified_generated_code_executor": False, - }, - ) - write( - output / "runtime-locators.private.json", - { - "python_root": str(base), - "site_root": str(site), - "python_relative": executable, - "bwrap": bwrap, - "system_libraries": dependencies, - }, - ) - - -def command(locators, probe, namespaces, sentinel): - args = [ - locators["bwrap"], - "--unshare-user", - "--uid", - "65534", - "--gid", - "65534", - "--unshare-net", - "--unshare-pid", - "--unshare-ipc", - "--unshare-uts", - "--hostname", - "sandbox", - "--disable-userns", - "--assert-userns-disabled", - "--cap-drop", - "ALL", - "--die-with-parent", - "--new-session", - "--clearenv", - "--ro-bind", - locators["python_root"], - "/opt/python", - "--ro-bind", - locators["site_root"], - "/opt/site-packages", - "--ro-bind", - str(probe), - "/opt/probe.py", - ] - for destination, source in sorted(locators["system_libraries"].items()): - args.extend(["--ro-bind", source, destination]) - args += [ - "--proc", - "/proc", - "--dev", - "/dev", - "--size", - "33554432", - "--tmpfs", - "/dev/shm", - "--size", - "67108864", - "--tmpfs", - "/work", - "--dir", - "/work/tmp", - "--dir", - "/work/output", - "--chdir", - "/work", - "--remount-ro", - "/", - ] - environment = { - "HOME": "/work", - "TMPDIR": "/work/tmp", - "PATH": "/opt/python/bin", - "LD_LIBRARY_PATH": "/opt/python/lib", - "PYTHONDONTWRITEBYTECODE": "1", - "OMP_NUM_THREADS": "1", - "OPENBLAS_NUM_THREADS": "1", - "LC_ALL": "C.UTF-8", - "PROBE_PARENT_NAMESPACES": json.dumps(namespaces), - "PROBE_UNMOUNTED_SENTINEL": str(sentinel), - } - for name, value in environment.items(): - args += ["--setenv", name, value] - return args + ["/opt/python/" + locators["python_relative"], "-I", "/opt/probe.py"] - - -def limits(): - os.sched_setaffinity(0, {min(os.sched_getaffinity(0))}) - for name, value in ( - ("AS", 2147483648), - ("CPU", 30), - ("NOFILE", 64), - ("FSIZE", 1048576), - ("CORE", 0), - ): - resource.setrlimit(getattr(resource, "RLIMIT_" + name), (value, value)) - - -def probe(inventory_path, output): - rootfs = json.loads((inventory_path / "rootfs.json").read_bytes()) - locators = json.loads((inventory_path / "runtime-locators.private.json").read_bytes()) - if sha(Path(locators["bwrap"]).read_bytes()) != rootfs["executor"]["binary_sha256"]: - raise ValueError("bubblewrap binary changed after inventory") - output.mkdir(parents=True, exist_ok=False) - sentinel = output / "unmounted-sentinel.private" - sentinel.write_text("synthetic host marker, not a credential") - namespaces = { - name: os.readlink("/proc/self/ns/" + name) for name in ("user", "net", "pid", "uts", "ipc") - } - args = command(locators, Path(__file__).with_name("synthetic_probe.py"), namespaces, sentinel) - timed_out = False - # Sample from the supervisor after Popen. The evaluator process remains free - # of an added observer thread before its own default multiprocessing fork. - import psutil - sampled = {"processes": 0, "threads": 0, "sum_vms_bytes": 0, - "sum_rss_bytes": 0, "sum_fds": 0, "samples": 0, "errors": 0} - known = {} - sampling_failures = [] - stop_sampling = threading.Event() - def observe(pid): - try: - parent = psutil.Process(pid) - except psutil.NoSuchProcess: - sampling_failures.append("launcher exited before sampling") - return - while not stop_sampling.is_set(): - totals = {"processes": 0, "threads": 0, "sum_vms_bytes": 0, - "sum_rss_bytes": 0, "sum_fds": 0} - try: - processes = [parent, *parent.children(recursive=True)] - except psutil.NoSuchProcess: - break - except psutil.AccessDenied: - sampling_failures.append("cannot inspect owned launcher descendants") - return - for child in processes: - try: - known[child.pid] = child.create_time() - memory = child.memory_info() - totals["processes"] += 1 - totals["threads"] += child.num_threads() - totals["sum_vms_bytes"] += memory.vms - totals["sum_rss_bytes"] += memory.rss - totals["sum_fds"] += child.num_fds() - except (psutil.NoSuchProcess, psutil.AccessDenied): - sampled["errors"] += 1 - for name, value in totals.items(): - sampled[name] = max(sampled[name], value) - sampled["samples"] += 1 - stop_sampling.wait(0.01) - with ( - (output / "stdout.json").open("xb") as stdout, - (output / "stderr.private.log").open("xb") as stderr, - ): - process = subprocess.Popen( - [sys.executable, str(Path(__file__).resolve()), "_exec", json.dumps(args)], - stdin=subprocess.DEVNULL, - stdout=stdout, - stderr=stderr, - start_new_session=True, - env={**os.environ, "PROBE_INHERITED_SECRET": "synthetic-must-be-cleared"}, - ) - observer = threading.Thread(target=observe, args=(process.pid,), daemon=True) - observer.start() - try: - code = process.wait(timeout=60) - except subprocess.TimeoutExpired: - timed_out = True - if os.getpgid(process.pid) != process.pid: - raise RuntimeError("owned sandbox process group changed unexpectedly") - os.killpg(process.pid, signal.SIGKILL) - code = process.wait(timeout=10) - finally: - stop_sampling.set() - observer.join(timeout=2) - remaining = [] - for pid, created in known.items(): - try: - child = psutil.Process(pid) - if child.create_time() == created and child.status() != psutil.STATUS_ZOMBIE: - remaining.append(pid) - except psutil.NoSuchProcess: - pass - result = { - "sampled_peaks": sampled, - "sampling_thread_joined": not observer.is_alive(), - "sampling_failures": sampling_failures, - "known_live_descendants_after_exit": remaining, - "sampling_limitation": "Polling may miss transient processes; sampled peaks are not hard bounds or exhaustive cleanup proof.", - "phase": "trusted_synthetic_evaluator", - "exit_code": code, - "timed_out": timed_out, - "rootfs_digest": rootfs["rootfs_digest"], - "executor": rootfs["executor"], - "policy_script_sha256": sha(Path(__file__).read_bytes()), - "probe_script_sha256": sha(Path(__file__).with_name("synthetic_probe.py").read_bytes()), - "stdout_sha256": sha((output / "stdout.json").read_bytes()), - "generated_code_executed": False, - "full_evaluator_attempt_requested": True, - "qualified_generated_code_executor": False, - "verdict": "INCONCLUSIVE", - } - write(output / "execution.json", result) - if code != 0 or timed_out or remaining or observer.is_alive() or sampling_failures or not sampled["samples"]: - raise ValueError("trusted synthetic evaluator probe failed; retain evidence without fallback") - payload = json.loads((output / "stdout.json").read_bytes()) - if payload.get("passed") is not True: - raise ValueError("trusted synthetic evaluator assertions failed") - - -def main(): - parser = argparse.ArgumentParser(description=__doc__) - parser.add_argument("phase", choices=("requirements", "verify-source", "inventory", "probe")) - parser.add_argument("--bundle", type=Path, required=True) - parser.add_argument("--output", type=Path) - parser.add_argument("--inventory", type=Path) - args = parser.parse_args() - if not os.environ.get("SLURM_JOB_ID") or os.environ.get("SLURM_GPUS_ON_NODE", "0") != "0": - raise ValueError("CPU-only Slurm allocation required") - if args.phase == "requirements": - requirements(args.bundle) - elif args.phase == "verify-source": - verify_source(args.bundle) - elif args.phase == "inventory": - inventory(args.bundle, args.output) - else: - probe(args.inventory, args.output) - - -if __name__ == "__main__": - if len(sys.argv) == 3 and sys.argv[1] == "_exec": - # A fresh interpreter sets limits before exec; no Python preexec callback - # runs between fork and exec in the monitoring supervisor. - if not os.environ.get("SLURM_JOB_ID") or os.environ.get("SLURM_GPUS_ON_NODE", "0") != "0": - raise ValueError("CPU-only Slurm allocation required") - command_args = json.loads(sys.argv[2]) - if not command_args or any(not isinstance(value, str) for value in command_args): - raise ValueError("invalid internal command") - limits() - os.execv(command_args[0], command_args) - else: - main() diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-full164-v1/sources.sha256 b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-full164-v1/sources.sha256 deleted file mode 100644 index a67f2f031..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-full164-v1/sources.sha256 +++ /dev/null @@ -1,12 +0,0 @@ -997cbfed8398e8ad62d1beb4f6b4bc6e61a0f4572b9156cfd0c6f3fb52b03233 runtime.py -bfeeb6ea794c0ebf24e72a0127458ff74951253471daee080b3a3b1d82a062ba bounds_driver.py -236920f27573447b9f3833afb72739033c965e22a9e406b09f28da815529ca64 synthetic_probe.py -080788af20e7132e7602570ec6eba620439e000a934cc6c15adcac19e31c0c12 contract.py -dd69193feb2b824b5bdc4d037ec63ede64c9fe809faaf78f974542b23935a19f inner.py -05e59d9928f06c47c235bc5e878c97d9c49260631ae020fd1b6baf851ea2d5b3 executor.py -3527f2fe807c1399df245d5a17b84bef0bdd16f21edcc7cf69b00ac895b3b902 data_binding.py -b5949144fd33dafd78a1ed89d23ba1e8ce1bc41775b2765d5fe5991930aae74f data-lock.json -ad0bb37a1898afee20acf1e828e62f32828cc5841cab43f884353d01738e0f8f exercise.py -415bf4636f9b717ed22a589fe6a08d88370aca6998f23ac5fb66251797f7d9e6 test_binding.py -0445dc9a7c281911da4701d4e6ecdd59f083a306aca6509fbb1817cd220af46a policy.json -078961e8190a9903b07e70af96f469f2429884fe9a882fdcdb559e90771b4391 run-executor.sbatch diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-full164-v1/synthetic_probe.py b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-full164-v1/synthetic_probe.py deleted file mode 100644 index 51cb12f67..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-full164-v1/synthetic_probe.py +++ /dev/null @@ -1,529 +0,0 @@ -"""Finite handwritten kernel-boundary probes; no model programs or task data.""" - -import ctypes -import errno -import json -import mmap -import os -import platform -import resource -import select -import signal -import subprocess -import sys -import threading -import time -from pathlib import Path - -TASK_LIMIT = 16 -MEMORY_LIMIT = 2147483648 -START_TOKEN = b"inferrsi-trusted-start-v2\n" -IPC_SYSCALLS = (29, 30, 31, 64, 65, 66, 67, 68, 69, 70, 71, 220, 240) - - -def authorize_start(fd=0, timeout=5): - """Require the supervisor's complete token and EOF before any dispatch.""" - deadline = time.monotonic() + timeout - received = bytearray() - while len(received) <= len(START_TOKEN): - remaining = deadline - time.monotonic() - if remaining <= 0 or not select.select([fd], [], [], remaining)[0]: - raise TimeoutError("startup token deadline exceeded") - part = os.read(fd, len(START_TOKEN) + 1 - len(received)) - if not part: - if bytes(received) != START_TOKEN: - raise ValueError("startup token absent or invalid") - return - received.extend(part) - raise ValueError("startup token too long") - - -def filter_instructions(): - # Linux v6.12 x86 syscall table; reject other architectures and the x32 ABI. - result = [ - (0x20, 0, 0, 4), - (0x15, 1, 0, 0xC000003E), - (0x06, 0, 0, 0x80000000), - (0x20, 0, 0, 0), - (0x35, 0, 1, 0x40000000), - (0x06, 0, 0, 0x80000000), - ] - for number in IPC_SYSCALLS: - result.extend([(0x15, 0, 1, number), (0x06, 0, 0, 0x00050000 | errno.EPERM)]) - return [*result, (0x06, 0, 0, 0x7FFF0000)] - - -def install_ipc_filter(): - if platform.machine() != "x86_64" or sys.maxsize != 2**63 - 1: - raise ValueError("trusted filter requires Linux x86_64 LP64") - - class Instruction(ctypes.Structure): - _fields_ = [ - ("code", ctypes.c_ushort), - ("jt", ctypes.c_ubyte), - ("jf", ctypes.c_ubyte), - ("k", ctypes.c_uint32), - ] - - class Program(ctypes.Structure): - _fields_ = [ - ("length", ctypes.c_ushort), - ("instructions", ctypes.POINTER(Instruction)), - ] - - instructions = filter_instructions() - array = (Instruction * len(instructions))( - *(Instruction(*row) for row in instructions) - ) - program = Program(len(instructions), array) - libc = ctypes.CDLL(None, use_errno=True) - libc.prctl.restype = ctypes.c_int - if ( - libc.prctl(38, 1, 0, 0, 0) != 0 - or libc.prctl(22, 2, ctypes.byref(program), 0, 0) != 0 - ): - raise OSError(ctypes.get_errno(), "fixed seccomp policy could not be installed") - - -def ipc_attempts(): - libc = ctypes.CDLL(None, use_errno=True) - libc.syscall.restype = ctypes.c_long - attempts = [] - # Finite minimal allocations only, even if the expected rejection is absent. - for number, arguments in ( - (29, (0, 4096, 0o600)), - (64, (0, 1, 0o600)), - (68, (0, 0o600)), - ): - ctypes.set_errno(0) - value = libc.syscall( - ctypes.c_long(number), *(ctypes.c_long(v) for v in arguments) - ) - attempts.append( - {"syscall": number, "return": value, "errno": ctypes.get_errno()} - ) - ctypes.set_errno(0) - value = libc.syscall( - ctypes.c_long(240), - ctypes.c_char_p(b"trusted-probe"), - ctypes.c_long(os.O_CREAT | os.O_WRONLY), - ctypes.c_long(0o600), - ctypes.c_void_p(), - ) - attempts.append({"syscall": 240, "return": value, "errno": ctypes.get_errno()}) - return attempts - - -def emit(value): - print(json.dumps(value, sort_keys=True), flush=True) - - -def limits_record(): - return { - name: list(resource.getrlimit(getattr(resource, "RLIMIT_" + name))) - for name in ( - "AS", - "CPU", - "NOFILE", - "FSIZE", - "CORE", - "NPROC", - "MEMLOCK", - "MSGQUEUE", - ) - } - - -def finite_forks(): - children = [] - read_fd, write_fd = os.pipe() - rejected = None - try: - for _ in range(TASK_LIMIT + 1): - try: - child = os.fork() - except OSError as error: - rejected = error.errno - break - if child == 0: - os.close(write_fd) - signal.alarm(8) - os.read(read_fd, 1) - os._exit(0) - children.append(child) - finally: - os.close(write_fd) - os.close(read_fd) - for child in children: - os.waitpid(child, 0) - return { - "created": len(children), - "rejection_errno": rejected, - "attempt_cap": TASK_LIMIT + 1, - } - - -def finite_threads(): - threads = [] - stop = threading.Event() - rejected = None - try: - for _ in range(TASK_LIMIT + 1): - thread = threading.Thread(target=stop.wait, args=(8,)) - try: - thread.start() - except RuntimeError as error: - rejected = str(error) - break - threads.append(thread) - finally: - stop.set() - for thread in threads: - thread.join(timeout=2) - return { - "created": len(threads), - "rejection": rejected, - "all_joined": all(not thread.is_alive() for thread in threads), - "attempt_cap": TASK_LIMIT + 1, - } - - -def bounds(): - before = limits_record() - raises = {} - for name in ( - "NPROC", - "AS", - "CPU", - "NOFILE", - "FSIZE", - "CORE", - "MEMLOCK", - "MSGQUEUE", - ): - code = getattr(resource, "RLIMIT_" + name) - soft, hard = resource.getrlimit(code) - try: - resource.setrlimit(code, (soft, hard + 1)) - except (ValueError, PermissionError) as error: - raises[name] = type(error).__name__ - else: - raises[name] = "unexpected_success" - try: - mapping = mmap.mmap(-1, MEMORY_LIMIT + mmap.PAGESIZE) - except OSError as error: - as_errno = error.errno - else: - mapping.close() - as_errno = None - inherited = subprocess.run( - [sys.executable, "-I", "-B", "/opt/probe.py", "inherit"], - input=START_TOKEN, - capture_output=True, - timeout=5, - check=False, - ) - inherited_payload = json.loads(inherited.stdout) - forks = finite_forks() - threads = finite_threads() - descriptors = [] - fd_errno = None - try: - for _ in range(65): - try: - descriptors.append(os.open("/dev/null", os.O_RDONLY)) - except OSError as error: - fd_errno = error.errno - break - finally: - for fd in descriptors: - os.close(fd) - old_handler = signal.signal(signal.SIGXFSZ, signal.SIG_IGN) - try: - with open("/work/size-check", "wb") as stream: - try: - stream.truncate(1048577) - except OSError as error: - size_errno = error.errno - else: - size_errno = None - finally: - signal.signal(signal.SIGXFSZ, old_handler) - cpu = subprocess.run( - [sys.executable, "-I", "-B", "/opt/probe.py", "cpu"], - input=START_TOKEN, - capture_output=True, - timeout=5, - check=False, - ) - status = dict( - line.split(":", 1) - for line in Path("/proc/self/status").read_text().splitlines() - if ":" in line - ) - expected = { - "AS": [MEMORY_LIMIT] * 2, - "CPU": [30] * 2, - "NOFILE": [64] * 2, - "FSIZE": [1048576] * 2, - "CORE": [0, 0], - "NPROC": [TASK_LIMIT] * 2, - "MEMLOCK": [0, 0], - "MSGQUEUE": [0, 0], - } - ipc = ipc_attempts() - try: - with open("/dev/creation-check", "xb"): - pass - except OSError as error: - dev_errno = error.errno - else: - dev_errno = None - with open("/dev/zero", "rb") as stream: - zero = stream.read(4) - checks = { - "exact_limits": before == expected and limits_record() == expected, - "nonroot": os.getuid() == os.geteuid() == 65534, - "no_effective_capabilities": int(status["CapEff"].strip(), 16) == 0, - "no_new_privileges": status["NoNewPrivs"].strip() == "1", - "seccomp_filter_present": status["Seccomp"].strip() == "2", - "sysv_creation_denied": all( - row["return"] == -1 and row["errno"] == errno.EPERM for row in ipc - ), - "device_mount_readonly": dev_errno == errno.EROFS, - "device_read_available": zero == b"\x00" * 4, - "raise_hard_limit_denied": all( - value != "unexpected_success" for value in raises.values() - ), - "address_space_denied": as_errno == errno.ENOMEM, - "fork_exec_inheritance": inherited.returncode == 0 - and inherited_payload["limits"] == expected, - "fork_exec_filter_inheritance": inherited.returncode == 0 - and all( - row["return"] == -1 and row["errno"] == errno.EPERM - for row in inherited_payload["ipc"] - ), - "bounded_fork_denied": 0 < forks["created"] < TASK_LIMIT - and forks["rejection_errno"] == errno.EAGAIN, - "bounded_thread_denied": 0 < threads["created"] < TASK_LIMIT - and threads["rejection"] is not None - and threads["all_joined"], - "descriptor_limit_denied": fd_errno == errno.EMFILE, - "file_size_denied": size_errno == errno.EFBIG, - "cpu_limit_kills": cpu.returncode == -signal.SIGKILL, - } - emit( - { - "mode": "bounds", - "checks": checks, - "passed": all(checks.values()), - "limits": before, - "hard_limit_raise_results": raises, - "as_errno": as_errno, - "forks": forks, - "threads": threads, - "descriptor_errno": fd_errno, - "opened_descriptors": len(descriptors), - "file_size_errno": size_errno, - "cpu_exit_code": cpu.returncode, - "ipc_attempts": ipc, - "device_creation_errno": dev_errno, - "inherited_exit_code": inherited.returncode, - "generated_code_executed": False, - } - ) - return 0 if all(checks.values()) else 2 - - -def cleanup(mode): - read_fd, write_fd = os.pipe() - direct = os.fork() - if direct == 0: - os.close(read_fd) - os.write(write_fd, f"{os.getpid()}\n".encode()) - os.close(write_fd) - time.sleep(15) - os._exit(0) - intermediate = os.fork() - if intermediate == 0: - os.close(read_fd) - os.setsid() - grandchild = os.fork() - if grandchild == 0: - os.write(write_fd, f"{os.getpid()}\n".encode()) - os.close(write_fd) - time.sleep(15) - os._exit(0) - os._exit(0) - os.close(write_fd) - with os.fdopen(read_fd, "r") as stream: - live = sorted(int(line) for line in stream) - os.waitpid(intermediate, 0) - emit( - { - "mode": mode, - "ready": True, - "probe_pid": os.getpid(), - "live_descendant_pids": live, - "direct_pid": direct, - "namespace_pid": os.readlink("/proc/self/ns/pid"), - "generated_code_executed": False, - } - ) - if mode == "normal": - # Allow the supervisor to open pidfds for both deliberately live children. - time.sleep(2) - return 0 - time.sleep(15) - return 3 - - -def synthetic_evaluator(): - import base64 - import contextlib - import hashlib - import importlib - import io - - sys.path.insert(0, "/opt/site-packages") - import psutil - - def digest(raw): - return hashlib.sha256(raw).hexdigest() - - os.environ["EVALPLUS_MAX_MEMORY_BYTES"] = str(MEMORY_LIMIT) - os.environ.pop("EVALPLUS_TIMEOUT_PER_TASK", None) - os.environ["XDG_CACHE_HOME"] = "/work/cache" - os.environ["HUMANEVAL_OVERRIDE_PATH"] = "/work/synthetic-dataset.jsonl" - work = Path("/work") - samples = work / "synthetic-samples.jsonl" - dataset = work / "synthetic-dataset.jsonl" - output = work / "output/synthetic-results.json" - assert not samples.exists() and not dataset.exists() and not output.exists() - assert not (work / "cache").exists() - solutions = { - "Synthetic/correct": "def f(x):\n return x + 1\n", - "Synthetic/plus_wrong": "def f(x):\n return 1\n", - "Synthetic/exception": "def f(x):\n raise ArithmeticError('synthetic')\n", - "Synthetic/input_timeout": "import time\ndef f(x):\n time.sleep(5)\n return x + 1\n", - } - rows = [ - { - "task_id": task_id, - "prompt": "def f(x):\n", - "canonical_solution": " return x + 1\n", - "contract": "", - "entry_point": "f", - "base_input": [[0]], - "plus_input": [[1]], - "atol": 0, - } - for task_id in solutions - ] - dataset.write_text("".join(json.dumps(row, sort_keys=True) + "\n" for row in rows)) - samples.write_text( - "".join( - json.dumps({"task_id": key, "solution": value}, sort_keys=True) + "\n" - for key, value in solutions.items() - ) - ) - parent = psutil.Process() - captured_out, captured_err = io.StringIO(), io.StringIO() - started = time.monotonic() - with ( - contextlib.redirect_stdout(captured_out), - contextlib.redirect_stderr(captured_err), - ): - evaluator = importlib.import_module("evalplus.evaluate") - evaluator.evaluate( - dataset="humaneval", - samples=str(samples), - output_file=str(output), - parallel=1, - base_only=False, - mini=False, - noextreme=False, - test_details=True, - ) - result = json.loads(output.read_bytes()) - statuses = { - task_id: [[row["base_status"], row["plus_status"]] for row in values] - for task_id, values in result["eval"].items() - } - # The upstream per-input exception handler reports time-limit exceptions as - # fail. This is distinct from its outer child-process timeout classification. - expected = { - "Synthetic/correct": [["pass", "pass"]], - "Synthetic/plus_wrong": [["pass", "fail"]], - "Synthetic/exception": [["fail", "fail"]], - "Synthetic/input_timeout": [["fail", "fail"]], - } - checks = { - "four_exact_status_pairs": statuses == expected, - "dataset_hash": result["hash"] == hashlib.md5(dataset.read_bytes()).hexdigest(), - "bounded_output": output.stat().st_size <= 1048576, - "timeout_override_absent": "EVALPLUS_TIMEOUT_PER_TASK" not in os.environ, - } - payload = { - "checks": checks, - "passed": all(checks.values()), - "statuses": statuses, - "elapsed_seconds": time.monotonic() - started, - "dataset_sha256": digest(dataset.read_bytes()), - "samples_sha256": digest(samples.read_bytes()), - "results_sha256": digest(output.read_bytes()), - "results": result, - "results_raw_base64": base64.b64encode(output.read_bytes()).decode("ascii"), - "evaluator_module_sha256": digest(Path(evaluator.__file__).read_bytes()), - "evaluator_stdout": captured_out.getvalue(), - "evaluator_stderr": captured_err.getvalue(), - "live_children_after_evaluate": len(parent.children(recursive=True)), - "generated_code_executed": False, - "real_benchmark_data_mounted": False, - "full_evaluator_executed": True, - "qualified_generated_code_executor": False, - } - print(json.dumps(payload, sort_keys=True)) - return 0 if payload["passed"] else 2 - - -def main(): - try: - authorize_start() - except (ValueError, OSError): - os.write(2, b"PAYLOAD_START_REJECTED\n") - return 23 - # The authorization pipe is never available to evaluator code or children. - devnull = os.open("/dev/null", os.O_RDONLY) - os.dup2(devnull, 0) - if devnull != 0: - os.close(devnull) - os.write(2, b"PAYLOAD_START_AUTHORIZED\n") - mode = sys.argv[1] - if mode == "gate_check": - emit({"passed": True, "mode": "gate_check"}) - return 0 - if mode == "inherit": - emit({"limits": limits_record(), "ipc": ipc_attempts()}) - return 0 - if mode == "cpu": - resource.setrlimit(resource.RLIMIT_CPU, (1, 1)) - signal.alarm(4) - while True: - pass - resource.setrlimit(resource.RLIMIT_NPROC, (TASK_LIMIT, TASK_LIMIT)) - resource.setrlimit(resource.RLIMIT_MEMLOCK, (0, 0)) - resource.setrlimit(resource.RLIMIT_MSGQUEUE, (0, 0)) - install_ipc_filter() - signal.alarm(40) - if mode == "synthetic": - return synthetic_evaluator() - if mode == "bounds": - return bounds() - if mode in ("normal", "timeout", "launcher_kill"): - return cleanup(mode) - raise ValueError("unknown trusted probe mode") - - -if __name__ == "__main__": - raise SystemExit(main()) diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-full164-v1/test_binding.py b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-full164-v1/test_binding.py deleted file mode 100644 index 0825897f1..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-full164-v1/test_binding.py +++ /dev/null @@ -1,111 +0,0 @@ -"""Host-only binding/capacity controls; never execute a solution.""" - -import gzip -import json -import sys -import tempfile -import unittest -from pathlib import Path -from unittest.mock import patch - -sys.path.insert(0, str(Path(__file__).resolve().parent)) -import contract -import data_binding - - -class BindingTests(unittest.TestCase): - def setUp(self): - self.dataset = ( - json.dumps( - { - "task_id": "Synthetic/one", - "prompt": "def f():\n", - "canonical_solution": " return 1\n", - "base_input": [[1]], - "plus_input": [[2, "\u0000😀"]], - }, - ensure_ascii=False, - ) - + "\n" - ).encode() - self.compressed = gzip.compress(self.dataset) - self.lock = { - "gzip_sha256": contract.sha(self.compressed), - "dataset_sha256": contract.sha(self.dataset), - "dataset_bytes": len(self.dataset), - "task_count": 1, - "cohort_sha256": contract.sha(contract.canonical(["Synthetic/one"])), - } - self.temp = tempfile.TemporaryDirectory() - self.addCleanup(self.temp.cleanup) - self.root = Path(self.temp.name) - self.gz = self.root / "data.gz" - self.gz.write_bytes(self.compressed) - - def payload(self): - with patch.object(data_binding, "lock", return_value=self.lock): - identity = data_binding.create(self.gz, self.root / "input") - return contract.inputs(self.root / "input", **identity, maximum=67108864) - - def test_complete_binding(self): - raw, export, samples = self.payload() - with patch.object(data_binding, "lock", return_value=self.lock): - data_binding.validate(raw, export, samples) - - def test_gzip_drift(self): - self.gz.write_bytes(b"wrong") - with ( - patch.object(data_binding, "lock", return_value=self.lock), - self.assertRaisesRegex(ValueError, "compressed dataset"), - ): - data_binding.create(self.gz, self.root / "input") - - def test_dataset_drift(self): - raw, export, samples = self.payload() - raw["humaneval-subset.jsonl"] += b"\n" - with ( - patch.object(data_binding, "lock", return_value=self.lock), - self.assertRaisesRegex(ValueError, "full dataset"), - ): - data_binding.validate(raw, export, samples) - - def test_model_solution_rejected(self): - raw, export, samples = self.payload() - raw["humaneval-solutions.jsonl"] = raw["humaneval-solutions.jsonl"].replace( - b"return 1", b"return 2" - ) - with ( - patch.object(data_binding, "lock", return_value=self.lock), - self.assertRaisesRegex(ValueError, "canonical references"), - ): - data_binding.validate(raw, export, samples) - - def test_missing_task_rejected(self): - raw, export, samples = self.payload() - with ( - patch.object(data_binding, "lock", return_value=self.lock), - self.assertRaisesRegex(ValueError, "full cohort"), - ): - data_binding.validate(raw, export, {}) - - def test_capacity_includes_escaping_and_failure_details(self): - report = data_binding.capacity(self.dataset, 67108864) - self.assertEqual(report["solution_serialization_bound_bytes"], 402653184) - self.assertGreater(report["result_upper_bound_bytes"], 402653184) - self.assertLess(report["result_upper_bound_bytes"], 536870912) - - def test_policy_preserves_official_defaults(self): - path = Path(__file__).parent / "policy.json" - selected = contract.policy(path, contract.sha(path.read_bytes())) - self.assertGreater(selected["cpu_seconds"], selected["wall_seconds"]) - self.assertEqual(selected["evalplus"]["memory_override"], None) - self.assertEqual(selected["evalplus"]["timeout_override"], None) - self.assertIs(selected["qualified"], False) - - def test_duplicate_json_key_rejected(self): - with self.assertRaisesRegex(ValueError, "duplicate JSON"): - contract.decode(b'{"task_id":"x","task_id":"y"}') - - -if __name__ == "__main__": - unittest.main() diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-install-retry.sbatch b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-install-retry.sbatch deleted file mode 100644 index f0beba259..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-install-retry.sbatch +++ /dev/null @@ -1,139 +0,0 @@ -#!/usr/bin/env bash -# Install a reviewed dependency freeze and run only the trusted benign probe. -#SBATCH --job-name=qwen38-eval-sandbox-install -#SBATCH --nodes=1 -#SBATCH --ntasks=1 -#SBATCH --cpus-per-task=4 -#SBATCH --mem=16G -#SBATCH --time=00:30:00 -set -euo pipefail -umask 077 -: "${TASK_ROOT:?}" -: "${BASE_PYTHON:?}" -: "${UV_BIN:?}" -: "${SANDBOX_BUNDLE:?}" -: "${SANDBOX_FREEZE_SHA256:?}" -: "${SANDBOX_VENV:?}" -: "${INSTALL_SOURCE_DIR:?}" -: "${INSTALL_SCRIPT_SHA256:?}" -: "${PACKAGE_AUDIT_SHA256:?}" -task_uv_executable=$UV_BIN -[[ -n ${SLURM_JOB_ID:-} && ${SLURM_GPUS_ON_NODE:-0} == 0 ]] -root=$(realpath -e "$TASK_ROOT") -check_path() { - local value - value=$(realpath -m "$1") - case "$value" in "$root"/*) ;; *) echo 'Path escapes task root.' >&2; exit 2;; esac -} -for path in "$BASE_PYTHON" "$task_uv_executable" "$SANDBOX_BUNDLE" "$SANDBOX_VENV" "$INSTALL_SOURCE_DIR"; do - check_path "$path" -done -source_dir=$(cd -- "$INSTALL_SOURCE_DIR" && pwd -P) -check_digest() { - if [[ ! "$2" =~ ^[a-f0-9]{64}$ ]]; then return 2; fi - [[ $(sha256sum "$1" | cut -d' ' -f1) == "$2" ]] -} -check_digest "$source_dir/install-reviewed.sbatch" "$INSTALL_SCRIPT_SHA256" -check_digest "$source_dir/audit_packages.py" "$PACKAGE_AUDIT_SHA256" -cmp "${BASH_SOURCE[0]}" "$source_dir/install-reviewed.sbatch" -check_digest "$SANDBOX_BUNDLE/freeze.sha256" "$SANDBOX_FREEZE_SHA256" -(cd "$SANDBOX_BUNDLE" && sha256sum --check --strict freeze.sha256) -check_digest "$task_uv_executable" "$(cat "$SANDBOX_BUNDLE/uv-sha256.txt")" -check_digest "$BASE_PYTHON" "$(cat "$SANDBOX_BUNDLE/python-sha256.txt")" -for path in "$SANDBOX_BUNDLE/runtime.py" "$SANDBOX_BUNDLE/benign_probe.py" \ - "$SANDBOX_BUNDLE/source" "$SANDBOX_BUNDLE/wheelhouse"; do check_path "$path"; done -[[ ! -e "$SANDBOX_VENV" && ! -L "$SANDBOX_VENV" ]] -for task_resolver_variable in "${!UV_@}" "${!PIP_@}"; do - if [[ -n "$task_resolver_variable" ]]; then unset "$task_resolver_variable"; fi -done -export TMPDIR="$root/tmp/qwen38-eval-sandbox-install-${SLURM_JOB_ID}" -export XDG_CACHE_HOME="$root/cache/qwen38-eval-sandbox" -export UV_CACHE_DIR="$XDG_CACHE_HOME/uv" -export HF_HOME="$XDG_CACHE_HOME/huggingface" -export TORCH_HOME="$XDG_CACHE_HOME/torch" -export TRITON_CACHE_DIR="$XDG_CACHE_HOME/triton" -export VLLM_CACHE_ROOT="$XDG_CACHE_HOME/vllm" -export UV_PYTHON_INSTALL_DIR="$root/software/python" -export UV_PYTHON_DOWNLOADS=never -export UV_HTTP_TIMEOUT=300 -export UV_CONCURRENT_DOWNLOADS=2 -export PYTHONNOUSERSITE=1 -export PYTHONDONTWRITEBYTECODE=1 -unset PYTHONPATH PYTHONHOME PYTHONOPTIMIZE PYTHONINSPECT PYTHONSTARTUP VIRTUAL_ENV -for path in "$TMPDIR" "$XDG_CACHE_HOME" "$UV_CACHE_DIR" "$HF_HOME" "$TORCH_HOME" \ - "$TRITON_CACHE_DIR" "$VLLM_CACHE_ROOT" "$UV_PYTHON_INSTALL_DIR"; do check_path "$path"; done -mkdir -p "$TMPDIR" "$UV_CACHE_DIR" "$HF_HOME" "$TORCH_HOME" "$TRITON_CACHE_DIR" "$VLLM_CACHE_ROOT" -cd "$root" -output="$SANDBOX_BUNDLE/installation-v2" -check_path "$output" -mkdir "$output" -# BEGIN TERMINAL RECORD: an EXIT code alone never authenticates completion. -task_completed=false -task_signal=none -finish_installation() { - local status=$? - trap - EXIT - if [[ "$status" == 0 && "$task_completed" != true ]]; then status=1; fi - if [[ "$status" != 0 ]]; then task_completed=false; fi - printf '%s\n' "$status" > "$output/exit-code" - printf '{"completed":%s,"signal":"%s","exit_code":%s,"job_id":"%s"}\n' \ - "$task_completed" "$task_signal" "$status" "$SLURM_JOB_ID" > "$output/terminal-status.json" - exit "$status" -} -trap finish_installation EXIT -trap 'task_completed=false; task_signal=TERM; exit 143' TERM -trap 'task_completed=false; task_signal=INT; exit 130' INT -trap 'task_completed=false; task_signal=HUP; exit 129' HUP -# END TERMINAL RECORD -cp "$source_dir/install-reviewed.sbatch" "$source_dir/audit_packages.py" "$output/" -"$BASE_PYTHON" "$SANDBOX_BUNDLE/runtime.py" verify-source --bundle "$SANDBOX_BUNDLE" -[[ -z $(git -C "$SANDBOX_BUNDLE/source" ls-files --others --exclude-standard) ]] -"$task_uv_executable" --no-config venv --python "$BASE_PYTHON" "$SANDBOX_VENV" -"$task_uv_executable" --no-config pip install --python "$SANDBOX_VENV/bin/python" \ - --no-deps --no-build --require-hashes --default-index https://pypi.org/simple -r "$SANDBOX_BUNDLE/build.lock" -"$task_uv_executable" --no-config pip install --python "$SANDBOX_VENV/bin/python" \ - --no-deps --no-build --require-hashes --default-index https://pypi.org/simple \ - --find-links "$SANDBOX_BUNDLE/wheelhouse" -r "$SANDBOX_BUNDLE/dependencies.lock" -# Both reviewed lockfiles contain the full closure; no implicit dependencies. -"$SANDBOX_VENV/bin/python" -IB "$source_dir/audit_packages.py" \ - "$SANDBOX_BUNDLE/build.lock" "$SANDBOX_BUNDLE/dependencies.lock" dependencies \ - "$output/dependencies-verified.json" -# Build only the authenticated EvalPlus checkout using the already locked tools. -"$task_uv_executable" --no-config --offline pip install --python "$SANDBOX_VENV/bin/python" \ - --no-deps --no-build-isolation "$SANDBOX_BUNDLE/source" -"$SANDBOX_VENV/bin/python" -IB "$source_dir/audit_packages.py" \ - "$SANDBOX_BUNDLE/build.lock" "$SANDBOX_BUNDLE/dependencies.lock" evaluator \ - "$output/evaluator-packages-verified.json" -"$task_uv_executable" --no-config pip check --python "$SANDBOX_VENV/bin/python" -"$task_uv_executable" --no-config pip freeze --python "$SANDBOX_VENV/bin/python" > "$output/installed.private.txt" -"$BASE_PYTHON" "$SANDBOX_BUNDLE/runtime.py" verify-source --bundle "$SANDBOX_BUNDLE" -"$SANDBOX_VENV/bin/python" "$SANDBOX_BUNDLE/runtime.py" inventory --bundle "$SANDBOX_BUNDLE" \ - --output "$output/rootfs-before" -probe_status=0 -"$SANDBOX_VENV/bin/python" "$SANDBOX_BUNDLE/runtime.py" probe --bundle "$SANDBOX_BUNDLE" \ - --inventory "$output/rootfs-before" --output "$output/probe" || probe_status=$? -printf '%s\n' "$probe_status" > "$output/probe-status" -post_status=0 -run_postcheck() { - local name=$1 status=0 - shift - "$@" || status=$? - printf '%s\t%s\n' "$name" "$status" >> "$output/postflight-status.tsv" - if [[ "$status" != 0 ]]; then post_status=$status; fi -} -check_frozen_files() { (cd "$SANDBOX_BUNDLE" && sha256sum --check --strict freeze.sha256); } -run_postcheck inventory "$SANDBOX_VENV/bin/python" "$SANDBOX_BUNDLE/runtime.py" inventory \ - --bundle "$SANDBOX_BUNDLE" --output "$output/rootfs-after" -run_postcheck runtime-byte-comparison cmp "$output/rootfs-before/rootfs.json" "$output/rootfs-after/rootfs.json" -run_postcheck source "$BASE_PYTHON" "$SANDBOX_BUNDLE/runtime.py" verify-source --bundle "$SANDBOX_BUNDLE" -run_postcheck package-set "$SANDBOX_VENV/bin/python" -IB "$source_dir/audit_packages.py" \ - "$SANDBOX_BUNDLE/build.lock" "$SANDBOX_BUNDLE/dependencies.lock" evaluator \ - "$output/evaluator-packages-after.json" -run_postcheck batch-script check_digest "$source_dir/install-reviewed.sbatch" "$INSTALL_SCRIPT_SHA256" -run_postcheck audit-script check_digest "$source_dir/audit_packages.py" "$PACKAGE_AUDIT_SHA256" -run_postcheck freeze-manifest check_digest "$SANDBOX_BUNDLE/freeze.sha256" "$SANDBOX_FREEZE_SHA256" -run_postcheck freeze-files check_frozen_files -if [[ "$probe_status" != 0 ]]; then exit "$probe_status"; fi -if [[ "$post_status" != 0 ]]; then exit "$post_status"; fi -echo 'Trusted benign sandbox preparation completed; generated-code executor remains unqualified.' -task_completed=true diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-install-timeout.sbatch b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-install-timeout.sbatch deleted file mode 100644 index 4136e4c65..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-install-timeout.sbatch +++ /dev/null @@ -1,106 +0,0 @@ -#!/usr/bin/env bash -# Install a reviewed dependency freeze and run only the trusted benign probe. -#SBATCH --job-name=qwen38-eval-sandbox-install -#SBATCH --nodes=1 -#SBATCH --ntasks=1 -#SBATCH --cpus-per-task=4 -#SBATCH --mem=16G -#SBATCH --time=00:30:00 -set -euo pipefail -umask 077 -: "${TASK_ROOT:?}" -: "${BASE_PYTHON:?}" -: "${UV_BIN:?}" -: "${SANDBOX_BUNDLE:?}" -: "${SANDBOX_FREEZE_SHA256:?}" -: "${SANDBOX_VENV:?}" -: "${INSTALL_SOURCE_DIR:?}" -: "${INSTALL_SCRIPT_SHA256:?}" -task_uv_executable=$UV_BIN -[[ -n ${SLURM_JOB_ID:-} && ${SLURM_GPUS_ON_NODE:-0} == 0 ]] -root=$(realpath -e "$TASK_ROOT") -check_path() { - local value - value=$(realpath -m "$1") - case "$value" in "$root"/*) ;; *) echo 'Path escapes task root.' >&2; exit 2;; esac -} -for path in "$BASE_PYTHON" "$task_uv_executable" "$SANDBOX_BUNDLE" "$SANDBOX_VENV" "$INSTALL_SOURCE_DIR"; do - check_path "$path" -done -source_dir=$(cd -- "$INSTALL_SOURCE_DIR" && pwd -P) -check_digest() { - [[ "$2" =~ ^[a-f0-9]{64}$ ]] - [[ $(sha256sum "$1" | cut -d' ' -f1) == "$2" ]] -} -check_digest "$source_dir/install-reviewed.sbatch" "$INSTALL_SCRIPT_SHA256" -cmp "${BASH_SOURCE[0]}" "$source_dir/install-reviewed.sbatch" -check_digest "$SANDBOX_BUNDLE/freeze.sha256" "$SANDBOX_FREEZE_SHA256" -(cd "$SANDBOX_BUNDLE" && sha256sum --check --strict freeze.sha256) -check_digest "$task_uv_executable" "$(cat "$SANDBOX_BUNDLE/uv-sha256.txt")" -check_digest "$BASE_PYTHON" "$(cat "$SANDBOX_BUNDLE/python-sha256.txt")" -for path in "$SANDBOX_BUNDLE/runtime.py" "$SANDBOX_BUNDLE/benign_probe.py" \ - "$SANDBOX_BUNDLE/source" "$SANDBOX_BUNDLE/wheelhouse"; do check_path "$path"; done -[[ ! -e "$SANDBOX_VENV" && ! -L "$SANDBOX_VENV" ]] -for task_resolver_variable in "${!UV_@}" "${!PIP_@}"; do - if [[ -n "$task_resolver_variable" ]]; then unset "$task_resolver_variable"; fi -done -export TMPDIR="$root/tmp/qwen38-eval-sandbox-install-${SLURM_JOB_ID}" -export XDG_CACHE_HOME="$root/cache/qwen38-eval-sandbox" -export UV_CACHE_DIR="$XDG_CACHE_HOME/uv" -export HF_HOME="$XDG_CACHE_HOME/huggingface" -export TORCH_HOME="$XDG_CACHE_HOME/torch" -export TRITON_CACHE_DIR="$XDG_CACHE_HOME/triton" -export VLLM_CACHE_ROOT="$XDG_CACHE_HOME/vllm" -export UV_PYTHON_INSTALL_DIR="$root/software/python" -export UV_PYTHON_DOWNLOADS=never -export UV_HTTP_TIMEOUT=60 -export PYTHONNOUSERSITE=1 -export PYTHONDONTWRITEBYTECODE=1 -unset PYTHONPATH VIRTUAL_ENV -for path in "$TMPDIR" "$XDG_CACHE_HOME" "$UV_CACHE_DIR" "$HF_HOME" "$TORCH_HOME" \ - "$TRITON_CACHE_DIR" "$VLLM_CACHE_ROOT" "$UV_PYTHON_INSTALL_DIR"; do check_path "$path"; done -mkdir -p "$TMPDIR" "$UV_CACHE_DIR" "$HF_HOME" "$TORCH_HOME" "$TRITON_CACHE_DIR" "$VLLM_CACHE_ROOT" -cd "$root" -output="$SANDBOX_BUNDLE/installation-v1" -check_path "$output" -mkdir "$output" -trap 'status=$?; printf "%s\n" "$status" > "$output/exit-code"; exit "$status"' EXIT -cp "$source_dir/install-reviewed.sbatch" "$output/" -"$BASE_PYTHON" "$SANDBOX_BUNDLE/runtime.py" verify-source --bundle "$SANDBOX_BUNDLE" -[[ -z $(git -C "$SANDBOX_BUNDLE/source" ls-files --others --exclude-standard) ]] -"$task_uv_executable" --no-config venv --python "$BASE_PYTHON" "$SANDBOX_VENV" -"$task_uv_executable" --no-config pip install --python "$SANDBOX_VENV/bin/python" \ - --no-build --require-hashes --default-index https://pypi.org/simple -r "$SANDBOX_BUNDLE/build.lock" -"$task_uv_executable" --no-config pip install --python "$SANDBOX_VENV/bin/python" \ - --no-build --require-hashes --default-index https://pypi.org/simple \ - --find-links "$SANDBOX_BUNDLE/wheelhouse" -r "$SANDBOX_BUNDLE/dependencies.lock" -# A runtime dependency must not silently replace the reviewed build backend. -"$SANDBOX_VENV/bin/python" - "$SANDBOX_BUNDLE/build.lock" "$output/build-tools-verified.json" <<'PY' -import importlib.metadata -import json -import re -import sys -from pathlib import Path -expected = dict(re.findall(r"^([A-Za-z0-9_.-]+)==([^\s\\]+)", Path(sys.argv[1]).read_text(), re.M)) -assert expected, "No build-tool pins found" -actual = {name: importlib.metadata.version(name) for name in expected} -Path(sys.argv[2]).write_text(json.dumps({"expected": expected, "actual": actual}, sort_keys=True) + "\n") -assert actual == expected, "Runtime installation changed the reviewed build tools" -PY -# Build only the authenticated EvalPlus checkout using the already locked tools. -"$task_uv_executable" --no-config --offline pip install --python "$SANDBOX_VENV/bin/python" \ - --no-deps --no-build-isolation "$SANDBOX_BUNDLE/source" -"$task_uv_executable" --no-config pip check --python "$SANDBOX_VENV/bin/python" -"$task_uv_executable" --no-config pip freeze --python "$SANDBOX_VENV/bin/python" > "$output/installed.private.txt" -"$BASE_PYTHON" "$SANDBOX_BUNDLE/runtime.py" verify-source --bundle "$SANDBOX_BUNDLE" -"$SANDBOX_VENV/bin/python" "$SANDBOX_BUNDLE/runtime.py" inventory --bundle "$SANDBOX_BUNDLE" \ - --output "$output/rootfs-before" -"$SANDBOX_VENV/bin/python" "$SANDBOX_BUNDLE/runtime.py" probe --bundle "$SANDBOX_BUNDLE" \ - --inventory "$output/rootfs-before" --output "$output/probe" -"$SANDBOX_VENV/bin/python" "$SANDBOX_BUNDLE/runtime.py" inventory --bundle "$SANDBOX_BUNDLE" \ - --output "$output/rootfs-after" -cmp "$output/rootfs-before/rootfs.json" "$output/rootfs-after/rootfs.json" -check_digest "$source_dir/install-reviewed.sbatch" "$INSTALL_SCRIPT_SHA256" -check_digest "$SANDBOX_BUNDLE/freeze.sha256" "$SANDBOX_FREEZE_SHA256" -(cd "$SANDBOX_BUNDLE" && sha256sum --check --strict freeze.sha256) -echo 'Trusted benign sandbox preparation completed; generated-code executor remains unqualified.' diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-installed-package-audit.py b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-installed-package-audit.py deleted file mode 100644 index 946efc625..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-installed-package-audit.py +++ /dev/null @@ -1,59 +0,0 @@ -"""Verify the complete installed set against authenticated exact dependency pins.""" - -import importlib.metadata -import json -import re -import sys -from pathlib import Path - - -def normalize(name): - return re.sub(r"[-_.]+", "-", name).lower() - - -def locked(paths): - expected = {} - for path in paths: - rows = re.findall(r"^([A-Za-z0-9_.-]+)==([^\s\\]+)", path.read_text(), re.MULTILINE) - if not rows: - raise ValueError("Empty lock") - for name, version in rows: - key = normalize(name) - if key in expected and expected[key] != version: - raise ValueError("Conflicting build/runtime pins") - expected[key] = version - return expected - - -def validate(expected, rows, phase): - actual = {} - for name, version in rows: - key = normalize(name) - if key in actual: - raise ValueError("Duplicate installed distribution") - actual[key] = version - wanted = dict(expected) - if phase == "evaluator": - if "evalplus" not in actual: - raise ValueError("EvalPlus missing") - # Its version is SCM-derived; source bytes are independently authenticated. - wanted["evalplus"] = actual["evalplus"] - elif phase != "dependencies": - raise ValueError("Unknown phase") - if actual != wanted: - raise ValueError("Installed distribution set or versions differ from locks") - return actual - - -def main(): - build, runtime, phase, output = sys.argv[1:] - expected = locked([Path(build), Path(runtime)]) - rows = [(dist.metadata["Name"], dist.version) for dist in importlib.metadata.distributions()] - actual = validate(expected, rows, phase) - with Path(output).open("x") as stream: - json.dump({"phase": phase, "expected": expected, "actual": actual}, stream, sort_keys=True) - stream.write("\n") - - -if __name__ == "__main__": - main() diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-offline-install-v3/audit_packages.py b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-offline-install-v3/audit_packages.py deleted file mode 100644 index 946efc625..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-offline-install-v3/audit_packages.py +++ /dev/null @@ -1,59 +0,0 @@ -"""Verify the complete installed set against authenticated exact dependency pins.""" - -import importlib.metadata -import json -import re -import sys -from pathlib import Path - - -def normalize(name): - return re.sub(r"[-_.]+", "-", name).lower() - - -def locked(paths): - expected = {} - for path in paths: - rows = re.findall(r"^([A-Za-z0-9_.-]+)==([^\s\\]+)", path.read_text(), re.MULTILINE) - if not rows: - raise ValueError("Empty lock") - for name, version in rows: - key = normalize(name) - if key in expected and expected[key] != version: - raise ValueError("Conflicting build/runtime pins") - expected[key] = version - return expected - - -def validate(expected, rows, phase): - actual = {} - for name, version in rows: - key = normalize(name) - if key in actual: - raise ValueError("Duplicate installed distribution") - actual[key] = version - wanted = dict(expected) - if phase == "evaluator": - if "evalplus" not in actual: - raise ValueError("EvalPlus missing") - # Its version is SCM-derived; source bytes are independently authenticated. - wanted["evalplus"] = actual["evalplus"] - elif phase != "dependencies": - raise ValueError("Unknown phase") - if actual != wanted: - raise ValueError("Installed distribution set or versions differ from locks") - return actual - - -def main(): - build, runtime, phase, output = sys.argv[1:] - expected = locked([Path(build), Path(runtime)]) - rows = [(dist.metadata["Name"], dist.version) for dist in importlib.metadata.distributions()] - actual = validate(expected, rows, phase) - with Path(output).open("x") as stream: - json.dump({"phase": phase, "expected": expected, "actual": actual}, stream, sort_keys=True) - stream.write("\n") - - -if __name__ == "__main__": - main() diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-offline-install-v3/check_wheels.py b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-offline-install-v3/check_wheels.py deleted file mode 100644 index 5f3a39fcc..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-offline-install-v3/check_wheels.py +++ /dev/null @@ -1,71 +0,0 @@ -"""Read-only controls over the retained complete wheelhouse; no installation.""" -import copy -import hashlib -import importlib.util -import io -import json -import sys -from pathlib import Path -from unittest.mock import patch - -from packaging.tags import parse_tag - -spec = importlib.util.spec_from_file_location("verify", Path(__file__).with_name("verify_wheels.py")) -m = importlib.util.module_from_spec(spec) -spec.loader.exec_module(m) -root = Path(sys.argv[1]) -bundle = root / "sandbox-offline-prefetch-v1/fetched" -build = root / "sandbox-resolve-1991/build.lock" -runtime = root / "sandbox-resolve-1991/dependencies.lock" -raw = (bundle / "manifest.json").read_bytes() -digest = hashlib.sha256(raw).hexdigest() -manifest = json.loads(raw) -tags = [next(iter(parse_tag(tag))) for tag in json.loads( - (root / "sandbox-platform-fs-v1/observations.json").read_bytes())["platform"]["supported_tags"]] -checks = [] - - -def run(): - return m.verify(bundle, build, runtime, digest) - - -def rejects(function, name): - try: - function() - except ValueError: - checks.append(name) - else: - raise RuntimeError(name) - - -with patch.object(m, "sys_tags", return_value=tags), patch.object(m.platform, "python_version", return_value="3.12.13"): - result = run() - if result["package_count"] != 96 or result["total_wheel_bytes"] != 172422937: - raise RuntimeError("Complete wheelhouse") - checks.append("complete-wheelhouse-with-recorded-target-tags") - for field, value in (("completed", False), ("installed", True), ("executed_downloads", True), - ("errors", [{"name": "synthetic"}]), ("files", manifest["files"][:-1])): - modified = copy.deepcopy(manifest) - modified[field] = value - with patch.object(m.helper, "strict_json", return_value=modified): - rejects(run, "manifest-" + field) - for field, value in (("version", "0.0.0"), ("sha256", "0" * 64), ("size", True), - ("requires_python", ">=99")): - modified = copy.deepcopy(manifest) - modified["files"][0][field] = value - with patch.object(m.helper, "strict_json", return_value=modified): - rejects(run, "wheel-" + field) - actual_open = Path.open - - def bad_open(path, *args, **kwargs): - if path.name == manifest["files"][0]["filename"]: - return io.BytesIO(b"synthetic-corruption") - return actual_open(path, *args, **kwargs) - - with patch.object(Path, "open", bad_open): - rejects(run, "corrupted-wheel-bytes") - with patch.object(m, "sys_tags", return_value=[]): - rejects(run, "no-compatible-target-tags") - rejects(lambda: m.verify(bundle, build, runtime, "0" * 64), "manifest-digest") -print(json.dumps({"checks_passed": len(checks), "checks": checks, - "actual_linux_execution": False, "installed": False}, indent=2)) diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-offline-install-v3/controls.json b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-offline-install-v3/controls.json deleted file mode 100644 index a26fd995f..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-offline-install-v3/controls.json +++ /dev/null @@ -1,20 +0,0 @@ -{ - "checks_passed": 13, - "checks": [ - "complete-wheelhouse-with-recorded-target-tags", - "manifest-completed", - "manifest-installed", - "manifest-executed_downloads", - "manifest-errors", - "manifest-files", - "wheel-version", - "wheel-sha256", - "wheel-size", - "wheel-requires_python", - "corrupted-wheel-bytes", - "no-compatible-target-tags", - "manifest-digest" - ], - "actual_linux_execution": false, - "installed": false -} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-offline-install-v3/install-reviewed.sbatch b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-offline-install-v3/install-reviewed.sbatch deleted file mode 100644 index 78e02493a..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-offline-install-v3/install-reviewed.sbatch +++ /dev/null @@ -1,154 +0,0 @@ -#!/usr/bin/env bash -# Install a reviewed dependency freeze and run only the trusted benign probe. -#SBATCH --job-name=qwen38-eval-sandbox-install -#SBATCH --nodes=1 -#SBATCH --ntasks=1 -#SBATCH --cpus-per-task=4 -#SBATCH --mem=16G -#SBATCH --time=00:30:00 -set -euo pipefail -umask 077 -: "${TASK_ROOT:?}" -: "${BASE_PYTHON:?}" -: "${UV_BIN:?}" -: "${SANDBOX_BUNDLE:?}" -: "${SANDBOX_FREEZE_SHA256:?}" -: "${SANDBOX_VENV:?}" -: "${INSTALL_SOURCE_DIR:?}" -: "${INSTALL_SCRIPT_SHA256:?}" -: "${PACKAGE_AUDIT_SHA256:?}" -: "${WHEEL_BUNDLE:?}" "${WHEEL_MANIFEST_SHA256:?}" -: "${WHEEL_VERIFIER_SHA256:?}" "${PREFETCH_HELPER_SHA256:?}" -task_uv_executable=$UV_BIN -[[ -n ${SLURM_JOB_ID:-} && ${SLURM_GPUS_ON_NODE:-0} == 0 ]] -root=$(realpath -e "$TASK_ROOT") -check_path() { - local value - value=$(realpath -m "$1") - case "$value" in "$root"/*) ;; *) echo 'Path escapes task root.' >&2; exit 2;; esac -} -for path in "$BASE_PYTHON" "$task_uv_executable" "$SANDBOX_BUNDLE" "$SANDBOX_VENV" "$INSTALL_SOURCE_DIR" "$WHEEL_BUNDLE"; do - check_path "$path" -done -source_dir=$(cd -- "$INSTALL_SOURCE_DIR" && pwd -P) -check_digest() { - if [[ ! "$2" =~ ^[a-f0-9]{64}$ ]]; then return 2; fi - [[ $(sha256sum "$1" | cut -d' ' -f1) == "$2" ]] -} -check_digest "$source_dir/install-reviewed.sbatch" "$INSTALL_SCRIPT_SHA256" -check_digest "$source_dir/audit_packages.py" "$PACKAGE_AUDIT_SHA256" -check_digest "$source_dir/verify_wheels.py" "$WHEEL_VERIFIER_SHA256" -check_digest "$source_dir/prefetch.py" "$PREFETCH_HELPER_SHA256" -check_path "$WHEEL_BUNDLE/wheelhouse" -cmp "${BASH_SOURCE[0]}" "$source_dir/install-reviewed.sbatch" -check_digest "$SANDBOX_BUNDLE/freeze.sha256" "$SANDBOX_FREEZE_SHA256" -(cd "$SANDBOX_BUNDLE" && sha256sum --check --strict freeze.sha256) -check_digest "$task_uv_executable" "$(cat "$SANDBOX_BUNDLE/uv-sha256.txt")" -check_digest "$BASE_PYTHON" "$(cat "$SANDBOX_BUNDLE/python-sha256.txt")" -for path in "$SANDBOX_BUNDLE/runtime.py" "$SANDBOX_BUNDLE/benign_probe.py" \ - "$SANDBOX_BUNDLE/source" "$SANDBOX_BUNDLE/wheelhouse"; do check_path "$path"; done -[[ ! -e "$SANDBOX_VENV" && ! -L "$SANDBOX_VENV" ]] -for task_resolver_variable in "${!UV_@}" "${!PIP_@}"; do - if [[ -n "$task_resolver_variable" ]]; then unset "$task_resolver_variable"; fi -done -export TMPDIR="$root/tmp/qwen38-eval-sandbox-install-${SLURM_JOB_ID}" -export XDG_CACHE_HOME="$root/cache/qwen38-eval-sandbox" -export UV_CACHE_DIR="$XDG_CACHE_HOME/uv" -export HF_HOME="$XDG_CACHE_HOME/huggingface" -export TORCH_HOME="$XDG_CACHE_HOME/torch" -export TRITON_CACHE_DIR="$XDG_CACHE_HOME/triton" -export VLLM_CACHE_ROOT="$XDG_CACHE_HOME/vllm" -export UV_PYTHON_INSTALL_DIR="$root/software/python" -export UV_PYTHON_DOWNLOADS=never -export UV_OFFLINE=true -export PYTHONNOUSERSITE=1 -export PYTHONDONTWRITEBYTECODE=1 -unset PYTHONPATH PYTHONHOME PYTHONOPTIMIZE PYTHONINSPECT PYTHONSTARTUP VIRTUAL_ENV -for path in "$TMPDIR" "$XDG_CACHE_HOME" "$UV_CACHE_DIR" "$HF_HOME" "$TORCH_HOME" \ - "$TRITON_CACHE_DIR" "$VLLM_CACHE_ROOT" "$UV_PYTHON_INSTALL_DIR"; do check_path "$path"; done -mkdir -p "$TMPDIR" "$UV_CACHE_DIR" "$HF_HOME" "$TORCH_HOME" "$TRITON_CACHE_DIR" "$VLLM_CACHE_ROOT" -cd "$root" -output="$SANDBOX_BUNDLE/installation-v3" -check_path "$output" -mkdir "$output" -# BEGIN TERMINAL RECORD: an EXIT code alone never authenticates completion. -task_completed=false -task_signal=none -finish_installation() { - local status=$? - trap - EXIT - if [[ "$status" == 0 && "$task_completed" != true ]]; then status=1; fi - if [[ "$status" != 0 ]]; then task_completed=false; fi - printf '%s\n' "$status" > "$output/exit-code" - printf '{"completed":%s,"signal":"%s","exit_code":%s,"job_id":"%s"}\n' \ - "$task_completed" "$task_signal" "$status" "$SLURM_JOB_ID" > "$output/terminal-status.json" - exit "$status" -} -trap finish_installation EXIT -trap 'task_completed=false; task_signal=TERM; exit 143' TERM -trap 'task_completed=false; task_signal=INT; exit 130' INT -trap 'task_completed=false; task_signal=HUP; exit 129' HUP -# END TERMINAL RECORD -cp "$source_dir/install-reviewed.sbatch" "$source_dir/audit_packages.py" \ - "$source_dir/verify_wheels.py" "$source_dir/prefetch.py" "$output/" -verify_offline_wheels() { - "$BASE_PYTHON" -IB "$source_dir/verify_wheels.py" --bundle "$WHEEL_BUNDLE" \ - --build "$SANDBOX_BUNDLE/build.lock" --runtime "$SANDBOX_BUNDLE/dependencies.lock" \ - --sha256 "$WHEEL_MANIFEST_SHA256" --output "$1" -} -verify_offline_wheels "$output/wheels-before.json" -"$BASE_PYTHON" "$SANDBOX_BUNDLE/runtime.py" verify-source --bundle "$SANDBOX_BUNDLE" -[[ -z $(git -C "$SANDBOX_BUNDLE/source" ls-files --others --exclude-standard) ]] -"$task_uv_executable" --no-config venv --python "$BASE_PYTHON" "$SANDBOX_VENV" -"$task_uv_executable" --no-config --offline pip install --python "$SANDBOX_VENV/bin/python" \ - --no-deps --no-build --require-hashes --no-index \ - --find-links "$WHEEL_BUNDLE/wheelhouse" -r "$SANDBOX_BUNDLE/build.lock" -"$task_uv_executable" --no-config --offline pip install --python "$SANDBOX_VENV/bin/python" \ - --no-deps --no-build --require-hashes --no-index \ - --find-links "$WHEEL_BUNDLE/wheelhouse" -r "$SANDBOX_BUNDLE/dependencies.lock" -# Both reviewed lockfiles contain the full closure; no implicit dependencies. -"$SANDBOX_VENV/bin/python" -IB "$source_dir/audit_packages.py" \ - "$SANDBOX_BUNDLE/build.lock" "$SANDBOX_BUNDLE/dependencies.lock" dependencies \ - "$output/dependencies-verified.json" -# Build only the authenticated EvalPlus checkout using the already locked tools. -"$task_uv_executable" --no-config --offline pip install --python "$SANDBOX_VENV/bin/python" \ - --no-deps --no-build-isolation "$SANDBOX_BUNDLE/source" -"$SANDBOX_VENV/bin/python" -IB "$source_dir/audit_packages.py" \ - "$SANDBOX_BUNDLE/build.lock" "$SANDBOX_BUNDLE/dependencies.lock" evaluator \ - "$output/evaluator-packages-verified.json" -"$task_uv_executable" --no-config pip check --python "$SANDBOX_VENV/bin/python" -"$task_uv_executable" --no-config pip freeze --python "$SANDBOX_VENV/bin/python" > "$output/installed.private.txt" -"$BASE_PYTHON" "$SANDBOX_BUNDLE/runtime.py" verify-source --bundle "$SANDBOX_BUNDLE" -"$SANDBOX_VENV/bin/python" "$SANDBOX_BUNDLE/runtime.py" inventory --bundle "$SANDBOX_BUNDLE" \ - --output "$output/rootfs-before" -probe_status=0 -"$SANDBOX_VENV/bin/python" "$SANDBOX_BUNDLE/runtime.py" probe --bundle "$SANDBOX_BUNDLE" \ - --inventory "$output/rootfs-before" --output "$output/probe" || probe_status=$? -printf '%s\n' "$probe_status" > "$output/probe-status" -post_status=0 -run_postcheck() { - local name=$1 status=0 - shift - "$@" || status=$? - printf '%s\t%s\n' "$name" "$status" >> "$output/postflight-status.tsv" - if [[ "$status" != 0 ]]; then post_status=$status; fi -} -check_frozen_files() { (cd "$SANDBOX_BUNDLE" && sha256sum --check --strict freeze.sha256); } -run_postcheck inventory "$SANDBOX_VENV/bin/python" "$SANDBOX_BUNDLE/runtime.py" inventory \ - --bundle "$SANDBOX_BUNDLE" --output "$output/rootfs-after" -run_postcheck runtime-byte-comparison cmp "$output/rootfs-before/rootfs.json" "$output/rootfs-after/rootfs.json" -run_postcheck source "$BASE_PYTHON" "$SANDBOX_BUNDLE/runtime.py" verify-source --bundle "$SANDBOX_BUNDLE" -run_postcheck package-set "$SANDBOX_VENV/bin/python" -IB "$source_dir/audit_packages.py" \ - "$SANDBOX_BUNDLE/build.lock" "$SANDBOX_BUNDLE/dependencies.lock" evaluator \ - "$output/evaluator-packages-after.json" -run_postcheck batch-script check_digest "$source_dir/install-reviewed.sbatch" "$INSTALL_SCRIPT_SHA256" -run_postcheck audit-script check_digest "$source_dir/audit_packages.py" "$PACKAGE_AUDIT_SHA256" -run_postcheck wheel-verifier check_digest "$source_dir/verify_wheels.py" "$WHEEL_VERIFIER_SHA256" -run_postcheck prefetch-helper check_digest "$source_dir/prefetch.py" "$PREFETCH_HELPER_SHA256" -run_postcheck wheel-bytes verify_offline_wheels "$output/wheels-after.json" -run_postcheck freeze-manifest check_digest "$SANDBOX_BUNDLE/freeze.sha256" "$SANDBOX_FREEZE_SHA256" -run_postcheck freeze-files check_frozen_files -if [[ "$probe_status" != 0 ]]; then exit "$probe_status"; fi -if [[ "$post_status" != 0 ]]; then exit "$post_status"; fi -echo 'Trusted benign sandbox preparation completed; generated-code executor remains unqualified.' -task_completed=true diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-offline-install-v3/prefetch.py b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-offline-install-v3/prefetch.py deleted file mode 100644 index 471b82868..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-offline-install-v3/prefetch.py +++ /dev/null @@ -1,241 +0,0 @@ -"""Fetch locked wheels as data using recorded target tags; never install or import.""" - -import argparse -import hashlib -import json -import re -import signal -import urllib.parse -import urllib.request -from pathlib import Path - -from packaging.specifiers import SpecifierSet -from packaging.utils import canonicalize_name, parse_wheel_filename -from packaging.version import Version - - -def sha(raw): - return hashlib.sha256(raw).hexdigest() - - -def strict_json(raw): - def pairs(items): - result = {} - for key, value in items: - if key in result: - raise ValueError("Duplicate JSON key") - result[key] = value - return result - return json.loads(raw, object_pairs_hook=pairs) - - -def parse_lock(raw): - result, pending = {}, "" - for line in raw.decode("utf-8").splitlines(): - line = line.strip() - if not line or line.startswith("#"): - if pending: - raise ValueError("Interrupted lock continuation") - continue - pending += " " + line.removesuffix("\\").strip() - if line.endswith("\\"): - continue - match = re.fullmatch( - r"\s*([A-Za-z0-9][A-Za-z0-9._-]*)==([A-Za-z0-9.!+_-]+)" - r"((?:\s+--hash=sha256:[a-f0-9]{64})+)\s*", pending - ) - if not match: - raise ValueError("Unsupported lock grammar") - name = canonicalize_name(match[1]) - version = str(Version(match[2])) - hashes = set(re.findall(r"--hash=sha256:([a-f0-9]{64})", match[3])) - if name in result: - raise ValueError("Duplicate lock package") - result[name] = (version, hashes) - pending = "" - if pending or not result: - raise ValueError("Incomplete or empty lock") - return result - - -def merged_locks(locks): - result = {} - for lock in locks: - for name, (version, hashes) in lock.items(): - if name in result: - previous, accepted = result[name] - if previous != version or not accepted & hashes: - raise ValueError("Conflicting lock package") - hashes = accepted & hashes - result[name] = (version, hashes) - return result - - -def checked_url(url, host): - parsed = urllib.parse.urlsplit(url) - if (parsed.scheme != "https" or parsed.netloc != host - or parsed.username or parsed.password or parsed.query or parsed.fragment): - raise ValueError("Unexpected public artifact URL") - return url - - -class SameHostRedirect(urllib.request.HTTPRedirectHandler): - def redirect_request(self, request, fp, code, message, headers, newurl): - checked_url(newurl, urllib.parse.urlsplit(request.full_url).netloc) - return super().redirect_request(request, fp, code, message, headers, newurl) - - -def fetch(url, host, maximum): - checked_url(url, host) - opener = urllib.request.build_opener(SameHostRedirect()) - request = urllib.request.Request(url, headers={"Accept-Encoding": "identity"}) - - def timeout(_signum, _frame): - raise TimeoutError("Artifact read deadline") - - # Main-thread sequential downloads: SIGALRM interrupts a blocked read, unlike - # a deadline checked only between reads or the socket's inactivity timeout. - if signal.getitimer(signal.ITIMER_REAL) != (0.0, 0.0): - raise ValueError("An existing alarm would be overwritten") - previous = signal.signal(signal.SIGALRM, timeout) - signal.setitimer(signal.ITIMER_REAL, 120) - try: - with opener.open(request, timeout=30) as response: - checked_url(response.geturl(), host) - if response.status != 200 or response.headers.get("Content-Encoding", "identity") != "identity": - raise ValueError("Unexpected HTTP response") - chunks, count = [], 0 - while True: - chunk = response.read(min(262144, maximum + 1 - count)) - if not chunk: - break - chunks.append(chunk) - count += len(chunk) - if count > maximum: - raise ValueError("Oversized artifact") - return b"".join(chunks) - finally: - signal.setitimer(signal.ITIMER_REAL, 0) - signal.signal(signal.SIGALRM, previous) - - -def choose_wheel(name, version, hashes, metadata, ranks, python_version): - if (canonicalize_name(metadata["info"]["name"]) != name - or Version(metadata["info"]["version"]) != Version(version)): - raise ValueError("Metadata package identity mismatch") - candidates = [] - for entry in metadata["urls"]: - if entry["packagetype"] != "bdist_wheel" or entry.get("yanked"): - continue - filename = entry["filename"] - if Path(filename).name != filename or "\\" in filename: - raise ValueError("Invalid wheel filename") - package, release, _, tags = parse_wheel_filename(filename) - if package != name or release != Version(version): - continue - matching = [ranks[str(tag)] for tag in tags if str(tag) in ranks] - if not matching or entry["digests"]["sha256"] not in hashes: - continue - requires = entry.get("requires_python") - if requires and not SpecifierSet(requires).contains(python_version, prereleases=True): - continue - url = checked_url(entry["url"], "files.pythonhosted.org") - if urllib.parse.unquote(urllib.parse.urlsplit(url).path.rsplit("/", 1)[-1]) != filename: - raise ValueError("URL filename mismatch") - if type(entry["size"]) is not int or not 0 < entry["size"] <= 200 * 1024 * 1024: - raise ValueError("Unexpected wheel size") - candidates.append((min(matching), filename, entry)) - if not candidates: - raise ValueError("No compatible locked wheel") - rank, _, entry = min(candidates, key=lambda item: item[:2]) - return entry, rank - - -def main(): - parser = argparse.ArgumentParser(description=__doc__) - parser.add_argument("--policy", type=Path, required=True) - parser.add_argument("--inputs", type=Path, required=True) - parser.add_argument("--platform", type=Path, required=True) - parser.add_argument("--output", type=Path, required=True) - args = parser.parse_args() - policy_raw = args.policy.read_bytes() - policy = strict_json(policy_raw) - platform_raw = args.platform.read_bytes() - if sha(platform_raw) != policy["platform_sha256"]: - raise ValueError("Platform digest mismatch") - observed = strict_json(platform_raw) - target = observed["platform"] - if (observed["completed"] is not True or target["completed"] is not True - or target["system"] != "Linux" or target["machine"] != "x86_64" - or target["implementation"] != "cpython" or target["python"] != "3.12.13"): - raise ValueError("Unexpected target platform") - tags = target["supported_tags"] - if not tags or len(set(tags)) != len(tags): - raise ValueError("Empty or duplicate target tags") - ranks = {tag: index for index, tag in enumerate(tags)} - locks = [] - for filename in ("build.lock", "dependencies.lock"): - raw = (args.inputs / filename).read_bytes() - if sha(raw) != policy["locks"][filename]: - raise ValueError("Lock digest mismatch") - locks.append(parse_lock(raw)) - packages = merged_locks(locks) - if [len(lock) for lock in locks] != [5, 92] or len(packages) != 96: - raise ValueError("Unexpected locked closure") - args.output.mkdir(mode=0o700, exist_ok=False) - wheelhouse = args.output / "wheelhouse" - wheelhouse.mkdir() - metadata_dir = args.output / "metadata" - metadata_dir.mkdir() - report = {"scope": "local_offline_wheel_prefetch", "verdict": "INCONCLUSIVE", - "completed": False, "installed": False, "executed_downloads": False, - "policy_sha256": sha(policy_raw), "platform_sha256": sha(platform_raw), - "script_sha256": sha(Path(__file__).read_bytes()), "files": [], "errors": []} - - def package_task(name, version, hashes): - if name in ("wget", "tempdir"): - filename = f"{name}-{version}-py3-none-any.whl" - raw = (args.inputs / "wheelhouse" / filename).read_bytes() - digest = sha(raw) - if digest not in hashes or "py3-none-any" not in ranks: - raise ValueError("Reviewed local wheel mismatch") - record = {"name": name, "version": version, "filename": filename, - "sha256": digest, "size": len(raw), - "source": "reviewed_cpu1986_build", "tag_rank": ranks["py3-none-any"]} - else: - url = f"https://pypi.org/pypi/{name}/{version}/json" - metadata_raw = fetch(url, "pypi.org", 8 * 1024 * 1024) - (metadata_dir / f"{name}.json").write_bytes(metadata_raw) - entry, rank = choose_wheel(name, version, hashes, strict_json(metadata_raw), - ranks, target["python"]) - raw = fetch(entry["url"], "files.pythonhosted.org", entry["size"]) - digest = sha(raw) - if len(raw) != entry["size"] or digest != entry["digests"]["sha256"]: - raise ValueError("Wheel size or digest mismatch") - filename = entry["filename"] - record = {"name": name, "version": version, "filename": filename, - "sha256": digest, "size": len(raw), "source": entry["url"], - "metadata_url": url, "metadata_sha256": sha(metadata_raw), - "requires_python": entry.get("requires_python"), "tag_rank": rank} - (wheelhouse / filename).write_bytes(raw) - if sha((wheelhouse / filename).read_bytes()) != digest: - raise ValueError("Written wheel differs") - return record - - for name, (version, hashes) in sorted(packages.items()): - try: - record = package_task(name, version, hashes) - report["files"].append(record) - print(json.dumps({"downloaded": record["name"], "size": record["size"]}), flush=True) - except (OSError, ValueError, KeyError, TypeError) as error: - report["errors"].append({"name": name, "error_type": type(error).__name__}) - (args.output / "manifest.json").write_text(json.dumps(report, indent=2) + "\n") - if report["errors"]: - break - report["completed"] = not report["errors"] and len(report["files"]) == len(packages) - (args.output / "manifest.json").write_text(json.dumps(report, indent=2) + "\n") - return 0 if report["completed"] else 1 - - -if __name__ == "__main__": - raise SystemExit(main()) diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-offline-install-v3/verify_wheels.py b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-offline-install-v3/verify_wheels.py deleted file mode 100644 index c4557c06a..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-offline-install-v3/verify_wheels.py +++ /dev/null @@ -1,86 +0,0 @@ -"""Verify all offline wheels against unchanged locks and the actual interpreter.""" - -import argparse -import hashlib -import importlib.util -import json -import platform -from pathlib import Path - -from packaging.specifiers import SpecifierSet -from packaging.tags import sys_tags -from packaging.utils import parse_wheel_filename -from packaging.version import Version - -spec = importlib.util.spec_from_file_location("prefetch", Path(__file__).with_name("prefetch.py")) -helper = importlib.util.module_from_spec(spec) -spec.loader.exec_module(helper) - - -def verify(bundle, build, runtime, expected_sha): - raw = (bundle / "manifest.json").read_bytes() - if hashlib.sha256(raw).hexdigest() != expected_sha: - raise ValueError("Manifest digest mismatch") - manifest = helper.strict_json(raw) - if (manifest["completed"] is not True or manifest["errors"] - or manifest["installed"] is not False or manifest["executed_downloads"] is not False): - raise ValueError("Unexpected prefetch state") - locks = [helper.parse_lock(path.read_bytes()) for path in (build, runtime)] - expected = helper.merged_locks(locks) - if [len(lock) for lock in locks] != [5, 92] or len(expected) != 96: - raise ValueError("Unexpected locked package set") - entries = manifest["files"] - if len(entries) != 96 or {entry["name"] for entry in entries} != set(expected): - raise ValueError("Manifest closure differs") - wheelhouse = bundle / "wheelhouse" - names = {entry["filename"] for entry in entries} - if len(names) != 96 or {p.name for p in wheelhouse.iterdir()} != names: - raise ValueError("Wheelhouse inventory differs") - tags = list(sys_tags()) - accepted = set(tags) - for entry in entries: - filename = entry["filename"] - if Path(filename).name != filename or "\\" in filename: - raise ValueError("Unsafe filename") - path = wheelhouse / filename - if path.is_symlink() or not path.is_file(): - raise ValueError("Wheel is not a regular file") - name, version, _, wheel_tags = parse_wheel_filename(filename) - wanted_version, hashes = expected[entry["name"]] - if (name != entry["name"] or version != Version(wanted_version) - or entry["version"] != wanted_version or not accepted & wheel_tags - or entry["sha256"] not in hashes or type(entry["size"]) is not int): - raise ValueError("Wheel package/tag/hash identity differs") - requires = entry.get("requires_python") - if requires and not SpecifierSet(requires).contains(platform.python_version(), prereleases=True): - raise ValueError("Requires-Python mismatch") - digest = hashlib.sha256() - size = 0 - with path.open("rb") as stream: - while chunk := stream.read(1024 * 1024): - digest.update(chunk) - size += len(chunk) - if size != entry["size"] or digest.hexdigest() != entry["sha256"]: - raise ValueError("Wheel bytes differ") - return {"completed": True, "package_count": len(entries), - "manifest_sha256": expected_sha, "python": platform.python_version(), - "supported_tags_sha256": hashlib.sha256("\n".join(map(str, tags)).encode()).hexdigest(), - "total_wheel_bytes": sum(entry["size"] for entry in entries)} - - -def main(): - parser = argparse.ArgumentParser(description=__doc__) - parser.add_argument("--bundle", required=True, type=Path) - parser.add_argument("--build", required=True, type=Path) - parser.add_argument("--runtime", required=True, type=Path) - parser.add_argument("--sha256", required=True) - parser.add_argument("--output", required=True, type=Path) - args = parser.parse_args() - report = verify(args.bundle, args.build, args.runtime, args.sha256) - with args.output.open("x") as stream: - json.dump(report, stream, sort_keys=True) - stream.write("\n") - - -if __name__ == "__main__": - main() diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-offline-prefetch-v1/check_prefetch.py b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-offline-prefetch-v1/check_prefetch.py deleted file mode 100644 index 66d2fea7c..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-offline-prefetch-v1/check_prefetch.py +++ /dev/null @@ -1,136 +0,0 @@ -"""Offline controls for lock grammar and target wheel authentication/selection.""" -import copy -import importlib.util -import json -import signal -import time -from pathlib import Path -from unittest.mock import patch - -spec = importlib.util.spec_from_file_location("prefetch", Path(__file__).with_name("prefetch.py")) -m = importlib.util.module_from_spec(spec) -spec.loader.exec_module(m) -checks = [] - - -def require(condition, name): - if not condition: - raise RuntimeError(name) - checks.append(name) - - -def rejects(function, name): - try: - function() - except ValueError: - checks.append(name) - else: - raise RuntimeError(name) - - -digest = "a" * 64 -line = f"sample-pkg==1.0 --hash=sha256:{digest}\n".encode() -require(m.parse_lock(line) == {"sample-pkg": ("1.0", {digest})}, "valid-lock") -require(m.parse_lock(line.replace(b" --", b" \\\n --")) == m.parse_lock(line), "continuation") -for raw, name in ((line + line, "duplicate-package"), (line + b"\\\n", "unfinished"), - (b"sample>=1.0\n", "unpinned"), (b"--index-url https://invalid.example\n", "extra-option"), - (line.replace(b"sha256", b"sha512"), "hash-algorithm"), - (line.replace(b"1.0 ", b"1.0; python_version>'3' "), "marker")): - rejects(lambda raw=raw: m.parse_lock(raw), name) -require(m.merged_locks([m.parse_lock(line), m.parse_lock(line)]) == m.parse_lock(line), "merge-shared") -rejects(lambda: m.merged_locks([m.parse_lock(line), m.parse_lock(line.replace(b"1.0", b"2.0"))]), "version-conflict") -rejects(lambda: m.merged_locks([m.parse_lock(line), m.parse_lock(line.replace(b"a" * 64, b"b" * 64))]), "hash-conflict") -rejects(lambda: m.strict_json(b'{"a":1,"a":2}'), "duplicate-json") -for url in ("http://pypi.org/x", "https://pypi.org.evil.example/x", "https://user@pypi.org/x", "https://pypi.org/x?secret=1"): - rejects(lambda url=url: m.checked_url(url, "pypi.org"), "url-boundary") -filename = "sample_pkg-1.0-cp312-cp312-manylinux_2_28_x86_64.whl" -entry = {"packagetype": "bdist_wheel", "filename": filename, "yanked": False, - "url": "https://files.pythonhosted.org/packages/" + filename, - "digests": {"sha256": digest}, "size": 123, "requires_python": ">=3.10"} -metadata = {"info": {"name": "sample-pkg", "version": "1.0"}, "urls": [entry]} -ranks = {"cp312-cp312-manylinux_2_28_x86_64": 0, "py3-none-any": 1} -choose = lambda data: m.choose_wheel("sample-pkg", "1.0", {digest}, data, ranks, "3.12.13") -require(choose(metadata) == (entry, 0), "target-wheel") -for field, value, name in (("requires_python", ">=3.13", "python-requirement"), - ("yanked", True, "yanked"), ("size", True, "boolean-size"), - ("digests", {"sha256": "b" * 64}, "unlocked-hash"), - ("filename", "sample_pkg-1.0-cp312-cp312-win_amd64.whl", "platform-mismatch"), - ("url", "https://files.pythonhosted.org/packages/other.whl", "filename-mismatch")): - modified = copy.deepcopy(metadata) - modified["urls"][0][field] = value - rejects(lambda modified=modified: choose(modified), name) - -class Response: - status = 200 - - def __init__(self): - self.headers = {} - self.chunks = [b"ab", b""] - - def __enter__(self): - return self - - def __exit__(self, *_): - return None - - def geturl(self): - return "https://pypi.org/artifact" - - def read(self, _size): - return self.chunks.pop(0) - - -class Opener: - def __init__(self, response): - self.response = response - - def open(self, *_args, **_kwargs): - return self.response - - -with patch.object(m.urllib.request, "build_opener", return_value=Opener(Response())): - require(m.fetch("https://pypi.org/artifact", "pypi.org", 2) == b"ab", "fetch-complete") -response = Response() -response.chunks = [b"abc"] -with patch.object(m.urllib.request, "build_opener", return_value=Opener(response)): - rejects(lambda: m.fetch("https://pypi.org/artifact", "pypi.org", 2), "oversized-fetch") -for field, value in (("status", 206), ("headers", {"Content-Encoding": "gzip"})): - response = Response() - setattr(response, field, value) - with patch.object(m.urllib.request, "build_opener", return_value=Opener(response)): - rejects(lambda: m.fetch("https://pypi.org/artifact", "pypi.org", 2), "http-response") - - -class BlockedResponse(Response): - def read(self, _size): - time.sleep(0.3) - return b"x" - - -class TrickleResponse(Response): - def read(self, _size): - time.sleep(0.005) - return b"x" - - -actual_setitimer = signal.setitimer -old_handler = signal.getsignal(signal.SIGALRM) - - -def short_alarm(which, seconds): - return actual_setitimer(which, 0.03 if seconds else 0) - - -for response, name in ((BlockedResponse(), "blocked-read-alarm"), (TrickleResponse(), "trickle-alarm")): - with patch.object(m.urllib.request, "build_opener", return_value=Opener(response)), \ - patch.object(m.signal, "setitimer", side_effect=short_alarm): - started = time.monotonic() - try: - m.fetch("https://pypi.org/artifact", "pypi.org", 100000) - except TimeoutError: - require(time.monotonic() - started < 0.25, name) - else: - raise RuntimeError(name) - require(signal.getitimer(signal.ITIMER_REAL) == (0.0, 0.0) - and signal.getsignal(signal.SIGALRM) == old_handler, "alarm-restored") -print(json.dumps({"checks_passed": len(checks), "checks": checks, "network_used": False}, indent=2)) diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-offline-prefetch-v1/policy.json b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-offline-prefetch-v1/policy.json deleted file mode 100644 index 582f9033d..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-offline-prefetch-v1/policy.json +++ /dev/null @@ -1,7 +0,0 @@ -{ - "platform_sha256": "9d13f8c349b340b3a697c4892794557f17d0cac10a87a28f15afe15aa5a674db", - "locks": { - "build.lock": "8f033a8ba24ddabb64ac12e4e5e701d7c52986902a7943cd3c2aee2dc46c0bd1", - "dependencies.lock": "2c60d833d1a705050a152c767da55c90d9f30a4b831d140e115dbffe4d6bb823" - } -} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-offline-prefetch-v1/prefetch.py b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-offline-prefetch-v1/prefetch.py deleted file mode 100644 index 471b82868..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-offline-prefetch-v1/prefetch.py +++ /dev/null @@ -1,241 +0,0 @@ -"""Fetch locked wheels as data using recorded target tags; never install or import.""" - -import argparse -import hashlib -import json -import re -import signal -import urllib.parse -import urllib.request -from pathlib import Path - -from packaging.specifiers import SpecifierSet -from packaging.utils import canonicalize_name, parse_wheel_filename -from packaging.version import Version - - -def sha(raw): - return hashlib.sha256(raw).hexdigest() - - -def strict_json(raw): - def pairs(items): - result = {} - for key, value in items: - if key in result: - raise ValueError("Duplicate JSON key") - result[key] = value - return result - return json.loads(raw, object_pairs_hook=pairs) - - -def parse_lock(raw): - result, pending = {}, "" - for line in raw.decode("utf-8").splitlines(): - line = line.strip() - if not line or line.startswith("#"): - if pending: - raise ValueError("Interrupted lock continuation") - continue - pending += " " + line.removesuffix("\\").strip() - if line.endswith("\\"): - continue - match = re.fullmatch( - r"\s*([A-Za-z0-9][A-Za-z0-9._-]*)==([A-Za-z0-9.!+_-]+)" - r"((?:\s+--hash=sha256:[a-f0-9]{64})+)\s*", pending - ) - if not match: - raise ValueError("Unsupported lock grammar") - name = canonicalize_name(match[1]) - version = str(Version(match[2])) - hashes = set(re.findall(r"--hash=sha256:([a-f0-9]{64})", match[3])) - if name in result: - raise ValueError("Duplicate lock package") - result[name] = (version, hashes) - pending = "" - if pending or not result: - raise ValueError("Incomplete or empty lock") - return result - - -def merged_locks(locks): - result = {} - for lock in locks: - for name, (version, hashes) in lock.items(): - if name in result: - previous, accepted = result[name] - if previous != version or not accepted & hashes: - raise ValueError("Conflicting lock package") - hashes = accepted & hashes - result[name] = (version, hashes) - return result - - -def checked_url(url, host): - parsed = urllib.parse.urlsplit(url) - if (parsed.scheme != "https" or parsed.netloc != host - or parsed.username or parsed.password or parsed.query or parsed.fragment): - raise ValueError("Unexpected public artifact URL") - return url - - -class SameHostRedirect(urllib.request.HTTPRedirectHandler): - def redirect_request(self, request, fp, code, message, headers, newurl): - checked_url(newurl, urllib.parse.urlsplit(request.full_url).netloc) - return super().redirect_request(request, fp, code, message, headers, newurl) - - -def fetch(url, host, maximum): - checked_url(url, host) - opener = urllib.request.build_opener(SameHostRedirect()) - request = urllib.request.Request(url, headers={"Accept-Encoding": "identity"}) - - def timeout(_signum, _frame): - raise TimeoutError("Artifact read deadline") - - # Main-thread sequential downloads: SIGALRM interrupts a blocked read, unlike - # a deadline checked only between reads or the socket's inactivity timeout. - if signal.getitimer(signal.ITIMER_REAL) != (0.0, 0.0): - raise ValueError("An existing alarm would be overwritten") - previous = signal.signal(signal.SIGALRM, timeout) - signal.setitimer(signal.ITIMER_REAL, 120) - try: - with opener.open(request, timeout=30) as response: - checked_url(response.geturl(), host) - if response.status != 200 or response.headers.get("Content-Encoding", "identity") != "identity": - raise ValueError("Unexpected HTTP response") - chunks, count = [], 0 - while True: - chunk = response.read(min(262144, maximum + 1 - count)) - if not chunk: - break - chunks.append(chunk) - count += len(chunk) - if count > maximum: - raise ValueError("Oversized artifact") - return b"".join(chunks) - finally: - signal.setitimer(signal.ITIMER_REAL, 0) - signal.signal(signal.SIGALRM, previous) - - -def choose_wheel(name, version, hashes, metadata, ranks, python_version): - if (canonicalize_name(metadata["info"]["name"]) != name - or Version(metadata["info"]["version"]) != Version(version)): - raise ValueError("Metadata package identity mismatch") - candidates = [] - for entry in metadata["urls"]: - if entry["packagetype"] != "bdist_wheel" or entry.get("yanked"): - continue - filename = entry["filename"] - if Path(filename).name != filename or "\\" in filename: - raise ValueError("Invalid wheel filename") - package, release, _, tags = parse_wheel_filename(filename) - if package != name or release != Version(version): - continue - matching = [ranks[str(tag)] for tag in tags if str(tag) in ranks] - if not matching or entry["digests"]["sha256"] not in hashes: - continue - requires = entry.get("requires_python") - if requires and not SpecifierSet(requires).contains(python_version, prereleases=True): - continue - url = checked_url(entry["url"], "files.pythonhosted.org") - if urllib.parse.unquote(urllib.parse.urlsplit(url).path.rsplit("/", 1)[-1]) != filename: - raise ValueError("URL filename mismatch") - if type(entry["size"]) is not int or not 0 < entry["size"] <= 200 * 1024 * 1024: - raise ValueError("Unexpected wheel size") - candidates.append((min(matching), filename, entry)) - if not candidates: - raise ValueError("No compatible locked wheel") - rank, _, entry = min(candidates, key=lambda item: item[:2]) - return entry, rank - - -def main(): - parser = argparse.ArgumentParser(description=__doc__) - parser.add_argument("--policy", type=Path, required=True) - parser.add_argument("--inputs", type=Path, required=True) - parser.add_argument("--platform", type=Path, required=True) - parser.add_argument("--output", type=Path, required=True) - args = parser.parse_args() - policy_raw = args.policy.read_bytes() - policy = strict_json(policy_raw) - platform_raw = args.platform.read_bytes() - if sha(platform_raw) != policy["platform_sha256"]: - raise ValueError("Platform digest mismatch") - observed = strict_json(platform_raw) - target = observed["platform"] - if (observed["completed"] is not True or target["completed"] is not True - or target["system"] != "Linux" or target["machine"] != "x86_64" - or target["implementation"] != "cpython" or target["python"] != "3.12.13"): - raise ValueError("Unexpected target platform") - tags = target["supported_tags"] - if not tags or len(set(tags)) != len(tags): - raise ValueError("Empty or duplicate target tags") - ranks = {tag: index for index, tag in enumerate(tags)} - locks = [] - for filename in ("build.lock", "dependencies.lock"): - raw = (args.inputs / filename).read_bytes() - if sha(raw) != policy["locks"][filename]: - raise ValueError("Lock digest mismatch") - locks.append(parse_lock(raw)) - packages = merged_locks(locks) - if [len(lock) for lock in locks] != [5, 92] or len(packages) != 96: - raise ValueError("Unexpected locked closure") - args.output.mkdir(mode=0o700, exist_ok=False) - wheelhouse = args.output / "wheelhouse" - wheelhouse.mkdir() - metadata_dir = args.output / "metadata" - metadata_dir.mkdir() - report = {"scope": "local_offline_wheel_prefetch", "verdict": "INCONCLUSIVE", - "completed": False, "installed": False, "executed_downloads": False, - "policy_sha256": sha(policy_raw), "platform_sha256": sha(platform_raw), - "script_sha256": sha(Path(__file__).read_bytes()), "files": [], "errors": []} - - def package_task(name, version, hashes): - if name in ("wget", "tempdir"): - filename = f"{name}-{version}-py3-none-any.whl" - raw = (args.inputs / "wheelhouse" / filename).read_bytes() - digest = sha(raw) - if digest not in hashes or "py3-none-any" not in ranks: - raise ValueError("Reviewed local wheel mismatch") - record = {"name": name, "version": version, "filename": filename, - "sha256": digest, "size": len(raw), - "source": "reviewed_cpu1986_build", "tag_rank": ranks["py3-none-any"]} - else: - url = f"https://pypi.org/pypi/{name}/{version}/json" - metadata_raw = fetch(url, "pypi.org", 8 * 1024 * 1024) - (metadata_dir / f"{name}.json").write_bytes(metadata_raw) - entry, rank = choose_wheel(name, version, hashes, strict_json(metadata_raw), - ranks, target["python"]) - raw = fetch(entry["url"], "files.pythonhosted.org", entry["size"]) - digest = sha(raw) - if len(raw) != entry["size"] or digest != entry["digests"]["sha256"]: - raise ValueError("Wheel size or digest mismatch") - filename = entry["filename"] - record = {"name": name, "version": version, "filename": filename, - "sha256": digest, "size": len(raw), "source": entry["url"], - "metadata_url": url, "metadata_sha256": sha(metadata_raw), - "requires_python": entry.get("requires_python"), "tag_rank": rank} - (wheelhouse / filename).write_bytes(raw) - if sha((wheelhouse / filename).read_bytes()) != digest: - raise ValueError("Written wheel differs") - return record - - for name, (version, hashes) in sorted(packages.items()): - try: - record = package_task(name, version, hashes) - report["files"].append(record) - print(json.dumps({"downloaded": record["name"], "size": record["size"]}), flush=True) - except (OSError, ValueError, KeyError, TypeError) as error: - report["errors"].append({"name": name, "error_type": type(error).__name__}) - (args.output / "manifest.json").write_text(json.dumps(report, indent=2) + "\n") - if report["errors"]: - break - report["completed"] = not report["errors"] and len(report["files"]) == len(packages) - (args.output / "manifest.json").write_text(json.dumps(report, indent=2) + "\n") - return 0 if report["completed"] else 1 - - -if __name__ == "__main__": - raise SystemExit(main()) diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-platform-fs-v1/check_driver.py b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-platform-fs-v1/check_driver.py deleted file mode 100644 index 9a61224d6..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-platform-fs-v1/check_driver.py +++ /dev/null @@ -1,78 +0,0 @@ -"""Mocked driver completion controls; no cluster, network or model work.""" - -import importlib.util -import json -import tempfile -from pathlib import Path -from unittest.mock import patch - -ROOT = Path(__file__).parent -spec = importlib.util.spec_from_file_location("platform_fs", ROOT / "platform_fs.py") -driver = importlib.util.module_from_spec(spec) -spec.loader.exec_module(driver) - - -def main(): - checks = [] - for case in ( - "complete", - "metadata-failure", - "filesystem-failure", - "late-timeout", - "deadline", - ): - now = [0.0] - - def metadata(case=case, now=now): - if case == "metadata-failure": - raise OSError("synthetic metadata error") - if case == "deadline": - now[0] = 119.5 - return {"completed": True, "fixture_only": True} - - def filesystem(_output, state, case=case): - state["fixture_partial_bytes"] = 7 - if case == "filesystem-failure": - raise ValueError("synthetic fs error") - state["completed"] = True - if case == "late-timeout": - raise TimeoutError("synthetic late timeout") - - with tempfile.TemporaryDirectory(dir=ROOT) as temporary: - result = {"completed": False} - with ( - patch.object(driver, "platform_tags", side_effect=metadata), - patch.object( - driver._probe, "filesystem_probe", side_effect=filesystem - ) as fs, - patch.object(driver.time, "monotonic", side_effect=lambda now=now: now[0]), - patch.object(driver.signal, "alarm"), - patch("builtins.print"), - ): - status = driver.diagnose(Path(temporary), result) - expected = case == "complete" - if (status == 0) != expected or result["completed"] is not expected: - raise RuntimeError("Incorrect driver completion") - if case == "deadline" and ( - fs.called - or result["filesystem"].get("skipped_reason") - != "overall_deadline_exhausted" - ): - raise RuntimeError("Skipped phase was executed or unrecorded") - if case in ("filesystem-failure", "late-timeout") and ( - result["filesystem"]["completed"] is not False - or result["filesystem"]["fixture_partial_bytes"] != 7 - ): - raise RuntimeError("Failure state lost") - checks.append(case) - report = { - "checks_passed": len(checks), - "checks": checks, - "real_network_or_filesystem_probe": False, - } - (ROOT / "controls.json").write_text(json.dumps(report, indent=2) + "\n") - print(json.dumps(report)) - - -if __name__ == "__main__": - main() diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-platform-fs-v1/platform_fs.py b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-platform-fs-v1/platform_fs.py deleted file mode 100644 index 9077d35d0..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-platform-fs-v1/platform_fs.py +++ /dev/null @@ -1,112 +0,0 @@ -"""Compute-node wheel tags and the previously skipped bounded filesystem probe.""" - -import argparse -import hashlib -import importlib.metadata -import importlib.util -import os -import platform -import signal -import sys -import sysconfig -import time -from pathlib import Path - -import packaging -import packaging.tags - -# Isolated Python deliberately omits the script directory from sys.path. -_spec = importlib.util.spec_from_file_location( - "reviewed_fs_probe", Path(__file__).with_name("probe.py") -) -_probe = importlib.util.module_from_spec(_spec) -_spec.loader.exec_module(_probe) - - -def platform_tags(): - package = Path(packaging.__file__).parent - sources = { - p.name: hashlib.sha256(p.read_bytes()).hexdigest() - for p in sorted(package.glob("*.py")) - } - tags = [str(tag) for tag in packaging.tags.sys_tags()] - if ( - sys.version_info[:2] != (3, 12) - or platform.system() != "Linux" - or platform.machine() != "x86_64" - ): - raise ValueError("Unexpected target interpreter/platform") - if not tags or len(tags) != len(set(tags)): - raise ValueError("Empty or duplicate wheel tag set") - return { - "completed": True, - "python": platform.python_version(), - "implementation": sys.implementation.name, - "system": platform.system(), - "machine": platform.machine(), - "libc": platform.libc_ver(), - "soabi": sysconfig.get_config_var("SOABI"), - "packaging_version": importlib.metadata.version("packaging"), - "packaging_sources": sources, - "supported_tags": tags, - } - - -def diagnose(output, result): - deadline = time.monotonic() + 120 - try: - for name, operation in ( - ("platform", lambda state: state.update(platform_tags())), - ("filesystem", lambda state: _probe.filesystem_probe(output, state)), - ): - state = result[name] = {"completed": False} - remaining = deadline - time.monotonic() - if remaining <= 1: - state["skipped_reason"] = "overall_deadline_exhausted" - else: - signal.alarm(max(1, int(remaining))) - try: - operation(state) - except (OSError, ValueError) as error: - state["completed"] = False - state["error_type"] = type(error).__name__ - _probe.persist(output, result) - result["completed"] = all( - result[name].get("completed") is True for name in ("platform", "filesystem") - ) - _probe.persist(output, result) - return 0 if result["completed"] else 1 - finally: - signal.alarm(0) - - -def main(): - parser = argparse.ArgumentParser(description=__doc__) - parser.add_argument("--output", type=Path, required=True) - args = parser.parse_args() - if ( - not os.environ.get("SLURM_JOB_ID") - or os.environ.get("SLURM_GPUS_ON_NODE", "0") != "0" - ): - raise ValueError("CPU-only Slurm allocation required") - root = Path(os.environ["TASK_ROOT"]).resolve(strict=True) - output = args.output.resolve(strict=False) - if output == root or not output.is_relative_to(root): - raise ValueError("Output escapes task root") - output.mkdir(mode=0o700, exist_ok=False) - result = { - "scope": "compute_platform_and_filesystem_readiness", - "verdict": "INCONCLUSIVE", - "completed": False, - "job_id": os.environ["SLURM_JOB_ID"], - "network_requested": False, - "package_changes": False, - "evaluator_executed": False, - "gpu_used": False, - } - signal.signal(signal.SIGALRM, _probe.fail_timeout) - return diagnose(output, result) - - -if __name__ == "__main__": - raise SystemExit(main()) diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-platform-fs-v1/probe.py b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-platform-fs-v1/probe.py deleted file mode 100644 index ae1656637..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-platform-fs-v1/probe.py +++ /dev/null @@ -1,215 +0,0 @@ -"""Bounded CPU diagnosis of wheel-prefix reads and task-filesystem writes. - -No retrieved bytes are installed, imported, extracted or reused for installation. -The prefix hash does not authenticate the complete wheel. Synthetic files remain -in a new private output directory. This does not measure service performance. -""" - -import argparse -import hashlib -import json -import os -import signal -import time -import urllib.request -from pathlib import Path -from urllib.parse import urlsplit - - -def sha(raw): - return hashlib.sha256(raw).hexdigest() - - -def fail_timeout(_number, _frame): - raise TimeoutError("bounded diagnostic deadline") - - -def measured(state, name, operation): - event = {"operation": name, "completed": False} - state["operations"].append(event) - started = time.monotonic() - try: - value = operation() - event["completed"] = True - return value - except (OSError, ValueError) as error: - event["error_type"] = type(error).__name__ - raise - finally: - event["seconds"] = time.monotonic() - started - - -def network_probe(policy, state): - length = policy["prefix_bytes"] - state.update( - completed=False, - received_bytes=0, - operations=[], - whole_wheel_authenticated=False, - payload_written_to_disk=False, - ) - request = urllib.request.Request( - policy["url"], headers={"Range": f"bytes=0-{length - 1}"} - ) - started = time.monotonic() - digest = hashlib.sha256() - try: - with measured( - state, "open", lambda: urllib.request.urlopen(request, timeout=15) - ) as response: - if ( - response.status != 206 - or urlsplit(response.url).hostname != "files.pythonhosted.org" - ): - raise ValueError("unexpected wheel-prefix endpoint or response") - expected = f"bytes 0-{length - 1}/{policy['whole_wheel_bytes']}" - if response.headers.get("Content-Range") != expected: - raise ValueError("unexpected wheel-prefix range") - while state["received_bytes"] < length: - count = min(262144, length - state["received_bytes"]) - chunk = measured( - state, "read", lambda count=count: response.read(count) - ) - state["operations"][-1]["received_bytes"] = len(chunk) - if not chunk: - raise ValueError("truncated prefix") - digest.update(chunk) - state["received_bytes"] += len(chunk) - state["completed"] = True - finally: - state["received_prefix_sha256"] = digest.hexdigest() - state["elapsed_seconds"] = time.monotonic() - started - - -def filesystem_probe(output, state): - state.update( - completed=False, - stream_bytes=0, - small_file_count=0, - small_file_bytes=0, - operations=[], - small_files_fsynced=False, - physical_storage_throughput_claimed=False, - ) - block = (b"diagnostic-file-content-" * 11400)[:262144] - if len(block) != 262144: - raise ValueError("synthetic block length") - started = time.monotonic() - try: - handle = measured( - state, - "stream_open", - lambda: (output / "synthetic-stream.bin").open("xb", buffering=0), - ) - try: - for _ in range(32): - size = measured(state, "stream_write", lambda: handle.write(block)) - state["stream_bytes"] += size - if size != len(block): - raise ValueError("short synthetic write") - measured(state, "stream_fsync", lambda: os.fsync(handle.fileno())) - finally: - measured(state, "stream_close", handle.close) - small = output / "synthetic-small-files" - measured(state, "small_mkdir", small.mkdir) - for i in range(128): - handle = measured( - state, - "small_open", - lambda i=i: (small / f"{i}.bin").open("xb", buffering=0), - ) - try: - size = measured( - state, - "small_write", - lambda handle=handle: handle.write(block[:4096]), - ) - state["small_file_bytes"] += size - if size != 4096: - raise ValueError("short synthetic small write") - finally: - measured(state, "small_close", handle.close) - state["small_file_count"] += 1 - state["completed"] = True - finally: - state["elapsed_seconds"] = time.monotonic() - started - - -def persist(output, result): - raw = json.dumps(result, sort_keys=True) - # Preserve observations in the Slurm log even if the result-file write fails. - print(raw, flush=True) - (output / "observations.json").write_text(raw + "\n") - - -def run_phases(policy, output, result): - deadline = time.monotonic() + 300 - try: - for name, operation in ( - ("network", lambda state: network_probe(policy, state)), - ("filesystem", lambda state: filesystem_probe(output, state)), - ): - state = result[name] = {"completed": False} - remaining = deadline - time.monotonic() - if remaining <= 1: - state["skipped_reason"] = "overall_deadline_exhausted" - else: - signal.alarm(max(1, int(remaining))) - try: - operation(state) - except (OSError, ValueError) as error: - state["completed"] = False - state["error_type"] = type(error).__name__ - persist(output, result) - result["completed"] = all( - result[name].get("completed") is True for name in ("network", "filesystem") - ) - persist(output, result) - return 0 if result["completed"] else 1 - finally: - signal.alarm(0) - - -def main(): - parser = argparse.ArgumentParser(description=__doc__) - parser.add_argument("--policy", type=Path, required=True) - parser.add_argument("--policy-sha256", required=True) - parser.add_argument("--output", type=Path, required=True) - args = parser.parse_args() - if ( - not os.environ.get("SLURM_JOB_ID") - or os.environ.get("SLURM_GPUS_ON_NODE", "0") != "0" - ): - raise ValueError("CPU-only Slurm allocation required") - root = Path(os.environ["TASK_ROOT"]).resolve(strict=True) - output = args.output.resolve(strict=False) - if not output.is_relative_to(root) or output == root: - raise ValueError("output escapes task root") - raw = args.policy.read_bytes() - if sha(raw) != args.policy_sha256: - raise ValueError("diagnostic policy changed") - policy = json.loads(raw) - if ( - policy["prefix_bytes"] != 8388608 - or policy["whole_wheel_bytes"] != 53931932 - or urlsplit(policy["url"]).scheme != "https" - or urlsplit(policy["url"]).hostname != "files.pythonhosted.org" - ): - raise ValueError("unexpected bounded wheel-prefix policy") - output.mkdir(mode=0o700, exist_ok=False) - result = { - "scope": "bounded_transport_and_filesystem_diagnostic", - "verdict": "INCONCLUSIVE", - "completed": False, - "job_id": os.environ["SLURM_JOB_ID"], - "policy_sha256": args.policy_sha256, - "gpu_used": False, - "installation_performed": False, - "evaluator_executed": False, - } - signal.signal(signal.SIGALRM, fail_timeout) - return run_phases(policy, output, result) - - -if __name__ == "__main__": - raise SystemExit(main()) diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-platform-fs-v1/run.sbatch b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-platform-fs-v1/run.sbatch deleted file mode 100644 index 2b1fe004f..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-platform-fs-v1/run.sbatch +++ /dev/null @@ -1,53 +0,0 @@ -#!/usr/bin/env bash -#SBATCH --job-name=qwen38-eval-platform-fs -#SBATCH --nodes=1 -#SBATCH --ntasks=1 -#SBATCH --cpus-per-task=1 -#SBATCH --mem=2G -#SBATCH --time=00:03:00 -set -euo pipefail -umask 077 -: "${TASK_ROOT:?}" "${BASE_PYTHON:?}" "${SOURCE_DIR:?}" "${OUTPUT_DIR:?}" -: "${SCRIPT_SHA256:?}" "${PROBE_SHA256:?}" "${DRIVER_SHA256:?}" "${PYTHON_SHA256:?}" -[[ -n ${SLURM_JOB_ID:-} && ${SLURM_GPUS_ON_NODE:-0} == 0 ]] -root=$(realpath -e "$TASK_ROOT") -inside_root() { - local path - path=$(realpath -m "$1") - case "$path" in "$root"/*) ;; *) exit 2;; esac -} -for path in "$BASE_PYTHON" "$SOURCE_DIR" "$OUTPUT_DIR"; do inside_root "$path"; done -check_digest() { - [[ $2 =~ ^[a-f0-9]{64}$ ]] - [[ $(sha256sum "$1" | cut -d' ' -f1) == "$2" ]] -} -check_digest "$SOURCE_DIR/run.sbatch" "$SCRIPT_SHA256" -check_digest "$SOURCE_DIR/probe.py" "$PROBE_SHA256" -check_digest "$SOURCE_DIR/platform_fs.py" "$DRIVER_SHA256" -check_digest "$BASE_PYTHON" "$PYTHON_SHA256" -cmp "${BASH_SOURCE[0]}" "$SOURCE_DIR/run.sbatch" -[[ ! -e "$OUTPUT_DIR" && ! -L "$OUTPUT_DIR" ]] -export TMPDIR="$root/tmp/qwen38-eval-platform-fs-${SLURM_JOB_ID}" -export XDG_CACHE_HOME="$root/cache/qwen38-eval-sandbox" -export UV_CACHE_DIR="$XDG_CACHE_HOME/uv" -export HF_HOME="$XDG_CACHE_HOME/huggingface" -export TORCH_HOME="$XDG_CACHE_HOME/torch" -export TRITON_CACHE_DIR="$XDG_CACHE_HOME/triton" -export VLLM_CACHE_ROOT="$XDG_CACHE_HOME/vllm" -export PYTHONNOUSERSITE=1 PYTHONDONTWRITEBYTECODE=1 -unset PYTHONPATH PYTHONHOME PYTHONINSPECT PYTHONSTARTUP PYTHONOPTIMIZE VIRTUAL_ENV -for path in "$TMPDIR" "$XDG_CACHE_HOME" "$UV_CACHE_DIR" "$HF_HOME" "$TORCH_HOME" "$TRITON_CACHE_DIR" "$VLLM_CACHE_ROOT"; do - inside_root "$path" - mkdir -p "$path" -done -cd "$root" -status=0 -"$BASE_PYTHON" -IB "$SOURCE_DIR/platform_fs.py" --output "$OUTPUT_DIR" || status=$? -if [[ -d "$OUTPUT_DIR" ]]; then - printf '%s\n' "$status" > "$OUTPUT_DIR/helper-exit-code" - cp "$SOURCE_DIR/run.sbatch" "$SOURCE_DIR/probe.py" "$SOURCE_DIR/platform_fs.py" "$OUTPUT_DIR/" -fi -check_digest "$SOURCE_DIR/run.sbatch" "$SCRIPT_SHA256" -check_digest "$SOURCE_DIR/probe.py" "$PROBE_SHA256" -check_digest "$SOURCE_DIR/platform_fs.py" "$DRIVER_SHA256" -exit "$status" diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-resolve-reviewed-wheels.sbatch b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-resolve-reviewed-wheels.sbatch deleted file mode 100644 index 40f6fceb4..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-resolve-reviewed-wheels.sbatch +++ /dev/null @@ -1,117 +0,0 @@ -#!/usr/bin/env bash -# Resolve-only successor using independently reviewed local dependency wheels. -#SBATCH --job-name=qwen38-eval-sandbox-prepare -#SBATCH --nodes=1 -#SBATCH --ntasks=1 -#SBATCH --cpus-per-task=4 -#SBATCH --mem=16G -#SBATCH --time=00:30:00 -set -euo pipefail -umask 077 -: "${TASK_ROOT:?set the authorized task root}" -: "${BASE_PYTHON:?set a registered Python3.12 under TASK_ROOT}" -: "${UV_BIN:?set the existing uv executable under TASK_ROOT}" -: "${SANDBOX_SOURCE_DIR:?set the explicit staged sandbox helper directory}" -: "${SANDBOX_HELPER_SHA256:?set the reviewed runtime.py digest}" -: "${SANDBOX_PROBE_SHA256:?set the reviewed benign_probe.py digest}" -: "${SANDBOX_SCRIPT_SHA256:?set the reviewed submission-script digest}" -: "${SANDBOX_BUNDLE:?set a new resolution bundle under TASK_ROOT}" -: "${SANDBOX_VENV:?name the future dedicated environment; never created here}" -: "${WHEEL_BUILD_BUNDLE:?set the reviewed build output directory}" -: "${WHEEL_BUILD_MANIFEST_SHA256:?set its independently reviewed output manifest hash}" -task_uv_executable=$UV_BIN -phase=${1:-} -case "$phase" in resolve) ;; *) echo 'Use resolve.' >&2; exit 2;; esac -[[ -n ${SLURM_JOB_ID:-} && ${SLURM_GPUS_ON_NODE:-0} == 0 ]] -root=$(realpath -e "$TASK_ROOT") -check_path() { - local value - value=$(realpath -m "$1") - case "$value" in "$root"/*) ;; *) echo 'Path escapes TASK_ROOT.' >&2; exit 2;; esac -} -for path in "$BASE_PYTHON" "$task_uv_executable" "$SANDBOX_SOURCE_DIR" "$SANDBOX_BUNDLE" "$SANDBOX_VENV" "$WHEEL_BUILD_BUNDLE"; do - check_path "$path" -done -source_dir=$(cd -- "$SANDBOX_SOURCE_DIR" && pwd -P) -check_digest() { - [[ "$2" =~ ^[a-f0-9]{64}$ ]] - [[ $(sha256sum "$1" | cut -d' ' -f1) == "$2" ]] -} -check_sources() { - check_digest "$source_dir/runtime.py" "$SANDBOX_HELPER_SHA256" - check_digest "$source_dir/benign_probe.py" "$SANDBOX_PROBE_SHA256" - check_digest "$source_dir/prepare-sandbox.sbatch" "$SANDBOX_SCRIPT_SHA256" -} -cmp "${BASH_SOURCE[0]}" "$source_dir/prepare-sandbox.sbatch" -check_sources -check_digest "$WHEEL_BUILD_BUNDLE/output.sha256" "$WHEEL_BUILD_MANIFEST_SHA256" -(cd "$WHEEL_BUILD_BUNDLE" && sha256sum --check --strict output.sha256) -for path in "$WHEEL_BUILD_BUNDLE/wheelhouse/wget-3.2-py3-none-any.whl" \ - "$WHEEL_BUILD_BUNDLE/wheelhouse/tempdir-0.7.1-py3-none-any.whl" \ - "$WHEEL_BUILD_BUNDLE/build.lock"; do check_path "$path"; done -# Ignore inherited resolver source/constraint/build policy; never print its values. -for task_resolver_variable in "${!UV_@}" "${!PIP_@}"; do - if [[ -n "$task_resolver_variable" ]]; then unset "$task_resolver_variable"; fi -done -export TMPDIR="$root/tmp/qwen38-eval-sandbox-${SLURM_JOB_ID}" -export XDG_CACHE_HOME="$root/cache/qwen38-eval-sandbox" -export UV_CACHE_DIR="$XDG_CACHE_HOME/uv" -export HF_HOME="$XDG_CACHE_HOME/huggingface" -export TORCH_HOME="$XDG_CACHE_HOME/torch" -export TRITON_CACHE_DIR="$XDG_CACHE_HOME/triton" -export VLLM_CACHE_ROOT="$XDG_CACHE_HOME/vllm" -export UV_PYTHON_INSTALL_DIR="$root/software/python" -export UV_PYTHON_DOWNLOADS=never -export UV_HTTP_TIMEOUT=60 -export PYTHONNOUSERSITE=1 -export PYTHONDONTWRITEBYTECODE=1 -unset PYTHONPATH VIRTUAL_ENV -for path in "$TMPDIR" "$XDG_CACHE_HOME" "$UV_CACHE_DIR" "$HF_HOME" "$TORCH_HOME" \ - "$TRITON_CACHE_DIR" "$VLLM_CACHE_ROOT" "$UV_PYTHON_INSTALL_DIR"; do - check_path "$path" -done -mkdir -p "$TMPDIR" "$XDG_CACHE_HOME" "$UV_CACHE_DIR" "$HF_HOME" "$TORCH_HOME" \ - "$TRITON_CACHE_DIR" "$VLLM_CACHE_ROOT" -cd "$root" -"$BASE_PYTHON" -c 'import sys; assert sys.version_info[:2] == (3, 12)' -source_revision=26d6d00bb1fd0fa37f39c99d5290da67891d1c5e -if [[ "$phase" == resolve ]]; then - [[ ! -e "$SANDBOX_BUNDLE" && ! -L "$SANDBOX_BUNDLE" ]] - mkdir "$SANDBOX_BUNDLE" - trap 'status=$?; printf "%s\n" "$status" > "$SANDBOX_BUNDLE/resolve.exit-code"; exit "$status"' EXIT - cp "$source_dir/runtime.py" "$source_dir/benign_probe.py" "$source_dir/prepare-sandbox.sbatch" "$SANDBOX_BUNDLE/" - mkdir "$SANDBOX_BUNDLE/wheelhouse" - cp "$WHEEL_BUILD_BUNDLE/wheelhouse/wget-3.2-py3-none-any.whl" \ - "$WHEEL_BUILD_BUNDLE/wheelhouse/tempdir-0.7.1-py3-none-any.whl" "$SANDBOX_BUNDLE/wheelhouse/" - cp "$WHEEL_BUILD_BUNDLE/build.lock" "$SANDBOX_BUNDLE/" - cp "$WHEEL_BUILD_BUNDLE/output.sha256" "$SANDBOX_BUNDLE/wheel-build-output.sha256" - git -c http.lowSpeedLimit=1 -c http.lowSpeedTime=60 clone --filter=blob:none --no-checkout \ - https://github.com/evalplus/evalplus.git "$SANDBOX_BUNDLE/source" - git -C "$SANDBOX_BUNDLE/source" fetch --depth=1 origin "$source_revision" - git -C "$SANDBOX_BUNDLE/source" checkout --detach "$source_revision" - "$BASE_PYTHON" "$source_dir/runtime.py" requirements --bundle "$SANDBOX_BUNDLE" - cat > "$SANDBOX_BUNDLE/resolver-policy.json" <<'POLICY' -{"phase":"resolve","config_files":false,"inherited_UV_and_PIP_variables":"cleared","default_index":"https://pypi.org/simple","additional_indexes":[],"find_links":["reviewed_local_wheelhouse"],"build_allowed":false,"python_downloads":"never","http_timeout_seconds":60,"generated_code_executed":false} -POLICY - "$task_uv_executable" --no-config pip compile "$SANDBOX_BUNDLE/dependencies.in" --python "$BASE_PYTHON" \ - --no-build --generate-hashes --default-index https://pypi.org/simple --no-header --no-annotate \ - --find-links "$SANDBOX_BUNDLE/wheelhouse" \ - --output-file "$SANDBOX_BUNDLE/dependencies.lock" - "$task_uv_executable" --version > "$SANDBOX_BUNDLE/uv-version.txt" - sha256sum "$task_uv_executable" | cut -d' ' -f1 > "$SANDBOX_BUNDLE/uv-sha256.txt" - sha256sum "$BASE_PYTHON" | cut -d' ' -f1 > "$SANDBOX_BUNDLE/python-sha256.txt" - check_sources - check_digest "$WHEEL_BUILD_BUNDLE/output.sha256" "$WHEEL_BUILD_MANIFEST_SHA256" - (cd "$WHEEL_BUILD_BUNDLE" && sha256sum --check --strict output.sha256) - for name in wget-3.2-py3-none-any.whl tempdir-0.7.1-py3-none-any.whl; do - cmp "$WHEEL_BUILD_BUNDLE/wheelhouse/$name" "$SANDBOX_BUNDLE/wheelhouse/$name" - done - cmp "$WHEEL_BUILD_BUNDLE/build.lock" "$SANDBOX_BUNDLE/build.lock" - ( - cd "$SANDBOX_BUNDLE" - sha256sum runtime.py benign_probe.py prepare-sandbox.sbatch resolver-policy.json source.json build.in dependencies.in \ - build.lock dependencies.lock wheel-build-output.sha256 wheelhouse/*.whl \ - uv-version.txt uv-sha256.txt python-sha256.txt > freeze.sha256 - ) - echo 'Resolution ready for independent source/dependency freeze review; no environment installed.' -fi diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-transport-v1/check_probe.py b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-transport-v1/check_probe.py deleted file mode 100644 index 7f0c39e92..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-transport-v1/check_probe.py +++ /dev/null @@ -1,226 +0,0 @@ -"""Synthetic I/O controls; no network request, installation or evaluator.""" - -import hashlib -import json -import tempfile -from pathlib import Path -from unittest.mock import patch - -import probe - -ROOT = Path(__file__).parent -POLICY = json.loads((ROOT / "policy.json").read_bytes()) - - -class Response: - def __init__(self, status=206, url=POLICY["url"], range_header=None, limit=8388608): - self.status, self.url, self.limit = status, url, limit - self.headers = {"Content-Range": range_header or "bytes 0-8388607/53931932"} - self.read_sizes = [] - - def __enter__(self): - return self - - def __exit__(self, *_args): - pass - - def read(self, count): - self.read_sizes.append(count) - actual = min(count, self.limit) - self.limit -= actual - return b"x" * actual - - -def main(): - checks = [] - for name, response in ( - ("wrong-status", Response(status=200)), - ("wrong-host", Response(url="https://example.com/ignored")), - ("wrong-range", Response(range_header="bytes 0-7/53931932")), - ("truncated", Response(limit=100)), - ): - with patch.object(probe.urllib.request, "urlopen", return_value=response): - try: - probe.network_probe(POLICY, {}) - except ValueError: - checks.append(name) - else: - raise RuntimeError("Invalid fixture accepted") - response = Response() - with patch.object(probe.urllib.request, "urlopen", return_value=response) as mocked: - result = {} - probe.network_probe(POLICY, result) - request = mocked.call_args.args[0] - if request.headers["Range"] != "bytes=0-8388607": - raise RuntimeError("Wrong requested range") - if sum(response.read_sizes) != 8388608 or max(response.read_sizes) != 262144: - raise RuntimeError("Read bound changed") - if ( - result["received_bytes"] != 8388608 - or result["whole_wheel_authenticated"] is not False - ): - raise RuntimeError("Prefix mistaken for whole wheel") - if ( - result["received_prefix_sha256"] - != hashlib.sha256(b"x" * 8388608).hexdigest() - ): - raise RuntimeError("Prefix hash differs") - checks.append("fixed-prefix-byte-and-range-bound") - with tempfile.TemporaryDirectory(dir=ROOT) as temporary: - directory = Path(temporary) - result = {} - probe.filesystem_probe(directory, result) - if (directory / "synthetic-stream.bin").stat().st_size != 8388608: - raise RuntimeError("Stream size differs") - sizes = [ - p.stat().st_size for p in (directory / "synthetic-small-files").iterdir() - ] - if len(sizes) != 128 or set(sizes) != {4096}: - raise RuntimeError("Small file bound changed") - if ( - sum(e["operation"] == "stream_write" for e in result["operations"]) != 32 - or result["small_files_fsynced"] is not False - ): - raise RuntimeError("Invalid filesystem observation") - checks.append("filesystem-exact-file-and-byte-bounds") - try: - probe.filesystem_probe(directory, {}) - except FileExistsError: - checks.append("existing-output-not-overwritten") - else: - raise RuntimeError("Existing output overwritten") - response = Response(limit=100) - partial = {} - with patch.object(probe.urllib.request, "urlopen", return_value=response): - try: - probe.network_probe(POLICY, partial) - except ValueError: - if ( - partial["received_bytes"] != 100 - or partial["completed"] is not False - or not partial["operations"] - ): - raise RuntimeError("Partial read observations lost") - checks.append("truncated-read-retains-partial-observations") - else: - raise RuntimeError("Truncation accepted") - - class BrokenWriter: - count = 0 - - def write(self, block): - self.count += 1 - if self.count > 1: - raise OSError("synthetic write error") - return len(block) - - def close(self): - pass - - partial = {} - with patch.object(Path, "open", return_value=BrokenWriter()): - try: - probe.filesystem_probe(ROOT, partial) - except OSError: - failed = [ - e for e in partial["operations"] if e.get("error_type") == "OSError" - ] - if ( - partial["stream_bytes"] != 262144 - or len(failed) != 1 - or failed[0]["operation"] != "stream_write" - ): - raise RuntimeError("Failed write observations lost") - checks.append("mid-write-retains-byte-count-and-failed-operation") - else: - raise RuntimeError("Write failure accepted") - with tempfile.TemporaryDirectory(dir=ROOT) as temporary: - partial = {} - with patch.object( - probe.os, "fsync", side_effect=OSError("synthetic sync error") - ): - try: - probe.filesystem_probe(Path(temporary), partial) - except OSError: - failed = [ - e for e in partial["operations"] if e.get("error_type") == "OSError" - ] - if ( - partial["stream_bytes"] != 8388608 - or failed[0]["operation"] != "stream_fsync" - ): - raise RuntimeError("Fsync failure observations lost") - checks.append("fsync-retains-partial-observations") - else: - raise RuntimeError("Fsync failure accepted") - - now = [0.0] - - def slow_network(_policy, state): - state["completed"] = True - now[0] = 299.5 - - with tempfile.TemporaryDirectory(dir=ROOT) as temporary: - observed = {"completed": False} - with ( - patch.object(probe.time, "monotonic", side_effect=lambda: now[0]), - patch.object(probe.signal, "alarm"), - patch.object(probe, "network_probe", side_effect=slow_network), - patch.object(probe, "filesystem_probe") as filesystem, - patch("builtins.print"), - ): - status = probe.run_phases(POLICY, Path(temporary), observed) - if status != 1 or observed["completed"] is not False or filesystem.called: - raise RuntimeError("Skipped phase reported completion") - if observed["filesystem"].get("skipped_reason") != "overall_deadline_exhausted": - raise RuntimeError("Missing skip reason") - checks.append("deadline-skip-is-nonzero-and-incomplete") - with ( - patch.object(Path, "write_text", side_effect=OSError("synthetic output error")), - patch("builtins.print") as output, - ): - try: - probe.persist(ROOT, {"partial": True}) - except OSError: - if not output.called or '"partial": true' not in output.call_args.args[0]: - raise RuntimeError("Partial result missing from log fallback") - checks.append("result-file-error-retains-log-output") - else: - raise RuntimeError("Result write error ignored") - - def late_timeout(_policy, state): - state["completed"] = True - raise TimeoutError("synthetic return-boundary timeout") - - def complete_fs(_output, state): - state["completed"] = True - - with tempfile.TemporaryDirectory(dir=ROOT) as temporary: - observed = {"completed": False} - with ( - patch.object(probe.signal, "alarm"), - patch.object(probe, "network_probe", side_effect=late_timeout), - patch.object(probe, "filesystem_probe", side_effect=complete_fs), - patch("builtins.print"), - ): - status = probe.run_phases(POLICY, Path(temporary), observed) - if ( - status != 1 - or observed["completed"] is not False - or observed["network"]["completed"] is not False - ): - raise RuntimeError("Late phase timeout reported success") - checks.append("late-timeout-clears-phase-completion") - report = { - "checks_passed": len(checks), - "checks": checks, - "network_called": False, - "gpu_used": False, - "evaluator_executed": False, - } - (ROOT / "controls.json").write_text(json.dumps(report, indent=2) + "\n") - print(json.dumps(report)) - - -if __name__ == "__main__": - main() diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-transport-v1/policy.json b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-transport-v1/policy.json deleted file mode 100644 index 14c52196a..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-transport-v1/policy.json +++ /dev/null @@ -1,9 +0,0 @@ -{ - "schema_version": 1, - "purpose": "diagnostic_prefix_only_not_installable", - "url": "https://files.pythonhosted.org/packages/88/e4/9cd356d906e71bd79b0c3fc5c9a54e01a0020dcf14c152ccfbcb503c7298/pyarrow-26.0.0-cp312-cp312-manylinux_2_28_x86_64.whl", - "whole_wheel_bytes": 53931932, - "whole_wheel_sha256": "6a628922ba20705fa964ca73e4ef959c2fb2f14b9bbec5589a6a1e68e6257c85", - "pypi_metadata_sha256": "820e0086a99265aeb261219e687d72eae106dba6290a66683315b4fe85276be5", - "prefix_bytes": 8388608 -} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-transport-v1/probe.py b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-transport-v1/probe.py deleted file mode 100644 index ae1656637..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-transport-v1/probe.py +++ /dev/null @@ -1,215 +0,0 @@ -"""Bounded CPU diagnosis of wheel-prefix reads and task-filesystem writes. - -No retrieved bytes are installed, imported, extracted or reused for installation. -The prefix hash does not authenticate the complete wheel. Synthetic files remain -in a new private output directory. This does not measure service performance. -""" - -import argparse -import hashlib -import json -import os -import signal -import time -import urllib.request -from pathlib import Path -from urllib.parse import urlsplit - - -def sha(raw): - return hashlib.sha256(raw).hexdigest() - - -def fail_timeout(_number, _frame): - raise TimeoutError("bounded diagnostic deadline") - - -def measured(state, name, operation): - event = {"operation": name, "completed": False} - state["operations"].append(event) - started = time.monotonic() - try: - value = operation() - event["completed"] = True - return value - except (OSError, ValueError) as error: - event["error_type"] = type(error).__name__ - raise - finally: - event["seconds"] = time.monotonic() - started - - -def network_probe(policy, state): - length = policy["prefix_bytes"] - state.update( - completed=False, - received_bytes=0, - operations=[], - whole_wheel_authenticated=False, - payload_written_to_disk=False, - ) - request = urllib.request.Request( - policy["url"], headers={"Range": f"bytes=0-{length - 1}"} - ) - started = time.monotonic() - digest = hashlib.sha256() - try: - with measured( - state, "open", lambda: urllib.request.urlopen(request, timeout=15) - ) as response: - if ( - response.status != 206 - or urlsplit(response.url).hostname != "files.pythonhosted.org" - ): - raise ValueError("unexpected wheel-prefix endpoint or response") - expected = f"bytes 0-{length - 1}/{policy['whole_wheel_bytes']}" - if response.headers.get("Content-Range") != expected: - raise ValueError("unexpected wheel-prefix range") - while state["received_bytes"] < length: - count = min(262144, length - state["received_bytes"]) - chunk = measured( - state, "read", lambda count=count: response.read(count) - ) - state["operations"][-1]["received_bytes"] = len(chunk) - if not chunk: - raise ValueError("truncated prefix") - digest.update(chunk) - state["received_bytes"] += len(chunk) - state["completed"] = True - finally: - state["received_prefix_sha256"] = digest.hexdigest() - state["elapsed_seconds"] = time.monotonic() - started - - -def filesystem_probe(output, state): - state.update( - completed=False, - stream_bytes=0, - small_file_count=0, - small_file_bytes=0, - operations=[], - small_files_fsynced=False, - physical_storage_throughput_claimed=False, - ) - block = (b"diagnostic-file-content-" * 11400)[:262144] - if len(block) != 262144: - raise ValueError("synthetic block length") - started = time.monotonic() - try: - handle = measured( - state, - "stream_open", - lambda: (output / "synthetic-stream.bin").open("xb", buffering=0), - ) - try: - for _ in range(32): - size = measured(state, "stream_write", lambda: handle.write(block)) - state["stream_bytes"] += size - if size != len(block): - raise ValueError("short synthetic write") - measured(state, "stream_fsync", lambda: os.fsync(handle.fileno())) - finally: - measured(state, "stream_close", handle.close) - small = output / "synthetic-small-files" - measured(state, "small_mkdir", small.mkdir) - for i in range(128): - handle = measured( - state, - "small_open", - lambda i=i: (small / f"{i}.bin").open("xb", buffering=0), - ) - try: - size = measured( - state, - "small_write", - lambda handle=handle: handle.write(block[:4096]), - ) - state["small_file_bytes"] += size - if size != 4096: - raise ValueError("short synthetic small write") - finally: - measured(state, "small_close", handle.close) - state["small_file_count"] += 1 - state["completed"] = True - finally: - state["elapsed_seconds"] = time.monotonic() - started - - -def persist(output, result): - raw = json.dumps(result, sort_keys=True) - # Preserve observations in the Slurm log even if the result-file write fails. - print(raw, flush=True) - (output / "observations.json").write_text(raw + "\n") - - -def run_phases(policy, output, result): - deadline = time.monotonic() + 300 - try: - for name, operation in ( - ("network", lambda state: network_probe(policy, state)), - ("filesystem", lambda state: filesystem_probe(output, state)), - ): - state = result[name] = {"completed": False} - remaining = deadline - time.monotonic() - if remaining <= 1: - state["skipped_reason"] = "overall_deadline_exhausted" - else: - signal.alarm(max(1, int(remaining))) - try: - operation(state) - except (OSError, ValueError) as error: - state["completed"] = False - state["error_type"] = type(error).__name__ - persist(output, result) - result["completed"] = all( - result[name].get("completed") is True for name in ("network", "filesystem") - ) - persist(output, result) - return 0 if result["completed"] else 1 - finally: - signal.alarm(0) - - -def main(): - parser = argparse.ArgumentParser(description=__doc__) - parser.add_argument("--policy", type=Path, required=True) - parser.add_argument("--policy-sha256", required=True) - parser.add_argument("--output", type=Path, required=True) - args = parser.parse_args() - if ( - not os.environ.get("SLURM_JOB_ID") - or os.environ.get("SLURM_GPUS_ON_NODE", "0") != "0" - ): - raise ValueError("CPU-only Slurm allocation required") - root = Path(os.environ["TASK_ROOT"]).resolve(strict=True) - output = args.output.resolve(strict=False) - if not output.is_relative_to(root) or output == root: - raise ValueError("output escapes task root") - raw = args.policy.read_bytes() - if sha(raw) != args.policy_sha256: - raise ValueError("diagnostic policy changed") - policy = json.loads(raw) - if ( - policy["prefix_bytes"] != 8388608 - or policy["whole_wheel_bytes"] != 53931932 - or urlsplit(policy["url"]).scheme != "https" - or urlsplit(policy["url"]).hostname != "files.pythonhosted.org" - ): - raise ValueError("unexpected bounded wheel-prefix policy") - output.mkdir(mode=0o700, exist_ok=False) - result = { - "scope": "bounded_transport_and_filesystem_diagnostic", - "verdict": "INCONCLUSIVE", - "completed": False, - "job_id": os.environ["SLURM_JOB_ID"], - "policy_sha256": args.policy_sha256, - "gpu_used": False, - "installation_performed": False, - "evaluator_executed": False, - } - signal.signal(signal.SIGALRM, fail_timeout) - return run_phases(policy, output, result) - - -if __name__ == "__main__": - raise SystemExit(main()) diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-transport-v1/run.sbatch b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-transport-v1/run.sbatch deleted file mode 100644 index 8ab9715ff..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/diagnostics/sandbox-transport-v1/run.sbatch +++ /dev/null @@ -1,54 +0,0 @@ -#!/usr/bin/env bash -#SBATCH --job-name=qwen38-eval-transport-diagnostic -#SBATCH --nodes=1 -#SBATCH --ntasks=1 -#SBATCH --cpus-per-task=1 -#SBATCH --mem=2G -#SBATCH --time=00:06:00 -set -euo pipefail -umask 077 -: "${TASK_ROOT:?}" "${BASE_PYTHON:?}" "${SOURCE_DIR:?}" "${OUTPUT_DIR:?}" -: "${SCRIPT_SHA256:?}" "${PROBE_SHA256:?}" "${POLICY_SHA256:?}" "${PYTHON_SHA256:?}" -[[ -n ${SLURM_JOB_ID:-} && ${SLURM_GPUS_ON_NODE:-0} == 0 ]] -root=$(realpath -e "$TASK_ROOT") -inside_root() { - local path - path=$(realpath -m "$1") - case "$path" in "$root"/*) ;; *) exit 2;; esac -} -for path in "$BASE_PYTHON" "$SOURCE_DIR" "$OUTPUT_DIR"; do inside_root "$path"; done -check_digest() { - [[ $2 =~ ^[a-f0-9]{64}$ ]] - [[ $(sha256sum "$1" | cut -d' ' -f1) == "$2" ]] -} -check_digest "$SOURCE_DIR/run.sbatch" "$SCRIPT_SHA256" -check_digest "$SOURCE_DIR/probe.py" "$PROBE_SHA256" -check_digest "$SOURCE_DIR/policy.json" "$POLICY_SHA256" -check_digest "$BASE_PYTHON" "$PYTHON_SHA256" -cmp "${BASH_SOURCE[0]}" "$SOURCE_DIR/run.sbatch" -[[ ! -e "$OUTPUT_DIR" && ! -L "$OUTPUT_DIR" ]] -export TMPDIR="$root/tmp/qwen38-eval-transport-${SLURM_JOB_ID}" -export XDG_CACHE_HOME="$root/cache/qwen38-eval-sandbox" -export UV_CACHE_DIR="$XDG_CACHE_HOME/uv" -export HF_HOME="$XDG_CACHE_HOME/huggingface" -export TORCH_HOME="$XDG_CACHE_HOME/torch" -export TRITON_CACHE_DIR="$XDG_CACHE_HOME/triton" -export VLLM_CACHE_ROOT="$XDG_CACHE_HOME/vllm" -export PYTHONNOUSERSITE=1 PYTHONDONTWRITEBYTECODE=1 -unset PYTHONPATH PYTHONHOME PYTHONINSPECT PYTHONSTARTUP PYTHONOPTIMIZE VIRTUAL_ENV -for path in "$TMPDIR" "$XDG_CACHE_HOME" "$UV_CACHE_DIR" "$HF_HOME" "$TORCH_HOME" "$TRITON_CACHE_DIR" "$VLLM_CACHE_ROOT"; do - inside_root "$path" - mkdir -p "$path" -done -cd "$root" -status=0 -"$BASE_PYTHON" -IB "$SOURCE_DIR/probe.py" --policy "$SOURCE_DIR/policy.json" \ - --policy-sha256 "$POLICY_SHA256" --output "$OUTPUT_DIR" || status=$? -if [[ -d "$OUTPUT_DIR" ]]; then - printf '%s\n' "$status" > "$OUTPUT_DIR/helper-exit-code" - cp "$SOURCE_DIR/run.sbatch" "$SOURCE_DIR/probe.py" "$SOURCE_DIR/policy.json" "$OUTPUT_DIR/" -fi -check_digest "$SOURCE_DIR/run.sbatch" "$SCRIPT_SHA256" -check_digest "$SOURCE_DIR/probe.py" "$PROBE_SHA256" -check_digest "$SOURCE_DIR/policy.json" "$POLICY_SHA256" -exit "$status" diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/flashinfer-small-m/engine.patch b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/flashinfer-small-m/engine.patch deleted file mode 100644 index e478f5461..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/flashinfer-small-m/engine.patch +++ /dev/null @@ -1,228 +0,0 @@ -diff --git a/docs/qwen38-phase1.md b/docs/qwen38-phase1.md -index a14d5b7..9f78c58 100644 ---- a/docs/qwen38-phase1.md -+++ b/docs/qwen38-phase1.md -@@ -96,3 +96,18 @@ CPU socket tests do not establish that service acceptance. - The model math is derived from the Apache-2.0 Transformers Qwen3.5 source. - See [NOTICE-qwen38](../NOTICE-qwen38) and [Apache license](../LICENSES/Apache-2.0.txt). - The original nano-vllm code retains its [MIT license](../LICENSE). -+# Optional small-M FP8 GEMM -+ -+`NANOVLLM_FP8_GEMM=flashinfer` selects FlashInfer's SM90 block-scaled -+FP8 GEMM for fewer than 32 activation rows, including single-token decoding. -+Larger inputs retain the existing Triton prefill implementation. This opt-in -+requires `flashinfer-python` with `flashinfer.gemm.fp8_blockscale_gemm_sm90`; -+missing dependencies or unsupported shapes raise an error. The default remains -+`triton`. Weights stay E4M3 and checkpoint block multipliers stay FP32; FlashInfer -+quantizes BF16 activations internally and returns BF16 outputs. -+ -+The loader reports this mixed backend, and provenance separates FlashInfer and -+Triton Python call counts. These counts include warmup/capture and exclude CUDA -+Graph replay. Selecting the backend does not establish model quality or service -+performance; those require the governing acceptance campaign's evidence. -+ -diff --git a/nanovllm/qwen38/engine.py b/nanovllm/qwen38/engine.py -index 1d770cb..f9f93cd 100644 ---- a/nanovllm/qwen38/engine.py -+++ b/nanovllm/qwen38/engine.py -@@ -166,7 +166,7 @@ class DedicatedEngine: - origins[name] = {"path": str(Path(path).resolve()), - "sha256": hashlib.sha256(Path(path).read_bytes()).hexdigest()} - versions = {name: str(getattr(sys.modules[name], "__version__", "unknown")) -- for name in ("torch", "transformers", "triton") if name in sys.modules} -+ for name in ("torch", "transformers", "triton", "flashinfer") if name in sys.modules} - device = {"device_type": "synthetic", "device_count": 0, "device_model": "test_only"} - if self.loader_report["dtype"] != "test_only": - import torch -@@ -185,6 +185,8 @@ class DedicatedEngine: - "backend": self.loader_report["backend"], - "kernel_name": self.loader_report["kernel_name"], - "executed_modules": sum(count > 0 for count in calls.values()), "per_module_calls": calls, -+ "kernel_calls": {kernel: sum(module.kernel_calls[kernel] for module in modules.values()) -+ for kernel in ("flashinfer", "triton")}, - "weight_dtype": "float8_e4m3fn", "scale_dtype": "float32", - "actual_dtypes_valid": all(module.weight.dtype == torch.float8_e4m3fn - and module.weight_scale_inv.dtype == torch.float32 -diff --git a/nanovllm/qwen38/loader.py b/nanovllm/qwen38/loader.py -index f3abdae..444c6b0 100644 ---- a/nanovllm/qwen38/loader.py -+++ b/nanovllm/qwen38/loader.py -@@ -126,7 +126,7 @@ def load_weights(model: torch.nn.Module, model_path: str | Path) -> dict: - report = {"loaded_parameters": len(mapped), "excluded_tensors": skipped, - "scope": "text_only", "dtype": "bfloat16"} - if fp8_modules: -- report.update(fp8_report(len(fp8_modules))) -+ report.update(fp8_report(fp8_modules.values())) - return report - - -diff --git a/nanovllm/qwen38/quant.py b/nanovllm/qwen38/quant.py -index 6df0144..1a06f0a 100644 ---- a/nanovllm/qwen38/quant.py -+++ b/nanovllm/qwen38/quant.py -@@ -24,11 +24,15 @@ class FP8Linear(nn.Module): - def __init__(self, in_features: int, out_features: int, *, device=None): - super().__init__() - requested = os.environ.get("NANOVLLM_FP8_GEMM", "triton") -- if requested not in ("triton", self.backend): -+ if requested not in ("triton", self.backend, "flashinfer"): - raise ValueError( - f"Unsupported NANOVLLM_FP8_GEMM={requested!r}; this build implements " -- f"only {self.backend}. No fallback was selected." -+ "triton and flashinfer (small-M only). No fallback was selected." - ) -+ self.use_flashinfer = requested == "flashinfer" -+ if self.use_flashinfer: -+ self.backend = "flashinfer_small_m_triton_prefill" -+ self.kernel_name = "fp8_blockscale_gemm_sm90+qwen38_block_fp8" - if in_features <= 0 or out_features <= 0: - raise ValueError("FP8 linear dimensions must be positive") - self.in_features = in_features -@@ -41,6 +45,7 @@ class FP8Linear(nn.Module): - device=device, dtype=torch.float32, - ), requires_grad=False) - self.native_fp8_calls = 0 -+ self.kernel_calls = {"flashinfer": 0, "triton": 0} - - def forward(self, inputs: torch.Tensor) -> torch.Tensor: - if inputs.ndim < 2 or inputs.shape[-1] != self.in_features or inputs.numel() == 0: -@@ -51,12 +56,31 @@ class FP8Linear(nn.Module): - raise ValueError("Native FP8 execution requires colocated CUDA tensors") - if torch.cuda.get_device_capability(inputs.device) != (9, 0): - raise ValueError("Dedicated FP8 kernel requires SM90") -- from ._triton_fp8 import matmul -- -- result = matmul(inputs.contiguous(), self.weight, self.weight_scale_inv) -+ result = self._matmul(inputs) - self.native_fp8_calls += 1 - return result - -+ def _matmul(self, inputs: torch.Tensor) -> torch.Tensor: -+ # Match vLLM's small-M SM90 choice. Large prefill keeps the existing -+ # kernel; FlashInfer's BF16 input path quantizes activations internally. -+ rows = inputs.numel() // self.in_features -+ if self.use_flashinfer and rows < 32: -+ if self.in_features % 128 or self.out_features % 64: -+ raise ValueError("FlashInfer FP8 requires K divisible by 128 and N by 64") -+ from flashinfer.gemm import fp8_blockscale_gemm_sm90 -+ -+ result = fp8_blockscale_gemm_sm90( -+ inputs.reshape(rows, self.in_features).contiguous(), self.weight, -+ weight_scale=self.weight_scale_inv, out_dtype=torch.bfloat16, -+ ).reshape(*inputs.shape[:-1], self.out_features) -+ self.kernel_calls["flashinfer"] += 1 -+ else: -+ from ._triton_fp8 import matmul -+ -+ result = matmul(inputs.contiguous(), self.weight, self.weight_scale_inv) -+ self.kernel_calls["triton"] += 1 -+ return result -+ - - def convert_fp8_linears(model: nn.Module, exclusions: tuple[str, ...]) -> int: - """Replace precisely the supported text projections, checking HF exclusions. -@@ -93,12 +117,17 @@ def convert_fp8_linears(model: nn.Module, exclusions: tuple[str, ...]) -> int: - return len(selected) - - --def fp8_report(count: int) -> dict: -+def fp8_report(modules) -> dict: -+ modules = tuple(modules) -+ backends = {(module.backend, module.kernel_name) for module in modules} -+ if len(backends) != 1: -+ raise ValueError("FP8 loading requires one consistent backend selection") -+ backend, kernel_name = next(iter(backends)) - return { - "dtype": "fp8", "activation_dtype": "bfloat16", - "weight_dtype": "float8_e4m3fn", "scale_dtype": "float32", - "checkpoint_scale_dtype": "bfloat16", "quantization": dict(FP8_QUANTIZATION), -- "execution": "native_fp8", "fp8_linear_modules": count, -- "dequantized": False, "backend": FP8Linear.backend, -- "kernel_name": FP8Linear.kernel_name, -+ "execution": "native_fp8", "fp8_linear_modules": len(modules), -+ "dequantized": False, "backend": backend, -+ "kernel_name": kernel_name, - } -diff --git a/tests/test_qwen38_quant.py b/tests/test_qwen38_quant.py -index 8dd0767..2f8f0ce 100644 ---- a/tests/test_qwen38_quant.py -+++ b/tests/test_qwen38_quant.py -@@ -1,6 +1,8 @@ - """CPU structural and checkpoint tests, not evidence of CUDA FP8 execution.""" - - import json -+import sys -+from types import ModuleType - - import pytest - import torch -@@ -9,10 +11,10 @@ from safetensors.torch import save_file - from nanovllm.qwen38.config import Qwen38TextConfig - from nanovllm.qwen38.loader import load_weights - from nanovllm.qwen38.model import Qwen38ForCausalLM --from nanovllm.qwen38.quant import FP8Linear, convert_fp8_linears -+from nanovllm.qwen38.quant import FP8Linear, convert_fp8_linears, fp8_report - - --@pytest.mark.parametrize("backend", ["flashinfer", "deepgemm", "typo"]) -+@pytest.mark.parametrize("backend", ["deepgemm", "typo"]) - def test_unimplemented_backend_is_not_silently_ignored(monkeypatch, backend): - monkeypatch.setenv("NANOVLLM_FP8_GEMM", backend) - with pytest.raises(ValueError, match="Unsupported NANOVLLM_FP8_GEMM"): -@@ -27,6 +29,56 @@ def test_supported_backend_aliases_report_implemented_kernel(monkeypatch, backen - assert module.kernel_name == "qwen38_block_fp8" - - -+@pytest.mark.parametrize("shape,expected", [((1, 128), "flashinfer"), ((1, 31, 128), "flashinfer"), -+ ((2, 16, 128), "triton"), ((33, 128), "triton")]) -+def test_small_m_dispatch_preserves_shape_and_checkpoint_scales(monkeypatch, shape, expected): -+ monkeypatch.setenv("NANOVLLM_FP8_GEMM", "flashinfer") -+ module = FP8Linear(128, 64) -+ inputs = torch.zeros(shape, dtype=torch.bfloat16) -+ seen = [] -+ -+ def flashinfer(x, weight, *, weight_scale, out_dtype): -+ assert x.ndim == 2 and x.is_contiguous() and out_dtype == torch.bfloat16 -+ assert weight is module.weight and weight_scale is module.weight_scale_inv -+ seen.append("flashinfer") -+ return x.new_zeros(x.shape[0], 64) -+ -+ def triton(x, weight, scales): -+ assert weight is module.weight and scales is module.weight_scale_inv -+ seen.append("triton") -+ return x.new_zeros(*x.shape[:-1], 64) -+ -+ fake_flashinfer = ModuleType("flashinfer.gemm") -+ fake_flashinfer.fp8_blockscale_gemm_sm90 = flashinfer -+ fake_triton = ModuleType("nanovllm.qwen38._triton_fp8") -+ fake_triton.matmul = triton -+ monkeypatch.setitem(sys.modules, "flashinfer.gemm", fake_flashinfer) -+ monkeypatch.setitem(sys.modules, "nanovllm.qwen38._triton_fp8", fake_triton) -+ output = module._matmul(inputs) -+ assert output.shape == (*shape[:-1], 64) and output.dtype == torch.bfloat16 -+ assert seen == [expected] and module.kernel_calls[expected] == 1 -+ -+ -+def test_flashinfer_missing_dependency_does_not_fallback(monkeypatch): -+ monkeypatch.setenv("NANOVLLM_FP8_GEMM", "flashinfer") -+ module = FP8Linear(128, 64) -+ monkeypatch.setitem(sys.modules, "flashinfer.gemm", None) -+ with pytest.raises(ModuleNotFoundError): -+ module._matmul(torch.zeros(1, 128, dtype=torch.bfloat16)) -+ assert module.native_fp8_calls == 0 and sum(module.kernel_calls.values()) == 0 -+ -+ -+def test_report_uses_constructed_backend_not_current_environment(monkeypatch, tmp_path): -+ monkeypatch.setenv("NANOVLLM_FP8_GEMM", "flashinfer") -+ model, _ = tiny_fp8_model() -+ write_fp8_checkpoint(tmp_path, model) -+ monkeypatch.setenv("NANOVLLM_FP8_GEMM", "triton") -+ assert load_weights(model, tmp_path)["backend"] == "flashinfer_small_m_triton_prefill" -+ modules = [module for module in model.modules() if isinstance(module, FP8Linear)] -+ with pytest.raises(ValueError, match="consistent backend"): -+ fp8_report([*modules, FP8Linear(128, 64)]) -+ -+ - def tiny_fp8_model(): - model = Qwen38ForCausalLM(Qwen38TextConfig.for_testing()).to(torch.bfloat16) - exclusions = ("lm_head", "model.language_model.layers.0.linear_attn.in_proj_a", diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/flashinfer-small-m/gpu.redacted.sbatch b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/flashinfer-small-m/gpu.redacted.sbatch deleted file mode 100644 index 396395357..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/flashinfer-small-m/gpu.redacted.sbatch +++ /dev/null @@ -1,33 +0,0 @@ -# Redacted operational copy; supplied privately. Hash differs from submitted wrapper. -#!/usr/bin/env bash -#SBATCH --job-name=qwen38-fi-gemm -#SBATCH --partition=h20 -#SBATCH --nodes=1 -#SBATCH --ntasks=1 -#SBATCH --gres=gpu:1 -#SBATCH --cpus-per-task=8 -#SBATCH --mem=96G -#SBATCH --time=00:30:00 -#SBATCH --chdir=/artifacts/qwen38-flashinfer-gemm-v3 -#SBATCH --output=/artifacts/qwen38-flashinfer-gemm-v3/baseline-%j.log -set -euo pipefail -umask 077 -TASK_ROOT= -TASK_OUT="$TASK_ROOT/artifacts/qwen38-flashinfer-gemm-v3" -export TMPDIR="$TASK_OUT/tmp" XDG_CACHE_HOME="$TASK_OUT/baseline-cache" -export HF_HOME="$TASK_OUT/baseline-cache/hf" TORCH_HOME="$TASK_OUT/baseline-cache/torch" -export UV_CACHE_DIR="$TASK_OUT/baseline-cache/uv" TRITON_CACHE_DIR="$TASK_OUT/baseline-cache/triton" -export VLLM_CACHE_ROOT="$TASK_OUT/baseline-cache/vllm" CUDA_CACHE_PATH="$TASK_OUT/baseline-cache/cuda" -export FLASHINFER_WORKSPACE_BASE="$TASK_ROOT/artifacts/qwen38-benchmark-http/baseline-cache/flashinfer" -export PYTHONDONTWRITEBYTECODE=1 TOKENIZERS_PARALLELISM=false HF_HUB_OFFLINE=1 -unset PYTHONPATH OMP_NUM_THREADS -mkdir -p "$TMPDIR" "$XDG_CACHE_HOME" "$HF_HOME" "$TORCH_HOME" "$UV_CACHE_DIR" "$TRITON_CACHE_DIR" "$VLLM_CACHE_ROOT" "$CUDA_CACHE_PATH" "$FLASHINFER_WORKSPACE_BASE" -trap 'task_rc=$?; printf "%s\n" "$task_rc" > "$TASK_OUT/baseline-exit-$SLURM_JOB_ID.txt"' EXIT -date -u +%FT%TZ -printf 'job=%s visible=%s\n' "$SLURM_JOB_ID" "${CUDA_VISIBLE_DEVICES:-unset}" -nvidia-smi --query-gpu=name,uuid,memory.total,driver_version --format=csv,noheader -sha256sum source.tar operator_probe.py http_probe.py > "$TASK_OUT/sources-$SLURM_JOB_ID.sha256" -export PYTHONPATH="$TASK_ROOT/sources/nanovllm-qwen38-flashinfer-gemm" -timeout --signal=TERM --kill-after=20s 400s "$TASK_ROOT/software/qwen38-vllm032/bin/python" -u operator_probe.py --output "$TASK_OUT/operator-$SLURM_JOB_ID.json" -timeout --signal=TERM --kill-after=35s 1300s "$TASK_ROOT/software/qwen38-vllm032/bin/python" -u http_probe.py --source "$PYTHONPATH" --model "$TASK_ROOT/models/Qwen3.8-27B-FP8" --fixture "$TASK_ROOT/artifacts/qwen38-vllm032-readiness/fixture-v1/fixture.json" --output "$TASK_OUT/http-$SLURM_JOB_ID" -date -u +%FT%TZ diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/flashinfer-small-m/http_probe.py b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/flashinfer-small-m/http_probe.py deleted file mode 100644 index 63bb41b71..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/flashinfer-small-m/http_probe.py +++ /dev/null @@ -1,181 +0,0 @@ -"""Bounded baseline feasibility; no performance acceptance or candidate search.""" -import argparse -import asyncio -import dataclasses -import hashlib -import importlib.metadata -import json -import os -from pathlib import Path -import signal -import socket -import subprocess -import sys -import time -import urllib.request - -import aiohttp -import vllm.benchmarks.lib.endpoint_request_func as client - - -class RecordingResponse: - def __init__(self, response, chunks): - self.response, self.chunks = response, chunks - self.status, self.reason, self.content = response.status, response.reason, self - - async def iter_any(self): - async for chunk in self.response.content.iter_any(): - self.chunks.append(chunk) - yield chunk - - -class RecordingContext: - def __init__(self, context, chunks): - self.context, self.chunks = context, chunks - - async def __aenter__(self): - return RecordingResponse(await self.context.__aenter__(), self.chunks) - - async def __aexit__(self, *args): - return await self.context.__aexit__(*args) - - -class RecordingSession: - def __init__(self, session, chunks): - self.session, self.chunks = session, chunks - - def post(self, *args, **kwargs): - return RecordingContext(self.session.post(*args, **kwargs), self.chunks) - - -def digest(path): - return hashlib.sha256(Path(path).read_bytes()).hexdigest() - - -def group_members(pgid): - rows = subprocess.check_output(['ps', '-eo', 'pid=,pgid='], text=True) - return [int(pid) for pid, group in (r.split() for r in rows.splitlines()) - if int(group) == pgid] - - -def terminate_owned(process): - forced = False - try: - os.killpg(process.pid, signal.SIGTERM) - except ProcessLookupError: - pass - try: - process.wait(timeout=30) - except subprocess.TimeoutExpired: - forced = True - if group_members(process.pid): - forced = True - try: - os.killpg(process.pid, signal.SIGKILL) - except ProcessLookupError: - pass - process.wait(timeout=10) - return {'exit_code': process.returncode, 'forced': forced, - 'owned_group_remaining': len(group_members(process.pid))} - - -async def main(): - p = argparse.ArgumentParser() - for name in ('model', 'fixture', 'output', 'source'): - p.add_argument('--'+name, required=True) - args = p.parse_args() - assert digest(args.fixture) == '4f4256cc4edc9a23f55f66151738a1ecef07eb752c695e7b7fbdfa17e01d6aa9' - assert digest(client.__file__) == 'f31fae8c573ecc4bf8e31dae03823502e71404333e8111f4b535432758302b29' - samples = {r['id']: r['input_ids'] for r in json.loads(Path(args.fixture).read_text())['samples']} - prompt = samples['n8192-s0'] - assert len(prompt) == 8192 - out = Path(args.output); out.mkdir(parents=True, exist_ok=False) - with (out/'observations.jsonl').open('x') as stream: - def emit(**row): - stream.write(json.dumps(row)+'\n'); stream.flush() - print(json.dumps({k:v for k,v in row.items() if k not in ('token_ids','itl')}), flush=True) - emit(phase='protocol', scope='single_case_feasibility_not_formal_acceptance', verdict='INCONCLUSIVE', - arms=['triton', 'flashinfer', 'vllm'], cases=[['warmup',8192,16],['measurement',8192,2048]], - observer_sha256=digest(__file__), fixture_sha256=digest(args.fixture), - client_sha256=digest(client.__file__), - source_hashes={str(f.relative_to(args.source)):digest(f) for f in sorted(Path(args.source).glob('nanovllm/qwen38/*.py'))}, - versions={k:importlib.metadata.version(k) for k in ('vllm','torch','triton','flashinfer-python','transformers')}) - for arm in ('triton','flashinfer','vllm'): - with socket.socket() as sock: - sock.bind(('127.0.0.1',0)); port=sock.getsockname()[1] - url=f'http://127.0.0.1:{port}' - ready=out/f'{arm}-ready.json'; shutdown=out/f'{arm}-shutdown.json' - env=dict(os.environ, PYTHONPATH=args.source, NANOVLLM_FP8_GEMM=arm, NANOVLLM_CUDA_GRAPH='1') - if arm=='vllm': - env.pop('PYTHONPATH',None) - command=[sys.executable,'-u','-m','vllm.entrypoints.openai.api_server', - '--model',args.model,'--host','127.0.0.1','--port',str(port), - '--served-model-name','qwen38-fp8','--max-model-len','10240', - '--no-enable-prefix-caching','--language-model-only'] - else: - command=[sys.executable,'-u','-m','nanovllm.qwen38.server','--model',args.model, - '--port',str(port),'--max-context-tokens','10240','--max-active-requests','1', - '--ready-file',str(ready),'--shutdown-file',str(shutdown)] - (out/f'{arm}-command.private.json').write_text(json.dumps(command)+'\n') - with (out/f'{arm}-server.private.log').open('x') as log: - start=time.monotonic() - process=subprocess.Popen(command,cwd=args.source,env=env,stdout=log,stderr=log,start_new_session=True) - try: - while True: - if process.poll() is not None: raise RuntimeError('server_exited_before_readiness') - try: - with urllib.request.urlopen(url+'/health',timeout=2) as response: - assert response.status==200 - health=json.load(response) if arm!='vllm' else {} - break - except (OSError,AssertionError): - if time.monotonic()-start>600: raise TimeoutError('startup_timeout') - await asyncio.sleep(0.5) - model_id=health.get('model_id','qwen38-fp8') - emit(phase='ready',arm=arm,startup_s=time.monotonic()-start, - loader_report=health.get('loader_report'),graph_prewarm=health.get('graph_prewarm')) - async with aiohttp.ClientSession(timeout=aiohttp.ClientTimeout(total=500)) as session: - for index,(purpose,cap) in enumerate((('warmup',16),('measurement',2048))): - chunks=[] - request=client.RequestFuncInput(prompt=prompt,api_url=url+'/v1/completions',prompt_len=8192, - output_len=cap,model=model_id,ignore_eos=True,extra_body={'temperature':0,'return_token_ids':True}) - result=await client.async_request_openai_completions(request,RecordingSession(session,chunks)) - wire=b''.join(chunks) - (out/f'{arm}-{index}.sse').write_bytes(wire) - (out/f'{arm}-{index}-client.json').write_text(json.dumps(dataclasses.asdict(result),indent=2)+'\n') - payloads=[line[5:].strip() for line in wire.splitlines() if line.startswith(b'data:')] - framing=(payloads.count(b'[DONE]')==1 and payloads[-1:]==[b'[DONE]']) - events=[json.loads(payload) for payload in payloads if payload!=b'[DONE]'] - choices=[c for e in events for c in e.get('choices',[])] - tokens=[t for c in choices for t in c.get('token_ids',[])] - usage=[e['usage'] for e in events if e.get('usage')] - echoes=[c['prompt_token_ids'] for c in choices if c.get('prompt_token_ids') is not None] - terminal=[c['finish_reason'] for c in choices if c.get('finish_reason')] - valid=(result.success and not result.error and not any('error' in e for e in events) - and echoes==[prompt] and terminal==['length'] and framing - and len(usage)==1 and len(tokens)==cap==result.output_tokens - and usage[0]['completion_tokens']==cap and usage[0]['prompt_tokens']==8192==result.prompt_len - and len(result.itl)==len([e for e in events if e.get('choices')])-1) - emit(phase='request',arm=arm,purpose=purpose,input_tokens=8192,output_tokens=len(tokens), - valid=valid,token_ids=tokens,usage=usage,terminal=terminal, - chunk_token_counts=[len(c.get('token_ids',[])) for c in choices], - client_error=bool(result.error),ttft_s=result.ttft,latency_s=result.latency,itl=result.itl, - tpot_s=(result.latency-result.ttft)/(cap-1),wire_sha256=hashlib.sha256(wire).hexdigest()) - if not valid: raise RuntimeError('HTTP_accounting_failed') - if arm!='vllm': - with urllib.request.urlopen(url+'/stats',timeout=10) as response: stats=json.load(response) - with urllib.request.urlopen(url+'/health',timeout=10) as response: health=json.load(response) - (out/f'{arm}-health.private.json').write_text(json.dumps(health,indent=2)+'\n') - emit(phase='postflight',arm=arm,active_requests=stats['active_requests'], - runner_state_count=stats['runner_state_count'],fp8_execution=health['fp8_execution']) - assert not stats['active_requests'] and not stats['runner_state_count'] - except BaseException as exc: - emit(phase='error',arm=arm,exception=type(exc).__name__) - raise - finally: - cleanup=terminate_owned(process) - emit(phase='cleanup',arm=arm,**cleanup) - assert cleanup['owned_group_remaining']==0 - emit(phase='complete',verdict='INCONCLUSIVE',scope='single_case_feasibility_not_formal_acceptance') - -asyncio.run(main()) diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/flashinfer-small-m/operator_probe.py b/evidence/qwen38-h20-fp8-single-request-vllm032/patch/flashinfer-small-m/operator_probe.py deleted file mode 100644 index 5cd7af9a8..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/patch/flashinfer-small-m/operator_probe.py +++ /dev/null @@ -1,71 +0,0 @@ -"""Bounded integration probe; comparisons are diagnostic, not quality gates.""" -import argparse -import hashlib -import importlib.metadata -import inspect -import json -import os -from pathlib import Path - -import torch -from nanovllm.qwen38.quant import FP8Linear - -p = argparse.ArgumentParser() -p.add_argument('--output', required=True) -args = p.parse_args() -torch.manual_seed(670032) -assert torch.cuda.device_count() == 1 -rows = [] -for m, k, n in ((1, 128, 64), (31, 128, 64), (32, 128, 64), - (1, 5120, 5120), (1, 5120, 17408), (1, 17408, 5120)): - os.environ['NANOVLLM_FP8_GEMM'] = 'flashinfer' - candidate = FP8Linear(k, n, device='cuda') - os.environ['NANOVLLM_FP8_GEMM'] = 'triton' - control = FP8Linear(k, n, device='cuda') - candidate.weight.data.copy_(torch.randn(n, k, device='cuda').clamp(-4, 4).to(torch.float8_e4m3fn)) - candidate.weight_scale_inv.data.fill_(0.125) - control.weight.data.copy_(candidate.weight) - control.weight_scale_inv.data.copy_(candidate.weight_scale_inv) - x = torch.randn(m, k, device='cuda', dtype=torch.bfloat16) - with torch.inference_mode(): - a, b = control(x), candidate(x) - assert torch.isfinite(a).all() and torch.isfinite(b).all() - nrmse = ((a.float()-b.float()).square().mean().sqrt()/a.float().square().mean().sqrt()).item() - # Warm on a side stream, capture, then replay after changing input bytes. - stream = torch.cuda.Stream() - stream.wait_stream(torch.cuda.current_stream()) - with torch.cuda.stream(stream): - for _ in range(3): candidate(x) - torch.cuda.current_stream().wait_stream(stream) - graph = torch.cuda.CUDAGraph() - with torch.cuda.graph(graph): - captured = candidate(x) - exact = [] - for _ in range(2): - x.copy_(torch.randn_like(x)) - eager = candidate(x) - graph.replay() - torch.cuda.synchronize() - exact.append(torch.equal(eager, captured)) - assert all(exact), 'Graph replay differs from eager for changed inputs' - timing = {} - for name, module in [('triton', control), ('flashinfer', candidate)]: - for _ in range(3): module(x) - start, end = torch.cuda.Event(enable_timing=True), torch.cuda.Event(enable_timing=True) - start.record() - for _ in range(20): module(x) - end.record(); end.synchronize() - timing[name] = start.elapsed_time(end)/20 - row = dict(shape=[m, k, n], diagnostic_nrmse=nrmse, graph_changed_inputs_exact=exact, - kernel_calls=candidate.kernel_calls, milliseconds=timing) - rows.append(row) - print(json.dumps(row), flush=True) - del graph, candidate, control, x, a, b, captured, eager - torch.cuda.empty_cache() - -from flashinfer.gemm import fp8_blockscale_gemm_sm90 -implementation = Path(inspect.getfile(inspect.unwrap(fp8_blockscale_gemm_sm90))) -result = dict(scope='operator_integration_not_model_quality_or_service_acceptance', rows=rows, - versions={name: importlib.metadata.version(name) for name in ['torch', 'triton', 'flashinfer-python']}, - flashinfer_source_sha256=hashlib.sha256(implementation.read_bytes()).hexdigest()) -Path(args.output).write_text(json.dumps(result, indent=2)+'\n') diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/plan/README.md b/evidence/qwen38-h20-fp8-single-request-vllm032/plan/README.md new file mode 100644 index 000000000..1a0c79039 --- /dev/null +++ b/evidence/qwen38-h20-fp8-single-request-vllm032/plan/README.md @@ -0,0 +1,15 @@ +# Frozen plan and projection + +Campaign [#67](https://github.com/InferMatrixLab/InferRSI/issues/67) +owns the acceptance criteria and source revisions. Retained original files keep +their exact bytes; the complete plan is also preserved in the external archive. + +Configuration entry points: +- `plan/search-8k-v1/activation.json` +- `plan/formal-8k-baseline-identities.json` +- `plan/quality-v5-source-identities.json` + +`archive-projection.json` declares which original source records supply the +compact summary, sample arrays and configuration references. The identical +projection and its digest are pinned in `results/artifacts.json`. It is a +read-only storage projection, not a new controller or retrospective freeze. diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/plan/approved-statistics-8k-v1.md b/evidence/qwen38-h20-fp8-single-request-vllm032/plan/approved-statistics-8k-v1.md deleted file mode 100644 index 3e280f158..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/plan/approved-statistics-8k-v1.md +++ /dev/null @@ -1,38 +0,0 @@ -# Approved 8K statistical decision component - -Refs [Issue #67 owner decision](https://github.com/InferMatrixLab/InferRSI/issues/67#issuecomment-6357921043). -The owner explicitly approved the pending statistical question in the active -conversation. This prospective record supersedes the pending-statistics wording -in earlier proposals and the 8K scope amendment. Those historical bytes remain -unchanged. No historical diagnostic is upgraded to formal acceptance. - -For each of TTFT and TPOT, pair the same ten fixed 8192-token input IDs and -three repetition IDs across baseline and candidate. Each request must generate -2048 actual tokens. The primary statistic is the median of all 30 paired -candidate/baseline ratios. Resample ten input IDs with replacement as clusters, -retaining the three repetitions within each sampled input, for 10000 draws -using Python random.Random seed 670032. Reuse the same sampled indices for both -metrics. Quantiles use linear interpolation; the 95th percentile of bootstrap -medians is the one-sided upper bound. - -A metric is PASS only when both median and upper bound are <=1.03. A median -above1.03 is FAIL; median<=1.03 with upper>1.03 is INCONCLUSIVE. This does not -claim simultaneous confidence for both metrics or quantify shared service -restart effects. All raw samples and repetition-level results remain required. - -The versioned reducer in validator/statistics-8k-v1 checks the complete60-row -matrix, identity types, duplicate records/JSON keys, actual output counts, -positive finite measurements and finite ratios. TPOT is derived as -(latency-TTFT)/2047. Invalid or incomplete input emits no metric verdict. -The valid flag must be independently authenticated from the original client, -wire response and protocol evidence. This reducer cannot authenticate a caller's -flag, hardware, source, model, cleanup or task quality; campaign_verdict therefore -remains INCONCLUSIVE. statistics_approved records owner policy only and is not -full campaign activation. The request timeout belongs to the separate execution -protocol; the numeric reducer introduces no unapproved latency cap. - -The old three-length proposal remains byte-for-byte intact. The new component -uses8192 only. Synthetic checks are CPU-only, consume no GPU budget and are not -model measurements. The surrounding full plan, runtime/source/quality bindings -and execution commands still require completed qualification and freeze before -formal candidate search. Approved phased GPU caps remain2/4/8/30GPU-hours. diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/plan/archive-projection.json b/evidence/qwen38-h20-fp8-single-request-vllm032/plan/archive-projection.json new file mode 100644 index 000000000..929671aec --- /dev/null +++ b/evidence/qwen38-h20-fp8-single-request-vllm032/plan/archive-projection.json @@ -0,0 +1,85 @@ +{ + "schema_version": 1, + "campaign": "qwen38-h20-fp8-single-request-vllm032", + "configuration": [ + "plan/search-8k-v1/activation.json", + "plan/formal-8k-baseline-identities.json", + "plan/quality-v5-source-identities.json" + ], + "integrity_manifests": [ + "plan/retrieval-integrity.json", + "plan/search-8k-v1-integrity.json", + "results/search-observations-2163-2175-integrity.json", + "results/diagnostics/sandbox-offline-prefetch-integrity.json" + ], + "verdict": { + "path": "results/fp8-projections-search-v1/bound-result.json", + "pointer": "/campaign_verdict" + }, + "metrics": [ + { + "name": "baseline-ttft_s", + "path": "results/fp8-projections-search-v1/bound-result.json", + "pointer": "/rows", + "field": "ttft_s", + "where": { + "arm": "baseline" + }, + "unit": "s", + "scope": "Published paired 8K search: 10 inputs x 3 repetitions per arm; pooled descriptive distribution" + }, + { + "name": "baseline-tpot_s", + "path": "results/fp8-projections-search-v1/bound-result.json", + "pointer": "/rows", + "field": "tpot_s", + "where": { + "arm": "baseline" + }, + "unit": "s", + "scope": "Published paired 8K search: 10 inputs x 3 repetitions per arm; pooled descriptive distribution" + }, + { + "name": "candidate-ttft_s", + "path": "results/fp8-projections-search-v1/bound-result.json", + "pointer": "/rows", + "field": "ttft_s", + "where": { + "arm": "candidate" + }, + "unit": "s", + "scope": "Published paired 8K search: 10 inputs x 3 repetitions per arm; pooled descriptive distribution" + }, + { + "name": "candidate-tpot_s", + "path": "results/fp8-projections-search-v1/bound-result.json", + "pointer": "/rows", + "field": "tpot_s", + "where": { + "arm": "candidate" + }, + "unit": "s", + "scope": "Published paired 8K search: 10 inputs x 3 repetitions per arm; pooled descriptive distribution" + } + ], + "recorded_results": [ + { + "path": "results/fp8-projections-search-v1/bound-result.json", + "pointer": "/statistics" + }, + { + "path": "results/merged-flash-prefill-v1/parent-review.json" + }, + { + "path": "results/profile-merged-v1/phase-summary.json" + } + ], + "limitations": [ + "Campaign acceptance remains INCONCLUSIVE; this is a storage-only projection.", + "The index covers all published result JSON documents and their named/nested attempt observations, not unpublished controller history.", + "Repeated identical observations of one named attempt are merged with every source pointer retained. Changed observations and anonymous records remain distinct.", + "Record/index entry counts are not independent trial counts. Text-only history and per-request traces remain in the byte-preserved original archive.", + "Only declared sample distributions are recomputed. Other retained results are explicitly reported frozen values, not independently requalified quality or statistics.", + "Original private models, media, raw receipts and runtime records were not published; archiving does not restore them or complete outstanding acceptance." + ] +} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/plan/deepgemm-prefill/metadata.json b/evidence/qwen38-h20-fp8-single-request-vllm032/plan/deepgemm-prefill/metadata.json deleted file mode 100644 index 9e9eaf741..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/plan/deepgemm-prefill/metadata.json +++ /dev/null @@ -1,11 +0,0 @@ -{ - "issue": 67, - "engine_candidate": "b412062e13c92d2429b3758ca39af7439df4f7ec", - "base": "8859ef319486eb5ed4a444fc66c1018cbebbab33", - "plane": "model", - "source_archive_sha256": "25a15d6fd18147dc1ad2a772d618ebd8c483cb07c101c0847f18319d422d9109", - "knowledge_revision": "sha256:a98efeae08df1e252dc35968c6b9ddda249f57f1ea6da91fd6c3e44cfe05387c", - "next_scope": "GPUoperator/TMA/quantizer integration and fixed8K2048HTTP diagnostic; unexecuted at this snapshot", - "packaging": "candidate-only repackagedoperator; baselinePYTHONPATHremoved", - "formal_acceptance": "INCONCLUSIVE" -} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/plan/diagnostics/baseline-long-readiness.md b/evidence/qwen38-h20-fp8-single-request-vllm032/plan/diagnostics/baseline-long-readiness.md deleted file mode 100644 index 6a11f0948..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/plan/diagnostics/baseline-long-readiness.md +++ /dev/null @@ -1,22 +0,0 @@ -# Baseline long-context feasibility - -Prospective workload: [Issue67 comment](https://github.com/InferMatrixLab/InferRSI/issues/67#issuecomment-6347246284). -Infrastructure retry: [separate prospective comment](https://github.com/InferMatrixLab/InferRSI/issues/67#issuecomment-6347328808). -One H20,35-minute cap from readiness for each declared attempt;180-second request -and1200-second startup limits,2000-second outer command limit. Same authenticated -official FP8 weights, unmodified vLLM0.32.0 and pinned official client; exact -fixture/client/model/observer identities are recorded by the retained observer. - -Sequential fixed inputs s0:32/8 warmup,32/8 framing,8192/32,65536/8,131072/8. -Greedy,ignoreEOS; validate every prompt echo, generated token ID, usage, terminal -reason and timing event. Stop on error. Log actual dispatch and clean only the -owned process group and allocation. This is a feasibility/cost probe, not the -2048-output performance matrix, retrieval validation or candidate search. - -Necessary server settings: loopback/name,133120 maximum context, prefix caching -off, language-model-only. Dtype, quantization, KV, batching, compilation, Graph, -attention and kernel selection retain defaults. All file caches are task-root -scoped. Job1959 used filesystem IPC and lacked the environment bin on PATH. -CPU1960 confirms filesystem EOPNOTSUPP even at70-byte paths; abstract IPC works. -The retry exposes existing ninja/nvcc and uses a job-unique Linux abstract -namespace. No package/source change or filesystem socket outside the task root. diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/plan/diagnostics/benchmark-http.md b/evidence/qwen38-h20-fp8-single-request-vllm032/plan/diagnostics/benchmark-http.md deleted file mode 100644 index 81072ffbd..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/plan/diagnostics/benchmark-http.md +++ /dev/null @@ -1,50 +0,0 @@ -# Official-client accounting and longer Graph readiness - -Governing issue: https://github.com/InferMatrixLab/InferRSI/issues/67. -The harness slice and prospective GPU bounds were declared before execution in -issue comments 6346158087 and 6346252423. This is not an acceptance freeze. - -One plane: harness. Source23417bbf13548127fdd0b49dbb26e495a38d9af6 changes only -HTTP protocol handling, socket tests and service documentation from merged -896c9214739b24adeae5c677511877e89e625a99. Model/runner/Graph kernels are unchanged. -The reference is vLLM0.32.0 source5df63a7203cdb9328d824a27b682e77c7b9928be: -completion protocol/serving and benchmarks/lib/endpoint_request_func.py. -The latter installed module must match SHA256 -f31fae8c573ecc4bf8e31dae03823502e71404333e8111f4b535432758302b29. - -The adapter accepts neutral repetition_penalty/logprobs, ignore_eos, requested -stream usage and first-choice prompt IDs. Normal completion sends finish_reason -on its final token. The upstream client times any nonempty choices list, -including empty-text choices; an extra terminal choice therefore adds a spurious -inter-token interval. Usage-only chunks must have empty choices. The client's -success flag does not validate streamed error objects or terminal reasons. - -CPU protocol checks use synthetic real HTTP sockets. The original server must -reject the client's payload without model work; the repaired server must return -one token under natural EOS and four under ignore_eos, actual usage replacing an -intentionally wrong caller prompt-length estimate, and zero/three ITL values. -Version1 failed before requests because it imported a pytest-dependent server -fixture inside the unchanged client environment. Version2 runs that fixture in -the existing test environment and keeps the official client environment intact. -Both source versions and the failed attempt are retained. - -The subsequent GPU diagnostic is bounded at one GPU for30minutes, charged to -the remaining1.96GPU-hours in the approved readiness stage. Two sequential -service processes use the same assigned GPU and official FP8 weights: Graph, -then eager; context4096 and concurrency1. Cases783/1/424 use natural EOS/max2048; -case783 then uses ignore_eos/exact1024 to cross the selected1024 Graph capacity. -All8 requests are retained, including differences. No reduced replacement work. - -The official unmodified request function uses a transparent response-chunk -recorder. Original wire/client data stay private because model identifiers can -contain local paths. Selected token sequences, hashes, counters and timing are -published. Validate exact input echo, usage/output agreement, ITL count, DONE -and healthy terminal framing, then zero active requests/state. Compare complete -Graph/eager token sequences. Startup/prewarm are separate; fixed arm order, -small scope and response recording preclude a performance or quality PASS. - -Bounds: startup180seconds/service, HTTP300seconds/request, command1720seconds, -Slurm1800seconds; owned-process cleanup with20-second graceful deadline. -Preserve Slurm device visibility and task-local paths/caches. No GPU dependency -install or vLLM baseline server is included. Historical full model-byte intake -is reused; a fresh full-shard scan is not claimed. diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/plan/diagnostics/fp8-graph-takeover.md b/evidence/qwen38-h20-fp8-single-request-vllm032/plan/diagnostics/fp8-graph-takeover.md deleted file mode 100644 index 820cb5ea6..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/plan/diagnostics/fp8-graph-takeover.md +++ /dev/null @@ -1,44 +0,0 @@ -# FP8 dispatch and graph correctness diagnosis - -Status: INCONCLUSIVE; diagnostic preparation, not an acceptance freeze. -Authority: owner continuation on 2026-10-22; governing acceptance Issue #67. -The owner requests actual FP8 path verification and graph correctness repair, -followed by a reviewable acceptance freeze and itemized GPU budget. - -## Bounded first contrast - -- Plane: harness-only observation; the engine remains unchanged. -- One physical H20-family GPU, one process, at most 20 minutes (0.333 allocated - GPU-hours). No multi-GPU allocation, server, benchmark, dependency installation, - quality scoring, or performance candidate search. -- Existing official FP8 checkpoint and candidate runtime; inspect source, - model/config identities, installed versions, GPU identity and all cache paths. -- The three historical screening prompts with reported graph/eager divergence: - GSM8K test 783, 1 and 424. At most 16 teacher-forced decode steps per prompt. - All paths consume identical tokens selected from compact eager execution. -- Compare compact eager, fixed-capacity eager, and fixed-capacity CUDA Graph. - Record prefill outputs, layer boundaries, live state, capture restoration, - decode logits, greedy choices and exact equality; numerical distances remain - diagnostics, not substitute acceptance thresholds. -- Identical fixed-capacity eager/replay results isolate shape/layout effects. - A difference between them triggers capture/state/address investigation. -- Retain every partial result and failure. Timeout is INCONCLUSIVE. No changed - threshold or post-result sample exclusion. Exit ends the Slurm allocation; - no background child services are launched. - -The full baseline/search/finalization budget and statistical/retrieval freeze -remain pending owner confirmation. This contrast cannot establish parity. - -## Prospective correctness repair contrast (2026-10-23) - -The owner approved the 20-minute single-GPU diagnostic cap. The metadata failure -and initial contrast used 6 + 82 allocated GPU-seconds. The initial contrast -found replay equal to fixed-capacity eager at all 48 steps; the prefill layout -changed outputs before capture. Preserve the first contrast and its limitations. - -After CPU regression, spend at most another three allocated GPU-minutes on the -same three prompts / 16 forced steps, changing only the model plane: initial -prefill attends to its current K/V operands while populating fixed decode -storage. The capture and FP8 implementation remain unchanged. Retain all -numerical differences; neither exact equality on these samples nor successful -CPU tests establish full-service or task-quality acceptance. diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/plan/diagnostics/graph-demotion.md b/evidence/qwen38-h20-fp8-single-request-vllm032/plan/diagnostics/graph-demotion.md deleted file mode 100644 index 526eb9148..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/plan/diagnostics/graph-demotion.md +++ /dev/null @@ -1,29 +0,0 @@ -# Hybrid Graph demotion readiness - -Authority and prospective limits: InferRSI Issue #67 comments6346532120, -6346586062 and6346638737. This is not an acceptance freeze or performance search. - -Model-plane candidate80e5108e44c43f3f54fabb5723e6154d18088516 on test-only -base7ec5d224e983e437530355b94b0d6d08153bc27d. Its model patch and all runtime -bytes equal reviewed e69035df55d7fd956a7a2529c75a402164cdf1ed; the sole rebase -delta is the independently reviewed readiness test. Only full-attention KV -is compacted; linear conv/recurrent history remains available to eager decode. - -After CPU regressions reproduce the old exception, verify the same original -four requests per Graph/eager arm from benchmark-http.md, plus a final16-token -ignore-EOS request for case783 after demotion. Total10 requests; same allocated -GPU, official FP8 weights, exact pinned client, context4096 and concurrency1. -Preserve every output difference. No candidate selection or reduced replacement -work. Observer v2 is separate; original observer and failed1951 remain frozen. - -One GPU maximum30minutes, within approved readiness2 GPU-hours and overall44. -Startup180seconds/service, request300seconds, command1720seconds; owned-process -cleanup20seconds then bounded kill. Readiness before retry uses375 GPU seconds; -initial separate allowance147 seconds is not charged again. - -1952 failed at CPU preflight before model execution (73passed/1failed,5 GPU -seconds). CPU-only1953 repeated the same existing ready-file race. Separate -harness test repair94eed335c365fccc39289551837818e333758263 waits for complete JSON -within unchanged3 seconds;72 remote CPU tests passed in1954, then merged7ec5d22. -Combined source74 local tests pass. Runtime and GPU diagnostic definitions were -not changed by the test correction. diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/plan/diagnostics/graph-forced96.md b/evidence/qwen38-h20-fp8-single-request-vllm032/plan/diagnostics/graph-forced96.md deleted file mode 100644 index ed297a02c..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/plan/diagnostics/graph-forced96.md +++ /dev/null @@ -1,17 +0,0 @@ -# Fixed-history Graph numerical diagnosis - -Prospective scope: [Issue #67 comment](https://github.com/InferMatrixLab/InferRSI/issues/67#issuecomment-6347094636). -Harness-only observer, production unchanged. One H20, at most15 minutes from -the approved readiness stage. Historical case gsm8k/test/783,149 input tokens, -96 predicted positions (prefill plus95 decode), capacity1024. Force the prior -job1955 eager token history for all three arms: compact eager, capacity eager, -and the real prewarmed pooled Graph state. Compare full logits and all live -KV, conv and recurrent tensors. Reproduce the compact history and Graph prefix -through the previous first divergence at zero-based index81. No hooks, kernel -changes, quality scoring, performance search or acceptance gate changes. - -The exact retained observer records fixture/history/source digests; its output -is a diagnostic INCONCLUSIVE regardless of exact comparisons. The source is -engine head80e5108e44c43f3f54fabb5723e6154d18088516, tree-equivalent to merged -3e918df2dd7b478d3d2985350bb5efb523ea4981. Checkpoint/config and fixture identities -reuse the authenticated intake; this does not claim a new full-shard rehash. diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/plan/diagnostics/http-publication-readiness.md b/evidence/qwen38-h20-fp8-single-request-vllm032/plan/diagnostics/http-publication-readiness.md deleted file mode 100644 index 96476c8d6..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/plan/diagnostics/http-publication-readiness.md +++ /dev/null @@ -1,46 +0,0 @@ -# Publication-head HTTP readiness protocol - -Governing acceptance: https://github.com/InferMatrixLab/InferRSI/issues/67 -Protocol declared before allocation in issue comment 6345756537. -This file records that declaration; it does not activate an acceptance freeze. - -Purpose: validate the independently published prefill repair through the actual -HTTP service, including startup prewarm, pooled Graph reuse and state cleanup. -One GPU, at most 15 minutes, charged to the owner-approved 2 GPU-hour readiness -stage within the 44 GPU-hour phased ceiling. No performance candidate search. - -Source arms run sequentially on the same allocated GPU: - -1. Original main, Graph enabled. -2. Publication repair, Graph enabled. -3. Publication repair, eager. - -Source revisions and actual source hashes are in results/diagnostics/http-summary.json -and http-readiness.jsonl. Hardware is NVIDIA H20-3e, driver 580.105.08; private -device identity is retained in controlled operational logs. Runtime remains -Python 3.12.13, torch 2.14.1+cu130, Triton 3.8.0, Transformers 5.20.2, official -FP8 checkpoint and Triton block-FP8 implementation. FlashInfer is not installed. -Model bytes reuse the earlier authenticated intake; no fresh full-shard scan. - -Inputs: historical GSM8K cases 783, 1 and 424, token lengths 149, 70 and 106. -Each arm performs two repetitions in that case order. Greedy, natural EOS, -16-token maximum, concurrency 1, context bound 2048. Input hashes are retained. -Graph startup captures all existing buckets; these requests use bucket 1024. -No additional warmup request or capacity-transition probe is included. - -The exact retained client measures token-ID SSE events, with first-token delay, -mean interval between the first and last returned token, and complete HTTP -request latency. It retains every token, timestamp, terminal marker and cleanup -record. These semantics are **not the official vLLM benchmark client**. Main's -unsupported benchmark fields remain a separate prerequisite. - -Startup is bounded at 180 seconds per service; requests at 120 seconds; server -shutdown at 20 seconds followed by owned-PID kill if necessary. The job has an -850-second command timeout and 900-second Slurm limit. Slurm device visibility -is preserved. No server is externally exposed. All mutable paths and caches -remain within the task's authorized root. - -Startup and first-use effects are reported separately from request samples. -The fixed arm order, different generated sequences on the failing base case, -short caps and lack of an acceptance freeze preclude causal speedup, quality, -long-context or vLLM-parity claims. Verdict remains INCONCLUSIVE. diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/plan/execution-proposal-v1.md b/evidence/qwen38-h20-fp8-single-request-vllm032/plan/execution-proposal-v1.md deleted file mode 100644 index 875d18051..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/plan/execution-proposal-v1.md +++ /dev/null @@ -1,80 +0,0 @@ -# Proposed paired performance execution protocol - -Refs [Issue67](https://github.com/InferMatrixLab/InferRSI/issues/67). -**INCONCLUSIVE / NOT ACTIVATED.** This supplements `freeze-proposal.md`. -Owner confirmation of the statistical/retrieval policy remains necessary. -This document is prospective; it does not relabel the readiness runs. - -## Request and timing unit - -Use the authenticated official vLLM 0.32.0 streaming request function for both -arms, with the exact pretokenized performance fixture and return-token-ID -extension. Concurrency is one and no requests overlap. Greedy generation uses -ignore_eos and exactly 2048 output tokens, with cross-request prefix caching off. -Authenticate every actual input echo, generated ID count, usage count, terminal -length reason and unique final DONE. Missing/error/partial responses are retained -and prevent a complete comparison. Multiple token IDs in one SSE choice are -valid upstream coalescing; count timing events and output tokens separately. - -TTFT and total latency come from the official client's timing boundaries. The -observer derives TPOT as `(latency - TTFT) / (usage_output_tokens - 1)`. It is not -the mean SSE-event ITL. Report complete latency, 2048/latency output tokens/s, -ITL event distributions and coalescing counts separately. Validate all raw -client records and raw SSE before applying the proposed numerical reducer. - -## Pairing, order and restarts - -A complete comparison keeps one physical allocated GPU for both arms throughout -all three repetitions. Record private device identity and provide a consistent -public device placeholder. Both services never hold the GPU simultaneously. -Repetition order is baseline/candidate, candidate/baseline, baseline/candidate. -Each arm starts a fresh server once per repetition (six starts total), and serves -all thirty fixed cases in order: 8192 s0..s9, 65536 s0..s9, 131072 s0..s9. -Pair observations by exact input ID and repetition, never by nearest timing. - -After each server is healthy, warm each length once using its fixed s0 input and -2048 ignore-EOS output tokens. Those three warmups are recorded and excluded. -Then execute the thirty timed requests. Prefix reuse remains disabled during -warmup and measurement. Keep task-owned compilation caches between restarts; -compilation, graph capture and warmup work stay outside timed requests. A failed -warmup stops that arm and leaves the comparison incomplete. No adaptive warmup, -outlier removal, extra repetitions or request retry after seeing timing. - -Startup limit is 1200 seconds and request limit is 1800 seconds. The approved -stage/allocation cap always takes precedence. Stop rather than silently shorten -work to fit the cap. A GPU/environment change invalidates pairing; retain the -partial attempt and declare a fresh full comparison prospectively. Final -acceptance reruns the clean delivered candidate with its matched baseline. -The complete paired matrix contains 180 timed and 18 warmup requests: 405504 -generated tokens across six service starts, before prefill and startup cost. -Readiness and baseline-only cost measurements cannot be reused as one arm of a -paired final performance comparison. - -## Proposed reducer and scope - -`validator/proposals/paired_statistics_v1.py` rejects missing, duplicate, -unexpected or invalid records in the 180-request matrix. It reports median of -thirty paired ratios per metric/length, p10/p90, and the proposed input-cluster -bootstrap from `freeze-proposal.md`. A fixed Python `random.Random` seed draws -ten input indices with replacement per draw, preserves all three repetitions, -and reuses the same 10000 draws for every metric/length. The sampled-index digest -is emitted. Linear interpolation defines all reported quantiles. - -Only a proposed metric verdict is emitted; campaign verdict stays INCONCLUSIVE -and activated=false. The reducer does not establish source/device/authenticity, -quality, cleanup, approval or validity of a caller's `valid` flag. The final -validator must authenticate those independently before any campaign verdict. -The bootstrap is a per-metric uncertainty rule, not a simultaneous six-metric -confidence claim or a claim about production workloads. Input-cluster resampling -conditions on these same three paired repetitions; it does not estimate -uncertainty from service-restart effects shared across all inputs. Record this -limitation with the owner decision, together with all repetition-level results. - -## Remaining activation work - -Bind actual service commands/runtime inventories, authenticated sources and all -quality preparation/scoring/sandbox artifacts in an immutable manifest. Resolve -quality readiness, independent review and owner policy confirmation before -search. Baseline-only performance/quality runs require their own declared order, -bounds and complete protocol before execution. No engine performance candidate -is authorized by committing this proposal or by successful synthetic checks. diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/plan/flashinfer-small-m/feasibility-v3.json b/evidence/qwen38-h20-fp8-single-request-vllm032/plan/flashinfer-small-m/feasibility-v3.json deleted file mode 100644 index 7741743fc..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/plan/flashinfer-small-m/feasibility-v3.json +++ /dev/null @@ -1,46 +0,0 @@ -{ - "issue": 67, - "candidate": "8859ef319486eb5ed4a444fc66c1018cbebbab33", - "plane": "model", - "single_hypothesis": "FlashInfer SM90 M<32 replacing singleton-packed Triton decode", - "gpu_count": 1, - "max_seconds": 1800, - "budget_phase": "readiness", - "readiness_used_before_gpu_seconds": 2388, - "scope": "bounded integration and single-case E2E feasibility; not full acceptance or search grid", - "arms": [ - "native_triton", - "native_flashinfer", - "vllm_0.32.0" - ], - "environment": "all arms use existing vllm032 runtime; prior native runtime is not a matched control", - "input_id": "n8192-s0", - "warmup_output_tokens": 16, - "measurement_output_tokens": 2048, - "concurrency": 1, - "ignore_eos": true, - "required_checks": [ - "finite operator outputs", - "changed-input Graph replay exactly matches eager", - "authenticated fixture and official client", - "exact 8192 prompt echo and 2048 output token ids and usage", - "owned process cleanup" - ], - "quality": "not evaluated; remains pending", - "performance": "descriptive observations only; no statistical PASS", - "runtime_timeout_s": { - "operator": 400, - "http": 1300, - "server_start": 600, - "request": 500 - }, - "artifacts": { - "source.tar": "2f431b13e97bc0a26fe4d685be1b7896a53d9a0b93d582e692dc2ea53976fc3d", - "operator_probe.py": "fa45568dcc07a227b513d03dd2606e000b96dc68b7b099a65ce220c62423f8e2", - "http_probe.py": "e20a54b81e8f5a1b6d6dcfd86c41fd1bc98b7e910763ae3445637cce69dc7512", - "gpu.sbatch": "d48c728806168fa862b9afa4e6f099e8450d9b17b03f2008b69a046f4800a703" - }, - "supersedes": "v2 syntax check failed: unmatched parenthesis; local shell did not stop after failure and submitted2040, cancelled before allocation. v1 pending2038 also cancelled. Both0GPU.", - "observer_correction": "Allow official vLLM SSE multi-token choices; ITL counts choice events, not tokens. Exactly one final DONE.", - "OMP_NUM_THREADS": "unset for all arms; vLLM internal defaults retained" -} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/plan/freeze-proposal.md b/evidence/qwen38-h20-fp8-single-request-vllm032/plan/freeze-proposal.md deleted file mode 100644 index bca2bedb0..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/plan/freeze-proposal.md +++ /dev/null @@ -1,102 +0,0 @@ -# Prospective acceptance freeze — owner review draft - -Status: INCONCLUSIVE / NOT ACTIVATED. Governing acceptance Issue #67. -Prepared 2026-10-23. Nothing here changes a historical result or authorizes -performance search. The owner-approved diagnostic cap is separately recorded -in diagnostics/fp8-graph-takeover.md. - -## Existing accepted requirements - -Same physical allocated H20-family GPU, same official FP8 model/tokenizer, -vLLM 0.32.0 default optimizations, the pinned official streaming client, -concurrency one, no prefix reuse. Exact input lengths 8192/65536/131072, -ten distinct fixed inputs each, exactly 2048 generated tokens, three paired -repetitions with alternating arm order. Every length must satisfy both -candidate/baseline TTFT and TPOT <= 1.03. No averaging across lengths. -Final task-quality datasets and one-percentage-point score floor remain as -specified in #67. No W8A16 substitution, truncation or reduced-work acceptance. - -## Proposed statistical rule — requires owner confirmation - -- Pair each input and repetition across arms (30 pairs per length). -- Primary statistic for each metric/length: median of the 30 paired ratios. - Publish all raw measurements, arm medians, paired ratios and p10/p90. -- Resample the ten input IDs as clusters, retaining all three repetitions - within each chosen ID: 10,000 bootstrap samples, seed 670032, percentile - quantiles with linear interpolation. Retain the 95th percentile as a - one-sided upper uncertainty bound. This is a proposed handling of sampling - variation, not a claim that ten inputs establish general production behavior. -- Complete valid results: primary ratio > 1.03 means FAIL for that metric. - Primary ratio <= 1.03 but upper bound > 1.03 means INCONCLUSIVE; both - <= 1.03 permit the scoped metric gate to pass. Missing/error/timeout requests, - changed environment or an unqualified correctness path mean INCONCLUSIVE, - never a performance PASS. Predeclared correctness failures remain FAIL. -- Freeze exact commands, source hashes, order, warmup and restart policy - before baseline measurement. Do not add repeats or remove outliers after - seeing results. Startup/JIT/warmup are separately recorded and excluded; - timed requests use the same warmup policy in both arms. - -## Proposed retrieval construction — baseline then owner confirmation - -- Ten fixed synthetic inputs at each required length; four distinct key/value - targets near 10/35/65/90 percent of actual token positions, with decoys. -- Authenticate exact token lengths, positions, identities and expected strings - before baseline. Ask for all four values; freeze extraction before scoring. -- Report both complete-case accuracy (all four exact) and per-position counts. -- Proposal: the baseline must solve at least 9/10 complete cases per length; - otherwise the set is not qualified. Candidate complete-case counts and each - position count must be no lower than the matched baseline at that length. -- Baseline qualification and owner approval precede candidate search. Do not - silently regenerate hard cases or alter extraction after observing outputs. - -## Prior-art-first implementation queue - -Each item is a separate model-plane hypothesis, after correctness and freeze: - -1. Adopt the vLLM 0.32.0 FP8 dispatch design: FlashInfer small-M blockscaled - GEMM, DeepGEMM large-M branch, with exact dtype/scale-layout checks and - logged actual selected callables. Preserve official W8A8 semantics. -2. Replace quadratic materialized attention with an upstream fused attention - implementation supporting actual sequence lengths and graph capture. - This is also required for meaningful 64K/128K feasibility on one GPU. -3. Adapt vLLM's GDN prefill / recurrent decode / causal convolution fusion and - state ownership. Preserve FP32 recurrent state and checkpoint semantics. -4. Consider projection and normalization fusion only after measuring remaining - service bottlenecks. BF16-only historical fusion is not an FP8 result. - -Every attempted/rejected/inconclusive variant retains source, environment, -raw outcomes, cost and a candidate knowledge record. No automatic promotion. - -## Proposed allocated GPU budget — hard caps, not a completion promise - -| Stage | Proposed cap | Work and stop rule | -| --- | ---: | --- | -| Readiness and baseline qualification | 2 GPU-hours | Correctness follow-up, bounded long-context feasibility, source/client/model checks and measured runtime estimation. Stop on unsupported lengths or invalid timing. | -| Frozen baseline measurements | 4 GPU-hours | Performance matrix, full matched task-quality baseline and retrieval. Reuse outputs only under identical declared identities. | -| Candidate search | 8 GPU-hours | Prior-art-first operator/model screening and small service contrasts; include failed jobs and JIT. No final-set tuning. | -| Finalization reserve | 30 GPU-hours | Clean delivered restart, full performance/quality/retrieval gates, raw accounting and cleanup. Do not spend this reserve on search. | -| Total new authorization requested | **44 GPU-hours** | Sequential single-GPU jobs by default; allocation count times elapsed time, including idle allocated time. | - -The earlier DSH estimates are not adopted as approvals. For scale only, its -566,000-token quality estimate at the observed 9.1 tokens/s is approximately -17.3 hours of generation, before loading/prefill/scoring. The full performance -matrix generates 184,320 tokens per arm; an 88 ms TPOT corresponds to about -4.5 hours of decode for one arm, excluding prefill. These measurements used -different paths and are cost scenarios, not matched performance results. - -The maximum quality workload is 2,483 * 2,048 = 5,085,184 generated tokens per -arm. At 9.1 tokens/s that alone would take approximately 155.2 hours. Therefore -44 GPU-hours cannot guarantee completion if outputs approach the cap or current -slow paths persist. Recompute stage costs after readiness using actual lengths -and launch/prefill measurements. If a stage cannot finish within its cap, stop -and present the revised itemization before allocating more; preserve every -partial result as INCONCLUSIVE. Do not reduce workload or relax gates to fit. - -Owner decision, 2026-10-23: **44 GPU-hours approved as a phased total cap**, with -the four stage caps above. This is prospective authority; the prior 147-second -diagnosis was separately approved and remains recorded. Budget approval does -not approve the proposed statistical/retrieval gates or activate a freeze. -Candidate search remains blocked until baseline qualification, -retrieval/statistical approval, identity binding and validator/freeze activation -are all complete. Re-budgeting a stage or exhausting the total requires a new -owner decision; finalization reserve is not available to candidate search. diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/plan/owner-amendment-8k-only.md b/evidence/qwen38-h20-fp8-single-request-vllm032/plan/owner-amendment-8k-only.md deleted file mode 100644 index a64660216..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/plan/owner-amendment-8k-only.md +++ /dev/null @@ -1,48 +0,0 @@ -# Owner amendment: 8K-only performance and retrieval acceptance - -Refs [Issue #67](https://github.com/InferMatrixLab/InferRSI/issues/67). -The owner explicitly confirmed that both retrieval and formal TTFT/TPOT -acceptance cover only 8192 input tokens. This -prospective scope change supersedes the earlier required 64K/128K lengths. -Historical artifacts, source identities, observations, failed attempts and -verdicts remain unchanged. No old result is relabeled by this amendment. - -The two engines still use the same pinned official vLLM0.32.0 streaming HTTP -client, authenticated official FP8 model/tokenizer, physical H20, concurrency1, -10 fixed 8K inputs, 2048 actual output tokens ignoring EOS and three paired -alternating repetitions. Candidate/baseline TTFT and TPOT must each be <=1.03. -Metric aggregation and variability handling remain pending owner confirmation. -The prior three-length reducer cannot be treated as the new activated validator. -A versioned 8K-only reducer and complete freeze are still required before search. - -Retrieval retains the existing ten 8K inputs, all four positions and scorer. -The matched baseline has10/10 complete matches and10/10 at every position. -Candidate complete and per-position counts must not regress, thus10/10 is -required at8K. No resampling, input mutation or post-result scorer change. -The prior64K/128K short-output vLLM retrieval requests succeeded. The native -attention path has a static long-context memory limitation; full2048-output -performance acceptance was not established. The new scope follows the owner's -explicit choice and does not assert that the prior baseline requests failed. - -GSM8K1319/HumanEval+164/MMLU1000 quality scope, frozen screening subsets and each -per-dataset one-percentage-point gate remain unchanged. Full164 CPU reference -preparation remains applicable. The approved44GPUh phased cap2/4/8/30 remains an -upper limit; actual GPU usage is2388readiness seconds plus147separately approved -initial seconds. Formal baseline/search/finalization stages remain unused. - -Under the pending six-restart AB/BA/AB proposal, one8K warmup plus ten timed -requests per restart would produce60timed+6warmup requests=135168 output tokens, -compared with405504 in the older three-length proposal. This is a workload count, -not a measured time estimate or a new budget authorization. Cost estimates will -be revised after matching bounded measurements. No additional GPU is used by -this scope-record update. - -Campaign remains INCONCLUSIVE. Evaluator qualification, statistical decision, -final source/runtime/command/validator freeze and full acceptance remain pending. -No search, quality PASS or E2E gain is activated. Autonomous reviewed PR merge -is authorized; final campaign acceptance and Issue closure remain owner actions. - -KnowledgeStore candidate revisions (readback verified; no protected promotion): - -- `case.qwen38-statistics-proposal-cand`: `sha256:dff55bb95214f4192c1fd4f8544c57be838a50a3ede294814be92f1bc24feff1` -- `case.qwen38-retrieval-construction-cand`: `sha256:90cbe064f0885f4bd2089e0b0c66e7445c2ddc7380b4ffb46bbbc5ab470c7d1f` diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/plan/quality-v4-source-identities.json b/evidence/qwen38-h20-fp8-single-request-vllm032/plan/quality-v4-source-identities.json deleted file mode 100644 index d52980416..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/plan/quality-v4-source-identities.json +++ /dev/null @@ -1,17 +0,0 @@ -{ - "state": "proposal_not_activated", - "scope": "quality_transport_termination_successor", - "files": { - "protocol.py": "5865b15e2101a17a024004be4c44b7715737b0a5face69e095a3aacd8fd1f2fd", - "prepare.py": "7a3fa40367871fbe775f9ddaa2c97e90c2672d49cbe4faf91a69b176898d72c3", - "run.py": "c0102c31c713758c0a35379e79cd162189ad44511d8ba90e1e5f85ab0d3f4af9", - "score.py": "79e93cf0cd73750821a438dc17380eead9883c1c6b84c75dff99c29c3cf96005", - "retrieval.py": "45f05a85cc1721dbe69beadc370cd84f7d1d5b1bb1d3593cc7d054d499cb53ec", - "source-lock.json": "d4f38283b893aebaa8ebd917c04ca78479370fd5d3e6a5e48552d54fb9e0df5f", - "prepare_guard.py": "8ca992130d624d659bf7c1d5deef53abf0df1ef5af07ca60753706c188fd429c", - "prepare-quality.sbatch": "cee87ec17bbcb45ab3cab16bd3990903b35e02aafdb4e82f9015144b590a0a70" - }, - "generation_config_sha256": "e70c136c1b78ddc1fb0905bac8e733a4dc448d4f852a5dd75143fffc70be550e", - "materialization": "Apply retained null-echo patch then EOS-v4 patch to the authenticated v2 archive; full source remains in controlled task storage.", - "original_archive_sha256": "9c9cddd3242c00b6c39e4a6342a4ab7d17ab3e74f07bbd7c628c6952f4517099" -} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/plan/retrieval-integrity.json b/evidence/qwen38-h20-fp8-single-request-vllm032/plan/retrieval-integrity.json deleted file mode 100644 index 14ec90cf0..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/plan/retrieval-integrity.json +++ /dev/null @@ -1,11 +0,0 @@ -{ - "plan/retrieval-proposal-v1/metadata.json": "17b78b353ecd3c60598e003ff27aad9ae30babe3e5e64bf868683becf5c21eff", - "plan/retrieval-proposal-v1/inputs.jsonl.gz": "72c21dcd655df6851451dd4e2f3e6f0b056d777494fe2723219a83dbcc90a009", - "plan/retrieval-proposal-v2/metadata.json": "08a410773c441f3e8c648e6015751a80da50454d4331e21ad040d9fd83f8e7a9", - "plan/retrieval-proposal-v2/inputs.jsonl.gz": "721811a4302b022f2ff2a14dbcfb90c2744e58104d92885c13be09a37a0ce30d", - "validator/diagnostics/prepare_retrieval.py": "0571ed279b9ef750654b2bc077e70a61d722506e05804c1c632425e57c19d38b", - "validator/diagnostics/prepare_retrieval_v2.py": "035c1b317c4a1d0d09c47eb07ebeb95da144751b0cd2b04fea85b39e7c96339d", - "validator/diagnostics/retrieval_baseline.py": "128f3e7903a94aea37f1234fe8850b47822b2fd8843fa68fa154a7908c0a8bf1", - "validator/diagnostics/retrieval_baseline_v2.py": "f79c744e4285fd7ff31af16376884cf67905a3b1faca7c584630537344870456", - "validator/diagnostics/retrieval_baseline_v3.py": "10f7d81bd971337f237ed9a20ff704ee7bcd84cfbd769a9e94b049bf6d21d671" -} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/plan/retrieval-proposal-v1/README.md b/evidence/qwen38-h20-fp8-single-request-vllm032/plan/retrieval-proposal-v1/README.md deleted file mode 100644 index 12b74d16c..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/plan/retrieval-proposal-v1/README.md +++ /dev/null @@ -1,41 +0,0 @@ -# Retrieval input proposal — not an activated acceptance freeze - -CPU-only job1964 completed0 in28 seconds, with30 exact-length inputs and2,048,000 -tokens in total. Inputs use the authenticated official tokenizer/chat template -and thinking disabled. At each of8192/65536/131072 tokens,10 distinct cases each -contain four independent key/value targets near10/35/65/90 percent of absolute -input token positions. The recorded starts include the newline preceding each -record. Synthetic decoys are deterministically generated; no private corpus. - -Every target key appears exactly twice in the decoded prompt (record and query); -every target value appears once. The exact needle slices decode to their stated -record text. The generated archive is compressed losslessly with deterministic -gzip metadata; manifest records both compressed and uncompressed SHA256 plus -per-case input hashes, positions, expected answers and tokenizer identities. -The local readback independently authenticated all30 records, lengths, hashes, -manifest correspondence and positions. This is preparation, not retrieval -accuracy or model execution. The generator is retained unchanged under -validator/diagnostics/prepare_retrieval.py. - -Extraction is predeclared in metadata.json: strip surrounding whitespace; -optionally unwrap a single enclosing lowercase `json` code fence; parse the -entire remainder as JSON; reject duplicate keys, missing/extra keys and nonstring -values. Score exact expected strings, without case folding or substring matching. -Report complete-case accuracy and each position. The7 local extraction checks -cover exact object, permitted fence, duplicates, extra keys, wrong case, invalid -JSON and nonobject output. - -No answers have been observed at input preparation. Baseline qualification and -owner approval precede any activation; no regeneration based on model mistakes. -The9/10 baseline proposal and candidate nonregression gate remain owner-review -drafts from plan/freeze-proposal.md. No final-set candidate tuning is authorized. - -Disposition before any model answers: independent review rejected this version -for freezing because targets differ in format and decoys repeat. See sibling -retrieval-proposal-v2 for the revised inputs; original bytes are retained. - -Publication layout: the generator output originally named `manifest.json` is -retained byte-for-byte as `metadata.json`. The existing bundle-sha256 schema -classifies and authenticates this metadata, compressed inputs and observer -sources through `plan/retrieval-integrity.json`; no validator schema or gate is -weakened. Original remote artifacts retain their generated names. diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/plan/retrieval-proposal-v1/inputs.jsonl.gz b/evidence/qwen38-h20-fp8-single-request-vllm032/plan/retrieval-proposal-v1/inputs.jsonl.gz deleted file mode 100644 index 0332bdc78..000000000 Binary files a/evidence/qwen38-h20-fp8-single-request-vllm032/plan/retrieval-proposal-v1/inputs.jsonl.gz and /dev/null differ diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/plan/retrieval-proposal-v1/metadata.json b/evidence/qwen38-h20-fp8-single-request-vllm032/plan/retrieval-proposal-v1/metadata.json deleted file mode 100644 index f517d69a0..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/plan/retrieval-proposal-v1/metadata.json +++ /dev/null @@ -1,1488 +0,0 @@ -{ - "schema": "retrieval-proposal-v1", - "status": "PROPOSED_NOT_ACTIVATED", - "cases": [ - { - "id": "retrieval-n8192-s0", - "input_tokens": 8192, - "input_ids_sha256": "f023d5f9bc2f0486df95fdfe9306dc0adb75ddde06ff1fd45da924313ef73b8a", - "expected": { - "KEY_6D40C7185207": "VAL_A4FC936A7635", - "KEY_29507F8D704C": "VAL_64E39505D8E0", - "KEY_C08D82170F07": "VAL_52B53BADE83E", - "KEY_0101F12DE978": "VAL_E4CE4FC35FEA" - }, - "placements": [ - { - "index": 0, - "target_percent": 10, - "record_start_token": 819, - "record_tokens": 32, - "key": "KEY_6D40C7185207", - "value": "VAL_A4FC936A7635" - }, - { - "index": 1, - "target_percent": 35, - "record_start_token": 2867, - "record_tokens": 34, - "key": "KEY_29507F8D704C", - "value": "VAL_64E39505D8E0" - }, - { - "index": 2, - "target_percent": 65, - "record_start_token": 5324, - "record_tokens": 31, - "key": "KEY_C08D82170F07", - "value": "VAL_52B53BADE83E" - }, - { - "index": 3, - "target_percent": 90, - "record_start_token": 7372, - "record_tokens": 29, - "key": "KEY_0101F12DE978", - "value": "VAL_E4CE4FC35FEA" - } - ], - "max_output_tokens": 2048, - "ignore_eos": false, - "temperature": 0, - "thinking": false - }, - { - "id": "retrieval-n8192-s1", - "input_tokens": 8192, - "input_ids_sha256": "b4b971b7981cfc46382dc9502e60819b9408bc97bbb0b882aad28ba85018b705", - "expected": { - "KEY_B028448F3330": "VAL_F497FBCECAAF", - "KEY_321BB3FD1153": "VAL_49D7BE6AA9C6", - "KEY_D15BFC3FF318": "VAL_8B68C8EF2C12", - "KEY_632D25557BBC": "VAL_FC11601AC939" - }, - "placements": [ - { - "index": 0, - "target_percent": 10, - "record_start_token": 819, - "record_tokens": 28, - "key": "KEY_B028448F3330", - "value": "VAL_F497FBCECAAF" - }, - { - "index": 1, - "target_percent": 35, - "record_start_token": 2867, - "record_tokens": 30, - "key": "KEY_321BB3FD1153", - "value": "VAL_49D7BE6AA9C6" - }, - { - "index": 2, - "target_percent": 65, - "record_start_token": 5324, - "record_tokens": 30, - "key": "KEY_D15BFC3FF318", - "value": "VAL_8B68C8EF2C12" - }, - { - "index": 3, - "target_percent": 90, - "record_start_token": 7372, - "record_tokens": 29, - "key": "KEY_632D25557BBC", - "value": "VAL_FC11601AC939" - } - ], - "max_output_tokens": 2048, - "ignore_eos": false, - "temperature": 0, - "thinking": false - }, - { - "id": "retrieval-n8192-s2", - "input_tokens": 8192, - "input_ids_sha256": "88f52c864bffada1b55d32a9ea4f2239e9904314962693aed8e5335eb90c2c49", - "expected": { - "KEY_4215D36753C6": "VAL_A2E0CE8ADD7F", - "KEY_712D91B2AADC": "VAL_43CF4195334D", - "KEY_E1282770324D": "VAL_9197B16A1C9F", - "KEY_019F12ACF217": "VAL_5449D270270E" - }, - "placements": [ - { - "index": 0, - "target_percent": 10, - "record_start_token": 819, - "record_tokens": 30, - "key": "KEY_4215D36753C6", - "value": "VAL_A2E0CE8ADD7F" - }, - { - "index": 1, - "target_percent": 35, - "record_start_token": 2867, - "record_tokens": 31, - "key": "KEY_712D91B2AADC", - "value": "VAL_43CF4195334D" - }, - { - "index": 2, - "target_percent": 65, - "record_start_token": 5324, - "record_tokens": 33, - "key": "KEY_E1282770324D", - "value": "VAL_9197B16A1C9F" - }, - { - "index": 3, - "target_percent": 90, - "record_start_token": 7372, - "record_tokens": 33, - "key": "KEY_019F12ACF217", - "value": "VAL_5449D270270E" - } - ], - "max_output_tokens": 2048, - "ignore_eos": false, - "temperature": 0, - "thinking": false - }, - { - "id": "retrieval-n8192-s3", - "input_tokens": 8192, - "input_ids_sha256": "4e090adc2c88f98840e589e297e9094fd509989d45017d88d900a0027cd0dffd", - "expected": { - "KEY_66E016CE6604": "VAL_7EC506AAC095", - "KEY_A1FA3F7B5DF9": "VAL_E33C1B8A470A", - "KEY_D4036EE7A178": "VAL_26C26F3DFC44", - "KEY_A22288E39B25": "VAL_A377217ECD60" - }, - "placements": [ - { - "index": 0, - "target_percent": 10, - "record_start_token": 819, - "record_tokens": 30, - "key": "KEY_66E016CE6604", - "value": "VAL_7EC506AAC095" - }, - { - "index": 1, - "target_percent": 35, - "record_start_token": 2867, - "record_tokens": 30, - "key": "KEY_A1FA3F7B5DF9", - "value": "VAL_E33C1B8A470A" - }, - { - "index": 2, - "target_percent": 65, - "record_start_token": 5324, - "record_tokens": 31, - "key": "KEY_D4036EE7A178", - "value": "VAL_26C26F3DFC44" - }, - { - "index": 3, - "target_percent": 90, - "record_start_token": 7372, - "record_tokens": 30, - "key": "KEY_A22288E39B25", - "value": "VAL_A377217ECD60" - } - ], - "max_output_tokens": 2048, - "ignore_eos": false, - "temperature": 0, - "thinking": false - }, - { - "id": "retrieval-n8192-s4", - "input_tokens": 8192, - "input_ids_sha256": "0d22ee699f7bc7588af1ce6755fe6a1394d704bdc1c9bc576f4fcc5e661c36d3", - "expected": { - "KEY_88E9F681F7A1": "VAL_D195BEBC34B3", - "KEY_51DB7FBCAA61": "VAL_2EADADDEDD61", - "KEY_DFD91DFB0ED7": "VAL_33232F89BE38", - "KEY_5A15B15B7CE0": "VAL_ABC8F5980F9C" - }, - "placements": [ - { - "index": 0, - "target_percent": 10, - "record_start_token": 819, - "record_tokens": 31, - "key": "KEY_88E9F681F7A1", - "value": "VAL_D195BEBC34B3" - }, - { - "index": 1, - "target_percent": 35, - "record_start_token": 2867, - "record_tokens": 27, - "key": "KEY_51DB7FBCAA61", - "value": "VAL_2EADADDEDD61" - }, - { - "index": 2, - "target_percent": 65, - "record_start_token": 5324, - "record_tokens": 29, - "key": "KEY_DFD91DFB0ED7", - "value": "VAL_33232F89BE38" - }, - { - "index": 3, - "target_percent": 90, - "record_start_token": 7372, - "record_tokens": 30, - "key": "KEY_5A15B15B7CE0", - "value": "VAL_ABC8F5980F9C" - } - ], - "max_output_tokens": 2048, - "ignore_eos": false, - "temperature": 0, - "thinking": false - }, - { - "id": "retrieval-n8192-s5", - "input_tokens": 8192, - "input_ids_sha256": "e149df52f14ddffc02c12a868dc6f990f2d8e46bb08e0cc9bd5b3922090ab845", - "expected": { - "KEY_589F55CE6B6B": "VAL_D42A03B12EAF", - "KEY_963B6CBD37C6": "VAL_903CEE8EFD93", - "KEY_0944063BD472": "VAL_F8EBF1548534", - "KEY_22E7EAA46B61": "VAL_EEAD4DC1F582" - }, - "placements": [ - { - "index": 0, - "target_percent": 10, - "record_start_token": 819, - "record_tokens": 31, - "key": "KEY_589F55CE6B6B", - "value": "VAL_D42A03B12EAF" - }, - { - "index": 1, - "target_percent": 35, - "record_start_token": 2867, - "record_tokens": 29, - "key": "KEY_963B6CBD37C6", - "value": "VAL_903CEE8EFD93" - }, - { - "index": 2, - "target_percent": 65, - "record_start_token": 5324, - "record_tokens": 31, - "key": "KEY_0944063BD472", - "value": "VAL_F8EBF1548534" - }, - { - "index": 3, - "target_percent": 90, - "record_start_token": 7372, - "record_tokens": 30, - "key": "KEY_22E7EAA46B61", - "value": "VAL_EEAD4DC1F582" - } - ], - "max_output_tokens": 2048, - "ignore_eos": false, - "temperature": 0, - "thinking": false - }, - { - "id": "retrieval-n8192-s6", - "input_tokens": 8192, - "input_ids_sha256": "17859d4a875abd8489337644541e0e95403e0afaa9393302be19069030eb7f41", - "expected": { - "KEY_FA32E1A3AD31": "VAL_E24CB474CA36", - "KEY_4F7F423B73C6": "VAL_E885E8354CDA", - "KEY_741BC0F81657": "VAL_073D7E6A0EFB", - "KEY_CD3F9567FEC7": "VAL_47D46A22B4E8" - }, - "placements": [ - { - "index": 0, - "target_percent": 10, - "record_start_token": 819, - "record_tokens": 28, - "key": "KEY_FA32E1A3AD31", - "value": "VAL_E24CB474CA36" - }, - { - "index": 1, - "target_percent": 35, - "record_start_token": 2867, - "record_tokens": 32, - "key": "KEY_4F7F423B73C6", - "value": "VAL_E885E8354CDA" - }, - { - "index": 2, - "target_percent": 65, - "record_start_token": 5324, - "record_tokens": 32, - "key": "KEY_741BC0F81657", - "value": "VAL_073D7E6A0EFB" - }, - { - "index": 3, - "target_percent": 90, - "record_start_token": 7372, - "record_tokens": 31, - "key": "KEY_CD3F9567FEC7", - "value": "VAL_47D46A22B4E8" - } - ], - "max_output_tokens": 2048, - "ignore_eos": false, - "temperature": 0, - "thinking": false - }, - { - "id": "retrieval-n8192-s7", - "input_tokens": 8192, - "input_ids_sha256": "4466b727b6d8e4b803ad8f7050f6496fa687eea5b3d6463418415a3a00133267", - "expected": { - "KEY_9C8FA0551A77": "VAL_860EF143333A", - "KEY_4627A60B4817": "VAL_3FB18EE80E74", - "KEY_42AD07FE03DB": "VAL_A4513AA7606C", - "KEY_3579A87B332C": "VAL_E6EB1DD7CE27" - }, - "placements": [ - { - "index": 0, - "target_percent": 10, - "record_start_token": 819, - "record_tokens": 32, - "key": "KEY_9C8FA0551A77", - "value": "VAL_860EF143333A" - }, - { - "index": 1, - "target_percent": 35, - "record_start_token": 2867, - "record_tokens": 32, - "key": "KEY_4627A60B4817", - "value": "VAL_3FB18EE80E74" - }, - { - "index": 2, - "target_percent": 65, - "record_start_token": 5324, - "record_tokens": 29, - "key": "KEY_42AD07FE03DB", - "value": "VAL_A4513AA7606C" - }, - { - "index": 3, - "target_percent": 90, - "record_start_token": 7372, - "record_tokens": 30, - "key": "KEY_3579A87B332C", - "value": "VAL_E6EB1DD7CE27" - } - ], - "max_output_tokens": 2048, - "ignore_eos": false, - "temperature": 0, - "thinking": false - }, - { - "id": "retrieval-n8192-s8", - "input_tokens": 8192, - "input_ids_sha256": "3ae82224c14f62bfff032bcb2f842caf482090744c0de0e2150da830d64d6f48", - "expected": { - "KEY_28DA23D73F91": "VAL_15E2CE2DB22E", - "KEY_AF71C29B5398": "VAL_BA169E71ABFA", - "KEY_30633C955CDA": "VAL_21E4AEBCC025", - "KEY_DC55B764D47A": "VAL_03A63F6F8A59" - }, - "placements": [ - { - "index": 0, - "target_percent": 10, - "record_start_token": 819, - "record_tokens": 31, - "key": "KEY_28DA23D73F91", - "value": "VAL_15E2CE2DB22E" - }, - { - "index": 1, - "target_percent": 35, - "record_start_token": 2867, - "record_tokens": 29, - "key": "KEY_AF71C29B5398", - "value": "VAL_BA169E71ABFA" - }, - { - "index": 2, - "target_percent": 65, - "record_start_token": 5324, - "record_tokens": 31, - "key": "KEY_30633C955CDA", - "value": "VAL_21E4AEBCC025" - }, - { - "index": 3, - "target_percent": 90, - "record_start_token": 7372, - "record_tokens": 32, - "key": "KEY_DC55B764D47A", - "value": "VAL_03A63F6F8A59" - } - ], - "max_output_tokens": 2048, - "ignore_eos": false, - "temperature": 0, - "thinking": false - }, - { - "id": "retrieval-n8192-s9", - "input_tokens": 8192, - "input_ids_sha256": "558dc98da88cc5d6507f872ac54409f7615e70e4218c3afbd94336d42ac99f72", - "expected": { - "KEY_EA04F9A2EB86": "VAL_5AA754731DD0", - "KEY_DEF1FEBC6B2A": "VAL_7EBC5696FB73", - "KEY_672BA14E63E5": "VAL_00B68689C59D", - "KEY_FAA91316F7F2": "VAL_156EF068452A" - }, - "placements": [ - { - "index": 0, - "target_percent": 10, - "record_start_token": 819, - "record_tokens": 30, - "key": "KEY_EA04F9A2EB86", - "value": "VAL_5AA754731DD0" - }, - { - "index": 1, - "target_percent": 35, - "record_start_token": 2867, - "record_tokens": 27, - "key": "KEY_DEF1FEBC6B2A", - "value": "VAL_7EBC5696FB73" - }, - { - "index": 2, - "target_percent": 65, - "record_start_token": 5324, - "record_tokens": 33, - "key": "KEY_672BA14E63E5", - "value": "VAL_00B68689C59D" - }, - { - "index": 3, - "target_percent": 90, - "record_start_token": 7372, - "record_tokens": 31, - "key": "KEY_FAA91316F7F2", - "value": "VAL_156EF068452A" - } - ], - "max_output_tokens": 2048, - "ignore_eos": false, - "temperature": 0, - "thinking": false - }, - { - "id": "retrieval-n65536-s0", - "input_tokens": 65536, - "input_ids_sha256": "c05d38b4f50e1541b0db6b651c2e5d0078b4526574f4da4fc8ef5bd06bd73e88", - "expected": { - "KEY_D110644FE1BC": "VAL_EC9D0CAFE91A", - "KEY_61E5C31CC06A": "VAL_4302277DCEEF", - "KEY_565460FA50C0": "VAL_EE564E2E2F59", - "KEY_ED4E96750FB3": "VAL_23E3908791DF" - }, - "placements": [ - { - "index": 0, - "target_percent": 10, - "record_start_token": 6553, - "record_tokens": 27, - "key": "KEY_D110644FE1BC", - "value": "VAL_EC9D0CAFE91A" - }, - { - "index": 1, - "target_percent": 35, - "record_start_token": 22937, - "record_tokens": 31, - "key": "KEY_61E5C31CC06A", - "value": "VAL_4302277DCEEF" - }, - { - "index": 2, - "target_percent": 65, - "record_start_token": 42598, - "record_tokens": 32, - "key": "KEY_565460FA50C0", - "value": "VAL_EE564E2E2F59" - }, - { - "index": 3, - "target_percent": 90, - "record_start_token": 58982, - "record_tokens": 30, - "key": "KEY_ED4E96750FB3", - "value": "VAL_23E3908791DF" - } - ], - "max_output_tokens": 2048, - "ignore_eos": false, - "temperature": 0, - "thinking": false - }, - { - "id": "retrieval-n65536-s1", - "input_tokens": 65536, - "input_ids_sha256": "d674c2041868743297a0fbcdfc2f7264b40c6a03087deb78b1f732b9ebea784c", - "expected": { - "KEY_44FA0BB13EEE": "VAL_A0D73B8E332F", - "KEY_C2EC095B488C": "VAL_ACBBC756608D", - "KEY_546C78A3C74A": "VAL_D7F0D06223E0", - "KEY_D31CF5336B6E": "VAL_8306E869EB68" - }, - "placements": [ - { - "index": 0, - "target_percent": 10, - "record_start_token": 6553, - "record_tokens": 29, - "key": "KEY_44FA0BB13EEE", - "value": "VAL_A0D73B8E332F" - }, - { - "index": 1, - "target_percent": 35, - "record_start_token": 22937, - "record_tokens": 29, - "key": "KEY_C2EC095B488C", - "value": "VAL_ACBBC756608D" - }, - { - "index": 2, - "target_percent": 65, - "record_start_token": 42598, - "record_tokens": 33, - "key": "KEY_546C78A3C74A", - "value": "VAL_D7F0D06223E0" - }, - { - "index": 3, - "target_percent": 90, - "record_start_token": 58982, - "record_tokens": 31, - "key": "KEY_D31CF5336B6E", - "value": "VAL_8306E869EB68" - } - ], - "max_output_tokens": 2048, - "ignore_eos": false, - "temperature": 0, - "thinking": false - }, - { - "id": "retrieval-n65536-s2", - "input_tokens": 65536, - "input_ids_sha256": "0333c6a7b79f2fb4e2fce0be823d4299ce9e00eb332f28c40d2f9be14cb9345a", - "expected": { - "KEY_FAE626ED216A": "VAL_2A7C5EF723DB", - "KEY_FDCCB859E71D": "VAL_8CFBD8668BF3", - "KEY_25E644AAD70D": "VAL_F5F7CAB7DA91", - "KEY_6731635C4690": "VAL_4A4A90570EC4" - }, - "placements": [ - { - "index": 0, - "target_percent": 10, - "record_start_token": 6553, - "record_tokens": 29, - "key": "KEY_FAE626ED216A", - "value": "VAL_2A7C5EF723DB" - }, - { - "index": 1, - "target_percent": 35, - "record_start_token": 22937, - "record_tokens": 28, - "key": "KEY_FDCCB859E71D", - "value": "VAL_8CFBD8668BF3" - }, - { - "index": 2, - "target_percent": 65, - "record_start_token": 42598, - "record_tokens": 29, - "key": "KEY_25E644AAD70D", - "value": "VAL_F5F7CAB7DA91" - }, - { - "index": 3, - "target_percent": 90, - "record_start_token": 58982, - "record_tokens": 33, - "key": "KEY_6731635C4690", - "value": "VAL_4A4A90570EC4" - } - ], - "max_output_tokens": 2048, - "ignore_eos": false, - "temperature": 0, - "thinking": false - }, - { - "id": "retrieval-n65536-s3", - "input_tokens": 65536, - "input_ids_sha256": "5b7dad42dabc9dfbb782555134295383668fe4a387218258c33f9018480f7387", - "expected": { - "KEY_9C8496424102": "VAL_CBFFB4A7728D", - "KEY_C0A02A8BFD5F": "VAL_697398AA7015", - "KEY_1BCB66B7E4EE": "VAL_38AD6A66871E", - "KEY_223324B8EE36": "VAL_032A72253E9D" - }, - "placements": [ - { - "index": 0, - "target_percent": 10, - "record_start_token": 6553, - "record_tokens": 31, - "key": "KEY_9C8496424102", - "value": "VAL_CBFFB4A7728D" - }, - { - "index": 1, - "target_percent": 35, - "record_start_token": 22937, - "record_tokens": 31, - "key": "KEY_C0A02A8BFD5F", - "value": "VAL_697398AA7015" - }, - { - "index": 2, - "target_percent": 65, - "record_start_token": 42598, - "record_tokens": 31, - "key": "KEY_1BCB66B7E4EE", - "value": "VAL_38AD6A66871E" - }, - { - "index": 3, - "target_percent": 90, - "record_start_token": 58982, - "record_tokens": 33, - "key": "KEY_223324B8EE36", - "value": "VAL_032A72253E9D" - } - ], - "max_output_tokens": 2048, - "ignore_eos": false, - "temperature": 0, - "thinking": false - }, - { - "id": "retrieval-n65536-s4", - "input_tokens": 65536, - "input_ids_sha256": "3f903ce88b220a26f2137c824b34533bc15f051c80035e3e44c3b1bffce2d2ba", - "expected": { - "KEY_D9684E2B8C5C": "VAL_E38BD16E9B82", - "KEY_9A7689BCA2AE": "VAL_5733B58B2A7E", - "KEY_940EFF5D8E6F": "VAL_EAC6BFCB7BF6", - "KEY_EAB028A79FEB": "VAL_53718861A3A2" - }, - "placements": [ - { - "index": 0, - "target_percent": 10, - "record_start_token": 6553, - "record_tokens": 31, - "key": "KEY_D9684E2B8C5C", - "value": "VAL_E38BD16E9B82" - }, - { - "index": 1, - "target_percent": 35, - "record_start_token": 22937, - "record_tokens": 32, - "key": "KEY_9A7689BCA2AE", - "value": "VAL_5733B58B2A7E" - }, - { - "index": 2, - "target_percent": 65, - "record_start_token": 42598, - "record_tokens": 28, - "key": "KEY_940EFF5D8E6F", - "value": "VAL_EAC6BFCB7BF6" - }, - { - "index": 3, - "target_percent": 90, - "record_start_token": 58982, - "record_tokens": 31, - "key": "KEY_EAB028A79FEB", - "value": "VAL_53718861A3A2" - } - ], - "max_output_tokens": 2048, - "ignore_eos": false, - "temperature": 0, - "thinking": false - }, - { - "id": "retrieval-n65536-s5", - "input_tokens": 65536, - "input_ids_sha256": "0b62fe7bc348bf9639ff7460e87c2527763fbde5de2b7d35d1c211fafe982e54", - "expected": { - "KEY_6DE73A97331F": "VAL_D3BB2A34A29E", - "KEY_E41B14834F4D": "VAL_06A8565CCD9A", - "KEY_D1C293566BB0": "VAL_8BB5091BAF46", - "KEY_5C6DA432051B": "VAL_E3A5D42D5589" - }, - "placements": [ - { - "index": 0, - "target_percent": 10, - "record_start_token": 6553, - "record_tokens": 31, - "key": "KEY_6DE73A97331F", - "value": "VAL_D3BB2A34A29E" - }, - { - "index": 1, - "target_percent": 35, - "record_start_token": 22937, - "record_tokens": 32, - "key": "KEY_E41B14834F4D", - "value": "VAL_06A8565CCD9A" - }, - { - "index": 2, - "target_percent": 65, - "record_start_token": 42598, - "record_tokens": 30, - "key": "KEY_D1C293566BB0", - "value": "VAL_8BB5091BAF46" - }, - { - "index": 3, - "target_percent": 90, - "record_start_token": 58982, - "record_tokens": 32, - "key": "KEY_5C6DA432051B", - "value": "VAL_E3A5D42D5589" - } - ], - "max_output_tokens": 2048, - "ignore_eos": false, - "temperature": 0, - "thinking": false - }, - { - "id": "retrieval-n65536-s6", - "input_tokens": 65536, - "input_ids_sha256": "87ca61fec32f58ee868319534bf8a80f8633a704267fa58dd1192ab26f7ae3e9", - "expected": { - "KEY_6CC3A5ECA10E": "VAL_9C1BC3BA2EF7", - "KEY_8EB0719834B2": "VAL_7EE2C96C2D10", - "KEY_257D73149273": "VAL_6B3712DA7665", - "KEY_3C872903DFB6": "VAL_F4E5E8E354FD" - }, - "placements": [ - { - "index": 0, - "target_percent": 10, - "record_start_token": 6553, - "record_tokens": 28, - "key": "KEY_6CC3A5ECA10E", - "value": "VAL_9C1BC3BA2EF7" - }, - { - "index": 1, - "target_percent": 35, - "record_start_token": 22937, - "record_tokens": 32, - "key": "KEY_8EB0719834B2", - "value": "VAL_7EE2C96C2D10" - }, - { - "index": 2, - "target_percent": 65, - "record_start_token": 42598, - "record_tokens": 33, - "key": "KEY_257D73149273", - "value": "VAL_6B3712DA7665" - }, - { - "index": 3, - "target_percent": 90, - "record_start_token": 58982, - "record_tokens": 31, - "key": "KEY_3C872903DFB6", - "value": "VAL_F4E5E8E354FD" - } - ], - "max_output_tokens": 2048, - "ignore_eos": false, - "temperature": 0, - "thinking": false - }, - { - "id": "retrieval-n65536-s7", - "input_tokens": 65536, - "input_ids_sha256": "52ae98b3b27afa4003ff2cd3fe4ddcebb6a06e67718403f74d8c7850abf6d560", - "expected": { - "KEY_0BC2A1E4FF48": "VAL_1D877AC1F667", - "KEY_1B4538EDD0DD": "VAL_A601533AAD6B", - "KEY_3247F58DF0A8": "VAL_AD687A8CEB46", - "KEY_44C197F8F532": "VAL_9A5CEAEAF080" - }, - "placements": [ - { - "index": 0, - "target_percent": 10, - "record_start_token": 6553, - "record_tokens": 31, - "key": "KEY_0BC2A1E4FF48", - "value": "VAL_1D877AC1F667" - }, - { - "index": 1, - "target_percent": 35, - "record_start_token": 22937, - "record_tokens": 29, - "key": "KEY_1B4538EDD0DD", - "value": "VAL_A601533AAD6B" - }, - { - "index": 2, - "target_percent": 65, - "record_start_token": 42598, - "record_tokens": 30, - "key": "KEY_3247F58DF0A8", - "value": "VAL_AD687A8CEB46" - }, - { - "index": 3, - "target_percent": 90, - "record_start_token": 58982, - "record_tokens": 31, - "key": "KEY_44C197F8F532", - "value": "VAL_9A5CEAEAF080" - } - ], - "max_output_tokens": 2048, - "ignore_eos": false, - "temperature": 0, - "thinking": false - }, - { - "id": "retrieval-n65536-s8", - "input_tokens": 65536, - "input_ids_sha256": "fd9af1f6ab2feb6e6c426b0e9f6439875bc5847c68efef3b96947e5e4d908347", - "expected": { - "KEY_B60083D32F6F": "VAL_14E4687993AC", - "KEY_0560A1851F8C": "VAL_0FAA51D3E3E6", - "KEY_7D7EB0CD6D66": "VAL_5415F89A5F83", - "KEY_440EE7B89BDC": "VAL_FEA0D606BE4E" - }, - "placements": [ - { - "index": 0, - "target_percent": 10, - "record_start_token": 6553, - "record_tokens": 32, - "key": "KEY_B60083D32F6F", - "value": "VAL_14E4687993AC" - }, - { - "index": 1, - "target_percent": 35, - "record_start_token": 22937, - "record_tokens": 33, - "key": "KEY_0560A1851F8C", - "value": "VAL_0FAA51D3E3E6" - }, - { - "index": 2, - "target_percent": 65, - "record_start_token": 42598, - "record_tokens": 32, - "key": "KEY_7D7EB0CD6D66", - "value": "VAL_5415F89A5F83" - }, - { - "index": 3, - "target_percent": 90, - "record_start_token": 58982, - "record_tokens": 29, - "key": "KEY_440EE7B89BDC", - "value": "VAL_FEA0D606BE4E" - } - ], - "max_output_tokens": 2048, - "ignore_eos": false, - "temperature": 0, - "thinking": false - }, - { - "id": "retrieval-n65536-s9", - "input_tokens": 65536, - "input_ids_sha256": "689ef67730b92f94a6615dd486e69a13c07b12e9cae8d4965d0a9ddd4286df15", - "expected": { - "KEY_C804FBB370A2": "VAL_C9E4C4C8B591", - "KEY_33A414B076A5": "VAL_860C58D8C858", - "KEY_DAF6090677E0": "VAL_D93FA35AF08A", - "KEY_C121DEEDF7D1": "VAL_79D4D7349B40" - }, - "placements": [ - { - "index": 0, - "target_percent": 10, - "record_start_token": 6553, - "record_tokens": 31, - "key": "KEY_C804FBB370A2", - "value": "VAL_C9E4C4C8B591" - }, - { - "index": 1, - "target_percent": 35, - "record_start_token": 22937, - "record_tokens": 34, - "key": "KEY_33A414B076A5", - "value": "VAL_860C58D8C858" - }, - { - "index": 2, - "target_percent": 65, - "record_start_token": 42598, - "record_tokens": 29, - "key": "KEY_DAF6090677E0", - "value": "VAL_D93FA35AF08A" - }, - { - "index": 3, - "target_percent": 90, - "record_start_token": 58982, - "record_tokens": 31, - "key": "KEY_C121DEEDF7D1", - "value": "VAL_79D4D7349B40" - } - ], - "max_output_tokens": 2048, - "ignore_eos": false, - "temperature": 0, - "thinking": false - }, - { - "id": "retrieval-n131072-s0", - "input_tokens": 131072, - "input_ids_sha256": "8a11a9f797db9cccf9443f4dcf691d76457dbf3a598c607a251001c88edc27b6", - "expected": { - "KEY_737F4BEE7628": "VAL_778624E378DE", - "KEY_6531CCE40387": "VAL_73B7DA3FE02A", - "KEY_DCE4FD63CB23": "VAL_B438467187EF", - "KEY_9EE7B7C5FE26": "VAL_C31439E4C184" - }, - "placements": [ - { - "index": 0, - "target_percent": 10, - "record_start_token": 13107, - "record_tokens": 32, - "key": "KEY_737F4BEE7628", - "value": "VAL_778624E378DE" - }, - { - "index": 1, - "target_percent": 35, - "record_start_token": 45875, - "record_tokens": 31, - "key": "KEY_6531CCE40387", - "value": "VAL_73B7DA3FE02A" - }, - { - "index": 2, - "target_percent": 65, - "record_start_token": 85196, - "record_tokens": 28, - "key": "KEY_DCE4FD63CB23", - "value": "VAL_B438467187EF" - }, - { - "index": 3, - "target_percent": 90, - "record_start_token": 117964, - "record_tokens": 31, - "key": "KEY_9EE7B7C5FE26", - "value": "VAL_C31439E4C184" - } - ], - "max_output_tokens": 2048, - "ignore_eos": false, - "temperature": 0, - "thinking": false - }, - { - "id": "retrieval-n131072-s1", - "input_tokens": 131072, - "input_ids_sha256": "dcd4391fe3b78c88b83174e3cfbdec00ec897aca4b659ee8eb33458218240271", - "expected": { - "KEY_366F554BD828": "VAL_855DEFE56D16", - "KEY_5F7945F5DDDA": "VAL_4A05F879BF07", - "KEY_D529E3DC0FE0": "VAL_61143D9165F7", - "KEY_64E68AED524A": "VAL_BAEAEEE993B9" - }, - "placements": [ - { - "index": 0, - "target_percent": 10, - "record_start_token": 13107, - "record_tokens": 31, - "key": "KEY_366F554BD828", - "value": "VAL_855DEFE56D16" - }, - { - "index": 1, - "target_percent": 35, - "record_start_token": 45875, - "record_tokens": 31, - "key": "KEY_5F7945F5DDDA", - "value": "VAL_4A05F879BF07" - }, - { - "index": 2, - "target_percent": 65, - "record_start_token": 85196, - "record_tokens": 31, - "key": "KEY_D529E3DC0FE0", - "value": "VAL_61143D9165F7" - }, - { - "index": 3, - "target_percent": 90, - "record_start_token": 117964, - "record_tokens": 28, - "key": "KEY_64E68AED524A", - "value": "VAL_BAEAEEE993B9" - } - ], - "max_output_tokens": 2048, - "ignore_eos": false, - "temperature": 0, - "thinking": false - }, - { - "id": "retrieval-n131072-s2", - "input_tokens": 131072, - "input_ids_sha256": "1423ffcb445ee70f42e33fe01a07f900de134bd91e8e949046e59cb5021dc9f3", - "expected": { - "KEY_370C3E69A270": "VAL_441B26467203", - "KEY_C2C6302C0229": "VAL_763264FCA241", - "KEY_9E4189450A04": "VAL_1506A6EBF88B", - "KEY_30CF378A481A": "VAL_7247945426B5" - }, - "placements": [ - { - "index": 0, - "target_percent": 10, - "record_start_token": 13107, - "record_tokens": 34, - "key": "KEY_370C3E69A270", - "value": "VAL_441B26467203" - }, - { - "index": 1, - "target_percent": 35, - "record_start_token": 45875, - "record_tokens": 32, - "key": "KEY_C2C6302C0229", - "value": "VAL_763264FCA241" - }, - { - "index": 2, - "target_percent": 65, - "record_start_token": 85196, - "record_tokens": 33, - "key": "KEY_9E4189450A04", - "value": "VAL_1506A6EBF88B" - }, - { - "index": 3, - "target_percent": 90, - "record_start_token": 117964, - "record_tokens": 33, - "key": "KEY_30CF378A481A", - "value": "VAL_7247945426B5" - } - ], - "max_output_tokens": 2048, - "ignore_eos": false, - "temperature": 0, - "thinking": false - }, - { - "id": "retrieval-n131072-s3", - "input_tokens": 131072, - "input_ids_sha256": "ecb6f2ad85c4a0ea74ddcc17b976e781c14aa53242a22e09cf1c5a75fd3a21eb", - "expected": { - "KEY_270979768147": "VAL_3F640637A148", - "KEY_AD1CF330D2EF": "VAL_EB3BA9D1305E", - "KEY_41E18DFF1FA8": "VAL_0D3CB49A7466", - "KEY_473AE1872C7A": "VAL_01EF80777BC1" - }, - "placements": [ - { - "index": 0, - "target_percent": 10, - "record_start_token": 13107, - "record_tokens": 34, - "key": "KEY_270979768147", - "value": "VAL_3F640637A148" - }, - { - "index": 1, - "target_percent": 35, - "record_start_token": 45875, - "record_tokens": 28, - "key": "KEY_AD1CF330D2EF", - "value": "VAL_EB3BA9D1305E" - }, - { - "index": 2, - "target_percent": 65, - "record_start_token": 85196, - "record_tokens": 31, - "key": "KEY_41E18DFF1FA8", - "value": "VAL_0D3CB49A7466" - }, - { - "index": 3, - "target_percent": 90, - "record_start_token": 117964, - "record_tokens": 31, - "key": "KEY_473AE1872C7A", - "value": "VAL_01EF80777BC1" - } - ], - "max_output_tokens": 2048, - "ignore_eos": false, - "temperature": 0, - "thinking": false - }, - { - "id": "retrieval-n131072-s4", - "input_tokens": 131072, - "input_ids_sha256": "e8561332bff2595b9f473427c761afde90a6acf70c92de5fd757c7de717eaa72", - "expected": { - "KEY_D265428D86BD": "VAL_6B595EB4A1D1", - "KEY_A6C7E29974D8": "VAL_778D462EC6A7", - "KEY_7FE9958C0C0B": "VAL_F4C0F943B0A8", - "KEY_137375882964": "VAL_D041C393CB7F" - }, - "placements": [ - { - "index": 0, - "target_percent": 10, - "record_start_token": 13107, - "record_tokens": 31, - "key": "KEY_D265428D86BD", - "value": "VAL_6B595EB4A1D1" - }, - { - "index": 1, - "target_percent": 35, - "record_start_token": 45875, - "record_tokens": 32, - "key": "KEY_A6C7E29974D8", - "value": "VAL_778D462EC6A7" - }, - { - "index": 2, - "target_percent": 65, - "record_start_token": 85196, - "record_tokens": 32, - "key": "KEY_7FE9958C0C0B", - "value": "VAL_F4C0F943B0A8" - }, - { - "index": 3, - "target_percent": 90, - "record_start_token": 117964, - "record_tokens": 32, - "key": "KEY_137375882964", - "value": "VAL_D041C393CB7F" - } - ], - "max_output_tokens": 2048, - "ignore_eos": false, - "temperature": 0, - "thinking": false - }, - { - "id": "retrieval-n131072-s5", - "input_tokens": 131072, - "input_ids_sha256": "542a8863b7ea8ad346eff5e7dc3f7bd88f5e28f3927f3b10b3a7e83b7caefe1e", - "expected": { - "KEY_3E19BF9BD574": "VAL_347F0779911E", - "KEY_4F25061922B5": "VAL_374EC4E8F9FD", - "KEY_153FA8B5F454": "VAL_910BE09D7C29", - "KEY_79A1FC632493": "VAL_4BAB5538CE1D" - }, - "placements": [ - { - "index": 0, - "target_percent": 10, - "record_start_token": 13107, - "record_tokens": 32, - "key": "KEY_3E19BF9BD574", - "value": "VAL_347F0779911E" - }, - { - "index": 1, - "target_percent": 35, - "record_start_token": 45875, - "record_tokens": 32, - "key": "KEY_4F25061922B5", - "value": "VAL_374EC4E8F9FD" - }, - { - "index": 2, - "target_percent": 65, - "record_start_token": 85196, - "record_tokens": 32, - "key": "KEY_153FA8B5F454", - "value": "VAL_910BE09D7C29" - }, - { - "index": 3, - "target_percent": 90, - "record_start_token": 117964, - "record_tokens": 30, - "key": "KEY_79A1FC632493", - "value": "VAL_4BAB5538CE1D" - } - ], - "max_output_tokens": 2048, - "ignore_eos": false, - "temperature": 0, - "thinking": false - }, - { - "id": "retrieval-n131072-s6", - "input_tokens": 131072, - "input_ids_sha256": "b90f4117796b73c4f04e07c50872d174924160fd4b63b0d4a4433eb3a95af079", - "expected": { - "KEY_6E4545E96448": "VAL_1E2BFE6C3C8F", - "KEY_7D815A31D62B": "VAL_C34972128FCE", - "KEY_05903353F314": "VAL_A7030CFD04A1", - "KEY_90C8AB6D04F1": "VAL_122D122B0115" - }, - "placements": [ - { - "index": 0, - "target_percent": 10, - "record_start_token": 13107, - "record_tokens": 33, - "key": "KEY_6E4545E96448", - "value": "VAL_1E2BFE6C3C8F" - }, - { - "index": 1, - "target_percent": 35, - "record_start_token": 45875, - "record_tokens": 32, - "key": "KEY_7D815A31D62B", - "value": "VAL_C34972128FCE" - }, - { - "index": 2, - "target_percent": 65, - "record_start_token": 85196, - "record_tokens": 32, - "key": "KEY_05903353F314", - "value": "VAL_A7030CFD04A1" - }, - { - "index": 3, - "target_percent": 90, - "record_start_token": 117964, - "record_tokens": 33, - "key": "KEY_90C8AB6D04F1", - "value": "VAL_122D122B0115" - } - ], - "max_output_tokens": 2048, - "ignore_eos": false, - "temperature": 0, - "thinking": false - }, - { - "id": "retrieval-n131072-s7", - "input_tokens": 131072, - "input_ids_sha256": "66ad050daa93438d28031e590dcc89ebd87210fb7400eda8b4c3328c386fd3bf", - "expected": { - "KEY_054CCD110AFB": "VAL_9697F65CA4C4", - "KEY_585E90552DE2": "VAL_BB59CCE01603", - "KEY_BEF278FA5857": "VAL_2A920529B120", - "KEY_B30DF53E8537": "VAL_971C2C2E4A90" - }, - "placements": [ - { - "index": 0, - "target_percent": 10, - "record_start_token": 13107, - "record_tokens": 31, - "key": "KEY_054CCD110AFB", - "value": "VAL_9697F65CA4C4" - }, - { - "index": 1, - "target_percent": 35, - "record_start_token": 45875, - "record_tokens": 30, - "key": "KEY_585E90552DE2", - "value": "VAL_BB59CCE01603" - }, - { - "index": 2, - "target_percent": 65, - "record_start_token": 85196, - "record_tokens": 31, - "key": "KEY_BEF278FA5857", - "value": "VAL_2A920529B120" - }, - { - "index": 3, - "target_percent": 90, - "record_start_token": 117964, - "record_tokens": 32, - "key": "KEY_B30DF53E8537", - "value": "VAL_971C2C2E4A90" - } - ], - "max_output_tokens": 2048, - "ignore_eos": false, - "temperature": 0, - "thinking": false - }, - { - "id": "retrieval-n131072-s8", - "input_tokens": 131072, - "input_ids_sha256": "3b8c05ea04015e74c20d3c92dd97ab29226e7be1896195357dfd40789dc08bd0", - "expected": { - "KEY_1FD01B4361C6": "VAL_BFD602A5485F", - "KEY_576049A5872E": "VAL_927C9E8442E4", - "KEY_1FAEB3FF131B": "VAL_62F6E0422C83", - "KEY_B1E29D911821": "VAL_A255678658E3" - }, - "placements": [ - { - "index": 0, - "target_percent": 10, - "record_start_token": 13107, - "record_tokens": 31, - "key": "KEY_1FD01B4361C6", - "value": "VAL_BFD602A5485F" - }, - { - "index": 1, - "target_percent": 35, - "record_start_token": 45875, - "record_tokens": 34, - "key": "KEY_576049A5872E", - "value": "VAL_927C9E8442E4" - }, - { - "index": 2, - "target_percent": 65, - "record_start_token": 85196, - "record_tokens": 31, - "key": "KEY_1FAEB3FF131B", - "value": "VAL_62F6E0422C83" - }, - { - "index": 3, - "target_percent": 90, - "record_start_token": 117964, - "record_tokens": 32, - "key": "KEY_B1E29D911821", - "value": "VAL_A255678658E3" - } - ], - "max_output_tokens": 2048, - "ignore_eos": false, - "temperature": 0, - "thinking": false - }, - { - "id": "retrieval-n131072-s9", - "input_tokens": 131072, - "input_ids_sha256": "891f89f0bd4b1f029e2c0fa5eca027f818d1ab8bf311d5b70870c011b1dd5966", - "expected": { - "KEY_478F5C1E2F73": "VAL_DD268DF2C2DA", - "KEY_BC7BEEDB6B4A": "VAL_FFA37CEFA91D", - "KEY_463B41944939": "VAL_1EFD6A2DD4F2", - "KEY_338C48A7990B": "VAL_63D4394AF760" - }, - "placements": [ - { - "index": 0, - "target_percent": 10, - "record_start_token": 13107, - "record_tokens": 30, - "key": "KEY_478F5C1E2F73", - "value": "VAL_DD268DF2C2DA" - }, - { - "index": 1, - "target_percent": 35, - "record_start_token": 45875, - "record_tokens": 26, - "key": "KEY_BC7BEEDB6B4A", - "value": "VAL_FFA37CEFA91D" - }, - { - "index": 2, - "target_percent": 65, - "record_start_token": 85196, - "record_tokens": 32, - "key": "KEY_463B41944939", - "value": "VAL_1EFD6A2DD4F2" - }, - { - "index": 3, - "target_percent": 90, - "record_start_token": 117964, - "record_tokens": 33, - "key": "KEY_338C48A7990B", - "value": "VAL_63D4394AF760" - } - ], - "max_output_tokens": 2048, - "ignore_eos": false, - "temperature": 0, - "thinking": false - } - ], - "generator_sha256": "0571ed279b9ef750654b2bc077e70a61d722506e05804c1c632425e57c19d38b", - "inputs_sha256": "285ddc88444b082236aa11cc0469f9112b93225c660ed9cdc4a079316ca35db1", - "compressed_sha256": "72c21dcd655df6851451dd4e2f3e6f0b056d777494fe2723219a83dbcc90a009", - "tokenizer_files": { - "chat_template.jinja": "c3cf9e34abf4f9e36c2d72165aa9c132d3e2a725b6c2586aaa3a8af9d7a81041", - "config.json": "74227dd615bf1ea975aa676bdf355a0379858c12f394b5365cd9dfa5fc2c70bc", - "tokenizer.json": "0997f410c57a1f4e53b09e4be8f4a172d90edd9564368fb0847030937229b9f3", - "tokenizer_config.json": "b11349aafa7cdc6a320767cf7ceb29ed82f7eda5d65e8e0819e76f0ce947bf27" - }, - "transformers": "5.19.0", - "extraction": "Strip whitespace; optionally unwrap one enclosing json code fence; json.loads entire remaining response; require exactly the four expected keys, all string values. Duplicate JSON keys are rejected. Exact string equality; no case folding or substring matching.", - "gate_status": "baseline qualification and owner approval pending" -} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/plan/retrieval-proposal-v2/README.md b/evidence/qwen38-h20-fp8-single-request-vllm032/plan/retrieval-proposal-v2/README.md deleted file mode 100644 index b173b7762..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/plan/retrieval-proposal-v2/README.md +++ /dev/null @@ -1,52 +0,0 @@ -# Revised retrieval proposal — unactivated, no model answers - -Version1 is retained in the sibling directory and is not selected for freezing. -Independent reviewer review_fp8_graph found two shortcuts: decoys had different -identifier prefixes and an extra sentence;128 decoy records were repeated. -Version2 uses the same record template and identifier lengths for targets and -decoys, with unique deterministic decoy identities rather than a repeating pool. -All changes precede model answers; this is design review, not result-driven tuning. - -CPU1965 prepared30 cases totaling2,048,000 tokens. Exact sizes and absolute -record start positions remain8192/65536/131072 ×10 and10/35/65/90 percent. The -same authenticated tokenizer and thinking-disabled chat template are used; -manifest binds the generator, inputs and tokenizer files. Target key/value -occurrence and needle-slice decode assertions all passed. Local compressed -readback independently authenticated all records, hashes, sizes and manifest -correspondence. Fixed positions, synthetic content and token-splice boundaries -still constrain generalization; this is not a universal long-context benchmark. - -Extraction now distinguishes independent per-position matches from whole-object -validity. Complete-case success requires an exact four-key JSON object with -string values matching all expected values. Each position may match an available -string value even when another key is missing or an extra key occurs. Invalid -JSON or duplicate keys score all positions false. One enclosing lowercase `json` -fence is permitted. No normalization or substring matching. Eight focused local -cases check those rules. Version1's globally schema-conditioned scores are not -used for this proposal; neither version has been evaluated on model answers. - -The prepared v2 observer accepts supported multi-token SSE deltas, validates -choice-event ITLs separately from usage-token counts, and requires one final -DONE. The pinned official client remains unchanged. Its source is retained but -has not run on a GPU. A separate prospective workload/cost declaration and -independent review precede baseline execution. The proposed9/10 qualification -and candidate nonregression gates still require owner confirmation. - -Publication layout: the generator output originally named `manifest.json` is -retained byte-for-byte as `metadata.json`. The existing bundle-sha256 schema -classifies and authenticates this metadata, compressed inputs and observer -sources through `plan/retrieval-integrity.json`; no validator schema or gate is -weakened. Original remote artifacts retain their generated names. - -Independent review of v2 found no blocking issue. A v3 observer successor makes -the intended CPU policy enforceable: assert no OMP_NUM_THREADS override before -launch, record the actual environment value, and let vLLM apply its default. -It also requests30 seconds for vLLM shutdown and allows45 seconds before observer -escalation. Wrapper unsets OMP explicitly; scorer/inputs/client/kernels unchanged. -All earlier observers remain immutable and unexecuted retrieval prototypes are -labelled as such. The added source snapshot increases inventory coverage to121. -Five original1961 raw streams satisfy the prospective v2/v3 validity expression, -including coalesced choices; five synthetic mutations (missing DONE, error event, -missing usage, wrong prompt and timing count) are rejected. That CPU check does -not change the original1961 job verdict. Eight local scorer boundary cases and -ten independent reviewer scorer cases passed. No retrieval GPU result yet. diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/plan/retrieval-proposal-v2/inputs.jsonl.gz b/evidence/qwen38-h20-fp8-single-request-vllm032/plan/retrieval-proposal-v2/inputs.jsonl.gz deleted file mode 100644 index c6626f0b5..000000000 Binary files a/evidence/qwen38-h20-fp8-single-request-vllm032/plan/retrieval-proposal-v2/inputs.jsonl.gz and /dev/null differ diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/plan/retrieval-proposal-v2/metadata.json b/evidence/qwen38-h20-fp8-single-request-vllm032/plan/retrieval-proposal-v2/metadata.json deleted file mode 100644 index 6ec93e806..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/plan/retrieval-proposal-v2/metadata.json +++ /dev/null @@ -1,1488 +0,0 @@ -{ - "schema": "retrieval-proposal-v2", - "status": "PROPOSED_NOT_ACTIVATED", - "cases": [ - { - "id": "retrieval-n8192-s0", - "input_tokens": 8192, - "input_ids_sha256": "60fe39746eab8f146ed905b2ddbc20859a8391b624db52d7dc50b3db3cd23bd4", - "expected": { - "KEY_48D0482FB109": "VAL_00C19710A484", - "KEY_9625E218DE62": "VAL_D10F531B5345", - "KEY_B4D07F699C59": "VAL_3D11490EF091", - "KEY_84088D6AA1FE": "VAL_148ABC144720" - }, - "placements": [ - { - "index": 0, - "target_percent": 10, - "record_start_token": 819, - "record_tokens": 33, - "key": "KEY_48D0482FB109", - "value": "VAL_00C19710A484" - }, - { - "index": 1, - "target_percent": 35, - "record_start_token": 2867, - "record_tokens": 32, - "key": "KEY_9625E218DE62", - "value": "VAL_D10F531B5345" - }, - { - "index": 2, - "target_percent": 65, - "record_start_token": 5324, - "record_tokens": 32, - "key": "KEY_B4D07F699C59", - "value": "VAL_3D11490EF091" - }, - { - "index": 3, - "target_percent": 90, - "record_start_token": 7372, - "record_tokens": 30, - "key": "KEY_84088D6AA1FE", - "value": "VAL_148ABC144720" - } - ], - "max_output_tokens": 2048, - "ignore_eos": false, - "temperature": 0, - "thinking": false - }, - { - "id": "retrieval-n8192-s1", - "input_tokens": 8192, - "input_ids_sha256": "156ae570678e4e71a5055a672f35a15db554e550b19fb416068ab05803bde489", - "expected": { - "KEY_658357C7B44B": "VAL_0A075A1112A6", - "KEY_BF4A1A24CE68": "VAL_B3E0DCAC2698", - "KEY_B2D170FC8F40": "VAL_3B54EC2AD5EC", - "KEY_ACCA18675051": "VAL_F62C097883A9" - }, - "placements": [ - { - "index": 0, - "target_percent": 10, - "record_start_token": 819, - "record_tokens": 34, - "key": "KEY_658357C7B44B", - "value": "VAL_0A075A1112A6" - }, - { - "index": 1, - "target_percent": 35, - "record_start_token": 2867, - "record_tokens": 29, - "key": "KEY_BF4A1A24CE68", - "value": "VAL_B3E0DCAC2698" - }, - { - "index": 2, - "target_percent": 65, - "record_start_token": 5324, - "record_tokens": 29, - "key": "KEY_B2D170FC8F40", - "value": "VAL_3B54EC2AD5EC" - }, - { - "index": 3, - "target_percent": 90, - "record_start_token": 7372, - "record_tokens": 30, - "key": "KEY_ACCA18675051", - "value": "VAL_F62C097883A9" - } - ], - "max_output_tokens": 2048, - "ignore_eos": false, - "temperature": 0, - "thinking": false - }, - { - "id": "retrieval-n8192-s2", - "input_tokens": 8192, - "input_ids_sha256": "ed9b66e09d9662812ccb8fa78a107de066434db19033553a75d5b5d69ce4d4c6", - "expected": { - "KEY_8CACAB1A5C8D": "VAL_6036D54D7373", - "KEY_7EB1C16A83A4": "VAL_BE12EF59BE37", - "KEY_5EAE4B0C7C6E": "VAL_23C6CDC883BA", - "KEY_09A2CE135CD8": "VAL_80F4CF7086F9" - }, - "placements": [ - { - "index": 0, - "target_percent": 10, - "record_start_token": 819, - "record_tokens": 31, - "key": "KEY_8CACAB1A5C8D", - "value": "VAL_6036D54D7373" - }, - { - "index": 1, - "target_percent": 35, - "record_start_token": 2867, - "record_tokens": 29, - "key": "KEY_7EB1C16A83A4", - "value": "VAL_BE12EF59BE37" - }, - { - "index": 2, - "target_percent": 65, - "record_start_token": 5324, - "record_tokens": 30, - "key": "KEY_5EAE4B0C7C6E", - "value": "VAL_23C6CDC883BA" - }, - { - "index": 3, - "target_percent": 90, - "record_start_token": 7372, - "record_tokens": 31, - "key": "KEY_09A2CE135CD8", - "value": "VAL_80F4CF7086F9" - } - ], - "max_output_tokens": 2048, - "ignore_eos": false, - "temperature": 0, - "thinking": false - }, - { - "id": "retrieval-n8192-s3", - "input_tokens": 8192, - "input_ids_sha256": "eea5317ddb724bf5bf328681bb42fec29b5bd2786eb20ba4d8ff2aa0d3a594a7", - "expected": { - "KEY_7C17A7C224BD": "VAL_975AF3765BE1", - "KEY_9A0B8D3D5847": "VAL_E1384C5C0A7C", - "KEY_2919BD59FEDC": "VAL_1A6A4BBC3A34", - "KEY_C4CCAE40EC9B": "VAL_F4E762A0BCE7" - }, - "placements": [ - { - "index": 0, - "target_percent": 10, - "record_start_token": 819, - "record_tokens": 31, - "key": "KEY_7C17A7C224BD", - "value": "VAL_975AF3765BE1" - }, - { - "index": 1, - "target_percent": 35, - "record_start_token": 2867, - "record_tokens": 33, - "key": "KEY_9A0B8D3D5847", - "value": "VAL_E1384C5C0A7C" - }, - { - "index": 2, - "target_percent": 65, - "record_start_token": 5324, - "record_tokens": 30, - "key": "KEY_2919BD59FEDC", - "value": "VAL_1A6A4BBC3A34" - }, - { - "index": 3, - "target_percent": 90, - "record_start_token": 7372, - "record_tokens": 28, - "key": "KEY_C4CCAE40EC9B", - "value": "VAL_F4E762A0BCE7" - } - ], - "max_output_tokens": 2048, - "ignore_eos": false, - "temperature": 0, - "thinking": false - }, - { - "id": "retrieval-n8192-s4", - "input_tokens": 8192, - "input_ids_sha256": "8da5a29ea13e5c920810fda65352ee839f2ab752bf31e0581e9689da055a0388", - "expected": { - "KEY_34E16D590568": "VAL_4DF45120C441", - "KEY_D844BF781F00": "VAL_0E815DD3A3FE", - "KEY_34F0672C565F": "VAL_A5480F57AC37", - "KEY_7CA1CBFFCFE2": "VAL_83AF30750C36" - }, - "placements": [ - { - "index": 0, - "target_percent": 10, - "record_start_token": 819, - "record_tokens": 33, - "key": "KEY_34E16D590568", - "value": "VAL_4DF45120C441" - }, - { - "index": 1, - "target_percent": 35, - "record_start_token": 2867, - "record_tokens": 30, - "key": "KEY_D844BF781F00", - "value": "VAL_0E815DD3A3FE" - }, - { - "index": 2, - "target_percent": 65, - "record_start_token": 5324, - "record_tokens": 32, - "key": "KEY_34F0672C565F", - "value": "VAL_A5480F57AC37" - }, - { - "index": 3, - "target_percent": 90, - "record_start_token": 7372, - "record_tokens": 29, - "key": "KEY_7CA1CBFFCFE2", - "value": "VAL_83AF30750C36" - } - ], - "max_output_tokens": 2048, - "ignore_eos": false, - "temperature": 0, - "thinking": false - }, - { - "id": "retrieval-n8192-s5", - "input_tokens": 8192, - "input_ids_sha256": "bb4e838888511cd92415080f6ddb13442d96a5db48b6c02e54ae851d8b546494", - "expected": { - "KEY_C957D9C07257": "VAL_2A575856A1BE", - "KEY_143ECD9B470B": "VAL_F5C7F636E406", - "KEY_9E1A8D57F08A": "VAL_3B35C77D0E43", - "KEY_837DD3E1EA3D": "VAL_24F0FCE9E515" - }, - "placements": [ - { - "index": 0, - "target_percent": 10, - "record_start_token": 819, - "record_tokens": 32, - "key": "KEY_C957D9C07257", - "value": "VAL_2A575856A1BE" - }, - { - "index": 1, - "target_percent": 35, - "record_start_token": 2867, - "record_tokens": 31, - "key": "KEY_143ECD9B470B", - "value": "VAL_F5C7F636E406" - }, - { - "index": 2, - "target_percent": 65, - "record_start_token": 5324, - "record_tokens": 34, - "key": "KEY_9E1A8D57F08A", - "value": "VAL_3B35C77D0E43" - }, - { - "index": 3, - "target_percent": 90, - "record_start_token": 7372, - "record_tokens": 31, - "key": "KEY_837DD3E1EA3D", - "value": "VAL_24F0FCE9E515" - } - ], - "max_output_tokens": 2048, - "ignore_eos": false, - "temperature": 0, - "thinking": false - }, - { - "id": "retrieval-n8192-s6", - "input_tokens": 8192, - "input_ids_sha256": "80f9519dcff063b8355074607681b3416a754255da972b39578c0733760b6a43", - "expected": { - "KEY_51C062623F76": "VAL_00C8B1F0EE3B", - "KEY_5860EDCDBA1F": "VAL_D0AC821B1C73", - "KEY_D1F424F3850C": "VAL_EBE14AF8B77A", - "KEY_A757B77E7E51": "VAL_94C0C64D7904" - }, - "placements": [ - { - "index": 0, - "target_percent": 10, - "record_start_token": 819, - "record_tokens": 33, - "key": "KEY_51C062623F76", - "value": "VAL_00C8B1F0EE3B" - }, - { - "index": 1, - "target_percent": 35, - "record_start_token": 2867, - "record_tokens": 30, - "key": "KEY_5860EDCDBA1F", - "value": "VAL_D0AC821B1C73" - }, - { - "index": 2, - "target_percent": 65, - "record_start_token": 5324, - "record_tokens": 30, - "key": "KEY_D1F424F3850C", - "value": "VAL_EBE14AF8B77A" - }, - { - "index": 3, - "target_percent": 90, - "record_start_token": 7372, - "record_tokens": 33, - "key": "KEY_A757B77E7E51", - "value": "VAL_94C0C64D7904" - } - ], - "max_output_tokens": 2048, - "ignore_eos": false, - "temperature": 0, - "thinking": false - }, - { - "id": "retrieval-n8192-s7", - "input_tokens": 8192, - "input_ids_sha256": "5fd80d5d781dd370d7e37987abfca39be15931393f93a04c2ee90f8ed52d1e6c", - "expected": { - "KEY_C3830AF22CA3": "VAL_F96D78A77312", - "KEY_7AF82659EEDD": "VAL_CE1C0DA20FF8", - "KEY_5E3C3F3158D5": "VAL_2CC4F1C5B0E4", - "KEY_5A99CD921008": "VAL_61DA90A62470" - }, - "placements": [ - { - "index": 0, - "target_percent": 10, - "record_start_token": 819, - "record_tokens": 30, - "key": "KEY_C3830AF22CA3", - "value": "VAL_F96D78A77312" - }, - { - "index": 1, - "target_percent": 35, - "record_start_token": 2867, - "record_tokens": 28, - "key": "KEY_7AF82659EEDD", - "value": "VAL_CE1C0DA20FF8" - }, - { - "index": 2, - "target_percent": 65, - "record_start_token": 5324, - "record_tokens": 33, - "key": "KEY_5E3C3F3158D5", - "value": "VAL_2CC4F1C5B0E4" - }, - { - "index": 3, - "target_percent": 90, - "record_start_token": 7372, - "record_tokens": 32, - "key": "KEY_5A99CD921008", - "value": "VAL_61DA90A62470" - } - ], - "max_output_tokens": 2048, - "ignore_eos": false, - "temperature": 0, - "thinking": false - }, - { - "id": "retrieval-n8192-s8", - "input_tokens": 8192, - "input_ids_sha256": "469f4f91864269692ff74b5ea131b5be48dd3b0ffda60350af92516986a6fb00", - "expected": { - "KEY_EF7F95C8C310": "VAL_5DE51EB3B32C", - "KEY_68CC3411901D": "VAL_2273CB504F70", - "KEY_9D1AA5AF4ADF": "VAL_8059F047F934", - "KEY_E0AA0BBC9D57": "VAL_E1A082136237" - }, - "placements": [ - { - "index": 0, - "target_percent": 10, - "record_start_token": 819, - "record_tokens": 31, - "key": "KEY_EF7F95C8C310", - "value": "VAL_5DE51EB3B32C" - }, - { - "index": 1, - "target_percent": 35, - "record_start_token": 2867, - "record_tokens": 32, - "key": "KEY_68CC3411901D", - "value": "VAL_2273CB504F70" - }, - { - "index": 2, - "target_percent": 65, - "record_start_token": 5324, - "record_tokens": 30, - "key": "KEY_9D1AA5AF4ADF", - "value": "VAL_8059F047F934" - }, - { - "index": 3, - "target_percent": 90, - "record_start_token": 7372, - "record_tokens": 29, - "key": "KEY_E0AA0BBC9D57", - "value": "VAL_E1A082136237" - } - ], - "max_output_tokens": 2048, - "ignore_eos": false, - "temperature": 0, - "thinking": false - }, - { - "id": "retrieval-n8192-s9", - "input_tokens": 8192, - "input_ids_sha256": "38a57672595c4fae09b49289772cc49b26cc781a547c63a2c0963882ce9b5219", - "expected": { - "KEY_EF1EBFE0AAB8": "VAL_A57016B6636B", - "KEY_8CD68D503195": "VAL_D13978547AC6", - "KEY_B0BF1CE73D7E": "VAL_2E936A05D1A9", - "KEY_260766402CE1": "VAL_7350F5E3B561" - }, - "placements": [ - { - "index": 0, - "target_percent": 10, - "record_start_token": 819, - "record_tokens": 29, - "key": "KEY_EF1EBFE0AAB8", - "value": "VAL_A57016B6636B" - }, - { - "index": 1, - "target_percent": 35, - "record_start_token": 2867, - "record_tokens": 31, - "key": "KEY_8CD68D503195", - "value": "VAL_D13978547AC6" - }, - { - "index": 2, - "target_percent": 65, - "record_start_token": 5324, - "record_tokens": 31, - "key": "KEY_B0BF1CE73D7E", - "value": "VAL_2E936A05D1A9" - }, - { - "index": 3, - "target_percent": 90, - "record_start_token": 7372, - "record_tokens": 33, - "key": "KEY_260766402CE1", - "value": "VAL_7350F5E3B561" - } - ], - "max_output_tokens": 2048, - "ignore_eos": false, - "temperature": 0, - "thinking": false - }, - { - "id": "retrieval-n65536-s0", - "input_tokens": 65536, - "input_ids_sha256": "07c810fee031cc4459911748c565743ead76dd9806b95474f040ea749df2355d", - "expected": { - "KEY_F199ADFDE5EC": "VAL_4DC9E697A944", - "KEY_145B316B246A": "VAL_054F40195969", - "KEY_D504F8218514": "VAL_BD2095C7EF7C", - "KEY_1FFEA3476750": "VAL_CD7AB7F38418" - }, - "placements": [ - { - "index": 0, - "target_percent": 10, - "record_start_token": 6553, - "record_tokens": 28, - "key": "KEY_F199ADFDE5EC", - "value": "VAL_4DC9E697A944" - }, - { - "index": 1, - "target_percent": 35, - "record_start_token": 22937, - "record_tokens": 34, - "key": "KEY_145B316B246A", - "value": "VAL_054F40195969" - }, - { - "index": 2, - "target_percent": 65, - "record_start_token": 42598, - "record_tokens": 31, - "key": "KEY_D504F8218514", - "value": "VAL_BD2095C7EF7C" - }, - { - "index": 3, - "target_percent": 90, - "record_start_token": 58982, - "record_tokens": 29, - "key": "KEY_1FFEA3476750", - "value": "VAL_CD7AB7F38418" - } - ], - "max_output_tokens": 2048, - "ignore_eos": false, - "temperature": 0, - "thinking": false - }, - { - "id": "retrieval-n65536-s1", - "input_tokens": 65536, - "input_ids_sha256": "2c4b26986ba9b73f2e72c243658fec02916c53f3710634bd8476b72bd8650510", - "expected": { - "KEY_D149BAB85691": "VAL_AA85D60A28A8", - "KEY_EEF286EF5EDC": "VAL_34DEFF8B16EC", - "KEY_411578C5828D": "VAL_128D4D609ACA", - "KEY_67DE09AC6DCE": "VAL_AECE17ACD968" - }, - "placements": [ - { - "index": 0, - "target_percent": 10, - "record_start_token": 6553, - "record_tokens": 29, - "key": "KEY_D149BAB85691", - "value": "VAL_AA85D60A28A8" - }, - { - "index": 1, - "target_percent": 35, - "record_start_token": 22937, - "record_tokens": 27, - "key": "KEY_EEF286EF5EDC", - "value": "VAL_34DEFF8B16EC" - }, - { - "index": 2, - "target_percent": 65, - "record_start_token": 42598, - "record_tokens": 32, - "key": "KEY_411578C5828D", - "value": "VAL_128D4D609ACA" - }, - { - "index": 3, - "target_percent": 90, - "record_start_token": 58982, - "record_tokens": 27, - "key": "KEY_67DE09AC6DCE", - "value": "VAL_AECE17ACD968" - } - ], - "max_output_tokens": 2048, - "ignore_eos": false, - "temperature": 0, - "thinking": false - }, - { - "id": "retrieval-n65536-s2", - "input_tokens": 65536, - "input_ids_sha256": "118b5941352be4021847354a0d199d9def3f40c098152ef973ca1b08b2684c39", - "expected": { - "KEY_171981E46F20": "VAL_E0A822C2E874", - "KEY_0AE521510615": "VAL_94709834762E", - "KEY_443C031EBE71": "VAL_1920A538E912", - "KEY_4559964CF82F": "VAL_E7BCCF21D6E4" - }, - "placements": [ - { - "index": 0, - "target_percent": 10, - "record_start_token": 6553, - "record_tokens": 33, - "key": "KEY_171981E46F20", - "value": "VAL_E0A822C2E874" - }, - { - "index": 1, - "target_percent": 35, - "record_start_token": 22937, - "record_tokens": 33, - "key": "KEY_0AE521510615", - "value": "VAL_94709834762E" - }, - { - "index": 2, - "target_percent": 65, - "record_start_token": 42598, - "record_tokens": 33, - "key": "KEY_443C031EBE71", - "value": "VAL_1920A538E912" - }, - { - "index": 3, - "target_percent": 90, - "record_start_token": 58982, - "record_tokens": 31, - "key": "KEY_4559964CF82F", - "value": "VAL_E7BCCF21D6E4" - } - ], - "max_output_tokens": 2048, - "ignore_eos": false, - "temperature": 0, - "thinking": false - }, - { - "id": "retrieval-n65536-s3", - "input_tokens": 65536, - "input_ids_sha256": "f417abfeb092edfa11cb9dede94635966095a68a8e7f139bc26aa8ae754a64ec", - "expected": { - "KEY_B2115793251B": "VAL_616A8972FFEF", - "KEY_32C0D8CD2496": "VAL_3E80F953CA33", - "KEY_7C45203F80D3": "VAL_D993F1B00109", - "KEY_EEEDBD9EDA93": "VAL_AB6D4F5362AC" - }, - "placements": [ - { - "index": 0, - "target_percent": 10, - "record_start_token": 6553, - "record_tokens": 31, - "key": "KEY_B2115793251B", - "value": "VAL_616A8972FFEF" - }, - { - "index": 1, - "target_percent": 35, - "record_start_token": 22937, - "record_tokens": 32, - "key": "KEY_32C0D8CD2496", - "value": "VAL_3E80F953CA33" - }, - { - "index": 2, - "target_percent": 65, - "record_start_token": 42598, - "record_tokens": 33, - "key": "KEY_7C45203F80D3", - "value": "VAL_D993F1B00109" - }, - { - "index": 3, - "target_percent": 90, - "record_start_token": 58982, - "record_tokens": 26, - "key": "KEY_EEEDBD9EDA93", - "value": "VAL_AB6D4F5362AC" - } - ], - "max_output_tokens": 2048, - "ignore_eos": false, - "temperature": 0, - "thinking": false - }, - { - "id": "retrieval-n65536-s4", - "input_tokens": 65536, - "input_ids_sha256": "f3509cb3bbd1f07fbcb4b05a2679e1fefe17966140296015aee2f5de5fb6b950", - "expected": { - "KEY_8699EF52D026": "VAL_84F827532C15", - "KEY_9C2DDEC0F196": "VAL_069ACF9EA29C", - "KEY_A4ACB8658F04": "VAL_7644D34AE0F9", - "KEY_5123C1DD6BE8": "VAL_81578A2A4507" - }, - "placements": [ - { - "index": 0, - "target_percent": 10, - "record_start_token": 6553, - "record_tokens": 33, - "key": "KEY_8699EF52D026", - "value": "VAL_84F827532C15" - }, - { - "index": 1, - "target_percent": 35, - "record_start_token": 22937, - "record_tokens": 30, - "key": "KEY_9C2DDEC0F196", - "value": "VAL_069ACF9EA29C" - }, - { - "index": 2, - "target_percent": 65, - "record_start_token": 42598, - "record_tokens": 31, - "key": "KEY_A4ACB8658F04", - "value": "VAL_7644D34AE0F9" - }, - { - "index": 3, - "target_percent": 90, - "record_start_token": 58982, - "record_tokens": 32, - "key": "KEY_5123C1DD6BE8", - "value": "VAL_81578A2A4507" - } - ], - "max_output_tokens": 2048, - "ignore_eos": false, - "temperature": 0, - "thinking": false - }, - { - "id": "retrieval-n65536-s5", - "input_tokens": 65536, - "input_ids_sha256": "ba761250776978d855008333188471bfd443702a288b1fc8cd0da22e64fec0bf", - "expected": { - "KEY_38FD70C43096": "VAL_EE1BDA4688E9", - "KEY_BF77F05BE6E4": "VAL_667B69AC1A10", - "KEY_0C61812A34A9": "VAL_9D82AE136DEE", - "KEY_D4DE64056B19": "VAL_8A6F6053080D" - }, - "placements": [ - { - "index": 0, - "target_percent": 10, - "record_start_token": 6553, - "record_tokens": 31, - "key": "KEY_38FD70C43096", - "value": "VAL_EE1BDA4688E9" - }, - { - "index": 1, - "target_percent": 35, - "record_start_token": 22937, - "record_tokens": 31, - "key": "KEY_BF77F05BE6E4", - "value": "VAL_667B69AC1A10" - }, - { - "index": 2, - "target_percent": 65, - "record_start_token": 42598, - "record_tokens": 32, - "key": "KEY_0C61812A34A9", - "value": "VAL_9D82AE136DEE" - }, - { - "index": 3, - "target_percent": 90, - "record_start_token": 58982, - "record_tokens": 32, - "key": "KEY_D4DE64056B19", - "value": "VAL_8A6F6053080D" - } - ], - "max_output_tokens": 2048, - "ignore_eos": false, - "temperature": 0, - "thinking": false - }, - { - "id": "retrieval-n65536-s6", - "input_tokens": 65536, - "input_ids_sha256": "8950b1276aba9ad1aa14a692d044312c0b7c0ca3e83bd2978e753f637be90e09", - "expected": { - "KEY_880F23B3AEB1": "VAL_19009431F219", - "KEY_D404B6DFC7DF": "VAL_F6238EB3EF83", - "KEY_39B55FA9E793": "VAL_B6C1FF528106", - "KEY_C0B0373FD4E3": "VAL_8F7050CDB233" - }, - "placements": [ - { - "index": 0, - "target_percent": 10, - "record_start_token": 6553, - "record_tokens": 33, - "key": "KEY_880F23B3AEB1", - "value": "VAL_19009431F219" - }, - { - "index": 1, - "target_percent": 35, - "record_start_token": 22937, - "record_tokens": 28, - "key": "KEY_D404B6DFC7DF", - "value": "VAL_F6238EB3EF83" - }, - { - "index": 2, - "target_percent": 65, - "record_start_token": 42598, - "record_tokens": 31, - "key": "KEY_39B55FA9E793", - "value": "VAL_B6C1FF528106" - }, - { - "index": 3, - "target_percent": 90, - "record_start_token": 58982, - "record_tokens": 31, - "key": "KEY_C0B0373FD4E3", - "value": "VAL_8F7050CDB233" - } - ], - "max_output_tokens": 2048, - "ignore_eos": false, - "temperature": 0, - "thinking": false - }, - { - "id": "retrieval-n65536-s7", - "input_tokens": 65536, - "input_ids_sha256": "7b773fc34023ccb43390f81df42f60f7b34ba4573618f1c8dd90677fdd9b1014", - "expected": { - "KEY_EF3AAA7B95B4": "VAL_459DC3229BBD", - "KEY_8EE66A56A687": "VAL_45B85352F16D", - "KEY_F9DB04A30085": "VAL_E6FCAA87F646", - "KEY_58D5FBD6608A": "VAL_49A2E1F7A24E" - }, - "placements": [ - { - "index": 0, - "target_percent": 10, - "record_start_token": 6553, - "record_tokens": 29, - "key": "KEY_EF3AAA7B95B4", - "value": "VAL_459DC3229BBD" - }, - { - "index": 1, - "target_percent": 35, - "record_start_token": 22937, - "record_tokens": 33, - "key": "KEY_8EE66A56A687", - "value": "VAL_45B85352F16D" - }, - { - "index": 2, - "target_percent": 65, - "record_start_token": 42598, - "record_tokens": 29, - "key": "KEY_F9DB04A30085", - "value": "VAL_E6FCAA87F646" - }, - { - "index": 3, - "target_percent": 90, - "record_start_token": 58982, - "record_tokens": 33, - "key": "KEY_58D5FBD6608A", - "value": "VAL_49A2E1F7A24E" - } - ], - "max_output_tokens": 2048, - "ignore_eos": false, - "temperature": 0, - "thinking": false - }, - { - "id": "retrieval-n65536-s8", - "input_tokens": 65536, - "input_ids_sha256": "79abc720269b4e05a2ebff85e0c12512f6d429e8f50144e52b25e5d377412680", - "expected": { - "KEY_FE3D0A329FBA": "VAL_12CF84845CD1", - "KEY_8CC64479CDE7": "VAL_1E65FA0E02C8", - "KEY_FB9F29729FA0": "VAL_33EBEE672177", - "KEY_EB4F30C29F54": "VAL_4A3D76474E4E" - }, - "placements": [ - { - "index": 0, - "target_percent": 10, - "record_start_token": 6553, - "record_tokens": 29, - "key": "KEY_FE3D0A329FBA", - "value": "VAL_12CF84845CD1" - }, - { - "index": 1, - "target_percent": 35, - "record_start_token": 22937, - "record_tokens": 31, - "key": "KEY_8CC64479CDE7", - "value": "VAL_1E65FA0E02C8" - }, - { - "index": 2, - "target_percent": 65, - "record_start_token": 42598, - "record_tokens": 29, - "key": "KEY_FB9F29729FA0", - "value": "VAL_33EBEE672177" - }, - { - "index": 3, - "target_percent": 90, - "record_start_token": 58982, - "record_tokens": 33, - "key": "KEY_EB4F30C29F54", - "value": "VAL_4A3D76474E4E" - } - ], - "max_output_tokens": 2048, - "ignore_eos": false, - "temperature": 0, - "thinking": false - }, - { - "id": "retrieval-n65536-s9", - "input_tokens": 65536, - "input_ids_sha256": "5800726390639e6440a58fa21f7d8d65d2b72241c2d385937ad7e8191b5fe803", - "expected": { - "KEY_033985BF7C13": "VAL_ACDDDD7004B6", - "KEY_AD798726BF07": "VAL_9F1D28803ABD", - "KEY_CA7212A0DA15": "VAL_C71860C5B090", - "KEY_90FCD15E8307": "VAL_D81DD45CCEAE" - }, - "placements": [ - { - "index": 0, - "target_percent": 10, - "record_start_token": 6553, - "record_tokens": 29, - "key": "KEY_033985BF7C13", - "value": "VAL_ACDDDD7004B6" - }, - { - "index": 1, - "target_percent": 35, - "record_start_token": 22937, - "record_tokens": 30, - "key": "KEY_AD798726BF07", - "value": "VAL_9F1D28803ABD" - }, - { - "index": 2, - "target_percent": 65, - "record_start_token": 42598, - "record_tokens": 30, - "key": "KEY_CA7212A0DA15", - "value": "VAL_C71860C5B090" - }, - { - "index": 3, - "target_percent": 90, - "record_start_token": 58982, - "record_tokens": 29, - "key": "KEY_90FCD15E8307", - "value": "VAL_D81DD45CCEAE" - } - ], - "max_output_tokens": 2048, - "ignore_eos": false, - "temperature": 0, - "thinking": false - }, - { - "id": "retrieval-n131072-s0", - "input_tokens": 131072, - "input_ids_sha256": "acb475f8203322978ebec844b5df1d226c92b88fa48efffb3d96b6f022c05dcd", - "expected": { - "KEY_1D7DE61CB245": "VAL_15E5B80A5BB9", - "KEY_3376BD5C4999": "VAL_6058CFFAB853", - "KEY_FB52141FEAA6": "VAL_A378E0DEB51A", - "KEY_3B931512F097": "VAL_6B7264F3B5F9" - }, - "placements": [ - { - "index": 0, - "target_percent": 10, - "record_start_token": 13107, - "record_tokens": 31, - "key": "KEY_1D7DE61CB245", - "value": "VAL_15E5B80A5BB9" - }, - { - "index": 1, - "target_percent": 35, - "record_start_token": 45875, - "record_tokens": 31, - "key": "KEY_3376BD5C4999", - "value": "VAL_6058CFFAB853" - }, - { - "index": 2, - "target_percent": 65, - "record_start_token": 85196, - "record_tokens": 28, - "key": "KEY_FB52141FEAA6", - "value": "VAL_A378E0DEB51A" - }, - { - "index": 3, - "target_percent": 90, - "record_start_token": 117964, - "record_tokens": 34, - "key": "KEY_3B931512F097", - "value": "VAL_6B7264F3B5F9" - } - ], - "max_output_tokens": 2048, - "ignore_eos": false, - "temperature": 0, - "thinking": false - }, - { - "id": "retrieval-n131072-s1", - "input_tokens": 131072, - "input_ids_sha256": "c263318cc3e3e829f3d9fffaa85971e9af87c666598805eb549a5b9746a83972", - "expected": { - "KEY_BA5A6B7C3728": "VAL_37FF2EB72750", - "KEY_62C6A258F632": "VAL_4DC8403FE68F", - "KEY_07C7FF8DBC99": "VAL_9F6931691352", - "KEY_6D13C79B4107": "VAL_4BAB1C50E88F" - }, - "placements": [ - { - "index": 0, - "target_percent": 10, - "record_start_token": 13107, - "record_tokens": 31, - "key": "KEY_BA5A6B7C3728", - "value": "VAL_37FF2EB72750" - }, - { - "index": 1, - "target_percent": 35, - "record_start_token": 45875, - "record_tokens": 32, - "key": "KEY_62C6A258F632", - "value": "VAL_4DC8403FE68F" - }, - { - "index": 2, - "target_percent": 65, - "record_start_token": 85196, - "record_tokens": 31, - "key": "KEY_07C7FF8DBC99", - "value": "VAL_9F6931691352" - }, - { - "index": 3, - "target_percent": 90, - "record_start_token": 117964, - "record_tokens": 32, - "key": "KEY_6D13C79B4107", - "value": "VAL_4BAB1C50E88F" - } - ], - "max_output_tokens": 2048, - "ignore_eos": false, - "temperature": 0, - "thinking": false - }, - { - "id": "retrieval-n131072-s2", - "input_tokens": 131072, - "input_ids_sha256": "b1b7f97f50856fabdbdd367a99a85660167a0b5482d0057f31e5bb9934f2453c", - "expected": { - "KEY_B0E1F87FB6D0": "VAL_7CAF6CF37C7F", - "KEY_44A1F0EABBD6": "VAL_D392021D4836", - "KEY_2B5041437D6B": "VAL_5767CA6BC6B2", - "KEY_8F0CFD23CB09": "VAL_B372C4F1AEF7" - }, - "placements": [ - { - "index": 0, - "target_percent": 10, - "record_start_token": 13107, - "record_tokens": 29, - "key": "KEY_B0E1F87FB6D0", - "value": "VAL_7CAF6CF37C7F" - }, - { - "index": 1, - "target_percent": 35, - "record_start_token": 45875, - "record_tokens": 31, - "key": "KEY_44A1F0EABBD6", - "value": "VAL_D392021D4836" - }, - { - "index": 2, - "target_percent": 65, - "record_start_token": 85196, - "record_tokens": 32, - "key": "KEY_2B5041437D6B", - "value": "VAL_5767CA6BC6B2" - }, - { - "index": 3, - "target_percent": 90, - "record_start_token": 117964, - "record_tokens": 30, - "key": "KEY_8F0CFD23CB09", - "value": "VAL_B372C4F1AEF7" - } - ], - "max_output_tokens": 2048, - "ignore_eos": false, - "temperature": 0, - "thinking": false - }, - { - "id": "retrieval-n131072-s3", - "input_tokens": 131072, - "input_ids_sha256": "fc4abf5851ec8135b6bb7e60852b3c1356256d1772c1073170dea0bab2036a6e", - "expected": { - "KEY_11ECFE992D7D": "VAL_67B0199E1814", - "KEY_46B8F2CAE751": "VAL_178F5396AD53", - "KEY_401CFE21BBD2": "VAL_2689382834E7", - "KEY_CBFC39ABE091": "VAL_37B51CFC6EB9" - }, - "placements": [ - { - "index": 0, - "target_percent": 10, - "record_start_token": 13107, - "record_tokens": 32, - "key": "KEY_11ECFE992D7D", - "value": "VAL_67B0199E1814" - }, - { - "index": 1, - "target_percent": 35, - "record_start_token": 45875, - "record_tokens": 32, - "key": "KEY_46B8F2CAE751", - "value": "VAL_178F5396AD53" - }, - { - "index": 2, - "target_percent": 65, - "record_start_token": 85196, - "record_tokens": 32, - "key": "KEY_401CFE21BBD2", - "value": "VAL_2689382834E7" - }, - { - "index": 3, - "target_percent": 90, - "record_start_token": 117964, - "record_tokens": 28, - "key": "KEY_CBFC39ABE091", - "value": "VAL_37B51CFC6EB9" - } - ], - "max_output_tokens": 2048, - "ignore_eos": false, - "temperature": 0, - "thinking": false - }, - { - "id": "retrieval-n131072-s4", - "input_tokens": 131072, - "input_ids_sha256": "b599eaa99b1155b1fc802901d8cf230a1e0d008f57a5516c76a26a61fd7d437a", - "expected": { - "KEY_FA7DF2E2511E": "VAL_D0A643F8B6F5", - "KEY_B171AACE1ED2": "VAL_6A6DCDE46388", - "KEY_2AD587AE30D4": "VAL_8BD555A16B79", - "KEY_5D6A51DBAC61": "VAL_2916C63FA7A0" - }, - "placements": [ - { - "index": 0, - "target_percent": 10, - "record_start_token": 13107, - "record_tokens": 30, - "key": "KEY_FA7DF2E2511E", - "value": "VAL_D0A643F8B6F5" - }, - { - "index": 1, - "target_percent": 35, - "record_start_token": 45875, - "record_tokens": 28, - "key": "KEY_B171AACE1ED2", - "value": "VAL_6A6DCDE46388" - }, - { - "index": 2, - "target_percent": 65, - "record_start_token": 85196, - "record_tokens": 31, - "key": "KEY_2AD587AE30D4", - "value": "VAL_8BD555A16B79" - }, - { - "index": 3, - "target_percent": 90, - "record_start_token": 117964, - "record_tokens": 31, - "key": "KEY_5D6A51DBAC61", - "value": "VAL_2916C63FA7A0" - } - ], - "max_output_tokens": 2048, - "ignore_eos": false, - "temperature": 0, - "thinking": false - }, - { - "id": "retrieval-n131072-s5", - "input_tokens": 131072, - "input_ids_sha256": "12892515f83a716b2880e733c8a2b3f8fccc6ae53b3eae6f49b55a37a6884cd7", - "expected": { - "KEY_B6E9C81511D5": "VAL_B7CC31A093A4", - "KEY_28DC460BC080": "VAL_3F9B063DB035", - "KEY_009AB49B10D8": "VAL_FA9AAA83C638", - "KEY_41259044966F": "VAL_5A7C37B697D1" - }, - "placements": [ - { - "index": 0, - "target_percent": 10, - "record_start_token": 13107, - "record_tokens": 31, - "key": "KEY_B6E9C81511D5", - "value": "VAL_B7CC31A093A4" - }, - { - "index": 1, - "target_percent": 35, - "record_start_token": 45875, - "record_tokens": 31, - "key": "KEY_28DC460BC080", - "value": "VAL_3F9B063DB035" - }, - { - "index": 2, - "target_percent": 65, - "record_start_token": 85196, - "record_tokens": 29, - "key": "KEY_009AB49B10D8", - "value": "VAL_FA9AAA83C638" - }, - { - "index": 3, - "target_percent": 90, - "record_start_token": 117964, - "record_tokens": 34, - "key": "KEY_41259044966F", - "value": "VAL_5A7C37B697D1" - } - ], - "max_output_tokens": 2048, - "ignore_eos": false, - "temperature": 0, - "thinking": false - }, - { - "id": "retrieval-n131072-s6", - "input_tokens": 131072, - "input_ids_sha256": "9a03e29d250ab9a2c7ddc4d8f0ff1dba6bdeca5bf440368d9a5186efbab1004d", - "expected": { - "KEY_2DEC79C0A476": "VAL_B8D0DA50F844", - "KEY_4E6B89164AC9": "VAL_B31777B11B6C", - "KEY_9424061BDD36": "VAL_778404A0DF8E", - "KEY_AD95C05D361F": "VAL_DCE753B1D85B" - }, - "placements": [ - { - "index": 0, - "target_percent": 10, - "record_start_token": 13107, - "record_tokens": 30, - "key": "KEY_2DEC79C0A476", - "value": "VAL_B8D0DA50F844" - }, - { - "index": 1, - "target_percent": 35, - "record_start_token": 45875, - "record_tokens": 32, - "key": "KEY_4E6B89164AC9", - "value": "VAL_B31777B11B6C" - }, - { - "index": 2, - "target_percent": 65, - "record_start_token": 85196, - "record_tokens": 32, - "key": "KEY_9424061BDD36", - "value": "VAL_778404A0DF8E" - }, - { - "index": 3, - "target_percent": 90, - "record_start_token": 117964, - "record_tokens": 30, - "key": "KEY_AD95C05D361F", - "value": "VAL_DCE753B1D85B" - } - ], - "max_output_tokens": 2048, - "ignore_eos": false, - "temperature": 0, - "thinking": false - }, - { - "id": "retrieval-n131072-s7", - "input_tokens": 131072, - "input_ids_sha256": "d4e80e826578e7a80f40d25156570949857cdbda6c0ee01d15d29b68c82324b0", - "expected": { - "KEY_5EE1BEA30178": "VAL_A91589D23FF0", - "KEY_5FCAB7E42EDC": "VAL_F0E6FC236486", - "KEY_169EC8E3E10F": "VAL_ABEAF259FBCC", - "KEY_91FAFFBC08C1": "VAL_3323D4E2EEE8" - }, - "placements": [ - { - "index": 0, - "target_percent": 10, - "record_start_token": 13107, - "record_tokens": 30, - "key": "KEY_5EE1BEA30178", - "value": "VAL_A91589D23FF0" - }, - { - "index": 1, - "target_percent": 35, - "record_start_token": 45875, - "record_tokens": 29, - "key": "KEY_5FCAB7E42EDC", - "value": "VAL_F0E6FC236486" - }, - { - "index": 2, - "target_percent": 65, - "record_start_token": 85196, - "record_tokens": 28, - "key": "KEY_169EC8E3E10F", - "value": "VAL_ABEAF259FBCC" - }, - { - "index": 3, - "target_percent": 90, - "record_start_token": 117964, - "record_tokens": 29, - "key": "KEY_91FAFFBC08C1", - "value": "VAL_3323D4E2EEE8" - } - ], - "max_output_tokens": 2048, - "ignore_eos": false, - "temperature": 0, - "thinking": false - }, - { - "id": "retrieval-n131072-s8", - "input_tokens": 131072, - "input_ids_sha256": "d72e343e0e44604ee66be34f2f7ef16dd6ab0a5444e0066706494e8084cd6ef5", - "expected": { - "KEY_6AB0243D81A0": "VAL_310AE8CD38F3", - "KEY_FB2B98D7F0FF": "VAL_AC78F7C75A1B", - "KEY_BC9200996E46": "VAL_03FC2E983876", - "KEY_0F4197453005": "VAL_3284D482A2AD" - }, - "placements": [ - { - "index": 0, - "target_percent": 10, - "record_start_token": 13107, - "record_tokens": 31, - "key": "KEY_6AB0243D81A0", - "value": "VAL_310AE8CD38F3" - }, - { - "index": 1, - "target_percent": 35, - "record_start_token": 45875, - "record_tokens": 29, - "key": "KEY_FB2B98D7F0FF", - "value": "VAL_AC78F7C75A1B" - }, - { - "index": 2, - "target_percent": 65, - "record_start_token": 85196, - "record_tokens": 31, - "key": "KEY_BC9200996E46", - "value": "VAL_03FC2E983876" - }, - { - "index": 3, - "target_percent": 90, - "record_start_token": 117964, - "record_tokens": 33, - "key": "KEY_0F4197453005", - "value": "VAL_3284D482A2AD" - } - ], - "max_output_tokens": 2048, - "ignore_eos": false, - "temperature": 0, - "thinking": false - }, - { - "id": "retrieval-n131072-s9", - "input_tokens": 131072, - "input_ids_sha256": "583692b94597295169f98e496db7b8e37db084dabb06a5140d921624bf564685", - "expected": { - "KEY_085A834BA79A": "VAL_59D4EBF50164", - "KEY_8627554907F8": "VAL_6E20721FEE7F", - "KEY_05871F38A4E4": "VAL_A77A20C2564E", - "KEY_00F87ADA09A5": "VAL_B3E46FBE4032" - }, - "placements": [ - { - "index": 0, - "target_percent": 10, - "record_start_token": 13107, - "record_tokens": 32, - "key": "KEY_085A834BA79A", - "value": "VAL_59D4EBF50164" - }, - { - "index": 1, - "target_percent": 35, - "record_start_token": 45875, - "record_tokens": 33, - "key": "KEY_8627554907F8", - "value": "VAL_6E20721FEE7F" - }, - { - "index": 2, - "target_percent": 65, - "record_start_token": 85196, - "record_tokens": 33, - "key": "KEY_05871F38A4E4", - "value": "VAL_A77A20C2564E" - }, - { - "index": 3, - "target_percent": 90, - "record_start_token": 117964, - "record_tokens": 30, - "key": "KEY_00F87ADA09A5", - "value": "VAL_B3E46FBE4032" - } - ], - "max_output_tokens": 2048, - "ignore_eos": false, - "temperature": 0, - "thinking": false - } - ], - "generator_sha256": "035c1b317c4a1d0d09c47eb07ebeb95da144751b0cd2b04fea85b39e7c96339d", - "inputs_sha256": "935b24411e897b0a1909fe94587ded8b809d7c6777cceac4e2decce8d239f75e", - "compressed_sha256": "721811a4302b022f2ff2a14dbcfb90c2744e58104d92885c13be09a37a0ce30d", - "tokenizer_files": { - "chat_template.jinja": "c3cf9e34abf4f9e36c2d72165aa9c132d3e2a725b6c2586aaa3a8af9d7a81041", - "config.json": "74227dd615bf1ea975aa676bdf355a0379858c12f394b5365cd9dfa5fc2c70bc", - "tokenizer.json": "0997f410c57a1f4e53b09e4be8f4a172d90edd9564368fb0847030937229b9f3", - "tokenizer_config.json": "b11349aafa7cdc6a320767cf7ceb29ed82f7eda5d65e8e0819e76f0ce947bf27" - }, - "transformers": "5.19.0", - "extraction": "Strip whitespace; optionally unwrap one enclosing json code fence; json.loads entire remaining response; require exactly the four expected keys, all string values. Duplicate JSON keys are rejected. Exact string equality; no case folding or substring matching. Complete-case success additionally requires the exact four-key schema; per-position counts use available parsed key values even if another key is absent or extra. Invalid JSON or duplicate keys score all positions false.", - "gate_status": "baseline qualification and owner approval pending" -} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/plan/sandbox-build-tools.lock b/evidence/qwen38-h20-fp8-single-request-vllm032/plan/sandbox-build-tools.lock deleted file mode 100644 index d75a11383..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/plan/sandbox-build-tools.lock +++ /dev/null @@ -1,15 +0,0 @@ -packaging==26.3 \ - --hash=sha256:94edc256424af38762eb31306eed28beb9f0efc50a8837492c9d6fd6004aed79 \ - --hash=sha256:d7193f7c8e4e93f444fde0262bf90af30e16fa0ad0ad44cb553c87339b23cd1c -setuptools==84.2.1 \ - --hash=sha256:3ff8b3b5492a0aa66960f4419365a86b84e013070d4313e4d4a6894f5d64d6bc \ - --hash=sha256:ca140b45d5325a9478f880e351484f62fef15f336462e449ca0c03c873d589c2 -setuptools-scm==10.3.4 \ - --hash=sha256:82f34c3e3084fc2b57d397200637cc13f2338004052d2fee47baa5f0e902464e \ - --hash=sha256:a69f28bfc245608781205e912faae437c2b2165773afa4e7b979d77447a69dd2 -vcs-versioning==2.6.0 \ - --hash=sha256:22e9159288e2d8bca2fa6f6c31c34c9c66aec157f60e9e526f5d160d2516d60f \ - --hash=sha256:f8bebd1302ebc12e67ceb30fc7dd698ec969414b6fa43b30c3f14ac9b1e72c41 -wheel==0.48.0 \ - --hash=sha256:3217dcc807155e45db462d7ef2431f5ddda0d7273b700d05a67b271ceb1287ab \ - --hash=sha256:94800765601e9171bf5d58d066e640662842bcedcbab982b2c90787a2c987322 diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/plan/sandbox-runtime-proposal.lock b/evidence/qwen38-h20-fp8-single-request-vllm032/plan/sandbox-runtime-proposal.lock deleted file mode 100644 index 55395d9ef..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/plan/sandbox-runtime-proposal.lock +++ /dev/null @@ -1,2366 +0,0 @@ -aiohappyeyeballs==2.7.1 \ - --hash=sha256:065665c041c42a5938ed220bdcd7230f22527fbec085e1853d2402c8a3615d9d \ - --hash=sha256:9243213661e29250eb41368e5daa826fc017156c3b8a11440826b2e3ed376472 -aiohttp==3.14.5 \ - --hash=sha256:0133c3c3b54a0bf1e71fa5c1ad95c93f07fd54e24ef1fe182f5122e1573d2bf1 \ - --hash=sha256:038c2c7e8caa26b6c8423779b5eaf1893904048a512c19b32fe841ffa5592b50 \ - --hash=sha256:0684952aeae1f5dbfe02d46039338513b94009baecd15d8e4098a357c4c4a2a6 \ - --hash=sha256:0790ec66fa4013e83c53b9025a45d454723da1a2fce28b3208c9b32d08af162f \ - --hash=sha256:083673c7a94c3ea035caaa5ca04288bdb44887abfe1f5ba23294e6a4b03efd2d \ - --hash=sha256:09e0eb18c7e0c8777e2f9149de63799195b9b3ca1b5c81ba6f32f2c6b8628210 \ - --hash=sha256:09ec102b4b8c9a920275733bbc11fdbb615efe6f9231a06007c0218d336fb77a \ - --hash=sha256:0a8ea271867e360ac985ae607f4a23ad9a38414b9aca1d49ec98839ae660e49f \ - --hash=sha256:0e6f16f5e49c4b8267988c05ab07760d7064cea57d077c3d068d04b0fbb992cb \ - --hash=sha256:1061b364556e8172e8d46b0b183adeeb73e8c42d30ebc745591e1bd89acad52e \ - --hash=sha256:1220353657ad49493551f089ce02f1a348fd57ffd585bfec77f2f3c4fe3a7346 \ - --hash=sha256:137351bf20bbed9a65e839f4a4452ac377389bdb2f2857d2acffef38f5e9f2d1 \ - --hash=sha256:149fb56caf7acb67073126f675d0958d9c4b3125fcd3f6d4877df98aa8a97ce9 \ - --hash=sha256:14f04769cfefe4734016a856a83af36133cd17779cef9ae817f812b8ba9d6d51 \ - --hash=sha256:15a310d3c71398e3d7bfc93a1a73fbe664315cd9e9016b8efc1cff85eeab7155 \ - --hash=sha256:1612fa5857b37bf32e5c1eaeefb96e3b01e9c70679eec81f0934e8a600080863 \ - --hash=sha256:16c8abd5bca220a47efe667d26f8460124c81810787e79ee87b242677563d9dd \ - --hash=sha256:19e2ba471507c34f8252402ab50f5ab512398b9ea8c8f1cb26beb3f75793ba30 \ - --hash=sha256:1aead151c3abbac6b32942e452020cb66d7efc099d253cc6c20f748e926c858b \ - --hash=sha256:1b438b73c38111818d0c9d6a5c2bfed8584c8e503a49ef085d70e874ec846738 \ - --hash=sha256:1b5416552740edf07234cc9437d0706f2acb67b93c198670b1a68e1b2b587dec \ - --hash=sha256:1d2d981b53dd09a319e3570ef8cc3bbc3ef86f5a7abef0f6b2bff3867db3a9e7 \ - --hash=sha256:1eb8167961ec4dfcc8cb9dd50bd0ee72519f7ef496be95203e49e27b01618382 \ - --hash=sha256:1ffa3a523a36d8628f98c06492ae16a31a23d14c0b4ec721757b477319f656d6 \ - --hash=sha256:20064a177a070d789ee64a50b01a9161d3468e989baacfc6c714aa685c4b332f \ - --hash=sha256:20726f9782d5c2744c1c66255842d1d163bb3edcf768b8de25216bf47f7b6ccf \ - --hash=sha256:20f085697d7e911f1f73c43ed03fafbed1e7121797e2eb5428efa80398060584 \ - --hash=sha256:225c579c23b68b343cccea27a7e06e3bd8ec23a09c30b427eb3f1e4ca6239b20 \ - --hash=sha256:24409db442e2fb6e766bc7f3943851a8381dec3098140e43bb2e843b79e31b12 \ - --hash=sha256:248d779ad720b49d4fb355720e60c9e5f444f95887bc16974fea48fc56c41789 \ - --hash=sha256:2528cb4c6b92008c76ac9ac6298624069bb2db91ff4929905512d1d84485f658 \ - --hash=sha256:276a4fc00b1d9ae492b802763a789c5b86328b989c5ea169f2faa447d6a11c7c \ - --hash=sha256:27c2322e03f66101acb09869ce1cf1efc04994ee95e1735b69827bf8c8b9d781 \ - --hash=sha256:293d3ae7c6a0ed176a42e59a1b5fde825ead65c835360f734148e96729f928d2 \ - --hash=sha256:2cc38a4f2b516bef1714e690df87a0e043faf1a7693c82d860091684453d5111 \ - --hash=sha256:3093b72c215bda16ce961a6d073f6e71d46e022962a9d5d457c5d4d421c78b57 \ - --hash=sha256:32e8fa6644e541fcd7e02430588c7fc93b602c1778ea0bc345505db76b61cfb4 \ - --hash=sha256:330900acd0dc4cb8b27f9c127fbaad770964845338493e7906ae3822e82dbf8d \ - --hash=sha256:33f706574e32c6e694f352a856e05caf18f7f2c871b3e87b41c55ea452b409ab \ - --hash=sha256:3ae800a20947e2c2e53088047d021e6bf7d51560cc49f6a0737a1f79d2e3a13c \ - --hash=sha256:3be7dd397d64ca3e1869626fa9318aaebb54b7bf93bc72d7a205448d83e4f748 \ - --hash=sha256:3e0eb43bed3c6801a6cee315195377789e90b2a72c2277a475b578535312488d \ - --hash=sha256:3e51a27980c3788e6e6b3325d694fdd4898087fa8a86b2763af77b39353da41e \ - --hash=sha256:3edbece0379b8b4aaa67619b8aa2399bb66fce372cd5911098a434ea77220aa0 \ - --hash=sha256:3f2dcc00191fd563e9075181a14ec31d7dd63223ced7582cc70a15a499de0c79 \ - --hash=sha256:42b5e616946dbaf505e2bff18c9af2cd4ef9e7ef300ee58a6e951a5b7cf147ae \ - --hash=sha256:42f320d4a5b00b9af0bddcfec5407dc6f2d9816f006b2f79ebbaa31f16895df3 \ - --hash=sha256:43351bdb5e4c3cb7d1772368e988534e869a74db7778079a83782c11c69535c7 \ - --hash=sha256:43e1b7994a8b038125f722bff07492ef501110722c2727c408995d9fb864c421 \ - --hash=sha256:4887d130a7bbfed3a85493bb5a25e5b5b558d40c1d986dd16970d2bb26d63793 \ - --hash=sha256:4f07fe3ac408d8b3f768be471dc3f56d43843c47d97c66120534467a15ead197 \ - --hash=sha256:50343c1757b4b6f6708eeaf24534b32f19dfb99fb1b762c00420867a62fc81e0 \ - --hash=sha256:50983e3be33d8c0942ab88cec3905b10602f64c469b20153c48c5d4e558dd016 \ - --hash=sha256:50a195903119008fe9cc68710535eb37f556ffffd6a7759afe70a2c145587045 \ - --hash=sha256:5558a7f5a05af9ecf744af91e5baefc436f93c9333e656c27ec253f9a6bbe178 \ - --hash=sha256:56572c42e3ecd636de8d2c3dd54cf5fc939cb5c32eb56297f176a0d366fac622 \ - --hash=sha256:56d9828f204331a5ca8850fcfe2bcce95a149f1f223f60cc7216e5524978e480 \ - --hash=sha256:579f97d5120f2971876d2ddca2968135f6944d00c44c3a6590ad7d86ca9b403f \ - --hash=sha256:5af42135fdfebdadbc2bcd9c0842a48ccf0d62794c36a260b21dc4b94d1e0119 \ - --hash=sha256:5ba14a839fbe87cf7c12a6b5661c05f324a296eb8363141edb3944ba63d4c9d3 \ - --hash=sha256:5c76f1802bab718a68ac3cce447160605c734551f95c67ae90fa1132b215cb29 \ - --hash=sha256:5e8f97c0488ffda3082766ac0f2c8150a9a58c4d05788330e479cfd449b37939 \ - --hash=sha256:5f3e96071686755d9cd3600c3880183eb94b012178e92746d68101800f0ed8a3 \ - --hash=sha256:5fb6a6e919bfb703227bc1ce6579281b84b1a2ba57deb9794dfdbec7dcd1e40c \ - --hash=sha256:602c1e9b718a3275c580149f947e7fac65044c0a20e599553fb12e9700da9eca \ - --hash=sha256:657291433bf4dd3142f3abac495764cd47d0c7c92087751e6666c6447e65fcef \ - --hash=sha256:6774814fd5c338e72ee0da5cbb9432816df450e69c019f72b5d29bdec2a1792d \ - --hash=sha256:6da32b5ff3fd78d244e37300463434c7145162bfd2b6e9e915ab164da37f7343 \ - --hash=sha256:6e1d8637cf73eebc92eba2e11d4cfff98a3b562f2505bd75bba766d908926e8d \ - --hash=sha256:6e4251c0ba4624a68a2c11471a1ac54c3306876c21f0ae86de085cc9241c8905 \ - --hash=sha256:6f275c11d1aa6d4c458e05a68be084efe3c55a113d99e3f46a318098e52948fc \ - --hash=sha256:6f967dde489ca6a8c02d093ab245d2cbf50ccb5c36adf0188b17b0ca39d24b67 \ - --hash=sha256:6fdcd6af7e2e51d1ba1b4bea16e97b074bcb7b5dd0246a9d8201341bb28085a0 \ - --hash=sha256:70cb4008ac2ed1e0ca9e824deb4b53d3aa0d939109698ebf1e723a84337bd794 \ - --hash=sha256:7457580535e019e1247ea35d6a02bf081ad30c26d0cbc210c93f6c3ab67a0835 \ - --hash=sha256:74b0a9c8270f9b0a11410e124ff8d4f18bfc1f1837440ec84da5ae7b50927b5d \ - --hash=sha256:74efb69332b85675b1eabd760a8cfc2e2cf42c60607c66f88014c1bdfb40942d \ - --hash=sha256:755933b107ea7a6a9ac916f635a70595a5b1a32fac10a8ff0b9f2ab88555550c \ - --hash=sha256:7779cd97e61ebe583ec2f1c5616cdd038aa08a4453b1848c67842176d054948e \ - --hash=sha256:788ecaa9c10533b786ce5ba70c4f2df78ad41819fd00a6c99d92b66f9a32e1da \ - --hash=sha256:7bd8ac754ebd6733a3e2a0dd1674c4d8ab086196803fd8dcd776f07b4e2607d9 \ - --hash=sha256:81c2b3dfd56c62bee6108e4852d5970b4cf9086390b6983f52b666e878c1f115 \ - --hash=sha256:823c910f046f23f4c713b8d99a2242dc65f591cb45ee86418fa11762a3c2963c \ - --hash=sha256:82c7583cd3dfdc7dcc927835b4f6c7faae7ecc1ba3ca5879321621ae2e6f8e84 \ - --hash=sha256:8966ecac808dd5f473c9c4cefd10cd3ffda71c18a4d3493b7c7d2ae1803bf2cc \ - --hash=sha256:8df7d481654ac96fe1ba9a02a9f67770fdd367823e0d5ef01b922725c4bd2cfa \ - --hash=sha256:8e317e0fb6b16212c881d2205a7d87414c29acd69320b3aa6dce9d9c7b86fe4f \ - --hash=sha256:932ce7e694bbc29b2bf6f64f2343c27d148d4997c771d01bdade4639b6749ff4 \ - --hash=sha256:939042d5cda21d41a6f512e7cc8b8e33a2aebff863352251da495fbd91b673b5 \ - --hash=sha256:94684b879ac1d71e4238850c99b62dc1b28d9086b156a2555f082010b85a865c \ - --hash=sha256:96a2e584f0b9ed8f1fa33211397dcf67bb7069866402cb405d191c2f0defb9a3 \ - --hash=sha256:9ad7e6aa38c20da1be697874349c4c273c8a03b7887169665081706398d0439a \ - --hash=sha256:9b42db919715e91eb76acf3bc492a9a7ccd8bd9adc6745c1412b689735269f14 \ - --hash=sha256:9bab2045550c4fe0f7baf89574db1b455c195750702ba96fef1f16972b146617 \ - --hash=sha256:9bf1d5dcc15204d9ec8b8ea4c18fd66e6b80e5de1f4ecbafb3a2f2740f8039d4 \ - --hash=sha256:9c061aa954daaf57d2a4b8374f9fca621ef0e1b603584431c220c22458c59b6d \ - --hash=sha256:9c428eb2bd8817588d16a0ab898aa4eb5d141f896aa2b394cc79a4cf61d9a8e2 \ - --hash=sha256:9cc882cf8619109583c906b4d4a85d6a111a98afa34b7a450d1e08118d016820 \ - --hash=sha256:9ce66feae6ac65327379460380549bf1b8df8e17c4e25df2a2bcf168272e3bed \ - --hash=sha256:a23fe35d776bc03cb495938b9594450d047e3bc08c5255315a82323e9cb7d2dd \ - --hash=sha256:a2c473a355f9239efcb72c92d5abfd8fcdb0cc78c8e9af607e72ca12dbb36593 \ - --hash=sha256:a63afd1f757de949028387e65a7127b61ad0f775432dbb0e62816ae619fe69ac \ - --hash=sha256:a6d02b4c38de03d9c7617813433e6a0fb6b522797974177d69d9dad431900833 \ - --hash=sha256:a7d470cf7b206e6359fc77b1b860632fde400d5a2ed59cd0181b93a686bc81ee \ - --hash=sha256:a95529a92a446db351675f4aab518feaf5e99842f63f5dd17160c2b74f382db3 \ - --hash=sha256:a9918e58faf62ba2c7147927d06057aec78f42475aff5048047ec47e7265a600 \ - --hash=sha256:a9d3983bd6ab7aa1cfd573544ae98df9b6cb6912a5185a198263e024a636861d \ - --hash=sha256:ab52d8f1fc1b64821c1fbad64a647ed6203627004059a6d1ed4f0858a1499703 \ - --hash=sha256:abfda5cb094a829f7bc25216a32f7db2e85cc65bd59910f8e7b40b3d9b224764 \ - --hash=sha256:adbeee7d6fd4cf5fe0aece2fb3edc4243615d3180430ba8149d01a90670cac99 \ - --hash=sha256:ae53924aa853a7a2ca20ed4142c7c6b56338e4d4cd999e2980075b9efc2e257a \ - --hash=sha256:b032a0023eb41d768ce77d83210ab2a3c389bc0b09313273c7e1eca48c10a755 \ - --hash=sha256:b1b8ece1e71132d2afba4dbc0c3d62c766e25165990b25db1196c04969eb3d84 \ - --hash=sha256:b2966998927d7bed9db12c0a4647b0c7b179755878fc9c357fe1ffd3e3b0c1a5 \ - --hash=sha256:b3cc509327c7b27f6f4727a8830f4004f6df7766e179f2f4b8e54e65c0bec5d3 \ - --hash=sha256:b7806e804889231b0e06469fd4a5c06313d1c0a3377322b6d9237fa5e0fe4167 \ - --hash=sha256:bea559ad70218d230663e4210875735076a9bfea5994cef34a55a25faeaf2544 \ - --hash=sha256:bf163cc701f3d4ac43ba7d97771bf5fd955220ef5500ef3ee847bc0ecfbf4ec1 \ - --hash=sha256:c147451b4a58e7050f7f7394e6c467867c84161560001f9ad4fb2d1446743946 \ - --hash=sha256:c172db893e516e1358e65a95ee20b7ce7173963eefe318b6ab2a2220688b999e \ - --hash=sha256:c1d60eafd9c7e8e74abd03a5b00df44e7febfe6d9b89b559c0a6551eef0699d4 \ - --hash=sha256:c2c30484dd1417ef98b51021ffa2cc0d7f3c78918adaaaab7e70817335ab3e02 \ - --hash=sha256:c32e26310cc10e547f53cd13d39a369034f69dcb7d749d5cb0e5f67bc196b6ba \ - --hash=sha256:c5ed596aedb9c42afd3fe0aae3117725378ac73d2cc5ddc735056fbdb96c5d02 \ - --hash=sha256:c8859a013ae0de1074660992139a1a440df3e6b219b86cf0d3f11c2692bb4fe3 \ - --hash=sha256:c8c4478bef6d57fcfda15dae461ea3c9f06aa7b257c58df3f2300174ccbb185a \ - --hash=sha256:cb11a971a3aea10f9b8373be628f1df932964fc6c6b174516d318a48c3ac4412 \ - --hash=sha256:cb131d775a1573c1aee66656bd78b023577bbdb6cb8349a07773bd4f73e68a6e \ - --hash=sha256:cd88b01f3d37b7a2a34f91d98f14720206f1ea3d540843fab2d649dd5fb91fec \ - --hash=sha256:d05e94cdfe0d15d0206f970722d2554780ce562787b21b218b275447f8751319 \ - --hash=sha256:d079c0a0135c36e7beb6f1c88087c8f108dc5891cdd0b5eafa778421bda70ed2 \ - --hash=sha256:d3112585250b199296c26ca6e0131640b6a8d01bab8b232d2eb3763ed469de11 \ - --hash=sha256:d418ce2af40c6bb685b3f663e9e8de27cb0a22431d8e88a167348d7f01878073 \ - --hash=sha256:d51db97c96384fbfcaf8f4c65922183a68b94f891c3c10c862ef5f6df2adbb1f \ - --hash=sha256:d94e44be379e569758fee8a9a58431cfc3c2598c708b92b1cfe96c66b4c94aef \ - --hash=sha256:dab9ac5a67c8d1f070c00fa8fccb7cbd1b8dcc1a8d6b42f37540df9b3d4cc603 \ - --hash=sha256:dbf53ae2601b7fd5a93c3944deea3a78d40f495226d582c35ef7a433425ce2b2 \ - --hash=sha256:dca3fa8d8a0a26679862eccb0b1a9151b2b9f1cd2c212e7a6335778faaff5833 \ - --hash=sha256:df37b620684e19b5e25724412518ccafc3b1a49cdac706fdbd2f983fad943450 \ - --hash=sha256:e1cc2bfaee8c214f06080a7c7d5772419b8a1108e8e5349236189811823fb02a \ - --hash=sha256:e29347c142cf6e99e0dff5e2995ead1d50fa3b51bf37a7c726a7ccfe5419745a \ - --hash=sha256:e4f5cf4dc72a71c4cfa9751b4950be22f733626670230d46e7d606592aa22d59 \ - --hash=sha256:e724a7b6091f0b1ac064f9d1b15ff9ec52e6033a86cdae649e5f086e32a3c0db \ - --hash=sha256:e87046c8ff77a8decdb6a41d8ab25824b47531b2da933aeab0c1e21c7acff329 \ - --hash=sha256:e95c8def4b81c5d68d5cf1f54c07acd7c0d2577af244e5b6da802120825737c6 \ - --hash=sha256:eb324e2009fb54db30a071dad7caf6998ee2879c4704007efb244514dad1fec1 \ - --hash=sha256:ef60869969180ec2464f1349aff07138ae35ca2200f0946cb3552e49e8f301a8 \ - --hash=sha256:ef692a24087a699c0a4a26af45e746e0c1eae2116f6d8a5ff91d8aae2b867b45 \ - --hash=sha256:efc21a454892828368b11c2c780de0ff8bc991f73f6b99c6b66e56205470929b \ - --hash=sha256:f001b571ead90ca1770f1e616db255351a1703317f20374c361ef22f12c06d09 \ - --hash=sha256:f2a7966bda23dd85051f1661ce0ace38d6890e05ec6c357ecae9d2479cba377e \ - --hash=sha256:f2ebb54b3f932210503072f09974b4fb574d823e497a944adfdcd140a6a00255 \ - --hash=sha256:f2ed8b64dc0c651c0f5a9c926777719770021251b8f336d97c4b80b660836ce1 \ - --hash=sha256:f375db73a39f5cf83696d500e21a67f418dc9a988955756f254be8f03b7b3651 \ - --hash=sha256:f59c7673465908cbe506117176156c127f29f917677afceada34957179221d91 \ - --hash=sha256:f8d40ce41991e9d56fab4f5dc4a51fe59bc3b5c77c27f4b148963064d00232e8 \ - --hash=sha256:f9033b43f511f27547c557dcaba0177649e10a3725336ccd2cce0fdc1dc4850d \ - --hash=sha256:fba47bc2c3d7303c3d027c6cf4d07626c37b1314ac81f5820c31032e0ca1f677 \ - --hash=sha256:fbdc5ec49f9ca3cd24955cf3520b10a4d4c901ba2572094c84274e9e7eb30534 \ - --hash=sha256:fce9523df31cea6284f3e2c479876750d7687cf671d7b25d32b19effc0e86441 \ - --hash=sha256:ff75a7537413a86e7cafe98e0e1d6e3dc4b15c6349896e7d5c6b881bfdb6d550 -aiosignal==1.4.0 \ - --hash=sha256:053243f8b92b990551949e63930a839ff0cf0b0ebbe0597b0f3fb19e1a0fe82e \ - --hash=sha256:f47eecd9468083c2029cc99945502cb7708b082c232f9aca65da147157b251c7 -annotated-doc==0.0.5 \ - --hash=sha256:117bac03a25ede5df5440e855b32d556049ca169ead221505badf432fed4b101 \ - --hash=sha256:c7e58ce09192557605d8bbd92836d7e1d520ac9580096042c0bfd197efacf1bb -annotated-types==0.8.0 \ - --hash=sha256:13b2beaad985e05e2d6407ee4c4f35590b11f8d693a258a561055cac8f64cab7 \ - --hash=sha256:f072f4d804ea359e4eaf198b1af7a8b0943881a87f31bb764f8bf219bb9419e0 -anthropic==2.1.0 \ - --hash=sha256:7533cc9035ef056159ec44f54e309de7b7d5ae329ff8f7df9da9ba05188d3c25 \ - --hash=sha256:d776fa2ea9a07e20336a0668d1ef3a12357fdef75c895f54a050b3746b8e4de8 -anyio==4.15.1 \ - --hash=sha256:6152fdbbf9a77fdec97731721bebf7c4c44f7c29b424b0065826173efc7ed101 \ - --hash=sha256:9f28306018cbd6d329e64a36d58256edff76dd996fe423bc957326e578b82a94 -appdirs==1.4.4 \ - --hash=sha256:7d5d0167b2b1ba821647616af46a749d1c653740dd0d2415100fe26e27afdf41 \ - --hash=sha256:a841dacd6b99318a741b166adb07e19ee71a274450e68237b4650ca1055ab128 -attrs==26.1.0 \ - --hash=sha256:c647aa4a12dfbad9333ca4e71fe62ddc36f4e63b2d260a37a8b83d2f043ac309 \ - --hash=sha256:d03ceb89cb322a8fd706d4fb91940737b6642aa36998fe130a9bc96c985eff32 -boto3==1.43.117 \ - --hash=sha256:a5b6cd7e67653987fcf0e212187c16f7b58af2ae192e368f7212ddab7c05b25d \ - --hash=sha256:f3d314908d866b577ee6b74ed432c21e565c4c4dbd640ef3fd5e0da30fa13a98 -botocore==1.43.117 \ - --hash=sha256:5920ef9b3002e1361846549078549d9154a6e0004c22853c01ce7b279d05d495 \ - --hash=sha256:71993702832cc37c2495048b9945f2929d306fccb3fb767e9964e437e637fa6c -certifi==2026.10.19 \ - --hash=sha256:035c1e85572b141bf1c3a3eff0e33a8eb74917522f2559075ff8d2fea780a1ae \ - --hash=sha256:42d35340cc5b9ba78e818ddbff0c5b1c2b779de448a340fd1b989f5fa8670e1b -cffi==2.1.1 \ - --hash=sha256:046bfc24911b37851ee1b51aab8bffe713d89c68c6a057b09484ce9fd5f69b4e \ - --hash=sha256:06c72bb76605a4b0cd0aad6930b69d4baf7dd5d806cfc409b824191099700e66 \ - --hash=sha256:0beceaabe56af686895136a2de78db54ecd8e4046b236b8fd6d6cb61389e9bf2 \ - --hash=sha256:154852545011f779917b11c78db2358d095da62a9a172b78ad0a583ee5adc0d0 \ - --hash=sha256:194cffa889098ced9976c3fc6340305e43f6303657d298da55366907c05c22d6 \ - --hash=sha256:19ee6127ee34de7d83ce3d371ebc5ed91addbdcc39f9ab15ce4eb35a4e534971 \ - --hash=sha256:1a18a57b58cfb21fc28d72e876acf10eaed67a1ed96226f92af4df681d571c4c \ - --hash=sha256:1aa5645c30469b09530c4ebca77ebf8f17618293c58f8549cb1a543a50236e7d \ - --hash=sha256:1dea0e4d7d4f11f619fe8c1d76caf49e24405b4b5743c0e3be16a500ecd930c9 \ - --hash=sha256:208f941bb9d18e768138677f0a6d2ce01f590df56043dda1df1535ac57c88517 \ - --hash=sha256:210019b6c7cf07f081b4c54635c8cf744377001350e29cc0f81c4377b4797735 \ - --hash=sha256:246fa40ce8645a614ff682e0b70f37134e460eaf93a775e0cbe3cca585a67a80 \ - --hash=sha256:25792eac27877609e7bb06d42ff88278a6624fff2ba9bbb523c09616b117e80f \ - --hash=sha256:27350daa11d4f10c540e6e89dada4c54feb7256ad03e9a4dc075ebad7ba360d1 \ - --hash=sha256:28907ab9bfb6aa13184cfc17c6b8e1023c5ab6fd7076d8c20a35e59fe04f8f29 \ - --hash=sha256:2ae64be792b8966f2c69538199728b290e34726562896df1e5dc8ffd8d8188e8 \ - --hash=sha256:31348097ff5bbe827ccc41795d4dd099d9f0625e7def00ee653c137a490c2a6c \ - --hash=sha256:3143d81e29e1e20a9ce10901ec369012947876596f75a222235965f2b7ae832e \ - --hash=sha256:3222ba5d678f80a030e6afbcc33dc1ae5cb45facabb61cee2c7016b8432fde48 \ - --hash=sha256:3311ed60d36f83378794e1009ac6258bafbf81f7888b4caa7b35a521e3f95813 \ - --hash=sha256:334644fbac4eff73d985a17a91226df55d0f394160c4cfb880e084c8f7161cac \ - --hash=sha256:34e261f78cb6ceaaa36f42f2613f4380d94d9c759a9c73c769ee6e0247364632 \ - --hash=sha256:363e05fa78e15116c3c32c210ee36884fd6b9afa6d440e47112c3bd511d64cb6 \ - --hash=sha256:398aff33cee2767e3e781d2554c54bd0dff386bb437581e0d8011fde1a942ec1 \ - --hash=sha256:3d22a20b1fb1632cc72c22f95f7b0d2961c3e1c235f245ba4c606c4771035659 \ - --hash=sha256:42a494cee34437f05546455144f2b5d9ac09b1face62bcfce597d2e521066688 \ - --hash=sha256:42e2f76b9455f5a9a844f770bf3e200ed3da0e15f5df3db9c31fe80b04b3d004 \ - --hash=sha256:42f6930c31dc7f50732c9ae793c2786c7b6b044195967bbdde40bb9be81c4cc0 \ - --hash=sha256:456a61fa52d579ebf9df2e9552ead5129855dbaff6c1e5a9b1bc408809bdc062 \ - --hash=sha256:471cee653ae88de62096552e6d24ccb4a5adb8c8c9f10b5054d0122c15bf2779 \ - --hash=sha256:49cbc70e6542d4ccccb936558d1064a8012541e78f821f955cff24e357776c94 \ - --hash=sha256:4a7c934f7360e8cd64fe9efadcbd10c7c6364f531e432b9a4bf5ccbc9e0e8b50 \ - --hash=sha256:4be96343e422f2dfcd12ab5c9f5aebe03f82f737c6bffeca6830b3875cb44aab \ - --hash=sha256:4f42141fc14250de6dde5ee7ea4432be017252d91f19c5ad043c084cea629cac \ - --hash=sha256:507a24c282e0f42f8ed737cf048572cbf580468da5555764a8331735e9c736b6 \ - --hash=sha256:51b31d1c98274844cfd7838ce00bfc27c7423a4dc00fc0772fc3331c2cc90676 \ - --hash=sha256:58acb8ab8e295e6c5ea12f888cbb13cf21511ef2a3303a23f4325c29d17fe5c1 \ - --hash=sha256:5a59cc1c4442bc3d5c703bf720b51138d0bfc173618807c9ee2490a7541dd3d9 \ - --hash=sha256:5bb4e7ea95dcd6a014a6fef62e62467d67d8e582326443f3d68e71d6320a9fcf \ - --hash=sha256:5c58fe613dc5e5336357eff555824a314d8e43282600435c8d1cb6a7a2fedd13 \ - --hash=sha256:5e7cecbaadb83884793e05828cee59b210b24583b9c7425d0ba6a754fe22eb4e \ - --hash=sha256:616f097f2fe415bc92a247f02e11f634e1f9e9a83d327e3c915c15089c87869e \ - --hash=sha256:63bbfd5ded17c4840ac07cd8f1c21ba9d9708141f840b324f422f41b207e3973 \ - --hash=sha256:64faea20f4e2613363a1a9b9c7dd73058f3ecd00133a511e72ad7c511658f527 \ - --hash=sha256:661c298b4821edebead0c91edd2b00374d67ad7c5a1f7a91d4442633b79d6a72 \ - --hash=sha256:68e62fe11f30d5ca8289242866f0a5291402d8529ca2178ab8afc5c9694ae890 \ - --hash=sha256:6a8dddef476fab96d066d578fc88526767b836ab5ab21754e1d5bf3879c31c7c \ - --hash=sha256:6e192623c49c94421616a5778fba35cf0d5a8d000650c1967ef4448ee5cdd990 \ - --hash=sha256:7225e4514edb64eb6740324353e0da0711954fd8d7da4576755b1c6e09b697cd \ - --hash=sha256:75f80557d1389eddbd0de2681f6a390a0c5338c31ddaa821381c203fc3fd50d9 \ - --hash=sha256:770de9db11e84213beec501cfcaa013b019820ca881e03344dea5844f7876d94 \ - --hash=sha256:7750c6449dff7864bb9bb27ddfb0267756189201a3afc911d82b3caacd70dfc3 \ - --hash=sha256:7bde5e4cc5c10140859842b9d383af292b22639a4dffb725314baf45968cef80 \ - --hash=sha256:7ce713ace7c0e4520535b42b77eaa742c16dab813978064913e5a3cf82973b41 \ - --hash=sha256:7da0c5eff80f0197f3b3d1232ec5a682a9325f4ae9016a78f5f5ca35f9ced1f5 \ - --hash=sha256:7dbb61fe3a7699468030f71bbe5f8a0e326a151daa91beb11a6fc1f980c55e1c \ - --hash=sha256:811bd1e21d32de12efca32393a0ab3f5133b54fce9bd44b8bd77ab07da14bf6a \ - --hash=sha256:8ef53b2de9bcb9197d31854256575d59dbac0cba72ac627bb291ef5eceb74be4 \ - --hash=sha256:937c0052c05a31ca1daf18de3158eed4dbfcb9cc107adbea227728d647be701e \ - --hash=sha256:9d2055050ea716bd38b7f7f1579c275386646b4894c155a3e2f3cd62ed41b7c6 \ - --hash=sha256:9f8d177621de5cb38ee3e731eda45d421db093ec0739f46a5594babda7987a98 \ - --hash=sha256:a2d7755bef5a12ed488f4ef1f1b69ee9191d7396083b755a5d2295f6edb4768b \ - --hash=sha256:a48d62ab9d6f4f98c983223a547af44be6ca3691074c31cecced6facd3ba2dc1 \ - --hash=sha256:a4f00aa42f75d6e4595e8866e748cc1705adc0cddfeb2ca86d0d03993d63ba03 \ - --hash=sha256:a6e721d4b0e45d5b65e87534470e67b18dcd092c83f68fba09f152b9cbc061af \ - --hash=sha256:a730a083190634c65cca36ba5f489531576ebd79bcd5c8e172130f6453127231 \ - --hash=sha256:a931079504ecc49efed7744c476a5c343a92fabf66dec2db95edb1b2fdc770e2 \ - --hash=sha256:aa9511c62d14da7aacc9b4bf51f3f697a621e83b2d6919008243c3aad168eea3 \ - --hash=sha256:ab36d55f9ed2d067327667c2fea18dda018eb628dd6347aa01dda6cf1f5d3836 \ - --hash=sha256:ad2c86c495b899d862ea0f4b42891b8713a3bd45dd4105c7fd51c2a72f39f3a5 \ - --hash=sha256:aeae0e330c9f6acd681f647d46cefd30c29f93e3392882e792e82080c9691399 \ - --hash=sha256:b0431303acaea1089ad4b3e9ce4e6518193def1118d4073ca848635ee4ea2e96 \ - --hash=sha256:b5bdfd1c873d4e093aabc0ca84c4ca6dbc4f752afb5c86f146d9742580c9da2e \ - --hash=sha256:baed1e86cc735622097354b9d1281406caf42ff42a886d29faa8e8d1630333be \ - --hash=sha256:c1453022f490d2459a11819d83ad1d586e9ff65a12ac3e705ffebd46d3685dcf \ - --hash=sha256:c26608d2222fb1e94487e4a387d85f13eb55d5ed725cb25a0c589ac4ee60e7bc \ - --hash=sha256:c7659f22557c5a0bc4855cd635f55edec690cc008a40768527762cb9fb263455 \ - --hash=sha256:c8c69575568085ba0b1b10c0249d779a214aea6f6522e949a0fc9fb0fcb449d0 \ - --hash=sha256:c8d2c9fd1f2d16f780d15127abb050d13d1a76c03a4bd87d7e4980e45e511e12 \ - --hash=sha256:ca82be1a1d406ecfe1d25dc16cb33488e5a16bf4438c9fb590484ea29d92478b \ - --hash=sha256:cc572dace3f60ef98d7b12ff411d20f5362feb31a0439eab0085bbfd349982d7 \ - --hash=sha256:d18e5ac0f2f03f4f518d3e23db0f0cad7faa1da8620e9c09461d443bbf6e6692 \ - --hash=sha256:d28630f5854ab07ab1fd4aba756de52326c82e6be15d414b12793f1975048b54 \ - --hash=sha256:d9c275eaacd24aa73f94ffd6de08fc3f932424d8b6c376f4bed7cde376fe7bc3 \ - --hash=sha256:da0e573f9f97159390c89d9f1a9e41908b66d408cc5b58d08cf3847d844c531b \ - --hash=sha256:dd31f52ea1086513bb9df30f8fcee9b8918323ae067a3d5b78bc826a000712be \ - --hash=sha256:dddad92b554513a31f272570678ba307fb9f618f05e3d4a5eacafff9eae03e1d \ - --hash=sha256:df423d40ee8654634421812bc3b196da3f9bd7d32929da813f8394c4348a5358 \ - --hash=sha256:df913725b79db7bcf03448f36b7bf8815363417d5b58deecf9305e3e30f0f21a \ - --hash=sha256:e0bcb7e0f677f543555d2adff3bf19c05f66cdb4796e5ff602442ab2fe3c4ef7 \ - --hash=sha256:e2d65b31f36619cda3999b78b2aa9632e76b78448e7a56fc4240824200e7c4fc \ - --hash=sha256:e6e8cff14d6fb0be70a09c0bdc58096f501952d04624ebf867e0e56da2df8960 \ - --hash=sha256:f16c709686a78c727bbbf059f92b0bf41c6fc60deec706d2dc19f529175a6125 \ - --hash=sha256:f24fb43132a4c6b4cb4eb029492919b2db645be6808d738f244fd146c03c32cb \ - --hash=sha256:f53e442b08449d42821fa4a4fba000095af9f62742a500f978a9f557ec44339a \ - --hash=sha256:f5cfbc5fe74540d335175b656c725d74d90e3730c626d92575eea35029d9afaa \ - --hash=sha256:f81b3b8f3d4e343550fa4baa0e479bba9f2d29ce9c2e9b51d1ce1718d7442fcf \ - --hash=sha256:f8ec5e643a9a937f64e1999eb9f75d072263751912dc5cd06d3c85f8f44be7c3 \ - --hash=sha256:fb92203a88b3d3053034db775110081c49d28be6551923805e039924093761e4 \ - --hash=sha256:fcd22650c908d7b7da162bbfaab594a1227a15d1643a98c68b122ac642fa2264 -charset-normalizer==3.5.2 \ - --hash=sha256:01077390b03f7988f11d700a2194e69b119741a86b1a638b1db88891e3eced8e \ - --hash=sha256:01b0c0d2262a9e28e8484a278c7e1b5d650e3ac8cf2683d2967e25899f208bdf \ - --hash=sha256:04851f73ae72b8413dddadb16a49dfee95263553741fd42d546f7d66907e6be5 \ - --hash=sha256:0521c5665880b33d603717defa76c094048900010897909952397feb3039da56 \ - --hash=sha256:0774bf9bf620249fee3e0b8b9fd3065de213be30f3aa94ce2494b3b638949e26 \ - --hash=sha256:0891b9d3903c5571c03771ca669a4b0ec5618ca722a5c957d3d29cd4e5062848 \ - --hash=sha256:0c951d5e6dd9c2ff60609476752bee49da4206adde960ebc247766937f72e718 \ - --hash=sha256:0fed1d06615f022ee3b13caf5e8b180cfea32bb2c5aded8a9d44277afc040f93 \ - --hash=sha256:114e4d0c92d618409ed82a99e22b5c5e768fe995f2973f78265f4524f49d4640 \ - --hash=sha256:11912e4bb14baae7c5d8791aa55ba0a3a03ec6729073307b0f57270abaa713d3 \ - --hash=sha256:11a4d68a6ecda3292cb1e50239e111543ba5d709bb62a6b4ea1afcfa729d8875 \ - --hash=sha256:124fbf1a8ff966d87ae05bb8bd45a71f966055ed8bba320d0c7cf450bc5f4d0e \ - --hash=sha256:1461ac396c4fdb983a675f20aa555624f0ee18ac83d832b9244ffff3d8055275 \ - --hash=sha256:1503bccbeb36d5527790c3930327704c39af22de3112f1b1666a9f3ce15ee204 \ - --hash=sha256:15bb4005af6320d259dc7593ca84a38d7fe06a421dbcf7b910ae23979101e787 \ - --hash=sha256:15c44f7edfd477b06f517a5cc317fc1707edb9de2c865f43d4b6513907473234 \ - --hash=sha256:16fa0eccf81304b79c5cd87f9271c3b85dd9dd99245e4422ae9c0dd45e0f99d3 \ - --hash=sha256:183b88127acdb4fabe59d951ab424faf1af7b63cdbb5f776186c1ea2ffcaed98 \ - --hash=sha256:195c26fb65950f8fce54e26349852b7bdd7c5f120aeefbcc440b8a20faaed4a3 \ - --hash=sha256:1afb975bd5d68d5ce9f6b6d44fdf2f7e34b895a35e95708a7a91b20a3b51d187 \ - --hash=sha256:1b4cbc7c3491ccb4aa17fcd8165649d01cf39f76de1696da8631b5f71b85401d \ - --hash=sha256:1bc0baf5ef96b6ede57d47f4b8fe4d9d84019c3bfcbeb20a41edc6a6ee341f1f \ - --hash=sha256:1c50fe28bbc2ced33386f298650d91218076c05420e6cbd790b913adc41659e7 \ - --hash=sha256:1db38f4c5496827c1a501846d64d14c3b80c7e6714e406cd7dc36a9899fa1011 \ - --hash=sha256:211d5a3eb6af8f513b8d4ca19a8c1b7accab1b5f0d3175f9826b03c1a920dc1f \ - --hash=sha256:23851fb4e1b85ed3f6c2a27b777cdfe2e19fb5b38429a8faf38c7542b7665869 \ - --hash=sha256:254eb48b9fa5ee9898a3c445825a1f340fe53712a098904b39b0bddba8ea3cb1 \ - --hash=sha256:2625388c6c754520c37abaf3b41eb34d1cc4a373f457898f08606c8e362b891d \ - --hash=sha256:281cb91036248400f4cc957495cccd44c275c2e0c5854f7e45ac5cf7dc193847 \ - --hash=sha256:28a15fdad492a99b6eccfaaed66ef3f74050680545ea61ec8b2f4c538f1f1320 \ - --hash=sha256:28b4f0d66fb834ff90f28209ac7bce77868c45d8c93e26f906709d9b7c2e1af9 \ - --hash=sha256:2a925889534b3748302dae5dead07cc13480de1dac3aea80a941b729b471ef93 \ - --hash=sha256:2b7b3bbfb4fe8ef40600792d762fbaa9057559f9d3fad209525b7a22b99e91fd \ - --hash=sha256:2c9ad19a6cfcd5ea5c0d41161d22f9df1dcc277e9bef2751391334546a314c00 \ - --hash=sha256:2cc961b171b3f3440f410489ab3573e86aea8736134ebbb40ea1338b7f0831bc \ - --hash=sha256:2ce45c6627b22c47e390bc91a41c3d13032192e699fa0bea96e9671b373d69b0 \ - --hash=sha256:2e06a3a98f916dd41d27f3105e02e7a40181c98c94b9158733d03a6f80506c09 \ - --hash=sha256:304d5463e65a35d7bb0850550e0780395395f6fcf452f04db7d5ca7cecc425ac \ - --hash=sha256:304d8e4d493af723536393eee0c689eb7813f4a474c8b479dee63f1fdd98f621 \ - --hash=sha256:30fcd120b732aa79317f08dee04d7de0847822e4cf7ee0e9f445bb958832252c \ - --hash=sha256:31f3930700408d211f13378ccbe1c40845d8da54bd0681fac3a9b5aae81c7aa8 \ - --hash=sha256:34276fd796040bf0993ab33a369aa572e6979c7aab225a88893667ad8eac8f7a \ - --hash=sha256:355ad8011081dec5412240c087a9a0c9d4d5039f3ed11a3f13e18c2b29b56c51 \ - --hash=sha256:38a873987f3be698494da8b2e3085e29da02da7b633dce73e79c699a113d7bf0 \ - --hash=sha256:39de2a259fc954455c57274dc94c79d5842774e1247a016aff30bc0efed0f4ef \ - --hash=sha256:3d14b50de6bf4d0edf857a9386836846f982b8f524e188e2e68b96d702bcf4aa \ - --hash=sha256:3d21b8b13c7592db2ac5e544a6d83187b995257472b0c9e8351b6d507ae37ed6 \ - --hash=sha256:3d31298449090ab8d47b7b1b2a555ff73cac7ed438a08b7ac160980c7ebed649 \ - --hash=sha256:3ddacd27458c45bdacd6bd6db644bfb730efbf9e830310186e3045c9c5be8fb2 \ - --hash=sha256:3df041de8887954562c9b261cba85ca0e9ded74048daf125f45edcfaa4832229 \ - --hash=sha256:40ab6bffa02ae10a0581e6c198be7d2d8ca5c2a0c64e4ed3465d766df457573e \ - --hash=sha256:4275811936e2f06feff5e598fb42a1b7ae852da8e39605211892b56b81a34efd \ - --hash=sha256:443eae2bf318abeaf6f15d785138f71fd6de770e99a92158b8b814265e079115 \ - --hash=sha256:447441e76ec720b15e64418d32e092297340387053047c7c694f579efb0ee1d9 \ - --hash=sha256:4495c5002a7b28557e7e222e77e0b661183e432b7d6d2e788101e3f240e05b8c \ - --hash=sha256:44bd4fbb29dfbeba60e7d2bd000c59e4b21ddb3cc53912b14048d37092706d7c \ - --hash=sha256:4685902cf26edf013ed7a3da0f426ebba7a00ebb9541386d835afbf002c11cab \ - --hash=sha256:498dc3188ca05a68231ac3fdbfc7f57eb67e1343c30e0fea17f8218c1599b253 \ - --hash=sha256:4c2b5031f63e331e3839b40aed2dd6f191e9c07edbde303e7876846ea1946995 \ - --hash=sha256:4d48f2d08b9de5864e2c8744d4461b862fb149a18274abc8b698c45975573438 \ - --hash=sha256:4f87960d57feabfb618e4e0af6e7371645fa26a277860739d6e5d6e0012c92f0 \ - --hash=sha256:50e3adfb96fc189eb27b1cf62d3b598b89b4bb0420d93a3d3e42e137409011be \ - --hash=sha256:51cf45226a9b588d0d2b4880c62d686934b63ab0bd79ca23ab0e9762eb27441b \ - --hash=sha256:52aa6992700996af31f375de0c6bacd402b0097fe40b53c426b9f51a90ebabc7 \ - --hash=sha256:55ea99acb17b9325618de155a0cd6a2e8f5d10be008113e1d433bbb58db543b2 \ - --hash=sha256:56bc200a365efb37383b7852e4cc5898d3b2da5987289b543956cf8cad71018a \ - --hash=sha256:588461c2e8384d309bd63e5826019b6977bc66d629b99ac8737bb795d7b2cb5a \ - --hash=sha256:58ca3755ee7ff7f59b57789ec9833c9de9ea275405cdd240eda1f193112e398a \ - --hash=sha256:58f361dcbab699cf8f42db3f47c8e7fd1036f138c23a5d08de9fde5f425a730c \ - --hash=sha256:598a11a2c7ebaa5334bf698bf29568c9c390abac6a154d8170fedecd1cea38c5 \ - --hash=sha256:59f63901b0031c3136cf64704dcb21de0bbae62ce2c9529bc39d27665463de37 \ - --hash=sha256:5cde776b7cc66e4f6c99612cea4aa7269aa65863f7a15841b2c264f103822f4e \ - --hash=sha256:5e2b6b57e9733d39f0c9fd3185efa6b8e29652c4cd8fe94180272cf6ed9a78c4 \ - --hash=sha256:5fb29fb8cd1a46c27a1bf9613ad5ec2599310d46b4025d9556404a6b6a292800 \ - --hash=sha256:6045373d5a89a5ec71afde535db987ca28e76dfa276c2d4c818265b375d4b055 \ - --hash=sha256:619799369eeef6366ed3e8755a5670f4f2f0fb6b30a0fd7264dc0fdc2357058e \ - --hash=sha256:62588a277bfb59def052abd940703fa35107152bf479781a878617d60faf8fb5 \ - --hash=sha256:62603db9a7caa0802eaa28c1c46fecd7b3a263a774069c24c3c28c302448721c \ - --hash=sha256:65cd72beeeca9d3aaea1201e5923859f308f952f9c71de93f06063c79f0f7a3b \ - --hash=sha256:68eb192d85ab8e5f6ec69c2bc6ac0179fbf04a5ac1569d12fbef74883fe102d0 \ - --hash=sha256:6bd128f206a7752ae1f2ab6c61bf8a24ba28913a10df8b14c2637b973ff97a80 \ - --hash=sha256:6be488a102b8cf28d0391d8c4ba7748938ae28b78ad901f8585520fca33ead1a \ - --hash=sha256:7218e8f32b0956cfcd048fd42d9d5779809745ca1d86113ca56f66e7ae1549c4 \ - --hash=sha256:7441d755b7ab94f8d4eb3e43ec05482d760842fd263d003a99102d742cd835e2 \ - --hash=sha256:749e97e1b32313717a565abbe321bc2190bc8b35f1a67e4cdbc7c56c8d8ffe58 \ - --hash=sha256:75a3ceed0724d625d64b86ca20aba182e4df462e04c2414fc941c0f523f06aac \ - --hash=sha256:780fbe7cab297b81dad9fb8dc5eb003c0468ffb0d9e5f65068c53a34661a96bc \ - --hash=sha256:78456a747de8dc58360ffa581f30a002baf5aa28cb262536545e91f113ed7639 \ - --hash=sha256:7967d08cf06dee78443b874f98c98036f624f3a4e73e11f9f64f5be4d25393cf \ - --hash=sha256:7a881931aa470808df94a8c380eed2bbbc76cd9dc622310f99665658c821eb6d \ - --hash=sha256:7dcd882da75ef9adf94903b1e3b9419e8aa8fb4c7396822b834b9ef7fb96954f \ - --hash=sha256:7e841fb9010836c992c9f12fcbd43a831de93a5f726fc1ccd8ca1d0268c5014c \ - --hash=sha256:7fdde2c9fd9e3eca40631e024664cf2584272cc8f96308cbe5fdfc930f51d8bc \ - --hash=sha256:8024d00c3faf3fc0c16e07a69f4405e8eac7cc0ab15f65fe6cf43827c4cf72b4 \ - --hash=sha256:80d02b6f04e92601a081dd97b23d3128033098bff5d35d392ddcc0476ea11253 \ - --hash=sha256:838dcc90063569a0448120554591a1d6c4a4ffe11babf048908793154ab86ade \ - --hash=sha256:849df64e889b2e17230d58410a03dba311a65b163508fd33679b2b737d4b7858 \ - --hash=sha256:87475fabc8d9996fd9c27debb395e642e8c838d78a00b6e932227a0e06b81e26 \ - --hash=sha256:87e50a3e7cb90af586b6c5faf23e302a970415ac73bd7bd90a515a04b427ef96 \ - --hash=sha256:89b53f3cda69831909888e0494f4fa0bcd3537e3e138dabeb620bd6ad946bae8 \ - --hash=sha256:8a893cc101149f80a653f82062ebc95b34525a2614382e1da5458fe7c6997249 \ - --hash=sha256:8b2bfab86aa71ae13aa41a6a26aab338e0db2b8bc75434b05aea89e011ff35a4 \ - --hash=sha256:8d86d6fc60743dc916eb79e2eb1ec4818e21e427731543af40a3021851174a13 \ - --hash=sha256:915563965d418f986e7e145accc592eae9e1a1be3566ff98a05d7a9ec42a76e1 \ - --hash=sha256:92888bb3187c5ba50500b00b3b310c9f2c651709d28036077680cb5255450a03 \ - --hash=sha256:93223adc95033dd47133a46ccfc316a0139176fd79085762e27202ec56018f03 \ - --hash=sha256:9373ad13ef0d2c0fb761e04e55bfdee5a08b52cef2c882c8fbe9935b1517152e \ - --hash=sha256:9409a8bf35cf78353942504b24a57de3d75b708997a1e4bd8db71ac8633ce364 \ - --hash=sha256:9b7f416ff0978e2f2249330527f0ad6fa02f4932e6199692d3b52da2048c19e4 \ - --hash=sha256:9bde855991b7e362c146535e3136a50bfaffc0487d38b33ca7e5edefc6e23849 \ - --hash=sha256:9cae88599c7219005d879f98e5ed53341e9a122af585e1091200358a3003d2a0 \ - --hash=sha256:9cf9b1a857e25c4baceeb3624e92a56df3668f398c4acba74e174d81fb4d1d3a \ - --hash=sha256:9f56f72050826f63dcee7a7f55b0a77168cb3bfc553fd405e7f8f9ece75a4036 \ - --hash=sha256:a090bb2c68df85450502e3e20d665e3a5af9c65a84d6508ed477badd49166fd3 \ - --hash=sha256:a192e2c40070d92c3ccf777e3a5c4ff515573cd2bb7ed0c537fdadbbec5bbf21 \ - --hash=sha256:a19a731138fc27d5682277d3b9df22855cea1239bce7fcec5f78f42ef2d1f3c3 \ - --hash=sha256:a66c3bc5ab1f0ff2164fc9965ddd611ff0802173f4b9d24554c563f6ab7e1d6e \ - --hash=sha256:a815775b6c38d4e0ff7bcffbeba67feded90202bb6a226b8dd35f1c855217413 \ - --hash=sha256:a89012d6d5476ee112d20d998570ed58df2260a852afb1758809cd6900411d21 \ - --hash=sha256:ae4f5fea5b8b8ccff88238cc8569303e5ee95efae67fa62922a311397a71f346 \ - --hash=sha256:b6856554c4f44d79fc2307d5768854310a8f0096e501c75637542c82292b0429 \ - --hash=sha256:b6b751274acb69d77b3323d6b7dbaa3c7fdfc1eb829b7eb61d262f32e1af9685 \ - --hash=sha256:b736353c0a625bbd5fcec108576e2385db3496f4f771f785ff32e108d3c3bc45 \ - --hash=sha256:b7fd005a73d9e657273b7a10dc71a9e03c8fb9ee6999798d6918ce095b81ac7f \ - --hash=sha256:b91363207bd9dc966a691e959bb47f64b30f7ac4b072be9968b366982f7db77c \ - --hash=sha256:ba0b1d2620edf869789c3879223f52bf2afc5d31b3cb47cc57b3a12c05e2aa9d \ - --hash=sha256:bbbfc8e28816f19d7c0f1816664980c0a9875d01b27cdf8eedddb639d9e108ad \ - --hash=sha256:bd16aabe4a02a297c23417aa17ac6299dbd8c49f673bcd645b4929b11f5a4400 \ - --hash=sha256:c0afc6800ba57ccc350374c5bd6150419915d95ce93cdbab2d783d75eaf30ecb \ - --hash=sha256:c6708715abcf3c73b99508253e961a9967f02fe536532834149574eda6de0d1c \ - --hash=sha256:c7c9ab723cde841fefb34efbad91e87f00a674b1fe1cd0784fde742bf2c154dc \ - --hash=sha256:c8f3d67aeaf55f017982b73683f0e7342ba2f6635a78f69ce89ebb26aa411e5c \ - --hash=sha256:c9790464842f85f437dbbb54417eda1e0e6bfc52dd8d22d6fd1c994b73b2dc74 \ - --hash=sha256:ca403d7e4798f525fdfc78e258820419cbbd0f0ecbab9de7840e3c017cf6b8cf \ - --hash=sha256:d008d90a7f2471519aef0c90dfbe73b3e6e4d5e66ac48e19154c17e89e98b604 \ - --hash=sha256:d19fbd981a488e22cd04883659ca6b08f50b5974f9fd7c95655ef6a043e5893f \ - --hash=sha256:d1befeed746d247c81127bb14de9dc3d30edb6e5976d34f83f86ed262b1d9105 \ - --hash=sha256:d2374b62878abb00cd8309b32af6c0b715cd02dec0ca74ef12e5069bdc64144a \ - --hash=sha256:d376bbd28b3a8999db1a103b3b388aee6f1ddeb3e51bc2172993efdcd86e064d \ - --hash=sha256:d4a7319f304a774bed22115bc891618e45f85065ab44ea6acd07d274e750519a \ - --hash=sha256:d6734d2ef8a50fbf8445c139477da401f50d62a0606bf00e20ec6d87773fefb1 \ - --hash=sha256:d760fe2a4d7c3b226cb9026d6a842868d52a7901bd98420e1baf14e80da85cf5 \ - --hash=sha256:d913de495d90407cd859d263bee2e5d1a4ed3eb6573c04e70d9ec619a7cbed7f \ - --hash=sha256:db19d07e2e0129e974a0e65d0064fc222a446cd5122c2fd4184d2af9fc734a9e \ - --hash=sha256:dca9ab98072a5a54ebacebdc45f53e645336b320c667410b061be1ca588ae709 \ - --hash=sha256:ddc7dacc8ece3a182e7f15cb862d1fd616b46d076cb1ae9dd232b2c38b655874 \ - --hash=sha256:ddf19c062bea7a0cc80f519243d2c01dd091be0cf952a0750d4ad576709559f5 \ - --hash=sha256:def79fa35ef0cef8d2accec024f4fdc7ead3012ff02f5215c783f39f03ef8cfc \ - --hash=sha256:df29a0a7107f7011e77f4eebdddec4c7331e24d787a0b21a46d63bdf7445da95 \ - --hash=sha256:e09a3942ecbdee5cce73ea9d42da82b81b72ac1bf031ce069b93b5adf4eac8cd \ - --hash=sha256:e242bb1c5e76e97dfa9e7f209a71e93a01d7f19ffdd5cfbb2e2d55b4f08f8ab0 \ - --hash=sha256:e243bd13217235fc7290c621941c3f5cc8b66e4872495be821d7436ba2fb838d \ - --hash=sha256:e2af3aad578aa6bd1384bcf4750fc285e5a9de53f40b7d41e5a0bf748edeb2b3 \ - --hash=sha256:e4e81e09c1578b8df602e3db08b0b3ea0a6947ad612f52bf8dc5ea8d47691f0c \ - --hash=sha256:e54da4baf05720032d527874d40b65fa4d7e5c6c6a43d0c3adbeffcaf275a2b3 \ - --hash=sha256:e80e6c2f55656b4824d72065abb4ddd6a525c74bd78a0aab5d9fc2cf4fb5af50 \ - --hash=sha256:ed2a239c0ea213acc1908150a3037257083c7c083128f1a4cec2ec4b97dca491 \ - --hash=sha256:ed905975ab14056a2e5eb1c376cb2e1ebc5396baf84163939c518556fccde9f5 \ - --hash=sha256:ee21e28f0430bd6dc9086c6e525d5e818a44a5ad19720c8a0ef766792f3eb5e5 \ - --hash=sha256:ee43c17b173d46a3212baa6ead3ae258eeabdae48c263a01ccf0218c366dd655 \ - --hash=sha256:ef4fcbf3327382cd4c9f540babd61248208af7b93eec4de397b4d5f58a09e288 \ - --hash=sha256:eff0ac9dbe711a4aee69bf04a83896aa9b85f19641264053a9f6d48573abb7dd \ - --hash=sha256:f0aa869112ef88429ae17820d99c3dd9504c9e9c671d3c246f3d7442cb051084 \ - --hash=sha256:f3c96f633825733f735c5a9cf21d21a257d8e1edf0b1cee0a064b9c424ca0f7d \ - --hash=sha256:f5833ad231be5eb6553de524a70f48d71b2c8563101750531e0b80184e175cd4 \ - --hash=sha256:f5ec61164adcec446f8969a3358ec3f9b26bbda3b9213e5586d219afa8df2915 \ - --hash=sha256:f7d486c83842422badd511868fd8a9a20e9407ace71564b6af47ce7e60a336c1 \ - --hash=sha256:fb9e68df06293761f9fe66ade60a9bc6d0f5e42b8acf2939a9158af86ab0e5bd \ - --hash=sha256:fc14a032f813bf5fe624d991960ea83e9715adc27e4c1830a2361eb1d02ac341 \ - --hash=sha256:fcff63213e8e6e47770541a4607175404f47cbb3ebea7b6058cc82d524a0e424 \ - --hash=sha256:fd1fbe0f116b6e55da77aca2c6ddcddcfac2186cbf78bdebf40fc156efca389d \ - --hash=sha256:fe9753dfee015c570d73df76f899f18444d41388bffcde097deba51c4fadbb9f -click==8.5.0 \ - --hash=sha256:255bc9599cf7748b4b1a446ccc735421bd08a2ae529a8b88597d3de5664ee360 \ - --hash=sha256:ba0d2089de75ea0310e2dde03160e6ca10009947fb95a182f9b54021bb272e34 -cryptography==50.0.2 \ - --hash=sha256:0ddc924c04591c2811ca024d62ecad4f7f6f08af8939c211438f48a16bd23602 \ - --hash=sha256:0ec5f09541743261e66e291b4a0cbf0fb2997aeaab6d9e9c740b9dba1b58d1c2 \ - --hash=sha256:0ecbc5652bdb6fc9eaf89a7d196e20941adfe812f43bc4ca05d9150496821047 \ - --hash=sha256:1981f1db4630889b9ef7803fadef12b056f428cb6b85c27ba57b774793b6093c \ - --hash=sha256:1ba34f04897fcdaa73f74145c25f3ec146fbd56593853e88adc2e811303c5f42 \ - --hash=sha256:241449bf940a5d27309bd317e6f9a2af6932113818bb2b8f5c59ddc7ef16da18 \ - --hash=sha256:25784ce8b9621c90c643efb9e1e2162ab3b0224cae446ad5e70e7fcb1ce18b51 \ - --hash=sha256:3dc4fd8058cea1644971207d530e1a03a184a805ffc8ebdddf0599d78a331b81 \ - --hash=sha256:4061c0079120205fb760c58acab6443e217307dcf05e3702cf970e0689972856 \ - --hash=sha256:4a20ce1e5cb4284a86692fdcba7cb8754185c6b2e5c56fcef3751cf451d3cdc2 \ - --hash=sha256:4e81d95e5bafc2d6e34e4bed780e53e4d5b9a2f928573428aa4d35fbec1eb0de \ - --hash=sha256:58a0c478eeca76fe5e07993c5a0703def34a6dc6a0cda4f5564639b33112ffe7 \ - --hash=sha256:58ddb5a8e3179d12f19e4ea34d2d32e9d63a4baa142c875c1eb59f41b7243acd \ - --hash=sha256:630ebfea3bf689d075f82316324ff7433dc447fe6bc1bfc76524b74b4a9567d2 \ - --hash=sha256:6f8700550aa1474a91e5dc07049c46f98b423b5b1ddd0483e0b51362eeeaf5be \ - --hash=sha256:78198641e5be9521beea5aa782bb551a58068d10e6eb04c9c680c1b69f2e7d45 \ - --hash=sha256:79def8d059362e7831389ed3be0ecdf58a89386e1271e35dd9f5af84e81bffd0 \ - --hash=sha256:7a8701d6b584d76e909e3d305b7d126b41439876a5aaf76cddc67fc230eafa2e \ - --hash=sha256:7afa5a6602a9f29af1f3a2965f831bae7c9d5d597b7cbb716d41ab3b7d89879c \ - --hash=sha256:7b46165bb56eb4704e2eaaf86f3c940d19154535d9b0ca7d6d590b04060e00d5 \ - --hash=sha256:7b75de3c8b3be1cdb1052747c929440c3eea46c1bc2cb8a6e3a48388e9b7b452 \ - --hash=sha256:7c6d0330c472d96f6a6afe24d80dfdf15176c33096f0a4397ae4c60f3dd3be48 \ - --hash=sha256:828d49b0ff5a0e3975865571c5d91dbbdd0d38d8289b249a163e9425413a5e05 \ - --hash=sha256:84f964e537f916e2cc85199e5a88742e964939b575ac8598b3f9d6cc416cdaf1 \ - --hash=sha256:85d0d9a31b9098e98534226d5686b47264b95e62ce459dc2e62fdfc809f9fe93 \ - --hash=sha256:87e9ce85beb6b328ba370cc6e6aea483c92617b4c95b1d33a49297eb662bfb04 \ - --hash=sha256:8c71ba2cd31fc93748c38e1b613200ff1c2665cbfd5341fe3a61cfde35a1430e \ - --hash=sha256:92e665960f25fcdc73725b9cec7a3824f279ba97a98653afe9ffac2e43668f67 \ - --hash=sha256:94e5e9f108ee10471288214d3d233fbfbb492840a8457eb85178d643ddeb32c7 \ - --hash=sha256:9c8402a82ea0dc4ceeab793db05f0fafa8ca139ca34fcde5df0f596103c74107 \ - --hash=sha256:9dab55f57c74c3cad24c323bacbbd04be4705ba6eb0d92e920b1fc4837ed5079 \ - --hash=sha256:a582ab2ae1d34f67112cadc86702774c9ea4374df6bca6afe672817203c99134 \ - --hash=sha256:a6557e5f38e065ca9fbdaf7cfc7435ecb1d113aa81a022d1b51921ee7432e227 \ - --hash=sha256:a9f7355e6fab51f6c369b86fb7571cffa05edee2c2121e0380a37fb9ac1cd5c1 \ - --hash=sha256:ab50ee449bf968271e820086f10a33d101dd060370abc10bcd22279be2656539 \ - --hash=sha256:ac9ed99d81760c62fe89d5f0815cdfa1ba9a35141cf30f1c2d044f04b4803d2e \ - --hash=sha256:b13478603dcd0a2479ff8e87e2c19a7d525734686fe3c49542472293a204212d \ - --hash=sha256:c423ab384a46c4dff7217b2ea5ba2e11cffdeab6441acd04cf65a369caf0366c \ - --hash=sha256:c5e67125c7dca78d199ec4e116aa93dbb83494808ecbb8211a2cb09b1bf41dbd \ - --hash=sha256:c71be1cbfa5cd9a41ee452acf1eccd82b2c05950358b106ec8ceb83411d1a020 \ - --hash=sha256:cbc8738fd8526d80f35cb3a40d41f41a2e7030bb3b18b09a6778ef63d291c2fd \ - --hash=sha256:ce47f66801c20ec6c6632453bb5960fe38939e9306970b48b3a5a26de7745d94 \ - --hash=sha256:d370b8d1dfcdf7130178137f6fbee6140774a1acc6cacefc4b42643ec11d0a3a \ - --hash=sha256:d38cdff612d06fa6a32840d5e1b1f7a27cee4a349aa9085d94a67789d6bfd408 \ - --hash=sha256:d8947001be83df1394050758ce0e745dd74fb134eef0a4b5124208dfc3a68c37 \ - --hash=sha256:deb9fde5c60e437ee4821bc9bc39ff31b42135c27e1dc61ef0a629389c1de62e \ - --hash=sha256:dfe9763530994147d9af1def057a5b9658b00e8f8fe8743d144d1e0911c2e454 \ - --hash=sha256:e105ab60406787da31fccc883fc0f733af1efd78f0136a4599692c4083a73d0c \ - --hash=sha256:e275096ea1e60cc595cda2836fd4a6c725d1125108b868be17f53684d164e2cc \ - --hash=sha256:edc3342adf8f697fc5f59c887a304356f147b397809440ed64e2fa6af2f50f37 \ - --hash=sha256:ee247f5c245c9a2fe7c8e2214e295918838e44e00a45a6718451e4004219e767 \ - --hash=sha256:eef4c2f3423810b3070ab391f85436d2f8bbfcb286ac15cbc73190b3563b1f1a \ - --hash=sha256:f21e8a22c8605750c7af886bab299a363721264061b4ac0a30efb73cfd58efc5 \ - --hash=sha256:f265528741e048bce55c3463ed721fb0aa45a5888d8add8cfeccb3035451bbdc \ - --hash=sha256:f2f9bd7f90c64fe89253f0a2c05e3c4856072660429ce8831b4235bf29403a67 \ - --hash=sha256:f785f6161f202ab04d8ca194158968798e480ca058943907972da5f12e2881e8 \ - --hash=sha256:f9f6143a8c75945eb960d9eb98905a441394abfa24afaae239d514ffb2586480 \ - --hash=sha256:fa8f5efb344d6908a1ce62f4a24e2e5780f825d6f53f5f50ec5ffacac72936cb \ - --hash=sha256:fdd28f912fccfec1846a94e2e1e8f9b0012f557f0c46fe4f3eb0d7a87afcf90b -datasets==5.1.0 \ - --hash=sha256:1561347daa8841b51351ef2ad1647b4ee4af406852c0f7bd1df587ac1ca7888e \ - --hash=sha256:86fd08b280485aab82894d2c81842c4151d93b575efea0054315104cb7bc3cec -dill==0.4.1 \ - --hash=sha256:1e1ce33e978ae97fcfcff5638477032b801c46c7c65cf717f95fbc2248f79a9d \ - --hash=sha256:423092df4182177d4d8ba8290c8a5b640c66ab35ec7da59ccfa00f6fa3eea5fa -docstring-parser==0.18.0 \ - --hash=sha256:292510982205c12b1248696f44959db3cdd1740237a968ea1e2e7a900eeb2015 \ - --hash=sha256:b3fcbed555c47d8479be0796ef7e19c2670d428d72e96da63f3a40122860374b -filelock==4.2.4 \ - --hash=sha256:301241438c57936ef93f4243a94f2559022268bfcf9b375a9a72eb0f8db8358e \ - --hash=sha256:c6d8663535e1650ee430507b4ffbac7b464dd64b7647ea0112a9f406499d163d -fire==0.7.1 \ - --hash=sha256:3b208f05c736de98fb343310d090dcc4d8c78b2a89ea4f32b837c586270a9cbf \ - --hash=sha256:e43fd8a5033a9001e7e2973bab96070694b9f12f2e0ecf96d4683971b5ab1882 -frozenlist==1.8.0 \ - --hash=sha256:0325024fe97f94c41c08872db482cf8ac4800d80e79222c6b0b7b162d5b13686 \ - --hash=sha256:032efa2674356903cd0261c4317a561a6850f3ac864a63fc1583147fb05a79b0 \ - --hash=sha256:03ae967b4e297f58f8c774c7eabcce57fe3c2434817d4385c50661845a058121 \ - --hash=sha256:06be8f67f39c8b1dc671f5d83aaefd3358ae5cdcf8314552c57e7ed3e6475bdd \ - --hash=sha256:073f8bf8becba60aa931eb3bc420b217bb7d5b8f4750e6f8b3be7f3da85d38b7 \ - --hash=sha256:07cdca25a91a4386d2e76ad992916a85038a9b97561bf7a3fd12d5d9ce31870c \ - --hash=sha256:09474e9831bc2b2199fad6da3c14c7b0fbdd377cce9d3d77131be28906cb7d84 \ - --hash=sha256:0c18a16eab41e82c295618a77502e17b195883241c563b00f0aa5106fc4eaa0d \ - --hash=sha256:0f96534f8bfebc1a394209427d0f8a63d343c9779cda6fc25e8e121b5fd8555b \ - --hash=sha256:102e6314ca4da683dca92e3b1355490fed5f313b768500084fbe6371fddfdb79 \ - --hash=sha256:11847b53d722050808926e785df837353bd4d75f1d494377e59b23594d834967 \ - --hash=sha256:119fb2a1bd47307e899c2fac7f28e85b9a543864df47aa7ec9d3c1b4545f096f \ - --hash=sha256:13d23a45c4cebade99340c4165bd90eeb4a56c6d8a9d8aa49568cac19a6d0dc4 \ - --hash=sha256:154e55ec0655291b5dd1b8731c637ecdb50975a2ae70c606d100750a540082f7 \ - --hash=sha256:168c0969a329b416119507ba30b9ea13688fafffac1b7822802537569a1cb0ef \ - --hash=sha256:17c883ab0ab67200b5f964d2b9ed6b00971917d5d8a92df149dc2c9779208ee9 \ - --hash=sha256:1a7607e17ad33361677adcd1443edf6f5da0ce5e5377b798fba20fae194825f3 \ - --hash=sha256:1a7fa382a4a223773ed64242dbe1c9c326ec09457e6b8428efb4118c685c3dfd \ - --hash=sha256:1aa77cb5697069af47472e39612976ed05343ff2e84a3dcf15437b232cbfd087 \ - --hash=sha256:1b9290cf81e95e93fdf90548ce9d3c1211cf574b8e3f4b3b7cb0537cf2227068 \ - --hash=sha256:20e63c9493d33ee48536600d1a5c95eefc870cd71e7ab037763d1fbb89cc51e7 \ - --hash=sha256:21900c48ae04d13d416f0e1e0c4d81f7931f73a9dfa0b7a8746fb2fe7dd970ed \ - --hash=sha256:229bf37d2e4acdaf808fd3f06e854a4a7a3661e871b10dc1f8f1896a3b05f18b \ - --hash=sha256:2552f44204b744fba866e573be4c1f9048d6a324dfe14475103fd51613eb1d1f \ - --hash=sha256:27c6e8077956cf73eadd514be8fb04d77fc946a7fe9f7fe167648b0b9085cc25 \ - --hash=sha256:28bd570e8e189d7f7b001966435f9dac6718324b5be2990ac496cf1ea9ddb7fe \ - --hash=sha256:294e487f9ec720bd8ffcebc99d575f7eff3568a08a253d1ee1a0378754b74143 \ - --hash=sha256:29548f9b5b5e3460ce7378144c3010363d8035cea44bc0bf02d57f5a685e084e \ - --hash=sha256:2c5dcbbc55383e5883246d11fd179782a9d07a986c40f49abe89ddf865913930 \ - --hash=sha256:2dc43a022e555de94c3b68a4ef0b11c4f747d12c024a520c7101709a2144fb37 \ - --hash=sha256:2f05983daecab868a31e1da44462873306d3cbfd76d1f0b5b69c473d21dbb128 \ - --hash=sha256:33139dc858c580ea50e7e60a1b0ea003efa1fd42e6ec7fdbad78fff65fad2fd2 \ - --hash=sha256:332db6b2563333c5671fecacd085141b5800cb866be16d5e3eb15a2086476675 \ - --hash=sha256:33f48f51a446114bc5d251fb2954ab0164d5be02ad3382abcbfe07e2531d650f \ - --hash=sha256:34187385b08f866104f0c0617404c8eb08165ab1272e884abc89c112e9c00746 \ - --hash=sha256:342c97bf697ac5480c0a7ec73cd700ecfa5a8a40ac923bd035484616efecc2df \ - --hash=sha256:3462dd9475af2025c31cc61be6652dfa25cbfb56cbbf52f4ccfe029f38decaf8 \ - --hash=sha256:39ecbc32f1390387d2aa4f5a995e465e9e2f79ba3adcac92d68e3e0afae6657c \ - --hash=sha256:3e0761f4d1a44f1d1a47996511752cf3dcec5bbdd9cc2b4fe595caf97754b7a0 \ - --hash=sha256:3ede829ed8d842f6cd48fc7081d7a41001a56f1f38603f9d49bf3020d59a31ad \ - --hash=sha256:3ef2d026f16a2b1866e1d86fc4e1291e1ed8a387b2c333809419a2f8b3a77b82 \ - --hash=sha256:405e8fe955c2280ce66428b3ca55e12b3c4e9c336fb2103a4937e891c69a4a29 \ - --hash=sha256:42145cd2748ca39f32801dad54aeea10039da6f86e303659db90db1c4b614c8c \ - --hash=sha256:4314debad13beb564b708b4a496020e5306c7333fa9a3ab90374169a20ffab30 \ - --hash=sha256:433403ae80709741ce34038da08511d4a77062aa924baf411ef73d1146e74faf \ - --hash=sha256:44389d135b3ff43ba8cc89ff7f51f5a0bb6b63d829c8300f79a2fe4fe61bcc62 \ - --hash=sha256:48e6d3f4ec5c7273dfe83ff27c91083c6c9065af655dc2684d2c200c94308bb5 \ - --hash=sha256:494a5952b1c597ba44e0e78113a7266e656b9794eec897b19ead706bd7074383 \ - --hash=sha256:4970ece02dbc8c3a92fcc5228e36a3e933a01a999f7094ff7c23fbd2beeaa67c \ - --hash=sha256:4e0c11f2cc6717e0a741f84a527c52616140741cd812a50422f83dc31749fb52 \ - --hash=sha256:50066c3997d0091c411a66e710f4e11752251e6d2d73d70d8d5d4c76442a199d \ - --hash=sha256:517279f58009d0b1f2e7c1b130b377a349405da3f7621ed6bfae50b10adf20c1 \ - --hash=sha256:54b2077180eb7f83dd52c40b2750d0a9f175e06a42e3213ce047219de902717a \ - --hash=sha256:5500ef82073f599ac84d888e3a8c1f77ac831183244bfd7f11eaa0289fb30714 \ - --hash=sha256:581ef5194c48035a7de2aefc72ac6539823bb71508189e5de01d60c9dcd5fa65 \ - --hash=sha256:59a6a5876ca59d1b63af8cd5e7ffffb024c3dc1e9cf9301b21a2e76286505c95 \ - --hash=sha256:5a3a935c3a4e89c733303a2d5a7c257ea44af3a56c8202df486b7f5de40f37e1 \ - --hash=sha256:5c1c8e78426e59b3f8005e9b19f6ff46e5845895adbde20ece9218319eca6506 \ - --hash=sha256:5d63a068f978fc69421fb0e6eb91a9603187527c86b7cd3f534a5b77a592b888 \ - --hash=sha256:667c3777ca571e5dbeb76f331562ff98b957431df140b54c85fd4d52eea8d8f6 \ - --hash=sha256:6da155091429aeba16851ecb10a9104a108bcd32f6c1642867eadaee401c1c41 \ - --hash=sha256:6dc4126390929823e2d2d9dc79ab4046ed74680360fc5f38b585c12c66cdf459 \ - --hash=sha256:7398c222d1d405e796970320036b1b563892b65809d9e5261487bb2c7f7b5c6a \ - --hash=sha256:74c51543498289c0c43656701be6b077f4b265868fa7f8a8859c197006efb608 \ - --hash=sha256:776f352e8329135506a1d6bf16ac3f87bc25b28e765949282dcc627af36123aa \ - --hash=sha256:778a11b15673f6f1df23d9586f83c4846c471a8af693a22e066508b77d201ec8 \ - --hash=sha256:78f7b9e5d6f2fdb88cdde9440dc147259b62b9d3b019924def9f6478be254ac1 \ - --hash=sha256:799345ab092bee59f01a915620b5d014698547afd011e691a208637312db9186 \ - --hash=sha256:7bf6cdf8e07c8151fba6fe85735441240ec7f619f935a5205953d58009aef8c6 \ - --hash=sha256:8009897cdef112072f93a0efdce29cd819e717fd2f649ee3016efd3cd885a7ed \ - --hash=sha256:80f85f0a7cc86e7a54c46d99c9e1318ff01f4687c172ede30fd52d19d1da1c8e \ - --hash=sha256:8585e3bb2cdea02fc88ffa245069c36555557ad3609e83be0ec71f54fd4abb52 \ - --hash=sha256:878be833caa6a3821caf85eb39c5ba92d28e85df26d57afb06b35b2efd937231 \ - --hash=sha256:8a76ea0f0b9dfa06f254ee06053d93a600865b3274358ca48a352ce4f0798450 \ - --hash=sha256:8b7b94a067d1c504ee0b16def57ad5738701e4ba10cec90529f13fa03c833496 \ - --hash=sha256:8d92f1a84bb12d9e56f818b3a746f3efba93c1b63c8387a73dde655e1e42282a \ - --hash=sha256:908bd3f6439f2fef9e85031b59fd4f1297af54415fb60e4254a95f75b3cab3f3 \ - --hash=sha256:92db2bf818d5cc8d9c1f1fc56b897662e24ea5adb36ad1f1d82875bd64e03c24 \ - --hash=sha256:940d4a017dbfed9daf46a3b086e1d2167e7012ee297fef9e1c545c4d022f5178 \ - --hash=sha256:957e7c38f250991e48a9a73e6423db1bb9dd14e722a10f6b8bb8e16a0f55f695 \ - --hash=sha256:96153e77a591c8adc2ee805756c61f59fef4cf4073a9275ee86fe8cba41241f7 \ - --hash=sha256:96f423a119f4777a4a056b66ce11527366a8bb92f54e541ade21f2374433f6d4 \ - --hash=sha256:97260ff46b207a82a7567b581ab4190bd4dfa09f4db8a8b49d1a958f6aa4940e \ - --hash=sha256:974b28cf63cc99dfb2188d8d222bc6843656188164848c4f679e63dae4b0708e \ - --hash=sha256:9ff15928d62a0b80bb875655c39bf517938c7d589554cbd2669be42d97c2cb61 \ - --hash=sha256:a6483e309ca809f1efd154b4d37dc6d9f61037d6c6a81c2dc7a15cb22c8c5dca \ - --hash=sha256:a88f062f072d1589b7b46e951698950e7da00442fc1cacbe17e19e025dc327ad \ - --hash=sha256:ac913f8403b36a2c8610bbfd25b8013488533e71e62b4b4adce9c86c8cea905b \ - --hash=sha256:adbeebaebae3526afc3c96fad434367cafbfd1b25d72369a9e5858453b1bb71a \ - --hash=sha256:b2a095d45c5d46e5e79ba1e5b9cb787f541a8dee0433836cea4b96a2c439dcd8 \ - --hash=sha256:b3210649ee28062ea6099cfda39e147fa1bc039583c8ee4481cb7811e2448c51 \ - --hash=sha256:b37f6d31b3dcea7deb5e9696e529a6aa4a898adc33db82da12e4c60a7c4d2011 \ - --hash=sha256:b4dec9482a65c54a5044486847b8a66bf10c9cb4926d42927ec4e8fd5db7fed8 \ - --hash=sha256:b4f3b365f31c6cd4af24545ca0a244a53688cad8834e32f56831c4923b50a103 \ - --hash=sha256:b6db2185db9be0a04fecf2f241c70b63b1a242e2805be291855078f2b404dd6b \ - --hash=sha256:b9be22a69a014bc47e78072d0ecae716f5eb56c15238acca0f43d6eb8e4a5bda \ - --hash=sha256:bac9c42ba2ac65ddc115d930c78d24ab8d4f465fd3fc473cdedfccadb9429806 \ - --hash=sha256:bf0a7e10b077bf5fb9380ad3ae8ce20ef919a6ad93b4552896419ac7e1d8e042 \ - --hash=sha256:c23c3ff005322a6e16f71bf8692fcf4d5a304aaafe1e262c98c6d4adc7be863e \ - --hash=sha256:c4c800524c9cd9bac5166cd6f55285957fcfc907db323e193f2afcd4d9abd69b \ - --hash=sha256:c7366fe1418a6133d5aa824ee53d406550110984de7637d65a178010f759c6ef \ - --hash=sha256:c8d1634419f39ea6f5c427ea2f90ca85126b54b50837f31497f3bf38266e853d \ - --hash=sha256:c9a63152fe95756b85f31186bddf42e4c02c6321207fd6601a1c89ebac4fe567 \ - --hash=sha256:cb89a7f2de3602cfed448095bab3f178399646ab7c61454315089787df07733a \ - --hash=sha256:cba69cb73723c3f329622e34bdbf5ce1f80c21c290ff04256cff1cd3c2036ed2 \ - --hash=sha256:cee686f1f4cadeb2136007ddedd0aaf928ab95216e7691c63e50a8ec066336d0 \ - --hash=sha256:cf253e0e1c3ceb4aaff6df637ce033ff6535fb8c70a764a8f46aafd3d6ab798e \ - --hash=sha256:d1eaff1d00c7751b7c6662e9c5ba6eb2c17a2306ba5e2a37f24ddf3cc953402b \ - --hash=sha256:d3bb933317c52d7ea5004a1c442eef86f426886fba134ef8cf4226ea6ee1821d \ - --hash=sha256:d4d3214a0f8394edfa3e303136d0575eece0745ff2b47bd2cb2e66dd92d4351a \ - --hash=sha256:d6a5df73acd3399d893dafc71663ad22534b5aa4f94e8a2fabfe856c3c1b6a52 \ - --hash=sha256:d8b7138e5cd0647e4523d6685b0eac5d4be9a184ae9634492f25c6eb38c12a47 \ - --hash=sha256:db1e72ede2d0d7ccb213f218df6a078a9c09a7de257c2fe8fcef16d5925230b1 \ - --hash=sha256:e25ac20a2ef37e91c1b39938b591457666a0fa835c7783c3a8f33ea42870db94 \ - --hash=sha256:e2de870d16a7a53901e41b64ffdf26f2fbb8917b3e6ebf398098d72c5b20bd7f \ - --hash=sha256:e4a3408834f65da56c83528fb52ce7911484f0d1eaf7b761fc66001db1646eff \ - --hash=sha256:eaa352d7047a31d87dafcacbabe89df0aa506abb5b1b85a2fb91bc3faa02d822 \ - --hash=sha256:eab8145831a0d56ec9c4139b6c3e594c7a83c2c8be25d5bcf2d86136a532287a \ - --hash=sha256:ec3cc8c5d4084591b4237c0a272cc4f50a5b03396a47d9caaf76f5d7b38a4f11 \ - --hash=sha256:edee74874ce20a373d62dc28b0b18b93f645633c2943fd90ee9d898550770581 \ - --hash=sha256:eefdba20de0d938cec6a89bd4d70f346a03108a19b9df4248d3cf0d88f1b0f51 \ - --hash=sha256:ef2b7b394f208233e471abc541cc6991f907ffd47dc72584acee3147899d6565 \ - --hash=sha256:f21f00a91358803399890ab167098c131ec2ddd5f8f5fd5fe9c9f2c6fcd91e40 \ - --hash=sha256:f4be2e3d8bc8aabd566f8d5b8ba7ecc09249d74ba3c9ed52e54dc23a293f0b92 \ - --hash=sha256:f57fb59d9f385710aa7060e89410aeb5058b99e62f4d16b08b91986b9a2140c2 \ - --hash=sha256:f6292f1de555ffcc675941d65fffffb0a5bcd992905015f85d0592201793e0e5 \ - --hash=sha256:f833670942247a14eafbb675458b4e61c82e002a148f49e68257b79296e865c4 \ - --hash=sha256:fa47e444b8ba08fffd1c18e8cdb9a75db1b6a27f17507522834ad13ed5922b93 \ - --hash=sha256:fb30f9626572a76dfe4293c7194a09fb1fe93ba94c7d4f720dfae3b646b45027 \ - --hash=sha256:fe3c58d2f5db5fbd18c2987cba06d51b0529f52bc3a6cdc33d3f4eab725104bd -fsspec==2026.7.0 \ - --hash=sha256:b57ddbafedfaef7018c1ecab32aa200a9d7ca26b77965f64e48b70061249d279 \ - --hash=sha256:c803c40f4cf860b49dea58ee3e1c33cb9c790520e233537e1340049f89b82a88 -google-ai-generativelanguage==0.6.15 \ - --hash=sha256:5a03ef86377aa184ffef3662ca28f19eeee158733e45d7947982eb953c6ebb6c \ - --hash=sha256:8f6d9dc4c12b065fe2d0289026171acea5183ebf2d0b11cefe12f3821e159ec3 -google-api-core==2.33.0 \ - --hash=sha256:3a36bcc3e319783f4c97da41f6f45ea6ffcaa55848e341de16e09cb70243c2bb \ - --hash=sha256:a2e22a0c1d0f03eafff1858b38cf46f832d5902b0c052235bf0ab8402929fbdc -google-api-python-client==2.201.0 \ - --hash=sha256:2d9bf1ba3f12eee8ed3d0f1791ce0605d163432f496baa72d3677faa2cf097d6 \ - --hash=sha256:d5691982abd7287f53cb0b0e0c6a9984d4103cf864ea0a88cb6e4347bbaf70de -google-auth==2.62.0 \ - --hash=sha256:0bef0ce54bdf9ce226c5d66e4264413bd918141c31bbe49fb52eac882f513d69 \ - --hash=sha256:4ff4319aeb4ad128409759d397a9fcafad126d0031d241cc0dd6b9a00b43e3f3 -google-auth-httplib2==0.4.4 \ - --hash=sha256:b931de392c20cfaa351cd789274922bd8cdc001e0e9e96de31b39d71347f8e16 \ - --hash=sha256:bbe5d7b2401bb3a4017f4720e1e91bd273ab9a2bb60b84e65edbc0de127852da -google-generativeai==0.8.6 \ - --hash=sha256:37a0eaaa95e5bbf888828e20a4a1b2c196cc9527d194706e58a68ff388aeb0fa -googleapis-common-protos==1.75.0 \ - --hash=sha256:53a062ff3c32552fbd62c11fe23768b78e4ddf0494d5e5fd97d3f4689c75fbbd \ - --hash=sha256:961ed60399c457ceb0ee8f285a84c870aabc9c6a832b9d37bb281b5bebde43ed -grpcio==1.84.1 \ - --hash=sha256:0225c1e29d1506594e75ac46fd7b327bbfd24c6306a3c479b2164c1c4e87901f \ - --hash=sha256:02612f3c19ed3dd885e2fff1236bf5629e5b43b5aa84c3290b0d5f62f26b0ef7 \ - --hash=sha256:0b3d2b21939a6a773d1a692a1ac8564ec31702b752b38b913eb5b1bcfc280d55 \ - --hash=sha256:116b4796da0e7fca3fdf6edf422ea7521c25d8c4be64ab7e3b5e2a77c832b0ed \ - --hash=sha256:11c1d07c58922f5dad022b422d3f7998755c3e35e2f1949bac47cfd7a5eb31fd \ - --hash=sha256:1a221fe42e444cd0099752028810a289efafce1b5690dfe19a831d2a2fe6ea62 \ - --hash=sha256:1e25f45414b3a53bfc1fe43b980f9d0cae9e194a62541d5f6bef0846758162ac \ - --hash=sha256:28caf3df58096e2cb06f0f7e672160e65e9df5e053f392594d93d33a15fe1ef1 \ - --hash=sha256:2ce575e400e0dec9484f5d038fa12d6b3919d493c2beb897a23b9a32fa3bfd12 \ - --hash=sha256:380b58bf32707b4e3019c9567dab30375bd21f9a5c98aa24401aff1e73f3b262 \ - --hash=sha256:3875f4c4caf0c54ac370b66322f0bf2d46e9d21414be36ea8b7f618badb3b87c \ - --hash=sha256:3bb28f8c1911de119135adbabb21b6e6563e7a5c9fc950c6b8ab2a387afb4372 \ - --hash=sha256:3c5b41ac9a337f5633d9474b9ae6b1b7f2f4e3eca024a3855536d6980e817d43 \ - --hash=sha256:3cf14e9cf02d12628f20ec205d699838c47e2d3741690fbf14ae626bfa32ac0b \ - --hash=sha256:4052f86fdb5105ac8b02c77d17cd9beff78790a0c3117afdd5316154cf4348a5 \ - --hash=sha256:42ffbbc11fe2e4b02a4b51b090f8ca6727414748cf551539665b5592372263e8 \ - --hash=sha256:43d8b84b3e2b7505e4902ab6288ceee699cfec5435ed879bb53f876b2966ee33 \ - --hash=sha256:446e7876b9b991fa124c1d414899fa7cf95f89a3e71237ea0161863dc0b2c09f \ - --hash=sha256:474408c92b27c823ea6aab2315e2ea52d5a4e0040b37ce1e65e89d9149c48205 \ - --hash=sha256:4aaf596ea35c7d022a3bfe0402cdee30d86c6ccea2b6fbbe03d1e4826cc7179c \ - --hash=sha256:5842cc52e13a5cf0d1beced73e43276a41e22c0e8a8f9f7c606536efce187e5a \ - --hash=sha256:68ff21d6cee6f38f7554cdd263e6db25c95a3efe20726350ab4de4a81808be9c \ - --hash=sha256:6a22342e59b5b2f4c5ed276ef72a3668ec84f38d2f5fad8a64af0b3618e510eb \ - --hash=sha256:6a4a06d757e7ea5ddf636a4c40914aa36fb4f7e2a03abf727a16e61822331343 \ - --hash=sha256:6ce0835704f30d4954aa0a68352fc9cc30242cc85f39d7b6cd20c0790e05925b \ - --hash=sha256:772ff116d0d31b2817e23fb9a1dfa52fa9fed08be0cd87b593450f034e2968b1 \ - --hash=sha256:775f98cb476625936061b91e3c1da229112b212a6b2a70fefc9c87e5aaf9fba6 \ - --hash=sha256:79b3ee4683290b3cfabadc718a008c3474073b585bc450231a842c21c2ef1f27 \ - --hash=sha256:912694cac76adb4f45563b38ae4d06d77a2081e8ed5cb20a3918c1affba83c63 \ - --hash=sha256:9c24b1ebbdf867b33d74868044fed5c7c5a6a6e00787140b5f582572f7a7e633 \ - --hash=sha256:a1ce9e360d0a4a329b925ae8b0bc49d97d663bcce50822eb862873798a86eb10 \ - --hash=sha256:a37234085137ba26aa89bb377bba839894616eb1ed3873e1cc4a1badce885880 \ - --hash=sha256:a5ea4e8d1e9745ba6c2c9d14ed0861e9496a8051bf9fd8a17b667d9b9a0c1019 \ - --hash=sha256:a92bb13dd1b31a16f631ab415710de59f404c37364c018e7486496fb449a89da \ - --hash=sha256:af0f0c10281c2b62c68868e0a98de24f89c31308c68dbf19320d30c3a329aba5 \ - --hash=sha256:b7452ccc592e40e4167adec46195d85387c8036f0aadf6f96ab6ed931205d0ce \ - --hash=sha256:b87f8696bb78a31328e815629d320304c3d5cca1c654d3fd7e5518685bcfb8d2 \ - --hash=sha256:b8fb04fe0ded6fb1c5edd4c90ed39193d7b6226214685111b25a153ccb33a52a \ - --hash=sha256:bdd1181edaf42b6e79a457789fd43e85858db1df3861e4e5128ff4d6a49cb638 \ - --hash=sha256:c1c04d2876ccb063507a976602bc88b5c31330982e650c620bf8e730b120c9d5 \ - --hash=sha256:c36f794a507bbda13e1b656e4fba775a08f3ca0a1e1be5f65ed47ceb0db47812 \ - --hash=sha256:c61a0d5c60506b2371444716b56d6c633d3c56a17c703566bcca39604c02c12c \ - --hash=sha256:c6e9be93efe763bb2b12a4c0d5fc13082f24de6cbb53a9d11078c9d0c8cdb2b8 \ - --hash=sha256:c8ae49802f09cc272546592b3656207de6f719e50a6a0d1654e613aaba6336ac \ - --hash=sha256:c8e1c15e81f65efb317906754fdba4a85d2eac9cff250d8a3fb3ccb3a4e52db9 \ - --hash=sha256:ccd8be89a73d276336cbf223b22bd5c677d9fd15233100f208845c6a0c7cbc72 \ - --hash=sha256:ced0c5ef2709964aca16e6af6f74f682b57872d2f7c7c553916c6b86dac64fa4 \ - --hash=sha256:cfdc7530e5707ab6e5c80fc624f89e26ddf8f38d3e89e5226dd7e55c080e9b65 \ - --hash=sha256:d82bfc0cc4e9120a52f7657caa6af1727a79093612bd7e91a8444a29e4df21de \ - --hash=sha256:db2dbe5380dccc2f5df967f31308a744e65d46e21b473a9c3ca033c4299ec3f4 \ - --hash=sha256:de3c31abc0d052511744a817506e03534f14e32dadab8ec19f63f2d4facd05f3 \ - --hash=sha256:e0dcc7b588e7b7dd3363317296983db23a26af8baa5884db15a10fc0c8534186 \ - --hash=sha256:e44786d5c166944a53fb1fa3384d03f1dbade6f68007be6d0c5a1a0971cbaca6 \ - --hash=sha256:eb881326607275ad2d2d17dce09b823472414dcc559e8f694d5607fe8af3cbc1 \ - --hash=sha256:f216832aa84ad7e697ae1ed993c33905d27e27cd1b02016af1441be2091e287d \ - --hash=sha256:f2482ebe5e3a9c6d1f99859e135cc0ca370362e3b2a94cb6304f64943997bfcb \ - --hash=sha256:f4b64099afaccdeeb987240af47aa1fc14f57285d3d2ff000c7327228624999a \ - --hash=sha256:f6d12757eea5b6b642a171e7701ecfb360ce0bf13d262dad1bfdebcd267bcaf5 \ - --hash=sha256:f7533921b52ef94210437dedd3911e00b7642ca8d11eda64eb1715778630715e \ - --hash=sha256:f821954e4d89e97f7d51c03073d6964e177f2d11181550a5a22c082fac614f43 \ - --hash=sha256:ffc0be893d8df55f0d23afbd3b776ede444847260a4a00ac80326a32dac91ad3 -grpcio-status==1.71.2 \ - --hash=sha256:803c98cb6a8b7dc6dbb785b1111aed739f241ab5e9da0bba96888aa74704cfd3 \ - --hash=sha256:c7a97e176df71cdc2c179cd1847d7fc86cca5832ad12e9798d7fed6b7a1aab50 -h11==0.16.0 \ - --hash=sha256:4e35b956cf45792e4caa5885e69fba00bdbc6ffafbfa020300e549b208ee5ff1 \ - --hash=sha256:63cf8bbe7522de3bf65932fda1d9c2772064ffb3dae62d55932da54b31cb6c86 -hf-xet==1.7.0 \ - --hash=sha256:0a9e802f33bf50c851abe45fc5380e61f959e2d369647d6742b79ad9d6c27cab \ - --hash=sha256:19c0e64f14175ccb6a1aff69e0d2ab9ec5269a560e6687abaf2b3fa4f73de7cd \ - --hash=sha256:2814a6e999d13464c4d679b788cc5d784eb5a4edfc638a31f10e9a11ab531ef8 \ - --hash=sha256:2b7bb5727889b0f2436dbaaad8fc4c3e66b8240d992716989e0c086b4278b1bc \ - --hash=sha256:4ee5e05a627f5ab5bad7a86582277d645556ea1e199903aae19e033a392aa13a \ - --hash=sha256:57bc157b8b7fe3bee9dcb9af7f3da8de41801c3b31a9ef68a77a33c6a6be382f \ - --hash=sha256:59fba37039233c7fcbe196817d6cdcf1b40dfb17b410f229d85b0cf0a1848da4 \ - --hash=sha256:757168feb5679647c0bb13ee5d0faebe799c4dff9051419885a566ebd79f949d \ - --hash=sha256:80f79dae613ce9e0ea1fd1ae15616ca9ac74aed4c770aabc199c4f03ebecc863 \ - --hash=sha256:87dab080f8f7d32781c2586904e3603f4e60d09bfc727706c3ae419e0829beeb \ - --hash=sha256:acc3851cf2576a8fb2ae926da863f4efabe21303cf292e9a44332802ab0dcc6a \ - --hash=sha256:b01fe18dbbd151a2403d2c64ed30dc6547b00d6babab9a617d77c7acdb81ee66 \ - --hash=sha256:b91569d5f1b61c34b043687da02c05dd3604f3d329e7868510bf3f7971599006 \ - --hash=sha256:d406ec79053c0871817f700c2ac8c36ba0d87f9c34b7458b0f0063bb218b0466 \ - --hash=sha256:e3e88a7a75d7d95cbee1f37dc31341d6201124cf21c6c4b1dfab8ccba9b09e0f \ - --hash=sha256:fa029678be1ba7f953c409b0b27bf15cc69cd1c9b3a674fbd78856ebefca1052 \ - --hash=sha256:fcfd6c22418e57dd5b3aea649e813b2e2cfb2aebf317b210d90f1fe4b3018b52 -httpcore==1.0.9 \ - --hash=sha256:2d400746a40668fc9dec9810239072b40b4484b640a8c38fd654a024c7a1bf55 \ - --hash=sha256:6e34463af53fd2ab5d807f399a9b45ea31c3dfa2276f15a2c3f00afff6e176e8 -httpcore2==2.16.0 \ - --hash=sha256:5c4358319f2602d984ad93182c9fee5160f22e9da7babdda870a5e7c4e2dc34f \ - --hash=sha256:c72cd0413563b17e1cd8354677dff887466f8ec2551679759e93126534428643 -httplib2==0.32.0 \ - --hash=sha256:48a0ef30a42db65d8f3399045e1d09ab0ba66e3b9efc360d07f80ea55d286025 \ - --hash=sha256:dc6705cacdf3fb0a2aba7629fa33c90fd93e30035db0c157325826be177e4816 -httpx==0.28.1 \ - --hash=sha256:75e98c5f16b0f35b567856f597f06ff2270a374470a5c2392242528e3e3e42fc \ - --hash=sha256:d909fcccc110f8c7faf814ca82a9a4d816bc5a6dbfea25d6591d6985b8ba59ad -httpx2==2.16.0 \ - --hash=sha256:0d54c98128e8f2ff7fbbde0fca8c359e5f032edeaaaac443cdb7f31f685bcd7f \ - --hash=sha256:fd2c5d7909e8d86fe92d53020c310ca0e4e431b838c207c5754b9bbf0308d49a -huggingface-hub==2.2.0 \ - --hash=sha256:1667f145dc56dc210d60966069397df9ecfca9607a5d43db88b308c89dae56b3 \ - --hash=sha256:5d1b47537394e4215cb858aa12fd493d0f7ef7f58990f5dcd24bc173107b2871 -idna==3.20 \ - --hash=sha256:a7db850025b95ded1eae8a46181a1a6c56c92c96f0e2b005d9ff8dc0210cab44 \ - --hash=sha256:ab7ae7122974553370f0bdb919e1a960b2cd1bc1ef0276416d896db81c14582c -jiter==0.17.0 \ - --hash=sha256:00b5a98df3e3a3e8cf7b619f4ac2f8bf975bbf3d95d02c5d17b8dbfe5c8b8245 \ - --hash=sha256:00d783a779c5664e16dbad5e3a3c3a75e128b07dd5f4765159658d9210a50ca5 \ - --hash=sha256:0239520085cac678e77a606fd7e3f1c60c371d719790c5e3807388d3da4354c2 \ - --hash=sha256:02a360707033d8cef53f7f3480817a1489177a259ec6ec01e98c37e0b922ddca \ - --hash=sha256:02adebb7ce6413c44d40af9ad59d1c1cd79630ccdcb6f7bdd2d461e48c03d8f9 \ - --hash=sha256:03e432f226a453851079fb84cd17c6da9991eab723e28d716f14ae3d906e0c12 \ - --hash=sha256:0619d806e260ecf0c2a64521942c94af5d547c9ec99b55ae4f51b538b5576a76 \ - --hash=sha256:073dc68c1a700c8fc480e877864a6b6ffc887533e261f4380c08c16bf09d057a \ - --hash=sha256:0b52d52035b3907c5b1f6277857b29c1cbfc965e24e0f27330dbed83edb591ec \ - --hash=sha256:10c5349312e5cb02b7a21e123a57665afa895953f05bf252a9dd4c13a572b7ab \ - --hash=sha256:10cd64a5720ad7f809ac5466ff1705813f1b6b510f195a73acafba0ac0e1f675 \ - --hash=sha256:10f5558eed511b830488003449d942bd75829ad6257dc58cb9a03e596a7777b1 \ - --hash=sha256:11902505d401691720f5785c15b02204248526edee11b635cd6c40cd52b81599 \ - --hash=sha256:155be7355bdb7ca76ab0961be8982c225f964a5c073a83984183f22391cc29fc \ - --hash=sha256:16dd0c1baf098ae70b8f3616574eb3fedf34e26670b89e16a7e67561f737ed2d \ - --hash=sha256:1b18434638228c0c184281609bf3d9459026a0f1ea48fb76c205e3ef72069caa \ - --hash=sha256:29f49b325e0234e4ad9ecca5b861ffbd09b95ccac9bd46fa55841b6e56eea5fe \ - --hash=sha256:2c45ad7c973ef33fe5114a953377b35a95240f4542c0724d9f781e47dc24bac7 \ - --hash=sha256:300ce01ab0215e3dea4d00090143c909aedc65c0f809b3c07983e1d038f291b9 \ - --hash=sha256:30793a24a31e968969757c9e08d830cbb15a2cd3c4959b4498b38f4b1c2258eb \ - --hash=sha256:30c692d567ba206c7cca38c9d1d0ccc70c9786290173c184d871ca12e9981ed7 \ - --hash=sha256:32aaaa764604496610a3ad2d98503ae88ccb2fbe769e892ff4533e778e85f708 \ - --hash=sha256:362bb47423886d45a9f705d2d9d4008c6eedd4e41eb1bab4e96fb6daa06b33fd \ - --hash=sha256:36ee6e69027396664e59995b9a635a947a5304ee9837279584a0bb8145c8f6b8 \ - --hash=sha256:370d8fe5bf201dc6925e8a84c81ac7291f74d9fd1778234fc79d517064a5c76b \ - --hash=sha256:37150a9e02e869475854fa20b7d0d5e26d18d0f8bc17293999973ff27e99ae7a \ - --hash=sha256:37f33d327900bf2879613b3363fd48df97b4232d0c41f54bcf2e790c2fc40a71 \ - --hash=sha256:3ad556afc289f15d2b181b941982d01f06190863c07440185b9f354e1bd2def3 \ - --hash=sha256:3bf4dc2b84a464117fb097d15a25c58d100d2692888e3b0d92df5b48ed16b7c0 \ - --hash=sha256:3c1a5336c04a41b1f1cf9572e294aec27cc569767ff73de7bf87a91f0bea7cb9 \ - --hash=sha256:3e05f5adbf68c4bd11e1610f394034d984152988e84be6f8314235ce6f2139e5 \ - --hash=sha256:40d2c240f8f80b5b0f201b29f0ae129c81448c60c772227a41747b5e0026f6a2 \ - --hash=sha256:42b0260445251b1bc520a63baa94a32d88e0f931fba234f1764db7feb7c72174 \ - --hash=sha256:454c4997d73cc466c71fd565d91e603b0274e48ea0c6b0b7a7aee6967e4ceb7c \ - --hash=sha256:455e4ab35cb2a4a91a8404e08fd3c621bae433922e59bf1c494fe20a426b013b \ - --hash=sha256:4607ec7d93355fbc25b8dc5189153cf21d66063b9f9cd04dd2774e6e783f9b6a \ - --hash=sha256:470e1b1e4c42f1ead2189166a299691871a2df5056c976e7fb96feafaf5f9d44 \ - --hash=sha256:492f37230bbf9581ab2c17bcda862c249afb9ae2e3ab2dd6db59943bc4cc3153 \ - --hash=sha256:4dfbfe5a6e1e80a7082af559f66386405025ec278833e0c649f69cbc6e1004cc \ - --hash=sha256:4e3f052c671d5f425cca5ea5901cf11a831369fba4a55a3862cab93c323b4c3b \ - --hash=sha256:5078ab00664307fab2019b522a93aeb191122789f085daf5fd9e362154021d4a \ - --hash=sha256:51e1519d676a9f14dad9c2a411170d43b022ddb7989562df4e849b261ce127b2 \ - --hash=sha256:523c499235fb65add25d4bb01b1c4709ce695efdc7deb6c0a7bc515b5c44e0fb \ - --hash=sha256:545c36a0f3b2238c242cc9785439d3242a871b7bc39fe3f441bcaa07bf3aa83e \ - --hash=sha256:55d0e0e613a3f9ad600cf436e0e2b8057d1b52bcf1d91b2d36ac53451231e6a8 \ - --hash=sha256:5888fe5abc1ca2fa834a3e1b4c7ef0dcece286a7d7e95a609ef0934b777b9fc9 \ - --hash=sha256:58df29268a95e910f17db7ec9178eb7f15aa8619aaca3575275c4e6b3f4fe4c5 \ - --hash=sha256:59bddbe6f9ffecc68d641e1e2d619ce64cf8a9e9eeb74e5c518f74fc87abf1b0 \ - --hash=sha256:5a52a430d04225ffde633e6840bf2381d34c019ff98526b5929755b9052fb199 \ - --hash=sha256:5bf350452a43173e69e1fc74847c57a60e3d7515807287f29849baa2a85d8718 \ - --hash=sha256:5c23849235d2142ce444b2b8c6eceee9f82f4cc0bd5c9081602e4155c6197807 \ - --hash=sha256:61aed66ee042b3b49ef85fdf75714234d055d89d8496ac1c6e47f89e7a30d5e4 \ - --hash=sha256:6219adaf59711ba7063a52496e8ec6d3fa3e209d7827d83eee3b2abc780a1744 \ - --hash=sha256:64846211a2debe7c071d2146d2283d2b0c1c93dc8fd5fb7794faac2ca6061b5c \ - --hash=sha256:686c93d86f2b426c803024b805bd161a6cd10e9627c23e901640eab646c0ad8a \ - --hash=sha256:6871973bfbd4408f7f1c632b30bbb5bbd9671c1bc8650af6823e24b7be13709b \ - --hash=sha256:6af5b74073bd25bae695e6d00919f6a9be7ed5a9f8836d981eb1ffe84139e6fb \ - --hash=sha256:6b303d88e6a0bda789ec4b7801c7bad68e27230ba1fe4baffc756d1fbd32dc9d \ - --hash=sha256:6cb41cd1432f1dc19a231cf70b54d42b2c9f05085155859263fce06fa4d41388 \ - --hash=sha256:6cf564d43c4388149ca58ee571d0f5ccf875e20d1fd4662fd94cc0d1ea3b10ef \ - --hash=sha256:6eb6aedeb7352b8f3b6af9cbd67983840165c00428e63f1b420a85885128ea31 \ - --hash=sha256:70f19a2ca8429f91e82eeffb2f51cb87bc2d6e953b009b91a92d29c3a16ccb03 \ - --hash=sha256:71dbd74314c5df52a1bccf7b8bca46d14e943af7a2012e73b23f49977ef194c8 \ - --hash=sha256:73b64e69c4150748e020356d958af94bec33c70a0a93d665cfa8f6d580fe1a63 \ - --hash=sha256:746243a080b4ca790b8499af3d7cf9825d5f5987933950cd818e767ee353d826 \ - --hash=sha256:755079792868ce5d4938e83b91a0939b34fb858a1ca65a104f2d771bea57faa1 \ - --hash=sha256:7573e80232c5bcf80c24c038cf7e53a463f5c3b1dd1dd4109d66304f4dccc233 \ - --hash=sha256:76eb4a5c20e86f9f848286f167024890f2862258a965d254774deb7fc1545ca1 \ - --hash=sha256:77f6aac0137309b31448c1bdcda4c6c77077664a6d018ece8d94019c68a5a5b9 \ - --hash=sha256:785a216bbaf8f15fc974e964ced7322cd3d774bb0e86949edd78c6bffd6ba35b \ - --hash=sha256:7b68d3495d95da120651a5628c7ebadee84ed001a1b76e6afc325c42482f15b5 \ - --hash=sha256:8079849db9a1371bfd90bad088458a8fb836261879df2233cc9632464ecf64e1 \ - --hash=sha256:81c83c0abe614446a283d994d2c07c4f58632dea2cdf66ba9e2921bb8ccd593e \ - --hash=sha256:826871c42cebaae22f0a2b5673a4a1a75c851bb2d13b3c17764a630a6b298984 \ - --hash=sha256:84963d3f395ef5e9a32ce47155e08a7962fa292c159a10cb98b931cef1416925 \ - --hash=sha256:84ac78df457e1ee3f7e733bd114823302ae8c5ad5542d7e6647d92ffaa090a04 \ - --hash=sha256:86d703d9faa1ffc8ae4e9de0fa007712ed2171b5c0d93811a8e2e105ac729b0d \ - --hash=sha256:86f3f9343a288eb85a81ef20a752b2f84564296636db54a9fff0b5c8deaf1df2 \ - --hash=sha256:8adca2e793288e5f1bb29279bb439d0d3cfbb50eddca7e7e6ffd42ff4f482406 \ - --hash=sha256:8c21265b251d99bbb40080d178a8953e35601d3a1564e05c4de4c0d2ca616797 \ - --hash=sha256:8c286860abfe8b100cac1c02e225e5776eb9216edd71ba17cdb237da4af32bc9 \ - --hash=sha256:8f770b0c77e5fac482e1ba03ca1a7e18286bfb213d749932a00a7e4cd5de5e06 \ - --hash=sha256:93946d89fa04d5ba64dd323a8dd8d901676cb8a3c81d99ae4f6c051a9b4c3f2f \ - --hash=sha256:96b8b0c6dc5d78682f54a450785e075aa929cde768304cad363cd4efba5a82ac \ - --hash=sha256:9bd3caac219df476dd0cc3fe01d2f1581ed588906feac767abd9614c1c12f8b3 \ - --hash=sha256:a277f97eba7d66b1ee27eb5dab5b774ff46a10c78d89a1d3dcce04ce1357c8ca \ - --hash=sha256:a3cebb1fe4a1abb00465f3f8a17e09112603e8b7c59e5c3adbcd9f7815a64acd \ - --hash=sha256:ac3c6ee3264d6f5c44c617f90bc7e8b9e1587e7d6708c9d8f811cb65582ee312 \ - --hash=sha256:af2f7501580f274b63c4b2283bc425f5df7edf06ae5b171e5f87d912ff359a20 \ - --hash=sha256:b550585523339b71cb852b811aae49d08d7601ad8ffe9f5dc1562f4c3d22fd87 \ - --hash=sha256:b75f85660108965a94be77911a25a253429307294d9415b3c597118977a614de \ - --hash=sha256:b847b18d066c46b3b7ae49d6c94a7634c5e4a8983146ee25562a092000f5e3ad \ - --hash=sha256:bcc064f99183a9cbe7f26ed648c352031a74145cd61ed75d34632c73eb46a5a8 \ - --hash=sha256:c19b9357309b8cc6de8a48fca8e44a8c9c2feaaa2f5896d037fa505d48fcab80 \ - --hash=sha256:c4289293e5278d9314b00f15c37f2120fa51d3d68565292e715524c750e775a9 \ - --hash=sha256:cfafd7be8b16ceadd298db542cead37cddc211c4c49e04ad2596924df18625b1 \ - --hash=sha256:d0ce4feb52493e3513335b2accdcd75605652e4632772d3c8c2f7b86954d7f39 \ - --hash=sha256:d2c0bf24c72fd0491405dce5d40194f2070e9021ce648c1a1d46234b93d848ff \ - --hash=sha256:d47687806f9c54c84ea38733507081337922beca90ce819c7d852dd485bc0f23 \ - --hash=sha256:d85c558c9f8532bba287a990ac63767c7daf756f0d8c030219f62499b1fa228a \ - --hash=sha256:da139721f4b7cafdbff580a4f511ea24cb91f4909330c6b926a1ca53836c0a59 \ - --hash=sha256:dbbfe4e3c21c8166980cddc5bee1a315df082454f007947dfb6fb73800768165 \ - --hash=sha256:dc0288ce39190ee33fe6e4ec73161eed34e7e2da509b525546ca061778d62b64 \ - --hash=sha256:e088612ff90ebc9247e1a43074b72835804261c47e6a6c01cb3ddcb55360d688 \ - --hash=sha256:e654b6b04e39c9cb19cb8b04c6ddf1f2db07751fa14156413969fd78bad0e5cb \ - --hash=sha256:eaba834b72d573547b9d966465b3394b749d5e14208cc70acb63aca37619ab33 \ - --hash=sha256:eae86b1f027031e39db2e0e9c4842221edb7b8cd474d23f87a79b3bd4b651768 \ - --hash=sha256:eb2295da7c3769f6719b227a237aa6a5cfa6550e478bc838001b592c57e16575 \ - --hash=sha256:ebf918dfd6a74adc1b9ad71f63c4ab00902fcd3b7fd39f2e24d871db8d713b91 \ - --hash=sha256:ec89771f4272b989487a6364e519db6bbaba323e8bbf949ac89a45ea9c18b7a3 \ - --hash=sha256:ed1a24005daac667d577402d75a2922f9775a165b146b883ff1ad3602d8be689 \ - --hash=sha256:efe9f61bb30174d2f5c8396445c360c96c44e78164d0815dfe627ccf57849574 \ - --hash=sha256:f0bc7f684b65bcda9c20434267577db71bf9905ceddd32b60d1d93278d8c8d3a \ - --hash=sha256:f3d7f7b34114f7ddc6d72a8e882d49de636b35d9fd12b4d420d3c5729f6c9812 \ - --hash=sha256:f753eb70b1474a29e635e7542ff7312e6d6b951e0b25e8a2e8c34eeb1ddcd478 \ - --hash=sha256:fa13acf1046f95df808c64b1310705e143fab87aee73ae00cc42d640867fd2c1 \ - --hash=sha256:fd7790aa79c8b518e512ebcdfce9f11d8ef5f30efd43720c8a19a548b39fa489 \ - --hash=sha256:fe15ddf316f1f1f643347d3a474e74ce61880c79a11ec5dca53df20c071bd3e8 \ - --hash=sha256:ffa0380ad091de7d3fc33e17a97ff479851ee18a0a2a3ee56ff3215cdc886656 -jmespath==1.1.0 \ - --hash=sha256:472c87d80f36026ae83c6ddd0f1d05d4e510134ed462851fd5f754c8c3cbb88d \ - --hash=sha256:a5663118de4908c91729bea0acadca56526eb2698e83de10cd116ae0f4e97c64 -markdown-it-py==4.2.0 \ - --hash=sha256:04a21681d6fbb623de53f6f364d352309d4094dd4194040a10fd51833e418d49 \ - --hash=sha256:9f7ebbcd14fe59494226453aed97c1070d83f8d24b6fc3a3bcf9a38092641c4a -mdurl==0.1.2 \ - --hash=sha256:84008a41e51615a49fc9966191ff91509e3c40b939176e643fd50a5c2196b8f8 \ - --hash=sha256:bb413d29f5eea38f31dd4754dd7377d4465116fb207585f97bf925588687c1ba -multidict==7.1.0 \ - --hash=sha256:0179698c3c913eb64f32397083747fad20ed0f0a2b7469a08cd1a8a95d14d90e \ - --hash=sha256:034b0dc1b7fb8279599c5d8563f86abb4d2454735b06544ecab23c54572ad2bd \ - --hash=sha256:05d12b4bac53abe0c65f3163af2b45894e2e1c0cc55493ac784d52a350047d88 \ - --hash=sha256:0604ff025497a050a2b2dcc4ae0e5cb6477c525e57b89825152c707e88d74d28 \ - --hash=sha256:0631eb5f49f67de10bbdc3f64141326dbc62e8d319900966648381ce0845d8ca \ - --hash=sha256:0747a83e7ae617793181a4763ee8b84863cec5c0bbbde70c4394e4c0276c36de \ - --hash=sha256:08834fb8b20e1a985c70e8380a10940234b4162de62694458727330376e58b33 \ - --hash=sha256:0aa1ba3ff7cdda05a1242490612976b2ae1c90fc6200903ef8f53815dcb35c5d \ - --hash=sha256:0ae91de396d5c4ac97cb24dbada3d5c91a51454781e0a70476b008f4e879e4f0 \ - --hash=sha256:0e79ed92b1dece6bb57e9b46effd74d7a5d3d00187c85466d880ed184239a698 \ - --hash=sha256:0ead852a5e906a43fcb6784eeac480f6a67919a51d480c1f80d32ddf9d615475 \ - --hash=sha256:10202ba98cfb3f7eb60da7ca87a2c458a69b7d0d6e4d4388cd6773ebbce89085 \ - --hash=sha256:1101aea5c3eb1d26e090b931c693488af0db9f3d52e68be8d4cdd807dad9841d \ - --hash=sha256:128ea4142f81a79d430f3d0eb55206093e5eda03a12abbc7b03c34748ff6116b \ - --hash=sha256:1348ddc076251cd542f4a99ccda4b7c1f8444e8ab489d3541a978ca5901c7c1f \ - --hash=sha256:1401caec21fd7f002e79ab6806bbfd1f54bb3de6d5e12bd91c6685dce16ad2be \ - --hash=sha256:14b1ce8579a43dfc0e592d93fb1d63dea693e4977980ac4166f26d494cc7a358 \ - --hash=sha256:159976f9c40f96e3fe0952b708846a43a76bacb114e9cc828816f5080bddd5ec \ - --hash=sha256:160bdb3520fdadcaa21e1b98aab2e011265070814ecab3804eb61674becbd400 \ - --hash=sha256:16b21164797bde6f417066d02775975cc2e15ab8abf80efa55fe85e0b4894020 \ - --hash=sha256:170ba61761f59ab92afcc86ce5534a3f3d0b07c38b339b950a83213f22dd86ec \ - --hash=sha256:18a447d46a3a2f1e61b365cbf5627db7030fdb707dad70c4f2760e5144166ecc \ - --hash=sha256:1df055e51fe7491120cc84f3362bd43db186be78d0e4c476acad45e435af9ffb \ - --hash=sha256:1fed3d721f75c25a9fcdd0e362af53f4b20acbcdc63081112f85419ba0ce3444 \ - --hash=sha256:2128f3358335e0c83688ecb40c19d9d6606cd60784dfbf2e24e980ac2ba87b0d \ - --hash=sha256:23f6d325241b0db006ca2841309ed17622137e134930a740a8f1331ec4404791 \ - --hash=sha256:248dabb89b5aa90b2f7e43e045f048f7e5392ec77b6446d80853ba7117d7bbdf \ - --hash=sha256:24ad4921135a1410d95b1f1504f4901e1c64cea680014ce2c3c7a825f4f259fc \ - --hash=sha256:274023bf952f849e0d05eba28a4c1f65f9796430d2b09ec16539386c0f76554c \ - --hash=sha256:2a964dfeb2aba3663f0536c809aa1ff385f065e89fae57e883fb7edfb4067c2f \ - --hash=sha256:2ba6611fc93c4b169d0e0ea376ebf4b8a529933d1f5f2c2ec7d8f8b93ef58ec2 \ - --hash=sha256:33376418ab2846b931a72b36cfa16810befc4f49485d0b3f4dc054a4d6d00038 \ - --hash=sha256:33389fe084e5426d9fd85d7d9ca91a29cd0d88a83c7c96e411aca49a3f9967bc \ - --hash=sha256:34a35be8fb82d37087e8176aba907b9459f03d0e293c80f574c6337a436f4eaa \ - --hash=sha256:34d2ee98e15d5cfe782a431bc913fce3b58cf3fdb34fcb437aeb275cdf9007ab \ - --hash=sha256:35534b366410a36bb3d6f788691e37a76e4d1da48326b0ada3e5032580dd76af \ - --hash=sha256:36b14886aa3e0b8786ecdaa196374422c7b1c1dcc8764d02b2409f74d47914bc \ - --hash=sha256:379f477b98a1e9a77ddc3ccaa8c709d3fb4a288ff54b96e171e637b55b4adbae \ - --hash=sha256:396ba9917fe489ec3a5942ae3e29e91324c8b9956f371f7e124c971c71379e7a \ - --hash=sha256:3dbaa7f7c2f0ca8578895fc61fb8c8e50ebb405dad8982f92f4343285c7a3fda \ - --hash=sha256:40f586bc8a084a3671ddcae9e5fbd3228a596bfb63d9f0380f153f9a65b69f08 \ - --hash=sha256:41e0c3350d08994ee8640c39884e16514e282f70ba40f5b2299582509a327774 \ - --hash=sha256:41ff3202cc23c800507777df5a4805b402f262b31008c60fdc652aeb6db2f278 \ - --hash=sha256:439a19f7fbbff232ce96682c57e27030b8ac3a4b8121484c94f04bf99d08bfff \ - --hash=sha256:44f7e5dd83a615636b80182bdf446ece57ed61d5d51854acc5d9840631136d4e \ - --hash=sha256:46d4af0afc6eb9867b3ae50605787c80b868e2f52eac3801246034925fe578b8 \ - --hash=sha256:4b5c41e44da74383c924cc5d75ef0a268f301d69305b3c42bd17af685d55e412 \ - --hash=sha256:4b87ad54e8d4adeb0a1f04889504d6ec7f04fb02609220810f51f1b6c66bc1cc \ - --hash=sha256:4cba2b0b9235fe10e12301d6b4cfba0f353fa668d635f6e988b03623c2cd42ba \ - --hash=sha256:4e11e7299079718c78f8147e7206c22fe35bab4466d38992420795288a0b8096 \ - --hash=sha256:507151e1e3dee95e9e8159e329aed4f75aa5205ecd6505a4f6be546890eafbe1 \ - --hash=sha256:50acd7ee7096949b04482cd7720cb6b85eb9cd9dd5d7ffb6704bfda250261a22 \ - --hash=sha256:50fdfcb03be719d9573597b095b1175d2e9d0b30d065791dfd9fca727c499442 \ - --hash=sha256:5129cc1f5fec6888e2db0be936dab67242e32c738811c8769aeea93aab4257a8 \ - --hash=sha256:51d7f33be9a4a1a2801430846d72841deea0894eae8381a07e7d90e0f71b3c4b \ - --hash=sha256:535173fbcc3933d84f9929d49d7a59a0faec259ee07d07c34c7d2a980b4e3683 \ - --hash=sha256:53daa47dd176db64bb35170e3d5d0ae2388c060121201883696278f055a0e70c \ - --hash=sha256:542429c796430de924d03b68a6173bb6d79d5c4967d4e9a18de3e501cad55593 \ - --hash=sha256:544f2642a456fa264614e975d921540ee8c3b368b04d5aa1ddbec33241b13e08 \ - --hash=sha256:55392202cb374dd1a1f89a8ce1586644870d9e936752059d053e576acc50bc89 \ - --hash=sha256:5c6455f2c11daeee40665c67494cedb426f67dba7375710524071c0c56d739a6 \ - --hash=sha256:5c8074ad4d67067c87bd0663dfda654f786336078c8fd7d2f6c1aa41de8494cc \ - --hash=sha256:5c93473d0d7cd9bbb370973a9679a62f381c7050d7dff4ad6aaa92e8650f5a79 \ - --hash=sha256:5cc58ebb731200ddb64d55f1b345630fb5f7a8138cdbd242af9dce964a7cb03d \ - --hash=sha256:5d19bb1ec12e385c09215d5d53a243c060c7e8a0aacdba16d933e22902ee380d \ - --hash=sha256:5f21fda91bd6c34455bd5c312e42aa1334da46cdafb4c533ecd01e0f7f19250b \ - --hash=sha256:5fa1484f74d011addf2e5f5a0378ec41521989839a05d6051d8067d8ce732423 \ - --hash=sha256:5fa296f14068538fced53c6eec86520a2ef3d3d27a0fb134640d03e067986d5f \ - --hash=sha256:6120aab922bb3e15800b6655558cf8e0a5cc79518e954d457f064e5b3d5e9bf6 \ - --hash=sha256:61a4e5d81b8d4e4ad61964b230129e7a2b914793d96289029078fc9009f074ec \ - --hash=sha256:67fcf28db77b385820881521db7435e9f1c607cfaf07db6eb78aa9d1146bde86 \ - --hash=sha256:6ab323f0c5490abaf35a78563e1043c7a772eb86d93f359ecc0fd286d1cd3807 \ - --hash=sha256:6ad60de1f4c702448fc8f1449f05e810f6b7957c08a5b3950c8a792dfb13b50a \ - --hash=sha256:6b7cd1cb0b363cd43ebf499beca26d201dd8b89eee49fae60205c82ba13ee03a \ - --hash=sha256:6c2144785e42527404bbd5cfd11981fee4abe59a22aded0e498eb711a831d3f3 \ - --hash=sha256:6c9fd50f636a8fa9cb6324cd3eac962fec2bc5bb432452a3b583583a1059acfc \ - --hash=sha256:6e7f70d912a589e30290ed926f90ddbc3160998359cbad7c9ede1bcee481748c \ - --hash=sha256:71196ebb8d523148e5975396a444de02367f204b53b14e26794c96b2be0ed742 \ - --hash=sha256:71acdc6eded0f4b86b5e16c96314887cf2572a8eb5d8038b78583d0c0eb3aa1c \ - --hash=sha256:77024596b9046572c4e90b34c1ff212346756dc48933f90c53cf6e233660788d \ - --hash=sha256:7a90453a79423cd7145cc08fc92322dcd7aca4862258f533e03f473226d4b835 \ - --hash=sha256:7b25c335fc53acf29d4d21dbc19fe39d2824201cdda0448623152cc5917bd259 \ - --hash=sha256:7d0b4fec6a8d02d7e95de5cfa913261820f1ce04bd4c0381924de0da523179b8 \ - --hash=sha256:7de54b49e6da811b0321e412d14efdaa1ee0c0b6609296ea5b9022bc5b2bd843 \ - --hash=sha256:7e0bfa161df365ba3c88899ee3b7c94755200967284bdedef8c1b8b43e2c0f2b \ - --hash=sha256:7fac4250b37d994e3fe42b46ba3c8bfa1614d1d7d8cf1cf23f303099082a9565 \ - --hash=sha256:7ff8dd079e7b5f3438332499233a2a5acfca0741fd0eb3d4ddba0c2d9bc04d19 \ - --hash=sha256:8090c35199d6b7bc6426bb8bdaf341e64f295cc2624a1fda7860c0837f1acc03 \ - --hash=sha256:81a0e08c64dfdad27dab687b96f572b23bafa1999a39d1b6f70b3ddbb73e8bd0 \ - --hash=sha256:852c921217f330b3e81a822647ebadeae7e42cf503ec1992d0bfbc90121c09fb \ - --hash=sha256:85cb3ced4fa84949cee12bfe78208b6ece7baf3cbd242b26dcaf773efff8d206 \ - --hash=sha256:86bc779a0896e59e4be30a5be5cd6eeffd0b40b6f0e75e730218736b7bfc6f5c \ - --hash=sha256:88811f890db240a1c82bf0bcd52973763707a552c8113ac3fcebca183afb2fa8 \ - --hash=sha256:88ec4d16e9f58071c9896ea01c4da97cce9d01418fe844ff06eebb00e0a1386a \ - --hash=sha256:8a844b8b1685f38a2e8b2f3213b286e2a7abfe67508381780a0d4599ac337c1c \ - --hash=sha256:8b8429361241da973e594d15344a0989f44fd288ea58d33a6221fb7cc0daf27e \ - --hash=sha256:9161eb81b8062da824426d3700d4b0d287f0cb0b05923713adfe3bd25e7937ac \ - --hash=sha256:9267bf8261a779abb2a6eab5f107f5db85b2d1745f2494081c731aaf28738ce3 \ - --hash=sha256:939d8cd2d8c35e3956f6bc858390b6ccb611e6152b4920d64ab5e98f3fcf39e4 \ - --hash=sha256:943a9bce22180ad0f4d32d1b402a0949a4ecfe5a1257b47f54a1b51981d81b86 \ - --hash=sha256:966ae0588ac9959a040220063733b33f321d04eaf4e60349b42cd855d232202f \ - --hash=sha256:98beff85392ce435b28a0971ec21cade61ce8be8b632c9d855475a28ef92d31a \ - --hash=sha256:99cf27791129d37e191ff013bfc29bf6631c29edb21680c00978567b91fc5d6b \ - --hash=sha256:9a8c826caeb7c08264e0a556df1267531c6ed90cc70506e7e5f4119e2d09f3d7 \ - --hash=sha256:9b24e1f93b9b586ec03bc7bea1bf021ec90bf2528c729195028a3ca1c266b3f9 \ - --hash=sha256:9bc5e7f843d14a167cdc26fe2d22f6f3aa2feb57919cf3ff034262a57d8d95d0 \ - --hash=sha256:9c10791e9f5ef132effc8fdce2009482c1cfb26618c5fc1b7952a47dd5eb632e \ - --hash=sha256:a177a0ee5cf19931dcaeb3f662bc562754cfa4f4ace2351d9da24a954ef7db94 \ - --hash=sha256:a2e575129c048bc286d696ed8e49ca148591768b2d77debcc6569f6fb64d0668 \ - --hash=sha256:a5f0bebb10aae010d3c9ee3abaf83ab2069c718457aea09c15532355dd7e061f \ - --hash=sha256:a5f721a2437390ab69c10c6df5c142478d399af8dfb02e6d823cf2358e8a4748 \ - --hash=sha256:a60b720c329c0007feae692b7bf91cf17b3f9bd3727be96cc6f9a3336651041b \ - --hash=sha256:a8bba9d1f6db4ef2a6ebfc937a65d36e80e3aada00b382eaf56fea8f639322d5 \ - --hash=sha256:abeec7a89d698aa1c9b4c36bd5e3c746faef0867076e6a2ca27fa5077c4ece26 \ - --hash=sha256:aeba2c750102051aa51e087c2ccbc79f2724a41c94168f8731e36f54c453551a \ - --hash=sha256:afe36ca503c2ffe30fb6df82b20389fa3c4035b5d65888a61310921cf3ae91c5 \ - --hash=sha256:b0e0040b0d8dd89bd0af9ab18901981e344ffba68bb30b8eabb4eab6c303279b \ - --hash=sha256:b117ed1cd1a23df0902461c38093408b95971833dcee629112acda25b603c8d0 \ - --hash=sha256:b4674b12701c3fcbdf7f88b9e4479701c93bec5da9eb576140d5fcc0092990af \ - --hash=sha256:b4908e17867930b7ac77f89a18dc67308c67c511f037d8580489be86fb585912 \ - --hash=sha256:b57d4d7021bfd159db9f8f6f862a85a7a6027934643c512f028d6e5c60c4cbd2 \ - --hash=sha256:b5ed78742502b8d90ff2816688d407a097c8b5cc6af4343fc5ad7a98df53a7cd \ - --hash=sha256:b78de22bae456a976f33df34d598dfd16edc9a03df8f4cc8b7c17bdba4c97b4a \ - --hash=sha256:b7cc5333fcbfb27327d12612ed72322f221b61c2b69deb1155078c964f86e1a1 \ - --hash=sha256:b9d9b7d72975521434368fe8aed3f6b522060bf271adabaa5ca6c87c0c08e168 \ - --hash=sha256:bbcae7a54050b7ad7bc7bf425ba63dea7d2cd31a92246ba787a2ce69a9b98dbc \ - --hash=sha256:bf14cfcc30b097583d698a6e2b8b68c9bcffab277c485d481881958360c2938d \ - --hash=sha256:c39dfcaa0bf23443474c0cb58d8d8aea9529c1841d99654cb38e4dada7b1948a \ - --hash=sha256:c44ca6d3cdf4cfcbcd4f928fdcbe87af5fd7319f6ad4169617b7fd6b4527c33c \ - --hash=sha256:c44ced5e5168cdf677f0ae39900863bf2bda7d14a5e13502014005cfe040b8b4 \ - --hash=sha256:c45629c0049fbdef932dbe408ac2b271fdc8c7d9962ca31160f4a0fc3455fe4f \ - --hash=sha256:c53be0dd676484a660acc56e4f1cd0dd74bc1255d12fa285e86a3fa9d5f22bf9 \ - --hash=sha256:c54ae1b89e582aa25f213cd8b5eac0bda1724e79299f486baeb3f562bbf82ca5 \ - --hash=sha256:c564d0758748f38aec56a6b98c6801a427b3a63f39b7cac538b2b2d18ca32740 \ - --hash=sha256:c5e4a362a95b85301d262ef6bed06cc8e4a144ac7e2be874cb4c3c46ae89d754 \ - --hash=sha256:c6b67f08014bfc4aedc22cf6a21010c2530cd5fbeb655730406827fe196296be \ - --hash=sha256:c7aafa4dd2f702ee2198005d6cba4309c1e25ed1c201d77beddefa47411bead8 \ - --hash=sha256:c81062e947f4b5a624135a843f6ac4b3c7fe6508300c9fb27347f022ba0c513d \ - --hash=sha256:cbec738d2ad551c6f70955d7eec95e339380ee1564e2afe86bfee05fed52ceec \ - --hash=sha256:ccf98ee859fe29f874ddd8e637f14ba59108a333492b521acb885a9095244a9c \ - --hash=sha256:ccfb950359a80de0fcd2030ad60ac1b1a861462de3e2ef746697c9256659af21 \ - --hash=sha256:cf606cfe3f67984b4064ac605d71e1eba12515fbabf5bd5a34a8952b8800dc66 \ - --hash=sha256:d02cd23b5af182a49d635ee72be38053767711987a9fd82625b16b93828a0d8c \ - --hash=sha256:d1b1b32f3c32f734dde8f36ac1df8e275e768a7b333241cd637cb2538628a4b4 \ - --hash=sha256:d7dd46a8fcd7653c09ebe67eae9d4cb6636c7a905d9cbaf587dabcbd4eca6013 \ - --hash=sha256:d9ef29cfd98e17085b4f91bba8fa1570bec6787d5c52ce653ed33a58785585d0 \ - --hash=sha256:db77888081431aaa69f3fd3480891746ddce6c2a571f6201869a24e2f06cf423 \ - --hash=sha256:dd8a6b3e8f9edb07fe671b02d8c3241c8b641fecce7eb1e36432db3e55e243da \ - --hash=sha256:df03e392cae1e05462918abbae06d6100f1e53f67db971ff0ac6c07d9edf7321 \ - --hash=sha256:e0d91a4bcb59ac0d7af0d8e0da737332e1b7fe6831e53e47819b1b5349d431b2 \ - --hash=sha256:e2e718fa9d1d900decbc240a533d5d0baf0947ef464c78a8cd4fa32b4e8f590c \ - --hash=sha256:e4ef15d0a29fc2da67fe8ba2301ecabd6f8733696cc2bf0a0cf96a144a20328c \ - --hash=sha256:e50f7775b66c7802f4cb697e986c5acf30ec07301efee95b396c08114e890d67 \ - --hash=sha256:e6906aa4bc62cde2c8aeb8a99a7b4401b241e274ae7b11df67d863d61ab3d5de \ - --hash=sha256:e96d67914ddbf5466e4476a1cd7ff30a332cbab85ed895207acc3e58c979b6a7 \ - --hash=sha256:ea027bdeca1d7e498237634ee4e3a852e2723eef39996dec0ff0f77dff8a2336 \ - --hash=sha256:eb0228c809b2e7eb47921876050af0bc4214b351bad8d8112f70b6ed4288763c \ - --hash=sha256:ecc68f5e47bc6f6f889bbed5bc657b22bb2237ad9ccab8229cb5a0d64f4cb536 \ - --hash=sha256:ed6b7f402f3dabd1d72c798b96cf947005ddd796a5bea7b041bccbd517859a42 \ - --hash=sha256:f16ac8af2804855d3cae5fc3c5ab609c9fd0fc8ecacd92579c05ed3c173396fd \ - --hash=sha256:f376224572d1f5da1c871f969ab04765727f180e70d012d93e07bfc08442c64b \ - --hash=sha256:f76ceb623f7ff50df46ac57e1587c479d87a5766319c4f43d0c0a5158896afab \ - --hash=sha256:f79def86aee67b5ba01b2565f1610f262bf88ae53c379f93e5fa29c50fe793be \ - --hash=sha256:f8e95c95039eab6a2dad8c83c38ab87fc5431d28849e0c8a7e2a4e70ba38710d \ - --hash=sha256:f979a077d1c0a9a36dd4fab0d3a36b8de7b593bf935e13df85a380395b2c11ad \ - --hash=sha256:f996b19ac89e0dae65821ce65f788619e4286f78c62d005ecd3b75b5d9c0892b \ - --hash=sha256:fab380fcff8b3555eb2bd04304fa4330909a771a9a9b0dc07666cfc23148a711 \ - --hash=sha256:fabfdd4cf97db033196b51af46b8a681d4785c2a66347f2a5af1b4bbb1182629 \ - --hash=sha256:fadcc96cd6155f35e6d85845fa4fcd37b35885dc8fda77b9f851cdfa538194c1 \ - --hash=sha256:fed6b7705d49dd07e5e0dd5f5c873fc44047e92d714299b13245b5fecac49d01 \ - --hash=sha256:ff15531a376dc6f35984443fd1429e4b150c36ce27633e7cc52a9e5318546e20 -multipledispatch==1.0.0 \ - --hash=sha256:0c53cd8b077546da4e48869f49b13164bebafd0c2a5afceb6bb6a316e7fb46e4 \ - --hash=sha256:5c839915465c68206c3e9c473357908216c28383b425361e5d144594bf85a7e0 -multiprocess==0.70.19 \ - --hash=sha256:02e5c35d7d6cd2bdc89c1858867f7bde4012837411023a4696c148c1bdd7c80e \ - --hash=sha256:0d4b4397ed669d371c81dcd1ef33fd384a44d6c3de1bd0ca7ac06d837720d3c5 \ - --hash=sha256:1bbf1b69af1cf64cd05f65337d9215b88079ec819cd0ea7bac4dab84e162efe7 \ - --hash=sha256:1c3dce098845a0db43b32a0b76a228ca059a668071cfeaa0f40c36c0b1585d45 \ - --hash=sha256:3a56c0e85dd5025161bac5ce138dcac1e49174c7d8e74596537e729fd5c53c28 \ - --hash=sha256:5be9ec7f0c1c49a4f4a6fd20d5dda4aeabc2d39a50f4ad53720f1cd02b3a7c2e \ - --hash=sha256:79576c02d1207ec405b00cabf2c643c36070800cca433860e14539df7818b2aa \ - --hash=sha256:8d5eb4ec5017ba2fab4e34a747c6d2c2b6fecfe9e7236e77988db91580ada952 \ - --hash=sha256:928851ae7973aea4ce0eaf330bbdafb2e01398a91518d5c8818802845564f45c \ - --hash=sha256:952021e0e6c55a4a9fe4cd787895b86e239a40e76802a789d6305398d3975897 \ - --hash=sha256:97404393419dcb2a8385910864eedf47a3cadf82c66345b44f036420eb0b5d87 \ - --hash=sha256:c6b6d78d43a03b68014ca1f0b7937d965393a670c5de7c29026beb2258f2f896 \ - --hash=sha256:d6db91ca6391eebc139c352f34578cea382df6bfa03d3b4146ed12b18b01cc14 \ - --hash=sha256:e5e7dc3e3e1732e88c07aaec17eeb9917f9ed1107d9e60d5ab985cdc14bac43a \ - --hash=sha256:e6c0674d34b8adac22533f6786576b3de4e396aaeda9e0c15378af9b8ada2702 \ - --hash=sha256:e8cc7fbdff15c0613f0a1f1f8744bef961b0a164c0ca29bdff53e9d2d93c5e5f -numpy==2.5.4 \ - --hash=sha256:067374eb538c34c745436365cf7b0112595c1d326f21ce4ff340f61230239fbb \ - --hash=sha256:0b4724a19de67bea8cfc4970798efa78bcbbe2ac2613cfac16721a42d44de2a5 \ - --hash=sha256:0f02a46e49cfb6c73bdb7aea1c0d3461dbae9aba613542b65f657cd3d17b9fab \ - --hash=sha256:1c2e71b04c6cad90026e544501bbe0ab9290fa8a4d845e7e8c0d124fb429c988 \ - --hash=sha256:1ef3aa6d7e29bb13677323114280b05acc57607fa2300e66432d665d5418a162 \ - --hash=sha256:2132418bf8dd124a427ca9e6a1daf9ee1a87185344c95119ceae868b99466da1 \ - --hash=sha256:2199ed071f460487c8db2c0e5c0b564494190edb4772fe80f9aad88b2604def5 \ - --hash=sha256:2377da2dd3ba2c1200956acbab2a358c83b8e1f8531191672d1cd6ad83250d53 \ - --hash=sha256:298eca75243f2cbbfdb460560b9fb2a1792a33cf2ab4286efd43d92e8d3df508 \ - --hash=sha256:2c2c4afffdeb7920e445028dd71eb932cac3e704792e964bc2a232426d4f1255 \ - --hash=sha256:2ca144f15135b6212a5c47b1e2aeca6e412f102f95a2d5d88d8aec77eb255de3 \ - --hash=sha256:2fa3328f784fc8277fc48026f6cad516f5c561c5d8e2e39b3c9e0c8f23223b34 \ - --hash=sha256:325518d4245b9e331387702aa58c2ce1dc4cdcbb41dfb4ccd5dcbc7e08db1266 \ - --hash=sha256:332f3378fe077dd850e677ec01bdcc4f22368fb5d50ef10b2c79230b1bf5a592 \ - --hash=sha256:3573cd22564692a5b899ec344e5d5b9cc4576f2985b96f22af3564ed54f2710f \ - --hash=sha256:381a7a3d2e65e64c0ec302795ab9dc12bb1e73f150904699c153716177eebdaf \ - --hash=sha256:38f47be9f74ab870d2633b5456ae519c43758a8d1fd05342f0ce4ecc034396ee \ - --hash=sha256:4054173604cd8658796053f1f3bc0befb68ec1c0762c57fdad61e199256a8617 \ - --hash=sha256:468397ba3c64427474706e5c9123fe266395496714dc684294eac75cd4930d1e \ - --hash=sha256:4e263278bfb5ee6409db8aedbc4cc32973b1b82bc1e8d3c668551d04d83a7e37 \ - --hash=sha256:5258bc06526964be5face2fc6f756857a3f24f21ec3e72ca131337a75b165d6c \ - --hash=sha256:56733449d2544178beaa4545cee357370440cf056c197f9c7bfb19dbfdd0e86d \ - --hash=sha256:5ec3753760c1a6d8bb91200666e545c3a9728e6269dfb5d6ce02340996698aa3 \ - --hash=sha256:5fbf7141bbfd63aea22f435c9062a032b9ea0082fe9845dad7f021d3f1234e71 \ - --hash=sha256:64d1c8ac28a4077cf987e0a71a7a0ef7e2df70722f07f0baa42dbb7eb6938647 \ - --hash=sha256:64f9c9878c1938476365e11ccfb6b770f3b9e5f045ccddc514235041e6959365 \ - --hash=sha256:6c109eac9cd439193678f69d70733c1108487546ca8eafc107b510ae10c1aecd \ - --hash=sha256:6d6a71b9d9a97c03633aa12565ef2825ffa036cc1d99cfd50dacf0f128af4fe2 \ - --hash=sha256:6ffa07666f8da0eef81d149934a626d0d95fbd6838432a33e66245423a9062c0 \ - --hash=sha256:7415db95818b39ec475a5eea54d9e3b6bc83e3912158e46da3438cdce399804d \ - --hash=sha256:77045a4b175bbf5316ec08003880804336c78f92281a1b72222b274ea85ec5ac \ - --hash=sha256:7a14a461d9340f1b46b8648578aed9cdb8b3b018a8fac6c1dde2c9192a01a87f \ - --hash=sha256:80d6ef6e8620eb2c2b4c4caad50b5935d6db3cde2d51581b55dcc79e14016d1d \ - --hash=sha256:81e3420b27048b65eb14c3acf0c174a8cb0e023277716110347d2dcb26026dad \ - --hash=sha256:823874a507a84af050493b622affde94b6f7c3a0dc22cb2801381bc03b871c00 \ - --hash=sha256:8b4d2fd2d34e5f8c9235ee787de5631a37a28402b15cb80814df973d2be54129 \ - --hash=sha256:8dddfbee2e68d26d0d7d7d9cb247b1fd4409241cce32d815a11d97ec2cfde179 \ - --hash=sha256:950ea81d57ef070665581b6e1b5f6a029306423cd1739c5b95fe78aa30db6b9d \ - --hash=sha256:956555e0603a4d38019ae6925711cb9dc43195c076a928accf7ea5d50bddfe53 \ - --hash=sha256:98b053943e5a0474ec0da309d2cb9d3f18ea57f8a2067c2ab7b5f763d1068380 \ - --hash=sha256:9968ab7e49b93ac6e1c3b2239732183152c9150f16308d30b66a372cffe3483c \ - --hash=sha256:9a94cf751c9ad8ebaa835bcd3d40dacf8534ad086b88c38029b65123c7999d2a \ - --hash=sha256:9cb18a327b49c5c337f972b03682f6a49855525faaf3c0d3e9c96cd0fd8880a8 \ - --hash=sha256:a7b1b6353e36a7e50de2973a38d705c88ee93adcf120673cee7f45a4a3fa223a \ - --hash=sha256:a813ed7719bf45463c51779e6a98d0385fe905e48447526938a4b8337333d551 \ - --hash=sha256:aa1cce2ff3f8d953de38b76bf44602caeb69f101430208f64a10067f7cb4b1d3 \ - --hash=sha256:ad62a416ddcf863bf44bba76fbf6b53366ab0692e294f51cae4b5fbe0d246788 \ - --hash=sha256:aec3fc4b32ff82421274f5d205c559c51c840c8df66a78efd7f3612dd005a26a \ - --hash=sha256:b1185012870173de7ae33d370bd45b1cf5baee747ea4b97036b65f4e93016877 \ - --hash=sha256:b11e8fda06a7d69f15ebf542660b74466c2e51094800c1fb794f47ad4faeef17 \ - --hash=sha256:b64a85f40e154983960a4167d4c1d57a50c7f109b3d3264a3a984154e90a8454 \ - --hash=sha256:b86966fbe4ad7de710422175572bcdc75fdedadfb54bc6fab7deabccddd7780b \ - --hash=sha256:b89d0aaae2fe498c648f4c4795c084db535af5bd98ef942b2a3681fb74ce8645 \ - --hash=sha256:bc39ac66a7a9a3fbd6134fda43136b60ffde99c8f4501e64e0d2b24da137babf \ - --hash=sha256:c05ede731b03fb1b7591faca9389ade3267d2bddf1ad8882bb3f2cc5e101694f \ - --hash=sha256:c6342f54c67093cae5c0227eb0eb772fdb79f2a2c37a6eb278b9909ee06aa356 \ - --hash=sha256:c668b2f0d651605b58892644b0e302c7157f7159544227758c896982ef384b18 \ - --hash=sha256:c9b80cdf5cedba0e90d93fa5f9a333c4d65bd545cd669b71bb97ce2b703c9d73 \ - --hash=sha256:cfd73180400042a7c532d30c5e287bdd03c59ff9ee1b4c0316af0539e29dfe23 \ - --hash=sha256:d4cccbbc78717966f764cd3af4fb70276fa01fc7a2688af11c78901fa5c04f05 \ - --hash=sha256:d549420b8858885cea8838a727842249218b9c1da24dd517e25c9c7a948310a3 \ - --hash=sha256:d8200f16437b289a5bb927c6e184eccc3e8389bc0070fea4cd5b9e13c1757959 \ - --hash=sha256:e94aef2c639da4a960ad0db8e06471208d8589974953d78b61d345b4eb99e394 \ - --hash=sha256:fbde6962867ee75b48b0ee29b2b9372ec5d617799dbaf38e82dc0596f2f7738a \ - --hash=sha256:fe4d21ab149f15e4e6043dfb0de87e6e5f34ac176cde83060e9802981fca2ac2 \ - --hash=sha256:ffa6ce09a1c6a08e9667dd9c97aa0b14184e8d18f2a14b78b2a2328c9147f076 -ollama==0.6.3 \ - --hash=sha256:41fc49a8095c4a75939c4c1f8582e4d0671692fb6eac2a5a7ede8c9872b67096 \ - --hash=sha256:6a20bc42c1a5f889295d7ec490d35e5132fc31f339561530f43a8abd4dbfe508 -openai==3.36.1 \ - --hash=sha256:4c15b6765fe1792c5ad753556f44c7f82c2920c5e9e40219c0fca97369e923d3 \ - --hash=sha256:ad8e5d56f20198915d98726ac1edb12ae38e7ecfa9bd560057e84642168b20c9 -opentelemetry-api==1.45.1 \ - --hash=sha256:aa38ed19bcc084ba42782a73255b3582283eced7ad6dddbd6695189e69adfb75 \ - --hash=sha256:b31553efa588ae44bc306f863c785c5333a9ecc091248c6ee68b4b6c87fdedfb -packaging==26.3 \ - --hash=sha256:94edc256424af38762eb31306eed28beb9f0efc50a8837492c9d6fd6004aed79 \ - --hash=sha256:d7193f7c8e4e93f444fde0262bf90af30e16fa0ad0ad44cb553c87339b23cd1c -pandas==3.1.0 \ - --hash=sha256:04ca88727dfa86837030c493d7512bb1c008aa4f5f14149c5f8d3ff446791340 \ - --hash=sha256:07807035ad66de0d833d68f9bee05801dbf08b88d56a233b0e9fd189372c3261 \ - --hash=sha256:0a28602573f50453b57802d282f9f8cb7663aa10201127ecbbe95ed4a41c64ce \ - --hash=sha256:0d2ca3b380f0f6cc788780430eb23762619730c11d53989f3982ec96835bfb9e \ - --hash=sha256:1340f823a1d189fae8365de314c4d3c2ea0173c61bca66cc76a91385c64d9d3d \ - --hash=sha256:224a9267e5fa70c16bf4f8a317435eef98387940ce6c8c611f1f68f2c3415e10 \ - --hash=sha256:2a8dc7780720b9e2182799ee86279d4cc7d1fd30b230ea1b202da5b6f7e14c3a \ - --hash=sha256:2bb3f8cc42fae0c0de5186778f4a4957656b9b3955aca48383abfdac44a83598 \ - --hash=sha256:37f88576f3449bfd86c1052f6b42274b61d7250c8d9dcc55f9dc4de7fe9883ae \ - --hash=sha256:388fad0c683ad061baabe23a76f43bf43c1d2c2fd46572b687082cfa195b7275 \ - --hash=sha256:39c67ed9e56ad3ac0727dc2ae6a1eef766988c762d5ce2e2a162613d81c8990b \ - --hash=sha256:3cba20fd269808e224d6212273008173978033a1dee0f3f217bfdbd58775a72c \ - --hash=sha256:3fe234bfb9999552dc54c45851752e56e163218d7663897dca950b4a7c32af39 \ - --hash=sha256:461b79db058639e446ddf90dd7f4804fae385acf0669f6e31321333e14cb6bc5 \ - --hash=sha256:4b5ce18612ca955e7a95d0bacc20120aedf97e2fbbebbe4de029fc24479e79b8 \ - --hash=sha256:54fed4a50b8321ff8d413b46b7e31a023cecefd06c7210c02aa980a9836428b2 \ - --hash=sha256:5611344f9f0ed51619ac7acdbf72b86f5043801a66e3f8730c8971257082e8ad \ - --hash=sha256:684ef3add5a92abb99c1622f2bd04b54b8843075a266e8240106ca94ce1d1237 \ - --hash=sha256:6c51273b89784c9d54020ed2f60e20524169aef349bddf6cba9f5e83b46077a7 \ - --hash=sha256:73b6cea22090b7123c78ee662a5108ff36616b0bec4fe47026f86fb04a49bce7 \ - --hash=sha256:743554ed96755ad1963397ccadbe9965d92db8777ca2ae8c1fe58bd9549d7f28 \ - --hash=sha256:756fd67995f3efb6b25f8eaea95c0e893f84e0068e2c53c5fdbb65275041d4db \ - --hash=sha256:84377fa82d0b40165379a0058d8a1a267ceb09e65160de6fcc75475d4d595884 \ - --hash=sha256:849d5019dead65944875ce21fbf71a2ca18e982164cd7367623cbda8567c3122 \ - --hash=sha256:84d73e0814a38711e4f2a24b272f748a0b4335169d2f9752407ae26bde22990f \ - --hash=sha256:89b66823678cdd53cf97187e7cd6861402bdd147d11a383706093136a4e83f45 \ - --hash=sha256:8e4de3fe8017e3445f1174707cd977e4e1154a704af2c060a1f648862f81acb9 \ - --hash=sha256:9433b5dc2f26ef9d9b56daaf4d41d078bf4372c5d88c42ed5b397a6eea80de16 \ - --hash=sha256:9b4e696ff70ed06e986feeec09356c491ce878d0df69a36c8e591b6a38afa29b \ - --hash=sha256:9ea2983fe096a3d997c8c4b9f26775fa51e7fb251b10634bd7f87e6fe3542d52 \ - --hash=sha256:a13199463b129363701a540c7b79586581b246b10b6a7ad682c41b268e7991d5 \ - --hash=sha256:a1d61e0eae5219cd024fcbc21304de759964b5642ddcdbb28d9a8ba442ab6901 \ - --hash=sha256:a7704e3d7367cfae71d3f8c95ee81e178c7a90bef8d6eac30df9815010722259 \ - --hash=sha256:a8c44d40cbb6d00128bf2d8f8bb66c8d561744b6d44686b5a1a01f3b3fd1245f \ - --hash=sha256:ac2f833557874a79fc0b5fac4c86e402ea6498beba8753dc29ca23d08481689a \ - --hash=sha256:b8c75dbb27c7d170b25c13dd35515da4bb92d40fe8bb3bd2c69a95de2218889b \ - --hash=sha256:bbb121205af7bc5e587b94a07b5f67b5ba687a918665b9e8748f21388a88b044 \ - --hash=sha256:bbfb087db83eb79ca348047961d367bdd4bf9820d90a5945f3652c7060da7363 \ - --hash=sha256:bfac077874a396b7ae61a307b27aa6f0256cf31c143b7f96ac622765754cc300 \ - --hash=sha256:c0bfec4d188fb0c05bba47b9f9def87624b8c545913aa3585a28f19f27ce847e \ - --hash=sha256:c1009f5b6819b81319cf177b4f01a2823ed54c81db8bd11cdf2375f42d064c02 \ - --hash=sha256:c1e084760f3b275eed5863407d025b933d25f3247515fcc84bf664405e8bb3d2 \ - --hash=sha256:c261c7a808982557c2917d40ffd27916769f26aacb4e6775c01e4467bcd34c3b \ - --hash=sha256:c4d9252b57bded194d09cd5ebf18d9b4edac238b16f56984ea8281d2ecfe39cd \ - --hash=sha256:cc5e4e12d287ea77bbf4c4d4bf47ed152d82a990d3a6794dee7c350f887532b5 \ - --hash=sha256:d3920d9b4de89fbef76ba6b863d58a330b5bfb720c9f8f58eb9b7da9b711d034 \ - --hash=sha256:d3e660af30a6683fa57a5a2969555df117e873eaaeb418ad3b2e59249405ee70 \ - --hash=sha256:d53387f8bbff4e7aa567619d4034ffdb1836db1e768a482851b904bbeeed8ef9 \ - --hash=sha256:dde9a55ecd3c4f9100ac50ab15e852e4c54fcf372d800e4495e0b2159a0122ab \ - --hash=sha256:de6d0fc96e8551b83f400bf212611a0b3a2b389594d20a2540dacc61b21e7bce \ - --hash=sha256:e4455cdcc03e42c17233184e8e4ea65c3ed2925d8b1d9d7dd8e6fced3f6e43c1 \ - --hash=sha256:e6f83e44846f9386e3593965bac017a54a5b5c611ea7b836ceab7f465bb4cb24 \ - --hash=sha256:e77112cfcabb3bef7281a72170670db9a5b31409a1cd3199939abb1fba67a7ad \ - --hash=sha256:ebbc0e652844e52e06d1525e5084ec1609158c086c8646aa6ccb34802102e533 \ - --hash=sha256:ecb9b2d8a817b1e1ba1fdb90c0abe9eab31b4fdf2562b7a56b86ac66ace17e49 \ - --hash=sha256:eee72bba8a72f0bf9522132e5c87cd10fce9d77440b50f65326b093ec9fb8d43 \ - --hash=sha256:f1a93c24175129bfc9922ef748ddae1d8137c743ed835e2615be329f606fed4d \ - --hash=sha256:f4772320b7735d96dbbef437129f840dec4175f8a18a2391bd1222ed8943825c -propcache==0.5.4 \ - --hash=sha256:004e685b315646c410771836e72a44f143bbe624f29653a42687815069a303d5 \ - --hash=sha256:02c0a34f16889cf800f10f0247a564d8ce6eeab6ffcd7c87198f769067eb8432 \ - --hash=sha256:03969626faf0783a592dfa17e28eac06018bd0b44dafae6943d53b92421a7f72 \ - --hash=sha256:03b229037d25b801e7af53fd52b9fc49d9439b036fca1e087e02780631adfa97 \ - --hash=sha256:0951315a6b3142ee2167404d707743f0157c110091342b1aa0accac5cf0e4acf \ - --hash=sha256:0a095db8e15a6020db149ecbed6461939fe74f6acaa3ae8b702a1fe8c38cd983 \ - --hash=sha256:0c889f6fa84957bc7e8b4eab71fd16a0455068d5045e3aa40c733071d2b2fd77 \ - --hash=sha256:0d21d0d2c82bbfeb1677a9711f38df968f9837576102bb4add1bd449d28d88f1 \ - --hash=sha256:10ef33a68a61ce317e095fd2e202a592ea92392b90944a78c993f0d9a73ab06c \ - --hash=sha256:12682126712ddc19b70ff819debbd279e58adf1f0c8f8f8138c18ade2044b284 \ - --hash=sha256:135036c5cfc93864affb0f9af9a27e5d7a71cb7bd745e7b6dbfc2d56cc30e827 \ - --hash=sha256:13e52b6e0bde97dee98ab66552dbff2931649c96f1ac432eac299fe689ec373b \ - --hash=sha256:141fdbd73748db0cf7636035030aaac383d2efde8f34e7bc24594cc776d225b8 \ - --hash=sha256:146f48a9e4812611a7581003b1a39de56c34967046310c4171a68ef908c9a745 \ - --hash=sha256:174507f82d3594622acb1dd2dafecf2d899d6d506335494e7107767bf05f3aae \ - --hash=sha256:1783582065a1f07f9d9ee1e992e13f15d7dc8fb1eb3a7476d43eb3f2e69d26bb \ - --hash=sha256:17a7400cec0256f0a71ae71f9da398f9894c956ff6668a1c9d317b3367316320 \ - --hash=sha256:1b2f3bec4261a94019575481c726c29850f72e27907773c75b1de421e20e9f9d \ - --hash=sha256:1d759d05634f1b038fb625a66662a8c85e5a8fec912da381b5149ddac107482b \ - --hash=sha256:1df8d8561b21465c5dd56110a01caf897e026d065b4b84e98a488209094272ec \ - --hash=sha256:213bb68d9ced5cf2bf717b1071bf2b09b4b04c426256f9fe6d054c60318424c4 \ - --hash=sha256:23278f808cd81d5ada7184a76606b925fb3389c60e1077b2cd7da7b1fcf0553c \ - --hash=sha256:251c63dd46a0659bb875cb254dc4c1e79ee91a847c737cd62373295afc2235dc \ - --hash=sha256:279655a16973f1ee2bd2fe79973137681642fd9ae0d89215bba263726eb0dc3a \ - --hash=sha256:2814ecd8e818f487bee4b0f921bc4d1c176cc5fc71ac0f072d0fa67eda4ac14b \ - --hash=sha256:286867fb156488c251a3721766e380ac4495e4fd6b51aaa1403d89ce7f4359d9 \ - --hash=sha256:2dba2f02d2d5c09ef8a0e6c1a42aeaa451f4be9898cb00b04fe98717da2eb23b \ - --hash=sha256:30cc1cebaf9aef49db06357a50398323ae04d70460c0491837d026ab7d6452ea \ - --hash=sha256:31eb43ba2edc704ab2ec27815315dd8a19def0fb16215be4cfe8d32fe78ffd51 \ - --hash=sha256:350b272b2279f4135a64fc0c304a5d08e28a137c9573442c606152446638a831 \ - --hash=sha256:36c0d9db44b523ef93d03341b1c42d69ff01d673c053d1b1c6c3a363bcaa39ba \ - --hash=sha256:3af0c8642b2da4815d86e631232ac8286e17644fad907c19508aa8e7cb4ba8ad \ - --hash=sha256:3cd3a7edb6b95b9b33998135ebfa18d709da82290fb8f27c858970b5a12c8b56 \ - --hash=sha256:3d605bb239b796e82a81c6709548b2bd460ab73b4590cb0c83de8a2dd9694d0f \ - --hash=sha256:3e413d7a4a9b4866b7a761d6060d434b64d23cd35122eda3b026a0bbe8196b25 \ - --hash=sha256:3eb2e820e8e2101407da93f17c57cbb7d225461955fc60105daaba14cd421ee2 \ - --hash=sha256:3fa15757fea1dfcd5b7745cad9f4638929605531bd4018ab2adff7955f1a403d \ - --hash=sha256:3fc24f209c1b7f7f688b66b98293954f5504279760999b58920ee12dd8471c1d \ - --hash=sha256:4054acf80d40456a0537f2913b349718649d8d6458a14ab7f48d0ce28c30869d \ - --hash=sha256:40e94adb1e7d39ff28a8bd8d8b8fbd1df6b9f40976dbe379134f1ce058e532dd \ - --hash=sha256:420162a77f94eb1cf5ef7893f500016dabd548e73de956785a1dd899cc73006a \ - --hash=sha256:425f8cc86ab5018b4b8d4a23bc8e74d964bd3d757c3702e301aa79be76c53f6c \ - --hash=sha256:44149f46500a0a41b95b4d99c2e586a77319539730607b9892974a092788b111 \ - --hash=sha256:445ee3bfb46e85838387fb3c536a73cc0b994dc192b004e40e170adc54aa2a7e \ - --hash=sha256:45488d1a5f9ab5bd90aaa1ca20f50fe1922b8ffad71a2009d2adf41355897aac \ - --hash=sha256:45bebbe252550fec975ba3b62bc6f931643cfd3b5464ef47619cf3fef154e01c \ - --hash=sha256:45bf2e730ab8905d0527fe05a86500f406e64305c34cc81ebe64b4617cab9760 \ - --hash=sha256:48cb48c5346a97de792254af77715aa2529c2a1ebc5f586aa0aae44a02f1fe57 \ - --hash=sha256:4a1f4f5ffa55dce6307631f3cb2948e117e665966ea512e0d502b16c24f567e7 \ - --hash=sha256:4cfe0a92ae30151869e67a4b5f5e105e4e03ad30b3f38e5211b5bf77d0881993 \ - --hash=sha256:4d86476a935c88963d9b8e1a9a0d38188790e9622169bfbafa173046846709d3 \ - --hash=sha256:4e985382be6d15da8d0c2710a6fa7b9070fc9ecdeefb7f580e88373984ec8be3 \ - --hash=sha256:4f2d880ff60f45898f4acfa152aac8d04e3ee627d90ff4003491bf92239d5757 \ - --hash=sha256:4fbc1a15dc8cd1689508758d626b372b1f09d28d9577667feaf9e6bfcd8efcbc \ - --hash=sha256:50e337653721d20ead710da33bf44487fbe8a0db8782714b60306481e9f95b51 \ - --hash=sha256:53eaa697c4d0422ff4cb714d00231b43352064d97b944033b30c1d57cc506ec0 \ - --hash=sha256:56fc3f7599528db40b1efa0889a620116e2704144495273d66066e8164e45838 \ - --hash=sha256:58134228927cee6c047d626c08e60a81be604a20578a12ce752cc5c9a84d4826 \ - --hash=sha256:594eb4c6ec35e7179b058481f4e9f02521b56de16fa577c4b85c76fb1bf8a9f8 \ - --hash=sha256:5cacf3c9efd09df409dc33654dd077e1c245ba8fb747b0f0236ef41b7c49b589 \ - --hash=sha256:60a64cbccaa11b7760ce705a14ada17ba459e7ca9f23ba587eb013821032d7ef \ - --hash=sha256:62530ca89187827e4a4fe733f971abe81a7542eeea48ff61995f19b64d7199c8 \ - --hash=sha256:62c60aec739ed00124573cce1178138fd690c7676352d67a37328c1cf51d7468 \ - --hash=sha256:69fc35c0779522da366c563e5faf203ffc1f8ff0021d5b1337fa4efa5be73177 \ - --hash=sha256:6af4693716bfb03f1752ef1b30faa593db2c01d5272e9b8564a1549452a979ab \ - --hash=sha256:6c7599df2b57ebeea8de011b5f2f7b85de95e76037d43d34b95e328430275487 \ - --hash=sha256:6e9368e87a3efc285e559131092c5db643eb8e56de4ee42064d5baec22ef2bb5 \ - --hash=sha256:6f0093ac3e9daada202c2082439d414a625c57184727a46e112a3fb2a81cb788 \ - --hash=sha256:7177c43eddf10a0893c4fec52ebb408fdcd7f7d63962caace9180d8f81b14ece \ - --hash=sha256:720cf832eb2d0b0dfee129cb3335a26f6ce3cc45ee1187e8f0731758caa16792 \ - --hash=sha256:770e8209d018175fc0063936fa9583b6d27e88c5ad31543f3383d66080efdd62 \ - --hash=sha256:7a8d5ff04eb1f85698a78d20c62a14676e7b960dcafde09a388d60ad377d355d \ - --hash=sha256:7b9100a93b372418d8688f3f2a3e5b45c64d70ca4d6176e121aca1e3bfc1e32f \ - --hash=sha256:7cc528e760a8af06f2b13e9b9f362cd90c7c718ea61228a96dbd31ba16ed7f47 \ - --hash=sha256:7ffafcbfc7b549ab940047e505c831eabac5e67de53e1bc174adbc5285c55944 \ - --hash=sha256:87a3caecf8095e48dc72f84bfa42e23a848cf410cc9cc13031fba4869b706a21 \ - --hash=sha256:886b59c4d28ca97dd23b025fdfc50a0356be934efbbbca89ad26230067f86fe5 \ - --hash=sha256:8876b39961e33d912afe3c1bee18ee564fdad0206f873cc15d522756b7f50737 \ - --hash=sha256:897d1ddf6716e8f47200f7aad9a0efa6cc7586df66c6defa572f9eab379c078e \ - --hash=sha256:8a1fc236528c457cd739c88abe823da851b7ab645d72792f88658114cc340c12 \ - --hash=sha256:8a235f73d6e020855dc29dff012d920c02ee0feab8d73a24185a7569f4be1161 \ - --hash=sha256:8f911c395cef73c510bac566da9507bb6a43e7763d0c79138dc60ee53f11207e \ - --hash=sha256:96f7c5c15656040ddcbc51e56dc59b58aa25999d743c126abd425b9766ab43e9 \ - --hash=sha256:978f28401afbc76cdc3df9e1717b4229a06b626a1dcc75db4e1f2beb3884c3e9 \ - --hash=sha256:98914de2c4d7f0f9f4a8c6ea4bf05841f4175796941e3ef7d47eb718f22311fb \ - --hash=sha256:9a2a8a50a93dee0268a860a07fa3b4bd968f8ce4dbd794957da772f395368526 \ - --hash=sha256:9cbfff4423eef4cc6cafc021469641a2b835f610b2647a6c5281903e21b8670d \ - --hash=sha256:9e9ab13760aa8b6d0881ae7cb04fd891d8d490cd2554ea8e79bb278399169bcc \ - --hash=sha256:9f3551b8a35c1df3e7ea4d2d86edee15f0dde1bddd434a71744048683544d0ef \ - --hash=sha256:9f86f7259efe2c951f43e57d471c9b41daa5bfc7db9f67189059cf1ae6d77fd9 \ - --hash=sha256:9fb0a5be8d9aa213150e8d8148a42aca4984b285bcad1e69587dc4298edd929b \ - --hash=sha256:a219f0ac59817a9114dd2aa57c13180f993e819ba658c7ddab4b66ed1ee0d370 \ - --hash=sha256:a419ee85e654927baabda3929c03c0cc1112bf472ff0dfd6142f4e3a81ca4162 \ - --hash=sha256:a4d7a54719b67338a305dca2ce6aafe366817df94ddfd4b5514374356f5ca546 \ - --hash=sha256:a5793c7698a53f56f4a1889a4737c7eeb1b7ad0842fa6b1abca22913ff79c8c1 \ - --hash=sha256:a5e8ef588c109725dc713ba69aadcac00a1ef90c2ce9c0a8c7075128f569f47f \ - --hash=sha256:a74bfa37147cc08fb29df10bd9c16f40fa7f860cd3a6d2fff853323a94f6e17f \ - --hash=sha256:ada748108a43d29b7c328ba7db3755327cd94f028bcc1a7ee3f0addcfacd9c38 \ - --hash=sha256:ae58f361bd5dae942717c65d3413b478c70aea9c462599e7b9adad3731db3894 \ - --hash=sha256:b28f41fa3b8c6900457f858ec5b03998f3a6d535fbc1bb2edec5961ea05ec429 \ - --hash=sha256:b3083bfe87f95c756e610bd8025f26cbd1cd4aaa03a422f2d65efb7a97cd53d8 \ - --hash=sha256:b61805357d966680acf68b3b6d49772631ed9df44ebece10ff1460e117a7da8a \ - --hash=sha256:b77c313314524ca9c38fbd70f73515d04597ac58c40c939bc0e71eeb4abff680 \ - --hash=sha256:bee7d3aed13d56f54e681df38c3a23031bc9e3863f687d9d598825c9146acd7d \ - --hash=sha256:c02c0e570c5c7e077b0181a9f3cdb7d4c3617d1cda6b5c95bd5d34022923d82c \ - --hash=sha256:c174bfd1c48a1b51a3078e95586dde718374bac79719ab3541ec9e74aec40574 \ - --hash=sha256:c2ba30a89035b57b73e00475de948521602f543d79ce01db10b04b36c4c76fc8 \ - --hash=sha256:c3e98c55bde2bcf7db3c70d1aed7ae9aa8aebbf19a250c66645cde44cdb8b867 \ - --hash=sha256:c3ef2818d63bc86071e9d2989ae75a1bc32b8f7059cfd9f5abbbee70c32e2ed6 \ - --hash=sha256:c83acbce9f2b5e3f5f5eda9e53d2001fed22fcdfef81274a9e02d8fd53b70a30 \ - --hash=sha256:c9281e922c072158c91974d4589f1dbe0fee6d467f284c28e463f9f5a4d933f4 \ - --hash=sha256:cc07876cfb079b6f6f36d21ce75784ad6c2c6b563eeac0ed26c2fa2669b85df9 \ - --hash=sha256:ccf4f7a79e26bb7efb06ecd50c177833b71df05cbc748701372325e6bcc17f6f \ - --hash=sha256:cdee8205a44d0be91bbac4c41b95d86641b72dfc7aef1279400e4fda3f26a937 \ - --hash=sha256:ceb3e879afac028f93d272c957814695dc5569e4904262dbee92f6c41bd5e4a3 \ - --hash=sha256:d1f5a500bfcbb2c0ab85e98a0dcd70f5899d34efe365a0187700369a79603031 \ - --hash=sha256:d42a9a856a4a6e2f6c10f1318c07e7daa498d6593abe745c71dae4521a26ca39 \ - --hash=sha256:d83b12902eb8bce151259c86c03ba746600b2d994543de46e370cecf96c452f2 \ - --hash=sha256:d8e017eeb7482bed34cdb0d61cf2bcfc88d104bbab296a17cd16a6af8aabc70e \ - --hash=sha256:db3ae52ccc150dbc84704e9d642743897f3e1c54742ff34cacb661e52e3818a9 \ - --hash=sha256:dbab5f5ff6897c81f355d079010cdae85b02e5a0b518b5251523b8ad8ae9ac3c \ - --hash=sha256:dc4242ca653c9b30ab51c5f8193323e7bc0928f897ee9103201e59a43abcb72e \ - --hash=sha256:dcbf346a318a5e30063f547630b02bb787ce2f45b6368d5da143660b6a3835d8 \ - --hash=sha256:dd2ac8f5b643454c2cc6b6118b13da16e88f4a6434fc3ba61aca384029f04f36 \ - --hash=sha256:e1d52a05dc417279f7e5c7618c5dfbbc29923aaf9bc0a5c1802ddcebf54c61a0 \ - --hash=sha256:e6720ba44ad7e72174314d0e1fb0172494cff5c73a3a8a2159c3d2402ff15565 \ - --hash=sha256:e738ab81179510ce79b2eac9a6ecf47feffd9e76d1c72e403005dddb6e36c06c \ - --hash=sha256:e904d4d01f36bd6e197590be1533c44e06058771e0746dd073a8ebb3ef880858 \ - --hash=sha256:e9f165403b81fea7e89c932d89046a1e3d9a3a60e8d7ef2f249dccdcb0982bf5 \ - --hash=sha256:ec6a85f424afa8d23e0d9a094e5dbb6eda01da91c92b9183cd433768247ffc97 \ - --hash=sha256:ee19113bce2f3acd46432050688b70f61acd6857d75abb9ec96341b7e9ced123 \ - --hash=sha256:ef3b928d9c984322b5c44e6964d8dbc653da87d2d8ee1647fa6da43072e650a9 \ - --hash=sha256:f273dcf7149a50527c4fd1f55cfe9eac0f60753f5af544b4c9352578e20c0874 \ - --hash=sha256:f5470694918830da62fac9e69133b53d23b736d7070e587b27a4a2be37e08e68 \ - --hash=sha256:f574e460d1c8a08384a016fdb09ccf3543433263ed6b2f97104f979e64ea57c2 \ - --hash=sha256:f85915e00dcb1cd9f2f890ead064ed40a27df06f0db65be427b29482ae357572 \ - --hash=sha256:fc2461ecc45f17893f8207e73b46ea8ba93e33630e51cf4af3fbc21d47462b1a \ - --hash=sha256:ff6b113f50bc066a698db5d944d2c6dc7507168dd3341e255a8892fd0715a558 -proto-plus==1.28.2 \ - --hash=sha256:26d843eb99c1e32fdf1d20ff0faae56607f7748fe774acf9ecd5cfe6c6472501 \ - --hash=sha256:b874236fcac2358f601e4330bcb76cb8b89c851303ccf4078408b3d4774d1c52 -protobuf==5.29.6 \ - --hash=sha256:36ade6ff88212e91aef4e687a971a11d7d24d6948a66751abc1b3238648f5d05 \ - --hash=sha256:62e8a3114992c7c647bce37dcc93647575fc52d50e48de30c6fcb28a6a291eb1 \ - --hash=sha256:6b9edb641441b2da9fa8f428760fc136a49cf97a52076010cf22a2ff73438a86 \ - --hash=sha256:76e07e6567f8baf827137e8d5b8204b6c7b6488bbbff1bf0a72b383f77999c18 \ - --hash=sha256:7e6ad413275be172f67fdee0f43484b6de5a904cc1c3ea9804cb6fe2ff366eda \ - --hash=sha256:831e2da16b6cc9d8f1654c041dd594eda43391affd3c03a91bea7f7f6da106d6 \ - --hash=sha256:a8866b2cff111f0f863c1b3b9e7572dc7eaea23a7fae27f6fc613304046483e6 \ - --hash=sha256:b5a169e664b4057183a34bdc424540e86eea47560f3c123a0d64de4e137f9269 \ - --hash=sha256:cb4c86de9cd8a7f3a256b9744220d87b847371c6b2f10bde87768918ef33ba49 \ - --hash=sha256:da9ee6a5424b6b30fd5e45c5ea663aef540ca95f9ad99d1e887e819cdf9b8723 \ - --hash=sha256:e3387f44798ac1106af0233c04fb8abf543772ff241169946f698b3a9a3d3ab9 -psutil==7.2.2 \ - --hash=sha256:0746f5f8d406af344fd547f1c8daa5f5c33dbc293bb8d6a16d80b4bb88f59372 \ - --hash=sha256:076a2d2f923fd4821644f5ba89f059523da90dc9014e85f8e45a5774ca5bc6f9 \ - --hash=sha256:11fe5a4f613759764e79c65cf11ebdf26e33d6dd34336f8a337aa2996d71c841 \ - --hash=sha256:1a571f2330c966c62aeda00dd24620425d4b0cc86881c89861fbc04549e5dc63 \ - --hash=sha256:1a7b04c10f32cc88ab39cbf606e117fd74721c831c98a27dc04578deb0c16979 \ - --hash=sha256:1fa4ecf83bcdf6e6c8f4449aff98eefb5d0604bf88cb883d7da3d8d2d909546a \ - --hash=sha256:2edccc433cbfa046b980b0df0171cd25bcaeb3a68fe9022db0979e7aa74a826b \ - --hash=sha256:7b6d09433a10592ce39b13d7be5a54fbac1d1228ed29abc880fb23df7cb694c9 \ - --hash=sha256:8c233660f575a5a89e6d4cb65d9f938126312bca76d8fe087b947b3a1aaac9ee \ - --hash=sha256:917e891983ca3c1887b4ef36447b1e0873e70c933afc831c6b6da078ba474312 \ - --hash=sha256:ab486563df44c17f5173621c7b198955bd6b613fb87c71c161f827d3fb149a9b \ - --hash=sha256:ae0aefdd8796a7737eccea863f80f81e468a1e4cf14d926bd9b6f5f2d5f90ca9 \ - --hash=sha256:b0726cecd84f9474419d67252add4ac0cd9811b04d61123054b9fb6f57df6e9e \ - --hash=sha256:b58fabe35e80b264a4e3bb23e6b96f9e45a3df7fb7eed419ac0e5947c61e47cc \ - --hash=sha256:c7663d4e37f13e884d13994247449e9f8f574bc4655d509c3b95e9ec9e2b9dc1 \ - --hash=sha256:e452c464a02e7dc7822a05d25db4cde564444a67e58539a00f929c51eddda0cf \ - --hash=sha256:e78c8603dcd9a04c7364f1a3e670cea95d51ee865e4efb3556a3a63adef958ea \ - --hash=sha256:eb7e81434c8d223ec4a219b5fc1c47d0417b12be7ea866e24fb5ad6e84b3d988 \ - --hash=sha256:ed0cace939114f62738d808fdcecd4c869222507e266e574799e9c0faa17d486 \ - --hash=sha256:eed63d3b4d62449571547b60578c5b2c4bcccc5387148db46e0c2313dad0ee00 \ - --hash=sha256:fd04ef36b4a6d599bbdb225dd1d3f51e00105f6d48a28f006da7f9822f2606d8 -pyarrow==26.0.0 \ - --hash=sha256:01c863a18bd9c8412453dd0d92de6d0ee7b2b3d6fb079d9734a4b2a3c8bd4453 \ - --hash=sha256:0cccd36e00ea3afeb52ded61f2721ce71f604853d70c45365c58324eb773d6ae \ - --hash=sha256:106bb9290fc6fd9a84138a9440038ef184bac86463543c5ff099229cb30d996c \ - --hash=sha256:13b0972a3dc71b642050d1bc72664a3916e14f59c943d8c1368154d6e4b0c2d5 \ - --hash=sha256:210cc9b83888b87cdc8f793eebb264f22b20d0dedbedefc73b9687a7047b4747 \ - --hash=sha256:240bd18a7487f8767616a948a69dd4e740a8bc36a1c9da49e4dc9a32c5c2faed \ - --hash=sha256:24f892fdf1ae1942d69d3f7742e2f49960ec95277cfb1a70b8a1d91f4a96d935 \ - --hash=sha256:290a74c48e9491b436fd5edacfadf357943f82aa45c81110bd83a69aab33d1cf \ - --hash=sha256:2b5fcd69c0e1107b79e55839877db5a6ed04651b73fd6fec581d09e230bed5e4 \ - --hash=sha256:2e4a413046eba9896e632925066c74095182200ba32e19ff0166bf64d2f936ac \ - --hash=sha256:3a4d235876f14b4136b4d616ec42eb469ea0d6ead336cae631aa1dd29b21c962 \ - --hash=sha256:3de30a7432b48b98b9decbd9e25a53bb9251d202c2e6c5a29a50869592ccb117 \ - --hash=sha256:41dd3661ef40790a78870052ad7a58ad827b27c67a4511f06962eb9e9b74d19b \ - --hash=sha256:4a5fa8dc70dd50808990ff36faf44088e357b353d86c7682dd92d4b78d4c97d5 \ - --hash=sha256:4bcba83299cb2b8f8e443d36c6ba6269a5034431879015fb0719495df8a14de2 \ - --hash=sha256:515a10dae2a1d236bc9c9209d0317acb6746ea63cd4f98704904af7156d90ed1 \ - --hash=sha256:5780d487ff6c6ed7b42298609680d87fe0036e529a9dc2e1105364bce9697f50 \ - --hash=sha256:5b827650e874f1f9f9392524ea3e9e3e8a245de5ba64acca1f81ab188090afb9 \ - --hash=sha256:5d5768d03426abe6526d5274adefa00abf00a7f81118c46e98b5a46390f5549e \ - --hash=sha256:645917e976671debabf854abab6e2b75c571ca4f82adc33a2d338697f7c27d93 \ - --hash=sha256:68cd662e9e2b00876a131950cf32336ace2d0865e1f9418763e3d3be8481dfa4 \ - --hash=sha256:6a628922ba20705fa964ca73e4ef959c2fb2f14b9bbec5589a6a1e68e6257c85 \ - --hash=sha256:6e89dee53aaeb50505ed6152ea55bc7ddfd4f4df264f5427ea255288d8f0e580 \ - --hash=sha256:6e949744dcfc2d379808f7013c5f9cafaf0f817656dff7d46c6931528dd1784b \ - --hash=sha256:734312d3d99088d9ec28c5b17bad40389bd8373a1afc10acb60b83fd217af087 \ - --hash=sha256:7aa12ab8e236789b1ecd2d6ecaef036b4e63d675ddf1864a43c6799d18f2d028 \ - --hash=sha256:7c3fda041e7078802589cf257750323ee3d0cd1e56e53a9b20ec845697fb3d28 \ - --hash=sha256:879331ddea2a26479fa18fade71e6facf684a6cf19f67daec3775c871569e8e5 \ - --hash=sha256:8e8e28c464552b5ca03e30d4504168c4425ce383884f8611b00e972f9fd933fc \ - --hash=sha256:90ddaf7c625307ad52f31a9b25c34fe5e4897c7529ee3481135822b2b6842ff1 \ - --hash=sha256:954d971b363b16ee41f89389a4053315dc71265f2ce5c2468eb0a910b1166268 \ - --hash=sha256:9db18a9dc0af52135c9eac549d80a7a882696efbe5406cf882b044525d4ecc2e \ - --hash=sha256:a0e4e92eeb088f1d7c2c04d6c7de8434c75abb4b4ccf0bbcd045aa7164c68d93 \ - --hash=sha256:a6ca849f90cf73fe361f08a5762c783ead9671e4548c1f558cc637b54c9103f2 \ - --hash=sha256:ab6914db225d7f399652ae1f08588dfbc9efe617612715701e3d9d5cfa5ca19f \ - --hash=sha256:c2ba350957076b1b3a22f549261dc3e9c67ca20816d8bd5f79d7b9c69be4c4c2 \ - --hash=sha256:ca77c43ca55bfc9a4eeb1f0cd5f093f08731b77c24cdba0829035f084959b0bb \ - --hash=sha256:cc903e1069e9dd5e9dcf780324c0112e27e051e422ecfaff574fb33ed65d9160 \ - --hash=sha256:ce28748cbeb0f29c3ce9603782979c7117580fc76f16aa3ca448b38a22281adb \ - --hash=sha256:d58798c4d8d629700058e9afc1e16b9801023f3ce4dc1c92d945e79b5ffe4e98 \ - --hash=sha256:e2a1856e9565fe2679863b372478c681806aebbf7d0a6e72f33e77f804e647d6 \ - --hash=sha256:e3b190ba1d3d22a5a8758597f797111b77d433473744352a184a5ee0a42d672e \ - --hash=sha256:e890816e5ee89c74a0f8b9379fe8b5ba83f46132b2a0bbb9b1c21359ec30dfda \ - --hash=sha256:eaf9e7cc7ab59f6c760232bbde18f64d559bbc50544841303bfb32be53533297 \ - --hash=sha256:ee341973f78a0b46e073d065e88e75026a9c584051e97f98a0d05d96c6bac7dd \ - --hash=sha256:f1c1b4263fd13abbc339a16f2bf19f3a5cbf2a620853d812b1256f03c5342cb8 \ - --hash=sha256:f7444ea6975c49a857c68f9bd8fa11acae96dede63d120ffb3bf0a603ea82516 \ - --hash=sha256:f800e9e722c145ccd18012d82a864cb21bfee4ba4ceffde77100d25eced511a9 \ - --hash=sha256:fcdd1e04982637c6042337d3e24d472f938f01fdc502e2b994844b726d12c3f4 \ - --hash=sha256:ff1e816af7abff71f289242e109217036723ce36aca74ad6691e52d964a74afa -pyasn1==0.6.4 \ - --hash=sha256:9c447d8431c947fe4c8febc4ed9e760bc29011a5b01e5c74b67025bd9fb8ce81 \ - --hash=sha256:deda9277cfd454080ec40b207fb6df82206a3a2688735233cdcd8d3d565f088b -pyasn1-modules==0.4.2 \ - --hash=sha256:29253a9207ce32b64c3ac6600edc75368f98473906e8fd1043bd6b5b1de2c14a \ - --hash=sha256:677091de870a80aae844b1ca6134f54652fa2c8c5a52aa396440ac3106e941e6 -pycparser==3.12 \ - --hash=sha256:a076f3f489b39b43ed13563a1dafe42447445bec2e294154ec5c57fcdda4c85e \ - --hash=sha256:f03387877eab82ce01bbdfd0ba7252ee55a90b1c708d5129e2ce915475cf1db6 -pydantic==2.14.1 \ - --hash=sha256:9195d967ec791692a04438115466764fb8b9a27b31f14a760437694f40d6b454 \ - --hash=sha256:94f478203dd03404682a1ada216965651dd74b1d2d5ffd62e00e0837caab5c26 -pydantic-core==2.50.1 \ - --hash=sha256:0036473f5583e6a60e50b8b21651511564277a3f05cc5dab8cf579f552cd5f6c \ - --hash=sha256:0048b6dddc8ef4b64fccaad878bd143b0c3882ea9936279dc11d613f6b7dd1bc \ - --hash=sha256:009634b83993777ddcd69cad0ffcace43dabde692109528e35f0fde91e386a8b \ - --hash=sha256:028e2f212273d4a39b1ec1e0de8166b1165a65fc0f1111452a9d94fc7c625c63 \ - --hash=sha256:062e891facce5ca296a1c37098e5e466780457f86413894b399f0cf22934f769 \ - --hash=sha256:06e01fbbfdb9be777b316a71b6c49efaf4a08b615d0a98d678cda3023f79d019 \ - --hash=sha256:06ead20d39ffd6f2f6f2a8f8a6de67ff8bb1b4f14a8a30e058502514ee2ac685 \ - --hash=sha256:0b3a6f334c6a2345ca15318ff894502a90012536404b37c844a976c76c846e0b \ - --hash=sha256:0c003c3b7f49debb893d2d85ae099ac5959c9839e2f330fadb1fcdf7a6594482 \ - --hash=sha256:131059670f1d2444269b8585cb888963994871932447c08b39ac6a51fcfef658 \ - --hash=sha256:132529c83901437ff642f585216831bf5fd7a91df66829907e155192ead62498 \ - --hash=sha256:17e722e156d0444ecaefbe640bdb60928752bf2013e2b7a11cdb099aaae19bec \ - --hash=sha256:1a9006395dece0e32e704c315eff8a00bede494f6108546cfc5539c89fef4f9a \ - --hash=sha256:1c8632d4ac04e6f91128fca584b3a8a507d81604c24eeaaad00d4be42765c32b \ - --hash=sha256:1c96fd793b73d1b92e65570132505498fe7b21eaef73cdf74e67e5dfba7ac9e4 \ - --hash=sha256:1cf41f1ae3fa155cf167a72689ad044bcc1e3c97e064123677149bdfb5dafc4a \ - --hash=sha256:1deeacb112d14d3f4fcb16b165f7dbaf76c70ba6e82f37ba042bdab51970a0b8 \ - --hash=sha256:1ef800dd7d85bcdadf4c3076e4c94e43939493558a3b69a1ea830c706d4617bb \ - --hash=sha256:1fa4c8bc12c1354c5550c0c35c1852c8c1901e89e06561724e03f8d0342e1f87 \ - --hash=sha256:2005207aafe1231315718bf6ed5d064a7300fb4772754af35ee72fc68159492e \ - --hash=sha256:23923ab9292c40da026330b1ecf4dc2618c8e86e0422e5d1fbf50d94d64ca4f8 \ - --hash=sha256:23edad659e8dbd8ca7e4e877fe6c81573abbdf215bd25a68b53e1272f58b80c7 \ - --hash=sha256:2ab756b72bd5054e4c7ef3ded331b35786cbd3cf931531a508f79a9537517064 \ - --hash=sha256:2cbd1b75b09e976ed0d6b6ca297675632ca35df86130088457cdc60ef36970ae \ - --hash=sha256:2cf91809d0721ab81592ba67bea7694821679c10b1a2e3c3460082b286c1918a \ - --hash=sha256:2df1ff41884de2bc4b307bafd7c40a691094fad2ff8e767e5b45a319257bcf4e \ - --hash=sha256:2eb75304506894a281d346220a4f7481a1b8729577c5ed2a05395991966a8396 \ - --hash=sha256:2eedf82ee4753cdab8e50044c6bd569577eebc3859b11fecf4eb9223761ff966 \ - --hash=sha256:30ddf019d082c117b5d309e5b86710c2a78909907ec1a9381feec3eec02eca0b \ - --hash=sha256:325c23f3e35cfbf0fe3486fa5f7260d1e45885173002d30a28ca019994124255 \ - --hash=sha256:32fad3a91e51b6d2039c572db04a5a873260b399f6bd62c3552671fa7a4a2899 \ - --hash=sha256:36c426eac0af8d1529ff8467e612b933346caec1fdc0d774f78f67a1a11e16c1 \ - --hash=sha256:3a5fce22f1e87d181e924e12da7d81cfe031fb3881a5ddf26ad28f141756ca43 \ - --hash=sha256:3aa9de446b793de2beb6fa2d9d0961803126c4e2a99c2f25ab59b9fd6ea125c0 \ - --hash=sha256:3e46a9eb0a0901dd6275e6b06ac3a464885ef350ec4121fe486869de8053e4bb \ - --hash=sha256:3fde4fdc6487a58d944ca87cf5adc95d5f266e872c19599f5f4c0a8a1b1f9f9f \ - --hash=sha256:409e0ea40ec30d9158f33574fd758e689f6045a0f2596701828c27816ca9687d \ - --hash=sha256:40f523349960fa30f3ea51404308ff50f9997a90df639590f47a057c1f32b415 \ - --hash=sha256:41bc8237121bd8dc8d888dfd6279fc166ffc88c1f1bf3a8bf00869680533ca4c \ - --hash=sha256:42b54c2c90ad348b5e3a85e03e715d572c1fde357ef104cdfe3b03b697a404ea \ - --hash=sha256:455a773617b5913bf5c20d0692e5787b119e52c4d40ea644ca31f5758fd31be2 \ - --hash=sha256:45b11cac094aa25725581d9304eee93c9028516b9ea80dd9e175e13a5a2c840e \ - --hash=sha256:45c6266d071c241f2a168d45bf8c54344f0effce35e7e6b73afdec11f3687568 \ - --hash=sha256:46b3301d3b5c886f77de7546e47274a5842c622ea2020b8c6524c6b66913b4a6 \ - --hash=sha256:476f6ed8e43cd1e0b460920e23571700872b284e77331cb30c4faf459cf48a4b \ - --hash=sha256:48569b0ade9edfbe065cad1d700175546592aebbb42f02adcebcc26e75b896fe \ - --hash=sha256:49c2cbb2397fe4d0987e84606e691af6cb87bc0ee1bd3e7b737f7e10b4c142f9 \ - --hash=sha256:4a53d13cdfbedbfa87f08b83c1a0a5efcc767d785a4b41934fa9cb672670493a \ - --hash=sha256:4be846f55c9477f5f3ddde8f2ce941137e16862a56d018ed885d422bb6ae02f2 \ - --hash=sha256:4df197990c15b5a37c5a277d131d9f2c67de6133f2e5dafd80d9bba4b99f46f9 \ - --hash=sha256:4e834f6a8e4ff772dcc34f58ef5504147a3ea5b0f4eeb13b0f8eb2ca75ac57f1 \ - --hash=sha256:57f51b31ff826e2859120cf4737c5a758a48d96f3e97da40ccee1796d58078ff \ - --hash=sha256:5958c72adb417c39b12ac87525ac60b0d73315fcdc59e21f44ee4a5e2512c9ef \ - --hash=sha256:5dfe41f232befddb9c4377f6cfc702b51595e2d78ed082672adf8758d2c4619f \ - --hash=sha256:5f3cae32fc46121f787cb2486de9cf95a8bf72aec5cc78f64c606fa1735a6ef5 \ - --hash=sha256:64f6047f62a6c5ae08d0a6afb035667aa2d97c3d20d69762e034c5ea144d92a5 \ - --hash=sha256:6a733778df2f7087ec1100ed0b41533e4f3001976e99570fa34f57c66e7f8e3e \ - --hash=sha256:6b20a4bffabdad0db2927ac034ae3b8a681b1f7a0182f3e60b479ad2fde21ebb \ - --hash=sha256:6dbcbee53bf17196a7f745aa9bf5a9603953a1e365b1f020be3207c676a3e7c4 \ - --hash=sha256:6ed4f3cef55164b026fefb41341b7754cc6b624c75dfe7142d2ecceb5ad21c87 \ - --hash=sha256:704075d10b74f2f3c6e15407c696d88701df35fc8953f434a431add0d0074db0 \ - --hash=sha256:739dc730e6be3bd5ec2f4ab5cfc7eb047cc45fc1497b3bafec74ff2ed07df597 \ - --hash=sha256:7456d699b13954e9c0164dcb267250a10ae0dfb03e6e26d6796ab0d46e189c84 \ - --hash=sha256:756d669f04e62ec4148ecfe22be6a4484d9b1181a6ef32e205ebfd200540858b \ - --hash=sha256:7689580e72a642ab5ec64d5f55b2e33636fa43b4ebe63c0c2c965ef307c7d1aa \ - --hash=sha256:76e2e83fa6ec8cdc972d438dafc2522b3a47bee4ec0ae668b29cfb1977ab5242 \ - --hash=sha256:7816e98acc08119dc0f340ab167048ecc54126316330c1f0caf7c6756c88e28f \ - --hash=sha256:79490e33c4c0fcb933bbbcfc3a62184d8803b99f535863dfbb925e1bcb6945ad \ - --hash=sha256:7f476456ac2bb0d937f75191494a09c83a30765fea4f70f3b404942fe25f6cdf \ - --hash=sha256:844b869f118e22a41a091bdcedda8a71bc1b0f62c38d1a0c3211cece47e1d8fc \ - --hash=sha256:84bc765b282a9d5b7fe0348b8648904f25a6a04b2139da52b1dd30c8ac3a2c8f \ - --hash=sha256:84f34323a61a365b4e9295de6028474754829aaddd59c7bf1a040e7487ef8f3c \ - --hash=sha256:8812592c85d0edf423f10eadcef42716d71e8219085ad9e85b775057b7306133 \ - --hash=sha256:88e492e8b9d0312e7dc13667c30222abf284dc3b79b5302b3607b41a5784ce61 \ - --hash=sha256:8a6791afa2245e6c6b180122d105941644f5bd410bb18623b408808cc41a3102 \ - --hash=sha256:8b4c3df25bd323bf1d36a648d563cf1fc69d717451569927151bdad7cad07a77 \ - --hash=sha256:8daa7ee75245d43ad7d747e5c9ecc1b1d06552f72b14887e9276f787d57375f4 \ - --hash=sha256:93ba4e9d8210d941c200431a56b2c0400b131865947903937ed3ec5404307d2e \ - --hash=sha256:94845ff54dc5193f228cab81b2662a04bfbb892e95bdc15edf7399000ce57d54 \ - --hash=sha256:94be440c03fede26969a5ce75468e0e6a9927a1b46d9b679ee8adc1b057b0350 \ - --hash=sha256:9572c1369e9c9da2d64a7b7992c786d90ff295abc93964cfe3125e4290768070 \ - --hash=sha256:983a662de2571cb2502fc8ff47b6770b03d025d2eb314c92f77b3f07c74720ed \ - --hash=sha256:992c3514ec891fa7858099183e4d64e6bd5a5d4ff452fae29df22faa77a006bb \ - --hash=sha256:99ba9bc2b8062ea0c326a990f7f00e6530c23579de66dd246e72c4cafef950a5 \ - --hash=sha256:9d1bed94af6a63835461f3cf7502058eb166c58c4778e11d0f433cfb1bd69e19 \ - --hash=sha256:9e4472072de0137ee0d8e72d6620e85939c271d2f90f6bbb4b15c24638b79f92 \ - --hash=sha256:a27c09d86600f1bf2fe3f37e1ae697faf3143931c09322cd799da94deee923b5 \ - --hash=sha256:a29a061fec0b4e2d714f277e70a3a18125ecff803f2fea6eade2f2e53711d112 \ - --hash=sha256:a3cda0e538208e5d722bbf3698b24f19c0a7d05bc8d5f8a7f9b121ea7fa243d9 \ - --hash=sha256:a44101320cfe99432db74237545a63057dc7a88dfe792cbcad0647f2af56cb81 \ - --hash=sha256:a4aaaa791bdae1c972a7e81765f4f3571c926b8e0b9b6e47346499fb80079665 \ - --hash=sha256:a51eee75939cf811ac09b278745a6cee7dc873ccfbc8b9af3cc88fe4b7ce25b5 \ - --hash=sha256:a7c58106de36ac6a56314182958de20db8d3a29dfd5db527192cc754e4f8e7fb \ - --hash=sha256:aa8224f10880d9bf1b5993988ba153d42a8b4f3f4f511f93b1f09c93ff613c72 \ - --hash=sha256:acbf31f37c53a5ac0c34706c80b4f5107ba20b05fdd3816124bf236ef0c57dd2 \ - --hash=sha256:adc06d218a1cadfd2ec4628424d7d79ce4eba69c2965e7e7b55106f0da5208c8 \ - --hash=sha256:ae28183297fb0d2b8dc46a1f01d51f5e45825fc5afe76a835a6cb7fb34821295 \ - --hash=sha256:b0135bcdcaa0f23573f286e4cb5e0fd2962700964ed13df085b85f2b97aeab9e \ - --hash=sha256:b087b1c5be7ac687cf22eabfe4b6b608d40df23610651e93611e1f49118baf84 \ - --hash=sha256:b0d955195bbbe489ad343fcc956eacea9357b79cb22192c66cacdefcbc14b32f \ - --hash=sha256:b281a3b0f0822618fe5e3e0d8a2048b6356b14388505dc9374ccffeb69989713 \ - --hash=sha256:b6d0c2183008c188e19f4906d426b293bdc4f67ab17df8e180fe16cda208fa71 \ - --hash=sha256:bbce99252ba3167b2b6277f1829d5bf4b43b754524bddf7f944707c3db7d2253 \ - --hash=sha256:bc1f08f68dac9f9e83845a8039880aba2ab553eb9b2259c3243a313182c253fe \ - --hash=sha256:bc94f474417604bd383d2cd445d071b07dd55fedceed3ce33407bf1fcc107290 \ - --hash=sha256:bed5163e03b98bc1fa2eb05d74c63d9c5c95d8ed6254985481640fbf5e237dea \ - --hash=sha256:c17799a62c142d61b8a3c51752a7cbc87fe2ad4ccfab10e628a77b405075c662 \ - --hash=sha256:c18db21573bd2c6489f9a544b7499f0df2853958c568e5e783536ee1f690af41 \ - --hash=sha256:c3ede305158e75510be50869b319550ab072008c13d64d4ab1e094fb286b6f44 \ - --hash=sha256:c516cc5367ca3448995d42cb994bf3f4c9002d2a7c22eac9622551269ad1b807 \ - --hash=sha256:c531166c42ea7bdfecc8c50049581f05dd1993b09cc7c52bb36a14e96deaec7d \ - --hash=sha256:c73622ef819328873b53109ee4f77ceb598bffedd02daf916102be3228866b78 \ - --hash=sha256:c8dce1f1e0e5358b682a6ad3fa5e31b31d4560997b8e61417e9217c8d60f8a0c \ - --hash=sha256:cb57f304525a5e3c13333b772bf9a473f36326e9c821b2e8e1b2fd36f80ae2c3 \ - --hash=sha256:ce8c25ca38cc0e3d7753ba180808de2c0c8cb24eae0df64491e40921454e9831 \ - --hash=sha256:ceff0acc940be2715bd6ad17b24c0e5304abf44f6efd0f81ee8499e640f9dc86 \ - --hash=sha256:cf356f70551d40374eaffb1aa63f1eb6d2006681cbd7a9faea173ce0f4dd7cd2 \ - --hash=sha256:d2d82aa62521c55ddfb000ae70f88cdd8de974078f6024e821dfe5addd0c818f \ - --hash=sha256:d32f3acc081cc3923386d88f422cde8892335e95f034e0104bb4cf9310d9915f \ - --hash=sha256:d4193206b6587047437f6f11d7e776df23e1c1e23af2a54d9347275614791e10 \ - --hash=sha256:d5c0e32fdbce7f1e8ef4d11f655694bf5f4175c757a9f1dc2be09b8864e5bcf5 \ - --hash=sha256:d5e062c01286d861fd6a1c4ff6e063547b3e713067f2df033c0ff97ac2ca006b \ - --hash=sha256:d8f9e8a6c4ab04b78d61f78627370d834eb004b2869dcb28cfffa647b4ea1980 \ - --hash=sha256:d939de9c82e2126f7f48a7e658f8a85ed46d57662d53f44c49b8895fe94a3eb7 \ - --hash=sha256:de531ce1e2a3364e8767878b58f4ff728a434b4fde089781fe30b1e08e2396e0 \ - --hash=sha256:e50d7b94baac6c7d09927fa5ca5800a0c7ee5015c7fcff65beb3a1931b5a6e09 \ - --hash=sha256:e5faeaee74a57d32b3ab3aebad2e348f06d3ba946fc5d28c1728455f00a3d13a \ - --hash=sha256:e6f0cc1bb9900dc558960894adeb30b0c083366fc1d69b856209fb2ca5c36fe5 \ - --hash=sha256:e8e1d6ce820aa23317e8209a86bd65a540973c12dc7552b48a4f6c8e9926815e \ - --hash=sha256:ed1e728b39a383c81035b2459cfcb35d99dfb01f7d6ebe3a913bc1cc5b81e459 \ - --hash=sha256:ee6db2fbed51a7991302e8fac498cd67e336246026d0dfa84cf5166ce1412760 \ - --hash=sha256:efbecf43d321f7b9281441f1f213f7c21c66988b0e06c2730ba13ed47a46bb08 \ - --hash=sha256:f2c634642694e6a0dad2ab1d375589fa671fd442edd5caf7d9737b8f6ca22906 \ - --hash=sha256:f3377c8c2b3ce898423c5e5dd94c7982e30aa7717a7e6ab2470b9de364963709 \ - --hash=sha256:f5187624823423e1d1b82b1072ac41dc837389e18d3d0572cc19bbee46cd550a \ - --hash=sha256:f77ac30b19221cd9bd3fcfa3d4614eff93140d0572ab730cded17b64adca05f3 \ - --hash=sha256:fe90228920fd8ff2be62622b6bb8a2b11acd65046d50c6b130614b5879605a20 -pygments==2.21.0 \ - --hash=sha256:2363c69b61c4a97c838da3b130dcd6468f4848992b21a82f2a63ec34377137d9 \ - --hash=sha256:610ca751c9bc2492b38eb9a38a7fbc93edbbb2d7182edaf34e66ae493dee5c8c -pyparsing==3.3.3 \ - --hash=sha256:928ae7e20211f3b6f3915a72f06a0cfd29ab9d24279dd6346b6b1a7146397d36 \ - --hash=sha256:ece8c00a69cf01b45d0b1dedabb469c90d8caf996d4fda40f147627a122849a4 -python-dateutil==2.9.0.post0 \ - --hash=sha256:37dd54208da7e1cd875388217d5e00ebd4179249f90fb72437e91a35459a0ad3 \ - --hash=sha256:a8b2bc7bffae282281c8140a97d3aa9c14da0b136dfe83f850eea9a5f7470427 -pyyaml==6.0.3 \ - --hash=sha256:00c4bdeba853cc34e7dd471f16b4114f4162dc03e6b7afcc2128711f0eca823c \ - --hash=sha256:0150219816b6a1fa26fb4699fb7daa9caf09eb1999f3b70fb6e786805e80375a \ - --hash=sha256:02893d100e99e03eda1c8fd5c441d8c60103fd175728e23e431db1b589cf5ab3 \ - --hash=sha256:02ea2dfa234451bbb8772601d7b8e426c2bfa197136796224e50e35a78777956 \ - --hash=sha256:0f29edc409a6392443abf94b9cf89ce99889a1dd5376d94316ae5145dfedd5d6 \ - --hash=sha256:10892704fc220243f5305762e276552a0395f7beb4dbf9b14ec8fd43b57f126c \ - --hash=sha256:16249ee61e95f858e83976573de0f5b2893b3677ba71c9dd36b9cf8be9ac6d65 \ - --hash=sha256:1d37d57ad971609cf3c53ba6a7e365e40660e3be0e5175fa9f2365a379d6095a \ - --hash=sha256:1ebe39cb5fc479422b83de611d14e2c0d3bb2a18bbcb01f229ab3cfbd8fee7a0 \ - --hash=sha256:214ed4befebe12df36bcc8bc2b64b396ca31be9304b8f59e25c11cf94a4c033b \ - --hash=sha256:2283a07e2c21a2aa78d9c4442724ec1eb15f5e42a723b99cb3d822d48f5f7ad1 \ - --hash=sha256:22ba7cfcad58ef3ecddc7ed1db3409af68d023b7f940da23c6c2a1890976eda6 \ - --hash=sha256:27c0abcb4a5dac13684a37f76e701e054692a9b2d3064b70f5e4eb54810553d7 \ - --hash=sha256:28c8d926f98f432f88adc23edf2e6d4921ac26fb084b028c733d01868d19007e \ - --hash=sha256:2e71d11abed7344e42a8849600193d15b6def118602c4c176f748e4583246007 \ - --hash=sha256:34d5fcd24b8445fadc33f9cf348c1047101756fd760b4dacb5c3e99755703310 \ - --hash=sha256:37503bfbfc9d2c40b344d06b2199cf0e96e97957ab1c1b546fd4f87e53e5d3e4 \ - --hash=sha256:3c5677e12444c15717b902a5798264fa7909e41153cdf9ef7ad571b704a63dd9 \ - --hash=sha256:3ff07ec89bae51176c0549bc4c63aa6202991da2d9a6129d7aef7f1407d3f295 \ - --hash=sha256:41715c910c881bc081f1e8872880d3c650acf13dfa8214bad49ed4cede7c34ea \ - --hash=sha256:418cf3f2111bc80e0933b2cd8cd04f286338bb88bdc7bc8e6dd775ebde60b5e0 \ - --hash=sha256:44edc647873928551a01e7a563d7452ccdebee747728c1080d881d68af7b997e \ - --hash=sha256:4a2e8cebe2ff6ab7d1050ecd59c25d4c8bd7e6f400f5f82b96557ac0abafd0ac \ - --hash=sha256:4ad1906908f2f5ae4e5a8ddfce73c320c2a1429ec52eafd27138b7f1cbe341c9 \ - --hash=sha256:501a031947e3a9025ed4405a168e6ef5ae3126c59f90ce0cd6f2bfc477be31b7 \ - --hash=sha256:5190d403f121660ce8d1d2c1bb2ef1bd05b5f68533fc5c2ea899bd15f4399b35 \ - --hash=sha256:5498cd1645aa724a7c71c8f378eb29ebe23da2fc0d7a08071d89469bf1d2defb \ - --hash=sha256:5cf4e27da7e3fbed4d6c3d8e797387aaad68102272f8f9752883bc32d61cb87b \ - --hash=sha256:5e0b74767e5f8c593e8c9b5912019159ed0533c70051e9cce3e8b6aa699fcd69 \ - --hash=sha256:5ed875a24292240029e4483f9d4a4b8a1ae08843b9c54f43fcc11e404532a8a5 \ - --hash=sha256:5fcd34e47f6e0b794d17de1b4ff496c00986e1c83f7ab2fb8fcfe9616ff7477b \ - --hash=sha256:5fdec68f91a0c6739b380c83b951e2c72ac0197ace422360e6d5a959d8d97b2c \ - --hash=sha256:6344df0d5755a2c9a276d4473ae6b90647e216ab4757f8426893b5dd2ac3f369 \ - --hash=sha256:64386e5e707d03a7e172c0701abfb7e10f0fb753ee1d773128192742712a98fd \ - --hash=sha256:652cb6edd41e718550aad172851962662ff2681490a8a711af6a4d288dd96824 \ - --hash=sha256:66291b10affd76d76f54fad28e22e51719ef9ba22b29e1d7d03d6777a9174198 \ - --hash=sha256:66e1674c3ef6f541c35191caae2d429b967b99e02040f5ba928632d9a7f0f065 \ - --hash=sha256:6adc77889b628398debc7b65c073bcb99c4a0237b248cacaf3fe8a557563ef6c \ - --hash=sha256:79005a0d97d5ddabfeeea4cf676af11e647e41d81c9a7722a193022accdb6b7c \ - --hash=sha256:7c6610def4f163542a622a73fb39f534f8c101d690126992300bf3207eab9764 \ - --hash=sha256:7f047e29dcae44602496db43be01ad42fc6f1cc0d8cd6c83d342306c32270196 \ - --hash=sha256:8098f252adfa6c80ab48096053f512f2321f0b998f98150cea9bd23d83e1467b \ - --hash=sha256:850774a7879607d3a6f50d36d04f00ee69e7fc816450e5f7e58d7f17f1ae5c00 \ - --hash=sha256:8d1fab6bb153a416f9aeb4b8763bc0f22a5586065f86f7664fc23339fc1c1fac \ - --hash=sha256:8da9669d359f02c0b91ccc01cac4a67f16afec0dac22c2ad09f46bee0697eba8 \ - --hash=sha256:8dc52c23056b9ddd46818a57b78404882310fb473d63f17b07d5c40421e47f8e \ - --hash=sha256:9149cad251584d5fb4981be1ecde53a1ca46c891a79788c0df828d2f166bda28 \ - --hash=sha256:93dda82c9c22deb0a405ea4dc5f2d0cda384168e466364dec6255b293923b2f3 \ - --hash=sha256:96b533f0e99f6579b3d4d4995707cf36df9100d67e0c8303a0c55b27b5f99bc5 \ - --hash=sha256:9c57bb8c96f6d1808c030b1687b9b5fb476abaa47f0db9c0101f5e9f394e97f4 \ - --hash=sha256:9c7708761fccb9397fe64bbc0395abcae8c4bf7b0eac081e12b809bf47700d0b \ - --hash=sha256:9f3bfb4965eb874431221a3ff3fdcddc7e74e3b07799e0e84ca4a0f867d449bf \ - --hash=sha256:a33284e20b78bd4a18c8c2282d549d10bc8408a2a7ff57653c0cf0b9be0afce5 \ - --hash=sha256:a80cb027f6b349846a3bf6d73b5e95e782175e52f22108cfa17876aaeff93702 \ - --hash=sha256:b30236e45cf30d2b8e7b3e85881719e98507abed1011bf463a8fa23e9c3e98a8 \ - --hash=sha256:b3bc83488de33889877a0f2543ade9f70c67d66d9ebb4ac959502e12de895788 \ - --hash=sha256:b865addae83924361678b652338317d1bd7e79b1f4596f96b96c77a5a34b34da \ - --hash=sha256:b8bb0864c5a28024fac8a632c443c87c5aa6f215c0b126c449ae1a150412f31d \ - --hash=sha256:ba1cc08a7ccde2d2ec775841541641e4548226580ab850948cbfda66a1befcdc \ - --hash=sha256:bdb2c67c6c1390b63c6ff89f210c8fd09d9a1217a465701eac7316313c915e4c \ - --hash=sha256:c1ff362665ae507275af2853520967820d9124984e0f7466736aea23d8611fba \ - --hash=sha256:c2514fceb77bc5e7a2f7adfaa1feb2fb311607c9cb518dbc378688ec73d8292f \ - --hash=sha256:c3355370a2c156cffb25e876646f149d5d68f5e0a3ce86a5084dd0b64a994917 \ - --hash=sha256:c458b6d084f9b935061bc36216e8a69a7e293a2f1e68bf956dcd9e6cbcd143f5 \ - --hash=sha256:d0eae10f8159e8fdad514efdc92d74fd8d682c933a6dd088030f3834bc8e6b26 \ - --hash=sha256:d76623373421df22fb4cf8817020cbb7ef15c725b9d5e45f17e189bfc384190f \ - --hash=sha256:ebc55a14a21cb14062aa4162f906cd962b28e2e9ea38f9b4391244cd8de4ae0b \ - --hash=sha256:eda16858a3cab07b80edaf74336ece1f986ba330fdb8ee0d6c0d68fe82bc96be \ - --hash=sha256:ee2922902c45ae8ccada2c5b501ab86c36525b883eff4255313a253a3160861c \ - --hash=sha256:efd7b85f94a6f21e4932043973a7ba2613b059c4a000551892ac9f1d11f5baf3 \ - --hash=sha256:f7057c9a337546edc7973c0d3ba84ddcdf0daa14533c2065749c9075001090e6 \ - --hash=sha256:fa160448684b4e94d80416c0fa4aac48967a969efe22931448d853ada8baf926 \ - --hash=sha256:fc09d0aa354569bc501d4e787133afc08552722d3ab34836a80547331bb5d4a0 -regex==2026.9.29 \ - --hash=sha256:01000ddf0e3ffef97f2413ceb514f6313040106b6d18a03ee00a4fe35c1eb1db \ - --hash=sha256:0166844493626c5015c6088ee15c9ca2fd060ca15b7641d1657da6a58432ae33 \ - --hash=sha256:044265d77d94f5e3cb2fd72c76723807c429cb8c533e9d4672d0334a6f14f588 \ - --hash=sha256:0476e5bcbe6e1ba3d1c4cc7bbb1c3ba78e3b979b5c8a88d0a6a8cdd4992b8c84 \ - --hash=sha256:066d0e3dbfdd739bce2bf8c2a41dd16f73e3d8adc2eb06dd803a36a307f56075 \ - --hash=sha256:0b65c72739f981377c9c22e0c5c3cd7f42da7bd8a3c9209330fac772c7d893ed \ - --hash=sha256:0c992c19cd45058a4b92f68f139c93db168b48fb1f322c9a7cd620806afb6b51 \ - --hash=sha256:0cc63b5e47c12a48d90c7e9d7de6a035dd14f62868aaedbb4e0ff8ba2b8bfe7b \ - --hash=sha256:0dd8af32e9f7b56b7f95cc1fd79b23054c3bdc172392ae560acc24d57b7ffe71 \ - --hash=sha256:0def9fb6abac55492d6d51cddb7225d07d6f279e774e0adc08569a54a5fc8d46 \ - --hash=sha256:0fd2c901cc307a745ad4bc87f20060d7a0825a3371d1e93488af22e7a387f78f \ - --hash=sha256:1043aedf5917caa861bcb25a9c11460049656bdf0017a90a309fa8f255467725 \ - --hash=sha256:121a76a0985db80ceae9e171c337f8c927868e37d01b54e3ce87bc87f9c6a208 \ - --hash=sha256:143533cc4b6fbc5b95aca0a5b8d541088d374831593def000ec89322c220221d \ - --hash=sha256:14e953ff3607c92d7675bf79c4d4509ef6782aa8c08509f179f9b3d6d0679e86 \ - --hash=sha256:18ae8eed4526e35bdb754d61562b90bf5c00a67fdcf3cc1380dd59597486631b \ - --hash=sha256:19959129885356df0e97556856f77eb2888380dac18bed075a7c05c5128c618d \ - --hash=sha256:1ba8c6a416569ce0d37e83e28a254a61dc99a419084dfb6476cea02d997f74fa \ - --hash=sha256:1c2a0026062abcc321a53db4a185ceba0b59a66b5d37b0808917a88b55a5257f \ - --hash=sha256:1d9fe8091b2e89d470df68a9331111ed008ae8aae6bf1e8e1fba4086a495c84e \ - --hash=sha256:2089fe39c406784d90101c726755ffa1497bb74638fd434300d2b88006186de8 \ - --hash=sha256:23ae6fdad9e63e54038f5ef78aba2933faca61e24d432786589e737bc5522ebb \ - --hash=sha256:26ec4ccce55aa533fbd603d08911b01101a8fcfec987845ac3ae2c7087b2bde3 \ - --hash=sha256:2f7f7aa47b229f2b39a2ae2596d2ad5625d77b5eb9856fac2dab3eb506cdd0a0 \ - --hash=sha256:31b003f9a070335e2a8233ee9b14a3ca8e6d792012ae011f741bf0aaf11744c5 \ - --hash=sha256:32ab11df9677ca80bcbb5fe4eb1da9109a5019239a054836efc6fa1c64e683cf \ - --hash=sha256:33026515aebc0e70d1c89978e53e8d695d35d9e472f8d5b34465ba3c74028650 \ - --hash=sha256:34b6925af9853bf461950e6508910f179fd6e9b1a7ec8548e069606b7e51a26b \ - --hash=sha256:352cf115a810b357caa35193ab656ecf5ef41056855e82f292c99e8514f8d954 \ - --hash=sha256:39ab5894d971f9ac68baa6eca5c50387db579cfcacf36ae8df3feceb1815e6d0 \ - --hash=sha256:3a21a9509d0ee88e7a70e1ad228cd2f0e0fd1e187458db132e8a8d18c97daf9d \ - --hash=sha256:3c5c2ef13797466aa64170cbb66ad98a32351dd4127694cea7199f80f213750d \ - --hash=sha256:3e778bfccd63075167709136afbc251c1f683758d5bf49c803c60ac3f894ce6b \ - --hash=sha256:3f1e6cb402a89457582cd696f982559217d13484a193202c394015297968c86d \ - --hash=sha256:42e82e578c904445d4c8a35b8f28052cf567593215fa5db06266fbc6f77aaa2e \ - --hash=sha256:4408b2b27a95ca8cc48b7411945753773353b5c93b307754781086c99d3a576f \ - --hash=sha256:446654b29bfaa30500d80947eda42cef1449dc8a87f4e3cf061cc8485d3a1f0b \ - --hash=sha256:45010bcfe66df41522d56c9b6114e87ecc597a08970ff6a2ced24415c141ae5f \ - --hash=sha256:49ee178ca31c94621294bf9b8b676a92a2e6bba8af0529591753719e57edb621 \ - --hash=sha256:4d7d93613b01b0199961330e49cfc52d479b3d5776c56c691db31130c0a07d91 \ - --hash=sha256:4fb41211d2333eb930a51e0546a65999761cf1f572a4da56ef9b8a62966c06f2 \ - --hash=sha256:4fe97894d1b306c919b4e50def1e6f6c522f4d03a7283811f4d108f1ce5d3ac2 \ - --hash=sha256:554bffadcbcb6d5f4e5fb10a61cc52084b9a63d1dab5f10bcd2c4343972e8e2c \ - --hash=sha256:59b49507f47479e299a9e1bc41b5cb83a7afda0540625f1dbae886615978acbf \ - --hash=sha256:5eeb8edc6110d9194a4d0d54610f64c37a31c605b5dbb7e407fc6ec7fa34a4a1 \ - --hash=sha256:612b709381c0355b70d89cdb51b7f670591ed5cbbc0e3b5337488019dc667b65 \ - --hash=sha256:61956f074ecd123f55adca68ee3eab46e6a07ad3f8e64e6db95dfacb444f55c4 \ - --hash=sha256:6398d5145689503412cc1748895242598d8846b8967b851133b20dc2ed1e21e8 \ - --hash=sha256:65b408d8fcb273e3499e7ef2ce796810da1becd208c7fb4373692a242d79d461 \ - --hash=sha256:686ac5350fceae63830bb98805fcb8039325bf4c06d9f6f048ff65229d5bffa5 \ - --hash=sha256:6a1a824fbed817e0a891103886b68f063b1e83cc51bc97192a90a60195a9291f \ - --hash=sha256:6abb75ab16bc3281714a5b99548a2225db70dba1f995f6d7f7419b76eb5a8fbe \ - --hash=sha256:6f7121a8914ed13fcfe2099f895341bfb789f004d4c5a0bdece8fa667da10849 \ - --hash=sha256:7020ed44df30b3aa492c00ee3b52d0548c1f30c2c6c5bb13ae897680900d3413 \ - --hash=sha256:720537c7ea6f80dc61913184edb0ce2497a306b39ef19f28505b322553d52bdb \ - --hash=sha256:724184b4aafed865e4f13ca313fdcb43024300c028ec67319cfa16847d84685e \ - --hash=sha256:7c03031610e3e6ed1768a2b7a8fc84637c1257b50c5eacaf094c6e17a84fc563 \ - --hash=sha256:80a5ea3b4fd9d6a5b9a44f7976a9acaaab35aa3c1f6b29e5bd857dfabaded223 \ - --hash=sha256:80c7cadd3fd2bfde5df8aa0787e315812cad0c313a753095d02f4c2b6c01677b \ - --hash=sha256:80ea96f5c1a30bf09007d48466521d9c294bebe197c708c3359096e3e3691632 \ - --hash=sha256:864e9b87ac33c3fb9fb4ad48166d4fdb579c351d5c77deb0d34bccb36a775cd9 \ - --hash=sha256:87fb80cbe3557e27e7b28b995c2b2eedf689b8886f941ab93e0e288f0976518a \ - --hash=sha256:8873c4a11c50b9989168881aeb3f08859f469d809941866aa1feefd8be5431f6 \ - --hash=sha256:888d60953908dcf761aa320c3e390ab8556efbdb551ace63921de90f6ae0848d \ - --hash=sha256:8b5fcc4771732191b2b7d1dd68d8f0353f47f8d90b6150f6dce58bf1112442cb \ - --hash=sha256:8f39588af4731c8923c26810eb3b33f76f17633985e40f59c3cd45a33805a895 \ - --hash=sha256:9173db3be74a35cb6731701094b98120f7ee4876a287882a59cdea1fa7da342f \ - --hash=sha256:92f05c9c42bde5785dc48770bc2194d9f7442544156f951e19cd31b096cec562 \ - --hash=sha256:951733b1bbdb71e377cec567b409f1a7881b47cfcad84121aa74cb575fa425ea \ - --hash=sha256:957bb708e8057ab1649ba566456429d691ec9b90d1c9ad1af1ba7ffbbeaf05f2 \ - --hash=sha256:9916fda742cd4eede63b286f58c06718324265d727ce0856eb1aac86d0d150d6 \ - --hash=sha256:9e1d3a4cb7993b708f0ada8d0c84590efd853f169e7147d2202c9da503180242 \ - --hash=sha256:9e4482589065c8ecd761cff522dcd85f2d39e62f551e37e025d1c7d54772def3 \ - --hash=sha256:a5300757f8a68f5b6cc33f57338d72a0e3589c5cc9ad5f8504ea06f028be582a \ - --hash=sha256:a540abfab208e1b7ef2df231c40ef3b6cbb30a0aad6204e9b6a81c10a6794628 \ - --hash=sha256:a5758353650079898dc1b2b0e95aa51fa23a30d020e06f62c430dd08ee56cdd8 \ - --hash=sha256:a64b85a4760337cfefdb27d42da6ed8b58e8cde3f2d57b6ef43e76ef6ea9ef47 \ - --hash=sha256:a655d34b2a6943af32401f3d94f72e9d731f6ad16285815550bf2b4ee69d420a \ - --hash=sha256:a714befaacbd10092ffe4cea0d3c5f008fb9efe9bc322c715bcdfdee414b9a3d \ - --hash=sha256:a760da040b47767b4b873adfb7c3b691e9ba2fc60f113f9d0b88f1a62f323e85 \ - --hash=sha256:addd736a0547d553283adaf4e05d7104e7f2c7b0b092e9b4d28756825f14531f \ - --hash=sha256:ae4613d7d9dda60fcba95f846cc6f808017f1843f392cf9daad14a6534493d71 \ - --hash=sha256:b11b589e00095ec69cf79841a76360f9b079e95b0368a25b5ebb951ab0c157ff \ - --hash=sha256:b3e445b66c80b4eb4234e855ce94d9adc183eedbd632816228d89930b91b2c5b \ - --hash=sha256:b7b893976e7fe42053da64f2aa27239c24252fd2ec6df471e1be197c0addc3b1 \ - --hash=sha256:b84f186a7f0536fe4ff9a9fa12d06d007b9b71d4b5352ddcc41f59ad6522a312 \ - --hash=sha256:b89efc38431793d28b7cd91227e2f952ad7c48df19132b17f43a5fec3c14143b \ - --hash=sha256:b97a38fb4c732b6832db6bf108963adbcd82ef1268ba2025dce390f45af75efa \ - --hash=sha256:b9d74e4eee9ddb64c2e92d5d61472c59c21684c059eb7b68767be9628e977859 \ - --hash=sha256:bb90e7177944b6684738c1fc36aabd2dd00d1de3be7dbe09f91e196f1bc0dc81 \ - --hash=sha256:bec37990e3d6121f29ecfb594bd8f1bf009e9f7926daba2e50e3b27d3892a783 \ - --hash=sha256:bf3c49863c23a1ad6da9c30351aed6cff8d5ddbeb63c5c8420ae54e98c7d0138 \ - --hash=sha256:bf48516e35cf848390ea68850aba53e7c333720d2945b4d2c25b69fc5171723f \ - --hash=sha256:bfc71e6d970419c1309b3640305298643e2a734cad3f7cfb6d2ddee4175ab53d \ - --hash=sha256:c0094897d7d01f184b2d7fe8c56c66d64efe01b31f4b7d34205b391387df1111 \ - --hash=sha256:c03c6eb6ece86dfdcbb34799efaa339b093132e1aceed491ba5e08fe06cdf699 \ - --hash=sha256:c1a9a6651197fbed6f0212591418b9def774fc3f8324f78d1bf0e6a63e5f8aa1 \ - --hash=sha256:c3589f40749acce747510bf5d589d54e376cb0930ea58b35effac97e5312b0c1 \ - --hash=sha256:c4e38dd8f39c43a91d2410ad2b85610701b0979342c3df1d69eaf8e838c757d8 \ - --hash=sha256:c6c8fabf1dafc1f1ddcbb67896d3f93efb092e8c4b6322d7389b944e76a484e5 \ - --hash=sha256:c90fcf7804ea0a54b896ce0f2b9565350220b8d4890fd0db461a476a4c687963 \ - --hash=sha256:c9b602fae1e00b7c035d661ce85575365719192a7b46784bd71cf64c68053aa0 \ - --hash=sha256:ccb64d887a9db1cd76dbc0f92051a1a478a2a67e7f56c62d915cb881d7734704 \ - --hash=sha256:d06fcdecc10fc7954d7c8f27a03c96055fe525274dc84a7b0dbdc3d6b9e03dab \ - --hash=sha256:d0c3082bf79bcd6a614d55916590ad4b8f93200e10b97f463ea5d9d07c9b5f23 \ - --hash=sha256:d49c18f1ea294cf4adde2e5ac256e98c82ea9d708462ce4bf799dffa7cfe8a2c \ - --hash=sha256:d60030baaa7bfbb02d650c126cdcddcb6e33dbff14d819434c8fa2fdcaeeeba5 \ - --hash=sha256:d7cab119d0df0b9413f106b4d7fc34f2872d3574ed3806fb48959c830b1537da \ - --hash=sha256:d9b77b25b4f395f92de6099ab08e8ae2bc7e51dfe157f22900902243a5cc90c7 \ - --hash=sha256:dabee8f4935e731fb46b2a3091bdda0d3d94b3bbfb907d2b4f12eefce4009619 \ - --hash=sha256:db5e82ba15c142425b8406690032df89e39cca4a2e8afbbb9a3d84edc2373ac3 \ - --hash=sha256:dc79d36d0618752265f0d575915bdc5c5130ecb9c9f6b3bcefeae32e4bdfafcf \ - --hash=sha256:ddfa987262763c3c22a8367d2a49c244b018a74c3a8e3ab1a864119ad45c5633 \ - --hash=sha256:e1172147d28d8fbcf8cb8d26c41506169f5ad8fe9ec969cb116835a19d4d8eca \ - --hash=sha256:e11edba5bc344a32b029a7af9d4b3173982dd79eeafa0b9dbd787364414b0509 \ - --hash=sha256:e2c89e9b762c57f59d5e99ee8b20202adb892e35f8d3485741340999ca55058e \ - --hash=sha256:e31f72490b7c12f7790e1e25c3afffd20503ee1bfb43461d7838b871ff244b19 \ - --hash=sha256:e8c65ef3862a8ad6e86492b6ed9327805dd66904c012bd3649dc67d822ed6c34 \ - --hash=sha256:ebb8912f565b8cdbbf27debfe00df04202c20e2f651b9e32767930c5eace3621 \ - --hash=sha256:ed511a0708e2297e1d6431e7fb217e3402791e491e02da800658ace4973df1bb \ - --hash=sha256:edf06545875f3efa31560d94121e95c7fd70d98b1dfedc0157097d79b13b52ea \ - --hash=sha256:f0fe9834e5aeccaf19a0d8feb296d66a24be1a7c9922002f842a682cd5abb787 \ - --hash=sha256:f1a0d5117230dd46b399a30a38afa44f79c99f3168988fdc4f425c3f928b39df \ - --hash=sha256:f37964e4a5e993d2fd45147741e9dff7f34a2d8c00ab94c4ea0514a4677f959e \ - --hash=sha256:f57dc6b8fef170f105d2cf5cdce254f47b137d7755086cf7050f47e16582abba \ - --hash=sha256:f93bc1c3486ef3747e07c9d7c1d0a147b8fbaab975f80e348aed6f71309dfaca \ - --hash=sha256:fb00027a09a8f9f08028b40dce4c933cf73e4833240ed356583fdc9cfa721566 \ - --hash=sha256:fb99cc9d45f48895d9d67f6a0b8a57f08d39c174d9f25ad97a313e0470267b1c \ - --hash=sha256:fdd88ed5e20b1bcdd234421e454962c971aa44b653bdb7f1ea9ef683e90fb649 \ - --hash=sha256:fe3fa1dd453ed5c7f5ea23a26218329790ed7197a99b90e94330e313959a7f52 -requests==2.34.2 \ - --hash=sha256:2a0d60c172f83ac6ab31e4554906c0f3b3588d37b5cb939b1c061f4907e278e0 \ - --hash=sha256:f288924cae4e29463698d6d60bc6a4da69c89185ad1e0bcc4104f584e960b9ed -rich==15.0.0 \ - --hash=sha256:33bd4ef74232fb73fe9279a257718407f169c09b78a87ad3d296f548e27de0bb \ - --hash=sha256:edd07a4824c6b40189fb7ac9bc4c52536e9780fbbfbddf6f1e2502c31b068c36 -s3transfer==0.19.3 \ - --hash=sha256:24eb7535804306355e31c952a90022bf5b2f9d760f9a11062c7493a217c08553 \ - --hash=sha256:8326acc3d056d7f7b8a37110c7d2aea41f67e83f6ae79cd3b5fbcca34c726406 -safetensors==0.8.0 \ - --hash=sha256:040070828e36dc8e122178bbbd5830ff9e97920affb84cbe0f46442497bed358 \ - --hash=sha256:096ec1a98435df7beb08853bb5aa9081a84f23d0adc67ed1a0a10550f608373f \ - --hash=sha256:2ddf52eac562eda224f99acfa7889d02968c1fd59a5b011ae7d8137c37e9c02d \ - --hash=sha256:3ae091f16662658bdc019a4ff6cb4c085bb7d725eb5978b183ffd265863b6d2d \ - --hash=sha256:4124502b78f03534117c848f87a39b8f31e577b15eff423bf8bfb95f2a8c30d0 \ - --hash=sha256:4a95ae2b05d7726d751da4ebf626a2ca782b706e101bd894c95bc2450b1cffcc \ - --hash=sha256:7a46e5ff292c356d6991e60942ba7f79817682d3a2cef0702136448cb9c4d235 \ - --hash=sha256:7bc0a787ba8a35be368ee3574edfa2b1ad389eebd0a72e482ae275490e3f6c98 \ - --hash=sha256:87eec7ffed2b809f05a398a8becb7d013f19f7837cd15d9748580d6cf30dbaf4 \ - --hash=sha256:8e080062fcde23be189565e1c3305d16751a218ecf9412c8601e64204eb6f846 \ - --hash=sha256:8e9f537aa183a38ace122d27303dcd986b26bd2a7591f9181d7f0c396f4677ca \ - --hash=sha256:c554f85858e05226d3c2828e32395e677434685d6d94594a41643361c5e837f0 \ - --hash=sha256:c80201d22cbf405b80647a60ada77bba06c8fba2da2743ba1e89cdcc39a81f25 \ - --hash=sha256:f7838e5135a406ad3e02efdcb8cf2e5397d368b0154537c4fec682dbc544d452 \ - --hash=sha256:fabaf3e0f18a6618d9b36560682562157f77c2b71fcffc7b432be2baed9d753d \ - --hash=sha256:fcdd41ec4628fee5799f807c73c353629130fbd942aa23d83c623dd6c9d52d78 \ - --hash=sha256:fd6f3f93c9a0a7cc2788ee63fb763353d4bd2e89b0751bc78fcf7dda00bea774 -shellingham==1.5.4 \ - --hash=sha256:7ecfff8f2fd72616f7481040475a65b2bf8af90a56c89140852d1120324e8686 \ - --hash=sha256:8dbca0739d487e5bd35ab3ca4b36e11c4078f3a234bfce294b0a0291363404de -six==1.17.0 \ - --hash=sha256:4721f391ed90541fddacab5acf947aa0d3dc7d27b2e1e8eda2be8970586c3274 \ - --hash=sha256:ff70335d468e7eb6ec65b95b99d3a2836546063f63acc5171de367e834932a81 -sniffio==1.3.1 \ - --hash=sha256:2f6da418d1f1e0fddd844478f41680e794e6051915791a034ff65e5f100525a2 \ - --hash=sha256:f4324edc670a0f49750a81b895f35c3adb843cca46f0530f79fc1babb23789dc -tempdir==0.7.1 \ - --hash=sha256:b70640b48d5379585b85ff7dc46a989a8a82ef1292dc22c057a79cf47842256c -termcolor==3.3.0 \ - --hash=sha256:348871ca648ec6a9a983a13ab626c0acce02f515b9e1983332b17af7979521c5 \ - --hash=sha256:cf642efadaf0a8ebbbf4bc7a31cec2f9b5f21a9f726f4ccbb08192c9c26f43a5 -tokenizers==0.23.3 \ - --hash=sha256:114e2b55ed177179d59f4ab98200a4471e11e78f9e4b5a922d146740f96fcf52 \ - --hash=sha256:1554a6eed34d9d6a78d23360f4e06df8dffab1ae08c7e8488e0b3e3b36cc266f \ - --hash=sha256:1ebf28794e7e4954e20a7f70fbea410b2d1f0418f7dbbca97ca384fcfef38c25 \ - --hash=sha256:1f0823bb00c5fdc98e487354d54dd55a03848d61a1a0bf29a68c77f24f3b26c3 \ - --hash=sha256:2a89614730d7b80940a5d2ed9320e1ec8add5a745c6151d8d05071b7215505b6 \ - --hash=sha256:376851d22bcf9d650a5c3090bb83e6cf9e895fbf0595369fa4cd43c1f69b5f87 \ - --hash=sha256:5cc24bb457dd4a8af89c8fcb40074d570129ec473df2a866c276ee55db4749d7 \ - --hash=sha256:68649e97d5b43c44c031d8d848874a6eecae8f8fe40ea989aa777a5a83aca716 \ - --hash=sha256:7e48734d2de9260d86f03ab056d2cfeeff3869f61dbd49aaa15a2793b5f3458b \ - --hash=sha256:82eb480f6f1c21cea3349dec32cf1a6384c6c1e775f00f83b0d51197bc013687 \ - --hash=sha256:84513ef0aeb8bf8f4ea11a2e8a7ac163ec5288aa115e649a59b470ac5c3107df \ - --hash=sha256:9d2b5c97daf61688c2ad1803ca851800feaba50fb68d5821779e9ea5880d968c \ - --hash=sha256:a4fbb3662f9f59d199d61338e54b4bcc11d07ebbb1aeb3540dacb2be9c521cb7 \ - --hash=sha256:acd5c57b4bd3e56e246e2731a3a3a6825a7a7d89b7e3b761ba80bc521710f04b \ - --hash=sha256:bf501c40b72d2d5c8623620210430e9cac1ce47a46e45b34107b70a1557d46b0 \ - --hash=sha256:c64a0713180ff16829d4e7f39a658b77ea11443af4e1aa46523692943c9b1414 \ - --hash=sha256:cded33237c77caeef62944d32aa9a7ef42bdce2b3497e18d137e072a8c4be438 \ - --hash=sha256:d3407fb7b9c4d75dd68850ffd7180bc0a5d2dbaf0762d888e612f31fec3f9c6b \ - --hash=sha256:ddedfd4b3b4be6be24ff6ca645c4a37fddfd305f6f3e354c54cf10b715c48215 \ - --hash=sha256:de536665495cb4b409d25bade41963f801aff4225c19a6b804b048f7d14e34c7 \ - --hash=sha256:e05ab7baf7f47b406a95fea6f3b0a484b2ddcd9e1d14b68844c457eb755085a3 \ - --hash=sha256:e88646b8580c5ad7f4361477f1298e9cc01771a1ee9aecfe32c47b8ff614cc38 \ - --hash=sha256:ec82e80e65a862275b97c3d90b7a523df8d9519ee48aeb4e9625b2cc909274e0 \ - --hash=sha256:efa3d7318406b4d115dce61ad5061953f1f44b128e79c020ce4615d763e23b6e -tqdm==4.70.1 \ - --hash=sha256:c293e525e6fef9c20e8728fd4612df02a0aa31bb5fe91ecd93e123b1b7bffa73 \ - --hash=sha256:cefd0eca11b2a37a3aee776544d4f4ae913f02688135b5556b8788dfa474afc4 -transformers==5.20.2 \ - --hash=sha256:080871e061546d720be57f5846fce426f46b44d61574907cc7404c2cba49d180 \ - --hash=sha256:7ae75dbc1fb3922aff1b6c97b9e5479d6cf26fd86a8866bfb71324190479f807 -tree-sitter==0.26.0 \ - --hash=sha256:00289bfe7978f3e0dc0ce69813a20fa9f44ea4c100b3ec62043e5eb74ccfc3a2 \ - --hash=sha256:0f8793fd18ad7eec276ed4b51c097b4bf2002b357259b66b0d75db1f3f41c754 \ - --hash=sha256:10f0d4eb94aa7242dcb7f554bcd24dd7ba1c114f00d58759ba08c7a46c8ec51a \ - --hash=sha256:17a1c5cfd3a05d5c7c86bf4282b6ef8092c91dc0a98390499669c3fedb7d1814 \ - --hash=sha256:1d6fe0e8fb4df77b5ee816228e2c4475a63d8cc1d4d3a7ffd7097b2b87fc3e95 \ - --hash=sha256:253df7ab82cc0a9d311cd65f06e9f99fb3eac55996ae9fc94da22f123a861b90 \ - --hash=sha256:26c996c1edfee86e977bb3f5462e74fcec0d0b0db1e85a3c475875763caa03be \ - --hash=sha256:2f941cea06128c1f74f8937a8e2a90c7db49cf4be6647cd9e07d92a306d91517 \ - --hash=sha256:30a88be89ff1f2755297f81e8080d88b795dd98720c3f9fa2acf93873182cc95 \ - --hash=sha256:335294ce0504fcefde5245dff596778ffaf820205b98ae0b549c72e48855f1d8 \ - --hash=sha256:3f3c44339dd34fe8eb2b8d5aa7610660499a795f70376b130bbee7a437337280 \ - --hash=sha256:514a9bf8993e5210e7970736aaf6020d1759b670e195ef17b1c48f586aa30736 \ - --hash=sha256:526a165a2cb1d1f79e247d400f0e0acd8d49a817d6f312d543513af200b1f886 \ - --hash=sha256:5a3c93a352b7e6f70f73e121bbfa2d0117ba7478bd51114ed35c91b0b78814fa \ - --hash=sha256:5a6b333b0282d8bb0af741f9b018bd2523d4eecb2686bf6717066a625fecfaa4 \ - --hash=sha256:5fc2f41bf246ff2f70a9cc3690be35ec7580a4923151873d898c8bcb1a4503d3 \ - --hash=sha256:6189c6c340c7384357711e3d92645e96bfb79f7a502f86de1ebdb23eb43f7dab \ - --hash=sha256:6cb2bd20efb2544c19ac54486ab7cb8ec7b36f913bbe1ce95df84acb96743d9c \ - --hash=sha256:7075ef857ef86f327dbb72d1e2574dda78db5754b3a1fca6506acd7fe5d561a7 \ - --hash=sha256:763627db05db34f12333081bd7422cc1c675893d373cc870b3e9249e200700e4 \ - --hash=sha256:7bcbadfa614326debef581957d5c780a9d7f66065c13deea61aa21d1dd36263f \ - --hash=sha256:823251c4b6725a7c03ed497a339135ede7ae4bdde75bb8be7ef5e305aeb4ff52 \ - --hash=sha256:8ff2e0750b7daa722302838356d7b65e303829b7eb73c915df127ddba115e1d1 \ - --hash=sha256:918d89529786873f0982a0f59c2a303cd065fbfd1b903d71a8e4e1584f67b42e \ - --hash=sha256:93e220cab7e6a823efeb2046c49171427de92ef71c7c681c01820d14d8d3721f \ - --hash=sha256:94550e13b6ae576969da40246f4c4abb206380b5375ad43f26dd9151d55438e3 \ - --hash=sha256:a4033fecc8f606c7f2e8b8014d0057b74668a7f0152763606f7bc25c5f9ec64c \ - --hash=sha256:b31a8195d2f224224c530ac814632d98c1dcc123d227442c07c736e86b70d564 \ - --hash=sha256:b40c219edccc4564530c96f8f1556f6202b37cda964d1cbd7bd2b7e68b40a245 \ - --hash=sha256:b8ea92a255c91671a7ec4625aba3ab7bb5220c423630ffbf83c45d7312abe084 \ - --hash=sha256:bc6cb01d5ee75c85424aa1f1c72a82d8f07fd52539a0f3c4a6ed3e8721079b84 \ - --hash=sha256:c56581ad256c4195a21bfe449fed5d44a02fe83a4a7d6e70e6ec302c881191c7 \ - --hash=sha256:ca89e361a276dbc934b28a43dd881199e25d34ff5493ee0ce45f3c52a6124a37 \ - --hash=sha256:dea4b4e27d49e9ec5b785d4f994da000e6726882fcc6ad05ec98478500c71aef \ - --hash=sha256:e9e46b664887d8c1014f1fb33e09454bbdd9ec1fe29b7fd02dde7b46bc1bb81a \ - --hash=sha256:ed0889dbed843ce45ede9f5169c0b2dea2222f12685844a03fadb81f12705867 \ - --hash=sha256:f289be0225ba2ace8e87d6c9639b2bc9ff2b5271afb7c5d39282a4a00e248682 \ - --hash=sha256:f665510f0fcf4636fb9696f1f7853bed7a3bd764b7bb0cb8494e619c14ed5a0c \ - --hash=sha256:f9997ba61368c48ed54e715676afadf703947a1542464e39d047764fb3624b01 \ - --hash=sha256:ff527388df14cb5009f9274faf78cc69a7393ae6acf3b04784b8acca249519c5 \ - --hash=sha256:ff80d4833d330a73184a3ac5132abe93c575d2dea31975c6f15c0d21fef238aa -tree-sitter-python==0.25.0 \ - --hash=sha256:0fbf6a3774ad7e89ee891851204c2e2c47e12b63a5edbe2e9156997731c128bb \ - --hash=sha256:14a79a47ddef72f987d5a2c122d148a812169d7484ff5c75a3db9609d419f361 \ - --hash=sha256:480c21dbd995b7fe44813e741d71fed10ba695e7caab627fb034e3828469d762 \ - --hash=sha256:71959832fc5d9642e52c11f2f7d79ae520b461e63334927e93ca46cd61cd9683 \ - --hash=sha256:86f118e5eecad616ecdb81d171a36dde9bef5a0b21ed71ea9c3e390813c3baf5 \ - --hash=sha256:9bcde33f18792de54ee579b00e1b4fe186b7926825444766f849bf7181793a76 \ - --hash=sha256:b13e090f725f5b9c86aa455a268553c65cadf325471ad5b65cd29cac8a1a68ac \ - --hash=sha256:be71650ca2b93b6e9649e5d65c6811aad87a7614c8c1003246b303f6b150f61b \ - --hash=sha256:e6d5b5799628cc0f24691ab2a172a8e676f668fe90dc60468bee14084a35c16d -truststore==0.10.5 \ - --hash=sha256:30d36967ccaded5cbb38d602c433f53600036c79d502f4533a49b60a03bbefcd \ - --hash=sha256:9aaaedaefaf06d8b206278cf8b5012bc897f485a874503501e12d776df78951c -typer==0.27.3 \ - --hash=sha256:d0396f770a560ab1b0a8504e13b5f254b728cedb05c61cf0359e944e50ce8901 \ - --hash=sha256:e50022f28b82a86313e54501317a1db64bf8f8d036ff8cfe5ca7e47675454aff -typing-extensions==4.16.0 \ - --hash=sha256:481caa481374e813c1b176ada14e97f1f67a4539ce9cfeb3f350d78d6370c2e8 \ - --hash=sha256:dc983d19a509c94dba722ee6abd33940f7c05a89e243c47e907eb4db6f1a43e5 -typing-inspection==0.4.4 \ - --hash=sha256:547274fa6b0a561ccf549cc9524b999a578e737d015d8709d021f9d0d13bea47 \ - --hash=sha256:65b8397ba37ccbce054456aaccddfc91e6e3083c92824df348d96ca832f3f147 -uritemplate==4.2.0 \ - --hash=sha256:480c2ed180878955863323eea31b0ede668795de182617fef9c6ca09e6ec9d0e \ - --hash=sha256:962201ba1c4edcab02e60f9a0d3821e82dfc5d2d6662a21abd533879bdb8a686 -urllib3==2.8.0 \ - --hash=sha256:0cf3cae568d36aa9576b28dfb35f11328f1cb974ca7647d9475ebb86c75ac6e3 \ - --hash=sha256:63bf2ead4c879426ebf22ef2a781eeb4aa3b4ae798a0435506f8687fd5bb9b63 -wget==3.2 \ - --hash=sha256:c00a5fd7032ccf1c4af99f26155f5a379892d2e9d94520b36859d3f010922553 -xxhash==4.0.2 \ - --hash=sha256:00329536c281e13725283421dcdf4633f5710123e06a608b910d6a6f3bc891cf \ - --hash=sha256:03b1e97d14fd1b185c75808f72dfc7d9ea5866df3f8661566f2a23d0d07dcf7e \ - --hash=sha256:043dcb74ba30a068a2821c136b61262db67f1e1d174ff4dbe83a3e24996861e7 \ - --hash=sha256:04fe6c267bb6a04d8e5f56d8058eac0621359dc8fc03faa0c03cf1dd4343304e \ - --hash=sha256:050c1d66587e8f1ce70ccb7f661ec117e01497d48da817c14543beba0b8c677e \ - --hash=sha256:07aa7644899f97e4ff208f54b9d57fe4c077bfdf2dea5bd5a4a165e9b57309f1 \ - --hash=sha256:07abbc84c3b22064294e8d7f8e8d71a3c0dd3ae9aca8dfda55ad1977d3a7af61 \ - --hash=sha256:0b96b2694c97084c5e2bff8a6a3dc3c2fb3b9a07178286e8d69869efe76eeb50 \ - --hash=sha256:0c9ce49f326c85958df18789486d4c28812b52890e74d833b121735200d04bce \ - --hash=sha256:0d57b09c8f01945ab789bf8dad0e78a82b47f8e09b91d740cbefb52480a45744 \ - --hash=sha256:0e1832b53b321f742f46d692ca3c25ef23298e422f146d34b0e0010944fdf875 \ - --hash=sha256:102b2da60cf1b76fa4cbed4b03acaae4c4be1238e534a00fd608ace5027e3653 \ - --hash=sha256:12d5a3c0687c990124a98a1eb027655075deebd985388b4e8ba4302992b90850 \ - --hash=sha256:131d73e1467e9f782048d7f4901eb5ced6997e55ca761b503b598618271da8d8 \ - --hash=sha256:13bc7f3ba5b048fb5191d01f3971ef4c1c04a7f8a22621d6f8e94eb3c7cbfd04 \ - --hash=sha256:144a93e0efa8dafaa2826200edd377680981830fd3082b414cf8a96ba12a578c \ - --hash=sha256:14873a759b512ff875e01e7f8bd187ff613362cf189c44ba604e5cff833a308f \ - --hash=sha256:16e30b4a045f1456af45016a888fda09a45b5adb5606da0463a3a0d58130852d \ - --hash=sha256:17c05e116555586f584c6dce34af2517b6648d56ef5ae7e4d84ea1a19d6743ef \ - --hash=sha256:180b349bbc89af61240adab8519426e0cb6546c2487e4b26b464364bf59dcbed \ - --hash=sha256:1837c43b06cd2823b44ad27e80e51bbceed172a4afd0c3cc2ef543f2be74155e \ - --hash=sha256:1ab636ddc4f257bb2713c078d3f6112f194e92d8a32442233ea957fdddb51de6 \ - --hash=sha256:1b0f2dc4556ed6466707b6343266d90ba6cd6707822f5d567a6f8b51f5ed70d4 \ - --hash=sha256:1ba7e798c28e327b221270af073f7319c771daaf5974d18856c0ed3d0fa1c6de \ - --hash=sha256:1c5242bcd736259c1600cb2588736c736cbbf8872fd7a091379c036bc59c5bd8 \ - --hash=sha256:1d549419687aac3f25c17a0fb534cc6828b5a4c3ae301544b9393127e413ac59 \ - --hash=sha256:1fce8df7958f0828bd78d2ecfd9b11510abff9edc0e28fa7f7ca0ddcee2f0a8b \ - --hash=sha256:202c18875cf93d652c506d0e7c8a50847713f963a8aa9c06ce5117bdd14ba771 \ - --hash=sha256:2076eee9f06fcb611754f98b9c272046e4a8a3fcfde353575dd8e6e2f33e80ed \ - --hash=sha256:250048412cb196e1f705134dadbc415eca399e74d2678b935660be0b2e4fc380 \ - --hash=sha256:26f8343b7a5d65f704029a0ebeb7e98a47149cf76dcab0072f0874307181bfb5 \ - --hash=sha256:28881f55fe85fef47921f4cbd3636bbd75506f1e616b2ae61e266d53c6c56637 \ - --hash=sha256:2889ac17629a786dec7991112b1dde2de377f8035dc0bd9590bd7bc640b01ca4 \ - --hash=sha256:28ae2c583d923aa57c0a4301ea4e071bccf1f84b5e16a31f86b854d9f8943151 \ - --hash=sha256:2a99c2d36df2d9b238a27e1fd8547311dba51c997e4c7681a383a98c0831aee0 \ - --hash=sha256:2acdb1cabc41140e97d4adba7422dbeb7fabef9caace27d56f8de662d505fdfa \ - --hash=sha256:2b5849d45e5c12ddd23316c999eafda73c3ad00b589c474285dcf4135205921e \ - --hash=sha256:2bc6cfaa4a39bfca4c31878ad769986d2d039673eaf527068f668e88ff65ab99 \ - --hash=sha256:2c2d0dbcf64ac20300508b5a7bd763924d58f19feb13cf693922fe7188bbf431 \ - --hash=sha256:2ca60825069a19e381f4ed03d0e036907b2b98460d33d4f8d1a93cf2d7e37c7b \ - --hash=sha256:2cd94476503f0d277e4188cc117f72e7bc92e2287e415931f3d91bf0e13a3d21 \ - --hash=sha256:2cf332a7b823e867cc0d8e4c69eb5c82ae96ed827677fe4d96f990a6a1fd86b8 \ - --hash=sha256:2f164b788017a4a766884aa9acb3490bbc5f03a79adc7d02894393df44675c1e \ - --hash=sha256:30b1798c6e6814504f32e81a0b87072f9b38628b0d610c22d53918e5059e086e \ - --hash=sha256:30ffd0deaf0e9ea112c38656922323ae23dbaa90878cebdb243f646d121e52db \ - --hash=sha256:318e6c3a31d75ccacd1f9b864e3b857d44c9f4bed5e9d5fb5db1dc2fec82f019 \ - --hash=sha256:329bf14bab110bf111aa7211c613a94e97e4554407d2cc964c01338278cd4213 \ - --hash=sha256:33f89d2a64716d7108f49b38d92ebe5344d3c50440f15f48f0f65cbcf4ffb781 \ - --hash=sha256:3473e7ac274a9a63c90acd365dd4b481659c6e061db153851f6e5d7b61e264cf \ - --hash=sha256:360343b1d61efb44be005c2180cb19f8c0351d7dc63c8ebbdd8e219ca5cbbb4a \ - --hash=sha256:3764bf2b5c139932b4951092103be2f7567950b1441cbda69c94107744d9010f \ - --hash=sha256:38ad1aef6d84d644b4a1afed0a7d226449f09cc2de1c9a0d0021c51f680ae2f9 \ - --hash=sha256:3a5529d43d09851dffa38936040f19e938e8368ecaddc646035a1dc7801bfa09 \ - --hash=sha256:3bdbde088f54a5cccb92f0d199e018bd676dbe2e5d2eabf1f9dc8148c6f297e2 \ - --hash=sha256:3c368344f91263272aaf0fc5c1420f0ec9196466d507f8b1ba52d0e29645d1af \ - --hash=sha256:3d7f484b338ddf6a8e58ffb73c4f9d13ea157f01152fdfdf9c44c9a59666f3ac \ - --hash=sha256:3d8b5da050138043edb95fbd4b79e240a0994d2f272e872cf074dc6acd68249b \ - --hash=sha256:3e0d6d7ec5072ed81e54de4fa778e6c1c8fe70cf8ccfc3907abde59147fac84e \ - --hash=sha256:3e229bf3c7fa330c84ca25bbc29a83a6d078ef2cf3b7b826a5c9b3828aac6f07 \ - --hash=sha256:40fab713c1d2f5c5e7450b8731615d8580757599ebf1f8683b211310ed8e5ad4 \ - --hash=sha256:42fcaec8c705d1ea362d4cf011aea0dad1ebe13096ed227f9cfecfcac396c72c \ - --hash=sha256:438fdfda0fb1198b7e9a5b4da6b073f36cec5dd7129dd486bbe72c8ecd623adf \ - --hash=sha256:45b10d7adffa2d5805279a123ad0cf5dc635b7e72542215d0e0f221cf49038e8 \ - --hash=sha256:46fb84bad2ca32eb68eb11882159d0a0af19576026946a185c957d8d306624a7 \ - --hash=sha256:49b93898657ba0dd4517576b5c94fb0aa6738200f6d1ec2dbc9d0981154828dc \ - --hash=sha256:4a5955614bf744ac52faaf535ff94df3e935ec057c6242a3d8aa4e1007efa0f0 \ - --hash=sha256:4c3834e32060ab46ad9cfe4981d48c306cd1f124b8efe0aed559a765f6e1d332 \ - --hash=sha256:4d1fa6f15a913dbafa50f3ee143a13440cd0b9849a1bde290c72ebde7838c937 \ - --hash=sha256:4d3a90ec615e72934be372bf734190626614929054809ac12e59950aca91d8a4 \ - --hash=sha256:4d5f0e2bf5257900f60339cf439a23b35fd29a45cb7effed4b9f1b87c094df20 \ - --hash=sha256:4d86d5aaae8312de06d25b91fbc901b891b3b5ce2f5c8359b340b4bcc3d01b8f \ - --hash=sha256:5065d7e74b7b3e3df03e11b47d9b14bdcdd0171603ae77cf39c89df140e3e948 \ - --hash=sha256:515a5f4405a29bc447bcee2ee13455dfc67ce41bf8f1da5258bc75bbdfb6dee6 \ - --hash=sha256:5223b3d133b81197fdbede79601b9b0f0bdf07818a71b751e9d2480bdba72a70 \ - --hash=sha256:52357b9fee08af3a5c81a793c97110d8be63baba91b88584beee400a2a0113c6 \ - --hash=sha256:52dff0cfe99e1f03614a40867e39d5c5a1d55047e114265db4ec2355d65fc542 \ - --hash=sha256:52fd50828b49caf80824fff67bdbd8cd2db7e033dd8b3edf024c98b326d2a8c4 \ - --hash=sha256:531a1056fdfae837ef56c81e62bb4a0ce3274b813a3164bb558151d913684cce \ - --hash=sha256:5359b14f7467972b50ee264f0a711ec0bdc7d0a3d05c11f5ed3fe462fcbba0ee \ - --hash=sha256:5424d3754239b3080d2c0719dae2e6fb0b0e74cc38125f2722942c054d172257 \ - --hash=sha256:56ee43c284daffc12d269b3fb36b1fc624c7ab007064a24d54099049ee71ce80 \ - --hash=sha256:570ba2952f1fe3b4f7a272314db64903e5c9121032e1dbe20800c7e0e5a13145 \ - --hash=sha256:5846aa6fe15c5b1cda6738ef9b1ba3cf49028b729507ce44a1e9fc2642586b06 \ - --hash=sha256:58d8fc67453d337a8cbeed4f5543493c3792cb5f3839c17bc08ecf307592466f \ - --hash=sha256:5a4e73f8a17c433640d452a94455414a9888b88be0443ca37541d673343b261f \ - --hash=sha256:5d51b2159376d98e99f7a33e19aed3d58d30d24ca641086e2ca0d54875d92ea6 \ - --hash=sha256:5da15e375188f4239277559e4efaab4cf4434162e622b2ba1dea956457994cf7 \ - --hash=sha256:5eb07be4bbf9b0b751576edc37423017e5666d37d24b993b476295ca0d2138cb \ - --hash=sha256:5fb7bfdd0d1136fb73e4efa9a1eaff71a72f45ff7409647f601bf714d7838922 \ - --hash=sha256:6010eab32dd57dcb8070b81a5d459c422d09de3ffd1bf6f11f0ee381582fe5a5 \ - --hash=sha256:60208a7c31d936045e911ff0c3170ac0e432611b6800c220cc66b908c33b16dd \ - --hash=sha256:603eac9723ac2becbf126cf0d237b6ac5cd645592809d697aeced51842ae09bd \ - --hash=sha256:6049b09b27d9226dfe0a8b0dee2b867063c8df9a7ef64748bf48110d8fc896c2 \ - --hash=sha256:605b582e3895fc240b972d9a28606c53fb5444c118c9f6392bffe8795a4c2467 \ - --hash=sha256:60f30791faec3e96c8ed5921045231f7d3355649af47809e5a75c8a17dba8b6b \ - --hash=sha256:61ded7dee4f95da69295c228fc18a65447f5741c45a315aa3f9a3adda6038611 \ - --hash=sha256:61f60d0233612bfe3502a6cb1e233c469ea074e027328d72ac2c63b58fbbbce9 \ - --hash=sha256:63522739ce94e4e73c1605aa60a52b3863e77e835f6292cc8a5e7e7b626ff00c \ - --hash=sha256:66fda3f72176bd7d1d3aa71de1f4bc116276e3ca3ac583bf06e5f561f2b1fc74 \ - --hash=sha256:68ed6108f9ef16bf90752582aadaef4c14caa00a50f8b340e282348365a351eb \ - --hash=sha256:6e736b549811153e3b54779708725c68e105a6e6860f9f30fcba896a5f28ef0c \ - --hash=sha256:7016f753d9c79d2ed53a1517b3eb1bd475f05d026fa3fbc3acec393d14ddd396 \ - --hash=sha256:711b8d7d6a4588c14a0874c3d4afdb0b9cd501a11705e231d79d6660ae937f60 \ - --hash=sha256:730fba89df435ac081bc2ae28e4fa0ca5009a9eb5f4947b648abc27acd96e7b3 \ - --hash=sha256:74a92b9104c4af158ce37021b32cb165aa74e69c39bd10bacf569f39604bf2c9 \ - --hash=sha256:763a9b3447ece8a5a1e9e27749ef3910a2f6e8ac251b8983a3da2e0801956e02 \ - --hash=sha256:768aba59c7fb2e3e976560c91223cb01f00f83809088e451e59f05b841486fbf \ - --hash=sha256:7779b0f388972029f0de4c83dee4523544b3ee33272425683a34c801c8afe6a3 \ - --hash=sha256:78219e648979877aeb1782486f598ca99c5788da388155510daf6527fee57ef6 \ - --hash=sha256:792e7c12ac0d300908a99f644e583f4091c3a5a0035ab7b1b76aadc908480381 \ - --hash=sha256:79cfd336c2ecaceef1061e7a7b3da079eab5920ba1352a28be690e857f67f2ea \ - --hash=sha256:79d832049ab43b88846a13161351887892f3d8cb0bcd3e7c91b3f125caada384 \ - --hash=sha256:7b244e3e1d6eadb5061ce777632a822ed75c2fb6174c65882f5829f31b46e8c1 \ - --hash=sha256:7b4855f9a4b7e2134d248af852b4e5b1381baedc01d70afa9736efb40dc5bc3d \ - --hash=sha256:7bc036ceb030d468473b9e19e3d49617c48b378f92c4cc07eee83966771a2b55 \ - --hash=sha256:7c1087b8dc4a08709e1ce358a79671f9aa4d029211aff537d0f821e4a6483305 \ - --hash=sha256:7c7c95dba490be3b4e5a2fa5595b4d4f088fe7b8726e5998b8c1d83b1f9eb3e4 \ - --hash=sha256:7d66a2b02c02cf5a165454ec62372bc4a97071c2652acc3f917814c2014ccc16 \ - --hash=sha256:7d8d913ecbd21dff2f47f458a8cd37beccb3e41504004341f42234829b973309 \ - --hash=sha256:7f290205f86f984507ab409a018986dddcb0724c4197705edafb80d21be85355 \ - --hash=sha256:8050eb0d892d11308f81b6980d13098ae065efa57ec68f0e9ec44b8bc6b9988f \ - --hash=sha256:8113a8cb51a5121b8017f275e9aff2cb9203d57e614c989f943f81937442896b \ - --hash=sha256:827580bfb04f69c6c40283cb299be15508e151af066aa012f07732e9579b3f30 \ - --hash=sha256:829ae333a35219c24584755b8f5c718b87e4713ac4233c8531a4e5d3d67fa7b7 \ - --hash=sha256:838dae9c2efa4f4cd0be082a1a4077aa3043b407d8bc72f82864cd0807bd9945 \ - --hash=sha256:846189d5cc9a4e6815a04c595148b2a82721da1fcdacfdff13fb0225d0945ea3 \ - --hash=sha256:86fdc2ae36f5d11446896c4456883b3740be2ec3ff56f700a7dc7aeb635e236f \ - --hash=sha256:87255720aa1a1be35256b688c5ee4151d3273894186cda5c8a5b82f1dc05debf \ - --hash=sha256:873ee53a7d8f020ddfcd3bc101707ae46fcd1eb702f809ff29a765d0f88b654f \ - --hash=sha256:8841fb5f071edf93655bd304e7d43005630e71c6aa58d345cb312a867aa1a6c3 \ - --hash=sha256:88fb30a81b5c7cb4b4dac3b4974deb00de06f5ed4b15fbacf5c7e14d670ec049 \ - --hash=sha256:892a66641657cf9aa0ae5c2c88258bc281221a33bd4af34bed786eea23e06d83 \ - --hash=sha256:89d02277a7d5ce8b3fe21182ce4e32cca54963768c4b9609ce29e1ac82922acf \ - --hash=sha256:8c24094617758a4953fbf56e96637262b56d5007fcf34756ecb8d0b60231f53e \ - --hash=sha256:9478e4ae8b1db3f6fd72a5d17fc5805fc0ad576e31b58e2acb6bdbeefda2a549 \ - --hash=sha256:97903f3df852ca5840a26cc27beef15f9f93b0071e2aa550a03d8ee5c65e7765 \ - --hash=sha256:983a14a70a018dcccd847e780ba630771d5525d7604b6f519c963634414180c9 \ - --hash=sha256:987a83e7a7762bd8d40d6ee75bb047a33f6ed34a4269dd6bdda561666c3e0c54 \ - --hash=sha256:99fec645143af9dc697620aebff07e99388c16d4208a2ee8b6a2a9bb7d68a21b \ - --hash=sha256:9a1f6598dc8f7ccdc9bd5ac51264c71431185194e8c88042502a04c1f53b5780 \ - --hash=sha256:9a23f0c1dd31aa4136ef8a678a40e73dbaf070b54ee6f2a907b78ba40331b3c3 \ - --hash=sha256:9c22ed1ea72cc23cec070fc5b9c3574844e67236f3f6e5bf7058e512abefe086 \ - --hash=sha256:aa60df2de8dc4f59ae4faddd77237514ff88d15c2afd6d501e21dbe82a4711db \ - --hash=sha256:aeda53dfb6bcae94adb1f5df4246fe13b9c4202f531dd9d39e0d55b36ba8db0e \ - --hash=sha256:af3d1b3cfa5280e84d8968e9f30c9eca75781db4915eb6d6acb0dd4fc93e0f47 \ - --hash=sha256:b3428dc057d1d483723d597e60e778bc58dd117daf23b866c1a6b5debb8eb300 \ - --hash=sha256:b4c405512c813a47cf70e47c9886b07702a38e991756216b65ac19f9cbb4f8f8 \ - --hash=sha256:b5fbcdf157caa642de731565310bae6a5c3b1e15f28507feb3eddf04bd4a56a9 \ - --hash=sha256:b7f62733495384b424bae45ac457fc80d2143c80de2564416a2a7cd8a4dded4e \ - --hash=sha256:b881b67c01929df867d30fed94d4221205639072a63049818732c1bae54e74f0 \ - --hash=sha256:b91b42f62e6cf6c1aaec9cea6b9e4b8ae18119517242d22ffbd967b275f9d12d \ - --hash=sha256:b97cc2b874d0656e88385fd7ba4c2dedd903f8e23767f69207fbdab79bd0861a \ - --hash=sha256:b9bff92881cfd1d4f98f27e1275f125f90ddf324571ac9a3e35d1dfcf6f42d19 \ - --hash=sha256:b9c7c18d235991edcfa2275889e84a1458dff0b9315cb7351f5368c5b942bc0d \ - --hash=sha256:ba2daf714d2c87374cfdab5ff22736b19cc12a200130b64610feb3b0b08bbe11 \ - --hash=sha256:bb99a2c5c35cad78284e057f8207856c911ac088ff89d0807dccf07b65775895 \ - --hash=sha256:bdcdedbee2922898f4457cf3a86ca857e442770d71e847db36d7df82912184fd \ - --hash=sha256:bdec072bdb457c65e4b96eab7610af72d1535ce38adb32b4e02fa6957d7cd2b8 \ - --hash=sha256:bdee724c52954f6d3a551adc30eac1555a6ab1444dd3e02bffcc51832cbcd5da \ - --hash=sha256:beb27fc243c6975ea44f74ea2e2c8c5014f9956ba0bf95ed6ee6dd3707e0add2 \ - --hash=sha256:bf93a290a49285f75d0d6ec801aa247592a16bd043892a7b48d93f718dfc99c8 \ - --hash=sha256:c04c1aeff5cbe56714dfa49aa5d2edbb69c1a1a798926307b8d6af86549e7534 \ - --hash=sha256:c06c5fd89304f2c274d5129bd3a555b78f7a8af5ea16edd5b1360010f3359bd4 \ - --hash=sha256:c37fcacb7f5b8590c822bbf3bc41690e40a7581d8c4bcea7b15148f0743e716d \ - --hash=sha256:c3f96730e700636db76860d2710147202470e3059d3d37bc018088ebc70b7573 \ - --hash=sha256:c45f489790caf482840d2c108d50b8b0d9e77f2f6d17e2b345e2ae3dc48849ff \ - --hash=sha256:c5d199912f885f381ac518d0bb300dc2c4d543adf8f98bc41f33d9cd503d2493 \ - --hash=sha256:c60fa1038814c3c682a8e3b68aeca598e7b90eb4a65d4a48b7e0e6fbae825507 \ - --hash=sha256:c7a8faa852e40cda4d23dff1243b3fb0e6eb5b1943c646559c2f8488fcfb7b98 \ - --hash=sha256:c9b7b91d072e239d4d0c6d868ae4a2f75dee55856227638488db156a4b462588 \ - --hash=sha256:c9d48e67fc623850ac65b350742b758085ea3aeada2f79aacd0200676abc29b3 \ - --hash=sha256:c9f3129e086297db6b24b83db59a0dbfdd8f325ffc7dcc184fc573473b13293f \ - --hash=sha256:c9f9c25738cbab7ca51763f724db95bc7f50f3174793389fee7f97e154bd9c83 \ - --hash=sha256:ca848e5edfe3aa31ea7355df790b25e55f910dbb540d587e738e1b0c644e1263 \ - --hash=sha256:cb66d346593cb6851498b9ed554582aede153e32c9835edb76995746e563ac6b \ - --hash=sha256:ce0fd7c57fa4fe9e0a9c390a04e1668c09fc09253e8fd76f1785069508686059 \ - --hash=sha256:cf390edab741574cc770e0b587cc2d6f2573152ba09bf85fdae10e556333ac77 \ - --hash=sha256:d42a7b02f204d5133d38d173766ff47a5da46fcf2c3876718b819f1f5500a55f \ - --hash=sha256:d621fdcb127f9cdc87e35388e064f43db519bffef74a3fd34b6cd1f19364e74f \ - --hash=sha256:d63ab508ecb528c9ede793a362411c064e8c55973e7968415bb0159144ba0583 \ - --hash=sha256:d695979588da837a36d5195d10812c4118515d33e4f41a9b4d0cd92bf7c2f716 \ - --hash=sha256:d8dc7e4071794c0405dc6a40559dbe77d7eec1e4bbf6eed5dbebc4e6577d12f3 \ - --hash=sha256:d9cd501f5d6c737e26b8d45f0facff43d3b7f2f5a8c2ef1c88800f6f310da418 \ - --hash=sha256:da1a626793a5fde9c97466d443ae7c62f3d9bdc8658fd8363f435852952996d3 \ - --hash=sha256:dc290f7366f2a31093d31aae3ebad13299d7e4de9daf4ef3e544868bb7755364 \ - --hash=sha256:dd86600a7083aecd5e0fee0680e5a328cbd1bf18046faa217e2a76cf2e01faf4 \ - --hash=sha256:dee8f3352ac0fce896bb0d2493221dad6d6af5b909d2c38d76827d7a6769766d \ - --hash=sha256:e24e9093771e8a27089176f202991199ebedad8f4791a6e2abf88acd5c69bd9d \ - --hash=sha256:e2b98f3de3e6ec78333c6d4db6b8d40634ce6dc05140815a195916385f912176 \ - --hash=sha256:e359ff2f38be3c856a129c1c5c4cbd70e219188456c1bf266fd4f94e2baae278 \ - --hash=sha256:e4111b5d46d1488399562897f7b7b827bdc065bba0355449348a34cc7545fef3 \ - --hash=sha256:e553d545d05094f3950ef36b328f75f5ef8568a0438c26966d9a0f1fc187a42b \ - --hash=sha256:e79bc2bef5c7ca6cf4c2561aab00340d369b80ea56976a4a4c6ce5e3ed8fb59d \ - --hash=sha256:e83c54a9b8477079d6dbd3814ab444f43819ae1aa06b5098c1020d77747897c0 \ - --hash=sha256:e956b1a4e8fd8ea6fdb823670bf6ee3eade44c5ba3bbca06aa67805e4815a9e6 \ - --hash=sha256:ea7b157d98bd8dd2cae312422657a122c059cbbfa67fd7b53c49d8aa2b5f4202 \ - --hash=sha256:eb90b2efd8d76777995ffdf3a134f079a79d59727a3301ca98f818fc142e841a \ - --hash=sha256:ec851cf4a64d98e09101a18190fd20de2928b2ae1541093236664a47fb101271 \ - --hash=sha256:eff2109cbac3a8b04000d1b687331ff538d9d72d266c5aede545fd6bf90d303d \ - --hash=sha256:eff48c792788e33ace8e8e3ef6a0db45741d6acdc35c9b89dc914469fa040536 \ - --hash=sha256:f064c47d2a281378c3d173d47ddd5b9faa4273dbcc40bffe51ff5d6817aad7ec \ - --hash=sha256:f1a31825aa626279c25d797c105641d398865a93e96a042cb89c7faaa8905e90 \ - --hash=sha256:f26362daeb8e71de918899c3f590129858ede3559bc1d251ac26441bd0221ab6 \ - --hash=sha256:f2a9839e13bdb0b611cbe55f6ef865dfcbeff1bb2d48de313fe94b9dc33de1ba \ - --hash=sha256:f2b2839a83b9a9d03ddf89f440bb43627832e77b9a4276988d8aa0bb49aed0df \ - --hash=sha256:f2b936ead8be8e1771b3a4508f5b8ba4087a57c59e3f6e91bbbb6849000ee302 \ - --hash=sha256:f53479d8e13f93e1e3ef0e933a1901cbb4d8745c61c3876ff5b7c544e40f13ed \ - --hash=sha256:f5a509325aea8802d2ab710a5f2fb7c4d7b1a6c90f9a148dbc9f84311a1aa15e \ - --hash=sha256:f64380470531ca49aceb15179ea47dd7be2fc38e41207eca4f7d05e4d6c9f795 \ - --hash=sha256:f6831526d8bacbaa6a0a54673cc63855facfe12424d0e4cd68ee8e089ee60a22 \ - --hash=sha256:f6efd3d8a3ee0b86dd7ce3e6ffe47dcc115f71c38cd9f7392e6387fa541a243e \ - --hash=sha256:f8e86a0b5e28418162dfa68d6dc4bee873ea2efbd6ee67789903a0ee37a81c14 \ - --hash=sha256:f99686036e0575b4bd1947f9efcf12c5bdf310b2348fb72db84a4861287df6fb \ - --hash=sha256:f9f528029f32c081622f36a0149898929c0a89f601f1cdc6620c57cd836762db \ - --hash=sha256:f9ffee37361effc04fc1ae120f00e5dfe2491b05d637cd45e62b8a8b35bc66fd \ - --hash=sha256:fa1febd6e8ad9eedfbbeac0cb5d86346ed0b78a846d822c3af36fddde503516b \ - --hash=sha256:fb3dce09d52103f29a4648649df1ab217e28ccc2e9a9c08e8daab472e5cda9c9 \ - --hash=sha256:fb83c398f50f1c030cce9dd8accfab5fb02ab5f695c157e051ac2a21840d7574 \ - --hash=sha256:fbf323d9c22c6ac25cd8c4633566393282fa22db583ef0320f82bded10d7cc2e \ - --hash=sha256:fdc8799f1a4835b40c615cbc9ce73c8a42f51c3218726d285367dcf853f36128 \ - --hash=sha256:fe3ae82b1c7d031da7bb300188fe31018fbc05e1e496e0edb7c930d6ecf88d1a \ - --hash=sha256:fe5a0d96cb339d383d3f720ffe94d8a7a51ccc8abf4d348a3ee058b5625f948e \ - --hash=sha256:fe765b3c7e7fcd14ef9b77836295847cb73229cdbe7eced958ed013d2f1c4025 \ - --hash=sha256:fe86ae86241d0aee7561b19bf892c9250a976e4a56fc052405cd537180e7af15 -yarl==1.25.1 \ - --hash=sha256:0136d640dfa9b0523853e411430a99f8a91eca85774c6420285a33b755bc6de3 \ - --hash=sha256:03dd38de09bc213e9a8b29761eec33ee1d5318dac0e49d8af36e4d27830e23a7 \ - --hash=sha256:0a191bfdb30a79b98e5d175d75285f9fcb78bf0e46ba5efda042e1c72071a0de \ - --hash=sha256:0a66db89ea473abeac4b70523cafd94db3772380e565f9d28af7a179b7af71fa \ - --hash=sha256:0ae12ff2b805fa02c4dab838005caef735e39986322698c48588d3beacb65c62 \ - --hash=sha256:0f12afda4eea8c8994a76d4df1875c765194f5fbe8a9d197929ea303caee29ec \ - --hash=sha256:10b2fd95332f0d716d5eee3c9fb2ce8eada19082de7fee83d32e37992fd75c26 \ - --hash=sha256:126a2533570c554719ca40a1288fdee1700b6bc82e7131aa69fa85252d92e651 \ - --hash=sha256:12b6bc4906e11f5e1a1cdcb12296e7afbd366c783cc8073403cd2fb74334e453 \ - --hash=sha256:142c06c4d6a35ee3ec5da08499805e879cb3ca7c1fbfbecb0140fe72403818d6 \ - --hash=sha256:14b79a30a93a3ce2e8832603fd0ab780ada281b0ba5110b519a634f2d7d7d1fc \ - --hash=sha256:17c9877a89fb6e2bca6f9087eb24cd7fb434653946ef5075e470d23d49b52287 \ - --hash=sha256:192a866877a49993949ef1975864ad8728bea28ee810f6abe1a0729c2b500426 \ - --hash=sha256:1ab7618921a93767387a4b83776f751588f5b5ae9bb5bc96620e2e2e00bca868 \ - --hash=sha256:1e80dcf1446e1b080b1932b0d103c464a04112f5bc31f0f983ad418172063cde \ - --hash=sha256:1f51020b2eb8a003c84925638ec63c21a750a4bddd3a22ec8eac6a742dadf1b9 \ - --hash=sha256:1fb2a01ba8cd9c5d2c5dc1ec35e0fc951d04b4f037541d4ac090c993ce58b3d7 \ - --hash=sha256:2239a02249d9326655419e0168a28ca9008938eaab31dc29fc875c217927a6c0 \ - --hash=sha256:23bf5b403c879a54964e0feac7285688e04bb220074878d737d331522da0a5bf \ - --hash=sha256:24ce942011a61953e7d313438038f4d32ff21387b775f58a957f7a07dd55ef95 \ - --hash=sha256:25868beca8b6765f8f7d0e11fe6dd7c66dd4b0793b9500286d20cc92352126a5 \ - --hash=sha256:287e99ff5aa4dc1c7630bfc683ded6f106d756c99dec432a2d7f197a784f51c6 \ - --hash=sha256:29273edf1530e397bd07cb784db1fbe0d2590b77569f2e24679a9c0a2d763b94 \ - --hash=sha256:2b49375d22299b0a834c2bca72f39aaecc270d96fb24c30424899676f487b22a \ - --hash=sha256:30eec96e8a91bd588ce897c9543f6d5d8d34b28fbcba28a4dedf20ebeae9fe57 \ - --hash=sha256:319e070a01db9920fb63761843f96a104c8e2b9427266731810dc1e22595b17c \ - --hash=sha256:35dcbea443fafb3eece757ad4e514560ddeb6c34cfae1582c620d7b293d7feee \ - --hash=sha256:3f4d48a6112712973e676bd792121fee470e432d749177162d9949d5c9460a1b \ - --hash=sha256:3feb99222553a8cbedfa52c2f59dd84c3f50d5b582c728d522caf8d72769a54b \ - --hash=sha256:419f392a1da624877975709e3864dfe833af6cc7671b39318086d456e288380c \ - --hash=sha256:42a66563d8cc056ee32e6191e05097a7b2b3bc302e0bc3133daf8710eb18bd26 \ - --hash=sha256:48796ea00a303961507dc6c8437c4b325a6fc3f95f7c36c71b91ea9a8150963c \ - --hash=sha256:4bd6340d20ae2c7ca719b87b426e808e90743b676d05d4c26c4fb5ca71f41184 \ - --hash=sha256:4ca89e4e21854ed27ec753297dde84b16c9f8e53b14a4866fb44457d643c19f8 \ - --hash=sha256:4d781294bb815ecb5ea57ff6bbf8038e0a31a95fdf3e1788f66e0dc100d64b58 \ - --hash=sha256:4f1c91f5a5980a937ff8e238e98e6897e1ad74a4b1e2c0d68c73b5ffbb3f5c0b \ - --hash=sha256:564fdc7085d2245ab84f88882fdb1d6ac0723124bff6ded35bfb1c00f812630d \ - --hash=sha256:59ba3a6e1aa8cfe5adf4bd270fd965db21955401b7ca6f1696010c55ed4daec2 \ - --hash=sha256:5df89f769cc8ff94c3d7e7603386fba309d25ce5240132d26c15baa8d0e96c4c \ - --hash=sha256:632da579b2d879f6bad20f2cfa35ded1efe2f4f77f8abb26a6234a5b236acd2f \ - --hash=sha256:65b5b2066651b7432d389e9799d979c703bcc6ef44266bb8153ef54e91e4aab3 \ - --hash=sha256:664ec6a520b74a1df2810666eb67695fcb77fa663e6ea0a25aaf2e529cb24dfa \ - --hash=sha256:681c758b0490f9e96b78e5fa8e8dc6e648e9185bb6eaebe73183c33ea0c445f3 \ - --hash=sha256:683e362b8ba453080f7489c66f4ea794e751c35b72e7eab3575ef784c2fbc7fb \ - --hash=sha256:68782fdb4027b8d1eee25ec35e9a6db05e863b899eb0310b3a33b6c3fef55707 \ - --hash=sha256:6efaf45df6a849cef613a03a94c845647456662f85438c886bb67a9c027c8c2c \ - --hash=sha256:71f42c5b9a948c113bbdebfa544598321431d064ff959d32e99b1feb61d68345 \ - --hash=sha256:72849d892954be4d09e569b8b831ac39ce58417fedc767d4308a0fe542018a40 \ - --hash=sha256:72c34ac7ad4314c19362d5ce27626dcc8429bd30bbf8c179f4234078851f9492 \ - --hash=sha256:734f6e5400352ac4254456003d462866c684703570929cff7a7bde015d0cb371 \ - --hash=sha256:75baa6cf9b6d1c52f3e111a130e202fd8cf0a5b3a066c3f73d615e885092e4ec \ - --hash=sha256:77716e245c90f058466a05e6a465bb8600f767a8f4b18b4d40f3aff958e5f73c \ - --hash=sha256:77e5099b99b37f3cf79c246998ca9f7313a78054cd1809ec46bc1afad47e1c4c \ - --hash=sha256:783dd1467083f4d3f7722ad6a313f24c173e7571372738fcb7a6e6d1ba48df25 \ - --hash=sha256:7a5c3115595995779ee21f2567035793911c3802a43c74f3fbb0314929ec67ac \ - --hash=sha256:7c88edaec8c349ad4c5ad4c486a3defcc4b80ceb2f074436ffa0a87caf5e76a6 \ - --hash=sha256:7cb414a73e21a7ab58254926073f2930cb22f5b4314ea4260a687e2b3fd4dce3 \ - --hash=sha256:7d42e7e3ca399555578b4d617e3a6ecf13371b3743a115995fa010c7bf341459 \ - --hash=sha256:7d575b54cb3863ef9bc290ea4b009999d55dc237326131e4853cf33e888fee03 \ - --hash=sha256:7e4de3ac4adbad3d0bc7c6f4360a7dbff5de2f15e3b723be3198074e17fd9c40 \ - --hash=sha256:7efc9f082dfed77c316edffa9deb52888e1bc6789171887cc1f68e06d65465c8 \ - --hash=sha256:80a063f8297fc796296f00f100be520f209b23dc98f93ce8eba6ee7122598209 \ - --hash=sha256:80e47012e730da131c9f059c80936783f9659aae22dc31c03c0595590d11ed54 \ - --hash=sha256:83d4a37e4b95da4d8bda930d6d35b75b4cdadbacbb4980cae290ea3100b5d51d \ - --hash=sha256:83e9f4a25085bd4b7214701a0794ff1f50fc633ffb8bdfebf07abdd81c2db126 \ - --hash=sha256:85a18376073f8a39aa07be34f9fc77e2869aa72c55c441efdd2cf79a0407504d \ - --hash=sha256:87796fedc3ba97ec14fab55acb48584276e6c1e4c1e89c422bda62c838e754a9 \ - --hash=sha256:882569ff613758cac762a457a5d72d6e211b28d4bcfea89d1d71ea942b02eac0 \ - --hash=sha256:8ce4d6ccafb33d39bd78444612d14938ead674c25702ded2ee9c54a47735d225 \ - --hash=sha256:8e7d98cdbb6d71e726f7d525952867096053d1f290dd4e3c50d7d313a136f414 \ - --hash=sha256:8ee202350cf57abf0e9502a41601841019c25d3db7ff52d980aaf31446254059 \ - --hash=sha256:8fb0eb4955adf0579001581f2f71a126e8781ba61bcd120f127b0401163c6c2d \ - --hash=sha256:90c30ed53546da833c700115c0064c22120d1b1560f474699fd31f22dd668233 \ - --hash=sha256:9489e6abf47ba37f332075a91444c7cfedb03e6ce99fbb2f116bfe1ce810da3b \ - --hash=sha256:94d7aa6debf92a1dd14cb5280b083a764169a13cfb23a452111160274ed989f4 \ - --hash=sha256:98d370568f393215d605304cdb77b3d5539bd192c75b623c7304c42c8d6d8273 \ - --hash=sha256:9b1bdaae98bc016825dd3c9d8ee1832f829b3341f9cc6ebd1a1b0a7fef7367cc \ - --hash=sha256:9d693bf4bf534e9ba3ae2780cfd577f5135629f7b5ac653490859d0b77864865 \ - --hash=sha256:9d6ed3d17bccce4c05343e1ca8da13bc5c02c812a4e7282ddd05e8769322d3fc \ - --hash=sha256:9e23c82b63cd7652fc24d33ed6cc17099d607aa3b4fc4ddc75e95062f3d82df4 \ - --hash=sha256:a1daf47cd95a7c3a63456336bc5aaa8c86dd3a47d07ed3d0e76132ae4666a5a1 \ - --hash=sha256:a1e32763e641a1566507d90a8d3b19bfc3cc04a9d4e5ae3e32189874ed4b58a3 \ - --hash=sha256:a2059a2d891bd156bc5184e7ab7a56e78a84dfcfdeac8c501b552533ad1c36ee \ - --hash=sha256:a2ed0ba415ccdf08f14bf544cb78346d0f76086707ffee24921a2c84dbf1305a \ - --hash=sha256:a3faadac7d812ddac258feb57b9846b60c1b437c4f4b9ad42595c6f6fe4390df \ - --hash=sha256:a5877f2255aab518ebe528289037699201d5dc5f045f2396cb30aa02db22f57f \ - --hash=sha256:a78b50b4f7918a3de71105d5c0b93bbc57bb8339a4d03a9dfd449f9068e76f3d \ - --hash=sha256:a8c2b841478068440d8b733005d13a5ef535b9928cbc05f17182d410f32ba449 \ - --hash=sha256:a9ca696eb02e5c02a8afd872ada510eba9b7fe6e68b9572c2e9a9b1941e31e2e \ - --hash=sha256:aa4ed3dd308548f9e707d9caaf005d2d7f8c1e7868f858dfeb47fe76e16b391d \ - --hash=sha256:ab2054c5531af2a9ba7b69b8ec91e4f884420e83a8c5e579b013084cb57e5e5d \ - --hash=sha256:abb1384477f5901d436b5d2e5465954de46ea6098f59163d243660b5c4461d35 \ - --hash=sha256:acae6b45d1ace09b6ba3876da43b88366ef368f73b988c7f57e14231753d4420 \ - --hash=sha256:acfa7e22aa6c6e7a5996a41d275bfa01efa7ea56ab890590280e9063e2cf5c1b \ - --hash=sha256:af4ea5b37403ef4e30f3927eaed540db942bde01d8d3ff083527c0704d1c9c68 \ - --hash=sha256:b10dd0557ba422715b5206b3743192135a6022acca8baec51aa127d0a75db8fe \ - --hash=sha256:b13b88747769537f3d32e89e3a735da10c0a9e35d7322928c701b5f93d3afffd \ - --hash=sha256:b51c159a9794633f5e0db7ecec7b2b6e3734eca1f5d17dc989ff3552a43ff78b \ - --hash=sha256:b5402a340723fa7da00b5cff987ddab61276be6d11251ea71ae02bcac54890d8 \ - --hash=sha256:b7abffdf37af1cec6a2ad69b827aa84320db5894791bc8ed932dc93fb274b7e9 \ - --hash=sha256:b8075fe90bc08e40b8b8a1874fab42ee4c7b56af05c5886e9cc841397f916908 \ - --hash=sha256:bc3ac7bf569f6b64dad04dd7808c7872dae8a97df657856eac05e9b7e3614a85 \ - --hash=sha256:bd0912757081f89b107d6c00b2ff8a194401b0b87eadcf4481de2b865a8fd44f \ - --hash=sha256:bdc8d8b8c22e9e43ac68316b5e6cf083dec537f4ec213cb4aa967b583bc3fa64 \ - --hash=sha256:be80550d9bfe83d9b62398a37081a90434e6df2d978ec345c3d2820de6beddab \ - --hash=sha256:c6f117789d22dce188e5754e8bc65b7e6ebf8cb73963b9fa761f672a5883769d \ - --hash=sha256:ca32926d7d77bcc8838425c4c95e040a3ace1cb7dfdae599013458dcda2607ca \ - --hash=sha256:cce0727fd5ac04d372fa9bbfde9febc2bcf209aadfcf0468e45dec72719895d1 \ - --hash=sha256:d0f1489233a254bb3643d2f05de7d59019254d81daeca6b9162fe9edef57e0c7 \ - --hash=sha256:d1c557dfd5e3db046053a0bdc72261ade790ebe8e2c7a41b36b0ca1f14cb95f3 \ - --hash=sha256:d21f0fa80a02d05299207eeaafef345d812ace96d5306e4ef265e1d419a615fa \ - --hash=sha256:d45673badd08456d0340e9364eddafe1c53a9d2896424294de4d7dd71ad3ee57 \ - --hash=sha256:d5add7b4ca7afeea91d52e4d4e4db3b1fe9885b71f07054560d8c4296b7441a2 \ - --hash=sha256:d5f90e44653c4e0f78501ed9bb7d3fce835a8d62b7c6ed0cb16557534087e743 \ - --hash=sha256:d7306dee25b8a0e737363f347362b875094b4dc4e367311470656ae420fdbf8e \ - --hash=sha256:dbcef5a9119ef653653132cccaf999b30a0af6f33bb0a4ba80bec30056868487 \ - --hash=sha256:def538065f9e4d4cf1ae164bd59aba00dfa84f03923e0de4c3788f252d6bcd17 \ - --hash=sha256:df23df54b5114a17c2d0ef192433e2e5a9f0c5178c32375e90b7cfc965f349d0 \ - --hash=sha256:dfbf531053a0935f2e871bcd4753f90313688772ff8c017f5ea402e315a78c1f \ - --hash=sha256:e029648f9c951db30e98a7d7ec90835db88ec4b32820efe2a9bdc2287e032eb6 \ - --hash=sha256:e07595c7d6f4db270ceede356a1bd1c07a34f1c26f958d1ed0cd7b48e0d2bba3 \ - --hash=sha256:e12c538e00e7c1b286a07061046b90e8124e6a9793efae2c70db6a4aad07faad \ - --hash=sha256:e546fe1d4a93ebc2910f0d768baff19faa09843ab3f2036a67ed6e69fae4419d \ - --hash=sha256:e5637ca8d0bd7fb72648a6c7934af4baaccb697657f7438c9d264fc2abb8b0b1 \ - --hash=sha256:e636b64d24fd9c38053c5e389a1174c66361fa49dcfd220f4dd35b4abde7cb89 \ - --hash=sha256:e7011b8fb8c4054bf0c12e5edc6cd83778b0028e99ce59b18586ed036f92cfdc \ - --hash=sha256:e80f557716fd765439577131e526b8942ffc2c07bdbc5e39fa62f660ba1e963f \ - --hash=sha256:e92b6bcc741b86d67606c40d3cb9c7cc8e6c737f81e31f4a94efc204456c92e3 \ - --hash=sha256:eb96ed1ae6c7d072d60840c0434aef07a2df611812810807fbc54263a6053e9a \ - --hash=sha256:eda19ea5ee88742f47a2340816e6f2d40b53bed3ab5b69794769f36af9f35bb4 \ - --hash=sha256:ef74070ac553c59eb4f04258722066d6c6135b7baa03b2e9f2da65c096e96d98 \ - --hash=sha256:efb01a106f971cb3752856bca2318bbdf7f01bd8823779c461586cbe5ffd5258 \ - --hash=sha256:f074e8d4aa0a5798920ddb6de3d08b228c614ff3724c3e8bd7577f4bafea867b \ - --hash=sha256:f38a70074041d3b7e138e452799f5174198bae5bd5ab2000917badf403908c5f \ - --hash=sha256:f41753a76f4f63927d03a0d8ba8f5ce0f2083bec29a8cfaccc55371b1564b96b \ - --hash=sha256:f53dcd26694f148f738edc052b5a69234833e739f10f4c3287bdfd8ec0f7b326 \ - --hash=sha256:f61964f235a43738bfac50da46fc4254943a7eea3051aeb0b6fc7c992c29fadc \ - --hash=sha256:fe01645169a2112aa1d4ebc3e4c5f029c5c8f97adfc32e5d37c993b39a994d75 diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/README.md b/evidence/qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/README.md deleted file mode 100644 index 84b42c491..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/README.md +++ /dev/null @@ -1,47 +0,0 @@ -# Activated 8K search protocol - -Governing acceptance: [Issue #67](https://github.com/InferMatrixLab/InferRSI/issues/67#issuecomment-6359192342). -The owner approved 8K-only retrieval and formal performance, the statistical -rule, and the 44 GPU-hour phased ceiling. The root activation record was frozen -before submitting candidate job 2163. Campaign verdict remains **INCONCLUSIVE**. - -The single new model change is merged FP8 projections in engine PR16. Baseline -and candidate sources are pinned in the Issue and activation record. Candidate -component, complete quality/scorer/executor binding, approved statistics, -raw-to-statistics adapter and independent review are content-locked. The copied -performance binding files are the exact reviewed private execution bytes. -Quality collection files here are exact source copies; host-specific wrappers, -full dataset/model/runtime identities and raw model outputs remain in controlled -storage. Their digests are retained in the root and component locks. Private -raw archives are not available from this public checkout. - -Sixteen CPU controls exercise the actual performance-binding logic with a mocked -raw audit; they are not GPU or SSE validation. The separate actual baseline run -2140 was independently replayed through the unchanged SSE auditor. Initial -binding review findings (ambiguous terminal fields, missing actual HTTP Graph -and module-source checks, incomplete per-operator checks) were fixed before -freeze. Wrong metric prose was corrected to TTFT/TPOT. No gate was relaxed. - -Quality qualification has a narrow executor scope. The original failed canonical -pilot, successful isolation/export/capacity observations and official assertion -classification remain retained. The production adapter reuses unchanged runtime -and supervision with independent source/consumer review; its first actual model -export still requires successful terminal and postflight sealing. No canonical -all-pass threshold was added, and this is not a model-quality PASS. - -Candidate job 2163 has a 3600 GPU-second search ceiling. Baseline quality job -2157 has a separate 9000 GPU-second reservation. These are running allocations, -not completed outcomes. No retry or budget transfer is implied. Complete quality -and retrieval evidence is required alongside performance for final acceptance. - -The production-binding observation and candidate-review records retain their -original pre-activation wording (including pending review or unexecuted status). -They are historical source-review snapshots. The later activation.json and its -prospective Issue comment define current execution authorization; snapshots are -not rewritten to imply they observed subsequent execution. The standalone binder -CPU tests require the original candidate component directory supplied through -PERFORMANCE_COMPONENT_DIR; that private component is not included here. - -KnowledgeStore search-freeze record: store:case.qwen38-8k-search-freeze-cand, -revision sha256:82267a195a48c2dc0616b05eb512938830141c85edf9fadf81f39a6dea27ecaf, -with verified readback and retained preparation failures. diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/artifacts.json b/evidence/qwen38-h20-fp8-single-request-vllm032/results/artifacts.json new file mode 100644 index 000000000..829b57516 --- /dev/null +++ b/evidence/qwen38-h20-fp8-single-request-vllm032/results/artifacts.json @@ -0,0 +1,84 @@ +{ + "schema_version": 2, + "archive": { + "id": "qwen38-h20-fp8-single-request-vllm032-6c7e237-v1", + "file": "qwen38-h20-fp8-single-request-vllm032-6c7e237-v1.tar.gz", + "url": "https://github.com/InferMatrixLab/InferRSI/releases/download/evidence-model-cleanup-v1/qwen38-h20-fp8-single-request-vllm032-6c7e237-v1.tar.gz", + "sha256": "4bf744194caef014eeaf213d5f9f6a6826c97b799b06b78bba8f9c354e228227", + "bytes": 2469784, + "format": "tar+gzip", + "retention_owner": "amy-why-3459", + "source_revision": "6c7e237935a22cc5e90e954d4f397376e753e7d8", + "root": "qwen38-h20-fp8-single-request-vllm032", + "files": 413, + "uncompressed_bytes": 5205664, + "key_files": { + "historical_narrative": "qwen38-h20-fp8-single-request-vllm032/README.md", + "snapshot": "snapshot.json", + "projection": "replay/projection.json", + "control": "qwen38-h20-fp8-single-request-vllm032/results/fp8-projections-search-v1/bound-result.json", + "validator": "qwen38-h20-fp8-single-request-vllm032/validator/performance-binding-8k-v1/paired_statistics_8k_v1.py" + } + }, + "verification": { + "summary": "results/summary.json", + "trials": "results/trials.jsonl", + "command": [ + "python", + "tools/rebuild_model_evidence.py", + "evidence/qwen38-h20-fp8-single-request-vllm032/results/artifacts.json", + "--archive", + "", + "--output", + "" + ], + "inputs": [ + { + "role": "control", + "path": "qwen38-h20-fp8-single-request-vllm032/results/fp8-projections-search-v1/bound-result.json", + "sha256": "818046ab9571161fb631ff07d17e9ac267c95d1e1f50ce2b741fdb14cac93137" + }, + { + "role": "configuration", + "path": "qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json", + "sha256": "5bd63870f80ba6b04526983f80d6880b0bf04fe97358a1e7784f80ddd925a664" + }, + { + "role": "validator", + "path": "qwen38-h20-fp8-single-request-vllm032/validator/performance-binding-8k-v1/paired_statistics_8k_v1.py", + "sha256": "328571c178612273dcf5a6176973e659c4d1f93a5208db5b0a356f78b1e29269" + }, + { + "role": "configuration", + "path": "replay/projection.json", + "sha256": "adc4f74f97c37f1b49da554b90ef1e1e51e3a826aa3fa1317a92ac21703bd3ce" + }, + { + "role": "control", + "path": "snapshot.json", + "sha256": "f71d52cfcb1d3c2a98219ee21282b035f75226e0f57872e882b528f9d4553f0f" + }, + { + "role": "validator", + "path": "replay/rebuild_model_evidence.py", + "sha256": "bf9472493aff02d52ed341eba46e71f3f620cad48fd6eddc86cd9fe34036f607" + }, + { + "role": "validator", + "path": "replay/evidence_archives.py", + "sha256": "0c141cf6d925627ffd3ddafc18f927ae19449bbf8ff4c26175d975801311e936" + } + ], + "absolute_tolerance": 1e-12, + "relative_tolerance": 1e-12 + }, + "availability_limits": [ + "Campaign acceptance remains INCONCLUSIVE; this is a storage-only projection.", + "The index covers all published result JSON documents and their named/nested attempt observations, not unpublished controller history.", + "Repeated identical observations of one named attempt are merged with every source pointer retained. Changed observations and anonymous records remain distinct.", + "Record/index entry counts are not independent trial counts. Text-only history and per-request traces remain in the byte-preserved original archive.", + "Only declared sample distributions are recomputed. Other retained results are explicitly reported frozen values, not independently requalified quality or statistics.", + "Original private models, media, raw receipts and runtime records were not published; archiving does not restore them or complete outstanding acceptance." + ], + "retention_policy": "Keep this exact asset retrievable while referenced; changed bytes require a new version." +} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1-integrity.json b/evidence/qwen38-h20-fp8-single-request-vllm032/results/core-integrity.json similarity index 67% rename from evidence/qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1-integrity.json rename to evidence/qwen38-h20-fp8-single-request-vllm032/results/core-integrity.json index 5c906e0e7..fb92af9b7 100644 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1-integrity.json +++ b/evidence/qwen38-h20-fp8-single-request-vllm032/results/core-integrity.json @@ -1,12 +1,16 @@ { - "plan/search-8k-v1/README.md": "5a0c327926f61871b38d027cce2801f0ae80d3795af6ce123ccd597d04e6a4c1", + "README.md": "6097a7e57a89f35183fc747fdb67583b6a8978592fb037d3d2253fa0e4fb18c8", + "patch/README.md": "d2264b896ce62986ebd5362d3a426375f7619e275d6da04346998b1231053dbe", + "plan/README.md": "ab20522ebc53585be55097574ac5cb612ba9f92ff363eb42e3cd8737d565762a", + "plan/archive-projection.json": "adc4f74f97c37f1b49da554b90ef1e1e51e3a826aa3fa1317a92ac21703bd3ce", + "plan/formal-8k-baseline-identities.json": "6741b193ff4b4674e9967af1fe6f2a019031a09e7d548cc3bbec7e810afacbf4", + "plan/quality-v5-source-identities.json": "5137bf2476dbe15847b979cb5caf9acca3e766ad7a0bec5afa9ae0cdf1cc73db", "plan/search-8k-v1/activation.json": "5bd63870f80ba6b04526983f80d6880b0bf04fe97358a1e7784f80ddd925a664", "plan/search-8k-v1/candidate-review.json": "821c3a46dc1f3b15ca6792b090275794d287591d15e23b27ff3374e5551d2862", - "results/formal-8k-baseline/independent-review.json": "95310bf77a74853660c81273b218584422d8e931f69d1c46783a02c5f8e727f9", - "results/formal-8k-baseline/observation.md": "b73fbd2892997e89cb89ec86f017db6cb1092fe0e4904a231eb10096b5033d9a", - "results/formal-8k-baseline/raw-audit.json": "9b7ee8644b05ae2bb59a99fe879c798f0bdcd5bdc2fa40a0534ed1b85d8fa2c2", - "results/quality-executor/production-binding-observation.md": "60306744619283de1842cfe7036672716ae36d4dfd2f41b163d2b5487c730598", - "results/quality-executor/production-binding-review.json": "35f4be1a39e78e8ca42424e053d68205c79ac54fd0578db37bf19095134b48f4", + "results/artifacts.json": "7e2bcb08836674a23501a7c05e35bbacbc14a735d14b9c9956f5349ccca4d94c", + "results/summary.json": "a003ed722986a0c9f8ec08fba71f2df5109eca4435e35d33357e86371107f569", + "results/trials.jsonl": "f6dc44cb82184f140ead302320ef166f6f67f1c6b6f960f5afc4cac01e7fc485", + "validator/README.md": "40a1cf59eb90c92e261094f3da02f981af46f881634185b00567c0128cc4ef8e", "validator/performance-binding-8k-v1/audit.py": "c1ddee3ccbc5815ab428a2ec36ea279fbb8c232a6f07d246302610042b925ff7", "validator/performance-binding-8k-v1/bind.py": "b0473d3f51c54f8b0d3da92754a49efaef339ea0888f66f65d0f4c8adcf374fa", "validator/performance-binding-8k-v1/freeze.sha256": "0046c51271b0831f9eae1d9d18f3a64d77f45b0e6058b9ff8fb5c29a2f5afc6f", diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/deepgemm-prefill/actual-2084.json b/evidence/qwen38-h20-fp8-single-request-vllm032/results/deepgemm-prefill/actual-2084.json deleted file mode 100644 index e97ab6182..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/deepgemm-prefill/actual-2084.json +++ /dev/null @@ -1,194 +0,0 @@ -{ - "issue": 67, - "scope": "single-case engineering diagnostic, not formal acceptance", - "verdict": "INCONCLUSIVE", - "job": 2084, - "execution": "COMPLETED 0:0", - "allocated_gpu_seconds": 956, - "engine_tested": "b412062e13c92d2429b3758ca39af7439df4f7ec", - "raw_archive_sha256": "0c39db0129ed4a4ef9027dc8356778d5a0486a426c33d55a1921b73f635d7468", - "requests_verified": 6, - "metrics": { - "vllm": { - "ttft_s": 1.6813895399682224, - "tpot_s": 0.01072323002054483, - "latency_s": 23.63184139202349 - }, - "flashinfer_deepgemm": { - "ttft_s": 3.520265935920179, - "tpot_s": 0.05371937867517674, - "latency_s": 113.48383408400696 - }, - "flashinfer": { - "ttft_s": 151.33197165396996, - "tpot_s": 0.05124317053737302, - "latency_s": 256.2267417439725 - } - }, - "deepgemm_ratios": { - "flashinfer": { - "ttft_s": 0.023261878487709706, - "tpot_s": 1.0483226957238674, - "latency_s": 0.4429039424674983 - }, - "vllm": { - "ttft_s": 2.0936647054356663, - "tpot_s": 5.009626630432697, - "latency_s": 4.802157910653184 - } - }, - "token_comparisons": { - "flashinfer": { - "matching_positions": 2048, - "total": 2048, - "first_mismatch": null - }, - "vllm": { - "matching_positions": 2048, - "total": 2048, - "first_mismatch": null - } - }, - "cleanup": [ - { - "phase": "cleanup", - "arm": "vllm", - "exit_code": 0, - "forced": false, - "owned_group_remaining": 0, - "cleanup_errors": [], - "cleanup_complete": true - }, - { - "phase": "cleanup", - "arm": "flashinfer_deepgemm", - "exit_code": 0, - "forced": false, - "owned_group_remaining": 0, - "cleanup_errors": [], - "cleanup_complete": true - }, - { - "phase": "cleanup", - "arm": "flashinfer", - "exit_code": 0, - "forced": false, - "owned_group_remaining": 0, - "cleanup_errors": [], - "cleanup_complete": true - } - ], - "operator_rows": [ - { - "shape": [ - 32, - 256, - 128 - ], - "input_kind": "random", - "quantized_byte_mismatches": 0, - "max_scale_abs_error": 0.0, - "diagnostic_gemm_nrmse": 0.0016899170586839318, - "graph_changed_input_exact": true, - "kernel_calls": { - "flashinfer": 0, - "triton": 0, - "deepgemm": 15 - }, - "milliseconds": 0.08652160167694092 - }, - { - "shape": [ - 33, - 256, - 128 - ], - "input_kind": "random", - "quantized_byte_mismatches": 0, - "max_scale_abs_error": 0.0, - "diagnostic_gemm_nrmse": 0.0016690513584762812, - "graph_changed_input_exact": true, - "kernel_calls": { - "flashinfer": 0, - "triton": 0, - "deepgemm": 15 - }, - "milliseconds": 0.08336319923400878 - }, - { - "shape": [ - 33, - 256, - 128 - ], - "input_kind": "zero", - "quantized_byte_mismatches": 0, - "max_scale_abs_error": 0.0, - "diagnostic_gemm_nrmse": 0.0, - "graph_changed_input_exact": true, - "kernel_calls": { - "flashinfer": 0, - "triton": 0, - "deepgemm": 15 - }, - "milliseconds": 0.07743359804153442 - }, - { - "shape": [ - 33, - 256, - 128 - ], - "input_kind": "tiny", - "quantized_byte_mismatches": 0, - "max_scale_abs_error": 0.0, - "diagnostic_gemm_nrmse": 0.0, - "graph_changed_input_exact": true, - "kernel_calls": { - "flashinfer": 0, - "triton": 0, - "deepgemm": 15 - }, - "milliseconds": 0.07818880081176757 - }, - { - "shape": [ - 32, - 5120, - 5120 - ], - "input_kind": "random", - "quantized_byte_mismatches": 0, - "max_scale_abs_error": 0.0, - "diagnostic_gemm_nrmse": 0.0016527068801224232, - "graph_changed_input_exact": true, - "kernel_calls": { - "flashinfer": 0, - "triton": 0, - "deepgemm": 15 - }, - "milliseconds": 0.08004480004310607 - }, - { - "shape": [ - 8192, - 5120, - 5120 - ], - "input_kind": "random", - "quantized_byte_mismatches": 5, - "max_scale_abs_error": 0.0, - "diagnostic_gemm_nrmse": 0.001660516019910574, - "graph_changed_input_exact": true, - "kernel_calls": { - "flashinfer": 0, - "triton": 0, - "deepgemm": 15 - }, - "milliseconds": 1.8489023208618165 - } - ], - "overlay_prepost_exact": true, - "readiness_used_gpu_seconds": 6475, - "readiness_remaining_gpu_seconds": 725 -} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/deepgemm-prefill/build-failure-2061.md b/evidence/qwen38-h20-fp8-single-request-vllm032/results/deepgemm-prefill/build-failure-2061.md deleted file mode 100644 index 2ea16625e..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/deepgemm-prefill/build-failure-2061.md +++ /dev/null @@ -1,9 +0,0 @@ -# Standalone DeepGEMM build2061 - -Issue67; candidateb412062e13c92d2429b3758ca39af7439df4f7ec; pinned DeepGEMM1e1842a833699298f7afc02eefb2ed168fff6938. INCONCLUSIVE for GPU/model/service. - -CPU-only job2061 FAILED2:0 after88seconds,8CPUs/0GPU, allocation released (parent inspected live final scheduler output; subsequent saved query expired and must not be represented as that receipt). The compiler stopped at third-party/deep_jit/include/deep_jit/utils/exception.hpp:3, missing elfutils/libdwfl.h. No usable wheel was produced. The runtime package inventory before the job and after failure compares identical. The wheel0.48.0 build-only overlay did not mutate the selected runtime. - -The pinned DeepJIT source unconditionally includes libdwfl and C++20format. Header-only elfutils0.190-3 files from the head's installed development package were copied into a task-owned overlay as prospective preparation, not a successful repaired build. The head GCC12.3 installation has no standardformat header; compute-node compiler availability is not established by that head observation. Do not repeatedly retry a header-only repair or claim a standalone binary is qualified. No replacement source revision, system installation, vendor-binary substitution, or DeepGEMM GPU run occurred. - -CPU115 tests and independent API/layout review remain valid at their recorded scope. Activation Triton quantization, TMA access and standalone GEMM execution remain untested. Preserve this environmental failure separately from numerical/performance outcomes. diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/deepgemm-prefill/build-failure-2077.md b/evidence/qwen38-h20-fp8-single-request-vllm032/results/deepgemm-prefill/build-failure-2077.md deleted file mode 100644 index 4fa6a488c..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/deepgemm-prefill/build-failure-2077.md +++ /dev/null @@ -1,7 +0,0 @@ -# Standalone DeepGEMM retry2077 - -INCONCLUSIVE for engine GPU/model/service; Issue67. Prior2061missingheader failure remains retained. - -Versionedbuild-v2 reused clean pinnedsource and task-ownedelfutilsheaderoverlay; it added a compiler preflight before recompiling. CPUjob2077 FAILED1:0 in3seconds,8CPUs/0GPU,released. The actualcompute-node compiler was GCC10.2.1 and rejected #include. No second fullcompile, wheel, GPUtest or baselinepackage modification occurred. This verifies a concretetoolchain prerequisite, not an algorithmic rejection. No further same-toolchain retry is justified. - -PR12 remains draft. To keep performance work moving, a separate explicit flashinfer_all candidate uses the already installed FlashInfer normal SM90 path for largeM. It does not silently substitute for the standaloneDeepGEMM option, and it has no measured GPU/model/service result yet. diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/deepgemm-prefill/e2e-observation-2084.md b/evidence/qwen38-h20-fp8-single-request-vllm032/results/deepgemm-prefill/e2e-observation-2084.md deleted file mode 100644 index 086258652..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/deepgemm-prefill/e2e-observation-2084.md +++ /dev/null @@ -1,21 +0,0 @@ -# DeepGEMM prefill: actual bounded E2E result - -Governing Issue67. Execution COMPLETED0:0; formal performance, retrieval and task-quality verdict INCONCLUSIVE. Slurm2084 used one H20 for956 allocated seconds and released it. Readiness usage becomes6475/7200 GPU-seconds;725 remain. The earlier147-second diagnosis is separate; total44GPU-hour phase limits remain unchanged. - -Tested engine b412062e13c92d2429b3758ca39af7439df4f7ec, same physical GPU/runtime, authenticated fixed8192-token input, pinned official vLLM0.32 streaming client, one8192/16 warmup and one8192/2048 measurement per arm, concurrency1, ignoreEOS. Arm order was vLLM, FlashInfer+DeepGEMM, FlashInfer-small-M control. The candidate overlay was absent from the baseline PYTHONPATH. Actual source/versions and operator origins are retained privately. - -| Arm | TTFT s | TPOT ms | Request s | -| --- | ---: | ---: | ---: | -| vLLM0.32 |1.681389540|10.723230021|23.631841392| -| FlashInfer decode + DeepGEMM prefill |3.520265936|53.719378675|113.483834084| -| FlashInfer decode + native Triton prefill |151.331971654|51.243170537|256.226741744| - -DeepGEMM/control ratios: TTFT0.0232618785, TPOT1.0483226957, latency0.4429039425. Relative to vLLM: TTFT2.0936647054, TPOT5.0096266304, latency4.8021579107. The prefill improvement is large on this case; decode TPOT is slightly worse in this one observation, with no causal/variability conclusion. Do not hide the regression or claim parity. All2048 outputtoken IDs match across all three arms on this synthetic input; this does not establish general task quality. - -All six raw SSE/client records passed prompt, count, usage, terminal, DONE framing, ITL and timing recomputation checks. All three observer cleanup receipts report exit0, zero remaining group members, no cleanup errors and no observer SIGKILL. This does not establish universally graceful shutdown: the vLLM server log reports an internal EngineCore abort-timeout force-kill and one leaked-semaphore cleanup warning. Startup also retained ten TritonBundler missing-cubin warning tracebacks before successful service execution. Native active-request/state counts were zero. Startup and warmup were excluded from measured requests. First DeepGEMM warmup TTFT18.413s includes first-use work; measured TTFT is3.520s. One case/repetition cannot substitute for formal10-case/3-repeat testing. - -Six operator cases completed with finite output and exact changed-input Graph/eager comparisons. For8192x5120 activations, five FP8 bytes differ from the independent PyTorch formula; FP32 scales are exact. Other five cases have zero byte differences. Diagnostic GEMM NRMSE against dequantized FP32 references is about0.00165–0.00169 on random cases and zero on zero/tiny cases. These are retained observations, not a newly invented numerical acceptance gate. Only original FP8 weight bytes and FP32 checkpoint multipliers were used. Full model quality remains unverified. - -Preflight and postflight each authenticated the frozen overlay manifest,899 operator files, licenses, allowed root entries and no symlinks. Imported package/extension paths and binary digest were checked before operator work. This remains an unmodified operator subtree repackaged from the installed vLLM wheel, not an independently rebuilt distribution. Source-build failures2061/2077 and prior launch failures remain retained. - -Archive SHA2560c39db0129ed4a4ef9027dc8356778d5a0486a426c33d55a1921b73f635d7468;33 retained file hashes verified. Raw paths, device identifiers and logs are private. Final implementation head980d8b1d141b69337531e52565afd250c2ff21c5 changes only two dispatch-comment lines from testedb412; independent review verified the actual archive, all six SSE/client records, source/overlay bindings and bounded integration limits. Final head a2ff24f61e0daeec128137806c193515a2a25f36 has the exact same tree as reviewed980d8b1; only merge ancestry changed after resolving documentation. PR12 merged at83d82067e72f500fd81163fc7ccfda979a499200. No Actions/status checks were configured; this opt-in integration leaves Issue67 acceptance open. The current graph capacity still demotes long decode to eager; the separately prepared full-context policy is not part of2084. diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/deepgemm-prefill/extraction-2083.md b/evidence/qwen38-h20-fp8-single-request-vllm032/results/deepgemm-prefill/extraction-2083.md deleted file mode 100644 index aae562c64..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/deepgemm-prefill/extraction-2083.md +++ /dev/null @@ -1,9 +0,0 @@ -# DeepGEMM external overlay from authenticated wheel subtree - -Issue67; CPU2083 COMPLETED0:0/17seconds/8CPU/0GPU,released. This is a repackaged unmodified vendored operator binary from the authenticatedvLLM0.32wheel, not an independently rebuilt wheel or an official standalone distribution. It replaces neither the installedvLLM package nor its baseline runtime import path. - -The reviewed preparation verifies the exact entiredeep_gemm RECORD member set, per-fileSHA256/size, no source symlinks and exactcopiedbytes;899files totaling64143962bytes passed. Theextension36595112bytes is SHA2560042598bc0770e70bc4ea4b9c3d27d8efc8e395d6f766d41e939c746c2bd3046. OriginalDeepGEMM/CUTLASS/DeepJITlicenses are retained. OriginalwheelSHA5e2ecdc5b0baf72921c288486417ca82778456a448e4fe60b62ee43c0dfc8611 comes from the earlier authenticated installation; this operation verifies installedRECORD, not a freshwheelarchivehash. - -Fresh isolatedCPUprocess imported top-leveldeep_gemm2.8.0 usingTorch2.13.0+cu130; an importfinder rejected anyvllm import; actualsys.modules and loadednative mappings contained no originalvllm subtree. Theextension loaded from the externaloverlay. ELFNEEDED containsCUDA/Torch/C++/system libraries, no vllm library; noRPATH/RUNPATHentry was reported. CUDAvisiblecount0; noGPU/JIT/GEMMexecuted. These controls establish packaging/import boundary only. - -The baseline must not receive this overlay: pinnedvLLM prefers an externaldeep_gemm before its own vendor. Inject only candidatechildPYTHONPATH and remove it forbaseline. The candidatePR12source/weights/scales remainunchanged; exact largeM GPUquantizer/TMA/GEMM andHTTP qualification remainpending. Prior2061/2077sourcebuildfailures are retained; successfulpackaging doesnotrelabelthem. diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/deepgemm-prefill/extraction-metadata.json b/evidence/qwen38-h20-fp8-single-request-vllm032/results/deepgemm-prefill/extraction-metadata.json deleted file mode 100644 index 6f6726298..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/deepgemm-prefill/extraction-metadata.json +++ /dev/null @@ -1,26 +0,0 @@ -{ - "packaging": "unmodified operator subtree repackaged from vLLM wheel; not independently rebuilt", - "wheel_sha256": "5e2ecdc5b0baf72921c288486417ca82778456a448e4fe60b62ee43c0dfc8611", - "wheel_identity_scope": "earlier authenticated installation; this operation verifies installed RECORD members", - "upstream": "1e1842a833699298f7afc02eefb2ed168fff6938", - "licenses": { - "DeepGEMM-LICENSE": "6c73a30c14a2938abaa6afe98ebb19e6d81dac5bd4ace0cb05c61c20afa37b85", - "DeepJIT-LICENSE": "ebb4f09972aee8608be255debaf78451a68e95c290f55c240dec2ecfa16ea6be", - "CUTLASS-LICENSE": "80a7a18b73d41f64dd9ca881af35938f8de88b18c728703251f4c94d1299884d" - }, - "files_count": 899, - "total_bytes": 64143962, - "binary": [ - { - "path": "_C_extension.abi3.so", - "sha256": "0042598bc0770e70bc4ea4b9c3d27d8efc8e395d6f766d41e939c746c2bd3046", - "bytes": 36595112 - } - ], - "private_manifest_sha256": "3c427abc1f5f76abca793e0f5696e641fca99867620a66540e594ee975a0e1bc", - "cpu_job": 2083, - "elapsed_seconds": 17, - "gpu_count": 0, - "no_vllm_imports": true, - "gpu_qualification": false -} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/deepgemm-prefill/implementation.md b/evidence/qwen38-h20-fp8-single-request-vllm032/results/deepgemm-prefill/implementation.md deleted file mode 100644 index 4b9b7994f..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/deepgemm-prefill/implementation.md +++ /dev/null @@ -1,18 +0,0 @@ -# DeepGEMM prefill without checkpoint requantization - -Historical implementation snapshot before build attempts2061/2077 and extraction2083. See the adjacent build-failure and extraction records for later outcomes. - -Scope: InferRSI acceptance Issue67, Qwen3.8 official FP8 independent engine. -Date2026-10-23; state candidate; verdict INCONCLUSIVE for GPU/model/service claims. - -The owner prioritized performance implementation while the first FlashInfer GPU measurement waits for allocation. This separate model-plane slice replaces M>=32 prefill GEMM only under the explicit flashinfer_deepgemm option. M<32 retains FlashInfer. Default and FlashInfer-only modes retain their existing compute paths. - -vLLM0.32.0 pins vllm-project/DeepGEMM1e1842a833699298f7afc02eefb2ed168fff6938. Its SM90 API accepts FP32 per-token/block128 activation scales and contiguous FP32 block128 weight scales. vLLM's optional E8M0 preprocessing can dequantize/requantize weights; copying that behavior would violate this campaign's original-weight constraint. The new adapter explicitly uses recipe(1,128,128), disable_ue8m0_cast=True, original E4M3 weights and unchanged multipliers. It imports external deep_gemm and has no vLLM-runtime fallback. - -The new Triton quantizer uses the old native formula max(abs(x))/448 and division by max(scale,1e-12), writing FP32 SFA at stride(1,ceil(M/4)*4). This preserves the existing native policy, not a claim of bit-identical vLLM quantization: the inspected upstream helper clamps amax differently. No E8M0 rounding or weight transformation is introduced. - -Implementation b412062e13c92d2429b3758ca39af7439df4f7ec is based on8859ef319486eb5ed4a444fc66c1018cbebbab33 and pushed as nano-vllm PR12, dependent onPR11. CPU Slurm2054 completed0:0 in16s,8CPU/0GPU;115 tests passed in11.93s and allocation released. The independent reviewer verified exact patch identity, API/layout/recipe compatibility, CPU log and scheduler receipt. No blocking interface/dispatch defect found. A stale large-prefill comment is a minor pending cleanup. - -Important test gap: CPU tests mock the activation quantizer. They do not execute its Triton code, TMA loads or DeepGEMM GEMM. The target runtime currently contains only vLLM's vendored package, not independent deep_gemm. CPU-only Slurm2061 is building the exact pinned standalone source, with pinned submodules, against the unchanged baseline Torch environment. A task-owned uv-installed wheel-only build overlay supplies the missing build tool; no baseline package mutation is intended, and before/after freeze comparison is required. Build outcome and binary identity are pending. - -No DeepGEMM GPU job is submitted. Next GPU controls must include M32/M33, multiple K blocks, zero/tiny activations and nonuniform scales, plus actual projection shapes; then exact8192/2048 HTTP measurement under the same physical device/runtime. Keep compilation, numerical, resource and performance failures distinct. No microbenchmark or CPU check can award service gain or final parity; task-quality and full formal acceptance remain outstanding. diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/deepgemm-prefill/knowledge-2084.json b/evidence/qwen38-h20-fp8-single-request-vllm032/results/deepgemm-prefill/knowledge-2084.json deleted file mode 100644 index add843fda..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/deepgemm-prefill/knowledge-2084.json +++ /dev/null @@ -1,5 +0,0 @@ -{ - "ref": "store:case.qwen38-deepgemm-prefill-integration-cand", - "revision": "sha256:12e5de9514a760bf52cc0fede31733f2fdcbce67f0e2402fb688b2fcbaf52ba6", - "readback_verified": true -} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/baseline-infrastructure-impact.md b/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/baseline-infrastructure-impact.md deleted file mode 100644 index 097533f8b..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/baseline-infrastructure-impact.md +++ /dev/null @@ -1,28 +0,0 @@ -# Baseline infrastructure readiness attempts - -Refs [Issue67](https://github.com/InferMatrixLab/InferRSI/issues/67). -**INCONCLUSIVE. No model, quality or performance result.** - -Job1959 failed before startup readiness and before any request. File-backed -ZeroMQ IPC bind on the task shared filesystem returned EOPNOTSUPP. The server -exited1 and the owned process group was empty, with no forced kill. Slurm -FAILED1:0 after68 allocated GPU seconds. It also warned that FlashInfer was -disabled: the environment contained ninja1.13.2, but its bin directory was not -on PATH. Existing nvcc was present; no dependency installation was needed. - -CPU-only job1960 completed0 in1 second. Both92-byte artifact and70-byte short -filesystem endpoints failed errno95; the59-byte Linux abstract endpoint passed -PAIR send/receive. This rejects a path-length-only explanation for1959. With -the environment bin on PATH, ninja and existing nvcc were found. No GPU was -allocated for this preflight. - -The prospective retry uses the unchanged observer with job-owned abstract IPC, -existing environment/toolkit PATH and explicit cache paths. Its result is -pending in this snapshot. No engine optimizations or acceptance thresholds were -changed. Selected raw observations are retained; private startup trace/command -and device identities stay outside the repository. The exact script is retained -under validator/diagnostics; the CPU operational helper is private because it -binds authorized task-local paths, with selected results below. - -Readiness cumulative allocation972/7200 GPU seconds, initial147 seconds separate. -All1959/1960 allocations released. Both environment dependencies are unchanged. diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/baseline-long-1961.json b/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/baseline-long-1961.json deleted file mode 100644 index 91832ad38..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/baseline-long-1961.json +++ /dev/null @@ -1,141 +0,0 @@ -{ - "verdict": "INCONCLUSIVE", - "job": 1961, - "slurm_status": "FAILED", - "exit_code": 1, - "allocated_gpu_seconds": 537, - "readiness_total_gpu_seconds": 1509, - "readiness_remaining_gpu_seconds": 5691, - "startup_s": 455.5554689109558, - "requests": [ - { - "index": 0, - "input_tokens": 32, - "output_tokens": 8, - "token_counts_per_sse_choice": [ - 1, - 1, - 1, - 1, - 1, - 1, - 1, - 1 - ], - "timing_events": 8, - "original_observer_valid": true, - "unique_final_done": true, - "prompt_and_output_identity_verified": true - }, - { - "index": 1, - "input_tokens": 32, - "output_tokens": 8, - "token_counts_per_sse_choice": [ - 1, - 1, - 1, - 1, - 1, - 1, - 1, - 1 - ], - "timing_events": 8, - "original_observer_valid": true, - "unique_final_done": true, - "prompt_and_output_identity_verified": true - }, - { - "index": 2, - "input_tokens": 8192, - "output_tokens": 32, - "token_counts_per_sse_choice": [ - 1, - 1, - 1, - 1, - 1, - 1, - 1, - 1, - 1, - 1, - 1, - 1, - 1, - 1, - 1, - 1, - 1, - 1, - 1, - 1, - 1, - 1, - 1, - 1, - 1, - 1, - 1, - 1, - 1, - 1, - 1, - 1 - ], - "timing_events": 32, - "original_observer_valid": true, - "unique_final_done": true, - "prompt_and_output_identity_verified": true - }, - { - "index": 3, - "input_tokens": 65536, - "output_tokens": 8, - "token_counts_per_sse_choice": [ - 1, - 1, - 1, - 1, - 1, - 1, - 1, - 1 - ], - "timing_events": 8, - "original_observer_valid": true, - "unique_final_done": true, - "prompt_and_output_identity_verified": true - }, - { - "index": 4, - "input_tokens": 131072, - "output_tokens": 8, - "token_counts_per_sse_choice": [ - 1, - 2, - 1, - 1, - 1, - 1, - 1 - ], - "timing_events": 7, - "original_observer_valid": false, - "unique_final_done": true, - "prompt_and_output_identity_verified": true - } - ], - "observer_cleanup": { - "phase": "cleanup", - "exit_code": 0, - "forced": false, - "owned_group_remaining": 0 - }, - "internal_cleanup": "vLLM default abort timeout0 force-killed EngineCore and reported one semaphore cleanup warning; observer itself did not escalate and owned group was empty", - "cause": "observer required one token per SSE choice; default vLLM DELTA collector may aggregate multiple tokens", - "retained_observer_sha256": "76685ff92da150ad1cae3faa259c4305dd9778a2f78b44c4be805370859d9be7", - "raw_observations_sha256": "59a99229cb4ad355163b2607b7185a31f8aa4f88d98378944debfa96363159d1", - "source_provenance": "https://github.com/vllm-project/vllm/blob/5df63a7203cdb9328d824a27b682e77c7b9928be/vllm/v1/engine/output_processor.py" -} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/baseline-long-1961.jsonl b/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/baseline-long-1961.jsonl deleted file mode 100644 index 8bff4742d..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/baseline-long-1961.jsonl +++ /dev/null @@ -1,9 +0,0 @@ -{"phase": "protocol", "observer_sha256": "76685ff92da150ad1cae3faa259c4305dd9778a2f78b44c4be805370859d9be7", "fixture_sha256": "4f4256cc4edc9a23f55f66151738a1ecef07eb752c695e7b7fbdfa17e01d6aa9", "client_sha256": "f31fae8c573ecc4bf8e31dae03823502e71404333e8111f4b535432758302b29", "cases": [["warmup", 32, 8], ["framing", 32, 8], ["feasibility", 8192, 32], ["feasibility", 65536, 8], ["feasibility", 131072, 8]], "verdict": "INCONCLUSIVE", "scope": "single_baseline_feasibility_no_candidate_no_formal_matrix", "versions": {"vllm": "0.32.0", "torch": "2.13.0", "transformers": "5.19.0", "flashinfer-python": "0.7.0.post1", "triton": "3.7.1", "aiohttp": "3.14.5"}, "server_settings": {"max_model_len": 133120, "prefix_caching": false, "language_model_only": true, "other_optimization_flags": "defaults"}} -{"phase": "ready", "startup_s": 455.5554689109558} -{"phase": "request", "index": 0, "purpose": "warmup", "input_tokens": 32, "requested_output_tokens": 8, "output_tokens": 8, "token_ids": [17, 23, 2708, 17, 23, 2708, 17, 23], "terminal": ["length"], "usage": [{"prompt_tokens": 32, "total_tokens": 40, "completion_tokens": 8}], "valid": true, "client_success": true, "client_error": false, "wire_sha256": "b0a324d872cadaaa510d8ea834429ce02150d1f115f7ab6af6f130fb4335cacc", "prompt_sha256": "1ed898856af30d303a2829755cbb760ee8a7a73cb21784783822696af4587471", "ttft_s": 0.07155771111138165, "latency_s": 0.14077040809206665, "itl": [0.008739197975955904, 0.010014739935286343, 0.010035554994828999, 0.010117141995579004, 0.010099678067490458, 0.010057984967716038, 0.010148399043828249], "tpot_s": 0.009887528140097857} -{"phase": "request", "index": 1, "purpose": "framing", "input_tokens": 32, "requested_output_tokens": 8, "output_tokens": 8, "token_ids": [17, 23, 2708, 17, 23, 2708, 17, 23], "terminal": ["length"], "usage": [{"prompt_tokens": 32, "total_tokens": 40, "completion_tokens": 8}], "valid": true, "client_success": true, "client_error": false, "wire_sha256": "f908a8271c0b9a61997d3d48f75fcb14a76e227e36d7d1901ce6d32c685d31d5", "prompt_sha256": "1ed898856af30d303a2829755cbb760ee8a7a73cb21784783822696af4587471", "ttft_s": 0.02883078099694103, "latency_s": 0.09875082492362708, "itl": [0.009402295923791826, 0.010069505078718066, 0.010107590933330357, 0.010068831965327263, 0.010107489069923759, 0.010072381934151053, 0.010091949021443725], "tpot_s": 0.009988577703812293} -{"phase": "request", "index": 2, "purpose": "feasibility", "input_tokens": 8192, "requested_output_tokens": 32, "output_tokens": 32, "token_ids": [220, 15, 220, 15, 220, 15, 220, 15, 220, 15, 220, 15, 220, 15, 220, 15, 220, 15, 220, 15, 220, 15, 220, 15, 220, 15, 220, 15, 220, 15, 220, 15], "terminal": ["length"], "usage": [{"prompt_tokens": 8192, "total_tokens": 8224, "completion_tokens": 32}], "valid": true, "client_success": true, "client_error": false, "wire_sha256": "2e1b811c1713c7216f530cc453ee306da3bf315c9a017227d007757b1f62991d", "prompt_sha256": "118c7838e318287aaba3138fe48c07aff172cbe41c337988691ab03cb8cd3843", "ttft_s": 1.6905596759170294, "latency_s": 2.0154057278996333, "itl": [0.006495462032034993, 0.010427915025502443, 0.01058577501680702, 0.010605121962726116, 0.010623298003338277, 0.010592453996650875, 0.0105877669993788, 0.010624018032103777, 0.010600145906209946, 0.010640258085913956, 0.010572591912932694, 0.010621741064824164, 0.010629624011926353, 0.01059205795172602, 0.010615212027914822, 0.010620897053740919, 0.010732156923040748, 0.010564220021478832, 0.010622882982715964, 0.010614713071845472, 0.01063494198024273, 0.01070735591929406, 0.010581104084849358, 0.010613097925670445, 0.01063842000439763, 0.010613518999889493, 0.010595062980428338, 0.01063729904126376, 0.01061701902654022, 0.010607316973619163, 0.010632602963596582], "tpot_s": 0.010478904902664643} -{"phase": "request", "index": 3, "purpose": "feasibility", "input_tokens": 65536, "requested_output_tokens": 8, "output_tokens": 8, "token_ids": [220, 15, 20, 220, 15, 198, 220, 15], "terminal": ["length"], "usage": [{"prompt_tokens": 65536, "total_tokens": 65544, "completion_tokens": 8}], "valid": true, "client_success": true, "client_error": false, "wire_sha256": "576105c92deb99d4e1af0ca2b5562d20515ca9f1e43141d8a0ae689904703c68", "prompt_sha256": "49b7a660c5a4da955c913a7d06a8de5bd04d97c17473a9658fb7aef98fbe21d8", "ttft_s": 18.649625079939142, "latency_s": 18.718206357909366, "itl": [0.0005742210196331143, 8.560600690543652e-05, 0.011006830958649516, 0.0141556600574404, 0.014217481017112732, 0.014181286911480129, 0.014360191999003291], "tpot_s": 0.009797325424317802} -{"phase": "request", "index": 4, "purpose": "feasibility", "input_tokens": 131072, "requested_output_tokens": 8, "output_tokens": 8, "token_ids": [220, 15, 13, 220, 15, 13, 220, 19], "terminal": ["length"], "usage": [{"prompt_tokens": 131072, "total_tokens": 131080, "completion_tokens": 8}], "valid": false, "client_success": true, "client_error": false, "wire_sha256": "72b8195b0af091869382a5efb61897a3648dd5bbef5372160a6bc2e135d431a1", "prompt_sha256": "86366fe6acf2aeb9bf237b7b0353856be9168ddfc0b95e5d7b4bf0fd209f4073", "ttft_s": 49.592377305962145, "latency_s": 49.65468656201847, "itl": [0.0010693400399759412, 1.0433956049382687e-05, 0.00571415398735553, 0.018424927024170756, 0.01843471801839769, 0.018655683030374348], "tpot_s": 0.00890132229376052} -{"phase": "error", "exception": "RuntimeError", "reason": "baseline_framing_or_accounting_invalid"} -{"phase": "cleanup", "exit_code": 0, "forced": false, "owned_group_remaining": 0} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/baseline-long-impact.md b/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/baseline-long-impact.md deleted file mode 100644 index 5a9811e89..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/baseline-long-impact.md +++ /dev/null @@ -1,62 +0,0 @@ -# Baseline long-context probe: retained observer rejection - -Refs [Issue67](https://github.com/InferMatrixLab/InferRSI/issues/67). -**Campaign INCONCLUSIVE; original job remains FAILED1:0.** - -The infrastructure retry1961 loaded the official FP8 model and executed all five -requests:32/8 warmup,32/8 framing,8192/32,65536/8,131072/8. Every returned token, -input echo, usage count, terminal reason and unique final DONE was independently -checked against the raw SSE. The last request has8 output tokens in7 choices, -with token counts[1,2,1,1,1,1,1], and6 ITLs. The retained observer rejected its -extra assumption that each choice contains exactly one token; earlier four -requests satisfied that assumption. No original valid flag or verdict is edited. - -Pinned upstream RequestOutputCollector explicitly aggregates DELTA outputs when -the producer gets ahead of the consumer. The official HTTP client timestamps -choices and returns usage token counts. This observer computes TPOT from those -returned values, independently of choice count. -Thus the observed aggregation is supported default behavior, not evidence of -missing tokens or invalid model execution. A future observer must preserve -this timing semantics, check timing-event count against choices, and separately -check all token IDs against usage. It must not infer one ITL per generated token. -The diagnosis does not activate a new acceptance validator or rescore the job. - -## Descriptive feasibility and cost observations - -No paired candidate, only one fixed input per length and short outputs. These -numbers estimate workload cost; they are not the2048-output matrix or parity. - -| Input / output tokens | TTFT (s) | HTTP latency (s) | Original observer | -| --- | ---: | ---: | --- | -| 8192 /32 | 1.690560 | 2.015406 | valid | -| 65536 /8 | 18.649625 | 18.718206 | valid | -| 131072 /8 | 49.592377 | 49.654687 | rejected: aggregated choice | - -Startup455.555469s; full allocation537s. Cumulative readiness1509/7200 GPU -seconds (0.419167 GPU-hours), remaining5691s. Initial147 seconds are separate. -All allocations released. Server returned0 and observer did not force-kill; -its group was empty. However, vLLM's default abort teardown (timeout0) did -force-kill its EngineCore and logged a semaphore cleanup warning. Do not call -this an entirely graceful shutdown. All original private startup/SSE/client -records are retained; selected diagnostics contain no private endpoints. - -Startup logs selected FlashInfer GDN prefill, FlashAttention3, norm_quant and -act_quant fusions, and built the fp8_blockscale_gemm_90 JIT module. This confirms -actual initialization/warmup activity; no per-request kernel trace or complete -FP8 branch accounting is claimed. Runtime flags leave kernel/Graph/compile -choices automatic. OMP_NUM_THREADS=8 was explicitly inherited from the wrapper, -and vLLM warned about CPU spin contention: this probe is not a certified default -CPU-thread performance baseline. Future formal settings must bind this choice -prospectively, preferably retaining vLLM's normal single-thread serving policy. - -The prior1959 failure and CPU1960 fix remain separate. No engine optimization or -dependency change occurred. No shortened request becomes final quality evidence. - -Sources: pinned upstream -[DELTA collector](https://github.com/vllm-project/vllm/blob/5df63a7203cdb9328d824a27b682e77c7b9928be/vllm/v1/engine/output_processor.py) -and -[official request client](https://github.com/vllm-project/vllm/blob/5df63a7203cdb9328d824a27b682e77c7b9928be/vllm/benchmarks/lib/endpoint_request_func.py). - -KnowledgeStore candidate `case.qwen38-vllm032-infrastructure-readiness-cand`, -revision `sha256:760988acd4e5f2485d9fc66fe653c50ae140389e513dac300ad170494188e5e0`, -retains1959/1960/1961 and prior snapshot with verified readback. diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/baseline-startup-1959.jsonl b/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/baseline-startup-1959.jsonl deleted file mode 100644 index 0c222be72..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/baseline-startup-1959.jsonl +++ /dev/null @@ -1,3 +0,0 @@ -{"phase": "protocol", "observer_sha256": "76685ff92da150ad1cae3faa259c4305dd9778a2f78b44c4be805370859d9be7", "fixture_sha256": "4f4256cc4edc9a23f55f66151738a1ecef07eb752c695e7b7fbdfa17e01d6aa9", "client_sha256": "f31fae8c573ecc4bf8e31dae03823502e71404333e8111f4b535432758302b29", "cases": [["warmup", 32, 8], ["framing", 32, 8], ["feasibility", 8192, 32], ["feasibility", 65536, 8], ["feasibility", 131072, 8]], "verdict": "INCONCLUSIVE", "scope": "single_baseline_feasibility_no_candidate_no_formal_matrix", "versions": {"vllm": "0.32.0", "torch": "2.13.0", "transformers": "5.19.0", "flashinfer-python": "0.7.0.post1", "triton": "3.7.1", "aiohttp": "3.14.5"}, "server_settings": {"max_model_len": 133120, "prefix_caching": false, "language_model_only": true, "other_optimization_flags": "defaults"}} -{"phase": "error", "exception": "RuntimeError", "reason": "server_exited_before_readiness"} -{"phase": "cleanup", "exit_code": 1, "forced": false, "owned_group_remaining": 0} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/benchmark-client-ledger.json b/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/benchmark-client-ledger.json deleted file mode 100644 index f9d4fd890..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/benchmark-client-ledger.json +++ /dev/null @@ -1,24 +0,0 @@ -{ - "campaign_verdict": "INCONCLUSIVE", - "candidate": "23417bbf13548127fdd0b49dbb26e495a38d9af6", - "upstream_vllm_revision": "5df63a7203cdb9328d824a27b682e77c7b9928be", - "client_sha256": "f31fae8c573ecc4bf8e31dae03823502e71404333e8111f4b535432758302b29", - "cpu_attempts": [ - { - "job": "1947", - "execution": "failed", - "gpu_seconds": 0, - "reason": "Probe imported pytest-dependent test server inside pristine baseline client environment; pytest unavailable; no requests issued.", - "observer_sha256": "bf9616dc55d422be9cae43a3a3578b5aa9e764d552e3af49eace3bbdbef3a353", - "private_log_sha256": "4570c457b0751936d5fdc5d7952e8a3cf5d7e8b9d52658714c1183ebe6769461" - }, - { - "job": "1950", - "execution": "completed", - "gpu_seconds": 0, - "reason": "Version2 isolates server in existing test environment and client in unchanged baseline environment. Original API rejected both probes; repair returned correct naturalEOS/ignoreEOS counts1/4 and ITL0/3. Exact-head72socket/runner/entrytests passed.", - "observer_sha256": "5ad0b37a6046f2ce304d7af8c0c3851019aa8227ad0cc39134c6e8a8722b9ac0", - "private_log_sha256": "86401bf0caa30b1e4e4b00d38e33326587943193bef007b87c18b0c6168d0de3" - } - ] -} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/benchmark-http-impact.md b/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/benchmark-http-impact.md deleted file mode 100644 index 07369658c..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/benchmark-http-impact.md +++ /dev/null @@ -1,36 +0,0 @@ -# Benchmark HTTP accounting: bounded impact - -Refs InferRSI Issue #67. Campaign verdict **INCONCLUSIVE**. - -Engine PR8 is merged as1ce49a19a72351668cdbb9d48996796f5ac2efb3. Its tree exactly -matches tested23417bbf13548127fdd0b49dbb26e495a38d9af6. Local and remote related -suites each passed72 tests. The installed official vLLM0.32.0 client was verified -byte-for-byte; old API rejected both synthetic probes, repaired API reported -correct natural-EOS/ignore-EOS token usage and token timing intervals. - -Actual model HTTP job1951 completed three Graph requests with529/169/314 tokens, -then failed after876 tokens at the1024-capacity boundary. Eager never ran. -Results graph-long-1951.jsonl retain every observed token and ITL; the summary -records partial coverage and226 GPU-seconds. This does not establish paired E2E -speedup. Neither these timings nor the official client's false success flag -can establish acceptance. Strict usage/terminal/length checks rejected failure. - -Independent review verified unchanged runner/model/graph/engine runtime hashes -and identified inherited hybrid demotion behavior; HTTP correctly exposed the -error. PR8 enables benchmark compatibility/accounting, not performance parity. - -CPU attempt1947 failed due a fixture dependency crossing into the pristine -client environment; version2 isolated it and1950 passed. Later1952/1953 exposed -a preexisting readiness-file test race; separate test-only PR10 merged as -7ec5d224e983e437530355b94b0d6d08153bc27d after72 remote CPU tests and independent -review. No checks were removed or deadlines increased. See both ledgers. - -KnowledgeStore candidates (verified readback, not library promotion): -- store:case.qwen38-official-http-accounting-cand at - sha256:2863b8b4c714ae8423c83178a2ddf4ff0160ecd67c5be41e0a66a942cf9d8125. -- store:case.qwen38-hybrid-graph-demotion-cand at - sha256:e39d8104adffd937f8e71f33465cd8afac6e66d714faf65381b77147652137d9. - -The latter retains model diagnosis and negative preflight attempts. Their GPU -retry remains pending in this snapshot. Readiness use before retry is375 -GPU-seconds; separate initial147 seconds remain separately accounted. diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/client-base-1950.json b/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/client-base-1950.json deleted file mode 100644 index 21ac8b0cc..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/client-base-1950.json +++ /dev/null @@ -1,42 +0,0 @@ -{ - "client_sha256": "f31fae8c573ecc4bf8e31dae03823502e71404333e8111f4b535432758302b29", - "expected": "rejected", - "rows": [ - { - "generated_text": "", - "success": false, - "latency": 0.0, - "output_tokens": 0, - "ttft": 0.0, - "itl": [], - "tpot": 0.0, - "prompt_len": 999, - "error": "Bad Request", - "start_time": 903857.965747661, - "client_queue_time": 0.0, - "input_audio_duration": 0.0, - "num_input_sequences": 1, - "ignore_eos": false, - "requests_left": 0 - }, - { - "generated_text": "", - "success": false, - "latency": 0.0, - "output_tokens": 0, - "ttft": 0.0, - "itl": [], - "tpot": 0.0, - "prompt_len": 999, - "error": "Bad Request", - "start_time": 903857.969535084, - "client_queue_time": 0.0, - "input_audio_duration": 0.0, - "num_input_sequences": 1, - "ignore_eos": true, - "requests_left": 0 - } - ], - "server_exit": 0, - "scope": "CPU synthetic protocol, not model performance acceptance" -} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/client-repair-1950.json b/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/client-repair-1950.json deleted file mode 100644 index a66b84f23..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/client-repair-1950.json +++ /dev/null @@ -1,46 +0,0 @@ -{ - "client_sha256": "f31fae8c573ecc4bf8e31dae03823502e71404333e8111f4b535432758302b29", - "expected": "supported", - "rows": [ - { - "generated_text": "\u4e01", - "success": true, - "latency": 0.003103488008491695, - "output_tokens": 1, - "ttft": 0.003103488008491695, - "itl": [], - "tpot": 0.0, - "prompt_len": 1, - "error": "", - "start_time": 903863.438265146, - "client_queue_time": 0.0, - "input_audio_duration": 0.0, - "num_input_sequences": 1, - "ignore_eos": false, - "requests_left": 0 - }, - { - "generated_text": "\u4e01\u4e02\u4e04\u4e00", - "success": true, - "latency": 0.008860887959599495, - "output_tokens": 4, - "ttft": 0.00261228799354285, - "itl": [ - 0.0020757809979841113, - 0.002080902922898531, - 0.0020919160451740026 - ], - "tpot": 0.0, - "prompt_len": 1, - "error": "", - "start_time": 903863.444553415, - "client_queue_time": 0.0, - "input_audio_duration": 0.0, - "num_input_sequences": 1, - "ignore_eos": true, - "requests_left": 0 - } - ], - "server_exit": 0, - "scope": "CPU synthetic protocol, not model performance acceptance" -} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/demotion-preflight.json b/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/demotion-preflight.json deleted file mode 100644 index 37338f8a0..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/demotion-preflight.json +++ /dev/null @@ -1,36 +0,0 @@ -{ - "campaign_verdict": "INCONCLUSIVE", - "attempts": [ - { - "job": "1952", - "source": "e69035df55d7fd956a7a2529c75a402164cdf1ed", - "gpu_seconds": 5, - "result": "73 passed,1 failed", - "note": "SIGTERM test read an empty ready document; no model execution", - "private_log_sha256": "bb6257432dcfd3ed44e3c9e346c54b40202be04d58c8a8b5071d6ebd5b12fd45", - "exit_code": 1, - "allocation_released": true - }, - { - "job": "1953", - "source": "e69035df55d7fd956a7a2529c75a402164cdf1ed", - "gpu_seconds": 0, - "result": "73 passed,1 failed", - "note": "CPU-only reproduction of same readiness JSON race", - "private_log_sha256": "73ecce101c62c0b561a80f10dd50db9bc55d8f926d241b15f3929fbdd26553c9", - "exit_code": 1, - "allocation_released": true - }, - { - "job": "1954", - "source": "94eed335c365fccc39289551837818e333758263", - "gpu_seconds": 0, - "result": "72 passed", - "note": "Only test wait corrected; remote CPU suite completed", - "private_log_sha256": "31cf1c0f9ee838cf7e2aff63cece570f33e67aac8321ab042731fa999f1ebcfd", - "exit_code": 0, - "allocation_released": true - } - ], - "readiness_used_gpu_seconds_before_retry": 375 -} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/evidence-layout-check.md b/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/evidence-layout-check.md deleted file mode 100644 index 706cd43d9..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/evidence-layout-check.md +++ /dev/null @@ -1,17 +0,0 @@ -# Publication layout check - -The first local inventory run after retrieval preparation failed one of19 tests: -18 passed,1 failed because two generated metadata files named manifest.json had -no declared schema. No GPU was used, and this was not a campaign verdict. - -The raw metadata bytes are preserved under metadata.json in both publication -folders; original operational copies retain manifest.json. An explicit existing -bundle-sha256 map now authenticates both metadata files, both compressed inputs -and all four prepared observer/generator snapshots. The inventory declares that -map, increasing checked hashes from112 to120. No inventory schema, workflow, -threshold, historical manifest or check is removed or weakened. This is metadata -classification and hash coverage, not a freeze activation or evidence replay. - -After classification, all19 inventory/path tests passed. KnowledgeStore -`case.qwen38-retrieval-construction-cand` revision `sha256:ed563c52804c2a39dc45d9d4e9f29fa4edba770fa076937ec1118de44f6d86ac` -retains the failed check and exact layout report with verified readback. diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/graph-baseline.jsonl b/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/graph-baseline.jsonl deleted file mode 100644 index 9b8a28ae8..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/graph-baseline.jsonl +++ /dev/null @@ -1,57 +0,0 @@ -{"phase": "identity", "python": "3.12.13 (main, May 10 2026, 19:30:01) [Clang 22.1.3 ]", "torch": "2.14.1+cu130", "cuda": "13.0", "device": "NVIDIA H20-3e", "capability": [9, 0], "dependencies": {"triton": "3.8.0", "transformers": "5.20.2", "flashinfer-python": null, "safetensors": "0.8.0"}} -{"phase": "loaded", "seconds": 21.887683575972915, "report": {"loaded_parameters": 1251, "excluded_tensors": {"vision": 333, "mtp": 22}, "scope": "text_only", "dtype": "fp8", "activation_dtype": "bfloat16", "weight_dtype": "float8_e4m3fn", "scale_dtype": "float32", "checkpoint_scale_dtype": "bfloat16", "quantization": {"activation_scheme": "dynamic", "fmt": "e4m3", "quant_method": "fp8", "weight_block_size": [128, 128]}, "execution": "native_fp8", "fp8_linear_modules": 400, "dequantized": false, "backend": "triton_block_fp8", "kernel_name": "qwen38_block_fp8"}} -{"phase": "prefill", "case": "gsm8k/test/783", "prompt_tokens": 149, "capacity": 1024, "prompt_sha256": "f2233be4ebb7b122d80433fe4081fdabeda64f735b7bc24f9706c4c4339b824f", "compact_buffered": {"exact": false, "max_abs": 5.5546875, "nrmse": 0.06517861038446426}, "buffered_repeated": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [1206, 1206, 1206], "layer_diff": [{"layer": 3, "exact": false, "max_abs": 0.001953125, "nrmse": 0.00013569257862400264}, {"layer": 4, "exact": false, "max_abs": 0.125, "nrmse": 0.0009780735708773136}, {"layer": 5, "exact": false, "max_abs": 0.25, "nrmse": 0.0027811273466795683}, {"layer": 6, "exact": false, "max_abs": 0.25, "nrmse": 0.004517734982073307}, {"layer": 7, "exact": false, "max_abs": 0.25, "nrmse": 0.00537494570016861}, {"layer": 8, "exact": false, "max_abs": 0.25, "nrmse": 0.00624054204672575}, {"layer": 9, "exact": false, "max_abs": 0.25, "nrmse": 0.007281075697392225}, {"layer": 10, "exact": false, "max_abs": 0.5, "nrmse": 0.008228786289691925}, {"layer": 11, "exact": false, "max_abs": 0.5, "nrmse": 0.008619681000709534}, {"layer": 12, "exact": false, "max_abs": 1.0, "nrmse": 0.009394780732691288}, {"layer": 13, "exact": false, "max_abs": 1.0, "nrmse": 0.010150868445634842}, {"layer": 14, "exact": false, "max_abs": 1.0, "nrmse": 0.010960409417748451}, {"layer": 15, "exact": false, "max_abs": 1.0, "nrmse": 0.011493951082229614}, {"layer": 16, "exact": false, "max_abs": 1.0, "nrmse": 0.012176775373518467}, {"layer": 17, "exact": false, "max_abs": 1.0, "nrmse": 0.013218415901064873}, {"layer": 18, "exact": false, "max_abs": 1.0, "nrmse": 0.014418303966522217}, {"layer": 19, "exact": false, "max_abs": 0.75, "nrmse": 0.01710168831050396}, {"layer": 20, "exact": false, "max_abs": 0.75, "nrmse": 0.019661009311676025}, {"layer": 21, "exact": false, "max_abs": 0.75, "nrmse": 0.019840963184833527}, {"layer": 22, "exact": false, "max_abs": 0.75, "nrmse": 0.01985854282975197}, {"layer": 23, "exact": false, "max_abs": 1.0, "nrmse": 0.021280594170093536}, {"layer": 24, "exact": false, "max_abs": 1.0, "nrmse": 0.022130679339170456}, {"layer": 25, "exact": false, "max_abs": 1.0, "nrmse": 0.02220517210662365}, {"layer": 26, "exact": false, "max_abs": 1.0, "nrmse": 0.022142359986901283}, {"layer": 27, "exact": false, "max_abs": 1.0, "nrmse": 0.024251600727438927}, {"layer": 28, "exact": false, "max_abs": 1.0, "nrmse": 0.025731991976499557}, {"layer": 29, "exact": false, "max_abs": 1.0, "nrmse": 0.02700507827103138}, {"layer": 30, "exact": false, "max_abs": 1.0, "nrmse": 0.027155021205544472}, {"layer": 31, "exact": false, "max_abs": 1.0, "nrmse": 0.029900016263127327}, {"layer": 32, "exact": false, "max_abs": 1.25, "nrmse": 0.031958311796188354}, {"layer": 33, "exact": false, "max_abs": 1.5, "nrmse": 0.0326150581240654}, {"layer": 34, "exact": false, "max_abs": 1.75, "nrmse": 0.031858786940574646}, {"layer": 35, "exact": false, "max_abs": 1.5, "nrmse": 0.03566876798868179}, {"layer": 36, "exact": false, "max_abs": 1.5, "nrmse": 0.03758247569203377}, {"layer": 37, "exact": false, "max_abs": 1.75, "nrmse": 0.038920894265174866}, {"layer": 38, "exact": false, "max_abs": 1.5, "nrmse": 0.03692947328090668}, {"layer": 39, "exact": false, "max_abs": 2.0, "nrmse": 0.04034900665283203}, {"layer": 40, "exact": false, "max_abs": 1.75, "nrmse": 0.04175692796707153}, {"layer": 41, "exact": false, "max_abs": 1.625, "nrmse": 0.0428168922662735}, {"layer": 42, "exact": false, "max_abs": 3.0, "nrmse": 0.04179355129599571}, {"layer": 43, "exact": false, "max_abs": 2.5, "nrmse": 0.044816236943006516}, {"layer": 44, "exact": false, "max_abs": 2.5, "nrmse": 0.04673221707344055}, {"layer": 45, "exact": false, "max_abs": 2.6796875, "nrmse": 0.047618214040994644}, {"layer": 46, "exact": false, "max_abs": 3.09375, "nrmse": 0.0465744286775589}, {"layer": 47, "exact": false, "max_abs": 2.75, "nrmse": 0.05059361457824707}, {"layer": 48, "exact": false, "max_abs": 2.34375, "nrmse": 0.05369478464126587}, {"layer": 49, "exact": false, "max_abs": 2.5, "nrmse": 0.054837726056575775}, {"layer": 50, "exact": false, "max_abs": 5.5, "nrmse": 0.05403049290180206}, {"layer": 51, "exact": false, "max_abs": 4.625, "nrmse": 0.05909387022256851}, {"layer": 52, "exact": false, "max_abs": 6.4375, "nrmse": 0.06207602098584175}, {"layer": 53, "exact": false, "max_abs": 7.75, "nrmse": 0.06648595631122589}, {"layer": 54, "exact": false, "max_abs": 150.0, "nrmse": 0.09059146046638489}, {"layer": 55, "exact": false, "max_abs": 49.0, "nrmse": 0.07521013915538788}, {"layer": 56, "exact": false, "max_abs": 45.0, "nrmse": 0.07565156370401382}, {"layer": 57, "exact": false, "max_abs": 60.0, "nrmse": 0.07733091711997986}, {"layer": 58, "exact": false, "max_abs": 280.0, "nrmse": 0.11470826715230942}, {"layer": 59, "exact": false, "max_abs": 46.0, "nrmse": 0.0782075896859169}, {"layer": 60, "exact": false, "max_abs": 25.5, "nrmse": 0.07591037452220917}, {"layer": 61, "exact": false, "max_abs": 26.0, "nrmse": 0.0754697173833847}, {"layer": 62, "exact": false, "max_abs": 64.0, "nrmse": 0.07897090166807175}, {"layer": 63, "exact": false, "max_abs": 44.0, "nrmse": 0.0806334987282753}], "state_diff": [{"layer": 4, "state": "conv", "exact": false, "max_abs": 0.03515625, "nrmse": 0.008410594426095486}, {"layer": 4, "state": "recurrent", "exact": false, "max_abs": 0.0004898719489574432, "nrmse": 0.0023321821354329586}, {"layer": 5, "state": "conv", "exact": false, "max_abs": 0.0625, "nrmse": 0.011460041627287865}, {"layer": 5, "state": "recurrent", "exact": false, "max_abs": 0.0024808943271636963, "nrmse": 0.0072328816168010235}, {"layer": 6, "state": "conv", "exact": false, "max_abs": 0.0546875, "nrmse": 0.01576853357255459}, {"layer": 6, "state": "recurrent", "exact": false, "max_abs": 0.0028396248817443848, "nrmse": 0.008456045761704445}, {"layer": 7, "state": "key", "exact": false, "max_abs": 0.171875, "nrmse": 0.016161663457751274}, {"layer": 7, "state": "value", "exact": false, "max_abs": 0.13671875, "nrmse": 0.018452880904078484}, {"layer": 8, "state": "conv", "exact": false, "max_abs": 0.0546875, "nrmse": 0.0188299547880888}, {"layer": 8, "state": "recurrent", "exact": false, "max_abs": 0.002139464020729065, "nrmse": 0.016938991844654083}, {"layer": 9, "state": "conv", "exact": false, "max_abs": 0.05078125, "nrmse": 0.01889151521027088}, {"layer": 9, "state": "recurrent", "exact": false, "max_abs": 0.0053014978766441345, "nrmse": 0.012813080102205276}, {"layer": 10, "state": "conv", "exact": false, "max_abs": 0.078125, "nrmse": 0.02033483050763607}, {"layer": 10, "state": "recurrent", "exact": false, "max_abs": 0.004199333488941193, "nrmse": 0.012261766940355301}, {"layer": 11, "state": "key", "exact": false, "max_abs": 0.2421875, "nrmse": 0.022564243525266647}, {"layer": 11, "state": "value", "exact": false, "max_abs": 0.15625, "nrmse": 0.025585854426026344}, {"layer": 12, "state": "conv", "exact": false, "max_abs": 0.078125, "nrmse": 0.02093205228447914}, {"layer": 12, "state": "recurrent", "exact": false, "max_abs": 0.003071354702115059, "nrmse": 0.015698662027716637}, {"layer": 13, "state": "conv", "exact": false, "max_abs": 0.080078125, "nrmse": 0.02451997436583042}, {"layer": 13, "state": "recurrent", "exact": false, "max_abs": 0.004504084587097168, "nrmse": 0.01583363488316536}, {"layer": 14, "state": "conv", "exact": false, "max_abs": 0.09375, "nrmse": 0.02279035374522209}, {"layer": 14, "state": "recurrent", "exact": false, "max_abs": 0.006101645529270172, "nrmse": 0.017694320529699326}, {"layer": 15, "state": "key", "exact": false, "max_abs": 0.36181640625, "nrmse": 0.027654122561216354}, {"layer": 15, "state": "value", "exact": false, "max_abs": 0.1787109375, "nrmse": 0.030548283830285072}, {"layer": 16, "state": "conv", "exact": false, "max_abs": 0.09375, "nrmse": 0.02620692551136017}, {"layer": 16, "state": "recurrent", "exact": false, "max_abs": 0.004688143730163574, "nrmse": 0.022312110289931297}, {"layer": 17, "state": "conv", "exact": false, "max_abs": 0.125, "nrmse": 0.02669060416519642}, {"layer": 17, "state": "recurrent", "exact": false, "max_abs": 0.004119458608329296, "nrmse": 0.021085284650325775}, {"layer": 18, "state": "conv", "exact": false, "max_abs": 0.15625, "nrmse": 0.024607254192233086}, {"layer": 18, "state": "recurrent", "exact": false, "max_abs": 0.013811580836772919, "nrmse": 0.01766715943813324}, {"layer": 19, "state": "key", "exact": false, "max_abs": 0.265625, "nrmse": 0.028365440666675568}, {"layer": 19, "state": "value", "exact": false, "max_abs": 0.24609375, "nrmse": 0.034957144409418106}, {"layer": 20, "state": "conv", "exact": false, "max_abs": 0.15625, "nrmse": 0.020073654130101204}, {"layer": 20, "state": "recurrent", "exact": false, "max_abs": 0.012838944792747498, "nrmse": 0.019972672685980797}, {"layer": 21, "state": "conv", "exact": false, "max_abs": 0.20257568359375, "nrmse": 0.02729034051299095}, {"layer": 21, "state": "recurrent", "exact": false, "max_abs": 0.016585752367973328, "nrmse": 0.025008520111441612}, {"layer": 22, "state": "conv", "exact": false, "max_abs": 0.28125, "nrmse": 0.03399784490466118}, {"layer": 22, "state": "recurrent", "exact": false, "max_abs": 0.010384783148765564, "nrmse": 0.03258032351732254}, {"layer": 23, "state": "key", "exact": false, "max_abs": 0.3359375, "nrmse": 0.031050749123096466}, {"layer": 23, "state": "value", "exact": false, "max_abs": 0.3046875, "nrmse": 0.04110908880829811}, {"layer": 24, "state": "conv", "exact": false, "max_abs": 0.1953125, "nrmse": 0.03933577612042427}, {"layer": 24, "state": "recurrent", "exact": false, "max_abs": 0.01117938756942749, "nrmse": 0.03183792531490326}, {"layer": 25, "state": "conv", "exact": false, "max_abs": 0.25, "nrmse": 0.03903399035334587}, {"layer": 25, "state": "recurrent", "exact": false, "max_abs": 0.009317323565483093, "nrmse": 0.028552260249853134}, {"layer": 26, "state": "conv", "exact": false, "max_abs": 0.21875, "nrmse": 0.0428243950009346}, {"layer": 26, "state": "recurrent", "exact": false, "max_abs": 0.012021340429782867, "nrmse": 0.03103220835328102}, {"layer": 27, "state": "key", "exact": false, "max_abs": 0.3681640625, "nrmse": 0.032478757202625275}, {"layer": 27, "state": "value", "exact": false, "max_abs": 0.359375, "nrmse": 0.03846028074622154}, {"layer": 28, "state": "conv", "exact": false, "max_abs": 0.2890625, "nrmse": 0.048020217567682266}, {"layer": 28, "state": "recurrent", "exact": false, "max_abs": 0.009120527654886246, "nrmse": 0.03970470279455185}, {"layer": 29, "state": "conv", "exact": false, "max_abs": 0.25, "nrmse": 0.05556854233145714}, {"layer": 29, "state": "recurrent", "exact": false, "max_abs": 0.010848216712474823, "nrmse": 0.04578101634979248}, {"layer": 30, "state": "conv", "exact": false, "max_abs": 0.34375, "nrmse": 0.05779753252863884}, {"layer": 30, "state": "recurrent", "exact": false, "max_abs": 0.054690882563591, "nrmse": 0.05238010361790657}, {"layer": 31, "state": "key", "exact": false, "max_abs": 0.578125, "nrmse": 0.03438999876379967}, {"layer": 31, "state": "value", "exact": false, "max_abs": 0.52734375, "nrmse": 0.04723092168569565}, {"layer": 32, "state": "conv", "exact": false, "max_abs": 0.453125, "nrmse": 0.06662819534540176}, {"layer": 32, "state": "recurrent", "exact": false, "max_abs": 0.018993139266967773, "nrmse": 0.05988849326968193}, {"layer": 33, "state": "conv", "exact": false, "max_abs": 0.46875, "nrmse": 0.06980583816766739}, {"layer": 33, "state": "recurrent", "exact": false, "max_abs": 0.03056234121322632, "nrmse": 0.07154297083616257}, {"layer": 34, "state": "conv", "exact": false, "max_abs": 0.46875, "nrmse": 0.06633146107196808}, {"layer": 34, "state": "recurrent", "exact": false, "max_abs": 0.03384364768862724, "nrmse": 0.058720264583826065}, {"layer": 35, "state": "key", "exact": false, "max_abs": 0.578125, "nrmse": 0.035733748227357864}, {"layer": 35, "state": "value", "exact": false, "max_abs": 0.5078125, "nrmse": 0.050750963389873505}, {"layer": 36, "state": "conv", "exact": false, "max_abs": 0.5, "nrmse": 0.05571591481566429}, {"layer": 36, "state": "recurrent", "exact": false, "max_abs": 0.03369385004043579, "nrmse": 0.038860056549310684}, {"layer": 37, "state": "conv", "exact": false, "max_abs": 0.50390625, "nrmse": 0.06112487241625786}, {"layer": 37, "state": "recurrent", "exact": false, "max_abs": 0.05428065359592438, "nrmse": 0.05420928820967674}, {"layer": 38, "state": "conv", "exact": false, "max_abs": 0.546875, "nrmse": 0.0771539956331253}, {"layer": 38, "state": "recurrent", "exact": false, "max_abs": 0.01704704761505127, "nrmse": 0.07323625683784485}, {"layer": 39, "state": "key", "exact": false, "max_abs": 1.171875, "nrmse": 0.03821973130106926}, {"layer": 39, "state": "value", "exact": false, "max_abs": 1.5390625, "nrmse": 0.05746109038591385}, {"layer": 40, "state": "conv", "exact": false, "max_abs": 0.6875, "nrmse": 0.0860784575343132}, {"layer": 40, "state": "recurrent", "exact": false, "max_abs": 0.038080811500549316, "nrmse": 0.03444210812449455}, {"layer": 41, "state": "conv", "exact": false, "max_abs": 0.71875, "nrmse": 0.09112459421157837}, {"layer": 41, "state": "recurrent", "exact": false, "max_abs": 0.027652859687805176, "nrmse": 0.0786505714058876}, {"layer": 42, "state": "conv", "exact": false, "max_abs": 0.75, "nrmse": 0.10304129868745804}, {"layer": 42, "state": "recurrent", "exact": false, "max_abs": 0.05603890120983124, "nrmse": 0.07925857603549957}, {"layer": 43, "state": "key", "exact": false, "max_abs": 1.109375, "nrmse": 0.044905420392751694}, {"layer": 43, "state": "value", "exact": false, "max_abs": 1.328125, "nrmse": 0.05676501989364624}, {"layer": 44, "state": "conv", "exact": false, "max_abs": 0.7421875, "nrmse": 0.11491772532463074}, {"layer": 44, "state": "recurrent", "exact": false, "max_abs": 0.036311328411102295, "nrmse": 0.10141896456480026}, {"layer": 45, "state": "conv", "exact": false, "max_abs": 0.83984375, "nrmse": 0.13108636438846588}, {"layer": 45, "state": "recurrent", "exact": false, "max_abs": 0.02536379173398018, "nrmse": 0.1294761598110199}, {"layer": 46, "state": "conv", "exact": false, "max_abs": 1.03125, "nrmse": 0.13657069206237793}, {"layer": 46, "state": "recurrent", "exact": false, "max_abs": 0.04649195075035095, "nrmse": 0.1341523677110672}, {"layer": 47, "state": "key", "exact": false, "max_abs": 0.921875, "nrmse": 0.045367442071437836}, {"layer": 47, "state": "value", "exact": false, "max_abs": 1.7109375, "nrmse": 0.06669075042009354}, {"layer": 48, "state": "conv", "exact": false, "max_abs": 1.65625, "nrmse": 0.15020841360092163}, {"layer": 48, "state": "recurrent", "exact": false, "max_abs": 0.03264278918504715, "nrmse": 0.14419467747211456}, {"layer": 49, "state": "conv", "exact": false, "max_abs": 1.5625, "nrmse": 0.16281495988368988}, {"layer": 49, "state": "recurrent", "exact": false, "max_abs": 0.056175678968429565, "nrmse": 0.15675988793373108}, {"layer": 50, "state": "conv", "exact": false, "max_abs": 1.92578125, "nrmse": 0.15526561439037323}, {"layer": 50, "state": "recurrent", "exact": false, "max_abs": 0.028942108154296875, "nrmse": 0.14094018936157227}, {"layer": 51, "state": "key", "exact": false, "max_abs": 1.5234375, "nrmse": 0.04533622786402702}, {"layer": 51, "state": "value", "exact": false, "max_abs": 3.125, "nrmse": 0.06317996233701706}, {"layer": 52, "state": "conv", "exact": false, "max_abs": 1.46875, "nrmse": 0.13614018261432648}, {"layer": 52, "state": "recurrent", "exact": false, "max_abs": 0.12307286262512207, "nrmse": 0.09064915031194687}, {"layer": 53, "state": "conv", "exact": false, "max_abs": 1.80078125, "nrmse": 0.15963412821292877}, {"layer": 53, "state": "recurrent", "exact": false, "max_abs": 0.08056384325027466, "nrmse": 0.13196563720703125}, {"layer": 54, "state": "conv", "exact": false, "max_abs": 1.60546875, "nrmse": 0.17995159327983856}, {"layer": 54, "state": "recurrent", "exact": false, "max_abs": 0.04900136962532997, "nrmse": 0.15859141945838928}, {"layer": 55, "state": "key", "exact": false, "max_abs": 6.421875, "nrmse": 0.05801068991422653}, {"layer": 55, "state": "value", "exact": false, "max_abs": 11.25, "nrmse": 0.07032923400402069}, {"layer": 56, "state": "conv", "exact": false, "max_abs": 1.8046875, "nrmse": 0.17940671741962433}, {"layer": 56, "state": "recurrent", "exact": false, "max_abs": 0.113425612449646, "nrmse": 0.11876164376735687}, {"layer": 57, "state": "conv", "exact": false, "max_abs": 2.125, "nrmse": 0.17346957325935364}, {"layer": 57, "state": "recurrent", "exact": false, "max_abs": 0.11382734775543213, "nrmse": 0.12613755464553833}, {"layer": 58, "state": "conv", "exact": false, "max_abs": 2.125, "nrmse": 0.1863757073879242}, {"layer": 58, "state": "recurrent", "exact": false, "max_abs": 0.15572810173034668, "nrmse": 0.10383633524179459}, {"layer": 59, "state": "key", "exact": false, "max_abs": 5.232421875, "nrmse": 0.05959430709481239}, {"layer": 59, "state": "value", "exact": false, "max_abs": 5.0, "nrmse": 0.055543072521686554}, {"layer": 60, "state": "conv", "exact": false, "max_abs": 3.5, "nrmse": 0.1593857705593109}, {"layer": 60, "state": "recurrent", "exact": false, "max_abs": 0.13369032740592957, "nrmse": 0.1249513253569603}, {"layer": 61, "state": "conv", "exact": false, "max_abs": 2.2998046875, "nrmse": 0.18737463653087616}, {"layer": 61, "state": "recurrent", "exact": false, "max_abs": 0.0852140486240387, "nrmse": 0.13187021017074585}, {"layer": 62, "state": "conv", "exact": false, "max_abs": 3.1328125, "nrmse": 0.1593492478132248}, {"layer": 62, "state": "recurrent", "exact": false, "max_abs": 0.11956894397735596, "nrmse": 0.09254644066095352}, {"layer": 63, "state": "key", "exact": false, "max_abs": 2.708984375, "nrmse": 0.05999032035470009}, {"layer": 63, "state": "value", "exact": false, "max_abs": 8.0625, "nrmse": 0.03741009160876274}]} -{"phase": "capture_restore", "case": "gsm8k/test/783", "state_diff": []} -{"phase": "decode", "case": "gsm8k/test/783", "step": 0, "teacher_token": 1206, "compact_buffered": {"exact": false, "max_abs": 0.974609375, "nrmse": 0.05625910311937332}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [11290, 1423, 1423], "layer_diff": [{"layer": 4, "compact_buffered": {"exact": false, "max_abs": 0.0078125, "nrmse": 0.002414735732600093}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 5, "compact_buffered": {"exact": false, "max_abs": 0.0078125, "nrmse": 0.00384372565895319}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 6, "compact_buffered": {"exact": false, "max_abs": 0.015625, "nrmse": 0.004754701629281044}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 7, "compact_buffered": {"exact": false, "max_abs": 0.015625, "nrmse": 0.005177075508981943}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 8, "compact_buffered": {"exact": false, "max_abs": 0.03125, "nrmse": 0.0057281446643173695}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 9, "compact_buffered": {"exact": false, "max_abs": 0.03125, "nrmse": 0.006632520351558924}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 10, "compact_buffered": {"exact": false, "max_abs": 0.03125, "nrmse": 0.007444664370268583}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 11, "compact_buffered": {"exact": false, "max_abs": 0.03125, "nrmse": 0.008246475830674171}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 12, "compact_buffered": {"exact": false, "max_abs": 0.03125, "nrmse": 0.010105502791702747}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 13, "compact_buffered": {"exact": false, "max_abs": 0.03125, "nrmse": 0.011514963582158089}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 14, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.013831562362611294}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 15, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.01455683447420597}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 16, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.015169491060078144}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 17, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.016614539548754692}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 18, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.018465878441929817}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 19, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.021609630435705185}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 20, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.028012724593281746}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 21, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.03261345997452736}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 22, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.035256434231996536}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 23, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.03834041580557823}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 24, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.039300840348005295}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 25, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.03992803022265434}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 26, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.04415490850806236}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 27, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.04781073331832886}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 28, "compact_buffered": {"exact": false, "max_abs": 0.16796875, "nrmse": 0.052162960171699524}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 29, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.05881553143262863}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 30, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.060613613575696945}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 31, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.06791423261165619}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 32, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.06846141815185547}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 33, "compact_buffered": {"exact": false, "max_abs": 0.2578125, "nrmse": 0.07021299004554749}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 34, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.06502822786569595}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 35, "compact_buffered": {"exact": false, "max_abs": 0.2421875, "nrmse": 0.06893744319677353}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 36, "compact_buffered": {"exact": false, "max_abs": 0.625, "nrmse": 0.07532521337270737}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 37, "compact_buffered": {"exact": false, "max_abs": 0.8125, "nrmse": 0.08114398270845413}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 38, "compact_buffered": {"exact": false, "max_abs": 0.9375, "nrmse": 0.08562914282083511}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 39, "compact_buffered": {"exact": false, "max_abs": 0.8125, "nrmse": 0.09281440824270248}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 40, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.10298649221658707}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 41, "compact_buffered": {"exact": false, "max_abs": 1.125, "nrmse": 0.10287047177553177}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 42, "compact_buffered": {"exact": false, "max_abs": 0.875, "nrmse": 0.10330410301685333}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 43, "compact_buffered": {"exact": false, "max_abs": 1.25, "nrmse": 0.10663195699453354}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 44, "compact_buffered": {"exact": false, "max_abs": 2.5, "nrmse": 0.1096184030175209}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 45, "compact_buffered": {"exact": false, "max_abs": 2.75, "nrmse": 0.11174024641513824}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 46, "compact_buffered": {"exact": false, "max_abs": 2.0, "nrmse": 0.1064935252070427}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 47, "compact_buffered": {"exact": false, "max_abs": 1.75, "nrmse": 0.10598213970661163}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 48, "compact_buffered": {"exact": false, "max_abs": 1.75, "nrmse": 0.10720738768577576}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 49, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.10565706342458725}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 50, "compact_buffered": {"exact": false, "max_abs": 1.375, "nrmse": 0.103025883436203}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 51, "compact_buffered": {"exact": false, "max_abs": 1.3125, "nrmse": 0.10396289825439453}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 52, "compact_buffered": {"exact": false, "max_abs": 1.0625, "nrmse": 0.10301118344068527}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 53, "compact_buffered": {"exact": false, "max_abs": 1.02197265625, "nrmse": 0.10523132234811783}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 54, "compact_buffered": {"exact": false, "max_abs": 1.20703125, "nrmse": 0.10553404688835144}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 55, "compact_buffered": {"exact": false, "max_abs": 1.5, "nrmse": 0.1074504628777504}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 56, "compact_buffered": {"exact": false, "max_abs": 1.669921875, "nrmse": 0.10849025845527649}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 57, "compact_buffered": {"exact": false, "max_abs": 1.7265625, "nrmse": 0.11142552644014359}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 58, "compact_buffered": {"exact": false, "max_abs": 1.484375, "nrmse": 0.11257363110780716}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 59, "compact_buffered": {"exact": false, "max_abs": 3.75, "nrmse": 0.11317288130521774}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 60, "compact_buffered": {"exact": false, "max_abs": 2.5, "nrmse": 0.10815378278493881}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 61, "compact_buffered": {"exact": false, "max_abs": 2.5, "nrmse": 0.1071418821811676}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 62, "compact_buffered": {"exact": false, "max_abs": 1.66796875, "nrmse": 0.10040110349655151}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 63, "compact_buffered": {"exact": false, "max_abs": 9.0, "nrmse": 0.09951882809400558}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}], "state_diff": []} -{"phase": "decode", "case": "gsm8k/test/783", "step": 1, "teacher_token": 11290, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.039853908121585846}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [411, 411, 411], "layer_diff": [{"layer": 4, "compact_buffered": {"exact": false, "max_abs": 0.00390625, "nrmse": 0.001937157823704183}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 5, "compact_buffered": {"exact": false, "max_abs": 0.00634765625, "nrmse": 0.0034559040796011686}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 6, "compact_buffered": {"exact": false, "max_abs": 0.01171875, "nrmse": 0.004881524946540594}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 7, "compact_buffered": {"exact": false, "max_abs": 0.015625, "nrmse": 0.005590427201241255}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 8, "compact_buffered": {"exact": false, "max_abs": 0.015625, "nrmse": 0.0061693983152508736}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 9, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.008426614105701447}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 10, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.009256651625037193}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 11, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.009782951325178146}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 12, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.010734330862760544}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 13, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.011729815043509007}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 14, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.012630747631192207}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 15, "compact_buffered": {"exact": false, "max_abs": 0.09375, "nrmse": 0.012731661088764668}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 16, "compact_buffered": {"exact": false, "max_abs": 0.09375, "nrmse": 0.013253632001578808}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 17, "compact_buffered": {"exact": false, "max_abs": 0.09375, "nrmse": 0.014254373498260975}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 18, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.015917813405394554}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 19, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.017942845821380615}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 20, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.020108921453356743}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 21, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.021238399669528008}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 22, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.021709738299250603}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 23, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.022708933800458908}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 24, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.023120995610952377}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 25, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.022887825965881348}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 26, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.02246869169175625}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 27, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.023418782278895378}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 28, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.0236501507461071}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 29, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.02560492418706417}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 30, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.02625681273639202}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 31, "compact_buffered": {"exact": false, "max_abs": 0.75, "nrmse": 0.028915071859955788}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 32, "compact_buffered": {"exact": false, "max_abs": 0.75, "nrmse": 0.032267168164253235}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 33, "compact_buffered": {"exact": false, "max_abs": 0.75, "nrmse": 0.03179093077778816}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 34, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.031700436025857925}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 35, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.03407158702611923}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 36, "compact_buffered": {"exact": false, "max_abs": 0.75, "nrmse": 0.035921867936849594}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 37, "compact_buffered": {"exact": false, "max_abs": 1.5, "nrmse": 0.04028245806694031}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 38, "compact_buffered": {"exact": false, "max_abs": 2.25, "nrmse": 0.04401605203747749}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 39, "compact_buffered": {"exact": false, "max_abs": 2.5, "nrmse": 0.04746328294277191}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 40, "compact_buffered": {"exact": false, "max_abs": 3.25, "nrmse": 0.05463847890496254}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 41, "compact_buffered": {"exact": false, "max_abs": 2.75, "nrmse": 0.05202601104974747}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 42, "compact_buffered": {"exact": false, "max_abs": 2.5, "nrmse": 0.05029167979955673}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 43, "compact_buffered": {"exact": false, "max_abs": 2.25, "nrmse": 0.04947904124855995}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 44, "compact_buffered": {"exact": false, "max_abs": 2.5, "nrmse": 0.05174306780099869}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 45, "compact_buffered": {"exact": false, "max_abs": 2.5, "nrmse": 0.05420791730284691}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 46, "compact_buffered": {"exact": false, "max_abs": 2.25, "nrmse": 0.05487946420907974}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 47, "compact_buffered": {"exact": false, "max_abs": 2.0, "nrmse": 0.0536222979426384}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 48, "compact_buffered": {"exact": false, "max_abs": 1.25, "nrmse": 0.05640975385904312}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 49, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.055657438933849335}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 50, "compact_buffered": {"exact": false, "max_abs": 1.75, "nrmse": 0.05937744677066803}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 51, "compact_buffered": {"exact": false, "max_abs": 1.0625, "nrmse": 0.06171370670199394}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 52, "compact_buffered": {"exact": false, "max_abs": 0.6484375, "nrmse": 0.06419426947832108}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 53, "compact_buffered": {"exact": false, "max_abs": 0.625, "nrmse": 0.06600390374660492}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 54, "compact_buffered": {"exact": false, "max_abs": 0.6875, "nrmse": 0.06675635278224945}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 55, "compact_buffered": {"exact": false, "max_abs": 1.375, "nrmse": 0.06845012307167053}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 56, "compact_buffered": {"exact": false, "max_abs": 2.0, "nrmse": 0.07039323449134827}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 57, "compact_buffered": {"exact": false, "max_abs": 1.75, "nrmse": 0.07223542779684067}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 58, "compact_buffered": {"exact": false, "max_abs": 3.25, "nrmse": 0.07357407361268997}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 59, "compact_buffered": {"exact": false, "max_abs": 4.5, "nrmse": 0.0725579485297203}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 60, "compact_buffered": {"exact": false, "max_abs": 5.0, "nrmse": 0.07378881424665451}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 61, "compact_buffered": {"exact": false, "max_abs": 4.5, "nrmse": 0.07220984995365143}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 62, "compact_buffered": {"exact": false, "max_abs": 1.5, "nrmse": 0.07046609371900558}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 63, "compact_buffered": {"exact": false, "max_abs": 4.0, "nrmse": 0.07392653822898865}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}], "state_diff": []} -{"phase": "decode", "case": "gsm8k/test/783", "step": 2, "teacher_token": 411, "compact_buffered": {"exact": false, "max_abs": 1.125, "nrmse": 0.038314443081617355}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [3377, 3377, 3377], "layer_diff": [{"layer": 4, "compact_buffered": {"exact": false, "max_abs": 0.001953125, "nrmse": 0.001413944410160184}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 5, "compact_buffered": {"exact": false, "max_abs": 0.00390625, "nrmse": 0.0025929908733814955}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 6, "compact_buffered": {"exact": false, "max_abs": 0.125, "nrmse": 0.0065256706438958645}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 7, "compact_buffered": {"exact": false, "max_abs": 0.125, "nrmse": 0.006489915773272514}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 8, "compact_buffered": {"exact": false, "max_abs": 0.125, "nrmse": 0.006863136310130358}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 9, "compact_buffered": {"exact": false, "max_abs": 0.125, "nrmse": 0.00780177116394043}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 10, "compact_buffered": {"exact": false, "max_abs": 0.03125, "nrmse": 0.007701684255152941}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 11, "compact_buffered": {"exact": false, "max_abs": 0.03125, "nrmse": 0.007918592542409897}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 12, "compact_buffered": {"exact": false, "max_abs": 0.03125, "nrmse": 0.0085104089230299}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 13, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.010960174724459648}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 14, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.011366435326635838}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 15, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.011874321848154068}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 16, "compact_buffered": {"exact": false, "max_abs": 0.029296875, "nrmse": 0.01109798438847065}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 17, "compact_buffered": {"exact": false, "max_abs": 0.03125, "nrmse": 0.012567637488245964}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 18, "compact_buffered": {"exact": false, "max_abs": 0.0546875, "nrmse": 0.014840496703982353}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 19, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.02067406103014946}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 20, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.02180362306535244}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 21, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.019383225589990616}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 22, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.018842944875359535}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 23, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.021620556712150574}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 24, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.023076236248016357}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 25, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.024538928642868996}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 26, "compact_buffered": {"exact": false, "max_abs": 0.75, "nrmse": 0.024735350161790848}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 27, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.02566756308078766}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 28, "compact_buffered": {"exact": false, "max_abs": 0.109375, "nrmse": 0.025314128026366234}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 29, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.025354573503136635}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 30, "compact_buffered": {"exact": false, "max_abs": 0.1171875, "nrmse": 0.02392597123980522}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 31, "compact_buffered": {"exact": false, "max_abs": 0.109375, "nrmse": 0.027462171390652657}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 32, "compact_buffered": {"exact": false, "max_abs": 0.1328125, "nrmse": 0.030694475397467613}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 33, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.031588438898324966}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 34, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.031929198652505875}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 35, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.03368670865893364}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 36, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.03559720888733864}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 37, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.0357658676803112}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 38, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.03379024937748909}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 39, "compact_buffered": {"exact": false, "max_abs": 0.1875, "nrmse": 0.03362782672047615}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 40, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.03444388508796692}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 41, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.03599659353494644}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 42, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.03479611873626709}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 43, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.03643143177032471}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 44, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.036703068763017654}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 45, "compact_buffered": {"exact": false, "max_abs": 0.2734375, "nrmse": 0.03716227039694786}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 46, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.04003520309925079}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 47, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.041164007037878036}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 48, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.044570911675691605}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 49, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.05190445855259895}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 50, "compact_buffered": {"exact": false, "max_abs": 0.75, "nrmse": 0.06345446407794952}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 51, "compact_buffered": {"exact": false, "max_abs": 0.5625, "nrmse": 0.06024077534675598}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 52, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.06320164352655411}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 53, "compact_buffered": {"exact": false, "max_abs": 1.375, "nrmse": 0.06556501984596252}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 54, "compact_buffered": {"exact": false, "max_abs": 2.375, "nrmse": 0.0643397867679596}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 55, "compact_buffered": {"exact": false, "max_abs": 2.25, "nrmse": 0.06505104899406433}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 56, "compact_buffered": {"exact": false, "max_abs": 2.75, "nrmse": 0.06728604435920715}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 57, "compact_buffered": {"exact": false, "max_abs": 2.25, "nrmse": 0.06634783744812012}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 58, "compact_buffered": {"exact": false, "max_abs": 4.0, "nrmse": 0.0666070505976677}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 59, "compact_buffered": {"exact": false, "max_abs": 2.0, "nrmse": 0.06136871874332428}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 60, "compact_buffered": {"exact": false, "max_abs": 1.3125, "nrmse": 0.062480222433805466}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 61, "compact_buffered": {"exact": false, "max_abs": 1.5, "nrmse": 0.06310445070266724}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 62, "compact_buffered": {"exact": false, "max_abs": 2.0, "nrmse": 0.06901755928993225}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 63, "compact_buffered": {"exact": false, "max_abs": 15.0, "nrmse": 0.08602791279554367}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}], "state_diff": []} -{"phase": "decode", "case": "gsm8k/test/783", "step": 3, "teacher_token": 3377, "compact_buffered": {"exact": false, "max_abs": 0.65625, "nrmse": 0.028664033859968185}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [11, 11, 11], "layer_diff": [{"layer": 4, "compact_buffered": {"exact": false, "max_abs": 0.00390625, "nrmse": 0.0020272277761250734}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 5, "compact_buffered": {"exact": false, "max_abs": 0.005859375, "nrmse": 0.0035160877741873264}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 6, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.00923036877065897}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 7, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.009353787638247013}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 8, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.009560019709169865}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 9, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.010283420793712139}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 10, "compact_buffered": {"exact": false, "max_abs": 0.03125, "nrmse": 0.00967183243483305}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 11, "compact_buffered": {"exact": false, "max_abs": 0.03125, "nrmse": 0.010194533504545689}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 12, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.01481553167104721}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 13, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.015400763601064682}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 14, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.016414429992437363}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 15, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.01728709042072296}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 16, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.01595035009086132}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 17, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.017050476744771004}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 18, "compact_buffered": {"exact": false, "max_abs": 0.0625, "nrmse": 0.017887074500322342}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 19, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.020061912015080452}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 20, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.021487759426236153}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 21, "compact_buffered": {"exact": false, "max_abs": 0.07861328125, "nrmse": 0.02155737578868866}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 22, "compact_buffered": {"exact": false, "max_abs": 0.09375, "nrmse": 0.02118222415447235}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 23, "compact_buffered": {"exact": false, "max_abs": 0.09375, "nrmse": 0.021802539005875587}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 24, "compact_buffered": {"exact": false, "max_abs": 0.09375, "nrmse": 0.023080717772245407}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 25, "compact_buffered": {"exact": false, "max_abs": 0.078125, "nrmse": 0.023335305973887444}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 26, "compact_buffered": {"exact": false, "max_abs": 0.08203125, "nrmse": 0.022973353043198586}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 27, "compact_buffered": {"exact": false, "max_abs": 0.1015625, "nrmse": 0.02471119351685047}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 28, "compact_buffered": {"exact": false, "max_abs": 0.09765625, "nrmse": 0.026273002848029137}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 29, "compact_buffered": {"exact": false, "max_abs": 0.1015625, "nrmse": 0.02716190740466118}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 30, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.027964988723397255}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 31, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.02931821160018444}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 32, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.029867947101593018}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 33, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.03068857640028}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 34, "compact_buffered": {"exact": false, "max_abs": 0.21875, "nrmse": 0.028890540823340416}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 35, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.031068040058016777}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 36, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.03262987360358238}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 37, "compact_buffered": {"exact": false, "max_abs": 0.75, "nrmse": 0.03478429093956947}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 38, "compact_buffered": {"exact": false, "max_abs": 0.75, "nrmse": 0.03447679057717323}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 39, "compact_buffered": {"exact": false, "max_abs": 0.75, "nrmse": 0.035309385508298874}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 40, "compact_buffered": {"exact": false, "max_abs": 0.75, "nrmse": 0.035356033593416214}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 41, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.03535252436995506}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 42, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.034423843026161194}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 43, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.03484973683953285}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 44, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.036613184958696365}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 45, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.03919495642185211}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 46, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.04241687431931496}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 47, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.04416446015238762}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 48, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.04741004481911659}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 49, "compact_buffered": {"exact": false, "max_abs": 0.46875, "nrmse": 0.05063139274716377}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 50, "compact_buffered": {"exact": false, "max_abs": 1.625, "nrmse": 0.055415648967027664}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 51, "compact_buffered": {"exact": false, "max_abs": 1.5625, "nrmse": 0.05665857717394829}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 52, "compact_buffered": {"exact": false, "max_abs": 1.6875, "nrmse": 0.059024836868047714}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 53, "compact_buffered": {"exact": false, "max_abs": 1.7421875, "nrmse": 0.061801839619874954}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 54, "compact_buffered": {"exact": false, "max_abs": 0.8125, "nrmse": 0.06024974212050438}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 55, "compact_buffered": {"exact": false, "max_abs": 0.875, "nrmse": 0.06329134851694107}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 56, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.06460049748420715}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 57, "compact_buffered": {"exact": false, "max_abs": 0.9375, "nrmse": 0.06627743691205978}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 58, "compact_buffered": {"exact": false, "max_abs": 2.125, "nrmse": 0.06858351826667786}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 59, "compact_buffered": {"exact": false, "max_abs": 0.9375, "nrmse": 0.06620936840772629}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 60, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.06742481887340546}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 61, "compact_buffered": {"exact": false, "max_abs": 1.125, "nrmse": 0.06578367203474045}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 62, "compact_buffered": {"exact": false, "max_abs": 1.53125, "nrmse": 0.0665970966219902}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 63, "compact_buffered": {"exact": false, "max_abs": 4.0, "nrmse": 0.05924805626273155}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}], "state_diff": []} -{"phase": "decode", "case": "gsm8k/test/783", "step": 4, "teacher_token": 11, "compact_buffered": {"exact": false, "max_abs": 1.25, "nrmse": 0.05096922442317009}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [567, 567, 567], "layer_diff": [{"layer": 4, "compact_buffered": {"exact": false, "max_abs": 0.001953125, "nrmse": 0.001373784034512937}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 5, "compact_buffered": {"exact": false, "max_abs": 0.00390625, "nrmse": 0.003010929562151432}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 6, "compact_buffered": {"exact": false, "max_abs": 0.125, "nrmse": 0.007279893383383751}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 7, "compact_buffered": {"exact": false, "max_abs": 0.125, "nrmse": 0.007290353532880545}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 8, "compact_buffered": {"exact": false, "max_abs": 0.0107421875, "nrmse": 0.0066311173141002655}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 9, "compact_buffered": {"exact": false, "max_abs": 0.01318359375, "nrmse": 0.008093317970633507}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 10, "compact_buffered": {"exact": false, "max_abs": 0.125, "nrmse": 0.010748814791440964}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 11, "compact_buffered": {"exact": false, "max_abs": 0.125, "nrmse": 0.011071505956351757}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 12, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.013522695749998093}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 13, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.013516333885490894}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 14, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.014426064677536488}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 15, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.014846136793494225}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 16, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.015206272713840008}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 17, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.015990158542990685}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 18, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.01690836250782013}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 19, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.020414285361766815}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 20, "compact_buffered": {"exact": false, "max_abs": 0.0625, "nrmse": 0.02224644087255001}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 21, "compact_buffered": {"exact": false, "max_abs": 0.0625, "nrmse": 0.022058825939893723}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 22, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.022227110341191292}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 23, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.025315746665000916}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 24, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.02626638673245907}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 25, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.028465455397963524}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 26, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.027179453521966934}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 27, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.030306659638881683}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 28, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.032765623182058334}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 29, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.03442922979593277}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 30, "compact_buffered": {"exact": false, "max_abs": 0.75, "nrmse": 0.0341387614607811}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 31, "compact_buffered": {"exact": false, "max_abs": 0.75, "nrmse": 0.03703862801194191}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 32, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.040002256631851196}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 33, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.04096589237451553}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 34, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.037004005163908005}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 35, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.04049525409936905}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 36, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.042188860476017}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 37, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.043170541524887085}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 38, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.041113272309303284}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 39, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.04240073636174202}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 40, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.04327702149748802}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 41, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.0451485775411129}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 42, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.043858014047145844}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 43, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.04596129432320595}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 44, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.04675419256091118}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 45, "compact_buffered": {"exact": false, "max_abs": 0.21875, "nrmse": 0.04538975656032562}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 46, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.049871090799570084}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 47, "compact_buffered": {"exact": false, "max_abs": 0.3125, "nrmse": 0.04762207716703415}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 48, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.04922783002257347}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 49, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.051972828805446625}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 50, "compact_buffered": {"exact": false, "max_abs": 1.75, "nrmse": 0.0565619170665741}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 51, "compact_buffered": {"exact": false, "max_abs": 1.75, "nrmse": 0.05577949061989784}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 52, "compact_buffered": {"exact": false, "max_abs": 1.75, "nrmse": 0.05686844512820244}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 53, "compact_buffered": {"exact": false, "max_abs": 1.375, "nrmse": 0.05972279980778694}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 54, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.059351153671741486}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 55, "compact_buffered": {"exact": false, "max_abs": 0.875, "nrmse": 0.06202330067753792}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 56, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.06281964480876923}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 57, "compact_buffered": {"exact": false, "max_abs": 2.125, "nrmse": 0.06374262273311615}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 58, "compact_buffered": {"exact": false, "max_abs": 5.5, "nrmse": 0.060139648616313934}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 59, "compact_buffered": {"exact": false, "max_abs": 1.125, "nrmse": 0.06404081732034683}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 60, "compact_buffered": {"exact": false, "max_abs": 3.5, "nrmse": 0.06312526762485504}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 61, "compact_buffered": {"exact": false, "max_abs": 3.5, "nrmse": 0.06351176649332047}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 62, "compact_buffered": {"exact": false, "max_abs": 6.0, "nrmse": 0.0649857372045517}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 63, "compact_buffered": {"exact": false, "max_abs": 12.0, "nrmse": 0.0770815834403038}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}], "state_diff": []} -{"phase": "decode", "case": "gsm8k/test/783", "step": 5, "teacher_token": 567, "compact_buffered": {"exact": false, "max_abs": 1.1875, "nrmse": 0.06808371841907501}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [1144, 1144, 1144], "layer_diff": [{"layer": 4, "compact_buffered": {"exact": false, "max_abs": 0.00390625, "nrmse": 0.0012614671140909195}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 5, "compact_buffered": {"exact": false, "max_abs": 0.125, "nrmse": 0.005090969614684582}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 6, "compact_buffered": {"exact": false, "max_abs": 0.125, "nrmse": 0.0057774861343204975}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 7, "compact_buffered": {"exact": false, "max_abs": 0.125, "nrmse": 0.005764863453805447}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 8, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.009035173803567886}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 9, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.009394921362400055}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 10, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.010179410688579082}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 11, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.01026198174804449}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 12, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.010717169381678104}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 13, "compact_buffered": {"exact": false, "max_abs": 0.0234375, "nrmse": 0.009204594418406487}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 14, "compact_buffered": {"exact": false, "max_abs": 0.0625, "nrmse": 0.010217724367976189}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 15, "compact_buffered": {"exact": false, "max_abs": 0.0625, "nrmse": 0.010805081576108932}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 16, "compact_buffered": {"exact": false, "max_abs": 0.0625, "nrmse": 0.011403422802686691}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 17, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.013414734043180943}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 18, "compact_buffered": {"exact": false, "max_abs": 0.0625, "nrmse": 0.014307253062725067}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 19, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.017344430088996887}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 20, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.02114364132285118}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 21, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.023102227598428726}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 22, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.02271069400012493}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 23, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.02264021337032318}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 24, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.025120921432971954}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 25, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.026062680408358574}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 26, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.03090566024184227}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 27, "compact_buffered": {"exact": false, "max_abs": 0.75, "nrmse": 0.03178852051496506}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 28, "compact_buffered": {"exact": false, "max_abs": 0.75, "nrmse": 0.0368795245885849}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 29, "compact_buffered": {"exact": false, "max_abs": 0.75, "nrmse": 0.040259018540382385}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 30, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.04555608704686165}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 31, "compact_buffered": {"exact": false, "max_abs": 0.75, "nrmse": 0.05103134363889694}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 32, "compact_buffered": {"exact": false, "max_abs": 0.546875, "nrmse": 0.05523514002561569}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 33, "compact_buffered": {"exact": false, "max_abs": 0.625, "nrmse": 0.061785172671079636}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 34, "compact_buffered": {"exact": false, "max_abs": 1.5, "nrmse": 0.06962373852729797}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 35, "compact_buffered": {"exact": false, "max_abs": 1.5, "nrmse": 0.07417062669992447}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 36, "compact_buffered": {"exact": false, "max_abs": 2.125, "nrmse": 0.08801506459712982}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 37, "compact_buffered": {"exact": false, "max_abs": 3.0, "nrmse": 0.10249558836221695}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 38, "compact_buffered": {"exact": false, "max_abs": 3.625, "nrmse": 0.11286698281764984}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 39, "compact_buffered": {"exact": false, "max_abs": 3.625, "nrmse": 0.12229280173778534}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 40, "compact_buffered": {"exact": false, "max_abs": 3.875, "nrmse": 0.127888485789299}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 41, "compact_buffered": {"exact": false, "max_abs": 3.75, "nrmse": 0.13254596292972565}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 42, "compact_buffered": {"exact": false, "max_abs": 5.5, "nrmse": 0.15752045810222626}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 43, "compact_buffered": {"exact": false, "max_abs": 5.5, "nrmse": 0.15920579433441162}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 44, "compact_buffered": {"exact": false, "max_abs": 6.5, "nrmse": 0.16609106957912445}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 45, "compact_buffered": {"exact": false, "max_abs": 5.5, "nrmse": 0.1619064062833786}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 46, "compact_buffered": {"exact": false, "max_abs": 6.5, "nrmse": 0.17398767173290253}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 47, "compact_buffered": {"exact": false, "max_abs": 6.0, "nrmse": 0.16531996428966522}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 48, "compact_buffered": {"exact": false, "max_abs": 5.5, "nrmse": 0.14914633333683014}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 49, "compact_buffered": {"exact": false, "max_abs": 4.75, "nrmse": 0.1418963223695755}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 50, "compact_buffered": {"exact": false, "max_abs": 4.0, "nrmse": 0.12723539769649506}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 51, "compact_buffered": {"exact": false, "max_abs": 4.5, "nrmse": 0.12362123280763626}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 52, "compact_buffered": {"exact": false, "max_abs": 5.0, "nrmse": 0.12033507972955704}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 53, "compact_buffered": {"exact": false, "max_abs": 3.75, "nrmse": 0.11800701171159744}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 54, "compact_buffered": {"exact": false, "max_abs": 1.375, "nrmse": 0.11124955862760544}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 55, "compact_buffered": {"exact": false, "max_abs": 1.5625, "nrmse": 0.11174485087394714}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 56, "compact_buffered": {"exact": false, "max_abs": 1.5, "nrmse": 0.10841432958841324}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 57, "compact_buffered": {"exact": false, "max_abs": 1.53125, "nrmse": 0.10755490511655807}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 58, "compact_buffered": {"exact": false, "max_abs": 1.5625, "nrmse": 0.10707105696201324}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 59, "compact_buffered": {"exact": false, "max_abs": 1.6875, "nrmse": 0.10603530704975128}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 60, "compact_buffered": {"exact": false, "max_abs": 2.5, "nrmse": 0.10026637464761734}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 61, "compact_buffered": {"exact": false, "max_abs": 5.75, "nrmse": 0.09927684813737869}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 62, "compact_buffered": {"exact": false, "max_abs": 3.0, "nrmse": 0.09297248721122742}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 63, "compact_buffered": {"exact": false, "max_abs": 4.5, "nrmse": 0.09697619080543518}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}], "state_diff": []} -{"phase": "decode", "case": "gsm8k/test/783", "step": 6, "teacher_token": 1144, "compact_buffered": {"exact": false, "max_abs": 0.53125, "nrmse": 0.022107649594545364}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [310, 310, 310], "layer_diff": [{"layer": 4, "compact_buffered": {"exact": false, "max_abs": 0.00390625, "nrmse": 0.0014658485306426883}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 5, "compact_buffered": {"exact": false, "max_abs": 0.0078125, "nrmse": 0.0027024145238101482}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 6, "compact_buffered": {"exact": false, "max_abs": 0.015625, "nrmse": 0.003956501372158527}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 7, "compact_buffered": {"exact": false, "max_abs": 0.015625, "nrmse": 0.00447174534201622}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 8, "compact_buffered": {"exact": false, "max_abs": 0.015625, "nrmse": 0.005355948116630316}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 9, "compact_buffered": {"exact": false, "max_abs": 0.015625, "nrmse": 0.006478843279182911}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 10, "compact_buffered": {"exact": false, "max_abs": 0.0234375, "nrmse": 0.007759894710034132}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 11, "compact_buffered": {"exact": false, "max_abs": 0.03125, "nrmse": 0.008516036905348301}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 12, "compact_buffered": {"exact": false, "max_abs": 0.03125, "nrmse": 0.009961629286408424}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 13, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.012231300584971905}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 14, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.01594468206167221}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 15, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.014188682660460472}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 16, "compact_buffered": {"exact": false, "max_abs": 0.046875, "nrmse": 0.013737623579800129}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 17, "compact_buffered": {"exact": false, "max_abs": 0.05078125, "nrmse": 0.014616722241044044}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 18, "compact_buffered": {"exact": false, "max_abs": 0.060546875, "nrmse": 0.01628836616873741}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 19, "compact_buffered": {"exact": false, "max_abs": 0.0673828125, "nrmse": 0.01813034899532795}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 20, "compact_buffered": {"exact": false, "max_abs": 0.0732421875, "nrmse": 0.020894164219498634}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 21, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.022797392681241035}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 22, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.023841029033064842}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 23, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.02434373088181019}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 24, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.023592747747898102}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 25, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.023042969405651093}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 26, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.023092757910490036}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 27, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.025494882836937904}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 28, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.026052629575133324}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 29, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.026727071031928062}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 30, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.026270244270563126}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 31, "compact_buffered": {"exact": false, "max_abs": 0.109375, "nrmse": 0.02634720876812935}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 32, "compact_buffered": {"exact": false, "max_abs": 0.125, "nrmse": 0.02719780243933201}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 33, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.02728606015443802}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 34, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.025952642783522606}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 35, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.028317514806985855}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 36, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.031278446316719055}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 37, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.03341677784919739}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 38, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.032064031809568405}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 39, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.03242671862244606}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 40, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.03192763403058052}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 41, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.0333852656185627}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 42, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.032194167375564575}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 43, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.033296700567007065}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 44, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.03404496610164642}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 45, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.035442352294921875}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 46, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.03637658804655075}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 47, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.03707294911146164}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 48, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.04024029150605202}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 49, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.0427166149020195}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 50, "compact_buffered": {"exact": false, "max_abs": 1.25, "nrmse": 0.04485674202442169}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 51, "compact_buffered": {"exact": false, "max_abs": 0.75, "nrmse": 0.04739827290177345}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 52, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.04909827932715416}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 53, "compact_buffered": {"exact": false, "max_abs": 0.5625, "nrmse": 0.05143430456519127}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 54, "compact_buffered": {"exact": false, "max_abs": 0.65625, "nrmse": 0.0517585352063179}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 55, "compact_buffered": {"exact": false, "max_abs": 0.734375, "nrmse": 0.05394992604851723}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 56, "compact_buffered": {"exact": false, "max_abs": 1.625, "nrmse": 0.05598875507712364}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 57, "compact_buffered": {"exact": false, "max_abs": 1.5, "nrmse": 0.05659451335668564}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 58, "compact_buffered": {"exact": false, "max_abs": 1.5, "nrmse": 0.0561491996049881}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 59, "compact_buffered": {"exact": false, "max_abs": 1.25, "nrmse": 0.053102195262908936}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 60, "compact_buffered": {"exact": false, "max_abs": 3.0, "nrmse": 0.05593951418995857}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 61, "compact_buffered": {"exact": false, "max_abs": 1.5, "nrmse": 0.05172210559248924}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 62, "compact_buffered": {"exact": false, "max_abs": 2.0, "nrmse": 0.054296910762786865}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 63, "compact_buffered": {"exact": false, "max_abs": 8.0, "nrmse": 0.05066690593957901}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}], "state_diff": []} -{"phase": "decode", "case": "gsm8k/test/783", "step": 7, "teacher_token": 310, "compact_buffered": {"exact": false, "max_abs": 0.75, "nrmse": 0.044079191982746124}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [7995, 7995, 7995], "layer_diff": [{"layer": 4, "compact_buffered": {"exact": false, "max_abs": 0.001953125, "nrmse": 0.001219821977429092}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 5, "compact_buffered": {"exact": false, "max_abs": 0.00390625, "nrmse": 0.0025177476927638054}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 6, "compact_buffered": {"exact": false, "max_abs": 0.005859375, "nrmse": 0.0035539385862648487}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 7, "compact_buffered": {"exact": false, "max_abs": 0.125, "nrmse": 0.0061644986271858215}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 8, "compact_buffered": {"exact": false, "max_abs": 0.009765625, "nrmse": 0.004536329302936792}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 9, "compact_buffered": {"exact": false, "max_abs": 0.009765625, "nrmse": 0.005366444122046232}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 10, "compact_buffered": {"exact": false, "max_abs": 0.125, "nrmse": 0.00718729617074132}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 11, "compact_buffered": {"exact": false, "max_abs": 0.01220703125, "nrmse": 0.0063633075915277}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 12, "compact_buffered": {"exact": false, "max_abs": 0.015625, "nrmse": 0.0066779134795069695}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 13, "compact_buffered": {"exact": false, "max_abs": 0.015625, "nrmse": 0.007104129530489445}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 14, "compact_buffered": {"exact": false, "max_abs": 0.017578125, "nrmse": 0.007440043147653341}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 15, "compact_buffered": {"exact": false, "max_abs": 0.0234375, "nrmse": 0.007615047041326761}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 16, "compact_buffered": {"exact": false, "max_abs": 0.0167236328125, "nrmse": 0.007936840876936913}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 17, "compact_buffered": {"exact": false, "max_abs": 0.03125, "nrmse": 0.009463869966566563}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 18, "compact_buffered": {"exact": false, "max_abs": 0.0419921875, "nrmse": 0.010411993600428104}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 19, "compact_buffered": {"exact": false, "max_abs": 0.046875, "nrmse": 0.013325626961886883}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 20, "compact_buffered": {"exact": false, "max_abs": 0.05078125, "nrmse": 0.016214948147535324}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 21, "compact_buffered": {"exact": false, "max_abs": 0.048828125, "nrmse": 0.01645831950008869}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 22, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.017713656648993492}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 23, "compact_buffered": {"exact": false, "max_abs": 0.0625, "nrmse": 0.018968846648931503}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 24, "compact_buffered": {"exact": false, "max_abs": 0.0625, "nrmse": 0.019592316821217537}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 25, "compact_buffered": {"exact": false, "max_abs": 0.0625, "nrmse": 0.020424608141183853}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 26, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.021344607695937157}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 27, "compact_buffered": {"exact": false, "max_abs": 0.0791015625, "nrmse": 0.02436346933245659}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 28, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.027358844876289368}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 29, "compact_buffered": {"exact": false, "max_abs": 0.1015625, "nrmse": 0.029535137116909027}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 30, "compact_buffered": {"exact": false, "max_abs": 0.1015625, "nrmse": 0.030151844024658203}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 31, "compact_buffered": {"exact": false, "max_abs": 0.196533203125, "nrmse": 0.041913047432899475}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 32, "compact_buffered": {"exact": false, "max_abs": 0.185546875, "nrmse": 0.04713185504078865}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 33, "compact_buffered": {"exact": false, "max_abs": 0.203125, "nrmse": 0.04683183506131172}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 34, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.048468127846717834}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 35, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.05521465837955475}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 36, "compact_buffered": {"exact": false, "max_abs": 0.75, "nrmse": 0.06139735132455826}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 37, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.06409084796905518}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 38, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.06435645371675491}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 39, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.06465471535921097}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 40, "compact_buffered": {"exact": false, "max_abs": 0.30078125, "nrmse": 0.06364911049604416}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 41, "compact_buffered": {"exact": false, "max_abs": 0.34375, "nrmse": 0.06298506259918213}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 42, "compact_buffered": {"exact": false, "max_abs": 0.375, "nrmse": 0.06386778503656387}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 43, "compact_buffered": {"exact": false, "max_abs": 0.3828125, "nrmse": 0.062307246029376984}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 44, "compact_buffered": {"exact": false, "max_abs": 0.3828125, "nrmse": 0.062091805040836334}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 45, "compact_buffered": {"exact": false, "max_abs": 0.390625, "nrmse": 0.0625607892870903}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 46, "compact_buffered": {"exact": false, "max_abs": 0.484375, "nrmse": 0.06764218956232071}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 47, "compact_buffered": {"exact": false, "max_abs": 0.5625, "nrmse": 0.06414414942264557}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 48, "compact_buffered": {"exact": false, "max_abs": 0.625, "nrmse": 0.05916459113359451}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 49, "compact_buffered": {"exact": false, "max_abs": 0.625, "nrmse": 0.06010358780622482}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 50, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.054383981972932816}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 51, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.05116857960820198}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 52, "compact_buffered": {"exact": false, "max_abs": 2.0, "nrmse": 0.05183149501681328}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 53, "compact_buffered": {"exact": false, "max_abs": 2.0, "nrmse": 0.05302620679140091}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 54, "compact_buffered": {"exact": false, "max_abs": 2.5, "nrmse": 0.05345532298088074}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 55, "compact_buffered": {"exact": false, "max_abs": 2.0, "nrmse": 0.05414202809333801}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 56, "compact_buffered": {"exact": false, "max_abs": 2.5, "nrmse": 0.053871214389801025}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 57, "compact_buffered": {"exact": false, "max_abs": 2.0, "nrmse": 0.053397126495838165}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 58, "compact_buffered": {"exact": false, "max_abs": 3.25, "nrmse": 0.05374930053949356}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 59, "compact_buffered": {"exact": false, "max_abs": 0.890625, "nrmse": 0.05373493582010269}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 60, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.05437732860445976}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 61, "compact_buffered": {"exact": false, "max_abs": 1.015625, "nrmse": 0.05392136424779892}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 62, "compact_buffered": {"exact": false, "max_abs": 1.5, "nrmse": 0.05543329566717148}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 63, "compact_buffered": {"exact": false, "max_abs": 24.0, "nrmse": 0.07644850760698318}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}], "state_diff": []} -{"phase": "decode", "case": "gsm8k/test/783", "step": 8, "teacher_token": 7995, "compact_buffered": {"exact": false, "max_abs": 0.96875, "nrmse": 0.0280954297631979}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [279, 279, 279], "layer_diff": [{"layer": 4, "compact_buffered": {"exact": false, "max_abs": 0.00390625, "nrmse": 0.0014123674482107162}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 5, "compact_buffered": {"exact": false, "max_abs": 0.0078125, "nrmse": 0.002687155269086361}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 6, "compact_buffered": {"exact": false, "max_abs": 0.015625, "nrmse": 0.0039189038798213005}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 7, "compact_buffered": {"exact": false, "max_abs": 0.015625, "nrmse": 0.0045326631516218185}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 8, "compact_buffered": {"exact": false, "max_abs": 0.015625, "nrmse": 0.005504998844116926}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 9, "compact_buffered": {"exact": false, "max_abs": 0.01953125, "nrmse": 0.00645186472684145}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 10, "compact_buffered": {"exact": false, "max_abs": 0.0234375, "nrmse": 0.007249526679515839}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 11, "compact_buffered": {"exact": false, "max_abs": 0.03125, "nrmse": 0.008078763261437416}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 12, "compact_buffered": {"exact": false, "max_abs": 0.03125, "nrmse": 0.009017722681164742}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 13, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.011195847764611244}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 14, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.012570066377520561}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 15, "compact_buffered": {"exact": false, "max_abs": 0.041015625, "nrmse": 0.012215463444590569}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 16, "compact_buffered": {"exact": false, "max_abs": 0.04296875, "nrmse": 0.012607634998857975}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 17, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.013774109072983265}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 18, "compact_buffered": {"exact": false, "max_abs": 0.0625, "nrmse": 0.013490420766174793}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 19, "compact_buffered": {"exact": false, "max_abs": 0.0703125, "nrmse": 0.015281866304576397}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 20, "compact_buffered": {"exact": false, "max_abs": 0.07421875, "nrmse": 0.01678861305117607}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 21, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.018704194575548172}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 22, "compact_buffered": {"exact": false, "max_abs": 0.078125, "nrmse": 0.018336355686187744}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 23, "compact_buffered": {"exact": false, "max_abs": 0.125, "nrmse": 0.018827730789780617}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 24, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.018915150314569473}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 25, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.018623190000653267}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 26, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.01828119158744812}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 27, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.019717805087566376}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 28, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.020309854298830032}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 29, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.0206205565482378}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 30, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.0207266453653574}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 31, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.022723965346813202}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 32, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.023895731195807457}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 33, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.02383849211037159}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 34, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.02435234934091568}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 35, "compact_buffered": {"exact": false, "max_abs": 0.75, "nrmse": 0.02680220454931259}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 36, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.026654040440917015}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 37, "compact_buffered": {"exact": false, "max_abs": 0.1875, "nrmse": 0.026837453246116638}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 38, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.025885475799441338}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 39, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.02693502977490425}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 40, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.02689548023045063}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 41, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.026547374203801155}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 42, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.02548922412097454}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 43, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.02653210423886776}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 44, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.02832336165010929}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 45, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.029334306716918945}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 46, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.03024740144610405}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 47, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.03192620351910591}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 48, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.03329094126820564}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 49, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.035280898213386536}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 50, "compact_buffered": {"exact": false, "max_abs": 0.75, "nrmse": 0.038192812353372574}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 51, "compact_buffered": {"exact": false, "max_abs": 0.3515625, "nrmse": 0.04029233381152153}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 52, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.040362030267715454}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 53, "compact_buffered": {"exact": false, "max_abs": 0.4375, "nrmse": 0.04108942300081253}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 54, "compact_buffered": {"exact": false, "max_abs": 0.4375, "nrmse": 0.040411777794361115}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 55, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.04106684774160385}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 56, "compact_buffered": {"exact": false, "max_abs": 1.5, "nrmse": 0.04340307414531708}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 57, "compact_buffered": {"exact": false, "max_abs": 1.5, "nrmse": 0.042992159724235535}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 58, "compact_buffered": {"exact": false, "max_abs": 2.75, "nrmse": 0.04456396773457527}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 59, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.0415910929441452}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 60, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.04173950478434563}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 61, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.04124514386057854}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 62, "compact_buffered": {"exact": false, "max_abs": 1.5, "nrmse": 0.042077742516994476}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 63, "compact_buffered": {"exact": false, "max_abs": 2.0, "nrmse": 0.04699288308620453}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}], "state_diff": []} -{"phase": "decode", "case": "gsm8k/test/783", "step": 9, "teacher_token": 279, "compact_buffered": {"exact": false, "max_abs": 1.1875, "nrmse": 0.04069231450557709}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [1324, 1324, 1324], "layer_diff": [{"layer": 4, "compact_buffered": {"exact": false, "max_abs": 0.001953125, "nrmse": 0.00105656823143363}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 5, "compact_buffered": {"exact": false, "max_abs": 0.015625, "nrmse": 0.002257892396301031}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 6, "compact_buffered": {"exact": false, "max_abs": 0.015625, "nrmse": 0.00317277479916811}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 7, "compact_buffered": {"exact": false, "max_abs": 0.015625, "nrmse": 0.0037533382419496775}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 8, "compact_buffered": {"exact": false, "max_abs": 0.125, "nrmse": 0.0063282186165452}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 9, "compact_buffered": {"exact": false, "max_abs": 0.125, "nrmse": 0.006650182418525219}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 10, "compact_buffered": {"exact": false, "max_abs": 0.125, "nrmse": 0.006996923591941595}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 11, "compact_buffered": {"exact": false, "max_abs": 0.01171875, "nrmse": 0.005972105078399181}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 12, "compact_buffered": {"exact": false, "max_abs": 0.012939453125, "nrmse": 0.0063162799924612045}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 13, "compact_buffered": {"exact": false, "max_abs": 0.015625, "nrmse": 0.006803423631936312}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 14, "compact_buffered": {"exact": false, "max_abs": 0.0234375, "nrmse": 0.0076719019562006}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 15, "compact_buffered": {"exact": false, "max_abs": 0.0234375, "nrmse": 0.008012678474187851}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 16, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.010063087567687035}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 17, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.010551840998232365}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 18, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.011761755682528019}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 19, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.014583844691514969}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 20, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.01704796776175499}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 21, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.018130846321582794}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 22, "compact_buffered": {"exact": false, "max_abs": 0.05859375, "nrmse": 0.0175704937428236}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 23, "compact_buffered": {"exact": false, "max_abs": 0.05859375, "nrmse": 0.019649209454655647}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 24, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.020867832005023956}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 25, "compact_buffered": {"exact": false, "max_abs": 0.0625, "nrmse": 0.021352160722017288}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 26, "compact_buffered": {"exact": false, "max_abs": 0.07421875, "nrmse": 0.02063579298555851}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 27, "compact_buffered": {"exact": false, "max_abs": 0.078125, "nrmse": 0.023067239671945572}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 28, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.024556487798690796}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 29, "compact_buffered": {"exact": false, "max_abs": 0.080078125, "nrmse": 0.025732368230819702}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 30, "compact_buffered": {"exact": false, "max_abs": 0.103515625, "nrmse": 0.026768164709210396}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 31, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.03131698817014694}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 32, "compact_buffered": {"exact": false, "max_abs": 0.125, "nrmse": 0.03229052945971489}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 33, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.03312164172530174}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 34, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.035867687314748764}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 35, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.04047549515962601}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 36, "compact_buffered": {"exact": false, "max_abs": 0.208984375, "nrmse": 0.04519141465425491}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 37, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.04799601435661316}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 38, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.04743845388293266}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 39, "compact_buffered": {"exact": false, "max_abs": 0.265625, "nrmse": 0.04900607466697693}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 40, "compact_buffered": {"exact": false, "max_abs": 0.2734375, "nrmse": 0.050619907677173615}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 41, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.052636586129665375}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 42, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.05469748005270958}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 43, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.05567435920238495}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 44, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.05702560022473335}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 45, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.054352737963199615}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 46, "compact_buffered": {"exact": false, "max_abs": 1.5, "nrmse": 0.055747952312231064}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 47, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.05368177965283394}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 48, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.05018778145313263}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 49, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.05095245689153671}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 50, "compact_buffered": {"exact": false, "max_abs": 1.5, "nrmse": 0.05053844675421715}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 51, "compact_buffered": {"exact": false, "max_abs": 1.5, "nrmse": 0.04812449589371681}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 52, "compact_buffered": {"exact": false, "max_abs": 1.5, "nrmse": 0.047550756484270096}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 53, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.04853009060025215}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 54, "compact_buffered": {"exact": false, "max_abs": 0.5078125, "nrmse": 0.047573380172252655}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 55, "compact_buffered": {"exact": false, "max_abs": 0.5625, "nrmse": 0.049200136214494705}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 56, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.050169408321380615}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 57, "compact_buffered": {"exact": false, "max_abs": 1.5, "nrmse": 0.04957564175128937}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 58, "compact_buffered": {"exact": false, "max_abs": 3.5, "nrmse": 0.050120674073696136}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 59, "compact_buffered": {"exact": false, "max_abs": 5.25, "nrmse": 0.055619414895772934}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 60, "compact_buffered": {"exact": false, "max_abs": 3.0, "nrmse": 0.04789874330163002}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 61, "compact_buffered": {"exact": false, "max_abs": 3.5, "nrmse": 0.04981551691889763}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 62, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.045381393283605576}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 63, "compact_buffered": {"exact": false, "max_abs": 4.0, "nrmse": 0.05474939942359924}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}], "state_diff": []} -{"phase": "decode", "case": "gsm8k/test/783", "step": 10, "teacher_token": 1324, "compact_buffered": {"exact": false, "max_abs": 0.875, "nrmse": 0.04968669265508652}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [314, 314, 314], "layer_diff": [{"layer": 4, "compact_buffered": {"exact": false, "max_abs": 0.00390625, "nrmse": 0.0013934295857325196}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 5, "compact_buffered": {"exact": false, "max_abs": 0.00390625, "nrmse": 0.002621519146487117}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 6, "compact_buffered": {"exact": false, "max_abs": 0.125, "nrmse": 0.005485396832227707}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 7, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.008560762740671635}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 8, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.00869427528232336}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 9, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.00946188997477293}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 10, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.01001245342195034}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 11, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.00970485433936119}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 12, "compact_buffered": {"exact": false, "max_abs": 0.03125, "nrmse": 0.008575868792831898}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 13, "compact_buffered": {"exact": false, "max_abs": 0.03125, "nrmse": 0.009101216681301594}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 14, "compact_buffered": {"exact": false, "max_abs": 0.03125, "nrmse": 0.009513943456113338}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 15, "compact_buffered": {"exact": false, "max_abs": 0.03125, "nrmse": 0.009955819696187973}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 16, "compact_buffered": {"exact": false, "max_abs": 0.033203125, "nrmse": 0.010497095994651318}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 17, "compact_buffered": {"exact": false, "max_abs": 0.0625, "nrmse": 0.011117560788989067}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 18, "compact_buffered": {"exact": false, "max_abs": 0.0625, "nrmse": 0.012512709945440292}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 19, "compact_buffered": {"exact": false, "max_abs": 0.078125, "nrmse": 0.01466429978609085}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 20, "compact_buffered": {"exact": false, "max_abs": 0.078125, "nrmse": 0.016453934833407402}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 21, "compact_buffered": {"exact": false, "max_abs": 0.0625, "nrmse": 0.017072515562176704}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 22, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.016936544328927994}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 23, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.01812702603638172}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 24, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.019999966025352478}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 25, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.020685408264398575}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 26, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.019750606268644333}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 27, "compact_buffered": {"exact": false, "max_abs": 0.75, "nrmse": 0.0231346283107996}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 28, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.02370394393801689}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 29, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.0246123019605875}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 30, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.024309305474162102}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 31, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.027158472687005997}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 32, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.02869129367172718}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 33, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.02842240035533905}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 34, "compact_buffered": {"exact": false, "max_abs": 0.128173828125, "nrmse": 0.027662664651870728}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 35, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.03015093132853508}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 36, "compact_buffered": {"exact": false, "max_abs": 0.138671875, "nrmse": 0.029140913859009743}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 37, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.029620442539453506}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 38, "compact_buffered": {"exact": false, "max_abs": 0.146484375, "nrmse": 0.027985146269202232}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 39, "compact_buffered": {"exact": false, "max_abs": 0.1533203125, "nrmse": 0.029470933601260185}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 40, "compact_buffered": {"exact": false, "max_abs": 0.169921875, "nrmse": 0.02882612869143486}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 41, "compact_buffered": {"exact": false, "max_abs": 0.177734375, "nrmse": 0.028117842972278595}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 42, "compact_buffered": {"exact": false, "max_abs": 0.1875, "nrmse": 0.02792433649301529}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 43, "compact_buffered": {"exact": false, "max_abs": 0.171875, "nrmse": 0.0292916651815176}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 44, "compact_buffered": {"exact": false, "max_abs": 0.1875, "nrmse": 0.030621608719229698}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 45, "compact_buffered": {"exact": false, "max_abs": 0.1962890625, "nrmse": 0.03172212094068527}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 46, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.03226688504219055}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 47, "compact_buffered": {"exact": false, "max_abs": 0.21875, "nrmse": 0.03392559289932251}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 48, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.03761869668960571}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 49, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.039973463863134384}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 50, "compact_buffered": {"exact": false, "max_abs": 0.625, "nrmse": 0.04147641733288765}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 51, "compact_buffered": {"exact": false, "max_abs": 0.625, "nrmse": 0.04523998871445656}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 52, "compact_buffered": {"exact": false, "max_abs": 0.75, "nrmse": 0.04651298001408577}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 53, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.04791542515158653}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 54, "compact_buffered": {"exact": false, "max_abs": 2.375, "nrmse": 0.04880497232079506}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 55, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.047592468559741974}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 56, "compact_buffered": {"exact": false, "max_abs": 1.375, "nrmse": 0.04848909005522728}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 57, "compact_buffered": {"exact": false, "max_abs": 1.25, "nrmse": 0.047973938286304474}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 58, "compact_buffered": {"exact": false, "max_abs": 2.5, "nrmse": 0.044373705983161926}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 59, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.03901958838105202}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 60, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.04221688210964203}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 61, "compact_buffered": {"exact": false, "max_abs": 1.125, "nrmse": 0.04068310931324959}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 62, "compact_buffered": {"exact": false, "max_abs": 2.0, "nrmse": 0.04674163833260536}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 63, "compact_buffered": {"exact": false, "max_abs": 12.0, "nrmse": 0.055913057178258896}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}], "state_diff": []} -{"phase": "decode", "case": "gsm8k/test/783", "step": 11, "teacher_token": 314, "compact_buffered": {"exact": false, "max_abs": 0.75, "nrmse": 0.029374774545431137}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [4022, 4022, 4022], "layer_diff": [{"layer": 4, "compact_buffered": {"exact": false, "max_abs": 0.001953125, "nrmse": 0.0011303412029519677}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 5, "compact_buffered": {"exact": false, "max_abs": 0.00390625, "nrmse": 0.0024327810388058424}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 6, "compact_buffered": {"exact": false, "max_abs": 0.015625, "nrmse": 0.0036020195111632347}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 7, "compact_buffered": {"exact": false, "max_abs": 0.015625, "nrmse": 0.004066918045282364}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 8, "compact_buffered": {"exact": false, "max_abs": 0.015625, "nrmse": 0.004466308280825615}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 9, "compact_buffered": {"exact": false, "max_abs": 0.015625, "nrmse": 0.005147476680576801}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 10, "compact_buffered": {"exact": false, "max_abs": 0.015625, "nrmse": 0.006079676561057568}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 11, "compact_buffered": {"exact": false, "max_abs": 0.03125, "nrmse": 0.00651041604578495}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 12, "compact_buffered": {"exact": false, "max_abs": 0.03125, "nrmse": 0.006910101510584354}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 13, "compact_buffered": {"exact": false, "max_abs": 0.03125, "nrmse": 0.007285292726010084}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 14, "compact_buffered": {"exact": false, "max_abs": 0.03125, "nrmse": 0.007628529332578182}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 15, "compact_buffered": {"exact": false, "max_abs": 0.03125, "nrmse": 0.008114292286336422}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 16, "compact_buffered": {"exact": false, "max_abs": 0.0234375, "nrmse": 0.008730215951800346}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 17, "compact_buffered": {"exact": false, "max_abs": 0.03125, "nrmse": 0.009634681046009064}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 18, "compact_buffered": {"exact": false, "max_abs": 0.0625, "nrmse": 0.0112794553861022}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 19, "compact_buffered": {"exact": false, "max_abs": 0.0625, "nrmse": 0.015065873041749}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 20, "compact_buffered": {"exact": false, "max_abs": 0.0625, "nrmse": 0.01683894917368889}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 21, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.017628079280257225}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 22, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.01980810984969139}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 23, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.02180766873061657}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 24, "compact_buffered": {"exact": false, "max_abs": 0.75, "nrmse": 0.025441374629735947}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 25, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.02550249919295311}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 26, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.02487722598016262}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 27, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.029965493828058243}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 28, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.03444591164588928}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 29, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.040054529905319214}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 30, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.04118270054459572}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 31, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.05088239535689354}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 32, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.05535091087222099}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 33, "compact_buffered": {"exact": false, "max_abs": 0.191162109375, "nrmse": 0.05248531326651573}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 34, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.05255214497447014}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 35, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.05546848103404045}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 36, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.058508072048425674}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 37, "compact_buffered": {"exact": false, "max_abs": 0.2578125, "nrmse": 0.05733279511332512}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 38, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.05457320064306259}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 39, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.057266782969236374}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 40, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.056330807507038116}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 41, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.05631415173411369}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 42, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.05675153434276581}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 43, "compact_buffered": {"exact": false, "max_abs": 1.5, "nrmse": 0.05723407864570618}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 44, "compact_buffered": {"exact": false, "max_abs": 1.5, "nrmse": 0.057046882808208466}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 45, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.05409333109855652}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 46, "compact_buffered": {"exact": false, "max_abs": 1.5, "nrmse": 0.05686958134174347}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 47, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.05598307028412819}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 48, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.05406636372208595}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 49, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.052184753119945526}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 50, "compact_buffered": {"exact": false, "max_abs": 0.75, "nrmse": 0.053773537278175354}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 51, "compact_buffered": {"exact": false, "max_abs": 0.625, "nrmse": 0.050821609795093536}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 52, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.05004865303635597}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 53, "compact_buffered": {"exact": false, "max_abs": 1.75, "nrmse": 0.052014466375112534}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 54, "compact_buffered": {"exact": false, "max_abs": 2.5, "nrmse": 0.04914158210158348}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 55, "compact_buffered": {"exact": false, "max_abs": 2.0, "nrmse": 0.049469247460365295}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 56, "compact_buffered": {"exact": false, "max_abs": 1.5, "nrmse": 0.04818030074238777}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 57, "compact_buffered": {"exact": false, "max_abs": 2.0, "nrmse": 0.04672151803970337}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 58, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.041887637227773666}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 59, "compact_buffered": {"exact": false, "max_abs": 2.5, "nrmse": 0.046330198645591736}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 60, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.04098816588521004}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 61, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.04269256070256233}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 62, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.040751319378614426}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 63, "compact_buffered": {"exact": false, "max_abs": 4.0, "nrmse": 0.04097844660282135}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}], "state_diff": []} -{"phase": "decode", "case": "gsm8k/test/783", "step": 12, "teacher_token": 4022, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.023178115487098694}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [7397, 7397, 7397], "layer_diff": [{"layer": 4, "compact_buffered": {"exact": false, "max_abs": 0.0029296875, "nrmse": 0.0014819870702922344}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 5, "compact_buffered": {"exact": false, "max_abs": 0.0078125, "nrmse": 0.002416919684037566}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 6, "compact_buffered": {"exact": false, "max_abs": 0.01171875, "nrmse": 0.003926156088709831}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 7, "compact_buffered": {"exact": false, "max_abs": 0.015625, "nrmse": 0.00529476348310709}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 8, "compact_buffered": {"exact": false, "max_abs": 0.015625, "nrmse": 0.006482032127678394}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 9, "compact_buffered": {"exact": false, "max_abs": 0.017578125, "nrmse": 0.007297571748495102}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 10, "compact_buffered": {"exact": false, "max_abs": 0.03125, "nrmse": 0.00791252963244915}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 11, "compact_buffered": {"exact": false, "max_abs": 0.03125, "nrmse": 0.00833815336227417}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 12, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.009739087894558907}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 13, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.010075084865093231}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 14, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.01009262353181839}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 15, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.01120231207460165}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 16, "compact_buffered": {"exact": false, "max_abs": 0.04296875, "nrmse": 0.011135725304484367}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 17, "compact_buffered": {"exact": false, "max_abs": 0.04443359375, "nrmse": 0.011625763028860092}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 18, "compact_buffered": {"exact": false, "max_abs": 0.05078125, "nrmse": 0.012628014199435711}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 19, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.015553480945527554}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 20, "compact_buffered": {"exact": false, "max_abs": 0.0625, "nrmse": 0.015840906649827957}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 21, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.016308216378092766}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 22, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.015878252685070038}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 23, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.01811651512980461}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 24, "compact_buffered": {"exact": false, "max_abs": 0.078125, "nrmse": 0.019261164590716362}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 25, "compact_buffered": {"exact": false, "max_abs": 0.08203125, "nrmse": 0.01866801455616951}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 26, "compact_buffered": {"exact": false, "max_abs": 0.0703125, "nrmse": 0.017529157921671867}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 27, "compact_buffered": {"exact": false, "max_abs": 0.0859375, "nrmse": 0.020766567438840866}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 28, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.022927559912204742}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 29, "compact_buffered": {"exact": false, "max_abs": 0.0859375, "nrmse": 0.02332173101603985}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 30, "compact_buffered": {"exact": false, "max_abs": 0.11328125, "nrmse": 0.022972678765654564}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 31, "compact_buffered": {"exact": false, "max_abs": 0.15625, "nrmse": 0.02815951779484749}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 32, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.03536241501569748}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 33, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.034467343240976334}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 34, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.03125406801700592}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 35, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.0357942134141922}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 36, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.034732382744550705}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 37, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.034418053925037384}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 38, "compact_buffered": {"exact": false, "max_abs": 0.1875, "nrmse": 0.02976539358496666}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 39, "compact_buffered": {"exact": false, "max_abs": 0.1875, "nrmse": 0.03192266449332237}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 40, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.031299009919166565}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 41, "compact_buffered": {"exact": false, "max_abs": 0.1875, "nrmse": 0.03064430132508278}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 42, "compact_buffered": {"exact": false, "max_abs": 0.1875, "nrmse": 0.02939450927078724}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 43, "compact_buffered": {"exact": false, "max_abs": 0.1875, "nrmse": 0.031239910051226616}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 44, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.032399147748947144}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 45, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.03270171955227852}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 46, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.03422107920050621}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 47, "compact_buffered": {"exact": false, "max_abs": 0.3125, "nrmse": 0.03617871552705765}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 48, "compact_buffered": {"exact": false, "max_abs": 0.265625, "nrmse": 0.03715778514742851}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 49, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.04035096615552902}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 50, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.042389921844005585}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 51, "compact_buffered": {"exact": false, "max_abs": 1.25, "nrmse": 0.04463879019021988}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 52, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.042811669409275055}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 53, "compact_buffered": {"exact": false, "max_abs": 1.25, "nrmse": 0.04372306913137436}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 54, "compact_buffered": {"exact": false, "max_abs": 1.75, "nrmse": 0.04421500116586685}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 55, "compact_buffered": {"exact": false, "max_abs": 1.25, "nrmse": 0.0438406728208065}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 56, "compact_buffered": {"exact": false, "max_abs": 1.25, "nrmse": 0.04412072151899338}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 57, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.04362652450799942}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 58, "compact_buffered": {"exact": false, "max_abs": 4.0, "nrmse": 0.04782693833112717}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 59, "compact_buffered": {"exact": false, "max_abs": 2.0, "nrmse": 0.04123862087726593}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 60, "compact_buffered": {"exact": false, "max_abs": 2.5, "nrmse": 0.040963251143693924}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 61, "compact_buffered": {"exact": false, "max_abs": 2.0, "nrmse": 0.03842664510011673}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 62, "compact_buffered": {"exact": false, "max_abs": 2.0, "nrmse": 0.04068988189101219}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 63, "compact_buffered": {"exact": false, "max_abs": 6.0, "nrmse": 0.04293791577219963}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}], "state_diff": []} -{"phase": "decode", "case": "gsm8k/test/783", "step": 13, "teacher_token": 7397, "compact_buffered": {"exact": false, "max_abs": 0.53125, "nrmse": 0.024438217282295227}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [13853, 13853, 13853], "layer_diff": [{"layer": 4, "compact_buffered": {"exact": false, "max_abs": 0.001953125, "nrmse": 0.0011653956025838852}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 5, "compact_buffered": {"exact": false, "max_abs": 0.0078125, "nrmse": 0.0029676274862140417}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 6, "compact_buffered": {"exact": false, "max_abs": 0.015625, "nrmse": 0.005186103284358978}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 7, "compact_buffered": {"exact": false, "max_abs": 0.015625, "nrmse": 0.006493800785392523}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 8, "compact_buffered": {"exact": false, "max_abs": 0.015625, "nrmse": 0.0074600642547011375}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 9, "compact_buffered": {"exact": false, "max_abs": 0.0234375, "nrmse": 0.008334903046488762}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 10, "compact_buffered": {"exact": false, "max_abs": 0.03125, "nrmse": 0.008948485367000103}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 11, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.010218795388936996}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 12, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.010241564363241196}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 13, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.010236929170787334}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 14, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.012513198889791965}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 15, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.012516784481704235}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 16, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.011084422469139099}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 17, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.01188205648213625}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 18, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.013124201446771622}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 19, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.015724318102002144}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 20, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.016249677166342735}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 21, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.01657100021839142}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 22, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.01623792015016079}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 23, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.017686160281300545}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 24, "compact_buffered": {"exact": false, "max_abs": 0.09765625, "nrmse": 0.01802895963191986}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 25, "compact_buffered": {"exact": false, "max_abs": 0.0859375, "nrmse": 0.018019581213593483}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 26, "compact_buffered": {"exact": false, "max_abs": 0.078125, "nrmse": 0.017453564330935478}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 27, "compact_buffered": {"exact": false, "max_abs": 0.08203125, "nrmse": 0.019026275724172592}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 28, "compact_buffered": {"exact": false, "max_abs": 0.09765625, "nrmse": 0.020550742745399475}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 29, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.021664490923285484}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 30, "compact_buffered": {"exact": false, "max_abs": 0.1015625, "nrmse": 0.02174435369670391}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 31, "compact_buffered": {"exact": false, "max_abs": 0.095703125, "nrmse": 0.02295711264014244}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 32, "compact_buffered": {"exact": false, "max_abs": 0.125, "nrmse": 0.024634070694446564}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 33, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.026367617771029472}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 34, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.02757667750120163}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 35, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.029724575579166412}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 36, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.030008502304553986}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 37, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.032021790742874146}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 38, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.029965173453092575}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 39, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.030245371162891388}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 40, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.02990644983947277}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 41, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.029173580929636955}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 42, "compact_buffered": {"exact": false, "max_abs": 0.1875, "nrmse": 0.027494505047798157}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 43, "compact_buffered": {"exact": false, "max_abs": 0.21875, "nrmse": 0.02712026797235012}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 44, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.027758639305830002}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 45, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.02827388420701027}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 46, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.029560618102550507}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 47, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.032087914645671844}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 48, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.034862421452999115}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 49, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.03753003105521202}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 50, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.039910994470119476}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 51, "compact_buffered": {"exact": false, "max_abs": 2.0, "nrmse": 0.04570294916629791}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 52, "compact_buffered": {"exact": false, "max_abs": 1.25, "nrmse": 0.04421960189938545}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 53, "compact_buffered": {"exact": false, "max_abs": 1.5, "nrmse": 0.04554176330566406}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 54, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.045045603066682816}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 55, "compact_buffered": {"exact": false, "max_abs": 0.5625, "nrmse": 0.04481031000614166}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 56, "compact_buffered": {"exact": false, "max_abs": 0.6875, "nrmse": 0.04537743702530861}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 57, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.046002671122550964}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 58, "compact_buffered": {"exact": false, "max_abs": 2.25, "nrmse": 0.04906235262751579}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 59, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.04593077301979065}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 60, "compact_buffered": {"exact": false, "max_abs": 2.0, "nrmse": 0.04238634556531906}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 61, "compact_buffered": {"exact": false, "max_abs": 2.0, "nrmse": 0.0419137179851532}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 62, "compact_buffered": {"exact": false, "max_abs": 2.0, "nrmse": 0.041266415268182755}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 63, "compact_buffered": {"exact": false, "max_abs": 9.0, "nrmse": 0.04803977161645889}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}], "state_diff": []} -{"phase": "decode", "case": "gsm8k/test/783", "step": 14, "teacher_token": 13853, "compact_buffered": {"exact": false, "max_abs": 0.591796875, "nrmse": 0.02568630874156952}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [10471, 10471, 10471], "layer_diff": [{"layer": 4, "compact_buffered": {"exact": false, "max_abs": 0.00390625, "nrmse": 0.0017262034816667438}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 5, "compact_buffered": {"exact": false, "max_abs": 0.015625, "nrmse": 0.004255622159689665}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 6, "compact_buffered": {"exact": false, "max_abs": 0.015625, "nrmse": 0.005797350313514471}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 7, "compact_buffered": {"exact": false, "max_abs": 0.015625, "nrmse": 0.006381942890584469}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 8, "compact_buffered": {"exact": false, "max_abs": 0.015625, "nrmse": 0.006915087811648846}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 9, "compact_buffered": {"exact": false, "max_abs": 0.01953125, "nrmse": 0.007839959114789963}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 10, "compact_buffered": {"exact": false, "max_abs": 0.0234375, "nrmse": 0.00903934333473444}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 11, "compact_buffered": {"exact": false, "max_abs": 0.03125, "nrmse": 0.00929130706936121}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 12, "compact_buffered": {"exact": false, "max_abs": 0.03125, "nrmse": 0.009786130860447884}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 13, "compact_buffered": {"exact": false, "max_abs": 0.03125, "nrmse": 0.009845618158578873}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 14, "compact_buffered": {"exact": false, "max_abs": 0.0390625, "nrmse": 0.010296978987753391}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 15, "compact_buffered": {"exact": false, "max_abs": 0.0625, "nrmse": 0.010555327869951725}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 16, "compact_buffered": {"exact": false, "max_abs": 0.0625, "nrmse": 0.0108405202627182}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 17, "compact_buffered": {"exact": false, "max_abs": 0.0625, "nrmse": 0.011715826578438282}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 18, "compact_buffered": {"exact": false, "max_abs": 0.078125, "nrmse": 0.013387253507971764}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 19, "compact_buffered": {"exact": false, "max_abs": 0.078125, "nrmse": 0.015018514357507229}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 20, "compact_buffered": {"exact": false, "max_abs": 0.078125, "nrmse": 0.016300570219755173}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 21, "compact_buffered": {"exact": false, "max_abs": 0.078125, "nrmse": 0.016179777681827545}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 22, "compact_buffered": {"exact": false, "max_abs": 0.078125, "nrmse": 0.015566335991024971}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 23, "compact_buffered": {"exact": false, "max_abs": 0.125, "nrmse": 0.016324101015925407}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 24, "compact_buffered": {"exact": false, "max_abs": 0.125, "nrmse": 0.01656726747751236}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 25, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.01759253814816475}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 26, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.01747935824096203}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 27, "compact_buffered": {"exact": false, "max_abs": 0.09375, "nrmse": 0.017723100259900093}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 28, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.01931561529636383}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 29, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.019665099680423737}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 30, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.019927628338336945}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 31, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.021268324926495552}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 32, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.02547619864344597}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 33, "compact_buffered": {"exact": false, "max_abs": 0.75, "nrmse": 0.024380847811698914}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 34, "compact_buffered": {"exact": false, "max_abs": 0.75, "nrmse": 0.024954816326498985}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 35, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.028751276433467865}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 36, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.026619210839271545}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 37, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.02611815743148327}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 38, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.024077845737338066}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 39, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.02707800827920437}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 40, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.02711658552289009}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 41, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.027146130800247192}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 42, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.026644764468073845}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 43, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.027279797941446304}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 44, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.027916554361581802}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 45, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.02856079861521721}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 46, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.03048820234835148}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 47, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.032141558825969696}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 48, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.034684933722019196}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 49, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.03723951429128647}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 50, "compact_buffered": {"exact": false, "max_abs": 0.75, "nrmse": 0.04048467054963112}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 51, "compact_buffered": {"exact": false, "max_abs": 0.75, "nrmse": 0.04565417766571045}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 52, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.04602130502462387}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 53, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.04777124896645546}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 54, "compact_buffered": {"exact": false, "max_abs": 1.5, "nrmse": 0.049740538001060486}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 55, "compact_buffered": {"exact": false, "max_abs": 1.5, "nrmse": 0.0491207055747509}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 56, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.0483270138502121}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 57, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.0473950020968914}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 58, "compact_buffered": {"exact": false, "max_abs": 1.75, "nrmse": 0.05037911608815193}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 59, "compact_buffered": {"exact": false, "max_abs": 0.8125, "nrmse": 0.04481876641511917}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 60, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.042147859930992126}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 61, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.04177042469382286}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 62, "compact_buffered": {"exact": false, "max_abs": 1.25, "nrmse": 0.04317623749375343}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 63, "compact_buffered": {"exact": false, "max_abs": 5.0, "nrmse": 0.048680152744054794}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}], "state_diff": []} -{"phase": "decode", "case": "gsm8k/test/783", "step": 15, "teacher_token": 10471, "compact_buffered": {"exact": false, "max_abs": 0.875, "nrmse": 0.03911370038986206}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [3018, 3018, 3018], "layer_diff": [{"layer": 4, "compact_buffered": {"exact": false, "max_abs": 0.00390625, "nrmse": 0.0012286490527912974}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 5, "compact_buffered": {"exact": false, "max_abs": 0.0078125, "nrmse": 0.0033882062416523695}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 6, "compact_buffered": {"exact": false, "max_abs": 0.015625, "nrmse": 0.005705875810235739}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 7, "compact_buffered": {"exact": false, "max_abs": 0.015625, "nrmse": 0.006364049855619669}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 8, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.009166628122329712}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 9, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.010291180573403835}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 10, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.01105175819247961}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 11, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.011329825036227703}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 12, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.011632080189883709}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 13, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.011839007027447224}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 14, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.012087204493582249}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 15, "compact_buffered": {"exact": false, "max_abs": 0.041015625, "nrmse": 0.012112760916352272}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 16, "compact_buffered": {"exact": false, "max_abs": 0.046875, "nrmse": 0.012411167845129967}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 17, "compact_buffered": {"exact": false, "max_abs": 0.05078125, "nrmse": 0.012977083213627338}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 18, "compact_buffered": {"exact": false, "max_abs": 0.064453125, "nrmse": 0.013730837032198906}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 19, "compact_buffered": {"exact": false, "max_abs": 0.078125, "nrmse": 0.01658768206834793}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 20, "compact_buffered": {"exact": false, "max_abs": 0.078125, "nrmse": 0.01803090050816536}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 21, "compact_buffered": {"exact": false, "max_abs": 0.0703125, "nrmse": 0.018002783879637718}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 22, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.01896592788398266}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 23, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.02090461179614067}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 24, "compact_buffered": {"exact": false, "max_abs": 0.75, "nrmse": 0.023335136473178864}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 25, "compact_buffered": {"exact": false, "max_abs": 0.75, "nrmse": 0.023163218051195145}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 26, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.02077263407409191}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 27, "compact_buffered": {"exact": false, "max_abs": 0.109375, "nrmse": 0.022804295644164085}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 28, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.025164062157273293}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 29, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.026497378945350647}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 30, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.024617314338684082}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 31, "compact_buffered": {"exact": false, "max_abs": 0.140625, "nrmse": 0.026828018948435783}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 32, "compact_buffered": {"exact": false, "max_abs": 0.1484375, "nrmse": 0.02916260063648224}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 33, "compact_buffered": {"exact": false, "max_abs": 0.15771484375, "nrmse": 0.02905306965112686}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 34, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.029402391985058784}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 35, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.03202325850725174}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 36, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.03147295117378235}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 37, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.030700786039233208}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 38, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.02791164442896843}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 39, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.028599219396710396}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 40, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.030222339555621147}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 41, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.029376186430454254}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 42, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.02828907035291195}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 43, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.029564106836915016}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 44, "compact_buffered": {"exact": false, "max_abs": 0.1875, "nrmse": 0.028498636558651924}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 45, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.029222587123513222}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 46, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.03218436986207962}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 47, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.03164277970790863}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 48, "compact_buffered": {"exact": false, "max_abs": 0.375, "nrmse": 0.035890739411115646}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 49, "compact_buffered": {"exact": false, "max_abs": 0.375, "nrmse": 0.04173807427287102}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 50, "compact_buffered": {"exact": false, "max_abs": 0.75, "nrmse": 0.04780912026762962}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 51, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.052482374012470245}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 52, "compact_buffered": {"exact": false, "max_abs": 1.5, "nrmse": 0.053916770964860916}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 53, "compact_buffered": {"exact": false, "max_abs": 1.625, "nrmse": 0.05531158670783043}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 54, "compact_buffered": {"exact": false, "max_abs": 1.6875, "nrmse": 0.05589951574802399}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 55, "compact_buffered": {"exact": false, "max_abs": 1.75, "nrmse": 0.05706340819597244}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 56, "compact_buffered": {"exact": false, "max_abs": 2.125, "nrmse": 0.05849846452474594}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 57, "compact_buffered": {"exact": false, "max_abs": 2.375, "nrmse": 0.05892575904726982}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 58, "compact_buffered": {"exact": false, "max_abs": 2.125, "nrmse": 0.0603349469602108}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 59, "compact_buffered": {"exact": false, "max_abs": 1.0625, "nrmse": 0.05594661459326744}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 60, "compact_buffered": {"exact": false, "max_abs": 1.5, "nrmse": 0.05573271960020065}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 61, "compact_buffered": {"exact": false, "max_abs": 1.1875, "nrmse": 0.0548352412879467}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 62, "compact_buffered": {"exact": false, "max_abs": 2.0, "nrmse": 0.0555439330637455}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 63, "compact_buffered": {"exact": false, "max_abs": 9.0, "nrmse": 0.0620729885995388}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}], "state_diff": []} -{"phase": "prefill", "case": "gsm8k/test/1", "prompt_tokens": 70, "capacity": 1024, "prompt_sha256": "78d335abe81363d4bb48e5aa256c120344c2ceadbcf8739afe4ec3923c747238", "compact_buffered": {"exact": false, "max_abs": 2.42578125, "nrmse": 0.029481491073966026}, "buffered_repeated": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [1206, 1206, 1206], "layer_diff": [{"layer": 3, "exact": false, "max_abs": 3.0517578125e-05, "nrmse": 1.2318825781676424e-07}, {"layer": 35, "exact": false, "max_abs": 0.00390625, "nrmse": 5.422154572443105e-06}, {"layer": 36, "exact": false, "max_abs": 0.00390625, "nrmse": 5.482363121700473e-06}, {"layer": 37, "exact": false, "max_abs": 0.0078125, "nrmse": 1.0673227734514512e-05}, {"layer": 38, "exact": false, "max_abs": 0.0078125, "nrmse": 1.0186482541030273e-05}, {"layer": 39, "exact": false, "max_abs": 0.0078125, "nrmse": 1.0453527465870138e-05}, {"layer": 40, "exact": false, "max_abs": 0.0625, "nrmse": 0.00033488470944575965}, {"layer": 41, "exact": false, "max_abs": 0.25, "nrmse": 0.0036053527146577835}, {"layer": 42, "exact": false, "max_abs": 0.5, "nrmse": 0.006831399165093899}, {"layer": 43, "exact": false, "max_abs": 0.5, "nrmse": 0.010408325120806694}, {"layer": 44, "exact": false, "max_abs": 0.5, "nrmse": 0.012871676124632359}, {"layer": 45, "exact": false, "max_abs": 1.0, "nrmse": 0.014820152893662453}, {"layer": 46, "exact": false, "max_abs": 1.0, "nrmse": 0.015897709876298904}, {"layer": 47, "exact": false, "max_abs": 1.0, "nrmse": 0.019001035019755363}, {"layer": 48, "exact": false, "max_abs": 1.6328125, "nrmse": 0.021166939288377762}, {"layer": 49, "exact": false, "max_abs": 2.046875, "nrmse": 0.022870704531669617}, {"layer": 50, "exact": false, "max_abs": 1.1875, "nrmse": 0.023309091106057167}, {"layer": 51, "exact": false, "max_abs": 1.5, "nrmse": 0.027060123160481453}, {"layer": 52, "exact": false, "max_abs": 1.140625, "nrmse": 0.02900853008031845}, {"layer": 53, "exact": false, "max_abs": 2.125, "nrmse": 0.03098701871931553}, {"layer": 54, "exact": false, "max_abs": 2.5, "nrmse": 0.026523586362600327}, {"layer": 55, "exact": false, "max_abs": 2.75, "nrmse": 0.03092687949538231}, {"layer": 56, "exact": false, "max_abs": 3.0, "nrmse": 0.031377729028463364}, {"layer": 57, "exact": false, "max_abs": 3.25, "nrmse": 0.03162383660674095}, {"layer": 58, "exact": false, "max_abs": 4.5, "nrmse": 0.027109256014227867}, {"layer": 59, "exact": false, "max_abs": 6.25, "nrmse": 0.034615010023117065}, {"layer": 60, "exact": false, "max_abs": 4.0, "nrmse": 0.03464360907673836}, {"layer": 61, "exact": false, "max_abs": 5.75, "nrmse": 0.03482685983181}, {"layer": 62, "exact": false, "max_abs": 6.0, "nrmse": 0.035836026072502136}, {"layer": 63, "exact": false, "max_abs": 12.0, "nrmse": 0.03712904453277588}], "state_diff": [{"layer": 40, "state": "recurrent", "exact": false, "max_abs": 2.775341272354126e-07, "nrmse": 7.024756598639215e-08}, {"layer": 41, "state": "recurrent", "exact": false, "max_abs": 0.00045558810234069824, "nrmse": 0.001104617607779801}, {"layer": 42, "state": "conv", "exact": false, "max_abs": 0.125, "nrmse": 0.020563345402479172}, {"layer": 42, "state": "recurrent", "exact": false, "max_abs": 0.007800176739692688, "nrmse": 0.016103915870189667}, {"layer": 43, "state": "key", "exact": false, "max_abs": 0.2265625, "nrmse": 0.011978358030319214}, {"layer": 43, "state": "value", "exact": false, "max_abs": 0.21875, "nrmse": 0.013276536948978901}, {"layer": 44, "state": "conv", "exact": false, "max_abs": 0.28125, "nrmse": 0.04075804725289345}, {"layer": 44, "state": "recurrent", "exact": false, "max_abs": 0.007756220176815987, "nrmse": 0.034294795244932175}, {"layer": 45, "state": "conv", "exact": false, "max_abs": 0.486328125, "nrmse": 0.05160902440547943}, {"layer": 45, "state": "recurrent", "exact": false, "max_abs": 0.011023305356502533, "nrmse": 0.0492597296833992}, {"layer": 46, "state": "conv", "exact": false, "max_abs": 0.4609375, "nrmse": 0.057862069457769394}, {"layer": 46, "state": "recurrent", "exact": false, "max_abs": 0.016843989491462708, "nrmse": 0.052261706441640854}, {"layer": 47, "state": "key", "exact": false, "max_abs": 0.435546875, "nrmse": 0.018985597416758537}, {"layer": 47, "state": "value", "exact": false, "max_abs": 0.890625, "nrmse": 0.024606995284557343}, {"layer": 48, "state": "conv", "exact": false, "max_abs": 0.5, "nrmse": 0.06813570857048035}, {"layer": 48, "state": "recurrent", "exact": false, "max_abs": 0.01282241940498352, "nrmse": 0.06380875408649445}, {"layer": 49, "state": "conv", "exact": false, "max_abs": 0.65625, "nrmse": 0.07430944591760635}, {"layer": 49, "state": "recurrent", "exact": false, "max_abs": 0.018826209008693695, "nrmse": 0.06936171650886536}, {"layer": 50, "state": "conv", "exact": false, "max_abs": 0.935791015625, "nrmse": 0.07415544241666794}, {"layer": 50, "state": "recurrent", "exact": false, "max_abs": 0.013631787151098251, "nrmse": 0.06511436402797699}, {"layer": 51, "state": "key", "exact": false, "max_abs": 0.484375, "nrmse": 0.021273180842399597}, {"layer": 51, "state": "value", "exact": false, "max_abs": 0.640625, "nrmse": 0.02814042568206787}, {"layer": 52, "state": "conv", "exact": false, "max_abs": 0.7265625, "nrmse": 0.07057075947523117}, {"layer": 52, "state": "recurrent", "exact": false, "max_abs": 0.05502641201019287, "nrmse": 0.05099900811910629}, {"layer": 53, "state": "conv", "exact": false, "max_abs": 0.84375, "nrmse": 0.08514342457056046}, {"layer": 53, "state": "recurrent", "exact": false, "max_abs": 0.05016501247882843, "nrmse": 0.06693451851606369}, {"layer": 54, "state": "conv", "exact": false, "max_abs": 0.892578125, "nrmse": 0.09433965384960175}, {"layer": 54, "state": "recurrent", "exact": false, "max_abs": 0.05852530896663666, "nrmse": 0.08190671354532242}, {"layer": 55, "state": "key", "exact": false, "max_abs": 0.578125, "nrmse": 0.02549578808248043}, {"layer": 55, "state": "value", "exact": false, "max_abs": 0.982421875, "nrmse": 0.035633765161037445}, {"layer": 56, "state": "conv", "exact": false, "max_abs": 0.87109375, "nrmse": 0.09353488683700562}, {"layer": 56, "state": "recurrent", "exact": false, "max_abs": 0.04837954044342041, "nrmse": 0.06559330970048904}, {"layer": 57, "state": "conv", "exact": false, "max_abs": 0.9375, "nrmse": 0.08754058927297592}, {"layer": 57, "state": "recurrent", "exact": false, "max_abs": 0.06442983448505402, "nrmse": 0.06771516054868698}, {"layer": 58, "state": "conv", "exact": false, "max_abs": 0.953125, "nrmse": 0.09323801845312119}, {"layer": 58, "state": "recurrent", "exact": false, "max_abs": 0.06945216655731201, "nrmse": 0.05448879674077034}, {"layer": 59, "state": "key", "exact": false, "max_abs": 0.578125, "nrmse": 0.025188324972987175}, {"layer": 59, "state": "value", "exact": false, "max_abs": 1.8125, "nrmse": 0.028554724529385567}, {"layer": 60, "state": "conv", "exact": false, "max_abs": 1.5, "nrmse": 0.07874731719493866}, {"layer": 60, "state": "recurrent", "exact": false, "max_abs": 0.07894952595233917, "nrmse": 0.06083955243229866}, {"layer": 61, "state": "conv", "exact": false, "max_abs": 0.996337890625, "nrmse": 0.09409598261117935}, {"layer": 61, "state": "recurrent", "exact": false, "max_abs": 0.04450759291648865, "nrmse": 0.06785402446985245}, {"layer": 62, "state": "conv", "exact": false, "max_abs": 1.53515625, "nrmse": 0.08058037608861923}, {"layer": 62, "state": "recurrent", "exact": false, "max_abs": 0.08402697741985321, "nrmse": 0.04325580224394798}, {"layer": 63, "state": "key", "exact": false, "max_abs": 0.6171875, "nrmse": 0.027296271175146103}, {"layer": 63, "state": "value", "exact": false, "max_abs": 2.265625, "nrmse": 0.018473410978913307}]} -{"phase": "capture_restore", "case": "gsm8k/test/1", "state_diff": []} -{"phase": "decode", "case": "gsm8k/test/1", "step": 0, "teacher_token": 1206, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.03530074283480644}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [7995, 7995, 7995], "layer_diff": [{"layer": 41, "compact_buffered": {"exact": false, "max_abs": 0.0625, "nrmse": 0.007775433827191591}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 42, "compact_buffered": {"exact": false, "max_abs": 0.125, "nrmse": 0.018040549010038376}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 43, "compact_buffered": {"exact": false, "max_abs": 0.125, "nrmse": 0.028837505728006363}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 44, "compact_buffered": {"exact": false, "max_abs": 0.150390625, "nrmse": 0.036955010145902634}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 45, "compact_buffered": {"exact": false, "max_abs": 0.197265625, "nrmse": 0.042504265904426575}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 46, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.044552356004714966}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 47, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.047775719314813614}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 48, "compact_buffered": {"exact": false, "max_abs": 0.283203125, "nrmse": 0.050229575484991074}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 49, "compact_buffered": {"exact": false, "max_abs": 0.328125, "nrmse": 0.05322303995490074}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 50, "compact_buffered": {"exact": false, "max_abs": 0.625, "nrmse": 0.05347342789173126}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 51, "compact_buffered": {"exact": false, "max_abs": 0.46875, "nrmse": 0.054443471133708954}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 52, "compact_buffered": {"exact": false, "max_abs": 0.58984375, "nrmse": 0.055172573775053024}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 53, "compact_buffered": {"exact": false, "max_abs": 0.625, "nrmse": 0.05643213540315628}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 54, "compact_buffered": {"exact": false, "max_abs": 1.5, "nrmse": 0.057832684367895126}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 55, "compact_buffered": {"exact": false, "max_abs": 1.3125, "nrmse": 0.05910389870405197}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 56, "compact_buffered": {"exact": false, "max_abs": 1.5, "nrmse": 0.05882955342531204}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 57, "compact_buffered": {"exact": false, "max_abs": 1.625, "nrmse": 0.05802130326628685}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 58, "compact_buffered": {"exact": false, "max_abs": 1.25, "nrmse": 0.058037079870700836}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 59, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.05826474353671074}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 60, "compact_buffered": {"exact": false, "max_abs": 1.25, "nrmse": 0.05683373659849167}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 61, "compact_buffered": {"exact": false, "max_abs": 1.5, "nrmse": 0.055655840784311295}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 62, "compact_buffered": {"exact": false, "max_abs": 2.0, "nrmse": 0.05386992171406746}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 63, "compact_buffered": {"exact": false, "max_abs": 6.0, "nrmse": 0.051288239657878876}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}], "state_diff": []} -{"phase": "decode", "case": "gsm8k/test/1", "step": 1, "teacher_token": 7995, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.022783564403653145}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [279, 279, 279], "layer_diff": [{"layer": 41, "compact_buffered": {"exact": false, "max_abs": 0.03125, "nrmse": 0.002497064881026745}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 42, "compact_buffered": {"exact": false, "max_abs": 0.0625, "nrmse": 0.005950317718088627}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 43, "compact_buffered": {"exact": false, "max_abs": 0.0625, "nrmse": 0.010397149249911308}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 44, "compact_buffered": {"exact": false, "max_abs": 0.09375, "nrmse": 0.013457856141030788}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 45, "compact_buffered": {"exact": false, "max_abs": 0.09375, "nrmse": 0.015509563498198986}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 46, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.018111733719706535}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 47, "compact_buffered": {"exact": false, "max_abs": 0.125, "nrmse": 0.01910177245736122}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 48, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.022413568571209908}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 49, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.02429608628153801}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 50, "compact_buffered": {"exact": false, "max_abs": 1.25, "nrmse": 0.028229212388396263}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 51, "compact_buffered": {"exact": false, "max_abs": 1.5, "nrmse": 0.030630284920334816}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 52, "compact_buffered": {"exact": false, "max_abs": 1.5, "nrmse": 0.03112686797976494}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 53, "compact_buffered": {"exact": false, "max_abs": 1.5, "nrmse": 0.032707810401916504}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 54, "compact_buffered": {"exact": false, "max_abs": 1.5, "nrmse": 0.0321192666888237}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 55, "compact_buffered": {"exact": false, "max_abs": 1.25, "nrmse": 0.032614585012197495}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 56, "compact_buffered": {"exact": false, "max_abs": 1.75, "nrmse": 0.03414294868707657}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 57, "compact_buffered": {"exact": false, "max_abs": 2.5, "nrmse": 0.03450377658009529}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 58, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.03242415934801102}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 59, "compact_buffered": {"exact": false, "max_abs": 0.625, "nrmse": 0.029972277581691742}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 60, "compact_buffered": {"exact": false, "max_abs": 2.0, "nrmse": 0.030913466587662697}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 61, "compact_buffered": {"exact": false, "max_abs": 2.0, "nrmse": 0.030147520825266838}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 62, "compact_buffered": {"exact": false, "max_abs": 3.0, "nrmse": 0.03682580590248108}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 63, "compact_buffered": {"exact": false, "max_abs": 8.0, "nrmse": 0.04310641065239906}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}], "state_diff": []} -{"phase": "decode", "case": "gsm8k/test/1", "step": 2, "teacher_token": 279, "compact_buffered": {"exact": false, "max_abs": 0.5625, "nrmse": 0.023349260911345482}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [2702, 2702, 2702], "layer_diff": [{"layer": 41, "compact_buffered": {"exact": false, "max_abs": 0.03125, "nrmse": 0.0040276492945849895}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 42, "compact_buffered": {"exact": false, "max_abs": 0.0625, "nrmse": 0.008030947297811508}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 43, "compact_buffered": {"exact": false, "max_abs": 0.0625, "nrmse": 0.01415078155696392}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 44, "compact_buffered": {"exact": false, "max_abs": 0.0859375, "nrmse": 0.0178099125623703}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 45, "compact_buffered": {"exact": false, "max_abs": 0.09375, "nrmse": 0.0195455402135849}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 46, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.021973488852381706}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 47, "compact_buffered": {"exact": false, "max_abs": 0.1875, "nrmse": 0.02338118851184845}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 48, "compact_buffered": {"exact": false, "max_abs": 0.1875, "nrmse": 0.025211341679096222}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 49, "compact_buffered": {"exact": false, "max_abs": 0.19140625, "nrmse": 0.030484117567539215}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 50, "compact_buffered": {"exact": false, "max_abs": 0.2265625, "nrmse": 0.03490211442112923}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 51, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.03682208061218262}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 52, "compact_buffered": {"exact": false, "max_abs": 0.359375, "nrmse": 0.037979014217853546}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 53, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.04061910882592201}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 54, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.03816542029380798}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 55, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.03853027895092964}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 56, "compact_buffered": {"exact": false, "max_abs": 0.625, "nrmse": 0.038530874997377396}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 57, "compact_buffered": {"exact": false, "max_abs": 0.625, "nrmse": 0.03839142620563507}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 58, "compact_buffered": {"exact": false, "max_abs": 0.75, "nrmse": 0.03695910796523094}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 59, "compact_buffered": {"exact": false, "max_abs": 0.75, "nrmse": 0.03974274918437004}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 60, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.03613412752747536}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 61, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.03659055382013321}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 62, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.03622285649180412}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 63, "compact_buffered": {"exact": false, "max_abs": 2.0, "nrmse": 0.04396894574165344}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}], "state_diff": []} -{"phase": "decode", "case": "gsm8k/test/1", "step": 3, "teacher_token": 2702, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.029369357973337173}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [1324, 1324, 1324], "layer_diff": [{"layer": 41, "compact_buffered": {"exact": false, "max_abs": 0.03125, "nrmse": 0.0030717267654836178}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 42, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.007898773066699505}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 43, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.012220322154462337}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 44, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.01429073791950941}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 45, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.015403177589178085}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 46, "compact_buffered": {"exact": false, "max_abs": 0.125, "nrmse": 0.015923844650387764}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 47, "compact_buffered": {"exact": false, "max_abs": 0.130859375, "nrmse": 0.018537497147917747}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 48, "compact_buffered": {"exact": false, "max_abs": 0.13671875, "nrmse": 0.020243994891643524}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 49, "compact_buffered": {"exact": false, "max_abs": 0.1875, "nrmse": 0.022297050803899765}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 50, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.0230008065700531}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 51, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.02585572749376297}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 52, "compact_buffered": {"exact": false, "max_abs": 0.328125, "nrmse": 0.027090294286608696}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 53, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.02905300445854664}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 54, "compact_buffered": {"exact": false, "max_abs": 0.375, "nrmse": 0.02827063389122486}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 55, "compact_buffered": {"exact": false, "max_abs": 0.3515625, "nrmse": 0.029413755983114243}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 56, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.031127894297242165}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 57, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.03127732127904892}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 58, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.03172437101602554}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 59, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.03405531123280525}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 60, "compact_buffered": {"exact": false, "max_abs": 0.5625, "nrmse": 0.03247629106044769}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 61, "compact_buffered": {"exact": false, "max_abs": 0.59375, "nrmse": 0.03296339139342308}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 62, "compact_buffered": {"exact": false, "max_abs": 2.0, "nrmse": 0.032747697085142136}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 63, "compact_buffered": {"exact": false, "max_abs": 1.5, "nrmse": 0.03714054077863693}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}], "state_diff": []} -{"phase": "decode", "case": "gsm8k/test/1", "step": 4, "teacher_token": 1324, "compact_buffered": {"exact": false, "max_abs": 0.375, "nrmse": 0.01972614787518978}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [314, 314, 314], "layer_diff": [{"layer": 41, "compact_buffered": {"exact": false, "max_abs": 0.03125, "nrmse": 0.0034973453730344772}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 42, "compact_buffered": {"exact": false, "max_abs": 0.0390625, "nrmse": 0.00747736357152462}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 43, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.014394775964319706}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 44, "compact_buffered": {"exact": false, "max_abs": 0.125, "nrmse": 0.01710880920290947}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 45, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.018772337585687637}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 46, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.021883992478251457}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 47, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.025333348661661148}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 48, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.028204916045069695}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 49, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.028524667024612427}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 50, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.031077388674020767}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 51, "compact_buffered": {"exact": false, "max_abs": 0.75, "nrmse": 0.03270306810736656}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 52, "compact_buffered": {"exact": false, "max_abs": 0.375, "nrmse": 0.03247910365462303}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 53, "compact_buffered": {"exact": false, "max_abs": 0.4375, "nrmse": 0.03292668238282204}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 54, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.03222402557730675}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 55, "compact_buffered": {"exact": false, "max_abs": 0.625, "nrmse": 0.03364887461066246}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 56, "compact_buffered": {"exact": false, "max_abs": 0.625, "nrmse": 0.03338215872645378}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 57, "compact_buffered": {"exact": false, "max_abs": 0.625, "nrmse": 0.03363557532429695}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 58, "compact_buffered": {"exact": false, "max_abs": 0.75, "nrmse": 0.031237555667757988}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 59, "compact_buffered": {"exact": false, "max_abs": 0.6875, "nrmse": 0.031019682064652443}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 60, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.031043188646435738}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 61, "compact_buffered": {"exact": false, "max_abs": 0.5625, "nrmse": 0.029424777254462242}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 62, "compact_buffered": {"exact": false, "max_abs": 2.0, "nrmse": 0.032424259930849075}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 63, "compact_buffered": {"exact": false, "max_abs": 6.0, "nrmse": 0.03422952815890312}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}], "state_diff": []} -{"phase": "decode", "case": "gsm8k/test/1", "step": 5, "teacher_token": 314, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.021810948848724365}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [47205, 47205, 47205], "layer_diff": [{"layer": 41, "compact_buffered": {"exact": false, "max_abs": 0.03125, "nrmse": 0.0036734428722411394}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 42, "compact_buffered": {"exact": false, "max_abs": 0.03515625, "nrmse": 0.007248974405229092}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 43, "compact_buffered": {"exact": false, "max_abs": 0.06298828125, "nrmse": 0.013772067613899708}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 44, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.018845224753022194}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 45, "compact_buffered": {"exact": false, "max_abs": 0.09375, "nrmse": 0.02027803286910057}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 46, "compact_buffered": {"exact": false, "max_abs": 0.109375, "nrmse": 0.023100169375538826}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 47, "compact_buffered": {"exact": false, "max_abs": 0.125, "nrmse": 0.02415597438812256}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 48, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.026708068326115608}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 49, "compact_buffered": {"exact": false, "max_abs": 0.27978515625, "nrmse": 0.03167141601443291}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 50, "compact_buffered": {"exact": false, "max_abs": 0.75, "nrmse": 0.03735828027129173}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 51, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.0369618721306324}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 52, "compact_buffered": {"exact": false, "max_abs": 1.25, "nrmse": 0.03789510950446129}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 53, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.03898381441831589}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 54, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.03747051581740379}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 55, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.03931336849927902}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 56, "compact_buffered": {"exact": false, "max_abs": 1.25, "nrmse": 0.03896882385015488}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 57, "compact_buffered": {"exact": false, "max_abs": 1.75, "nrmse": 0.039461154490709305}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 58, "compact_buffered": {"exact": false, "max_abs": 1.75, "nrmse": 0.03918009251356125}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 59, "compact_buffered": {"exact": false, "max_abs": 2.25, "nrmse": 0.04070711135864258}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 60, "compact_buffered": {"exact": false, "max_abs": 3.0, "nrmse": 0.03830813989043236}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 61, "compact_buffered": {"exact": false, "max_abs": 2.0, "nrmse": 0.03668176382780075}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 62, "compact_buffered": {"exact": false, "max_abs": 1.5, "nrmse": 0.036776334047317505}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 63, "compact_buffered": {"exact": false, "max_abs": 6.0, "nrmse": 0.044016122817993164}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}], "state_diff": []} -{"phase": "decode", "case": "gsm8k/test/1", "step": 6, "teacher_token": 47205, "compact_buffered": {"exact": false, "max_abs": 0.4375, "nrmse": 0.027601346373558044}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [2483, 2483, 2483], "layer_diff": [{"layer": 41, "compact_buffered": {"exact": false, "max_abs": 0.03125, "nrmse": 0.004141110461205244}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 42, "compact_buffered": {"exact": false, "max_abs": 0.0625, "nrmse": 0.007947721518576145}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 43, "compact_buffered": {"exact": false, "max_abs": 0.09375, "nrmse": 0.014475391246378422}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 44, "compact_buffered": {"exact": false, "max_abs": 0.09375, "nrmse": 0.016602372750639915}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 45, "compact_buffered": {"exact": false, "max_abs": 0.1015625, "nrmse": 0.01744982972741127}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 46, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.020117957144975662}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 47, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.022178461775183678}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 48, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.02452375367283821}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 49, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.028083449229598045}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 50, "compact_buffered": {"exact": false, "max_abs": 1.25, "nrmse": 0.02931784652173519}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 51, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.03130349516868591}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 52, "compact_buffered": {"exact": false, "max_abs": 0.625, "nrmse": 0.031928710639476776}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 53, "compact_buffered": {"exact": false, "max_abs": 0.625, "nrmse": 0.03222506120800972}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 54, "compact_buffered": {"exact": false, "max_abs": 0.75, "nrmse": 0.032288115471601486}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 55, "compact_buffered": {"exact": false, "max_abs": 0.5625, "nrmse": 0.03312955051660538}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 56, "compact_buffered": {"exact": false, "max_abs": 0.75, "nrmse": 0.033212415874004364}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 57, "compact_buffered": {"exact": false, "max_abs": 0.75, "nrmse": 0.03292194381356239}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 58, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.03422970324754715}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 59, "compact_buffered": {"exact": false, "max_abs": 0.75, "nrmse": 0.03387482836842537}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 60, "compact_buffered": {"exact": false, "max_abs": 0.625, "nrmse": 0.031795524060726166}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 61, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.0318218469619751}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 62, "compact_buffered": {"exact": false, "max_abs": 2.0, "nrmse": 0.032587818801403046}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 63, "compact_buffered": {"exact": false, "max_abs": 1.0625, "nrmse": 0.034014299511909485}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}], "state_diff": []} -{"phase": "decode", "case": "gsm8k/test/1", "step": 7, "teacher_token": 2483, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.019858064129948616}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [364, 364, 364], "layer_diff": [{"layer": 41, "compact_buffered": {"exact": false, "max_abs": 0.03125, "nrmse": 0.0034485762007534504}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 42, "compact_buffered": {"exact": false, "max_abs": 0.0625, "nrmse": 0.006945476867258549}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 43, "compact_buffered": {"exact": false, "max_abs": 0.125, "nrmse": 0.01029270887374878}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 44, "compact_buffered": {"exact": false, "max_abs": 0.125, "nrmse": 0.012644843198359013}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 45, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.014824660494923592}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 46, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.017936890944838524}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 47, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.019786689430475235}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 48, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.021439334377646446}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 49, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.023569904267787933}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 50, "compact_buffered": {"exact": false, "max_abs": 1.25, "nrmse": 0.027506649494171143}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 51, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.028770316392183304}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 52, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.029317548498511314}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 53, "compact_buffered": {"exact": false, "max_abs": 0.875, "nrmse": 0.030396899208426476}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 54, "compact_buffered": {"exact": false, "max_abs": 0.75, "nrmse": 0.030056513845920563}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 55, "compact_buffered": {"exact": false, "max_abs": 2.125, "nrmse": 0.033227864652872086}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 56, "compact_buffered": {"exact": false, "max_abs": 2.0, "nrmse": 0.03323650732636452}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 57, "compact_buffered": {"exact": false, "max_abs": 1.5, "nrmse": 0.032873786985874176}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 58, "compact_buffered": {"exact": false, "max_abs": 2.0, "nrmse": 0.03214018791913986}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 59, "compact_buffered": {"exact": false, "max_abs": 2.0, "nrmse": 0.032063309103250504}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 60, "compact_buffered": {"exact": false, "max_abs": 2.5, "nrmse": 0.03254409506917}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 61, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.030625052750110626}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 62, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.030440712347626686}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 63, "compact_buffered": {"exact": false, "max_abs": 3.0, "nrmse": 0.03304876759648323}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}], "state_diff": []} -{"phase": "decode", "case": "gsm8k/test/1", "step": 8, "teacher_token": 364, "compact_buffered": {"exact": false, "max_abs": 0.28125, "nrmse": 0.021358296275138855}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [279, 279, 279], "layer_diff": [{"layer": 41, "compact_buffered": {"exact": false, "max_abs": 0.03125, "nrmse": 0.003333813278004527}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 42, "compact_buffered": {"exact": false, "max_abs": 0.0625, "nrmse": 0.0067302156239748}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 43, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.011706427671015263}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 44, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.014242243953049183}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 45, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.015345202758908272}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 46, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.017238087952136993}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 47, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.018807219341397285}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 48, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.02110205590724945}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 49, "compact_buffered": {"exact": false, "max_abs": 0.1875, "nrmse": 0.021822309121489525}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 50, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.023714188486337662}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 51, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.02537180483341217}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 52, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.026377251371741295}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 53, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.02720579132437706}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 54, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.024994103237986565}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 55, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.02616407535970211}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 56, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.02692357636988163}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 57, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.02649606391787529}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 58, "compact_buffered": {"exact": false, "max_abs": 2.0, "nrmse": 0.026108063757419586}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 59, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.025614891201257706}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 60, "compact_buffered": {"exact": false, "max_abs": 2.0, "nrmse": 0.02537686936557293}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 61, "compact_buffered": {"exact": false, "max_abs": 2.0, "nrmse": 0.024992309510707855}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 62, "compact_buffered": {"exact": false, "max_abs": 2.0, "nrmse": 0.025494132190942764}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 63, "compact_buffered": {"exact": false, "max_abs": 4.0, "nrmse": 0.025960970669984818}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}], "state_diff": []} -{"phase": "decode", "case": "gsm8k/test/1", "step": 9, "teacher_token": 279, "compact_buffered": {"exact": false, "max_abs": 0.421875, "nrmse": 0.02206346206367016}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [60507, 60507, 60507], "layer_diff": [{"layer": 41, "compact_buffered": {"exact": false, "max_abs": 0.03125, "nrmse": 0.004006790928542614}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 42, "compact_buffered": {"exact": false, "max_abs": 0.03125, "nrmse": 0.007465104106813669}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 43, "compact_buffered": {"exact": false, "max_abs": 0.0703125, "nrmse": 0.01328459195792675}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 44, "compact_buffered": {"exact": false, "max_abs": 0.09375, "nrmse": 0.016330787912011147}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 45, "compact_buffered": {"exact": false, "max_abs": 0.09375, "nrmse": 0.01812141388654709}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 46, "compact_buffered": {"exact": false, "max_abs": 0.09375, "nrmse": 0.019993863999843597}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 47, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.02206403762102127}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 48, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.024696802720427513}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 49, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.029829684644937515}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 50, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.03488713875412941}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 51, "compact_buffered": {"exact": false, "max_abs": 0.2578125, "nrmse": 0.035751823335886}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 52, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.037326205521821976}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 53, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.038055483251810074}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 54, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.033683981746435165}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 55, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.035830527544021606}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 56, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.03621388226747513}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 57, "compact_buffered": {"exact": false, "max_abs": 0.75, "nrmse": 0.03486642241477966}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 58, "compact_buffered": {"exact": false, "max_abs": 0.75, "nrmse": 0.031729549169540405}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 59, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.035626787692308426}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 60, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.03233060613274574}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 61, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.031435925513505936}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 62, "compact_buffered": {"exact": false, "max_abs": 2.0, "nrmse": 0.030959220603108406}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 63, "compact_buffered": {"exact": false, "max_abs": 4.5, "nrmse": 0.03623591363430023}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}], "state_diff": []} -{"phase": "decode", "case": "gsm8k/test/1", "step": 10, "teacher_token": 60507, "compact_buffered": {"exact": false, "max_abs": 0.484375, "nrmse": 0.01969759725034237}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [11, 11, 11], "layer_diff": [{"layer": 41, "compact_buffered": {"exact": false, "max_abs": 0.03125, "nrmse": 0.0030685164965689182}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 42, "compact_buffered": {"exact": false, "max_abs": 0.0625, "nrmse": 0.006534868851304054}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 43, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.010814945213496685}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 44, "compact_buffered": {"exact": false, "max_abs": 0.109375, "nrmse": 0.012364295311272144}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 45, "compact_buffered": {"exact": false, "max_abs": 0.125, "nrmse": 0.014713520184159279}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 46, "compact_buffered": {"exact": false, "max_abs": 0.125, "nrmse": 0.017258044332265854}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 47, "compact_buffered": {"exact": false, "max_abs": 0.125, "nrmse": 0.01860966719686985}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 48, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.023461701348423958}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 49, "compact_buffered": {"exact": false, "max_abs": 0.2109375, "nrmse": 0.027242885902523994}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 50, "compact_buffered": {"exact": false, "max_abs": 0.375, "nrmse": 0.032446544617414474}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 51, "compact_buffered": {"exact": false, "max_abs": 0.375, "nrmse": 0.03295440226793289}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 52, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.034198250621557236}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 53, "compact_buffered": {"exact": false, "max_abs": 1.3125, "nrmse": 0.03698066994547844}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 54, "compact_buffered": {"exact": false, "max_abs": 1.875, "nrmse": 0.03697710111737251}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 55, "compact_buffered": {"exact": false, "max_abs": 1.625, "nrmse": 0.0367177277803421}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 56, "compact_buffered": {"exact": false, "max_abs": 1.75, "nrmse": 0.03832268714904785}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 57, "compact_buffered": {"exact": false, "max_abs": 1.75, "nrmse": 0.0383295901119709}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 58, "compact_buffered": {"exact": false, "max_abs": 1.625, "nrmse": 0.03977617993950844}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 59, "compact_buffered": {"exact": false, "max_abs": 0.5625, "nrmse": 0.03484763950109482}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 60, "compact_buffered": {"exact": false, "max_abs": 0.75, "nrmse": 0.03595894202589989}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 61, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.0340726301074028}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 62, "compact_buffered": {"exact": false, "max_abs": 2.0, "nrmse": 0.03740076348185539}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 63, "compact_buffered": {"exact": false, "max_abs": 4.0, "nrmse": 0.03612157702445984}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}], "state_diff": []} -{"phase": "decode", "case": "gsm8k/test/1", "step": 11, "teacher_token": 11, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.040291476994752884}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [567, 567, 567], "layer_diff": [{"layer": 41, "compact_buffered": {"exact": false, "max_abs": 0.03125, "nrmse": 0.0029816478490829468}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 42, "compact_buffered": {"exact": false, "max_abs": 0.03125, "nrmse": 0.007037548813968897}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 43, "compact_buffered": {"exact": false, "max_abs": 0.0625, "nrmse": 0.010430746711790562}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 44, "compact_buffered": {"exact": false, "max_abs": 0.125, "nrmse": 0.014418485574424267}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 45, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.01679215580224991}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 46, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.019445104524493217}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 47, "compact_buffered": {"exact": false, "max_abs": 0.375, "nrmse": 0.020998451858758926}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 48, "compact_buffered": {"exact": false, "max_abs": 0.375, "nrmse": 0.02317448891699314}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 49, "compact_buffered": {"exact": false, "max_abs": 0.375, "nrmse": 0.026117585599422455}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 50, "compact_buffered": {"exact": false, "max_abs": 0.375, "nrmse": 0.029593462124466896}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 51, "compact_buffered": {"exact": false, "max_abs": 0.75, "nrmse": 0.031576745212078094}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 52, "compact_buffered": {"exact": false, "max_abs": 0.75, "nrmse": 0.03470155969262123}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 53, "compact_buffered": {"exact": false, "max_abs": 0.875, "nrmse": 0.03791496530175209}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 54, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.03816435486078262}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 55, "compact_buffered": {"exact": false, "max_abs": 0.75, "nrmse": 0.03997182846069336}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 56, "compact_buffered": {"exact": false, "max_abs": 0.625, "nrmse": 0.04004881531000137}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 57, "compact_buffered": {"exact": false, "max_abs": 0.625, "nrmse": 0.040208570659160614}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 58, "compact_buffered": {"exact": false, "max_abs": 2.0, "nrmse": 0.03841681778430939}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 59, "compact_buffered": {"exact": false, "max_abs": 0.75, "nrmse": 0.03984598442912102}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 60, "compact_buffered": {"exact": false, "max_abs": 0.875, "nrmse": 0.03862430900335312}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 61, "compact_buffered": {"exact": false, "max_abs": 1.125, "nrmse": 0.038350917398929596}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 62, "compact_buffered": {"exact": false, "max_abs": 1.25, "nrmse": 0.03714261204004288}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 63, "compact_buffered": {"exact": false, "max_abs": 1.5, "nrmse": 0.04341091960668564}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}], "state_diff": []} -{"phase": "decode", "case": "gsm8k/test/1", "step": 12, "teacher_token": 567, "compact_buffered": {"exact": false, "max_abs": 0.453125, "nrmse": 0.03866123408079147}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [1144, 1144, 1144], "layer_diff": [{"layer": 41, "compact_buffered": {"exact": false, "max_abs": 0.03125, "nrmse": 0.0040316591039299965}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 42, "compact_buffered": {"exact": false, "max_abs": 0.0390625, "nrmse": 0.007981079630553722}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 43, "compact_buffered": {"exact": false, "max_abs": 0.09375, "nrmse": 0.011914466507732868}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 44, "compact_buffered": {"exact": false, "max_abs": 0.125, "nrmse": 0.015032190829515457}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 45, "compact_buffered": {"exact": false, "max_abs": 0.125, "nrmse": 0.016956642270088196}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 46, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.020260639488697052}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 47, "compact_buffered": {"exact": false, "max_abs": 0.125, "nrmse": 0.021396633237600327}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 48, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.02375609613955021}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 49, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.027629969641566277}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 50, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.030503343790769577}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 51, "compact_buffered": {"exact": false, "max_abs": 0.296875, "nrmse": 0.031005574390292168}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 52, "compact_buffered": {"exact": false, "max_abs": 0.75, "nrmse": 0.03423059359192848}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 53, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.03742779418826103}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 54, "compact_buffered": {"exact": false, "max_abs": 0.625, "nrmse": 0.037868622690439224}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 55, "compact_buffered": {"exact": false, "max_abs": 0.75, "nrmse": 0.03958245739340782}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 56, "compact_buffered": {"exact": false, "max_abs": 0.59375, "nrmse": 0.039692264050245285}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 57, "compact_buffered": {"exact": false, "max_abs": 0.5703125, "nrmse": 0.040117114782333374}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 58, "compact_buffered": {"exact": false, "max_abs": 0.75, "nrmse": 0.04049203172326088}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 59, "compact_buffered": {"exact": false, "max_abs": 0.6328125, "nrmse": 0.040304169058799744}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 60, "compact_buffered": {"exact": false, "max_abs": 0.75, "nrmse": 0.03950680419802666}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 61, "compact_buffered": {"exact": false, "max_abs": 0.78125, "nrmse": 0.03936716541647911}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 62, "compact_buffered": {"exact": false, "max_abs": 1.5, "nrmse": 0.039494555443525314}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 63, "compact_buffered": {"exact": false, "max_abs": 2.75, "nrmse": 0.04306350275874138}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}], "state_diff": []} -{"phase": "decode", "case": "gsm8k/test/1", "step": 13, "teacher_token": 1144, "compact_buffered": {"exact": false, "max_abs": 0.40625, "nrmse": 0.027117768302559853}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [310, 310, 310], "layer_diff": [{"layer": 41, "compact_buffered": {"exact": false, "max_abs": 0.01953125, "nrmse": 0.0036602807231247425}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 42, "compact_buffered": {"exact": false, "max_abs": 0.0625, "nrmse": 0.0071758790872991085}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 43, "compact_buffered": {"exact": false, "max_abs": 0.0703125, "nrmse": 0.010367976501584053}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 44, "compact_buffered": {"exact": false, "max_abs": 0.0703125, "nrmse": 0.01291560661047697}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 45, "compact_buffered": {"exact": false, "max_abs": 0.125, "nrmse": 0.015045356936752796}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 46, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.017600608989596367}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 47, "compact_buffered": {"exact": false, "max_abs": 0.140625, "nrmse": 0.019352437928318977}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 48, "compact_buffered": {"exact": false, "max_abs": 0.1875, "nrmse": 0.02107785828411579}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 49, "compact_buffered": {"exact": false, "max_abs": 0.171875, "nrmse": 0.022966448217630386}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 50, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.02533491887152195}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 51, "compact_buffered": {"exact": false, "max_abs": 0.75, "nrmse": 0.027785034850239754}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 52, "compact_buffered": {"exact": false, "max_abs": 0.3359375, "nrmse": 0.029153762385249138}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 53, "compact_buffered": {"exact": false, "max_abs": 0.375, "nrmse": 0.031042469665408134}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 54, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.03153139725327492}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 55, "compact_buffered": {"exact": false, "max_abs": 0.625, "nrmse": 0.03289324417710304}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 56, "compact_buffered": {"exact": false, "max_abs": 0.75, "nrmse": 0.03376872465014458}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 57, "compact_buffered": {"exact": false, "max_abs": 0.75, "nrmse": 0.03445173427462578}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 58, "compact_buffered": {"exact": false, "max_abs": 0.75, "nrmse": 0.034553162753582}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 59, "compact_buffered": {"exact": false, "max_abs": 0.75, "nrmse": 0.03276233747601509}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 60, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.03396955877542496}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 61, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.03251054137945175}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 62, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.03484716638922691}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 63, "compact_buffered": {"exact": false, "max_abs": 2.0, "nrmse": 0.03559885546565056}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}], "state_diff": []} -{"phase": "decode", "case": "gsm8k/test/1", "step": 14, "teacher_token": 310, "compact_buffered": {"exact": false, "max_abs": 0.375, "nrmse": 0.02413068898022175}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [10724, 10724, 10724], "layer_diff": [{"layer": 41, "compact_buffered": {"exact": false, "max_abs": 0.015625, "nrmse": 0.0038125773426145315}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 42, "compact_buffered": {"exact": false, "max_abs": 0.0625, "nrmse": 0.007754190359264612}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 43, "compact_buffered": {"exact": false, "max_abs": 0.0625, "nrmse": 0.011528944596648216}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 44, "compact_buffered": {"exact": false, "max_abs": 0.0859375, "nrmse": 0.014547341503202915}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 45, "compact_buffered": {"exact": false, "max_abs": 0.09375, "nrmse": 0.016438880935311317}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 46, "compact_buffered": {"exact": false, "max_abs": 0.125, "nrmse": 0.019069673493504524}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 47, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.020575927570462227}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 48, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.02243492752313614}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 49, "compact_buffered": {"exact": false, "max_abs": 0.1875, "nrmse": 0.02567751705646515}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 50, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.027163630351424217}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 51, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.02914813905954361}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 52, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.03101595863699913}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 53, "compact_buffered": {"exact": false, "max_abs": 1.5, "nrmse": 0.03417181596159935}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 54, "compact_buffered": {"exact": false, "max_abs": 2.0, "nrmse": 0.03385016322135925}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 55, "compact_buffered": {"exact": false, "max_abs": 2.5, "nrmse": 0.03490690514445305}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 56, "compact_buffered": {"exact": false, "max_abs": 2.5, "nrmse": 0.03480594605207443}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 57, "compact_buffered": {"exact": false, "max_abs": 3.0, "nrmse": 0.03543567284941673}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 58, "compact_buffered": {"exact": false, "max_abs": 3.0, "nrmse": 0.03572867438197136}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 59, "compact_buffered": {"exact": false, "max_abs": 4.25, "nrmse": 0.0386384017765522}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 60, "compact_buffered": {"exact": false, "max_abs": 4.0, "nrmse": 0.03595902770757675}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 61, "compact_buffered": {"exact": false, "max_abs": 3.0, "nrmse": 0.035663023591041565}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 62, "compact_buffered": {"exact": false, "max_abs": 2.0, "nrmse": 0.03509168326854706}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 63, "compact_buffered": {"exact": false, "max_abs": 5.5, "nrmse": 0.0344194695353508}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}], "state_diff": []} -{"phase": "decode", "case": "gsm8k/test/1", "step": 15, "teacher_token": 10724, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.0214045662432909}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [279, 279, 279], "layer_diff": [{"layer": 41, "compact_buffered": {"exact": false, "max_abs": 0.03125, "nrmse": 0.003364731092005968}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 42, "compact_buffered": {"exact": false, "max_abs": 0.0625, "nrmse": 0.006526120938360691}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 43, "compact_buffered": {"exact": false, "max_abs": 0.09375, "nrmse": 0.009761461988091469}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 44, "compact_buffered": {"exact": false, "max_abs": 0.09375, "nrmse": 0.012532729655504227}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 45, "compact_buffered": {"exact": false, "max_abs": 0.125, "nrmse": 0.014333542436361313}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 46, "compact_buffered": {"exact": false, "max_abs": 0.125, "nrmse": 0.015826622024178505}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 47, "compact_buffered": {"exact": false, "max_abs": 0.125, "nrmse": 0.01842726208269596}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 48, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.0203526820987463}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 49, "compact_buffered": {"exact": false, "max_abs": 0.1875, "nrmse": 0.021640026941895485}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 50, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.025472410023212433}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 51, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.028072938323020935}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 52, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.029975565150380135}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 53, "compact_buffered": {"exact": false, "max_abs": 0.75, "nrmse": 0.03094879724085331}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 54, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.03164401650428772}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 55, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.032205142080783844}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 56, "compact_buffered": {"exact": false, "max_abs": 0.75, "nrmse": 0.032185256481170654}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 57, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.03217645362019539}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 58, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.032963789999485016}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 59, "compact_buffered": {"exact": false, "max_abs": 0.625, "nrmse": 0.03032032400369644}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 60, "compact_buffered": {"exact": false, "max_abs": 0.6875, "nrmse": 0.03107144683599472}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 61, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.0310567244887352}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 62, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.032807573676109314}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 63, "compact_buffered": {"exact": false, "max_abs": 7.0, "nrmse": 0.037635188549757004}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}], "state_diff": []} -{"phase": "prefill", "case": "gsm8k/test/424", "prompt_tokens": 106, "capacity": 1024, "prompt_sha256": "03cbdf5804bab62d4efbc0ae89e191bd557840c6d9a1a5ada9669de953a0b783", "compact_buffered": {"exact": false, "max_abs": 3.890625, "nrmse": 0.029980963096022606}, "buffered_repeated": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [1206, 1206, 1206], "layer_diff": [{"layer": 35, "exact": false, "max_abs": 0.03125, "nrmse": 0.00021201255731284618}, {"layer": 36, "exact": false, "max_abs": 0.25, "nrmse": 0.0024990097153931856}, {"layer": 37, "exact": false, "max_abs": 0.5, "nrmse": 0.004719598218798637}, {"layer": 38, "exact": false, "max_abs": 0.5, "nrmse": 0.005951761268079281}, {"layer": 39, "exact": false, "max_abs": 0.5, "nrmse": 0.007887035608291626}, {"layer": 40, "exact": false, "max_abs": 0.75, "nrmse": 0.009706377983093262}, {"layer": 41, "exact": false, "max_abs": 0.875, "nrmse": 0.010712246410548687}, {"layer": 42, "exact": false, "max_abs": 0.90625, "nrmse": 0.011322427541017532}, {"layer": 43, "exact": false, "max_abs": 1.0, "nrmse": 0.013455981388688087}, {"layer": 44, "exact": false, "max_abs": 1.0546875, "nrmse": 0.01487346738576889}, {"layer": 45, "exact": false, "max_abs": 1.125, "nrmse": 0.015733681619167328}, {"layer": 46, "exact": false, "max_abs": 1.0, "nrmse": 0.01591903157532215}, {"layer": 47, "exact": false, "max_abs": 1.84375, "nrmse": 0.01874421164393425}, {"layer": 48, "exact": false, "max_abs": 1.5, "nrmse": 0.02047210931777954}, {"layer": 49, "exact": false, "max_abs": 2.4375, "nrmse": 0.021971283480525017}, {"layer": 50, "exact": false, "max_abs": 2.25, "nrmse": 0.02231096290051937}, {"layer": 51, "exact": false, "max_abs": 2.375, "nrmse": 0.025380097329616547}, {"layer": 52, "exact": false, "max_abs": 1.75, "nrmse": 0.027753403410315514}, {"layer": 53, "exact": false, "max_abs": 2.3125, "nrmse": 0.030110223218798637}, {"layer": 54, "exact": false, "max_abs": 2.5, "nrmse": 0.026986097916960716}, {"layer": 55, "exact": false, "max_abs": 2.5, "nrmse": 0.03034009411931038}, {"layer": 56, "exact": false, "max_abs": 4.625, "nrmse": 0.03133460879325867}, {"layer": 57, "exact": false, "max_abs": 6.25, "nrmse": 0.031590696424245834}, {"layer": 58, "exact": false, "max_abs": 8.0, "nrmse": 0.02827567607164383}, {"layer": 59, "exact": false, "max_abs": 10.5, "nrmse": 0.03306536749005318}, {"layer": 60, "exact": false, "max_abs": 12.75, "nrmse": 0.03319771960377693}, {"layer": 61, "exact": false, "max_abs": 15.25, "nrmse": 0.033511463552713394}, {"layer": 62, "exact": false, "max_abs": 10.0, "nrmse": 0.03524669259786606}, {"layer": 63, "exact": false, "max_abs": 25.0, "nrmse": 0.0364813506603241}], "state_diff": [{"layer": 36, "state": "recurrent", "exact": false, "max_abs": 0.0018756464123725891, "nrmse": 0.0011717906454578042}, {"layer": 37, "state": "conv", "exact": false, "max_abs": 0.15625, "nrmse": 0.016431430354714394}, {"layer": 37, "state": "recurrent", "exact": false, "max_abs": 0.00861323717981577, "nrmse": 0.012398558668792248}, {"layer": 38, "state": "conv", "exact": false, "max_abs": 0.21875, "nrmse": 0.030127285048365593}, {"layer": 38, "state": "recurrent", "exact": false, "max_abs": 0.008101731538772583, "nrmse": 0.026300549507141113}, {"layer": 39, "state": "key", "exact": false, "max_abs": 0.234375, "nrmse": 0.008645334281027317}, {"layer": 39, "state": "value", "exact": false, "max_abs": 0.2421875, "nrmse": 0.013015429489314556}, {"layer": 40, "state": "conv", "exact": false, "max_abs": 0.4375, "nrmse": 0.042933784425258636}, {"layer": 40, "state": "recurrent", "exact": false, "max_abs": 0.014508247375488281, "nrmse": 0.014950418844819069}, {"layer": 41, "state": "conv", "exact": false, "max_abs": 0.40625, "nrmse": 0.0507311187684536}, {"layer": 41, "state": "recurrent", "exact": false, "max_abs": 0.01569046825170517, "nrmse": 0.04031945765018463}, {"layer": 42, "state": "conv", "exact": false, "max_abs": 0.380859375, "nrmse": 0.059988826513290405}, {"layer": 42, "state": "recurrent", "exact": false, "max_abs": 0.012959972023963928, "nrmse": 0.039817579090595245}, {"layer": 43, "state": "key", "exact": false, "max_abs": 0.4501953125, "nrmse": 0.014691941440105438}, {"layer": 43, "state": "value", "exact": false, "max_abs": 0.4921875, "nrmse": 0.017778607085347176}, {"layer": 44, "state": "conv", "exact": false, "max_abs": 0.5625, "nrmse": 0.07260511815547943}, {"layer": 44, "state": "recurrent", "exact": false, "max_abs": 0.021238893270492554, "nrmse": 0.05692726746201515}, {"layer": 45, "state": "conv", "exact": false, "max_abs": 0.5625, "nrmse": 0.08554074168205261}, {"layer": 45, "state": "recurrent", "exact": false, "max_abs": 0.019624754786491394, "nrmse": 0.08326196670532227}, {"layer": 46, "state": "conv", "exact": false, "max_abs": 0.66796875, "nrmse": 0.08974196761846542}, {"layer": 46, "state": "recurrent", "exact": false, "max_abs": 0.030830055475234985, "nrmse": 0.0828576534986496}, {"layer": 47, "state": "key", "exact": false, "max_abs": 0.62109375, "nrmse": 0.01927304081618786}, {"layer": 47, "state": "value", "exact": false, "max_abs": 0.8125, "nrmse": 0.024891650304198265}, {"layer": 48, "state": "conv", "exact": false, "max_abs": 0.90234375, "nrmse": 0.10041088610887527}, {"layer": 48, "state": "recurrent", "exact": false, "max_abs": 0.02020172029733658, "nrmse": 0.0909489169716835}, {"layer": 49, "state": "conv", "exact": false, "max_abs": 0.875, "nrmse": 0.10831701010465622}, {"layer": 49, "state": "recurrent", "exact": false, "max_abs": 0.04584473371505737, "nrmse": 0.0973081886768341}, {"layer": 50, "state": "conv", "exact": false, "max_abs": 1.04296875, "nrmse": 0.10665299743413925}, {"layer": 50, "state": "recurrent", "exact": false, "max_abs": 0.02286107838153839, "nrmse": 0.09075934439897537}, {"layer": 51, "state": "key", "exact": false, "max_abs": 0.81640625, "nrmse": 0.021241847425699234}, {"layer": 51, "state": "value", "exact": false, "max_abs": 1.03125, "nrmse": 0.026385612785816193}, {"layer": 52, "state": "conv", "exact": false, "max_abs": 1.11376953125, "nrmse": 0.10195021331310272}, {"layer": 52, "state": "recurrent", "exact": false, "max_abs": 0.09245634078979492, "nrmse": 0.06528455018997192}, {"layer": 53, "state": "conv", "exact": false, "max_abs": 1.0625, "nrmse": 0.12166950851678848}, {"layer": 53, "state": "recurrent", "exact": false, "max_abs": 0.06181037425994873, "nrmse": 0.09399404376745224}, {"layer": 54, "state": "conv", "exact": false, "max_abs": 1.16015625, "nrmse": 0.13379767537117004}, {"layer": 54, "state": "recurrent", "exact": false, "max_abs": 0.0732591301202774, "nrmse": 0.11704912781715393}, {"layer": 55, "state": "key", "exact": false, "max_abs": 0.890625, "nrmse": 0.02375023066997528}, {"layer": 55, "state": "value", "exact": false, "max_abs": 1.45703125, "nrmse": 0.03451654687523842}, {"layer": 56, "state": "conv", "exact": false, "max_abs": 1.34375, "nrmse": 0.13237310945987701}, {"layer": 56, "state": "recurrent", "exact": false, "max_abs": 0.0661701187491417, "nrmse": 0.08466211706399918}, {"layer": 57, "state": "conv", "exact": false, "max_abs": 1.53125, "nrmse": 0.12896430492401123}, {"layer": 57, "state": "recurrent", "exact": false, "max_abs": 0.14541256427764893, "nrmse": 0.0912630558013916}, {"layer": 58, "state": "conv", "exact": false, "max_abs": 1.3828125, "nrmse": 0.1335846185684204}, {"layer": 58, "state": "recurrent", "exact": false, "max_abs": 0.07205022126436234, "nrmse": 0.0703577920794487}, {"layer": 59, "state": "key", "exact": false, "max_abs": 1.015625, "nrmse": 0.024760516360402107}, {"layer": 59, "state": "value", "exact": false, "max_abs": 2.33203125, "nrmse": 0.027468768879771233}, {"layer": 60, "state": "conv", "exact": false, "max_abs": 2.4296875, "nrmse": 0.11387389898300171}, {"layer": 60, "state": "recurrent", "exact": false, "max_abs": 0.31357789039611816, "nrmse": 0.08904241025447845}, {"layer": 61, "state": "conv", "exact": false, "max_abs": 1.7421875, "nrmse": 0.1345670074224472}, {"layer": 61, "state": "recurrent", "exact": false, "max_abs": 0.03928488492965698, "nrmse": 0.09569883346557617}, {"layer": 62, "state": "conv", "exact": false, "max_abs": 2.53125, "nrmse": 0.11435270309448242}, {"layer": 62, "state": "recurrent", "exact": false, "max_abs": 0.09763680398464203, "nrmse": 0.05918039754033089}, {"layer": 63, "state": "key", "exact": false, "max_abs": 0.76953125, "nrmse": 0.027863062918186188}, {"layer": 63, "state": "value", "exact": false, "max_abs": 5.3046875, "nrmse": 0.019731946289539337}]} -{"phase": "capture_restore", "case": "gsm8k/test/424", "state_diff": []} -{"phase": "decode", "case": "gsm8k/test/424", "step": 0, "teacher_token": 1206, "compact_buffered": {"exact": false, "max_abs": 1.125, "nrmse": 0.040244732052087784}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [7995, 7995, 7995], "layer_diff": [{"layer": 36, "compact_buffered": {"exact": false, "max_abs": 0.04296875, "nrmse": 0.01244552992284298}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 37, "compact_buffered": {"exact": false, "max_abs": 0.08984375, "nrmse": 0.023444542661309242}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 38, "compact_buffered": {"exact": false, "max_abs": 0.375, "nrmse": 0.03235524520277977}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 39, "compact_buffered": {"exact": false, "max_abs": 0.375, "nrmse": 0.041259441524744034}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 40, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.04973213002085686}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 41, "compact_buffered": {"exact": false, "max_abs": 0.5625, "nrmse": 0.05713227018713951}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 42, "compact_buffered": {"exact": false, "max_abs": 1.125, "nrmse": 0.06271854043006897}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 43, "compact_buffered": {"exact": false, "max_abs": 1.125, "nrmse": 0.06743495911359787}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 44, "compact_buffered": {"exact": false, "max_abs": 1.75, "nrmse": 0.07297339290380478}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 45, "compact_buffered": {"exact": false, "max_abs": 1.25, "nrmse": 0.07286365330219269}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 46, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.07104818522930145}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 47, "compact_buffered": {"exact": false, "max_abs": 1.75, "nrmse": 0.0734754279255867}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 48, "compact_buffered": {"exact": false, "max_abs": 1.25, "nrmse": 0.07380354404449463}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 49, "compact_buffered": {"exact": false, "max_abs": 0.75, "nrmse": 0.07314188033342361}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 50, "compact_buffered": {"exact": false, "max_abs": 0.4453125, "nrmse": 0.07120824605226517}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 51, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.06708870828151703}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 52, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.06709706038236618}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 53, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.06616181880235672}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 54, "compact_buffered": {"exact": false, "max_abs": 1.375, "nrmse": 0.06539811193943024}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 55, "compact_buffered": {"exact": false, "max_abs": 0.8828125, "nrmse": 0.06568265706300735}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 56, "compact_buffered": {"exact": false, "max_abs": 0.921875, "nrmse": 0.06618709862232208}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 57, "compact_buffered": {"exact": false, "max_abs": 0.8671875, "nrmse": 0.06357471644878387}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 58, "compact_buffered": {"exact": false, "max_abs": 1.5, "nrmse": 0.06459666788578033}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 59, "compact_buffered": {"exact": false, "max_abs": 1.0625, "nrmse": 0.06446419656276703}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 60, "compact_buffered": {"exact": false, "max_abs": 1.078125, "nrmse": 0.06245187669992447}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 61, "compact_buffered": {"exact": false, "max_abs": 1.015625, "nrmse": 0.06049462407827377}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 62, "compact_buffered": {"exact": false, "max_abs": 1.15625, "nrmse": 0.05936203524470329}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 63, "compact_buffered": {"exact": false, "max_abs": 23.0, "nrmse": 0.07453305274248123}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}], "state_diff": []} -{"phase": "decode", "case": "gsm8k/test/424", "step": 1, "teacher_token": 7995, "compact_buffered": {"exact": false, "max_abs": 0.4375, "nrmse": 0.0276047196239233}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [279, 279, 279], "layer_diff": [{"layer": 36, "compact_buffered": {"exact": false, "max_abs": 0.03125, "nrmse": 0.004077203571796417}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 37, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.010157985612750053}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 38, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.013922483660280704}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 39, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.013971911743283272}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 40, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.015538658015429974}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 41, "compact_buffered": {"exact": false, "max_abs": 0.0986328125, "nrmse": 0.01631973311305046}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 42, "compact_buffered": {"exact": false, "max_abs": 0.09375, "nrmse": 0.016399720683693886}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 43, "compact_buffered": {"exact": false, "max_abs": 0.109375, "nrmse": 0.018005607649683952}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 44, "compact_buffered": {"exact": false, "max_abs": 0.125, "nrmse": 0.0197128988802433}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 45, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.02176528237760067}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 46, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.023048415780067444}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 47, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.024183547124266624}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 48, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.02718840166926384}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 49, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.027714312076568604}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 50, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.03027355670928955}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 51, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.030646812170743942}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 52, "compact_buffered": {"exact": false, "max_abs": 0.3125, "nrmse": 0.030829910188913345}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 53, "compact_buffered": {"exact": false, "max_abs": 0.875, "nrmse": 0.032798971980810165}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 54, "compact_buffered": {"exact": false, "max_abs": 1.375, "nrmse": 0.03246929123997688}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 55, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.03209254518151283}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 56, "compact_buffered": {"exact": false, "max_abs": 1.75, "nrmse": 0.03368675708770752}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 57, "compact_buffered": {"exact": false, "max_abs": 1.5, "nrmse": 0.033042386174201965}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 58, "compact_buffered": {"exact": false, "max_abs": 0.6875, "nrmse": 0.03247496858239174}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 59, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.031637683510780334}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 60, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.03134135529398918}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 61, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.03172137215733528}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 62, "compact_buffered": {"exact": false, "max_abs": 1.5, "nrmse": 0.03233013674616814}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 63, "compact_buffered": {"exact": false, "max_abs": 2.0, "nrmse": 0.03353269025683403}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}], "state_diff": []} -{"phase": "decode", "case": "gsm8k/test/424", "step": 2, "teacher_token": 279, "compact_buffered": {"exact": false, "max_abs": 0.3935546875, "nrmse": 0.024922151118516922}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [2702, 2702, 2702], "layer_diff": [{"layer": 36, "compact_buffered": {"exact": false, "max_abs": 0.03125, "nrmse": 0.005190987139940262}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 37, "compact_buffered": {"exact": false, "max_abs": 0.0390625, "nrmse": 0.01049896515905857}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 38, "compact_buffered": {"exact": false, "max_abs": 0.05078125, "nrmse": 0.012528279796242714}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 39, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.017044927924871445}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 40, "compact_buffered": {"exact": false, "max_abs": 0.75, "nrmse": 0.02082565426826477}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 41, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.02136523276567459}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 42, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.02221645973622799}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 43, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.026102488860487938}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 44, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.027914486825466156}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 45, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.02995472587645054}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 46, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.032212696969509125}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 47, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.032495446503162384}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 48, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.032556191086769104}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 49, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.03661176934838295}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 50, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.037539493292570114}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 51, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.03685439005494118}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 52, "compact_buffered": {"exact": false, "max_abs": 1.25, "nrmse": 0.03824616223573685}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 53, "compact_buffered": {"exact": false, "max_abs": 1.25, "nrmse": 0.03884580358862877}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 54, "compact_buffered": {"exact": false, "max_abs": 1.5, "nrmse": 0.0366680882871151}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 55, "compact_buffered": {"exact": false, "max_abs": 1.25, "nrmse": 0.03784633055329323}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 56, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.03907844051718712}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 57, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.03902671858668327}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 58, "compact_buffered": {"exact": false, "max_abs": 0.59375, "nrmse": 0.03654983639717102}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 59, "compact_buffered": {"exact": false, "max_abs": 0.625, "nrmse": 0.038602314889431}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 60, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.03578142821788788}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 61, "compact_buffered": {"exact": false, "max_abs": 2.0, "nrmse": 0.037034206092357635}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 62, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.03700539842247963}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 63, "compact_buffered": {"exact": false, "max_abs": 2.5, "nrmse": 0.041409216821193695}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}], "state_diff": []} -{"phase": "decode", "case": "gsm8k/test/424", "step": 3, "teacher_token": 2702, "compact_buffered": {"exact": false, "max_abs": 0.46875, "nrmse": 0.021444527432322502}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [1324, 1324, 1324], "layer_diff": [{"layer": 36, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.005857264623045921}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 37, "compact_buffered": {"exact": false, "max_abs": 0.0625, "nrmse": 0.009698164649307728}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 38, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.01222863420844078}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 39, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.015225925482809544}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 40, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.017336057499051094}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 41, "compact_buffered": {"exact": false, "max_abs": 0.15625, "nrmse": 0.017838697880506516}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 42, "compact_buffered": {"exact": false, "max_abs": 0.1875, "nrmse": 0.017482690513134003}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 43, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.0188731886446476}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 44, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.02043158933520317}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 45, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.02102850377559662}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 46, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.021510647609829903}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 47, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.023520689457654953}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 48, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.025951111689209938}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 49, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.025944150984287262}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 50, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.025842957198619843}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 51, "compact_buffered": {"exact": false, "max_abs": 0.3125, "nrmse": 0.02821602299809456}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 52, "compact_buffered": {"exact": false, "max_abs": 0.3125, "nrmse": 0.030199483036994934}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 53, "compact_buffered": {"exact": false, "max_abs": 0.75, "nrmse": 0.03216719254851341}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 54, "compact_buffered": {"exact": false, "max_abs": 0.75, "nrmse": 0.03037448599934578}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 55, "compact_buffered": {"exact": false, "max_abs": 0.75, "nrmse": 0.03145371377468109}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 56, "compact_buffered": {"exact": false, "max_abs": 0.75, "nrmse": 0.03282389044761658}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 57, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.03311885893344879}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 58, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.03244725242257118}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 59, "compact_buffered": {"exact": false, "max_abs": 0.515625, "nrmse": 0.03398338332772255}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 60, "compact_buffered": {"exact": false, "max_abs": 1.5, "nrmse": 0.03262050822377205}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 61, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.03329955041408539}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 62, "compact_buffered": {"exact": false, "max_abs": 2.0, "nrmse": 0.033797282725572586}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 63, "compact_buffered": {"exact": false, "max_abs": 5.0, "nrmse": 0.041649676859378815}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}], "state_diff": []} -{"phase": "decode", "case": "gsm8k/test/424", "step": 4, "teacher_token": 1324, "compact_buffered": {"exact": false, "max_abs": 0.375, "nrmse": 0.015972768887877464}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [314, 314, 314], "layer_diff": [{"layer": 36, "compact_buffered": {"exact": false, "max_abs": 0.03125, "nrmse": 0.00594170531257987}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 37, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.010557836852967739}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 38, "compact_buffered": {"exact": false, "max_abs": 0.125, "nrmse": 0.012074205093085766}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 39, "compact_buffered": {"exact": false, "max_abs": 0.125, "nrmse": 0.016733717173337936}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 40, "compact_buffered": {"exact": false, "max_abs": 0.125, "nrmse": 0.01886039227247238}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 41, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.020372360944747925}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 42, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.021234968677163124}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 43, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.022323016077280045}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 44, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.02304401621222496}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 45, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.023489011451601982}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 46, "compact_buffered": {"exact": false, "max_abs": 0.15625, "nrmse": 0.024281123653054237}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 47, "compact_buffered": {"exact": false, "max_abs": 0.169921875, "nrmse": 0.025426309555768967}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 48, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.028356734663248062}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 49, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.029207896441221237}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 50, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.030683699995279312}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 51, "compact_buffered": {"exact": false, "max_abs": 0.296875, "nrmse": 0.033003076910972595}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 52, "compact_buffered": {"exact": false, "max_abs": 0.359375, "nrmse": 0.03328215703368187}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 53, "compact_buffered": {"exact": false, "max_abs": 0.375, "nrmse": 0.03335943445563316}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 54, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.03177928552031517}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 55, "compact_buffered": {"exact": false, "max_abs": 0.3671875, "nrmse": 0.032436270266771317}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 56, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.032857075333595276}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 57, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.03271995484828949}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 58, "compact_buffered": {"exact": false, "max_abs": 0.53125, "nrmse": 0.029944749549031258}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 59, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.029003864154219627}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 60, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.02892889268696308}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 61, "compact_buffered": {"exact": false, "max_abs": 0.625, "nrmse": 0.027917325496673584}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 62, "compact_buffered": {"exact": false, "max_abs": 0.6875, "nrmse": 0.03205031529068947}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 63, "compact_buffered": {"exact": false, "max_abs": 10.0, "nrmse": 0.03787039965391159}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}], "state_diff": []} -{"phase": "decode", "case": "gsm8k/test/424", "step": 5, "teacher_token": 314, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.03167033940553665}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [7998, 7998, 7998], "layer_diff": [{"layer": 36, "compact_buffered": {"exact": false, "max_abs": 0.0234375, "nrmse": 0.005269130691885948}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 37, "compact_buffered": {"exact": false, "max_abs": 0.0625, "nrmse": 0.009972062893211842}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 38, "compact_buffered": {"exact": false, "max_abs": 0.0390625, "nrmse": 0.012603159062564373}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 39, "compact_buffered": {"exact": false, "max_abs": 0.07421875, "nrmse": 0.01826065592467785}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 40, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.02114355005323887}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 41, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.023318737745285034}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 42, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.023670053109526634}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 43, "compact_buffered": {"exact": false, "max_abs": 0.1171875, "nrmse": 0.026957685127854347}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 44, "compact_buffered": {"exact": false, "max_abs": 0.1298828125, "nrmse": 0.030695343390107155}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 45, "compact_buffered": {"exact": false, "max_abs": 0.162109375, "nrmse": 0.03218967840075493}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 46, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.037444453686475754}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 47, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.036016643047332764}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 48, "compact_buffered": {"exact": false, "max_abs": 1.5, "nrmse": 0.03811931237578392}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 49, "compact_buffered": {"exact": false, "max_abs": 1.5, "nrmse": 0.043484196066856384}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 50, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.045388925820589066}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 51, "compact_buffered": {"exact": false, "max_abs": 0.3671875, "nrmse": 0.044342853128910065}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 52, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.044209472835063934}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 53, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.045242443680763245}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 54, "compact_buffered": {"exact": false, "max_abs": 1.75, "nrmse": 0.042860474437475204}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 55, "compact_buffered": {"exact": false, "max_abs": 1.5, "nrmse": 0.042069919407367706}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 56, "compact_buffered": {"exact": false, "max_abs": 2.0, "nrmse": 0.04198922961950302}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 57, "compact_buffered": {"exact": false, "max_abs": 2.0, "nrmse": 0.041471634060144424}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 58, "compact_buffered": {"exact": false, "max_abs": 1.5, "nrmse": 0.03862380608916283}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 59, "compact_buffered": {"exact": false, "max_abs": 0.65625, "nrmse": 0.037934862077236176}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 60, "compact_buffered": {"exact": false, "max_abs": 3.0, "nrmse": 0.03705381974577904}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 61, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.03476737439632416}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 62, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.03626803308725357}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 63, "compact_buffered": {"exact": false, "max_abs": 8.0, "nrmse": 0.04833865165710449}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}], "state_diff": []} -{"phase": "decode", "case": "gsm8k/test/424", "step": 6, "teacher_token": 7998, "compact_buffered": {"exact": false, "max_abs": 0.40625, "nrmse": 0.019859177991747856}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [421, 421, 421], "layer_diff": [{"layer": 36, "compact_buffered": {"exact": false, "max_abs": 0.03125, "nrmse": 0.005416165105998516}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 37, "compact_buffered": {"exact": false, "max_abs": 0.0625, "nrmse": 0.008834417909383774}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 38, "compact_buffered": {"exact": false, "max_abs": 0.0546875, "nrmse": 0.01071897428482771}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 39, "compact_buffered": {"exact": false, "max_abs": 0.0703125, "nrmse": 0.013888556510210037}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 40, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.016260342672467232}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 41, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.017351221293210983}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 42, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.01832660287618637}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 43, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.020870687440037727}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 44, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.02189326286315918}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 45, "compact_buffered": {"exact": false, "max_abs": 0.125, "nrmse": 0.021722877398133278}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 46, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.023409344255924225}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 47, "compact_buffered": {"exact": false, "max_abs": 0.1875, "nrmse": 0.02568940818309784}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 48, "compact_buffered": {"exact": false, "max_abs": 0.203125, "nrmse": 0.028154324740171432}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 49, "compact_buffered": {"exact": false, "max_abs": 0.27734375, "nrmse": 0.03044147789478302}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 50, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.03140253573656082}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 51, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.03321586549282074}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 52, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.033724378794431686}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 53, "compact_buffered": {"exact": false, "max_abs": 0.375, "nrmse": 0.034092340618371964}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 54, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.03373667597770691}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 55, "compact_buffered": {"exact": false, "max_abs": 0.75, "nrmse": 0.0336761474609375}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 56, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.03294450044631958}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 57, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.032394200563430786}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 58, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.03360024839639664}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 59, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.03382017835974693}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 60, "compact_buffered": {"exact": false, "max_abs": 1.5, "nrmse": 0.033255353569984436}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 61, "compact_buffered": {"exact": false, "max_abs": 0.5625, "nrmse": 0.031908996403217316}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 62, "compact_buffered": {"exact": false, "max_abs": 1.5, "nrmse": 0.03347127139568329}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 63, "compact_buffered": {"exact": false, "max_abs": 1.125, "nrmse": 0.038032133132219315}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}], "state_diff": []} -{"phase": "decode", "case": "gsm8k/test/424", "step": 7, "teacher_token": 421, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.01860646903514862}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [628, 628, 628], "layer_diff": [{"layer": 36, "compact_buffered": {"exact": false, "max_abs": 0.03125, "nrmse": 0.006132100243121386}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 37, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.011140950955450535}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 38, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.013274221681058407}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 39, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.018375961109995842}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 40, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.018968408927321434}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 41, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.02142685279250145}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 42, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.022730493918061256}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 43, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.02544895000755787}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 44, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.02646820992231369}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 45, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.027089500799775124}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 46, "compact_buffered": {"exact": false, "max_abs": 0.171875, "nrmse": 0.02906535193324089}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 47, "compact_buffered": {"exact": false, "max_abs": 0.17431640625, "nrmse": 0.030393190681934357}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 48, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.03312614932656288}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 49, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.03576743230223656}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 50, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.04062994569540024}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 51, "compact_buffered": {"exact": false, "max_abs": 0.75, "nrmse": 0.044214047491550446}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 52, "compact_buffered": {"exact": false, "max_abs": 0.75, "nrmse": 0.04441540315747261}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 53, "compact_buffered": {"exact": false, "max_abs": 0.75, "nrmse": 0.044887568801641464}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 54, "compact_buffered": {"exact": false, "max_abs": 1.75, "nrmse": 0.04128674417734146}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 55, "compact_buffered": {"exact": false, "max_abs": 2.5, "nrmse": 0.042607277631759644}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 56, "compact_buffered": {"exact": false, "max_abs": 2.75, "nrmse": 0.042433880269527435}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 57, "compact_buffered": {"exact": false, "max_abs": 2.5, "nrmse": 0.04149830341339111}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 58, "compact_buffered": {"exact": false, "max_abs": 2.0, "nrmse": 0.03905279189348221}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 59, "compact_buffered": {"exact": false, "max_abs": 2.0, "nrmse": 0.03970376029610634}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 60, "compact_buffered": {"exact": false, "max_abs": 2.0, "nrmse": 0.03662055358290672}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 61, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.03558027371764183}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 62, "compact_buffered": {"exact": false, "max_abs": 0.6875, "nrmse": 0.037708621472120285}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 63, "compact_buffered": {"exact": false, "max_abs": 1.5, "nrmse": 0.040384627878665924}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}], "state_diff": []} -{"phase": "decode", "case": "gsm8k/test/424", "step": 8, "teacher_token": 628, "compact_buffered": {"exact": false, "max_abs": 0.65625, "nrmse": 0.02266240492463112}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [4973, 4973, 4973], "layer_diff": [{"layer": 36, "compact_buffered": {"exact": false, "max_abs": 0.03125, "nrmse": 0.005554413888603449}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 37, "compact_buffered": {"exact": false, "max_abs": 0.05078125, "nrmse": 0.010291719809174538}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 38, "compact_buffered": {"exact": false, "max_abs": 0.06640625, "nrmse": 0.013310067355632782}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 39, "compact_buffered": {"exact": false, "max_abs": 0.09375, "nrmse": 0.017081845551729202}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 40, "compact_buffered": {"exact": false, "max_abs": 0.125, "nrmse": 0.018781069666147232}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 41, "compact_buffered": {"exact": false, "max_abs": 0.15625, "nrmse": 0.01947009190917015}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 42, "compact_buffered": {"exact": false, "max_abs": 0.1875, "nrmse": 0.020696215331554413}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 43, "compact_buffered": {"exact": false, "max_abs": 0.15625, "nrmse": 0.022585036233067513}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 44, "compact_buffered": {"exact": false, "max_abs": 0.1875, "nrmse": 0.022774072363972664}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 45, "compact_buffered": {"exact": false, "max_abs": 0.15625, "nrmse": 0.022785257548093796}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 46, "compact_buffered": {"exact": false, "max_abs": 0.1640625, "nrmse": 0.024264350533485413}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 47, "compact_buffered": {"exact": false, "max_abs": 0.21875, "nrmse": 0.026479482650756836}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 48, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.028272781521081924}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 49, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.02871895767748356}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 50, "compact_buffered": {"exact": false, "max_abs": 1.75, "nrmse": 0.03419705852866173}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 51, "compact_buffered": {"exact": false, "max_abs": 1.75, "nrmse": 0.03643788397312164}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 52, "compact_buffered": {"exact": false, "max_abs": 1.75, "nrmse": 0.03597934544086456}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 53, "compact_buffered": {"exact": false, "max_abs": 2.25, "nrmse": 0.036586958914995193}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 54, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.03376131132245064}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 55, "compact_buffered": {"exact": false, "max_abs": 1.5, "nrmse": 0.03489164263010025}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 56, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.0347992405295372}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 57, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.03375546261668205}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 58, "compact_buffered": {"exact": false, "max_abs": 0.5625, "nrmse": 0.03337429463863373}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 59, "compact_buffered": {"exact": false, "max_abs": 1.5, "nrmse": 0.03440816327929497}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 60, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.0316251739859581}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 61, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.03146897628903389}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 62, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.03161400184035301}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 63, "compact_buffered": {"exact": false, "max_abs": 4.0, "nrmse": 0.040216416120529175}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}], "state_diff": []} -{"phase": "decode", "case": "gsm8k/test/424", "step": 9, "teacher_token": 4973, "compact_buffered": {"exact": false, "max_abs": 0.4375, "nrmse": 0.02100093849003315}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [279, 279, 279], "layer_diff": [{"layer": 36, "compact_buffered": {"exact": false, "max_abs": 0.03125, "nrmse": 0.0040286933071911335}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 37, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.008947975933551788}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 38, "compact_buffered": {"exact": false, "max_abs": 0.0625, "nrmse": 0.010812376625835896}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 39, "compact_buffered": {"exact": false, "max_abs": 0.09375, "nrmse": 0.014051037840545177}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 40, "compact_buffered": {"exact": false, "max_abs": 0.09375, "nrmse": 0.015194041654467583}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 41, "compact_buffered": {"exact": false, "max_abs": 0.09375, "nrmse": 0.015775809064507484}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 42, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.016694970428943634}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 43, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.018743963912129402}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 44, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.019794989377260208}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 45, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.020452730357646942}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 46, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.022806327790021896}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 47, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.0236266627907753}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 48, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.026123525574803352}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 49, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.025906456634402275}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 50, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.029017720371484756}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 51, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.031157974153757095}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 52, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.03223520144820213}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 53, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.0316862054169178}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 54, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.030200494453310966}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 55, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.03143785148859024}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 56, "compact_buffered": {"exact": false, "max_abs": 0.5625, "nrmse": 0.032983794808387756}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 57, "compact_buffered": {"exact": false, "max_abs": 0.75, "nrmse": 0.03205791488289833}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 58, "compact_buffered": {"exact": false, "max_abs": 0.75, "nrmse": 0.03207436576485634}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 59, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.029867742210626602}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 60, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.030679786577820778}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 61, "compact_buffered": {"exact": false, "max_abs": 1.5, "nrmse": 0.029186036437749863}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 62, "compact_buffered": {"exact": false, "max_abs": 1.5, "nrmse": 0.03362181782722473}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 63, "compact_buffered": {"exact": false, "max_abs": 6.0, "nrmse": 0.03511303663253784}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}], "state_diff": []} -{"phase": "decode", "case": "gsm8k/test/424", "step": 10, "teacher_token": 279, "compact_buffered": {"exact": false, "max_abs": 2.28125, "nrmse": 0.0697738379240036}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [1834, 1834, 1834], "layer_diff": [{"layer": 36, "compact_buffered": {"exact": false, "max_abs": 0.017578125, "nrmse": 0.004533097613602877}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 37, "compact_buffered": {"exact": false, "max_abs": 0.033203125, "nrmse": 0.009889349341392517}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 38, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.01257106103003025}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 39, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.016805727034807205}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 40, "compact_buffered": {"exact": false, "max_abs": 0.0849609375, "nrmse": 0.01858266070485115}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 41, "compact_buffered": {"exact": false, "max_abs": 0.07421875, "nrmse": 0.02001667208969593}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 42, "compact_buffered": {"exact": false, "max_abs": 0.09375, "nrmse": 0.02001698687672615}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 43, "compact_buffered": {"exact": false, "max_abs": 0.125, "nrmse": 0.02404976636171341}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 44, "compact_buffered": {"exact": false, "max_abs": 0.15625, "nrmse": 0.025715259835124016}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 45, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.027685469016432762}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 46, "compact_buffered": {"exact": false, "max_abs": 0.1875, "nrmse": 0.028108909726142883}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 47, "compact_buffered": {"exact": false, "max_abs": 0.21875, "nrmse": 0.029126770794391632}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 48, "compact_buffered": {"exact": false, "max_abs": 0.3125, "nrmse": 0.03392261639237404}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 49, "compact_buffered": {"exact": false, "max_abs": 1.5, "nrmse": 0.058811429888010025}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 50, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.09662257879972458}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 51, "compact_buffered": {"exact": false, "max_abs": 1.375, "nrmse": 0.13217252492904663}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 52, "compact_buffered": {"exact": false, "max_abs": 2.0, "nrmse": 0.15010808408260345}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 53, "compact_buffered": {"exact": false, "max_abs": 2.25, "nrmse": 0.17399583756923676}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 54, "compact_buffered": {"exact": false, "max_abs": 13.5, "nrmse": 0.21187230944633484}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 55, "compact_buffered": {"exact": false, "max_abs": 9.0, "nrmse": 0.21502839028835297}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 56, "compact_buffered": {"exact": false, "max_abs": 13.0, "nrmse": 0.2312212437391281}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 57, "compact_buffered": {"exact": false, "max_abs": 18.0, "nrmse": 0.23784108459949493}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 58, "compact_buffered": {"exact": false, "max_abs": 21.0, "nrmse": 0.22308818995952606}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 59, "compact_buffered": {"exact": false, "max_abs": 7.5, "nrmse": 0.21472252905368805}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 60, "compact_buffered": {"exact": false, "max_abs": 15.0, "nrmse": 0.1804974228143692}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 61, "compact_buffered": {"exact": false, "max_abs": 9.0, "nrmse": 0.1687784194946289}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 62, "compact_buffered": {"exact": false, "max_abs": 9.0, "nrmse": 0.1513976752758026}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 63, "compact_buffered": {"exact": false, "max_abs": 30.0, "nrmse": 0.16640545427799225}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}], "state_diff": []} -{"phase": "decode", "case": "gsm8k/test/424", "step": 11, "teacher_token": 1834, "compact_buffered": {"exact": false, "max_abs": 0.66796875, "nrmse": 0.01849493570625782}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [8151, 8151, 8151], "layer_diff": [{"layer": 36, "compact_buffered": {"exact": false, "max_abs": 0.017578125, "nrmse": 0.005154166836291552}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 37, "compact_buffered": {"exact": false, "max_abs": 0.04296875, "nrmse": 0.010513018816709518}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 38, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.013927087187767029}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 39, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.017796609550714493}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 40, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.02008376456797123}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 41, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.021791480481624603}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 42, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.022558309137821198}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 43, "compact_buffered": {"exact": false, "max_abs": 0.1875, "nrmse": 0.026435550302267075}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 44, "compact_buffered": {"exact": false, "max_abs": 0.1875, "nrmse": 0.028176525607705116}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 45, "compact_buffered": {"exact": false, "max_abs": 0.1875, "nrmse": 0.02899898588657379}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 46, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.03205467760562897}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 47, "compact_buffered": {"exact": false, "max_abs": 0.1875, "nrmse": 0.032463621348142624}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 48, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.03636416420340538}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 49, "compact_buffered": {"exact": false, "max_abs": 0.265625, "nrmse": 0.04306873679161072}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 50, "compact_buffered": {"exact": false, "max_abs": 0.375, "nrmse": 0.05071976035833359}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 51, "compact_buffered": {"exact": false, "max_abs": 0.40625, "nrmse": 0.051728472113609314}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 52, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.05413447320461273}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 53, "compact_buffered": {"exact": false, "max_abs": 0.875, "nrmse": 0.05591907724738121}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 54, "compact_buffered": {"exact": false, "max_abs": 1.5, "nrmse": 0.050132766366004944}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 55, "compact_buffered": {"exact": false, "max_abs": 0.517578125, "nrmse": 0.050995804369449615}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 56, "compact_buffered": {"exact": false, "max_abs": 0.625, "nrmse": 0.05478179454803467}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 57, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.05387504771351814}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 58, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.0504019632935524}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 59, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.048536937683820724}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 60, "compact_buffered": {"exact": false, "max_abs": 0.875, "nrmse": 0.04997668042778969}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 61, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.046865273267030716}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 62, "compact_buffered": {"exact": false, "max_abs": 1.0625, "nrmse": 0.05137123912572861}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 63, "compact_buffered": {"exact": false, "max_abs": 16.0, "nrmse": 0.05501401424407959}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}], "state_diff": []} -{"phase": "decode", "case": "gsm8k/test/424", "step": 12, "teacher_token": 8151, "compact_buffered": {"exact": false, "max_abs": 0.40625, "nrmse": 0.020681021735072136}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [11, 11, 11], "layer_diff": [{"layer": 36, "compact_buffered": {"exact": false, "max_abs": 0.03125, "nrmse": 0.005567422602325678}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 37, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.010099058970808983}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 38, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.013319087214767933}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 39, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.016106029972434044}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 40, "compact_buffered": {"exact": false, "max_abs": 0.08984375, "nrmse": 0.01653122343122959}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 41, "compact_buffered": {"exact": false, "max_abs": 0.1015625, "nrmse": 0.0174028929322958}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 42, "compact_buffered": {"exact": false, "max_abs": 0.09375, "nrmse": 0.018393080681562424}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 43, "compact_buffered": {"exact": false, "max_abs": 0.1171875, "nrmse": 0.019646773114800453}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 44, "compact_buffered": {"exact": false, "max_abs": 0.125, "nrmse": 0.020728396251797676}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 45, "compact_buffered": {"exact": false, "max_abs": 0.15625, "nrmse": 0.022330541163682938}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 46, "compact_buffered": {"exact": false, "max_abs": 0.1484375, "nrmse": 0.025097161531448364}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 47, "compact_buffered": {"exact": false, "max_abs": 0.1875, "nrmse": 0.025653913617134094}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 48, "compact_buffered": {"exact": false, "max_abs": 0.21875, "nrmse": 0.02939108945429325}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 49, "compact_buffered": {"exact": false, "max_abs": 0.234375, "nrmse": 0.03333212062716484}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 50, "compact_buffered": {"exact": false, "max_abs": 0.75, "nrmse": 0.03707246109843254}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 51, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.03641389682888985}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 52, "compact_buffered": {"exact": false, "max_abs": 0.37109375, "nrmse": 0.03705570846796036}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 53, "compact_buffered": {"exact": false, "max_abs": 0.39453125, "nrmse": 0.03861463442444801}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 54, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.039400868117809296}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 55, "compact_buffered": {"exact": false, "max_abs": 0.4375, "nrmse": 0.0392465777695179}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 56, "compact_buffered": {"exact": false, "max_abs": 1.3125, "nrmse": 0.04117510840296745}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 57, "compact_buffered": {"exact": false, "max_abs": 0.875, "nrmse": 0.04116159304976463}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 58, "compact_buffered": {"exact": false, "max_abs": 0.625, "nrmse": 0.04184285178780556}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 59, "compact_buffered": {"exact": false, "max_abs": 0.59375, "nrmse": 0.03698941320180893}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 60, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.03965001925826073}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 61, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.03843707963824272}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 62, "compact_buffered": {"exact": false, "max_abs": 1.03125, "nrmse": 0.041994038969278336}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 63, "compact_buffered": {"exact": false, "max_abs": 8.0, "nrmse": 0.04064470902085304}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}], "state_diff": []} -{"phase": "decode", "case": "gsm8k/test/424", "step": 13, "teacher_token": 11, "compact_buffered": {"exact": false, "max_abs": 0.375, "nrmse": 0.023442843928933144}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [567, 567, 567], "layer_diff": [{"layer": 36, "compact_buffered": {"exact": false, "max_abs": 0.03125, "nrmse": 0.004116564989089966}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 37, "compact_buffered": {"exact": false, "max_abs": 0.048828125, "nrmse": 0.010274237021803856}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 38, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.013264822773635387}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 39, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.015930023044347763}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 40, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.01740264892578125}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 41, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.019058873876929283}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 42, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.018856078386306763}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 43, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.022295288741588593}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 44, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.024521060287952423}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 45, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.026159116998314857}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 46, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.026793107390403748}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 47, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.027372589334845543}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 48, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.03024895302951336}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 49, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.03158486261963844}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 50, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.033872056752443314}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 51, "compact_buffered": {"exact": false, "max_abs": 0.75, "nrmse": 0.03619193658232689}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 52, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.039070744067430496}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 53, "compact_buffered": {"exact": false, "max_abs": 1.625, "nrmse": 0.041919853538274765}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 54, "compact_buffered": {"exact": false, "max_abs": 1.5, "nrmse": 0.040716394782066345}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 55, "compact_buffered": {"exact": false, "max_abs": 1.5, "nrmse": 0.04210279509425163}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 56, "compact_buffered": {"exact": false, "max_abs": 2.375, "nrmse": 0.04318403825163841}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 57, "compact_buffered": {"exact": false, "max_abs": 2.25, "nrmse": 0.04335274547338486}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 58, "compact_buffered": {"exact": false, "max_abs": 2.0, "nrmse": 0.03966652974486351}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 59, "compact_buffered": {"exact": false, "max_abs": 4.0, "nrmse": 0.04356882721185684}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 60, "compact_buffered": {"exact": false, "max_abs": 2.5, "nrmse": 0.040180377662181854}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 61, "compact_buffered": {"exact": false, "max_abs": 3.0, "nrmse": 0.04049080237746239}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 62, "compact_buffered": {"exact": false, "max_abs": 2.0, "nrmse": 0.03858669847249985}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 63, "compact_buffered": {"exact": false, "max_abs": 2.0, "nrmse": 0.037129949778318405}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}], "state_diff": []} -{"phase": "decode", "case": "gsm8k/test/424", "step": 14, "teacher_token": 567, "compact_buffered": {"exact": false, "max_abs": 0.625, "nrmse": 0.025705164298415184}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [1144, 1144, 1144], "layer_diff": [{"layer": 36, "compact_buffered": {"exact": false, "max_abs": 0.017578125, "nrmse": 0.004214361310005188}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 37, "compact_buffered": {"exact": false, "max_abs": 0.0390625, "nrmse": 0.009674521163105965}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 38, "compact_buffered": {"exact": false, "max_abs": 0.0625, "nrmse": 0.012589294463396072}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 39, "compact_buffered": {"exact": false, "max_abs": 0.0625, "nrmse": 0.015898849815130234}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 40, "compact_buffered": {"exact": false, "max_abs": 0.078125, "nrmse": 0.017503997310996056}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 41, "compact_buffered": {"exact": false, "max_abs": 0.0830078125, "nrmse": 0.019246412441134453}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 42, "compact_buffered": {"exact": false, "max_abs": 0.1015625, "nrmse": 0.01923968270421028}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 43, "compact_buffered": {"exact": false, "max_abs": 0.140625, "nrmse": 0.020764263346791267}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 44, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.02243914268910885}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 45, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.023843474686145782}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 46, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.025149064138531685}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 47, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.026229577139019966}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 48, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.027647394686937332}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 49, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.030065318569540977}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 50, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.03139353543519974}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 51, "compact_buffered": {"exact": false, "max_abs": 0.375, "nrmse": 0.03142969310283661}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 52, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.03463907539844513}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 53, "compact_buffered": {"exact": false, "max_abs": 0.390625, "nrmse": 0.03702929615974426}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 54, "compact_buffered": {"exact": false, "max_abs": 0.4140625, "nrmse": 0.036779776215553284}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 55, "compact_buffered": {"exact": false, "max_abs": 0.484375, "nrmse": 0.03845982253551483}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 56, "compact_buffered": {"exact": false, "max_abs": 0.78125, "nrmse": 0.038922179490327835}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 57, "compact_buffered": {"exact": false, "max_abs": 0.53125, "nrmse": 0.0381280817091465}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 58, "compact_buffered": {"exact": false, "max_abs": 1.75, "nrmse": 0.038655925542116165}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 59, "compact_buffered": {"exact": false, "max_abs": 1.125, "nrmse": 0.03868322819471359}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 60, "compact_buffered": {"exact": false, "max_abs": 1.5, "nrmse": 0.03634640574455261}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 61, "compact_buffered": {"exact": false, "max_abs": 1.5, "nrmse": 0.03638196364045143}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 62, "compact_buffered": {"exact": false, "max_abs": 4.0, "nrmse": 0.039071690291166306}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 63, "compact_buffered": {"exact": false, "max_abs": 2.0, "nrmse": 0.03761620074510574}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}], "state_diff": []} -{"phase": "decode", "case": "gsm8k/test/424", "step": 15, "teacher_token": 1144, "compact_buffered": {"exact": false, "max_abs": 0.4375, "nrmse": 0.01926518976688385}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [310, 310, 310], "layer_diff": [{"layer": 36, "compact_buffered": {"exact": false, "max_abs": 0.03125, "nrmse": 0.0044249556958675385}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 37, "compact_buffered": {"exact": false, "max_abs": 0.09375, "nrmse": 0.009469624608755112}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 38, "compact_buffered": {"exact": false, "max_abs": 0.09375, "nrmse": 0.01184225082397461}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 39, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.015874620527029037}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 40, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.017191974446177483}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 41, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.018334856256842613}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 42, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.018283767625689507}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 43, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.01931038312613964}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 44, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.020621107891201973}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 45, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.023343132808804512}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 46, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.023303894326090813}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 47, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.02483632043004036}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 48, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.026323998346924782}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 49, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.02762913703918457}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 50, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.030338158831000328}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 51, "compact_buffered": {"exact": false, "max_abs": 0.375, "nrmse": 0.030881624668836594}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 52, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.032304637134075165}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 53, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.034970104694366455}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 54, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.03435663878917694}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 55, "compact_buffered": {"exact": false, "max_abs": 0.625, "nrmse": 0.03545057401061058}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 56, "compact_buffered": {"exact": false, "max_abs": 0.75, "nrmse": 0.03599783405661583}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 57, "compact_buffered": {"exact": false, "max_abs": 0.75, "nrmse": 0.036764372140169144}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 58, "compact_buffered": {"exact": false, "max_abs": 0.75, "nrmse": 0.03671498969197273}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 59, "compact_buffered": {"exact": false, "max_abs": 2.0, "nrmse": 0.03429407998919487}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 60, "compact_buffered": {"exact": false, "max_abs": 2.5, "nrmse": 0.034483857452869415}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 61, "compact_buffered": {"exact": false, "max_abs": 2.0, "nrmse": 0.0332176573574543}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 62, "compact_buffered": {"exact": false, "max_abs": 3.0, "nrmse": 0.03869050741195679}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 63, "compact_buffered": {"exact": false, "max_abs": 8.0, "nrmse": 0.03535183146595955}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}], "state_diff": []} -{"phase": "complete", "scope": "diagnostic_only", "verdict": "INCONCLUSIVE", "seconds": 77.0561493668938} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/graph-demotion-1955.json b/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/graph-demotion-1955.json deleted file mode 100644 index a4597e875..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/graph-demotion-1955.json +++ /dev/null @@ -1,428 +0,0 @@ -{ - "campaign_verdict": "INCONCLUSIVE", - "execution": "completed", - "job": "1955", - "candidate": "80e5108e44c43f3f54fabb5723e6154d18088516", - "base": "7ec5d224e983e437530355b94b0d6d08153bc27d", - "observer_sha256": "883c8b871df6837d7a04f7c712b6b606003980ec7ce09b41d71c884c6e9bc3e4", - "client_sha256": "f31fae8c573ecc4bf8e31dae03823502e71404333e8111f4b535432758302b29", - "allocated_gpu_seconds": 464, - "readiness_used_gpu_seconds": 839, - "approved_readiness_gpu_seconds": 7200, - "approved_total_gpu_hours": 44, - "initial_separate_allowance_used_gpu_seconds": 147, - "exit_code": 0, - "allocation_released": true, - "source_hashes_verified": true, - "previous_request_prefixes_preserved": [ - true, - true, - true, - true - ], - "requests": [ - { - "phase": "request", - "arm": "graph", - "case": "gsm8k/test/783", - "index": 0, - "cap": 2048, - "ignore_eos": false, - "prompt_tokens": 149, - "prompt_sha256": "f2233be4ebb7b122d80433fe4081fdabeda64f735b7bc24f9706c4c4339b824f", - "tokens": 529, - "terminal": [ - "stop" - ], - "usage": [ - { - "prompt_tokens": 149, - "completion_tokens": 529, - "total_tokens": 678 - } - ], - "output_text_sha256": "da51da33ffc217b8e774f28fe0fe3488b2710e0a9f9cb7ce4f315746a9b53d4f", - "wire_sha256": "3fc37ed942e24519f74bdaaf4bbfc19d77a6f18cc49966b3431fa83dbd479858", - "client_success": true, - "client_error": false, - "valid": true, - "ttft_s": 2.6729599660029635, - "latency_s": 49.13746222201735, - "tpot_s": 0.08800095124245148, - "active_requests_after": 0, - "runner_state_count_after": 0, - "itl_count": 528, - "token_ids_sha256": "61f3bb0e25b687ee1f5f41c47834b2b92dca0804ab488a500d5177e75d0bac5c" - }, - { - "phase": "request", - "arm": "graph", - "case": "gsm8k/test/1", - "index": 1, - "cap": 2048, - "ignore_eos": false, - "prompt_tokens": 70, - "prompt_sha256": "78d335abe81363d4bb48e5aa256c120344c2ceadbcf8739afe4ec3923c747238", - "tokens": 169, - "terminal": [ - "stop" - ], - "usage": [ - { - "prompt_tokens": 70, - "completion_tokens": 169, - "total_tokens": 239 - } - ], - "output_text_sha256": "53ef84987096a7daa32212812c15c7fce023a5b06f50ea52a2c0e026d0a1f1e6", - "wire_sha256": "41a86d42bcbbf6e255b1649b99def36a7383cf9497296231e8f870069467a847", - "client_success": true, - "client_error": false, - "valid": true, - "ttft_s": 1.3001440840307623, - "latency_s": 16.00403310393449, - "tpot_s": 0.08752314892799837, - "active_requests_after": 0, - "runner_state_count_after": 0, - "itl_count": 168, - "token_ids_sha256": "f7a498c4f29d1e6090babac836d7d128792a3e96300c81603ec90897fc82197a" - }, - { - "phase": "request", - "arm": "graph", - "case": "gsm8k/test/424", - "index": 2, - "cap": 2048, - "ignore_eos": false, - "prompt_tokens": 106, - "prompt_sha256": "03cbdf5804bab62d4efbc0ae89e191bd557840c6d9a1a5ada9669de953a0b783", - "tokens": 314, - "terminal": [ - "stop" - ], - "usage": [ - { - "prompt_tokens": 106, - "completion_tokens": 314, - "total_tokens": 420 - } - ], - "output_text_sha256": "bb797c1732c29c467b45fd24a2a27f80a1a5d8ce8dc38799eb28b29a5c056bb2", - "wire_sha256": "09f9bc44738ef9408b9f966a39309048c838dc71b0e9d24e729985c4a76ed075", - "client_success": true, - "client_error": false, - "valid": true, - "ttft_s": 1.9258344959234819, - "latency_s": 29.31772259902209, - "tpot_s": 0.0875140194987176, - "active_requests_after": 0, - "runner_state_count_after": 0, - "itl_count": 313, - "token_ids_sha256": "c11b16c562ef23718ab2334d57ba00db3da4a3d6aea61e73f2150de294188e25" - }, - { - "phase": "request", - "arm": "graph", - "case": "gsm8k/test/783", - "index": 3, - "cap": 1024, - "ignore_eos": true, - "prompt_tokens": 149, - "prompt_sha256": "f2233be4ebb7b122d80433fe4081fdabeda64f735b7bc24f9706c4c4339b824f", - "tokens": 1024, - "terminal": [ - "length" - ], - "usage": [ - { - "prompt_tokens": 149, - "completion_tokens": 1024, - "total_tokens": 1173 - } - ], - "output_text_sha256": "5436ae0c73cb99b957af31c6f3c5b905d66d815fbc4de621f633ada46a410e99", - "wire_sha256": "14f697729b3953772422c0c26f12eb3048acf5f7147c4af17788e474b8cfb6e8", - "client_success": true, - "client_error": false, - "valid": true, - "ttft_s": 2.5463140930514783, - "latency_s": 92.73048542998731, - "tpot_s": 0.08815657022183365, - "active_requests_after": 0, - "runner_state_count_after": 0, - "itl_count": 1023, - "token_ids_sha256": "12f7253c80514735684c1edd3ec782117aafb78a1302090bb23e66ae8cecceb8" - }, - { - "phase": "request", - "arm": "graph", - "case": "gsm8k/test/783", - "index": 4, - "cap": 16, - "ignore_eos": true, - "prompt_tokens": 149, - "prompt_sha256": "f2233be4ebb7b122d80433fe4081fdabeda64f735b7bc24f9706c4c4339b824f", - "tokens": 16, - "terminal": [ - "length" - ], - "usage": [ - { - "prompt_tokens": 149, - "completion_tokens": 16, - "total_tokens": 165 - } - ], - "output_text_sha256": "4b5d6370a617c539db1e6ee0df2c726c7165c1e9395eab1b873ff700a997510a", - "wire_sha256": "dea4820b34797b96c746b40c23dbea2743b33e34377054957236105b2dea2c9b", - "client_success": true, - "client_error": false, - "valid": true, - "ttft_s": 2.7129198129987344, - "latency_s": 4.026040954980999, - "tpot_s": 0.08754140946548432, - "active_requests_after": 0, - "runner_state_count_after": 0, - "itl_count": 15, - "token_ids_sha256": "43751399c86b34aac41331b293081f6fd0a047ba3a9ed488b775ed7fb6dbf879" - }, - { - "phase": "request", - "arm": "eager", - "case": "gsm8k/test/783", - "index": 0, - "cap": 2048, - "ignore_eos": false, - "prompt_tokens": 149, - "prompt_sha256": "f2233be4ebb7b122d80433fe4081fdabeda64f735b7bc24f9706c4c4339b824f", - "tokens": 503, - "terminal": [ - "stop" - ], - "usage": [ - { - "prompt_tokens": 149, - "completion_tokens": 503, - "total_tokens": 652 - } - ], - "output_text_sha256": "6e7b3fd3ae3c464a07b9014602a22a121e1a85fda0d3458a42a21dbb66708bf0", - "wire_sha256": "0190c4550c6e071e9bef20f6725f24082bbb4d629b6fd257875334f9cbb3f814", - "client_success": true, - "client_error": false, - "valid": true, - "ttft_s": 3.526063673081808, - "latency_s": 49.761497988016345, - "tpot_s": 0.09210245879469031, - "active_requests_after": 0, - "runner_state_count_after": 0, - "itl_count": 502, - "token_ids_sha256": "b6303882caf4474c11beabffebea3597698b6f0c8443ad72f92b08cae980d700" - }, - { - "phase": "request", - "arm": "eager", - "case": "gsm8k/test/1", - "index": 1, - "cap": 2048, - "ignore_eos": false, - "prompt_tokens": 70, - "prompt_sha256": "78d335abe81363d4bb48e5aa256c120344c2ceadbcf8739afe4ec3923c747238", - "tokens": 169, - "terminal": [ - "stop" - ], - "usage": [ - { - "prompt_tokens": 70, - "completion_tokens": 169, - "total_tokens": 239 - } - ], - "output_text_sha256": "53ef84987096a7daa32212812c15c7fce023a5b06f50ea52a2c0e026d0a1f1e6", - "wire_sha256": "7eec63e87c579da1d1ba607db57ed9efe90417c1c6b639544d003f139c7a1cfc", - "client_success": true, - "client_error": false, - "valid": true, - "ttft_s": 1.2559849520912394, - "latency_s": 16.675945091992617, - "tpot_s": 0.09178547702322248, - "active_requests_after": 0, - "runner_state_count_after": 0, - "itl_count": 168, - "token_ids_sha256": "f7a498c4f29d1e6090babac836d7d128792a3e96300c81603ec90897fc82197a" - }, - { - "phase": "request", - "arm": "eager", - "case": "gsm8k/test/424", - "index": 2, - "cap": 2048, - "ignore_eos": false, - "prompt_tokens": 106, - "prompt_sha256": "03cbdf5804bab62d4efbc0ae89e191bd557840c6d9a1a5ada9669de953a0b783", - "tokens": 314, - "terminal": [ - "stop" - ], - "usage": [ - { - "prompt_tokens": 106, - "completion_tokens": 314, - "total_tokens": 420 - } - ], - "output_text_sha256": "bb797c1732c29c467b45fd24a2a27f80a1a5d8ce8dc38799eb28b29a5c056bb2", - "wire_sha256": "0a8d0c7369364d65e2191c6a882a29d49d4ba452cf1800802dbb21ad7e436e82", - "client_success": true, - "client_error": false, - "valid": true, - "ttft_s": 1.8804057410452515, - "latency_s": 30.63704265409615, - "tpot_s": 0.09187423933882076, - "active_requests_after": 0, - "runner_state_count_after": 0, - "itl_count": 313, - "token_ids_sha256": "c11b16c562ef23718ab2334d57ba00db3da4a3d6aea61e73f2150de294188e25" - }, - { - "phase": "request", - "arm": "eager", - "case": "gsm8k/test/783", - "index": 3, - "cap": 1024, - "ignore_eos": true, - "prompt_tokens": 149, - "prompt_sha256": "f2233be4ebb7b122d80433fe4081fdabeda64f735b7bc24f9706c4c4339b824f", - "tokens": 1024, - "terminal": [ - "length" - ], - "usage": [ - { - "prompt_tokens": 149, - "completion_tokens": 1024, - "total_tokens": 1173 - } - ], - "output_text_sha256": "14382258fbbcb4c8029f7e172c087a42d22b0f3164bbcf2bdfded406d0a8906e", - "wire_sha256": "7e7f486a2470afb9cd6295da071798d5ec40e6c1338cec7807b1177e349a95fd", - "client_success": true, - "client_error": false, - "valid": true, - "ttft_s": 2.543290780042298, - "latency_s": 96.65085916907992, - "tpot_s": 0.09199175795604851, - "active_requests_after": 0, - "runner_state_count_after": 0, - "itl_count": 1023, - "token_ids_sha256": "1146456c252cad5f25c0ecce1463917c39e3e066f7ce93a3429fa23dd330fc99" - }, - { - "phase": "request", - "arm": "eager", - "case": "gsm8k/test/783", - "index": 4, - "cap": 16, - "ignore_eos": true, - "prompt_tokens": 149, - "prompt_sha256": "f2233be4ebb7b122d80433fe4081fdabeda64f735b7bc24f9706c4c4339b824f", - "tokens": 16, - "terminal": [ - "length" - ], - "usage": [ - { - "prompt_tokens": 149, - "completion_tokens": 16, - "total_tokens": 165 - } - ], - "output_text_sha256": "4b5d6370a617c539db1e6ee0df2c726c7165c1e9395eab1b873ff700a997510a", - "wire_sha256": "a22bd0ab71fb0706feeb594c95dcd6e006c7559f7ccf8fb4aac9780d33970522", - "client_success": true, - "client_error": false, - "valid": true, - "ttft_s": 2.5469864449696615, - "latency_s": 3.927188378991559, - "tpot_s": 0.09201346226812651, - "active_requests_after": 0, - "runner_state_count_after": 0, - "itl_count": 15, - "token_ids_sha256": "43751399c86b34aac41331b293081f6fd0a047ba3a9ed488b775ed7fb6dbf879" - } - ], - "comparisons": [ - { - "phase": "comparison", - "case": "gsm8k/test/783", - "cap": 2048, - "ignore_eos": false, - "graph_tokens": 529, - "eager_tokens": 503, - "exact": false, - "first_difference": 81 - }, - { - "phase": "comparison", - "case": "gsm8k/test/1", - "cap": 2048, - "ignore_eos": false, - "graph_tokens": 169, - "eager_tokens": 169, - "exact": true, - "first_difference": null - }, - { - "phase": "comparison", - "case": "gsm8k/test/424", - "cap": 2048, - "ignore_eos": false, - "graph_tokens": 314, - "eager_tokens": 314, - "exact": true, - "first_difference": null - }, - { - "phase": "comparison", - "case": "gsm8k/test/783", - "cap": 1024, - "ignore_eos": true, - "graph_tokens": 1024, - "eager_tokens": 1024, - "exact": false, - "first_difference": 81 - }, - { - "phase": "comparison", - "case": "gsm8k/test/783", - "cap": 16, - "ignore_eos": true, - "graph_tokens": 16, - "eager_tokens": 16, - "exact": true, - "first_difference": null - } - ], - "cleanup": [ - { - "phase": "cleanup", - "arm": "graph", - "exit_code": 0, - "forced": false, - "closed": true, - "active_requests": 0, - "runner_state_count": 0 - }, - { - "phase": "cleanup", - "arm": "eager", - "exit_code": 0, - "forced": false, - "closed": true, - "active_requests": 0, - "runner_state_count": 0 - } - ], - "limits": "Bounded readiness only; fixed arm order and recorder overhead; no vLLM baseline, formal matrix, task scoring or long-context qualification. Numerical output differences retained, no parity or speedup claim." -} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/graph-demotion-1955.jsonl b/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/graph-demotion-1955.jsonl deleted file mode 100644 index ee4de0565..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/graph-demotion-1955.jsonl +++ /dev/null @@ -1,21 +0,0 @@ -{"phase": "protocol", "cases": [["gsm8k/test/783", 2048, false], ["gsm8k/test/1", 2048, false], ["gsm8k/test/424", 2048, false], ["gsm8k/test/783", 1024, true], ["gsm8k/test/783", 16, true]], "client_sha256": "f31fae8c573ecc4bf8e31dae03823502e71404333e8111f4b535432758302b29", "observer_sha256": "883c8b871df6837d7a04f7c712b6b606003980ec7ce09b41d71c884c6e9bc3e4", "fixture_sha256": "f359c66f92b69d9a92a66564d8a087809dbc45b7e8b9e916229065806c4c188c", "timing_scope": "readiness_only_official_function_with_response_chunk_recording", "source_hashes": {"nanovllm/qwen38/__init__.py": "1f07e0cfd12bbf370c1f465c6a4bd73e72a5c826af1f29e60ccc7d6f45562cfc", "nanovllm/qwen38/__main__.py": "9e2a0881eedf8fa4abc6037511b42fd3a44171d121a7b2cb096023f485673ba2", "nanovllm/qwen38/_triton_fp8.py": "9f0cb68be017cb3858aebe8ba275039084e305df92cb4567d6e7da54d119f19e", "nanovllm/qwen38/_triton_rms.py": "ba99fa3ba82bfcab4cd602fd0db907acad0eee7c7a63ab23d560a17fd2f72272", "nanovllm/qwen38/config.py": "002ea25bbf9c8d17e5f1d3fb2b76c400e4291d7d2d06d5dd13fb5c3ac15c3d79", "nanovllm/qwen38/engine.py": "c47e1c72fcc24bc8eeb5497887da6be11c2de567abcc3efb275b061c061fd755", "nanovllm/qwen38/graph.py": "d4213aa68599564a01ced3b990993c9b17482615c154120bc4180f78a5da6098", "nanovllm/qwen38/loader.py": "1aa6b3321bc135fa28d1eebf6223bbe3bc5f646630193e292fb58a9f0ca2763f", "nanovllm/qwen38/model.py": "ad8386f156ff5315dfd73de313ab766fedd9ef892c7b94cec95303f11fd4e198", "nanovllm/qwen38/quant.py": "5590e4769a0fbbcaf5afeee1a93db5c4bccacf09f6f3b34e42d52ec31e9f50c3", "nanovllm/qwen38/runner.py": "3b1f04b42949616bf7b8c62dd802f1707d694c0a14bfa129d24fd6612881a22f", "nanovllm/qwen38/server.py": "d2ccb1ccb61dbf06b22f7f199854d58fc7fbb246c6d8e8ec9d5f6344c3016494"}} -{"phase": "ready", "arm": "graph", "startup_s": 36.32131540903356, "versions": {"torch": "2.14.1+cu130", "transformers": "5.20.2", "triton": "3.8.0"}, "device": "NVIDIA H20-3e", "backend": "triton_block_fp8", "graph_prewarm": {"buckets": {"256": {"capture_seconds": 2.062512908014469, "resident_MiB": 229.5}, "1024": {"capture_seconds": 0.17641336400993168, "resident_MiB": 215.224609375}, "2048": {"capture_seconds": 0.17611596698407084, "resident_MiB": 276.224609375}, "4096": {"capture_seconds": 0.17968802899122238, "resident_MiB": 415.224609375}, "8448": {"capture_seconds": 0.1833303360035643, "resident_MiB": 685.224609375}}, "started_unix": 1792689183.5669756, "total_seconds": 2.778060604003258, "resident_MiB_total": 1821.3984375, "completed_unix": 1792689186.346515}} -{"phase": "request", "arm": "graph", "case": "gsm8k/test/783", "index": 0, "cap": 2048, "ignore_eos": false, "prompt_tokens": 149, "prompt_sha256": "f2233be4ebb7b122d80433fe4081fdabeda64f735b7bc24f9706c4c4339b824f", "token_ids": [1206, 11290, 411, 3377, 11, 567, 1144, 310, 7995, 279, 1324, 314, 4022, 7397, 13853, 10471, 3018, 383, 279, 1534, 11013, 314, 18016, 310, 4022, 7397, 303, 780, 34393, 13, 271, 332, 16, 13, 36189, 2891, 279, 2763, 1528, 64700, 198, 9, 256, 3991, 18016, 7397, 25, 220, 18, 198, 9, 256, 3991, 4022, 7397, 25, 220, 19, 198, 9, 256, 10342, 2763, 7397, 25, 220, 22, 271, 332, 17, 13, 36189, 2891, 279, 4203, 64700, 198, 9, 256, 13853, 10471, 264, 2702, 314, 220, 16, 22, 7397, 13, 198, 9, 256, 6558, 393, 86, 3, 381, 279, 1324, 314, 4022, 7397, 10471, 13, 198, 9, 256, 6558, 393, 78, 3, 381, 279, 1324, 314, 18016, 7397, 10471, 13, 198, 9, 256, 1165, 1366, 421, 393, 86, 478, 296, 283, 220, 16, 22, 53031, 864, 23197, 393, 78, 283, 220, 16, 22, 471, 288, 12576, 271, 332, 18, 13, 36189, 2891, 279, 1534, 1528, 64700, 198, 9, 256, 12650, 1324, 314, 4022, 7397, 283, 3991, 4022, 478, 74522, 4022, 283, 393, 19, 478, 288, 3, 198, 9, 256, 12650, 1324, 314, 18016, 7397, 283, 3991, 18016, 478, 74522, 18016, 283, 393, 18, 478, 296, 3, 198, 9, 256, 3593, 3570, 10292, 393, 78, 283, 220, 16, 22, 471, 288, 3, 1083, 279, 18016, 22868, 25, 198, 262, 12650, 18016, 7397, 283, 393, 18, 478, 318, 16, 22, 471, 288, 8, 283, 220, 17, 15, 471, 288, 3, 271, 332, 19, 13, 19927, 279, 2876, 64700, 198, 760, 3377, 5134, 421, 303, 279, 809, 11, 13853, 995, 10598, 430, 1599, 18016, 7397, 430, 4022, 7397, 13, 198, 13682, 1088, 1272, 90, 18770, 21281, 92, 283, 220, 17, 1088, 14695, 1088, 1272, 90, 18770, 5626, 92, 25273, 271, 3036, 7417, 279, 22666, 494, 2923, 220, 18, 1083, 411, 22868, 25, 198, 13682, 220, 17, 15, 471, 288, 283, 220, 17, 1088, 14695, 318, 19, 478, 288, 8, 25273, 271, 332, 20, 13, 61148, 364, 393, 86, 60439, 332, 198, 13682, 220, 17, 15, 471, 288, 283, 220, 23, 478, 220, 17, 86, 25273, 198, 2143, 393, 86, 3, 310, 2107, 10744, 25, 198, 13682, 220, 17, 15, 283, 220, 23, 478, 220, 18, 86, 25273, 198, 3036, 2076, 220, 23, 494, 2107, 10744, 25, 198, 13682, 220, 16, 17, 283, 220, 18, 86, 25273, 198, 12151, 562, 539, 220, 18, 25, 198, 13682, 288, 283, 220, 19, 25273, 271, 332, 21, 13, 53611, 64700, 198, 9, 256, 1368, 13853, 10471, 220, 19, 4022, 7397, 11, 551, 10471, 393, 16, 22, 471, 220, 19, 283, 220, 16, 18, 3, 18016, 7397, 13, 198, 9, 256, 12650, 4022, 7397, 25, 393, 19, 19185, 1272, 90, 9147, 5242, 478, 220, 19, 19185, 1272, 90, 65, 2370, 5242, 283, 220, 23, 3, 198, 9, 256, 12650, 18016, 7397, 25, 393, 18, 19185, 1272, 90, 9147, 5242, 478, 220, 16, 18, 19185, 1272, 90, 65, 2370, 5242, 283, 220, 16, 21, 3, 198, 9, 256, 2091, 220, 16, 21, 10598, 430, 1599, 430, 220, 23, 30, 7179, 11, 393, 16, 21, 283, 220, 17, 1088, 14695, 220, 23, 12576, 271, 760, 1324, 314, 4022, 7397, 13853, 10471, 369, 220, 19, 13, 271, 794, 220, 19, 248046], "tokens": 529, "terminal": ["stop"], "usage": [{"prompt_tokens": 149, "completion_tokens": 529, "total_tokens": 678}], "output_text_sha256": "da51da33ffc217b8e774f28fe0fe3488b2710e0a9f9cb7ce4f315746a9b53d4f", "wire_sha256": "3fc37ed942e24519f74bdaaf4bbfc19d77a6f18cc49966b3431fa83dbd479858", "client_success": true, "client_error": false, "valid": true, "ttft_s": 2.6729599660029635, "latency_s": 49.13746222201735, "itl": [0.08801578299608082, 0.08808216801844537, 0.08802523801568896, 0.08799863595049828, 0.08800808107480407, 0.08798310498241335, 0.0881192060187459, 0.08798865298740566, 0.0880446060327813, 0.08794248197227716, 0.08797300898004323, 0.08809444704093039, 0.08796541998162866, 0.08807646995410323, 0.08793429099023342, 0.0880466450471431, 0.08798945404123515, 0.08805143996141851, 0.08800837001763284, 0.08793060597963631, 0.08806423004716635, 0.08798974100500345, 0.08812561596278101, 0.08797021303325891, 0.08797647291794419, 0.08796954003628343, 0.08794677897822112, 0.08803605497814715, 0.08810620999429375, 0.0884107310557738, 0.08794433600269258, 0.08810689998790622, 0.08795835601631552, 0.08801023894920945, 0.0879974770359695, 0.08796063996851444, 0.08808245207183063, 0.0879796699155122, 0.0880849399836734, 0.0879868030315265, 0.08810442104004323, 0.08797403797507286, 0.0879916170379147, 0.08797877398319542, 0.08791689993813634, 0.08804733003489673, 0.08798485202714801, 0.08815394202247262, 0.08788187999743968, 0.08802367292810231, 0.0879779610550031, 0.08808115404099226, 0.08808202692307532, 0.08803754206746817, 0.08807742490898818, 0.08786718500778079, 0.08807096898090094, 0.08800525299739093, 0.08803712204098701, 0.0879656330216676, 0.08795773796737194, 0.08802237105555832, 0.08801544096786529, 0.08803238498512655, 0.08797864802181721, 0.08791262202430516, 0.08804977999534458, 0.08816418692003936, 0.08800527406856418, 0.08799767796881497, 0.08810370601713657, 0.0878971399506554, 0.08810605306643993, 0.08802750799804926, 0.08796999498736113, 0.08796356897801161, 0.0880774330580607, 0.08800559595692903, 0.08795364305842668, 0.08805236394982785, 0.08792566100601107, 0.08811553497798741, 0.08797816606238484, 0.0879786629229784, 0.08792827907018363, 0.08802362496498972, 0.08794399001635611, 0.08794632100034505, 0.08810564503073692, 0.08789538289420307, 0.08801706100348383, 0.08804276108276099, 0.08797853800933808, 0.08806652191560715, 0.08791787107475102, 0.08807734097354114, 0.08792354702018201, 0.08805193495936692, 0.08795261301565915, 0.0880406319629401, 0.08798896905500442, 0.08798183803446591, 0.08804269891697913, 0.08796356106176972, 0.08801264793146402, 0.08790534606669098, 0.0879802550189197, 0.08796749694738537, 0.08797178894747049, 0.08799360902048647, 0.08785670704673976, 0.08799652999732643, 0.08792605297639966, 0.08807834202889353, 0.08794142596889287, 0.08809830900281668, 0.08797405706718564, 0.08797821099869907, 0.08801550895441324, 0.0880028250394389, 0.08804187795612961, 0.08823836094234139, 0.08800147310830653, 0.08791175088845193, 0.08799872105009854, 0.08800137601792812, 0.0879425669554621, 0.08789260603953153, 0.08790573896840215, 0.08807577006518841, 0.08785043994430453, 0.08799574500881135, 0.08794894604943693, 0.08804075792431831, 0.08811062306631356, 0.08811469899956137, 0.08808562101330608, 0.08789915998931974, 0.08806537301279604, 0.08793101191986352, 0.0879906719783321, 0.08804300799965858, 0.08797070407308638, 0.08803229697514325, 0.0879784170538187, 0.08805407490581274, 0.08787364908494055, 0.08810138900298625, 0.08793249097652733, 0.08813267201185226, 0.08807619696017355, 0.0879723159596324, 0.08797531703021377, 0.08798563200980425, 0.08805330796167254, 0.08791582705453038, 0.08815511094871908, 0.08799909404478967, 0.08801490697078407, 0.08795600500889122, 0.08802709507290274, 0.08797701797448099, 0.08789609791710973, 0.08804088004399091, 0.08806358301080763, 0.088003791985102, 0.08804978406988084, 0.08800598792731762, 0.08810343802906573, 0.08797972195316106, 0.0880487950053066, 0.0879355170764029, 0.08801459090318531, 0.08788230805657804, 0.08811011305078864, 0.08798605191987008, 0.08797265205066651, 0.08795657998416573, 0.08800649701151997, 0.08802246197592467, 0.08796052797697484, 0.08811763802077621, 0.08790491498075426, 0.08792227797675878, 0.0880910640116781, 0.0879850450437516, 0.08801831502933055, 0.0879339559469372, 0.08805644896347076, 0.08792070799972862, 0.08805608004331589, 0.08797151502221823, 0.08794031001161784, 0.08800659293774515, 0.08793637505732477, 0.0881155850365758, 0.08792396797798574, 0.0881199249997735, 0.08805139095056802, 0.08792795496992767, 0.08803890307899565, 0.08802344196010381, 0.08800219895783812, 0.08797861309722066, 0.08805506397038698, 0.08799266396090388, 0.08804129203781486, 0.08790165896061808, 0.08803792204707861, 0.08798654202837497, 0.08825509191956371, 0.08804706600494683, 0.08795567706692964, 0.08806177997030318, 0.08798432699404657, 0.08803686301689595, 0.08794513891916722, 0.08801889210008085, 0.0880280229030177, 0.08800113503821194, 0.08807080099359155, 0.0880014980211854, 0.08794468000996858, 0.0879267209675163, 0.08805213903542608, 0.08812617603689432, 0.08808518399018794, 0.08795883797574788, 0.0879483629250899, 0.08815197111107409, 0.08794078999198973, 0.08799392892979085, 0.08805051899980754, 0.08798040507826954, 0.0880292298970744, 0.08796984609216452, 0.08811975794378668, 0.08791560307145119, 0.08811515395063907, 0.08796978602185845, 0.08804116793908179, 0.08801750303246081, 0.08797168103046715, 0.08811699296347797, 0.08812569396104664, 0.08804080099798739, 0.08786258101463318, 0.08799951197579503, 0.08791278500575572, 0.0880850669927895, 0.0880154879996553, 0.08796892000827938, 0.08808640309143811, 0.0879516979912296, 0.08801294700242579, 0.08800753194373101, 0.08808892697561532, 0.08791881601791829, 0.08804805402178317, 0.08790866902563721, 0.088014384964481, 0.08799044496845454, 0.08800758607685566, 0.08801193593535572, 0.0879496120614931, 0.088032356929034, 0.08797790703829378, 0.08790783095173538, 0.08800642308779061, 0.08791817701421678, 0.08803518593776971, 0.08785767201334238, 0.08807790500577539, 0.08794527698773891, 0.0880127790151164, 0.08791372494306415, 0.08789317309856415, 0.08801482792478055, 0.08793550706468523, 0.08805527701042593, 0.08791449898853898, 0.0880548139102757, 0.08799203904345632, 0.08800260897260159, 0.08802391798235476, 0.08795258204918355, 0.08799931802786887, 0.08797344495542347, 0.08810376503970474, 0.08796518703456968, 0.0880114829633385, 0.0880759620340541, 0.08800922089722008, 0.08804318204056472, 0.0880071020219475, 0.08801860001403838, 0.08806582703255117, 0.08817214996088296, 0.0879384339787066, 0.08798870805185288, 0.0880703879520297, 0.08794953499455005, 0.0880273700458929, 0.08794506697449833, 0.0880776719423011, 0.08785542007535696, 0.08803033095318824, 0.08794685604516417, 0.08804972900543362, 0.0880606269929558, 0.08802638691850007, 0.08796311309561133, 0.08796753094065934, 0.08797784498892725, 0.08791163703426719, 0.08816152904182673, 0.08819769695401192, 0.08799009397625923, 0.0880117550259456, 0.08803344704210758, 0.08807967195753008, 0.08785851998254657, 0.08808721404056996, 0.08784309099428356, 0.08803951798472553, 0.087969767046161, 0.08798547391779721, 0.08804692700505257, 0.08803500898648053, 0.08802637201733887, 0.08801061799749732, 0.08803576300852001, 0.08791240304708481, 0.08818422595504671, 0.08804478903766721, 0.08804280799813569, 0.08798997895792127, 0.08789299102500081, 0.0881026970455423, 0.08799595991149545, 0.08804826904088259, 0.08805103995837271, 0.08810716506559402, 0.08795386296696961, 0.08792646997608244, 0.08800573600456119, 0.08793328807223588, 0.08808919799048454, 0.08796239993534982, 0.08802915003616363, 0.08796883700415492, 0.08798350393772125, 0.08802234509494156, 0.08797431690618396, 0.0880211100447923, 0.08792053302749991, 0.0881299739703536, 0.08786003000568599, 0.08795691898558289, 0.08797479199711233, 0.08803708397317678, 0.08807428902946413, 0.08798091800417751, 0.08806095598265529, 0.08793343207798898, 0.08799996296875179, 0.08795808092691004, 0.08793210610747337, 0.08798174699768424, 0.08801761292852461, 0.08811510598752648, 0.08791881601791829, 0.08803599106613547, 0.08803643495775759, 0.08798923098947853, 0.08805249095894396, 0.08799179806374013, 0.08797730994410813, 0.08794792997650802, 0.08804168307688087, 0.08788443496450782, 0.08799397700931877, 0.0879525060299784, 0.08806555194314569, 0.08810005302075297, 0.08792857697699219, 0.08797642006538808, 0.08792682597413659, 0.08808947796933353, 0.08785184705629945, 0.08802466397173703, 0.08789682702627033, 0.08796672092285007, 0.08802273403853178, 0.08799619402270764, 0.08805199596099555, 0.08788480900693685, 0.08801802806556225, 0.08789069193881005, 0.08808941498864442, 0.08796232007443905, 0.08797132992185652, 0.08800462598446757, 0.0879217330366373, 0.0879994350252673, 0.08790834003593773, 0.08804895891807973, 0.0879766009747982, 0.08800296008121222, 0.08796516596339643, 0.08800723496824503, 0.08806372398976237, 0.08790755900554359, 0.08809874509461224, 0.08790660498198122, 0.08799972094129771, 0.08797738002613187, 0.08795644098427147, 0.0879354439675808, 0.08792289206758142, 0.08802512101829052, 0.08792113291565329, 0.08804129203781486, 0.08794532797764987, 0.0879559350432828, 0.0879978749435395, 0.0879646900575608, 0.0880114670144394, 0.08796016592532396, 0.08802316803485155, 0.08788891800213605, 0.08801848895382136, 0.08793544711079448, 0.08800775895360857, 0.08801520999986678, 0.08793760696426034, 0.0879874030360952, 0.08793203695677221, 0.08805883699096739, 0.08789133408572525, 0.0880207559093833, 0.08800257404800504, 0.08836638298816979, 0.08804079506080598, 0.08792256494052708, 0.08809678605757654, 0.0878930379403755, 0.08803783496841788, 0.08797446708194911, 0.08800862391944975, 0.08798351103905588, 0.08795725496020168, 0.08797658304683864, 0.08796003903262317, 0.08803200291004032, 0.08792410709429532, 0.08806413493584841, 0.08789048402104527, 0.0879513289546594, 0.08799421810545027, 0.08797131094615906, 0.08799300598911941, 0.08788873802404851, 0.08827990794088691, 0.08800350001547486, 0.08804794703610241, 0.0879361629486084, 0.087908118031919, 0.08799843105953187, 0.08801860793028027, 0.08802503300830722, 0.08790370495989919, 0.08800852205604315, 0.08792619896121323, 0.088026283076033, 0.0879493709653616, 0.08796255302149802, 0.08796289400197566, 0.08797477895859629, 0.0880390650127083, 0.08784716494847089, 0.08797238604165614, 0.08800498105119914, 0.08800669794436544, 0.08795362000819296, 0.08794785803183913, 0.0880247459281236, 0.0880314470268786, 0.08800686406902969, 0.08810186199843884, 0.0880551419686526, 0.08786524902097881, 0.08790856099221855, 0.0880497939651832, 0.08796271297615021, 0.08795293106231838, 0.0879449169151485, 0.08812598802614957, 0.08790058805607259, 0.08801828394643962, 0.08797724498435855, 0.08795377798378468, 0.08811767504084855, 0.08784796297550201, 0.08794966398272663, 0.0879156980663538, 0.08801181893795729, 0.08801150205545127, 0.08801120100542903, 0.08796099503524601, 0.0879339330131188, 0.08804553688969463, 0.08791420410852879, 0.08814352191984653, 0.08793438400607556, 0.0880094770109281, 0.08793858997523785, 0.0880525300744921, 0.08789497392717749, 0.08799737703520805, 0.08807635598350316, 0.08783015003427863, 0.08804617403075099, 0.08794204099103808, 0.08799327199812979, 0.08799702196847647, 0.08795853902120143, 0.08809565694537014], "tpot_s": 0.08800095124245148, "active_requests_after": 0, "runner_state_count_after": 0} -{"phase": "request", "arm": "graph", "case": "gsm8k/test/1", "index": 1, "cap": 2048, "ignore_eos": false, "prompt_tokens": 70, "prompt_sha256": "78d335abe81363d4bb48e5aa256c120344c2ceadbcf8739afe4ec3923c747238", "token_ids": [1206, 7995, 279, 2702, 1324, 314, 47205, 2483, 364, 279, 60507, 11, 567, 1144, 310, 10724, 279, 3202, 314, 4022, 23043, 321, 884, 424, 310, 279, 3202, 314, 6105, 23043, 13, 271, 16, 13, 220, 2972, 10025, 23043, 92217, 561, 3377, 5134, 421, 279, 60507, 4829, 220, 17, 47205, 314, 6105, 23043, 13, 198, 17, 13, 220, 2972, 13672, 23043, 92217, 561, 3377, 5134, 421, 424, 4829, 328, 35778, 421, 1691, 1, 4022, 23043, 13, 328, 4639, 1691, 1, 18675, 310, 279, 3202, 314, 6105, 23043, 13, 198, 262, 348, 256, 71690, 25, 393, 17, 1088, 1272, 90, 47205, 92, 593, 220, 17, 283, 220, 16, 1088, 1272, 90, 30797, 92, 12576, 198, 18, 13, 220, 2972, 7354, 47205, 92217, 2604, 279, 6105, 23043, 321, 279, 4022, 23043, 3658, 13, 198, 262, 348, 256, 71690, 25, 393, 17, 1088, 1272, 90, 47205, 318, 11855, 8866, 478, 220, 16, 1088, 1272, 90, 30797, 318, 5603, 8866, 283, 220, 18, 1088, 1272, 90, 47205, 92, 12576, 271, 794, 220, 18, 248046], "tokens": 169, "terminal": ["stop"], "usage": [{"prompt_tokens": 70, "completion_tokens": 169, "total_tokens": 239}], "output_text_sha256": "53ef84987096a7daa32212812c15c7fce023a5b06f50ea52a2c0e026d0a1f1e6", "wire_sha256": "41a86d42bcbbf6e255b1649b99def36a7383cf9497296231e8f870069467a847", "client_success": true, "client_error": false, "valid": true, "ttft_s": 1.3001440840307623, "latency_s": 16.00403310393449, "itl": [0.08760336390696466, 0.08752772305160761, 0.0874659389955923, 0.0875271150143817, 0.08753990894183517, 0.08756203902885318, 0.08750876097474247, 0.08758655400015414, 0.08749075408559293, 0.08746394398622215, 0.08756021794397384, 0.08745211397763342, 0.08760256203822792, 0.0874046980170533, 0.08765546092763543, 0.0874475761083886, 0.08746426599100232, 0.08752766600809991, 0.08748558396473527, 0.0875573520315811, 0.08739252400118858, 0.08763847697991878, 0.0873973990092054, 0.08756640390492976, 0.0875026989961043, 0.08753265207633376, 0.08761524793226272, 0.08745209302287549, 0.0875794100575149, 0.08739217498805374, 0.08760493795853108, 0.08746322698425502, 0.08761836506891996, 0.08783055294770747, 0.08746119902934879, 0.08763942902442068, 0.08742707094643265, 0.08762283797841519, 0.08749645203351974, 0.087526450981386, 0.08751001500058919, 0.08748753706458956, 0.08754875091835856, 0.08754817605949938, 0.08772136503830552, 0.08743401896208525, 0.0875860039377585, 0.08747796306852251, 0.08756643394008279, 0.0875985020538792, 0.0874862769851461, 0.08749994297977537, 0.08743533201050013, 0.08761990000493824, 0.08739558001980186, 0.08760410896502435, 0.08753225102555007, 0.08746385597623885, 0.08755896403454244, 0.08745847700629383, 0.08767198596615344, 0.08738261403050274, 0.08758004102855921, 0.08739142795093358, 0.08755557704716921, 0.08755129389464855, 0.08750906703062356, 0.08752777008339763, 0.0873636519536376, 0.08768239803612232, 0.08763971494045109, 0.08755735098384321, 0.08750937005970627, 0.08753397793043405, 0.08747886202763766, 0.08745088102295995, 0.08766550896689296, 0.08731576800346375, 0.08747964899521321, 0.08745239698328078, 0.0876199840568006, 0.08749932597856969, 0.08745068102143705, 0.08757977595087141, 0.08746141102164984, 0.08758361905347556, 0.08738366095349193, 0.08763564296532422, 0.08751806803047657, 0.08742860902566463, 0.08758668298833072, 0.08746721502393484, 0.08758988499175757, 0.08748503099195659, 0.0875645800260827, 0.08755264698993415, 0.08755463000852615, 0.0873978539602831, 0.08745695697143674, 0.08770022902172059, 0.08738690603058785, 0.08762233494780958, 0.08742925408296287, 0.08755298692267388, 0.08748118800576776, 0.08764182799495757, 0.08747531601693481, 0.08747747098095715, 0.08761519205290824, 0.08739232097286731, 0.08765482797753066, 0.08742237999103963, 0.08750286605209112, 0.08742993592750281, 0.0875302100321278, 0.0875356380129233, 0.08740144898183644, 0.08770684106275439, 0.0873302809195593, 0.08759552205447108, 0.08747549704276025, 0.08749831293243915, 0.08752443897537887, 0.0873910520458594, 0.08761053194757551, 0.08741893107071519, 0.08758645702619106, 0.08739955199416727, 0.08753277396317571, 0.08754349697846919, 0.08784831198863685, 0.08754138799849898, 0.0873912270180881, 0.08765309897717088, 0.0872910360340029, 0.08756489201914519, 0.08747640100773424, 0.08749319601338357, 0.08753905794583261, 0.08748601900879294, 0.08760973904281855, 0.08732311998028308, 0.08760470396373421, 0.08740143605973572, 0.08770512090995908, 0.08770693698897958, 0.08754186506848782, 0.08759993501007557, 0.08741324802394956, 0.08757040998898447, 0.0873922900063917, 0.08772356493864208, 0.08746389404404908, 0.08751958492211998, 0.08750837703701109, 0.08748931100126356, 0.08766454795841128, 0.08736495801713318, 0.08756579703185707, 0.08752495096996427, 0.08752563199959695, 0.08743888605386019, 0.08741420903243124, 0.0877079889178276, 0.08746915706433356, 0.087603744934313, 0.08746186399366707, 0.08758087200112641], "tpot_s": 0.08752314892799837, "active_requests_after": 0, "runner_state_count_after": 0} -{"phase": "request", "arm": "graph", "case": "gsm8k/test/424", "index": 2, "cap": 2048, "ignore_eos": false, "prompt_tokens": 106, "prompt_sha256": "03cbdf5804bab62d4efbc0ae89e191bd557840c6d9a1a5ada9669de953a0b783", "token_ids": [1206, 7995, 279, 2702, 1324, 314, 7998, 421, 628, 4973, 279, 1834, 8151, 11, 567, 1144, 310, 10724, 279, 2702, 8390, 314, 660, 279, 47224, 11145, 13, 271, 16, 13, 220, 2972, 46270, 279, 8390, 314, 279, 32275, 64700, 198, 262, 2532, 513, 220, 19, 32275, 11, 321, 1754, 9687, 220, 21, 15, 7998, 13, 198, 262, 25273, 19, 1088, 14695, 220, 21, 15, 283, 220, 17, 19, 15, 1088, 1272, 90, 7998, 92, 13682, 271, 17, 13, 220, 2972, 46270, 279, 8390, 314, 279, 73320, 4623, 64700, 198, 262, 2532, 513, 220, 21, 73320, 4623, 11, 321, 1754, 9687, 220, 18, 15, 7998, 13, 198, 262, 25273, 21, 1088, 14695, 220, 18, 15, 283, 220, 16, 23, 15, 1088, 1272, 90, 7998, 92, 13682, 271, 18, 13, 220, 2972, 46270, 279, 8390, 314, 279, 1262, 340, 581, 64700, 198, 262, 2532, 513, 220, 16, 15, 1262, 340, 581, 11, 321, 1754, 9687, 220, 16, 20, 7998, 13, 198, 262, 25273, 16, 15, 1088, 14695, 220, 16, 20, 283, 220, 16, 20, 15, 1088, 1272, 90, 7998, 92, 13682, 271, 19, 13, 220, 2972, 8917, 279, 56468, 64700, 198, 262, 10342, 7998, 283, 318, 28432, 314, 32275, 8, 478, 318, 28432, 314, 73320, 4623, 8, 478, 318, 28432, 314, 1262, 340, 581, 8, 198, 262, 25273, 17, 19, 15, 478, 220, 16, 23, 15, 478, 220, 16, 20, 15, 13682, 271, 262, 5339, 11, 884, 220, 17, 19, 15, 321, 220, 16, 23, 15, 25, 198, 262, 25273, 17, 19, 15, 478, 220, 16, 23, 15, 283, 220, 19, 17, 15, 13682, 271, 262, 4844, 11, 884, 220, 16, 20, 15, 25, 198, 262, 25273, 19, 17, 15, 478, 220, 16, 20, 15, 283, 220, 20, 22, 15, 13682, 271, 760, 2702, 1324, 314, 7998, 421, 628, 4973, 279, 1834, 8151, 369, 220, 20, 22, 15, 13, 271, 794, 220, 20, 22, 15, 248046], "tokens": 314, "terminal": ["stop"], "usage": [{"prompt_tokens": 106, "completion_tokens": 314, "total_tokens": 420}], "output_text_sha256": "bb797c1732c29c467b45fd24a2a27f80a1a5d8ce8dc38799eb28b29a5c056bb2", "wire_sha256": "09f9bc44738ef9408b9f966a39309048c838dc71b0e9d24e729985c4a76ed075", "client_success": true, "client_error": false, "valid": true, "ttft_s": 1.9258344959234819, "latency_s": 29.31772259902209, "itl": [0.08758313208818436, 0.08763293700758368, 0.08737781993113458, 0.08762979204766452, 0.08738507993984967, 0.0875316180754453, 0.08751260000281036, 0.08752811094745994, 0.08760495099704713, 0.08746124303434044, 0.08758604200556874, 0.08747320692054927, 0.08763133408501744, 0.08746008900925517, 0.08744364196900278, 0.08759035903494805, 0.08746142592281103, 0.08765744906850159, 0.08747075800783932, 0.08762091398239136, 0.08743487193714827, 0.08747685805428773, 0.08759671193547547, 0.08755053800996393, 0.0875477580120787, 0.0874113580211997, 0.08769134897738695, 0.08741959603503346, 0.08756012201774865, 0.08751962298993021, 0.08754522097297013, 0.08759705803822726, 0.0873429209459573, 0.08765752706676722, 0.08745022700168192, 0.08757272793445736, 0.08750727807637304, 0.08788549900054932, 0.08753343194257468, 0.08745119394734502, 0.08757391001563519, 0.08739824406802654, 0.08767365792300552, 0.08744261600077152, 0.08752265106886625, 0.08746457402594388, 0.08743519696872681, 0.08762439794372767, 0.08749754703603685, 0.08756400202400982, 0.08742138394154608, 0.08751012699212879, 0.08753851999063045, 0.08751894708257169, 0.08745071792509407, 0.08752006408758461, 0.08757702098228037, 0.08745610795449466, 0.08761792501900345, 0.0874686170136556, 0.08767842897213995, 0.0874692170182243, 0.08741397794801742, 0.08758440299425274, 0.0874356710119173, 0.08754609699826688, 0.08748311502858996, 0.0875056330114603, 0.08741457702126354, 0.08753400994464755, 0.08750441705342382, 0.08741297398228198, 0.08758070599287748, 0.0874136220663786, 0.08774154295679182, 0.0875514840008691, 0.08766874694265425, 0.08748391701374203, 0.08744198002386838, 0.08754822902847081, 0.08751518500503153, 0.08761440601665527, 0.08745687094051391, 0.08762760600075126, 0.08741164696402848, 0.08754417509771883, 0.08755561895668507, 0.0874856150476262, 0.08760836790315807, 0.08743387006688863, 0.0875878439983353, 0.08746947499457747, 0.08762578503228724, 0.0874997329665348, 0.08746529498603195, 0.08745202200952917, 0.08749883598648012, 0.08759640296921134, 0.08742338209412992, 0.08753254590556026, 0.08745024004019797, 0.08751034503802657, 0.08745723497122526, 0.08749204501509666, 0.0876059029251337, 0.0874071610160172, 0.08758822898380458, 0.08753863209858537, 0.08748535893391818, 0.08745845197699964, 0.08754444005899131, 0.08761294500436634, 0.08751491492148489, 0.087654042057693, 0.08738958102185279, 0.08761200401932001, 0.08745525893755257, 0.08751870295964181, 0.08750369504559785, 0.08747140399646014, 0.0876491490053013, 0.08746872202027589, 0.08761117595713586, 0.08740641502663493, 0.0875009399605915, 0.08743931003846228, 0.087526187999174, 0.08751444902736694, 0.08749711001291871, 0.08757343294564635, 0.08743368601426482, 0.0876181679777801, 0.0874204650754109, 0.08755566889885813, 0.08759104507043958, 0.08778338902629912, 0.08754927199333906, 0.08742823090869933, 0.0876298100920394, 0.08738368796184659, 0.08751239604316652, 0.08747829392086715, 0.08754788106307387, 0.08752056502271444, 0.08746444992721081, 0.08757806604262441, 0.0873894690303132, 0.08757475996389985, 0.08762769203167409, 0.0876690240111202, 0.08748951193410903, 0.08745943801477551, 0.08757026994135231, 0.08743837010115385, 0.08760309591889381, 0.08743407705333084, 0.08757198695093393, 0.08752499299589545, 0.0874793539987877, 0.08752209402155131, 0.08747344999574125, 0.08749555400572717, 0.08731190499383956, 0.08753610402345657, 0.0873726459685713, 0.08752345805987716, 0.08748245891183615, 0.08745403110515326, 0.08773443091195077, 0.08741449599619955, 0.08767117909155786, 0.0874280259013176, 0.08758815901819617, 0.08743816404603422, 0.08750999299809337, 0.08754141500685364, 0.08740879292599857, 0.08768718200735748, 0.08742969005834311, 0.08758220495656133, 0.08745540608651936, 0.08752651489339769, 0.08751357009168714, 0.0875136919785291, 0.08756040397565812, 0.08736123703420162, 0.08758738299366087, 0.08741028199438006, 0.08750127395614982, 0.08748361200559884, 0.08752781699877232, 0.08766844402998686, 0.08740022999700159, 0.08752405794803053, 0.08742329909000546, 0.08757945301476866, 0.08743333991151303, 0.08753583999350667, 0.08751000103075057, 0.08749741199426353, 0.08749889605678618, 0.08747040992602706, 0.08755765308160335, 0.08743882889393717, 0.08760101709049195, 0.08743199601303786, 0.08748456498142332, 0.08758200891315937, 0.08740465110167861, 0.08750706992577761, 0.08739424706436694, 0.08760224096477032, 0.08744560496415943, 0.08760426903609186, 0.08737057296093553, 0.08743313001468778, 0.08751408604439348, 0.08742810797411948, 0.08759696199558675, 0.0874142280081287, 0.08763338602147996, 0.0874660819536075, 0.08744131901767105, 0.08755806204862893, 0.0874416489386931, 0.087521334993653, 0.08738373999949545, 0.0876638360787183, 0.0873892119852826, 0.08741597994230688, 0.08743900898844004, 0.0874880290357396, 0.0875582309672609, 0.08740905008744448, 0.08756641391664743, 0.0874755340628326, 0.08757808990776539, 0.08741748705506325, 0.08747582999058068, 0.08753539202734828, 0.08741559693589807, 0.08751505904365331, 0.08741631696466357, 0.08761773398146033, 0.08742110105231404, 0.0875647000502795, 0.08787536295130849, 0.08746540197171271, 0.08754815207794309, 0.08746820990927517, 0.08757829410023987, 0.08737320092041045, 0.08760723599698395, 0.08749439299572259, 0.08745359000749886, 0.0874976150225848, 0.0874461280182004, 0.0876677610212937, 0.08739971299655735, 0.0875260200118646, 0.08749556797556579, 0.08753507700748742, 0.0874233489157632, 0.08749264804646373, 0.08755764504894614, 0.08759884990286082, 0.08772005303762853, 0.08739405497908592, 0.08754102699458599, 0.08747030701488256, 0.08752551802899688, 0.0874226070009172, 0.08740011195186526, 0.08760314004030079, 0.08737990795634687, 0.08755549509078264, 0.08733957901131362, 0.08750187396071851, 0.087511220946908, 0.08746173908002675, 0.08759906399063766, 0.08740631700493395, 0.08761743700597435, 0.08740404399577528, 0.08757492795120925, 0.08741341298446059, 0.08753924502525479, 0.08751999004743993, 0.08754088694695383, 0.08769789698999375, 0.08744029700756073, 0.08761644002515823, 0.08750428294297308, 0.08758370904251933, 0.0875724529614672, 0.08737364201806486, 0.08760123606771231, 0.08746494492515922, 0.08755719102919102, 0.0874486289685592, 0.08750874502584338, 0.08747043798211962, 0.08745644602458924, 0.08754923695232719, 0.08740824006963521, 0.08757126203272492, 0.08745947398710996, 0.08758616691920906, 0.08745016506873071, 0.08755892293993384, 0.08747201599180698, 0.0875642200699076, 0.08754581701941788], "tpot_s": 0.0875140194987176, "active_requests_after": 0, "runner_state_count_after": 0} -{"phase": "request", "arm": "graph", "case": "gsm8k/test/783", "index": 3, "cap": 1024, "ignore_eos": true, "prompt_tokens": 149, "prompt_sha256": "f2233be4ebb7b122d80433fe4081fdabeda64f735b7bc24f9706c4c4339b824f", "token_ids": [1206, 11290, 411, 3377, 11, 567, 1144, 310, 7995, 279, 1324, 314, 4022, 7397, 13853, 10471, 3018, 383, 279, 1534, 11013, 314, 18016, 310, 4022, 7397, 303, 780, 34393, 13, 271, 332, 16, 13, 36189, 2891, 279, 2763, 1528, 64700, 198, 9, 256, 3991, 18016, 7397, 25, 220, 18, 198, 9, 256, 3991, 4022, 7397, 25, 220, 19, 198, 9, 256, 10342, 2763, 7397, 25, 220, 22, 271, 332, 17, 13, 36189, 2891, 279, 4203, 64700, 198, 9, 256, 13853, 10471, 264, 2702, 314, 220, 16, 22, 7397, 13, 198, 9, 256, 6558, 393, 86, 3, 381, 279, 1324, 314, 4022, 7397, 10471, 13, 198, 9, 256, 6558, 393, 78, 3, 381, 279, 1324, 314, 18016, 7397, 10471, 13, 198, 9, 256, 1165, 1366, 421, 393, 86, 478, 296, 283, 220, 16, 22, 53031, 864, 23197, 393, 78, 283, 220, 16, 22, 471, 288, 12576, 271, 332, 18, 13, 36189, 2891, 279, 1534, 1528, 64700, 198, 9, 256, 12650, 1324, 314, 4022, 7397, 283, 3991, 4022, 478, 74522, 4022, 283, 393, 19, 478, 288, 3, 198, 9, 256, 12650, 1324, 314, 18016, 7397, 283, 3991, 18016, 478, 74522, 18016, 283, 393, 18, 478, 296, 3, 198, 9, 256, 3593, 3570, 10292, 393, 78, 283, 220, 16, 22, 471, 288, 3, 1083, 279, 18016, 22868, 25, 198, 262, 12650, 18016, 7397, 283, 393, 18, 478, 318, 16, 22, 471, 288, 8, 283, 220, 17, 15, 471, 288, 3, 271, 332, 19, 13, 19927, 279, 2876, 64700, 198, 760, 3377, 5134, 421, 303, 279, 809, 11, 13853, 995, 10598, 430, 1599, 18016, 7397, 430, 4022, 7397, 13, 198, 13682, 1088, 1272, 90, 18770, 21281, 92, 283, 220, 17, 1088, 14695, 1088, 1272, 90, 18770, 5626, 92, 25273, 271, 3036, 7417, 279, 22666, 494, 2923, 220, 18, 1083, 411, 22868, 25, 198, 13682, 220, 17, 15, 471, 288, 283, 220, 17, 1088, 14695, 318, 19, 478, 288, 8, 25273, 271, 332, 20, 13, 61148, 364, 393, 86, 60439, 332, 198, 13682, 220, 17, 15, 471, 288, 283, 220, 23, 478, 220, 17, 86, 25273, 198, 2143, 393, 86, 3, 310, 2107, 10744, 25, 198, 13682, 220, 17, 15, 283, 220, 23, 478, 220, 18, 86, 25273, 198, 3036, 2076, 220, 23, 494, 2107, 10744, 25, 198, 13682, 220, 16, 17, 283, 220, 18, 86, 25273, 198, 12151, 562, 539, 220, 18, 25, 198, 13682, 288, 283, 220, 19, 25273, 271, 332, 21, 13, 53611, 64700, 198, 9, 256, 1368, 13853, 10471, 220, 19, 4022, 7397, 11, 551, 10471, 393, 16, 22, 471, 220, 19, 283, 220, 16, 18, 3, 18016, 7397, 13, 198, 9, 256, 12650, 4022, 7397, 25, 393, 19, 19185, 1272, 90, 9147, 5242, 478, 220, 19, 19185, 1272, 90, 65, 2370, 5242, 283, 220, 23, 3, 198, 9, 256, 12650, 18016, 7397, 25, 393, 18, 19185, 1272, 90, 9147, 5242, 478, 220, 16, 18, 19185, 1272, 90, 65, 2370, 5242, 283, 220, 16, 21, 3, 198, 9, 256, 2091, 220, 16, 21, 10598, 430, 1599, 430, 220, 23, 30, 7179, 11, 393, 16, 21, 283, 220, 17, 1088, 14695, 220, 23, 12576, 271, 760, 1324, 314, 4022, 7397, 13853, 10471, 369, 220, 19, 13, 271, 794, 220, 19, 248046, 198, 248044, 248045, 248045, 248045, 248045, 248046, 198, 248044, 248045, 248045, 248045, 248046, 198, 248045, 846, 198, 2523, 513, 264, 16097, 321, 16908, 28016, 13, 248046, 198, 248045, 248068, 198, 760, 1156, 6587, 728, 310, 11290, 279, 6673, 3377, 883, 13853, 579, 7397, 34393, 13, 6558, 728, 944, 1472, 424, 15060, 13, 271, 6140, 1528, 25, 198, 12, 220, 18, 18016, 7397, 198, 12, 220, 19, 4022, 7397, 198, 12, 10342, 25, 220, 22, 7397, 271, 31318, 47866, 220, 16, 22, 7397, 494, 264, 9238, 12526, 13, 6558, 579, 1910, 551, 47866, 288, 4022, 7397, 321, 296, 18016, 7397, 11, 1332, 288, 478, 296, 283, 220, 16, 22, 13, 271, 5836, 7594, 1070, 310, 780, 34393, 25, 198, 12, 10342, 4022, 7397, 25, 220, 19, 478, 288, 198, 12, 10342, 18016, 7397, 25, 220, 18, 478, 296, 283, 220, 18, 478, 318, 16, 22, 471, 288, 8, 283, 220, 17, 15, 471, 288, 271, 10234, 25, 1216, 1381, 682, 10598, 430, 1599, 18016, 7397, 430, 4022, 7397, 13, 198, 4272, 25, 220, 17, 15, 471, 288, 283, 220, 17, 7, 19, 478, 288, 8, 198, 17, 15, 471, 288, 283, 220, 23, 478, 220, 17, 86, 198, 16, 17, 283, 220, 18, 86, 198, 86, 283, 220, 19, 271, 9764, 728, 9845, 25, 198, 12, 5626, 7397, 10471, 25, 220, 19, 198, 12, 21281, 7397, 10471, 25, 220, 16, 18, 198, 12, 12650, 4022, 25, 220, 19, 478, 220, 19, 283, 220, 23, 198, 12, 12650, 18016, 25, 220, 18, 478, 220, 16, 18, 283, 220, 16, 21, 198, 12, 220, 16, 21, 283, 220, 17, 23985, 220, 23, 50615, 271, 760, 4087, 369, 220, 19, 13, 198, 248069, 271, 2, 11144, 4261, 13853, 579, 16120, 21397, 271, 550, 19427, 3038, 279, 21397, 271, 332, 6140, 34393, 64700, 198, 12, 21281, 7397, 25, 220, 18, 198, 12, 5626, 7397, 25, 220, 19, 271, 332, 62360, 14307, 64700, 220, 16, 22, 2702, 318, 64, 6311, 314, 4022, 321, 18016, 8, 271, 9764, 728, 6762, 25, 198, 12, 393, 86, 3, 283, 1324, 314, 4022, 7397, 10471, 198, 12, 393, 78, 3, 283, 1324, 314, 18016, 7397, 10471, 271, 12189, 551, 10471, 220, 16, 22, 7397, 2702, 25, 393, 86, 478, 296, 283, 220, 16, 22, 53031, 748, 393, 78, 283, 220, 16, 22, 471, 288, 3, 271, 550, 12650, 82699, 271, 5836, 7594, 279, 491, 7397, 310, 279, 34393, 25, 198, 12, 12650, 4022, 7397, 25, 393, 19, 478, 288, 3, 198, 12, 12650, 18016, 7397, 25, 393, 18, 478, 296, 283, 220, 18, 478, 318, 16, 22, 471, 288, 8, 283, 220, 17, 15, 471, 288, 3, 271, 550, 86521, 279, 14738, 271, 31318, 1381, 682, 2972, 15103, 543, 430, 1599, 18016, 7397, 430, 4022, 7397, 92217, 271, 13682, 17, 15, 471, 288, 283, 220, 17, 7, 19, 478, 288, 8, 13682, 271, 550, 11144, 4261, 271, 13682, 17, 15, 471, 288, 283, 220, 23, 478, 220, 17, 86, 13682, 198, 13682, 17, 15, 471], "tokens": 1024, "terminal": ["length"], "usage": [{"prompt_tokens": 149, "completion_tokens": 1024, "total_tokens": 1173}], "output_text_sha256": "5436ae0c73cb99b957af31c6f3c5b905d66d815fbc4de621f633ada46a410e99", "wire_sha256": "14f697729b3953772422c0c26f12eb3048acf5f7147c4af17788e474b8cfb6e8", "client_success": true, "client_error": false, "valid": true, "ttft_s": 2.5463140930514783, "latency_s": 92.73048542998731, "itl": [0.0873976789880544, 0.08747232903260738, 0.0875303999055177, 0.08752184500917792, 0.08750929799862206, 0.08756351005285978, 0.08751503401435912, 0.08765198301989585, 0.08738660893868655, 0.08753727097064257, 0.08750932104885578, 0.08751454600133002, 0.0874859819887206, 0.08749281405471265, 0.08753604395315051, 0.08743535296525806, 0.08765976305585355, 0.08787668193690479, 0.0875940170371905, 0.08755812095478177, 0.0874888370744884, 0.08754153491463512, 0.08738055708818138, 0.08763178391382098, 0.08746998209971935, 0.08757529291324317, 0.08745016809552908, 0.08748164889402688, 0.08763573109172285, 0.08757247193716466, 0.08754206902813166, 0.0874588389415294, 0.08753430109936744, 0.08737176493741572, 0.08752288005780429, 0.08754733100067824, 0.08746151300147176, 0.0876950059318915, 0.08743745798710734, 0.08760511805303395, 0.08746287797112018, 0.08760660805273801, 0.08748405892401934, 0.08746079006232321, 0.08753597096074373, 0.0874306249897927, 0.08762181701604277, 0.0874603750417009, 0.0874974150210619, 0.08743542688898742, 0.08755526307504624, 0.0874710789648816, 0.08746504399459809, 0.08756864198949188, 0.08756301598623395, 0.08765961299650371, 0.0873786601005122, 0.08754694496747106, 0.0875350299756974, 0.08746274805162102, 0.08745499094948173, 0.08747579297050834, 0.08757778408471495, 0.0874453759752214, 0.08757173595950007, 0.08745232899673283, 0.08762113703414798, 0.08748283493332565, 0.08743800001684576, 0.08757155400235206, 0.0874072799924761, 0.0875850870506838, 0.0873720080126077, 0.08762025192845613, 0.08745359699241817, 0.08751189708709717, 0.0875423839315772, 0.0874232720816508, 0.08757807989604771, 0.08737138204742223, 0.08760901202913374, 0.08734407695010304, 0.08752676798030734, 0.08750934107229114, 0.08742336195427924, 0.08756187604740262, 0.08744929498061538, 0.08760301303118467, 0.08746852597687393, 0.0876750759780407, 0.08747711905743927, 0.08757218590471894, 0.08756872802041471, 0.08748954196926206, 0.08755035407375544, 0.0874245249433443, 0.08758711908012629, 0.08740526693873107, 0.08756302704568952, 0.08745885593816638, 0.08750291308388114, 0.08755570196080953, 0.08748438896145672, 0.08761274302378297, 0.08737983601167798, 0.08760776394046843, 0.08741321507841349, 0.08754897292237729, 0.08749634202104062, 0.08738893002737314, 0.08755863003898412, 0.08734851598273963, 0.08765597303863615, 0.08737121999729425, 0.08751183398999274, 0.08784748800098896, 0.08747720497194678, 0.08750844898167998, 0.0875055049546063, 0.0876074080588296, 0.08737667195964605, 0.08754356007557362, 0.08742020290810615, 0.08756297908257693, 0.08752446295693517, 0.08745725604239851, 0.08756833197548985, 0.08739212900400162, 0.08756578993052244, 0.08765383099671453, 0.0876984290080145, 0.08745602809358388, 0.08745406393427402, 0.08758108504116535, 0.08745246194303036, 0.08753200399223715, 0.08745784603524953, 0.0876118519809097, 0.08746157097630203, 0.08752527204342186, 0.08749544096644968, 0.08747239201329648, 0.08754620305262506, 0.08741627098061144, 0.08763593400362879, 0.08742406393866986, 0.08757785405032337, 0.08743383397813886, 0.08759155205916613, 0.08770388294942677, 0.08740881097037345, 0.08757689199410379, 0.08737713808659464, 0.08759940101299435, 0.08745806396473199, 0.08746649394743145, 0.08755880105309188, 0.08745166100561619, 0.08754656603559852, 0.08740799897350371, 0.08760706195607781, 0.08737391897011548, 0.08755032601766288, 0.08742173598147929, 0.08749910106416792, 0.08753618597984314, 0.08737462002318352, 0.08769248600583524, 0.08737773902248591, 0.08753976598381996, 0.08742578199598938, 0.08749552594963461, 0.08751476497855037, 0.08755937009118497, 0.0875244359485805, 0.08744236500933766, 0.08763475902378559, 0.08739062794484198, 0.087593641015701, 0.08756876003462821, 0.08742745802737772, 0.08751708792988211, 0.08740673505235463, 0.08764070994220674, 0.08744541904889047, 0.08759115298744291, 0.08743785496335477, 0.08749528008047491, 0.08753926691133529, 0.0874065060634166, 0.0875301850028336, 0.08732901699841022, 0.08753641392104328, 0.08742682007141411, 0.08753317000810057, 0.08753915200941265, 0.08747350890189409, 0.08754018205218017, 0.08749563398305327, 0.08758070901967585, 0.08738152298610657, 0.08757220103871077, 0.08750542299821973, 0.08754349092487246, 0.08744878601282835, 0.08742176205851138, 0.08765547897201031, 0.0873998450115323, 0.08764852094464004, 0.08745064109098166, 0.08759302995167673, 0.08746362803503871, 0.08749128691852093, 0.08758965309243649, 0.08741924993228167, 0.08758612698875368, 0.08740167506039143, 0.08760802703909576, 0.0874587669968605, 0.08750965795479715, 0.08752775494940579, 0.08752966206520796, 0.08756332099437714, 0.08736445603426546, 0.08756049198564142, 0.08740890200715512, 0.08759783091954887, 0.0877236929954961, 0.08754801109898835, 0.08756315696518868, 0.08739170397166163, 0.08762621495407075, 0.08737625100184232, 0.08759151108097285, 0.08747103193309158, 0.08753448899369687, 0.08754514204338193, 0.08744393696542829, 0.08748608699534088, 0.08736528002191335, 0.08758196502458304, 0.08743782097008079, 0.08757246204186231, 0.08751348697114736, 0.08746158995199949, 0.08763144607655704, 0.08757511503063142, 0.08769369497895241, 0.08750752592459321, 0.08752989606000483, 0.08741254196502268, 0.08748942997772247, 0.08753895806148648, 0.08742611901834607, 0.08761006593704224, 0.0873889570357278, 0.08766057598404586, 0.088385576964356, 0.0875738300383091, 0.08839140797499567, 0.08743865008000284, 0.08742664696183056, 0.08742006996180862, 0.08845162903890014, 0.08736252796370536, 0.08760318299755454, 0.08829784498084337, 0.08752832398749888, 0.0875224550254643, 0.08759911800734699, 0.08765854802913964, 0.08744005102198571, 0.08752959303092211, 0.08733747096266598, 0.08847960398998111, 0.08747229201253504, 0.08748695999383926, 0.08752830803859979, 0.08752264094073325, 0.08836955798324198, 0.08736784500069916, 0.08755626005586237, 0.08731979294680059, 0.08836921001784503, 0.08737249404657632, 0.08737034990917891, 0.08744685002602637, 0.08734847197774798, 0.08752643608022481, 0.08737606194335967, 0.08741366700269282, 0.08739743405021727, 0.08739615394733846, 0.08750012400560081, 0.08740893297363073, 0.08749570907093585, 0.08732512092683464, 0.0875326810637489, 0.08736691693775356, 0.08741704002022743, 0.087428729981184, 0.08737570908851922, 0.08745888900011778, 0.08738049690145999, 0.0875069530447945, 0.08734005701262504, 0.08748187404125929, 0.08736453496385366, 0.08738769602496177, 0.08745615999214351, 0.08735322102438658, 0.08744497399311513, 0.08735888299997896, 0.08751844789367169, 0.08735909801907837, 0.08741779008414596, 0.08745500491932034, 0.08739021001383662, 0.08742717106360942, 0.0874041139613837, 0.08755351696163416, 0.08737708698026836, 0.08749967999756336, 0.0874230150366202, 0.08739596197847277, 0.08752465003635734, 0.08737650699913502, 0.08754289394710213, 0.08730510110035539, 0.08747017593123019, 0.08737412898335606, 0.08745762403123081, 0.08740172104444355, 0.08747045195195824, 0.0874782269820571, 0.08741101808845997, 0.08750164799857885, 0.08734714589081705, 0.08760687103495002, 0.08741879498120397, 0.08743509708438069, 0.08746127097401768, 0.08740481198765337, 0.08752462395932525, 0.08732053800486028, 0.08751448802649975, 0.08738333196379244, 0.08751269907224923, 0.08746365795377642, 0.08736979006789625, 0.08750496990978718, 0.08729277399834245, 0.08753500506281853, 0.08731714193709195, 0.0875298980390653, 0.08748460502829403, 0.08746825100388378, 0.08744323393329978, 0.08737165504135191, 0.0875133000081405, 0.08729116793256253, 0.08750095800496638, 0.0874106390401721, 0.08777708804700524, 0.0874432569835335, 0.08743940701242536, 0.08755109598860145, 0.08734330197330564, 0.08750639203935862, 0.08731459290720522, 0.087404819088988, 0.08738513896241784, 0.08744286303408444, 0.08750540693290532, 0.08738302707206458, 0.0875298649771139, 0.08727811893913895, 0.08742953010369092, 0.08742645790334791, 0.08737461606506258, 0.08748149091843516, 0.08737966208718717, 0.0875238929875195, 0.08728350896853954, 0.08749642898328602, 0.08753451402299106, 0.0875812079757452, 0.08737783902324736, 0.08736488595604897, 0.0875496621010825, 0.0873270999873057, 0.08755458099767566, 0.0872197370044887, 0.08746577601414174, 0.08743815193884075, 0.08740285504609346, 0.08740207192022353, 0.08744084008503705, 0.08739835501182824, 0.08735531091224402, 0.08745894802268595, 0.0873536110157147, 0.08741788403131068, 0.08738968090619892, 0.08743588300421834, 0.08743247110396624, 0.08729026291985065, 0.08750260900706053, 0.08737128600478172, 0.08742061199154705, 0.08739974303171039, 0.08743045898154378, 0.08751907595433295, 0.08744539401959628, 0.08758158702403307, 0.08727935503702611, 0.08750478795263916, 0.0872955369995907, 0.0874624770367518, 0.08747623697854578, 0.08759529504459351, 0.08727046300191432, 0.08725418196991086, 0.08753380295820534, 0.0873279890511185, 0.08740658895112574, 0.0874170350143686, 0.08744350296910852, 0.08745912904851139, 0.08734931703656912, 0.08750042296014726, 0.08729874703567475, 0.0875290329568088, 0.08738517400342971, 0.0874653389910236, 0.08739005797542632, 0.08741380809806287, 0.08752623398322612, 0.08735381590668112, 0.08754405507352203, 0.0874060170026496, 0.08756592101417482, 0.08743449195753783, 0.08751112397294492, 0.08744948299136013, 0.08741780300624669, 0.08751730003859848, 0.08738492394331843, 0.0874751100782305, 0.08734087401535362, 0.0874923539813608, 0.08743574295658618, 0.08740680804476142, 0.08755715598817915, 0.08735275699291378, 0.08749593293759972, 0.08725849201437086, 0.08749503502622247, 0.08738091704435647, 0.08746889792382717, 0.087493936996907, 0.08743659500032663, 0.08752119401469827, 0.08742047206033021, 0.08752597903367132, 0.08737466298043728, 0.08747519599273801, 0.08743503992445767, 0.08743307099211961, 0.08753039909061044, 0.08736288791988045, 0.0875796660548076, 0.08729380997829139, 0.08750027196947485, 0.0874444079818204, 0.08747173601295799, 0.08744679403025657, 0.0873446479672566, 0.08754886605311185, 0.08745160594116896, 0.0875841120723635, 0.08740645996294916, 0.08744411100633442, 0.08738229796290398, 0.08736685407347977, 0.08745189197361469, 0.08727751194965094, 0.08749154501128942, 0.0873129490064457, 0.08750368398614228, 0.08732877403963357, 0.08739894698373973, 0.08748861006461084, 0.08729612594470382, 0.08757377206347883, 0.0873394189402461, 0.08748082898091525, 0.08729237306397408, 0.08741837495472282, 0.08742161199916154, 0.087373310001567, 0.08745959703810513, 0.08722648397088051, 0.08752636704593897, 0.08740638592280447, 0.08738015603739768, 0.08738846995402128, 0.08738612709566951, 0.08745084097608924, 0.08725726802367717, 0.08755854598712176, 0.08729980990756303, 0.08751234202645719, 0.08730505197308958, 0.08749855705536902, 0.08743313304148614, 0.0873016269179061, 0.08747733803465962, 0.08731914195232093, 0.08746970305219293, 0.08733124600257725, 0.08742946700658649, 0.08745289302896708, 0.08737698895856738, 0.08738836902193725, 0.08740994695108384, 0.08775665401481092, 0.08736394497100264, 0.0874819440068677, 0.08736535906791687, 0.08744941500481218, 0.08737655798904598, 0.0873780669644475, 0.08746880199760199, 0.0874105190159753, 0.08746824599802494, 0.08731046901084483, 0.08748659503180534, 0.08740143291652203, 0.08733391005080193, 0.08734132093377411, 0.08736939006485045, 0.08748598396778107, 0.08728474599774927, 0.08763504796661437, 0.08729099610354751, 0.08746006595902145, 0.08738327398896217, 0.08734344295226038, 0.08747075800783932, 0.08730233204551041, 0.08745767804794014, 0.08728739095386118, 0.08758270705584437, 0.08758703188505024, 0.087368339067325, 0.0875139000127092, 0.08736102795228362, 0.08753808902110904, 0.08735017501749098, 0.08739898703061044, 0.08731907198671252, 0.08743122196756303, 0.0874889160040766, 0.08744965295773, 0.08748492505401373, 0.08727150899358094, 0.0874759720172733, 0.08735065197106451, 0.08747391903307289, 0.08737709000706673, 0.08738715795334429, 0.0874300489667803, 0.08733040106017143, 0.08748324296902865, 0.08733311598189175, 0.08742787002120167, 0.08737011102493852, 0.08745817502494901, 0.08748007100075483, 0.08736269397195429, 0.08751626394223422, 0.08725311607122421, 0.08751719002611935, 0.08737237995956093, 0.08739763498306274, 0.08747866796329618, 0.08751141803804785, 0.08744797995314002, 0.08731296507176012, 0.08755285199731588, 0.08737813995685428, 0.08742383902426809, 0.08739361097104847, 0.087434550980106, 0.08740805799607188, 0.08746945904567838, 0.08743446599692106, 0.08732832898385823, 0.0874586229911074, 0.08733720402233303, 0.08744517900049686, 0.08740329905413091, 0.08738453791011125, 0.0874791539972648, 0.08732462406624109, 0.08753640693612397, 0.08732162904925644, 0.08743870304897428, 0.0873857740079984, 0.08738452498801053, 0.08746121800504625, 0.08737366693094373, 0.08748702704906464, 0.08734232792630792, 0.08755869301967323, 0.08723773702513427, 0.08736629202030599, 0.08746394503396004, 0.08733188291080296, 0.08746431407053024, 0.087283446919173, 0.08748841099441051, 0.08737320802174509, 0.0874546670820564, 0.08734960993751884, 0.08736429898999631, 0.08746353501919657, 0.0872920589754358, 0.08752979501150548, 0.08732275804504752, 0.08749386202543974, 0.08732843189500272, 0.08749709010589868, 0.08736279094591737, 0.0873648039996624, 0.08752100705169141, 0.0872800590004772, 0.08751169592142105, 0.08727446605917066, 0.08742606895975769, 0.08743873599451035, 0.08736771496478468, 0.0873924270272255, 0.08736738900188357, 0.0874202799750492, 0.08731345506384969, 0.08749941899441183, 0.08737388299778104, 0.08744163298979402, 0.08743169601075351, 0.0873985739890486, 0.08745797199662775, 0.08723523397929966, 0.08747973502613604, 0.08734555204864591, 0.08742621389683336, 0.08734004909638315, 0.08739472599700093, 0.08747210097499192, 0.08737618592567742, 0.087422184064053, 0.08733857003971934, 0.08748720295261592, 0.08737095003016293, 0.0874425689689815, 0.08739366196095943, 0.08734304702375084, 0.08745688397902995, 0.0873308089794591, 0.08754492807202041, 0.08731606102082878, 0.08743520791176707, 0.08738466503564268, 0.08739832800347358, 0.08752275095321238, 0.08724123309366405, 0.08747313101775944, 0.08733482693787664, 0.08746857906226069, 0.08733070595189929, 0.08742795197758824, 0.08744460996240377, 0.08735885808710009, 0.08749053801875561, 0.08727893698960543, 0.08747603895608336, 0.08738198596984148, 0.08744530007243156, 0.08736608095932752, 0.0874591029714793, 0.08746060205157846, 0.08732546295505017, 0.08752761606592685, 0.08736257790587842, 0.08746222802437842, 0.08725102199241519, 0.08740086399484426, 0.08747492404654622, 0.08734397904481739, 0.08744969998952001, 0.0873352539492771, 0.08739158895332366, 0.08744398504495621, 0.08733064599800855, 0.08740403095725924, 0.08742969401646405, 0.08743802306707948, 0.0873392199864611, 0.08745671401266009, 0.0872701199259609, 0.08744760509580374, 0.08734281198121607, 0.08746315399184823, 0.08740999700967222, 0.0873936639400199, 0.0875016680220142, 0.08731091802474111, 0.08749755600001663, 0.08728841796983033, 0.08755787496920675, 0.08770412509329617, 0.08736841299105436, 0.08751894894521683, 0.08736574300564826, 0.08749298995826393, 0.08731286402326077, 0.08743790804874152, 0.08731223398353904, 0.08744689298328012, 0.08741623302921653, 0.08733078092336655, 0.08751454309094697, 0.08732732001226395, 0.08752635889686644, 0.08738987101241946, 0.08748031500726938, 0.08744025800842792, 0.0873851920478046, 0.08749955403618515, 0.0874097018968314, 0.08744633104652166, 0.08727803395595402, 0.08751635299995542, 0.08736005600076169, 0.08746450103353709, 0.08744732500053942, 0.08742133306805044, 0.08743472793139517, 0.08735318796243519, 0.08753066102508456, 0.08738032006658614, 0.08746167190838605, 0.08748269209172577, 0.08766756893601269, 0.0876272190362215, 0.08730826003011316, 0.08748134400229901, 0.08739974501077086, 0.08748842589557171, 0.08739889704156667, 0.08744854805991054, 0.0874811839312315, 0.08738730603363365, 0.08755539101548493, 0.08729520800989121, 0.08753720298409462, 0.08737935498356819, 0.08738932595588267, 0.08744161401409656, 0.0874466240638867, 0.08747628098353744, 0.08737639896571636, 0.08749874494969845, 0.08739336603321135, 0.08751032897271216, 0.08737048308830708, 0.08732063695788383, 0.08742276101838797, 0.0873914189869538, 0.0874805300263688, 0.0873106139479205, 0.08752047701273113, 0.08734480699058622, 0.08744341298006475, 0.08742420806083828, 0.08736981602851301, 0.08744786493480206, 0.08732536900788546, 0.08755559800192714, 0.08734790899325162, 0.08747271995525807, 0.08737148309592158, 0.08742448897100985, 0.08754354901611805, 0.08748466498218477, 0.08755290193948895, 0.08736781100742519, 0.08750223100651056, 0.08737608906812966, 0.08748348092194647, 0.0874285640893504, 0.08736462192609906, 0.08747770101763308, 0.0872778600314632, 0.08751872496213764, 0.08735880407039076, 0.08745187590830028, 0.08740718709304929, 0.08743045700248331, 0.08746682293713093, 0.08740934496745467, 0.0875070960028097, 0.08739694906398654, 0.08748333703260869, 0.08733772591222078, 0.08745099499356002, 0.08742191304918379, 0.0874330869410187, 0.08747110399417579, 0.08735887706279755, 0.08753623103257269, 0.08737999899312854, 0.08752081892453134, 0.08740055607631803, 0.08743471594061702, 0.08744373708032072, 0.08737448195461184, 0.08752853400073946, 0.08731653995346278, 0.08752934506628662, 0.08729008596856147, 0.08744691498577595, 0.08740984497126192, 0.08744135010056198, 0.08745143399573863, 0.08737353491596878, 0.08752815902698785, 0.08729584503453225, 0.08746181998867542, 0.08734174794517457, 0.08747059607412666, 0.08748025901149958, 0.08732938289176673, 0.08747706608846784, 0.08730324392672628, 0.08745352702680975, 0.08734053606167436, 0.08740721992217004, 0.08732625399716198, 0.08738803002052009, 0.08749461895786226, 0.08732650708407164, 0.08750936598517, 0.08732006594073027, 0.08747909008525312, 0.08740640291944146, 0.0873732800828293, 0.08744062192272395, 0.08729501906782389, 0.08753346302546561, 0.08736535592470318, 0.08746244199573994, 0.0873343599960208, 0.0874246220337227, 0.08738045301288366, 0.0872990059433505, 0.087445009034127, 0.08735120005439967, 0.08751975698396564, 0.08731774799525738, 0.08745179895777255, 0.08737258799374104, 0.08737955300603062, 0.08748610806651413, 0.10862210090272129, 0.09219435707200319, 0.09217357099987566, 0.09224512102082372, 0.09215296397451311, 0.0921607839409262, 0.09209811198525131, 0.0921127600595355, 0.09215297794435173, 0.09214425005484372, 0.0921968889888376, 0.09210553800221533, 0.09222706395667046, 0.09209025802556425, 0.09236405801493675, 0.09192251099739224, 0.09232125605922192, 0.09206420090049505, 0.09232780209276825, 0.09201401995960623, 0.09226787101943046, 0.09206473594531417, 0.09230716899037361, 0.09201061609201133, 0.09221729589626193, 0.09211504401173443, 0.09219123504590243, 0.09218828903976828, 0.0921474329661578, 0.09224745002575219, 0.09209472092334181, 0.09209767903666943, 0.09215482894796878, 0.0921810120344162, 0.09210415103007108, 0.09217817999888211, 0.09223545796703547, 0.09217240498401225, 0.09222999203484505, 0.09200545703060925, 0.09228510793764144, 0.09202570701017976, 0.09230444603599608, 0.09202274994459003, 0.0923755670664832, 0.09209222195204347, 0.09226062102243304, 0.09200418298132718, 0.09224976506084204, 0.09201096592005342, 0.0922016630647704, 0.09212675993330777, 0.09215332206804305, 0.09209366899449378, 0.09220454294700176, 0.0921524140285328, 0.09209398296661675, 0.09224163007456809, 0.09207644092384726, 0.0920910449931398, 0.09213681204710156, 0.09211795905139297, 0.09224248491227627, 0.09197979897726327, 0.09227764909155667, 0.09203968092333525, 0.09232056699693203, 0.09211026807315648, 0.09236158500425518, 0.09199582599103451, 0.09233420598320663, 0.09195311099756509, 0.09226551104802638, 0.09209330391604453, 0.0922466010088101, 0.09215254499576986, 0.09219130699057132, 0.09216768899932504, 0.09215556201525033, 0.09203745506238192, 0.09211888199206442, 0.09227565198671073, 0.09208122396375984, 0.09222243900876492, 0.09215085196774453, 0.092248177039437, 0.09223261405713856, 0.09202911495231092, 0.0921893670456484, 0.09206846298184246, 0.09228938794694841, 0.09201864607166499, 0.09227793791797012, 0.09205454704351723, 0.09237718395888805, 0.091858524014242, 0.09224454604554921, 0.09202646499034017, 0.09229579800739884, 0.09209258994087577, 0.09218572604004294, 0.09208842401858419, 0.09226580697577447, 0.09204603196121752, 0.09213922102935612, 0.09218776295892894, 0.0921651110984385, 0.09225438698194921, 0.09214188891928643, 0.09220850607380271, 0.09221710893325508, 0.09202665707562119, 0.09222343901637942, 0.0921464249258861, 0.09236225404310971, 0.0927153779193759, 0.0923294760286808, 0.09203617100138217, 0.09231986803933978, 0.0919750640168786, 0.09234195202589035, 0.09205771889537573, 0.09234901505988091, 0.09212691604625434, 0.09220063989050686, 0.09213949309196323, 0.09228976292070001, 0.09204570401925594, 0.09222246496938169, 0.09225213003810495, 0.09222214005421847, 0.09226209996268153, 0.09214704600162804, 0.09226855100132525, 0.09217820200137794, 0.09259787201881409, 0.09225098800379783, 0.09226768696680665, 0.0922852250514552, 0.09217450395226479, 0.09226924704853445, 0.09215275896713138, 0.092367002973333, 0.0919745359569788, 0.0923966180998832, 0.0920045409584418, 0.09232689195778221, 0.09224103100132197], "tpot_s": 0.08815657022183365, "active_requests_after": 0, "runner_state_count_after": 0} -{"phase": "request", "arm": "graph", "case": "gsm8k/test/783", "index": 4, "cap": 16, "ignore_eos": true, "prompt_tokens": 149, "prompt_sha256": "f2233be4ebb7b122d80433fe4081fdabeda64f735b7bc24f9706c4c4339b824f", "token_ids": [1206, 11290, 411, 3377, 11, 567, 1144, 310, 7995, 279, 1324, 314, 4022, 7397, 13853, 10471], "tokens": 16, "terminal": ["length"], "usage": [{"prompt_tokens": 149, "completion_tokens": 16, "total_tokens": 165}], "output_text_sha256": "4b5d6370a617c539db1e6ee0df2c726c7165c1e9395eab1b873ff700a997510a", "wire_sha256": "dea4820b34797b96c746b40c23dbea2743b33e34377054957236105b2dea2c9b", "client_success": true, "client_error": false, "valid": true, "ttft_s": 2.7129198129987344, "latency_s": 4.026040954980999, "itl": [0.08838639489840716, 0.08739197510294616, 0.08755896589718759, 0.08755458903033286, 0.08745490899309516, 0.08751123701222241, 0.08739926002454013, 0.08754899096675217, 0.08749840199016035, 0.08742117998190224, 0.08768221002537757, 0.08727028802968562, 0.0874713190132752, 0.08752693398855627, 0.08744448702782393], "tpot_s": 0.08754140946548432, "active_requests_after": 0, "runner_state_count_after": 0} -{"phase": "cleanup", "arm": "graph", "exit_code": 0, "forced": false, "closed": true, "active_requests": 0, "runner_state_count": 0} -{"phase": "ready", "arm": "eager", "startup_s": 21.871556909987703, "versions": {"torch": "2.14.1+cu130", "transformers": "5.20.2", "triton": "3.8.0"}, "device": "NVIDIA H20-3e", "backend": "triton_block_fp8", "graph_prewarm": {}} -{"phase": "request", "arm": "eager", "case": "gsm8k/test/783", "index": 0, "cap": 2048, "ignore_eos": false, "prompt_tokens": 149, "prompt_sha256": "f2233be4ebb7b122d80433fe4081fdabeda64f735b7bc24f9706c4c4339b824f", "token_ids": [1206, 11290, 411, 3377, 11, 567, 1144, 310, 7995, 279, 1324, 314, 4022, 7397, 13853, 10471, 3018, 383, 279, 1534, 11013, 314, 18016, 310, 4022, 7397, 303, 780, 34393, 13, 271, 332, 16, 13, 36189, 2891, 279, 2763, 1528, 64700, 198, 9, 256, 3991, 18016, 7397, 25, 220, 18, 198, 9, 256, 3991, 4022, 7397, 25, 220, 19, 198, 9, 256, 10342, 2763, 7397, 25, 220, 22, 271, 332, 17, 13, 36189, 2891, 279, 4203, 64700, 198, 9, 256, 13853, 10471, 220, 16, 22, 7397, 303, 2702, 13, 198, 9, 256, 6558, 393, 86, 3, 381, 279, 1324, 314, 4022, 7397, 10471, 13, 198, 9, 256, 6558, 393, 78, 3, 381, 279, 1324, 314, 18016, 7397, 10471, 13, 198, 9, 256, 1165, 1366, 421, 393, 86, 478, 296, 283, 220, 16, 22, 53031, 864, 23197, 393, 78, 283, 220, 16, 22, 471, 288, 12576, 271, 332, 18, 13, 36189, 2891, 279, 1534, 1528, 64700, 198, 9, 256, 12650, 1324, 314, 4022, 7397, 283, 3991, 4022, 478, 74522, 4022, 283, 393, 19, 478, 288, 3, 198, 9, 256, 12650, 1324, 314, 18016, 7397, 283, 3991, 18016, 478, 74522, 18016, 283, 393, 18, 478, 296, 3, 198, 9, 256, 84142, 393, 78, 283, 220, 16, 22, 471, 288, 3, 1083, 279, 18016, 22868, 25, 198, 262, 12650, 18016, 283, 393, 18, 478, 318, 16, 22, 471, 288, 8, 283, 220, 17, 15, 471, 288, 3, 271, 332, 19, 13, 19927, 279, 2876, 64700, 198, 760, 3377, 5134, 421, 303, 279, 809, 11, 13853, 995, 10598, 430, 1599, 18016, 7397, 430, 4022, 7397, 13, 198, 13682, 1088, 1272, 90, 18770, 21281, 92, 283, 220, 17, 1088, 14695, 1088, 1272, 90, 18770, 5626, 92, 25273, 271, 3036, 7417, 279, 22666, 494, 2923, 220, 18, 1083, 411, 22868, 25, 198, 13682, 220, 17, 15, 471, 288, 283, 220, 17, 1088, 14695, 318, 19, 478, 288, 8, 25273, 271, 332, 20, 13, 61148, 364, 393, 86, 60439, 332, 198, 13682, 220, 17, 15, 471, 288, 283, 220, 23, 478, 220, 17, 86, 25273, 198, 2143, 393, 86, 3, 310, 2107, 10744, 25, 198, 13682, 220, 17, 15, 283, 220, 23, 478, 220, 18, 86, 25273, 198, 3036, 2076, 220, 23, 494, 2107, 10744, 25, 198, 13682, 220, 16, 17, 283, 220, 18, 86, 25273, 198, 12151, 562, 539, 220, 18, 25, 198, 13682, 288, 283, 220, 19, 25273, 271, 332, 21, 13, 53611, 64700, 198, 9, 256, 1368, 13853, 10471, 220, 19, 4022, 7397, 11, 551, 10471, 393, 16, 22, 471, 220, 19, 283, 220, 16, 18, 3, 18016, 7397, 13, 198, 9, 256, 12650, 4022, 7397, 25, 393, 19, 478, 220, 19, 283, 220, 23, 3, 198, 9, 256, 12650, 18016, 7397, 25, 393, 18, 478, 220, 16, 18, 283, 220, 16, 21, 3, 198, 9, 256, 2091, 220, 16, 21, 10598, 430, 1599, 430, 220, 23, 30, 7179, 11, 393, 16, 21, 283, 220, 17, 1088, 14695, 220, 23, 12576, 271, 760, 1324, 314, 4022, 7397, 13853, 10471, 369, 220, 19, 13, 271, 794, 220, 19, 248046], "tokens": 503, "terminal": ["stop"], "usage": [{"prompt_tokens": 149, "completion_tokens": 503, "total_tokens": 652}], "output_text_sha256": "6e7b3fd3ae3c464a07b9014602a22a121e1a85fda0d3458a42a21dbb66708bf0", "wire_sha256": "0190c4550c6e071e9bef20f6725f24082bbb4d629b6fd257875334f9cbb3f814", "client_success": true, "client_error": false, "valid": true, "ttft_s": 3.526063673081808, "latency_s": 49.761497988016345, "itl": [0.1443011088995263, 0.09224375209305435, 0.09402337891515344, 0.09186256502289325, 0.09198092599399388, 0.09187716897577047, 0.09194194409064949, 0.09200621000491083, 0.09195813001133502, 0.09196901391260326, 0.09228342701680958, 0.09203403000719845, 0.09177269705105573, 0.09195708192419261, 0.09191116108559072, 0.09198117698542774, 0.09179784695152193, 0.09184921905398369, 0.09189187793526798, 0.09193509398028255, 0.09194973902776837, 0.09191990399267524, 0.09197579498868436, 0.09190719807520509, 0.09198754199314862, 0.09184529399499297, 0.09193790494464338, 0.09183291799854487, 0.09192062902729958, 0.09193777898326516, 0.09187620901502669, 0.09202477405779064, 0.09192382893525064, 0.09193624905310571, 0.09186607890296727, 0.0920818290906027, 0.09173844393808395, 0.09202720504254103, 0.09182030393276364, 0.0920386390062049, 0.09182934800628573, 0.09213398501742631, 0.09225882601458579, 0.09199530095793307, 0.09189207304734737, 0.0919452760135755, 0.0918796620098874, 0.09179574600420892, 0.09199754393193871, 0.0918444300768897, 0.09244179097004235, 0.09189065895043314, 0.09199236903805286, 0.09186714596580714, 0.09197114000562578, 0.09225387708283961, 0.09194837999530137, 0.09202081500552595, 0.09187306999228895, 0.09204381098970771, 0.09177846496459097, 0.09208885801490396, 0.09185343096032739, 0.09202803100924939, 0.09177761897444725, 0.09199591807555407, 0.0918588339118287, 0.09193431108724326, 0.09220563992857933, 0.09200556098949164, 0.09187844209372997, 0.09187805000692606, 0.0919792529894039, 0.09186373301781714, 0.09192861593328416, 0.09181456500664353, 0.09195915597956628, 0.09195321204606444, 0.09196262003388256, 0.09197144990321249, 0.09196936199441552, 0.09198817703872919, 0.0918399749789387, 0.09203548904042691, 0.09181102795992047, 0.09204122400842607, 0.09181969205383211, 0.09205643297173083, 0.09183611499611288, 0.0919809479964897, 0.09181870601605624, 0.09205223503522575, 0.09184049791656435, 0.09193039301317185, 0.0919629429699853, 0.09193570399656892, 0.09201417001895607, 0.0918527360772714, 0.09190089197363704, 0.09183835203293711, 0.0919715529307723, 0.09192327898927033, 0.09198142099194229, 0.09191611304413527, 0.09193281398620456, 0.09192960301879793, 0.09186778194271028, 0.09208497509825975, 0.09195678099058568, 0.09210507699754089, 0.09186591091565788, 0.0921166380867362, 0.09186407690867782, 0.09219125506933779, 0.09199276100844145, 0.0920513019664213, 0.09199996897950768, 0.09205329103861004, 0.09190493600908667, 0.09195400902535766, 0.09210690890904516, 0.0919089320814237, 0.09198823501355946, 0.09181525697931647, 0.09202063595876098, 0.09194621304050088, 0.09199847793206573, 0.09180488903075457, 0.09200000297278166, 0.0920174130005762, 0.09192413405980915, 0.09202082199044526, 0.09186982701066881, 0.09201039595063776, 0.09191653900779784, 0.0920713460072875, 0.09184225799981505, 0.09210588096175343, 0.09176929399836808, 0.09201044810470194, 0.09192623198032379, 0.09202473191544414, 0.091961967991665, 0.09196101408451796, 0.0919491439126432, 0.09187954606022686, 0.09250380494631827, 0.09197906404733658, 0.09200468403287232, 0.09194122499320656, 0.09202444297261536, 0.09189538494683802, 0.09202474902849644, 0.09197582700289786, 0.09194383001886308, 0.09202835999894887, 0.09186058701016009, 0.09202579001430422, 0.0918473299825564, 0.09208992100320756, 0.09179497300647199, 0.09215086803305894, 0.09231678093783557, 0.09207412996329367, 0.09186570509336889, 0.09199171594809741, 0.09181201306637377, 0.09204630192834884, 0.09211200301069766, 0.0919800839619711, 0.09198635909706354, 0.09194301092065871, 0.09194939699955285, 0.09199467301368713, 0.09201631403993815, 0.09202651400119066, 0.09199830994475633, 0.09193027205765247, 0.09196662600152194, 0.09202416497282684, 0.09190998296253383, 0.09201880008913577, 0.09182158298790455, 0.09201593894977123, 0.0919551580445841, 0.09207627095747739, 0.09175184997729957, 0.09209620405454189, 0.09186250099446625, 0.09208256495185196, 0.09180990408640355, 0.09200860094279051, 0.09191123698838055, 0.09389399201609194, 0.09186772699467838, 0.09183129400480539, 0.09200594900175929, 0.09186112799216062, 0.09204067604150623, 0.09182890201918781, 0.0920624869177118, 0.09191560407634825, 0.09193113399669528, 0.09194376692175865, 0.09189166803844273, 0.09201231400948018, 0.0919065389316529, 0.09190791903529316, 0.09189838997554034, 0.09200519102159888, 0.09185193607117981, 0.09208281093742698, 0.09180709102656692, 0.09208289999514818, 0.09188725601416081, 0.09207031095866114, 0.09197810001205653, 0.09195577597711235, 0.09194667299743742, 0.09193145902827382, 0.09204712696373463, 0.0918900080723688, 0.09201865398790687, 0.09183664002921432, 0.09208964696153998, 0.09186456596944481, 0.09191247005946934, 0.09194580698385835, 0.0919648869894445, 0.09190963197033852, 0.09195724804885685, 0.09199102001730353, 0.09194224898237735, 0.09198544092942029, 0.09245320199988782, 0.09203745203558356, 0.09185109799727798, 0.09205923904664814, 0.09190733300056309, 0.09196145692840219, 0.09194670501165092, 0.09198479598853737, 0.09188352804630995, 0.09202366997487843, 0.09201087197288871, 0.09197598102036864, 0.09195579402148724, 0.09189569903537631, 0.09203707391861826, 0.09182554902508855, 0.09195166605059057, 0.09197592898271978, 0.09214704995974898, 0.09203355701174587, 0.09202062606345862, 0.09201290493365377, 0.09197191195562482, 0.09206066804472357, 0.0918282630154863, 0.09202693204861134, 0.09189742000307888, 0.09197647194378078, 0.0918936999514699, 0.0921628640498966, 0.09198707598261535, 0.0920462830690667, 0.09190195892006159, 0.09198098699562252, 0.09196193004027009, 0.0918568120105192, 0.09206663398072124, 0.09186042298097163, 0.09221719799097627, 0.09176980098709464, 0.09199500898830593, 0.09183255105745047, 0.09202917199581861, 0.09196545497979969, 0.09200441604480147, 0.09191235702019185, 0.09194842493161559, 0.09202645299956203, 0.09183953399769962, 0.09199018008075655, 0.0918576359981671, 0.09205077600199729, 0.09190611890517175, 0.09216789598576725, 0.09197891503572464, 0.09206361800897866, 0.09186747495550662, 0.09227910509798676, 0.09194772399496287, 0.09206494898535311, 0.09199234494008124, 0.09197358402889222, 0.09208075795322657, 0.09192346304189414, 0.09203884098678827, 0.09187092806678265, 0.09212114696856588, 0.09196292294654995, 0.09197464503813535, 0.09204698202665895, 0.09194817102979869, 0.09197983797639608, 0.09191995591390878, 0.09206154302228242, 0.09195670997723937, 0.09213887806981802, 0.09196387103293091, 0.09207300399430096, 0.09184621495660394, 0.09211540804244578, 0.09185584099031985, 0.09205188497435302, 0.09204794897232205, 0.09196753206197172, 0.09196118894033134, 0.09202848700806499, 0.09208037599455565, 0.09192918904591352, 0.09194114501588047, 0.09197006293106824, 0.09213189303409308, 0.09203823795542121, 0.09213909006211907, 0.09194598998874426, 0.09214901097584516, 0.09204034006688744, 0.09202432888559997, 0.09200043405871838, 0.09190859296359122, 0.09213915804866701, 0.09191993402782828, 0.09207077696919441, 0.09183804993517697, 0.09209828602615744, 0.09180301602464169, 0.09204278397373855, 0.09199333703145385, 0.09200408903416246, 0.09284315991681069, 0.09198287699837238, 0.09185721003450453, 0.09269890701398253, 0.09204089804552495, 0.09193588199559599, 0.09289738198276609, 0.09180062497034669, 0.09200102998875082, 0.09280767198652029, 0.09219537302851677, 0.0919760880060494, 0.09198721800930798, 0.09205805801320821, 0.09196493495255709, 0.09284373407717794, 0.09192581393290311, 0.09205086901783943, 0.09193550399504602, 0.09210087300743908, 0.09182440396398306, 0.09296444605570287, 0.09189629892352968, 0.0920627360465005, 0.09197159495670348, 0.09201280400156975, 0.09276757598854601, 0.09196050104219466, 0.09194578498136252, 0.09199922904372215, 0.09205461095552891, 0.09208558104000986, 0.09212626598309726, 0.09195356606505811, 0.09204985399264842, 0.09194027294870466, 0.09199002396780998, 0.09201086300890893, 0.09194647800177336, 0.0920391840627417, 0.09193242993205786, 0.09203187806997448, 0.09191334899514914, 0.09207411692477763, 0.09178239200264215, 0.09217557904776186, 0.09182218695059419, 0.0920563000254333, 0.09187917097005993, 0.09197937708813697, 0.09196109394542873, 0.09196338499896228, 0.09191417705733329, 0.09198456292506307, 0.09198457398451865, 0.09189849207177758, 0.09205702797044069, 0.09187101200222969, 0.09206398704554886, 0.09187700098846108, 0.09191697300411761, 0.09195315395481884, 0.09202553296927363, 0.0920100110815838, 0.09194193000439554, 0.09199018497020006, 0.09199737594462931, 0.09212241705972701, 0.09180400997865945, 0.09204111795406789, 0.0918165760813281, 0.09211194596718997, 0.09192395501304418, 0.09213919192552567, 0.09194347308948636, 0.09201025194488466, 0.09186015604063869, 0.09200910700019449, 0.09212149295490235, 0.09196296602021903, 0.09198852896224707, 0.09185617906041443, 0.09215936902910471, 0.09194810793269426, 0.09210918098688126, 0.09191676403861493, 0.09199977898970246, 0.09199761098716408, 0.09193326800595969, 0.09202251001261175, 0.09194356296211481, 0.09207764302846044, 0.09181745199020952, 0.09216651297174394, 0.09187184309121221, 0.09216590190771967, 0.09189419401809573, 0.09214189008343965, 0.09194227494299412, 0.09214495006017387, 0.09237800899427384, 0.0920593929477036, 0.09197932295501232, 0.09198340901639313, 0.09206365700811148, 0.091921188053675, 0.09201429202221334, 0.09191584889777005, 0.091990398010239, 0.09190274903085083, 0.0919989210087806, 0.09203283803071827, 0.09204310295172036, 0.09195302100852132, 0.09196054795756936, 0.09206981910392642, 0.09187535394448787, 0.09216318000108004, 0.09188670897856355, 0.09219658002257347, 0.09186489100102335, 0.09215492603834718, 0.09196465299464762, 0.09210475301370025, 0.09190128499176353, 0.09206780896056443, 0.09199369105044752, 0.09201163693796843, 0.09199491399340332, 0.09193619398865849, 0.0920256320387125, 0.09190949297044426, 0.09199708898086101, 0.09198719798587263, 0.09206545609049499, 0.0919956419384107, 0.0920642539858818, 0.09198082098737359, 0.09196290909312665, 0.09202854696195573, 0.09189439902547747, 0.09204338700510561, 0.09189596399664879, 0.09212927601765841, 0.09187150397337973, 0.09216378501150757, 0.09188175201416016, 0.0920628699241206, 0.09181653300765902, 0.09211574005894363, 0.09237277996726334, 0.09199070802424103, 0.09203027398325503, 0.09194268495775759, 0.09199426101986319, 0.09197113895788789, 0.09194974403362721, 0.0919554199790582, 0.09202354098670185, 0.09218784503173083], "tpot_s": 0.09210245879469031, "active_requests_after": 0, "runner_state_count_after": 0} -{"phase": "request", "arm": "eager", "case": "gsm8k/test/1", "index": 1, "cap": 2048, "ignore_eos": false, "prompt_tokens": 70, "prompt_sha256": "78d335abe81363d4bb48e5aa256c120344c2ceadbcf8739afe4ec3923c747238", "token_ids": [1206, 7995, 279, 2702, 1324, 314, 47205, 2483, 364, 279, 60507, 11, 567, 1144, 310, 10724, 279, 3202, 314, 4022, 23043, 321, 884, 424, 310, 279, 3202, 314, 6105, 23043, 13, 271, 16, 13, 220, 2972, 10025, 23043, 92217, 561, 3377, 5134, 421, 279, 60507, 4829, 220, 17, 47205, 314, 6105, 23043, 13, 198, 17, 13, 220, 2972, 13672, 23043, 92217, 561, 3377, 5134, 421, 424, 4829, 328, 35778, 421, 1691, 1, 4022, 23043, 13, 328, 4639, 1691, 1, 18675, 310, 279, 3202, 314, 6105, 23043, 13, 198, 262, 348, 256, 71690, 25, 393, 17, 1088, 1272, 90, 47205, 92, 593, 220, 17, 283, 220, 16, 1088, 1272, 90, 30797, 92, 12576, 198, 18, 13, 220, 2972, 7354, 47205, 92217, 2604, 279, 6105, 23043, 321, 279, 4022, 23043, 3658, 13, 198, 262, 348, 256, 71690, 25, 393, 17, 1088, 1272, 90, 47205, 318, 11855, 8866, 478, 220, 16, 1088, 1272, 90, 30797, 318, 5603, 8866, 283, 220, 18, 1088, 1272, 90, 47205, 92, 12576, 271, 794, 220, 18, 248046], "tokens": 169, "terminal": ["stop"], "usage": [{"prompt_tokens": 70, "completion_tokens": 169, "total_tokens": 239}], "output_text_sha256": "53ef84987096a7daa32212812c15c7fce023a5b06f50ea52a2c0e026d0a1f1e6", "wire_sha256": "7eec63e87c579da1d1ba607db57ed9efe90417c1c6b639544d003f139c7a1cfc", "client_success": true, "client_error": false, "valid": true, "ttft_s": 1.2559849520912394, "latency_s": 16.675945091992617, "itl": [0.09187812393065542, 0.09215917298570275, 0.09171861805953085, 0.09175748797133565, 0.0918911820044741, 0.09175886097364128, 0.09186136501375586, 0.09162351698614657, 0.09191581199411303, 0.09164729597978294, 0.0918885130668059, 0.091626042034477, 0.09193414589390159, 0.09172028803732246, 0.09191478602588177, 0.09180231101345271, 0.09183143102563918, 0.09178045194130391, 0.09176972298882902, 0.09188946196809411, 0.09172277804464102, 0.09195102704688907, 0.09170157590415329, 0.09186248399782926, 0.09176168905105442, 0.09178484696894884, 0.09184341703075916, 0.09177924797404557, 0.09182140207849443, 0.09169779194053262, 0.0919224809622392, 0.09186464501544833, 0.09196603007148951, 0.09207313600927591, 0.09195714700035751, 0.09161122690420598, 0.09182659001089633, 0.09170735301449895, 0.09183385607320815, 0.09170566697139293, 0.09177136700600386, 0.09172288398258388, 0.09188121801707894, 0.09180203301366419, 0.09163444093428552, 0.09187781298533082, 0.09168421803042293, 0.09177799196913838, 0.09173856198322028, 0.09183673607185483, 0.09182051196694374, 0.09159477800130844, 0.09181163797620684, 0.0917549270670861, 0.09178372891619802, 0.09161980205681175, 0.09178863698616624, 0.09148883097805083, 0.09194846998434514, 0.09169350599404424, 0.09190093702636659, 0.09168832306750119, 0.0918254959397018, 0.09176858805585653, 0.0917337539140135, 0.09170236601494253, 0.09179432201199234, 0.09222543006762862, 0.09167280094698071, 0.09181598504073918, 0.09169154998380691, 0.09190551599022001, 0.09171011904254556, 0.09171459299977869, 0.09174730791710317, 0.09178288502153009, 0.09178290807176381, 0.09172526595648378, 0.09181655303109437, 0.09165991295594722, 0.09188088902737945, 0.09171727299690247, 0.09188983892090619, 0.09172617609146982, 0.09178921801503748, 0.09171483595855534, 0.09191243804525584, 0.09172385896090418, 0.09181372995954007, 0.09170608501881361, 0.09173331304918975, 0.0918909489409998, 0.09175736596807837, 0.09178063808940351, 0.09169169398956001, 0.0918332519941032, 0.09184690099209547, 0.09172099595889449, 0.09170836408156902, 0.09184926899615675, 0.09179071988910437, 0.09168036200571805, 0.09182954207062721, 0.09171878092456609, 0.09171577205415815, 0.09159320197068155, 0.09189511905424297, 0.09178202203474939, 0.09177273290697485, 0.09163459402043372, 0.09192372707184404, 0.09166941489093006, 0.0917867940152064, 0.09172611602116376, 0.09178068698383868, 0.09179133805446327, 0.09161975502502173, 0.09173683600965887, 0.09169395896606147, 0.09185183595400304, 0.09168944007251412, 0.09172940393909812, 0.09179095702711493, 0.09188309602905065, 0.09217723296023905, 0.091789431986399, 0.09179424401372671, 0.09168638393748552, 0.09181408304721117, 0.09158143796958029, 0.09188800503034145, 0.09166018606629223, 0.09188800095580518, 0.09165590500924736, 0.0918927559396252, 0.0917155280476436, 0.09184991393703967, 0.09165878407657146, 0.09182313398923725, 0.09184005204588175, 0.09173882589675486, 0.09181882604025304, 0.09171948197763413, 0.09191037598066032, 0.09165718907024711, 0.09161473694257438, 0.09177763503976166, 0.09191170602571219, 0.09174155502114445, 0.09166708495467901, 0.09185494703706354, 0.0917081160005182, 0.09181483893189579, 0.0916209340794012, 0.09187581297010183, 0.09159455099143088, 0.09194447693880647, 0.09178345208056271, 0.09192812792025506, 0.09161793801467866, 0.09183583303820342, 0.0917473710142076, 0.09185849502682686, 0.09179077995941043, 0.09178649494424462, 0.09186391904950142, 0.09174079995136708, 0.092087775003165], "tpot_s": 0.09178547702322248, "active_requests_after": 0, "runner_state_count_after": 0} -{"phase": "request", "arm": "eager", "case": "gsm8k/test/424", "index": 2, "cap": 2048, "ignore_eos": false, "prompt_tokens": 106, "prompt_sha256": "03cbdf5804bab62d4efbc0ae89e191bd557840c6d9a1a5ada9669de953a0b783", "token_ids": [1206, 7995, 279, 2702, 1324, 314, 7998, 421, 628, 4973, 279, 1834, 8151, 11, 567, 1144, 310, 10724, 279, 2702, 8390, 314, 660, 279, 47224, 11145, 13, 271, 16, 13, 220, 2972, 46270, 279, 8390, 314, 279, 32275, 64700, 198, 262, 2532, 513, 220, 19, 32275, 11, 321, 1754, 9687, 220, 21, 15, 7998, 13, 198, 262, 25273, 19, 1088, 14695, 220, 21, 15, 283, 220, 17, 19, 15, 1088, 1272, 90, 7998, 92, 13682, 271, 17, 13, 220, 2972, 46270, 279, 8390, 314, 279, 73320, 4623, 64700, 198, 262, 2532, 513, 220, 21, 73320, 4623, 11, 321, 1754, 9687, 220, 18, 15, 7998, 13, 198, 262, 25273, 21, 1088, 14695, 220, 18, 15, 283, 220, 16, 23, 15, 1088, 1272, 90, 7998, 92, 13682, 271, 18, 13, 220, 2972, 46270, 279, 8390, 314, 279, 1262, 340, 581, 64700, 198, 262, 2532, 513, 220, 16, 15, 1262, 340, 581, 11, 321, 1754, 9687, 220, 16, 20, 7998, 13, 198, 262, 25273, 16, 15, 1088, 14695, 220, 16, 20, 283, 220, 16, 20, 15, 1088, 1272, 90, 7998, 92, 13682, 271, 19, 13, 220, 2972, 8917, 279, 56468, 64700, 198, 262, 10342, 7998, 283, 318, 28432, 314, 32275, 8, 478, 318, 28432, 314, 73320, 4623, 8, 478, 318, 28432, 314, 1262, 340, 581, 8, 198, 262, 25273, 17, 19, 15, 478, 220, 16, 23, 15, 478, 220, 16, 20, 15, 13682, 271, 262, 5339, 11, 884, 220, 17, 19, 15, 321, 220, 16, 23, 15, 25, 198, 262, 25273, 17, 19, 15, 478, 220, 16, 23, 15, 283, 220, 19, 17, 15, 13682, 271, 262, 4844, 11, 884, 220, 16, 20, 15, 25, 198, 262, 25273, 19, 17, 15, 478, 220, 16, 20, 15, 283, 220, 20, 22, 15, 13682, 271, 760, 2702, 1324, 314, 7998, 421, 628, 4973, 279, 1834, 8151, 369, 220, 20, 22, 15, 13, 271, 794, 220, 20, 22, 15, 248046], "tokens": 314, "terminal": ["stop"], "usage": [{"prompt_tokens": 106, "completion_tokens": 314, "total_tokens": 420}], "output_text_sha256": "bb797c1732c29c467b45fd24a2a27f80a1a5d8ce8dc38799eb28b29a5c056bb2", "wire_sha256": "0a8d0c7369364d65e2191c6a882a29d49d4ba452cf1800802dbb21ad7e436e82", "client_success": true, "client_error": false, "valid": true, "ttft_s": 1.8804057410452515, "latency_s": 30.63704265409615, "itl": [0.09200994903221726, 0.09188687498681247, 0.09186827903613448, 0.09185629698913544, 0.09189935494214296, 0.0917300300206989, 0.09191106993239373, 0.09177832608111203, 0.09184441203251481, 0.09176273096818477, 0.09184768295381218, 0.09174961305689067, 0.0920221779961139, 0.0919568749377504, 0.09203012206126004, 0.0917795569403097, 0.09194785600993782, 0.09184053307399154, 0.09178312798030674, 0.09198678692337126, 0.09186489507555962, 0.09178452100604773, 0.09169409598689526, 0.09193541493732482, 0.09208949108142406, 0.09195012191776186, 0.09168193105142564, 0.0918164779432118, 0.09185855009127408, 0.0917272549122572, 0.09177555004134774, 0.09172607096843421, 0.09178184601478279, 0.09169432905036956, 0.09205421595834196, 0.09210142400115728, 0.09193753695581108, 0.09154252207372338, 0.09187588596250862, 0.09173929505050182, 0.0918292700080201, 0.09173779690172523, 0.09178261505439878, 0.09178417094517499, 0.09177911409642547, 0.09182625799439847, 0.09189470997080207, 0.09190505603328347, 0.0917537019122392, 0.09210379200521857, 0.09190273308195174, 0.09198030491825193, 0.09180991409812123, 0.09180616994854063, 0.09183281101286411, 0.0918521590065211, 0.09187393100000918, 0.09178574697580189, 0.09191595402080566, 0.09178219398017973, 0.09191121498588473, 0.09164862299803644, 0.09195049304980785, 0.09166684199590236, 0.091965117957443, 0.09180069901049137, 0.09195220400579274, 0.09183024300727993, 0.09189278504345566, 0.09176459698937833, 0.09189772291574627, 0.09194276400376111, 0.09183314407709986, 0.09182875091210008, 0.09177708299830556, 0.09202874707989395, 0.09190036891959608, 0.09184883208945394, 0.09181013400666416, 0.09186737099662423, 0.09184590692166239, 0.09183753002434969, 0.09180629695765674, 0.09177443303633481, 0.09193762706127018, 0.09168804297223687, 0.09200465492904186, 0.09174363105557859, 0.09193096694070846, 0.09175925108138472, 0.09197221894282848, 0.09190107206813991, 0.0920865919906646, 0.09178907296154648, 0.09184262005146593, 0.0919041259912774, 0.09191152499988675, 0.09194130799733102, 0.09172495594248176, 0.09192139597143978, 0.09176820702850819, 0.09192826505750418, 0.09187118895351887, 0.0918091160710901, 0.0918698109453544, 0.09182457497809082, 0.09194378706160933, 0.09186459693592042, 0.0918790950672701, 0.09171739500015974, 0.09204222192056477, 0.09182703401893377, 0.0920179559616372, 0.09167690004687756, 0.09192049503326416, 0.09177273802924901, 0.09190947690512985, 0.09178052202332765, 0.0918357620248571, 0.09189102798700333, 0.09190810401923954, 0.09198118699714541, 0.09176549699623138, 0.09192207001615316, 0.09178955899551511, 0.09185926895588636, 0.09190020908135921, 0.09185345098376274, 0.09183416399173439, 0.09186677902471274, 0.09193713997956365, 0.09187386895064265, 0.09196821600198746, 0.09165451000444591, 0.09188407403416932, 0.09171733097173274, 0.09197870397474617, 0.09170396300032735, 0.09210297802928835, 0.09227552299853414, 0.09206077095586807, 0.09173561504576355, 0.09191935905255377, 0.09176110092084855, 0.09187749505508691, 0.09184482193086296, 0.09183191903866827, 0.09193546394817531, 0.09172336803749204, 0.09194258297793567, 0.09190012398175895, 0.09191645006649196, 0.09177244501188397, 0.0918679719325155, 0.09196386008989066, 0.09182227496057749, 0.09199059195816517, 0.09174741699825972, 0.09206236898899078, 0.09176413202658296, 0.09199651598464698, 0.09178700402844697, 0.09191874298267066, 0.09179043106269091, 0.09196727094240487, 0.09178906807210296, 0.0919611279387027, 0.09185033396352082, 0.09191150299739093, 0.09198110399302095, 0.09184576105326414, 0.09193924895953387, 0.09177402802743018, 0.09192448505200446, 0.09183443698566407, 0.09195233101490885, 0.09197925101034343, 0.09187089593615383, 0.09180693503003567, 0.09181265102233738, 0.0920115759363398, 0.09180345200002193, 0.09199221897870302, 0.09174000401981175, 0.09201592404861003, 0.09175513801164925, 0.09202053502667695, 0.09179665788542479, 0.09186828706879169, 0.09179183398373425, 0.09191579394973814, 0.09185613004956394, 0.09194794995710254, 0.09181380900554359, 0.09193664009217173, 0.09203723398968577, 0.09180078096687794, 0.09189805504865944, 0.09185682190582156, 0.0920248650945723, 0.09189819789025933, 0.09193767909891903, 0.09184252889826894, 0.0918344110250473, 0.09193338104523718, 0.09180224593728781, 0.09195112204179168, 0.09172471601050347, 0.09200476994737983, 0.09173546801321208, 0.09202654799446464, 0.09173404006287456, 0.09206377598457038, 0.09229915600735694, 0.09197077993303537, 0.09186787705402821, 0.09193640702869743, 0.09183834702707827, 0.09188738896045834, 0.09188312396872789, 0.09188090905081481, 0.0919724489795044, 0.09183297795243561, 0.0919504800112918, 0.09189521404914558, 0.09192502894438803, 0.09189227502793074, 0.09181116404943168, 0.09194710094016045, 0.0918529590126127, 0.09192421694751829, 0.09184656909201294, 0.09209633700083941, 0.09171545098070055, 0.09210151201114058, 0.09182063594926149, 0.09195733396336436, 0.09180472604930401, 0.09210245101712644, 0.09186307701747864, 0.09182106400839984, 0.09189035696908832, 0.09185910096857697, 0.09186291706282645, 0.09179899096488953, 0.09181327105034143, 0.0917685569729656, 0.09191078098956496, 0.09184647793881595, 0.09195108304265887, 0.0918697000015527, 0.0918837389908731, 0.09176466206554323, 0.09187452995683998, 0.09200090100057423, 0.09182737895753235, 0.09192544408142567, 0.09176831599324942, 0.09199603798333555, 0.09178732696454972, 0.09198332100640982, 0.09172715002205223, 0.0919249210273847, 0.09175597899593413, 0.09182289300952107, 0.09183405293151736, 0.09185843204613775, 0.09192152495961636, 0.09177491697482765, 0.09188286506105214, 0.09188415401149541, 0.09206979791633785, 0.09176579304039478, 0.09196998702827841, 0.09187714592553675, 0.09214799106121063, 0.0919646320398897, 0.09176963195204735, 0.09189053799491376, 0.09183451102580875, 0.09186775796115398, 0.09178823803085834, 0.0919231059961021, 0.0917589080054313, 0.09200974903069437, 0.0917740958975628, 0.09196048008743674, 0.09182634390890598, 0.09198188909795135, 0.09188845392782241, 0.09192386898212135, 0.09183837403543293, 0.09180418297182769, 0.09188288205768913, 0.09183583897538483, 0.09194233396556228, 0.09182992903515697, 0.09205820003990084, 0.09187681693583727, 0.09195292100775987, 0.0917590360622853, 0.09187108499463648, 0.09181463695131242, 0.09174106596037745, 0.09199445811100304, 0.09179409791249782, 0.0919698600191623, 0.09175242204219103, 0.09190656500868499, 0.0917013919679448, 0.09203385794535279, 0.09178468002937734, 0.09209749207366258], "tpot_s": 0.09187423933882076, "active_requests_after": 0, "runner_state_count_after": 0} -{"phase": "request", "arm": "eager", "case": "gsm8k/test/783", "index": 3, "cap": 1024, "ignore_eos": true, "prompt_tokens": 149, "prompt_sha256": "f2233be4ebb7b122d80433fe4081fdabeda64f735b7bc24f9706c4c4339b824f", "token_ids": [1206, 11290, 411, 3377, 11, 567, 1144, 310, 7995, 279, 1324, 314, 4022, 7397, 13853, 10471, 3018, 383, 279, 1534, 11013, 314, 18016, 310, 4022, 7397, 303, 780, 34393, 13, 271, 332, 16, 13, 36189, 2891, 279, 2763, 1528, 64700, 198, 9, 256, 3991, 18016, 7397, 25, 220, 18, 198, 9, 256, 3991, 4022, 7397, 25, 220, 19, 198, 9, 256, 10342, 2763, 7397, 25, 220, 22, 271, 332, 17, 13, 36189, 2891, 279, 4203, 64700, 198, 9, 256, 13853, 10471, 220, 16, 22, 7397, 303, 2702, 13, 198, 9, 256, 6558, 393, 86, 3, 381, 279, 1324, 314, 4022, 7397, 10471, 13, 198, 9, 256, 6558, 393, 78, 3, 381, 279, 1324, 314, 18016, 7397, 10471, 13, 198, 9, 256, 1165, 1366, 421, 393, 86, 478, 296, 283, 220, 16, 22, 53031, 864, 23197, 393, 78, 283, 220, 16, 22, 471, 288, 12576, 271, 332, 18, 13, 36189, 2891, 279, 1534, 1528, 64700, 198, 9, 256, 12650, 1324, 314, 4022, 7397, 283, 3991, 4022, 478, 74522, 4022, 283, 393, 19, 478, 288, 3, 198, 9, 256, 12650, 1324, 314, 18016, 7397, 283, 3991, 18016, 478, 74522, 18016, 283, 393, 18, 478, 296, 3, 198, 9, 256, 84142, 393, 78, 283, 220, 16, 22, 471, 288, 3, 1083, 279, 18016, 22868, 25, 198, 262, 12650, 18016, 283, 393, 18, 478, 318, 16, 22, 471, 288, 8, 283, 220, 17, 15, 471, 288, 3, 271, 332, 19, 13, 19927, 279, 2876, 64700, 198, 760, 3377, 5134, 421, 303, 279, 809, 11, 13853, 995, 10598, 430, 1599, 18016, 7397, 430, 4022, 7397, 13, 198, 13682, 1088, 1272, 90, 18770, 21281, 92, 283, 220, 17, 1088, 14695, 1088, 1272, 90, 18770, 5626, 92, 25273, 271, 3036, 7417, 279, 22666, 494, 2923, 220, 18, 1083, 411, 22868, 25, 198, 13682, 220, 17, 15, 471, 288, 283, 220, 17, 1088, 14695, 318, 19, 478, 288, 8, 25273, 271, 332, 20, 13, 61148, 364, 393, 86, 60439, 332, 198, 13682, 220, 17, 15, 471, 288, 283, 220, 23, 478, 220, 17, 86, 25273, 198, 2143, 393, 86, 3, 310, 2107, 10744, 25, 198, 13682, 220, 17, 15, 283, 220, 23, 478, 220, 18, 86, 25273, 198, 3036, 2076, 220, 23, 494, 2107, 10744, 25, 198, 13682, 220, 16, 17, 283, 220, 18, 86, 25273, 198, 12151, 562, 539, 220, 18, 25, 198, 13682, 288, 283, 220, 19, 25273, 271, 332, 21, 13, 53611, 64700, 198, 9, 256, 1368, 13853, 10471, 220, 19, 4022, 7397, 11, 551, 10471, 393, 16, 22, 471, 220, 19, 283, 220, 16, 18, 3, 18016, 7397, 13, 198, 9, 256, 12650, 4022, 7397, 25, 393, 19, 478, 220, 19, 283, 220, 23, 3, 198, 9, 256, 12650, 18016, 7397, 25, 393, 18, 478, 220, 16, 18, 283, 220, 16, 21, 3, 198, 9, 256, 2091, 220, 16, 21, 10598, 430, 1599, 430, 220, 23, 30, 7179, 11, 393, 16, 21, 283, 220, 17, 1088, 14695, 220, 23, 12576, 271, 760, 1324, 314, 4022, 7397, 13853, 10471, 369, 220, 19, 13, 271, 794, 220, 19, 248046, 198, 248044, 248045, 248045, 248045, 248045, 248046, 198, 248044, 248045, 248045, 248045, 248046, 198, 248045, 846, 198, 2523, 513, 264, 6673, 24499, 13, 248046, 198, 248045, 248068, 271, 248069, 271, 15771, 49942, 0, 353, 1044, 5354, 310, 1438, 488, 430, 264, 6673, 24499, 13, 271, 5328, 3300, 279, 6673, 3377, 488, 1000, 1040, 728, 310, 11290, 466, 10033, 13, 353, 668, 2873, 488, 264, 2708, 11, 2923, 13909, 28253, 31626, 321, 809, 440, 279, 1534, 4087, 303, 279, 10897, 3443, 25, 1510, 794, 361, 3946, 29, 27653, 248046, 198, 248044, 248045, 248045, 248045, 248046, 198, 248045, 846, 198, 248046, 198, 248045, 248045, 248046, 198, 248045, 74455, 198, 248068, 271, 248069, 271, 1206, 1423, 680, 1204, 1599, 4022, 7397, 13853, 10471, 11, 567, 628, 716, 685, 449, 45309, 291, 22868, 3018, 383, 279, 1928, 3766, 13, 271, 332, 16, 13, 62366, 279, 2763, 1324, 314, 7397, 64700, 198, 9, 256, 3991, 18016, 7397, 25, 220, 18, 198, 9, 256, 3991, 4022, 7397, 25, 220, 19, 271, 332, 17, 13, 18053, 7101, 364, 279, 7397, 10471, 64700, 198, 9, 256, 6558, 393, 86, 3, 381, 279, 1324, 314, 4022, 7397, 10471, 13, 198, 9, 256, 6558, 393, 78, 3, 381, 279, 1324, 314, 18016, 7397, 10471, 13, 198, 9, 256, 1165, 1366, 279, 2702, 1324, 314, 7397, 10471, 369, 220, 16, 22, 11, 748, 25, 198, 262, 25273, 86, 478, 296, 283, 220, 16, 22, 13682, 198, 262, 5369, 411, 11, 567, 628, 3057, 279, 1324, 314, 18016, 7397, 10471, 303, 3665, 314, 4022, 7397, 25, 198, 262, 25273, 78, 283, 220, 16, 22, 471, 288, 13682, 271, 332, 18, 13, 28924, 279, 1534, 1324, 314, 7397, 303, 279, 34393, 64700, 198, 9, 256, 12650, 4022, 7397, 283, 3991, 4022, 478, 74522, 4022, 283, 393, 19, 478, 288, 3, 198, 9, 256, 12650, 18016, 7397, 283, 3991, 18016, 478, 74522, 18016, 283, 393, 18, 478, 296, 3, 198, 262, 3593, 3570, 10292, 393, 78, 283, 220, 16, 22, 471, 288, 60439, 198, 262, 12650, 18016, 7397, 283, 393, 18, 478, 318, 16, 22, 471, 288, 8, 283, 220, 17, 15, 471, 288, 3, 271, 332, 19, 13, 2488, 685, 279, 22868, 3018, 383, 279, 1534, 11013, 64700, 198, 760, 3377, 5134, 421, 279, 1534, 1324, 314, 18016, 7397, 369, 10598, 279, 1324, 314, 4022, 7397, 13, 198, 13682, 59, 1272, 90, 18770, 21281, 92, 283, 220, 17, 1088, 14695, 1088, 1272, 90, 18770, 5626, 92, 13682, 198, 13682, 17, 15, 471, 288, 283, 220, 17, 7, 19, 478, 288, 8, 13682, 271, 332, 20, 13, 61148, 364, 393, 86, 60439, 332, 198, 13682, 17, 15, 471, 288, 283, 220, 23, 478, 220, 17, 86, 13682, 198, 2143, 393, 86, 3, 310, 2107, 10744, 25, 198, 13682, 17, 15, 283, 220, 23, 478, 220, 18, 86, 13682, 198, 3036, 2076, 220, 23, 494, 2107, 10744, 25, 198, 13682, 16, 17, 283, 220, 18, 86, 13682, 198, 12151, 562, 539, 220, 18, 25, 198, 13682, 86, 283, 220, 19, 13682, 271, 332, 60239, 64700, 198, 9, 256, 1368, 13853, 10471, 220, 19, 4022, 7397, 11, 551, 10471, 393, 16, 22, 471, 220, 19, 283, 220], "tokens": 1024, "terminal": ["length"], "usage": [{"prompt_tokens": 149, "completion_tokens": 1024, "total_tokens": 1173}], "output_text_sha256": "14382258fbbcb4c8029f7e172c087a42d22b0f3164bbcf2bdfded406d0a8906e", "wire_sha256": "7e7f486a2470afb9cd6295da071798d5ec40e6c1338cec7807b1177e349a95fd", "client_success": true, "client_error": false, "valid": true, "ttft_s": 2.543290780042298, "latency_s": 96.65085916907992, "itl": [0.09182190604042262, 0.09191396296955645, 0.09156402596272528, 0.09196639305446297, 0.09157405700534582, 0.09187878097873181, 0.09179128799587488, 0.09184563905000687, 0.0917270400095731, 0.09185544797219336, 0.09178692603018135, 0.09175084298476577, 0.09177637193351984, 0.09180801699403673, 0.09178854699712247, 0.09171283501200378, 0.09189396898727864, 0.09176848107017577, 0.09186041296925396, 0.09187726397067308, 0.09192742803134024, 0.09180649800691754, 0.09170162794180214, 0.09176402108278126, 0.091767902020365, 0.09173575788736343, 0.09166273009032011, 0.09195855201687664, 0.09166096698027104, 0.09192418400198221, 0.09172474697697908, 0.0918817080091685, 0.09179008193314075, 0.09176363400183618, 0.09174078807700425, 0.0918350089341402, 0.09167014807462692, 0.09180782292969525, 0.09177637204993516, 0.09175090992357582, 0.09185859106946737, 0.09171539696399122, 0.09190081199631095, 0.09171816206071526, 0.09174983098637313, 0.09169745293911546, 0.0918348190607503, 0.09187020501121879, 0.09183547296561301, 0.0917682449799031, 0.09164346207398921, 0.09181463590357453, 0.09166999300941825, 0.09196868899744004, 0.09169612708501518, 0.09191480290610343, 0.09179790399502963, 0.09186884399969131, 0.09167750901542604, 0.09179551608394831, 0.09174620895646513, 0.09174854995217174, 0.09182373899966478, 0.09175628703087568, 0.091833092039451, 0.0916860660072416, 0.09177040902432054, 0.09183150692842901, 0.09188602305948734, 0.09165820095222443, 0.09179806697648019, 0.09182753204368055, 0.09167682100087404, 0.09175515593960881, 0.09171498706564307, 0.09196817292831838, 0.09168308810330927, 0.09202714590355754, 0.09166770300362259, 0.09201437199953943, 0.09173446404747665, 0.0919100500177592, 0.09179247601423413, 0.09190612798556685, 0.09175507898908108, 0.09177028900012374, 0.09176190197467804, 0.091805006028153, 0.09193441399838775, 0.09175156301353127, 0.09189401299227029, 0.09177625295706093, 0.09192185697611421, 0.09179641003720462, 0.09178831602912396, 0.09184812300372869, 0.09181752696167678, 0.09186454396694899, 0.09161669504828751, 0.09188164700753987, 0.09165590303018689, 0.09205759596079588, 0.0916849080240354, 0.0918763029621914, 0.09173665998969227, 0.09183752303943038, 0.09176167694386095, 0.09197205305099487, 0.09178028302267194, 0.09172529994975775, 0.09194472106173635, 0.09182257193606347, 0.09184271306730807, 0.09179510292597115, 0.09188422898296267, 0.09173335507512093, 0.09186414501164109, 0.09184030489996076, 0.0918118180707097, 0.09196853800676763, 0.09175921592395753, 0.09187088208273053, 0.09181310597341508, 0.09188984800130129, 0.09175014903303236, 0.09198999998625368, 0.09171245503239334, 0.09203475993126631, 0.0918047979939729, 0.0919172529829666, 0.09176573005970567, 0.09187284298241138, 0.09177308296784759, 0.0919198669726029, 0.09176991903223097, 0.09187193401157856, 0.09185354097280651, 0.09175771404989064, 0.09189189597964287, 0.09168255096301436, 0.09188180300407112, 0.09178493707440794, 0.09180009993724525, 0.09181992406956851, 0.09178573393728584, 0.0918471299810335, 0.09170133108273149, 0.09197555691935122, 0.0917163590202108, 0.09194030496291816, 0.09173366206232458, 0.09190780099015683, 0.09173710504546762, 0.09194456692785025, 0.09169950499199331, 0.0920167570002377, 0.09177466901019216, 0.09189549006987363, 0.09178756200708449, 0.09184344997629523, 0.0918808359419927, 0.09181974502280354, 0.09191326203290373, 0.09182905999477953, 0.09184516198001802, 0.09199810004793108, 0.09195296990219504, 0.0917203261051327, 0.09183658193796873, 0.09188249299768358, 0.09170634800102562, 0.09196872496977448, 0.09183599904645234, 0.09196561097633094, 0.09166559705045074, 0.09188757091760635, 0.09160912910010666, 0.0919526549987495, 0.09162546193692833, 0.09187197696883231, 0.09174354106653482, 0.09193149697966874, 0.0917607699520886, 0.09187007299624383, 0.09187647805083543, 0.09183487005066127, 0.09174142999108881, 0.09170803090091795, 0.09192592604085803, 0.09177616203669459, 0.09191247401759028, 0.09177198691759259, 0.09185541502665728, 0.09183767205104232, 0.0917669190093875, 0.09187695290893316, 0.09165822202339768, 0.09198283904697746, 0.0917248249752447, 0.09203589602839202, 0.09169349900912493, 0.09192976797930896, 0.09188270999584347, 0.09184985503088683, 0.09177896392066032, 0.09182310802862048, 0.09173971705604345, 0.0917613849742338, 0.09181648900266737, 0.09170806303154677, 0.09179678792133927, 0.09167055506259203, 0.09192221600096673, 0.09182084701023996, 0.09188298997469246, 0.09183316002599895, 0.09181492996867746, 0.0917881679488346, 0.09165273897815496, 0.09180215210653841, 0.09184364997781813, 0.0917751039378345, 0.09166514407843351, 0.09229682793375105, 0.09208845300599933, 0.09184777596965432, 0.09170468908268958, 0.09189451392740011, 0.091774114058353, 0.09181360295042396, 0.09186635003425181, 0.09188579104375094, 0.09184561891015619, 0.09179063502233475, 0.09181295498274267, 0.09175130899529904, 0.09192145406268537, 0.0916108819656074, 0.09190416801720858, 0.09185542201157659, 0.09184785501565784, 0.09183210099581629, 0.09179965395014733, 0.09186737099662423, 0.0917659989790991, 0.09191583504434675, 0.09167749399784952, 0.09204783395398408, 0.09166609705425799, 0.0918636069400236, 0.09174652106594294, 0.0918937079841271, 0.09181810403242707, 0.09192145394627005, 0.09185143606737256, 0.09175546700134873, 0.0918412929167971, 0.0918441649992019, 0.09189950907602906, 0.09173146693501621, 0.0920105540426448, 0.09172248793765903, 0.09177238505799323, 0.09179887000937015, 0.09183691896032542, 0.09181490307673812, 0.09174572490155697, 0.0919537910958752, 0.09175795398186892, 0.09195493895094842, 0.0918505690060556, 0.09187880204990506, 0.09181247290689498, 0.09185933810658753, 0.09165657893754542, 0.091984594007954, 0.09173186495900154, 0.09184599108994007, 0.09180904796812683, 0.09180686296895146, 0.09191248298157007, 0.09193335904274136, 0.09185212303418666, 0.0917184449499473, 0.09193219803273678, 0.09180474397726357, 0.09192117699421942, 0.09185061801690608, 0.09186225198209286, 0.09178373997565359, 0.09186003601644188, 0.09190762904472649, 0.0918675649445504, 0.09205975604709238, 0.09172922594007105, 0.0919040150474757, 0.09170632692985237, 0.09203608508687466, 0.09162223897874355, 0.09202036296483129, 0.09172677202150226, 0.09194729302544147, 0.09178582101594657, 0.09194632200524211, 0.09183886600658298, 0.09177936194464564, 0.09178088000044227, 0.09188466903287917, 0.0919683949323371, 0.09179238101933151, 0.09187485603615642, 0.09178174403496087, 0.09198636491782963, 0.09237900597508997, 0.09190502006094903, 0.09195710602216423, 0.0919098089216277, 0.0920889989938587, 0.09174952108878642, 0.09202319593168795, 0.09190817002672702, 0.09206924599129707, 0.09175728005357087, 0.09210948494728655, 0.09183252102229744, 0.09204986901022494, 0.0918819799553603, 0.09204285603482276, 0.09186974097974598, 0.09187303599901497, 0.09194926102645695, 0.09192239493131638, 0.09192987601272762, 0.09185654809698462, 0.09196433599572629, 0.09193460992537439, 0.09200528298970312, 0.0918918390525505, 0.09181419794913381, 0.09186377003788948, 0.09193107997998595, 0.0920418850146234, 0.09184428304433823, 0.09199735696893185, 0.09184285195078701, 0.09202532202471048, 0.09172818798106164, 0.09207419608719647, 0.09179323492571712, 0.09199588000774384, 0.09182871400844306, 0.09204612404573709, 0.09185127995442599, 0.09191574004944414, 0.09185321896802634, 0.0918688690289855, 0.09194639197085053, 0.09197633597068489, 0.09196286497171968, 0.09185542003251612, 0.09198648505844176, 0.09188926592469215, 0.09207164903637022, 0.09194470604415983, 0.09193137299735099, 0.0920013029826805, 0.09188527800142765, 0.0920232490170747, 0.09187103691510856, 0.09206703002564609, 0.09182005701586604, 0.09214737999718636, 0.09180004301015288, 0.09211822599172592, 0.09183310205116868, 0.09205453796312213, 0.09193926595617086, 0.09195803396869451, 0.09188221499789506, 0.09193856408819556, 0.09193452401086688, 0.09193946793675423, 0.09204556199256331, 0.09191351500339806, 0.09199469606392086, 0.09190578095149249, 0.09194861201103777, 0.09189972805324942, 0.09199585393071175, 0.09195154102053493, 0.09192377596627921, 0.09202726802323014, 0.09187287103850394, 0.09200522396713495, 0.09178843803238124, 0.09209214092697948, 0.09183211508207023, 0.09206749894656241, 0.09188319195527583, 0.09204071504063904, 0.091936290031299, 0.0919990959810093, 0.09184608305804431, 0.09199364390224218, 0.09195000503677875, 0.09199700003955513, 0.0919801399577409, 0.09186112799216062, 0.09204037697054446, 0.09182244609110057, 0.09198449901305139, 0.09187610598746687, 0.09207998297642916, 0.0919197719777003, 0.09195088199339807, 0.09204319899436086, 0.09192829497624189, 0.0919941479805857, 0.09177860501222312, 0.09201241401024163, 0.09179881098680198, 0.09203040006104857, 0.09184598596766591, 0.09211323596537113, 0.09182797302491963, 0.09209193906281143, 0.09185870294459164, 0.09200606204103678, 0.09202245599590242, 0.09194885694887489, 0.09203227504622191, 0.09192419203463942, 0.09201472392305732, 0.09186665806919336, 0.09195567690767348, 0.0919127460801974, 0.09200849593617022, 0.09194114501588047, 0.09194369206670672, 0.09207529097329825, 0.09199359500780702, 0.0919377279933542, 0.0918637829599902, 0.09210085403174162, 0.09183644398581237, 0.09208165796007961, 0.09185373806394637, 0.09205024398397654, 0.09187009604647756, 0.09207932197023183, 0.09175990801304579, 0.09203404991421849, 0.09193877200596035, 0.09201615606434643, 0.09191704902332276, 0.09192791092209518, 0.09202526602894068, 0.09194390301126987, 0.09195923304650933, 0.09190965199377388, 0.09200137294828892, 0.09192207304295152, 0.0919716899516061, 0.09192150004673749, 0.09187046892475337, 0.09194666403345764, 0.09178514196537435, 0.09198619704693556, 0.09188186493702233, 0.09220698603894562, 0.09181096707470715, 0.09211359999608248, 0.0917862969217822, 0.09215619007591158, 0.09180799394380301, 0.09207928401883692, 0.09186515002511442, 0.09191952098626643, 0.09190037997905165, 0.09193562204018235, 0.09200869395863265, 0.09187087800819427, 0.09186513803433627, 0.09187306999228895, 0.09208071499597281, 0.09192083601374179, 0.09205224993638694, 0.09193781996145844, 0.09195121203083545, 0.09188838698901236, 0.09190628200303763, 0.0920537420315668, 0.09189462603535503, 0.09205875394400209, 0.09188609605189413, 0.09226699394639581, 0.09183455200400203, 0.09206136898137629, 0.09177853108849376, 0.09208392794243991, 0.09189471905119717, 0.09206615993753076, 0.09193584497552365, 0.09198235301300883, 0.0919139210600406, 0.0919606409734115, 0.09192059596534818, 0.09184516198001802, 0.09203322301618755, 0.0918899760581553, 0.09197894297540188, 0.09187529305927455, 0.09197290998417884, 0.09193164191674441, 0.09184619004372507, 0.09197714598849416, 0.09187845699489117, 0.09206982096657157, 0.0918660790193826, 0.09212639706674963, 0.0918736649909988, 0.09207990299910307, 0.09173109801486135, 0.09204366593621671, 0.0918516080128029, 0.09202662599273026, 0.09198200306855142, 0.09248416090849787, 0.09195144998375326, 0.09198763908352703, 0.09193289198447019, 0.09193081397097558, 0.09204180398955941, 0.09190165100153536, 0.09207663801498711, 0.09183674596715719, 0.09197668207343668, 0.09191647893749177, 0.09187337104231119, 0.09200496796984226, 0.09193989401683211, 0.09197750897146761, 0.09186692605726421, 0.0920864469371736, 0.09180377807933837, 0.09208159090485424, 0.09171520103700459, 0.09214384004008025, 0.09189908695407212, 0.09203491802327335, 0.09197631396818906, 0.09203239297494292, 0.09189166908618063, 0.09196527500171214, 0.09196429199073464, 0.09194253000896424, 0.09197876101825386, 0.09192750591319054, 0.09215955203399062, 0.09215894003864378, 0.09208148892503232, 0.09199087205342948, 0.09197642700746655, 0.0920068429550156, 0.0919603219954297, 0.09200543607585132, 0.0919052679091692, 0.09206926799379289, 0.09188407100737095, 0.09208839805796742, 0.09174299600999802, 0.09213789098430425, 0.0919998970348388, 0.09204599098302424, 0.09205044200643897, 0.09205441793892533, 0.09192873002029955, 0.09202462597750127, 0.0919385040178895, 0.09196641703601927, 0.09199224598705769, 0.09197752096224576, 0.09206994797568768, 0.09190364007372409, 0.09206572198309004, 0.09198303695302457, 0.0919120010221377, 0.09199247206561267, 0.09200944297481328, 0.09195846493821591, 0.09195863804779947, 0.09208406799007207, 0.09192119294311851, 0.09208437299821526, 0.09167195810005069, 0.09217577893286943, 0.0918318519834429, 0.09213350201025605, 0.09190281899645925, 0.09209416899830103, 0.09190360398497432, 0.09202238207217306, 0.09193304099608213, 0.09196225693449378, 0.09202335600275546, 0.09192410204559565, 0.09199816302862018, 0.09194666193798184, 0.09211370104458183, 0.09190904500428587, 0.09190585999749601, 0.09195122099481523, 0.09202240000013262, 0.09201471996493638, 0.09203880408313125, 0.0920738959684968, 0.09192999394144863, 0.09207022609189153, 0.09182812692597508, 0.09218199807219207, 0.09189343394245952, 0.0920914129819721, 0.09191889397334307, 0.09214853507000953, 0.09189875703305006, 0.09206952294334769, 0.0924101349664852, 0.09198967006523162, 0.0920221209526062, 0.09202380501665175, 0.09209459694102407, 0.09201546909753233, 0.09214576391968876, 0.09194906905759126, 0.09194311697501689, 0.09189772803802043, 0.09197429998312145, 0.09190719097387046, 0.09198602405376732, 0.09200511290691793, 0.09197919408325106, 0.09204217395745218, 0.09185256599448621, 0.09217952296603471, 0.09192743699532002, 0.09213256905786693, 0.0918656310532242, 0.09214805497322232, 0.09187694091815501, 0.09214362502098083, 0.0918170060031116, 0.09205421397928149, 0.09193090407643467, 0.09200918697752059, 0.09200587496161461, 0.09212442906573415, 0.09213008393999189, 0.09197579207830131, 0.09196256590075791, 0.09192023100331426, 0.09206155501306057, 0.09198484802618623, 0.09202532703056931, 0.0920943699311465, 0.09199466404970735, 0.09203750302549452, 0.09188918094150722, 0.09211580303963274, 0.09188372897915542, 0.09212623804342002, 0.09191199101042002, 0.09219201095402241, 0.09192049305420369, 0.09213590691797435, 0.09185797604732215, 0.09212896495591849, 0.09199317707680166, 0.09210908296518028, 0.09209330298472196, 0.0919679410289973, 0.09210184996481985, 0.09188260405790061, 0.09198840195313096, 0.09197611000854522, 0.09210170700680465, 0.09193716803565621, 0.09213409689255059, 0.09201370610389858, 0.0920254219090566, 0.09198690904304385, 0.09188604902010411, 0.09211102302651852, 0.0919626778922975, 0.0920791020616889, 0.09194136399310082, 0.09211545903235674, 0.09196042991243303, 0.0920374960405752, 0.09184432402253151, 0.09201219794340432, 0.0919870100915432, 0.09207948995754123, 0.09208697301801294, 0.0920118410140276, 0.09204469597898424, 0.09199348802212626, 0.09201447491068393, 0.09198230807669461, 0.09207747492473572, 0.09202880901284516, 0.09206997510045767, 0.0920085069956258, 0.09203663689550012, 0.092781953047961, 0.09191877394914627, 0.09212721500080079, 0.09278926905244589, 0.09206594002898782, 0.09200287598650903, 0.09305899299215525, 0.09196116402745247, 0.09214454598259181, 0.09185330802574754, 0.09305674093775451, 0.0921688349917531, 0.0926177849760279, 0.0920744170434773, 0.09212870500050485, 0.0920792770339176, 0.09216656698845327, 0.092842360958457, 0.09203789406456053, 0.09221433801576495, 0.0920499429339543, 0.09307012206409127, 0.09204375697299838, 0.09215881791897118, 0.09202962601557374, 0.09193003200925887, 0.09292629698757082, 0.09204861207399517, 0.09211317694280297, 0.09205798606853932, 0.09216314600780606, 0.09199171490035951, 0.09218637610320002, 0.09179331292398274, 0.09216746804304421, 0.09194319392554462, 0.09221591101959348, 0.09208122303243726, 0.0921303479699418, 0.09206593106500804, 0.09205808700062335, 0.09192058502230793, 0.09207929391413927, 0.09210260899271816, 0.0920018469914794, 0.0922182050999254, 0.09207643393892795, 0.0921431330498308, 0.0919767189770937, 0.09203220903873444, 0.09202851098962128, 0.09207383799366653, 0.09215730091091245, 0.09208059508819133, 0.09214874694589525, 0.09206325700506568, 0.09212269796989858, 0.09188232605811208, 0.09221680497284979, 0.09200450696516782, 0.09222370001953095, 0.09196361107751727, 0.09224668191745877, 0.09209529706276953, 0.092233671923168, 0.09200690407305956, 0.09208875393960625, 0.0920730329817161, 0.09203165804501623, 0.09212117502465844, 0.09201551496516913, 0.0922562669729814, 0.09196576301474124, 0.09202023199759424, 0.0920302839949727, 0.09208620607387275, 0.09204739599954337, 0.09211505390703678, 0.09209724806714803, 0.09206469100899994, 0.09220862295478582, 0.09192515001632273, 0.09231309604365379, 0.091919059981592, 0.09220905601978302, 0.09205636999104172, 0.09225342702120543, 0.09197892399970442, 0.09211265889462084, 0.09190509805921465, 0.09218849102035165, 0.09212364896666259, 0.09213362901937217, 0.09212949103675783, 0.09213661996182054, 0.0922414039960131, 0.0920315800467506, 0.09201964200474322, 0.09201730892527848, 0.09212280798237771, 0.09204783604945987, 0.09213068103417754, 0.09207919798791409, 0.09207758994307369, 0.0921837619971484, 0.09186919697094709, 0.09227017010562122, 0.09204566094558686, 0.09220639104023576, 0.09199404995888472, 0.09222139697521925, 0.09195493103470653, 0.09214944101404399, 0.09194974193815142, 0.09216312307398766, 0.0920756709529087, 0.09218217304442078, 0.09216394799295813, 0.09209027502220124, 0.09219370491337031, 0.09201572404708713, 0.09205183095764369, 0.09200033801607788, 0.09222257905639708, 0.09208247798960656, 0.09216738701798022, 0.09210393892135471, 0.09230501600541174, 0.09213939402252436, 0.09193536604288965, 0.09215011494234204, 0.09208801004569978, 0.09212211996782571, 0.09201648004818708, 0.09221749298740178, 0.09193016600329429, 0.09229928895365447, 0.09189846296794713, 0.09220719209406525, 0.0919992309063673, 0.09212506900075823, 0.0920882560312748, 0.09215975599363446, 0.0920580750098452, 0.09206590906251222, 0.09205621189903468, 0.09209809207823128, 0.09213763196021318, 0.09203915996477008, 0.09218844608403742, 0.09206282196100801, 0.09208659501746297, 0.09309075295459479, 0.09204201807733625, 0.09302575793117285, 0.0920460430206731, 0.09223148401360959, 0.09278440498746932, 0.09195823804475367, 0.09177832899149507, 0.09200069901999086, 0.09260790795087814, 0.092218452016823, 0.09201289399061352, 0.09222400095313787, 0.0921108879847452, 0.09220030205324292, 0.0929471559356898, 0.09215476003009826, 0.09203291707672179, 0.09216306696180254, 0.09219891601242125, 0.09279614395927638, 0.09206866705790162, 0.09201198094524443, 0.09210422507021576, 0.09196120291016996, 0.09289402305148542, 0.0921342249494046, 0.09204405907075852, 0.09211581794079393, 0.09197026200126857, 0.09207702707499266, 0.09200587193481624, 0.09215415397193283, 0.09181589900981635, 0.09227010107133538, 0.09193933196365833, 0.09219506103545427, 0.09195641300175339, 0.09219015494454652, 0.09201185405254364, 0.09206316294148564, 0.09194275899790227, 0.0920624160207808, 0.09294326603412628, 0.09199887490831316, 0.09282254008576274, 0.09211950493045151, 0.09220734098926187, 0.09291504707653075, 0.09204831602983177, 0.09204681089613587, 0.09211232303641737, 0.09297379897907376, 0.09217781003098935, 0.09212062403094023, 0.09210602997336537, 0.09222323598805815, 0.09269112604670227, 0.09217413701117039, 0.09197724796831608, 0.09224869194440544, 0.09197886008769274, 0.09213383495807648, 0.09204202401451766, 0.0921527580358088, 0.09268558491021395, 0.09215574408881366, 0.0921189779182896, 0.09209821000695229, 0.09299727203324437, 0.09212764096446335, 0.09220326505601406, 0.09202485193964094, 0.09205742704216391, 0.09207838401198387, 0.09214113093912601, 0.09205708105582744, 0.09212929394561797, 0.09215059003327042, 0.0920745680341497, 0.09216981299687177, 0.09189671801868826, 0.09221148700453341, 0.0920473849400878, 0.09226397797465324, 0.0920162940165028, 0.0922716420609504, 0.09289152198471129, 0.09223369392566383, 0.09270524606108665, 0.09224885795265436, 0.09286267904099077, 0.09211067797150463, 0.09208454296458513, 0.0920261749997735, 0.09304858499672264, 0.09210995608009398, 0.09212766599375755, 0.09220757300499827, 0.09228995302692056, 0.09212306991685182, 0.09307925205212086, 0.09212216699961573, 0.09203599998727441, 0.0920665180310607, 0.09195851592812687, 0.09213958703912795, 0.09284500393550843, 0.09216472203843296, 0.09198491298593581, 0.09225408604834229, 0.09282029001042247, 0.09217788092792034, 0.09179093805141747, 0.09218717401381582, 0.0919437250122428, 0.09213268698658794, 0.09198686992749572, 0.09208127704914659, 0.09202633798122406, 0.09196080104447901, 0.09188654099125415, 0.09202391502913088, 0.09216247696895152, 0.09201611601747572, 0.09214156097732484, 0.09197820699773729, 0.09205825999379158, 0.09200680395588279, 0.0917818620800972, 0.09202567196916789, 0.09290794504340738, 0.09299872000701725, 0.09204009594395757, 0.09223418706096709, 0.09285935491789132, 0.09226737206336111, 0.09192139899823815, 0.09229340497404337, 0.09292395994998515, 0.09225686907302588, 0.09208290895912796, 0.09219517698511481, 0.09216902998741716, 0.09213799401186407, 0.09337641496676952, 0.09215677808970213, 0.09224006393924356, 0.09210938098840415, 0.09303572599310428, 0.09205316903535277, 0.09221493906807154, 0.09205288288649172, 0.09198751405347139, 0.09315340302418917, 0.09221944899763912, 0.0922139179892838, 0.09232570801395923], "tpot_s": 0.09199175795604851, "active_requests_after": 0, "runner_state_count_after": 0} -{"phase": "request", "arm": "eager", "case": "gsm8k/test/783", "index": 4, "cap": 16, "ignore_eos": true, "prompt_tokens": 149, "prompt_sha256": "f2233be4ebb7b122d80433fe4081fdabeda64f735b7bc24f9706c4c4339b824f", "token_ids": [1206, 11290, 411, 3377, 11, 567, 1144, 310, 7995, 279, 1324, 314, 4022, 7397, 13853, 10471], "tokens": 16, "terminal": ["length"], "usage": [{"prompt_tokens": 149, "completion_tokens": 16, "total_tokens": 165}], "output_text_sha256": "4b5d6370a617c539db1e6ee0df2c726c7165c1e9395eab1b873ff700a997510a", "wire_sha256": "a22bd0ab71fb0706feeb594c95dcd6e006c7559f7ccf8fb4aac9780d33970522", "client_success": true, "client_error": false, "valid": true, "ttft_s": 2.5469864449696615, "latency_s": 3.927188378991559, "itl": [0.09257348102983087, 0.09188807907048613, 0.09196719992905855, 0.09173286706209183, 0.09195811790414155, 0.09161947702523321, 0.09183989802841097, 0.09174554899800569, 0.09275991294998676, 0.09192328702192754, 0.09182386100292206, 0.09197691606823355, 0.09186255093663931, 0.09264334803447127, 0.09188738896045834], "tpot_s": 0.09201346226812651, "active_requests_after": 0, "runner_state_count_after": 0} -{"phase": "cleanup", "arm": "eager", "exit_code": 0, "forced": false, "closed": true, "active_requests": 0, "runner_state_count": 0} -{"phase": "comparison", "case": "gsm8k/test/783", "cap": 2048, "ignore_eos": false, "graph_tokens": 529, "eager_tokens": 503, "exact": false, "first_difference": 81} -{"phase": "comparison", "case": "gsm8k/test/1", "cap": 2048, "ignore_eos": false, "graph_tokens": 169, "eager_tokens": 169, "exact": true, "first_difference": null} -{"phase": "comparison", "case": "gsm8k/test/424", "cap": 2048, "ignore_eos": false, "graph_tokens": 314, "eager_tokens": 314, "exact": true, "first_difference": null} -{"phase": "comparison", "case": "gsm8k/test/783", "cap": 1024, "ignore_eos": true, "graph_tokens": 1024, "eager_tokens": 1024, "exact": false, "first_difference": 81} -{"phase": "comparison", "case": "gsm8k/test/783", "cap": 16, "ignore_eos": true, "graph_tokens": 16, "eager_tokens": 16, "exact": true, "first_difference": null} -{"phase": "complete", "verdict": "INCONCLUSIVE", "scope": "bounded_readiness"} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/graph-demotion-impact.md b/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/graph-demotion-impact.md deleted file mode 100644 index 46524b85c..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/graph-demotion-impact.md +++ /dev/null @@ -1,66 +0,0 @@ -# Hybrid Graph boundary repair: HTTP impact - -Refs InferRSI Issue #67. **Campaign INCONCLUSIVE.** - -Candidate80e5108e44c43f3f54fabb5723e6154d18088516 builds on -7ec5d224e983e437530355b94b0d6d08153bc27d. The rebase changes only the separately -reviewed readiness test; all runtime bytes equal previously reviewed e69035d. -The model patch only distinguishes real KV from linear state during demotion. - -Job1955 completed10/10 requests with exact usage, healthy terminal framing and -no remaining request state. Both owned processes exited0 without forced kill. -The previous876-token boundary error now completes1024 output tokens; the next -16-token request succeeds. All1951 observed Graph prefixes remain identical. - -Full Graph/eager sequences match3/5 pairs. Case783 first differs at zero-based -index81 in both natural-EOS and forced1024-token variants, before demotion. -NaturalEOS output lengths differ529 vs503. This is retained unresolved numerical -evidence. Neither request success nor these timings establishes full numerical, -quality, long-context or vLLM parity qualification. - -## Descriptive HTTP observations - -Official pinned client with response recording; fixed Graph-then-eager order, -short inputs, no matched vLLM baseline. The table records actual per-request -latency, not a causal speedup estimate. Different naturalEOS outputs further -preclude equal-work latency comparisons. Full tokens/ITLs are in the JSONL. - -| Arm | Case | Output policy | Tokens | TTFT (s) | Latency (s) | TPOT (ms) | -| --- | --- | --- | ---: | ---: | ---: | ---: | -| graph | gsm8k/test/783 | naturalEOS | 529 | 2.672960 | 49.137462 | 88.000951 | -| graph | gsm8k/test/1 | naturalEOS | 169 | 1.300144 | 16.004033 | 87.523149 | -| graph | gsm8k/test/424 | naturalEOS | 314 | 1.925834 | 29.317723 | 87.514019 | -| graph | gsm8k/test/783 | ignoreEOS/1024 | 1024 | 2.546314 | 92.730485 | 88.156570 | -| graph | gsm8k/test/783 | ignoreEOS/16 | 16 | 2.712920 | 4.026041 | 87.541409 | -| eager | gsm8k/test/783 | naturalEOS | 503 | 3.526064 | 49.761498 | 92.102459 | -| eager | gsm8k/test/1 | naturalEOS | 169 | 1.255985 | 16.675945 | 91.785477 | -| eager | gsm8k/test/424 | naturalEOS | 314 | 1.880406 | 30.637043 | 91.874239 | -| eager | gsm8k/test/783 | ignoreEOS/1024 | 1024 | 2.543291 | 96.650859 | 91.991758 | -| eager | gsm8k/test/783 | ignoreEOS/16 | 16 | 2.546986 | 3.927188 | 92.013462 | - -GPU allocation464 seconds; readiness total839 seconds, of approved7200 seconds -(total campaign ceiling44 GPU-hours). Initial147 seconds remain separate. -1952/1953 preflight failures and1954 CPU repair are retained in demotion-preflight. -Combined74 local tests passed; no full numerical suite rerun is claimed. - -KnowledgeStore (candidate state, verified readback): -case.qwen38-hybrid-graph-demotion-cand at sha256:48e720c89ca6cf511186b36a83a76ce236d6adcb99d7f3138d444191c671db49. - -Source-only upstream research is also retained under -store:case.qwen38-vllm032-upstream-dispatch-cand at -sha256:6824c5489a6775534e00163405d7e9228891bf709c7af14d7623f50554fa6a89. -It distinguishes FlashInfer/DeepGEMM and GDN eligibility from measured dispatch. -The current unchunked attention materializes a192-GiB BF16 score tensor at64K -(24 heads), before FP32 softmax and weights. This is a static memory bound, -not a GPU measurement; no64K/128K candidate request has been run here. - -## Integration - -Independent reviewer Codex review_fp8_graph verified all10 raw SSE streams, -source/observer hashes and cleanup against base7ec5d224e983e437530355b94b0d6d08153bc27d -and head80e5108e44c43f3f54fabb5723e6154d18088516. No introduced blocking finding; -GPU conditions for this limited boundary repair are satisfied. The reviewer -explicitly retained pre-boundary numerical divergence and INCONCLUSIVE. -Engine PR9 merged as3e918df2dd7b478d3d2985350bb5efb523ea4981 under the owner's -review/integration delegation. Fetched main tree exactly matches tested head. -No governing acceptance Issue was closed. diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/graph-forced96-1957.json b/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/graph-forced96-1957.json deleted file mode 100644 index 607e7bae7..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/graph-forced96-1957.json +++ /dev/null @@ -1,99 +0,0 @@ -{ - "verdict": "INCONCLUSIVE", - "scope": "fixed_history_numerical_diagnostic_not_quality_or_performance_acceptance", - "job": 1957, - "exit_code": 0, - "allocated_gpu_seconds": 65, - "readiness_total_gpu_seconds": 904, - "readiness_remaining_gpu_seconds": 6296, - "separate_initial_gpu_seconds": 147, - "observer_sha256": "6e759f732793f9410efa9afc9173d110a5a5a36705a18f0cc5a81ff0d744ec48", - "observations_sha256": "18fe5c4fc2f5d920af20a3f41dd1ff8dbdadd1d7952892428439b6b9fbd567bb", - "steps": 96, - "buffered_graph_logits_exact_steps": 96, - "buffered_graph_live_state_exact_steps": 96, - "compact_buffered_first_logit_difference": 23, - "compact_buffered_argmax_differences": [ - 81 - ], - "history_reproduction": { - "phase": "complete", - "verdict": "INCONCLUSIVE", - "compact_history_matches": true, - "graph_original_prefix_matches": true, - "compact_history_mismatches": [], - "graph_prefix_mismatches": [], - "seconds": 57.72265838296153, - "scope": "numerical_diagnostic_not_task_quality_or_service_acceptance" - }, - "index81": { - "phase": "step", - "index": 81, - "position": 229, - "teacher_token": 10471, - "compact": { - "ids": [ - 264, - 220, - 393, - 1010, - 4945 - ], - "logits": [ - 25.625, - 25.625, - 21.0, - 18.75, - 18.125 - ], - "margin": 0.0, - "argmax": 220 - }, - "buffered": { - "ids": [ - 264, - 220, - 393, - 1010, - 4945 - ], - "logits": [ - 25.75, - 25.625, - 21.0, - 18.5, - 18.125 - ], - "margin": 0.125, - "argmax": 264 - }, - "graph": { - "ids": [ - 264, - 220, - 393, - 1010, - 4945 - ], - "logits": [ - 25.75, - 25.625, - 21.0, - 18.5, - 18.125 - ], - "margin": 0.125, - "argmax": 264 - }, - "compact_buffered": { - "exact": false, - "max_abs": 0.328125, - "nrmse": 0.020043984055519104 - }, - "buffered_graph": { - "exact": true, - "max_abs": 0.0, - "nrmse": 0.0 - } - } -} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/graph-forced96-1957.jsonl b/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/graph-forced96-1957.jsonl deleted file mode 100644 index 9eddf5b99..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/graph-forced96-1957.jsonl +++ /dev/null @@ -1,99 +0,0 @@ -{"phase": "protocol", "case": "gsm8k/test/783", "steps": 96, "capacity": 1024, "fixture_sha256": "f359c66f92b69d9a92a66564d8a087809dbc45b7e8b9e916229065806c4c188c", "history_sha256": "7fcf761c8d664f40ecf4a5f08a83fcf4b362e4127ecc4c932fd9d5c4ceb6cb54", "observer_sha256": "6e759f732793f9410efa9afc9173d110a5a5a36705a18f0cc5a81ff0d744ec48", "source_hashes": {"nanovllm/qwen38/__init__.py": "1f07e0cfd12bbf370c1f465c6a4bd73e72a5c826af1f29e60ccc7d6f45562cfc", "nanovllm/qwen38/__main__.py": "9e2a0881eedf8fa4abc6037511b42fd3a44171d121a7b2cb096023f485673ba2", "nanovllm/qwen38/_triton_fp8.py": "9f0cb68be017cb3858aebe8ba275039084e305df92cb4567d6e7da54d119f19e", "nanovllm/qwen38/_triton_rms.py": "ba99fa3ba82bfcab4cd602fd0db907acad0eee7c7a63ab23d560a17fd2f72272", "nanovllm/qwen38/config.py": "002ea25bbf9c8d17e5f1d3fb2b76c400e4291d7d2d06d5dd13fb5c3ac15c3d79", "nanovllm/qwen38/engine.py": "c47e1c72fcc24bc8eeb5497887da6be11c2de567abcc3efb275b061c061fd755", "nanovllm/qwen38/graph.py": "d4213aa68599564a01ced3b990993c9b17482615c154120bc4180f78a5da6098", "nanovllm/qwen38/loader.py": "1aa6b3321bc135fa28d1eebf6223bbe3bc5f646630193e292fb58a9f0ca2763f", "nanovllm/qwen38/model.py": "ad8386f156ff5315dfd73de313ab766fedd9ef892c7b94cec95303f11fd4e198", "nanovllm/qwen38/quant.py": "5590e4769a0fbbcaf5afeee1a93db5c4bccacf09f6f3b34e42d52ec31e9f50c3", "nanovllm/qwen38/runner.py": "3b1f04b42949616bf7b8c62dd802f1707d694c0a14bfa129d24fd6612881a22f", "nanovllm/qwen38/server.py": "d2ccb1ccb61dbf06b22f7f199854d58fc7fbb246c6d8e8ec9d5f6344c3016494"}, "torch": "2.14.1+cu130", "device": "NVIDIA H20-3e", "triton": "3.8.0", "scope": "fixed_eager_history_same_model_no_quality_or_performance_claim"} -{"phase": "loaded", "elapsed_s": 19.252109191031195, "report": {"loaded_parameters": 1251, "excluded_tensors": {"vision": 333, "mtp": 22}, "scope": "text_only", "dtype": "fp8", "activation_dtype": "bfloat16", "weight_dtype": "float8_e4m3fn", "scale_dtype": "float32", "checkpoint_scale_dtype": "bfloat16", "quantization": {"activation_scheme": "dynamic", "fmt": "e4m3", "quant_method": "fp8", "weight_block_size": [128, 128]}, "execution": "native_fp8", "fp8_linear_modules": 400, "dequantized": false, "backend": "triton_block_fp8", "kernel_name": "qwen38_block_fp8"}} -{"phase": "step", "index": 0, "position": 148, "teacher_token": null, "compact": {"ids": [1206, 9764, 31318, 332, 1596], "logits": [24.875, 24.625, 20.25, 19.375, 19.125], "margin": 0.25, "argmax": 1206}, "buffered": {"ids": [1206, 9764, 31318, 332, 1596], "logits": [24.875, 24.625, 20.25, 19.375, 19.125], "margin": 0.25, "argmax": 1206}, "graph": {"ids": [1206, 9764, 31318, 332, 1596], "logits": [24.875, 24.625, 20.25, 19.375, 19.125], "margin": 0.25, "argmax": 1206}, "compact_buffered": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "compact_buffered_state_diff": [], "buffered_graph_state_diff": []} -{"phase": "step", "index": 1, "position": 149, "teacher_token": 1206, "compact": {"ids": [11290, 1423, 7995, 38406, 3462], "logits": [27.0, 26.5, 24.875, 16.5, 15.6875], "margin": 0.5, "argmax": 11290}, "buffered": {"ids": [11290, 1423, 7995, 38406, 3462], "logits": [27.0, 26.5, 24.875, 16.5, 15.6875], "margin": 0.5, "argmax": 11290}, "graph": {"ids": [11290, 1423, 7995, 38406, 3462], "logits": [27.0, 26.5, 24.875, 16.5, 15.6875], "margin": 0.5, "argmax": 11290}, "compact_buffered": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "compact_buffered_state_diff": [], "buffered_graph_state_diff": []} -{"phase": "step", "index": 2, "position": 150, "teacher_token": 11290, "compact": {"ids": [411, 364, 279, 104362, 3377], "logits": [29.625, 19.375, 18.625, 16.375, 15.25], "margin": 10.25, "argmax": 411}, "buffered": {"ids": [411, 364, 279, 104362, 3377], "logits": [29.625, 19.375, 18.625, 16.375, 15.25], "margin": 10.25, "argmax": 411}, "graph": {"ids": [411, 364, 279, 104362, 3377], "logits": [29.625, 19.375, 18.625, 16.375, 15.25], "margin": 10.25, "argmax": 411}, "compact_buffered": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "compact_buffered_state_diff": [], "buffered_graph_state_diff": []} -{"phase": "step", "index": 3, "position": 151, "teacher_token": 411, "compact": {"ids": [3377, 11, 3299, 96304, 5154], "logits": [31.625, 20.75, 17.25, 16.875, 16.125], "margin": 10.875, "argmax": 3377}, "buffered": {"ids": [3377, 11, 3299, 96304, 5154], "logits": [31.625, 20.75, 17.25, 16.875, 16.125], "margin": 10.875, "argmax": 3377}, "graph": {"ids": [3377, 11, 3299, 96304, 5154], "logits": [31.625, 20.75, 17.25, 16.875, 16.125], "margin": 10.875, "argmax": 3377}, "compact_buffered": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "compact_buffered_state_diff": [], "buffered_graph_state_diff": []} -{"phase": "step", "index": 4, "position": 152, "teacher_token": 3377, "compact": {"ids": [11, 25, 45309, 567, 2923], "logits": [24.5, 10.5625, 10.4375, 10.1875, 10.125], "margin": 13.9375, "argmax": 11}, "buffered": {"ids": [11, 25, 45309, 567, 2923], "logits": [24.5, 10.5625, 10.4375, 10.1875, 10.125], "margin": 13.9375, "argmax": 11}, "graph": {"ids": [11, 25, 45309, 567, 2923], "logits": [24.5, 10.5625, 10.4375, 10.1875, 10.125], "margin": 13.9375, "argmax": 11}, "compact_buffered": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "compact_buffered_state_diff": [], "buffered_graph_state_diff": []} -{"phase": "step", "index": 5, "position": 153, "teacher_token": 11, "compact": {"ids": [567, 1042, 1165, 6762, 109849], "logits": [29.625, 25.875, 15.5625, 15.5625, 15.3125], "margin": 3.75, "argmax": 567}, "buffered": {"ids": [567, 1042, 1165, 6762, 109849], "logits": [29.625, 25.875, 15.5625, 15.5625, 15.3125], "margin": 3.75, "argmax": 567}, "graph": {"ids": [567, 1042, 1165, 6762, 109849], "logits": [29.625, 25.875, 15.5625, 15.5625, 15.3125], "margin": 3.75, "argmax": 567}, "compact_buffered": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "compact_buffered_state_diff": [], "buffered_graph_state_diff": []} -{"phase": "step", "index": 6, "position": 154, "teacher_token": 567, "compact": {"ids": [1144, 628, 668, 6762, 1118], "logits": [30.5, 29.0, 25.25, 23.5, 23.5], "margin": 1.5, "argmax": 1144}, "buffered": {"ids": [1144, 628, 668, 6762, 1118], "logits": [30.5, 29.0, 25.25, 23.5, 23.5], "margin": 1.5, "argmax": 1144}, "graph": {"ids": [1144, 628, 668, 6762, 1118], "logits": [30.5, 29.0, 25.25, 23.5, 23.5], "margin": 1.5, "argmax": 1144}, "compact_buffered": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "compact_buffered_state_diff": [], "buffered_graph_state_diff": []} -{"phase": "step", "index": 7, "position": 155, "teacher_token": 1144, "compact": {"ids": [310, 7995, 121697, 10521, 716], "logits": [26.875, 14.4375, 13.0, 12.5, 11.8125], "margin": 12.4375, "argmax": 310}, "buffered": {"ids": [310, 7995, 121697, 10521, 716], "logits": [26.875, 14.4375, 13.0, 12.5, 11.8125], "margin": 12.4375, "argmax": 310}, "graph": {"ids": [310, 7995, 121697, 10521, 716], "logits": [26.875, 14.4375, 13.0, 12.5, 11.8125], "margin": 12.4375, "argmax": 310}, "compact_buffered": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "compact_buffered_state_diff": [], "buffered_graph_state_diff": []} -{"phase": "step", "index": 8, "position": 156, "teacher_token": 310, "compact": {"ids": [7995, 716, 3627, 5517, 6762], "logits": [30.25, 25.5, 24.875, 19.375, 19.25], "margin": 4.75, "argmax": 7995}, "buffered": {"ids": [7995, 716, 3627, 5517, 6762], "logits": [30.25, 25.5, 24.875, 19.375, 19.25], "margin": 4.75, "argmax": 7995}, "graph": {"ids": [7995, 716, 3627, 5517, 6762], "logits": [30.25, 25.5, 24.875, 19.375, 19.25], "margin": 4.75, "argmax": 7995}, "compact_buffered": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "compact_buffered_state_diff": [], "buffered_graph_state_diff": []} -{"phase": "step", "index": 9, "position": 157, "teacher_token": 7995, "compact": {"ids": [279, 1204, 13853, 1324, 1330], "logits": [27.5, 25.5, 15.625, 15.5, 14.5], "margin": 2.0, "argmax": 279}, "buffered": {"ids": [279, 1204, 13853, 1324, 1330], "logits": [27.5, 25.5, 15.625, 15.5, 14.5], "margin": 2.0, "argmax": 279}, "graph": {"ids": [279, 1204, 13853, 1324, 1330], "logits": [27.5, 25.5, 15.625, 15.5, 14.5], "margin": 2.0, "argmax": 279}, "compact_buffered": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "compact_buffered_state_diff": [], "buffered_graph_state_diff": []} -{"phase": "step", "index": 10, "position": 158, "teacher_token": 279, "compact": {"ids": [1324, 2702, 1534, 2222, 17498], "logits": [28.875, 21.125, 21.125, 21.125, 21.0], "margin": 7.75, "argmax": 1324}, "buffered": {"ids": [1324, 2702, 1534, 2222, 17498], "logits": [28.875, 21.125, 21.125, 21.125, 21.0], "margin": 7.75, "argmax": 1324}, "graph": {"ids": [1324, 2702, 1534, 2222, 17498], "logits": [28.875, 21.125, 21.125, 21.125, 21.0], "margin": 7.75, "argmax": 1324}, "compact_buffered": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "compact_buffered_state_diff": [], "buffered_graph_state_diff": []} -{"phase": "step", "index": 11, "position": 159, "teacher_token": 1324, "compact": {"ids": [314, 4022, 321, 491, 3018], "logits": [27.875, 16.125, 13.6875, 13.1875, 12.0], "margin": 11.75, "argmax": 314}, "buffered": {"ids": [314, 4022, 321, 491, 3018], "logits": [27.875, 16.125, 13.6875, 13.1875, 12.0], "margin": 11.75, "argmax": 314}, "graph": {"ids": [314, 4022, 321, 491, 3018], "logits": [27.875, 16.125, 13.6875, 13.1875, 12.0], "margin": 11.75, "argmax": 314}, "compact_buffered": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "compact_buffered_state_diff": [], "buffered_graph_state_diff": []} -{"phase": "step", "index": 12, "position": 160, "teacher_token": 314, "compact": {"ids": [4022, 491, 18016, 7397, 4945], "logits": [25.625, 21.375, 20.875, 19.375, 18.125], "margin": 4.25, "argmax": 4022}, "buffered": {"ids": [4022, 491, 18016, 7397, 4945], "logits": [25.625, 21.375, 20.875, 19.375, 18.125], "margin": 4.25, "argmax": 4022}, "graph": {"ids": [4022, 491, 18016, 7397, 4945], "logits": [25.625, 21.375, 20.875, 19.375, 18.125], "margin": 4.25, "argmax": 4022}, "compact_buffered": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "compact_buffered_state_diff": [], "buffered_graph_state_diff": []} -{"phase": "step", "index": 13, "position": 161, "teacher_token": 4022, "compact": {"ids": [7397, 321, 13853, 17629, 18016], "logits": [25.375, 23.25, 16.875, 15.9375, 15.875], "margin": 2.125, "argmax": 7397}, "buffered": {"ids": [7397, 321, 13853, 17629, 18016], "logits": [25.375, 23.25, 16.875, 15.9375, 15.875], "margin": 2.125, "argmax": 7397}, "graph": {"ids": [7397, 321, 13853, 17629, 18016], "logits": [25.375, 23.25, 16.875, 15.9375, 15.875], "margin": 2.125, "argmax": 7397}, "compact_buffered": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "compact_buffered_state_diff": [], "buffered_graph_state_diff": []} -{"phase": "step", "index": 14, "position": 162, "teacher_token": 7397, "compact": {"ids": [13853, 321, 3568, 10471, 14307], "logits": [24.875, 18.875, 17.875, 16.75, 15.9375], "margin": 6.0, "argmax": 13853}, "buffered": {"ids": [13853, 321, 3568, 10471, 14307], "logits": [24.875, 18.875, 17.875, 16.75, 15.9375], "margin": 6.0, "argmax": 13853}, "graph": {"ids": [13853, 321, 3568, 10471, 14307], "logits": [24.875, 18.875, 17.875, 16.75, 15.9375], "margin": 6.0, "argmax": 13853}, "compact_buffered": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "compact_buffered_state_diff": [], "buffered_graph_state_diff": []} -{"phase": "step", "index": 15, "position": 163, "teacher_token": 13853, "compact": {"ids": [10471, 14307, 3568, 6895, 3018], "logits": [27.25, 24.25, 23.875, 21.0, 18.5], "margin": 3.0, "argmax": 10471}, "buffered": {"ids": [10471, 14307, 3568, 6895, 3018], "logits": [27.25, 24.25, 23.875, 21.0, 18.5], "margin": 3.0, "argmax": 10471}, "graph": {"ids": [10471, 14307, 3568, 6895, 3018], "logits": [27.25, 24.25, 23.875, 21.0, 18.5], "margin": 3.0, "argmax": 10471}, "compact_buffered": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "compact_buffered_state_diff": [], "buffered_graph_state_diff": []} -{"phase": "step", "index": 16, "position": 164, "teacher_token": 10471, "compact": {"ids": [3018, 13, 11, 539, 506], "logits": [25.375, 25.0, 24.625, 24.5, 23.25], "margin": 0.375, "argmax": 3018}, "buffered": {"ids": [3018, 13, 11, 539, 506], "logits": [25.375, 25.0, 24.625, 24.5, 23.25], "margin": 0.375, "argmax": 3018}, "graph": {"ids": [3018, 13, 11, 539, 506], "logits": [25.375, 25.0, 24.625, 24.5, 23.25], "margin": 0.375, "argmax": 3018}, "compact_buffered": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "compact_buffered_state_diff": [], "buffered_graph_state_diff": []} -{"phase": "step", "index": 17, "position": 165, "teacher_token": 3018, "compact": {"ids": [383, 279, 263, 314, 4203], "logits": [26.25, 17.25, 12.8125, 12.1875, 12.0625], "margin": 9.0, "argmax": 383}, "buffered": {"ids": [383, 279, 263, 314, 4203], "logits": [26.25, 17.25, 12.8125, 12.1875, 12.0625], "margin": 9.0, "argmax": 383}, "graph": {"ids": [383, 279, 263, 314, 4203], "logits": [26.25, 17.25, 12.8125, 12.1875, 12.0625], "margin": 9.0, "argmax": 383}, "compact_buffered": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "compact_buffered_state_diff": [], "buffered_graph_state_diff": []} -{"phase": "step", "index": 18, "position": 166, "teacher_token": 383, "compact": {"ids": [279, 780, 4203, 2222, 1534], "logits": [28.125, 19.75, 16.75, 15.5, 14.875], "margin": 8.375, "argmax": 279}, "buffered": {"ids": [279, 780, 4203, 2222, 1534], "logits": [28.125, 19.75, 16.75, 15.5, 14.875], "margin": 8.375, "argmax": 279}, "graph": {"ids": [279, 780, 4203, 2222, 1534], "logits": [28.125, 19.75, 16.75, 15.5, 14.875], "margin": 8.375, "argmax": 279}, "compact_buffered": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "compact_buffered_state_diff": [], "buffered_graph_state_diff": []} -{"phase": "step", "index": 19, "position": 167, "teacher_token": 279, "compact": {"ids": [1534, 2222, 2763, 4203, 2702], "logits": [27.0, 24.75, 24.125, 24.125, 23.125], "margin": 2.25, "argmax": 1534}, "buffered": {"ids": [1534, 2222, 2763, 4203, 2702], "logits": [27.0, 24.75, 24.125, 24.125, 23.125], "margin": 2.25, "argmax": 1534}, "graph": {"ids": [1534, 2222, 2763, 4203, 2702], "logits": [27.0, 24.75, 24.125, 24.125, 23.125], "margin": 2.25, "argmax": 1534}, "compact_buffered": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "compact_buffered_state_diff": [], "buffered_graph_state_diff": []} -{"phase": "step", "index": 20, "position": 168, "teacher_token": 1534, "compact": {"ids": [11013, 17498, 14159, 7731, 39875], "logits": [26.25, 25.875, 24.375, 22.5, 22.25], "margin": 0.375, "argmax": 11013}, "buffered": {"ids": [11013, 17498, 14159, 7731, 39875], "logits": [26.25, 25.875, 24.375, 22.5, 22.25], "margin": 0.375, "argmax": 11013}, "graph": {"ids": [11013, 17498, 14159, 7731, 39875], "logits": [26.25, 25.875, 24.375, 22.5, 22.25], "margin": 0.375, "argmax": 11013}, "compact_buffered": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "compact_buffered_state_diff": [], "buffered_graph_state_diff": []} -{"phase": "step", "index": 21, "position": 169, "teacher_token": 11013, "compact": {"ids": [314, 303, 321, 13, 3766], "logits": [28.125, 24.875, 21.0, 18.375, 17.5], "margin": 3.25, "argmax": 314}, "buffered": {"ids": [314, 303, 321, 13, 3766], "logits": [28.125, 24.875, 21.0, 18.375, 17.5], "margin": 3.25, "argmax": 314}, "graph": {"ids": [314, 303, 321, 13, 3766], "logits": [28.125, 24.875, 21.0, 18.375, 17.5], "margin": 3.25, "argmax": 314}, "compact_buffered": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "compact_buffered_state_diff": [], "buffered_graph_state_diff": []} -{"phase": "step", "index": 22, "position": 170, "teacher_token": 314, "compact": {"ids": [18016, 7397, 7736, 780, 279], "logits": [24.0, 23.625, 19.625, 18.375, 17.75], "margin": 0.375, "argmax": 18016}, "buffered": {"ids": [18016, 7397, 7736, 780, 279], "logits": [24.0, 23.625, 19.625, 18.375, 17.75], "margin": 0.375, "argmax": 18016}, "graph": {"ids": [18016, 7397, 7736, 780, 279], "logits": [24.0, 23.625, 19.625, 18.375, 17.75], "margin": 0.375, "argmax": 18016}, "compact_buffered": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "compact_buffered_state_diff": [], "buffered_graph_state_diff": []} -{"phase": "step", "index": 23, "position": 171, "teacher_token": 18016, "compact": {"ids": [310, 7397, 321, 17629, 4534], "logits": [28.5, 22.75, 18.375, 17.25, 15.875], "margin": 5.75, "argmax": 310}, "buffered": {"ids": [310, 7397, 321, 17629, 4534], "logits": [28.625, 22.625, 18.125, 17.375, 15.875], "margin": 6.0, "argmax": 310}, "graph": {"ids": [310, 7397, 321, 17629, 4534], "logits": [28.625, 22.625, 18.125, 17.375, 15.875], "margin": 6.0, "argmax": 310}, "compact_buffered": {"exact": false, "max_abs": 0.2764892578125, "nrmse": 0.017587712034583092}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "compact_buffered_state_diff": [{"layer": 44, "field": "conv", "exact": false, "max_abs": 0.0625, "nrmse": 0.0030684019438922405}, {"layer": 44, "field": "recurrent", "exact": false, "max_abs": 0.001776188611984253, "nrmse": 0.0007243468426167965}, {"layer": 45, "field": "conv", "exact": false, "max_abs": 0.06640625, "nrmse": 0.008581223897635937}, {"layer": 45, "field": "recurrent", "exact": false, "max_abs": 0.0017136186361312866, "nrmse": 0.002778188558295369}, {"layer": 46, "field": "conv", "exact": false, "max_abs": 0.109375, "nrmse": 0.00989687442779541}, {"layer": 46, "field": "recurrent", "exact": false, "max_abs": 0.005556466057896614, "nrmse": 0.005136620718985796}, {"layer": 47, "field": "key", "exact": false, "max_abs": 0.1328125, "nrmse": 0.0012538685696199536}, {"layer": 47, "field": "value", "exact": false, "max_abs": 0.109375, "nrmse": 0.0016844975762069225}, {"layer": 48, "field": "conv", "exact": false, "max_abs": 0.1328125, "nrmse": 0.012064914219081402}, {"layer": 48, "field": "recurrent", "exact": false, "max_abs": 0.0025483909994363785, "nrmse": 0.0033667879179120064}, {"layer": 49, "field": "conv", "exact": false, "max_abs": 0.1875, "nrmse": 0.01195996068418026}, {"layer": 49, "field": "recurrent", "exact": false, "max_abs": 0.004310250282287598, "nrmse": 0.006455709226429462}, {"layer": 50, "field": "conv", "exact": false, "max_abs": 0.171875, "nrmse": 0.012615606188774109}, {"layer": 50, "field": "recurrent", "exact": false, "max_abs": 0.005076214671134949, "nrmse": 0.005148074124008417}, {"layer": 51, "field": "key", "exact": false, "max_abs": 0.15625, "nrmse": 0.0017117471434175968}, {"layer": 51, "field": "value", "exact": false, "max_abs": 0.17578125, "nrmse": 0.002343573607504368}, {"layer": 52, "field": "conv", "exact": false, "max_abs": 0.21875, "nrmse": 0.011075295507907867}, {"layer": 52, "field": "recurrent", "exact": false, "max_abs": 0.019721388816833496, "nrmse": 0.004348557908087969}, {"layer": 53, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.012180065736174583}, {"layer": 53, "field": "recurrent", "exact": false, "max_abs": 0.012011170387268066, "nrmse": 0.006310894154012203}, {"layer": 54, "field": "conv", "exact": false, "max_abs": 0.25, "nrmse": 0.013797740451991558}, {"layer": 54, "field": "recurrent", "exact": false, "max_abs": 0.00830298662185669, "nrmse": 0.008672994561493397}, {"layer": 55, "field": "key", "exact": false, "max_abs": 0.125, "nrmse": 0.0018120916793122888}, {"layer": 55, "field": "value", "exact": false, "max_abs": 0.26171875, "nrmse": 0.0028039978351444006}, {"layer": 56, "field": "conv", "exact": false, "max_abs": 0.21875, "nrmse": 0.01142491027712822}, {"layer": 56, "field": "recurrent", "exact": false, "max_abs": 0.015242815017700195, "nrmse": 0.00630442937836051}, {"layer": 57, "field": "conv", "exact": false, "max_abs": 0.375, "nrmse": 0.01068636029958725}, {"layer": 57, "field": "recurrent", "exact": false, "max_abs": 0.02260875701904297, "nrmse": 0.008974018506705761}, {"layer": 58, "field": "conv", "exact": false, "max_abs": 0.21875, "nrmse": 0.01129329577088356}, {"layer": 58, "field": "recurrent", "exact": false, "max_abs": 0.01868385076522827, "nrmse": 0.007724203635007143}, {"layer": 59, "field": "key", "exact": false, "max_abs": 0.125, "nrmse": 0.001612626831047237}, {"layer": 59, "field": "value", "exact": false, "max_abs": 0.34637451171875, "nrmse": 0.002165580168366432}, {"layer": 60, "field": "conv", "exact": false, "max_abs": 0.25, "nrmse": 0.009277026169002056}, {"layer": 60, "field": "recurrent", "exact": false, "max_abs": 0.010733306407928467, "nrmse": 0.004245422780513763}, {"layer": 61, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.013534123077988625}, {"layer": 61, "field": "recurrent", "exact": false, "max_abs": 0.010991841554641724, "nrmse": 0.009339485317468643}, {"layer": 62, "field": "conv", "exact": false, "max_abs": 0.203125, "nrmse": 0.00938336830586195}, {"layer": 62, "field": "recurrent", "exact": false, "max_abs": 0.00716051459312439, "nrmse": 0.002739340765401721}, {"layer": 63, "field": "key", "exact": false, "max_abs": 0.1484375, "nrmse": 0.002177826128900051}, {"layer": 63, "field": "value", "exact": false, "max_abs": 0.46875, "nrmse": 0.0012068833457306027}], "buffered_graph_state_diff": []} -{"phase": "step", "index": 24, "position": 172, "teacher_token": 310, "compact": {"ids": [4022, 7397, 5603, 5626, 15392], "logits": [29.5, 19.25, 16.625, 16.0, 14.875], "margin": 10.25, "argmax": 4022}, "buffered": {"ids": [4022, 7397, 5603, 5626, 15223], "logits": [29.5, 19.25, 16.625, 15.9375, 14.875], "margin": 10.25, "argmax": 4022}, "graph": {"ids": [4022, 7397, 5603, 5626, 15223], "logits": [29.5, 19.25, 16.625, 15.9375, 14.875], "margin": 10.25, "argmax": 4022}, "compact_buffered": {"exact": false, "max_abs": 0.3671875, "nrmse": 0.013163435272872448}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "compact_buffered_state_diff": [{"layer": 44, "field": "conv", "exact": false, "max_abs": 0.0625, "nrmse": 0.0030998974107205868}, {"layer": 44, "field": "recurrent", "exact": false, "max_abs": 0.0006057396531105042, "nrmse": 0.0013203236740082502}, {"layer": 45, "field": "conv", "exact": false, "max_abs": 0.0703125, "nrmse": 0.011526606976985931}, {"layer": 45, "field": "recurrent", "exact": false, "max_abs": 0.003116115927696228, "nrmse": 0.00470480602234602}, {"layer": 46, "field": "conv", "exact": false, "max_abs": 0.109375, "nrmse": 0.014038067311048508}, {"layer": 46, "field": "recurrent", "exact": false, "max_abs": 0.005031665787100792, "nrmse": 0.0075767431408166885}, {"layer": 47, "field": "key", "exact": false, "max_abs": 0.1328125, "nrmse": 0.0018090145895257592}, {"layer": 47, "field": "value", "exact": false, "max_abs": 0.13671875, "nrmse": 0.0025365205947309732}, {"layer": 48, "field": "conv", "exact": false, "max_abs": 0.1796875, "nrmse": 0.017777662724256516}, {"layer": 48, "field": "recurrent", "exact": false, "max_abs": 0.0028557926416397095, "nrmse": 0.005973083432763815}, {"layer": 49, "field": "conv", "exact": false, "max_abs": 0.1875, "nrmse": 0.017791956663131714}, {"layer": 49, "field": "recurrent", "exact": false, "max_abs": 0.007599383592605591, "nrmse": 0.009837957099080086}, {"layer": 50, "field": "conv", "exact": false, "max_abs": 0.171875, "nrmse": 0.01827160269021988}, {"layer": 50, "field": "recurrent", "exact": false, "max_abs": 0.004038766026496887, "nrmse": 0.008256028406322002}, {"layer": 51, "field": "key", "exact": false, "max_abs": 0.171875, "nrmse": 0.002429067390039563}, {"layer": 51, "field": "value", "exact": false, "max_abs": 0.21875, "nrmse": 0.003368081757798791}, {"layer": 52, "field": "conv", "exact": false, "max_abs": 0.25, "nrmse": 0.016546761617064476}, {"layer": 52, "field": "recurrent", "exact": false, "max_abs": 0.016048789024353027, "nrmse": 0.004988973494619131}, {"layer": 53, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.017332011833786964}, {"layer": 53, "field": "recurrent", "exact": false, "max_abs": 0.011740967631340027, "nrmse": 0.008330872282385826}, {"layer": 54, "field": "conv", "exact": false, "max_abs": 0.25, "nrmse": 0.01919926144182682}, {"layer": 54, "field": "recurrent", "exact": false, "max_abs": 0.008114248514175415, "nrmse": 0.010682012885808945}, {"layer": 55, "field": "key", "exact": false, "max_abs": 0.15625, "nrmse": 0.002549079479649663}, {"layer": 55, "field": "value", "exact": false, "max_abs": 0.26171875, "nrmse": 0.0035520836245268583}, {"layer": 56, "field": "conv", "exact": false, "max_abs": 0.21875, "nrmse": 0.016473842784762383}, {"layer": 56, "field": "recurrent", "exact": false, "max_abs": 0.009501904249191284, "nrmse": 0.00735496124252677}, {"layer": 57, "field": "conv", "exact": false, "max_abs": 0.375, "nrmse": 0.015380625613033772}, {"layer": 57, "field": "recurrent", "exact": false, "max_abs": 0.026046574115753174, "nrmse": 0.010611897334456444}, {"layer": 58, "field": "conv", "exact": false, "max_abs": 0.21875, "nrmse": 0.016044583171606064}, {"layer": 58, "field": "recurrent", "exact": false, "max_abs": 0.016187310218811035, "nrmse": 0.009086411446332932}, {"layer": 59, "field": "key", "exact": false, "max_abs": 0.17431640625, "nrmse": 0.0025790068320930004}, {"layer": 59, "field": "value", "exact": false, "max_abs": 0.34637451171875, "nrmse": 0.002800857415422797}, {"layer": 60, "field": "conv", "exact": false, "max_abs": 0.25, "nrmse": 0.013174155727028847}, {"layer": 60, "field": "recurrent", "exact": false, "max_abs": 0.015123292803764343, "nrmse": 0.006795981433242559}, {"layer": 61, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.017341051250696182}, {"layer": 61, "field": "recurrent", "exact": false, "max_abs": 0.007877245545387268, "nrmse": 0.010607891716063023}, {"layer": 62, "field": "conv", "exact": false, "max_abs": 0.25, "nrmse": 0.012782243080437183}, {"layer": 62, "field": "recurrent", "exact": false, "max_abs": 0.012845814228057861, "nrmse": 0.004572160542011261}, {"layer": 63, "field": "key", "exact": false, "max_abs": 0.15625, "nrmse": 0.0029625201132148504}, {"layer": 63, "field": "value", "exact": false, "max_abs": 0.46875, "nrmse": 0.0015391798224300146}], "buffered_graph_state_diff": []} -{"phase": "step", "index": 25, "position": 173, "teacher_token": 4022, "compact": {"ids": [7397, 17629, 91103, 303, 18698], "logits": [27.75, 16.875, 16.25, 15.375, 13.9375], "margin": 10.875, "argmax": 7397}, "buffered": {"ids": [7397, 17629, 91103, 303, 18698], "logits": [27.375, 16.625, 16.0, 15.3125, 13.6875], "margin": 10.75, "argmax": 7397}, "graph": {"ids": [7397, 17629, 91103, 303, 18698], "logits": [27.375, 16.625, 16.0, 15.3125, 13.6875], "margin": 10.75, "argmax": 7397}, "compact_buffered": {"exact": false, "max_abs": 0.375, "nrmse": 0.016292285174131393}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "compact_buffered_state_diff": [{"layer": 44, "field": "conv", "exact": false, "max_abs": 0.0625, "nrmse": 0.00312014645896852}, {"layer": 44, "field": "recurrent", "exact": false, "max_abs": 0.0005614124238491058, "nrmse": 0.0012291048187762499}, {"layer": 45, "field": "conv", "exact": false, "max_abs": 0.0703125, "nrmse": 0.013347189873456955}, {"layer": 45, "field": "recurrent", "exact": false, "max_abs": 0.0025259852409362793, "nrmse": 0.005287976935505867}, {"layer": 46, "field": "conv", "exact": false, "max_abs": 0.109375, "nrmse": 0.016683271154761314}, {"layer": 46, "field": "recurrent", "exact": false, "max_abs": 0.0042104572057724, "nrmse": 0.007280308287590742}, {"layer": 47, "field": "key", "exact": false, "max_abs": 0.1484375, "nrmse": 0.0022980705834925175}, {"layer": 47, "field": "value", "exact": false, "max_abs": 0.13671875, "nrmse": 0.0030274544842541218}, {"layer": 48, "field": "conv", "exact": false, "max_abs": 0.1796875, "nrmse": 0.02189135178923607}, {"layer": 48, "field": "recurrent", "exact": false, "max_abs": 0.003824777901172638, "nrmse": 0.0073199281468987465}, {"layer": 49, "field": "conv", "exact": false, "max_abs": 0.1875, "nrmse": 0.021954797208309174}, {"layer": 49, "field": "recurrent", "exact": false, "max_abs": 0.0057404860854148865, "nrmse": 0.012164085172116756}, {"layer": 50, "field": "conv", "exact": false, "max_abs": 0.21875, "nrmse": 0.022818991914391518}, {"layer": 50, "field": "recurrent", "exact": false, "max_abs": 0.0041518062353134155, "nrmse": 0.009543257765471935}, {"layer": 51, "field": "key", "exact": false, "max_abs": 0.23779296875, "nrmse": 0.0032619438134133816}, {"layer": 51, "field": "value", "exact": false, "max_abs": 0.2265625, "nrmse": 0.004296057857573032}, {"layer": 52, "field": "conv", "exact": false, "max_abs": 0.298828125, "nrmse": 0.020919904112815857}, {"layer": 52, "field": "recurrent", "exact": false, "max_abs": 0.01857733726501465, "nrmse": 0.005687769502401352}, {"layer": 53, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.0220721997320652}, {"layer": 53, "field": "recurrent", "exact": false, "max_abs": 0.012117743492126465, "nrmse": 0.009676119312644005}, {"layer": 54, "field": "conv", "exact": false, "max_abs": 0.25, "nrmse": 0.023788874968886375}, {"layer": 54, "field": "recurrent", "exact": false, "max_abs": 0.012822568416595459, "nrmse": 0.012072643265128136}, {"layer": 55, "field": "key", "exact": false, "max_abs": 0.15625, "nrmse": 0.003149081254377961}, {"layer": 55, "field": "value", "exact": false, "max_abs": 0.287109375, "nrmse": 0.004448418039828539}, {"layer": 56, "field": "conv", "exact": false, "max_abs": 0.2734375, "nrmse": 0.020678753033280373}, {"layer": 56, "field": "recurrent", "exact": false, "max_abs": 0.013139903545379639, "nrmse": 0.008448116481304169}, {"layer": 57, "field": "conv", "exact": false, "max_abs": 0.375, "nrmse": 0.01946233958005905}, {"layer": 57, "field": "recurrent", "exact": false, "max_abs": 0.01775050163269043, "nrmse": 0.011545231565833092}, {"layer": 58, "field": "conv", "exact": false, "max_abs": 0.25, "nrmse": 0.020651113241910934}, {"layer": 58, "field": "recurrent", "exact": false, "max_abs": 0.015999913215637207, "nrmse": 0.009856054559350014}, {"layer": 59, "field": "key", "exact": false, "max_abs": 0.17431640625, "nrmse": 0.003377198474481702}, {"layer": 59, "field": "value", "exact": false, "max_abs": 0.37109375, "nrmse": 0.0035408090334385633}, {"layer": 60, "field": "conv", "exact": false, "max_abs": 0.625, "nrmse": 0.017432335764169693}, {"layer": 60, "field": "recurrent", "exact": false, "max_abs": 0.012438341975212097, "nrmse": 0.007434377912431955}, {"layer": 61, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.020906049758195877}, {"layer": 61, "field": "recurrent", "exact": false, "max_abs": 0.012971535325050354, "nrmse": 0.01062857173383236}, {"layer": 62, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.016197126358747482}, {"layer": 62, "field": "recurrent", "exact": false, "max_abs": 0.011558860540390015, "nrmse": 0.0047526792623102665}, {"layer": 63, "field": "key", "exact": false, "max_abs": 0.2109375, "nrmse": 0.003840430174022913}, {"layer": 63, "field": "value", "exact": false, "max_abs": 0.5, "nrmse": 0.00192305410746485}], "buffered_graph_state_diff": []} -{"phase": "step", "index": 26, "position": 174, "teacher_token": 7397, "compact": {"ids": [303, 13, 321, 551, 1238], "logits": [26.25, 24.0, 17.625, 15.4375, 14.75], "margin": 2.25, "argmax": 303}, "buffered": {"ids": [303, 13, 321, 551, 1238], "logits": [26.375, 24.0, 17.625, 15.375, 14.6875], "margin": 2.375, "argmax": 303}, "graph": {"ids": [303, 13, 321, 551, 1238], "logits": [26.375, 24.0, 17.625, 15.375, 14.6875], "margin": 2.375, "argmax": 303}, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.01847914233803749}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "compact_buffered_state_diff": [{"layer": 44, "field": "conv", "exact": false, "max_abs": 0.0625, "nrmse": 0.0031188977882266045}, {"layer": 44, "field": "recurrent", "exact": false, "max_abs": 0.0005038082599639893, "nrmse": 0.0010994551703333855}, {"layer": 45, "field": "conv", "exact": false, "max_abs": 0.0703125, "nrmse": 0.01502121053636074}, {"layer": 45, "field": "recurrent", "exact": false, "max_abs": 0.0023186951875686646, "nrmse": 0.005496059078723192}, {"layer": 46, "field": "conv", "exact": false, "max_abs": 0.109375, "nrmse": 0.018706584349274635}, {"layer": 46, "field": "recurrent", "exact": false, "max_abs": 0.004307925701141357, "nrmse": 0.007337612099945545}, {"layer": 47, "field": "key", "exact": false, "max_abs": 0.1484375, "nrmse": 0.0027686075773090124}, {"layer": 47, "field": "value", "exact": false, "max_abs": 0.13671875, "nrmse": 0.0035241683945059776}, {"layer": 48, "field": "conv", "exact": false, "max_abs": 0.1796875, "nrmse": 0.02483254112303257}, {"layer": 48, "field": "recurrent", "exact": false, "max_abs": 0.003030940890312195, "nrmse": 0.007923978380858898}, {"layer": 49, "field": "conv", "exact": false, "max_abs": 0.1875, "nrmse": 0.025311090052127838}, {"layer": 49, "field": "recurrent", "exact": false, "max_abs": 0.00666499137878418, "nrmse": 0.013366514816880226}, {"layer": 50, "field": "conv", "exact": false, "max_abs": 0.21875, "nrmse": 0.026588594540953636}, {"layer": 50, "field": "recurrent", "exact": false, "max_abs": 0.004898160696029663, "nrmse": 0.010867545381188393}, {"layer": 51, "field": "key", "exact": false, "max_abs": 0.23779296875, "nrmse": 0.003900021081790328}, {"layer": 51, "field": "value", "exact": false, "max_abs": 0.3125, "nrmse": 0.005102069117128849}, {"layer": 52, "field": "conv", "exact": false, "max_abs": 0.298828125, "nrmse": 0.023820027709007263}, {"layer": 52, "field": "recurrent", "exact": false, "max_abs": 0.01246500015258789, "nrmse": 0.006277591921389103}, {"layer": 53, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.024984221905469894}, {"layer": 53, "field": "recurrent", "exact": false, "max_abs": 0.012613728642463684, "nrmse": 0.009957578033208847}, {"layer": 54, "field": "conv", "exact": false, "max_abs": 0.25, "nrmse": 0.02734456956386566}, {"layer": 54, "field": "recurrent", "exact": false, "max_abs": 0.0071926116943359375, "nrmse": 0.012335355393588543}, {"layer": 55, "field": "key", "exact": false, "max_abs": 0.203125, "nrmse": 0.003808452980592847}, {"layer": 55, "field": "value", "exact": false, "max_abs": 0.287109375, "nrmse": 0.005211383104324341}, {"layer": 56, "field": "conv", "exact": false, "max_abs": 0.2734375, "nrmse": 0.023554660379886627}, {"layer": 56, "field": "recurrent", "exact": false, "max_abs": 0.014546602964401245, "nrmse": 0.008702428080141544}, {"layer": 57, "field": "conv", "exact": false, "max_abs": 0.375, "nrmse": 0.02259988710284233}, {"layer": 57, "field": "recurrent", "exact": false, "max_abs": 0.015061426907777786, "nrmse": 0.012029689736664295}, {"layer": 58, "field": "conv", "exact": false, "max_abs": 0.25, "nrmse": 0.024239717051386833}, {"layer": 58, "field": "recurrent", "exact": false, "max_abs": 0.01914045214653015, "nrmse": 0.010084000416100025}, {"layer": 59, "field": "key", "exact": false, "max_abs": 0.1796875, "nrmse": 0.0041398098692297935}, {"layer": 59, "field": "value", "exact": false, "max_abs": 0.4296875, "nrmse": 0.004238666035234928}, {"layer": 60, "field": "conv", "exact": false, "max_abs": 0.625, "nrmse": 0.020267518237233162}, {"layer": 60, "field": "recurrent", "exact": false, "max_abs": 0.014546990394592285, "nrmse": 0.008216354064643383}, {"layer": 61, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.023734716698527336}, {"layer": 61, "field": "recurrent", "exact": false, "max_abs": 0.006846003234386444, "nrmse": 0.010458974167704582}, {"layer": 62, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.018880680203437805}, {"layer": 62, "field": "recurrent", "exact": false, "max_abs": 0.009744912385940552, "nrmse": 0.00499359518289566}, {"layer": 63, "field": "key", "exact": false, "max_abs": 0.2109375, "nrmse": 0.004536193795502186}, {"layer": 63, "field": "value", "exact": false, "max_abs": 0.5, "nrmse": 0.0022526211105287075}], "buffered_graph_state_diff": []} -{"phase": "step", "index": 27, "position": 175, "teacher_token": 303, "compact": {"ids": [780, 279, 836, 34393, 1141], "logits": [27.625, 21.5, 14.75, 14.4375, 14.4375], "margin": 6.125, "argmax": 780}, "buffered": {"ids": [780, 279, 836, 1141, 34393], "logits": [27.75, 21.625, 14.8125, 14.5625, 14.5], "margin": 6.125, "argmax": 780}, "graph": {"ids": [780, 279, 836, 1141, 34393], "logits": [27.75, 21.625, 14.8125, 14.5625, 14.5], "margin": 6.125, "argmax": 780}, "compact_buffered": {"exact": false, "max_abs": 0.234375, "nrmse": 0.01507080439478159}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "compact_buffered_state_diff": [{"layer": 44, "field": "recurrent", "exact": false, "max_abs": 0.0004266873002052307, "nrmse": 0.000996770104393363}, {"layer": 45, "field": "conv", "exact": false, "max_abs": 0.0703125, "nrmse": 0.013669734820723534}, {"layer": 45, "field": "recurrent", "exact": false, "max_abs": 0.003061160445213318, "nrmse": 0.005890420638024807}, {"layer": 46, "field": "conv", "exact": false, "max_abs": 0.1015625, "nrmse": 0.017981024459004402}, {"layer": 46, "field": "recurrent", "exact": false, "max_abs": 0.005652010440826416, "nrmse": 0.0073966714553534985}, {"layer": 47, "field": "key", "exact": false, "max_abs": 0.1484375, "nrmse": 0.0030109118670225143}, {"layer": 47, "field": "value", "exact": false, "max_abs": 0.13671875, "nrmse": 0.0037659239023923874}, {"layer": 48, "field": "conv", "exact": false, "max_abs": 0.1796875, "nrmse": 0.024416958913207054}, {"layer": 48, "field": "recurrent", "exact": false, "max_abs": 0.0032118111848831177, "nrmse": 0.00853046216070652}, {"layer": 49, "field": "conv", "exact": false, "max_abs": 0.1875, "nrmse": 0.02511255070567131}, {"layer": 49, "field": "recurrent", "exact": false, "max_abs": 0.0057115256786346436, "nrmse": 0.014217900112271309}, {"layer": 50, "field": "conv", "exact": false, "max_abs": 0.21875, "nrmse": 0.02649008482694626}, {"layer": 50, "field": "recurrent", "exact": false, "max_abs": 0.004892934113740921, "nrmse": 0.011930996552109718}, {"layer": 51, "field": "key", "exact": false, "max_abs": 0.23779296875, "nrmse": 0.004138512071222067}, {"layer": 51, "field": "value", "exact": false, "max_abs": 0.3125, "nrmse": 0.005416129250079393}, {"layer": 52, "field": "conv", "exact": false, "max_abs": 0.298828125, "nrmse": 0.023790959268808365}, {"layer": 52, "field": "recurrent", "exact": false, "max_abs": 0.015648484230041504, "nrmse": 0.007058687042444944}, {"layer": 53, "field": "conv", "exact": false, "max_abs": 0.28125, "nrmse": 0.024397574365139008}, {"layer": 53, "field": "recurrent", "exact": false, "max_abs": 0.014892742037773132, "nrmse": 0.010576128959655762}, {"layer": 54, "field": "conv", "exact": false, "max_abs": 0.234375, "nrmse": 0.026327479630708694}, {"layer": 54, "field": "recurrent", "exact": false, "max_abs": 0.00995597243309021, "nrmse": 0.013576802797615528}, {"layer": 55, "field": "key", "exact": false, "max_abs": 0.203125, "nrmse": 0.0042528072372078896}, {"layer": 55, "field": "value", "exact": false, "max_abs": 0.287109375, "nrmse": 0.005642397329211235}, {"layer": 56, "field": "conv", "exact": false, "max_abs": 0.2734375, "nrmse": 0.02311546914279461}, {"layer": 56, "field": "recurrent", "exact": false, "max_abs": 0.01963619887828827, "nrmse": 0.010097124613821507}, {"layer": 57, "field": "conv", "exact": false, "max_abs": 0.328125, "nrmse": 0.022232860326766968}, {"layer": 57, "field": "recurrent", "exact": false, "max_abs": 0.017313972115516663, "nrmse": 0.01261198054999113}, {"layer": 58, "field": "conv", "exact": false, "max_abs": 0.25, "nrmse": 0.02374468743801117}, {"layer": 58, "field": "recurrent", "exact": false, "max_abs": 0.021460920572280884, "nrmse": 0.010596529580652714}, {"layer": 59, "field": "key", "exact": false, "max_abs": 0.1796875, "nrmse": 0.00457134610041976}, {"layer": 59, "field": "value", "exact": false, "max_abs": 0.4296875, "nrmse": 0.004653829615563154}, {"layer": 60, "field": "conv", "exact": false, "max_abs": 0.625, "nrmse": 0.020141201093792915}, {"layer": 60, "field": "recurrent", "exact": false, "max_abs": 0.01348055899143219, "nrmse": 0.008879966102540493}, {"layer": 61, "field": "conv", "exact": false, "max_abs": 0.2734375, "nrmse": 0.021647578105330467}, {"layer": 61, "field": "recurrent", "exact": false, "max_abs": 0.012698644772171974, "nrmse": 0.011469818651676178}, {"layer": 62, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.018701225519180298}, {"layer": 62, "field": "recurrent", "exact": false, "max_abs": 0.01367156207561493, "nrmse": 0.005794625263661146}, {"layer": 63, "field": "key", "exact": false, "max_abs": 0.2109375, "nrmse": 0.004962374921888113}, {"layer": 63, "field": "value", "exact": false, "max_abs": 0.5, "nrmse": 0.0024725657422095537}], "buffered_graph_state_diff": []} -{"phase": "step", "index": 28, "position": 176, "teacher_token": 780, "compact": {"ids": [34393, 3396, 7087, 85312, 12526], "logits": [28.375, 16.875, 16.0, 15.4375, 15.375], "margin": 11.5, "argmax": 34393}, "buffered": {"ids": [34393, 3396, 7087, 85312, 28523], "logits": [28.375, 16.75, 16.0, 15.25, 15.25], "margin": 11.625, "argmax": 34393}, "graph": {"ids": [34393, 3396, 7087, 85312, 28523], "logits": [28.375, 16.75, 16.0, 15.25, 15.25], "margin": 11.625, "argmax": 34393}, "compact_buffered": {"exact": false, "max_abs": 0.375, "nrmse": 0.026475626975297928}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "compact_buffered_state_diff": [{"layer": 36, "field": "recurrent", "exact": false, "max_abs": 5.960464477539062e-07, "nrmse": 1.5438365608133608e-07}, {"layer": 37, "field": "conv", "exact": false, "max_abs": 0.0625, "nrmse": 0.003298611845821142}, {"layer": 37, "field": "recurrent", "exact": false, "max_abs": 0.0025909245014190674, "nrmse": 0.0016202675178647041}, {"layer": 38, "field": "conv", "exact": false, "max_abs": 0.09375, "nrmse": 0.007835827767848969}, {"layer": 38, "field": "recurrent", "exact": false, "max_abs": 0.0062901973724365234, "nrmse": 0.005308365914970636}, {"layer": 39, "field": "key", "exact": false, "max_abs": 0.125, "nrmse": 0.0014060799730941653}, {"layer": 39, "field": "value", "exact": false, "max_abs": 0.07303714752197266, "nrmse": 0.00151668896432966}, {"layer": 40, "field": "conv", "exact": false, "max_abs": 0.125, "nrmse": 0.010174120776355267}, {"layer": 40, "field": "recurrent", "exact": false, "max_abs": 0.0048596858978271484, "nrmse": 0.0018728034337982535}, {"layer": 41, "field": "conv", "exact": false, "max_abs": 0.15625, "nrmse": 0.012156014330685139}, {"layer": 41, "field": "recurrent", "exact": false, "max_abs": 0.0022789686918258667, "nrmse": 0.0036645804066210985}, {"layer": 42, "field": "conv", "exact": false, "max_abs": 0.1279296875, "nrmse": 0.012366200797259808}, {"layer": 42, "field": "recurrent", "exact": false, "max_abs": 0.00651269406080246, "nrmse": 0.004844251554459333}, {"layer": 43, "field": "key", "exact": false, "max_abs": 0.16015625, "nrmse": 0.0017111217603087425}, {"layer": 43, "field": "value", "exact": false, "max_abs": 0.125, "nrmse": 0.0017324859509244561}, {"layer": 44, "field": "conv", "exact": false, "max_abs": 0.11328125, "nrmse": 0.012295419350266457}, {"layer": 44, "field": "recurrent", "exact": false, "max_abs": 0.0035738786682486534, "nrmse": 0.003298657014966011}, {"layer": 45, "field": "conv", "exact": false, "max_abs": 0.125, "nrmse": 0.018066246062517166}, {"layer": 45, "field": "recurrent", "exact": false, "max_abs": 0.005233297124505043, "nrmse": 0.007893381640315056}, {"layer": 46, "field": "conv", "exact": false, "max_abs": 0.15625, "nrmse": 0.020335078239440918}, {"layer": 46, "field": "recurrent", "exact": false, "max_abs": 0.005906910635530949, "nrmse": 0.009689252823591232}, {"layer": 47, "field": "key", "exact": false, "max_abs": 0.25, "nrmse": 0.003526342799887061}, {"layer": 47, "field": "value", "exact": false, "max_abs": 0.13671875, "nrmse": 0.0043679471127688885}, {"layer": 48, "field": "conv", "exact": false, "max_abs": 0.140625, "nrmse": 0.024575844407081604}, {"layer": 48, "field": "recurrent", "exact": false, "max_abs": 0.0028948932886123657, "nrmse": 0.008889268152415752}, {"layer": 49, "field": "conv", "exact": false, "max_abs": 0.1875, "nrmse": 0.02513451874256134}, {"layer": 49, "field": "recurrent", "exact": false, "max_abs": 0.005156241357326508, "nrmse": 0.015080521814525127}, {"layer": 50, "field": "conv", "exact": false, "max_abs": 0.21875, "nrmse": 0.0267336443066597}, {"layer": 50, "field": "recurrent", "exact": false, "max_abs": 0.006207913160324097, "nrmse": 0.01285432931035757}, {"layer": 51, "field": "key", "exact": false, "max_abs": 0.23779296875, "nrmse": 0.004438433796167374}, {"layer": 51, "field": "value", "exact": false, "max_abs": 0.3125, "nrmse": 0.00587433110922575}, {"layer": 52, "field": "conv", "exact": false, "max_abs": 0.298828125, "nrmse": 0.023532500490546227}, {"layer": 52, "field": "recurrent", "exact": false, "max_abs": 0.014620274305343628, "nrmse": 0.007639180403202772}, {"layer": 53, "field": "conv", "exact": false, "max_abs": 0.28125, "nrmse": 0.02418258786201477}, {"layer": 53, "field": "recurrent", "exact": false, "max_abs": 0.010413885116577148, "nrmse": 0.010588320903480053}, {"layer": 54, "field": "conv", "exact": false, "max_abs": 0.234375, "nrmse": 0.02591981366276741}, {"layer": 54, "field": "recurrent", "exact": false, "max_abs": 0.010165631771087646, "nrmse": 0.014110561460256577}, {"layer": 55, "field": "key", "exact": false, "max_abs": 0.203125, "nrmse": 0.004702648613601923}, {"layer": 55, "field": "value", "exact": false, "max_abs": 0.287109375, "nrmse": 0.006093751639127731}, {"layer": 56, "field": "conv", "exact": false, "max_abs": 0.2734375, "nrmse": 0.022559914737939835}, {"layer": 56, "field": "recurrent", "exact": false, "max_abs": 0.013530967757105827, "nrmse": 0.00986373983323574}, {"layer": 57, "field": "conv", "exact": false, "max_abs": 0.34375, "nrmse": 0.021549776196479797}, {"layer": 57, "field": "recurrent", "exact": false, "max_abs": 0.029377609491348267, "nrmse": 0.012797139585018158}, {"layer": 58, "field": "conv", "exact": false, "max_abs": 0.25, "nrmse": 0.02362957037985325}, {"layer": 58, "field": "recurrent", "exact": false, "max_abs": 0.013305485248565674, "nrmse": 0.010668792761862278}, {"layer": 59, "field": "key", "exact": false, "max_abs": 0.1796875, "nrmse": 0.005004212725907564}, {"layer": 59, "field": "value", "exact": false, "max_abs": 0.525390625, "nrmse": 0.005056861788034439}, {"layer": 60, "field": "conv", "exact": false, "max_abs": 0.625, "nrmse": 0.0200160164386034}, {"layer": 60, "field": "recurrent", "exact": false, "max_abs": 0.0217103511095047, "nrmse": 0.010001263581216335}, {"layer": 61, "field": "conv", "exact": false, "max_abs": 0.2734375, "nrmse": 0.021736761555075645}, {"layer": 61, "field": "recurrent", "exact": false, "max_abs": 0.008070014417171478, "nrmse": 0.012234177440404892}, {"layer": 62, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.018837114796042442}, {"layer": 62, "field": "recurrent", "exact": false, "max_abs": 0.0168645977973938, "nrmse": 0.006667186040431261}, {"layer": 63, "field": "key", "exact": false, "max_abs": 0.2109375, "nrmse": 0.0053483229130506516}, {"layer": 63, "field": "value", "exact": false, "max_abs": 0.5, "nrmse": 0.00269137741997838}], "buffered_graph_state_diff": []} -{"phase": "step", "index": 29, "position": 177, "teacher_token": 34393, "compact": {"ids": [13, 271, 321, 11, 1238], "logits": [24.0, 12.625, 12.25, 12.0625, 11.8125], "margin": 11.375, "argmax": 13}, "buffered": {"ids": [13, 271, 321, 1238, 11], "logits": [24.0, 12.625, 12.3125, 12.0625, 12.0625], "margin": 11.375, "argmax": 13}, "graph": {"ids": [13, 271, 321, 1238, 11], "logits": [24.0, 12.625, 12.3125, 12.0625, 12.0625], "margin": 11.375, "argmax": 13}, "compact_buffered": {"exact": false, "max_abs": 0.359375, "nrmse": 0.01977214217185974}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "compact_buffered_state_diff": [{"layer": 36, "field": "recurrent", "exact": false, "max_abs": 5.960464477539062e-07, "nrmse": 1.523983286233488e-07}, {"layer": 37, "field": "conv", "exact": false, "max_abs": 0.0625, "nrmse": 0.00327504426240921}, {"layer": 37, "field": "recurrent", "exact": false, "max_abs": 0.0026026368141174316, "nrmse": 0.001326326048001647}, {"layer": 38, "field": "conv", "exact": false, "max_abs": 0.09375, "nrmse": 0.010760122910141945}, {"layer": 38, "field": "recurrent", "exact": false, "max_abs": 0.005262494087219238, "nrmse": 0.0061406628228724}, {"layer": 39, "field": "key", "exact": false, "max_abs": 0.125, "nrmse": 0.0017352624563500285}, {"layer": 39, "field": "value", "exact": false, "max_abs": 0.0830078125, "nrmse": 0.002019644482061267}, {"layer": 40, "field": "conv", "exact": false, "max_abs": 0.125, "nrmse": 0.013486634939908981}, {"layer": 40, "field": "recurrent", "exact": false, "max_abs": 0.007325857877731323, "nrmse": 0.003096088068559766}, {"layer": 41, "field": "conv", "exact": false, "max_abs": 0.15625, "nrmse": 0.016047948971390724}, {"layer": 41, "field": "recurrent", "exact": false, "max_abs": 0.005078732967376709, "nrmse": 0.006397385615855455}, {"layer": 42, "field": "conv", "exact": false, "max_abs": 0.1279296875, "nrmse": 0.017342308536171913}, {"layer": 42, "field": "recurrent", "exact": false, "max_abs": 0.005451895296573639, "nrmse": 0.0056550148874521255}, {"layer": 43, "field": "key", "exact": false, "max_abs": 0.16015625, "nrmse": 0.0022446291986852884}, {"layer": 43, "field": "value", "exact": false, "max_abs": 0.125, "nrmse": 0.0024176056031137705}, {"layer": 44, "field": "conv", "exact": false, "max_abs": 0.1484375, "nrmse": 0.017345113679766655}, {"layer": 44, "field": "recurrent", "exact": false, "max_abs": 0.008807212114334106, "nrmse": 0.004813707433640957}, {"layer": 45, "field": "conv", "exact": false, "max_abs": 0.125, "nrmse": 0.021606311202049255}, {"layer": 45, "field": "recurrent", "exact": false, "max_abs": 0.00466996431350708, "nrmse": 0.00976291298866272}, {"layer": 46, "field": "conv", "exact": false, "max_abs": 0.15625, "nrmse": 0.023040732368826866}, {"layer": 46, "field": "recurrent", "exact": false, "max_abs": 0.009523987770080566, "nrmse": 0.012891607359051704}, {"layer": 47, "field": "key", "exact": false, "max_abs": 0.25, "nrmse": 0.003941274713724852}, {"layer": 47, "field": "value", "exact": false, "max_abs": 0.15625, "nrmse": 0.00491897389292717}, {"layer": 48, "field": "conv", "exact": false, "max_abs": 0.201171875, "nrmse": 0.025463148951530457}, {"layer": 48, "field": "recurrent", "exact": false, "max_abs": 0.0036963820457458496, "nrmse": 0.009437368251383305}, {"layer": 49, "field": "conv", "exact": false, "max_abs": 0.224609375, "nrmse": 0.026101911440491676}, {"layer": 49, "field": "recurrent", "exact": false, "max_abs": 0.005205336958169937, "nrmse": 0.016330810263752937}, {"layer": 50, "field": "conv", "exact": false, "max_abs": 0.2421875, "nrmse": 0.028021620586514473}, {"layer": 50, "field": "recurrent", "exact": false, "max_abs": 0.0056074149906635284, "nrmse": 0.01354991178959608}, {"layer": 51, "field": "key", "exact": false, "max_abs": 0.23779296875, "nrmse": 0.00470945006236434}, {"layer": 51, "field": "value", "exact": false, "max_abs": 0.3125, "nrmse": 0.0064892699010670185}, {"layer": 52, "field": "conv", "exact": false, "max_abs": 0.23828125, "nrmse": 0.023256195709109306}, {"layer": 52, "field": "recurrent", "exact": false, "max_abs": 0.03228425979614258, "nrmse": 0.007975570857524872}, {"layer": 53, "field": "conv", "exact": false, "max_abs": 0.25, "nrmse": 0.023908454924821854}, {"layer": 53, "field": "recurrent", "exact": false, "max_abs": 0.009990602731704712, "nrmse": 0.01097980234771967}, {"layer": 54, "field": "conv", "exact": false, "max_abs": 0.25, "nrmse": 0.02661287970840931}, {"layer": 54, "field": "recurrent", "exact": false, "max_abs": 0.012155026197433472, "nrmse": 0.014879632741212845}, {"layer": 55, "field": "key", "exact": false, "max_abs": 0.203125, "nrmse": 0.0050735133700072765}, {"layer": 55, "field": "value", "exact": false, "max_abs": 0.287109375, "nrmse": 0.006726428400725126}, {"layer": 56, "field": "conv", "exact": false, "max_abs": 0.265625, "nrmse": 0.02272261120378971}, {"layer": 56, "field": "recurrent", "exact": false, "max_abs": 0.013109207153320312, "nrmse": 0.010326111689209938}, {"layer": 57, "field": "conv", "exact": false, "max_abs": 0.34375, "nrmse": 0.021454360336065292}, {"layer": 57, "field": "recurrent", "exact": false, "max_abs": 0.016015291213989258, "nrmse": 0.013513700105249882}, {"layer": 58, "field": "conv", "exact": false, "max_abs": 0.25, "nrmse": 0.02375240996479988}, {"layer": 58, "field": "recurrent", "exact": false, "max_abs": 0.013843744993209839, "nrmse": 0.010912948288023472}, {"layer": 59, "field": "key", "exact": false, "max_abs": 0.1796875, "nrmse": 0.005310228560119867}, {"layer": 59, "field": "value", "exact": false, "max_abs": 0.525390625, "nrmse": 0.005636544432491064}, {"layer": 60, "field": "conv", "exact": false, "max_abs": 0.75, "nrmse": 0.020131254568696022}, {"layer": 60, "field": "recurrent", "exact": false, "max_abs": 0.019167862832546234, "nrmse": 0.010166854597628117}, {"layer": 61, "field": "conv", "exact": false, "max_abs": 0.2734375, "nrmse": 0.022365953773260117}, {"layer": 61, "field": "recurrent", "exact": false, "max_abs": 0.007702067494392395, "nrmse": 0.012553741224110126}, {"layer": 62, "field": "conv", "exact": false, "max_abs": 0.53125, "nrmse": 0.019082963466644287}, {"layer": 62, "field": "recurrent", "exact": false, "max_abs": 0.015890419483184814, "nrmse": 0.006710229441523552}, {"layer": 63, "field": "key", "exact": false, "max_abs": 0.2109375, "nrmse": 0.005773698445409536}, {"layer": 63, "field": "value", "exact": false, "max_abs": 0.5, "nrmse": 0.0029913855250924826}], "buffered_graph_state_diff": []} -{"phase": "step", "index": 30, "position": 178, "teacher_token": 13, "compact": {"ids": [271, 6558, 5514, 2972, 1532], "logits": [26.125, 15.3125, 14.6875, 11.25, 11.0], "margin": 10.8125, "argmax": 271}, "buffered": {"ids": [271, 6558, 5514, 2972, 1532], "logits": [26.0, 15.3125, 14.75, 11.1875, 11.125], "margin": 10.6875, "argmax": 271}, "graph": {"ids": [271, 6558, 5514, 2972, 1532], "logits": [26.0, 15.3125, 14.75, 11.1875, 11.125], "margin": 10.6875, "argmax": 271}, "compact_buffered": {"exact": false, "max_abs": 0.3125, "nrmse": 0.014291426166892052}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "compact_buffered_state_diff": [{"layer": 36, "field": "recurrent", "exact": false, "max_abs": 5.960464477539062e-07, "nrmse": 1.504206750269077e-07}, {"layer": 37, "field": "conv", "exact": false, "max_abs": 0.0625, "nrmse": 0.0032421546056866646}, {"layer": 37, "field": "recurrent", "exact": false, "max_abs": 0.0020457208156585693, "nrmse": 0.0009619743213988841}, {"layer": 38, "field": "conv", "exact": false, "max_abs": 0.09375, "nrmse": 0.012235855683684349}, {"layer": 38, "field": "recurrent", "exact": false, "max_abs": 0.010132163763046265, "nrmse": 0.007110693957656622}, {"layer": 39, "field": "key", "exact": false, "max_abs": 0.125, "nrmse": 0.002131725661456585}, {"layer": 39, "field": "value", "exact": false, "max_abs": 0.0830078125, "nrmse": 0.0024740942753851414}, {"layer": 40, "field": "conv", "exact": false, "max_abs": 0.125, "nrmse": 0.01579064317047596}, {"layer": 40, "field": "recurrent", "exact": false, "max_abs": 0.006963133811950684, "nrmse": 0.002913985401391983}, {"layer": 41, "field": "conv", "exact": false, "max_abs": 0.15625, "nrmse": 0.018949098885059357}, {"layer": 41, "field": "recurrent", "exact": false, "max_abs": 0.004861496388912201, "nrmse": 0.006379321217536926}, {"layer": 42, "field": "conv", "exact": false, "max_abs": 0.1279296875, "nrmse": 0.020456548780202866}, {"layer": 42, "field": "recurrent", "exact": false, "max_abs": 0.005590997636318207, "nrmse": 0.006548497825860977}, {"layer": 43, "field": "key", "exact": false, "max_abs": 0.16015625, "nrmse": 0.0027453412767499685}, {"layer": 43, "field": "value", "exact": false, "max_abs": 0.125, "nrmse": 0.0029851042199879885}, {"layer": 44, "field": "conv", "exact": false, "max_abs": 0.1484375, "nrmse": 0.020381202921271324}, {"layer": 44, "field": "recurrent", "exact": false, "max_abs": 0.002939365804195404, "nrmse": 0.005796715617179871}, {"layer": 45, "field": "conv", "exact": false, "max_abs": 0.125, "nrmse": 0.023555103689432144}, {"layer": 45, "field": "recurrent", "exact": false, "max_abs": 0.0065735578536987305, "nrmse": 0.010396377183496952}, {"layer": 46, "field": "conv", "exact": false, "max_abs": 0.15625, "nrmse": 0.02425544522702694}, {"layer": 46, "field": "recurrent", "exact": false, "max_abs": 0.008649080991744995, "nrmse": 0.01276238914579153}, {"layer": 47, "field": "key", "exact": false, "max_abs": 0.25, "nrmse": 0.004348364658653736}, {"layer": 47, "field": "value", "exact": false, "max_abs": 0.1875, "nrmse": 0.0054127490147948265}, {"layer": 48, "field": "conv", "exact": false, "max_abs": 0.201171875, "nrmse": 0.02585102990269661}, {"layer": 48, "field": "recurrent", "exact": false, "max_abs": 0.003755968064069748, "nrmse": 0.0098269609734416}, {"layer": 49, "field": "conv", "exact": false, "max_abs": 0.224609375, "nrmse": 0.026762938126921654}, {"layer": 49, "field": "recurrent", "exact": false, "max_abs": 0.006457030773162842, "nrmse": 0.017325809225440025}, {"layer": 50, "field": "conv", "exact": false, "max_abs": 0.2421875, "nrmse": 0.027950040996074677}, {"layer": 50, "field": "recurrent", "exact": false, "max_abs": 0.00575004518032074, "nrmse": 0.015344783663749695}, {"layer": 51, "field": "key", "exact": false, "max_abs": 0.23779296875, "nrmse": 0.005098164547234774}, {"layer": 51, "field": "value", "exact": false, "max_abs": 0.3125, "nrmse": 0.006940989755094051}, {"layer": 52, "field": "conv", "exact": false, "max_abs": 0.24609375, "nrmse": 0.023939384147524834}, {"layer": 52, "field": "recurrent", "exact": false, "max_abs": 0.022550731897354126, "nrmse": 0.009624708443880081}, {"layer": 53, "field": "conv", "exact": false, "max_abs": 0.3046875, "nrmse": 0.02523924969136715}, {"layer": 53, "field": "recurrent", "exact": false, "max_abs": 0.017098404467105865, "nrmse": 0.013642564415931702}, {"layer": 54, "field": "conv", "exact": false, "max_abs": 0.296875, "nrmse": 0.028200607746839523}, {"layer": 54, "field": "recurrent", "exact": false, "max_abs": 0.014403298497200012, "nrmse": 0.018752742558717728}, {"layer": 55, "field": "key", "exact": false, "max_abs": 0.203125, "nrmse": 0.005265590734779835}, {"layer": 55, "field": "value", "exact": false, "max_abs": 0.287109375, "nrmse": 0.007306888233870268}, {"layer": 56, "field": "conv", "exact": false, "max_abs": 0.265625, "nrmse": 0.024360118433833122}, {"layer": 56, "field": "recurrent", "exact": false, "max_abs": 0.017659828066825867, "nrmse": 0.013118765316903591}, {"layer": 57, "field": "conv", "exact": false, "max_abs": 0.34375, "nrmse": 0.022760048508644104}, {"layer": 57, "field": "recurrent", "exact": false, "max_abs": 0.024019986391067505, "nrmse": 0.017076721414923668}, {"layer": 58, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.02542339637875557}, {"layer": 58, "field": "recurrent", "exact": false, "max_abs": 0.036575257778167725, "nrmse": 0.01509578712284565}, {"layer": 59, "field": "key", "exact": false, "max_abs": 0.1796875, "nrmse": 0.005474989302456379}, {"layer": 59, "field": "value", "exact": false, "max_abs": 0.525390625, "nrmse": 0.005909747444093227}, {"layer": 60, "field": "conv", "exact": false, "max_abs": 0.75, "nrmse": 0.021330608054995537}, {"layer": 60, "field": "recurrent", "exact": false, "max_abs": 0.01888003945350647, "nrmse": 0.011380147188901901}, {"layer": 61, "field": "conv", "exact": false, "max_abs": 0.25, "nrmse": 0.024206696078181267}, {"layer": 61, "field": "recurrent", "exact": false, "max_abs": 0.017560288310050964, "nrmse": 0.014719594269990921}, {"layer": 62, "field": "conv", "exact": false, "max_abs": 0.53125, "nrmse": 0.02113671414554119}, {"layer": 62, "field": "recurrent", "exact": false, "max_abs": 0.015967726707458496, "nrmse": 0.007758997846394777}, {"layer": 63, "field": "key", "exact": false, "max_abs": 0.2109375, "nrmse": 0.006061076652258635}, {"layer": 63, "field": "value", "exact": false, "max_abs": 0.5, "nrmse": 0.0032708521466702223}], "buffered_graph_state_diff": []} -{"phase": "step", "index": 31, "position": 179, "teacher_token": 271, "compact": {"ids": [332, 16, 9764, 5170, 8046], "logits": [29.5, 25.375, 20.5, 18.625, 18.0], "margin": 4.125, "argmax": 332}, "buffered": {"ids": [332, 16, 9764, 5170, 8046], "logits": [29.25, 25.0, 20.375, 18.625, 17.875], "margin": 4.25, "argmax": 332}, "graph": {"ids": [332, 16, 9764, 5170, 8046], "logits": [29.25, 25.0, 20.375, 18.625, 17.875], "margin": 4.25, "argmax": 332}, "compact_buffered": {"exact": false, "max_abs": 0.625732421875, "nrmse": 0.05110752210021019}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "compact_buffered_state_diff": [{"layer": 36, "field": "recurrent", "exact": false, "max_abs": 5.960464477539062e-07, "nrmse": 1.4493147659777605e-07}, {"layer": 37, "field": "conv", "exact": false, "max_abs": 0.0625, "nrmse": 0.003086460055783391}, {"layer": 37, "field": "recurrent", "exact": false, "max_abs": 0.0007747113704681396, "nrmse": 0.0007464935770258307}, {"layer": 38, "field": "conv", "exact": false, "max_abs": 0.16015625, "nrmse": 0.017723754048347473}, {"layer": 38, "field": "recurrent", "exact": false, "max_abs": 0.007885664701461792, "nrmse": 0.013256145641207695}, {"layer": 39, "field": "key", "exact": false, "max_abs": 0.1875, "nrmse": 0.003298182273283601}, {"layer": 39, "field": "value", "exact": false, "max_abs": 0.21875, "nrmse": 0.004303768742829561}, {"layer": 40, "field": "conv", "exact": false, "max_abs": 0.25, "nrmse": 0.0264253169298172}, {"layer": 40, "field": "recurrent", "exact": false, "max_abs": 0.008935511112213135, "nrmse": 0.005646335426717997}, {"layer": 41, "field": "conv", "exact": false, "max_abs": 0.26953125, "nrmse": 0.029883122071623802}, {"layer": 41, "field": "recurrent", "exact": false, "max_abs": 0.006231071427464485, "nrmse": 0.011680535972118378}, {"layer": 42, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.0339195691049099}, {"layer": 42, "field": "recurrent", "exact": false, "max_abs": 0.019006669521331787, "nrmse": 0.016813796013593674}, {"layer": 43, "field": "key", "exact": false, "max_abs": 0.3515625, "nrmse": 0.004808841738849878}, {"layer": 43, "field": "value", "exact": false, "max_abs": 0.31640625, "nrmse": 0.0055929445661604404}, {"layer": 44, "field": "conv", "exact": false, "max_abs": 0.32421875, "nrmse": 0.036790791898965836}, {"layer": 44, "field": "recurrent", "exact": false, "max_abs": 0.010948538780212402, "nrmse": 0.013402611017227173}, {"layer": 45, "field": "conv", "exact": false, "max_abs": 0.390625, "nrmse": 0.0418575182557106}, {"layer": 45, "field": "recurrent", "exact": false, "max_abs": 0.019751042127609253, "nrmse": 0.022487713024020195}, {"layer": 46, "field": "conv", "exact": false, "max_abs": 0.3828125, "nrmse": 0.04051075875759125}, {"layer": 46, "field": "recurrent", "exact": false, "max_abs": 0.016344696283340454, "nrmse": 0.02226710133254528}, {"layer": 47, "field": "key", "exact": false, "max_abs": 0.55078125, "nrmse": 0.00680410023778677}, {"layer": 47, "field": "value", "exact": false, "max_abs": 0.33203125, "nrmse": 0.007920870557427406}, {"layer": 48, "field": "conv", "exact": false, "max_abs": 0.4296875, "nrmse": 0.041673749685287476}, {"layer": 48, "field": "recurrent", "exact": false, "max_abs": 0.017584241926670074, "nrmse": 0.021025564521551132}, {"layer": 49, "field": "conv", "exact": false, "max_abs": 0.4560546875, "nrmse": 0.042722560465335846}, {"layer": 49, "field": "recurrent", "exact": false, "max_abs": 0.0195886492729187, "nrmse": 0.0305391363799572}, {"layer": 50, "field": "conv", "exact": false, "max_abs": 0.3701171875, "nrmse": 0.03999033942818642}, {"layer": 50, "field": "recurrent", "exact": false, "max_abs": 0.014995735138654709, "nrmse": 0.02807954140007496}, {"layer": 51, "field": "key", "exact": false, "max_abs": 0.32421875, "nrmse": 0.006342010106891394}, {"layer": 51, "field": "value", "exact": false, "max_abs": 0.4921875, "nrmse": 0.00851869210600853}, {"layer": 52, "field": "conv", "exact": false, "max_abs": 0.625, "nrmse": 0.03352968394756317}, {"layer": 52, "field": "recurrent", "exact": false, "max_abs": 0.03178302198648453, "nrmse": 0.017533643171191216}, {"layer": 53, "field": "conv", "exact": false, "max_abs": 0.515625, "nrmse": 0.03576846420764923}, {"layer": 53, "field": "recurrent", "exact": false, "max_abs": 0.03178834915161133, "nrmse": 0.02573339454829693}, {"layer": 54, "field": "conv", "exact": false, "max_abs": 0.46875, "nrmse": 0.03992859274148941}, {"layer": 54, "field": "recurrent", "exact": false, "max_abs": 0.02106001228094101, "nrmse": 0.02968214638531208}, {"layer": 55, "field": "key", "exact": false, "max_abs": 0.236328125, "nrmse": 0.006255665328353643}, {"layer": 55, "field": "value", "exact": false, "max_abs": 0.6962890625, "nrmse": 0.00939540658146143}, {"layer": 56, "field": "conv", "exact": false, "max_abs": 0.4609375, "nrmse": 0.03628179803490639}, {"layer": 56, "field": "recurrent", "exact": false, "max_abs": 0.026403896510601044, "nrmse": 0.023533152416348457}, {"layer": 57, "field": "conv", "exact": false, "max_abs": 0.515625, "nrmse": 0.03383828327059746}, {"layer": 57, "field": "recurrent", "exact": false, "max_abs": 0.02957908809185028, "nrmse": 0.028165588155388832}, {"layer": 58, "field": "conv", "exact": false, "max_abs": 0.52294921875, "nrmse": 0.037602610886096954}, {"layer": 58, "field": "recurrent", "exact": false, "max_abs": 0.036099597811698914, "nrmse": 0.02422865480184555}, {"layer": 59, "field": "key", "exact": false, "max_abs": 0.2284088134765625, "nrmse": 0.006234961096197367}, {"layer": 59, "field": "value", "exact": false, "max_abs": 0.79296875, "nrmse": 0.007332841400057077}, {"layer": 60, "field": "conv", "exact": false, "max_abs": 0.7607421875, "nrmse": 0.032043371349573135}, {"layer": 60, "field": "recurrent", "exact": false, "max_abs": 0.03432118892669678, "nrmse": 0.021770112216472626}, {"layer": 61, "field": "conv", "exact": false, "max_abs": 0.49609375, "nrmse": 0.03600942716002464}, {"layer": 61, "field": "recurrent", "exact": false, "max_abs": 0.03417626768350601, "nrmse": 0.027859240770339966}, {"layer": 62, "field": "conv", "exact": false, "max_abs": 0.9375, "nrmse": 0.03060036152601242}, {"layer": 62, "field": "recurrent", "exact": false, "max_abs": 0.02531254291534424, "nrmse": 0.015782751142978668}, {"layer": 63, "field": "key", "exact": false, "max_abs": 0.24609375, "nrmse": 0.0069368877448141575}, {"layer": 63, "field": "value", "exact": false, "max_abs": 0.84375, "nrmse": 0.00400109076872468}], "buffered_graph_state_diff": []} -{"phase": "step", "index": 32, "position": 180, "teacher_token": 332, "compact": {"ids": [16, 8046, 6140, 21365, 14392], "logits": [32.75, 31.75, 21.375, 18.875, 18.5], "margin": 1.0, "argmax": 16}, "buffered": {"ids": [16, 8046, 6140, 21365, 14392], "logits": [32.75, 31.5, 21.0, 18.5, 18.375], "margin": 1.25, "argmax": 16}, "graph": {"ids": [16, 8046, 6140, 21365, 14392], "logits": [32.75, 31.5, 21.0, 18.5, 18.375], "margin": 1.25, "argmax": 16}, "compact_buffered": {"exact": false, "max_abs": 0.75, "nrmse": 0.06362005323171616}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "compact_buffered_state_diff": [{"layer": 36, "field": "recurrent", "exact": false, "max_abs": 5.960464477539062e-07, "nrmse": 1.4276984927619196e-07}, {"layer": 37, "field": "recurrent", "exact": false, "max_abs": 0.0006485693156719208, "nrmse": 0.0006434605456888676}, {"layer": 38, "field": "conv", "exact": false, "max_abs": 0.16015625, "nrmse": 0.016960110515356064}, {"layer": 38, "field": "recurrent", "exact": false, "max_abs": 0.015389561653137207, "nrmse": 0.012803838588297367}, {"layer": 39, "field": "key", "exact": false, "max_abs": 0.1875, "nrmse": 0.003593417815864086}, {"layer": 39, "field": "value", "exact": false, "max_abs": 0.21875, "nrmse": 0.0046581667847931385}, {"layer": 40, "field": "conv", "exact": false, "max_abs": 0.25, "nrmse": 0.026426605880260468}, {"layer": 40, "field": "recurrent", "exact": false, "max_abs": 0.00897284597158432, "nrmse": 0.007598417345434427}, {"layer": 41, "field": "conv", "exact": false, "max_abs": 0.26953125, "nrmse": 0.030277103185653687}, {"layer": 41, "field": "recurrent", "exact": false, "max_abs": 0.007326975464820862, "nrmse": 0.015400713309645653}, {"layer": 42, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.03436211496591568}, {"layer": 42, "field": "recurrent", "exact": false, "max_abs": 0.009532123804092407, "nrmse": 0.017832642421126366}, {"layer": 43, "field": "key", "exact": false, "max_abs": 0.3515625, "nrmse": 0.005125741008669138}, {"layer": 43, "field": "value", "exact": false, "max_abs": 0.31640625, "nrmse": 0.006021962966769934}, {"layer": 44, "field": "conv", "exact": false, "max_abs": 0.32421875, "nrmse": 0.03675293177366257}, {"layer": 44, "field": "recurrent", "exact": false, "max_abs": 0.012836121022701263, "nrmse": 0.02218480408191681}, {"layer": 45, "field": "conv", "exact": false, "max_abs": 0.390625, "nrmse": 0.04161406680941582}, {"layer": 45, "field": "recurrent", "exact": false, "max_abs": 0.014271482825279236, "nrmse": 0.030138716101646423}, {"layer": 46, "field": "conv", "exact": false, "max_abs": 0.3828125, "nrmse": 0.03987395763397217}, {"layer": 46, "field": "recurrent", "exact": false, "max_abs": 0.021270722150802612, "nrmse": 0.03052538074553013}, {"layer": 47, "field": "key", "exact": false, "max_abs": 0.55078125, "nrmse": 0.007087238132953644}, {"layer": 47, "field": "value", "exact": false, "max_abs": 0.33203125, "nrmse": 0.00840931199491024}, {"layer": 48, "field": "conv", "exact": false, "max_abs": 0.4296875, "nrmse": 0.04170951247215271}, {"layer": 48, "field": "recurrent", "exact": false, "max_abs": 0.012110225856304169, "nrmse": 0.02800837904214859}, {"layer": 49, "field": "conv", "exact": false, "max_abs": 0.4560546875, "nrmse": 0.0431620329618454}, {"layer": 49, "field": "recurrent", "exact": false, "max_abs": 0.026474297046661377, "nrmse": 0.03496020659804344}, {"layer": 50, "field": "conv", "exact": false, "max_abs": 0.3701171875, "nrmse": 0.039874546229839325}, {"layer": 50, "field": "recurrent", "exact": false, "max_abs": 0.011813990771770477, "nrmse": 0.02994166687130928}, {"layer": 51, "field": "key", "exact": false, "max_abs": 0.32421875, "nrmse": 0.006669850088655949}, {"layer": 51, "field": "value", "exact": false, "max_abs": 0.4921875, "nrmse": 0.00896296463906765}, {"layer": 52, "field": "conv", "exact": false, "max_abs": 0.625, "nrmse": 0.034522198140621185}, {"layer": 52, "field": "recurrent", "exact": false, "max_abs": 0.0289895161986351, "nrmse": 0.020052289590239525}, {"layer": 53, "field": "conv", "exact": false, "max_abs": 0.515625, "nrmse": 0.03731080889701843}, {"layer": 53, "field": "recurrent", "exact": false, "max_abs": 0.04228025674819946, "nrmse": 0.03074413537979126}, {"layer": 54, "field": "conv", "exact": false, "max_abs": 0.46875, "nrmse": 0.04285800829529762}, {"layer": 54, "field": "recurrent", "exact": false, "max_abs": 0.02343234419822693, "nrmse": 0.03591478243470192}, {"layer": 55, "field": "key", "exact": false, "max_abs": 0.236328125, "nrmse": 0.006643708795309067}, {"layer": 55, "field": "value", "exact": false, "max_abs": 0.6962890625, "nrmse": 0.010017763823270798}, {"layer": 56, "field": "conv", "exact": false, "max_abs": 0.4609375, "nrmse": 0.03901185840368271}, {"layer": 56, "field": "recurrent", "exact": false, "max_abs": 0.020014971494674683, "nrmse": 0.025053400546312332}, {"layer": 57, "field": "conv", "exact": false, "max_abs": 0.515625, "nrmse": 0.03727808594703674}, {"layer": 57, "field": "recurrent", "exact": false, "max_abs": 0.027817413210868835, "nrmse": 0.029883461073040962}, {"layer": 58, "field": "conv", "exact": false, "max_abs": 0.52294921875, "nrmse": 0.04052743688225746}, {"layer": 58, "field": "recurrent", "exact": false, "max_abs": 0.027088791131973267, "nrmse": 0.02504885010421276}, {"layer": 59, "field": "key", "exact": false, "max_abs": 0.2284088134765625, "nrmse": 0.006742994301021099}, {"layer": 59, "field": "value", "exact": false, "max_abs": 0.79296875, "nrmse": 0.007909189909696579}, {"layer": 60, "field": "conv", "exact": false, "max_abs": 0.7607421875, "nrmse": 0.03411811217665672}, {"layer": 60, "field": "recurrent", "exact": false, "max_abs": 0.037253886461257935, "nrmse": 0.024820350110530853}, {"layer": 61, "field": "conv", "exact": false, "max_abs": 0.49609375, "nrmse": 0.03793107718229294}, {"layer": 61, "field": "recurrent", "exact": false, "max_abs": 0.01974216103553772, "nrmse": 0.029575437307357788}, {"layer": 62, "field": "conv", "exact": false, "max_abs": 0.9375, "nrmse": 0.03233107179403305}, {"layer": 62, "field": "recurrent", "exact": false, "max_abs": 0.026410799473524094, "nrmse": 0.018765637651085854}, {"layer": 63, "field": "key", "exact": false, "max_abs": 0.24609375, "nrmse": 0.007304164581000805}, {"layer": 63, "field": "value", "exact": false, "max_abs": 0.84375, "nrmse": 0.004303631838411093}], "buffered_graph_state_diff": []} -{"phase": "step", "index": 33, "position": 181, "teacher_token": 16, "compact": {"ids": [13, 36189, 62366, 25, 28924], "logits": [27.0, 13.875, 13.5625, 13.25, 12.875], "margin": 13.125, "argmax": 13}, "buffered": {"ids": [13, 36189, 62366, 25, 28924], "logits": [27.0, 13.6875, 13.5, 13.125, 12.6875], "margin": 13.3125, "argmax": 13}, "graph": {"ids": [13, 36189, 62366, 25, 28924], "logits": [27.0, 13.6875, 13.5, 13.125, 12.6875], "margin": 13.3125, "argmax": 13}, "compact_buffered": {"exact": false, "max_abs": 0.5390625, "nrmse": 0.015391683205962181}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "compact_buffered_state_diff": [{"layer": 36, "field": "recurrent", "exact": false, "max_abs": 4.76837158203125e-07, "nrmse": 1.3801518150557968e-07}, {"layer": 37, "field": "recurrent", "exact": false, "max_abs": 0.0006206147372722626, "nrmse": 0.0006052535027265549}, {"layer": 38, "field": "conv", "exact": false, "max_abs": 0.16015625, "nrmse": 0.016151810064911842}, {"layer": 38, "field": "recurrent", "exact": false, "max_abs": 0.013136208057403564, "nrmse": 0.010954787023365498}, {"layer": 39, "field": "key", "exact": false, "max_abs": 0.1875, "nrmse": 0.0038457238115370274}, {"layer": 39, "field": "value", "exact": false, "max_abs": 0.21875, "nrmse": 0.004908365197479725}, {"layer": 40, "field": "conv", "exact": false, "max_abs": 0.25, "nrmse": 0.02638959139585495}, {"layer": 40, "field": "recurrent", "exact": false, "max_abs": 0.007813774980604649, "nrmse": 0.006794786546379328}, {"layer": 41, "field": "conv", "exact": false, "max_abs": 0.26953125, "nrmse": 0.030037585645914078}, {"layer": 41, "field": "recurrent", "exact": false, "max_abs": 0.007454127073287964, "nrmse": 0.014777620323002338}, {"layer": 42, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.03377934545278549}, {"layer": 42, "field": "recurrent", "exact": false, "max_abs": 0.009474582970142365, "nrmse": 0.01656625047326088}, {"layer": 43, "field": "key", "exact": false, "max_abs": 0.3515625, "nrmse": 0.00532806059345603}, {"layer": 43, "field": "value", "exact": false, "max_abs": 0.31640625, "nrmse": 0.006263213697820902}, {"layer": 44, "field": "conv", "exact": false, "max_abs": 0.32421875, "nrmse": 0.036220431327819824}, {"layer": 44, "field": "recurrent", "exact": false, "max_abs": 0.013155847787857056, "nrmse": 0.021726073697209358}, {"layer": 45, "field": "conv", "exact": false, "max_abs": 0.390625, "nrmse": 0.040986787527799606}, {"layer": 45, "field": "recurrent", "exact": false, "max_abs": 0.012921571731567383, "nrmse": 0.02663079835474491}, {"layer": 46, "field": "conv", "exact": false, "max_abs": 0.3828125, "nrmse": 0.03885717689990997}, {"layer": 46, "field": "recurrent", "exact": false, "max_abs": 0.016282200813293457, "nrmse": 0.02610710822045803}, {"layer": 47, "field": "key", "exact": false, "max_abs": 0.55078125, "nrmse": 0.007281593978404999}, {"layer": 47, "field": "value", "exact": false, "max_abs": 0.33203125, "nrmse": 0.008701952174305916}, {"layer": 48, "field": "conv", "exact": false, "max_abs": 0.4296875, "nrmse": 0.04124858230352402}, {"layer": 48, "field": "recurrent", "exact": false, "max_abs": 0.012051083147525787, "nrmse": 0.027079038321971893}, {"layer": 49, "field": "conv", "exact": false, "max_abs": 0.4560546875, "nrmse": 0.042715009301900864}, {"layer": 49, "field": "recurrent", "exact": false, "max_abs": 0.015579640865325928, "nrmse": 0.031771138310432434}, {"layer": 50, "field": "conv", "exact": false, "max_abs": 0.3701171875, "nrmse": 0.0393376499414444}, {"layer": 50, "field": "recurrent", "exact": false, "max_abs": 0.013967156410217285, "nrmse": 0.02850847877562046}, {"layer": 51, "field": "key", "exact": false, "max_abs": 0.32421875, "nrmse": 0.00685054762288928}, {"layer": 51, "field": "value", "exact": false, "max_abs": 0.4921875, "nrmse": 0.009327782317996025}, {"layer": 52, "field": "conv", "exact": false, "max_abs": 0.625, "nrmse": 0.03546183183789253}, {"layer": 52, "field": "recurrent", "exact": false, "max_abs": 0.029098626226186752, "nrmse": 0.019828742370009422}, {"layer": 53, "field": "conv", "exact": false, "max_abs": 0.515625, "nrmse": 0.03850090503692627}, {"layer": 53, "field": "recurrent", "exact": false, "max_abs": 0.042403459548950195, "nrmse": 0.029560726135969162}, {"layer": 54, "field": "conv", "exact": false, "max_abs": 0.46875, "nrmse": 0.04386157914996147}, {"layer": 54, "field": "recurrent", "exact": false, "max_abs": 0.022021636366844177, "nrmse": 0.03436800837516785}, {"layer": 55, "field": "key", "exact": false, "max_abs": 0.236328125, "nrmse": 0.007051833905279636}, {"layer": 55, "field": "value", "exact": false, "max_abs": 0.6962890625, "nrmse": 0.01053436379879713}, {"layer": 56, "field": "conv", "exact": false, "max_abs": 0.4609375, "nrmse": 0.04037889093160629}, {"layer": 56, "field": "recurrent", "exact": false, "max_abs": 0.0374140739440918, "nrmse": 0.02456822246313095}, {"layer": 57, "field": "conv", "exact": false, "max_abs": 0.515625, "nrmse": 0.03930385410785675}, {"layer": 57, "field": "recurrent", "exact": false, "max_abs": 0.027750462293624878, "nrmse": 0.02871423028409481}, {"layer": 58, "field": "conv", "exact": false, "max_abs": 0.52294921875, "nrmse": 0.042380087077617645}, {"layer": 58, "field": "recurrent", "exact": false, "max_abs": 0.02685609459877014, "nrmse": 0.025070425122976303}, {"layer": 59, "field": "key", "exact": false, "max_abs": 0.2284088134765625, "nrmse": 0.007089788559824228}, {"layer": 59, "field": "value", "exact": false, "max_abs": 0.79296875, "nrmse": 0.008293412625789642}, {"layer": 60, "field": "conv", "exact": false, "max_abs": 0.7607421875, "nrmse": 0.03453662246465683}, {"layer": 60, "field": "recurrent", "exact": false, "max_abs": 0.040150582790374756, "nrmse": 0.024078870192170143}, {"layer": 61, "field": "conv", "exact": false, "max_abs": 0.49609375, "nrmse": 0.03910624980926514}, {"layer": 61, "field": "recurrent", "exact": false, "max_abs": 0.018094927072525024, "nrmse": 0.02842883951961994}, {"layer": 62, "field": "conv", "exact": false, "max_abs": 0.9375, "nrmse": 0.03315363824367523}, {"layer": 62, "field": "recurrent", "exact": false, "max_abs": 0.02692890539765358, "nrmse": 0.01820455677807331}, {"layer": 63, "field": "key", "exact": false, "max_abs": 0.24609375, "nrmse": 0.007573901675641537}, {"layer": 63, "field": "value", "exact": false, "max_abs": 0.84375, "nrmse": 0.004556052852421999}], "buffered_graph_state_diff": []} -{"phase": "step", "index": 34, "position": 182, "teacher_token": 13, "compact": {"ids": [36189, 28924, 62366, 18053, 3991], "logits": [27.875, 27.0, 26.75, 26.0, 25.5], "margin": 0.875, "argmax": 36189}, "buffered": {"ids": [36189, 28924, 62366, 18053, 3991], "logits": [27.75, 26.75, 26.5, 25.875, 25.25], "margin": 1.0, "argmax": 36189}, "graph": {"ids": [36189, 28924, 62366, 18053, 3991], "logits": [27.75, 26.75, 26.5, 25.875, 25.25], "margin": 1.0, "argmax": 36189}, "compact_buffered": {"exact": false, "max_abs": 0.4375, "nrmse": 0.02326270565390587}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "compact_buffered_state_diff": [{"layer": 36, "field": "recurrent", "exact": false, "max_abs": 4.76837158203125e-07, "nrmse": 1.345304525557367e-07}, {"layer": 37, "field": "recurrent", "exact": false, "max_abs": 0.0006210580468177795, "nrmse": 0.0005864796694368124}, {"layer": 38, "field": "conv", "exact": false, "max_abs": 0.16015625, "nrmse": 0.0162355974316597}, {"layer": 38, "field": "recurrent", "exact": false, "max_abs": 0.00622183084487915, "nrmse": 0.010066945105791092}, {"layer": 39, "field": "key", "exact": false, "max_abs": 0.1875, "nrmse": 0.004014794249087572}, {"layer": 39, "field": "value", "exact": false, "max_abs": 0.21875, "nrmse": 0.005135311279445887}, {"layer": 40, "field": "conv", "exact": false, "max_abs": 0.25, "nrmse": 0.027538876980543137}, {"layer": 40, "field": "recurrent", "exact": false, "max_abs": 0.00745844841003418, "nrmse": 0.0069232843816280365}, {"layer": 41, "field": "conv", "exact": false, "max_abs": 0.26953125, "nrmse": 0.031307708472013474}, {"layer": 41, "field": "recurrent", "exact": false, "max_abs": 0.0071281567215919495, "nrmse": 0.014718578197062016}, {"layer": 42, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.035296082496643066}, {"layer": 42, "field": "recurrent", "exact": false, "max_abs": 0.00760236382484436, "nrmse": 0.015520557761192322}, {"layer": 43, "field": "key", "exact": false, "max_abs": 0.3515625, "nrmse": 0.005746412090957165}, {"layer": 43, "field": "value", "exact": false, "max_abs": 0.31640625, "nrmse": 0.006603763438761234}, {"layer": 44, "field": "conv", "exact": false, "max_abs": 0.32421875, "nrmse": 0.03847923129796982}, {"layer": 44, "field": "recurrent", "exact": false, "max_abs": 0.013162165880203247, "nrmse": 0.021782662719488144}, {"layer": 45, "field": "conv", "exact": false, "max_abs": 0.390625, "nrmse": 0.04318029060959816}, {"layer": 45, "field": "recurrent", "exact": false, "max_abs": 0.012925103306770325, "nrmse": 0.025666167959570885}, {"layer": 46, "field": "conv", "exact": false, "max_abs": 0.3828125, "nrmse": 0.04122593253850937}, {"layer": 46, "field": "recurrent", "exact": false, "max_abs": 0.015983223915100098, "nrmse": 0.024648085236549377}, {"layer": 47, "field": "key", "exact": false, "max_abs": 0.55078125, "nrmse": 0.007721875794231892}, {"layer": 47, "field": "value", "exact": false, "max_abs": 0.33203125, "nrmse": 0.009116355329751968}, {"layer": 48, "field": "conv", "exact": false, "max_abs": 0.4296875, "nrmse": 0.044225458055734634}, {"layer": 48, "field": "recurrent", "exact": false, "max_abs": 0.011796645820140839, "nrmse": 0.026118479669094086}, {"layer": 49, "field": "conv", "exact": false, "max_abs": 0.4560546875, "nrmse": 0.04515489190816879}, {"layer": 49, "field": "recurrent", "exact": false, "max_abs": 0.013860702514648438, "nrmse": 0.030194882303476334}, {"layer": 50, "field": "conv", "exact": false, "max_abs": 0.3701171875, "nrmse": 0.04137164354324341}, {"layer": 50, "field": "recurrent", "exact": false, "max_abs": 0.0077513158321380615, "nrmse": 0.028149012476205826}, {"layer": 51, "field": "key", "exact": false, "max_abs": 0.32421875, "nrmse": 0.007257627788931131}, {"layer": 51, "field": "value", "exact": false, "max_abs": 0.4921875, "nrmse": 0.009635534137487411}, {"layer": 52, "field": "conv", "exact": false, "max_abs": 0.625, "nrmse": 0.03584799915552139}, {"layer": 52, "field": "recurrent", "exact": false, "max_abs": 0.03273731470108032, "nrmse": 0.020470499992370605}, {"layer": 53, "field": "conv", "exact": false, "max_abs": 0.515625, "nrmse": 0.038515761494636536}, {"layer": 53, "field": "recurrent", "exact": false, "max_abs": 0.04153689742088318, "nrmse": 0.02921690046787262}, {"layer": 54, "field": "conv", "exact": false, "max_abs": 0.46875, "nrmse": 0.04379388689994812}, {"layer": 54, "field": "recurrent", "exact": false, "max_abs": 0.021568529307842255, "nrmse": 0.03326877951622009}, {"layer": 55, "field": "key", "exact": false, "max_abs": 0.236328125, "nrmse": 0.007274681236594915}, {"layer": 55, "field": "value", "exact": false, "max_abs": 0.6962890625, "nrmse": 0.010807917453348637}, {"layer": 56, "field": "conv", "exact": false, "max_abs": 0.4609375, "nrmse": 0.040650662034749985}, {"layer": 56, "field": "recurrent", "exact": false, "max_abs": 0.03663438558578491, "nrmse": 0.02438308298587799}, {"layer": 57, "field": "conv", "exact": false, "max_abs": 0.515625, "nrmse": 0.038386475294828415}, {"layer": 57, "field": "recurrent", "exact": false, "max_abs": 0.027377814054489136, "nrmse": 0.026332568377256393}, {"layer": 58, "field": "conv", "exact": false, "max_abs": 0.52294921875, "nrmse": 0.04195566847920418}, {"layer": 58, "field": "recurrent", "exact": false, "max_abs": 0.025882378220558167, "nrmse": 0.023627085611224174}, {"layer": 59, "field": "key", "exact": false, "max_abs": 0.2284088134765625, "nrmse": 0.007355138193815947}, {"layer": 59, "field": "value", "exact": false, "max_abs": 0.79296875, "nrmse": 0.008574452251195908}, {"layer": 60, "field": "conv", "exact": false, "max_abs": 0.8125, "nrmse": 0.0333540216088295}, {"layer": 60, "field": "recurrent", "exact": false, "max_abs": 0.04357641935348511, "nrmse": 0.02402353100478649}, {"layer": 61, "field": "conv", "exact": false, "max_abs": 0.49609375, "nrmse": 0.03828210383653641}, {"layer": 61, "field": "recurrent", "exact": false, "max_abs": 0.01813378743827343, "nrmse": 0.027710789814591408}, {"layer": 62, "field": "conv", "exact": false, "max_abs": 0.9375, "nrmse": 0.03276621922850609}, {"layer": 62, "field": "recurrent", "exact": false, "max_abs": 0.04687884449958801, "nrmse": 0.018544351682066917}, {"layer": 63, "field": "key", "exact": false, "max_abs": 0.25, "nrmse": 0.007879354991018772}, {"layer": 63, "field": "value", "exact": false, "max_abs": 0.84375, "nrmse": 0.004751788917928934}], "buffered_graph_state_diff": []} -{"phase": "step", "index": 35, "position": 183, "teacher_token": 36189, "compact": {"ids": [2891, 537, 279, 89, 19752], "logits": [25.875, 16.25, 14.6875, 14.5625, 13.5625], "margin": 9.625, "argmax": 2891}, "buffered": {"ids": [2891, 537, 89, 279, 19752], "logits": [25.875, 16.25, 14.5625, 14.4375, 13.625], "margin": 9.625, "argmax": 2891}, "graph": {"ids": [2891, 537, 89, 279, 19752], "logits": [25.875, 16.25, 14.5625, 14.4375, 13.625], "margin": 9.625, "argmax": 2891}, "compact_buffered": {"exact": false, "max_abs": 0.375, "nrmse": 0.017478033900260925}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "compact_buffered_state_diff": [{"layer": 36, "field": "recurrent", "exact": false, "max_abs": 4.76837158203125e-07, "nrmse": 1.3384192243393045e-07}, {"layer": 37, "field": "recurrent", "exact": false, "max_abs": 0.0006080903112888336, "nrmse": 0.0005664105992764235}, {"layer": 38, "field": "conv", "exact": false, "max_abs": 0.09375, "nrmse": 0.011426367796957493}, {"layer": 38, "field": "recurrent", "exact": false, "max_abs": 0.006250649690628052, "nrmse": 0.009731589816510677}, {"layer": 39, "field": "key", "exact": false, "max_abs": 0.1875, "nrmse": 0.004188989754766226}, {"layer": 39, "field": "value", "exact": false, "max_abs": 0.21875, "nrmse": 0.005342117045074701}, {"layer": 40, "field": "conv", "exact": false, "max_abs": 0.125, "nrmse": 0.020277412608265877}, {"layer": 40, "field": "recurrent", "exact": false, "max_abs": 0.007194876670837402, "nrmse": 0.00675444770604372}, {"layer": 41, "field": "conv", "exact": false, "max_abs": 0.16796875, "nrmse": 0.023929059505462646}, {"layer": 41, "field": "recurrent", "exact": false, "max_abs": 0.007106348872184753, "nrmse": 0.014941244386136532}, {"layer": 42, "field": "conv", "exact": false, "max_abs": 0.1484375, "nrmse": 0.02720390073955059}, {"layer": 42, "field": "recurrent", "exact": false, "max_abs": 0.007697053253650665, "nrmse": 0.016095925122499466}, {"layer": 43, "field": "key", "exact": false, "max_abs": 0.3515625, "nrmse": 0.00594598613679409}, {"layer": 43, "field": "value", "exact": false, "max_abs": 0.31640625, "nrmse": 0.00680775148794055}, {"layer": 44, "field": "conv", "exact": false, "max_abs": 0.1875, "nrmse": 0.02686259336769581}, {"layer": 44, "field": "recurrent", "exact": false, "max_abs": 0.013092942535877228, "nrmse": 0.021905377507209778}, {"layer": 45, "field": "conv", "exact": false, "max_abs": 0.1904296875, "nrmse": 0.030553705990314484}, {"layer": 45, "field": "recurrent", "exact": false, "max_abs": 0.012883245944976807, "nrmse": 0.02566552348434925}, {"layer": 46, "field": "conv", "exact": false, "max_abs": 0.1875, "nrmse": 0.029744315892457962}, {"layer": 46, "field": "recurrent", "exact": false, "max_abs": 0.01612633466720581, "nrmse": 0.024890240281820297}, {"layer": 47, "field": "key", "exact": false, "max_abs": 0.55078125, "nrmse": 0.007951784878969193}, {"layer": 47, "field": "value", "exact": false, "max_abs": 0.33203125, "nrmse": 0.009378538466989994}, {"layer": 48, "field": "conv", "exact": false, "max_abs": 0.2890625, "nrmse": 0.03159920126199722}, {"layer": 48, "field": "recurrent", "exact": false, "max_abs": 0.011822715401649475, "nrmse": 0.026757758110761642}, {"layer": 49, "field": "conv", "exact": false, "max_abs": 0.220703125, "nrmse": 0.03239736333489418}, {"layer": 49, "field": "recurrent", "exact": false, "max_abs": 0.014679014682769775, "nrmse": 0.031348954886198044}, {"layer": 50, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.030983516946434975}, {"layer": 50, "field": "recurrent", "exact": false, "max_abs": 0.007288455963134766, "nrmse": 0.0277634859085083}, {"layer": 51, "field": "key", "exact": false, "max_abs": 0.32421875, "nrmse": 0.00744575634598732}, {"layer": 51, "field": "value", "exact": false, "max_abs": 0.4921875, "nrmse": 0.009901263751089573}, {"layer": 52, "field": "conv", "exact": false, "max_abs": 0.3359375, "nrmse": 0.02686849795281887}, {"layer": 52, "field": "recurrent", "exact": false, "max_abs": 0.032699644565582275, "nrmse": 0.020476585254073143}, {"layer": 53, "field": "conv", "exact": false, "max_abs": 0.28125, "nrmse": 0.02913588285446167}, {"layer": 53, "field": "recurrent", "exact": false, "max_abs": 0.043855488300323486, "nrmse": 0.028468448668718338}, {"layer": 54, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.033714476972818375}, {"layer": 54, "field": "recurrent", "exact": false, "max_abs": 0.021416135132312775, "nrmse": 0.032616760581731796}, {"layer": 55, "field": "key", "exact": false, "max_abs": 0.236328125, "nrmse": 0.007442714646458626}, {"layer": 55, "field": "value", "exact": false, "max_abs": 0.6962890625, "nrmse": 0.011097214184701443}, {"layer": 56, "field": "conv", "exact": false, "max_abs": 0.296875, "nrmse": 0.030723538249731064}, {"layer": 56, "field": "recurrent", "exact": false, "max_abs": 0.03686040639877319, "nrmse": 0.02362510934472084}, {"layer": 57, "field": "conv", "exact": false, "max_abs": 0.318359375, "nrmse": 0.028626101091504097}, {"layer": 57, "field": "recurrent", "exact": false, "max_abs": 0.026910558342933655, "nrmse": 0.025852849707007408}, {"layer": 58, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.0316300131380558}, {"layer": 58, "field": "recurrent", "exact": false, "max_abs": 0.026953592896461487, "nrmse": 0.02264029160141945}, {"layer": 59, "field": "key", "exact": false, "max_abs": 0.2284088134765625, "nrmse": 0.007599120028316975}, {"layer": 59, "field": "value", "exact": false, "max_abs": 0.79296875, "nrmse": 0.008907780051231384}, {"layer": 60, "field": "conv", "exact": false, "max_abs": 0.8125, "nrmse": 0.025191528722643852}, {"layer": 60, "field": "recurrent", "exact": false, "max_abs": 0.04466554522514343, "nrmse": 0.02416723407804966}, {"layer": 61, "field": "conv", "exact": false, "max_abs": 0.375, "nrmse": 0.029296834021806717}, {"layer": 61, "field": "recurrent", "exact": false, "max_abs": 0.017907138913869858, "nrmse": 0.027187807485461235}, {"layer": 62, "field": "conv", "exact": false, "max_abs": 0.40625, "nrmse": 0.02553611993789673}, {"layer": 62, "field": "recurrent", "exact": false, "max_abs": 0.047786712646484375, "nrmse": 0.01848667301237583}, {"layer": 63, "field": "key", "exact": false, "max_abs": 0.25, "nrmse": 0.00815790705382824}, {"layer": 63, "field": "value", "exact": false, "max_abs": 0.84375, "nrmse": 0.00499261450022459}], "buffered_graph_state_diff": []} -{"phase": "step", "index": 36, "position": 184, "teacher_token": 2891, "compact": {"ids": [279, 3991, 13853, 2763, 561], "logits": [28.125, 19.625, 19.5, 17.25, 13.25], "margin": 8.5, "argmax": 279}, "buffered": {"ids": [279, 13853, 3991, 2763, 561], "logits": [28.0, 19.5, 19.375, 16.625, 13.3125], "margin": 8.5, "argmax": 279}, "graph": {"ids": [279, 13853, 3991, 2763, 561], "logits": [28.0, 19.5, 19.375, 16.625, 13.3125], "margin": 8.5, "argmax": 279}, "compact_buffered": {"exact": false, "max_abs": 0.875, "nrmse": 0.02174369804561138}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "compact_buffered_state_diff": [{"layer": 36, "field": "recurrent", "exact": false, "max_abs": 4.76837158203125e-07, "nrmse": 1.328979095660543e-07}, {"layer": 37, "field": "recurrent", "exact": false, "max_abs": 0.0005772821605205536, "nrmse": 0.0005528760375455022}, {"layer": 38, "field": "conv", "exact": false, "max_abs": 0.09375, "nrmse": 0.011688323691487312}, {"layer": 38, "field": "recurrent", "exact": false, "max_abs": 0.006233245134353638, "nrmse": 0.009722073562443256}, {"layer": 39, "field": "key", "exact": false, "max_abs": 0.1875, "nrmse": 0.004329552408307791}, {"layer": 39, "field": "value", "exact": false, "max_abs": 0.21875, "nrmse": 0.005511152558028698}, {"layer": 40, "field": "conv", "exact": false, "max_abs": 0.125, "nrmse": 0.020539719611406326}, {"layer": 40, "field": "recurrent", "exact": false, "max_abs": 0.007149100303649902, "nrmse": 0.0068372623063623905}, {"layer": 41, "field": "conv", "exact": false, "max_abs": 0.15625, "nrmse": 0.023095879703760147}, {"layer": 41, "field": "recurrent", "exact": false, "max_abs": 0.007032208144664764, "nrmse": 0.01518576592206955}, {"layer": 42, "field": "conv", "exact": false, "max_abs": 0.1484375, "nrmse": 0.026674553751945496}, {"layer": 42, "field": "recurrent", "exact": false, "max_abs": 0.010091610252857208, "nrmse": 0.016439164057374}, {"layer": 43, "field": "key", "exact": false, "max_abs": 0.3515625, "nrmse": 0.006221218034625053}, {"layer": 43, "field": "value", "exact": false, "max_abs": 0.31640625, "nrmse": 0.00705134030431509}, {"layer": 44, "field": "conv", "exact": false, "max_abs": 0.171875, "nrmse": 0.026603523641824722}, {"layer": 44, "field": "recurrent", "exact": false, "max_abs": 0.013043396174907684, "nrmse": 0.021758591756224632}, {"layer": 45, "field": "conv", "exact": false, "max_abs": 0.16015625, "nrmse": 0.03087339922785759}, {"layer": 45, "field": "recurrent", "exact": false, "max_abs": 0.01288217306137085, "nrmse": 0.024599356576800346}, {"layer": 46, "field": "conv", "exact": false, "max_abs": 0.1875, "nrmse": 0.03012860007584095}, {"layer": 46, "field": "recurrent", "exact": false, "max_abs": 0.016071796417236328, "nrmse": 0.024235980585217476}, {"layer": 47, "field": "key", "exact": false, "max_abs": 0.55078125, "nrmse": 0.008261055685579777}, {"layer": 47, "field": "value", "exact": false, "max_abs": 0.33203125, "nrmse": 0.009716419503092766}, {"layer": 48, "field": "conv", "exact": false, "max_abs": 0.2890625, "nrmse": 0.03157876059412956}, {"layer": 48, "field": "recurrent", "exact": false, "max_abs": 0.011792145669460297, "nrmse": 0.025784624740481377}, {"layer": 49, "field": "conv", "exact": false, "max_abs": 0.21875, "nrmse": 0.032069917768239975}, {"layer": 49, "field": "recurrent", "exact": false, "max_abs": 0.012481153011322021, "nrmse": 0.03025246411561966}, {"layer": 50, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.03129620850086212}, {"layer": 50, "field": "recurrent", "exact": false, "max_abs": 0.007376410067081451, "nrmse": 0.027478987351059914}, {"layer": 51, "field": "key", "exact": false, "max_abs": 0.32421875, "nrmse": 0.007747367024421692}, {"layer": 51, "field": "value", "exact": false, "max_abs": 0.4921875, "nrmse": 0.010229232721030712}, {"layer": 52, "field": "conv", "exact": false, "max_abs": 0.3359375, "nrmse": 0.02580096572637558}, {"layer": 52, "field": "recurrent", "exact": false, "max_abs": 0.032455265522003174, "nrmse": 0.02047731727361679}, {"layer": 53, "field": "conv", "exact": false, "max_abs": 0.28125, "nrmse": 0.027205582708120346}, {"layer": 53, "field": "recurrent", "exact": false, "max_abs": 0.043797701597213745, "nrmse": 0.02809896320104599}, {"layer": 54, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.03088461607694626}, {"layer": 54, "field": "recurrent", "exact": false, "max_abs": 0.021687254309654236, "nrmse": 0.03188623487949371}, {"layer": 55, "field": "key", "exact": false, "max_abs": 0.236328125, "nrmse": 0.007922379299998283}, {"layer": 55, "field": "value", "exact": false, "max_abs": 0.6962890625, "nrmse": 0.01139891892671585}, {"layer": 56, "field": "conv", "exact": false, "max_abs": 0.28125, "nrmse": 0.02867073565721512}, {"layer": 56, "field": "recurrent", "exact": false, "max_abs": 0.037067949771881104, "nrmse": 0.023162126541137695}, {"layer": 57, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.026141250506043434}, {"layer": 57, "field": "recurrent", "exact": false, "max_abs": 0.026821449398994446, "nrmse": 0.02509908936917782}, {"layer": 58, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.02942846529185772}, {"layer": 58, "field": "recurrent", "exact": false, "max_abs": 0.026931658387184143, "nrmse": 0.022122252732515335}, {"layer": 59, "field": "key", "exact": false, "max_abs": 0.2734375, "nrmse": 0.00829966738820076}, {"layer": 59, "field": "value", "exact": false, "max_abs": 0.79296875, "nrmse": 0.009202993474900723}, {"layer": 60, "field": "conv", "exact": false, "max_abs": 0.8125, "nrmse": 0.023767918348312378}, {"layer": 60, "field": "recurrent", "exact": false, "max_abs": 0.06109321117401123, "nrmse": 0.025512931868433952}, {"layer": 61, "field": "conv", "exact": false, "max_abs": 0.28125, "nrmse": 0.027609366923570633}, {"layer": 61, "field": "recurrent", "exact": false, "max_abs": 0.018429215997457504, "nrmse": 0.02727317623794079}, {"layer": 62, "field": "conv", "exact": false, "max_abs": 0.4296875, "nrmse": 0.024430835619568825}, {"layer": 62, "field": "recurrent", "exact": false, "max_abs": 0.02976427972316742, "nrmse": 0.018610209226608276}, {"layer": 63, "field": "key", "exact": false, "max_abs": 0.25, "nrmse": 0.008570685982704163}, {"layer": 63, "field": "value", "exact": false, "max_abs": 0.84375, "nrmse": 0.005159083288162947}], "buffered_graph_state_diff": []} -{"phase": "step", "index": 37, "position": 185, "teacher_token": 279, "compact": {"ids": [2763, 3991, 104542, 5732, 1428], "logits": [31.125, 26.5, 19.75, 18.75, 18.75], "margin": 4.625, "argmax": 2763}, "buffered": {"ids": [2763, 3991, 104542, 9147, 1428], "logits": [31.125, 26.75, 19.75, 18.875, 18.625], "margin": 4.375, "argmax": 2763}, "graph": {"ids": [2763, 3991, 104542, 9147, 1428], "logits": [31.125, 26.75, 19.75, 18.875, 18.625], "margin": 4.375, "argmax": 2763}, "compact_buffered": {"exact": false, "max_abs": 0.421875, "nrmse": 0.023020662367343903}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "compact_buffered_state_diff": [{"layer": 36, "field": "recurrent", "exact": false, "max_abs": 4.76837158203125e-07, "nrmse": 1.334346819703569e-07}, {"layer": 37, "field": "recurrent", "exact": false, "max_abs": 0.0005527138710021973, "nrmse": 0.0005471715121529996}, {"layer": 38, "field": "conv", "exact": false, "max_abs": 0.09375, "nrmse": 0.012917231768369675}, {"layer": 38, "field": "recurrent", "exact": false, "max_abs": 0.006217151880264282, "nrmse": 0.009863466024398804}, {"layer": 39, "field": "key", "exact": false, "max_abs": 0.1875, "nrmse": 0.00455069076269865}, {"layer": 39, "field": "value", "exact": false, "max_abs": 0.21875, "nrmse": 0.005732604768127203}, {"layer": 40, "field": "conv", "exact": false, "max_abs": 0.125, "nrmse": 0.020671123638749123}, {"layer": 40, "field": "recurrent", "exact": false, "max_abs": 0.007893085479736328, "nrmse": 0.006969752721488476}, {"layer": 41, "field": "conv", "exact": false, "max_abs": 0.15625, "nrmse": 0.02397056668996811}, {"layer": 41, "field": "recurrent", "exact": false, "max_abs": 0.007037550210952759, "nrmse": 0.015255909413099289}, {"layer": 42, "field": "conv", "exact": false, "max_abs": 0.1484375, "nrmse": 0.027885962277650833}, {"layer": 42, "field": "recurrent", "exact": false, "max_abs": 0.009352900087833405, "nrmse": 0.01628902740776539}, {"layer": 43, "field": "key", "exact": false, "max_abs": 0.3515625, "nrmse": 0.006424249149858952}, {"layer": 43, "field": "value", "exact": false, "max_abs": 0.31640625, "nrmse": 0.007250201888382435}, {"layer": 44, "field": "conv", "exact": false, "max_abs": 0.171875, "nrmse": 0.027959179133176804}, {"layer": 44, "field": "recurrent", "exact": false, "max_abs": 0.012992218136787415, "nrmse": 0.021739881485700607}, {"layer": 45, "field": "conv", "exact": false, "max_abs": 0.171875, "nrmse": 0.032398149371147156}, {"layer": 45, "field": "recurrent", "exact": false, "max_abs": 0.012875378131866455, "nrmse": 0.02474111132323742}, {"layer": 46, "field": "conv", "exact": false, "max_abs": 0.1875, "nrmse": 0.03162236139178276}, {"layer": 46, "field": "recurrent", "exact": false, "max_abs": 0.01614391803741455, "nrmse": 0.02431575022637844}, {"layer": 47, "field": "key", "exact": false, "max_abs": 0.55078125, "nrmse": 0.008499869145452976}, {"layer": 47, "field": "value", "exact": false, "max_abs": 0.33203125, "nrmse": 0.00998661294579506}, {"layer": 48, "field": "conv", "exact": false, "max_abs": 0.2890625, "nrmse": 0.033225614577531815}, {"layer": 48, "field": "recurrent", "exact": false, "max_abs": 0.011806607246398926, "nrmse": 0.02588375099003315}, {"layer": 49, "field": "conv", "exact": false, "max_abs": 0.21875, "nrmse": 0.032965682446956635}, {"layer": 49, "field": "recurrent", "exact": false, "max_abs": 0.0122145414352417, "nrmse": 0.03004380129277706}, {"layer": 50, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.03163781017065048}, {"layer": 50, "field": "recurrent", "exact": false, "max_abs": 0.007421448826789856, "nrmse": 0.02752923220396042}, {"layer": 51, "field": "key", "exact": false, "max_abs": 0.32421875, "nrmse": 0.007993847131729126}, {"layer": 51, "field": "value", "exact": false, "max_abs": 0.4921875, "nrmse": 0.010454055853188038}, {"layer": 52, "field": "conv", "exact": false, "max_abs": 0.3359375, "nrmse": 0.024965912103652954}, {"layer": 52, "field": "recurrent", "exact": false, "max_abs": 0.03329133987426758, "nrmse": 0.020914604887366295}, {"layer": 53, "field": "conv", "exact": false, "max_abs": 0.2578125, "nrmse": 0.025612443685531616}, {"layer": 53, "field": "recurrent", "exact": false, "max_abs": 0.043294697999954224, "nrmse": 0.028009505942463875}, {"layer": 54, "field": "conv", "exact": false, "max_abs": 0.25, "nrmse": 0.029323143884539604}, {"layer": 54, "field": "recurrent", "exact": false, "max_abs": 0.022912174463272095, "nrmse": 0.03142014518380165}, {"layer": 55, "field": "key", "exact": false, "max_abs": 0.236328125, "nrmse": 0.008156424388289452}, {"layer": 55, "field": "value", "exact": false, "max_abs": 0.6962890625, "nrmse": 0.011610480025410652}, {"layer": 56, "field": "conv", "exact": false, "max_abs": 0.28125, "nrmse": 0.02743494138121605}, {"layer": 56, "field": "recurrent", "exact": false, "max_abs": 0.037168800830841064, "nrmse": 0.02313222736120224}, {"layer": 57, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.024706192314624786}, {"layer": 57, "field": "recurrent", "exact": false, "max_abs": 0.026507019996643066, "nrmse": 0.02473885379731655}, {"layer": 58, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.0274957362562418}, {"layer": 58, "field": "recurrent", "exact": false, "max_abs": 0.0274505615234375, "nrmse": 0.02164418436586857}, {"layer": 59, "field": "key", "exact": false, "max_abs": 0.2734375, "nrmse": 0.008506758138537407}, {"layer": 59, "field": "value", "exact": false, "max_abs": 0.79296875, "nrmse": 0.009387367404997349}, {"layer": 60, "field": "conv", "exact": false, "max_abs": 0.8125, "nrmse": 0.02252259850502014}, {"layer": 60, "field": "recurrent", "exact": false, "max_abs": 0.05063314735889435, "nrmse": 0.02470758557319641}, {"layer": 61, "field": "conv", "exact": false, "max_abs": 0.28125, "nrmse": 0.02555863931775093}, {"layer": 61, "field": "recurrent", "exact": false, "max_abs": 0.017617033794522285, "nrmse": 0.026608906686306}, {"layer": 62, "field": "conv", "exact": false, "max_abs": 0.5625, "nrmse": 0.024791035801172256}, {"layer": 62, "field": "recurrent", "exact": false, "max_abs": 0.03086341917514801, "nrmse": 0.01844562217593193}, {"layer": 63, "field": "key", "exact": false, "max_abs": 0.25, "nrmse": 0.008815706707537174}, {"layer": 63, "field": "value", "exact": false, "max_abs": 0.84375, "nrmse": 0.005269655957818031}], "buffered_graph_state_diff": []} -{"phase": "step", "index": 38, "position": 186, "teacher_token": 2763, "compact": {"ids": [1528, 6331, 4530, 14159, 1324], "logits": [29.5, 22.125, 21.625, 21.0, 21.0], "margin": 7.375, "argmax": 1528}, "buffered": {"ids": [1528, 6331, 4530, 1324, 14159], "logits": [29.75, 22.125, 21.75, 21.125, 20.75], "margin": 7.625, "argmax": 1528}, "graph": {"ids": [1528, 6331, 4530, 1324, 14159], "logits": [29.75, 22.125, 21.75, 21.125, 20.75], "margin": 7.625, "argmax": 1528}, "compact_buffered": {"exact": false, "max_abs": 0.375, "nrmse": 0.022250259295105934}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "compact_buffered_state_diff": [{"layer": 36, "field": "recurrent", "exact": false, "max_abs": 4.76837158203125e-07, "nrmse": 1.3415333910415939e-07}, {"layer": 37, "field": "recurrent", "exact": false, "max_abs": 0.0005453452467918396, "nrmse": 0.0005402601673267782}, {"layer": 38, "field": "conv", "exact": false, "max_abs": 0.09375, "nrmse": 0.012096039019525051}, {"layer": 38, "field": "recurrent", "exact": false, "max_abs": 0.006219416856765747, "nrmse": 0.009546088054776192}, {"layer": 39, "field": "key", "exact": false, "max_abs": 0.1875, "nrmse": 0.004658186808228493}, {"layer": 39, "field": "value", "exact": false, "max_abs": 0.21875, "nrmse": 0.005895523354411125}, {"layer": 40, "field": "conv", "exact": false, "max_abs": 0.125, "nrmse": 0.019819537177681923}, {"layer": 40, "field": "recurrent", "exact": false, "max_abs": 0.007161974906921387, "nrmse": 0.006759262178093195}, {"layer": 41, "field": "conv", "exact": false, "max_abs": 0.15625, "nrmse": 0.022354906424880028}, {"layer": 41, "field": "recurrent", "exact": false, "max_abs": 0.007002152502536774, "nrmse": 0.014978687278926373}, {"layer": 42, "field": "conv", "exact": false, "max_abs": 0.1875, "nrmse": 0.026533469557762146}, {"layer": 42, "field": "recurrent", "exact": false, "max_abs": 0.008419737219810486, "nrmse": 0.015919828787446022}, {"layer": 43, "field": "key", "exact": false, "max_abs": 0.3515625, "nrmse": 0.006551465485244989}, {"layer": 43, "field": "value", "exact": false, "max_abs": 0.31640625, "nrmse": 0.007425319403409958}, {"layer": 44, "field": "conv", "exact": false, "max_abs": 0.16796875, "nrmse": 0.025993622839450836}, {"layer": 44, "field": "recurrent", "exact": false, "max_abs": 0.01282542198896408, "nrmse": 0.021624188870191574}, {"layer": 45, "field": "conv", "exact": false, "max_abs": 0.171875, "nrmse": 0.030607957392930984}, {"layer": 45, "field": "recurrent", "exact": false, "max_abs": 0.012875735759735107, "nrmse": 0.024819452315568924}, {"layer": 46, "field": "conv", "exact": false, "max_abs": 0.1875, "nrmse": 0.029791569337248802}, {"layer": 46, "field": "recurrent", "exact": false, "max_abs": 0.016169726848602295, "nrmse": 0.024362986907362938}, {"layer": 47, "field": "key", "exact": false, "max_abs": 0.55078125, "nrmse": 0.008593300357460976}, {"layer": 47, "field": "value", "exact": false, "max_abs": 0.33203125, "nrmse": 0.010181366465985775}, {"layer": 48, "field": "conv", "exact": false, "max_abs": 0.193359375, "nrmse": 0.03035786561667919}, {"layer": 48, "field": "recurrent", "exact": false, "max_abs": 0.011668123304843903, "nrmse": 0.02597501128911972}, {"layer": 49, "field": "conv", "exact": false, "max_abs": 0.21875, "nrmse": 0.030084140598773956}, {"layer": 49, "field": "recurrent", "exact": false, "max_abs": 0.011119470000267029, "nrmse": 0.02899843081831932}, {"layer": 50, "field": "conv", "exact": false, "max_abs": 0.25, "nrmse": 0.029794247820973396}, {"layer": 50, "field": "recurrent", "exact": false, "max_abs": 0.00942334532737732, "nrmse": 0.027090366929769516}, {"layer": 51, "field": "key", "exact": false, "max_abs": 0.32421875, "nrmse": 0.008093971759080887}, {"layer": 51, "field": "value", "exact": false, "max_abs": 0.4921875, "nrmse": 0.010656247846782207}, {"layer": 52, "field": "conv", "exact": false, "max_abs": 0.28125, "nrmse": 0.02427748776972294}, {"layer": 52, "field": "recurrent", "exact": false, "max_abs": 0.038974374532699585, "nrmse": 0.020957443863153458}, {"layer": 53, "field": "conv", "exact": false, "max_abs": 0.2578125, "nrmse": 0.024730827659368515}, {"layer": 53, "field": "recurrent", "exact": false, "max_abs": 0.042664170265197754, "nrmse": 0.02787609212100506}, {"layer": 54, "field": "conv", "exact": false, "max_abs": 0.25, "nrmse": 0.02757790870964527}, {"layer": 54, "field": "recurrent", "exact": false, "max_abs": 0.023243263363838196, "nrmse": 0.03099982999265194}, {"layer": 55, "field": "key", "exact": false, "max_abs": 0.236328125, "nrmse": 0.008303233422338963}, {"layer": 55, "field": "value", "exact": false, "max_abs": 0.6962890625, "nrmse": 0.011795259080827236}, {"layer": 56, "field": "conv", "exact": false, "max_abs": 0.267578125, "nrmse": 0.025696653872728348}, {"layer": 56, "field": "recurrent", "exact": false, "max_abs": 0.03669413924217224, "nrmse": 0.02257409878075123}, {"layer": 57, "field": "conv", "exact": false, "max_abs": 0.296875, "nrmse": 0.023783229291439056}, {"layer": 57, "field": "recurrent", "exact": false, "max_abs": 0.028847217559814453, "nrmse": 0.024281634017825127}, {"layer": 58, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.026048077270388603}, {"layer": 58, "field": "recurrent", "exact": false, "max_abs": 0.026928231120109558, "nrmse": 0.021243572235107422}, {"layer": 59, "field": "key", "exact": false, "max_abs": 0.2734375, "nrmse": 0.00864479597657919}, {"layer": 59, "field": "value", "exact": false, "max_abs": 0.79296875, "nrmse": 0.009572832845151424}, {"layer": 60, "field": "conv", "exact": false, "max_abs": 0.75, "nrmse": 0.02162538841366768}, {"layer": 60, "field": "recurrent", "exact": false, "max_abs": 0.05072250962257385, "nrmse": 0.024414775893092155}, {"layer": 61, "field": "conv", "exact": false, "max_abs": 0.28125, "nrmse": 0.024638114497065544}, {"layer": 61, "field": "recurrent", "exact": false, "max_abs": 0.017862534150481224, "nrmse": 0.02599717304110527}, {"layer": 62, "field": "conv", "exact": false, "max_abs": 0.5625, "nrmse": 0.02360047586262226}, {"layer": 62, "field": "recurrent", "exact": false, "max_abs": 0.02715715765953064, "nrmse": 0.018391059711575508}, {"layer": 63, "field": "key", "exact": false, "max_abs": 0.25, "nrmse": 0.008961561135947704}, {"layer": 63, "field": "value", "exact": false, "max_abs": 0.84375, "nrmse": 0.005395293235778809}], "buffered_graph_state_diff": []} -{"phase": "step", "index": 39, "position": 187, "teacher_token": 1528, "compact": {"ids": [64700, 314, 303, 318, 321], "logits": [29.0, 26.125, 18.125, 17.375, 17.25], "margin": 2.875, "argmax": 64700}, "buffered": {"ids": [64700, 314, 303, 318, 321], "logits": [29.125, 26.0, 18.0, 17.375, 17.125], "margin": 3.125, "argmax": 64700}, "graph": {"ids": [64700, 314, 303, 318, 321], "logits": [29.125, 26.0, 18.0, 17.375, 17.125], "margin": 3.125, "argmax": 64700}, "compact_buffered": {"exact": false, "max_abs": 0.36328125, "nrmse": 0.024131333455443382}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "compact_buffered_state_diff": [{"layer": 36, "field": "recurrent", "exact": false, "max_abs": 4.76837158203125e-07, "nrmse": 1.327385206195686e-07}, {"layer": 37, "field": "recurrent", "exact": false, "max_abs": 0.000545985996723175, "nrmse": 0.0005404270486906171}, {"layer": 38, "field": "conv", "exact": false, "max_abs": 0.09375, "nrmse": 0.012168703600764275}, {"layer": 38, "field": "recurrent", "exact": false, "max_abs": 0.00692063570022583, "nrmse": 0.009940451011061668}, {"layer": 39, "field": "key", "exact": false, "max_abs": 0.1875, "nrmse": 0.004723666235804558}, {"layer": 39, "field": "value", "exact": false, "max_abs": 0.21875, "nrmse": 0.0060279276221990585}, {"layer": 40, "field": "conv", "exact": false, "max_abs": 0.125, "nrmse": 0.019503453746438026}, {"layer": 40, "field": "recurrent", "exact": false, "max_abs": 0.007282733917236328, "nrmse": 0.006654667668044567}, {"layer": 41, "field": "conv", "exact": false, "max_abs": 0.125, "nrmse": 0.021788591518998146}, {"layer": 41, "field": "recurrent", "exact": false, "max_abs": 0.007050611078739166, "nrmse": 0.015477548353374004}, {"layer": 42, "field": "conv", "exact": false, "max_abs": 0.1875, "nrmse": 0.02583906054496765}, {"layer": 42, "field": "recurrent", "exact": false, "max_abs": 0.011024177074432373, "nrmse": 0.01648710109293461}, {"layer": 43, "field": "key", "exact": false, "max_abs": 0.3515625, "nrmse": 0.006694328505545855}, {"layer": 43, "field": "value", "exact": false, "max_abs": 0.31640625, "nrmse": 0.007559658493846655}, {"layer": 44, "field": "conv", "exact": false, "max_abs": 0.16796875, "nrmse": 0.026116913184523582}, {"layer": 44, "field": "recurrent", "exact": false, "max_abs": 0.01283998042345047, "nrmse": 0.022230956703424454}, {"layer": 45, "field": "conv", "exact": false, "max_abs": 0.171875, "nrmse": 0.030168095603585243}, {"layer": 45, "field": "recurrent", "exact": false, "max_abs": 0.012859940528869629, "nrmse": 0.02543499879539013}, {"layer": 46, "field": "conv", "exact": false, "max_abs": 0.1875, "nrmse": 0.029303232207894325}, {"layer": 46, "field": "recurrent", "exact": false, "max_abs": 0.016149818897247314, "nrmse": 0.024476703256368637}, {"layer": 47, "field": "key", "exact": false, "max_abs": 0.55078125, "nrmse": 0.008692082948982716}, {"layer": 47, "field": "value", "exact": false, "max_abs": 0.33203125, "nrmse": 0.010362716391682625}, {"layer": 48, "field": "conv", "exact": false, "max_abs": 0.193359375, "nrmse": 0.029513882473111153}, {"layer": 48, "field": "recurrent", "exact": false, "max_abs": 0.011647291481494904, "nrmse": 0.025453688576817513}, {"layer": 49, "field": "conv", "exact": false, "max_abs": 0.21875, "nrmse": 0.02963108941912651}, {"layer": 49, "field": "recurrent", "exact": false, "max_abs": 0.011118568480014801, "nrmse": 0.028600826859474182}, {"layer": 50, "field": "conv", "exact": false, "max_abs": 0.25, "nrmse": 0.029582129791378975}, {"layer": 50, "field": "recurrent", "exact": false, "max_abs": 0.007363460958003998, "nrmse": 0.027369270101189613}, {"layer": 51, "field": "key", "exact": false, "max_abs": 0.32421875, "nrmse": 0.008192023262381554}, {"layer": 51, "field": "value", "exact": false, "max_abs": 0.4921875, "nrmse": 0.01087966188788414}, {"layer": 52, "field": "conv", "exact": false, "max_abs": 0.28125, "nrmse": 0.02407948113977909}, {"layer": 52, "field": "recurrent", "exact": false, "max_abs": 0.03274112939834595, "nrmse": 0.020664865151047707}, {"layer": 53, "field": "conv", "exact": false, "max_abs": 0.28125, "nrmse": 0.02438822202384472}, {"layer": 53, "field": "recurrent", "exact": false, "max_abs": 0.04220786690711975, "nrmse": 0.02787451446056366}, {"layer": 54, "field": "conv", "exact": false, "max_abs": 0.25, "nrmse": 0.026783863082528114}, {"layer": 54, "field": "recurrent", "exact": false, "max_abs": 0.0232001394033432, "nrmse": 0.030910272151231766}, {"layer": 55, "field": "key", "exact": false, "max_abs": 0.236328125, "nrmse": 0.008392943069338799}, {"layer": 55, "field": "value", "exact": false, "max_abs": 0.6962890625, "nrmse": 0.012059712782502174}, {"layer": 56, "field": "conv", "exact": false, "max_abs": 0.32421875, "nrmse": 0.02532101236283779}, {"layer": 56, "field": "recurrent", "exact": false, "max_abs": 0.036803603172302246, "nrmse": 0.022615553811192513}, {"layer": 57, "field": "conv", "exact": false, "max_abs": 0.296875, "nrmse": 0.023257916793227196}, {"layer": 57, "field": "recurrent", "exact": false, "max_abs": 0.03647291660308838, "nrmse": 0.024299893528223038}, {"layer": 58, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.02552690915763378}, {"layer": 58, "field": "recurrent", "exact": false, "max_abs": 0.03580892086029053, "nrmse": 0.02107112854719162}, {"layer": 59, "field": "key", "exact": false, "max_abs": 0.2734375, "nrmse": 0.008753511123359203}, {"layer": 59, "field": "value", "exact": false, "max_abs": 0.79296875, "nrmse": 0.009768159128725529}, {"layer": 60, "field": "conv", "exact": false, "max_abs": 0.75, "nrmse": 0.02089497819542885}, {"layer": 60, "field": "recurrent", "exact": false, "max_abs": 0.045840129256248474, "nrmse": 0.024267902597784996}, {"layer": 61, "field": "conv", "exact": false, "max_abs": 0.28125, "nrmse": 0.023383568972349167}, {"layer": 61, "field": "recurrent", "exact": false, "max_abs": 0.01731722429394722, "nrmse": 0.026049010455608368}, {"layer": 62, "field": "conv", "exact": false, "max_abs": 0.5625, "nrmse": 0.02249140664935112}, {"layer": 62, "field": "recurrent", "exact": false, "max_abs": 0.026059821248054504, "nrmse": 0.018171099945902824}, {"layer": 63, "field": "key", "exact": false, "max_abs": 0.25, "nrmse": 0.009082436561584473}, {"layer": 63, "field": "value", "exact": false, "max_abs": 0.84375, "nrmse": 0.005524291191250086}], "buffered_graph_state_diff": []} -{"phase": "step", "index": 40, "position": 188, "teacher_token": 64700, "compact": {"ids": [198, 13853, 271, 31318, 348], "logits": [27.875, 14.1875, 13.125, 12.1875, 11.875], "margin": 13.6875, "argmax": 198}, "buffered": {"ids": [198, 13853, 271, 31318, 348], "logits": [27.75, 14.125, 13.125, 12.125, 11.8125], "margin": 13.625, "argmax": 198}, "graph": {"ids": [198, 13853, 271, 31318, 348], "logits": [27.75, 14.125, 13.125, 12.125, 11.8125], "margin": 13.625, "argmax": 198}, "compact_buffered": {"exact": false, "max_abs": 0.2578125, "nrmse": 0.01303589902818203}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "compact_buffered_state_diff": [{"layer": 36, "field": "recurrent", "exact": false, "max_abs": 4.76837158203125e-07, "nrmse": 1.3388698505423235e-07}, {"layer": 37, "field": "recurrent", "exact": false, "max_abs": 0.0005646198987960815, "nrmse": 0.0005445280694402754}, {"layer": 38, "field": "conv", "exact": false, "max_abs": 0.09375, "nrmse": 0.012207113206386566}, {"layer": 38, "field": "recurrent", "exact": false, "max_abs": 0.006210446357727051, "nrmse": 0.009631644003093243}, {"layer": 39, "field": "key", "exact": false, "max_abs": 0.1875, "nrmse": 0.0048860521055758}, {"layer": 39, "field": "value", "exact": false, "max_abs": 0.21875, "nrmse": 0.00617792084813118}, {"layer": 40, "field": "conv", "exact": false, "max_abs": 0.1015625, "nrmse": 0.019021738320589066}, {"layer": 40, "field": "recurrent", "exact": false, "max_abs": 0.007752418518066406, "nrmse": 0.006438793148845434}, {"layer": 41, "field": "conv", "exact": false, "max_abs": 0.125, "nrmse": 0.02156742289662361}, {"layer": 41, "field": "recurrent", "exact": false, "max_abs": 0.0071364641189575195, "nrmse": 0.01497619692236185}, {"layer": 42, "field": "conv", "exact": false, "max_abs": 0.1875, "nrmse": 0.02527107298374176}, {"layer": 42, "field": "recurrent", "exact": false, "max_abs": 0.007470071315765381, "nrmse": 0.015680236741900444}, {"layer": 43, "field": "key", "exact": false, "max_abs": 0.3515625, "nrmse": 0.006854101549834013}, {"layer": 43, "field": "value", "exact": false, "max_abs": 0.31640625, "nrmse": 0.007697917986661196}, {"layer": 44, "field": "conv", "exact": false, "max_abs": 0.16796875, "nrmse": 0.02489091269671917}, {"layer": 44, "field": "recurrent", "exact": false, "max_abs": 0.012917943298816681, "nrmse": 0.022001037374138832}, {"layer": 45, "field": "conv", "exact": false, "max_abs": 0.1875, "nrmse": 0.028803851455450058}, {"layer": 45, "field": "recurrent", "exact": false, "max_abs": 0.01285463571548462, "nrmse": 0.024986527860164642}, {"layer": 46, "field": "conv", "exact": false, "max_abs": 0.15625, "nrmse": 0.02746683731675148}, {"layer": 46, "field": "recurrent", "exact": false, "max_abs": 0.016165196895599365, "nrmse": 0.02418598346412182}, {"layer": 47, "field": "key", "exact": false, "max_abs": 0.55078125, "nrmse": 0.00885316263884306}, {"layer": 47, "field": "value", "exact": false, "max_abs": 0.33203125, "nrmse": 0.010591083206236362}, {"layer": 48, "field": "conv", "exact": false, "max_abs": 0.193359375, "nrmse": 0.02821224182844162}, {"layer": 48, "field": "recurrent", "exact": false, "max_abs": 0.01166023313999176, "nrmse": 0.024770068004727364}, {"layer": 49, "field": "conv", "exact": false, "max_abs": 0.21875, "nrmse": 0.028792155906558037}, {"layer": 49, "field": "recurrent", "exact": false, "max_abs": 0.011123798787593842, "nrmse": 0.02836088091135025}, {"layer": 50, "field": "conv", "exact": false, "max_abs": 0.26171875, "nrmse": 0.028531644493341446}, {"layer": 50, "field": "recurrent", "exact": false, "max_abs": 0.007311083376407623, "nrmse": 0.026783965528011322}, {"layer": 51, "field": "key", "exact": false, "max_abs": 0.32421875, "nrmse": 0.008404606953263283}, {"layer": 51, "field": "value", "exact": false, "max_abs": 0.4921875, "nrmse": 0.011150326579809189}, {"layer": 52, "field": "conv", "exact": false, "max_abs": 0.265625, "nrmse": 0.024133170023560524}, {"layer": 52, "field": "recurrent", "exact": false, "max_abs": 0.03478839993476868, "nrmse": 0.02042274735867977}, {"layer": 53, "field": "conv", "exact": false, "max_abs": 0.28125, "nrmse": 0.02543165162205696}, {"layer": 53, "field": "recurrent", "exact": false, "max_abs": 0.04090988636016846, "nrmse": 0.02799059823155403}, {"layer": 54, "field": "conv", "exact": false, "max_abs": 0.25, "nrmse": 0.027732381597161293}, {"layer": 54, "field": "recurrent", "exact": false, "max_abs": 0.021167777478694916, "nrmse": 0.030889546498656273}, {"layer": 55, "field": "key", "exact": false, "max_abs": 0.236328125, "nrmse": 0.008583519607782364}, {"layer": 55, "field": "value", "exact": false, "max_abs": 0.6962890625, "nrmse": 0.01232876442372799}, {"layer": 56, "field": "conv", "exact": false, "max_abs": 0.32421875, "nrmse": 0.025342432782053947}, {"layer": 56, "field": "recurrent", "exact": false, "max_abs": 0.0369604229927063, "nrmse": 0.023029234260320663}, {"layer": 57, "field": "conv", "exact": false, "max_abs": 0.30078125, "nrmse": 0.023536691442131996}, {"layer": 57, "field": "recurrent", "exact": false, "max_abs": 0.027797818183898926, "nrmse": 0.024442533031105995}, {"layer": 58, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.025522105395793915}, {"layer": 58, "field": "recurrent", "exact": false, "max_abs": 0.0342177152633667, "nrmse": 0.02159038372337818}, {"layer": 59, "field": "key", "exact": false, "max_abs": 0.2734375, "nrmse": 0.008881129324436188}, {"layer": 59, "field": "value", "exact": false, "max_abs": 0.79296875, "nrmse": 0.009960968978703022}, {"layer": 60, "field": "conv", "exact": false, "max_abs": 0.5, "nrmse": 0.020518582314252853}, {"layer": 60, "field": "recurrent", "exact": false, "max_abs": 0.0437110960483551, "nrmse": 0.024350561201572418}, {"layer": 61, "field": "conv", "exact": false, "max_abs": 0.375, "nrmse": 0.0245591402053833}, {"layer": 61, "field": "recurrent", "exact": false, "max_abs": 0.01904451847076416, "nrmse": 0.026046814396977425}, {"layer": 62, "field": "conv", "exact": false, "max_abs": 0.75, "nrmse": 0.023982366546988487}, {"layer": 62, "field": "recurrent", "exact": false, "max_abs": 0.02550867199897766, "nrmse": 0.017949670553207397}, {"layer": 63, "field": "key", "exact": false, "max_abs": 0.25, "nrmse": 0.009242753498256207}, {"layer": 63, "field": "value", "exact": false, "max_abs": 0.84375, "nrmse": 0.005665380973368883}], "buffered_graph_state_diff": []} -{"phase": "step", "index": 41, "position": 189, "teacher_token": 198, "compact": {"ids": [9, 31318, 12, 256, 86888], "logits": [25.75, 24.0, 21.375, 17.5, 16.625], "margin": 1.75, "argmax": 9}, "buffered": {"ids": [9, 31318, 12, 256, 86888], "logits": [25.625, 24.0, 21.25, 17.25, 16.625], "margin": 1.625, "argmax": 9}, "graph": {"ids": [9, 31318, 12, 256, 86888], "logits": [25.625, 24.0, 21.25, 17.25, 16.625], "margin": 1.625, "argmax": 9}, "compact_buffered": {"exact": false, "max_abs": 0.3125, "nrmse": 0.030159842222929}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "compact_buffered_state_diff": [{"layer": 36, "field": "recurrent", "exact": false, "max_abs": 4.76837158203125e-07, "nrmse": 1.3478413052325777e-07}, {"layer": 37, "field": "recurrent", "exact": false, "max_abs": 0.0005768649280071259, "nrmse": 0.0005525847664102912}, {"layer": 38, "field": "conv", "exact": false, "max_abs": 0.0625, "nrmse": 0.010796060785651207}, {"layer": 38, "field": "recurrent", "exact": false, "max_abs": 0.005941480398178101, "nrmse": 0.009664127603173256}, {"layer": 39, "field": "key", "exact": false, "max_abs": 0.1875, "nrmse": 0.005068168044090271}, {"layer": 39, "field": "value", "exact": false, "max_abs": 0.21875, "nrmse": 0.006339035928249359}, {"layer": 40, "field": "conv", "exact": false, "max_abs": 0.125, "nrmse": 0.017882607877254486}, {"layer": 40, "field": "recurrent", "exact": false, "max_abs": 0.008502483367919922, "nrmse": 0.0064371852204203606}, {"layer": 41, "field": "conv", "exact": false, "max_abs": 0.125, "nrmse": 0.020617272704839706}, {"layer": 41, "field": "recurrent", "exact": false, "max_abs": 0.007090628147125244, "nrmse": 0.014859721064567566}, {"layer": 42, "field": "conv", "exact": false, "max_abs": 0.1875, "nrmse": 0.02409210242331028}, {"layer": 42, "field": "recurrent", "exact": false, "max_abs": 0.008699759840965271, "nrmse": 0.016068698838353157}, {"layer": 43, "field": "key", "exact": false, "max_abs": 0.3515625, "nrmse": 0.007040789350867271}, {"layer": 43, "field": "value", "exact": false, "max_abs": 0.31640625, "nrmse": 0.007857880555093288}, {"layer": 44, "field": "conv", "exact": false, "max_abs": 0.1328125, "nrmse": 0.023554332554340363}, {"layer": 44, "field": "recurrent", "exact": false, "max_abs": 0.012838371098041534, "nrmse": 0.0220404751598835}, {"layer": 45, "field": "conv", "exact": false, "max_abs": 0.1875, "nrmse": 0.0269610658288002}, {"layer": 45, "field": "recurrent", "exact": false, "max_abs": 0.012846827507019043, "nrmse": 0.025077098980545998}, {"layer": 46, "field": "conv", "exact": false, "max_abs": 0.15625, "nrmse": 0.026253100484609604}, {"layer": 46, "field": "recurrent", "exact": false, "max_abs": 0.016130447387695312, "nrmse": 0.024108393117785454}, {"layer": 47, "field": "key", "exact": false, "max_abs": 0.55078125, "nrmse": 0.009026985615491867}, {"layer": 47, "field": "value", "exact": false, "max_abs": 0.33203125, "nrmse": 0.010811416432261467}, {"layer": 48, "field": "conv", "exact": false, "max_abs": 0.1875, "nrmse": 0.02703041397035122}, {"layer": 48, "field": "recurrent", "exact": false, "max_abs": 0.011529140174388885, "nrmse": 0.02543739229440689}, {"layer": 49, "field": "conv", "exact": false, "max_abs": 0.21875, "nrmse": 0.02798590064048767}, {"layer": 49, "field": "recurrent", "exact": false, "max_abs": 0.011069066822528839, "nrmse": 0.02870052121579647}, {"layer": 50, "field": "conv", "exact": false, "max_abs": 0.26171875, "nrmse": 0.027858661487698555}, {"layer": 50, "field": "recurrent", "exact": false, "max_abs": 0.009143531322479248, "nrmse": 0.026965539902448654}, {"layer": 51, "field": "key", "exact": false, "max_abs": 0.32421875, "nrmse": 0.008598697371780872}, {"layer": 51, "field": "value", "exact": false, "max_abs": 0.4921875, "nrmse": 0.011408654972910881}, {"layer": 52, "field": "conv", "exact": false, "max_abs": 0.265625, "nrmse": 0.024574823677539825}, {"layer": 52, "field": "recurrent", "exact": false, "max_abs": 0.03472602367401123, "nrmse": 0.020238284021615982}, {"layer": 53, "field": "conv", "exact": false, "max_abs": 0.28125, "nrmse": 0.026247015222907066}, {"layer": 53, "field": "recurrent", "exact": false, "max_abs": 0.04239094257354736, "nrmse": 0.02740323171019554}, {"layer": 54, "field": "conv", "exact": false, "max_abs": 0.25, "nrmse": 0.02895573154091835}, {"layer": 54, "field": "recurrent", "exact": false, "max_abs": 0.02442244440317154, "nrmse": 0.030186181887984276}, {"layer": 55, "field": "key", "exact": false, "max_abs": 0.236328125, "nrmse": 0.00870700553059578}, {"layer": 55, "field": "value", "exact": false, "max_abs": 0.6962890625, "nrmse": 0.012618517503142357}, {"layer": 56, "field": "conv", "exact": false, "max_abs": 0.32421875, "nrmse": 0.025862671434879303}, {"layer": 56, "field": "recurrent", "exact": false, "max_abs": 0.035632044076919556, "nrmse": 0.022642675787210464}, {"layer": 57, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.02441134676337242}, {"layer": 57, "field": "recurrent", "exact": false, "max_abs": 0.02809000387787819, "nrmse": 0.02390105091035366}, {"layer": 58, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.026226093992590904}, {"layer": 58, "field": "recurrent", "exact": false, "max_abs": 0.033820509910583496, "nrmse": 0.021050790324807167}, {"layer": 59, "field": "key", "exact": false, "max_abs": 0.2734375, "nrmse": 0.008961380459368229}, {"layer": 59, "field": "value", "exact": false, "max_abs": 0.79296875, "nrmse": 0.010172294452786446}, {"layer": 60, "field": "conv", "exact": false, "max_abs": 0.5078125, "nrmse": 0.02119332365691662}, {"layer": 60, "field": "recurrent", "exact": false, "max_abs": 0.04435551166534424, "nrmse": 0.02470196969807148}, {"layer": 61, "field": "conv", "exact": false, "max_abs": 0.375, "nrmse": 0.02524859830737114}, {"layer": 61, "field": "recurrent", "exact": false, "max_abs": 0.022649317979812622, "nrmse": 0.02530290000140667}, {"layer": 62, "field": "conv", "exact": false, "max_abs": 0.75, "nrmse": 0.022793196141719818}, {"layer": 62, "field": "recurrent", "exact": false, "max_abs": 0.02465727925300598, "nrmse": 0.01810494251549244}, {"layer": 63, "field": "key", "exact": false, "max_abs": 0.25, "nrmse": 0.00938794482499361}, {"layer": 63, "field": "value", "exact": false, "max_abs": 0.84375, "nrmse": 0.005764503963291645}], "buffered_graph_state_diff": []} -{"phase": "step", "index": 42, "position": 190, "teacher_token": 9, "compact": {"ids": [256, 3991, 13853, 262, 6140], "logits": [28.875, 17.5, 15.875, 14.5625, 14.125], "margin": 11.375, "argmax": 256}, "buffered": {"ids": [256, 3991, 13853, 262, 6140], "logits": [28.625, 17.375, 15.9375, 14.5, 13.8125], "margin": 11.25, "argmax": 256}, "graph": {"ids": [256, 3991, 13853, 262, 6140], "logits": [28.625, 17.375, 15.9375, 14.5, 13.8125], "margin": 11.25, "argmax": 256}, "compact_buffered": {"exact": false, "max_abs": 0.375, "nrmse": 0.015390248969197273}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "compact_buffered_state_diff": [{"layer": 36, "field": "recurrent", "exact": false, "max_abs": 4.76837158203125e-07, "nrmse": 1.3384050134845893e-07}, {"layer": 37, "field": "recurrent", "exact": false, "max_abs": 0.0005639530718326569, "nrmse": 0.0005419512162916362}, {"layer": 38, "field": "conv", "exact": false, "max_abs": 0.0625, "nrmse": 0.011477947235107422}, {"layer": 38, "field": "recurrent", "exact": false, "max_abs": 0.005911797285079956, "nrmse": 0.009602113626897335}, {"layer": 39, "field": "key", "exact": false, "max_abs": 0.1875, "nrmse": 0.005171678494662046}, {"layer": 39, "field": "value", "exact": false, "max_abs": 0.21875, "nrmse": 0.0064584966748952866}, {"layer": 40, "field": "conv", "exact": false, "max_abs": 0.125, "nrmse": 0.01774229109287262}, {"layer": 40, "field": "recurrent", "exact": false, "max_abs": 0.008275747299194336, "nrmse": 0.00646873377263546}, {"layer": 41, "field": "conv", "exact": false, "max_abs": 0.109375, "nrmse": 0.020879549905657768}, {"layer": 41, "field": "recurrent", "exact": false, "max_abs": 0.006979800760746002, "nrmse": 0.014928365126252174}, {"layer": 42, "field": "conv", "exact": false, "max_abs": 0.15625, "nrmse": 0.02367103099822998}, {"layer": 42, "field": "recurrent", "exact": false, "max_abs": 0.007375568151473999, "nrmse": 0.01554959174245596}, {"layer": 43, "field": "key", "exact": false, "max_abs": 0.3515625, "nrmse": 0.007201528176665306}, {"layer": 43, "field": "value", "exact": false, "max_abs": 0.31640625, "nrmse": 0.00798987690359354}, {"layer": 44, "field": "conv", "exact": false, "max_abs": 0.1328125, "nrmse": 0.022790446877479553}, {"layer": 44, "field": "recurrent", "exact": false, "max_abs": 0.012634657323360443, "nrmse": 0.021789107471704483}, {"layer": 45, "field": "conv", "exact": false, "max_abs": 0.1875, "nrmse": 0.026219908148050308}, {"layer": 45, "field": "recurrent", "exact": false, "max_abs": 0.012894392013549805, "nrmse": 0.024642156437039375}, {"layer": 46, "field": "conv", "exact": false, "max_abs": 0.15234375, "nrmse": 0.025548601523041725}, {"layer": 46, "field": "recurrent", "exact": false, "max_abs": 0.015988826751708984, "nrmse": 0.02424810826778412}, {"layer": 47, "field": "key", "exact": false, "max_abs": 0.55078125, "nrmse": 0.00917324610054493}, {"layer": 47, "field": "value", "exact": false, "max_abs": 0.33203125, "nrmse": 0.010974054224789143}, {"layer": 48, "field": "conv", "exact": false, "max_abs": 0.1875, "nrmse": 0.02737712487578392}, {"layer": 48, "field": "recurrent", "exact": false, "max_abs": 0.011191681027412415, "nrmse": 0.025377405807375908}, {"layer": 49, "field": "conv", "exact": false, "max_abs": 0.21875, "nrmse": 0.028482787311077118}, {"layer": 49, "field": "recurrent", "exact": false, "max_abs": 0.012634873390197754, "nrmse": 0.028180116787552834}, {"layer": 50, "field": "conv", "exact": false, "max_abs": 0.26171875, "nrmse": 0.02805715799331665}, {"layer": 50, "field": "recurrent", "exact": false, "max_abs": 0.01186513900756836, "nrmse": 0.026464883238077164}, {"layer": 51, "field": "key", "exact": false, "max_abs": 0.32421875, "nrmse": 0.008757677860558033}, {"layer": 51, "field": "value", "exact": false, "max_abs": 0.4921875, "nrmse": 0.01161952130496502}, {"layer": 52, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.025012504309415817}, {"layer": 52, "field": "recurrent", "exact": false, "max_abs": 0.03684702515602112, "nrmse": 0.02027258649468422}, {"layer": 53, "field": "conv", "exact": false, "max_abs": 0.28125, "nrmse": 0.026615412905812263}, {"layer": 53, "field": "recurrent", "exact": false, "max_abs": 0.04251730442047119, "nrmse": 0.027554046362638474}, {"layer": 54, "field": "conv", "exact": false, "max_abs": 0.265625, "nrmse": 0.03011750802397728}, {"layer": 54, "field": "recurrent", "exact": false, "max_abs": 0.023622021079063416, "nrmse": 0.02997032180428505}, {"layer": 55, "field": "key", "exact": false, "max_abs": 0.236328125, "nrmse": 0.008905447088181973}, {"layer": 55, "field": "value", "exact": false, "max_abs": 0.6962890625, "nrmse": 0.012883827090263367}, {"layer": 56, "field": "conv", "exact": false, "max_abs": 0.32421875, "nrmse": 0.026991957798600197}, {"layer": 56, "field": "recurrent", "exact": false, "max_abs": 0.03608635067939758, "nrmse": 0.02352498471736908}, {"layer": 57, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.025736942887306213}, {"layer": 57, "field": "recurrent", "exact": false, "max_abs": 0.034268081188201904, "nrmse": 0.025155676528811455}, {"layer": 58, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.027010861784219742}, {"layer": 58, "field": "recurrent", "exact": false, "max_abs": 0.03421509265899658, "nrmse": 0.021041257306933403}, {"layer": 59, "field": "key", "exact": false, "max_abs": 0.2734375, "nrmse": 0.009144577197730541}, {"layer": 59, "field": "value", "exact": false, "max_abs": 0.79296875, "nrmse": 0.010374827310442924}, {"layer": 60, "field": "conv", "exact": false, "max_abs": 0.75, "nrmse": 0.02172367088496685}, {"layer": 60, "field": "recurrent", "exact": false, "max_abs": 0.043128788471221924, "nrmse": 0.025002719834446907}, {"layer": 61, "field": "conv", "exact": false, "max_abs": 0.375, "nrmse": 0.0271099042147398}, {"layer": 61, "field": "recurrent", "exact": false, "max_abs": 0.01680074632167816, "nrmse": 0.025705503299832344}, {"layer": 62, "field": "conv", "exact": false, "max_abs": 0.75, "nrmse": 0.02307753637433052}, {"layer": 62, "field": "recurrent", "exact": false, "max_abs": 0.02959054708480835, "nrmse": 0.018319716677069664}, {"layer": 63, "field": "key", "exact": false, "max_abs": 0.25, "nrmse": 0.00955713540315628}, {"layer": 63, "field": "value", "exact": false, "max_abs": 0.84375, "nrmse": 0.005900999065488577}], "buffered_graph_state_diff": []} -{"phase": "step", "index": 43, "position": 191, "teacher_token": 256, "compact": {"ids": [3991, 13853, 21281, 10342, 5447], "logits": [25.25, 21.875, 21.5, 18.0, 18.0], "margin": 3.375, "argmax": 3991}, "buffered": {"ids": [3991, 13853, 21281, 16722, 10342], "logits": [25.125, 22.125, 21.375, 18.125, 18.125], "margin": 3.0, "argmax": 3991}, "graph": {"ids": [3991, 13853, 21281, 16722, 10342], "logits": [25.125, 22.125, 21.375, 18.125, 18.125], "margin": 3.0, "argmax": 3991}, "compact_buffered": {"exact": false, "max_abs": 0.375, "nrmse": 0.01906081847846508}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "compact_buffered_state_diff": [{"layer": 36, "field": "recurrent", "exact": false, "max_abs": 4.76837158203125e-07, "nrmse": 1.3256608610845433e-07}, {"layer": 37, "field": "recurrent", "exact": false, "max_abs": 0.0005485303699970245, "nrmse": 0.0005277839954942465}, {"layer": 38, "field": "conv", "exact": false, "max_abs": 0.0625, "nrmse": 0.011755445972084999}, {"layer": 38, "field": "recurrent", "exact": false, "max_abs": 0.005937457084655762, "nrmse": 0.009392594918608665}, {"layer": 39, "field": "key", "exact": false, "max_abs": 0.1875, "nrmse": 0.005329279229044914}, {"layer": 39, "field": "value", "exact": false, "max_abs": 0.21875, "nrmse": 0.006595497485250235}, {"layer": 40, "field": "conv", "exact": false, "max_abs": 0.12890625, "nrmse": 0.019005486741662025}, {"layer": 40, "field": "recurrent", "exact": false, "max_abs": 0.00837254524230957, "nrmse": 0.006824756972491741}, {"layer": 41, "field": "conv", "exact": false, "max_abs": 0.11328125, "nrmse": 0.022849190980196}, {"layer": 41, "field": "recurrent", "exact": false, "max_abs": 0.006926827132701874, "nrmse": 0.015245701186358929}, {"layer": 42, "field": "conv", "exact": false, "max_abs": 0.15625, "nrmse": 0.02545030042529106}, {"layer": 42, "field": "recurrent", "exact": false, "max_abs": 0.0073490142822265625, "nrmse": 0.015666507184505463}, {"layer": 43, "field": "key", "exact": false, "max_abs": 0.3515625, "nrmse": 0.007410459220409393}, {"layer": 43, "field": "value", "exact": false, "max_abs": 0.31640625, "nrmse": 0.00821293331682682}, {"layer": 44, "field": "conv", "exact": false, "max_abs": 0.14306640625, "nrmse": 0.024843785911798477}, {"layer": 44, "field": "recurrent", "exact": false, "max_abs": 0.012949749827384949, "nrmse": 0.021997589617967606}, {"layer": 45, "field": "conv", "exact": false, "max_abs": 0.1875, "nrmse": 0.028748566284775734}, {"layer": 45, "field": "recurrent", "exact": false, "max_abs": 0.012875556945800781, "nrmse": 0.02440011315047741}, {"layer": 46, "field": "conv", "exact": false, "max_abs": 0.15234375, "nrmse": 0.027551747858524323}, {"layer": 46, "field": "recurrent", "exact": false, "max_abs": 0.015866398811340332, "nrmse": 0.023897215723991394}, {"layer": 47, "field": "key", "exact": false, "max_abs": 0.55078125, "nrmse": 0.009389585815370083}, {"layer": 47, "field": "value", "exact": false, "max_abs": 0.33203125, "nrmse": 0.011291525326669216}, {"layer": 48, "field": "conv", "exact": false, "max_abs": 0.203125, "nrmse": 0.0302408579736948}, {"layer": 48, "field": "recurrent", "exact": false, "max_abs": 0.011468417942523956, "nrmse": 0.0264702457934618}, {"layer": 49, "field": "conv", "exact": false, "max_abs": 0.203125, "nrmse": 0.03015298955142498}, {"layer": 49, "field": "recurrent", "exact": false, "max_abs": 0.014187037944793701, "nrmse": 0.02898542769253254}, {"layer": 50, "field": "conv", "exact": false, "max_abs": 0.26171875, "nrmse": 0.029414886608719826}, {"layer": 50, "field": "recurrent", "exact": false, "max_abs": 0.01064559817314148, "nrmse": 0.026960674673318863}, {"layer": 51, "field": "key", "exact": false, "max_abs": 0.32421875, "nrmse": 0.009091817773878574}, {"layer": 51, "field": "value", "exact": false, "max_abs": 0.4921875, "nrmse": 0.011885520070791245}, {"layer": 52, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.02595696970820427}, {"layer": 52, "field": "recurrent", "exact": false, "max_abs": 0.03952687978744507, "nrmse": 0.020763620734214783}, {"layer": 53, "field": "conv", "exact": false, "max_abs": 0.25, "nrmse": 0.02766577899456024}, {"layer": 53, "field": "recurrent", "exact": false, "max_abs": 0.041829824447631836, "nrmse": 0.02745053730905056}, {"layer": 54, "field": "conv", "exact": false, "max_abs": 0.265625, "nrmse": 0.030820921063423157}, {"layer": 54, "field": "recurrent", "exact": false, "max_abs": 0.022525697946548462, "nrmse": 0.029351381585001945}, {"layer": 55, "field": "key", "exact": false, "max_abs": 0.236328125, "nrmse": 0.009150550700724125}, {"layer": 55, "field": "value", "exact": false, "max_abs": 0.6962890625, "nrmse": 0.01309738215059042}, {"layer": 56, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.02720026671886444}, {"layer": 56, "field": "recurrent", "exact": false, "max_abs": 0.036142587661743164, "nrmse": 0.02329106256365776}, {"layer": 57, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.02593201957643032}, {"layer": 57, "field": "recurrent", "exact": false, "max_abs": 0.02844172716140747, "nrmse": 0.024231065064668655}, {"layer": 58, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.02749745361506939}, {"layer": 58, "field": "recurrent", "exact": false, "max_abs": 0.033173441886901855, "nrmse": 0.021060984581708908}, {"layer": 59, "field": "key", "exact": false, "max_abs": 0.2734375, "nrmse": 0.009316322393715382}, {"layer": 59, "field": "value", "exact": false, "max_abs": 0.79296875, "nrmse": 0.010531486943364143}, {"layer": 60, "field": "conv", "exact": false, "max_abs": 0.75, "nrmse": 0.02189066633582115}, {"layer": 60, "field": "recurrent", "exact": false, "max_abs": 0.04307207465171814, "nrmse": 0.02511078678071499}, {"layer": 61, "field": "conv", "exact": false, "max_abs": 0.375, "nrmse": 0.027123911306262016}, {"layer": 61, "field": "recurrent", "exact": false, "max_abs": 0.016730472445487976, "nrmse": 0.025655163452029228}, {"layer": 62, "field": "conv", "exact": false, "max_abs": 0.75, "nrmse": 0.023945346474647522}, {"layer": 62, "field": "recurrent", "exact": false, "max_abs": 0.033360958099365234, "nrmse": 0.01858360879123211}, {"layer": 63, "field": "key", "exact": false, "max_abs": 0.25, "nrmse": 0.009785924106836319}, {"layer": 63, "field": "value", "exact": false, "max_abs": 0.84375, "nrmse": 0.006013277918100357}], "buffered_graph_state_diff": []} -{"phase": "step", "index": 44, "position": 192, "teacher_token": 3991, "compact": {"ids": [18016, 21281, 4022, 1324, 2702], "logits": [24.375, 21.875, 19.375, 19.125, 16.0], "margin": 2.5, "argmax": 18016}, "buffered": {"ids": [18016, 21281, 4022, 1324, 2702], "logits": [24.625, 21.75, 19.375, 19.125, 16.375], "margin": 2.875, "argmax": 18016}, "graph": {"ids": [18016, 21281, 4022, 1324, 2702], "logits": [24.625, 21.75, 19.375, 19.125, 16.375], "margin": 2.875, "argmax": 18016}, "compact_buffered": {"exact": false, "max_abs": 0.375, "nrmse": 0.017567405477166176}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "compact_buffered_state_diff": [{"layer": 36, "field": "recurrent", "exact": false, "max_abs": 4.76837158203125e-07, "nrmse": 1.343264699471547e-07}, {"layer": 37, "field": "recurrent", "exact": false, "max_abs": 0.0005423501133918762, "nrmse": 0.0005264241481199861}, {"layer": 38, "field": "conv", "exact": false, "max_abs": 0.0625, "nrmse": 0.011495694518089294}, {"layer": 38, "field": "recurrent", "exact": false, "max_abs": 0.0059368908405303955, "nrmse": 0.00957368966192007}, {"layer": 39, "field": "key", "exact": false, "max_abs": 0.1875, "nrmse": 0.005405505187809467}, {"layer": 39, "field": "value", "exact": false, "max_abs": 0.21875, "nrmse": 0.006704125087708235}, {"layer": 40, "field": "conv", "exact": false, "max_abs": 0.12890625, "nrmse": 0.018450260162353516}, {"layer": 40, "field": "recurrent", "exact": false, "max_abs": 0.008883476257324219, "nrmse": 0.006926099304109812}, {"layer": 41, "field": "conv", "exact": false, "max_abs": 0.11328125, "nrmse": 0.022349456325173378}, {"layer": 41, "field": "recurrent", "exact": false, "max_abs": 0.006951287388801575, "nrmse": 0.01540492381900549}, {"layer": 42, "field": "conv", "exact": false, "max_abs": 0.125, "nrmse": 0.024721674621105194}, {"layer": 42, "field": "recurrent", "exact": false, "max_abs": 0.007250458002090454, "nrmse": 0.015505199320614338}, {"layer": 43, "field": "key", "exact": false, "max_abs": 0.3515625, "nrmse": 0.007515692617744207}, {"layer": 43, "field": "value", "exact": false, "max_abs": 0.31640625, "nrmse": 0.008339153602719307}, {"layer": 44, "field": "conv", "exact": false, "max_abs": 0.14306640625, "nrmse": 0.02444334141910076}, {"layer": 44, "field": "recurrent", "exact": false, "max_abs": 0.013028167188167572, "nrmse": 0.021891966462135315}, {"layer": 45, "field": "conv", "exact": false, "max_abs": 0.1796875, "nrmse": 0.0282332431524992}, {"layer": 45, "field": "recurrent", "exact": false, "max_abs": 0.012872755527496338, "nrmse": 0.024539919570088387}, {"layer": 46, "field": "conv", "exact": false, "max_abs": 0.1484375, "nrmse": 0.027513014152646065}, {"layer": 46, "field": "recurrent", "exact": false, "max_abs": 0.015874147415161133, "nrmse": 0.0239030122756958}, {"layer": 47, "field": "key", "exact": false, "max_abs": 0.55078125, "nrmse": 0.009491193108260632}, {"layer": 47, "field": "value", "exact": false, "max_abs": 0.33203125, "nrmse": 0.0114476028829813}, {"layer": 48, "field": "conv", "exact": false, "max_abs": 0.203125, "nrmse": 0.03021278604865074}, {"layer": 48, "field": "recurrent", "exact": false, "max_abs": 0.0115986168384552, "nrmse": 0.026857959106564522}, {"layer": 49, "field": "conv", "exact": false, "max_abs": 0.21875, "nrmse": 0.029842976480722427}, {"layer": 49, "field": "recurrent", "exact": false, "max_abs": 0.014001846313476562, "nrmse": 0.028987525030970573}, {"layer": 50, "field": "conv", "exact": false, "max_abs": 0.25, "nrmse": 0.029451582580804825}, {"layer": 50, "field": "recurrent", "exact": false, "max_abs": 0.009713709354400635, "nrmse": 0.026615628972649574}, {"layer": 51, "field": "key", "exact": false, "max_abs": 0.32421875, "nrmse": 0.009215839207172394}, {"layer": 51, "field": "value", "exact": false, "max_abs": 0.4921875, "nrmse": 0.012084996327757835}, {"layer": 52, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.025634026154875755}, {"layer": 52, "field": "recurrent", "exact": false, "max_abs": 0.04571431875228882, "nrmse": 0.021188998594880104}, {"layer": 53, "field": "conv", "exact": false, "max_abs": 0.25, "nrmse": 0.026658006012439728}, {"layer": 53, "field": "recurrent", "exact": false, "max_abs": 0.05443084239959717, "nrmse": 0.027694901451468468}, {"layer": 54, "field": "conv", "exact": false, "max_abs": 0.328125, "nrmse": 0.029767977073788643}, {"layer": 54, "field": "recurrent", "exact": false, "max_abs": 0.022090330719947815, "nrmse": 0.029202939942479134}, {"layer": 55, "field": "key", "exact": false, "max_abs": 0.236328125, "nrmse": 0.009262504987418652}, {"layer": 55, "field": "value", "exact": false, "max_abs": 0.6962890625, "nrmse": 0.013268301263451576}, {"layer": 56, "field": "conv", "exact": false, "max_abs": 0.30078125, "nrmse": 0.02644040435552597}, {"layer": 56, "field": "recurrent", "exact": false, "max_abs": 0.03565514087677002, "nrmse": 0.023238299414515495}, {"layer": 57, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.024960149079561234}, {"layer": 57, "field": "recurrent", "exact": false, "max_abs": 0.026845410466194153, "nrmse": 0.023621318861842155}, {"layer": 58, "field": "conv", "exact": false, "max_abs": 0.28125, "nrmse": 0.02652430534362793}, {"layer": 58, "field": "recurrent", "exact": false, "max_abs": 0.04366934299468994, "nrmse": 0.021199354901909828}, {"layer": 59, "field": "key", "exact": false, "max_abs": 0.2734375, "nrmse": 0.009453403763473034}, {"layer": 59, "field": "value", "exact": false, "max_abs": 0.79296875, "nrmse": 0.010726804845035076}, {"layer": 60, "field": "conv", "exact": false, "max_abs": 0.75, "nrmse": 0.02207695133984089}, {"layer": 60, "field": "recurrent", "exact": false, "max_abs": 0.04268178343772888, "nrmse": 0.025083158165216446}, {"layer": 61, "field": "conv", "exact": false, "max_abs": 0.375, "nrmse": 0.025205068290233612}, {"layer": 61, "field": "recurrent", "exact": false, "max_abs": 0.016841977834701538, "nrmse": 0.025570353493094444}, {"layer": 62, "field": "conv", "exact": false, "max_abs": 0.75, "nrmse": 0.021501168608665466}, {"layer": 62, "field": "recurrent", "exact": false, "max_abs": 0.03081512451171875, "nrmse": 0.018613412976264954}, {"layer": 63, "field": "key", "exact": false, "max_abs": 0.25, "nrmse": 0.00996606145054102}, {"layer": 63, "field": "value", "exact": false, "max_abs": 0.84375, "nrmse": 0.0061271036975085735}], "buffered_graph_state_diff": []} -{"phase": "step", "index": 45, "position": 193, "teacher_token": 18016, "compact": {"ids": [7397, 91103, 17629, 1651, 117500], "logits": [30.5, 18.75, 18.0, 17.75, 17.375], "margin": 11.75, "argmax": 7397}, "buffered": {"ids": [7397, 91103, 17629, 1651, 117500], "logits": [30.375, 18.625, 18.0, 17.75, 17.375], "margin": 11.75, "argmax": 7397}, "graph": {"ids": [7397, 91103, 17629, 1651, 117500], "logits": [30.375, 18.625, 18.0, 17.75, 17.375], "margin": 11.75, "argmax": 7397}, "compact_buffered": {"exact": false, "max_abs": 0.32421875, "nrmse": 0.02009403333067894}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "compact_buffered_state_diff": [{"layer": 36, "field": "recurrent", "exact": false, "max_abs": 4.76837158203125e-07, "nrmse": 1.3398393150509946e-07}, {"layer": 37, "field": "recurrent", "exact": false, "max_abs": 0.0005329661071300507, "nrmse": 0.0005248020752333105}, {"layer": 38, "field": "conv", "exact": false, "max_abs": 0.0625, "nrmse": 0.011194484308362007}, {"layer": 38, "field": "recurrent", "exact": false, "max_abs": 0.005938023328781128, "nrmse": 0.009482130408287048}, {"layer": 39, "field": "key", "exact": false, "max_abs": 0.1875, "nrmse": 0.005451532546430826}, {"layer": 39, "field": "value", "exact": false, "max_abs": 0.21875, "nrmse": 0.006775594316422939}, {"layer": 40, "field": "conv", "exact": false, "max_abs": 0.12890625, "nrmse": 0.018523329868912697}, {"layer": 40, "field": "recurrent", "exact": false, "max_abs": 0.00865793228149414, "nrmse": 0.0069081708788871765}, {"layer": 41, "field": "conv", "exact": false, "max_abs": 0.11328125, "nrmse": 0.022063620388507843}, {"layer": 41, "field": "recurrent", "exact": false, "max_abs": 0.006976902484893799, "nrmse": 0.015594418160617352}, {"layer": 42, "field": "conv", "exact": false, "max_abs": 0.125, "nrmse": 0.023968815803527832}, {"layer": 42, "field": "recurrent", "exact": false, "max_abs": 0.006905406713485718, "nrmse": 0.015428881160914898}, {"layer": 43, "field": "key", "exact": false, "max_abs": 0.3515625, "nrmse": 0.007621565368026495}, {"layer": 43, "field": "value", "exact": false, "max_abs": 0.31640625, "nrmse": 0.008455893024802208}, {"layer": 44, "field": "conv", "exact": false, "max_abs": 0.14306640625, "nrmse": 0.024438394233584404}, {"layer": 44, "field": "recurrent", "exact": false, "max_abs": 0.01298021525144577, "nrmse": 0.021899603307247162}, {"layer": 45, "field": "conv", "exact": false, "max_abs": 0.1875, "nrmse": 0.0286736898124218}, {"layer": 45, "field": "recurrent", "exact": false, "max_abs": 0.012874126434326172, "nrmse": 0.02449801377952099}, {"layer": 46, "field": "conv", "exact": false, "max_abs": 0.1484375, "nrmse": 0.02770698443055153}, {"layer": 46, "field": "recurrent", "exact": false, "max_abs": 0.01583045721054077, "nrmse": 0.023262714967131615}, {"layer": 47, "field": "key", "exact": false, "max_abs": 0.55078125, "nrmse": 0.009560530073940754}, {"layer": 47, "field": "value", "exact": false, "max_abs": 0.33203125, "nrmse": 0.011606315150856972}, {"layer": 48, "field": "conv", "exact": false, "max_abs": 0.203125, "nrmse": 0.029947930946946144}, {"layer": 48, "field": "recurrent", "exact": false, "max_abs": 0.011618368327617645, "nrmse": 0.026167696341872215}, {"layer": 49, "field": "conv", "exact": false, "max_abs": 0.21875, "nrmse": 0.030012382194399834}, {"layer": 49, "field": "recurrent", "exact": false, "max_abs": 0.014087438583374023, "nrmse": 0.02938680723309517}, {"layer": 50, "field": "conv", "exact": false, "max_abs": 0.25, "nrmse": 0.029974451288580894}, {"layer": 50, "field": "recurrent", "exact": false, "max_abs": 0.009910553693771362, "nrmse": 0.02663629874587059}, {"layer": 51, "field": "key", "exact": false, "max_abs": 0.32421875, "nrmse": 0.009308496490120888}, {"layer": 51, "field": "value", "exact": false, "max_abs": 0.4921875, "nrmse": 0.012300175614655018}, {"layer": 52, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.025096863508224487}, {"layer": 52, "field": "recurrent", "exact": false, "max_abs": 0.046072423458099365, "nrmse": 0.021488752216100693}, {"layer": 53, "field": "conv", "exact": false, "max_abs": 0.25, "nrmse": 0.025982694700360298}, {"layer": 53, "field": "recurrent", "exact": false, "max_abs": 0.056150734424591064, "nrmse": 0.0276144128292799}, {"layer": 54, "field": "conv", "exact": false, "max_abs": 0.328125, "nrmse": 0.028582461178302765}, {"layer": 54, "field": "recurrent", "exact": false, "max_abs": 0.022776737809181213, "nrmse": 0.02907905913889408}, {"layer": 55, "field": "key", "exact": false, "max_abs": 0.236328125, "nrmse": 0.009379449300467968}, {"layer": 55, "field": "value", "exact": false, "max_abs": 0.6962890625, "nrmse": 0.01351115945726633}, {"layer": 56, "field": "conv", "exact": false, "max_abs": 0.296875, "nrmse": 0.025879018008708954}, {"layer": 56, "field": "recurrent", "exact": false, "max_abs": 0.0355360209941864, "nrmse": 0.023343196138739586}, {"layer": 57, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.02359599433839321}, {"layer": 57, "field": "recurrent", "exact": false, "max_abs": 0.02807268500328064, "nrmse": 0.02363942191004753}, {"layer": 58, "field": "conv", "exact": false, "max_abs": 0.28125, "nrmse": 0.025581030175089836}, {"layer": 58, "field": "recurrent", "exact": false, "max_abs": 0.02801799774169922, "nrmse": 0.02107270061969757}, {"layer": 59, "field": "key", "exact": false, "max_abs": 0.2734375, "nrmse": 0.00956522487103939}, {"layer": 59, "field": "value", "exact": false, "max_abs": 0.79296875, "nrmse": 0.010958678089082241}, {"layer": 60, "field": "conv", "exact": false, "max_abs": 0.75, "nrmse": 0.022704247385263443}, {"layer": 60, "field": "recurrent", "exact": false, "max_abs": 0.04271093010902405, "nrmse": 0.025388283655047417}, {"layer": 61, "field": "conv", "exact": false, "max_abs": 0.375, "nrmse": 0.025945113971829414}, {"layer": 61, "field": "recurrent", "exact": false, "max_abs": 0.016772404313087463, "nrmse": 0.026101769879460335}, {"layer": 62, "field": "conv", "exact": false, "max_abs": 0.75, "nrmse": 0.020850474014878273}, {"layer": 62, "field": "recurrent", "exact": false, "max_abs": 0.032317884266376495, "nrmse": 0.01848716102540493}, {"layer": 63, "field": "key", "exact": false, "max_abs": 0.25, "nrmse": 0.010115300305187702}, {"layer": 63, "field": "value", "exact": false, "max_abs": 0.84375, "nrmse": 0.0062980991788208485}], "buffered_graph_state_diff": []} -{"phase": "step", "index": 46, "position": 194, "teacher_token": 7397, "compact": {"ids": [25, 1651, 283, 303, 393], "logits": [26.875, 26.625, 23.875, 18.625, 17.0], "margin": 0.25, "argmax": 25}, "buffered": {"ids": [25, 1651, 283, 303, 393], "logits": [27.0, 26.625, 24.0, 18.875, 17.125], "margin": 0.375, "argmax": 25}, "graph": {"ids": [25, 1651, 283, 303, 393], "logits": [27.0, 26.625, 24.0, 18.875, 17.125], "margin": 0.375, "argmax": 25}, "compact_buffered": {"exact": false, "max_abs": 0.4375, "nrmse": 0.028476128354668617}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "compact_buffered_state_diff": [{"layer": 36, "field": "recurrent", "exact": false, "max_abs": 4.76837158203125e-07, "nrmse": 1.3201159276832186e-07}, {"layer": 37, "field": "recurrent", "exact": false, "max_abs": 0.0005319640040397644, "nrmse": 0.0005147720221430063}, {"layer": 38, "field": "conv", "exact": false, "max_abs": 0.0625, "nrmse": 0.01096185389906168}, {"layer": 38, "field": "recurrent", "exact": false, "max_abs": 0.005935460329055786, "nrmse": 0.009520981460809708}, {"layer": 39, "field": "key", "exact": false, "max_abs": 0.1875, "nrmse": 0.005514972377568483}, {"layer": 39, "field": "value", "exact": false, "max_abs": 0.21875, "nrmse": 0.00685932207852602}, {"layer": 40, "field": "conv", "exact": false, "max_abs": 0.12890625, "nrmse": 0.018401658162474632}, {"layer": 40, "field": "recurrent", "exact": false, "max_abs": 0.009465217590332031, "nrmse": 0.007026456296443939}, {"layer": 41, "field": "conv", "exact": false, "max_abs": 0.11328125, "nrmse": 0.02127266861498356}, {"layer": 41, "field": "recurrent", "exact": false, "max_abs": 0.0069767385721206665, "nrmse": 0.015743574127554893}, {"layer": 42, "field": "conv", "exact": false, "max_abs": 0.125, "nrmse": 0.023989060893654823}, {"layer": 42, "field": "recurrent", "exact": false, "max_abs": 0.00709080696105957, "nrmse": 0.015559244900941849}, {"layer": 43, "field": "key", "exact": false, "max_abs": 0.3515625, "nrmse": 0.007718701846897602}, {"layer": 43, "field": "value", "exact": false, "max_abs": 0.31640625, "nrmse": 0.008565734140574932}, {"layer": 44, "field": "conv", "exact": false, "max_abs": 0.14306640625, "nrmse": 0.024915607646107674}, {"layer": 44, "field": "recurrent", "exact": false, "max_abs": 0.012827903032302856, "nrmse": 0.022089490666985512}, {"layer": 45, "field": "conv", "exact": false, "max_abs": 0.1875, "nrmse": 0.029184114187955856}, {"layer": 45, "field": "recurrent", "exact": false, "max_abs": 0.012873530387878418, "nrmse": 0.025024166330695152}, {"layer": 46, "field": "conv", "exact": false, "max_abs": 0.1484375, "nrmse": 0.02829824946820736}, {"layer": 46, "field": "recurrent", "exact": false, "max_abs": 0.01583653688430786, "nrmse": 0.02391468547284603}, {"layer": 47, "field": "key", "exact": false, "max_abs": 0.55078125, "nrmse": 0.009695684537291527}, {"layer": 47, "field": "value", "exact": false, "max_abs": 0.33203125, "nrmse": 0.011779295280575752}, {"layer": 48, "field": "conv", "exact": false, "max_abs": 0.203125, "nrmse": 0.02981698326766491}, {"layer": 48, "field": "recurrent", "exact": false, "max_abs": 0.011526510119438171, "nrmse": 0.02599947899580002}, {"layer": 49, "field": "conv", "exact": false, "max_abs": 0.21875, "nrmse": 0.029719777405261993}, {"layer": 49, "field": "recurrent", "exact": false, "max_abs": 0.014104664325714111, "nrmse": 0.0289937611669302}, {"layer": 50, "field": "conv", "exact": false, "max_abs": 0.25, "nrmse": 0.03005947917699814}, {"layer": 50, "field": "recurrent", "exact": false, "max_abs": 0.00967743992805481, "nrmse": 0.026942197233438492}, {"layer": 51, "field": "key", "exact": false, "max_abs": 0.32421875, "nrmse": 0.009416474960744381}, {"layer": 51, "field": "value", "exact": false, "max_abs": 0.4921875, "nrmse": 0.012477084994316101}, {"layer": 52, "field": "conv", "exact": false, "max_abs": 0.28125, "nrmse": 0.024680595844984055}, {"layer": 52, "field": "recurrent", "exact": false, "max_abs": 0.04208827018737793, "nrmse": 0.021340183913707733}, {"layer": 53, "field": "conv", "exact": false, "max_abs": 0.25, "nrmse": 0.025636350736021996}, {"layer": 53, "field": "recurrent", "exact": false, "max_abs": 0.056269168853759766, "nrmse": 0.027402164414525032}, {"layer": 54, "field": "conv", "exact": false, "max_abs": 0.328125, "nrmse": 0.028619971126317978}, {"layer": 54, "field": "recurrent", "exact": false, "max_abs": 0.022175714373588562, "nrmse": 0.02915925160050392}, {"layer": 55, "field": "key", "exact": false, "max_abs": 0.236328125, "nrmse": 0.009541500359773636}, {"layer": 55, "field": "value", "exact": false, "max_abs": 0.6962890625, "nrmse": 0.01377277821302414}, {"layer": 56, "field": "conv", "exact": false, "max_abs": 0.296875, "nrmse": 0.025395063683390617}, {"layer": 56, "field": "recurrent", "exact": false, "max_abs": 0.03592836856842041, "nrmse": 0.023352552205324173}, {"layer": 57, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.023266466334462166}, {"layer": 57, "field": "recurrent", "exact": false, "max_abs": 0.025988757610321045, "nrmse": 0.02369498834013939}, {"layer": 58, "field": "conv", "exact": false, "max_abs": 0.28125, "nrmse": 0.026285165920853615}, {"layer": 58, "field": "recurrent", "exact": false, "max_abs": 0.028946101665496826, "nrmse": 0.021381713449954987}, {"layer": 59, "field": "key", "exact": false, "max_abs": 0.2734375, "nrmse": 0.00970112718641758}, {"layer": 59, "field": "value", "exact": false, "max_abs": 0.79296875, "nrmse": 0.01120081264525652}, {"layer": 60, "field": "conv", "exact": false, "max_abs": 0.75, "nrmse": 0.022844437509775162}, {"layer": 60, "field": "recurrent", "exact": false, "max_abs": 0.042443275451660156, "nrmse": 0.025752639397978783}, {"layer": 61, "field": "conv", "exact": false, "max_abs": 0.375, "nrmse": 0.02507554180920124}, {"layer": 61, "field": "recurrent", "exact": false, "max_abs": 0.02202778309583664, "nrmse": 0.026269640773534775}, {"layer": 62, "field": "conv", "exact": false, "max_abs": 0.75, "nrmse": 0.02089415118098259}, {"layer": 62, "field": "recurrent", "exact": false, "max_abs": 0.03098607063293457, "nrmse": 0.01864277571439743}, {"layer": 63, "field": "key", "exact": false, "max_abs": 0.25, "nrmse": 0.010283022187650204}, {"layer": 63, "field": "value", "exact": false, "max_abs": 0.84375, "nrmse": 0.006393881514668465}], "buffered_graph_state_diff": []} -{"phase": "step", "index": 47, "position": 195, "teacher_token": 25, "compact": {"ids": [220, 393, 17237, 56014, 198], "logits": [28.375, 25.75, 15.6875, 15.5625, 14.1875], "margin": 2.625, "argmax": 220}, "buffered": {"ids": [220, 393, 17237, 56014, 198], "logits": [28.25, 25.5, 15.6875, 15.5625, 14.125], "margin": 2.75, "argmax": 220}, "graph": {"ids": [220, 393, 17237, 56014, 198], "logits": [28.25, 25.5, 15.6875, 15.5625, 14.125], "margin": 2.75, "argmax": 220}, "compact_buffered": {"exact": false, "max_abs": 0.296875, "nrmse": 0.0177061278373003}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "compact_buffered_state_diff": [{"layer": 36, "field": "recurrent", "exact": false, "max_abs": 4.76837158203125e-07, "nrmse": 1.3249984931462677e-07}, {"layer": 37, "field": "recurrent", "exact": false, "max_abs": 0.0005270130932331085, "nrmse": 0.0005136676481924951}, {"layer": 38, "field": "conv", "exact": false, "max_abs": 0.0625, "nrmse": 0.010475452058017254}, {"layer": 38, "field": "recurrent", "exact": false, "max_abs": 0.005919694900512695, "nrmse": 0.009471471421420574}, {"layer": 39, "field": "key", "exact": false, "max_abs": 0.1875, "nrmse": 0.005648030433803797}, {"layer": 39, "field": "value", "exact": false, "max_abs": 0.21875, "nrmse": 0.0069602131843566895}, {"layer": 40, "field": "conv", "exact": false, "max_abs": 0.09375, "nrmse": 0.017453745007514954}, {"layer": 40, "field": "recurrent", "exact": false, "max_abs": 0.010013043880462646, "nrmse": 0.007034321315586567}, {"layer": 41, "field": "conv", "exact": false, "max_abs": 0.109375, "nrmse": 0.02020220458507538}, {"layer": 41, "field": "recurrent", "exact": false, "max_abs": 0.007060721516609192, "nrmse": 0.01558213122189045}, {"layer": 42, "field": "conv", "exact": false, "max_abs": 0.125, "nrmse": 0.0224512480199337}, {"layer": 42, "field": "recurrent", "exact": false, "max_abs": 0.007059425115585327, "nrmse": 0.015668341889977455}, {"layer": 43, "field": "key", "exact": false, "max_abs": 0.3515625, "nrmse": 0.007845553569495678}, {"layer": 43, "field": "value", "exact": false, "max_abs": 0.31640625, "nrmse": 0.008703663013875484}, {"layer": 44, "field": "conv", "exact": false, "max_abs": 0.140625, "nrmse": 0.023197999224066734}, {"layer": 44, "field": "recurrent", "exact": false, "max_abs": 0.012779735028743744, "nrmse": 0.02209298685193062}, {"layer": 45, "field": "conv", "exact": false, "max_abs": 0.1875, "nrmse": 0.026747247204184532}, {"layer": 45, "field": "recurrent", "exact": false, "max_abs": 0.012858867645263672, "nrmse": 0.025020373985171318}, {"layer": 46, "field": "conv", "exact": false, "max_abs": 0.171875, "nrmse": 0.026045523583889008}, {"layer": 46, "field": "recurrent", "exact": false, "max_abs": 0.015830695629119873, "nrmse": 0.02384703792631626}, {"layer": 47, "field": "key", "exact": false, "max_abs": 0.55078125, "nrmse": 0.009819027967751026}, {"layer": 47, "field": "value", "exact": false, "max_abs": 0.33203125, "nrmse": 0.011934103444218636}, {"layer": 48, "field": "conv", "exact": false, "max_abs": 0.1875, "nrmse": 0.027837136760354042}, {"layer": 48, "field": "recurrent", "exact": false, "max_abs": 0.01176752895116806, "nrmse": 0.02587905526161194}, {"layer": 49, "field": "conv", "exact": false, "max_abs": 0.21875, "nrmse": 0.028507648035883904}, {"layer": 49, "field": "recurrent", "exact": false, "max_abs": 0.014077723026275635, "nrmse": 0.02949407510459423}, {"layer": 50, "field": "conv", "exact": false, "max_abs": 0.21875, "nrmse": 0.02850695699453354}, {"layer": 50, "field": "recurrent", "exact": false, "max_abs": 0.009652942419052124, "nrmse": 0.02698070928454399}, {"layer": 51, "field": "key", "exact": false, "max_abs": 0.32421875, "nrmse": 0.009549187496304512}, {"layer": 51, "field": "value", "exact": false, "max_abs": 0.4921875, "nrmse": 0.01263189222663641}, {"layer": 52, "field": "conv", "exact": false, "max_abs": 0.3671875, "nrmse": 0.024212181568145752}, {"layer": 52, "field": "recurrent", "exact": false, "max_abs": 0.04090636968612671, "nrmse": 0.02131311036646366}, {"layer": 53, "field": "conv", "exact": false, "max_abs": 0.25, "nrmse": 0.02511153742671013}, {"layer": 53, "field": "recurrent", "exact": false, "max_abs": 0.0586320161819458, "nrmse": 0.028034748509526253}, {"layer": 54, "field": "conv", "exact": false, "max_abs": 0.328125, "nrmse": 0.02881328947842121}, {"layer": 54, "field": "recurrent", "exact": false, "max_abs": 0.02163653075695038, "nrmse": 0.02960255928337574}, {"layer": 55, "field": "key", "exact": false, "max_abs": 0.236328125, "nrmse": 0.00969445239752531}, {"layer": 55, "field": "value", "exact": false, "max_abs": 0.6962890625, "nrmse": 0.014020846225321293}, {"layer": 56, "field": "conv", "exact": false, "max_abs": 0.296875, "nrmse": 0.025626065209507942}, {"layer": 56, "field": "recurrent", "exact": false, "max_abs": 0.035790711641311646, "nrmse": 0.02400698885321617}, {"layer": 57, "field": "conv", "exact": false, "max_abs": 0.34375, "nrmse": 0.023827295750379562}, {"layer": 57, "field": "recurrent", "exact": false, "max_abs": 0.028936833143234253, "nrmse": 0.024959959089756012}, {"layer": 58, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.02685493230819702}, {"layer": 58, "field": "recurrent", "exact": false, "max_abs": 0.028855443000793457, "nrmse": 0.022116119042038918}, {"layer": 59, "field": "key", "exact": false, "max_abs": 0.2734375, "nrmse": 0.009798352606594563}, {"layer": 59, "field": "value", "exact": false, "max_abs": 0.79296875, "nrmse": 0.011333717033267021}, {"layer": 60, "field": "conv", "exact": false, "max_abs": 0.75, "nrmse": 0.02309279702603817}, {"layer": 60, "field": "recurrent", "exact": false, "max_abs": 0.0425468385219574, "nrmse": 0.025731181725859642}, {"layer": 61, "field": "conv", "exact": false, "max_abs": 0.34375, "nrmse": 0.02590082958340645}, {"layer": 61, "field": "recurrent", "exact": false, "max_abs": 0.015303537249565125, "nrmse": 0.026476511731743813}, {"layer": 62, "field": "conv", "exact": false, "max_abs": 0.625, "nrmse": 0.02063637599349022}, {"layer": 62, "field": "recurrent", "exact": false, "max_abs": 0.0300065279006958, "nrmse": 0.01869914121925831}, {"layer": 63, "field": "key", "exact": false, "max_abs": 0.25, "nrmse": 0.010416578501462936}, {"layer": 63, "field": "value", "exact": false, "max_abs": 0.890625, "nrmse": 0.006518833804875612}], "buffered_graph_state_diff": []} -{"phase": "step", "index": 48, "position": 196, "teacher_token": 220, "compact": {"ids": [18, 22, 19, 15, 20], "logits": [28.125, 18.5, 15.875, 14.25, 13.5], "margin": 9.625, "argmax": 18}, "buffered": {"ids": [18, 22, 19, 15, 20], "logits": [28.375, 18.5, 15.875, 14.5, 13.5625], "margin": 9.875, "argmax": 18}, "graph": {"ids": [18, 22, 19, 15, 20], "logits": [28.375, 18.5, 15.875, 14.5, 13.5625], "margin": 9.875, "argmax": 18}, "compact_buffered": {"exact": false, "max_abs": 0.3125, "nrmse": 0.015546518377959728}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "compact_buffered_state_diff": [{"layer": 36, "field": "recurrent", "exact": false, "max_abs": 4.76837158203125e-07, "nrmse": 1.311289992145248e-07}, {"layer": 37, "field": "recurrent", "exact": false, "max_abs": 0.0005115121603012085, "nrmse": 0.0005062049604021013}, {"layer": 38, "field": "conv", "exact": false, "max_abs": 0.0625, "nrmse": 0.010978257283568382}, {"layer": 38, "field": "recurrent", "exact": false, "max_abs": 0.005929142236709595, "nrmse": 0.00962452869862318}, {"layer": 39, "field": "key", "exact": false, "max_abs": 0.1875, "nrmse": 0.005753740668296814}, {"layer": 39, "field": "value", "exact": false, "max_abs": 0.21875, "nrmse": 0.007129991427063942}, {"layer": 40, "field": "conv", "exact": false, "max_abs": 0.109375, "nrmse": 0.019023356959223747}, {"layer": 40, "field": "recurrent", "exact": false, "max_abs": 0.008731603622436523, "nrmse": 0.0073180957697331905}, {"layer": 41, "field": "conv", "exact": false, "max_abs": 0.140625, "nrmse": 0.022011535242199898}, {"layer": 41, "field": "recurrent", "exact": false, "max_abs": 0.007085636258125305, "nrmse": 0.015767347067594528}, {"layer": 42, "field": "conv", "exact": false, "max_abs": 0.14013671875, "nrmse": 0.025007842108607292}, {"layer": 42, "field": "recurrent", "exact": false, "max_abs": 0.007114678621292114, "nrmse": 0.015589781105518341}, {"layer": 43, "field": "key", "exact": false, "max_abs": 0.3515625, "nrmse": 0.00807138904929161}, {"layer": 43, "field": "value", "exact": false, "max_abs": 0.31640625, "nrmse": 0.008984953165054321}, {"layer": 44, "field": "conv", "exact": false, "max_abs": 0.1484375, "nrmse": 0.02637482061982155}, {"layer": 44, "field": "recurrent", "exact": false, "max_abs": 0.012743927538394928, "nrmse": 0.022281253710389137}, {"layer": 45, "field": "conv", "exact": false, "max_abs": 0.1875, "nrmse": 0.02988491579890251}, {"layer": 45, "field": "recurrent", "exact": false, "max_abs": 0.012832701206207275, "nrmse": 0.024660613387823105}, {"layer": 46, "field": "conv", "exact": false, "max_abs": 0.203125, "nrmse": 0.02874166890978813}, {"layer": 46, "field": "recurrent", "exact": false, "max_abs": 0.01583319902420044, "nrmse": 0.023539872840046883}, {"layer": 47, "field": "key", "exact": false, "max_abs": 0.55078125, "nrmse": 0.010022263042628765}, {"layer": 47, "field": "value", "exact": false, "max_abs": 0.33203125, "nrmse": 0.01232505775988102}, {"layer": 48, "field": "conv", "exact": false, "max_abs": 0.234375, "nrmse": 0.030808942392468452}, {"layer": 48, "field": "recurrent", "exact": false, "max_abs": 0.011630184948444366, "nrmse": 0.026275139302015305}, {"layer": 49, "field": "conv", "exact": false, "max_abs": 0.208984375, "nrmse": 0.03051215410232544}, {"layer": 49, "field": "recurrent", "exact": false, "max_abs": 0.011351704597473145, "nrmse": 0.02934708632528782}, {"layer": 50, "field": "conv", "exact": false, "max_abs": 0.21875, "nrmse": 0.030724991112947464}, {"layer": 50, "field": "recurrent", "exact": false, "max_abs": 0.009285926818847656, "nrmse": 0.026788920164108276}, {"layer": 51, "field": "key", "exact": false, "max_abs": 0.32421875, "nrmse": 0.009723694995045662}, {"layer": 51, "field": "value", "exact": false, "max_abs": 0.5625, "nrmse": 0.012981518171727657}, {"layer": 52, "field": "conv", "exact": false, "max_abs": 0.3671875, "nrmse": 0.02586713433265686}, {"layer": 52, "field": "recurrent", "exact": false, "max_abs": 0.034606754779815674, "nrmse": 0.021410273388028145}, {"layer": 53, "field": "conv", "exact": false, "max_abs": 0.2568359375, "nrmse": 0.026513002812862396}, {"layer": 53, "field": "recurrent", "exact": false, "max_abs": 0.05870378017425537, "nrmse": 0.028308171778917313}, {"layer": 54, "field": "conv", "exact": false, "max_abs": 0.28125, "nrmse": 0.03043118305504322}, {"layer": 54, "field": "recurrent", "exact": false, "max_abs": 0.02237352728843689, "nrmse": 0.02985704317688942}, {"layer": 55, "field": "key", "exact": false, "max_abs": 0.236328125, "nrmse": 0.009901842102408409}, {"layer": 55, "field": "value", "exact": false, "max_abs": 0.6962890625, "nrmse": 0.01423312071710825}, {"layer": 56, "field": "conv", "exact": false, "max_abs": 0.296875, "nrmse": 0.026587411761283875}, {"layer": 56, "field": "recurrent", "exact": false, "max_abs": 0.03574031591415405, "nrmse": 0.023852068930864334}, {"layer": 57, "field": "conv", "exact": false, "max_abs": 0.34375, "nrmse": 0.025188859552145004}, {"layer": 57, "field": "recurrent", "exact": false, "max_abs": 0.04009509086608887, "nrmse": 0.025066766887903214}, {"layer": 58, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.028555678203701973}, {"layer": 58, "field": "recurrent", "exact": false, "max_abs": 0.03210103511810303, "nrmse": 0.022220401093363762}, {"layer": 59, "field": "key", "exact": false, "max_abs": 0.2734375, "nrmse": 0.00999691616743803}, {"layer": 59, "field": "value", "exact": false, "max_abs": 0.79296875, "nrmse": 0.011468426324427128}, {"layer": 60, "field": "conv", "exact": false, "max_abs": 0.5, "nrmse": 0.023503879085183144}, {"layer": 60, "field": "recurrent", "exact": false, "max_abs": 0.04309338331222534, "nrmse": 0.025929922237992287}, {"layer": 61, "field": "conv", "exact": false, "max_abs": 0.34375, "nrmse": 0.026860173791646957}, {"layer": 61, "field": "recurrent", "exact": false, "max_abs": 0.015076026320457458, "nrmse": 0.027001600712537766}, {"layer": 62, "field": "conv", "exact": false, "max_abs": 0.625, "nrmse": 0.021484285593032837}, {"layer": 62, "field": "recurrent", "exact": false, "max_abs": 0.030377447605133057, "nrmse": 0.01887456513941288}, {"layer": 63, "field": "key", "exact": false, "max_abs": 0.25, "nrmse": 0.010604423470795155}, {"layer": 63, "field": "value", "exact": false, "max_abs": 0.890625, "nrmse": 0.006638674531131983}], "buffered_graph_state_diff": []} -{"phase": "step", "index": 49, "position": 197, "teacher_token": 18, "compact": {"ids": [198, 271, 318, 13, 11], "logits": [28.875, 13.75, 13.375, 12.9375, 12.125], "margin": 15.125, "argmax": 198}, "buffered": {"ids": [198, 271, 318, 13, 11], "logits": [28.75, 13.875, 13.3125, 12.9375, 12.0], "margin": 14.875, "argmax": 198}, "graph": {"ids": [198, 271, 318, 13, 11], "logits": [28.75, 13.875, 13.3125, 12.9375, 12.0], "margin": 14.875, "argmax": 198}, "compact_buffered": {"exact": false, "max_abs": 0.296875, "nrmse": 0.015063206665217876}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "compact_buffered_state_diff": [{"layer": 36, "field": "recurrent", "exact": false, "max_abs": 4.76837158203125e-07, "nrmse": 1.3149157496172847e-07}, {"layer": 37, "field": "recurrent", "exact": false, "max_abs": 0.000509403645992279, "nrmse": 0.0005077684763818979}, {"layer": 38, "field": "conv", "exact": false, "max_abs": 0.0625, "nrmse": 0.01115813385695219}, {"layer": 38, "field": "recurrent", "exact": false, "max_abs": 0.005921721458435059, "nrmse": 0.009701229631900787}, {"layer": 39, "field": "key", "exact": false, "max_abs": 0.1875, "nrmse": 0.005803022999316454}, {"layer": 39, "field": "value", "exact": false, "max_abs": 0.21875, "nrmse": 0.007213492877781391}, {"layer": 40, "field": "conv", "exact": false, "max_abs": 0.109375, "nrmse": 0.01927514560520649}, {"layer": 40, "field": "recurrent", "exact": false, "max_abs": 0.008681416511535645, "nrmse": 0.007317723240703344}, {"layer": 41, "field": "conv", "exact": false, "max_abs": 0.140625, "nrmse": 0.02268199436366558}, {"layer": 41, "field": "recurrent", "exact": false, "max_abs": 0.007069885730743408, "nrmse": 0.015799522399902344}, {"layer": 42, "field": "conv", "exact": false, "max_abs": 0.14013671875, "nrmse": 0.025928981602191925}, {"layer": 42, "field": "recurrent", "exact": false, "max_abs": 0.007120072841644287, "nrmse": 0.01577954925596714}, {"layer": 43, "field": "key", "exact": false, "max_abs": 0.3515625, "nrmse": 0.008164042606949806}, {"layer": 43, "field": "value", "exact": false, "max_abs": 0.31640625, "nrmse": 0.009095714427530766}, {"layer": 44, "field": "conv", "exact": false, "max_abs": 0.1484375, "nrmse": 0.02688182331621647}, {"layer": 44, "field": "recurrent", "exact": false, "max_abs": 0.01270640641450882, "nrmse": 0.022448476403951645}, {"layer": 45, "field": "conv", "exact": false, "max_abs": 0.169921875, "nrmse": 0.030589750036597252}, {"layer": 45, "field": "recurrent", "exact": false, "max_abs": 0.012828528881072998, "nrmse": 0.024934958666563034}, {"layer": 46, "field": "conv", "exact": false, "max_abs": 0.203125, "nrmse": 0.029344962909817696}, {"layer": 46, "field": "recurrent", "exact": false, "max_abs": 0.015757262706756592, "nrmse": 0.02360348403453827}, {"layer": 47, "field": "key", "exact": false, "max_abs": 0.55078125, "nrmse": 0.010116048157215118}, {"layer": 47, "field": "value", "exact": false, "max_abs": 0.33203125, "nrmse": 0.012499559670686722}, {"layer": 48, "field": "conv", "exact": false, "max_abs": 0.234375, "nrmse": 0.0316784493625164}, {"layer": 48, "field": "recurrent", "exact": false, "max_abs": 0.011098720133304596, "nrmse": 0.026413297280669212}, {"layer": 49, "field": "conv", "exact": false, "max_abs": 0.23828125, "nrmse": 0.031209388747811317}, {"layer": 49, "field": "recurrent", "exact": false, "max_abs": 0.0115814208984375, "nrmse": 0.029606083407998085}, {"layer": 50, "field": "conv", "exact": false, "max_abs": 0.21875, "nrmse": 0.03133271262049675}, {"layer": 50, "field": "recurrent", "exact": false, "max_abs": 0.00914764404296875, "nrmse": 0.026894578710198402}, {"layer": 51, "field": "key", "exact": false, "max_abs": 0.32421875, "nrmse": 0.0098037701100111}, {"layer": 51, "field": "value", "exact": false, "max_abs": 0.5625, "nrmse": 0.013161475770175457}, {"layer": 52, "field": "conv", "exact": false, "max_abs": 0.3671875, "nrmse": 0.025787953287363052}, {"layer": 52, "field": "recurrent", "exact": false, "max_abs": 0.03308743238449097, "nrmse": 0.02099742367863655}, {"layer": 53, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.026361485943198204}, {"layer": 53, "field": "recurrent", "exact": false, "max_abs": 0.05873894691467285, "nrmse": 0.027970116585493088}, {"layer": 54, "field": "conv", "exact": false, "max_abs": 0.28125, "nrmse": 0.030657056719064713}, {"layer": 54, "field": "recurrent", "exact": false, "max_abs": 0.022321045398712158, "nrmse": 0.02980484813451767}, {"layer": 55, "field": "key", "exact": false, "max_abs": 0.236328125, "nrmse": 0.009982488118112087}, {"layer": 55, "field": "value", "exact": false, "max_abs": 0.6962890625, "nrmse": 0.01440930925309658}, {"layer": 56, "field": "conv", "exact": false, "max_abs": 0.25390625, "nrmse": 0.02662375383079052}, {"layer": 56, "field": "recurrent", "exact": false, "max_abs": 0.03541913628578186, "nrmse": 0.023554449900984764}, {"layer": 57, "field": "conv", "exact": false, "max_abs": 0.34375, "nrmse": 0.02543499879539013}, {"layer": 57, "field": "recurrent", "exact": false, "max_abs": 0.0334928035736084, "nrmse": 0.024602102115750313}, {"layer": 58, "field": "conv", "exact": false, "max_abs": 0.359375, "nrmse": 0.029638901352882385}, {"layer": 58, "field": "recurrent", "exact": false, "max_abs": 0.034371912479400635, "nrmse": 0.021757233887910843}, {"layer": 59, "field": "key", "exact": false, "max_abs": 0.2734375, "nrmse": 0.0101168779656291}, {"layer": 59, "field": "value", "exact": false, "max_abs": 0.79296875, "nrmse": 0.011594745330512524}, {"layer": 60, "field": "conv", "exact": false, "max_abs": 0.375, "nrmse": 0.0223823394626379}, {"layer": 60, "field": "recurrent", "exact": false, "max_abs": 0.043007612228393555, "nrmse": 0.025679506361484528}, {"layer": 61, "field": "conv", "exact": false, "max_abs": 0.25, "nrmse": 0.02574126236140728}, {"layer": 61, "field": "recurrent", "exact": false, "max_abs": 0.015434414148330688, "nrmse": 0.026411041617393494}, {"layer": 62, "field": "conv", "exact": false, "max_abs": 0.625, "nrmse": 0.022041598334908485}, {"layer": 62, "field": "recurrent", "exact": false, "max_abs": 0.03037962317466736, "nrmse": 0.018697844818234444}, {"layer": 63, "field": "key", "exact": false, "max_abs": 0.25, "nrmse": 0.0107706468552351}, {"layer": 63, "field": "value", "exact": false, "max_abs": 0.890625, "nrmse": 0.006731683388352394}], "buffered_graph_state_diff": []} -{"phase": "step", "index": 50, "position": 198, "teacher_token": 198, "compact": {"ids": [9, 332, 262, 348, 9764], "logits": [27.25, 13.0625, 12.6875, 12.4375, 11.125], "margin": 14.1875, "argmax": 9}, "buffered": {"ids": [9, 332, 262, 348, 9764], "logits": [27.25, 13.0625, 12.5625, 12.4375, 11.5625], "margin": 14.1875, "argmax": 9}, "graph": {"ids": [9, 332, 262, 348, 9764], "logits": [27.25, 13.0625, 12.5625, 12.4375, 11.5625], "margin": 14.1875, "argmax": 9}, "compact_buffered": {"exact": false, "max_abs": 0.59375, "nrmse": 0.020686622709035873}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "compact_buffered_state_diff": [{"layer": 36, "field": "recurrent", "exact": false, "max_abs": 4.76837158203125e-07, "nrmse": 1.3158970091353694e-07}, {"layer": 37, "field": "recurrent", "exact": false, "max_abs": 0.0005142092704772949, "nrmse": 0.0005114871310070157}, {"layer": 38, "field": "conv", "exact": false, "max_abs": 0.0625, "nrmse": 0.01208989042788744}, {"layer": 38, "field": "recurrent", "exact": false, "max_abs": 0.00588718056678772, "nrmse": 0.01023452915251255}, {"layer": 39, "field": "key", "exact": false, "max_abs": 0.1875, "nrmse": 0.0058858259581029415}, {"layer": 39, "field": "value", "exact": false, "max_abs": 0.21875, "nrmse": 0.007345929741859436}, {"layer": 40, "field": "conv", "exact": false, "max_abs": 0.109375, "nrmse": 0.02061852440237999}, {"layer": 40, "field": "recurrent", "exact": false, "max_abs": 0.008969902992248535, "nrmse": 0.007343911100178957}, {"layer": 41, "field": "conv", "exact": false, "max_abs": 0.140625, "nrmse": 0.025127213448286057}, {"layer": 41, "field": "recurrent", "exact": false, "max_abs": 0.007066607475280762, "nrmse": 0.01579505205154419}, {"layer": 42, "field": "conv", "exact": false, "max_abs": 0.140625, "nrmse": 0.027988916262984276}, {"layer": 42, "field": "recurrent", "exact": false, "max_abs": 0.008296683430671692, "nrmse": 0.01642332412302494}, {"layer": 43, "field": "key", "exact": false, "max_abs": 0.3515625, "nrmse": 0.008379901759326458}, {"layer": 43, "field": "value", "exact": false, "max_abs": 0.31640625, "nrmse": 0.009320504032075405}, {"layer": 44, "field": "conv", "exact": false, "max_abs": 0.1875, "nrmse": 0.02887585759162903}, {"layer": 44, "field": "recurrent", "exact": false, "max_abs": 0.012667641043663025, "nrmse": 0.02246224693953991}, {"layer": 45, "field": "conv", "exact": false, "max_abs": 0.169921875, "nrmse": 0.031965069472789764}, {"layer": 45, "field": "recurrent", "exact": false, "max_abs": 0.012817800045013428, "nrmse": 0.025201458483934402}, {"layer": 46, "field": "conv", "exact": false, "max_abs": 0.203125, "nrmse": 0.030874762684106827}, {"layer": 46, "field": "recurrent", "exact": false, "max_abs": 0.015816450119018555, "nrmse": 0.023624222725629807}, {"layer": 47, "field": "key", "exact": false, "max_abs": 0.55078125, "nrmse": 0.010266491211950779}, {"layer": 47, "field": "value", "exact": false, "max_abs": 0.33203125, "nrmse": 0.012808382511138916}, {"layer": 48, "field": "conv", "exact": false, "max_abs": 0.2421875, "nrmse": 0.03439188003540039}, {"layer": 48, "field": "recurrent", "exact": false, "max_abs": 0.01094353199005127, "nrmse": 0.026592709124088287}, {"layer": 49, "field": "conv", "exact": false, "max_abs": 0.375, "nrmse": 0.034397028386592865}, {"layer": 49, "field": "recurrent", "exact": false, "max_abs": 0.011068172752857208, "nrmse": 0.030526747927069664}, {"layer": 50, "field": "conv", "exact": false, "max_abs": 0.9375, "nrmse": 0.03397870063781738}, {"layer": 50, "field": "recurrent", "exact": false, "max_abs": 0.008620679378509521, "nrmse": 0.027402248233556747}, {"layer": 51, "field": "key", "exact": false, "max_abs": 0.32421875, "nrmse": 0.010026687756180763}, {"layer": 51, "field": "value", "exact": false, "max_abs": 0.5625, "nrmse": 0.013419936411082745}, {"layer": 52, "field": "conv", "exact": false, "max_abs": 0.3671875, "nrmse": 0.02804662100970745}, {"layer": 52, "field": "recurrent", "exact": false, "max_abs": 0.033114492893218994, "nrmse": 0.02125123143196106}, {"layer": 53, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.02892323210835457}, {"layer": 53, "field": "recurrent", "exact": false, "max_abs": 0.0587388277053833, "nrmse": 0.028482750058174133}, {"layer": 54, "field": "conv", "exact": false, "max_abs": 0.283203125, "nrmse": 0.03352605178952217}, {"layer": 54, "field": "recurrent", "exact": false, "max_abs": 0.022816628217697144, "nrmse": 0.030430031940340996}, {"layer": 55, "field": "key", "exact": false, "max_abs": 0.236328125, "nrmse": 0.010185427963733673}, {"layer": 55, "field": "value", "exact": false, "max_abs": 0.75, "nrmse": 0.014687738381326199}, {"layer": 56, "field": "conv", "exact": false, "max_abs": 0.34375, "nrmse": 0.02931254357099533}, {"layer": 56, "field": "recurrent", "exact": false, "max_abs": 0.03569447994232178, "nrmse": 0.02390500158071518}, {"layer": 57, "field": "conv", "exact": false, "max_abs": 0.34375, "nrmse": 0.02674846537411213}, {"layer": 57, "field": "recurrent", "exact": false, "max_abs": 0.03053702786564827, "nrmse": 0.024912575259804726}, {"layer": 58, "field": "conv", "exact": false, "max_abs": 0.359375, "nrmse": 0.03120892494916916}, {"layer": 58, "field": "recurrent", "exact": false, "max_abs": 0.0338592529296875, "nrmse": 0.022211899980902672}, {"layer": 59, "field": "key", "exact": false, "max_abs": 0.2734375, "nrmse": 0.010259965434670448}, {"layer": 59, "field": "value", "exact": false, "max_abs": 0.79296875, "nrmse": 0.011724372394382954}, {"layer": 60, "field": "conv", "exact": false, "max_abs": 0.40625, "nrmse": 0.023675458505749702}, {"layer": 60, "field": "recurrent", "exact": false, "max_abs": 0.042922377586364746, "nrmse": 0.02586599998176098}, {"layer": 61, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.026926545426249504}, {"layer": 61, "field": "recurrent", "exact": false, "max_abs": 0.01651996374130249, "nrmse": 0.026591280475258827}, {"layer": 62, "field": "conv", "exact": false, "max_abs": 0.6875, "nrmse": 0.024061046540737152}, {"layer": 62, "field": "recurrent", "exact": false, "max_abs": 0.029678761959075928, "nrmse": 0.01880800351500511}, {"layer": 63, "field": "key", "exact": false, "max_abs": 0.25, "nrmse": 0.011028052307665348}, {"layer": 63, "field": "value", "exact": false, "max_abs": 0.890625, "nrmse": 0.006884791888296604}], "buffered_graph_state_diff": []} -{"phase": "step", "index": 51, "position": 199, "teacher_token": 9, "compact": {"ids": [256, 3991, 262, 220, 6140], "logits": [31.0, 16.875, 16.75, 15.25, 14.25], "margin": 14.125, "argmax": 256}, "buffered": {"ids": [256, 3991, 262, 220, 6140], "logits": [31.125, 16.875, 16.75, 15.125, 14.25], "margin": 14.25, "argmax": 256}, "graph": {"ids": [256, 3991, 262, 220, 6140], "logits": [31.125, 16.875, 16.75, 15.125, 14.25], "margin": 14.25, "argmax": 256}, "compact_buffered": {"exact": false, "max_abs": 0.4296875, "nrmse": 0.01646372675895691}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "compact_buffered_state_diff": [{"layer": 36, "field": "recurrent", "exact": false, "max_abs": 4.76837158203125e-07, "nrmse": 1.3253840336346911e-07}, {"layer": 37, "field": "recurrent", "exact": false, "max_abs": 0.0005092956125736237, "nrmse": 0.0005217301077209413}, {"layer": 38, "field": "conv", "exact": false, "max_abs": 0.0625, "nrmse": 0.011600391007959843}, {"layer": 38, "field": "recurrent", "exact": false, "max_abs": 0.005880922079086304, "nrmse": 0.00993130262941122}, {"layer": 39, "field": "key", "exact": false, "max_abs": 0.1875, "nrmse": 0.005937259178608656}, {"layer": 39, "field": "value", "exact": false, "max_abs": 0.21875, "nrmse": 0.0074109178967773914}, {"layer": 40, "field": "conv", "exact": false, "max_abs": 0.109375, "nrmse": 0.02090422809123993}, {"layer": 40, "field": "recurrent", "exact": false, "max_abs": 0.008839964866638184, "nrmse": 0.007192197255790234}, {"layer": 41, "field": "conv", "exact": false, "max_abs": 0.140625, "nrmse": 0.025026608258485794}, {"layer": 41, "field": "recurrent", "exact": false, "max_abs": 0.007077261805534363, "nrmse": 0.015870295464992523}, {"layer": 42, "field": "conv", "exact": false, "max_abs": 0.140625, "nrmse": 0.028004510328173637}, {"layer": 42, "field": "recurrent", "exact": false, "max_abs": 0.007171124219894409, "nrmse": 0.016206908971071243}, {"layer": 43, "field": "key", "exact": false, "max_abs": 0.3515625, "nrmse": 0.008484926074743271}, {"layer": 43, "field": "value", "exact": false, "max_abs": 0.31640625, "nrmse": 0.009429569356143475}, {"layer": 44, "field": "conv", "exact": false, "max_abs": 0.1875, "nrmse": 0.02861226163804531}, {"layer": 44, "field": "recurrent", "exact": false, "max_abs": 0.012679442763328552, "nrmse": 0.022326556965708733}, {"layer": 45, "field": "conv", "exact": false, "max_abs": 0.169921875, "nrmse": 0.03217114508152008}, {"layer": 45, "field": "recurrent", "exact": false, "max_abs": 0.012818217277526855, "nrmse": 0.025077078491449356}, {"layer": 46, "field": "conv", "exact": false, "max_abs": 0.203125, "nrmse": 0.031137721613049507}, {"layer": 46, "field": "recurrent", "exact": false, "max_abs": 0.015831947326660156, "nrmse": 0.023826951161026955}, {"layer": 47, "field": "key", "exact": false, "max_abs": 0.55078125, "nrmse": 0.010358838364481926}, {"layer": 47, "field": "value", "exact": false, "max_abs": 0.33203125, "nrmse": 0.012982863932847977}, {"layer": 48, "field": "conv", "exact": false, "max_abs": 0.2421875, "nrmse": 0.03518145903944969}, {"layer": 48, "field": "recurrent", "exact": false, "max_abs": 0.011071383953094482, "nrmse": 0.026610538363456726}, {"layer": 49, "field": "conv", "exact": false, "max_abs": 0.375, "nrmse": 0.034714389592409134}, {"layer": 49, "field": "recurrent", "exact": false, "max_abs": 0.01116955280303955, "nrmse": 0.031073210760951042}, {"layer": 50, "field": "conv", "exact": false, "max_abs": 0.9375, "nrmse": 0.03528732433915138}, {"layer": 50, "field": "recurrent", "exact": false, "max_abs": 0.01283868495374918, "nrmse": 0.02726105786859989}, {"layer": 51, "field": "key", "exact": false, "max_abs": 0.32421875, "nrmse": 0.010198844596743584}, {"layer": 51, "field": "value", "exact": false, "max_abs": 0.5625, "nrmse": 0.01363171637058258}, {"layer": 52, "field": "conv", "exact": false, "max_abs": 0.30078125, "nrmse": 0.029102472588419914}, {"layer": 52, "field": "recurrent", "exact": false, "max_abs": 0.03310203552246094, "nrmse": 0.021217428147792816}, {"layer": 53, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.029926950111985207}, {"layer": 53, "field": "recurrent", "exact": false, "max_abs": 0.058866024017333984, "nrmse": 0.028597790747880936}, {"layer": 54, "field": "conv", "exact": false, "max_abs": 0.283203125, "nrmse": 0.034211788326501846}, {"layer": 54, "field": "recurrent", "exact": false, "max_abs": 0.02639526128768921, "nrmse": 0.03073194995522499}, {"layer": 55, "field": "key", "exact": false, "max_abs": 0.236328125, "nrmse": 0.010397861711680889}, {"layer": 55, "field": "value", "exact": false, "max_abs": 0.75, "nrmse": 0.014882992021739483}, {"layer": 56, "field": "conv", "exact": false, "max_abs": 0.34375, "nrmse": 0.02940717712044716}, {"layer": 56, "field": "recurrent", "exact": false, "max_abs": 0.03571292757987976, "nrmse": 0.024074172601103783}, {"layer": 57, "field": "conv", "exact": false, "max_abs": 0.34375, "nrmse": 0.026760917156934738}, {"layer": 57, "field": "recurrent", "exact": false, "max_abs": 0.029140233993530273, "nrmse": 0.025008153170347214}, {"layer": 58, "field": "conv", "exact": false, "max_abs": 0.359375, "nrmse": 0.03110630251467228}, {"layer": 58, "field": "recurrent", "exact": false, "max_abs": 0.03296595811843872, "nrmse": 0.022277381271123886}, {"layer": 59, "field": "key", "exact": false, "max_abs": 0.2734375, "nrmse": 0.010400637984275818}, {"layer": 59, "field": "value", "exact": false, "max_abs": 0.79296875, "nrmse": 0.011902354657649994}, {"layer": 60, "field": "conv", "exact": false, "max_abs": 0.40625, "nrmse": 0.023691941052675247}, {"layer": 60, "field": "recurrent", "exact": false, "max_abs": 0.04273676872253418, "nrmse": 0.02604602836072445}, {"layer": 61, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.02667301520705223}, {"layer": 61, "field": "recurrent", "exact": false, "max_abs": 0.015707924962043762, "nrmse": 0.026406094431877136}, {"layer": 62, "field": "conv", "exact": false, "max_abs": 0.6875, "nrmse": 0.024209894239902496}, {"layer": 62, "field": "recurrent", "exact": false, "max_abs": 0.028339684009552002, "nrmse": 0.018845807760953903}, {"layer": 63, "field": "key", "exact": false, "max_abs": 0.25, "nrmse": 0.01122361235320568}, {"layer": 63, "field": "value", "exact": false, "max_abs": 0.890625, "nrmse": 0.007027504034340382}], "buffered_graph_state_diff": []} -{"phase": "step", "index": 52, "position": 200, "teacher_token": 256, "compact": {"ids": [3991, 104542, 2763, 56578, 6140], "logits": [30.25, 19.375, 19.25, 17.75, 17.5], "margin": 10.875, "argmax": 3991}, "buffered": {"ids": [3991, 104542, 2763, 56578, 1478], "logits": [30.125, 19.25, 19.25, 17.875, 17.5], "margin": 10.875, "argmax": 3991}, "graph": {"ids": [3991, 104542, 2763, 56578, 1478], "logits": [30.125, 19.25, 19.25, 17.875, 17.5], "margin": 10.875, "argmax": 3991}, "compact_buffered": {"exact": false, "max_abs": 0.46875, "nrmse": 0.022283852100372314}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "compact_buffered_state_diff": [{"layer": 36, "field": "recurrent", "exact": false, "max_abs": 4.76837158203125e-07, "nrmse": 1.319066456062501e-07}, {"layer": 37, "field": "recurrent", "exact": false, "max_abs": 0.0005031824111938477, "nrmse": 0.0005096098757348955}, {"layer": 38, "field": "conv", "exact": false, "max_abs": 0.0703125, "nrmse": 0.011773967184126377}, {"layer": 38, "field": "recurrent", "exact": false, "max_abs": 0.006268501281738281, "nrmse": 0.01009841077029705}, {"layer": 39, "field": "key", "exact": false, "max_abs": 0.1875, "nrmse": 0.006012330297380686}, {"layer": 39, "field": "value", "exact": false, "max_abs": 0.21875, "nrmse": 0.007543704938143492}, {"layer": 40, "field": "conv", "exact": false, "max_abs": 0.125, "nrmse": 0.021534770727157593}, {"layer": 40, "field": "recurrent", "exact": false, "max_abs": 0.008652091026306152, "nrmse": 0.007507008966058493}, {"layer": 41, "field": "conv", "exact": false, "max_abs": 0.125, "nrmse": 0.024891367182135582}, {"layer": 41, "field": "recurrent", "exact": false, "max_abs": 0.007072247564792633, "nrmse": 0.0157387163490057}, {"layer": 42, "field": "conv", "exact": false, "max_abs": 0.140625, "nrmse": 0.027006546035408974}, {"layer": 42, "field": "recurrent", "exact": false, "max_abs": 0.0071128010749816895, "nrmse": 0.015922509133815765}, {"layer": 43, "field": "key", "exact": false, "max_abs": 0.3515625, "nrmse": 0.008578155189752579}, {"layer": 43, "field": "value", "exact": false, "max_abs": 0.31640625, "nrmse": 0.009587648324668407}, {"layer": 44, "field": "conv", "exact": false, "max_abs": 0.1875, "nrmse": 0.027553053572773933}, {"layer": 44, "field": "recurrent", "exact": false, "max_abs": 0.012757569551467896, "nrmse": 0.02230980433523655}, {"layer": 45, "field": "conv", "exact": false, "max_abs": 0.171875, "nrmse": 0.031583141535520554}, {"layer": 45, "field": "recurrent", "exact": false, "max_abs": 0.012814044952392578, "nrmse": 0.024990417063236237}, {"layer": 46, "field": "conv", "exact": false, "max_abs": 0.177734375, "nrmse": 0.030873095616698265}, {"layer": 46, "field": "recurrent", "exact": false, "max_abs": 0.015829503536224365, "nrmse": 0.023538023233413696}, {"layer": 47, "field": "key", "exact": false, "max_abs": 0.55078125, "nrmse": 0.010493661276996136}, {"layer": 47, "field": "value", "exact": false, "max_abs": 0.33203125, "nrmse": 0.01322692260146141}, {"layer": 48, "field": "conv", "exact": false, "max_abs": 0.2421875, "nrmse": 0.03426394984126091}, {"layer": 48, "field": "recurrent", "exact": false, "max_abs": 0.011104077100753784, "nrmse": 0.02718745917081833}, {"layer": 49, "field": "conv", "exact": false, "max_abs": 0.375, "nrmse": 0.03453969955444336}, {"layer": 49, "field": "recurrent", "exact": false, "max_abs": 0.01148080825805664, "nrmse": 0.030469676479697227}, {"layer": 50, "field": "conv", "exact": false, "max_abs": 0.9375, "nrmse": 0.03476759418845177}, {"layer": 50, "field": "recurrent", "exact": false, "max_abs": 0.011809766292572021, "nrmse": 0.02673504874110222}, {"layer": 51, "field": "key", "exact": false, "max_abs": 0.32421875, "nrmse": 0.01033940538764}, {"layer": 51, "field": "value", "exact": false, "max_abs": 0.5625, "nrmse": 0.013890130445361137}, {"layer": 52, "field": "conv", "exact": false, "max_abs": 0.359375, "nrmse": 0.02961452305316925}, {"layer": 52, "field": "recurrent", "exact": false, "max_abs": 0.033263564109802246, "nrmse": 0.021549789234995842}, {"layer": 53, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.030479053035378456}, {"layer": 53, "field": "recurrent", "exact": false, "max_abs": 0.05896735191345215, "nrmse": 0.02878691256046295}, {"layer": 54, "field": "conv", "exact": false, "max_abs": 0.33203125, "nrmse": 0.034856781363487244}, {"layer": 54, "field": "recurrent", "exact": false, "max_abs": 0.022786781191825867, "nrmse": 0.03096703067421913}, {"layer": 55, "field": "key", "exact": false, "max_abs": 0.236328125, "nrmse": 0.010566708631813526}, {"layer": 55, "field": "value", "exact": false, "max_abs": 0.75, "nrmse": 0.01526200957596302}, {"layer": 56, "field": "conv", "exact": false, "max_abs": 0.34375, "nrmse": 0.03064442053437233}, {"layer": 56, "field": "recurrent", "exact": false, "max_abs": 0.0357033908367157, "nrmse": 0.02419482357800007}, {"layer": 57, "field": "conv", "exact": false, "max_abs": 0.4375, "nrmse": 0.02741914428770542}, {"layer": 57, "field": "recurrent", "exact": false, "max_abs": 0.026030734181404114, "nrmse": 0.025933030992746353}, {"layer": 58, "field": "conv", "exact": false, "max_abs": 0.40625, "nrmse": 0.03238258510828018}, {"layer": 58, "field": "recurrent", "exact": false, "max_abs": 0.0439608097076416, "nrmse": 0.022899361327290535}, {"layer": 59, "field": "key", "exact": false, "max_abs": 0.2734375, "nrmse": 0.010613100603222847}, {"layer": 59, "field": "value", "exact": false, "max_abs": 0.79296875, "nrmse": 0.012259695678949356}, {"layer": 60, "field": "conv", "exact": false, "max_abs": 0.5, "nrmse": 0.024789992719888687}, {"layer": 60, "field": "recurrent", "exact": false, "max_abs": 0.04286155104637146, "nrmse": 0.02628001570701599}, {"layer": 61, "field": "conv", "exact": false, "max_abs": 0.328125, "nrmse": 0.027913672849535942}, {"layer": 61, "field": "recurrent", "exact": false, "max_abs": 0.018038064241409302, "nrmse": 0.027231737971305847}, {"layer": 62, "field": "conv", "exact": false, "max_abs": 0.6875, "nrmse": 0.02502923458814621}, {"layer": 62, "field": "recurrent", "exact": false, "max_abs": 0.027950584888458252, "nrmse": 0.01915309950709343}, {"layer": 63, "field": "key", "exact": false, "max_abs": 0.25, "nrmse": 0.011472190730273724}, {"layer": 63, "field": "value", "exact": false, "max_abs": 0.890625, "nrmse": 0.007184951100498438}], "buffered_graph_state_diff": []} -{"phase": "step", "index": 53, "position": 201, "teacher_token": 3991, "compact": {"ids": [4022, 18016, 7397, 5626, 6105], "logits": [31.25, 16.875, 15.375, 15.375, 14.75], "margin": 14.375, "argmax": 4022}, "buffered": {"ids": [4022, 18016, 5626, 7397, 6105], "logits": [31.0, 16.75, 15.375, 15.3125, 14.6875], "margin": 14.25, "argmax": 4022}, "graph": {"ids": [4022, 18016, 5626, 7397, 6105], "logits": [31.0, 16.75, 15.375, 15.3125, 14.6875], "margin": 14.25, "argmax": 4022}, "compact_buffered": {"exact": false, "max_abs": 0.328125, "nrmse": 0.013677905313670635}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "compact_buffered_state_diff": [{"layer": 36, "field": "recurrent", "exact": false, "max_abs": 4.76837158203125e-07, "nrmse": 1.319779698860657e-07}, {"layer": 37, "field": "recurrent", "exact": false, "max_abs": 0.0004995986819267273, "nrmse": 0.000508722267113626}, {"layer": 38, "field": "conv", "exact": false, "max_abs": 0.0703125, "nrmse": 0.011922214180231094}, {"layer": 38, "field": "recurrent", "exact": false, "max_abs": 0.005865544080734253, "nrmse": 0.009955604560673237}, {"layer": 39, "field": "key", "exact": false, "max_abs": 0.1875, "nrmse": 0.006073134019970894}, {"layer": 39, "field": "value", "exact": false, "max_abs": 0.21875, "nrmse": 0.007628638297319412}, {"layer": 40, "field": "conv", "exact": false, "max_abs": 0.125, "nrmse": 0.022057626396417618}, {"layer": 40, "field": "recurrent", "exact": false, "max_abs": 0.008784770965576172, "nrmse": 0.007665571290999651}, {"layer": 41, "field": "conv", "exact": false, "max_abs": 0.125, "nrmse": 0.025175394490361214}, {"layer": 41, "field": "recurrent", "exact": false, "max_abs": 0.007067866623401642, "nrmse": 0.016003336757421494}, {"layer": 42, "field": "conv", "exact": false, "max_abs": 0.140625, "nrmse": 0.027594629675149918}, {"layer": 42, "field": "recurrent", "exact": false, "max_abs": 0.007452443242073059, "nrmse": 0.016283903270959854}, {"layer": 43, "field": "key", "exact": false, "max_abs": 0.3515625, "nrmse": 0.008662800304591656}, {"layer": 43, "field": "value", "exact": false, "max_abs": 0.31640625, "nrmse": 0.009723523631691933}, {"layer": 44, "field": "conv", "exact": false, "max_abs": 0.1875, "nrmse": 0.02881898172199726}, {"layer": 44, "field": "recurrent", "exact": false, "max_abs": 0.012746945023536682, "nrmse": 0.022472480311989784}, {"layer": 45, "field": "conv", "exact": false, "max_abs": 0.18701171875, "nrmse": 0.032350409775972366}, {"layer": 45, "field": "recurrent", "exact": false, "max_abs": 0.01282113790512085, "nrmse": 0.025328190997242928}, {"layer": 46, "field": "conv", "exact": false, "max_abs": 0.177734375, "nrmse": 0.031862907111644745}, {"layer": 46, "field": "recurrent", "exact": false, "max_abs": 0.01580911874771118, "nrmse": 0.024189284071326256}, {"layer": 47, "field": "key", "exact": false, "max_abs": 0.55078125, "nrmse": 0.010631020180881023}, {"layer": 47, "field": "value", "exact": false, "max_abs": 0.33203125, "nrmse": 0.013457716442644596}, {"layer": 48, "field": "conv", "exact": false, "max_abs": 0.2421875, "nrmse": 0.03566179797053337}, {"layer": 48, "field": "recurrent", "exact": false, "max_abs": 0.011082150042057037, "nrmse": 0.027208447456359863}, {"layer": 49, "field": "conv", "exact": false, "max_abs": 0.375, "nrmse": 0.035935308784246445}, {"layer": 49, "field": "recurrent", "exact": false, "max_abs": 0.011299729347229004, "nrmse": 0.03088853694498539}, {"layer": 50, "field": "conv", "exact": false, "max_abs": 0.9375, "nrmse": 0.03556199371814728}, {"layer": 50, "field": "recurrent", "exact": false, "max_abs": 0.009479761123657227, "nrmse": 0.02692257985472679}, {"layer": 51, "field": "key", "exact": false, "max_abs": 0.32421875, "nrmse": 0.010452758520841599}, {"layer": 51, "field": "value", "exact": false, "max_abs": 0.5625, "nrmse": 0.014098710380494595}, {"layer": 52, "field": "conv", "exact": false, "max_abs": 0.375, "nrmse": 0.03064228966832161}, {"layer": 52, "field": "recurrent", "exact": false, "max_abs": 0.03319364786148071, "nrmse": 0.021574515849351883}, {"layer": 53, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.03092389926314354}, {"layer": 53, "field": "recurrent", "exact": false, "max_abs": 0.05888640880584717, "nrmse": 0.028591549023985863}, {"layer": 54, "field": "conv", "exact": false, "max_abs": 0.33203125, "nrmse": 0.03476135805249214}, {"layer": 54, "field": "recurrent", "exact": false, "max_abs": 0.02139270305633545, "nrmse": 0.03063247725367546}, {"layer": 55, "field": "key", "exact": false, "max_abs": 0.236328125, "nrmse": 0.010682663880288601}, {"layer": 55, "field": "value", "exact": false, "max_abs": 0.75, "nrmse": 0.015376468189060688}, {"layer": 56, "field": "conv", "exact": false, "max_abs": 0.34375, "nrmse": 0.03058859519660473}, {"layer": 56, "field": "recurrent", "exact": false, "max_abs": 0.03550940752029419, "nrmse": 0.023881947621703148}, {"layer": 57, "field": "conv", "exact": false, "max_abs": 0.4375, "nrmse": 0.026975717395544052}, {"layer": 57, "field": "recurrent", "exact": false, "max_abs": 0.026249736547470093, "nrmse": 0.02483336813747883}, {"layer": 58, "field": "conv", "exact": false, "max_abs": 0.40625, "nrmse": 0.03193490207195282}, {"layer": 58, "field": "recurrent", "exact": false, "max_abs": 0.03246796131134033, "nrmse": 0.022062236443161964}, {"layer": 59, "field": "key", "exact": false, "max_abs": 0.2734375, "nrmse": 0.010753984563052654}, {"layer": 59, "field": "value", "exact": false, "max_abs": 0.79296875, "nrmse": 0.01238974928855896}, {"layer": 60, "field": "conv", "exact": false, "max_abs": 0.5, "nrmse": 0.02524341642856598}, {"layer": 60, "field": "recurrent", "exact": false, "max_abs": 0.04264092445373535, "nrmse": 0.02646055445075035}, {"layer": 61, "field": "conv", "exact": false, "max_abs": 0.328125, "nrmse": 0.027795296162366867}, {"layer": 61, "field": "recurrent", "exact": false, "max_abs": 0.018083348870277405, "nrmse": 0.026786895468831062}, {"layer": 62, "field": "conv", "exact": false, "max_abs": 0.6875, "nrmse": 0.02479003556072712}, {"layer": 62, "field": "recurrent", "exact": false, "max_abs": 0.02944357693195343, "nrmse": 0.019345348700881004}, {"layer": 63, "field": "key", "exact": false, "max_abs": 0.25, "nrmse": 0.01161795761436224}, {"layer": 63, "field": "value", "exact": false, "max_abs": 0.890625, "nrmse": 0.007257561665028334}], "buffered_graph_state_diff": []} -{"phase": "step", "index": 54, "position": 202, "teacher_token": 4022, "compact": {"ids": [7397, 17629, 97255, 318, 220], "logits": [31.0, 17.625, 15.625, 15.625, 13.75], "margin": 13.375, "argmax": 7397}, "buffered": {"ids": [7397, 17629, 318, 97255, 25], "logits": [31.0, 17.625, 15.625, 15.5, 13.6875], "margin": 13.375, "argmax": 7397}, "graph": {"ids": [7397, 17629, 318, 97255, 25], "logits": [31.0, 17.625, 15.625, 15.5, 13.6875], "margin": 13.375, "argmax": 7397}, "compact_buffered": {"exact": false, "max_abs": 0.375, "nrmse": 0.01508445292711258}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "compact_buffered_state_diff": [{"layer": 36, "field": "recurrent", "exact": false, "max_abs": 4.76837158203125e-07, "nrmse": 1.3235050744242471e-07}, {"layer": 37, "field": "recurrent", "exact": false, "max_abs": 0.0004965178668498993, "nrmse": 0.0005099096451885998}, {"layer": 38, "field": "conv", "exact": false, "max_abs": 0.0703125, "nrmse": 0.010553959757089615}, {"layer": 38, "field": "recurrent", "exact": false, "max_abs": 0.005863547325134277, "nrmse": 0.009765522554516792}, {"layer": 39, "field": "key", "exact": false, "max_abs": 0.1875, "nrmse": 0.006101346109062433}, {"layer": 39, "field": "value", "exact": false, "max_abs": 0.21875, "nrmse": 0.007699910551309586}, {"layer": 40, "field": "conv", "exact": false, "max_abs": 0.125, "nrmse": 0.020772278308868408}, {"layer": 40, "field": "recurrent", "exact": false, "max_abs": 0.008125662803649902, "nrmse": 0.007432177662849426}, {"layer": 41, "field": "conv", "exact": false, "max_abs": 0.140625, "nrmse": 0.023406976833939552}, {"layer": 41, "field": "recurrent", "exact": false, "max_abs": 0.0070663318037986755, "nrmse": 0.015899958088994026}, {"layer": 42, "field": "conv", "exact": false, "max_abs": 0.125, "nrmse": 0.025561805814504623}, {"layer": 42, "field": "recurrent", "exact": false, "max_abs": 0.00919320434331894, "nrmse": 0.01638774201273918}, {"layer": 43, "field": "key", "exact": false, "max_abs": 0.3515625, "nrmse": 0.008733434602618217}, {"layer": 43, "field": "value", "exact": false, "max_abs": 0.31640625, "nrmse": 0.009825987741351128}, {"layer": 44, "field": "conv", "exact": false, "max_abs": 0.1875, "nrmse": 0.02770818956196308}, {"layer": 44, "field": "recurrent", "exact": false, "max_abs": 0.012764304876327515, "nrmse": 0.022566096857190132}, {"layer": 45, "field": "conv", "exact": false, "max_abs": 0.18701171875, "nrmse": 0.03175366297364235}, {"layer": 45, "field": "recurrent", "exact": false, "max_abs": 0.012817621231079102, "nrmse": 0.02571115829050541}, {"layer": 46, "field": "conv", "exact": false, "max_abs": 0.17333984375, "nrmse": 0.031077900901436806}, {"layer": 46, "field": "recurrent", "exact": false, "max_abs": 0.015810608863830566, "nrmse": 0.024266140535473824}, {"layer": 47, "field": "key", "exact": false, "max_abs": 0.55078125, "nrmse": 0.010689652524888515}, {"layer": 47, "field": "value", "exact": false, "max_abs": 0.33203125, "nrmse": 0.013584375381469727}, {"layer": 48, "field": "conv", "exact": false, "max_abs": 0.23828125, "nrmse": 0.03425855189561844}, {"layer": 48, "field": "recurrent", "exact": false, "max_abs": 0.011090777814388275, "nrmse": 0.02737320028245449}, {"layer": 49, "field": "conv", "exact": false, "max_abs": 0.28125, "nrmse": 0.03377564251422882}, {"layer": 49, "field": "recurrent", "exact": false, "max_abs": 0.011085867881774902, "nrmse": 0.03105716034770012}, {"layer": 50, "field": "conv", "exact": false, "max_abs": 0.25, "nrmse": 0.033164557069540024}, {"layer": 50, "field": "recurrent", "exact": false, "max_abs": 0.007362760603427887, "nrmse": 0.027058053761720657}, {"layer": 51, "field": "key", "exact": false, "max_abs": 0.32421875, "nrmse": 0.010523946024477482}, {"layer": 51, "field": "value", "exact": false, "max_abs": 0.5625, "nrmse": 0.01422532182186842}, {"layer": 52, "field": "conv", "exact": false, "max_abs": 0.375, "nrmse": 0.028679464012384415}, {"layer": 52, "field": "recurrent", "exact": false, "max_abs": 0.033164799213409424, "nrmse": 0.021425114944577217}, {"layer": 53, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.028752122074365616}, {"layer": 53, "field": "recurrent", "exact": false, "max_abs": 0.058939576148986816, "nrmse": 0.02819313295185566}, {"layer": 54, "field": "conv", "exact": false, "max_abs": 0.33203125, "nrmse": 0.031657952815294266}, {"layer": 54, "field": "recurrent", "exact": false, "max_abs": 0.02139081060886383, "nrmse": 0.03023090586066246}, {"layer": 55, "field": "key", "exact": false, "max_abs": 0.236328125, "nrmse": 0.010742992162704468}, {"layer": 55, "field": "value", "exact": false, "max_abs": 0.75, "nrmse": 0.015464449301362038}, {"layer": 56, "field": "conv", "exact": false, "max_abs": 0.296875, "nrmse": 0.027686920017004013}, {"layer": 56, "field": "recurrent", "exact": false, "max_abs": 0.03581330180168152, "nrmse": 0.02355886995792389}, {"layer": 57, "field": "conv", "exact": false, "max_abs": 0.4375, "nrmse": 0.02481941133737564}, {"layer": 57, "field": "recurrent", "exact": false, "max_abs": 0.026186078786849976, "nrmse": 0.024456949904561043}, {"layer": 58, "field": "conv", "exact": false, "max_abs": 0.40625, "nrmse": 0.029491310939192772}, {"layer": 58, "field": "recurrent", "exact": false, "max_abs": 0.03243190050125122, "nrmse": 0.021562770009040833}, {"layer": 59, "field": "key", "exact": false, "max_abs": 0.2734375, "nrmse": 0.010817459784448147}, {"layer": 59, "field": "value", "exact": false, "max_abs": 0.79296875, "nrmse": 0.012512472458183765}, {"layer": 60, "field": "conv", "exact": false, "max_abs": 0.5, "nrmse": 0.023561742156744003}, {"layer": 60, "field": "recurrent", "exact": false, "max_abs": 0.04287591576576233, "nrmse": 0.026260752230882645}, {"layer": 61, "field": "conv", "exact": false, "max_abs": 0.328125, "nrmse": 0.025794154033064842}, {"layer": 61, "field": "recurrent", "exact": false, "max_abs": 0.018225297331809998, "nrmse": 0.02640463598072529}, {"layer": 62, "field": "conv", "exact": false, "max_abs": 0.375, "nrmse": 0.02378351055085659}, {"layer": 62, "field": "recurrent", "exact": false, "max_abs": 0.02944815158843994, "nrmse": 0.019239457324147224}, {"layer": 63, "field": "key", "exact": false, "max_abs": 0.25, "nrmse": 0.011762038804590702}, {"layer": 63, "field": "value", "exact": false, "max_abs": 0.890625, "nrmse": 0.00731819961220026}], "buffered_graph_state_diff": []} -{"phase": "step", "index": 55, "position": 203, "teacher_token": 7397, "compact": {"ids": [25, 198, 220, 283, 318], "logits": [26.25, 12.6875, 12.4375, 11.4375, 11.3125], "margin": 13.5625, "argmax": 25}, "buffered": {"ids": [25, 198, 220, 283, 318], "logits": [26.5, 12.875, 12.6875, 11.4375, 11.125], "margin": 13.625, "argmax": 25}, "graph": {"ids": [25, 198, 220, 283, 318], "logits": [26.5, 12.875, 12.6875, 11.4375, 11.125], "margin": 13.625, "argmax": 25}, "compact_buffered": {"exact": false, "max_abs": 0.32861328125, "nrmse": 0.013355189003050327}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "compact_buffered_state_diff": [{"layer": 36, "field": "recurrent", "exact": false, "max_abs": 4.76837158203125e-07, "nrmse": 1.3239011309451598e-07}, {"layer": 37, "field": "recurrent", "exact": false, "max_abs": 0.0004956163465976715, "nrmse": 0.0005108244367875159}, {"layer": 38, "field": "conv", "exact": false, "max_abs": 0.0703125, "nrmse": 0.010878868401050568}, {"layer": 38, "field": "recurrent", "exact": false, "max_abs": 0.005861073732376099, "nrmse": 0.009792807511985302}, {"layer": 39, "field": "key", "exact": false, "max_abs": 0.1875, "nrmse": 0.00613522669300437}, {"layer": 39, "field": "value", "exact": false, "max_abs": 0.21875, "nrmse": 0.007763063535094261}, {"layer": 40, "field": "conv", "exact": false, "max_abs": 0.125, "nrmse": 0.02097807452082634}, {"layer": 40, "field": "recurrent", "exact": false, "max_abs": 0.008437037467956543, "nrmse": 0.00742711778730154}, {"layer": 41, "field": "conv", "exact": false, "max_abs": 0.140625, "nrmse": 0.024379167705774307}, {"layer": 41, "field": "recurrent", "exact": false, "max_abs": 0.007077634334564209, "nrmse": 0.01596146821975708}, {"layer": 42, "field": "conv", "exact": false, "max_abs": 0.1875, "nrmse": 0.026848988607525826}, {"layer": 42, "field": "recurrent", "exact": false, "max_abs": 0.0069390833377838135, "nrmse": 0.0163588784635067}, {"layer": 43, "field": "key", "exact": false, "max_abs": 0.3515625, "nrmse": 0.008846464566886425}, {"layer": 43, "field": "value", "exact": false, "max_abs": 0.31640625, "nrmse": 0.009956845082342625}, {"layer": 44, "field": "conv", "exact": false, "max_abs": 0.1875, "nrmse": 0.028843387961387634}, {"layer": 44, "field": "recurrent", "exact": false, "max_abs": 0.01271599531173706, "nrmse": 0.022634657099843025}, {"layer": 45, "field": "conv", "exact": false, "max_abs": 0.1875, "nrmse": 0.032767754048109055}, {"layer": 45, "field": "recurrent", "exact": false, "max_abs": 0.012819111347198486, "nrmse": 0.025554699823260307}, {"layer": 46, "field": "conv", "exact": false, "max_abs": 0.17333984375, "nrmse": 0.032247692346572876}, {"layer": 46, "field": "recurrent", "exact": false, "max_abs": 0.015823304653167725, "nrmse": 0.024001799523830414}, {"layer": 47, "field": "key", "exact": false, "max_abs": 0.55078125, "nrmse": 0.010782185010612011}, {"layer": 47, "field": "value", "exact": false, "max_abs": 0.33203125, "nrmse": 0.013780687935650349}, {"layer": 48, "field": "conv", "exact": false, "max_abs": 0.23828125, "nrmse": 0.034116849303245544}, {"layer": 48, "field": "recurrent", "exact": false, "max_abs": 0.011078111827373505, "nrmse": 0.027065768837928772}, {"layer": 49, "field": "conv", "exact": false, "max_abs": 0.28125, "nrmse": 0.034403447061777115}, {"layer": 49, "field": "recurrent", "exact": false, "max_abs": 0.011127471923828125, "nrmse": 0.03085166960954666}, {"layer": 50, "field": "conv", "exact": false, "max_abs": 0.2265625, "nrmse": 0.033769894391298294}, {"layer": 50, "field": "recurrent", "exact": false, "max_abs": 0.007357411086559296, "nrmse": 0.027060819789767265}, {"layer": 51, "field": "key", "exact": false, "max_abs": 0.32421875, "nrmse": 0.010677437297999859}, {"layer": 51, "field": "value", "exact": false, "max_abs": 0.5625, "nrmse": 0.014482892118394375}, {"layer": 52, "field": "conv", "exact": false, "max_abs": 0.375, "nrmse": 0.027957109734416008}, {"layer": 52, "field": "recurrent", "exact": false, "max_abs": 0.03315943479537964, "nrmse": 0.021147236227989197}, {"layer": 53, "field": "conv", "exact": false, "max_abs": 0.296875, "nrmse": 0.02848653867840767}, {"layer": 53, "field": "recurrent", "exact": false, "max_abs": 0.05887401103973389, "nrmse": 0.028243569657206535}, {"layer": 54, "field": "conv", "exact": false, "max_abs": 0.33203125, "nrmse": 0.03156919404864311}, {"layer": 54, "field": "recurrent", "exact": false, "max_abs": 0.02138151228427887, "nrmse": 0.030725812539458275}, {"layer": 55, "field": "key", "exact": false, "max_abs": 0.236328125, "nrmse": 0.010902577079832554}, {"layer": 55, "field": "value", "exact": false, "max_abs": 0.75, "nrmse": 0.015734978020191193}, {"layer": 56, "field": "conv", "exact": false, "max_abs": 0.296875, "nrmse": 0.02765217050909996}, {"layer": 56, "field": "recurrent", "exact": false, "max_abs": 0.035627782344818115, "nrmse": 0.02361035905778408}, {"layer": 57, "field": "conv", "exact": false, "max_abs": 0.4375, "nrmse": 0.024447888135910034}, {"layer": 57, "field": "recurrent", "exact": false, "max_abs": 0.02576005458831787, "nrmse": 0.024957703426480293}, {"layer": 58, "field": "conv", "exact": false, "max_abs": 0.40625, "nrmse": 0.029603321105241776}, {"layer": 58, "field": "recurrent", "exact": false, "max_abs": 0.0303499698638916, "nrmse": 0.021949511021375656}, {"layer": 59, "field": "key", "exact": false, "max_abs": 0.2734375, "nrmse": 0.011004315689206123}, {"layer": 59, "field": "value", "exact": false, "max_abs": 0.79296875, "nrmse": 0.012735666707158089}, {"layer": 60, "field": "conv", "exact": false, "max_abs": 0.5, "nrmse": 0.023549553006887436}, {"layer": 60, "field": "recurrent", "exact": false, "max_abs": 0.04284977912902832, "nrmse": 0.026148078963160515}, {"layer": 61, "field": "conv", "exact": false, "max_abs": 0.328125, "nrmse": 0.02571890316903591}, {"layer": 61, "field": "recurrent", "exact": false, "max_abs": 0.018650829792022705, "nrmse": 0.026332907378673553}, {"layer": 62, "field": "conv", "exact": false, "max_abs": 0.375, "nrmse": 0.023928547278046608}, {"layer": 62, "field": "recurrent", "exact": false, "max_abs": 0.029714733362197876, "nrmse": 0.019153859466314316}, {"layer": 63, "field": "key", "exact": false, "max_abs": 0.25, "nrmse": 0.011952684260904789}, {"layer": 63, "field": "value", "exact": false, "max_abs": 0.890625, "nrmse": 0.007410941179841757}], "buffered_graph_state_diff": []} -{"phase": "step", "index": 56, "position": 204, "teacher_token": 25, "compact": {"ids": [220, 198, 19, 10342, 393], "logits": [24.0, 12.25, 11.1875, 10.0, 10.0], "margin": 11.75, "argmax": 220}, "buffered": {"ids": [220, 198, 19, 10342, 393], "logits": [24.375, 12.5, 11.25, 10.4375, 10.375], "margin": 11.875, "argmax": 220}, "graph": {"ids": [220, 198, 19, 10342, 393], "logits": [24.375, 12.5, 11.25, 10.4375, 10.375], "margin": 11.875, "argmax": 220}, "compact_buffered": {"exact": false, "max_abs": 0.4375, "nrmse": 0.015886038541793823}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "compact_buffered_state_diff": [{"layer": 36, "field": "recurrent", "exact": false, "max_abs": 4.76837158203125e-07, "nrmse": 1.330869281446212e-07}, {"layer": 37, "field": "recurrent", "exact": false, "max_abs": 0.0004913024604320526, "nrmse": 0.000507319055031985}, {"layer": 38, "field": "conv", "exact": false, "max_abs": 0.0625, "nrmse": 0.009899611584842205}, {"layer": 38, "field": "recurrent", "exact": false, "max_abs": 0.005858719348907471, "nrmse": 0.009648477658629417}, {"layer": 39, "field": "key", "exact": false, "max_abs": 0.1875, "nrmse": 0.006213391665369272}, {"layer": 39, "field": "value", "exact": false, "max_abs": 0.21875, "nrmse": 0.007875281386077404}, {"layer": 40, "field": "conv", "exact": false, "max_abs": 0.1015625, "nrmse": 0.02032206580042839}, {"layer": 40, "field": "recurrent", "exact": false, "max_abs": 0.01218801736831665, "nrmse": 0.007634670939296484}, {"layer": 41, "field": "conv", "exact": false, "max_abs": 0.15625, "nrmse": 0.024206729605793953}, {"layer": 41, "field": "recurrent", "exact": false, "max_abs": 0.007075019180774689, "nrmse": 0.016074642539024353}, {"layer": 42, "field": "conv", "exact": false, "max_abs": 0.1875, "nrmse": 0.027241535484790802}, {"layer": 42, "field": "recurrent", "exact": false, "max_abs": 0.006890386343002319, "nrmse": 0.016184285283088684}, {"layer": 43, "field": "key", "exact": false, "max_abs": 0.3515625, "nrmse": 0.008950388990342617}, {"layer": 43, "field": "value", "exact": false, "max_abs": 0.31640625, "nrmse": 0.010106016881763935}, {"layer": 44, "field": "conv", "exact": false, "max_abs": 0.267578125, "nrmse": 0.029727382585406303}, {"layer": 44, "field": "recurrent", "exact": false, "max_abs": 0.012701988220214844, "nrmse": 0.022602224722504616}, {"layer": 45, "field": "conv", "exact": false, "max_abs": 0.1875, "nrmse": 0.03375372663140297}, {"layer": 45, "field": "recurrent", "exact": false, "max_abs": 0.012817800045013428, "nrmse": 0.025668885558843613}, {"layer": 46, "field": "conv", "exact": false, "max_abs": 0.189453125, "nrmse": 0.03313736990094185}, {"layer": 46, "field": "recurrent", "exact": false, "max_abs": 0.01582998037338257, "nrmse": 0.024110333994030952}, {"layer": 47, "field": "key", "exact": false, "max_abs": 0.55078125, "nrmse": 0.010915526188910007}, {"layer": 47, "field": "value", "exact": false, "max_abs": 0.33203125, "nrmse": 0.014021956361830235}, {"layer": 48, "field": "conv", "exact": false, "max_abs": 0.24609375, "nrmse": 0.034368570894002914}, {"layer": 48, "field": "recurrent", "exact": false, "max_abs": 0.011076688766479492, "nrmse": 0.026535654440522194}, {"layer": 49, "field": "conv", "exact": false, "max_abs": 0.28125, "nrmse": 0.0356883630156517}, {"layer": 49, "field": "recurrent", "exact": false, "max_abs": 0.011067382991313934, "nrmse": 0.03033921867609024}, {"layer": 50, "field": "conv", "exact": false, "max_abs": 0.28515625, "nrmse": 0.035388603806495667}, {"layer": 50, "field": "recurrent", "exact": false, "max_abs": 0.0075422972440719604, "nrmse": 0.027673112228512764}, {"layer": 51, "field": "key", "exact": false, "max_abs": 0.32421875, "nrmse": 0.010879185050725937}, {"layer": 51, "field": "value", "exact": false, "max_abs": 0.5625, "nrmse": 0.014782163314521313}, {"layer": 52, "field": "conv", "exact": false, "max_abs": 0.40625, "nrmse": 0.02842140384018421}, {"layer": 52, "field": "recurrent", "exact": false, "max_abs": 0.03325313329696655, "nrmse": 0.02114923670887947}, {"layer": 53, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.029270846396684647}, {"layer": 53, "field": "recurrent", "exact": false, "max_abs": 0.05877256393432617, "nrmse": 0.0284597035497427}, {"layer": 54, "field": "conv", "exact": false, "max_abs": 0.275634765625, "nrmse": 0.03193647041916847}, {"layer": 54, "field": "recurrent", "exact": false, "max_abs": 0.021584972739219666, "nrmse": 0.03112734854221344}, {"layer": 55, "field": "key", "exact": false, "max_abs": 0.236328125, "nrmse": 0.01117193978279829}, {"layer": 55, "field": "value", "exact": false, "max_abs": 0.75, "nrmse": 0.016048043966293335}, {"layer": 56, "field": "conv", "exact": false, "max_abs": 0.375, "nrmse": 0.027480829507112503}, {"layer": 56, "field": "recurrent", "exact": false, "max_abs": 0.035545140504837036, "nrmse": 0.023901546373963356}, {"layer": 57, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.024233857169747353}, {"layer": 57, "field": "recurrent", "exact": false, "max_abs": 0.025819897651672363, "nrmse": 0.025825276970863342}, {"layer": 58, "field": "conv", "exact": false, "max_abs": 0.375, "nrmse": 0.028790773823857307}, {"layer": 58, "field": "recurrent", "exact": false, "max_abs": 0.030276715755462646, "nrmse": 0.022451650351285934}, {"layer": 59, "field": "key", "exact": false, "max_abs": 0.296875, "nrmse": 0.011359039694070816}, {"layer": 59, "field": "value", "exact": false, "max_abs": 0.79296875, "nrmse": 0.013022923842072487}, {"layer": 60, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.022344382479786873}, {"layer": 60, "field": "recurrent", "exact": false, "max_abs": 0.04338717460632324, "nrmse": 0.02624046802520752}, {"layer": 61, "field": "conv", "exact": false, "max_abs": 0.2734375, "nrmse": 0.024951310828328133}, {"layer": 61, "field": "recurrent", "exact": false, "max_abs": 0.018250703811645508, "nrmse": 0.026883220300078392}, {"layer": 62, "field": "conv", "exact": false, "max_abs": 0.46875, "nrmse": 0.0233183391392231}, {"layer": 62, "field": "recurrent", "exact": false, "max_abs": 0.029713734984397888, "nrmse": 0.019213862717151642}, {"layer": 63, "field": "key", "exact": false, "max_abs": 0.25, "nrmse": 0.012209231965243816}, {"layer": 63, "field": "value", "exact": false, "max_abs": 0.890625, "nrmse": 0.007538353092968464}], "buffered_graph_state_diff": []} -{"phase": "step", "index": 57, "position": 205, "teacher_token": 220, "compact": {"ids": [19, 18, 20, 22, 16], "logits": [26.625, 15.75, 14.5, 14.5, 13.75], "margin": 10.875, "argmax": 19}, "buffered": {"ids": [19, 18, 22, 20, 16], "logits": [26.5, 15.75, 14.5625, 14.375, 13.5], "margin": 10.75, "argmax": 19}, "graph": {"ids": [19, 18, 22, 20, 16], "logits": [26.5, 15.75, 14.5625, 14.375, 13.5], "margin": 10.75, "argmax": 19}, "compact_buffered": {"exact": false, "max_abs": 0.51171875, "nrmse": 0.022687595337629318}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "compact_buffered_state_diff": [{"layer": 36, "field": "recurrent", "exact": false, "max_abs": 4.76837158203125e-07, "nrmse": 1.3353486849609908e-07}, {"layer": 37, "field": "recurrent", "exact": false, "max_abs": 0.0004890970885753632, "nrmse": 0.0005067970487289131}, {"layer": 38, "field": "conv", "exact": false, "max_abs": 0.0625, "nrmse": 0.010333894751966}, {"layer": 38, "field": "recurrent", "exact": false, "max_abs": 0.005843907594680786, "nrmse": 0.009666376747190952}, {"layer": 39, "field": "key", "exact": false, "max_abs": 0.1875, "nrmse": 0.006269354838877916}, {"layer": 39, "field": "value", "exact": false, "max_abs": 0.21875, "nrmse": 0.00799395889043808}, {"layer": 40, "field": "conv", "exact": false, "max_abs": 0.171875, "nrmse": 0.022448912262916565}, {"layer": 40, "field": "recurrent", "exact": false, "max_abs": 0.012094289064407349, "nrmse": 0.007863704115152359}, {"layer": 41, "field": "conv", "exact": false, "max_abs": 0.1875, "nrmse": 0.026038730517029762}, {"layer": 41, "field": "recurrent", "exact": false, "max_abs": 0.007069967687129974, "nrmse": 0.016286952421069145}, {"layer": 42, "field": "conv", "exact": false, "max_abs": 0.1875, "nrmse": 0.029071476310491562}, {"layer": 42, "field": "recurrent", "exact": false, "max_abs": 0.008417248725891113, "nrmse": 0.01649458520114422}, {"layer": 43, "field": "key", "exact": false, "max_abs": 0.3515625, "nrmse": 0.009087610989809036}, {"layer": 43, "field": "value", "exact": false, "max_abs": 0.31640625, "nrmse": 0.010309572331607342}, {"layer": 44, "field": "conv", "exact": false, "max_abs": 0.267578125, "nrmse": 0.03102125972509384}, {"layer": 44, "field": "recurrent", "exact": false, "max_abs": 0.012687034904956818, "nrmse": 0.022728893905878067}, {"layer": 45, "field": "conv", "exact": false, "max_abs": 0.203125, "nrmse": 0.035493046045303345}, {"layer": 45, "field": "recurrent", "exact": false, "max_abs": 0.012816905975341797, "nrmse": 0.025932304561138153}, {"layer": 46, "field": "conv", "exact": false, "max_abs": 0.189453125, "nrmse": 0.03471267223358154}, {"layer": 46, "field": "recurrent", "exact": false, "max_abs": 0.01582014560699463, "nrmse": 0.024267330765724182}, {"layer": 47, "field": "key", "exact": false, "max_abs": 0.55078125, "nrmse": 0.011067138984799385}, {"layer": 47, "field": "value", "exact": false, "max_abs": 0.33203125, "nrmse": 0.014324329793453217}, {"layer": 48, "field": "conv", "exact": false, "max_abs": 0.24609375, "nrmse": 0.03611207380890846}, {"layer": 48, "field": "recurrent", "exact": false, "max_abs": 0.011092379689216614, "nrmse": 0.027005134150385857}, {"layer": 49, "field": "conv", "exact": false, "max_abs": 0.375, "nrmse": 0.03815779834985733}, {"layer": 49, "field": "recurrent", "exact": false, "max_abs": 0.018383026123046875, "nrmse": 0.0310304407030344}, {"layer": 50, "field": "conv", "exact": false, "max_abs": 0.36865234375, "nrmse": 0.03908615559339523}, {"layer": 50, "field": "recurrent", "exact": false, "max_abs": 0.0073730722069740295, "nrmse": 0.02842061221599579}, {"layer": 51, "field": "key", "exact": false, "max_abs": 0.32421875, "nrmse": 0.011144822463393211}, {"layer": 51, "field": "value", "exact": false, "max_abs": 0.5625, "nrmse": 0.01524219661951065}, {"layer": 52, "field": "conv", "exact": false, "max_abs": 0.40625, "nrmse": 0.0321616493165493}, {"layer": 52, "field": "recurrent", "exact": false, "max_abs": 0.037529945373535156, "nrmse": 0.02139684557914734}, {"layer": 53, "field": "conv", "exact": false, "max_abs": 0.4765625, "nrmse": 0.03344419226050377}, {"layer": 53, "field": "recurrent", "exact": false, "max_abs": 0.05861485004425049, "nrmse": 0.0290413536131382}, {"layer": 54, "field": "conv", "exact": false, "max_abs": 0.4375, "nrmse": 0.03760480135679245}, {"layer": 54, "field": "recurrent", "exact": false, "max_abs": 0.02176445722579956, "nrmse": 0.031832873821258545}, {"layer": 55, "field": "key", "exact": false, "max_abs": 0.3046875, "nrmse": 0.01157810352742672}, {"layer": 55, "field": "value", "exact": false, "max_abs": 0.75, "nrmse": 0.016646094620227814}, {"layer": 56, "field": "conv", "exact": false, "max_abs": 0.421875, "nrmse": 0.03261712193489075}, {"layer": 56, "field": "recurrent", "exact": false, "max_abs": 0.03561237454414368, "nrmse": 0.02457759529352188}, {"layer": 57, "field": "conv", "exact": false, "max_abs": 0.40625, "nrmse": 0.02913585864007473}, {"layer": 57, "field": "recurrent", "exact": false, "max_abs": 0.035480618476867676, "nrmse": 0.027204619720578194}, {"layer": 58, "field": "conv", "exact": false, "max_abs": 0.375, "nrmse": 0.03279045224189758}, {"layer": 58, "field": "recurrent", "exact": false, "max_abs": 0.03038609027862549, "nrmse": 0.023177102208137512}, {"layer": 59, "field": "key", "exact": false, "max_abs": 0.296875, "nrmse": 0.011792678385972977}, {"layer": 59, "field": "value", "exact": false, "max_abs": 0.96875, "nrmse": 0.013467324897646904}, {"layer": 60, "field": "conv", "exact": false, "max_abs": 0.5625, "nrmse": 0.025868235155940056}, {"layer": 60, "field": "recurrent", "exact": false, "max_abs": 0.04311341047286987, "nrmse": 0.026389876380562782}, {"layer": 61, "field": "conv", "exact": false, "max_abs": 0.41796875, "nrmse": 0.029179096221923828}, {"layer": 61, "field": "recurrent", "exact": false, "max_abs": 0.02013280987739563, "nrmse": 0.02767263911664486}, {"layer": 62, "field": "conv", "exact": false, "max_abs": 0.6875, "nrmse": 0.026780210435390472}, {"layer": 62, "field": "recurrent", "exact": false, "max_abs": 0.03022770583629608, "nrmse": 0.019510434940457344}, {"layer": 63, "field": "key", "exact": false, "max_abs": 0.2734375, "nrmse": 0.012610879726707935}, {"layer": 63, "field": "value", "exact": false, "max_abs": 0.890625, "nrmse": 0.007760533597320318}], "buffered_graph_state_diff": []} -{"phase": "step", "index": 58, "position": 206, "teacher_token": 19, "compact": {"ids": [198, 271, 318, 2702, 10342], "logits": [25.375, 22.0, 13.25, 12.75, 11.375], "margin": 3.375, "argmax": 198}, "buffered": {"ids": [198, 271, 318, 2702, 10342], "logits": [25.375, 22.125, 13.375, 12.75, 11.375], "margin": 3.25, "argmax": 198}, "graph": {"ids": [198, 271, 318, 2702, 10342], "logits": [25.375, 22.125, 13.375, 12.75, 11.375], "margin": 3.25, "argmax": 198}, "compact_buffered": {"exact": false, "max_abs": 0.447265625, "nrmse": 0.017909426242113113}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "compact_buffered_state_diff": [{"layer": 36, "field": "recurrent", "exact": false, "max_abs": 4.76837158203125e-07, "nrmse": 1.3342324223231117e-07}, {"layer": 37, "field": "recurrent", "exact": false, "max_abs": 0.0004912465810775757, "nrmse": 0.0005052430205978453}, {"layer": 38, "field": "conv", "exact": false, "max_abs": 0.0625, "nrmse": 0.010919513180851936}, {"layer": 38, "field": "recurrent", "exact": false, "max_abs": 0.005833864212036133, "nrmse": 0.009690304286777973}, {"layer": 39, "field": "key", "exact": false, "max_abs": 0.1875, "nrmse": 0.006322578992694616}, {"layer": 39, "field": "value", "exact": false, "max_abs": 0.21875, "nrmse": 0.008072474040091038}, {"layer": 40, "field": "conv", "exact": false, "max_abs": 0.171875, "nrmse": 0.022959677502512932}, {"layer": 40, "field": "recurrent", "exact": false, "max_abs": 0.008628606796264648, "nrmse": 0.007773127872496843}, {"layer": 41, "field": "conv", "exact": false, "max_abs": 0.1875, "nrmse": 0.02684878744184971}, {"layer": 41, "field": "recurrent", "exact": false, "max_abs": 0.007068172097206116, "nrmse": 0.016337258741259575}, {"layer": 42, "field": "conv", "exact": false, "max_abs": 0.1875, "nrmse": 0.029756374657154083}, {"layer": 42, "field": "recurrent", "exact": false, "max_abs": 0.0070960670709609985, "nrmse": 0.016346754506230354}, {"layer": 43, "field": "key", "exact": false, "max_abs": 0.3515625, "nrmse": 0.009223938919603825}, {"layer": 43, "field": "value", "exact": false, "max_abs": 0.31640625, "nrmse": 0.010453863069415092}, {"layer": 44, "field": "conv", "exact": false, "max_abs": 0.267578125, "nrmse": 0.03157448396086693}, {"layer": 44, "field": "recurrent", "exact": false, "max_abs": 0.012758180499076843, "nrmse": 0.022951163351535797}, {"layer": 45, "field": "conv", "exact": false, "max_abs": 0.203125, "nrmse": 0.036055561155080795}, {"layer": 45, "field": "recurrent", "exact": false, "max_abs": 0.012819111347198486, "nrmse": 0.025861218571662903}, {"layer": 46, "field": "conv", "exact": false, "max_abs": 0.189453125, "nrmse": 0.03487679734826088}, {"layer": 46, "field": "recurrent", "exact": false, "max_abs": 0.01578700542449951, "nrmse": 0.02457726001739502}, {"layer": 47, "field": "key", "exact": false, "max_abs": 0.55078125, "nrmse": 0.011188900098204613}, {"layer": 47, "field": "value", "exact": false, "max_abs": 0.33203125, "nrmse": 0.014510572887957096}, {"layer": 48, "field": "conv", "exact": false, "max_abs": 0.24609375, "nrmse": 0.03545653820037842}, {"layer": 48, "field": "recurrent", "exact": false, "max_abs": 0.011155717074871063, "nrmse": 0.026771731674671173}, {"layer": 49, "field": "conv", "exact": false, "max_abs": 0.375, "nrmse": 0.03818595036864281}, {"layer": 49, "field": "recurrent", "exact": false, "max_abs": 0.01974862813949585, "nrmse": 0.030695490539073944}, {"layer": 50, "field": "conv", "exact": false, "max_abs": 0.36865234375, "nrmse": 0.03931300714612007}, {"layer": 50, "field": "recurrent", "exact": false, "max_abs": 0.008629798889160156, "nrmse": 0.028404271230101585}, {"layer": 51, "field": "key", "exact": false, "max_abs": 0.32421875, "nrmse": 0.011309619061648846}, {"layer": 51, "field": "value", "exact": false, "max_abs": 0.5625, "nrmse": 0.015481149777770042}, {"layer": 52, "field": "conv", "exact": false, "max_abs": 0.40625, "nrmse": 0.03250462934374809}, {"layer": 52, "field": "recurrent", "exact": false, "max_abs": 0.03509366512298584, "nrmse": 0.021303387358784676}, {"layer": 53, "field": "conv", "exact": false, "max_abs": 0.4765625, "nrmse": 0.03376218304038048}, {"layer": 53, "field": "recurrent", "exact": false, "max_abs": 0.0583796501159668, "nrmse": 0.028891537338495255}, {"layer": 54, "field": "conv", "exact": false, "max_abs": 0.4375, "nrmse": 0.03823082149028778}, {"layer": 54, "field": "recurrent", "exact": false, "max_abs": 0.021771714091300964, "nrmse": 0.03143991529941559}, {"layer": 55, "field": "key", "exact": false, "max_abs": 0.3046875, "nrmse": 0.011738034896552563}, {"layer": 55, "field": "value", "exact": false, "max_abs": 0.75, "nrmse": 0.016871096566319466}, {"layer": 56, "field": "conv", "exact": false, "max_abs": 0.421875, "nrmse": 0.03284740075469017}, {"layer": 56, "field": "recurrent", "exact": false, "max_abs": 0.035248637199401855, "nrmse": 0.024376126006245613}, {"layer": 57, "field": "conv", "exact": false, "max_abs": 0.40625, "nrmse": 0.029621662572026253}, {"layer": 57, "field": "recurrent", "exact": false, "max_abs": 0.025626391172409058, "nrmse": 0.02567925862967968}, {"layer": 58, "field": "conv", "exact": false, "max_abs": 0.375, "nrmse": 0.033538639545440674}, {"layer": 58, "field": "recurrent", "exact": false, "max_abs": 0.03273582458496094, "nrmse": 0.022453002631664276}, {"layer": 59, "field": "key", "exact": false, "max_abs": 0.296875, "nrmse": 0.01198496762663126}, {"layer": 59, "field": "value", "exact": false, "max_abs": 0.96875, "nrmse": 0.013616694137454033}, {"layer": 60, "field": "conv", "exact": false, "max_abs": 0.5625, "nrmse": 0.025691183283925056}, {"layer": 60, "field": "recurrent", "exact": false, "max_abs": 0.043458402156829834, "nrmse": 0.02653682790696621}, {"layer": 61, "field": "conv", "exact": false, "max_abs": 0.41796875, "nrmse": 0.029636861756443977}, {"layer": 61, "field": "recurrent", "exact": false, "max_abs": 0.01821283996105194, "nrmse": 0.026674756780266762}, {"layer": 62, "field": "conv", "exact": false, "max_abs": 0.6875, "nrmse": 0.025612574070692062}, {"layer": 62, "field": "recurrent", "exact": false, "max_abs": 0.029849112033843994, "nrmse": 0.019451972097158432}, {"layer": 63, "field": "key", "exact": false, "max_abs": 0.2734375, "nrmse": 0.012781952507793903}, {"layer": 63, "field": "value", "exact": false, "max_abs": 0.890625, "nrmse": 0.007839808240532875}], "buffered_graph_state_diff": []} -{"phase": "step", "index": 59, "position": 207, "teacher_token": 198, "compact": {"ids": [9, 7354, 6391, 262, 348], "logits": [27.125, 15.1875, 13.9375, 13.1875, 12.4375], "margin": 11.9375, "argmax": 9}, "buffered": {"ids": [9, 7354, 6391, 262, 348], "logits": [27.125, 15.0, 13.875, 13.25, 12.5], "margin": 12.125, "argmax": 9}, "graph": {"ids": [9, 7354, 6391, 262, 348], "logits": [27.125, 15.0, 13.875, 13.25, 12.5], "margin": 12.125, "argmax": 9}, "compact_buffered": {"exact": false, "max_abs": 0.484375, "nrmse": 0.02829395979642868}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "compact_buffered_state_diff": [{"layer": 36, "field": "recurrent", "exact": false, "max_abs": 4.76837158203125e-07, "nrmse": 1.3416894262263668e-07}, {"layer": 37, "field": "recurrent", "exact": false, "max_abs": 0.00047805532813072205, "nrmse": 0.0005104973679408431}, {"layer": 38, "field": "conv", "exact": false, "max_abs": 0.0625, "nrmse": 0.011729732155799866}, {"layer": 38, "field": "recurrent", "exact": false, "max_abs": 0.0058325231075286865, "nrmse": 0.009953396394848824}, {"layer": 39, "field": "key", "exact": false, "max_abs": 0.1875, "nrmse": 0.006402274128049612}, {"layer": 39, "field": "value", "exact": false, "max_abs": 0.21875, "nrmse": 0.008183407597243786}, {"layer": 40, "field": "conv", "exact": false, "max_abs": 0.171875, "nrmse": 0.02404775843024254}, {"layer": 40, "field": "recurrent", "exact": false, "max_abs": 0.009696781635284424, "nrmse": 0.007833538576960564}, {"layer": 41, "field": "conv", "exact": false, "max_abs": 0.1875, "nrmse": 0.027908995747566223}, {"layer": 41, "field": "recurrent", "exact": false, "max_abs": 0.007071703672409058, "nrmse": 0.016297737136483192}, {"layer": 42, "field": "conv", "exact": false, "max_abs": 0.21875, "nrmse": 0.03023255616426468}, {"layer": 42, "field": "recurrent", "exact": false, "max_abs": 0.008008748292922974, "nrmse": 0.016860289499163628}, {"layer": 43, "field": "key", "exact": false, "max_abs": 0.3515625, "nrmse": 0.009429726749658585}, {"layer": 43, "field": "value", "exact": false, "max_abs": 0.31640625, "nrmse": 0.010701361112296581}, {"layer": 44, "field": "conv", "exact": false, "max_abs": 0.267578125, "nrmse": 0.03318934142589569}, {"layer": 44, "field": "recurrent", "exact": false, "max_abs": 0.012851126492023468, "nrmse": 0.022977542132139206}, {"layer": 45, "field": "conv", "exact": false, "max_abs": 0.20556640625, "nrmse": 0.037979599088430405}, {"layer": 45, "field": "recurrent", "exact": false, "max_abs": 0.012813568115234375, "nrmse": 0.02637047879397869}, {"layer": 46, "field": "conv", "exact": false, "max_abs": 0.189453125, "nrmse": 0.03592666611075401}, {"layer": 46, "field": "recurrent", "exact": false, "max_abs": 0.015688598155975342, "nrmse": 0.0253855399787426}, {"layer": 47, "field": "key", "exact": false, "max_abs": 0.55078125, "nrmse": 0.011414715088903904}, {"layer": 47, "field": "value", "exact": false, "max_abs": 0.33203125, "nrmse": 0.0147950304672122}, {"layer": 48, "field": "conv", "exact": false, "max_abs": 0.28125, "nrmse": 0.03715215623378754}, {"layer": 48, "field": "recurrent", "exact": false, "max_abs": 0.0110822394490242, "nrmse": 0.027322763577103615}, {"layer": 49, "field": "conv", "exact": false, "max_abs": 0.375, "nrmse": 0.03938059136271477}, {"layer": 49, "field": "recurrent", "exact": false, "max_abs": 0.016491800546646118, "nrmse": 0.03125028684735298}, {"layer": 50, "field": "conv", "exact": false, "max_abs": 0.36865234375, "nrmse": 0.03988746926188469}, {"layer": 50, "field": "recurrent", "exact": false, "max_abs": 0.009121894836425781, "nrmse": 0.02893805503845215}, {"layer": 51, "field": "key", "exact": false, "max_abs": 0.359375, "nrmse": 0.01157599501311779}, {"layer": 51, "field": "value", "exact": false, "max_abs": 0.5625, "nrmse": 0.015740839764475822}, {"layer": 52, "field": "conv", "exact": false, "max_abs": 0.40625, "nrmse": 0.03350020572543144}, {"layer": 52, "field": "recurrent", "exact": false, "max_abs": 0.037459373474121094, "nrmse": 0.02141200192272663}, {"layer": 53, "field": "conv", "exact": false, "max_abs": 0.4765625, "nrmse": 0.034408874809741974}, {"layer": 53, "field": "recurrent", "exact": false, "max_abs": 0.05841219425201416, "nrmse": 0.028667692095041275}, {"layer": 54, "field": "conv", "exact": false, "max_abs": 0.4375, "nrmse": 0.03918756917119026}, {"layer": 54, "field": "recurrent", "exact": false, "max_abs": 0.024066850543022156, "nrmse": 0.03179242089390755}, {"layer": 55, "field": "key", "exact": false, "max_abs": 0.3046875, "nrmse": 0.011989438906311989}, {"layer": 55, "field": "value", "exact": false, "max_abs": 0.75, "nrmse": 0.017110522836446762}, {"layer": 56, "field": "conv", "exact": false, "max_abs": 0.421875, "nrmse": 0.03354737535119057}, {"layer": 56, "field": "recurrent", "exact": false, "max_abs": 0.0349481999874115, "nrmse": 0.024500520899891853}, {"layer": 57, "field": "conv", "exact": false, "max_abs": 0.5625, "nrmse": 0.03043387643992901}, {"layer": 57, "field": "recurrent", "exact": false, "max_abs": 0.032234251499176025, "nrmse": 0.02632012963294983}, {"layer": 58, "field": "conv", "exact": false, "max_abs": 0.375, "nrmse": 0.03392244130373001}, {"layer": 58, "field": "recurrent", "exact": false, "max_abs": 0.02935659885406494, "nrmse": 0.02257373370230198}, {"layer": 59, "field": "key", "exact": false, "max_abs": 0.296875, "nrmse": 0.012166382744908333}, {"layer": 59, "field": "value", "exact": false, "max_abs": 0.96875, "nrmse": 0.013765135779976845}, {"layer": 60, "field": "conv", "exact": false, "max_abs": 0.75, "nrmse": 0.02726534940302372}, {"layer": 60, "field": "recurrent", "exact": false, "max_abs": 0.04369184374809265, "nrmse": 0.02672881819307804}, {"layer": 61, "field": "conv", "exact": false, "max_abs": 0.41796875, "nrmse": 0.030013369396328926}, {"layer": 61, "field": "recurrent", "exact": false, "max_abs": 0.017756938934326172, "nrmse": 0.02677825838327408}, {"layer": 62, "field": "conv", "exact": false, "max_abs": 0.75, "nrmse": 0.026167118921875954}, {"layer": 62, "field": "recurrent", "exact": false, "max_abs": 0.028705596923828125, "nrmse": 0.019342124462127686}, {"layer": 63, "field": "key", "exact": false, "max_abs": 0.2734375, "nrmse": 0.012963779270648956}, {"layer": 63, "field": "value", "exact": false, "max_abs": 0.890625, "nrmse": 0.007951420731842518}], "buffered_graph_state_diff": []} -{"phase": "step", "index": 60, "position": 208, "teacher_token": 9, "compact": {"ids": [256, 10342, 262, 271, 7354], "logits": [28.875, 17.0, 16.625, 16.0, 15.125], "margin": 11.875, "argmax": 256}, "buffered": {"ids": [256, 10342, 262, 271, 7354], "logits": [28.5, 16.75, 16.5, 15.875, 14.875], "margin": 11.75, "argmax": 256}, "graph": {"ids": [256, 10342, 262, 271, 7354], "logits": [28.5, 16.75, 16.5, 15.875, 14.875], "margin": 11.75, "argmax": 256}, "compact_buffered": {"exact": false, "max_abs": 0.375, "nrmse": 0.015146064572036266}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "compact_buffered_state_diff": [{"layer": 36, "field": "recurrent", "exact": false, "max_abs": 4.76837158203125e-07, "nrmse": 1.349599045852301e-07}, {"layer": 37, "field": "recurrent", "exact": false, "max_abs": 0.0004780180752277374, "nrmse": 0.0005107020842842758}, {"layer": 38, "field": "conv", "exact": false, "max_abs": 0.078125, "nrmse": 0.012278178706765175}, {"layer": 38, "field": "recurrent", "exact": false, "max_abs": 0.008057132363319397, "nrmse": 0.010067018680274487}, {"layer": 39, "field": "key", "exact": false, "max_abs": 0.1875, "nrmse": 0.006515707354992628}, {"layer": 39, "field": "value", "exact": false, "max_abs": 0.21875, "nrmse": 0.008292553946375847}, {"layer": 40, "field": "conv", "exact": false, "max_abs": 0.171875, "nrmse": 0.023280741646885872}, {"layer": 40, "field": "recurrent", "exact": false, "max_abs": 0.008361339569091797, "nrmse": 0.007538685575127602}, {"layer": 41, "field": "conv", "exact": false, "max_abs": 0.1875, "nrmse": 0.027137767523527145}, {"layer": 41, "field": "recurrent", "exact": false, "max_abs": 0.007064566016197205, "nrmse": 0.016258010640740395}, {"layer": 42, "field": "conv", "exact": false, "max_abs": 0.21875, "nrmse": 0.029386885464191437}, {"layer": 42, "field": "recurrent", "exact": false, "max_abs": 0.01381005346775055, "nrmse": 0.017152149230241776}, {"layer": 43, "field": "key", "exact": false, "max_abs": 0.3515625, "nrmse": 0.009549295529723167}, {"layer": 43, "field": "value", "exact": false, "max_abs": 0.31640625, "nrmse": 0.010799415409564972}, {"layer": 44, "field": "conv", "exact": false, "max_abs": 0.2276611328125, "nrmse": 0.031585872173309326}, {"layer": 44, "field": "recurrent", "exact": false, "max_abs": 0.012833669781684875, "nrmse": 0.022901685908436775}, {"layer": 45, "field": "conv", "exact": false, "max_abs": 0.20556640625, "nrmse": 0.03575250878930092}, {"layer": 45, "field": "recurrent", "exact": false, "max_abs": 0.012814104557037354, "nrmse": 0.02599586732685566}, {"layer": 46, "field": "conv", "exact": false, "max_abs": 0.25, "nrmse": 0.03335237503051758}, {"layer": 46, "field": "recurrent", "exact": false, "max_abs": 0.015744566917419434, "nrmse": 0.02498982474207878}, {"layer": 47, "field": "key", "exact": false, "max_abs": 0.55078125, "nrmse": 0.011533188633620739}, {"layer": 47, "field": "value", "exact": false, "max_abs": 0.33203125, "nrmse": 0.014939900487661362}, {"layer": 48, "field": "conv", "exact": false, "max_abs": 0.28125, "nrmse": 0.03598366677761078}, {"layer": 48, "field": "recurrent", "exact": false, "max_abs": 0.011070095002651215, "nrmse": 0.026921460404992104}, {"layer": 49, "field": "conv", "exact": false, "max_abs": 0.375, "nrmse": 0.03723977506160736}, {"layer": 49, "field": "recurrent", "exact": false, "max_abs": 0.015094608068466187, "nrmse": 0.03156309202313423}, {"layer": 50, "field": "conv", "exact": false, "max_abs": 0.36865234375, "nrmse": 0.037308238446712494}, {"layer": 50, "field": "recurrent", "exact": false, "max_abs": 0.008631914854049683, "nrmse": 0.028225962072610855}, {"layer": 51, "field": "key", "exact": false, "max_abs": 0.359375, "nrmse": 0.011729156598448753}, {"layer": 51, "field": "value", "exact": false, "max_abs": 0.5625, "nrmse": 0.0158996619284153}, {"layer": 52, "field": "conv", "exact": false, "max_abs": 0.40625, "nrmse": 0.031855158507823944}, {"layer": 52, "field": "recurrent", "exact": false, "max_abs": 0.03314846754074097, "nrmse": 0.021359575912356377}, {"layer": 53, "field": "conv", "exact": false, "max_abs": 0.4765625, "nrmse": 0.032744523137807846}, {"layer": 53, "field": "recurrent", "exact": false, "max_abs": 0.05854690074920654, "nrmse": 0.027985230088233948}, {"layer": 54, "field": "conv", "exact": false, "max_abs": 0.4375, "nrmse": 0.03754506632685661}, {"layer": 54, "field": "recurrent", "exact": false, "max_abs": 0.03116874396800995, "nrmse": 0.030672522261738777}, {"layer": 55, "field": "key", "exact": false, "max_abs": 0.3046875, "nrmse": 0.012181469239294529}, {"layer": 55, "field": "value", "exact": false, "max_abs": 0.75, "nrmse": 0.01726716384291649}, {"layer": 56, "field": "conv", "exact": false, "max_abs": 0.421875, "nrmse": 0.03234175220131874}, {"layer": 56, "field": "recurrent", "exact": false, "max_abs": 0.03511083126068115, "nrmse": 0.024222686886787415}, {"layer": 57, "field": "conv", "exact": false, "max_abs": 0.5625, "nrmse": 0.02953450009226799}, {"layer": 57, "field": "recurrent", "exact": false, "max_abs": 0.025693506002426147, "nrmse": 0.0252053365111351}, {"layer": 58, "field": "conv", "exact": false, "max_abs": 0.375, "nrmse": 0.032566171139478683}, {"layer": 58, "field": "recurrent", "exact": false, "max_abs": 0.029216110706329346, "nrmse": 0.02207227610051632}, {"layer": 59, "field": "key", "exact": false, "max_abs": 0.296875, "nrmse": 0.01231044065207243}, {"layer": 59, "field": "value", "exact": false, "max_abs": 0.96875, "nrmse": 0.013902070932090282}, {"layer": 60, "field": "conv", "exact": false, "max_abs": 0.75, "nrmse": 0.026819396764039993}, {"layer": 60, "field": "recurrent", "exact": false, "max_abs": 0.05512833595275879, "nrmse": 0.02659212052822113}, {"layer": 61, "field": "conv", "exact": false, "max_abs": 0.41796875, "nrmse": 0.029037602245807648}, {"layer": 61, "field": "recurrent", "exact": false, "max_abs": 0.01748359203338623, "nrmse": 0.02622166834771633}, {"layer": 62, "field": "conv", "exact": false, "max_abs": 0.75, "nrmse": 0.025466809049248695}, {"layer": 62, "field": "recurrent", "exact": false, "max_abs": 0.028615832328796387, "nrmse": 0.019213082268834114}, {"layer": 63, "field": "key", "exact": false, "max_abs": 0.2734375, "nrmse": 0.013091932982206345}, {"layer": 63, "field": "value", "exact": false, "max_abs": 0.890625, "nrmse": 0.008014283142983913}], "buffered_graph_state_diff": []} -{"phase": "step", "index": 61, "position": 209, "teacher_token": 256, "compact": {"ids": [10342, 3991, 561, 8852, 6558], "logits": [26.625, 22.25, 16.0, 15.9375, 15.25], "margin": 4.375, "argmax": 10342}, "buffered": {"ids": [10342, 3991, 561, 8852, 6558], "logits": [26.5, 22.125, 16.0, 15.75, 15.25], "margin": 4.375, "argmax": 10342}, "graph": {"ids": [10342, 3991, 561, 8852, 6558], "logits": [26.5, 22.125, 16.0, 15.75, 15.25], "margin": 4.375, "argmax": 10342}, "compact_buffered": {"exact": false, "max_abs": 0.4375, "nrmse": 0.023444872349500656}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "compact_buffered_state_diff": [{"layer": 36, "field": "recurrent", "exact": false, "max_abs": 4.76837158203125e-07, "nrmse": 1.341017537015432e-07}, {"layer": 37, "field": "recurrent", "exact": false, "max_abs": 0.0004790462553501129, "nrmse": 0.0005042626289650798}, {"layer": 38, "field": "conv", "exact": false, "max_abs": 0.078125, "nrmse": 0.012452938593924046}, {"layer": 38, "field": "recurrent", "exact": false, "max_abs": 0.0058325231075286865, "nrmse": 0.009957639500498772}, {"layer": 39, "field": "key", "exact": false, "max_abs": 0.1875, "nrmse": 0.006668194197118282}, {"layer": 39, "field": "value", "exact": false, "max_abs": 0.21875, "nrmse": 0.008388975635170937}, {"layer": 40, "field": "conv", "exact": false, "max_abs": 0.125, "nrmse": 0.022591520100831985}, {"layer": 40, "field": "recurrent", "exact": false, "max_abs": 0.008518457412719727, "nrmse": 0.007824438624083996}, {"layer": 41, "field": "conv", "exact": false, "max_abs": 0.140625, "nrmse": 0.02633744105696678}, {"layer": 41, "field": "recurrent", "exact": false, "max_abs": 0.00706714391708374, "nrmse": 0.016267647966742516}, {"layer": 42, "field": "conv", "exact": false, "max_abs": 0.21875, "nrmse": 0.028513053432106972}, {"layer": 42, "field": "recurrent", "exact": false, "max_abs": 0.010127894580364227, "nrmse": 0.016809748485684395}, {"layer": 43, "field": "key", "exact": false, "max_abs": 0.3515625, "nrmse": 0.009697054512798786}, {"layer": 43, "field": "value", "exact": false, "max_abs": 0.31640625, "nrmse": 0.010937640443444252}, {"layer": 44, "field": "conv", "exact": false, "max_abs": 0.2276611328125, "nrmse": 0.030733412131667137}, {"layer": 44, "field": "recurrent", "exact": false, "max_abs": 0.012759588658809662, "nrmse": 0.022830909118056297}, {"layer": 45, "field": "conv", "exact": false, "max_abs": 0.21875, "nrmse": 0.035494059324264526}, {"layer": 45, "field": "recurrent", "exact": false, "max_abs": 0.012811481952667236, "nrmse": 0.025817742571234703}, {"layer": 46, "field": "conv", "exact": false, "max_abs": 0.25, "nrmse": 0.03260965645313263}, {"layer": 46, "field": "recurrent", "exact": false, "max_abs": 0.015711724758148193, "nrmse": 0.02444784715771675}, {"layer": 47, "field": "key", "exact": false, "max_abs": 0.55078125, "nrmse": 0.011672892607748508}, {"layer": 47, "field": "value", "exact": false, "max_abs": 0.33203125, "nrmse": 0.015164590440690517}, {"layer": 48, "field": "conv", "exact": false, "max_abs": 0.28125, "nrmse": 0.03564764931797981}, {"layer": 48, "field": "recurrent", "exact": false, "max_abs": 0.011070236563682556, "nrmse": 0.027472389861941338}, {"layer": 49, "field": "conv", "exact": false, "max_abs": 0.40625, "nrmse": 0.03688560053706169}, {"layer": 49, "field": "recurrent", "exact": false, "max_abs": 0.012983709573745728, "nrmse": 0.03130677342414856}, {"layer": 50, "field": "conv", "exact": false, "max_abs": 0.265625, "nrmse": 0.03542407974600792}, {"layer": 50, "field": "recurrent", "exact": false, "max_abs": 0.010165572166442871, "nrmse": 0.028068114072084427}, {"layer": 51, "field": "key", "exact": false, "max_abs": 0.359375, "nrmse": 0.011993941850960255}, {"layer": 51, "field": "value", "exact": false, "max_abs": 0.5625, "nrmse": 0.01613566465675831}, {"layer": 52, "field": "conv", "exact": false, "max_abs": 0.33203125, "nrmse": 0.02937288209795952}, {"layer": 52, "field": "recurrent", "exact": false, "max_abs": 0.03928195685148239, "nrmse": 0.021813886240124702}, {"layer": 53, "field": "conv", "exact": false, "max_abs": 0.2734375, "nrmse": 0.030278438702225685}, {"layer": 53, "field": "recurrent", "exact": false, "max_abs": 0.05869162082672119, "nrmse": 0.027923541143536568}, {"layer": 54, "field": "conv", "exact": false, "max_abs": 0.4375, "nrmse": 0.03387543559074402}, {"layer": 54, "field": "recurrent", "exact": false, "max_abs": 0.029123708605766296, "nrmse": 0.030269121751189232}, {"layer": 55, "field": "key", "exact": false, "max_abs": 0.3046875, "nrmse": 0.012326112017035484}, {"layer": 55, "field": "value", "exact": false, "max_abs": 0.75, "nrmse": 0.017442671582102776}, {"layer": 56, "field": "conv", "exact": false, "max_abs": 0.3828125, "nrmse": 0.029148587957024574}, {"layer": 56, "field": "recurrent", "exact": false, "max_abs": 0.03499680757522583, "nrmse": 0.02425229363143444}, {"layer": 57, "field": "conv", "exact": false, "max_abs": 0.5625, "nrmse": 0.026350941509008408}, {"layer": 57, "field": "recurrent", "exact": false, "max_abs": 0.028930187225341797, "nrmse": 0.024824529886245728}, {"layer": 58, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.02958119846880436}, {"layer": 58, "field": "recurrent", "exact": false, "max_abs": 0.029146313667297363, "nrmse": 0.021685205399990082}, {"layer": 59, "field": "key", "exact": false, "max_abs": 0.296875, "nrmse": 0.012479153461754322}, {"layer": 59, "field": "value", "exact": false, "max_abs": 0.96875, "nrmse": 0.014049994759261608}, {"layer": 60, "field": "conv", "exact": false, "max_abs": 0.75, "nrmse": 0.024730728939175606}, {"layer": 60, "field": "recurrent", "exact": false, "max_abs": 0.04316309094429016, "nrmse": 0.026925474405288696}, {"layer": 61, "field": "conv", "exact": false, "max_abs": 0.28125, "nrmse": 0.02594613842666149}, {"layer": 61, "field": "recurrent", "exact": false, "max_abs": 0.01751190423965454, "nrmse": 0.02647143416106701}, {"layer": 62, "field": "conv", "exact": false, "max_abs": 0.75, "nrmse": 0.023993009701371193}, {"layer": 62, "field": "recurrent", "exact": false, "max_abs": 0.028656005859375, "nrmse": 0.019551344215869904}, {"layer": 63, "field": "key", "exact": false, "max_abs": 0.2734375, "nrmse": 0.013226238079369068}, {"layer": 63, "field": "value", "exact": false, "max_abs": 0.890625, "nrmse": 0.008090982213616371}], "buffered_graph_state_diff": []} -{"phase": "step", "index": 62, "position": 210, "teacher_token": 10342, "compact": {"ids": [2763, 7397, 104542, 6149, 14661], "logits": [26.0, 18.875, 15.25, 14.8125, 14.5], "margin": 7.125, "argmax": 2763}, "buffered": {"ids": [2763, 7397, 104542, 6149, 14661], "logits": [26.125, 18.875, 15.25, 14.6875, 14.5], "margin": 7.25, "argmax": 2763}, "graph": {"ids": [2763, 7397, 104542, 6149, 14661], "logits": [26.125, 18.875, 15.25, 14.6875, 14.5], "margin": 7.25, "argmax": 2763}, "compact_buffered": {"exact": false, "max_abs": 0.3125, "nrmse": 0.017815295606851578}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "compact_buffered_state_diff": [{"layer": 36, "field": "recurrent", "exact": false, "max_abs": 4.76837158203125e-07, "nrmse": 1.3157342948488804e-07}, {"layer": 37, "field": "recurrent", "exact": false, "max_abs": 0.00047915056347846985, "nrmse": 0.0004956306656822562}, {"layer": 38, "field": "conv", "exact": false, "max_abs": 0.078125, "nrmse": 0.012717080302536488}, {"layer": 38, "field": "recurrent", "exact": false, "max_abs": 0.005826801061630249, "nrmse": 0.0099977171048522}, {"layer": 39, "field": "key", "exact": false, "max_abs": 0.1875, "nrmse": 0.006720168516039848}, {"layer": 39, "field": "value", "exact": false, "max_abs": 0.21875, "nrmse": 0.008454367518424988}, {"layer": 40, "field": "conv", "exact": false, "max_abs": 0.125, "nrmse": 0.021968431770801544}, {"layer": 40, "field": "recurrent", "exact": false, "max_abs": 0.00837409496307373, "nrmse": 0.007713976316154003}, {"layer": 41, "field": "conv", "exact": false, "max_abs": 0.140625, "nrmse": 0.025454895570874214}, {"layer": 41, "field": "recurrent", "exact": false, "max_abs": 0.007067270576953888, "nrmse": 0.016154929995536804}, {"layer": 42, "field": "conv", "exact": false, "max_abs": 0.21875, "nrmse": 0.027865827083587646}, {"layer": 42, "field": "recurrent", "exact": false, "max_abs": 0.007596266455948353, "nrmse": 0.016429536044597626}, {"layer": 43, "field": "key", "exact": false, "max_abs": 0.3515625, "nrmse": 0.009794262237846851}, {"layer": 43, "field": "value", "exact": false, "max_abs": 0.31640625, "nrmse": 0.011028100736439228}, {"layer": 44, "field": "conv", "exact": false, "max_abs": 0.2276611328125, "nrmse": 0.029833795502781868}, {"layer": 44, "field": "recurrent", "exact": false, "max_abs": 0.012687712907791138, "nrmse": 0.0227518267929554}, {"layer": 45, "field": "conv", "exact": false, "max_abs": 0.21875, "nrmse": 0.0347442552447319}, {"layer": 45, "field": "recurrent", "exact": false, "max_abs": 0.012811243534088135, "nrmse": 0.025770828127861023}, {"layer": 46, "field": "conv", "exact": false, "max_abs": 0.25, "nrmse": 0.03227892518043518}, {"layer": 46, "field": "recurrent", "exact": false, "max_abs": 0.01571786403656006, "nrmse": 0.024780938401818275}, {"layer": 47, "field": "key", "exact": false, "max_abs": 0.55078125, "nrmse": 0.011758366599678993}, {"layer": 47, "field": "value", "exact": false, "max_abs": 0.33203125, "nrmse": 0.01530095748603344}, {"layer": 48, "field": "conv", "exact": false, "max_abs": 0.28125, "nrmse": 0.036020927131175995}, {"layer": 48, "field": "recurrent", "exact": false, "max_abs": 0.01111101359128952, "nrmse": 0.02759925276041031}, {"layer": 49, "field": "conv", "exact": false, "max_abs": 0.40625, "nrmse": 0.03625659644603729}, {"layer": 49, "field": "recurrent", "exact": false, "max_abs": 0.011469140648841858, "nrmse": 0.031121715903282166}, {"layer": 50, "field": "conv", "exact": false, "max_abs": 0.265625, "nrmse": 0.035141587257385254}, {"layer": 50, "field": "recurrent", "exact": false, "max_abs": 0.011447921395301819, "nrmse": 0.028378646820783615}, {"layer": 51, "field": "key", "exact": false, "max_abs": 0.359375, "nrmse": 0.012099895626306534}, {"layer": 51, "field": "value", "exact": false, "max_abs": 0.5625, "nrmse": 0.01625393144786358}, {"layer": 52, "field": "conv", "exact": false, "max_abs": 0.33203125, "nrmse": 0.02898094803094864}, {"layer": 52, "field": "recurrent", "exact": false, "max_abs": 0.03616988658905029, "nrmse": 0.02169324830174446}, {"layer": 53, "field": "conv", "exact": false, "max_abs": 0.2734375, "nrmse": 0.02957194112241268}, {"layer": 53, "field": "recurrent", "exact": false, "max_abs": 0.059038519859313965, "nrmse": 0.028380967676639557}, {"layer": 54, "field": "conv", "exact": false, "max_abs": 0.4375, "nrmse": 0.03289320692420006}, {"layer": 54, "field": "recurrent", "exact": false, "max_abs": 0.029377982020378113, "nrmse": 0.030195992439985275}, {"layer": 55, "field": "key", "exact": false, "max_abs": 0.3046875, "nrmse": 0.012405216693878174}, {"layer": 55, "field": "value", "exact": false, "max_abs": 0.75, "nrmse": 0.017562929540872574}, {"layer": 56, "field": "conv", "exact": false, "max_abs": 0.3828125, "nrmse": 0.02891295589506626}, {"layer": 56, "field": "recurrent", "exact": false, "max_abs": 0.03487744927406311, "nrmse": 0.024286016821861267}, {"layer": 57, "field": "conv", "exact": false, "max_abs": 0.5625, "nrmse": 0.026276810094714165}, {"layer": 57, "field": "recurrent", "exact": false, "max_abs": 0.03980839252471924, "nrmse": 0.025365468114614487}, {"layer": 58, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.02883073501288891}, {"layer": 58, "field": "recurrent", "exact": false, "max_abs": 0.026542454957962036, "nrmse": 0.0218500979244709}, {"layer": 59, "field": "key", "exact": false, "max_abs": 0.296875, "nrmse": 0.012563401833176613}, {"layer": 59, "field": "value", "exact": false, "max_abs": 0.96875, "nrmse": 0.01417210977524519}, {"layer": 60, "field": "conv", "exact": false, "max_abs": 0.75, "nrmse": 0.026167919859290123}, {"layer": 60, "field": "recurrent", "exact": false, "max_abs": 0.0428658127784729, "nrmse": 0.02700258046388626}, {"layer": 61, "field": "conv", "exact": false, "max_abs": 0.28125, "nrmse": 0.025794148445129395}, {"layer": 61, "field": "recurrent", "exact": false, "max_abs": 0.017594948410987854, "nrmse": 0.026556428521871567}, {"layer": 62, "field": "conv", "exact": false, "max_abs": 0.75, "nrmse": 0.024071861058473587}, {"layer": 62, "field": "recurrent", "exact": false, "max_abs": 0.031037628650665283, "nrmse": 0.01982223056256771}, {"layer": 63, "field": "key", "exact": false, "max_abs": 0.2734375, "nrmse": 0.013342333026230335}, {"layer": 63, "field": "value", "exact": false, "max_abs": 0.890625, "nrmse": 0.008142192848026752}], "buffered_graph_state_diff": []} -{"phase": "step", "index": 63, "position": 211, "teacher_token": 2763, "compact": {"ids": [7397, 91103, 17629, 25, 97255], "logits": [27.375, 14.625, 14.0, 13.625, 13.0], "margin": 12.75, "argmax": 7397}, "buffered": {"ids": [7397, 91103, 17629, 25, 97255], "logits": [27.375, 14.5625, 14.0, 13.6875, 12.9375], "margin": 12.8125, "argmax": 7397}, "graph": {"ids": [7397, 91103, 17629, 25, 97255], "logits": [27.375, 14.5625, 14.0, 13.6875, 12.9375], "margin": 12.8125, "argmax": 7397}, "compact_buffered": {"exact": false, "max_abs": 0.375, "nrmse": 0.017400141805410385}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "compact_buffered_state_diff": [{"layer": 36, "field": "recurrent", "exact": false, "max_abs": 4.76837158203125e-07, "nrmse": 1.3179162294818525e-07}, {"layer": 37, "field": "recurrent", "exact": false, "max_abs": 0.00047944486141204834, "nrmse": 0.0004960958613082767}, {"layer": 38, "field": "conv", "exact": false, "max_abs": 0.078125, "nrmse": 0.012037193402647972}, {"layer": 38, "field": "recurrent", "exact": false, "max_abs": 0.00582459568977356, "nrmse": 0.010081163607537746}, {"layer": 39, "field": "key", "exact": false, "max_abs": 0.1875, "nrmse": 0.006758806295692921}, {"layer": 39, "field": "value", "exact": false, "max_abs": 0.21875, "nrmse": 0.0085074407979846}, {"layer": 40, "field": "conv", "exact": false, "max_abs": 0.125, "nrmse": 0.020542757585644722}, {"layer": 40, "field": "recurrent", "exact": false, "max_abs": 0.011111140251159668, "nrmse": 0.007616524584591389}, {"layer": 41, "field": "conv", "exact": false, "max_abs": 0.125, "nrmse": 0.02313922718167305}, {"layer": 41, "field": "recurrent", "exact": false, "max_abs": 0.007062852382659912, "nrmse": 0.01645810529589653}, {"layer": 42, "field": "conv", "exact": false, "max_abs": 0.1328125, "nrmse": 0.025484764948487282}, {"layer": 42, "field": "recurrent", "exact": false, "max_abs": 0.006922334432601929, "nrmse": 0.016478387638926506}, {"layer": 43, "field": "key", "exact": false, "max_abs": 0.3515625, "nrmse": 0.00987751130014658}, {"layer": 43, "field": "value", "exact": false, "max_abs": 0.31640625, "nrmse": 0.011105243116617203}, {"layer": 44, "field": "conv", "exact": false, "max_abs": 0.1953125, "nrmse": 0.027044177055358887}, {"layer": 44, "field": "recurrent", "exact": false, "max_abs": 0.01265145093202591, "nrmse": 0.022748511284589767}, {"layer": 45, "field": "conv", "exact": false, "max_abs": 0.21875, "nrmse": 0.031332068145275116}, {"layer": 45, "field": "recurrent", "exact": false, "max_abs": 0.012812018394470215, "nrmse": 0.02594238892197609}, {"layer": 46, "field": "conv", "exact": false, "max_abs": 0.25, "nrmse": 0.030200203880667686}, {"layer": 46, "field": "recurrent", "exact": false, "max_abs": 0.015722155570983887, "nrmse": 0.024441983550786972}, {"layer": 47, "field": "key", "exact": false, "max_abs": 0.55078125, "nrmse": 0.011818552389740944}, {"layer": 47, "field": "value", "exact": false, "max_abs": 0.33203125, "nrmse": 0.015389814972877502}, {"layer": 48, "field": "conv", "exact": false, "max_abs": 0.2578125, "nrmse": 0.033829279243946075}, {"layer": 48, "field": "recurrent", "exact": false, "max_abs": 0.011070214211940765, "nrmse": 0.02723771333694458}, {"layer": 49, "field": "conv", "exact": false, "max_abs": 0.40625, "nrmse": 0.034229978919029236}, {"layer": 49, "field": "recurrent", "exact": false, "max_abs": 0.013281971216201782, "nrmse": 0.03122064657509327}, {"layer": 50, "field": "conv", "exact": false, "max_abs": 0.265625, "nrmse": 0.03322392702102661}, {"layer": 50, "field": "recurrent", "exact": false, "max_abs": 0.00828564167022705, "nrmse": 0.02882796712219715}, {"layer": 51, "field": "key", "exact": false, "max_abs": 0.359375, "nrmse": 0.012207192368805408}, {"layer": 51, "field": "value", "exact": false, "max_abs": 0.5625, "nrmse": 0.016425032168626785}, {"layer": 52, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.027218231931328773}, {"layer": 52, "field": "recurrent", "exact": false, "max_abs": 0.03402137756347656, "nrmse": 0.02173011749982834}, {"layer": 53, "field": "conv", "exact": false, "max_abs": 0.265625, "nrmse": 0.02813282050192356}, {"layer": 53, "field": "recurrent", "exact": false, "max_abs": 0.05917227268218994, "nrmse": 0.02800728566944599}, {"layer": 54, "field": "conv", "exact": false, "max_abs": 0.26171875, "nrmse": 0.03116484172642231}, {"layer": 54, "field": "recurrent", "exact": false, "max_abs": 0.02909103035926819, "nrmse": 0.030081234872341156}, {"layer": 55, "field": "key", "exact": false, "max_abs": 0.3046875, "nrmse": 0.01258009858429432}, {"layer": 55, "field": "value", "exact": false, "max_abs": 0.75, "nrmse": 0.01776362583041191}, {"layer": 56, "field": "conv", "exact": false, "max_abs": 0.28125, "nrmse": 0.027158793061971664}, {"layer": 56, "field": "recurrent", "exact": false, "max_abs": 0.03489166498184204, "nrmse": 0.02402500994503498}, {"layer": 57, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.024422870948910713}, {"layer": 57, "field": "recurrent", "exact": false, "max_abs": 0.030752897262573242, "nrmse": 0.02438790537416935}, {"layer": 58, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.0267791748046875}, {"layer": 58, "field": "recurrent", "exact": false, "max_abs": 0.026562616229057312, "nrmse": 0.021345481276512146}, {"layer": 59, "field": "key", "exact": false, "max_abs": 0.296875, "nrmse": 0.012677266262471676}, {"layer": 59, "field": "value", "exact": false, "max_abs": 0.96875, "nrmse": 0.01429841946810484}, {"layer": 60, "field": "conv", "exact": false, "max_abs": 0.75, "nrmse": 0.023955320939421654}, {"layer": 60, "field": "recurrent", "exact": false, "max_abs": 0.04248139262199402, "nrmse": 0.026871997863054276}, {"layer": 61, "field": "conv", "exact": false, "max_abs": 0.25, "nrmse": 0.02447502873837948}, {"layer": 61, "field": "recurrent", "exact": false, "max_abs": 0.016171693801879883, "nrmse": 0.02639758214354515}, {"layer": 62, "field": "conv", "exact": false, "max_abs": 0.6875, "nrmse": 0.023281795904040337}, {"layer": 62, "field": "recurrent", "exact": false, "max_abs": 0.030632972717285156, "nrmse": 0.019694585353136063}, {"layer": 63, "field": "key", "exact": false, "max_abs": 0.2734375, "nrmse": 0.01345925685018301}, {"layer": 63, "field": "value", "exact": false, "max_abs": 0.890625, "nrmse": 0.008189340122044086}], "buffered_graph_state_diff": []} -{"phase": "step", "index": 64, "position": 212, "teacher_token": 7397, "compact": {"ids": [25, 303, 283, 318, 393], "logits": [28.25, 19.625, 17.25, 17.0, 14.75], "margin": 8.625, "argmax": 25}, "buffered": {"ids": [25, 303, 283, 318, 393], "logits": [28.375, 19.75, 17.125, 16.75, 14.6875], "margin": 8.625, "argmax": 25}, "graph": {"ids": [25, 303, 283, 318, 393], "logits": [28.375, 19.75, 17.125, 16.75, 14.6875], "margin": 8.625, "argmax": 25}, "compact_buffered": {"exact": false, "max_abs": 0.4375, "nrmse": 0.030717618763446808}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "compact_buffered_state_diff": [{"layer": 36, "field": "recurrent", "exact": false, "max_abs": 4.76837158203125e-07, "nrmse": 1.317620359486682e-07}, {"layer": 37, "field": "recurrent", "exact": false, "max_abs": 0.0004707425832748413, "nrmse": 0.0004887034301646054}, {"layer": 38, "field": "conv", "exact": false, "max_abs": 0.0703125, "nrmse": 0.011204981245100498}, {"layer": 38, "field": "recurrent", "exact": false, "max_abs": 0.005810797214508057, "nrmse": 0.009928419254720211}, {"layer": 39, "field": "key", "exact": false, "max_abs": 0.1875, "nrmse": 0.006775327958166599}, {"layer": 39, "field": "value", "exact": false, "max_abs": 0.21875, "nrmse": 0.008549429476261139}, {"layer": 40, "field": "conv", "exact": false, "max_abs": 0.1015625, "nrmse": 0.019885199144482613}, {"layer": 40, "field": "recurrent", "exact": false, "max_abs": 0.008555948734283447, "nrmse": 0.007579618599265814}, {"layer": 41, "field": "conv", "exact": false, "max_abs": 0.125, "nrmse": 0.02228730544447899}, {"layer": 41, "field": "recurrent", "exact": false, "max_abs": 0.007055714726448059, "nrmse": 0.01642274297773838}, {"layer": 42, "field": "conv", "exact": false, "max_abs": 0.1328125, "nrmse": 0.02481212094426155}, {"layer": 42, "field": "recurrent", "exact": false, "max_abs": 0.006901532411575317, "nrmse": 0.016241785138845444}, {"layer": 43, "field": "key", "exact": false, "max_abs": 0.3515625, "nrmse": 0.009945233352482319}, {"layer": 43, "field": "value", "exact": false, "max_abs": 0.31640625, "nrmse": 0.011194489896297455}, {"layer": 44, "field": "conv", "exact": false, "max_abs": 0.1953125, "nrmse": 0.026823759078979492}, {"layer": 44, "field": "recurrent", "exact": false, "max_abs": 0.012633807957172394, "nrmse": 0.022853337228298187}, {"layer": 45, "field": "conv", "exact": false, "max_abs": 0.21875, "nrmse": 0.031754009425640106}, {"layer": 45, "field": "recurrent", "exact": false, "max_abs": 0.012815654277801514, "nrmse": 0.026021378114819527}, {"layer": 46, "field": "conv", "exact": false, "max_abs": 0.2158203125, "nrmse": 0.03079683892428875}, {"layer": 46, "field": "recurrent", "exact": false, "max_abs": 0.015704870223999023, "nrmse": 0.024745818227529526}, {"layer": 47, "field": "key", "exact": false, "max_abs": 0.55078125, "nrmse": 0.011912536807358265}, {"layer": 47, "field": "value", "exact": false, "max_abs": 0.33203125, "nrmse": 0.015502545051276684}, {"layer": 48, "field": "conv", "exact": false, "max_abs": 0.25, "nrmse": 0.03418891876935959}, {"layer": 48, "field": "recurrent", "exact": false, "max_abs": 0.011060461401939392, "nrmse": 0.02721722610294819}, {"layer": 49, "field": "conv", "exact": false, "max_abs": 0.40625, "nrmse": 0.03480084240436554}, {"layer": 49, "field": "recurrent", "exact": false, "max_abs": 0.011041320860385895, "nrmse": 0.031158341094851494}, {"layer": 50, "field": "conv", "exact": false, "max_abs": 0.265625, "nrmse": 0.033834949135780334}, {"layer": 50, "field": "recurrent", "exact": false, "max_abs": 0.007741093635559082, "nrmse": 0.028803978115320206}, {"layer": 51, "field": "key", "exact": false, "max_abs": 0.359375, "nrmse": 0.012287819758057594}, {"layer": 51, "field": "value", "exact": false, "max_abs": 0.5625, "nrmse": 0.016569746658205986}, {"layer": 52, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.02783527411520481}, {"layer": 52, "field": "recurrent", "exact": false, "max_abs": 0.03327062726020813, "nrmse": 0.021622328087687492}, {"layer": 53, "field": "conv", "exact": false, "max_abs": 0.28125, "nrmse": 0.02882871963083744}, {"layer": 53, "field": "recurrent", "exact": false, "max_abs": 0.059037208557128906, "nrmse": 0.02810850739479065}, {"layer": 54, "field": "conv", "exact": false, "max_abs": 0.2939453125, "nrmse": 0.03208233416080475}, {"layer": 54, "field": "recurrent", "exact": false, "max_abs": 0.029112860560417175, "nrmse": 0.030643953010439873}, {"layer": 55, "field": "key", "exact": false, "max_abs": 0.3046875, "nrmse": 0.012676025740802288}, {"layer": 55, "field": "value", "exact": false, "max_abs": 0.75, "nrmse": 0.01795388199388981}, {"layer": 56, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.027484320104122162}, {"layer": 56, "field": "recurrent", "exact": false, "max_abs": 0.03491407632827759, "nrmse": 0.023909056559205055}, {"layer": 57, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.024766677990555763}, {"layer": 57, "field": "recurrent", "exact": false, "max_abs": 0.03294944763183594, "nrmse": 0.02474091202020645}, {"layer": 58, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.027413327246904373}, {"layer": 58, "field": "recurrent", "exact": false, "max_abs": 0.026504039764404297, "nrmse": 0.02167584002017975}, {"layer": 59, "field": "key", "exact": false, "max_abs": 0.296875, "nrmse": 0.012738650664687157}, {"layer": 59, "field": "value", "exact": false, "max_abs": 0.96875, "nrmse": 0.014431538060307503}, {"layer": 60, "field": "conv", "exact": false, "max_abs": 0.75, "nrmse": 0.024751003831624985}, {"layer": 60, "field": "recurrent", "exact": false, "max_abs": 0.04275941848754883, "nrmse": 0.02664216049015522}, {"layer": 61, "field": "conv", "exact": false, "max_abs": 0.25, "nrmse": 0.02515607699751854}, {"layer": 61, "field": "recurrent", "exact": false, "max_abs": 0.027143999934196472, "nrmse": 0.026562370359897614}, {"layer": 62, "field": "conv", "exact": false, "max_abs": 0.6875, "nrmse": 0.02383265271782875}, {"layer": 62, "field": "recurrent", "exact": false, "max_abs": 0.030590176582336426, "nrmse": 0.01963697001338005}, {"layer": 63, "field": "key", "exact": false, "max_abs": 0.2734375, "nrmse": 0.013582268729805946}, {"layer": 63, "field": "value", "exact": false, "max_abs": 0.890625, "nrmse": 0.008258823305368423}], "buffered_graph_state_diff": []} -{"phase": "step", "index": 65, "position": 213, "teacher_token": 25, "compact": {"ids": [220, 393, 17237, 56014, 3451], "logits": [24.5, 24.375, 15.375, 12.375, 11.5], "margin": 0.125, "argmax": 220}, "buffered": {"ids": [393, 220, 17237, 56014, 3451], "logits": [24.25, 24.25, 15.25, 12.125, 11.375], "margin": 0.0, "argmax": 220}, "graph": {"ids": [393, 220, 17237, 56014, 3451], "logits": [24.25, 24.25, 15.25, 12.125, 11.375], "margin": 0.0, "argmax": 220}, "compact_buffered": {"exact": false, "max_abs": 0.3125, "nrmse": 0.01519216038286686}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "compact_buffered_state_diff": [{"layer": 36, "field": "recurrent", "exact": false, "max_abs": 4.76837158203125e-07, "nrmse": 1.319516229614237e-07}, {"layer": 37, "field": "recurrent", "exact": false, "max_abs": 0.00046594440937042236, "nrmse": 0.0004902757355012}, {"layer": 38, "field": "conv", "exact": false, "max_abs": 0.0703125, "nrmse": 0.010721971280872822}, {"layer": 38, "field": "recurrent", "exact": false, "max_abs": 0.005802541971206665, "nrmse": 0.009900039061903954}, {"layer": 39, "field": "key", "exact": false, "max_abs": 0.1875, "nrmse": 0.0068567534908652306}, {"layer": 39, "field": "value", "exact": false, "max_abs": 0.21875, "nrmse": 0.008635692298412323}, {"layer": 40, "field": "conv", "exact": false, "max_abs": 0.1171875, "nrmse": 0.01889331080019474}, {"layer": 40, "field": "recurrent", "exact": false, "max_abs": 0.008986413478851318, "nrmse": 0.00749638956040144}, {"layer": 41, "field": "conv", "exact": false, "max_abs": 0.125, "nrmse": 0.02179292030632496}, {"layer": 41, "field": "recurrent", "exact": false, "max_abs": 0.007048889994621277, "nrmse": 0.016234736889600754}, {"layer": 42, "field": "conv", "exact": false, "max_abs": 0.11328125, "nrmse": 0.023885240778326988}, {"layer": 42, "field": "recurrent", "exact": false, "max_abs": 0.007767710834741592, "nrmse": 0.01631094142794609}, {"layer": 43, "field": "key", "exact": false, "max_abs": 0.3515625, "nrmse": 0.010035951621830463}, {"layer": 43, "field": "value", "exact": false, "max_abs": 0.31640625, "nrmse": 0.011301401071250439}, {"layer": 44, "field": "conv", "exact": false, "max_abs": 0.1875, "nrmse": 0.02494990825653076}, {"layer": 44, "field": "recurrent", "exact": false, "max_abs": 0.012611784040927887, "nrmse": 0.02280745469033718}, {"layer": 45, "field": "conv", "exact": false, "max_abs": 0.16162109375, "nrmse": 0.029433120042085648}, {"layer": 45, "field": "recurrent", "exact": false, "max_abs": 0.012811779975891113, "nrmse": 0.02594652585685253}, {"layer": 46, "field": "conv", "exact": false, "max_abs": 0.2158203125, "nrmse": 0.029091574251651764}, {"layer": 46, "field": "recurrent", "exact": false, "max_abs": 0.015705883502960205, "nrmse": 0.02451191656291485}, {"layer": 47, "field": "key", "exact": false, "max_abs": 0.55078125, "nrmse": 0.011993265710771084}, {"layer": 47, "field": "value", "exact": false, "max_abs": 0.33203125, "nrmse": 0.015637554228305817}, {"layer": 48, "field": "conv", "exact": false, "max_abs": 0.203125, "nrmse": 0.03127576410770416}, {"layer": 48, "field": "recurrent", "exact": false, "max_abs": 0.011013932526111603, "nrmse": 0.02691027894616127}, {"layer": 49, "field": "conv", "exact": false, "max_abs": 0.21875, "nrmse": 0.03150193765759468}, {"layer": 49, "field": "recurrent", "exact": false, "max_abs": 0.011018380522727966, "nrmse": 0.03175901249051094}, {"layer": 50, "field": "conv", "exact": false, "max_abs": 0.21875, "nrmse": 0.03167136386036873}, {"layer": 50, "field": "recurrent", "exact": false, "max_abs": 0.007252991199493408, "nrmse": 0.028710946440696716}, {"layer": 51, "field": "key", "exact": false, "max_abs": 0.359375, "nrmse": 0.012458245269954205}, {"layer": 51, "field": "value", "exact": false, "max_abs": 0.5625, "nrmse": 0.016742508858442307}, {"layer": 52, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.02714349329471588}, {"layer": 52, "field": "recurrent", "exact": false, "max_abs": 0.03487706184387207, "nrmse": 0.02169577218592167}, {"layer": 53, "field": "conv", "exact": false, "max_abs": 0.337890625, "nrmse": 0.028445161879062653}, {"layer": 53, "field": "recurrent", "exact": false, "max_abs": 0.0581517219543457, "nrmse": 0.028245337307453156}, {"layer": 54, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.03235010430216789}, {"layer": 54, "field": "recurrent", "exact": false, "max_abs": 0.028997167944908142, "nrmse": 0.03101927787065506}, {"layer": 55, "field": "key", "exact": false, "max_abs": 0.3046875, "nrmse": 0.012891429476439953}, {"layer": 55, "field": "value", "exact": false, "max_abs": 0.75, "nrmse": 0.018151672556996346}, {"layer": 56, "field": "conv", "exact": false, "max_abs": 0.40625, "nrmse": 0.02750670723617077}, {"layer": 56, "field": "recurrent", "exact": false, "max_abs": 0.03524750471115112, "nrmse": 0.0242528785020113}, {"layer": 57, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.024993566796183586}, {"layer": 57, "field": "recurrent", "exact": false, "max_abs": 0.026114150881767273, "nrmse": 0.025667931884527206}, {"layer": 58, "field": "conv", "exact": false, "max_abs": 0.40625, "nrmse": 0.0274082962423563}, {"layer": 58, "field": "recurrent", "exact": false, "max_abs": 0.027039647102355957, "nrmse": 0.022219346836209297}, {"layer": 59, "field": "key", "exact": false, "max_abs": 0.296875, "nrmse": 0.012833996675908566}, {"layer": 59, "field": "value", "exact": false, "max_abs": 0.96875, "nrmse": 0.014573655091226101}, {"layer": 60, "field": "conv", "exact": false, "max_abs": 0.75, "nrmse": 0.024317026138305664}, {"layer": 60, "field": "recurrent", "exact": false, "max_abs": 0.04257717728614807, "nrmse": 0.026925090700387955}, {"layer": 61, "field": "conv", "exact": false, "max_abs": 0.28125, "nrmse": 0.025076013058423996}, {"layer": 61, "field": "recurrent", "exact": false, "max_abs": 0.016568288207054138, "nrmse": 0.02679397165775299}, {"layer": 62, "field": "conv", "exact": false, "max_abs": 0.39453125, "nrmse": 0.022683415561914444}, {"layer": 62, "field": "recurrent", "exact": false, "max_abs": 0.031035661697387695, "nrmse": 0.01964288391172886}, {"layer": 63, "field": "key", "exact": false, "max_abs": 0.2734375, "nrmse": 0.013751834630966187}, {"layer": 63, "field": "value", "exact": false, "max_abs": 0.890625, "nrmse": 0.00831931084394455}], "buffered_graph_state_diff": []} -{"phase": "step", "index": 66, "position": 214, "teacher_token": 220, "compact": {"ids": [22, 18, 24, 19, 16], "logits": [27.375, 23.25, 14.6875, 13.875, 13.75], "margin": 4.125, "argmax": 22}, "buffered": {"ids": [22, 18, 24, 23, 19], "logits": [27.375, 23.375, 14.625, 13.6875, 13.6875], "margin": 4.0, "argmax": 22}, "graph": {"ids": [22, 18, 24, 23, 19], "logits": [27.375, 23.375, 14.625, 13.6875, 13.6875], "margin": 4.0, "argmax": 22}, "compact_buffered": {"exact": false, "max_abs": 0.375, "nrmse": 0.0220460407435894}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "compact_buffered_state_diff": [{"layer": 36, "field": "recurrent", "exact": false, "max_abs": 4.76837158203125e-07, "nrmse": 1.3107298002523748e-07}, {"layer": 37, "field": "recurrent", "exact": false, "max_abs": 0.0004642792046070099, "nrmse": 0.00048058637185022235}, {"layer": 38, "field": "conv", "exact": false, "max_abs": 0.0703125, "nrmse": 0.011463121511042118}, {"layer": 38, "field": "recurrent", "exact": false, "max_abs": 0.0058172643184661865, "nrmse": 0.01006445661187172}, {"layer": 39, "field": "key", "exact": false, "max_abs": 0.1875, "nrmse": 0.006925580557435751}, {"layer": 39, "field": "value", "exact": false, "max_abs": 0.21875, "nrmse": 0.008764209225773811}, {"layer": 40, "field": "conv", "exact": false, "max_abs": 0.1875, "nrmse": 0.021612534299492836}, {"layer": 40, "field": "recurrent", "exact": false, "max_abs": 0.009958922863006592, "nrmse": 0.00821105670183897}, {"layer": 41, "field": "conv", "exact": false, "max_abs": 0.15625, "nrmse": 0.024542206898331642}, {"layer": 41, "field": "recurrent", "exact": false, "max_abs": 0.007061898708343506, "nrmse": 0.016659686341881752}, {"layer": 42, "field": "conv", "exact": false, "max_abs": 0.15625, "nrmse": 0.026515409350395203}, {"layer": 42, "field": "recurrent", "exact": false, "max_abs": 0.007802367210388184, "nrmse": 0.016457438468933105}, {"layer": 43, "field": "key", "exact": false, "max_abs": 0.3515625, "nrmse": 0.010142017155885696}, {"layer": 43, "field": "value", "exact": false, "max_abs": 0.31640625, "nrmse": 0.011493762023746967}, {"layer": 44, "field": "conv", "exact": false, "max_abs": 0.17578125, "nrmse": 0.02806347794830799}, {"layer": 44, "field": "recurrent", "exact": false, "max_abs": 0.012611784040927887, "nrmse": 0.023099355399608612}, {"layer": 45, "field": "conv", "exact": false, "max_abs": 0.21875, "nrmse": 0.03301016986370087}, {"layer": 45, "field": "recurrent", "exact": false, "max_abs": 0.012803852558135986, "nrmse": 0.02604263462126255}, {"layer": 46, "field": "conv", "exact": false, "max_abs": 0.19140625, "nrmse": 0.032892655581235886}, {"layer": 46, "field": "recurrent", "exact": false, "max_abs": 0.01572132110595703, "nrmse": 0.02486160211265087}, {"layer": 47, "field": "key", "exact": false, "max_abs": 0.55078125, "nrmse": 0.012151086702942848}, {"layer": 47, "field": "value", "exact": false, "max_abs": 0.33203125, "nrmse": 0.015949828550219536}, {"layer": 48, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.034115079790353775}, {"layer": 48, "field": "recurrent", "exact": false, "max_abs": 0.010975025594234467, "nrmse": 0.02698897384107113}, {"layer": 49, "field": "conv", "exact": false, "max_abs": 0.359375, "nrmse": 0.035236965864896774}, {"layer": 49, "field": "recurrent", "exact": false, "max_abs": 0.013257734477519989, "nrmse": 0.031898245215415955}, {"layer": 50, "field": "conv", "exact": false, "max_abs": 0.3515625, "nrmse": 0.03490085527300835}, {"layer": 50, "field": "recurrent", "exact": false, "max_abs": 0.007275231182575226, "nrmse": 0.029177118092775345}, {"layer": 51, "field": "key", "exact": false, "max_abs": 0.359375, "nrmse": 0.012663537636399269}, {"layer": 51, "field": "value", "exact": false, "max_abs": 0.5625, "nrmse": 0.017019756138324738}, {"layer": 52, "field": "conv", "exact": false, "max_abs": 0.375, "nrmse": 0.029431991279125214}, {"layer": 52, "field": "recurrent", "exact": false, "max_abs": 0.033090412616729736, "nrmse": 0.021836457774043083}, {"layer": 53, "field": "conv", "exact": false, "max_abs": 0.359375, "nrmse": 0.030701540410518646}, {"layer": 53, "field": "recurrent", "exact": false, "max_abs": 0.057674407958984375, "nrmse": 0.028918176889419556}, {"layer": 54, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.0349639393389225}, {"layer": 54, "field": "recurrent", "exact": false, "max_abs": 0.028174445033073425, "nrmse": 0.03154224529862404}, {"layer": 55, "field": "key", "exact": false, "max_abs": 0.3046875, "nrmse": 0.013159043155610561}, {"layer": 55, "field": "value", "exact": false, "max_abs": 0.75, "nrmse": 0.018389565870165825}, {"layer": 56, "field": "conv", "exact": false, "max_abs": 0.40625, "nrmse": 0.029690027236938477}, {"layer": 56, "field": "recurrent", "exact": false, "max_abs": 0.03529155254364014, "nrmse": 0.02444147877395153}, {"layer": 57, "field": "conv", "exact": false, "max_abs": 0.326171875, "nrmse": 0.02689807489514351}, {"layer": 57, "field": "recurrent", "exact": false, "max_abs": 0.06928598880767822, "nrmse": 0.026160389184951782}, {"layer": 58, "field": "conv", "exact": false, "max_abs": 0.40625, "nrmse": 0.030110888183116913}, {"layer": 58, "field": "recurrent", "exact": false, "max_abs": 0.02655157446861267, "nrmse": 0.022649839520454407}, {"layer": 59, "field": "key", "exact": false, "max_abs": 0.296875, "nrmse": 0.013064769096672535}, {"layer": 59, "field": "value", "exact": false, "max_abs": 0.96875, "nrmse": 0.014761633239686489}, {"layer": 60, "field": "conv", "exact": false, "max_abs": 1.25, "nrmse": 0.024701528251171112}, {"layer": 60, "field": "recurrent", "exact": false, "max_abs": 0.04120251536369324, "nrmse": 0.027091283351182938}, {"layer": 61, "field": "conv", "exact": false, "max_abs": 0.4375, "nrmse": 0.026924967765808105}, {"layer": 61, "field": "recurrent", "exact": false, "max_abs": 0.01452113687992096, "nrmse": 0.027584506198763847}, {"layer": 62, "field": "conv", "exact": false, "max_abs": 0.8125, "nrmse": 0.024592431262135506}, {"layer": 62, "field": "recurrent", "exact": false, "max_abs": 0.031226396560668945, "nrmse": 0.01993986964225769}, {"layer": 63, "field": "key", "exact": false, "max_abs": 0.2734375, "nrmse": 0.013948993757367134}, {"layer": 63, "field": "value", "exact": false, "max_abs": 0.890625, "nrmse": 0.008419963531196117}], "buffered_graph_state_diff": []} -{"phase": "step", "index": 67, "position": 215, "teacher_token": 22, "compact": {"ids": [271, 318, 198, 1651, 73879], "logits": [28.125, 24.25, 17.5, 15.5625, 13.625], "margin": 3.875, "argmax": 271}, "buffered": {"ids": [271, 318, 198, 1651, 13], "logits": [28.375, 24.375, 17.625, 15.4375, 13.625], "margin": 4.0, "argmax": 271}, "graph": {"ids": [271, 318, 198, 1651, 13], "logits": [28.375, 24.375, 17.625, 15.4375, 13.625], "margin": 4.0, "argmax": 271}, "compact_buffered": {"exact": false, "max_abs": 0.4375, "nrmse": 0.021991174668073654}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "compact_buffered_state_diff": [{"layer": 36, "field": "recurrent", "exact": false, "max_abs": 4.76837158203125e-07, "nrmse": 1.310888819716638e-07}, {"layer": 37, "field": "recurrent", "exact": false, "max_abs": 0.00046476349234580994, "nrmse": 0.0004837981832679361}, {"layer": 38, "field": "conv", "exact": false, "max_abs": 0.09375, "nrmse": 0.012096703052520752}, {"layer": 38, "field": "recurrent", "exact": false, "max_abs": 0.0058179497718811035, "nrmse": 0.010278497822582722}, {"layer": 39, "field": "key", "exact": false, "max_abs": 0.1875, "nrmse": 0.006952171213924885}, {"layer": 39, "field": "value", "exact": false, "max_abs": 0.21875, "nrmse": 0.008831789717078209}, {"layer": 40, "field": "conv", "exact": false, "max_abs": 0.1875, "nrmse": 0.021725766360759735}, {"layer": 40, "field": "recurrent", "exact": false, "max_abs": 0.009490907192230225, "nrmse": 0.008103144355118275}, {"layer": 41, "field": "conv", "exact": false, "max_abs": 0.15625, "nrmse": 0.024802755564451218}, {"layer": 41, "field": "recurrent", "exact": false, "max_abs": 0.0070378780364990234, "nrmse": 0.016633441671729088}, {"layer": 42, "field": "conv", "exact": false, "max_abs": 0.15625, "nrmse": 0.027320511639118195}, {"layer": 42, "field": "recurrent", "exact": false, "max_abs": 0.006774187088012695, "nrmse": 0.01631789840757847}, {"layer": 43, "field": "key", "exact": false, "max_abs": 0.3515625, "nrmse": 0.01021080743521452}, {"layer": 43, "field": "value", "exact": false, "max_abs": 0.31640625, "nrmse": 0.011582923121750355}, {"layer": 44, "field": "conv", "exact": false, "max_abs": 0.17578125, "nrmse": 0.029071589931845665}, {"layer": 44, "field": "recurrent", "exact": false, "max_abs": 0.01255703717470169, "nrmse": 0.02319546975195408}, {"layer": 45, "field": "conv", "exact": false, "max_abs": 0.21875, "nrmse": 0.034488189965486526}, {"layer": 45, "field": "recurrent", "exact": false, "max_abs": 0.012805044651031494, "nrmse": 0.026219608262181282}, {"layer": 46, "field": "conv", "exact": false, "max_abs": 0.19140625, "nrmse": 0.03360294923186302}, {"layer": 46, "field": "recurrent", "exact": false, "max_abs": 0.015755295753479004, "nrmse": 0.025278467684984207}, {"layer": 47, "field": "key", "exact": false, "max_abs": 0.55078125, "nrmse": 0.012252116575837135}, {"layer": 47, "field": "value", "exact": false, "max_abs": 0.33203125, "nrmse": 0.016099579632282257}, {"layer": 48, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.035160623490810394}, {"layer": 48, "field": "recurrent", "exact": false, "max_abs": 0.010938875377178192, "nrmse": 0.027293965220451355}, {"layer": 49, "field": "conv", "exact": false, "max_abs": 0.359375, "nrmse": 0.037547990679740906}, {"layer": 49, "field": "recurrent", "exact": false, "max_abs": 0.016437247395515442, "nrmse": 0.032328251749277115}, {"layer": 50, "field": "conv", "exact": false, "max_abs": 0.3515625, "nrmse": 0.03797193989157677}, {"layer": 50, "field": "recurrent", "exact": false, "max_abs": 0.007286481559276581, "nrmse": 0.029770735651254654}, {"layer": 51, "field": "key", "exact": false, "max_abs": 0.359375, "nrmse": 0.012789592146873474}, {"layer": 51, "field": "value", "exact": false, "max_abs": 0.5625, "nrmse": 0.017213838174939156}, {"layer": 52, "field": "conv", "exact": false, "max_abs": 0.40625, "nrmse": 0.030818525701761246}, {"layer": 52, "field": "recurrent", "exact": false, "max_abs": 0.033189237117767334, "nrmse": 0.021829547360539436}, {"layer": 53, "field": "conv", "exact": false, "max_abs": 0.359375, "nrmse": 0.03156638890504837}, {"layer": 53, "field": "recurrent", "exact": false, "max_abs": 0.05752360820770264, "nrmse": 0.02895497716963291}, {"layer": 54, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.03587554395198822}, {"layer": 54, "field": "recurrent", "exact": false, "max_abs": 0.027863815426826477, "nrmse": 0.031742364168167114}, {"layer": 55, "field": "key", "exact": false, "max_abs": 0.3046875, "nrmse": 0.013292919844388962}, {"layer": 55, "field": "value", "exact": false, "max_abs": 0.75, "nrmse": 0.018548373132944107}, {"layer": 56, "field": "conv", "exact": false, "max_abs": 0.40625, "nrmse": 0.03054843842983246}, {"layer": 56, "field": "recurrent", "exact": false, "max_abs": 0.03538593649864197, "nrmse": 0.02436177246272564}, {"layer": 57, "field": "conv", "exact": false, "max_abs": 0.326171875, "nrmse": 0.02807268314063549}, {"layer": 57, "field": "recurrent", "exact": false, "max_abs": 0.06363213062286377, "nrmse": 0.02604454755783081}, {"layer": 58, "field": "conv", "exact": false, "max_abs": 0.40625, "nrmse": 0.03179581090807915}, {"layer": 58, "field": "recurrent", "exact": false, "max_abs": 0.027960270643234253, "nrmse": 0.022666269913315773}, {"layer": 59, "field": "key", "exact": false, "max_abs": 0.296875, "nrmse": 0.013235645368695259}, {"layer": 59, "field": "value", "exact": false, "max_abs": 0.96875, "nrmse": 0.014896702021360397}, {"layer": 60, "field": "conv", "exact": false, "max_abs": 1.25, "nrmse": 0.026232916861772537}, {"layer": 60, "field": "recurrent", "exact": false, "max_abs": 0.04189977049827576, "nrmse": 0.026875963434576988}, {"layer": 61, "field": "conv", "exact": false, "max_abs": 0.4375, "nrmse": 0.028101742267608643}, {"layer": 61, "field": "recurrent", "exact": false, "max_abs": 0.0154016874730587, "nrmse": 0.0274188332259655}, {"layer": 62, "field": "conv", "exact": false, "max_abs": 0.8125, "nrmse": 0.02525688335299492}, {"layer": 62, "field": "recurrent", "exact": false, "max_abs": 0.031109333038330078, "nrmse": 0.01979129947721958}, {"layer": 63, "field": "key", "exact": false, "max_abs": 0.2734375, "nrmse": 0.01413693092763424}, {"layer": 63, "field": "value", "exact": false, "max_abs": 0.890625, "nrmse": 0.008495582267642021}], "buffered_graph_state_diff": []} -{"phase": "step", "index": 68, "position": 216, "teacher_token": 271, "compact": {"ids": [332, 9764, 17, 8839, 31318], "logits": [29.0, 19.875, 15.5, 14.9375, 14.625], "margin": 9.125, "argmax": 332}, "buffered": {"ids": [332, 9764, 17, 8839, 31318], "logits": [29.0, 19.875, 15.4375, 14.875, 14.6875], "margin": 9.125, "argmax": 332}, "graph": {"ids": [332, 9764, 17, 8839, 31318], "logits": [29.0, 19.875, 15.4375, 14.875, 14.6875], "margin": 9.125, "argmax": 332}, "compact_buffered": {"exact": false, "max_abs": 0.34375, "nrmse": 0.02950025163590908}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "compact_buffered_state_diff": [{"layer": 36, "field": "recurrent", "exact": false, "max_abs": 4.76837158203125e-07, "nrmse": 1.338663793148953e-07}, {"layer": 37, "field": "recurrent", "exact": false, "max_abs": 0.0004643388092517853, "nrmse": 0.0004956851480528712}, {"layer": 38, "field": "conv", "exact": false, "max_abs": 0.09375, "nrmse": 0.01140000019222498}, {"layer": 38, "field": "recurrent", "exact": false, "max_abs": 0.0057901740074157715, "nrmse": 0.00932728685438633}, {"layer": 39, "field": "key", "exact": false, "max_abs": 0.1875, "nrmse": 0.0069938357919454575}, {"layer": 39, "field": "value", "exact": false, "max_abs": 0.21875, "nrmse": 0.00889302883297205}, {"layer": 40, "field": "conv", "exact": false, "max_abs": 0.1875, "nrmse": 0.021500153467059135}, {"layer": 40, "field": "recurrent", "exact": false, "max_abs": 0.00836646556854248, "nrmse": 0.007431531324982643}, {"layer": 41, "field": "conv", "exact": false, "max_abs": 0.15625, "nrmse": 0.0250054020434618}, {"layer": 41, "field": "recurrent", "exact": false, "max_abs": 0.007035255432128906, "nrmse": 0.015719713643193245}, {"layer": 42, "field": "conv", "exact": false, "max_abs": 0.15625, "nrmse": 0.02694537118077278}, {"layer": 42, "field": "recurrent", "exact": false, "max_abs": 0.0073366835713386536, "nrmse": 0.016090326011180878}, {"layer": 43, "field": "key", "exact": false, "max_abs": 0.3515625, "nrmse": 0.010297763161361217}, {"layer": 43, "field": "value", "exact": false, "max_abs": 0.31640625, "nrmse": 0.011698564514517784}, {"layer": 44, "field": "conv", "exact": false, "max_abs": 0.17578125, "nrmse": 0.02925056591629982}, {"layer": 44, "field": "recurrent", "exact": false, "max_abs": 0.012604296207427979, "nrmse": 0.02290099300444126}, {"layer": 45, "field": "conv", "exact": false, "max_abs": 0.21875, "nrmse": 0.03337447717785835}, {"layer": 45, "field": "recurrent", "exact": false, "max_abs": 0.012798786163330078, "nrmse": 0.02529366873204708}, {"layer": 46, "field": "conv", "exact": false, "max_abs": 0.19140625, "nrmse": 0.03213127329945564}, {"layer": 46, "field": "recurrent", "exact": false, "max_abs": 0.015730857849121094, "nrmse": 0.02489450015127659}, {"layer": 47, "field": "key", "exact": false, "max_abs": 0.55078125, "nrmse": 0.012384233996272087}, {"layer": 47, "field": "value", "exact": false, "max_abs": 0.33203125, "nrmse": 0.016231289133429527}, {"layer": 48, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.03431323170661926}, {"layer": 48, "field": "recurrent", "exact": false, "max_abs": 0.01078583300113678, "nrmse": 0.026955848559737206}, {"layer": 49, "field": "conv", "exact": false, "max_abs": 0.375, "nrmse": 0.03695172816514969}, {"layer": 49, "field": "recurrent", "exact": false, "max_abs": 0.012426912784576416, "nrmse": 0.03172943741083145}, {"layer": 50, "field": "conv", "exact": false, "max_abs": 0.3515625, "nrmse": 0.036995433270931244}, {"layer": 50, "field": "recurrent", "exact": false, "max_abs": 0.007292784750461578, "nrmse": 0.029433678835630417}, {"layer": 51, "field": "key", "exact": false, "max_abs": 0.359375, "nrmse": 0.01291137095540762}, {"layer": 51, "field": "value", "exact": false, "max_abs": 0.5625, "nrmse": 0.01737634278833866}, {"layer": 52, "field": "conv", "exact": false, "max_abs": 0.40625, "nrmse": 0.03063850849866867}, {"layer": 52, "field": "recurrent", "exact": false, "max_abs": 0.03516554832458496, "nrmse": 0.02138432487845421}, {"layer": 53, "field": "conv", "exact": false, "max_abs": 0.359375, "nrmse": 0.03172092139720917}, {"layer": 53, "field": "recurrent", "exact": false, "max_abs": 0.057280898094177246, "nrmse": 0.02867158129811287}, {"layer": 54, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.03552153334021568}, {"layer": 54, "field": "recurrent", "exact": false, "max_abs": 0.027003690600395203, "nrmse": 0.030904164537787437}, {"layer": 55, "field": "key", "exact": false, "max_abs": 0.3046875, "nrmse": 0.013350564986467361}, {"layer": 55, "field": "value", "exact": false, "max_abs": 0.75, "nrmse": 0.018703145906329155}, {"layer": 56, "field": "conv", "exact": false, "max_abs": 0.40625, "nrmse": 0.031281616538763046}, {"layer": 56, "field": "recurrent", "exact": false, "max_abs": 0.03537675738334656, "nrmse": 0.02459886111319065}, {"layer": 57, "field": "conv", "exact": false, "max_abs": 0.4375, "nrmse": 0.030053576454520226}, {"layer": 57, "field": "recurrent", "exact": false, "max_abs": 0.03226083517074585, "nrmse": 0.02606305666267872}, {"layer": 58, "field": "conv", "exact": false, "max_abs": 0.40625, "nrmse": 0.032232560217380524}, {"layer": 58, "field": "recurrent", "exact": false, "max_abs": 0.028638005256652832, "nrmse": 0.02247922308743}, {"layer": 59, "field": "key", "exact": false, "max_abs": 0.296875, "nrmse": 0.01327506173402071}, {"layer": 59, "field": "value", "exact": false, "max_abs": 0.96875, "nrmse": 0.014964742586016655}, {"layer": 60, "field": "conv", "exact": false, "max_abs": 1.25, "nrmse": 0.027069423347711563}, {"layer": 60, "field": "recurrent", "exact": false, "max_abs": 0.04323127865791321, "nrmse": 0.02709638886153698}, {"layer": 61, "field": "conv", "exact": false, "max_abs": 0.4375, "nrmse": 0.028518227860331535}, {"layer": 61, "field": "recurrent", "exact": false, "max_abs": 0.014349933713674545, "nrmse": 0.02713284082710743}, {"layer": 62, "field": "conv", "exact": false, "max_abs": 0.8125, "nrmse": 0.025455676019191742}, {"layer": 62, "field": "recurrent", "exact": false, "max_abs": 0.03216969966888428, "nrmse": 0.01975143328309059}, {"layer": 63, "field": "key", "exact": false, "max_abs": 0.2734375, "nrmse": 0.014241613447666168}, {"layer": 63, "field": "value", "exact": false, "max_abs": 0.890625, "nrmse": 0.00855769868940115}], "buffered_graph_state_diff": []} -{"phase": "step", "index": 69, "position": 217, "teacher_token": 332, "compact": {"ids": [17, 18, 16, 19, 8046], "logits": [31.5, 16.75, 15.3125, 14.75, 13.5], "margin": 14.75, "argmax": 17}, "buffered": {"ids": [17, 18, 16, 19, 8046], "logits": [31.375, 16.875, 15.625, 14.6875, 13.1875], "margin": 14.5, "argmax": 17}, "graph": {"ids": [17, 18, 16, 19, 8046], "logits": [31.375, 16.875, 15.625, 14.6875, 13.1875], "margin": 14.5, "argmax": 17}, "compact_buffered": {"exact": false, "max_abs": 1.125, "nrmse": 0.06049276515841484}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "compact_buffered_state_diff": [{"layer": 36, "field": "recurrent", "exact": false, "max_abs": 4.76837158203125e-07, "nrmse": 1.3247188235254725e-07}, {"layer": 37, "field": "recurrent", "exact": false, "max_abs": 0.0004588216543197632, "nrmse": 0.00047694711247459054}, {"layer": 38, "field": "conv", "exact": false, "max_abs": 0.09375, "nrmse": 0.012104312889277935}, {"layer": 38, "field": "recurrent", "exact": false, "max_abs": 0.00650632381439209, "nrmse": 0.010057136416435242}, {"layer": 39, "field": "key", "exact": false, "max_abs": 0.1875, "nrmse": 0.007201306987553835}, {"layer": 39, "field": "value", "exact": false, "max_abs": 0.21875, "nrmse": 0.0092015927657485}, {"layer": 40, "field": "conv", "exact": false, "max_abs": 0.23828125, "nrmse": 0.02554425597190857}, {"layer": 40, "field": "recurrent", "exact": false, "max_abs": 0.008050620555877686, "nrmse": 0.008245083503425121}, {"layer": 41, "field": "conv", "exact": false, "max_abs": 0.375, "nrmse": 0.030952991917729378}, {"layer": 41, "field": "recurrent", "exact": false, "max_abs": 0.006825767457485199, "nrmse": 0.016739053651690483}, {"layer": 42, "field": "conv", "exact": false, "max_abs": 0.257568359375, "nrmse": 0.03413825109601021}, {"layer": 42, "field": "recurrent", "exact": false, "max_abs": 0.01497158408164978, "nrmse": 0.01865740679204464}, {"layer": 43, "field": "key", "exact": false, "max_abs": 0.3515625, "nrmse": 0.010946733877062798}, {"layer": 43, "field": "value", "exact": false, "max_abs": 0.328125, "nrmse": 0.012479942291975021}, {"layer": 44, "field": "conv", "exact": false, "max_abs": 0.35546875, "nrmse": 0.03929540887475014}, {"layer": 44, "field": "recurrent", "exact": false, "max_abs": 0.012583866715431213, "nrmse": 0.024720488116145134}, {"layer": 45, "field": "conv", "exact": false, "max_abs": 0.3671875, "nrmse": 0.045286837965250015}, {"layer": 45, "field": "recurrent", "exact": false, "max_abs": 0.015311287716031075, "nrmse": 0.02789202891290188}, {"layer": 46, "field": "conv", "exact": false, "max_abs": 0.4375, "nrmse": 0.04347870871424675}, {"layer": 46, "field": "recurrent", "exact": false, "max_abs": 0.03315672278404236, "nrmse": 0.030963512137532234}, {"layer": 47, "field": "key", "exact": false, "max_abs": 0.55078125, "nrmse": 0.013170280493795872}, {"layer": 47, "field": "value", "exact": false, "max_abs": 0.34375, "nrmse": 0.016987264156341553}, {"layer": 48, "field": "conv", "exact": false, "max_abs": 0.36328125, "nrmse": 0.043895021080970764}, {"layer": 48, "field": "recurrent", "exact": false, "max_abs": 0.010857805609703064, "nrmse": 0.028079211711883545}, {"layer": 49, "field": "conv", "exact": false, "max_abs": 0.376953125, "nrmse": 0.046644747257232666}, {"layer": 49, "field": "recurrent", "exact": false, "max_abs": 0.014188073575496674, "nrmse": 0.03533266484737396}, {"layer": 50, "field": "conv", "exact": false, "max_abs": 0.390625, "nrmse": 0.04561449587345123}, {"layer": 50, "field": "recurrent", "exact": false, "max_abs": 0.016188323497772217, "nrmse": 0.03201121464371681}, {"layer": 51, "field": "key", "exact": false, "max_abs": 0.46875, "nrmse": 0.013577990233898163}, {"layer": 51, "field": "value", "exact": false, "max_abs": 0.5625, "nrmse": 0.01801944337785244}, {"layer": 52, "field": "conv", "exact": false, "max_abs": 0.48828125, "nrmse": 0.039052773267030716}, {"layer": 52, "field": "recurrent", "exact": false, "max_abs": 0.037687256932258606, "nrmse": 0.02338484115898609}, {"layer": 53, "field": "conv", "exact": false, "max_abs": 0.46875, "nrmse": 0.04097355529665947}, {"layer": 53, "field": "recurrent", "exact": false, "max_abs": 0.06095612049102783, "nrmse": 0.031742747873067856}, {"layer": 54, "field": "conv", "exact": false, "max_abs": 0.478515625, "nrmse": 0.04569440335035324}, {"layer": 54, "field": "recurrent", "exact": false, "max_abs": 0.027792617678642273, "nrmse": 0.03516238555312157}, {"layer": 55, "field": "key", "exact": false, "max_abs": 0.31005859375, "nrmse": 0.013823586516082287}, {"layer": 55, "field": "value", "exact": false, "max_abs": 0.75, "nrmse": 0.019580373540520668}, {"layer": 56, "field": "conv", "exact": false, "max_abs": 0.587646484375, "nrmse": 0.0420561283826828}, {"layer": 56, "field": "recurrent", "exact": false, "max_abs": 0.03583051264286041, "nrmse": 0.027710815891623497}, {"layer": 57, "field": "conv", "exact": false, "max_abs": 0.59375, "nrmse": 0.041331905871629715}, {"layer": 57, "field": "recurrent", "exact": false, "max_abs": 0.04678267240524292, "nrmse": 0.03210325539112091}, {"layer": 58, "field": "conv", "exact": false, "max_abs": 0.703125, "nrmse": 0.04544059932231903}, {"layer": 58, "field": "recurrent", "exact": false, "max_abs": 0.03686249256134033, "nrmse": 0.027951696887612343}, {"layer": 59, "field": "key", "exact": false, "max_abs": 0.3125, "nrmse": 0.013627300038933754}, {"layer": 59, "field": "value", "exact": false, "max_abs": 1.0625, "nrmse": 0.01564410887658596}, {"layer": 60, "field": "conv", "exact": false, "max_abs": 1.25, "nrmse": 0.03872884064912796}, {"layer": 60, "field": "recurrent", "exact": false, "max_abs": 0.04387390613555908, "nrmse": 0.029453305527567863}, {"layer": 61, "field": "conv", "exact": false, "max_abs": 0.578125, "nrmse": 0.043206699192523956}, {"layer": 61, "field": "recurrent", "exact": false, "max_abs": 0.023406073451042175, "nrmse": 0.03353738412261009}, {"layer": 62, "field": "conv", "exact": false, "max_abs": 1.125, "nrmse": 0.03812984749674797}, {"layer": 62, "field": "recurrent", "exact": false, "max_abs": 0.028552770614624023, "nrmse": 0.021915052086114883}, {"layer": 63, "field": "key", "exact": false, "max_abs": 0.41796875, "nrmse": 0.014889623038470745}, {"layer": 63, "field": "value", "exact": false, "max_abs": 0.890625, "nrmse": 0.008966682478785515}], "buffered_graph_state_diff": []} -{"phase": "step", "index": 70, "position": 218, "teacher_token": 17, "compact": {"ids": [13, 36189, 18053, 11, 25], "logits": [27.375, 15.8125, 13.375, 12.875, 12.8125], "margin": 11.5625, "argmax": 13}, "buffered": {"ids": [13, 36189, 18053, 11, 25], "logits": [27.5, 15.9375, 13.5, 12.875, 12.8125], "margin": 11.5625, "argmax": 13}, "graph": {"ids": [13, 36189, 18053, 11, 25], "logits": [27.5, 15.9375, 13.5, 12.875, 12.8125], "margin": 11.5625, "argmax": 13}, "compact_buffered": {"exact": false, "max_abs": 0.453125, "nrmse": 0.012063883244991302}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "compact_buffered_state_diff": [{"layer": 36, "field": "recurrent", "exact": false, "max_abs": 4.76837158203125e-07, "nrmse": 1.336196788770394e-07}, {"layer": 37, "field": "recurrent", "exact": false, "max_abs": 0.00045559555292129517, "nrmse": 0.0004704244784079492}, {"layer": 38, "field": "conv", "exact": false, "max_abs": 0.09375, "nrmse": 0.010597536340355873}, {"layer": 38, "field": "recurrent", "exact": false, "max_abs": 0.008592098951339722, "nrmse": 0.009367256425321102}, {"layer": 39, "field": "key", "exact": false, "max_abs": 0.1875, "nrmse": 0.007274201605468988}, {"layer": 39, "field": "value", "exact": false, "max_abs": 0.21875, "nrmse": 0.009265068918466568}, {"layer": 40, "field": "conv", "exact": false, "max_abs": 0.23828125, "nrmse": 0.023766979575157166}, {"layer": 40, "field": "recurrent", "exact": false, "max_abs": 0.010437369346618652, "nrmse": 0.008198881521821022}, {"layer": 41, "field": "conv", "exact": false, "max_abs": 0.375, "nrmse": 0.029036685824394226}, {"layer": 41, "field": "recurrent", "exact": false, "max_abs": 0.007728911936283112, "nrmse": 0.016349628567695618}, {"layer": 42, "field": "conv", "exact": false, "max_abs": 0.257568359375, "nrmse": 0.032349757850170135}, {"layer": 42, "field": "recurrent", "exact": false, "max_abs": 0.011084645986557007, "nrmse": 0.01797809638082981}, {"layer": 43, "field": "key", "exact": false, "max_abs": 0.3515625, "nrmse": 0.011042649857699871}, {"layer": 43, "field": "value", "exact": false, "max_abs": 0.328125, "nrmse": 0.012572910636663437}, {"layer": 44, "field": "conv", "exact": false, "max_abs": 0.35546875, "nrmse": 0.03654443472623825}, {"layer": 44, "field": "recurrent", "exact": false, "max_abs": 0.012726128101348877, "nrmse": 0.022509543225169182}, {"layer": 45, "field": "conv", "exact": false, "max_abs": 0.3671875, "nrmse": 0.04204227775335312}, {"layer": 45, "field": "recurrent", "exact": false, "max_abs": 0.013254880905151367, "nrmse": 0.026031870394945145}, {"layer": 46, "field": "conv", "exact": false, "max_abs": 0.4375, "nrmse": 0.039581168442964554}, {"layer": 46, "field": "recurrent", "exact": false, "max_abs": 0.03344517946243286, "nrmse": 0.0310506708920002}, {"layer": 47, "field": "key", "exact": false, "max_abs": 0.55078125, "nrmse": 0.013255106285214424}, {"layer": 47, "field": "value", "exact": false, "max_abs": 0.34375, "nrmse": 0.017082244157791138}, {"layer": 48, "field": "conv", "exact": false, "max_abs": 0.36328125, "nrmse": 0.04117904603481293}, {"layer": 48, "field": "recurrent", "exact": false, "max_abs": 0.00821775197982788, "nrmse": 0.027143821120262146}, {"layer": 49, "field": "conv", "exact": false, "max_abs": 0.376953125, "nrmse": 0.043302927166223526}, {"layer": 49, "field": "recurrent", "exact": false, "max_abs": 0.012269116938114166, "nrmse": 0.03411451354622841}, {"layer": 50, "field": "conv", "exact": false, "max_abs": 0.390625, "nrmse": 0.04220312088727951}, {"layer": 50, "field": "recurrent", "exact": false, "max_abs": 0.014789402484893799, "nrmse": 0.03132137656211853}, {"layer": 51, "field": "key", "exact": false, "max_abs": 0.46875, "nrmse": 0.013656691648066044}, {"layer": 51, "field": "value", "exact": false, "max_abs": 0.5625, "nrmse": 0.018149130046367645}, {"layer": 52, "field": "conv", "exact": false, "max_abs": 0.48828125, "nrmse": 0.03763965889811516}, {"layer": 52, "field": "recurrent", "exact": false, "max_abs": 0.042803406715393066, "nrmse": 0.022727716714143753}, {"layer": 53, "field": "conv", "exact": false, "max_abs": 0.46875, "nrmse": 0.03951867297291756}, {"layer": 53, "field": "recurrent", "exact": false, "max_abs": 0.06156265735626221, "nrmse": 0.031474243849515915}, {"layer": 54, "field": "conv", "exact": false, "max_abs": 0.478515625, "nrmse": 0.044185101985931396}, {"layer": 54, "field": "recurrent", "exact": false, "max_abs": 0.02845403552055359, "nrmse": 0.035871636122465134}, {"layer": 55, "field": "key", "exact": false, "max_abs": 0.31005859375, "nrmse": 0.013960768468677998}, {"layer": 55, "field": "value", "exact": false, "max_abs": 0.75, "nrmse": 0.019725361838936806}, {"layer": 56, "field": "conv", "exact": false, "max_abs": 0.587646484375, "nrmse": 0.04080943018198013}, {"layer": 56, "field": "recurrent", "exact": false, "max_abs": 0.038372308015823364, "nrmse": 0.027518736198544502}, {"layer": 57, "field": "conv", "exact": false, "max_abs": 0.59375, "nrmse": 0.040503084659576416}, {"layer": 57, "field": "recurrent", "exact": false, "max_abs": 0.04539847373962402, "nrmse": 0.030480882152915}, {"layer": 58, "field": "conv", "exact": false, "max_abs": 0.703125, "nrmse": 0.04402966797351837}, {"layer": 58, "field": "recurrent", "exact": false, "max_abs": 0.03818225860595703, "nrmse": 0.02657725103199482}, {"layer": 59, "field": "key", "exact": false, "max_abs": 0.3125, "nrmse": 0.013826477341353893}, {"layer": 59, "field": "value", "exact": false, "max_abs": 1.0625, "nrmse": 0.01579657569527626}, {"layer": 60, "field": "conv", "exact": false, "max_abs": 1.125, "nrmse": 0.038526151329278946}, {"layer": 60, "field": "recurrent", "exact": false, "max_abs": 0.0425126850605011, "nrmse": 0.029797285795211792}, {"layer": 61, "field": "conv", "exact": false, "max_abs": 0.578125, "nrmse": 0.0426747128367424}, {"layer": 61, "field": "recurrent", "exact": false, "max_abs": 0.021962717175483704, "nrmse": 0.03237559273838997}, {"layer": 62, "field": "conv", "exact": false, "max_abs": 1.125, "nrmse": 0.037136465311050415}, {"layer": 62, "field": "recurrent", "exact": false, "max_abs": 0.0706527829170227, "nrmse": 0.02295284904539585}, {"layer": 63, "field": "key", "exact": false, "max_abs": 0.41796875, "nrmse": 0.015033422969281673}, {"layer": 63, "field": "value", "exact": false, "max_abs": 0.890625, "nrmse": 0.009037822484970093}], "buffered_graph_state_diff": []} -{"phase": "step", "index": 71, "position": 219, "teacher_token": 13, "compact": {"ids": [36189, 18053, 28924, 62366, 6558], "logits": [27.625, 26.125, 21.875, 19.75, 19.5], "margin": 1.5, "argmax": 36189}, "buffered": {"ids": [36189, 18053, 28924, 62366, 6558], "logits": [27.375, 26.125, 21.625, 19.5, 19.25], "margin": 1.25, "argmax": 36189}, "graph": {"ids": [36189, 18053, 28924, 62366, 6558], "logits": [27.375, 26.125, 21.625, 19.5, 19.25], "margin": 1.25, "argmax": 36189}, "compact_buffered": {"exact": false, "max_abs": 0.5625, "nrmse": 0.023646889254450798}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "compact_buffered_state_diff": [{"layer": 36, "field": "recurrent", "exact": false, "max_abs": 4.76837158203125e-07, "nrmse": 1.3228437012458016e-07}, {"layer": 37, "field": "recurrent", "exact": false, "max_abs": 0.0004566274583339691, "nrmse": 0.00046268609003163874}, {"layer": 38, "field": "conv", "exact": false, "max_abs": 0.0849609375, "nrmse": 0.010744001716375351}, {"layer": 38, "field": "recurrent", "exact": false, "max_abs": 0.0048909783363342285, "nrmse": 0.00906725786626339}, {"layer": 39, "field": "key", "exact": false, "max_abs": 0.1875, "nrmse": 0.007343024015426636}, {"layer": 39, "field": "value", "exact": false, "max_abs": 0.21875, "nrmse": 0.009357805363833904}, {"layer": 40, "field": "conv", "exact": false, "max_abs": 0.23828125, "nrmse": 0.024836847558617592}, {"layer": 40, "field": "recurrent", "exact": false, "max_abs": 0.010274171829223633, "nrmse": 0.008204268291592598}, {"layer": 41, "field": "conv", "exact": false, "max_abs": 0.375, "nrmse": 0.029950378462672234}, {"layer": 41, "field": "recurrent", "exact": false, "max_abs": 0.007722288370132446, "nrmse": 0.0161893293261528}, {"layer": 42, "field": "conv", "exact": false, "max_abs": 0.257568359375, "nrmse": 0.03320468217134476}, {"layer": 42, "field": "recurrent", "exact": false, "max_abs": 0.011141389608383179, "nrmse": 0.0176676157861948}, {"layer": 43, "field": "key", "exact": false, "max_abs": 0.3515625, "nrmse": 0.01116135809570551}, {"layer": 43, "field": "value", "exact": false, "max_abs": 0.328125, "nrmse": 0.01272273063659668}, {"layer": 44, "field": "conv", "exact": false, "max_abs": 0.35546875, "nrmse": 0.03676751255989075}, {"layer": 44, "field": "recurrent", "exact": false, "max_abs": 0.012862592935562134, "nrmse": 0.022080516442656517}, {"layer": 45, "field": "conv", "exact": false, "max_abs": 0.3671875, "nrmse": 0.04215073585510254}, {"layer": 45, "field": "recurrent", "exact": false, "max_abs": 0.013265252113342285, "nrmse": 0.024558216333389282}, {"layer": 46, "field": "conv", "exact": false, "max_abs": 0.4375, "nrmse": 0.03980096057057381}, {"layer": 46, "field": "recurrent", "exact": false, "max_abs": 0.01420658826828003, "nrmse": 0.02596699818968773}, {"layer": 47, "field": "key", "exact": false, "max_abs": 0.55078125, "nrmse": 0.013382471166551113}, {"layer": 47, "field": "value", "exact": false, "max_abs": 0.34375, "nrmse": 0.017255231738090515}, {"layer": 48, "field": "conv", "exact": false, "max_abs": 0.36328125, "nrmse": 0.04173167049884796}, {"layer": 48, "field": "recurrent", "exact": false, "max_abs": 0.00816769152879715, "nrmse": 0.026457281783223152}, {"layer": 49, "field": "conv", "exact": false, "max_abs": 0.376953125, "nrmse": 0.042768966406583786}, {"layer": 49, "field": "recurrent", "exact": false, "max_abs": 0.013831496238708496, "nrmse": 0.03356076031923294}, {"layer": 50, "field": "conv", "exact": false, "max_abs": 0.390625, "nrmse": 0.04112488776445389}, {"layer": 50, "field": "recurrent", "exact": false, "max_abs": 0.013606637716293335, "nrmse": 0.030455145984888077}, {"layer": 51, "field": "key", "exact": false, "max_abs": 0.46875, "nrmse": 0.013822061009705067}, {"layer": 51, "field": "value", "exact": false, "max_abs": 0.5625, "nrmse": 0.018300946801900864}, {"layer": 52, "field": "conv", "exact": false, "max_abs": 0.48828125, "nrmse": 0.03735567256808281}, {"layer": 52, "field": "recurrent", "exact": false, "max_abs": 0.032747626304626465, "nrmse": 0.022738920524716377}, {"layer": 53, "field": "conv", "exact": false, "max_abs": 0.46875, "nrmse": 0.03941771388053894}, {"layer": 53, "field": "recurrent", "exact": false, "max_abs": 0.06146383285522461, "nrmse": 0.030324507504701614}, {"layer": 54, "field": "conv", "exact": false, "max_abs": 0.478515625, "nrmse": 0.04424508288502693}, {"layer": 54, "field": "recurrent", "exact": false, "max_abs": 0.026893481612205505, "nrmse": 0.034316014498472214}, {"layer": 55, "field": "key", "exact": false, "max_abs": 0.31005859375, "nrmse": 0.014065030962228775}, {"layer": 55, "field": "value", "exact": false, "max_abs": 0.75, "nrmse": 0.019881602376699448}, {"layer": 56, "field": "conv", "exact": false, "max_abs": 0.587646484375, "nrmse": 0.041230566799640656}, {"layer": 56, "field": "recurrent", "exact": false, "max_abs": 0.033292412757873535, "nrmse": 0.02773663029074669}, {"layer": 57, "field": "conv", "exact": false, "max_abs": 0.59375, "nrmse": 0.04106421396136284}, {"layer": 57, "field": "recurrent", "exact": false, "max_abs": 0.04863826557993889, "nrmse": 0.029873182997107506}, {"layer": 58, "field": "conv", "exact": false, "max_abs": 0.703125, "nrmse": 0.04409565031528473}, {"layer": 58, "field": "recurrent", "exact": false, "max_abs": 0.0508384108543396, "nrmse": 0.026104815304279327}, {"layer": 59, "field": "key", "exact": false, "max_abs": 0.3125, "nrmse": 0.013934161514043808}, {"layer": 59, "field": "value", "exact": false, "max_abs": 1.0625, "nrmse": 0.01592293195426464}, {"layer": 60, "field": "conv", "exact": false, "max_abs": 1.25, "nrmse": 0.03817065432667732}, {"layer": 60, "field": "recurrent", "exact": false, "max_abs": 0.08370861411094666, "nrmse": 0.030786199495196342}, {"layer": 61, "field": "conv", "exact": false, "max_abs": 0.578125, "nrmse": 0.042926251888275146}, {"layer": 61, "field": "recurrent", "exact": false, "max_abs": 0.02283354103565216, "nrmse": 0.03204447776079178}, {"layer": 62, "field": "conv", "exact": false, "max_abs": 1.125, "nrmse": 0.037204310297966}, {"layer": 62, "field": "recurrent", "exact": false, "max_abs": 0.06625711917877197, "nrmse": 0.02318998984992504}, {"layer": 63, "field": "key", "exact": false, "max_abs": 0.41796875, "nrmse": 0.015204988420009613}, {"layer": 63, "field": "value", "exact": false, "max_abs": 0.890625, "nrmse": 0.009135271422564983}], "buffered_graph_state_diff": []} -{"phase": "step", "index": 72, "position": 220, "teacher_token": 36189, "compact": {"ids": [2891, 537, 279, 89, 53983], "logits": [29.625, 19.125, 17.25, 16.125, 15.625], "margin": 10.5, "argmax": 2891}, "buffered": {"ids": [2891, 537, 279, 89, 53983], "logits": [28.875, 18.875, 17.0, 15.8125, 15.4375], "margin": 10.0, "argmax": 2891}, "graph": {"ids": [2891, 537, 279, 89, 53983], "logits": [28.875, 18.875, 17.0, 15.8125, 15.4375], "margin": 10.0, "argmax": 2891}, "compact_buffered": {"exact": false, "max_abs": 0.75, "nrmse": 0.026361217722296715}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "compact_buffered_state_diff": [{"layer": 36, "field": "recurrent", "exact": false, "max_abs": 4.76837158203125e-07, "nrmse": 1.3210890870141156e-07}, {"layer": 37, "field": "recurrent", "exact": false, "max_abs": 0.0004567466676235199, "nrmse": 0.0004599377862177789}, {"layer": 38, "field": "conv", "exact": false, "max_abs": 0.0849609375, "nrmse": 0.011227905750274658}, {"layer": 38, "field": "recurrent", "exact": false, "max_abs": 0.00588458776473999, "nrmse": 0.009230896830558777}, {"layer": 39, "field": "key", "exact": false, "max_abs": 0.1875, "nrmse": 0.007422963622957468}, {"layer": 39, "field": "value", "exact": false, "max_abs": 0.21875, "nrmse": 0.009418484754860401}, {"layer": 40, "field": "conv", "exact": false, "max_abs": 0.23828125, "nrmse": 0.02480284683406353}, {"layer": 40, "field": "recurrent", "exact": false, "max_abs": 0.010764360427856445, "nrmse": 0.008104723878204823}, {"layer": 41, "field": "conv", "exact": false, "max_abs": 0.375, "nrmse": 0.030065974220633507}, {"layer": 41, "field": "recurrent", "exact": false, "max_abs": 0.00766386091709137, "nrmse": 0.01628684438765049}, {"layer": 42, "field": "conv", "exact": false, "max_abs": 0.257568359375, "nrmse": 0.033998314291238785}, {"layer": 42, "field": "recurrent", "exact": false, "max_abs": 0.00959843397140503, "nrmse": 0.01758188009262085}, {"layer": 43, "field": "key", "exact": false, "max_abs": 0.3515625, "nrmse": 0.011232687160372734}, {"layer": 43, "field": "value", "exact": false, "max_abs": 0.328125, "nrmse": 0.012793149799108505}, {"layer": 44, "field": "conv", "exact": false, "max_abs": 0.35546875, "nrmse": 0.03662317246198654}, {"layer": 44, "field": "recurrent", "exact": false, "max_abs": 0.012934237718582153, "nrmse": 0.021883774548768997}, {"layer": 45, "field": "conv", "exact": false, "max_abs": 0.3671875, "nrmse": 0.04250043258070946}, {"layer": 45, "field": "recurrent", "exact": false, "max_abs": 0.013262152671813965, "nrmse": 0.024424690753221512}, {"layer": 46, "field": "conv", "exact": false, "max_abs": 0.4375, "nrmse": 0.040588315576314926}, {"layer": 46, "field": "recurrent", "exact": false, "max_abs": 0.01424187421798706, "nrmse": 0.025268331170082092}, {"layer": 47, "field": "key", "exact": false, "max_abs": 0.55078125, "nrmse": 0.013448663055896759}, {"layer": 47, "field": "value", "exact": false, "max_abs": 0.34375, "nrmse": 0.017331356182694435}, {"layer": 48, "field": "conv", "exact": false, "max_abs": 0.36328125, "nrmse": 0.04246462881565094}, {"layer": 48, "field": "recurrent", "exact": false, "max_abs": 0.008143886923789978, "nrmse": 0.026513706892728806}, {"layer": 49, "field": "conv", "exact": false, "max_abs": 0.376953125, "nrmse": 0.04273044690489769}, {"layer": 49, "field": "recurrent", "exact": false, "max_abs": 0.014377176761627197, "nrmse": 0.03293337672948837}, {"layer": 50, "field": "conv", "exact": false, "max_abs": 0.390625, "nrmse": 0.04210861399769783}, {"layer": 50, "field": "recurrent", "exact": false, "max_abs": 0.008432835340499878, "nrmse": 0.029853682965040207}, {"layer": 51, "field": "key", "exact": false, "max_abs": 0.46875, "nrmse": 0.013895120471715927}, {"layer": 51, "field": "value", "exact": false, "max_abs": 0.5625, "nrmse": 0.01840764284133911}, {"layer": 52, "field": "conv", "exact": false, "max_abs": 0.48828125, "nrmse": 0.03706900775432587}, {"layer": 52, "field": "recurrent", "exact": false, "max_abs": 0.033308207988739014, "nrmse": 0.02245386503636837}, {"layer": 53, "field": "conv", "exact": false, "max_abs": 0.46875, "nrmse": 0.038357242941856384}, {"layer": 53, "field": "recurrent", "exact": false, "max_abs": 0.06225007772445679, "nrmse": 0.029316548258066177}, {"layer": 54, "field": "conv", "exact": false, "max_abs": 0.478515625, "nrmse": 0.04382389411330223}, {"layer": 54, "field": "recurrent", "exact": false, "max_abs": 0.026995152235031128, "nrmse": 0.03318715840578079}, {"layer": 55, "field": "key", "exact": false, "max_abs": 0.31005859375, "nrmse": 0.014171088114380836}, {"layer": 55, "field": "value", "exact": false, "max_abs": 0.75, "nrmse": 0.02000272087752819}, {"layer": 56, "field": "conv", "exact": false, "max_abs": 0.587646484375, "nrmse": 0.04008537530899048}, {"layer": 56, "field": "recurrent", "exact": false, "max_abs": 0.033822476863861084, "nrmse": 0.026508722454309464}, {"layer": 57, "field": "conv", "exact": false, "max_abs": 0.59375, "nrmse": 0.038544971495866776}, {"layer": 57, "field": "recurrent", "exact": false, "max_abs": 0.042987871915102005, "nrmse": 0.028459716588258743}, {"layer": 58, "field": "conv", "exact": false, "max_abs": 0.703125, "nrmse": 0.04274402931332588}, {"layer": 58, "field": "recurrent", "exact": false, "max_abs": 0.039360761642456055, "nrmse": 0.025324268266558647}, {"layer": 59, "field": "key", "exact": false, "max_abs": 0.3125, "nrmse": 0.014032687991857529}, {"layer": 59, "field": "value", "exact": false, "max_abs": 1.0625, "nrmse": 0.016053760424256325}, {"layer": 60, "field": "conv", "exact": false, "max_abs": 1.25, "nrmse": 0.03754645213484764}, {"layer": 60, "field": "recurrent", "exact": false, "max_abs": 0.08748947829008102, "nrmse": 0.030370090156793594}, {"layer": 61, "field": "conv", "exact": false, "max_abs": 0.578125, "nrmse": 0.041929326951503754}, {"layer": 61, "field": "recurrent", "exact": false, "max_abs": 0.02311563491821289, "nrmse": 0.031047508120536804}, {"layer": 62, "field": "conv", "exact": false, "max_abs": 1.125, "nrmse": 0.03699565306305885}, {"layer": 62, "field": "recurrent", "exact": false, "max_abs": 0.06508505344390869, "nrmse": 0.023003336042165756}, {"layer": 63, "field": "key", "exact": false, "max_abs": 0.41796875, "nrmse": 0.015321971848607063}, {"layer": 63, "field": "value", "exact": false, "max_abs": 1.375, "nrmse": 0.009296406991779804}], "buffered_graph_state_diff": []} -{"phase": "step", "index": 73, "position": 221, "teacher_token": 2891, "compact": {"ids": [279, 1092, 4203, 22757, 491], "logits": [27.875, 18.75, 16.0, 15.9375, 15.75], "margin": 9.125, "argmax": 279}, "buffered": {"ids": [279, 1092, 22757, 4203, 491], "logits": [28.0, 18.75, 16.25, 16.0, 15.75], "margin": 9.25, "argmax": 279}, "graph": {"ids": [279, 1092, 22757, 4203, 491], "logits": [28.0, 18.75, 16.25, 16.0, 15.75], "margin": 9.25, "argmax": 279}, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.022288529202342033}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "compact_buffered_state_diff": [{"layer": 32, "field": "recurrent", "exact": false, "max_abs": 5.6438148021698e-06, "nrmse": 4.06127628593822e-06}, {"layer": 33, "field": "conv", "exact": false, "max_abs": 0.015625, "nrmse": 0.0006152583518996835}, {"layer": 33, "field": "recurrent", "exact": false, "max_abs": 0.0007846280932426453, "nrmse": 0.0004414279537741095}, {"layer": 34, "field": "conv", "exact": false, "max_abs": 0.09375, "nrmse": 0.008328281342983246}, {"layer": 34, "field": "recurrent", "exact": false, "max_abs": 0.002916768193244934, "nrmse": 0.002179440576583147}, {"layer": 35, "field": "key", "exact": false, "max_abs": 0.1328125, "nrmse": 0.0012519342126324773}, {"layer": 35, "field": "value", "exact": false, "max_abs": 0.09375, "nrmse": 0.0016867517260834575}, {"layer": 36, "field": "conv", "exact": false, "max_abs": 0.203125, "nrmse": 0.011943986639380455}, {"layer": 36, "field": "recurrent", "exact": false, "max_abs": 0.00292176753282547, "nrmse": 0.002486945828422904}, {"layer": 37, "field": "conv", "exact": false, "max_abs": 0.15625, "nrmse": 0.011752397753298283}, {"layer": 37, "field": "recurrent", "exact": false, "max_abs": 0.008039534091949463, "nrmse": 0.006296829786151648}, {"layer": 38, "field": "conv", "exact": false, "max_abs": 0.15869140625, "nrmse": 0.017451409250497818}, {"layer": 38, "field": "recurrent", "exact": false, "max_abs": 0.012884080410003662, "nrmse": 0.012493526563048363}, {"layer": 39, "field": "key", "exact": false, "max_abs": 0.1875, "nrmse": 0.007561651058495045}, {"layer": 39, "field": "value", "exact": false, "max_abs": 0.21875, "nrmse": 0.009671751409769058}, {"layer": 40, "field": "conv", "exact": false, "max_abs": 0.13818359375, "nrmse": 0.023609580472111702}, {"layer": 40, "field": "recurrent", "exact": false, "max_abs": 0.01089012622833252, "nrmse": 0.008253682404756546}, {"layer": 41, "field": "conv", "exact": false, "max_abs": 0.171875, "nrmse": 0.026377245783805847}, {"layer": 41, "field": "recurrent", "exact": false, "max_abs": 0.007654920220375061, "nrmse": 0.016471700742840767}, {"layer": 42, "field": "conv", "exact": false, "max_abs": 0.171875, "nrmse": 0.028677942231297493}, {"layer": 42, "field": "recurrent", "exact": false, "max_abs": 0.010280013084411621, "nrmse": 0.017677677795290947}, {"layer": 43, "field": "key", "exact": false, "max_abs": 0.3515625, "nrmse": 0.011314122937619686}, {"layer": 43, "field": "value", "exact": false, "max_abs": 0.328125, "nrmse": 0.012956766411662102}, {"layer": 44, "field": "conv", "exact": false, "max_abs": 0.15625, "nrmse": 0.027972804382443428}, {"layer": 44, "field": "recurrent", "exact": false, "max_abs": 0.012790091335773468, "nrmse": 0.021730560809373856}, {"layer": 45, "field": "conv", "exact": false, "max_abs": 0.19384765625, "nrmse": 0.03232812136411667}, {"layer": 45, "field": "recurrent", "exact": false, "max_abs": 0.013262271881103516, "nrmse": 0.02433759532868862}, {"layer": 46, "field": "conv", "exact": false, "max_abs": 0.23095703125, "nrmse": 0.030864814296364784}, {"layer": 46, "field": "recurrent", "exact": false, "max_abs": 0.014236807823181152, "nrmse": 0.025519080460071564}, {"layer": 47, "field": "key", "exact": false, "max_abs": 0.55078125, "nrmse": 0.013575066812336445}, {"layer": 47, "field": "value", "exact": false, "max_abs": 0.34375, "nrmse": 0.017500219866633415}, {"layer": 48, "field": "conv", "exact": false, "max_abs": 0.28125, "nrmse": 0.03506818041205406}, {"layer": 48, "field": "recurrent", "exact": false, "max_abs": 0.008163303136825562, "nrmse": 0.026277732104063034}, {"layer": 49, "field": "conv", "exact": false, "max_abs": 0.2265625, "nrmse": 0.03366834670305252}, {"layer": 49, "field": "recurrent", "exact": false, "max_abs": 0.013825654983520508, "nrmse": 0.0317775197327137}, {"layer": 50, "field": "conv", "exact": false, "max_abs": 0.25, "nrmse": 0.033743806183338165}, {"layer": 50, "field": "recurrent", "exact": false, "max_abs": 0.00908920168876648, "nrmse": 0.029653986915946007}, {"layer": 51, "field": "key", "exact": false, "max_abs": 0.46875, "nrmse": 0.014063066802918911}, {"layer": 51, "field": "value", "exact": false, "max_abs": 0.5625, "nrmse": 0.018619190901517868}, {"layer": 52, "field": "conv", "exact": false, "max_abs": 0.30078125, "nrmse": 0.027940917760133743}, {"layer": 52, "field": "recurrent", "exact": false, "max_abs": 0.03652620315551758, "nrmse": 0.022428713738918304}, {"layer": 53, "field": "conv", "exact": false, "max_abs": 0.28125, "nrmse": 0.02861110493540764}, {"layer": 53, "field": "recurrent", "exact": false, "max_abs": 0.06211578845977783, "nrmse": 0.029234392568469048}, {"layer": 54, "field": "conv", "exact": false, "max_abs": 0.375, "nrmse": 0.03248031809926033}, {"layer": 54, "field": "recurrent", "exact": false, "max_abs": 0.027820810675621033, "nrmse": 0.03279343619942665}, {"layer": 55, "field": "key", "exact": false, "max_abs": 0.34765625, "nrmse": 0.01447681337594986}, {"layer": 55, "field": "value", "exact": false, "max_abs": 0.75, "nrmse": 0.020184367895126343}, {"layer": 56, "field": "conv", "exact": false, "max_abs": 0.328125, "nrmse": 0.028409259393811226}, {"layer": 56, "field": "recurrent", "exact": false, "max_abs": 0.033270567655563354, "nrmse": 0.026339320465922356}, {"layer": 57, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.026545003056526184}, {"layer": 57, "field": "recurrent", "exact": false, "max_abs": 0.04552800953388214, "nrmse": 0.02792496420443058}, {"layer": 58, "field": "conv", "exact": false, "max_abs": 0.34375, "nrmse": 0.029326727613806725}, {"layer": 58, "field": "recurrent", "exact": false, "max_abs": 0.038390517234802246, "nrmse": 0.024378633126616478}, {"layer": 59, "field": "key", "exact": false, "max_abs": 0.3125, "nrmse": 0.014234576374292374}, {"layer": 59, "field": "value", "exact": false, "max_abs": 1.0625, "nrmse": 0.01619548723101616}, {"layer": 60, "field": "conv", "exact": false, "max_abs": 1.25, "nrmse": 0.025000276044011116}, {"layer": 60, "field": "recurrent", "exact": false, "max_abs": 0.04264214634895325, "nrmse": 0.030079469084739685}, {"layer": 61, "field": "conv", "exact": false, "max_abs": 0.34375, "nrmse": 0.027691733092069626}, {"layer": 61, "field": "recurrent", "exact": false, "max_abs": 0.023823857307434082, "nrmse": 0.03044002875685692}, {"layer": 62, "field": "conv", "exact": false, "max_abs": 1.125, "nrmse": 0.02472936362028122}, {"layer": 62, "field": "recurrent", "exact": false, "max_abs": 0.07810204476118088, "nrmse": 0.023706503212451935}, {"layer": 63, "field": "key", "exact": false, "max_abs": 0.41796875, "nrmse": 0.015493026934564114}, {"layer": 63, "field": "value", "exact": false, "max_abs": 1.375, "nrmse": 0.009364076890051365}], "buffered_graph_state_diff": []} -{"phase": "step", "index": 74, "position": 222, "teacher_token": 279, "compact": {"ids": [4203, 35983, 14307, 7397, 3568], "logits": [22.75, 22.625, 22.125, 22.125, 22.125], "margin": 0.125, "argmax": 4203}, "buffered": {"ids": [4203, 35983, 14307, 7397, 3568], "logits": [22.875, 22.75, 22.25, 22.125, 22.125], "margin": 0.125, "argmax": 4203}, "graph": {"ids": [4203, 35983, 14307, 7397, 3568], "logits": [22.875, 22.75, 22.25, 22.125, 22.125], "margin": 0.125, "argmax": 4203}, "compact_buffered": {"exact": false, "max_abs": 0.359375, "nrmse": 0.027318930253386497}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "compact_buffered_state_diff": [{"layer": 32, "field": "recurrent", "exact": false, "max_abs": 4.742294549942017e-06, "nrmse": 3.3900760172400624e-06}, {"layer": 33, "field": "conv", "exact": false, "max_abs": 0.015625, "nrmse": 0.0006384450243785977}, {"layer": 33, "field": "recurrent", "exact": false, "max_abs": 0.0004075281322002411, "nrmse": 0.0004496341571211815}, {"layer": 34, "field": "conv", "exact": false, "max_abs": 0.09375, "nrmse": 0.010846756398677826}, {"layer": 34, "field": "recurrent", "exact": false, "max_abs": 0.0038811340928077698, "nrmse": 0.003202161518856883}, {"layer": 35, "field": "key", "exact": false, "max_abs": 0.1328125, "nrmse": 0.0018411653582006693}, {"layer": 35, "field": "value", "exact": false, "max_abs": 0.09375, "nrmse": 0.0021498030982911587}, {"layer": 36, "field": "conv", "exact": false, "max_abs": 0.203125, "nrmse": 0.015762237831950188}, {"layer": 36, "field": "recurrent", "exact": false, "max_abs": 0.0035328399389982224, "nrmse": 0.0035602969583123922}, {"layer": 37, "field": "conv", "exact": false, "max_abs": 0.15625, "nrmse": 0.016031315550208092}, {"layer": 37, "field": "recurrent", "exact": false, "max_abs": 0.007846564054489136, "nrmse": 0.007026807405054569}, {"layer": 38, "field": "conv", "exact": false, "max_abs": 0.15869140625, "nrmse": 0.02218496985733509}, {"layer": 38, "field": "recurrent", "exact": false, "max_abs": 0.008524440228939056, "nrmse": 0.013720370829105377}, {"layer": 39, "field": "key", "exact": false, "max_abs": 0.1875, "nrmse": 0.0078117214143276215}, {"layer": 39, "field": "value", "exact": false, "max_abs": 0.21875, "nrmse": 0.009881800971925259}, {"layer": 40, "field": "conv", "exact": false, "max_abs": 0.15625, "nrmse": 0.026233423501253128}, {"layer": 40, "field": "recurrent", "exact": false, "max_abs": 0.012512624263763428, "nrmse": 0.008671916089951992}, {"layer": 41, "field": "conv", "exact": false, "max_abs": 0.171875, "nrmse": 0.029398558661341667}, {"layer": 41, "field": "recurrent", "exact": false, "max_abs": 0.007667645812034607, "nrmse": 0.016811072826385498}, {"layer": 42, "field": "conv", "exact": false, "max_abs": 0.171875, "nrmse": 0.031262557953596115}, {"layer": 42, "field": "recurrent", "exact": false, "max_abs": 0.010130703449249268, "nrmse": 0.01784270443022251}, {"layer": 43, "field": "key", "exact": false, "max_abs": 0.3515625, "nrmse": 0.011443674564361572}, {"layer": 43, "field": "value", "exact": false, "max_abs": 0.328125, "nrmse": 0.013080551289021969}, {"layer": 44, "field": "conv", "exact": false, "max_abs": 0.1953125, "nrmse": 0.03038344904780388}, {"layer": 44, "field": "recurrent", "exact": false, "max_abs": 0.012731455266475677, "nrmse": 0.022038375958800316}, {"layer": 45, "field": "conv", "exact": false, "max_abs": 0.19384765625, "nrmse": 0.03489701449871063}, {"layer": 45, "field": "recurrent", "exact": false, "max_abs": 0.013257265090942383, "nrmse": 0.025032514706254005}, {"layer": 46, "field": "conv", "exact": false, "max_abs": 0.23095703125, "nrmse": 0.0337018184363842}, {"layer": 46, "field": "recurrent", "exact": false, "max_abs": 0.014236152172088623, "nrmse": 0.02567165717482567}, {"layer": 47, "field": "key", "exact": false, "max_abs": 0.55078125, "nrmse": 0.013676445931196213}, {"layer": 47, "field": "value", "exact": false, "max_abs": 0.34375, "nrmse": 0.017615817487239838}, {"layer": 48, "field": "conv", "exact": false, "max_abs": 0.28125, "nrmse": 0.03647488355636597}, {"layer": 48, "field": "recurrent", "exact": false, "max_abs": 0.008046314120292664, "nrmse": 0.02673754282295704}, {"layer": 49, "field": "conv", "exact": false, "max_abs": 0.2265625, "nrmse": 0.034448299556970596}, {"layer": 49, "field": "recurrent", "exact": false, "max_abs": 0.01368016004562378, "nrmse": 0.03204294666647911}, {"layer": 50, "field": "conv", "exact": false, "max_abs": 0.25, "nrmse": 0.03456483036279678}, {"layer": 50, "field": "recurrent", "exact": false, "max_abs": 0.008813738822937012, "nrmse": 0.029682490974664688}, {"layer": 51, "field": "key", "exact": false, "max_abs": 0.46875, "nrmse": 0.01416928693652153}, {"layer": 51, "field": "value", "exact": false, "max_abs": 0.5625, "nrmse": 0.018701286986470222}, {"layer": 52, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.027743568643927574}, {"layer": 52, "field": "recurrent", "exact": false, "max_abs": 0.03423810005187988, "nrmse": 0.02238503471016884}, {"layer": 53, "field": "conv", "exact": false, "max_abs": 0.28125, "nrmse": 0.027918152511119843}, {"layer": 53, "field": "recurrent", "exact": false, "max_abs": 0.06192964315414429, "nrmse": 0.029064208269119263}, {"layer": 54, "field": "conv", "exact": false, "max_abs": 0.375, "nrmse": 0.03155788779258728}, {"layer": 54, "field": "recurrent", "exact": false, "max_abs": 0.02791842818260193, "nrmse": 0.03245311230421066}, {"layer": 55, "field": "key", "exact": false, "max_abs": 0.34765625, "nrmse": 0.014591234736144543}, {"layer": 55, "field": "value", "exact": false, "max_abs": 0.75, "nrmse": 0.020273808389902115}, {"layer": 56, "field": "conv", "exact": false, "max_abs": 0.359375, "nrmse": 0.027835138142108917}, {"layer": 56, "field": "recurrent", "exact": false, "max_abs": 0.03309550881385803, "nrmse": 0.026410870254039764}, {"layer": 57, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.02580138109624386}, {"layer": 57, "field": "recurrent", "exact": false, "max_abs": 0.04408489912748337, "nrmse": 0.027066867798566818}, {"layer": 58, "field": "conv", "exact": false, "max_abs": 0.34375, "nrmse": 0.028127409517765045}, {"layer": 58, "field": "recurrent", "exact": false, "max_abs": 0.038211941719055176, "nrmse": 0.02365933172404766}, {"layer": 59, "field": "key", "exact": false, "max_abs": 0.3125, "nrmse": 0.014386478811502457}, {"layer": 59, "field": "value", "exact": false, "max_abs": 1.0625, "nrmse": 0.01627439446747303}, {"layer": 60, "field": "conv", "exact": false, "max_abs": 1.25, "nrmse": 0.024607421830296516}, {"layer": 60, "field": "recurrent", "exact": false, "max_abs": 0.04272860288619995, "nrmse": 0.02930705063045025}, {"layer": 61, "field": "conv", "exact": false, "max_abs": 0.34375, "nrmse": 0.0268251970410347}, {"layer": 61, "field": "recurrent", "exact": false, "max_abs": 0.022817343473434448, "nrmse": 0.02937459573149681}, {"layer": 62, "field": "conv", "exact": false, "max_abs": 1.125, "nrmse": 0.024931583553552628}, {"layer": 62, "field": "recurrent", "exact": false, "max_abs": 0.06929995119571686, "nrmse": 0.02331545576453209}, {"layer": 63, "field": "key", "exact": false, "max_abs": 0.41796875, "nrmse": 0.015597795136272907}, {"layer": 63, "field": "value", "exact": false, "max_abs": 1.375, "nrmse": 0.00940717663615942}], "buffered_graph_state_diff": []} -{"phase": "step", "index": 75, "position": 223, "teacher_token": 4203, "compact": {"ids": [64700, 321, 1801, 506, 494], "logits": [29.25, 25.25, 22.5, 22.375, 20.25], "margin": 4.0, "argmax": 64700}, "buffered": {"ids": [64700, 321, 1801, 506, 318], "logits": [29.25, 25.375, 22.375, 22.375, 20.125], "margin": 3.875, "argmax": 64700}, "graph": {"ids": [64700, 321, 1801, 506, 318], "logits": [29.25, 25.375, 22.375, 22.375, 20.125], "margin": 3.875, "argmax": 64700}, "compact_buffered": {"exact": false, "max_abs": 0.625, "nrmse": 0.03149857744574547}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "compact_buffered_state_diff": [{"layer": 32, "field": "recurrent", "exact": false, "max_abs": 3.0230730772018433e-06, "nrmse": 2.2534243271365995e-06}, {"layer": 33, "field": "conv", "exact": false, "max_abs": 0.015625, "nrmse": 0.0006225475808605552}, {"layer": 33, "field": "recurrent", "exact": false, "max_abs": 0.0003053247928619385, "nrmse": 0.00036175240529701114}, {"layer": 34, "field": "conv", "exact": false, "max_abs": 0.09375, "nrmse": 0.012633733451366425}, {"layer": 34, "field": "recurrent", "exact": false, "max_abs": 0.0035903379321098328, "nrmse": 0.0033689739648252726}, {"layer": 35, "field": "key", "exact": false, "max_abs": 0.1328125, "nrmse": 0.0021105343475937843}, {"layer": 35, "field": "value", "exact": false, "max_abs": 0.09375, "nrmse": 0.0025316784158349037}, {"layer": 36, "field": "conv", "exact": false, "max_abs": 0.203125, "nrmse": 0.017384549602866173}, {"layer": 36, "field": "recurrent", "exact": false, "max_abs": 0.0037713609635829926, "nrmse": 0.004145785700529814}, {"layer": 37, "field": "conv", "exact": false, "max_abs": 0.15625, "nrmse": 0.01809561625123024}, {"layer": 37, "field": "recurrent", "exact": false, "max_abs": 0.006978511810302734, "nrmse": 0.007365521043539047}, {"layer": 38, "field": "conv", "exact": false, "max_abs": 0.15869140625, "nrmse": 0.02413962222635746}, {"layer": 38, "field": "recurrent", "exact": false, "max_abs": 0.008803963661193848, "nrmse": 0.014507892541587353}, {"layer": 39, "field": "key", "exact": false, "max_abs": 0.1875, "nrmse": 0.007897337898612022}, {"layer": 39, "field": "value", "exact": false, "max_abs": 0.21875, "nrmse": 0.0100339874625206}, {"layer": 40, "field": "conv", "exact": false, "max_abs": 0.15625, "nrmse": 0.025799207389354706}, {"layer": 40, "field": "recurrent", "exact": false, "max_abs": 0.01493459939956665, "nrmse": 0.008871383965015411}, {"layer": 41, "field": "conv", "exact": false, "max_abs": 0.1875, "nrmse": 0.028505563735961914}, {"layer": 41, "field": "recurrent", "exact": false, "max_abs": 0.007677994668483734, "nrmse": 0.01709579862654209}, {"layer": 42, "field": "conv", "exact": false, "max_abs": 0.171875, "nrmse": 0.03037954308092594}, {"layer": 42, "field": "recurrent", "exact": false, "max_abs": 0.009615376591682434, "nrmse": 0.017934992909431458}, {"layer": 43, "field": "key", "exact": false, "max_abs": 0.3515625, "nrmse": 0.011486544273793697}, {"layer": 43, "field": "value", "exact": false, "max_abs": 0.328125, "nrmse": 0.013158578425645828}, {"layer": 44, "field": "conv", "exact": false, "max_abs": 0.1953125, "nrmse": 0.028938768431544304}, {"layer": 44, "field": "recurrent", "exact": false, "max_abs": 0.012680977582931519, "nrmse": 0.022150665521621704}, {"layer": 45, "field": "conv", "exact": false, "max_abs": 0.19384765625, "nrmse": 0.03321851044893265}, {"layer": 45, "field": "recurrent", "exact": false, "max_abs": 0.013255953788757324, "nrmse": 0.024931633844971657}, {"layer": 46, "field": "conv", "exact": false, "max_abs": 0.23095703125, "nrmse": 0.0318806916475296}, {"layer": 46, "field": "recurrent", "exact": false, "max_abs": 0.01433331985026598, "nrmse": 0.025149453431367874}, {"layer": 47, "field": "key", "exact": false, "max_abs": 0.55078125, "nrmse": 0.013712058775126934}, {"layer": 47, "field": "value", "exact": false, "max_abs": 0.34375, "nrmse": 0.01768522523343563}, {"layer": 48, "field": "conv", "exact": false, "max_abs": 0.28125, "nrmse": 0.033454038202762604}, {"layer": 48, "field": "recurrent", "exact": false, "max_abs": 0.008005976676940918, "nrmse": 0.026309853419661522}, {"layer": 49, "field": "conv", "exact": false, "max_abs": 0.2265625, "nrmse": 0.032417088747024536}, {"layer": 49, "field": "recurrent", "exact": false, "max_abs": 0.012225210666656494, "nrmse": 0.031556352972984314}, {"layer": 50, "field": "conv", "exact": false, "max_abs": 0.328125, "nrmse": 0.032949164509773254}, {"layer": 50, "field": "recurrent", "exact": false, "max_abs": 0.007833823561668396, "nrmse": 0.02945203334093094}, {"layer": 51, "field": "key", "exact": false, "max_abs": 0.46875, "nrmse": 0.014199567958712578}, {"layer": 51, "field": "value", "exact": false, "max_abs": 0.5625, "nrmse": 0.018788443878293037}, {"layer": 52, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.02699434570968151}, {"layer": 52, "field": "recurrent", "exact": false, "max_abs": 0.034252896904945374, "nrmse": 0.022269925102591515}, {"layer": 53, "field": "conv", "exact": false, "max_abs": 0.28125, "nrmse": 0.027006303891539574}, {"layer": 53, "field": "recurrent", "exact": false, "max_abs": 0.061781227588653564, "nrmse": 0.02885461039841175}, {"layer": 54, "field": "conv", "exact": false, "max_abs": 0.375, "nrmse": 0.030600054189562798}, {"layer": 54, "field": "recurrent", "exact": false, "max_abs": 0.02824103832244873, "nrmse": 0.03259168937802315}, {"layer": 55, "field": "key", "exact": false, "max_abs": 0.34765625, "nrmse": 0.014657867141067982}, {"layer": 55, "field": "value", "exact": false, "max_abs": 0.75, "nrmse": 0.02036810666322708}, {"layer": 56, "field": "conv", "exact": false, "max_abs": 0.359375, "nrmse": 0.02654035948216915}, {"layer": 56, "field": "recurrent", "exact": false, "max_abs": 0.032958269119262695, "nrmse": 0.026142749935388565}, {"layer": 57, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.024019982665777206}, {"layer": 57, "field": "recurrent", "exact": false, "max_abs": 0.037169910967350006, "nrmse": 0.02665025368332863}, {"layer": 58, "field": "conv", "exact": false, "max_abs": 0.28125, "nrmse": 0.026315761730074883}, {"layer": 58, "field": "recurrent", "exact": false, "max_abs": 0.037054091691970825, "nrmse": 0.023644251748919487}, {"layer": 59, "field": "key", "exact": false, "max_abs": 0.3125, "nrmse": 0.014429103583097458}, {"layer": 59, "field": "value", "exact": false, "max_abs": 1.0625, "nrmse": 0.016352923586964607}, {"layer": 60, "field": "conv", "exact": false, "max_abs": 1.0, "nrmse": 0.022897303104400635}, {"layer": 60, "field": "recurrent", "exact": false, "max_abs": 0.0428333580493927, "nrmse": 0.029282884672284126}, {"layer": 61, "field": "conv", "exact": false, "max_abs": 0.34375, "nrmse": 0.02520022727549076}, {"layer": 61, "field": "recurrent", "exact": false, "max_abs": 0.022652029991149902, "nrmse": 0.02902240864932537}, {"layer": 62, "field": "conv", "exact": false, "max_abs": 1.125, "nrmse": 0.023422233760356903}, {"layer": 62, "field": "recurrent", "exact": false, "max_abs": 0.07203391194343567, "nrmse": 0.023330818861722946}, {"layer": 63, "field": "key", "exact": false, "max_abs": 0.41796875, "nrmse": 0.01566825993359089}, {"layer": 63, "field": "value", "exact": false, "max_abs": 1.375, "nrmse": 0.009456679224967957}], "buffered_graph_state_diff": []} -{"phase": "step", "index": 76, "position": 224, "teacher_token": 64700, "compact": {"ids": [198, 6558, 1042, 9764, 271], "logits": [29.0, 15.4375, 14.3125, 14.25, 13.75], "margin": 13.5625, "argmax": 198}, "buffered": {"ids": [198, 6558, 1042, 9764, 271], "logits": [29.0, 15.375, 14.1875, 14.0, 13.75], "margin": 13.625, "argmax": 198}, "graph": {"ids": [198, 6558, 1042, 9764, 271], "logits": [29.0, 15.375, 14.1875, 14.0, 13.75], "margin": 13.625, "argmax": 198}, "compact_buffered": {"exact": false, "max_abs": 0.34375, "nrmse": 0.017235970124602318}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "compact_buffered_state_diff": [{"layer": 32, "field": "recurrent", "exact": false, "max_abs": 1.642853021621704e-06, "nrmse": 1.716964447950886e-06}, {"layer": 33, "field": "conv", "exact": false, "max_abs": 0.015625, "nrmse": 0.0006218663766048849}, {"layer": 33, "field": "recurrent", "exact": false, "max_abs": 0.0003445260226726532, "nrmse": 0.00032461839145980775}, {"layer": 34, "field": "conv", "exact": false, "max_abs": 0.09375, "nrmse": 0.014767727814614773}, {"layer": 34, "field": "recurrent", "exact": false, "max_abs": 0.004135102033615112, "nrmse": 0.004367504734545946}, {"layer": 35, "field": "key", "exact": false, "max_abs": 0.1328125, "nrmse": 0.002486081328243017}, {"layer": 35, "field": "value", "exact": false, "max_abs": 0.09375, "nrmse": 0.0029675208497792482}, {"layer": 36, "field": "conv", "exact": false, "max_abs": 0.203125, "nrmse": 0.02054772339761257}, {"layer": 36, "field": "recurrent", "exact": false, "max_abs": 0.004115913063287735, "nrmse": 0.004918103106319904}, {"layer": 37, "field": "conv", "exact": false, "max_abs": 0.157684326171875, "nrmse": 0.021969744935631752}, {"layer": 37, "field": "recurrent", "exact": false, "max_abs": 0.0071801841259002686, "nrmse": 0.008316622115671635}, {"layer": 38, "field": "conv", "exact": false, "max_abs": 0.15869140625, "nrmse": 0.02854699082672596}, {"layer": 38, "field": "recurrent", "exact": false, "max_abs": 0.01377876102924347, "nrmse": 0.01735648326575756}, {"layer": 39, "field": "key", "exact": false, "max_abs": 0.21875, "nrmse": 0.00812201201915741}, {"layer": 39, "field": "value", "exact": false, "max_abs": 0.21875, "nrmse": 0.010284395888447762}, {"layer": 40, "field": "conv", "exact": false, "max_abs": 0.16015625, "nrmse": 0.028882058337330818}, {"layer": 40, "field": "recurrent", "exact": false, "max_abs": 0.009334683418273926, "nrmse": 0.008724353276193142}, {"layer": 41, "field": "conv", "exact": false, "max_abs": 0.1875, "nrmse": 0.03130771964788437}, {"layer": 41, "field": "recurrent", "exact": false, "max_abs": 0.007671363651752472, "nrmse": 0.017516877502202988}, {"layer": 42, "field": "conv", "exact": false, "max_abs": 0.171875, "nrmse": 0.03268676623702049}, {"layer": 42, "field": "recurrent", "exact": false, "max_abs": 0.014114707708358765, "nrmse": 0.01877172850072384}, {"layer": 43, "field": "key", "exact": false, "max_abs": 0.3515625, "nrmse": 0.011623098514974117}, {"layer": 43, "field": "value", "exact": false, "max_abs": 0.328125, "nrmse": 0.01330247987061739}, {"layer": 44, "field": "conv", "exact": false, "max_abs": 0.1953125, "nrmse": 0.031106265261769295}, {"layer": 44, "field": "recurrent", "exact": false, "max_abs": 0.012681059539318085, "nrmse": 0.022398099303245544}, {"layer": 45, "field": "conv", "exact": false, "max_abs": 0.19384765625, "nrmse": 0.03539252653717995}, {"layer": 45, "field": "recurrent", "exact": false, "max_abs": 0.013250291347503662, "nrmse": 0.024941273033618927}, {"layer": 46, "field": "conv", "exact": false, "max_abs": 0.23095703125, "nrmse": 0.03386956825852394}, {"layer": 46, "field": "recurrent", "exact": false, "max_abs": 0.0203419029712677, "nrmse": 0.025895407423377037}, {"layer": 47, "field": "key", "exact": false, "max_abs": 0.55078125, "nrmse": 0.013820692896842957}, {"layer": 47, "field": "value", "exact": false, "max_abs": 0.34375, "nrmse": 0.01784161478281021}, {"layer": 48, "field": "conv", "exact": false, "max_abs": 0.28125, "nrmse": 0.03442436456680298}, {"layer": 48, "field": "recurrent", "exact": false, "max_abs": 0.00920608639717102, "nrmse": 0.02598695456981659}, {"layer": 49, "field": "conv", "exact": false, "max_abs": 0.244140625, "nrmse": 0.034482549875974655}, {"layer": 49, "field": "recurrent", "exact": false, "max_abs": 0.011885970830917358, "nrmse": 0.03193752467632294}, {"layer": 50, "field": "conv", "exact": false, "max_abs": 0.328125, "nrmse": 0.034281402826309204}, {"layer": 50, "field": "recurrent", "exact": false, "max_abs": 0.007781878113746643, "nrmse": 0.02949250303208828}, {"layer": 51, "field": "key", "exact": false, "max_abs": 0.46875, "nrmse": 0.0143220704048872}, {"layer": 51, "field": "value", "exact": false, "max_abs": 0.5625, "nrmse": 0.018947692587971687}, {"layer": 52, "field": "conv", "exact": false, "max_abs": 0.34375, "nrmse": 0.028860976919531822}, {"layer": 52, "field": "recurrent", "exact": false, "max_abs": 0.03936636447906494, "nrmse": 0.022364700213074684}, {"layer": 53, "field": "conv", "exact": false, "max_abs": 0.296875, "nrmse": 0.028963236138224602}, {"layer": 53, "field": "recurrent", "exact": false, "max_abs": 0.06146669387817383, "nrmse": 0.0289583932608366}, {"layer": 54, "field": "conv", "exact": false, "max_abs": 0.375, "nrmse": 0.032591935247182846}, {"layer": 54, "field": "recurrent", "exact": false, "max_abs": 0.02748700976371765, "nrmse": 0.033043839037418365}, {"layer": 55, "field": "key", "exact": false, "max_abs": 0.34765625, "nrmse": 0.014838352799415588}, {"layer": 55, "field": "value", "exact": false, "max_abs": 0.75, "nrmse": 0.02056911028921604}, {"layer": 56, "field": "conv", "exact": false, "max_abs": 0.359375, "nrmse": 0.02820638008415699}, {"layer": 56, "field": "recurrent", "exact": false, "max_abs": 0.032857805490493774, "nrmse": 0.026602504774928093}, {"layer": 57, "field": "conv", "exact": false, "max_abs": 0.33203125, "nrmse": 0.025639832019805908}, {"layer": 57, "field": "recurrent", "exact": false, "max_abs": 0.023078203201293945, "nrmse": 0.027800677344202995}, {"layer": 58, "field": "conv", "exact": false, "max_abs": 0.3203125, "nrmse": 0.028358014300465584}, {"layer": 58, "field": "recurrent", "exact": false, "max_abs": 0.04324686527252197, "nrmse": 0.025088975206017494}, {"layer": 59, "field": "key", "exact": false, "max_abs": 0.3125, "nrmse": 0.01457844115793705}, {"layer": 59, "field": "value", "exact": false, "max_abs": 1.0625, "nrmse": 0.01655724085867405}, {"layer": 60, "field": "conv", "exact": false, "max_abs": 1.0, "nrmse": 0.023989029228687286}, {"layer": 60, "field": "recurrent", "exact": false, "max_abs": 0.04270085692405701, "nrmse": 0.029323497787117958}, {"layer": 61, "field": "conv", "exact": false, "max_abs": 0.34375, "nrmse": 0.027256913483142853}, {"layer": 61, "field": "recurrent", "exact": false, "max_abs": 0.022582799196243286, "nrmse": 0.029611194506287575}, {"layer": 62, "field": "conv", "exact": false, "max_abs": 1.125, "nrmse": 0.02417750470340252}, {"layer": 62, "field": "recurrent", "exact": false, "max_abs": 0.05875198543071747, "nrmse": 0.022695189341902733}, {"layer": 63, "field": "key", "exact": false, "max_abs": 0.41796875, "nrmse": 0.01582689769566059}, {"layer": 63, "field": "value", "exact": false, "max_abs": 1.375, "nrmse": 0.009550346061587334}], "buffered_graph_state_diff": []} -{"phase": "step", "index": 77, "position": 225, "teacher_token": 198, "compact": {"ids": [9, 9764, 31318, 1112, 332], "logits": [28.0, 25.125, 21.875, 18.125, 14.9375], "margin": 2.875, "argmax": 9}, "buffered": {"ids": [9, 9764, 31318, 1112, 6558], "logits": [28.25, 25.5, 21.75, 18.375, 15.1875], "margin": 2.75, "argmax": 9}, "graph": {"ids": [9, 9764, 31318, 1112, 6558], "logits": [28.25, 25.5, 21.75, 18.375, 15.1875], "margin": 2.75, "argmax": 9}, "compact_buffered": {"exact": false, "max_abs": 0.375, "nrmse": 0.030249204486608505}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "compact_buffered_state_diff": [{"layer": 32, "field": "recurrent", "exact": false, "max_abs": 1.4565885066986084e-06, "nrmse": 1.5766318028909154e-06}, {"layer": 33, "field": "recurrent", "exact": false, "max_abs": 0.00018903985619544983, "nrmse": 0.00030132816755212843}, {"layer": 34, "field": "conv", "exact": false, "max_abs": 0.0625, "nrmse": 0.01362182293087244}, {"layer": 34, "field": "recurrent", "exact": false, "max_abs": 0.004169180989265442, "nrmse": 0.004950695671141148}, {"layer": 35, "field": "key", "exact": false, "max_abs": 0.1328125, "nrmse": 0.0027254081796854734}, {"layer": 35, "field": "value", "exact": false, "max_abs": 0.09375, "nrmse": 0.003242136212065816}, {"layer": 36, "field": "conv", "exact": false, "max_abs": 0.1875, "nrmse": 0.018841542303562164}, {"layer": 36, "field": "recurrent", "exact": false, "max_abs": 0.003907367587089539, "nrmse": 0.005279834847897291}, {"layer": 37, "field": "conv", "exact": false, "max_abs": 0.157684326171875, "nrmse": 0.02058456465601921}, {"layer": 37, "field": "recurrent", "exact": false, "max_abs": 0.009029805660247803, "nrmse": 0.007990630343556404}, {"layer": 38, "field": "conv", "exact": false, "max_abs": 0.15625, "nrmse": 0.026098188012838364}, {"layer": 38, "field": "recurrent", "exact": false, "max_abs": 0.008243530988693237, "nrmse": 0.016406575217843056}, {"layer": 39, "field": "key", "exact": false, "max_abs": 0.21875, "nrmse": 0.00826101005077362}, {"layer": 39, "field": "value", "exact": false, "max_abs": 0.21875, "nrmse": 0.010435450822114944}, {"layer": 40, "field": "conv", "exact": false, "max_abs": 0.16015625, "nrmse": 0.026795966550707817}, {"layer": 40, "field": "recurrent", "exact": false, "max_abs": 0.016475319862365723, "nrmse": 0.008722987957298756}, {"layer": 41, "field": "conv", "exact": false, "max_abs": 0.1875, "nrmse": 0.029372084885835648}, {"layer": 41, "field": "recurrent", "exact": false, "max_abs": 0.007658451795578003, "nrmse": 0.0173318013548851}, {"layer": 42, "field": "conv", "exact": false, "max_abs": 0.171875, "nrmse": 0.030847033485770226}, {"layer": 42, "field": "recurrent", "exact": false, "max_abs": 0.009500786662101746, "nrmse": 0.018576502799987793}, {"layer": 43, "field": "key", "exact": false, "max_abs": 0.3515625, "nrmse": 0.011722125113010406}, {"layer": 43, "field": "value", "exact": false, "max_abs": 0.328125, "nrmse": 0.013395676389336586}, {"layer": 44, "field": "conv", "exact": false, "max_abs": 0.1953125, "nrmse": 0.028963832184672356}, {"layer": 44, "field": "recurrent", "exact": false, "max_abs": 0.01267378032207489, "nrmse": 0.02255702205002308}, {"layer": 45, "field": "conv", "exact": false, "max_abs": 0.17578125, "nrmse": 0.03163528814911842}, {"layer": 45, "field": "recurrent", "exact": false, "max_abs": 0.013235509395599365, "nrmse": 0.025278475135564804}, {"layer": 46, "field": "conv", "exact": false, "max_abs": 0.1875, "nrmse": 0.03042733296751976}, {"layer": 46, "field": "recurrent", "exact": false, "max_abs": 0.017797216773033142, "nrmse": 0.02606642059981823}, {"layer": 47, "field": "key", "exact": false, "max_abs": 0.55078125, "nrmse": 0.013894467614591122}, {"layer": 47, "field": "value", "exact": false, "max_abs": 0.34375, "nrmse": 0.017957206815481186}, {"layer": 48, "field": "conv", "exact": false, "max_abs": 0.265625, "nrmse": 0.03150388225913048}, {"layer": 48, "field": "recurrent", "exact": false, "max_abs": 0.008302554488182068, "nrmse": 0.026408307254314423}, {"layer": 49, "field": "conv", "exact": false, "max_abs": 0.244140625, "nrmse": 0.03254052251577377}, {"layer": 49, "field": "recurrent", "exact": false, "max_abs": 0.011953860521316528, "nrmse": 0.03258463740348816}, {"layer": 50, "field": "conv", "exact": false, "max_abs": 0.328125, "nrmse": 0.03262294828891754}, {"layer": 50, "field": "recurrent", "exact": false, "max_abs": 0.007744789123535156, "nrmse": 0.02918909676373005}, {"layer": 51, "field": "key", "exact": false, "max_abs": 0.46875, "nrmse": 0.014440094120800495}, {"layer": 51, "field": "value", "exact": false, "max_abs": 0.5625, "nrmse": 0.01909213699400425}, {"layer": 52, "field": "conv", "exact": false, "max_abs": 0.34375, "nrmse": 0.02869347482919693}, {"layer": 52, "field": "recurrent", "exact": false, "max_abs": 0.03717350959777832, "nrmse": 0.022223029285669327}, {"layer": 53, "field": "conv", "exact": false, "max_abs": 0.296875, "nrmse": 0.028476953506469727}, {"layer": 53, "field": "recurrent", "exact": false, "max_abs": 0.061560094356536865, "nrmse": 0.02858319692313671}, {"layer": 54, "field": "conv", "exact": false, "max_abs": 0.375, "nrmse": 0.032526396214962006}, {"layer": 54, "field": "recurrent", "exact": false, "max_abs": 0.025927573442459106, "nrmse": 0.031626153737306595}, {"layer": 55, "field": "key", "exact": false, "max_abs": 0.34765625, "nrmse": 0.014893846586346626}, {"layer": 55, "field": "value", "exact": false, "max_abs": 0.75, "nrmse": 0.020716773346066475}, {"layer": 56, "field": "conv", "exact": false, "max_abs": 0.359375, "nrmse": 0.02825482375919819}, {"layer": 56, "field": "recurrent", "exact": false, "max_abs": 0.03243982791900635, "nrmse": 0.025874314829707146}, {"layer": 57, "field": "conv", "exact": false, "max_abs": 0.33203125, "nrmse": 0.02564380131661892}, {"layer": 57, "field": "recurrent", "exact": false, "max_abs": 0.02239002287387848, "nrmse": 0.025981664657592773}, {"layer": 58, "field": "conv", "exact": false, "max_abs": 0.3203125, "nrmse": 0.028710784390568733}, {"layer": 58, "field": "recurrent", "exact": false, "max_abs": 0.04342818260192871, "nrmse": 0.02356407605111599}, {"layer": 59, "field": "key", "exact": false, "max_abs": 0.3125, "nrmse": 0.014615606516599655}, {"layer": 59, "field": "value", "exact": false, "max_abs": 1.0625, "nrmse": 0.016642723232507706}, {"layer": 60, "field": "conv", "exact": false, "max_abs": 0.5, "nrmse": 0.024081138893961906}, {"layer": 60, "field": "recurrent", "exact": false, "max_abs": 0.04244345426559448, "nrmse": 0.029443053528666496}, {"layer": 61, "field": "conv", "exact": false, "max_abs": 0.34375, "nrmse": 0.027217477560043335}, {"layer": 61, "field": "recurrent", "exact": false, "max_abs": 0.0221174955368042, "nrmse": 0.028523147106170654}, {"layer": 62, "field": "conv", "exact": false, "max_abs": 0.625, "nrmse": 0.0234852097928524}, {"layer": 62, "field": "recurrent", "exact": false, "max_abs": 0.05566832423210144, "nrmse": 0.022467492148280144}, {"layer": 63, "field": "key", "exact": false, "max_abs": 0.41796875, "nrmse": 0.01588108018040657}, {"layer": 63, "field": "value", "exact": false, "max_abs": 1.375, "nrmse": 0.00959083903580904}], "buffered_graph_state_diff": []} -{"phase": "step", "index": 78, "position": 226, "teacher_token": 9, "compact": {"ids": [256, 13853, 262, 6558, 31318], "logits": [29.625, 19.125, 18.375, 17.125, 16.375], "margin": 10.5, "argmax": 256}, "buffered": {"ids": [256, 13853, 262, 6558, 31318], "logits": [30.0, 19.25, 18.75, 17.125, 16.625], "margin": 10.75, "argmax": 256}, "graph": {"ids": [256, 13853, 262, 6558, 31318], "logits": [30.0, 19.25, 18.75, 17.125, 16.625], "margin": 10.75, "argmax": 256}, "compact_buffered": {"exact": false, "max_abs": 0.5625, "nrmse": 0.018448878079652786}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "compact_buffered_state_diff": [{"layer": 32, "field": "recurrent", "exact": false, "max_abs": 9.872019290924072e-07, "nrmse": 1.3180656424083281e-06}, {"layer": 33, "field": "recurrent", "exact": false, "max_abs": 0.00017127394676208496, "nrmse": 0.0002799583598971367}, {"layer": 34, "field": "conv", "exact": false, "max_abs": 0.0625, "nrmse": 0.013050748966634274}, {"layer": 34, "field": "recurrent", "exact": false, "max_abs": 0.005196750164031982, "nrmse": 0.005078074522316456}, {"layer": 35, "field": "key", "exact": false, "max_abs": 0.15234375, "nrmse": 0.0030151091050356627}, {"layer": 35, "field": "value", "exact": false, "max_abs": 0.09375, "nrmse": 0.003440913511440158}, {"layer": 36, "field": "conv", "exact": false, "max_abs": 0.1875, "nrmse": 0.018374774605035782}, {"layer": 36, "field": "recurrent", "exact": false, "max_abs": 0.005102980881929398, "nrmse": 0.005887201055884361}, {"layer": 37, "field": "conv", "exact": false, "max_abs": 0.157684326171875, "nrmse": 0.02000533603131771}, {"layer": 37, "field": "recurrent", "exact": false, "max_abs": 0.010732382535934448, "nrmse": 0.007871677167713642}, {"layer": 38, "field": "conv", "exact": false, "max_abs": 0.15234375, "nrmse": 0.025283562019467354}, {"layer": 38, "field": "recurrent", "exact": false, "max_abs": 0.020235657691955566, "nrmse": 0.018108339980244637}, {"layer": 39, "field": "key", "exact": false, "max_abs": 0.21875, "nrmse": 0.008496435359120369}, {"layer": 39, "field": "value", "exact": false, "max_abs": 0.21875, "nrmse": 0.010606073774397373}, {"layer": 40, "field": "conv", "exact": false, "max_abs": 0.1640625, "nrmse": 0.02616128884255886}, {"layer": 40, "field": "recurrent", "exact": false, "max_abs": 0.00970943458378315, "nrmse": 0.008596164174377918}, {"layer": 41, "field": "conv", "exact": false, "max_abs": 0.1875, "nrmse": 0.028350556269288063}, {"layer": 41, "field": "recurrent", "exact": false, "max_abs": 0.007601790130138397, "nrmse": 0.01714695803821087}, {"layer": 42, "field": "conv", "exact": false, "max_abs": 0.171875, "nrmse": 0.03017672337591648}, {"layer": 42, "field": "recurrent", "exact": false, "max_abs": 0.00983932614326477, "nrmse": 0.018445786088705063}, {"layer": 43, "field": "key", "exact": false, "max_abs": 0.3515625, "nrmse": 0.011840574443340302}, {"layer": 43, "field": "value", "exact": false, "max_abs": 0.328125, "nrmse": 0.013502201996743679}, {"layer": 44, "field": "conv", "exact": false, "max_abs": 0.1875, "nrmse": 0.02839106321334839}, {"layer": 44, "field": "recurrent", "exact": false, "max_abs": 0.012519687414169312, "nrmse": 0.02232961356639862}, {"layer": 45, "field": "conv", "exact": false, "max_abs": 0.17578125, "nrmse": 0.03037324547767639}, {"layer": 45, "field": "recurrent", "exact": false, "max_abs": 0.013226807117462158, "nrmse": 0.024847058579325676}, {"layer": 46, "field": "conv", "exact": false, "max_abs": 0.1875, "nrmse": 0.029104076325893402}, {"layer": 46, "field": "recurrent", "exact": false, "max_abs": 0.016625747084617615, "nrmse": 0.025930017232894897}, {"layer": 47, "field": "key", "exact": false, "max_abs": 0.55078125, "nrmse": 0.013986299745738506}, {"layer": 47, "field": "value", "exact": false, "max_abs": 0.34375, "nrmse": 0.018060054630041122}, {"layer": 48, "field": "conv", "exact": false, "max_abs": 0.265625, "nrmse": 0.030728217214345932}, {"layer": 48, "field": "recurrent", "exact": false, "max_abs": 0.008015066385269165, "nrmse": 0.02617304027080536}, {"layer": 49, "field": "conv", "exact": false, "max_abs": 0.244140625, "nrmse": 0.032485481351614}, {"layer": 49, "field": "recurrent", "exact": false, "max_abs": 0.01944500207901001, "nrmse": 0.03130501136183739}, {"layer": 50, "field": "conv", "exact": false, "max_abs": 0.328125, "nrmse": 0.03214627131819725}, {"layer": 50, "field": "recurrent", "exact": false, "max_abs": 0.01284945011138916, "nrmse": 0.028714586049318314}, {"layer": 51, "field": "key", "exact": false, "max_abs": 0.46875, "nrmse": 0.01453383732587099}, {"layer": 51, "field": "value", "exact": false, "max_abs": 0.5625, "nrmse": 0.019194718450307846}, {"layer": 52, "field": "conv", "exact": false, "max_abs": 0.34375, "nrmse": 0.02853352390229702}, {"layer": 52, "field": "recurrent", "exact": false, "max_abs": 0.04600870609283447, "nrmse": 0.022249745205044746}, {"layer": 53, "field": "conv", "exact": false, "max_abs": 0.296875, "nrmse": 0.028755268082022667}, {"layer": 53, "field": "recurrent", "exact": false, "max_abs": 0.061964333057403564, "nrmse": 0.02804851531982422}, {"layer": 54, "field": "conv", "exact": false, "max_abs": 0.375, "nrmse": 0.03273189812898636}, {"layer": 54, "field": "recurrent", "exact": false, "max_abs": 0.026226401329040527, "nrmse": 0.031321823596954346}, {"layer": 55, "field": "key", "exact": false, "max_abs": 0.34765625, "nrmse": 0.015070096589624882}, {"layer": 55, "field": "value", "exact": false, "max_abs": 0.75, "nrmse": 0.020859001204371452}, {"layer": 56, "field": "conv", "exact": false, "max_abs": 0.34375, "nrmse": 0.028223106637597084}, {"layer": 56, "field": "recurrent", "exact": false, "max_abs": 0.03299173712730408, "nrmse": 0.02583909220993519}, {"layer": 57, "field": "conv", "exact": false, "max_abs": 0.33203125, "nrmse": 0.02584414929151535}, {"layer": 57, "field": "recurrent", "exact": false, "max_abs": 0.027801871299743652, "nrmse": 0.025800392031669617}, {"layer": 58, "field": "conv", "exact": false, "max_abs": 0.3203125, "nrmse": 0.02918865717947483}, {"layer": 58, "field": "recurrent", "exact": false, "max_abs": 0.0440366268157959, "nrmse": 0.023414865136146545}, {"layer": 59, "field": "key", "exact": false, "max_abs": 0.3125, "nrmse": 0.014708702452480793}, {"layer": 59, "field": "value", "exact": false, "max_abs": 1.0625, "nrmse": 0.01674974337220192}, {"layer": 60, "field": "conv", "exact": false, "max_abs": 0.75, "nrmse": 0.024189505726099014}, {"layer": 60, "field": "recurrent", "exact": false, "max_abs": 0.04343211650848389, "nrmse": 0.02930130437016487}, {"layer": 61, "field": "conv", "exact": false, "max_abs": 0.34375, "nrmse": 0.02758951298892498}, {"layer": 61, "field": "recurrent", "exact": false, "max_abs": 0.02252328395843506, "nrmse": 0.028452148661017418}, {"layer": 62, "field": "conv", "exact": false, "max_abs": 0.5859375, "nrmse": 0.023048030212521553}, {"layer": 62, "field": "recurrent", "exact": false, "max_abs": 0.050555258989334106, "nrmse": 0.022290589287877083}, {"layer": 63, "field": "key", "exact": false, "max_abs": 0.41796875, "nrmse": 0.015977883711457253}, {"layer": 63, "field": "value", "exact": false, "max_abs": 1.375, "nrmse": 0.009690017439424992}], "buffered_graph_state_diff": []} -{"phase": "step", "index": 79, "position": 227, "teacher_token": 256, "compact": {"ids": [13853, 6558, 16120, 10342, 5447], "logits": [24.75, 23.0, 19.5, 17.5, 16.875], "margin": 1.75, "argmax": 13853}, "buffered": {"ids": [13853, 6558, 16120, 10342, 5447], "logits": [24.875, 23.25, 19.25, 17.375, 16.75], "margin": 1.625, "argmax": 13853}, "graph": {"ids": [13853, 6558, 16120, 10342, 5447], "logits": [24.875, 23.25, 19.25, 17.375, 16.75], "margin": 1.625, "argmax": 13853}, "compact_buffered": {"exact": false, "max_abs": 0.34375, "nrmse": 0.023930197581648827}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "compact_buffered_state_diff": [{"layer": 32, "field": "recurrent", "exact": false, "max_abs": 7.618218660354614e-07, "nrmse": 1.0577671218925389e-06}, {"layer": 33, "field": "recurrent", "exact": false, "max_abs": 0.00015683751553297043, "nrmse": 0.0002588875067885965}, {"layer": 34, "field": "conv", "exact": false, "max_abs": 0.0625, "nrmse": 0.01289492379873991}, {"layer": 34, "field": "recurrent", "exact": false, "max_abs": 0.0031456351280212402, "nrmse": 0.004667704924941063}, {"layer": 35, "field": "key", "exact": false, "max_abs": 0.15234375, "nrmse": 0.0033769954461604357}, {"layer": 35, "field": "value", "exact": false, "max_abs": 0.09375, "nrmse": 0.0037203410174697638}, {"layer": 36, "field": "conv", "exact": false, "max_abs": 0.1875, "nrmse": 0.020301157608628273}, {"layer": 36, "field": "recurrent", "exact": false, "max_abs": 0.006976068019866943, "nrmse": 0.006675771903246641}, {"layer": 37, "field": "conv", "exact": false, "max_abs": 0.1640625, "nrmse": 0.021734226495027542}, {"layer": 37, "field": "recurrent", "exact": false, "max_abs": 0.007875502109527588, "nrmse": 0.009034648537635803}, {"layer": 38, "field": "conv", "exact": false, "max_abs": 0.16162109375, "nrmse": 0.02702426351606846}, {"layer": 38, "field": "recurrent", "exact": false, "max_abs": 0.009928792715072632, "nrmse": 0.01736849546432495}, {"layer": 39, "field": "key", "exact": false, "max_abs": 0.2578125, "nrmse": 0.008735769428312778}, {"layer": 39, "field": "value", "exact": false, "max_abs": 0.21875, "nrmse": 0.010815204121172428}, {"layer": 40, "field": "conv", "exact": false, "max_abs": 0.171875, "nrmse": 0.02791946940124035}, {"layer": 40, "field": "recurrent", "exact": false, "max_abs": 0.009672433137893677, "nrmse": 0.009169253520667553}, {"layer": 41, "field": "conv", "exact": false, "max_abs": 0.15625, "nrmse": 0.03022744134068489}, {"layer": 41, "field": "recurrent", "exact": false, "max_abs": 0.0077315568923950195, "nrmse": 0.01765761896967888}, {"layer": 42, "field": "conv", "exact": false, "max_abs": 0.15625, "nrmse": 0.03215649724006653}, {"layer": 42, "field": "recurrent", "exact": false, "max_abs": 0.011891059577465057, "nrmse": 0.019286129623651505}, {"layer": 43, "field": "key", "exact": false, "max_abs": 0.3515625, "nrmse": 0.012064654380083084}, {"layer": 43, "field": "value", "exact": false, "max_abs": 0.328125, "nrmse": 0.013707154430449009}, {"layer": 44, "field": "conv", "exact": false, "max_abs": 0.1875, "nrmse": 0.030241118744015694}, {"layer": 44, "field": "recurrent", "exact": false, "max_abs": 0.012434221804141998, "nrmse": 0.0224409531801939}, {"layer": 45, "field": "conv", "exact": false, "max_abs": 0.17578125, "nrmse": 0.03278222680091858}, {"layer": 45, "field": "recurrent", "exact": false, "max_abs": 0.013223886489868164, "nrmse": 0.024893714115023613}, {"layer": 46, "field": "conv", "exact": false, "max_abs": 0.1875, "nrmse": 0.03114733099937439}, {"layer": 46, "field": "recurrent", "exact": false, "max_abs": 0.014327764511108398, "nrmse": 0.02510513737797737}, {"layer": 47, "field": "key", "exact": false, "max_abs": 0.55078125, "nrmse": 0.014136610552668571}, {"layer": 47, "field": "value", "exact": false, "max_abs": 0.34375, "nrmse": 0.018212754279375076}, {"layer": 48, "field": "conv", "exact": false, "max_abs": 0.265625, "nrmse": 0.032868269830942154}, {"layer": 48, "field": "recurrent", "exact": false, "max_abs": 0.007921881973743439, "nrmse": 0.026497557759284973}, {"layer": 49, "field": "conv", "exact": false, "max_abs": 0.244140625, "nrmse": 0.033734556287527084}, {"layer": 49, "field": "recurrent", "exact": false, "max_abs": 0.019296646118164062, "nrmse": 0.030740033835172653}, {"layer": 50, "field": "conv", "exact": false, "max_abs": 0.2578125, "nrmse": 0.03262823447585106}, {"layer": 50, "field": "recurrent", "exact": false, "max_abs": 0.010905802249908447, "nrmse": 0.028749234974384308}, {"layer": 51, "field": "key", "exact": false, "max_abs": 0.46875, "nrmse": 0.014643323607742786}, {"layer": 51, "field": "value", "exact": false, "max_abs": 0.5625, "nrmse": 0.01931416057050228}, {"layer": 52, "field": "conv", "exact": false, "max_abs": 0.34375, "nrmse": 0.02894677221775055}, {"layer": 52, "field": "recurrent", "exact": false, "max_abs": 0.04843449592590332, "nrmse": 0.022281594574451447}, {"layer": 53, "field": "conv", "exact": false, "max_abs": 0.34375, "nrmse": 0.029553446918725967}, {"layer": 53, "field": "recurrent", "exact": false, "max_abs": 0.06162619590759277, "nrmse": 0.027937062084674835}, {"layer": 54, "field": "conv", "exact": false, "max_abs": 0.322265625, "nrmse": 0.033042870461940765}, {"layer": 54, "field": "recurrent", "exact": false, "max_abs": 0.02568812668323517, "nrmse": 0.03092491813004017}, {"layer": 55, "field": "key", "exact": false, "max_abs": 0.34765625, "nrmse": 0.015167254954576492}, {"layer": 55, "field": "value", "exact": false, "max_abs": 0.75, "nrmse": 0.020987175405025482}, {"layer": 56, "field": "conv", "exact": false, "max_abs": 0.34375, "nrmse": 0.028845060616731644}, {"layer": 56, "field": "recurrent", "exact": false, "max_abs": 0.032743215560913086, "nrmse": 0.025944313034415245}, {"layer": 57, "field": "conv", "exact": false, "max_abs": 0.33203125, "nrmse": 0.027244215831160545}, {"layer": 57, "field": "recurrent", "exact": false, "max_abs": 0.03467404842376709, "nrmse": 0.02562435157597065}, {"layer": 58, "field": "conv", "exact": false, "max_abs": 0.3203125, "nrmse": 0.030053598806262016}, {"layer": 58, "field": "recurrent", "exact": false, "max_abs": 0.044269680976867676, "nrmse": 0.02303493209183216}, {"layer": 59, "field": "key", "exact": false, "max_abs": 0.3125, "nrmse": 0.014824838377535343}, {"layer": 59, "field": "value", "exact": false, "max_abs": 1.0625, "nrmse": 0.01684960350394249}, {"layer": 60, "field": "conv", "exact": false, "max_abs": 0.75, "nrmse": 0.02494494989514351}, {"layer": 60, "field": "recurrent", "exact": false, "max_abs": 0.04277569055557251, "nrmse": 0.029550204053521156}, {"layer": 61, "field": "conv", "exact": false, "max_abs": 0.34375, "nrmse": 0.028263438493013382}, {"layer": 61, "field": "recurrent", "exact": false, "max_abs": 0.022269517183303833, "nrmse": 0.028611740097403526}, {"layer": 62, "field": "conv", "exact": false, "max_abs": 0.5859375, "nrmse": 0.02412082813680172}, {"layer": 62, "field": "recurrent", "exact": false, "max_abs": 0.04894280433654785, "nrmse": 0.022417372092604637}, {"layer": 63, "field": "key", "exact": false, "max_abs": 0.41796875, "nrmse": 0.01609557867050171}, {"layer": 63, "field": "value", "exact": false, "max_abs": 1.375, "nrmse": 0.009753894060850143}], "buffered_graph_state_diff": []} -{"phase": "step", "index": 80, "position": 228, "teacher_token": 13853, "compact": {"ids": [10471, 47866, 11039, 3568, 6895], "logits": [25.375, 24.375, 18.375, 18.0, 16.375], "margin": 1.0, "argmax": 10471}, "buffered": {"ids": [10471, 47866, 11039, 3568, 14307], "logits": [25.375, 24.25, 18.25, 17.875, 16.625], "margin": 1.125, "argmax": 10471}, "graph": {"ids": [10471, 47866, 11039, 3568, 14307], "logits": [25.375, 24.25, 18.25, 17.875, 16.625], "margin": 1.125, "argmax": 10471}, "compact_buffered": {"exact": false, "max_abs": 0.40625, "nrmse": 0.019007675349712372}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "compact_buffered_state_diff": [{"layer": 20, "field": "conv", "exact": false, "max_abs": 0.0625, "nrmse": 0.003361027454957366}, {"layer": 20, "field": "recurrent", "exact": false, "max_abs": 0.003707587718963623, "nrmse": 0.0014752991264685988}, {"layer": 21, "field": "conv", "exact": false, "max_abs": 0.0625, "nrmse": 0.005970000755041838}, {"layer": 21, "field": "recurrent", "exact": false, "max_abs": 0.0035887956619262695, "nrmse": 0.003198103979229927}, {"layer": 22, "field": "conv", "exact": false, "max_abs": 0.0625, "nrmse": 0.008110334165394306}, {"layer": 22, "field": "recurrent", "exact": false, "max_abs": 0.00313718244433403, "nrmse": 0.005671757273375988}, {"layer": 23, "field": "key", "exact": false, "max_abs": 0.09765625, "nrmse": 0.0010748665081337094}, {"layer": 23, "field": "value", "exact": false, "max_abs": 0.060546875, "nrmse": 0.001274999463930726}, {"layer": 24, "field": "conv", "exact": false, "max_abs": 0.09375, "nrmse": 0.010537249967455864}, {"layer": 24, "field": "recurrent", "exact": false, "max_abs": 0.00735589861869812, "nrmse": 0.004691677168011665}, {"layer": 25, "field": "conv", "exact": false, "max_abs": 0.125, "nrmse": 0.010295705869793892}, {"layer": 25, "field": "recurrent", "exact": false, "max_abs": 0.002385467290878296, "nrmse": 0.0020364627707749605}, {"layer": 26, "field": "conv", "exact": false, "max_abs": 0.09375, "nrmse": 0.012062355875968933}, {"layer": 26, "field": "recurrent", "exact": false, "max_abs": 0.004123024642467499, "nrmse": 0.004019129555672407}, {"layer": 27, "field": "key", "exact": false, "max_abs": 0.125, "nrmse": 0.0015638181939721107}, {"layer": 27, "field": "value", "exact": false, "max_abs": 0.078125, "nrmse": 0.001589867053553462}, {"layer": 28, "field": "conv", "exact": false, "max_abs": 0.125, "nrmse": 0.013500534929335117}, {"layer": 28, "field": "recurrent", "exact": false, "max_abs": 0.0031043458729982376, "nrmse": 0.0028002234175801277}, {"layer": 29, "field": "conv", "exact": false, "max_abs": 0.15625, "nrmse": 0.01553949061781168}, {"layer": 29, "field": "recurrent", "exact": false, "max_abs": 0.002476505935192108, "nrmse": 0.0038385652005672455}, {"layer": 30, "field": "conv", "exact": false, "max_abs": 0.125, "nrmse": 0.015000732615590096}, {"layer": 30, "field": "recurrent", "exact": false, "max_abs": 0.024011924862861633, "nrmse": 0.00867390725761652}, {"layer": 31, "field": "key", "exact": false, "max_abs": 0.1875, "nrmse": 0.0017844515386968851}, {"layer": 31, "field": "value", "exact": false, "max_abs": 0.15625, "nrmse": 0.0019121516961604357}, {"layer": 32, "field": "conv", "exact": false, "max_abs": 0.13525390625, "nrmse": 0.017324350774288177}, {"layer": 32, "field": "recurrent", "exact": false, "max_abs": 0.0014785360544919968, "nrmse": 0.0039046977180987597}, {"layer": 33, "field": "conv", "exact": false, "max_abs": 0.15625, "nrmse": 0.017417704686522484}, {"layer": 33, "field": "recurrent", "exact": false, "max_abs": 0.009818203747272491, "nrmse": 0.010498945601284504}, {"layer": 34, "field": "conv", "exact": false, "max_abs": 0.1171875, "nrmse": 0.018000135198235512}, {"layer": 34, "field": "recurrent", "exact": false, "max_abs": 0.006982579827308655, "nrmse": 0.0065082707442343235}, {"layer": 35, "field": "key", "exact": false, "max_abs": 0.21875, "nrmse": 0.0038568987511098385}, {"layer": 35, "field": "value", "exact": false, "max_abs": 0.1302490234375, "nrmse": 0.004295917693525553}, {"layer": 36, "field": "conv", "exact": false, "max_abs": 0.1875, "nrmse": 0.022272150963544846}, {"layer": 36, "field": "recurrent", "exact": false, "max_abs": 0.011461079120635986, "nrmse": 0.007301708217710257}, {"layer": 37, "field": "conv", "exact": false, "max_abs": 0.1875, "nrmse": 0.022219447419047356}, {"layer": 37, "field": "recurrent", "exact": false, "max_abs": 0.012368366122245789, "nrmse": 0.010254555381834507}, {"layer": 38, "field": "conv", "exact": false, "max_abs": 0.16162109375, "nrmse": 0.026895297691226006}, {"layer": 38, "field": "recurrent", "exact": false, "max_abs": 0.01341855525970459, "nrmse": 0.018811646848917007}, {"layer": 39, "field": "key", "exact": false, "max_abs": 0.2578125, "nrmse": 0.008924352005124092}, {"layer": 39, "field": "value", "exact": false, "max_abs": 0.21875, "nrmse": 0.01103356946259737}, {"layer": 40, "field": "conv", "exact": false, "max_abs": 0.171875, "nrmse": 0.027780845761299133}, {"layer": 40, "field": "recurrent", "exact": false, "max_abs": 0.010508418083190918, "nrmse": 0.009294486604630947}, {"layer": 41, "field": "conv", "exact": false, "max_abs": 0.15625, "nrmse": 0.028845569118857384}, {"layer": 41, "field": "recurrent", "exact": false, "max_abs": 0.007622852921485901, "nrmse": 0.017633726820349693}, {"layer": 42, "field": "conv", "exact": false, "max_abs": 0.15625, "nrmse": 0.03150179609656334}, {"layer": 42, "field": "recurrent", "exact": false, "max_abs": 0.009709373116493225, "nrmse": 0.01912587694823742}, {"layer": 43, "field": "key", "exact": false, "max_abs": 0.3515625, "nrmse": 0.012186545878648758}, {"layer": 43, "field": "value", "exact": false, "max_abs": 0.328125, "nrmse": 0.013839037157595158}, {"layer": 44, "field": "conv", "exact": false, "max_abs": 0.16339111328125, "nrmse": 0.029742954298853874}, {"layer": 44, "field": "recurrent", "exact": false, "max_abs": 0.012603893876075745, "nrmse": 0.022285452112555504}, {"layer": 45, "field": "conv", "exact": false, "max_abs": 0.15234375, "nrmse": 0.03185570612549782}, {"layer": 45, "field": "recurrent", "exact": false, "max_abs": 0.013221144676208496, "nrmse": 0.02537590079009533}, {"layer": 46, "field": "conv", "exact": false, "max_abs": 0.171875, "nrmse": 0.03039732202887535}, {"layer": 46, "field": "recurrent", "exact": false, "max_abs": 0.014310479164123535, "nrmse": 0.02507288008928299}, {"layer": 47, "field": "key", "exact": false, "max_abs": 0.55078125, "nrmse": 0.014227514155209064}, {"layer": 47, "field": "value", "exact": false, "max_abs": 0.34375, "nrmse": 0.018329042941331863}, {"layer": 48, "field": "conv", "exact": false, "max_abs": 0.265625, "nrmse": 0.03270222619175911}, {"layer": 48, "field": "recurrent", "exact": false, "max_abs": 0.008031576871871948, "nrmse": 0.02661074325442314}, {"layer": 49, "field": "conv", "exact": false, "max_abs": 0.2109375, "nrmse": 0.03201735392212868}, {"layer": 49, "field": "recurrent", "exact": false, "max_abs": 0.017760932445526123, "nrmse": 0.030305802822113037}, {"layer": 50, "field": "conv", "exact": false, "max_abs": 0.2578125, "nrmse": 0.030891163274645805}, {"layer": 50, "field": "recurrent", "exact": false, "max_abs": 0.0084972083568573, "nrmse": 0.028478655964136124}, {"layer": 51, "field": "key", "exact": false, "max_abs": 0.46875, "nrmse": 0.014726531691849232}, {"layer": 51, "field": "value", "exact": false, "max_abs": 0.5625, "nrmse": 0.019451189786195755}, {"layer": 52, "field": "conv", "exact": false, "max_abs": 0.2578125, "nrmse": 0.02730063535273075}, {"layer": 52, "field": "recurrent", "exact": false, "max_abs": 0.056208252906799316, "nrmse": 0.02238805964589119}, {"layer": 53, "field": "conv", "exact": false, "max_abs": 0.34375, "nrmse": 0.027474451810121536}, {"layer": 53, "field": "recurrent", "exact": false, "max_abs": 0.05993020534515381, "nrmse": 0.027965392917394638}, {"layer": 54, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.030551260337233543}, {"layer": 54, "field": "recurrent", "exact": false, "max_abs": 0.026553243398666382, "nrmse": 0.030921828001737595}, {"layer": 55, "field": "key", "exact": false, "max_abs": 0.34765625, "nrmse": 0.015257272869348526}, {"layer": 55, "field": "value", "exact": false, "max_abs": 0.75, "nrmse": 0.021122194826602936}, {"layer": 56, "field": "conv", "exact": false, "max_abs": 0.34375, "nrmse": 0.026746239513158798}, {"layer": 56, "field": "recurrent", "exact": false, "max_abs": 0.03246772289276123, "nrmse": 0.025761160999536514}, {"layer": 57, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.02543981745839119}, {"layer": 57, "field": "recurrent", "exact": false, "max_abs": 0.026831388473510742, "nrmse": 0.025404149666428566}, {"layer": 58, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.028048358857631683}, {"layer": 58, "field": "recurrent", "exact": false, "max_abs": 0.03823620080947876, "nrmse": 0.023484600707888603}, {"layer": 59, "field": "key", "exact": false, "max_abs": 0.3125, "nrmse": 0.014959066174924374}, {"layer": 59, "field": "value", "exact": false, "max_abs": 1.0625, "nrmse": 0.016986440867185593}, {"layer": 60, "field": "conv", "exact": false, "max_abs": 0.75, "nrmse": 0.02320590615272522}, {"layer": 60, "field": "recurrent", "exact": false, "max_abs": 0.04654684662818909, "nrmse": 0.029794473201036453}, {"layer": 61, "field": "conv", "exact": false, "max_abs": 0.296875, "nrmse": 0.025400392711162567}, {"layer": 61, "field": "recurrent", "exact": false, "max_abs": 0.021627992391586304, "nrmse": 0.028081173077225685}, {"layer": 62, "field": "conv", "exact": false, "max_abs": 0.5859375, "nrmse": 0.022956104949116707}, {"layer": 62, "field": "recurrent", "exact": false, "max_abs": 0.04880046844482422, "nrmse": 0.02260305918753147}, {"layer": 63, "field": "key", "exact": false, "max_abs": 0.41796875, "nrmse": 0.016181176528334618}, {"layer": 63, "field": "value", "exact": false, "max_abs": 1.375, "nrmse": 0.009799848310649395}], "buffered_graph_state_diff": []} -{"phase": "step", "index": 81, "position": 229, "teacher_token": 10471, "compact": {"ids": [264, 220, 393, 1010, 4945], "logits": [25.625, 25.625, 21.0, 18.75, 18.125], "margin": 0.0, "argmax": 220}, "buffered": {"ids": [264, 220, 393, 1010, 4945], "logits": [25.75, 25.625, 21.0, 18.5, 18.125], "margin": 0.125, "argmax": 264}, "graph": {"ids": [264, 220, 393, 1010, 4945], "logits": [25.75, 25.625, 21.0, 18.5, 18.125], "margin": 0.125, "argmax": 264}, "compact_buffered": {"exact": false, "max_abs": 0.328125, "nrmse": 0.020043984055519104}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "compact_buffered_state_diff": [{"layer": 20, "field": "conv", "exact": false, "max_abs": 0.0625, "nrmse": 0.003238220000639558}, {"layer": 20, "field": "recurrent", "exact": false, "max_abs": 0.0033497214317321777, "nrmse": 0.001683499664068222}, {"layer": 21, "field": "conv", "exact": false, "max_abs": 0.0625, "nrmse": 0.007927672937512398}, {"layer": 21, "field": "recurrent", "exact": false, "max_abs": 0.008188486099243164, "nrmse": 0.004739583004266024}, {"layer": 22, "field": "conv", "exact": false, "max_abs": 0.078125, "nrmse": 0.01145197730511427}, {"layer": 22, "field": "recurrent", "exact": false, "max_abs": 0.005116403102874756, "nrmse": 0.007339758798480034}, {"layer": 23, "field": "key", "exact": false, "max_abs": 0.09765625, "nrmse": 0.001595001551322639}, {"layer": 23, "field": "value", "exact": false, "max_abs": 0.0625, "nrmse": 0.001956133171916008}, {"layer": 24, "field": "conv", "exact": false, "max_abs": 0.09375, "nrmse": 0.014656418934464455}, {"layer": 24, "field": "recurrent", "exact": false, "max_abs": 0.0033346868585795164, "nrmse": 0.006805642507970333}, {"layer": 25, "field": "conv", "exact": false, "max_abs": 0.125, "nrmse": 0.015634246170520782}, {"layer": 25, "field": "recurrent", "exact": false, "max_abs": 0.006082162261009216, "nrmse": 0.005533641204237938}, {"layer": 26, "field": "conv", "exact": false, "max_abs": 0.099609375, "nrmse": 0.01822802424430847}, {"layer": 26, "field": "recurrent", "exact": false, "max_abs": 0.0067693740129470825, "nrmse": 0.006911869160830975}, {"layer": 27, "field": "key", "exact": false, "max_abs": 0.171875, "nrmse": 0.002220097929239273}, {"layer": 27, "field": "value", "exact": false, "max_abs": 0.078125, "nrmse": 0.0022636372596025467}, {"layer": 28, "field": "conv", "exact": false, "max_abs": 0.125, "nrmse": 0.020016230642795563}, {"layer": 28, "field": "recurrent", "exact": false, "max_abs": 0.005199939012527466, "nrmse": 0.005711168982088566}, {"layer": 29, "field": "conv", "exact": false, "max_abs": 0.15625, "nrmse": 0.022252157330513}, {"layer": 29, "field": "recurrent", "exact": false, "max_abs": 0.004465529695153236, "nrmse": 0.008554836735129356}, {"layer": 30, "field": "conv", "exact": false, "max_abs": 0.125, "nrmse": 0.021033484488725662}, {"layer": 30, "field": "recurrent", "exact": false, "max_abs": 0.03635162115097046, "nrmse": 0.015566008165478706}, {"layer": 31, "field": "key", "exact": false, "max_abs": 0.1875, "nrmse": 0.002431110944598913}, {"layer": 31, "field": "value", "exact": false, "max_abs": 0.15625, "nrmse": 0.0028561539947986603}, {"layer": 32, "field": "conv", "exact": false, "max_abs": 0.140625, "nrmse": 0.025356058031320572}, {"layer": 32, "field": "recurrent", "exact": false, "max_abs": 0.0021752947941422462, "nrmse": 0.0072195883840322495}, {"layer": 33, "field": "conv", "exact": false, "max_abs": 0.15625, "nrmse": 0.025240696966648102}, {"layer": 33, "field": "recurrent", "exact": false, "max_abs": 0.010773450136184692, "nrmse": 0.013880569487810135}, {"layer": 34, "field": "conv", "exact": false, "max_abs": 0.15625, "nrmse": 0.023793509230017662}, {"layer": 34, "field": "recurrent", "exact": false, "max_abs": 0.00804564356803894, "nrmse": 0.00798184797167778}, {"layer": 35, "field": "key", "exact": false, "max_abs": 0.21875, "nrmse": 0.004196259193122387}, {"layer": 35, "field": "value", "exact": false, "max_abs": 0.15625, "nrmse": 0.004887334071099758}, {"layer": 36, "field": "conv", "exact": false, "max_abs": 0.1875, "nrmse": 0.023919647559523582}, {"layer": 36, "field": "recurrent", "exact": false, "max_abs": 0.007403671741485596, "nrmse": 0.008040493354201317}, {"layer": 37, "field": "conv", "exact": false, "max_abs": 0.1875, "nrmse": 0.023776425048708916}, {"layer": 37, "field": "recurrent", "exact": false, "max_abs": 0.011884182691574097, "nrmse": 0.010816446505486965}, {"layer": 38, "field": "conv", "exact": false, "max_abs": 0.16162109375, "nrmse": 0.029164066538214684}, {"layer": 38, "field": "recurrent", "exact": false, "max_abs": 0.01937335729598999, "nrmse": 0.02062797360122204}, {"layer": 39, "field": "key", "exact": false, "max_abs": 0.2578125, "nrmse": 0.009052345529198647}, {"layer": 39, "field": "value", "exact": false, "max_abs": 0.21875, "nrmse": 0.011315962299704552}, {"layer": 40, "field": "conv", "exact": false, "max_abs": 0.2265625, "nrmse": 0.02956352010369301}, {"layer": 40, "field": "recurrent", "exact": false, "max_abs": 0.00987708568572998, "nrmse": 0.009389720857143402}, {"layer": 41, "field": "conv", "exact": false, "max_abs": 0.15625, "nrmse": 0.030248211696743965}, {"layer": 41, "field": "recurrent", "exact": false, "max_abs": 0.009534848853945732, "nrmse": 0.018468603491783142}, {"layer": 42, "field": "conv", "exact": false, "max_abs": 0.1875, "nrmse": 0.03361322358250618}, {"layer": 42, "field": "recurrent", "exact": false, "max_abs": 0.010495854541659355, "nrmse": 0.01956837996840477}, {"layer": 43, "field": "key", "exact": false, "max_abs": 0.3515625, "nrmse": 0.01231664139777422}, {"layer": 43, "field": "value", "exact": false, "max_abs": 0.328125, "nrmse": 0.013996611349284649}, {"layer": 44, "field": "conv", "exact": false, "max_abs": 0.21875, "nrmse": 0.03111221455037594}, {"layer": 44, "field": "recurrent", "exact": false, "max_abs": 0.012761212885379791, "nrmse": 0.0223674476146698}, {"layer": 45, "field": "conv", "exact": false, "max_abs": 0.18359375, "nrmse": 0.03379956632852554}, {"layer": 45, "field": "recurrent", "exact": false, "max_abs": 0.013222038745880127, "nrmse": 0.026128435507416725}, {"layer": 46, "field": "conv", "exact": false, "max_abs": 0.21875, "nrmse": 0.032582927495241165}, {"layer": 46, "field": "recurrent", "exact": false, "max_abs": 0.015996724367141724, "nrmse": 0.025760050863027573}, {"layer": 47, "field": "key", "exact": false, "max_abs": 0.55078125, "nrmse": 0.014307287521660328}, {"layer": 47, "field": "value", "exact": false, "max_abs": 0.34375, "nrmse": 0.0184637401252985}, {"layer": 48, "field": "conv", "exact": false, "max_abs": 0.2578125, "nrmse": 0.03383893519639969}, {"layer": 48, "field": "recurrent", "exact": false, "max_abs": 0.007879287004470825, "nrmse": 0.026553772389888763}, {"layer": 49, "field": "conv", "exact": false, "max_abs": 0.2265625, "nrmse": 0.03279079869389534}, {"layer": 49, "field": "recurrent", "exact": false, "max_abs": 0.017641663551330566, "nrmse": 0.031484946608543396}, {"layer": 50, "field": "conv", "exact": false, "max_abs": 0.25, "nrmse": 0.031816914677619934}, {"layer": 50, "field": "recurrent", "exact": false, "max_abs": 0.008366147056221962, "nrmse": 0.02932436391711235}, {"layer": 51, "field": "key", "exact": false, "max_abs": 0.46875, "nrmse": 0.014811933040618896}, {"layer": 51, "field": "value", "exact": false, "max_abs": 0.5625, "nrmse": 0.0196121484041214}, {"layer": 52, "field": "conv", "exact": false, "max_abs": 0.2578125, "nrmse": 0.02672635018825531}, {"layer": 52, "field": "recurrent", "exact": false, "max_abs": 0.05252361297607422, "nrmse": 0.02265191450715065}, {"layer": 53, "field": "conv", "exact": false, "max_abs": 0.34375, "nrmse": 0.026979800313711166}, {"layer": 53, "field": "recurrent", "exact": false, "max_abs": 0.05795729160308838, "nrmse": 0.028047896921634674}, {"layer": 54, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.030254313722252846}, {"layer": 54, "field": "recurrent", "exact": false, "max_abs": 0.023876488208770752, "nrmse": 0.031239215284585953}, {"layer": 55, "field": "key", "exact": false, "max_abs": 0.34765625, "nrmse": 0.015328844077885151}, {"layer": 55, "field": "value", "exact": false, "max_abs": 0.75, "nrmse": 0.021236373111605644}, {"layer": 56, "field": "conv", "exact": false, "max_abs": 0.34375, "nrmse": 0.02675168402493}, {"layer": 56, "field": "recurrent", "exact": false, "max_abs": 0.0370834618806839, "nrmse": 0.026777712628245354}, {"layer": 57, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.025196053087711334}, {"layer": 57, "field": "recurrent", "exact": false, "max_abs": 0.030250191688537598, "nrmse": 0.02655748836696148}, {"layer": 58, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.027784314006567}, {"layer": 58, "field": "recurrent", "exact": false, "max_abs": 0.03343325853347778, "nrmse": 0.024125654250383377}, {"layer": 59, "field": "key", "exact": false, "max_abs": 0.3125, "nrmse": 0.01504741795361042}, {"layer": 59, "field": "value", "exact": false, "max_abs": 1.0625, "nrmse": 0.017137136310338974}, {"layer": 60, "field": "conv", "exact": false, "max_abs": 0.75, "nrmse": 0.023273907601833344}, {"layer": 60, "field": "recurrent", "exact": false, "max_abs": 0.042760252952575684, "nrmse": 0.029931865632534027}, {"layer": 61, "field": "conv", "exact": false, "max_abs": 0.296875, "nrmse": 0.02525276690721512}, {"layer": 61, "field": "recurrent", "exact": false, "max_abs": 0.032279789447784424, "nrmse": 0.0289892740547657}, {"layer": 62, "field": "conv", "exact": false, "max_abs": 0.5859375, "nrmse": 0.023136021569371223}, {"layer": 62, "field": "recurrent", "exact": false, "max_abs": 0.04896062612533569, "nrmse": 0.0225808247923851}, {"layer": 63, "field": "key", "exact": false, "max_abs": 0.41796875, "nrmse": 0.016251353546977043}, {"layer": 63, "field": "value", "exact": false, "max_abs": 1.375, "nrmse": 0.009857971221208572}], "buffered_graph_state_diff": []} -{"phase": "step", "index": 82, "position": 230, "teacher_token": 220, "compact": {"ids": [16, 22, 17, 20, 19], "logits": [29.5, 17.75, 16.0, 15.5625, 15.5], "margin": 11.75, "argmax": 16}, "buffered": {"ids": [16, 22, 17, 20, 18], "logits": [29.5, 17.625, 15.9375, 15.75, 15.5], "margin": 11.875, "argmax": 16}, "graph": {"ids": [16, 22, 17, 20, 18], "logits": [29.5, 17.625, 15.9375, 15.75, 15.5], "margin": 11.875, "argmax": 16}, "compact_buffered": {"exact": false, "max_abs": 0.3203125, "nrmse": 0.017478501424193382}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "compact_buffered_state_diff": [{"layer": 20, "field": "conv", "exact": false, "max_abs": 0.0625, "nrmse": 0.004891497083008289}, {"layer": 20, "field": "recurrent", "exact": false, "max_abs": 0.0025340765714645386, "nrmse": 0.0018022449221462011}, {"layer": 21, "field": "conv", "exact": false, "max_abs": 0.068359375, "nrmse": 0.010516401380300522}, {"layer": 21, "field": "recurrent", "exact": false, "max_abs": 0.006798982620239258, "nrmse": 0.005128654185682535}, {"layer": 22, "field": "conv", "exact": false, "max_abs": 0.08984375, "nrmse": 0.014700721949338913}, {"layer": 22, "field": "recurrent", "exact": false, "max_abs": 0.006006151437759399, "nrmse": 0.00908160675317049}, {"layer": 23, "field": "key", "exact": false, "max_abs": 0.125, "nrmse": 0.0020471455063670874}, {"layer": 23, "field": "value", "exact": false, "max_abs": 0.0625, "nrmse": 0.0023661041632294655}, {"layer": 24, "field": "conv", "exact": false, "max_abs": 0.09375, "nrmse": 0.01850752905011177}, {"layer": 24, "field": "recurrent", "exact": false, "max_abs": 0.005962297320365906, "nrmse": 0.008391380310058594}, {"layer": 25, "field": "conv", "exact": false, "max_abs": 0.125, "nrmse": 0.019321605563163757}, {"layer": 25, "field": "recurrent", "exact": false, "max_abs": 0.006306752562522888, "nrmse": 0.007751068565994501}, {"layer": 26, "field": "conv", "exact": false, "max_abs": 0.099609375, "nrmse": 0.02199520170688629}, {"layer": 26, "field": "recurrent", "exact": false, "max_abs": 0.0067092180252075195, "nrmse": 0.009210257790982723}, {"layer": 27, "field": "key", "exact": false, "max_abs": 0.171875, "nrmse": 0.0027477743569761515}, {"layer": 27, "field": "value", "exact": false, "max_abs": 0.10546875, "nrmse": 0.002798666013404727}, {"layer": 28, "field": "conv", "exact": false, "max_abs": 0.1328125, "nrmse": 0.025227779522538185}, {"layer": 28, "field": "recurrent", "exact": false, "max_abs": 0.005718126893043518, "nrmse": 0.008701802231371403}, {"layer": 29, "field": "conv", "exact": false, "max_abs": 0.171875, "nrmse": 0.027931934222579002}, {"layer": 29, "field": "recurrent", "exact": false, "max_abs": 0.006219945847988129, "nrmse": 0.013429363258183002}, {"layer": 30, "field": "conv", "exact": false, "max_abs": 0.15625, "nrmse": 0.027417974546551704}, {"layer": 30, "field": "recurrent", "exact": false, "max_abs": 0.025470077991485596, "nrmse": 0.014410583302378654}, {"layer": 31, "field": "key", "exact": false, "max_abs": 0.24658203125, "nrmse": 0.0032118686940521}, {"layer": 31, "field": "value", "exact": false, "max_abs": 0.1640625, "nrmse": 0.0038645805325359106}, {"layer": 32, "field": "conv", "exact": false, "max_abs": 0.203125, "nrmse": 0.034029699862003326}, {"layer": 32, "field": "recurrent", "exact": false, "max_abs": 0.0037054046988487244, "nrmse": 0.011929291300475597}, {"layer": 33, "field": "conv", "exact": false, "max_abs": 0.1875, "nrmse": 0.03297307714819908}, {"layer": 33, "field": "recurrent", "exact": false, "max_abs": 0.013304471969604492, "nrmse": 0.021084014326334}, {"layer": 34, "field": "conv", "exact": false, "max_abs": 0.1875, "nrmse": 0.0304563008248806}, {"layer": 34, "field": "recurrent", "exact": false, "max_abs": 0.005730889737606049, "nrmse": 0.010474284179508686}, {"layer": 35, "field": "key", "exact": false, "max_abs": 0.21875, "nrmse": 0.004665891174226999}, {"layer": 35, "field": "value", "exact": false, "max_abs": 0.15625, "nrmse": 0.0055845133028924465}, {"layer": 36, "field": "conv", "exact": false, "max_abs": 0.1875, "nrmse": 0.026546310633420944}, {"layer": 36, "field": "recurrent", "exact": false, "max_abs": 0.0072310566902160645, "nrmse": 0.009051607921719551}, {"layer": 37, "field": "conv", "exact": false, "max_abs": 0.25, "nrmse": 0.026481056585907936}, {"layer": 37, "field": "recurrent", "exact": false, "max_abs": 0.00992169976234436, "nrmse": 0.011738176457583904}, {"layer": 38, "field": "conv", "exact": false, "max_abs": 0.25927734375, "nrmse": 0.03260139748454094}, {"layer": 38, "field": "recurrent", "exact": false, "max_abs": 0.013074830174446106, "nrmse": 0.022773858159780502}, {"layer": 39, "field": "key", "exact": false, "max_abs": 0.2578125, "nrmse": 0.009214329533278942}, {"layer": 39, "field": "value", "exact": false, "max_abs": 0.21875, "nrmse": 0.011556142009794712}, {"layer": 40, "field": "conv", "exact": false, "max_abs": 0.28125, "nrmse": 0.032248932868242264}, {"layer": 40, "field": "recurrent", "exact": false, "max_abs": 0.009655565023422241, "nrmse": 0.01013333909213543}, {"layer": 41, "field": "conv", "exact": false, "max_abs": 0.1953125, "nrmse": 0.032366521656513214}, {"layer": 41, "field": "recurrent", "exact": false, "max_abs": 0.01016806811094284, "nrmse": 0.01976105384528637}, {"layer": 42, "field": "conv", "exact": false, "max_abs": 0.2421875, "nrmse": 0.035890065133571625}, {"layer": 42, "field": "recurrent", "exact": false, "max_abs": 0.012838553637266159, "nrmse": 0.02011692337691784}, {"layer": 43, "field": "key", "exact": false, "max_abs": 0.3515625, "nrmse": 0.012453943490982056}, {"layer": 43, "field": "value", "exact": false, "max_abs": 0.328125, "nrmse": 0.014188186265528202}, {"layer": 44, "field": "conv", "exact": false, "max_abs": 0.21875, "nrmse": 0.03371432051062584}, {"layer": 44, "field": "recurrent", "exact": false, "max_abs": 0.012799985706806183, "nrmse": 0.023395277559757233}, {"layer": 45, "field": "conv", "exact": false, "max_abs": 0.19140625, "nrmse": 0.037059981375932693}, {"layer": 45, "field": "recurrent", "exact": false, "max_abs": 0.013222754001617432, "nrmse": 0.027209926396608353}, {"layer": 46, "field": "conv", "exact": false, "max_abs": 0.21875, "nrmse": 0.035626765340566635}, {"layer": 46, "field": "recurrent", "exact": false, "max_abs": 0.01400899887084961, "nrmse": 0.025792639702558517}, {"layer": 47, "field": "key", "exact": false, "max_abs": 0.55078125, "nrmse": 0.014390919357538223}, {"layer": 47, "field": "value", "exact": false, "max_abs": 0.34375, "nrmse": 0.018627183511853218}, {"layer": 48, "field": "conv", "exact": false, "max_abs": 0.2578125, "nrmse": 0.0367831289768219}, {"layer": 48, "field": "recurrent", "exact": false, "max_abs": 0.007845006883144379, "nrmse": 0.027172675356268883}, {"layer": 49, "field": "conv", "exact": false, "max_abs": 0.337890625, "nrmse": 0.03504073992371559}, {"layer": 49, "field": "recurrent", "exact": false, "max_abs": 0.017428338527679443, "nrmse": 0.03219189867377281}, {"layer": 50, "field": "conv", "exact": false, "max_abs": 0.28125, "nrmse": 0.03396521881222725}, {"layer": 50, "field": "recurrent", "exact": false, "max_abs": 0.008138641715049744, "nrmse": 0.02983446791768074}, {"layer": 51, "field": "key", "exact": false, "max_abs": 0.46875, "nrmse": 0.014887404628098011}, {"layer": 51, "field": "value", "exact": false, "max_abs": 0.5625, "nrmse": 0.0197422094643116}, {"layer": 52, "field": "conv", "exact": false, "max_abs": 0.373046875, "nrmse": 0.027812575921416283}, {"layer": 52, "field": "recurrent", "exact": false, "max_abs": 0.047148704528808594, "nrmse": 0.022646846249699593}, {"layer": 53, "field": "conv", "exact": false, "max_abs": 0.34375, "nrmse": 0.027715250849723816}, {"layer": 53, "field": "recurrent", "exact": false, "max_abs": 0.057835936546325684, "nrmse": 0.028486544266343117}, {"layer": 54, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.03089176118373871}, {"layer": 54, "field": "recurrent", "exact": false, "max_abs": 0.022611960768699646, "nrmse": 0.03172457963228226}, {"layer": 55, "field": "key", "exact": false, "max_abs": 0.34765625, "nrmse": 0.015415373258292675}, {"layer": 55, "field": "value", "exact": false, "max_abs": 0.75, "nrmse": 0.021360494196414948}, {"layer": 56, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.027158550918102264}, {"layer": 56, "field": "recurrent", "exact": false, "max_abs": 0.03471583127975464, "nrmse": 0.026599928736686707}, {"layer": 57, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.025363864377141}, {"layer": 57, "field": "recurrent", "exact": false, "max_abs": 0.03126460313796997, "nrmse": 0.026791328564286232}, {"layer": 58, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.028029711917042732}, {"layer": 58, "field": "recurrent", "exact": false, "max_abs": 0.0339486300945282, "nrmse": 0.02368452586233616}, {"layer": 59, "field": "key", "exact": false, "max_abs": 0.3125, "nrmse": 0.015144978649914265}, {"layer": 59, "field": "value", "exact": false, "max_abs": 1.0625, "nrmse": 0.01722080633044243}, {"layer": 60, "field": "conv", "exact": false, "max_abs": 0.625, "nrmse": 0.02350635826587677}, {"layer": 60, "field": "recurrent", "exact": false, "max_abs": 0.06436294317245483, "nrmse": 0.030017703771591187}, {"layer": 61, "field": "conv", "exact": false, "max_abs": 0.28125, "nrmse": 0.02510450780391693}, {"layer": 61, "field": "recurrent", "exact": false, "max_abs": 0.021255552768707275, "nrmse": 0.028996989130973816}, {"layer": 62, "field": "conv", "exact": false, "max_abs": 0.5, "nrmse": 0.023689601570367813}, {"layer": 62, "field": "recurrent", "exact": false, "max_abs": 0.048400163650512695, "nrmse": 0.022916734218597412}, {"layer": 63, "field": "key", "exact": false, "max_abs": 0.41796875, "nrmse": 0.016322899609804153}, {"layer": 63, "field": "value", "exact": false, "max_abs": 1.375, "nrmse": 0.009922431781888008}], "buffered_graph_state_diff": []} -{"phase": "step", "index": 83, "position": 231, "teacher_token": 16, "compact": {"ids": [22, 7397, 23, 491, 20], "logits": [28.25, 17.0, 16.75, 15.75, 15.25], "margin": 11.25, "argmax": 22}, "buffered": {"ids": [22, 23, 7397, 491, 20], "logits": [28.875, 17.125, 17.0, 15.8125, 15.6875], "margin": 11.75, "argmax": 22}, "graph": {"ids": [22, 23, 7397, 491, 20], "logits": [28.875, 17.125, 17.0, 15.8125, 15.6875], "margin": 11.75, "argmax": 22}, "compact_buffered": {"exact": false, "max_abs": 1.8125, "nrmse": 0.025699317455291748}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "compact_buffered_state_diff": [{"layer": 20, "field": "conv", "exact": false, "max_abs": 0.0625, "nrmse": 0.004828171338886023}, {"layer": 20, "field": "recurrent", "exact": false, "max_abs": 0.0023653656244277954, "nrmse": 0.0018947913777083158}, {"layer": 21, "field": "conv", "exact": false, "max_abs": 0.068359375, "nrmse": 0.012136906385421753}, {"layer": 21, "field": "recurrent", "exact": false, "max_abs": 0.006969183683395386, "nrmse": 0.005295480601489544}, {"layer": 22, "field": "conv", "exact": false, "max_abs": 0.08984375, "nrmse": 0.016786858439445496}, {"layer": 22, "field": "recurrent", "exact": false, "max_abs": 0.0037270784378051758, "nrmse": 0.009076271206140518}, {"layer": 23, "field": "key", "exact": false, "max_abs": 0.125, "nrmse": 0.0024316292256116867}, {"layer": 23, "field": "value", "exact": false, "max_abs": 0.06640625, "nrmse": 0.002777372021228075}, {"layer": 24, "field": "conv", "exact": false, "max_abs": 0.09375, "nrmse": 0.021502302959561348}, {"layer": 24, "field": "recurrent", "exact": false, "max_abs": 0.005881115794181824, "nrmse": 0.008831913582980633}, {"layer": 25, "field": "conv", "exact": false, "max_abs": 0.125, "nrmse": 0.022448722273111343}, {"layer": 25, "field": "recurrent", "exact": false, "max_abs": 0.006246313452720642, "nrmse": 0.008475028909742832}, {"layer": 26, "field": "conv", "exact": false, "max_abs": 0.125, "nrmse": 0.025853846222162247}, {"layer": 26, "field": "recurrent", "exact": false, "max_abs": 0.0064539238810539246, "nrmse": 0.010265265591442585}, {"layer": 27, "field": "key", "exact": false, "max_abs": 0.171875, "nrmse": 0.003359576454386115}, {"layer": 27, "field": "value", "exact": false, "max_abs": 0.1171875, "nrmse": 0.0034596940968185663}, {"layer": 28, "field": "conv", "exact": false, "max_abs": 0.15625, "nrmse": 0.029566479846835136}, {"layer": 28, "field": "recurrent", "exact": false, "max_abs": 0.004385523498058319, "nrmse": 0.009416917338967323}, {"layer": 29, "field": "conv", "exact": false, "max_abs": 0.171875, "nrmse": 0.033240363001823425}, {"layer": 29, "field": "recurrent", "exact": false, "max_abs": 0.007290929555892944, "nrmse": 0.014794813469052315}, {"layer": 30, "field": "conv", "exact": false, "max_abs": 0.16015625, "nrmse": 0.03241482377052307}, {"layer": 30, "field": "recurrent", "exact": false, "max_abs": 0.0360184907913208, "nrmse": 0.017784660682082176}, {"layer": 31, "field": "key", "exact": false, "max_abs": 0.24658203125, "nrmse": 0.003811535658314824}, {"layer": 31, "field": "value", "exact": false, "max_abs": 0.1640625, "nrmse": 0.004860393702983856}, {"layer": 32, "field": "conv", "exact": false, "max_abs": 0.203125, "nrmse": 0.038909781724214554}, {"layer": 32, "field": "recurrent", "exact": false, "max_abs": 0.004269815981388092, "nrmse": 0.014585464261472225}, {"layer": 33, "field": "conv", "exact": false, "max_abs": 0.1875, "nrmse": 0.03801761940121651}, {"layer": 33, "field": "recurrent", "exact": false, "max_abs": 0.009836059994995594, "nrmse": 0.021994298323988914}, {"layer": 34, "field": "conv", "exact": false, "max_abs": 0.1875, "nrmse": 0.03558514639735222}, {"layer": 34, "field": "recurrent", "exact": false, "max_abs": 0.0103455213829875, "nrmse": 0.01227896474301815}, {"layer": 35, "field": "key", "exact": false, "max_abs": 0.21875, "nrmse": 0.005041928496211767}, {"layer": 35, "field": "value", "exact": false, "max_abs": 0.17578125, "nrmse": 0.006277663633227348}, {"layer": 36, "field": "conv", "exact": false, "max_abs": 0.1875, "nrmse": 0.028767796233296394}, {"layer": 36, "field": "recurrent", "exact": false, "max_abs": 0.007865794003009796, "nrmse": 0.009596860967576504}, {"layer": 37, "field": "conv", "exact": false, "max_abs": 0.25, "nrmse": 0.028467932716012}, {"layer": 37, "field": "recurrent", "exact": false, "max_abs": 0.01652812957763672, "nrmse": 0.012550730258226395}, {"layer": 38, "field": "conv", "exact": false, "max_abs": 0.25927734375, "nrmse": 0.03556821495294571}, {"layer": 38, "field": "recurrent", "exact": false, "max_abs": 0.010270051658153534, "nrmse": 0.024448435753583908}, {"layer": 39, "field": "key", "exact": false, "max_abs": 0.2578125, "nrmse": 0.009474205784499645}, {"layer": 39, "field": "value", "exact": false, "max_abs": 0.21875, "nrmse": 0.011906578205525875}, {"layer": 40, "field": "conv", "exact": false, "max_abs": 0.28125, "nrmse": 0.03440570458769798}, {"layer": 40, "field": "recurrent", "exact": false, "max_abs": 0.010283246636390686, "nrmse": 0.010886806063354015}, {"layer": 41, "field": "conv", "exact": false, "max_abs": 0.1953125, "nrmse": 0.03487512841820717}, {"layer": 41, "field": "recurrent", "exact": false, "max_abs": 0.009192235767841339, "nrmse": 0.020617634057998657}, {"layer": 42, "field": "conv", "exact": false, "max_abs": 0.2421875, "nrmse": 0.038044799119234085}, {"layer": 42, "field": "recurrent", "exact": false, "max_abs": 0.00940394401550293, "nrmse": 0.020748116075992584}, {"layer": 43, "field": "key", "exact": false, "max_abs": 0.3515625, "nrmse": 0.012651058845221996}, {"layer": 43, "field": "value", "exact": false, "max_abs": 0.328125, "nrmse": 0.014426078647375107}, {"layer": 44, "field": "conv", "exact": false, "max_abs": 0.21875, "nrmse": 0.03561961278319359}, {"layer": 44, "field": "recurrent", "exact": false, "max_abs": 0.012801244854927063, "nrmse": 0.024417968466877937}, {"layer": 45, "field": "conv", "exact": false, "max_abs": 0.21875, "nrmse": 0.03953752666711807}, {"layer": 45, "field": "recurrent", "exact": false, "max_abs": 0.013220429420471191, "nrmse": 0.028196517378091812}, {"layer": 46, "field": "conv", "exact": false, "max_abs": 0.28125, "nrmse": 0.038608502596616745}, {"layer": 46, "field": "recurrent", "exact": false, "max_abs": 0.014050781726837158, "nrmse": 0.026946475729346275}, {"layer": 47, "field": "key", "exact": false, "max_abs": 0.55078125, "nrmse": 0.014558441005647182}, {"layer": 47, "field": "value", "exact": false, "max_abs": 0.484375, "nrmse": 0.018928632140159607}, {"layer": 48, "field": "conv", "exact": false, "max_abs": 0.341796875, "nrmse": 0.03955777734518051}, {"layer": 48, "field": "recurrent", "exact": false, "max_abs": 0.007745847105979919, "nrmse": 0.027943037450313568}, {"layer": 49, "field": "conv", "exact": false, "max_abs": 0.59375, "nrmse": 0.04031681269407272}, {"layer": 49, "field": "recurrent", "exact": false, "max_abs": 0.01718658208847046, "nrmse": 0.033370330929756165}, {"layer": 50, "field": "conv", "exact": false, "max_abs": 1.015625, "nrmse": 0.04664059728384018}, {"layer": 50, "field": "recurrent", "exact": false, "max_abs": 0.009168028831481934, "nrmse": 0.03219473361968994}, {"layer": 51, "field": "key", "exact": false, "max_abs": 0.46875, "nrmse": 0.015400717034935951}, {"layer": 51, "field": "value", "exact": false, "max_abs": 0.8125, "nrmse": 0.020344655960798264}, {"layer": 52, "field": "conv", "exact": false, "max_abs": 1.15625, "nrmse": 0.046602435410022736}, {"layer": 52, "field": "recurrent", "exact": false, "max_abs": 0.04788863658905029, "nrmse": 0.02349192276597023}, {"layer": 53, "field": "conv", "exact": false, "max_abs": 0.7958984375, "nrmse": 0.04335857555270195}, {"layer": 53, "field": "recurrent", "exact": false, "max_abs": 0.05750882625579834, "nrmse": 0.03066587820649147}, {"layer": 54, "field": "conv", "exact": false, "max_abs": 1.130859375, "nrmse": 0.05018661916255951}, {"layer": 54, "field": "recurrent", "exact": false, "max_abs": 0.022365763783454895, "nrmse": 0.034966956824064255}, {"layer": 55, "field": "key", "exact": false, "max_abs": 0.44140625, "nrmse": 0.016022270545363426}, {"layer": 55, "field": "value", "exact": false, "max_abs": 0.8125, "nrmse": 0.022017773240804672}, {"layer": 56, "field": "conv", "exact": false, "max_abs": 1.28125, "nrmse": 0.042306262999773026}, {"layer": 56, "field": "recurrent", "exact": false, "max_abs": 0.0334932804107666, "nrmse": 0.028591878712177277}, {"layer": 57, "field": "conv", "exact": false, "max_abs": 0.869140625, "nrmse": 0.037126507610082626}, {"layer": 57, "field": "recurrent", "exact": false, "max_abs": 0.035423457622528076, "nrmse": 0.03046284429728985}, {"layer": 58, "field": "conv", "exact": false, "max_abs": 1.0, "nrmse": 0.04143190383911133}, {"layer": 58, "field": "recurrent", "exact": false, "max_abs": 0.035500138998031616, "nrmse": 0.02668646350502968}, {"layer": 59, "field": "key", "exact": false, "max_abs": 0.4453125, "nrmse": 0.015741819515824318}, {"layer": 59, "field": "value", "exact": false, "max_abs": 1.0625, "nrmse": 0.017636066302657127}, {"layer": 60, "field": "conv", "exact": false, "max_abs": 1.5, "nrmse": 0.03361251950263977}, {"layer": 60, "field": "recurrent", "exact": false, "max_abs": 0.04251524806022644, "nrmse": 0.03093636780977249}, {"layer": 61, "field": "conv", "exact": false, "max_abs": 1.0625, "nrmse": 0.03603186458349228}, {"layer": 61, "field": "recurrent", "exact": false, "max_abs": 0.032217174768447876, "nrmse": 0.03268538415431976}, {"layer": 62, "field": "conv", "exact": false, "max_abs": 2.65625, "nrmse": 0.03192347288131714}, {"layer": 62, "field": "recurrent", "exact": false, "max_abs": 0.05210220813751221, "nrmse": 0.02341170608997345}, {"layer": 63, "field": "key", "exact": false, "max_abs": 0.41796875, "nrmse": 0.016800958663225174}, {"layer": 63, "field": "value", "exact": false, "max_abs": 1.921875, "nrmse": 0.010165371932089329}], "buffered_graph_state_diff": []} -{"phase": "step", "index": 84, "position": 232, "teacher_token": 22, "compact": {"ids": [7397, 491, 4945, 777, 2702], "logits": [25.25, 23.25, 21.75, 19.875, 16.5], "margin": 2.0, "argmax": 7397}, "buffered": {"ids": [7397, 491, 4945, 777, 2702], "logits": [25.25, 23.5, 21.75, 19.875, 16.625], "margin": 1.75, "argmax": 7397}, "graph": {"ids": [7397, 491, 4945, 777, 2702], "logits": [25.25, 23.5, 21.75, 19.875, 16.625], "margin": 1.75, "argmax": 7397}, "compact_buffered": {"exact": false, "max_abs": 0.328125, "nrmse": 0.01676274463534355}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "compact_buffered_state_diff": [{"layer": 20, "field": "conv", "exact": false, "max_abs": 0.0625, "nrmse": 0.0035729624796658754}, {"layer": 20, "field": "recurrent", "exact": false, "max_abs": 0.0021203160285949707, "nrmse": 0.0016393914120271802}, {"layer": 21, "field": "conv", "exact": false, "max_abs": 0.068359375, "nrmse": 0.011911001056432724}, {"layer": 21, "field": "recurrent", "exact": false, "max_abs": 0.0044536590576171875, "nrmse": 0.005342028569430113}, {"layer": 22, "field": "conv", "exact": false, "max_abs": 0.08984375, "nrmse": 0.017191404476761818}, {"layer": 22, "field": "recurrent", "exact": false, "max_abs": 0.004576012492179871, "nrmse": 0.009287990629673004}, {"layer": 23, "field": "key", "exact": false, "max_abs": 0.13671875, "nrmse": 0.0027654145378619432}, {"layer": 23, "field": "value", "exact": false, "max_abs": 0.06640625, "nrmse": 0.00318478187546134}, {"layer": 24, "field": "conv", "exact": false, "max_abs": 0.109375, "nrmse": 0.0224589966237545}, {"layer": 24, "field": "recurrent", "exact": false, "max_abs": 0.008382201194763184, "nrmse": 0.00910236593335867}, {"layer": 25, "field": "conv", "exact": false, "max_abs": 0.109375, "nrmse": 0.023204296827316284}, {"layer": 25, "field": "recurrent", "exact": false, "max_abs": 0.00623108446598053, "nrmse": 0.008605518378317356}, {"layer": 26, "field": "conv", "exact": false, "max_abs": 0.1875, "nrmse": 0.02677549049258232}, {"layer": 26, "field": "recurrent", "exact": false, "max_abs": 0.007726244628429413, "nrmse": 0.010374678298830986}, {"layer": 27, "field": "key", "exact": false, "max_abs": 0.171875, "nrmse": 0.00370964827015996}, {"layer": 27, "field": "value", "exact": false, "max_abs": 0.1171875, "nrmse": 0.003931064624339342}, {"layer": 28, "field": "conv", "exact": false, "max_abs": 0.15625, "nrmse": 0.03015468642115593}, {"layer": 28, "field": "recurrent", "exact": false, "max_abs": 0.0051985327154397964, "nrmse": 0.009547626599669456}, {"layer": 29, "field": "conv", "exact": false, "max_abs": 0.171875, "nrmse": 0.03391844034194946}, {"layer": 29, "field": "recurrent", "exact": false, "max_abs": 0.006817162036895752, "nrmse": 0.015592352487146854}, {"layer": 30, "field": "conv", "exact": false, "max_abs": 0.171875, "nrmse": 0.03276938572525978}, {"layer": 30, "field": "recurrent", "exact": false, "max_abs": 0.022394970059394836, "nrmse": 0.017594503238797188}, {"layer": 31, "field": "key", "exact": false, "max_abs": 0.24658203125, "nrmse": 0.004112464841455221}, {"layer": 31, "field": "value", "exact": false, "max_abs": 0.1640625, "nrmse": 0.005423448979854584}, {"layer": 32, "field": "conv", "exact": false, "max_abs": 0.203125, "nrmse": 0.03924243152141571}, {"layer": 32, "field": "recurrent", "exact": false, "max_abs": 0.0038718581199645996, "nrmse": 0.0159370806068182}, {"layer": 33, "field": "conv", "exact": false, "max_abs": 0.1875, "nrmse": 0.03943144157528877}, {"layer": 33, "field": "recurrent", "exact": false, "max_abs": 0.009870782494544983, "nrmse": 0.023792490363121033}, {"layer": 34, "field": "conv", "exact": false, "max_abs": 0.1875, "nrmse": 0.03729487955570221}, {"layer": 34, "field": "recurrent", "exact": false, "max_abs": 0.010282482951879501, "nrmse": 0.013149654492735863}, {"layer": 35, "field": "key", "exact": false, "max_abs": 0.21875, "nrmse": 0.0052647665143013}, {"layer": 35, "field": "value", "exact": false, "max_abs": 0.1875, "nrmse": 0.0068284752778708935}, {"layer": 36, "field": "conv", "exact": false, "max_abs": 0.21875, "nrmse": 0.02897842787206173}, {"layer": 36, "field": "recurrent", "exact": false, "max_abs": 0.007354363799095154, "nrmse": 0.010197180323302746}, {"layer": 37, "field": "conv", "exact": false, "max_abs": 0.25, "nrmse": 0.02929697185754776}, {"layer": 37, "field": "recurrent", "exact": false, "max_abs": 0.00919070839881897, "nrmse": 0.012690425850450993}, {"layer": 38, "field": "conv", "exact": false, "max_abs": 0.25927734375, "nrmse": 0.0371580645442009}, {"layer": 38, "field": "recurrent", "exact": false, "max_abs": 0.009114861488342285, "nrmse": 0.02468099072575569}, {"layer": 39, "field": "key", "exact": false, "max_abs": 0.2578125, "nrmse": 0.009593439288437366}, {"layer": 39, "field": "value", "exact": false, "max_abs": 0.21875, "nrmse": 0.012138398364186287}, {"layer": 40, "field": "conv", "exact": false, "max_abs": 0.28125, "nrmse": 0.036111101508140564}, {"layer": 40, "field": "recurrent", "exact": false, "max_abs": 0.010189607739448547, "nrmse": 0.011259892955422401}, {"layer": 41, "field": "conv", "exact": false, "max_abs": 0.2578125, "nrmse": 0.036555591970682144}, {"layer": 41, "field": "recurrent", "exact": false, "max_abs": 0.009056977927684784, "nrmse": 0.020592758432030678}, {"layer": 42, "field": "conv", "exact": false, "max_abs": 0.2421875, "nrmse": 0.038976483047008514}, {"layer": 42, "field": "recurrent", "exact": false, "max_abs": 0.010723467916250229, "nrmse": 0.02089167758822441}, {"layer": 43, "field": "key", "exact": false, "max_abs": 0.3515625, "nrmse": 0.0127256466075778}, {"layer": 43, "field": "value", "exact": false, "max_abs": 0.328125, "nrmse": 0.014567791484296322}, {"layer": 44, "field": "conv", "exact": false, "max_abs": 0.21875, "nrmse": 0.03666991740465164}, {"layer": 44, "field": "recurrent", "exact": false, "max_abs": 0.012784257531166077, "nrmse": 0.024701159447431564}, {"layer": 45, "field": "conv", "exact": false, "max_abs": 0.21875, "nrmse": 0.04127257689833641}, {"layer": 45, "field": "recurrent", "exact": false, "max_abs": 0.013216674327850342, "nrmse": 0.028629150241613388}, {"layer": 46, "field": "conv", "exact": false, "max_abs": 0.28125, "nrmse": 0.03985514119267464}, {"layer": 46, "field": "recurrent", "exact": false, "max_abs": 0.014065384864807129, "nrmse": 0.027653375640511513}, {"layer": 47, "field": "key", "exact": false, "max_abs": 0.55078125, "nrmse": 0.014626153744757175}, {"layer": 47, "field": "value", "exact": false, "max_abs": 0.484375, "nrmse": 0.01904667727649212}, {"layer": 48, "field": "conv", "exact": false, "max_abs": 0.341796875, "nrmse": 0.040359072387218475}, {"layer": 48, "field": "recurrent", "exact": false, "max_abs": 0.008910030126571655, "nrmse": 0.028571441769599915}, {"layer": 49, "field": "conv", "exact": false, "max_abs": 0.59375, "nrmse": 0.0413958877325058}, {"layer": 49, "field": "recurrent", "exact": false, "max_abs": 0.016206979751586914, "nrmse": 0.03396384045481682}, {"layer": 50, "field": "conv", "exact": false, "max_abs": 1.015625, "nrmse": 0.04763832315802574}, {"layer": 50, "field": "recurrent", "exact": false, "max_abs": 0.012767525389790535, "nrmse": 0.03327571600675583}, {"layer": 51, "field": "key", "exact": false, "max_abs": 0.46875, "nrmse": 0.015470507554709911}, {"layer": 51, "field": "value", "exact": false, "max_abs": 0.8125, "nrmse": 0.020469818264245987}, {"layer": 52, "field": "conv", "exact": false, "max_abs": 1.15625, "nrmse": 0.04688723385334015}, {"layer": 52, "field": "recurrent", "exact": false, "max_abs": 0.0432964563369751, "nrmse": 0.023710336536169052}, {"layer": 53, "field": "conv", "exact": false, "max_abs": 0.7958984375, "nrmse": 0.04365629702806473}, {"layer": 53, "field": "recurrent", "exact": false, "max_abs": 0.0574573278427124, "nrmse": 0.03109951876103878}, {"layer": 54, "field": "conv", "exact": false, "max_abs": 1.130859375, "nrmse": 0.05097239837050438}, {"layer": 54, "field": "recurrent", "exact": false, "max_abs": 0.022235363721847534, "nrmse": 0.035812485963106155}, {"layer": 55, "field": "key", "exact": false, "max_abs": 0.44140625, "nrmse": 0.016098445281386375}, {"layer": 55, "field": "value", "exact": false, "max_abs": 0.8125, "nrmse": 0.02211316116154194}, {"layer": 56, "field": "conv", "exact": false, "max_abs": 1.28125, "nrmse": 0.042477402836084366}, {"layer": 56, "field": "recurrent", "exact": false, "max_abs": 0.03472975268959999, "nrmse": 0.028249526396393776}, {"layer": 57, "field": "conv", "exact": false, "max_abs": 0.869140625, "nrmse": 0.03701985254883766}, {"layer": 57, "field": "recurrent", "exact": false, "max_abs": 0.03381681442260742, "nrmse": 0.028704773634672165}, {"layer": 58, "field": "conv", "exact": false, "max_abs": 1.0, "nrmse": 0.041574627161026}, {"layer": 58, "field": "recurrent", "exact": false, "max_abs": 0.04164993762969971, "nrmse": 0.025740860030055046}, {"layer": 59, "field": "key", "exact": false, "max_abs": 0.4453125, "nrmse": 0.01586073450744152}, {"layer": 59, "field": "value", "exact": false, "max_abs": 1.0625, "nrmse": 0.01774630695581436}, {"layer": 60, "field": "conv", "exact": false, "max_abs": 1.5, "nrmse": 0.03395026922225952}, {"layer": 60, "field": "recurrent", "exact": false, "max_abs": 0.0440211296081543, "nrmse": 0.03172361105680466}, {"layer": 61, "field": "conv", "exact": false, "max_abs": 1.0625, "nrmse": 0.03649675101041794}, {"layer": 61, "field": "recurrent", "exact": false, "max_abs": 0.021396830677986145, "nrmse": 0.031160444021224976}, {"layer": 62, "field": "conv", "exact": false, "max_abs": 2.65625, "nrmse": 0.032009173184633255}, {"layer": 62, "field": "recurrent", "exact": false, "max_abs": 0.05096014589071274, "nrmse": 0.024207789450883865}, {"layer": 63, "field": "key", "exact": false, "max_abs": 0.41796875, "nrmse": 0.016865558922290802}, {"layer": 63, "field": "value", "exact": false, "max_abs": 1.921875, "nrmse": 0.010199910029768944}], "buffered_graph_state_diff": []} -{"phase": "step", "index": 85, "position": 233, "teacher_token": 7397, "compact": {"ids": [303, 494, 13, 2702, 11], "logits": [28.375, 24.75, 24.375, 22.25, 19.5], "margin": 3.625, "argmax": 303}, "buffered": {"ids": [303, 494, 13, 2702, 11], "logits": [28.25, 24.75, 24.375, 22.375, 19.625], "margin": 3.5, "argmax": 303}, "graph": {"ids": [303, 494, 13, 2702, 11], "logits": [28.25, 24.75, 24.375, 22.375, 19.625], "margin": 3.5, "argmax": 303}, "compact_buffered": {"exact": false, "max_abs": 0.4375, "nrmse": 0.023288877680897713}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "compact_buffered_state_diff": [{"layer": 20, "field": "conv", "exact": false, "max_abs": 0.0625, "nrmse": 0.003600219264626503}, {"layer": 20, "field": "recurrent", "exact": false, "max_abs": 0.001844063401222229, "nrmse": 0.001508052577264607}, {"layer": 21, "field": "conv", "exact": false, "max_abs": 0.068359375, "nrmse": 0.01209002174437046}, {"layer": 21, "field": "recurrent", "exact": false, "max_abs": 0.00782853364944458, "nrmse": 0.005425538867712021}, {"layer": 22, "field": "conv", "exact": false, "max_abs": 0.09375, "nrmse": 0.01756216399371624}, {"layer": 22, "field": "recurrent", "exact": false, "max_abs": 0.003441333770751953, "nrmse": 0.009873286820948124}, {"layer": 23, "field": "key", "exact": false, "max_abs": 0.13671875, "nrmse": 0.0030007350724190474}, {"layer": 23, "field": "value", "exact": false, "max_abs": 0.06640625, "nrmse": 0.003502965671941638}, {"layer": 24, "field": "conv", "exact": false, "max_abs": 0.109375, "nrmse": 0.022832166403532028}, {"layer": 24, "field": "recurrent", "exact": false, "max_abs": 0.004392087459564209, "nrmse": 0.009770686738193035}, {"layer": 25, "field": "conv", "exact": false, "max_abs": 0.109375, "nrmse": 0.023425789549946785}, {"layer": 25, "field": "recurrent", "exact": false, "max_abs": 0.006727978587150574, "nrmse": 0.010015498846769333}, {"layer": 26, "field": "conv", "exact": false, "max_abs": 0.1875, "nrmse": 0.02675560489296913}, {"layer": 26, "field": "recurrent", "exact": false, "max_abs": 0.006014987826347351, "nrmse": 0.012095245532691479}, {"layer": 27, "field": "key", "exact": false, "max_abs": 0.171875, "nrmse": 0.004024130757898092}, {"layer": 27, "field": "value", "exact": false, "max_abs": 0.1171875, "nrmse": 0.004381646402180195}, {"layer": 28, "field": "conv", "exact": false, "max_abs": 0.15625, "nrmse": 0.02986966446042061}, {"layer": 28, "field": "recurrent", "exact": false, "max_abs": 0.0043562427163124084, "nrmse": 0.011526776477694511}, {"layer": 29, "field": "conv", "exact": false, "max_abs": 0.171875, "nrmse": 0.03460207208991051}, {"layer": 29, "field": "recurrent", "exact": false, "max_abs": 0.008324667811393738, "nrmse": 0.017699260264635086}, {"layer": 30, "field": "conv", "exact": false, "max_abs": 0.171875, "nrmse": 0.033820830285549164}, {"layer": 30, "field": "recurrent", "exact": false, "max_abs": 0.028577864170074463, "nrmse": 0.018641404807567596}, {"layer": 31, "field": "key", "exact": false, "max_abs": 0.24658203125, "nrmse": 0.004503621254116297}, {"layer": 31, "field": "value", "exact": false, "max_abs": 0.1640625, "nrmse": 0.0059264954179525375}, {"layer": 32, "field": "conv", "exact": false, "max_abs": 0.203125, "nrmse": 0.03955518454313278}, {"layer": 32, "field": "recurrent", "exact": false, "max_abs": 0.0052518174052238464, "nrmse": 0.017486482858657837}, {"layer": 33, "field": "conv", "exact": false, "max_abs": 0.1875, "nrmse": 0.039923492819070816}, {"layer": 33, "field": "recurrent", "exact": false, "max_abs": 0.01041758805513382, "nrmse": 0.025449590757489204}, {"layer": 34, "field": "conv", "exact": false, "max_abs": 0.1875, "nrmse": 0.03751023858785629}, {"layer": 34, "field": "recurrent", "exact": false, "max_abs": 0.007211722433567047, "nrmse": 0.013573616743087769}, {"layer": 35, "field": "key", "exact": false, "max_abs": 0.21875, "nrmse": 0.005523355212062597}, {"layer": 35, "field": "value", "exact": false, "max_abs": 0.1875, "nrmse": 0.00724406260997057}, {"layer": 36, "field": "conv", "exact": false, "max_abs": 0.21875, "nrmse": 0.02996828965842724}, {"layer": 36, "field": "recurrent", "exact": false, "max_abs": 0.008052125573158264, "nrmse": 0.010757464915513992}, {"layer": 37, "field": "conv", "exact": false, "max_abs": 0.25, "nrmse": 0.0305648073554039}, {"layer": 37, "field": "recurrent", "exact": false, "max_abs": 0.012442082166671753, "nrmse": 0.01306900940835476}, {"layer": 38, "field": "conv", "exact": false, "max_abs": 0.25927734375, "nrmse": 0.038306862115859985}, {"layer": 38, "field": "recurrent", "exact": false, "max_abs": 0.011987030506134033, "nrmse": 0.02516697719693184}, {"layer": 39, "field": "key", "exact": false, "max_abs": 0.2578125, "nrmse": 0.009728494100272655}, {"layer": 39, "field": "value", "exact": false, "max_abs": 0.21875, "nrmse": 0.01236749067902565}, {"layer": 40, "field": "conv", "exact": false, "max_abs": 0.28125, "nrmse": 0.0367300920188427}, {"layer": 40, "field": "recurrent", "exact": false, "max_abs": 0.011795580387115479, "nrmse": 0.011498866602778435}, {"layer": 41, "field": "conv", "exact": false, "max_abs": 0.2578125, "nrmse": 0.03707967698574066}, {"layer": 41, "field": "recurrent", "exact": false, "max_abs": 0.009672209620475769, "nrmse": 0.02128043957054615}, {"layer": 42, "field": "conv", "exact": false, "max_abs": 0.2421875, "nrmse": 0.03913186118006706}, {"layer": 42, "field": "recurrent", "exact": false, "max_abs": 0.00950312614440918, "nrmse": 0.022085703909397125}, {"layer": 43, "field": "key", "exact": false, "max_abs": 0.3515625, "nrmse": 0.012842548079788685}, {"layer": 43, "field": "value", "exact": false, "max_abs": 0.328125, "nrmse": 0.01469625998288393}, {"layer": 44, "field": "conv", "exact": false, "max_abs": 0.1953125, "nrmse": 0.0364316925406456}, {"layer": 44, "field": "recurrent", "exact": false, "max_abs": 0.01272815465927124, "nrmse": 0.024581851437687874}, {"layer": 45, "field": "conv", "exact": false, "max_abs": 0.21875, "nrmse": 0.04123729094862938}, {"layer": 45, "field": "recurrent", "exact": false, "max_abs": 0.013214707374572754, "nrmse": 0.02876928076148033}, {"layer": 46, "field": "conv", "exact": false, "max_abs": 0.28125, "nrmse": 0.03944601118564606}, {"layer": 46, "field": "recurrent", "exact": false, "max_abs": 0.01406949758529663, "nrmse": 0.027040991932153702}, {"layer": 47, "field": "key", "exact": false, "max_abs": 0.55078125, "nrmse": 0.014698992483317852}, {"layer": 47, "field": "value", "exact": false, "max_abs": 0.484375, "nrmse": 0.019153248518705368}, {"layer": 48, "field": "conv", "exact": false, "max_abs": 0.341796875, "nrmse": 0.03956294804811478}, {"layer": 48, "field": "recurrent", "exact": false, "max_abs": 0.009447891265153885, "nrmse": 0.02864728309214115}, {"layer": 49, "field": "conv", "exact": false, "max_abs": 0.59375, "nrmse": 0.04093000292778015}, {"layer": 49, "field": "recurrent", "exact": false, "max_abs": 0.016236424446105957, "nrmse": 0.03319917991757393}, {"layer": 50, "field": "conv", "exact": false, "max_abs": 1.015625, "nrmse": 0.0470975860953331}, {"layer": 50, "field": "recurrent", "exact": false, "max_abs": 0.00851963460445404, "nrmse": 0.03232644125819206}, {"layer": 51, "field": "key", "exact": false, "max_abs": 0.46875, "nrmse": 0.015520459972321987}, {"layer": 51, "field": "value", "exact": false, "max_abs": 0.8125, "nrmse": 0.020567674189805984}, {"layer": 52, "field": "conv", "exact": false, "max_abs": 1.15625, "nrmse": 0.04622533544898033}, {"layer": 52, "field": "recurrent", "exact": false, "max_abs": 0.04730355739593506, "nrmse": 0.023326290771365166}, {"layer": 53, "field": "conv", "exact": false, "max_abs": 0.7958984375, "nrmse": 0.04312386363744736}, {"layer": 53, "field": "recurrent", "exact": false, "max_abs": 0.05718350410461426, "nrmse": 0.029355566948652267}, {"layer": 54, "field": "conv", "exact": false, "max_abs": 1.130859375, "nrmse": 0.05025115981698036}, {"layer": 54, "field": "recurrent", "exact": false, "max_abs": 0.022014550864696503, "nrmse": 0.03422843664884567}, {"layer": 55, "field": "key", "exact": false, "max_abs": 0.44140625, "nrmse": 0.0161741990596056}, {"layer": 55, "field": "value", "exact": false, "max_abs": 0.8125, "nrmse": 0.022275350987911224}, {"layer": 56, "field": "conv", "exact": false, "max_abs": 1.28125, "nrmse": 0.04202611744403839}, {"layer": 56, "field": "recurrent", "exact": false, "max_abs": 0.034387387335300446, "nrmse": 0.027374716475605965}, {"layer": 57, "field": "conv", "exact": false, "max_abs": 0.869140625, "nrmse": 0.037214916199445724}, {"layer": 57, "field": "recurrent", "exact": false, "max_abs": 0.030245065689086914, "nrmse": 0.028061795979738235}, {"layer": 58, "field": "conv", "exact": false, "max_abs": 1.0, "nrmse": 0.041550517082214355}, {"layer": 58, "field": "recurrent", "exact": false, "max_abs": 0.0338040292263031, "nrmse": 0.02471235580742359}, {"layer": 59, "field": "key", "exact": false, "max_abs": 0.4453125, "nrmse": 0.015947267413139343}, {"layer": 59, "field": "value", "exact": false, "max_abs": 1.0625, "nrmse": 0.017856480553746223}, {"layer": 60, "field": "conv", "exact": false, "max_abs": 1.5, "nrmse": 0.033464446663856506}, {"layer": 60, "field": "recurrent", "exact": false, "max_abs": 0.04240855574607849, "nrmse": 0.03125910833477974}, {"layer": 61, "field": "conv", "exact": false, "max_abs": 1.0625, "nrmse": 0.03653471916913986}, {"layer": 61, "field": "recurrent", "exact": false, "max_abs": 0.025579482316970825, "nrmse": 0.030457384884357452}, {"layer": 62, "field": "conv", "exact": false, "max_abs": 2.65625, "nrmse": 0.03129589930176735}, {"layer": 62, "field": "recurrent", "exact": false, "max_abs": 0.04340773820877075, "nrmse": 0.023680906742811203}, {"layer": 63, "field": "key", "exact": false, "max_abs": 0.41796875, "nrmse": 0.016951194033026695}, {"layer": 63, "field": "value", "exact": false, "max_abs": 1.921875, "nrmse": 0.010237044654786587}], "buffered_graph_state_diff": []} -{"phase": "step", "index": 86, "position": 234, "teacher_token": 303, "compact": {"ids": [2702, 264, 5089, 279, 4873], "logits": [33.0, 18.625, 17.5, 17.125, 16.875], "margin": 14.375, "argmax": 2702}, "buffered": {"ids": [2702, 264, 5089, 279, 4873], "logits": [32.75, 18.375, 17.5, 17.25, 16.75], "margin": 14.375, "argmax": 2702}, "graph": {"ids": [2702, 264, 5089, 279, 4873], "logits": [32.75, 18.375, 17.5, 17.25, 16.75], "margin": 14.375, "argmax": 2702}, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.02149682492017746}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "compact_buffered_state_diff": [{"layer": 20, "field": "recurrent", "exact": false, "max_abs": 0.0015242695808410645, "nrmse": 0.001373930019326508}, {"layer": 21, "field": "conv", "exact": false, "max_abs": 0.078125, "nrmse": 0.01156286895275116}, {"layer": 21, "field": "recurrent", "exact": false, "max_abs": 0.006305530667304993, "nrmse": 0.005510301794856787}, {"layer": 22, "field": "conv", "exact": false, "max_abs": 0.09375, "nrmse": 0.01761569455265999}, {"layer": 22, "field": "recurrent", "exact": false, "max_abs": 0.00602935254573822, "nrmse": 0.01138062123209238}, {"layer": 23, "field": "key", "exact": false, "max_abs": 0.1484375, "nrmse": 0.003337266156449914}, {"layer": 23, "field": "value", "exact": false, "max_abs": 0.08203125, "nrmse": 0.0039157383143901825}, {"layer": 24, "field": "conv", "exact": false, "max_abs": 0.109375, "nrmse": 0.022747060284018517}, {"layer": 24, "field": "recurrent", "exact": false, "max_abs": 0.004088550806045532, "nrmse": 0.01001946534961462}, {"layer": 25, "field": "conv", "exact": false, "max_abs": 0.1171875, "nrmse": 0.024112632498145103}, {"layer": 25, "field": "recurrent", "exact": false, "max_abs": 0.00655306875705719, "nrmse": 0.010435322299599648}, {"layer": 26, "field": "conv", "exact": false, "max_abs": 0.1875, "nrmse": 0.027459248900413513}, {"layer": 26, "field": "recurrent", "exact": false, "max_abs": 0.006281889975070953, "nrmse": 0.01289709284901619}, {"layer": 27, "field": "key", "exact": false, "max_abs": 0.171875, "nrmse": 0.00439284136518836}, {"layer": 27, "field": "value", "exact": false, "max_abs": 0.1171875, "nrmse": 0.004749168176203966}, {"layer": 28, "field": "conv", "exact": false, "max_abs": 0.15625, "nrmse": 0.029777199029922485}, {"layer": 28, "field": "recurrent", "exact": false, "max_abs": 0.00508325919508934, "nrmse": 0.011956088244915009}, {"layer": 29, "field": "conv", "exact": false, "max_abs": 0.171875, "nrmse": 0.034098993986845016}, {"layer": 29, "field": "recurrent", "exact": false, "max_abs": 0.007902272045612335, "nrmse": 0.01791188307106495}, {"layer": 30, "field": "conv", "exact": false, "max_abs": 0.171875, "nrmse": 0.03348588943481445}, {"layer": 30, "field": "recurrent", "exact": false, "max_abs": 0.026151299476623535, "nrmse": 0.018206866458058357}, {"layer": 31, "field": "key", "exact": false, "max_abs": 0.24658203125, "nrmse": 0.004947990644723177}, {"layer": 31, "field": "value", "exact": false, "max_abs": 0.1640625, "nrmse": 0.006275942083448172}, {"layer": 32, "field": "conv", "exact": false, "max_abs": 0.203125, "nrmse": 0.03785116970539093}, {"layer": 32, "field": "recurrent", "exact": false, "max_abs": 0.005354676395654678, "nrmse": 0.017722725868225098}, {"layer": 33, "field": "conv", "exact": false, "max_abs": 0.1875, "nrmse": 0.039424072951078415}, {"layer": 33, "field": "recurrent", "exact": false, "max_abs": 0.014234945178031921, "nrmse": 0.026810297742486}, {"layer": 34, "field": "conv", "exact": false, "max_abs": 0.1796875, "nrmse": 0.03742591664195061}, {"layer": 34, "field": "recurrent", "exact": false, "max_abs": 0.007366873789578676, "nrmse": 0.014098483137786388}, {"layer": 35, "field": "key", "exact": false, "max_abs": 0.22314453125, "nrmse": 0.006134856957942247}, {"layer": 35, "field": "value", "exact": false, "max_abs": 0.1875, "nrmse": 0.007644311524927616}, {"layer": 36, "field": "conv", "exact": false, "max_abs": 0.28125, "nrmse": 0.029880035668611526}, {"layer": 36, "field": "recurrent", "exact": false, "max_abs": 0.007679671049118042, "nrmse": 0.01119399443268776}, {"layer": 37, "field": "conv", "exact": false, "max_abs": 0.2421875, "nrmse": 0.031037764623761177}, {"layer": 37, "field": "recurrent", "exact": false, "max_abs": 0.011388391256332397, "nrmse": 0.013467147015035152}, {"layer": 38, "field": "conv", "exact": false, "max_abs": 0.29296875, "nrmse": 0.03913681209087372}, {"layer": 38, "field": "recurrent", "exact": false, "max_abs": 0.016912832856178284, "nrmse": 0.027408717200160027}, {"layer": 39, "field": "key", "exact": false, "max_abs": 0.265625, "nrmse": 0.01015495229512453}, {"layer": 39, "field": "value", "exact": false, "max_abs": 0.21875, "nrmse": 0.012803452089428902}, {"layer": 40, "field": "conv", "exact": false, "max_abs": 0.2421875, "nrmse": 0.03657122701406479}, {"layer": 40, "field": "recurrent", "exact": false, "max_abs": 0.009821534156799316, "nrmse": 0.011740929447114468}, {"layer": 41, "field": "conv", "exact": false, "max_abs": 0.2578125, "nrmse": 0.03860819339752197}, {"layer": 41, "field": "recurrent", "exact": false, "max_abs": 0.009683337062597275, "nrmse": 0.021778099238872528}, {"layer": 42, "field": "conv", "exact": false, "max_abs": 0.2060546875, "nrmse": 0.04113761708140373}, {"layer": 42, "field": "recurrent", "exact": false, "max_abs": 0.012621060013771057, "nrmse": 0.02289583347737789}, {"layer": 43, "field": "key", "exact": false, "max_abs": 0.38671875, "nrmse": 0.013182048685848713}, {"layer": 43, "field": "value", "exact": false, "max_abs": 0.328125, "nrmse": 0.01497377548366785}, {"layer": 44, "field": "conv", "exact": false, "max_abs": 0.203125, "nrmse": 0.03877111151814461}, {"layer": 44, "field": "recurrent", "exact": false, "max_abs": 0.012659363448619843, "nrmse": 0.024949075654149055}, {"layer": 45, "field": "conv", "exact": false, "max_abs": 0.21875, "nrmse": 0.043938878923654556}, {"layer": 45, "field": "recurrent", "exact": false, "max_abs": 0.013212621212005615, "nrmse": 0.029442232102155685}, {"layer": 46, "field": "conv", "exact": false, "max_abs": 0.28125, "nrmse": 0.04138675332069397}, {"layer": 46, "field": "recurrent", "exact": false, "max_abs": 0.013982057571411133, "nrmse": 0.027753721922636032}, {"layer": 47, "field": "key", "exact": false, "max_abs": 0.55078125, "nrmse": 0.014939481392502785}, {"layer": 47, "field": "value", "exact": false, "max_abs": 0.484375, "nrmse": 0.01937820017337799}, {"layer": 48, "field": "conv", "exact": false, "max_abs": 0.341796875, "nrmse": 0.04083571955561638}, {"layer": 48, "field": "recurrent", "exact": false, "max_abs": 0.010532602667808533, "nrmse": 0.02861199341714382}, {"layer": 49, "field": "conv", "exact": false, "max_abs": 0.59375, "nrmse": 0.042365092784166336}, {"layer": 49, "field": "recurrent", "exact": false, "max_abs": 0.016033709049224854, "nrmse": 0.03339935466647148}, {"layer": 50, "field": "conv", "exact": false, "max_abs": 1.015625, "nrmse": 0.04778861999511719}, {"layer": 50, "field": "recurrent", "exact": false, "max_abs": 0.008475586771965027, "nrmse": 0.031825266778469086}, {"layer": 51, "field": "key", "exact": false, "max_abs": 0.46875, "nrmse": 0.015680646523833275}, {"layer": 51, "field": "value", "exact": false, "max_abs": 0.8125, "nrmse": 0.020704077556729317}, {"layer": 52, "field": "conv", "exact": false, "max_abs": 1.15625, "nrmse": 0.045454464852809906}, {"layer": 52, "field": "recurrent", "exact": false, "max_abs": 0.062081098556518555, "nrmse": 0.023266049101948738}, {"layer": 53, "field": "conv", "exact": false, "max_abs": 0.7958984375, "nrmse": 0.04264881834387779}, {"layer": 53, "field": "recurrent", "exact": false, "max_abs": 0.05696582794189453, "nrmse": 0.029486825689673424}, {"layer": 54, "field": "conv", "exact": false, "max_abs": 1.130859375, "nrmse": 0.049586839973926544}, {"layer": 54, "field": "recurrent", "exact": false, "max_abs": 0.023245573043823242, "nrmse": 0.033469319343566895}, {"layer": 55, "field": "key", "exact": false, "max_abs": 0.44140625, "nrmse": 0.01627594605088234}, {"layer": 55, "field": "value", "exact": false, "max_abs": 0.8125, "nrmse": 0.022387608885765076}, {"layer": 56, "field": "conv", "exact": false, "max_abs": 1.28125, "nrmse": 0.04172409325838089}, {"layer": 56, "field": "recurrent", "exact": false, "max_abs": 0.033346474170684814, "nrmse": 0.027436424046754837}, {"layer": 57, "field": "conv", "exact": false, "max_abs": 0.869140625, "nrmse": 0.03671593591570854}, {"layer": 57, "field": "recurrent", "exact": false, "max_abs": 0.028089940547943115, "nrmse": 0.028039976954460144}, {"layer": 58, "field": "conv", "exact": false, "max_abs": 1.0, "nrmse": 0.04109365493059158}, {"layer": 58, "field": "recurrent", "exact": false, "max_abs": 0.036000341176986694, "nrmse": 0.024773042649030685}, {"layer": 59, "field": "key", "exact": false, "max_abs": 0.4453125, "nrmse": 0.016062544658780098}, {"layer": 59, "field": "value", "exact": false, "max_abs": 1.0625, "nrmse": 0.017929956316947937}, {"layer": 60, "field": "conv", "exact": false, "max_abs": 1.5, "nrmse": 0.03334565833210945}, {"layer": 60, "field": "recurrent", "exact": false, "max_abs": 0.04234018921852112, "nrmse": 0.03099524788558483}, {"layer": 61, "field": "conv", "exact": false, "max_abs": 1.0625, "nrmse": 0.0369538776576519}, {"layer": 61, "field": "recurrent", "exact": false, "max_abs": 0.021864041686058044, "nrmse": 0.030411511659622192}, {"layer": 62, "field": "conv", "exact": false, "max_abs": 2.65625, "nrmse": 0.030480461195111275}, {"layer": 62, "field": "recurrent", "exact": false, "max_abs": 0.04527842998504639, "nrmse": 0.02328120917081833}, {"layer": 63, "field": "key", "exact": false, "max_abs": 0.41796875, "nrmse": 0.017037220299243927}, {"layer": 63, "field": "value", "exact": false, "max_abs": 1.921875, "nrmse": 0.010269511491060257}], "buffered_graph_state_diff": []} -{"phase": "step", "index": 87, "position": 235, "teacher_token": 2702, "compact": {"ids": [13, 494, 11, 318, 506], "logits": [29.5, 25.75, 20.5, 18.75, 18.5], "margin": 3.75, "argmax": 13}, "buffered": {"ids": [13, 494, 11, 318, 506], "logits": [29.5, 25.75, 20.75, 18.875, 18.5], "margin": 3.75, "argmax": 13}, "graph": {"ids": [13, 494, 11, 318, 506], "logits": [29.5, 25.75, 20.75, 18.875, 18.5], "margin": 3.75, "argmax": 13}, "compact_buffered": {"exact": false, "max_abs": 0.296875, "nrmse": 0.02108565904200077}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "compact_buffered_state_diff": [{"layer": 20, "field": "recurrent", "exact": false, "max_abs": 0.0013697892427444458, "nrmse": 0.0012806811137124896}, {"layer": 21, "field": "conv", "exact": false, "max_abs": 0.078125, "nrmse": 0.011003689840435982}, {"layer": 21, "field": "recurrent", "exact": false, "max_abs": 0.007452480494976044, "nrmse": 0.005498890299350023}, {"layer": 22, "field": "conv", "exact": false, "max_abs": 0.09375, "nrmse": 0.017153803259134293}, {"layer": 22, "field": "recurrent", "exact": false, "max_abs": 0.005676291882991791, "nrmse": 0.010842449963092804}, {"layer": 23, "field": "key", "exact": false, "max_abs": 0.1484375, "nrmse": 0.0035478619392961264}, {"layer": 23, "field": "value", "exact": false, "max_abs": 0.08203125, "nrmse": 0.004168539773672819}, {"layer": 24, "field": "conv", "exact": false, "max_abs": 0.109375, "nrmse": 0.022876055911183357}, {"layer": 24, "field": "recurrent", "exact": false, "max_abs": 0.003844618797302246, "nrmse": 0.009760730899870396}, {"layer": 25, "field": "conv", "exact": false, "max_abs": 0.1171875, "nrmse": 0.023884721100330353}, {"layer": 25, "field": "recurrent", "exact": false, "max_abs": 0.0066401660442352295, "nrmse": 0.010492726229131222}, {"layer": 26, "field": "conv", "exact": false, "max_abs": 0.1875, "nrmse": 0.026925351470708847}, {"layer": 26, "field": "recurrent", "exact": false, "max_abs": 0.00620260089635849, "nrmse": 0.012894284911453724}, {"layer": 27, "field": "key", "exact": false, "max_abs": 0.1875, "nrmse": 0.004682323429733515}, {"layer": 27, "field": "value", "exact": false, "max_abs": 0.1171875, "nrmse": 0.005084770731627941}, {"layer": 28, "field": "conv", "exact": false, "max_abs": 0.1422119140625, "nrmse": 0.029068369418382645}, {"layer": 28, "field": "recurrent", "exact": false, "max_abs": 0.004550270736217499, "nrmse": 0.011827959679067135}, {"layer": 29, "field": "conv", "exact": false, "max_abs": 0.171875, "nrmse": 0.03312124311923981}, {"layer": 29, "field": "recurrent", "exact": false, "max_abs": 0.0075668469071388245, "nrmse": 0.01789884828031063}, {"layer": 30, "field": "conv", "exact": false, "max_abs": 0.171875, "nrmse": 0.03271803259849548}, {"layer": 30, "field": "recurrent", "exact": false, "max_abs": 0.030745744705200195, "nrmse": 0.020439879968762398}, {"layer": 31, "field": "key", "exact": false, "max_abs": 0.24658203125, "nrmse": 0.00526169640943408}, {"layer": 31, "field": "value", "exact": false, "max_abs": 0.1640625, "nrmse": 0.00674910144880414}, {"layer": 32, "field": "conv", "exact": false, "max_abs": 0.20703125, "nrmse": 0.03790292516350746}, {"layer": 32, "field": "recurrent", "exact": false, "max_abs": 0.005475666373968124, "nrmse": 0.017672298476099968}, {"layer": 33, "field": "conv", "exact": false, "max_abs": 0.2109375, "nrmse": 0.04021500051021576}, {"layer": 33, "field": "recurrent", "exact": false, "max_abs": 0.010959863662719727, "nrmse": 0.02708534523844719}, {"layer": 34, "field": "conv", "exact": false, "max_abs": 0.1875, "nrmse": 0.03718746080994606}, {"layer": 34, "field": "recurrent", "exact": false, "max_abs": 0.005956731736660004, "nrmse": 0.013963471166789532}, {"layer": 35, "field": "key", "exact": false, "max_abs": 0.22314453125, "nrmse": 0.006434467621147633}, {"layer": 35, "field": "value", "exact": false, "max_abs": 0.20703125, "nrmse": 0.008058997802436352}, {"layer": 36, "field": "conv", "exact": false, "max_abs": 0.28125, "nrmse": 0.028740372508764267}, {"layer": 36, "field": "recurrent", "exact": false, "max_abs": 0.007101982831954956, "nrmse": 0.011204647831618786}, {"layer": 37, "field": "conv", "exact": false, "max_abs": 0.2421875, "nrmse": 0.030433155596256256}, {"layer": 37, "field": "recurrent", "exact": false, "max_abs": 0.008357301354408264, "nrmse": 0.013177407905459404}, {"layer": 38, "field": "conv", "exact": false, "max_abs": 0.29296875, "nrmse": 0.03797803819179535}, {"layer": 38, "field": "recurrent", "exact": false, "max_abs": 0.010737955570220947, "nrmse": 0.027988653630018234}, {"layer": 39, "field": "key", "exact": false, "max_abs": 0.265625, "nrmse": 0.010303687304258347}, {"layer": 39, "field": "value", "exact": false, "max_abs": 0.21875, "nrmse": 0.013049442321062088}, {"layer": 40, "field": "conv", "exact": false, "max_abs": 0.2421875, "nrmse": 0.03578805923461914}, {"layer": 40, "field": "recurrent", "exact": false, "max_abs": 0.01468154788017273, "nrmse": 0.011944648809731007}, {"layer": 41, "field": "conv", "exact": false, "max_abs": 0.2578125, "nrmse": 0.03718250244855881}, {"layer": 41, "field": "recurrent", "exact": false, "max_abs": 0.009669769555330276, "nrmse": 0.021779917180538177}, {"layer": 42, "field": "conv", "exact": false, "max_abs": 0.203125, "nrmse": 0.040139660239219666}, {"layer": 42, "field": "recurrent", "exact": false, "max_abs": 0.011820521205663681, "nrmse": 0.023217154666781425}, {"layer": 43, "field": "key", "exact": false, "max_abs": 0.38671875, "nrmse": 0.01329469121992588}, {"layer": 43, "field": "value", "exact": false, "max_abs": 0.328125, "nrmse": 0.01513990294188261}, {"layer": 44, "field": "conv", "exact": false, "max_abs": 0.296875, "nrmse": 0.038474325090646744}, {"layer": 44, "field": "recurrent", "exact": false, "max_abs": 0.01262122392654419, "nrmse": 0.025006799027323723}, {"layer": 45, "field": "conv", "exact": false, "max_abs": 0.21875, "nrmse": 0.042838964611291885}, {"layer": 45, "field": "recurrent", "exact": false, "max_abs": 0.013210773468017578, "nrmse": 0.030175713822245598}, {"layer": 46, "field": "conv", "exact": false, "max_abs": 0.2392578125, "nrmse": 0.03991687297821045}, {"layer": 46, "field": "recurrent", "exact": false, "max_abs": 0.013993501663208008, "nrmse": 0.027912892401218414}, {"layer": 47, "field": "key", "exact": false, "max_abs": 0.55078125, "nrmse": 0.01503305695950985}, {"layer": 47, "field": "value", "exact": false, "max_abs": 0.484375, "nrmse": 0.01955881528556347}, {"layer": 48, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.039149459451436996}, {"layer": 48, "field": "recurrent", "exact": false, "max_abs": 0.010209441184997559, "nrmse": 0.028991159051656723}, {"layer": 49, "field": "conv", "exact": false, "max_abs": 0.34375, "nrmse": 0.03808443993330002}, {"layer": 49, "field": "recurrent", "exact": false, "max_abs": 0.015089690685272217, "nrmse": 0.03416548669338226}, {"layer": 50, "field": "conv", "exact": false, "max_abs": 0.40625, "nrmse": 0.0372067429125309}, {"layer": 50, "field": "recurrent", "exact": false, "max_abs": 0.008151225745677948, "nrmse": 0.03208152949810028}, {"layer": 51, "field": "key", "exact": false, "max_abs": 0.46875, "nrmse": 0.01576399989426136}, {"layer": 51, "field": "value", "exact": false, "max_abs": 0.8125, "nrmse": 0.020840533077716827}, {"layer": 52, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.027970053255558014}, {"layer": 52, "field": "recurrent", "exact": false, "max_abs": 0.05119514465332031, "nrmse": 0.02310735359787941}, {"layer": 53, "field": "conv", "exact": false, "max_abs": 0.4375, "nrmse": 0.027988718822598457}, {"layer": 53, "field": "recurrent", "exact": false, "max_abs": 0.05659306049346924, "nrmse": 0.02930554375052452}, {"layer": 54, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.03151698037981987}, {"layer": 54, "field": "recurrent", "exact": false, "max_abs": 0.018913626670837402, "nrmse": 0.03341086581349373}, {"layer": 55, "field": "key", "exact": false, "max_abs": 0.44140625, "nrmse": 0.01632804423570633}, {"layer": 55, "field": "value", "exact": false, "max_abs": 0.8125, "nrmse": 0.02250448241829872}, {"layer": 56, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.02701355330646038}, {"layer": 56, "field": "recurrent", "exact": false, "max_abs": 0.03323543071746826, "nrmse": 0.027214333415031433}, {"layer": 57, "field": "conv", "exact": false, "max_abs": 0.5, "nrmse": 0.024609459564089775}, {"layer": 57, "field": "recurrent", "exact": false, "max_abs": 0.029024958610534668, "nrmse": 0.02769404835999012}, {"layer": 58, "field": "conv", "exact": false, "max_abs": 0.34375, "nrmse": 0.02830025926232338}, {"layer": 58, "field": "recurrent", "exact": false, "max_abs": 0.03547191619873047, "nrmse": 0.024375012144446373}, {"layer": 59, "field": "key", "exact": false, "max_abs": 0.4453125, "nrmse": 0.016119124367833138}, {"layer": 59, "field": "value", "exact": false, "max_abs": 1.0625, "nrmse": 0.018023021519184113}, {"layer": 60, "field": "conv", "exact": false, "max_abs": 0.625, "nrmse": 0.023583488538861275}, {"layer": 60, "field": "recurrent", "exact": false, "max_abs": 0.04212617874145508, "nrmse": 0.030825497582554817}, {"layer": 61, "field": "conv", "exact": false, "max_abs": 0.6875, "nrmse": 0.026386123150587082}, {"layer": 61, "field": "recurrent", "exact": false, "max_abs": 0.021949991583824158, "nrmse": 0.02970912866294384}, {"layer": 62, "field": "conv", "exact": false, "max_abs": 0.59375, "nrmse": 0.021645136177539825}, {"layer": 62, "field": "recurrent", "exact": false, "max_abs": 0.04731839895248413, "nrmse": 0.02307807095348835}, {"layer": 63, "field": "key", "exact": false, "max_abs": 0.41796875, "nrmse": 0.01715851202607155}, {"layer": 63, "field": "value", "exact": false, "max_abs": 1.921875, "nrmse": 0.010323631577193737}], "buffered_graph_state_diff": []} -{"phase": "step", "index": 88, "position": 236, "teacher_token": 13, "compact": {"ids": [198, 6558, 4081, 4188, 561], "logits": [28.75, 25.125, 17.375, 16.25, 16.125], "margin": 3.625, "argmax": 198}, "buffered": {"ids": [198, 6558, 4081, 4188, 561], "logits": [28.625, 24.875, 17.25, 16.125, 15.9375], "margin": 3.75, "argmax": 198}, "graph": {"ids": [198, 6558, 4081, 4188, 561], "logits": [28.625, 24.875, 17.25, 16.125, 15.9375], "margin": 3.75, "argmax": 198}, "compact_buffered": {"exact": false, "max_abs": 0.375, "nrmse": 0.023172112181782722}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "compact_buffered_state_diff": [{"layer": 20, "field": "recurrent", "exact": false, "max_abs": 0.0009790211915969849, "nrmse": 0.0011554087977856398}, {"layer": 21, "field": "conv", "exact": false, "max_abs": 0.078125, "nrmse": 0.01086685061454773}, {"layer": 21, "field": "recurrent", "exact": false, "max_abs": 0.00425150990486145, "nrmse": 0.005093869753181934}, {"layer": 22, "field": "conv", "exact": false, "max_abs": 0.09375, "nrmse": 0.017180221155285835}, {"layer": 22, "field": "recurrent", "exact": false, "max_abs": 0.004316270351409912, "nrmse": 0.011198362335562706}, {"layer": 23, "field": "key", "exact": false, "max_abs": 0.1484375, "nrmse": 0.0037655457854270935}, {"layer": 23, "field": "value", "exact": false, "max_abs": 0.08203125, "nrmse": 0.004424429032951593}, {"layer": 24, "field": "conv", "exact": false, "max_abs": 0.109375, "nrmse": 0.02276427485048771}, {"layer": 24, "field": "recurrent", "exact": false, "max_abs": 0.003923457115888596, "nrmse": 0.009629178792238235}, {"layer": 25, "field": "conv", "exact": false, "max_abs": 0.1171875, "nrmse": 0.024189740419387817}, {"layer": 25, "field": "recurrent", "exact": false, "max_abs": 0.006371945142745972, "nrmse": 0.01086539775133133}, {"layer": 26, "field": "conv", "exact": false, "max_abs": 0.125, "nrmse": 0.02689138613641262}, {"layer": 26, "field": "recurrent", "exact": false, "max_abs": 0.0055297017097473145, "nrmse": 0.013304700143635273}, {"layer": 27, "field": "key", "exact": false, "max_abs": 0.1875, "nrmse": 0.004963284824043512}, {"layer": 27, "field": "value", "exact": false, "max_abs": 0.1171875, "nrmse": 0.0054381126537919044}, {"layer": 28, "field": "conv", "exact": false, "max_abs": 0.1422119140625, "nrmse": 0.02931194007396698}, {"layer": 28, "field": "recurrent", "exact": false, "max_abs": 0.004443801939487457, "nrmse": 0.01252487301826477}, {"layer": 29, "field": "conv", "exact": false, "max_abs": 0.1650390625, "nrmse": 0.03400137647986412}, {"layer": 29, "field": "recurrent", "exact": false, "max_abs": 0.0074738264083862305, "nrmse": 0.018954433500766754}, {"layer": 30, "field": "conv", "exact": false, "max_abs": 0.19140625, "nrmse": 0.03388110548257828}, {"layer": 30, "field": "recurrent", "exact": false, "max_abs": 0.03341805934906006, "nrmse": 0.02086263708770275}, {"layer": 31, "field": "key", "exact": false, "max_abs": 0.24658203125, "nrmse": 0.005608717445284128}, {"layer": 31, "field": "value", "exact": false, "max_abs": 0.171875, "nrmse": 0.007305982522666454}, {"layer": 32, "field": "conv", "exact": false, "max_abs": 0.2109375, "nrmse": 0.03858990594744682}, {"layer": 32, "field": "recurrent", "exact": false, "max_abs": 0.0056362636387348175, "nrmse": 0.019162368029356003}, {"layer": 33, "field": "conv", "exact": false, "max_abs": 0.26171875, "nrmse": 0.04104176536202431}, {"layer": 33, "field": "recurrent", "exact": false, "max_abs": 0.01583249494433403, "nrmse": 0.030700797215104103}, {"layer": 34, "field": "conv", "exact": false, "max_abs": 0.1953125, "nrmse": 0.03884047642350197}, {"layer": 34, "field": "recurrent", "exact": false, "max_abs": 0.011498797684907913, "nrmse": 0.015621948055922985}, {"layer": 35, "field": "key", "exact": false, "max_abs": 0.22314453125, "nrmse": 0.00677519291639328}, {"layer": 35, "field": "value", "exact": false, "max_abs": 0.20703125, "nrmse": 0.008616791106760502}, {"layer": 36, "field": "conv", "exact": false, "max_abs": 0.28125, "nrmse": 0.02996704913675785}, {"layer": 36, "field": "recurrent", "exact": false, "max_abs": 0.006966382265090942, "nrmse": 0.01121034100651741}, {"layer": 37, "field": "conv", "exact": false, "max_abs": 0.2421875, "nrmse": 0.03166725113987923}, {"layer": 37, "field": "recurrent", "exact": false, "max_abs": 0.010158389806747437, "nrmse": 0.014054752886295319}, {"layer": 38, "field": "conv", "exact": false, "max_abs": 0.29296875, "nrmse": 0.03929736837744713}, {"layer": 38, "field": "recurrent", "exact": false, "max_abs": 0.023077674210071564, "nrmse": 0.029062381014227867}, {"layer": 39, "field": "key", "exact": false, "max_abs": 0.265625, "nrmse": 0.010607200674712658}, {"layer": 39, "field": "value", "exact": false, "max_abs": 0.21875, "nrmse": 0.013476349413394928}, {"layer": 40, "field": "conv", "exact": false, "max_abs": 0.359375, "nrmse": 0.03648975118994713}, {"layer": 40, "field": "recurrent", "exact": false, "max_abs": 0.012587308883666992, "nrmse": 0.01188252866268158}, {"layer": 41, "field": "conv", "exact": false, "max_abs": 0.267578125, "nrmse": 0.03975217789411545}, {"layer": 41, "field": "recurrent", "exact": false, "max_abs": 0.009533777832984924, "nrmse": 0.022484805434942245}, {"layer": 42, "field": "conv", "exact": false, "max_abs": 0.21875, "nrmse": 0.04285769164562225}, {"layer": 42, "field": "recurrent", "exact": false, "max_abs": 0.012727826833724976, "nrmse": 0.023629358038306236}, {"layer": 43, "field": "key", "exact": false, "max_abs": 0.38671875, "nrmse": 0.013519395142793655}, {"layer": 43, "field": "value", "exact": false, "max_abs": 0.328125, "nrmse": 0.015391530469059944}, {"layer": 44, "field": "conv", "exact": false, "max_abs": 0.296875, "nrmse": 0.040350209921598434}, {"layer": 44, "field": "recurrent", "exact": false, "max_abs": 0.01257895678281784, "nrmse": 0.025606267154216766}, {"layer": 45, "field": "conv", "exact": false, "max_abs": 0.21875, "nrmse": 0.04454336315393448}, {"layer": 45, "field": "recurrent", "exact": false, "max_abs": 0.013216078281402588, "nrmse": 0.030363354831933975}, {"layer": 46, "field": "conv", "exact": false, "max_abs": 0.3359375, "nrmse": 0.04136192798614502}, {"layer": 46, "field": "recurrent", "exact": false, "max_abs": 0.014336496591567993, "nrmse": 0.028715763241052628}, {"layer": 47, "field": "key", "exact": false, "max_abs": 0.55078125, "nrmse": 0.015199447982013226}, {"layer": 47, "field": "value", "exact": false, "max_abs": 0.484375, "nrmse": 0.01978256367146969}, {"layer": 48, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.040262170135974884}, {"layer": 48, "field": "recurrent", "exact": false, "max_abs": 0.010407835245132446, "nrmse": 0.029574446380138397}, {"layer": 49, "field": "conv", "exact": false, "max_abs": 0.34375, "nrmse": 0.03948594629764557}, {"layer": 49, "field": "recurrent", "exact": false, "max_abs": 0.014935910701751709, "nrmse": 0.035080309957265854}, {"layer": 50, "field": "conv", "exact": false, "max_abs": 0.40625, "nrmse": 0.037912577390670776}, {"layer": 50, "field": "recurrent", "exact": false, "max_abs": 0.007645927369594574, "nrmse": 0.032295241951942444}, {"layer": 51, "field": "key", "exact": false, "max_abs": 0.46875, "nrmse": 0.01588324084877968}, {"layer": 51, "field": "value", "exact": false, "max_abs": 0.8125, "nrmse": 0.021003082394599915}, {"layer": 52, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.02852625772356987}, {"layer": 52, "field": "recurrent", "exact": false, "max_abs": 0.04861569404602051, "nrmse": 0.02308681420981884}, {"layer": 53, "field": "conv", "exact": false, "max_abs": 0.4375, "nrmse": 0.0288534015417099}, {"layer": 53, "field": "recurrent", "exact": false, "max_abs": 0.056154608726501465, "nrmse": 0.028938906267285347}, {"layer": 54, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.03236343711614609}, {"layer": 54, "field": "recurrent", "exact": false, "max_abs": 0.01955685019493103, "nrmse": 0.03265469893813133}, {"layer": 55, "field": "key", "exact": false, "max_abs": 0.44140625, "nrmse": 0.016376597806811333}, {"layer": 55, "field": "value", "exact": false, "max_abs": 0.8125, "nrmse": 0.02263736166059971}, {"layer": 56, "field": "conv", "exact": false, "max_abs": 0.328125, "nrmse": 0.02785823680460453}, {"layer": 56, "field": "recurrent", "exact": false, "max_abs": 0.04102569818496704, "nrmse": 0.027284547686576843}, {"layer": 57, "field": "conv", "exact": false, "max_abs": 0.5, "nrmse": 0.02555205300450325}, {"layer": 57, "field": "recurrent", "exact": false, "max_abs": 0.02734827995300293, "nrmse": 0.028271060436964035}, {"layer": 58, "field": "conv", "exact": false, "max_abs": 0.34375, "nrmse": 0.028978385031223297}, {"layer": 58, "field": "recurrent", "exact": false, "max_abs": 0.035031259059906006, "nrmse": 0.025050194934010506}, {"layer": 59, "field": "key", "exact": false, "max_abs": 0.4453125, "nrmse": 0.016170887276530266}, {"layer": 59, "field": "value", "exact": false, "max_abs": 1.0625, "nrmse": 0.018111974000930786}, {"layer": 60, "field": "conv", "exact": false, "max_abs": 0.40625, "nrmse": 0.02385924942791462}, {"layer": 60, "field": "recurrent", "exact": false, "max_abs": 0.0433211624622345, "nrmse": 0.03116864152252674}, {"layer": 61, "field": "conv", "exact": false, "max_abs": 0.375, "nrmse": 0.026720436289906502}, {"layer": 61, "field": "recurrent", "exact": false, "max_abs": 0.021943897008895874, "nrmse": 0.029372941702604294}, {"layer": 62, "field": "conv", "exact": false, "max_abs": 0.5625, "nrmse": 0.02197849191725254}, {"layer": 62, "field": "recurrent", "exact": false, "max_abs": 0.04775911569595337, "nrmse": 0.022911526262760162}, {"layer": 63, "field": "key", "exact": false, "max_abs": 0.41796875, "nrmse": 0.01723427139222622}, {"layer": 63, "field": "value", "exact": false, "max_abs": 1.921875, "nrmse": 0.010367661714553833}], "buffered_graph_state_diff": []} -{"phase": "step", "index": 89, "position": 237, "teacher_token": 198, "compact": {"ids": [9, 9764, 262, 6558, 1112], "logits": [29.75, 22.75, 17.75, 17.25, 15.9375], "margin": 7.0, "argmax": 9}, "buffered": {"ids": [9, 9764, 262, 6558, 256], "logits": [29.5, 22.375, 17.5, 17.125, 15.8125], "margin": 7.125, "argmax": 9}, "graph": {"ids": [9, 9764, 262, 6558, 256], "logits": [29.5, 22.375, 17.5, 17.125, 15.8125], "margin": 7.125, "argmax": 9}, "compact_buffered": {"exact": false, "max_abs": 0.4072265625, "nrmse": 0.0236914474517107}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "compact_buffered_state_diff": [{"layer": 20, "field": "recurrent", "exact": false, "max_abs": 0.0009719431400299072, "nrmse": 0.0010824337368831038}, {"layer": 21, "field": "conv", "exact": false, "max_abs": 0.078125, "nrmse": 0.010449852794408798}, {"layer": 21, "field": "recurrent", "exact": false, "max_abs": 0.005234897136688232, "nrmse": 0.004768009763211012}, {"layer": 22, "field": "conv", "exact": false, "max_abs": 0.09375, "nrmse": 0.016988838091492653}, {"layer": 22, "field": "recurrent", "exact": false, "max_abs": 0.004380226135253906, "nrmse": 0.011355623602867126}, {"layer": 23, "field": "key", "exact": false, "max_abs": 0.1484375, "nrmse": 0.003973201382905245}, {"layer": 23, "field": "value", "exact": false, "max_abs": 0.08203125, "nrmse": 0.004560314584523439}, {"layer": 24, "field": "conv", "exact": false, "max_abs": 0.109375, "nrmse": 0.023012934252619743}, {"layer": 24, "field": "recurrent", "exact": false, "max_abs": 0.008120715618133545, "nrmse": 0.01019704807549715}, {"layer": 25, "field": "conv", "exact": false, "max_abs": 0.15625, "nrmse": 0.025259893387556076}, {"layer": 25, "field": "recurrent", "exact": false, "max_abs": 0.007278710603713989, "nrmse": 0.01215940061956644}, {"layer": 26, "field": "conv", "exact": false, "max_abs": 0.125, "nrmse": 0.028181685134768486}, {"layer": 26, "field": "recurrent", "exact": false, "max_abs": 0.0075583308935165405, "nrmse": 0.014981337822973728}, {"layer": 27, "field": "key", "exact": false, "max_abs": 0.189453125, "nrmse": 0.005335950292646885}, {"layer": 27, "field": "value", "exact": false, "max_abs": 0.1171875, "nrmse": 0.005718710832297802}, {"layer": 28, "field": "conv", "exact": false, "max_abs": 0.1422119140625, "nrmse": 0.030615372583270073}, {"layer": 28, "field": "recurrent", "exact": false, "max_abs": 0.004057921469211578, "nrmse": 0.013257906772196293}, {"layer": 29, "field": "conv", "exact": false, "max_abs": 0.1875, "nrmse": 0.035818085074424744}, {"layer": 29, "field": "recurrent", "exact": false, "max_abs": 0.0071109384298324585, "nrmse": 0.020765377208590508}, {"layer": 30, "field": "conv", "exact": false, "max_abs": 0.203125, "nrmse": 0.03623393923044205}, {"layer": 30, "field": "recurrent", "exact": false, "max_abs": 0.017594128847122192, "nrmse": 0.01996038295328617}, {"layer": 31, "field": "key", "exact": false, "max_abs": 0.24658203125, "nrmse": 0.0059913587756454945}, {"layer": 31, "field": "value", "exact": false, "max_abs": 0.171875, "nrmse": 0.007846853695809841}, {"layer": 32, "field": "conv", "exact": false, "max_abs": 0.2734375, "nrmse": 0.04122664034366608}, {"layer": 32, "field": "recurrent", "exact": false, "max_abs": 0.005331400781869888, "nrmse": 0.01997256651520729}, {"layer": 33, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.04405699670314789}, {"layer": 33, "field": "recurrent", "exact": false, "max_abs": 0.009352494031190872, "nrmse": 0.030461451038718224}, {"layer": 34, "field": "conv", "exact": false, "max_abs": 0.1953125, "nrmse": 0.04147213324904442}, {"layer": 34, "field": "recurrent", "exact": false, "max_abs": 0.014574997127056122, "nrmse": 0.01635158434510231}, {"layer": 35, "field": "key", "exact": false, "max_abs": 0.234375, "nrmse": 0.007121617905795574}, {"layer": 35, "field": "value", "exact": false, "max_abs": 0.20703125, "nrmse": 0.009109861217439175}, {"layer": 36, "field": "conv", "exact": false, "max_abs": 0.28125, "nrmse": 0.03140322118997574}, {"layer": 36, "field": "recurrent", "exact": false, "max_abs": 0.010960668325424194, "nrmse": 0.011725407093763351}, {"layer": 37, "field": "conv", "exact": false, "max_abs": 0.25, "nrmse": 0.03197653964161873}, {"layer": 37, "field": "recurrent", "exact": false, "max_abs": 0.010322093963623047, "nrmse": 0.014513526111841202}, {"layer": 38, "field": "conv", "exact": false, "max_abs": 0.29296875, "nrmse": 0.0400095134973526}, {"layer": 38, "field": "recurrent", "exact": false, "max_abs": 0.014269605278968811, "nrmse": 0.028808100149035454}, {"layer": 39, "field": "key", "exact": false, "max_abs": 0.265625, "nrmse": 0.010819346643984318}, {"layer": 39, "field": "value", "exact": false, "max_abs": 0.21875, "nrmse": 0.013750254176557064}, {"layer": 40, "field": "conv", "exact": false, "max_abs": 0.359375, "nrmse": 0.03721676394343376}, {"layer": 40, "field": "recurrent", "exact": false, "max_abs": 0.01525890827178955, "nrmse": 0.012826662510633469}, {"layer": 41, "field": "conv", "exact": false, "max_abs": 0.267578125, "nrmse": 0.04092167690396309}, {"layer": 41, "field": "recurrent", "exact": false, "max_abs": 0.00829504057765007, "nrmse": 0.02343156188726425}, {"layer": 42, "field": "conv", "exact": false, "max_abs": 0.232421875, "nrmse": 0.04388975352048874}, {"layer": 42, "field": "recurrent", "exact": false, "max_abs": 0.01870042085647583, "nrmse": 0.024513758718967438}, {"layer": 43, "field": "key", "exact": false, "max_abs": 0.38671875, "nrmse": 0.01367280539125204}, {"layer": 43, "field": "value", "exact": false, "max_abs": 0.328125, "nrmse": 0.015580255538225174}, {"layer": 44, "field": "conv", "exact": false, "max_abs": 0.296875, "nrmse": 0.04202495887875557}, {"layer": 44, "field": "recurrent", "exact": false, "max_abs": 0.01262509822845459, "nrmse": 0.025915171951055527}, {"layer": 45, "field": "conv", "exact": false, "max_abs": 0.3046875, "nrmse": 0.04677319899201393}, {"layer": 45, "field": "recurrent", "exact": false, "max_abs": 0.013215959072113037, "nrmse": 0.03125431388616562}, {"layer": 46, "field": "conv", "exact": false, "max_abs": 0.3359375, "nrmse": 0.04389236122369766}, {"layer": 46, "field": "recurrent", "exact": false, "max_abs": 0.026434332132339478, "nrmse": 0.03136783465743065}, {"layer": 47, "field": "key", "exact": false, "max_abs": 0.55078125, "nrmse": 0.015313350595533848}, {"layer": 47, "field": "value", "exact": false, "max_abs": 0.484375, "nrmse": 0.019978251308202744}, {"layer": 48, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.04254262149333954}, {"layer": 48, "field": "recurrent", "exact": false, "max_abs": 0.011076822876930237, "nrmse": 0.030031798407435417}, {"layer": 49, "field": "conv", "exact": false, "max_abs": 0.34375, "nrmse": 0.041810303926467896}, {"layer": 49, "field": "recurrent", "exact": false, "max_abs": 0.014805376529693604, "nrmse": 0.03531487658619881}, {"layer": 50, "field": "conv", "exact": false, "max_abs": 0.40625, "nrmse": 0.03993183746933937}, {"layer": 50, "field": "recurrent", "exact": false, "max_abs": 0.011534228920936584, "nrmse": 0.031885776668787}, {"layer": 51, "field": "key", "exact": false, "max_abs": 0.46875, "nrmse": 0.015975503250956535}, {"layer": 51, "field": "value", "exact": false, "max_abs": 0.8125, "nrmse": 0.021160202100872993}, {"layer": 52, "field": "conv", "exact": false, "max_abs": 0.34375, "nrmse": 0.030126076191663742}, {"layer": 52, "field": "recurrent", "exact": false, "max_abs": 0.0465083122253418, "nrmse": 0.022984379902482033}, {"layer": 53, "field": "conv", "exact": false, "max_abs": 0.4375, "nrmse": 0.030778711661696434}, {"layer": 53, "field": "recurrent", "exact": false, "max_abs": 0.056189894676208496, "nrmse": 0.028834886848926544}, {"layer": 54, "field": "conv", "exact": false, "max_abs": 0.34375, "nrmse": 0.03514175862073898}, {"layer": 54, "field": "recurrent", "exact": false, "max_abs": 0.018618226051330566, "nrmse": 0.03323045372962952}, {"layer": 55, "field": "key", "exact": false, "max_abs": 0.44140625, "nrmse": 0.016522135585546494}, {"layer": 55, "field": "value", "exact": false, "max_abs": 0.8125, "nrmse": 0.022838512435555458}, {"layer": 56, "field": "conv", "exact": false, "max_abs": 0.375, "nrmse": 0.030565710738301277}, {"layer": 56, "field": "recurrent", "exact": false, "max_abs": 0.034885138273239136, "nrmse": 0.02752758003771305}, {"layer": 57, "field": "conv", "exact": false, "max_abs": 0.515625, "nrmse": 0.027561921626329422}, {"layer": 57, "field": "recurrent", "exact": false, "max_abs": 0.030178755521774292, "nrmse": 0.029203549027442932}, {"layer": 58, "field": "conv", "exact": false, "max_abs": 0.375, "nrmse": 0.03150186687707901}, {"layer": 58, "field": "recurrent", "exact": false, "max_abs": 0.03437057137489319, "nrmse": 0.025968024507164955}, {"layer": 59, "field": "key", "exact": false, "max_abs": 0.4453125, "nrmse": 0.016285009682178497}, {"layer": 59, "field": "value", "exact": false, "max_abs": 1.0625, "nrmse": 0.01827773079276085}, {"layer": 60, "field": "conv", "exact": false, "max_abs": 1.0, "nrmse": 0.025640428066253662}, {"layer": 60, "field": "recurrent", "exact": false, "max_abs": 0.042452871799468994, "nrmse": 0.031405191868543625}, {"layer": 61, "field": "conv", "exact": false, "max_abs": 0.5625, "nrmse": 0.02869461476802826}, {"layer": 61, "field": "recurrent", "exact": false, "max_abs": 0.023042991757392883, "nrmse": 0.030052905902266502}, {"layer": 62, "field": "conv", "exact": false, "max_abs": 0.5625, "nrmse": 0.023921644315123558}, {"layer": 62, "field": "recurrent", "exact": false, "max_abs": 0.043686628341674805, "nrmse": 0.02309058979153633}, {"layer": 63, "field": "key", "exact": false, "max_abs": 0.41796875, "nrmse": 0.017336074262857437}, {"layer": 63, "field": "value", "exact": false, "max_abs": 1.921875, "nrmse": 0.010426831431686878}], "buffered_graph_state_diff": []} -{"phase": "step", "index": 90, "position": 238, "teacher_token": 9, "compact": {"ids": [256, 262, 6558, 9764, 220], "logits": [34.25, 21.0, 20.375, 19.125, 17.25], "margin": 13.25, "argmax": 256}, "buffered": {"ids": [256, 262, 6558, 9764, 220], "logits": [34.25, 21.25, 20.25, 19.125, 17.375], "margin": 13.0, "argmax": 256}, "graph": {"ids": [256, 262, 6558, 9764, 220], "logits": [34.25, 21.25, 20.25, 19.125, 17.375], "margin": 13.0, "argmax": 256}, "compact_buffered": {"exact": false, "max_abs": 0.4375, "nrmse": 0.01627241261303425}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "compact_buffered_state_diff": [{"layer": 20, "field": "recurrent", "exact": false, "max_abs": 0.0009318888187408447, "nrmse": 0.0010167035507038236}, {"layer": 21, "field": "conv", "exact": false, "max_abs": 0.0625, "nrmse": 0.009376155212521553}, {"layer": 21, "field": "recurrent", "exact": false, "max_abs": 0.003729104995727539, "nrmse": 0.003936191089451313}, {"layer": 22, "field": "conv", "exact": false, "max_abs": 0.0703125, "nrmse": 0.015360853634774685}, {"layer": 22, "field": "recurrent", "exact": false, "max_abs": 0.002281550318002701, "nrmse": 0.010894980281591415}, {"layer": 23, "field": "key", "exact": false, "max_abs": 0.1484375, "nrmse": 0.00413383636623621}, {"layer": 23, "field": "value", "exact": false, "max_abs": 0.08203125, "nrmse": 0.004664710722863674}, {"layer": 24, "field": "conv", "exact": false, "max_abs": 0.109375, "nrmse": 0.02165108174085617}, {"layer": 24, "field": "recurrent", "exact": false, "max_abs": 0.003954708576202393, "nrmse": 0.008822849951684475}, {"layer": 25, "field": "conv", "exact": false, "max_abs": 0.15625, "nrmse": 0.023182131350040436}, {"layer": 25, "field": "recurrent", "exact": false, "max_abs": 0.006319105625152588, "nrmse": 0.011540560983121395}, {"layer": 26, "field": "conv", "exact": false, "max_abs": 0.125, "nrmse": 0.02634495496749878}, {"layer": 26, "field": "recurrent", "exact": false, "max_abs": 0.004849731922149658, "nrmse": 0.01399180293083191}, {"layer": 27, "field": "key", "exact": false, "max_abs": 0.189453125, "nrmse": 0.005562490783631802}, {"layer": 27, "field": "value", "exact": false, "max_abs": 0.1171875, "nrmse": 0.005866733845323324}, {"layer": 28, "field": "conv", "exact": false, "max_abs": 0.1422119140625, "nrmse": 0.028940550982952118}, {"layer": 28, "field": "recurrent", "exact": false, "max_abs": 0.0034791529178619385, "nrmse": 0.013009782880544662}, {"layer": 29, "field": "conv", "exact": false, "max_abs": 0.1875, "nrmse": 0.03439881280064583}, {"layer": 29, "field": "recurrent", "exact": false, "max_abs": 0.006181463599205017, "nrmse": 0.020353147760033607}, {"layer": 30, "field": "conv", "exact": false, "max_abs": 0.203125, "nrmse": 0.034549031406641006}, {"layer": 30, "field": "recurrent", "exact": false, "max_abs": 0.02697688341140747, "nrmse": 0.02070668339729309}, {"layer": 31, "field": "key", "exact": false, "max_abs": 0.24658203125, "nrmse": 0.006239200010895729}, {"layer": 31, "field": "value", "exact": false, "max_abs": 0.171875, "nrmse": 0.00800968799740076}, {"layer": 32, "field": "conv", "exact": false, "max_abs": 0.2734375, "nrmse": 0.040184296667575836}, {"layer": 32, "field": "recurrent", "exact": false, "max_abs": 0.00535598024725914, "nrmse": 0.020640011876821518}, {"layer": 33, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.04235748574137688}, {"layer": 33, "field": "recurrent", "exact": false, "max_abs": 0.012235734611749649, "nrmse": 0.03141066059470177}, {"layer": 34, "field": "conv", "exact": false, "max_abs": 0.1953125, "nrmse": 0.0397370308637619}, {"layer": 34, "field": "recurrent", "exact": false, "max_abs": 0.017319008708000183, "nrmse": 0.016723524779081345}, {"layer": 35, "field": "key", "exact": false, "max_abs": 0.234375, "nrmse": 0.0074087330140173435}, {"layer": 35, "field": "value", "exact": false, "max_abs": 0.20703125, "nrmse": 0.00933515839278698}, {"layer": 36, "field": "conv", "exact": false, "max_abs": 0.21875, "nrmse": 0.030569106340408325}, {"layer": 36, "field": "recurrent", "exact": false, "max_abs": 0.010411277413368225, "nrmse": 0.01133232843130827}, {"layer": 37, "field": "conv", "exact": false, "max_abs": 0.25, "nrmse": 0.03049295023083687}, {"layer": 37, "field": "recurrent", "exact": false, "max_abs": 0.012468725442886353, "nrmse": 0.013532830402255058}, {"layer": 38, "field": "conv", "exact": false, "max_abs": 0.2109375, "nrmse": 0.03776044771075249}, {"layer": 38, "field": "recurrent", "exact": false, "max_abs": 0.012006387114524841, "nrmse": 0.026617851108312607}, {"layer": 39, "field": "key", "exact": false, "max_abs": 0.265625, "nrmse": 0.011060556396842003}, {"layer": 39, "field": "value", "exact": false, "max_abs": 0.21875, "nrmse": 0.013938008807599545}, {"layer": 40, "field": "conv", "exact": false, "max_abs": 0.359375, "nrmse": 0.035638872534036636}, {"layer": 40, "field": "recurrent", "exact": false, "max_abs": 0.015015840530395508, "nrmse": 0.011940422467887402}, {"layer": 41, "field": "conv", "exact": false, "max_abs": 0.267578125, "nrmse": 0.03919345885515213}, {"layer": 41, "field": "recurrent", "exact": false, "max_abs": 0.007729046046733856, "nrmse": 0.022707024589180946}, {"layer": 42, "field": "conv", "exact": false, "max_abs": 0.232421875, "nrmse": 0.041784822940826416}, {"layer": 42, "field": "recurrent", "exact": false, "max_abs": 0.011862039566040039, "nrmse": 0.023410478606820107}, {"layer": 43, "field": "key", "exact": false, "max_abs": 0.38671875, "nrmse": 0.01382841169834137}, {"layer": 43, "field": "value", "exact": false, "max_abs": 0.328125, "nrmse": 0.015724534168839455}, {"layer": 44, "field": "conv", "exact": false, "max_abs": 0.296875, "nrmse": 0.03918464481830597}, {"layer": 44, "field": "recurrent", "exact": false, "max_abs": 0.012700535356998444, "nrmse": 0.025192741304636}, {"layer": 45, "field": "conv", "exact": false, "max_abs": 0.3046875, "nrmse": 0.04333069920539856}, {"layer": 45, "field": "recurrent", "exact": false, "max_abs": 0.013217628002166748, "nrmse": 0.030240152031183243}, {"layer": 46, "field": "conv", "exact": false, "max_abs": 0.3359375, "nrmse": 0.0410567931830883}, {"layer": 46, "field": "recurrent", "exact": false, "max_abs": 0.013905346393585205, "nrmse": 0.029285302385687828}, {"layer": 47, "field": "key", "exact": false, "max_abs": 0.55078125, "nrmse": 0.015429852530360222}, {"layer": 47, "field": "value", "exact": false, "max_abs": 0.484375, "nrmse": 0.02009936422109604}, {"layer": 48, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.03989657759666443}, {"layer": 48, "field": "recurrent", "exact": false, "max_abs": 0.010895892977714539, "nrmse": 0.029880251735448837}, {"layer": 49, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.03906143829226494}, {"layer": 49, "field": "recurrent", "exact": false, "max_abs": 0.016080141067504883, "nrmse": 0.035181958228349686}, {"layer": 50, "field": "conv", "exact": false, "max_abs": 0.34375, "nrmse": 0.038297075778245926}, {"layer": 50, "field": "recurrent", "exact": false, "max_abs": 0.011348187923431396, "nrmse": 0.032123710960149765}, {"layer": 51, "field": "key", "exact": false, "max_abs": 0.46875, "nrmse": 0.0161085557192564}, {"layer": 51, "field": "value", "exact": false, "max_abs": 0.8125, "nrmse": 0.021277323365211487}, {"layer": 52, "field": "conv", "exact": false, "max_abs": 0.34375, "nrmse": 0.030137676745653152}, {"layer": 52, "field": "recurrent", "exact": false, "max_abs": 0.047072410583496094, "nrmse": 0.022677456960082054}, {"layer": 53, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.030583146959543228}, {"layer": 53, "field": "recurrent", "exact": false, "max_abs": 0.056291937828063965, "nrmse": 0.02838275209069252}, {"layer": 54, "field": "conv", "exact": false, "max_abs": 0.34375, "nrmse": 0.035043586045503616}, {"layer": 54, "field": "recurrent", "exact": false, "max_abs": 0.016119837760925293, "nrmse": 0.032431863248348236}, {"layer": 55, "field": "key", "exact": false, "max_abs": 0.44140625, "nrmse": 0.01662370003759861}, {"layer": 55, "field": "value", "exact": false, "max_abs": 0.8125, "nrmse": 0.022956620901823044}, {"layer": 56, "field": "conv", "exact": false, "max_abs": 0.375, "nrmse": 0.0305588711053133}, {"layer": 56, "field": "recurrent", "exact": false, "max_abs": 0.03918112814426422, "nrmse": 0.027179023250937462}, {"layer": 57, "field": "conv", "exact": false, "max_abs": 0.515625, "nrmse": 0.028033599257469177}, {"layer": 57, "field": "recurrent", "exact": false, "max_abs": 0.026693522930145264, "nrmse": 0.02818811498582363}, {"layer": 58, "field": "conv", "exact": false, "max_abs": 0.375, "nrmse": 0.031447701156139374}, {"layer": 58, "field": "recurrent", "exact": false, "max_abs": 0.03441783785820007, "nrmse": 0.024748634546995163}, {"layer": 59, "field": "key", "exact": false, "max_abs": 0.4453125, "nrmse": 0.016373619437217712}, {"layer": 59, "field": "value", "exact": false, "max_abs": 1.0625, "nrmse": 0.018394015729427338}, {"layer": 60, "field": "conv", "exact": false, "max_abs": 1.0, "nrmse": 0.025177575647830963}, {"layer": 60, "field": "recurrent", "exact": false, "max_abs": 0.04303485155105591, "nrmse": 0.03133764490485191}, {"layer": 61, "field": "conv", "exact": false, "max_abs": 0.5625, "nrmse": 0.02795250155031681}, {"layer": 61, "field": "recurrent", "exact": false, "max_abs": 0.023818373680114746, "nrmse": 0.029598988592624664}, {"layer": 62, "field": "conv", "exact": false, "max_abs": 0.5625, "nrmse": 0.024568237364292145}, {"layer": 62, "field": "recurrent", "exact": false, "max_abs": 0.04358428716659546, "nrmse": 0.022998815402388573}, {"layer": 63, "field": "key", "exact": false, "max_abs": 0.41796875, "nrmse": 0.017447376623749733}, {"layer": 63, "field": "value", "exact": false, "max_abs": 1.921875, "nrmse": 0.010478360578417778}], "buffered_graph_state_diff": []} -{"phase": "step", "index": 91, "position": 239, "teacher_token": 256, "compact": {"ids": [6558, 4188, 1216, 1042, 1165], "logits": [31.0, 18.0, 17.75, 17.75, 17.25], "margin": 13.0, "argmax": 6558}, "buffered": {"ids": [6558, 4188, 1216, 1042, 1165], "logits": [30.875, 18.0, 17.875, 17.625, 17.375], "margin": 12.875, "argmax": 6558}, "graph": {"ids": [6558, 4188, 1216, 1042, 1165], "logits": [30.875, 18.0, 17.875, 17.625, 17.375], "margin": 12.875, "argmax": 6558}, "compact_buffered": {"exact": false, "max_abs": 0.375, "nrmse": 0.023544441908597946}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "compact_buffered_state_diff": [{"layer": 20, "field": "recurrent", "exact": false, "max_abs": 0.0009404271841049194, "nrmse": 0.0009631375432945788}, {"layer": 21, "field": "conv", "exact": false, "max_abs": 0.0625, "nrmse": 0.008918523788452148}, {"layer": 21, "field": "recurrent", "exact": false, "max_abs": 0.0045548975467681885, "nrmse": 0.004418568219989538}, {"layer": 22, "field": "conv", "exact": false, "max_abs": 0.0703125, "nrmse": 0.015271983109414577}, {"layer": 22, "field": "recurrent", "exact": false, "max_abs": 0.002812981605529785, "nrmse": 0.010819952003657818}, {"layer": 23, "field": "key", "exact": false, "max_abs": 0.1484375, "nrmse": 0.0043031638488173485}, {"layer": 23, "field": "value", "exact": false, "max_abs": 0.08203125, "nrmse": 0.004826589487493038}, {"layer": 24, "field": "conv", "exact": false, "max_abs": 0.08349609375, "nrmse": 0.020962996408343315}, {"layer": 24, "field": "recurrent", "exact": false, "max_abs": 0.004088640213012695, "nrmse": 0.009097772650420666}, {"layer": 25, "field": "conv", "exact": false, "max_abs": 0.15625, "nrmse": 0.0233987458050251}, {"layer": 25, "field": "recurrent", "exact": false, "max_abs": 0.005614385008811951, "nrmse": 0.011221680790185928}, {"layer": 26, "field": "conv", "exact": false, "max_abs": 0.125, "nrmse": 0.02658335492014885}, {"layer": 26, "field": "recurrent", "exact": false, "max_abs": 0.005194757133722305, "nrmse": 0.013706764206290245}, {"layer": 27, "field": "key", "exact": false, "max_abs": 0.189453125, "nrmse": 0.005849097855389118}, {"layer": 27, "field": "value", "exact": false, "max_abs": 0.1171875, "nrmse": 0.00608666380867362}, {"layer": 28, "field": "conv", "exact": false, "max_abs": 0.15625, "nrmse": 0.029510200023651123}, {"layer": 28, "field": "recurrent", "exact": false, "max_abs": 0.003915322944521904, "nrmse": 0.012737205252051353}, {"layer": 29, "field": "conv", "exact": false, "max_abs": 0.1875, "nrmse": 0.035138897597789764}, {"layer": 29, "field": "recurrent", "exact": false, "max_abs": 0.005547394976019859, "nrmse": 0.019453490152955055}, {"layer": 30, "field": "conv", "exact": false, "max_abs": 0.203125, "nrmse": 0.03509123623371124}, {"layer": 30, "field": "recurrent", "exact": false, "max_abs": 0.039093535393476486, "nrmse": 0.022536151111125946}, {"layer": 31, "field": "key", "exact": false, "max_abs": 0.24658203125, "nrmse": 0.006587543524801731}, {"layer": 31, "field": "value", "exact": false, "max_abs": 0.171875, "nrmse": 0.00842373725026846}, {"layer": 32, "field": "conv", "exact": false, "max_abs": 0.2734375, "nrmse": 0.04022256284952164}, {"layer": 32, "field": "recurrent", "exact": false, "max_abs": 0.004945892840623856, "nrmse": 0.02075369842350483}, {"layer": 33, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.042600225657224655}, {"layer": 33, "field": "recurrent", "exact": false, "max_abs": 0.010267980396747589, "nrmse": 0.031225483864545822}, {"layer": 34, "field": "conv", "exact": false, "max_abs": 0.21044921875, "nrmse": 0.04019106552004814}, {"layer": 34, "field": "recurrent", "exact": false, "max_abs": 0.0117567777633667, "nrmse": 0.01640901155769825}, {"layer": 35, "field": "key", "exact": false, "max_abs": 0.265625, "nrmse": 0.007833665236830711}, {"layer": 35, "field": "value", "exact": false, "max_abs": 0.20703125, "nrmse": 0.009682579897344112}, {"layer": 36, "field": "conv", "exact": false, "max_abs": 0.21875, "nrmse": 0.03152667358517647}, {"layer": 36, "field": "recurrent", "exact": false, "max_abs": 0.009498834609985352, "nrmse": 0.011750347912311554}, {"layer": 37, "field": "conv", "exact": false, "max_abs": 0.25, "nrmse": 0.03078586608171463}, {"layer": 37, "field": "recurrent", "exact": false, "max_abs": 0.011726021766662598, "nrmse": 0.013797663152217865}, {"layer": 38, "field": "conv", "exact": false, "max_abs": 0.21875, "nrmse": 0.03837371990084648}, {"layer": 38, "field": "recurrent", "exact": false, "max_abs": 0.010944783687591553, "nrmse": 0.026737648993730545}, {"layer": 39, "field": "key", "exact": false, "max_abs": 0.265625, "nrmse": 0.011317204684019089}, {"layer": 39, "field": "value", "exact": false, "max_abs": 0.21875, "nrmse": 0.014169624075293541}, {"layer": 40, "field": "conv", "exact": false, "max_abs": 0.359375, "nrmse": 0.036339323967695236}, {"layer": 40, "field": "recurrent", "exact": false, "max_abs": 0.01442265510559082, "nrmse": 0.011830182746052742}, {"layer": 41, "field": "conv", "exact": false, "max_abs": 0.267578125, "nrmse": 0.04098129644989967}, {"layer": 41, "field": "recurrent", "exact": false, "max_abs": 0.00769449770450592, "nrmse": 0.022683974355459213}, {"layer": 42, "field": "conv", "exact": false, "max_abs": 0.234375, "nrmse": 0.043013621121644974}, {"layer": 42, "field": "recurrent", "exact": false, "max_abs": 0.013334304094314575, "nrmse": 0.02345244027674198}, {"layer": 43, "field": "key", "exact": false, "max_abs": 0.38671875, "nrmse": 0.014069224707782269}, {"layer": 43, "field": "value", "exact": false, "max_abs": 0.328125, "nrmse": 0.0159290824085474}, {"layer": 44, "field": "conv", "exact": false, "max_abs": 0.28125, "nrmse": 0.03959409147500992}, {"layer": 44, "field": "recurrent", "exact": false, "max_abs": 0.012782678008079529, "nrmse": 0.025233477354049683}, {"layer": 45, "field": "conv", "exact": false, "max_abs": 0.3046875, "nrmse": 0.04361055791378021}, {"layer": 45, "field": "recurrent", "exact": false, "max_abs": 0.013213872909545898, "nrmse": 0.029318971559405327}, {"layer": 46, "field": "conv", "exact": false, "max_abs": 0.3359375, "nrmse": 0.041317205876111984}, {"layer": 46, "field": "recurrent", "exact": false, "max_abs": 0.013881683349609375, "nrmse": 0.028566397726535797}, {"layer": 47, "field": "key", "exact": false, "max_abs": 0.55078125, "nrmse": 0.015572534874081612}, {"layer": 47, "field": "value", "exact": false, "max_abs": 0.484375, "nrmse": 0.020319703966379166}, {"layer": 48, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.04148576408624649}, {"layer": 48, "field": "recurrent", "exact": false, "max_abs": 0.009879589080810547, "nrmse": 0.029529711231589317}, {"layer": 49, "field": "conv", "exact": false, "max_abs": 0.375, "nrmse": 0.04070493206381798}, {"layer": 49, "field": "recurrent", "exact": false, "max_abs": 0.016285359859466553, "nrmse": 0.0356045700609684}, {"layer": 50, "field": "conv", "exact": false, "max_abs": 0.359375, "nrmse": 0.03903084620833397}, {"layer": 50, "field": "recurrent", "exact": false, "max_abs": 0.01550823450088501, "nrmse": 0.031813375651836395}, {"layer": 51, "field": "key", "exact": false, "max_abs": 0.46875, "nrmse": 0.016291853040456772}, {"layer": 51, "field": "value", "exact": false, "max_abs": 0.8125, "nrmse": 0.0214498583227396}, {"layer": 52, "field": "conv", "exact": false, "max_abs": 0.34375, "nrmse": 0.030321964994072914}, {"layer": 52, "field": "recurrent", "exact": false, "max_abs": 0.04572868347167969, "nrmse": 0.02269539050757885}, {"layer": 53, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.03048805706202984}, {"layer": 53, "field": "recurrent", "exact": false, "max_abs": 0.05583357810974121, "nrmse": 0.028121594339609146}, {"layer": 54, "field": "conv", "exact": false, "max_abs": 0.359375, "nrmse": 0.03526356443762779}, {"layer": 54, "field": "recurrent", "exact": false, "max_abs": 0.0221174955368042, "nrmse": 0.031644634902477264}, {"layer": 55, "field": "key", "exact": false, "max_abs": 0.44140625, "nrmse": 0.016704194247722626}, {"layer": 55, "field": "value", "exact": false, "max_abs": 0.8125, "nrmse": 0.02306501567363739}, {"layer": 56, "field": "conv", "exact": false, "max_abs": 0.375, "nrmse": 0.03061005286872387}, {"layer": 56, "field": "recurrent", "exact": false, "max_abs": 0.03780275583267212, "nrmse": 0.026840034872293472}, {"layer": 57, "field": "conv", "exact": false, "max_abs": 0.515625, "nrmse": 0.028697798028588295}, {"layer": 57, "field": "recurrent", "exact": false, "max_abs": 0.025987088680267334, "nrmse": 0.027869591489434242}, {"layer": 58, "field": "conv", "exact": false, "max_abs": 0.375, "nrmse": 0.031220849603414536}, {"layer": 58, "field": "recurrent", "exact": false, "max_abs": 0.03590741753578186, "nrmse": 0.02412356063723564}, {"layer": 59, "field": "key", "exact": false, "max_abs": 0.4453125, "nrmse": 0.016441144049167633}, {"layer": 59, "field": "value", "exact": false, "max_abs": 1.0625, "nrmse": 0.018476979807019234}, {"layer": 60, "field": "conv", "exact": false, "max_abs": 1.0, "nrmse": 0.025094540789723396}, {"layer": 60, "field": "recurrent", "exact": false, "max_abs": 0.0434134304523468, "nrmse": 0.031219353899359703}, {"layer": 61, "field": "conv", "exact": false, "max_abs": 0.5625, "nrmse": 0.028140602633357048}, {"layer": 61, "field": "recurrent", "exact": false, "max_abs": 0.023471876978874207, "nrmse": 0.029381755739450455}, {"layer": 62, "field": "conv", "exact": false, "max_abs": 0.46875, "nrmse": 0.025256913155317307}, {"layer": 62, "field": "recurrent", "exact": false, "max_abs": 0.03719449043273926, "nrmse": 0.02304891124367714}, {"layer": 63, "field": "key", "exact": false, "max_abs": 0.41796875, "nrmse": 0.017536349594593048}, {"layer": 63, "field": "value", "exact": false, "max_abs": 1.921875, "nrmse": 0.010513663291931152}], "buffered_graph_state_diff": []} -{"phase": "step", "index": 92, "position": 240, "teacher_token": 6558, "compact": {"ids": [393, 17237, 579, 279, 1510], "logits": [32.5, 22.25, 18.75, 16.75, 16.5], "margin": 10.25, "argmax": 393}, "buffered": {"ids": [393, 17237, 579, 279, 1510], "logits": [32.5, 22.5, 18.5, 16.875, 16.625], "margin": 10.0, "argmax": 393}, "graph": {"ids": [393, 17237, 579, 279, 1510], "logits": [32.5, 22.5, 18.5, 16.875, 16.625], "margin": 10.0, "argmax": 393}, "compact_buffered": {"exact": false, "max_abs": 0.50390625, "nrmse": 0.027645766735076904}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "compact_buffered_state_diff": [{"layer": 20, "field": "recurrent", "exact": false, "max_abs": 0.0009229034185409546, "nrmse": 0.0009062652825377882}, {"layer": 21, "field": "conv", "exact": false, "max_abs": 0.0625, "nrmse": 0.0086925458163023}, {"layer": 21, "field": "recurrent", "exact": false, "max_abs": 0.005073487758636475, "nrmse": 0.0049275909550487995}, {"layer": 22, "field": "conv", "exact": false, "max_abs": 0.125, "nrmse": 0.015267514623701572}, {"layer": 22, "field": "recurrent", "exact": false, "max_abs": 0.005419410765171051, "nrmse": 0.012005923315882683}, {"layer": 23, "field": "key", "exact": false, "max_abs": 0.1484375, "nrmse": 0.004497346933931112}, {"layer": 23, "field": "value", "exact": false, "max_abs": 0.08203125, "nrmse": 0.005075070075690746}, {"layer": 24, "field": "conv", "exact": false, "max_abs": 0.083984375, "nrmse": 0.02005208283662796}, {"layer": 24, "field": "recurrent", "exact": false, "max_abs": 0.006038248538970947, "nrmse": 0.009293070994317532}, {"layer": 25, "field": "conv", "exact": false, "max_abs": 0.15625, "nrmse": 0.022908806800842285}, {"layer": 25, "field": "recurrent", "exact": false, "max_abs": 0.005658760666847229, "nrmse": 0.01113422866910696}, {"layer": 26, "field": "conv", "exact": false, "max_abs": 0.09375, "nrmse": 0.026132168248295784}, {"layer": 26, "field": "recurrent", "exact": false, "max_abs": 0.006755799055099487, "nrmse": 0.01370162982493639}, {"layer": 27, "field": "key", "exact": false, "max_abs": 0.189453125, "nrmse": 0.006100678350776434}, {"layer": 27, "field": "value", "exact": false, "max_abs": 0.1171875, "nrmse": 0.006354071199893951}, {"layer": 28, "field": "conv", "exact": false, "max_abs": 0.15625, "nrmse": 0.027917513623833656}, {"layer": 28, "field": "recurrent", "exact": false, "max_abs": 0.004373684525489807, "nrmse": 0.01224362850189209}, {"layer": 29, "field": "conv", "exact": false, "max_abs": 0.1875, "nrmse": 0.032866738736629486}, {"layer": 29, "field": "recurrent", "exact": false, "max_abs": 0.004587696865200996, "nrmse": 0.019074801355600357}, {"layer": 30, "field": "conv", "exact": false, "max_abs": 0.203125, "nrmse": 0.03264521434903145}, {"layer": 30, "field": "recurrent", "exact": false, "max_abs": 0.02187393605709076, "nrmse": 0.02103903330862522}, {"layer": 31, "field": "key", "exact": false, "max_abs": 0.24658203125, "nrmse": 0.00679181469604373}, {"layer": 31, "field": "value", "exact": false, "max_abs": 0.171875, "nrmse": 0.008653005585074425}, {"layer": 32, "field": "conv", "exact": false, "max_abs": 0.2734375, "nrmse": 0.03743883967399597}, {"layer": 32, "field": "recurrent", "exact": false, "max_abs": 0.004923876374959946, "nrmse": 0.019843991845846176}, {"layer": 33, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.039577823132276535}, {"layer": 33, "field": "recurrent", "exact": false, "max_abs": 0.01262945681810379, "nrmse": 0.03081388585269451}, {"layer": 34, "field": "conv", "exact": false, "max_abs": 0.21044921875, "nrmse": 0.03676985204219818}, {"layer": 34, "field": "recurrent", "exact": false, "max_abs": 0.007430468685925007, "nrmse": 0.015777183696627617}, {"layer": 35, "field": "key", "exact": false, "max_abs": 0.265625, "nrmse": 0.008023780770599842}, {"layer": 35, "field": "value", "exact": false, "max_abs": 0.20703125, "nrmse": 0.009945372119545937}, {"layer": 36, "field": "conv", "exact": false, "max_abs": 0.21875, "nrmse": 0.029430313035845757}, {"layer": 36, "field": "recurrent", "exact": false, "max_abs": 0.007220517843961716, "nrmse": 0.011530404910445213}, {"layer": 37, "field": "conv", "exact": false, "max_abs": 0.25, "nrmse": 0.028642231598496437}, {"layer": 37, "field": "recurrent", "exact": false, "max_abs": 0.011868506669998169, "nrmse": 0.013886075466871262}, {"layer": 38, "field": "conv", "exact": false, "max_abs": 0.21875, "nrmse": 0.03541230037808418}, {"layer": 38, "field": "recurrent", "exact": false, "max_abs": 0.012672215700149536, "nrmse": 0.025630775839090347}, {"layer": 39, "field": "key", "exact": false, "max_abs": 0.265625, "nrmse": 0.011456863954663277}, {"layer": 39, "field": "value", "exact": false, "max_abs": 0.21875, "nrmse": 0.014376357197761536}, {"layer": 40, "field": "conv", "exact": false, "max_abs": 0.2421875, "nrmse": 0.03390858694911003}, {"layer": 40, "field": "recurrent", "exact": false, "max_abs": 0.01175236701965332, "nrmse": 0.011107896454632282}, {"layer": 41, "field": "conv", "exact": false, "max_abs": 0.25390625, "nrmse": 0.037465691566467285}, {"layer": 41, "field": "recurrent", "exact": false, "max_abs": 0.008116841316223145, "nrmse": 0.021797997877001762}, {"layer": 42, "field": "conv", "exact": false, "max_abs": 0.234375, "nrmse": 0.0392899326980114}, {"layer": 42, "field": "recurrent", "exact": false, "max_abs": 0.0131726935505867, "nrmse": 0.022255906835198402}, {"layer": 43, "field": "key", "exact": false, "max_abs": 0.38671875, "nrmse": 0.01417852658778429}, {"layer": 43, "field": "value", "exact": false, "max_abs": 0.328125, "nrmse": 0.016080740839242935}, {"layer": 44, "field": "conv", "exact": false, "max_abs": 0.21875, "nrmse": 0.03689545392990112}, {"layer": 44, "field": "recurrent", "exact": false, "max_abs": 0.012874886393547058, "nrmse": 0.024743832647800446}, {"layer": 45, "field": "conv", "exact": false, "max_abs": 0.3046875, "nrmse": 0.04040638357400894}, {"layer": 45, "field": "recurrent", "exact": false, "max_abs": 0.013209521770477295, "nrmse": 0.02867654524743557}, {"layer": 46, "field": "conv", "exact": false, "max_abs": 0.2705078125, "nrmse": 0.03849515691399574}, {"layer": 46, "field": "recurrent", "exact": false, "max_abs": 0.013762235641479492, "nrmse": 0.027997750788927078}, {"layer": 47, "field": "key", "exact": false, "max_abs": 0.55078125, "nrmse": 0.015632282942533493}, {"layer": 47, "field": "value", "exact": false, "max_abs": 0.484375, "nrmse": 0.02042529545724392}, {"layer": 48, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.03914061188697815}, {"layer": 48, "field": "recurrent", "exact": false, "max_abs": 0.007843062281608582, "nrmse": 0.02892855741083622}, {"layer": 49, "field": "conv", "exact": false, "max_abs": 0.375, "nrmse": 0.03799210116267204}, {"layer": 49, "field": "recurrent", "exact": false, "max_abs": 0.01488068699836731, "nrmse": 0.034301403909921646}, {"layer": 50, "field": "conv", "exact": false, "max_abs": 0.359375, "nrmse": 0.03716493025422096}, {"layer": 50, "field": "recurrent", "exact": false, "max_abs": 0.008227527141571045, "nrmse": 0.03059479221701622}, {"layer": 51, "field": "key", "exact": false, "max_abs": 0.46875, "nrmse": 0.016360312700271606}, {"layer": 51, "field": "value", "exact": false, "max_abs": 0.8125, "nrmse": 0.021558266133069992}, {"layer": 52, "field": "conv", "exact": false, "max_abs": 0.421875, "nrmse": 0.029615001752972603}, {"layer": 52, "field": "recurrent", "exact": false, "max_abs": 0.042650699615478516, "nrmse": 0.02289387211203575}, {"layer": 53, "field": "conv", "exact": false, "max_abs": 0.34375, "nrmse": 0.03044798970222473}, {"layer": 53, "field": "recurrent", "exact": false, "max_abs": 0.06077682971954346, "nrmse": 0.028718572109937668}, {"layer": 54, "field": "conv", "exact": false, "max_abs": 0.359375, "nrmse": 0.035454388707876205}, {"layer": 54, "field": "recurrent", "exact": false, "max_abs": 0.02258312702178955, "nrmse": 0.03299524635076523}, {"layer": 55, "field": "key", "exact": false, "max_abs": 0.44140625, "nrmse": 0.01675735041499138}, {"layer": 55, "field": "value", "exact": false, "max_abs": 0.8125, "nrmse": 0.02316250652074814}, {"layer": 56, "field": "conv", "exact": false, "max_abs": 0.375, "nrmse": 0.031740352511405945}, {"layer": 56, "field": "recurrent", "exact": false, "max_abs": 0.03534102439880371, "nrmse": 0.028176037594676018}, {"layer": 57, "field": "conv", "exact": false, "max_abs": 0.515625, "nrmse": 0.029447682201862335}, {"layer": 57, "field": "recurrent", "exact": false, "max_abs": 0.03201384097337723, "nrmse": 0.02918490581214428}, {"layer": 58, "field": "conv", "exact": false, "max_abs": 0.375, "nrmse": 0.03177341818809509}, {"layer": 58, "field": "recurrent", "exact": false, "max_abs": 0.03611406683921814, "nrmse": 0.025899838656187057}, {"layer": 59, "field": "key", "exact": false, "max_abs": 0.4453125, "nrmse": 0.016511380672454834}, {"layer": 59, "field": "value", "exact": false, "max_abs": 1.0625, "nrmse": 0.018576456233859062}, {"layer": 60, "field": "conv", "exact": false, "max_abs": 1.0, "nrmse": 0.02570980414748192}, {"layer": 60, "field": "recurrent", "exact": false, "max_abs": 0.04334539175033569, "nrmse": 0.03141523152589798}, {"layer": 61, "field": "conv", "exact": false, "max_abs": 0.5625, "nrmse": 0.029318004846572876}, {"layer": 61, "field": "recurrent", "exact": false, "max_abs": 0.023942530155181885, "nrmse": 0.030342668294906616}, {"layer": 62, "field": "conv", "exact": false, "max_abs": 0.40625, "nrmse": 0.026052454486489296}, {"layer": 62, "field": "recurrent", "exact": false, "max_abs": 0.03544926643371582, "nrmse": 0.023382902145385742}, {"layer": 63, "field": "key", "exact": false, "max_abs": 0.41796875, "nrmse": 0.01760447770357132}, {"layer": 63, "field": "value", "exact": false, "max_abs": 1.921875, "nrmse": 0.010564749129116535}], "buffered_graph_state_diff": []} -{"phase": "step", "index": 93, "position": 241, "teacher_token": 393, "compact": {"ids": [86, 54, 46, 87, 78], "logits": [27.625, 26.375, 25.625, 25.5, 23.5], "margin": 1.25, "argmax": 86}, "buffered": {"ids": [86, 54, 87, 46, 78], "logits": [27.625, 26.25, 25.75, 25.375, 23.5], "margin": 1.375, "argmax": 86}, "graph": {"ids": [86, 54, 87, 46, 78], "logits": [27.625, 26.25, 25.75, 25.375, 23.5], "margin": 1.375, "argmax": 86}, "compact_buffered": {"exact": false, "max_abs": 0.40625, "nrmse": 0.02504904568195343}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "compact_buffered_state_diff": [{"layer": 20, "field": "recurrent", "exact": false, "max_abs": 0.0009144693613052368, "nrmse": 0.0008565144380554557}, {"layer": 21, "field": "conv", "exact": false, "max_abs": 0.0625, "nrmse": 0.009083443321287632}, {"layer": 21, "field": "recurrent", "exact": false, "max_abs": 0.0071637630462646484, "nrmse": 0.005142590031027794}, {"layer": 22, "field": "conv", "exact": false, "max_abs": 0.125, "nrmse": 0.015564052388072014}, {"layer": 22, "field": "recurrent", "exact": false, "max_abs": 0.004815772175788879, "nrmse": 0.011721177957952023}, {"layer": 23, "field": "key", "exact": false, "max_abs": 0.1484375, "nrmse": 0.004681220278143883}, {"layer": 23, "field": "value", "exact": false, "max_abs": 0.08203125, "nrmse": 0.005333219654858112}, {"layer": 24, "field": "conv", "exact": false, "max_abs": 0.0859375, "nrmse": 0.02028747648000717}, {"layer": 24, "field": "recurrent", "exact": false, "max_abs": 0.0034818649291992188, "nrmse": 0.010352390818297863}, {"layer": 25, "field": "conv", "exact": false, "max_abs": 0.12890625, "nrmse": 0.022051285952329636}, {"layer": 25, "field": "recurrent", "exact": false, "max_abs": 0.004857152700424194, "nrmse": 0.01173641998320818}, {"layer": 26, "field": "conv", "exact": false, "max_abs": 0.095703125, "nrmse": 0.025061488151550293}, {"layer": 26, "field": "recurrent", "exact": false, "max_abs": 0.005029767751693726, "nrmse": 0.014422552660107613}, {"layer": 27, "field": "key", "exact": false, "max_abs": 0.189453125, "nrmse": 0.006312807090580463}, {"layer": 27, "field": "value", "exact": false, "max_abs": 0.1171875, "nrmse": 0.006580659653991461}, {"layer": 28, "field": "conv", "exact": false, "max_abs": 0.15625, "nrmse": 0.026583466678857803}, {"layer": 28, "field": "recurrent", "exact": false, "max_abs": 0.003313690423965454, "nrmse": 0.012737679295241833}, {"layer": 29, "field": "conv", "exact": false, "max_abs": 0.1640625, "nrmse": 0.030488383024930954}, {"layer": 29, "field": "recurrent", "exact": false, "max_abs": 0.004426001571118832, "nrmse": 0.02013993263244629}, {"layer": 30, "field": "conv", "exact": false, "max_abs": 0.203125, "nrmse": 0.02910028025507927}, {"layer": 30, "field": "recurrent", "exact": false, "max_abs": 0.02551180124282837, "nrmse": 0.019614769145846367}, {"layer": 31, "field": "key", "exact": false, "max_abs": 0.24658203125, "nrmse": 0.007042946759611368}, {"layer": 31, "field": "value", "exact": false, "max_abs": 0.171875, "nrmse": 0.008899128064513206}, {"layer": 32, "field": "conv", "exact": false, "max_abs": 0.1640625, "nrmse": 0.03420929238200188}, {"layer": 32, "field": "recurrent", "exact": false, "max_abs": 0.004978185519576073, "nrmse": 0.02060086280107498}, {"layer": 33, "field": "conv", "exact": false, "max_abs": 0.265625, "nrmse": 0.036350663751363754}, {"layer": 33, "field": "recurrent", "exact": false, "max_abs": 0.013914629817008972, "nrmse": 0.03150292485952377}, {"layer": 34, "field": "conv", "exact": false, "max_abs": 0.21044921875, "nrmse": 0.03408067673444748}, {"layer": 34, "field": "recurrent", "exact": false, "max_abs": 0.008749641478061676, "nrmse": 0.016957176849246025}, {"layer": 35, "field": "key", "exact": false, "max_abs": 0.265625, "nrmse": 0.008225083351135254}, {"layer": 35, "field": "value", "exact": false, "max_abs": 0.20703125, "nrmse": 0.010188573971390724}, {"layer": 36, "field": "conv", "exact": false, "max_abs": 0.21875, "nrmse": 0.0274372436106205}, {"layer": 36, "field": "recurrent", "exact": false, "max_abs": 0.007889620959758759, "nrmse": 0.01275542937219143}, {"layer": 37, "field": "conv", "exact": false, "max_abs": 0.1953125, "nrmse": 0.02723313495516777}, {"layer": 37, "field": "recurrent", "exact": false, "max_abs": 0.013637721538543701, "nrmse": 0.014680391177535057}, {"layer": 38, "field": "conv", "exact": false, "max_abs": 0.21875, "nrmse": 0.03404321148991585}, {"layer": 38, "field": "recurrent", "exact": false, "max_abs": 0.01085541769862175, "nrmse": 0.026160957291722298}, {"layer": 39, "field": "key", "exact": false, "max_abs": 0.265625, "nrmse": 0.011624877341091633}, {"layer": 39, "field": "value", "exact": false, "max_abs": 0.21875, "nrmse": 0.014581848867237568}, {"layer": 40, "field": "conv", "exact": false, "max_abs": 0.2421875, "nrmse": 0.03235956281423569}, {"layer": 40, "field": "recurrent", "exact": false, "max_abs": 0.012198925018310547, "nrmse": 0.011803206987679005}, {"layer": 41, "field": "conv", "exact": false, "max_abs": 0.25390625, "nrmse": 0.03627721592783928}, {"layer": 41, "field": "recurrent", "exact": false, "max_abs": 0.007649712264537811, "nrmse": 0.02304067835211754}, {"layer": 42, "field": "conv", "exact": false, "max_abs": 0.234375, "nrmse": 0.03754078969359398}, {"layer": 42, "field": "recurrent", "exact": false, "max_abs": 0.0096406489610672, "nrmse": 0.022812139242887497}, {"layer": 43, "field": "key", "exact": false, "max_abs": 0.38671875, "nrmse": 0.014310491271317005}, {"layer": 43, "field": "value", "exact": false, "max_abs": 0.328125, "nrmse": 0.016219258308410645}, {"layer": 44, "field": "conv", "exact": false, "max_abs": 0.203125, "nrmse": 0.03483001887798309}, {"layer": 44, "field": "recurrent", "exact": false, "max_abs": 0.012812919914722443, "nrmse": 0.025902602821588516}, {"layer": 45, "field": "conv", "exact": false, "max_abs": 0.25, "nrmse": 0.03774102404713631}, {"layer": 45, "field": "recurrent", "exact": false, "max_abs": 0.013188362121582031, "nrmse": 0.029768256470561028}, {"layer": 46, "field": "conv", "exact": false, "max_abs": 0.234375, "nrmse": 0.036148760467767715}, {"layer": 46, "field": "recurrent", "exact": false, "max_abs": 0.013869225978851318, "nrmse": 0.0288896095007658}, {"layer": 47, "field": "key", "exact": false, "max_abs": 0.55078125, "nrmse": 0.01572083681821823}, {"layer": 47, "field": "value", "exact": false, "max_abs": 0.484375, "nrmse": 0.020527008920907974}, {"layer": 48, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.03721965104341507}, {"layer": 48, "field": "recurrent", "exact": false, "max_abs": 0.011912435293197632, "nrmse": 0.029850581660866737}, {"layer": 49, "field": "conv", "exact": false, "max_abs": 0.375, "nrmse": 0.03660301864147186}, {"layer": 49, "field": "recurrent", "exact": false, "max_abs": 0.014102458953857422, "nrmse": 0.03442862257361412}, {"layer": 50, "field": "conv", "exact": false, "max_abs": 0.359375, "nrmse": 0.03561534732580185}, {"layer": 50, "field": "recurrent", "exact": false, "max_abs": 0.011030875146389008, "nrmse": 0.03175622224807739}, {"layer": 51, "field": "key", "exact": false, "max_abs": 0.46875, "nrmse": 0.01645837537944317}, {"layer": 51, "field": "value", "exact": false, "max_abs": 0.8125, "nrmse": 0.021655933931469917}, {"layer": 52, "field": "conv", "exact": false, "max_abs": 0.421875, "nrmse": 0.02921297587454319}, {"layer": 52, "field": "recurrent", "exact": false, "max_abs": 0.04135817289352417, "nrmse": 0.023782767355442047}, {"layer": 53, "field": "conv", "exact": false, "max_abs": 0.34375, "nrmse": 0.030011862516403198}, {"layer": 53, "field": "recurrent", "exact": false, "max_abs": 0.060302138328552246, "nrmse": 0.02952905371785164}, {"layer": 54, "field": "conv", "exact": false, "max_abs": 0.359375, "nrmse": 0.03394174575805664}, {"layer": 54, "field": "recurrent", "exact": false, "max_abs": 0.018848896026611328, "nrmse": 0.0338069386780262}, {"layer": 55, "field": "key", "exact": false, "max_abs": 0.44140625, "nrmse": 0.01683635078370571}, {"layer": 55, "field": "value", "exact": false, "max_abs": 0.8125, "nrmse": 0.023277519270777702}, {"layer": 56, "field": "conv", "exact": false, "max_abs": 0.3203125, "nrmse": 0.030821628868579865}, {"layer": 56, "field": "recurrent", "exact": false, "max_abs": 0.053534358739852905, "nrmse": 0.028436295688152313}, {"layer": 57, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.028405161574482918}, {"layer": 57, "field": "recurrent", "exact": false, "max_abs": 0.02467978000640869, "nrmse": 0.029353002086281776}, {"layer": 58, "field": "conv", "exact": false, "max_abs": 0.28125, "nrmse": 0.03047799877822399}, {"layer": 58, "field": "recurrent", "exact": false, "max_abs": 0.03655630350112915, "nrmse": 0.02584904059767723}, {"layer": 59, "field": "key", "exact": false, "max_abs": 0.4453125, "nrmse": 0.016557637602090836}, {"layer": 59, "field": "value", "exact": false, "max_abs": 1.0625, "nrmse": 0.018639912828803062}, {"layer": 60, "field": "conv", "exact": false, "max_abs": 0.5, "nrmse": 0.024512793868780136}, {"layer": 60, "field": "recurrent", "exact": false, "max_abs": 0.056479454040527344, "nrmse": 0.03111260011792183}, {"layer": 61, "field": "conv", "exact": false, "max_abs": 0.4013671875, "nrmse": 0.028532132506370544}, {"layer": 61, "field": "recurrent", "exact": false, "max_abs": 0.02271847426891327, "nrmse": 0.031115764752030373}, {"layer": 62, "field": "conv", "exact": false, "max_abs": 0.5, "nrmse": 0.02596789412200451}, {"layer": 62, "field": "recurrent", "exact": false, "max_abs": 0.03162795305252075, "nrmse": 0.023551343008875847}, {"layer": 63, "field": "key", "exact": false, "max_abs": 0.41796875, "nrmse": 0.017692724242806435}, {"layer": 63, "field": "value", "exact": false, "max_abs": 1.921875, "nrmse": 0.010617366060614586}], "buffered_graph_state_diff": []} -{"phase": "step", "index": 94, "position": 242, "teacher_token": 86, "compact": {"ids": [3, 853, 14717, 13773, 43219], "logits": [31.0, 20.75, 19.0, 16.75, 16.375], "margin": 10.25, "argmax": 3}, "buffered": {"ids": [3, 853, 14717, 13773, 43219], "logits": [31.25, 20.5, 19.0, 16.625, 16.5], "margin": 10.75, "argmax": 3}, "graph": {"ids": [3, 853, 14717, 13773, 43219], "logits": [31.25, 20.5, 19.0, 16.625, 16.5], "margin": 10.75, "argmax": 3}, "compact_buffered": {"exact": false, "max_abs": 0.4375, "nrmse": 0.024665754288434982}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "compact_buffered_state_diff": [{"layer": 20, "field": "recurrent", "exact": false, "max_abs": 0.0008896738290786743, "nrmse": 0.0008165835752151906}, {"layer": 21, "field": "conv", "exact": false, "max_abs": 0.0625, "nrmse": 0.009469235315918922}, {"layer": 21, "field": "recurrent", "exact": false, "max_abs": 0.007000982761383057, "nrmse": 0.0053426772356033325}, {"layer": 22, "field": "conv", "exact": false, "max_abs": 0.125, "nrmse": 0.01662815362215042}, {"layer": 22, "field": "recurrent", "exact": false, "max_abs": 0.009708613157272339, "nrmse": 0.012337464839220047}, {"layer": 23, "field": "key", "exact": false, "max_abs": 0.1484375, "nrmse": 0.004826508928090334}, {"layer": 23, "field": "value", "exact": false, "max_abs": 0.08203125, "nrmse": 0.00556397857144475}, {"layer": 24, "field": "conv", "exact": false, "max_abs": 0.1015625, "nrmse": 0.021910130977630615}, {"layer": 24, "field": "recurrent", "exact": false, "max_abs": 0.005181938409805298, "nrmse": 0.010928703472018242}, {"layer": 25, "field": "conv", "exact": false, "max_abs": 0.12890625, "nrmse": 0.023316094651818275}, {"layer": 25, "field": "recurrent", "exact": false, "max_abs": 0.004880249500274658, "nrmse": 0.011648707091808319}, {"layer": 26, "field": "conv", "exact": false, "max_abs": 0.1015625, "nrmse": 0.026089651510119438}, {"layer": 26, "field": "recurrent", "exact": false, "max_abs": 0.006007061339914799, "nrmse": 0.014195446856319904}, {"layer": 27, "field": "key", "exact": false, "max_abs": 0.189453125, "nrmse": 0.006499798968434334}, {"layer": 27, "field": "value", "exact": false, "max_abs": 0.1171875, "nrmse": 0.006795537658035755}, {"layer": 28, "field": "conv", "exact": false, "max_abs": 0.15625, "nrmse": 0.027518877759575844}, {"layer": 28, "field": "recurrent", "exact": false, "max_abs": 0.0040898434817790985, "nrmse": 0.01260420586913824}, {"layer": 29, "field": "conv", "exact": false, "max_abs": 0.1640625, "nrmse": 0.03151387721300125}, {"layer": 29, "field": "recurrent", "exact": false, "max_abs": 0.004459535703063011, "nrmse": 0.019772587344050407}, {"layer": 30, "field": "conv", "exact": false, "max_abs": 0.203125, "nrmse": 0.03034025989472866}, {"layer": 30, "field": "recurrent", "exact": false, "max_abs": 0.025857001543045044, "nrmse": 0.01973160356283188}, {"layer": 31, "field": "key", "exact": false, "max_abs": 0.24658203125, "nrmse": 0.007220391184091568}, {"layer": 31, "field": "value", "exact": false, "max_abs": 0.171875, "nrmse": 0.00916900485754013}, {"layer": 32, "field": "conv", "exact": false, "max_abs": 0.171875, "nrmse": 0.033990420401096344}, {"layer": 32, "field": "recurrent", "exact": false, "max_abs": 0.0048302458599209785, "nrmse": 0.020179178565740585}, {"layer": 33, "field": "conv", "exact": false, "max_abs": 0.265625, "nrmse": 0.0367446132004261}, {"layer": 33, "field": "recurrent", "exact": false, "max_abs": 0.007443636655807495, "nrmse": 0.02949322760105133}, {"layer": 34, "field": "conv", "exact": false, "max_abs": 0.21044921875, "nrmse": 0.034371353685855865}, {"layer": 34, "field": "recurrent", "exact": false, "max_abs": 0.008378613740205765, "nrmse": 0.016676494851708412}, {"layer": 35, "field": "key", "exact": false, "max_abs": 0.265625, "nrmse": 0.00837584026157856}, {"layer": 35, "field": "value", "exact": false, "max_abs": 0.20703125, "nrmse": 0.010512280277907848}, {"layer": 36, "field": "conv", "exact": false, "max_abs": 0.21875, "nrmse": 0.02848018705844879}, {"layer": 36, "field": "recurrent", "exact": false, "max_abs": 0.009214427322149277, "nrmse": 0.012873565778136253}, {"layer": 37, "field": "conv", "exact": false, "max_abs": 0.1953125, "nrmse": 0.027569646015763283}, {"layer": 37, "field": "recurrent", "exact": false, "max_abs": 0.017051905393600464, "nrmse": 0.014818389900028706}, {"layer": 38, "field": "conv", "exact": false, "max_abs": 0.21875, "nrmse": 0.0345751978456974}, {"layer": 38, "field": "recurrent", "exact": false, "max_abs": 0.007741183042526245, "nrmse": 0.02608054131269455}, {"layer": 39, "field": "key", "exact": false, "max_abs": 0.265625, "nrmse": 0.011733930557966232}, {"layer": 39, "field": "value", "exact": false, "max_abs": 0.21875, "nrmse": 0.014765048399567604}, {"layer": 40, "field": "conv", "exact": false, "max_abs": 0.2421875, "nrmse": 0.033094532787799835}, {"layer": 40, "field": "recurrent", "exact": false, "max_abs": 0.013451099395751953, "nrmse": 0.01147170178592205}, {"layer": 41, "field": "conv", "exact": false, "max_abs": 0.25390625, "nrmse": 0.03574199602007866}, {"layer": 41, "field": "recurrent", "exact": false, "max_abs": 0.007218025624752045, "nrmse": 0.023012539371848106}, {"layer": 42, "field": "conv", "exact": false, "max_abs": 0.234375, "nrmse": 0.03682924062013626}, {"layer": 42, "field": "recurrent", "exact": false, "max_abs": 0.010237321257591248, "nrmse": 0.022541344165802002}, {"layer": 43, "field": "key", "exact": false, "max_abs": 0.38671875, "nrmse": 0.014415382407605648}, {"layer": 43, "field": "value", "exact": false, "max_abs": 0.328125, "nrmse": 0.01633993722498417}, {"layer": 44, "field": "conv", "exact": false, "max_abs": 0.203125, "nrmse": 0.03464433178305626}, {"layer": 44, "field": "recurrent", "exact": false, "max_abs": 0.012706033885478973, "nrmse": 0.02554648369550705}, {"layer": 45, "field": "conv", "exact": false, "max_abs": 0.25, "nrmse": 0.03773895651102066}, {"layer": 45, "field": "recurrent", "exact": false, "max_abs": 0.013180196285247803, "nrmse": 0.029609203338623047}, {"layer": 46, "field": "conv", "exact": false, "max_abs": 0.234375, "nrmse": 0.03623369708657265}, {"layer": 46, "field": "recurrent", "exact": false, "max_abs": 0.013888120651245117, "nrmse": 0.02849840745329857}, {"layer": 47, "field": "key", "exact": false, "max_abs": 0.55078125, "nrmse": 0.01576719619333744}, {"layer": 47, "field": "value", "exact": false, "max_abs": 0.484375, "nrmse": 0.020642105489969254}, {"layer": 48, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.03724897280335426}, {"layer": 48, "field": "recurrent", "exact": false, "max_abs": 0.0087309330701828, "nrmse": 0.02929851971566677}, {"layer": 49, "field": "conv", "exact": false, "max_abs": 0.375, "nrmse": 0.03684213012456894}, {"layer": 49, "field": "recurrent", "exact": false, "max_abs": 0.0161648690700531, "nrmse": 0.03474951535463333}, {"layer": 50, "field": "conv", "exact": false, "max_abs": 0.359375, "nrmse": 0.03567574545741081}, {"layer": 50, "field": "recurrent", "exact": false, "max_abs": 0.007865265011787415, "nrmse": 0.03184236213564873}, {"layer": 51, "field": "key", "exact": false, "max_abs": 0.46875, "nrmse": 0.01648811437189579}, {"layer": 51, "field": "value", "exact": false, "max_abs": 0.8125, "nrmse": 0.02173437923192978}, {"layer": 52, "field": "conv", "exact": false, "max_abs": 0.421875, "nrmse": 0.029693476855754852}, {"layer": 52, "field": "recurrent", "exact": false, "max_abs": 0.039684295654296875, "nrmse": 0.02400212548673153}, {"layer": 53, "field": "conv", "exact": false, "max_abs": 0.34375, "nrmse": 0.029785672202706337}, {"layer": 53, "field": "recurrent", "exact": false, "max_abs": 0.06094169616699219, "nrmse": 0.02961435168981552}, {"layer": 54, "field": "conv", "exact": false, "max_abs": 0.359375, "nrmse": 0.034151893109083176}, {"layer": 54, "field": "recurrent", "exact": false, "max_abs": 0.02002561092376709, "nrmse": 0.034247029572725296}, {"layer": 55, "field": "key", "exact": false, "max_abs": 0.44140625, "nrmse": 0.016861706972122192}, {"layer": 55, "field": "value", "exact": false, "max_abs": 0.8125, "nrmse": 0.023347673937678337}, {"layer": 56, "field": "conv", "exact": false, "max_abs": 0.28125, "nrmse": 0.030966440215706825}, {"layer": 56, "field": "recurrent", "exact": false, "max_abs": 0.049428701400756836, "nrmse": 0.02817451022565365}, {"layer": 57, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.028102684766054153}, {"layer": 57, "field": "recurrent", "exact": false, "max_abs": 0.02964305877685547, "nrmse": 0.02837851084768772}, {"layer": 58, "field": "conv", "exact": false, "max_abs": 0.28125, "nrmse": 0.031080693006515503}, {"layer": 58, "field": "recurrent", "exact": false, "max_abs": 0.037941932678222656, "nrmse": 0.025822965428233147}, {"layer": 59, "field": "key", "exact": false, "max_abs": 0.4453125, "nrmse": 0.01659248024225235}, {"layer": 59, "field": "value", "exact": false, "max_abs": 1.0625, "nrmse": 0.018741179257631302}, {"layer": 60, "field": "conv", "exact": false, "max_abs": 0.5, "nrmse": 0.025001181289553642}, {"layer": 60, "field": "recurrent", "exact": false, "max_abs": 0.06767749786376953, "nrmse": 0.03131486102938652}, {"layer": 61, "field": "conv", "exact": false, "max_abs": 0.4013671875, "nrmse": 0.029326986521482468}, {"layer": 61, "field": "recurrent", "exact": false, "max_abs": 0.024246960878372192, "nrmse": 0.03137681633234024}, {"layer": 62, "field": "conv", "exact": false, "max_abs": 0.5, "nrmse": 0.02602510340511799}, {"layer": 62, "field": "recurrent", "exact": false, "max_abs": 0.03183853626251221, "nrmse": 0.02361723966896534}, {"layer": 63, "field": "key", "exact": false, "max_abs": 0.41796875, "nrmse": 0.017737969756126404}, {"layer": 63, "field": "value", "exact": false, "max_abs": 1.921875, "nrmse": 0.0106901740655303}], "buffered_graph_state_diff": []} -{"phase": "step", "index": 95, "position": 243, "teacher_token": 3, "compact": {"ids": [381, 3875, 283, 1323, 321], "logits": [37.75, 27.0, 19.25, 18.25, 17.0], "margin": 10.75, "argmax": 381}, "buffered": {"ids": [381, 3875, 283, 1323, 75389], "logits": [38.0, 27.25, 19.125, 18.25, 17.0], "margin": 10.75, "argmax": 381}, "graph": {"ids": [381, 3875, 283, 1323, 75389], "logits": [38.0, 27.25, 19.125, 18.25, 17.0], "margin": 10.75, "argmax": 381}, "compact_buffered": {"exact": false, "max_abs": 0.4375, "nrmse": 0.017105914652347565}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "compact_buffered_state_diff": [{"layer": 20, "field": "recurrent", "exact": false, "max_abs": 0.0008105933666229248, "nrmse": 0.0007939138449728489}, {"layer": 21, "field": "conv", "exact": false, "max_abs": 0.0625, "nrmse": 0.009279033169150352}, {"layer": 21, "field": "recurrent", "exact": false, "max_abs": 0.004539430141448975, "nrmse": 0.0048974608071148396}, {"layer": 22, "field": "conv", "exact": false, "max_abs": 0.125, "nrmse": 0.01663587987422943}, {"layer": 22, "field": "recurrent", "exact": false, "max_abs": 0.006794095039367676, "nrmse": 0.012239920906722546}, {"layer": 23, "field": "key", "exact": false, "max_abs": 0.1484375, "nrmse": 0.004948160611093044}, {"layer": 23, "field": "value", "exact": false, "max_abs": 0.08203125, "nrmse": 0.005738179665058851}, {"layer": 24, "field": "conv", "exact": false, "max_abs": 0.1015625, "nrmse": 0.022153833881020546}, {"layer": 24, "field": "recurrent", "exact": false, "max_abs": 0.007352948188781738, "nrmse": 0.011207171715795994}, {"layer": 25, "field": "conv", "exact": false, "max_abs": 0.125, "nrmse": 0.023711159825325012}, {"layer": 25, "field": "recurrent", "exact": false, "max_abs": 0.005158521234989166, "nrmse": 0.012015498243272305}, {"layer": 26, "field": "conv", "exact": false, "max_abs": 0.1171875, "nrmse": 0.025919940322637558}, {"layer": 26, "field": "recurrent", "exact": false, "max_abs": 0.007606178522109985, "nrmse": 0.014206006191670895}, {"layer": 27, "field": "key", "exact": false, "max_abs": 0.189453125, "nrmse": 0.006696788128465414}, {"layer": 27, "field": "value", "exact": false, "max_abs": 0.1171875, "nrmse": 0.007031410466879606}, {"layer": 28, "field": "conv", "exact": false, "max_abs": 0.15625, "nrmse": 0.027909625321626663}, {"layer": 28, "field": "recurrent", "exact": false, "max_abs": 0.004867076873779297, "nrmse": 0.01365293562412262}, {"layer": 29, "field": "conv", "exact": false, "max_abs": 0.1484375, "nrmse": 0.031206542626023293}, {"layer": 29, "field": "recurrent", "exact": false, "max_abs": 0.004653226584196091, "nrmse": 0.020945169031620026}, {"layer": 30, "field": "conv", "exact": false, "max_abs": 0.171875, "nrmse": 0.030112581327557564}, {"layer": 30, "field": "recurrent", "exact": false, "max_abs": 0.015769124031066895, "nrmse": 0.019051965326070786}, {"layer": 31, "field": "key", "exact": false, "max_abs": 0.24658203125, "nrmse": 0.007455453742295504}, {"layer": 31, "field": "value", "exact": false, "max_abs": 0.171875, "nrmse": 0.009543745778501034}, {"layer": 32, "field": "conv", "exact": false, "max_abs": 0.1875, "nrmse": 0.03472639247775078}, {"layer": 32, "field": "recurrent", "exact": false, "max_abs": 0.005454001948237419, "nrmse": 0.02136504277586937}, {"layer": 33, "field": "conv", "exact": false, "max_abs": 0.16796875, "nrmse": 0.03596702963113785}, {"layer": 33, "field": "recurrent", "exact": false, "max_abs": 0.00948953628540039, "nrmse": 0.03068796917796135}, {"layer": 34, "field": "conv", "exact": false, "max_abs": 0.1875, "nrmse": 0.03379501774907112}, {"layer": 34, "field": "recurrent", "exact": false, "max_abs": 0.010764094069600105, "nrmse": 0.0174910556524992}, {"layer": 35, "field": "key", "exact": false, "max_abs": 0.265625, "nrmse": 0.00856748316437006}, {"layer": 35, "field": "value", "exact": false, "max_abs": 0.20703125, "nrmse": 0.010886842384934425}, {"layer": 36, "field": "conv", "exact": false, "max_abs": 0.203125, "nrmse": 0.028178172186017036}, {"layer": 36, "field": "recurrent", "exact": false, "max_abs": 0.013675667345523834, "nrmse": 0.013484437949955463}, {"layer": 37, "field": "conv", "exact": false, "max_abs": 0.1875, "nrmse": 0.02745320461690426}, {"layer": 37, "field": "recurrent", "exact": false, "max_abs": 0.014738976955413818, "nrmse": 0.01542662363499403}, {"layer": 38, "field": "conv", "exact": false, "max_abs": 0.2109375, "nrmse": 0.03399806097149849}, {"layer": 38, "field": "recurrent", "exact": false, "max_abs": 0.010211192071437836, "nrmse": 0.02677527815103531}, {"layer": 39, "field": "key", "exact": false, "max_abs": 0.265625, "nrmse": 0.01184308622032404}, {"layer": 39, "field": "value", "exact": false, "max_abs": 0.21875, "nrmse": 0.014980697073042393}, {"layer": 40, "field": "conv", "exact": false, "max_abs": 0.21875, "nrmse": 0.031664688140153885}, {"layer": 40, "field": "recurrent", "exact": false, "max_abs": 0.011178016662597656, "nrmse": 0.011674842797219753}, {"layer": 41, "field": "conv", "exact": false, "max_abs": 0.1875, "nrmse": 0.03350761905312538}, {"layer": 41, "field": "recurrent", "exact": false, "max_abs": 0.007214680314064026, "nrmse": 0.02318371646106243}, {"layer": 42, "field": "conv", "exact": false, "max_abs": 0.1875, "nrmse": 0.034682974219322205}, {"layer": 42, "field": "recurrent", "exact": false, "max_abs": 0.00982964038848877, "nrmse": 0.022635454311966896}, {"layer": 43, "field": "key", "exact": false, "max_abs": 0.38671875, "nrmse": 0.014497940428555012}, {"layer": 43, "field": "value", "exact": false, "max_abs": 0.328125, "nrmse": 0.016455063596367836}, {"layer": 44, "field": "conv", "exact": false, "max_abs": 0.203125, "nrmse": 0.033342864364385605}, {"layer": 44, "field": "recurrent", "exact": false, "max_abs": 0.01273026317358017, "nrmse": 0.025936616584658623}, {"layer": 45, "field": "conv", "exact": false, "max_abs": 0.21875, "nrmse": 0.03679586946964264}, {"layer": 45, "field": "recurrent", "exact": false, "max_abs": 0.013179779052734375, "nrmse": 0.030583824962377548}, {"layer": 46, "field": "conv", "exact": false, "max_abs": 0.19921875, "nrmse": 0.03528095781803131}, {"layer": 46, "field": "recurrent", "exact": false, "max_abs": 0.013932287693023682, "nrmse": 0.028769688680768013}, {"layer": 47, "field": "key", "exact": false, "max_abs": 0.55078125, "nrmse": 0.015826310962438583}, {"layer": 47, "field": "value", "exact": false, "max_abs": 0.484375, "nrmse": 0.020755449309945107}, {"layer": 48, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.03517535701394081}, {"layer": 48, "field": "recurrent", "exact": false, "max_abs": 0.008481070399284363, "nrmse": 0.029385989531874657}, {"layer": 49, "field": "conv", "exact": false, "max_abs": 0.28125, "nrmse": 0.03530501201748848}, {"layer": 49, "field": "recurrent", "exact": false, "max_abs": 0.014088034629821777, "nrmse": 0.03517090156674385}, {"layer": 50, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.03375828266143799}, {"layer": 50, "field": "recurrent", "exact": false, "max_abs": 0.00751844048500061, "nrmse": 0.0320756770670414}, {"layer": 51, "field": "key", "exact": false, "max_abs": 0.46875, "nrmse": 0.01656353287398815}, {"layer": 51, "field": "value", "exact": false, "max_abs": 0.8125, "nrmse": 0.021854205057024956}, {"layer": 52, "field": "conv", "exact": false, "max_abs": 0.421875, "nrmse": 0.030295806005597115}, {"layer": 52, "field": "recurrent", "exact": false, "max_abs": 0.04403245449066162, "nrmse": 0.023898661136627197}, {"layer": 53, "field": "conv", "exact": false, "max_abs": 0.34375, "nrmse": 0.030396748334169388}, {"layer": 53, "field": "recurrent", "exact": false, "max_abs": 0.0608290433883667, "nrmse": 0.02988540194928646}, {"layer": 54, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.034343279898166656}, {"layer": 54, "field": "recurrent", "exact": false, "max_abs": 0.020983338356018066, "nrmse": 0.03463062271475792}, {"layer": 55, "field": "key", "exact": false, "max_abs": 0.44140625, "nrmse": 0.017004162073135376}, {"layer": 55, "field": "value", "exact": false, "max_abs": 0.8125, "nrmse": 0.023517919704318047}, {"layer": 56, "field": "conv", "exact": false, "max_abs": 0.34375, "nrmse": 0.03175637871026993}, {"layer": 56, "field": "recurrent", "exact": false, "max_abs": 0.0382649302482605, "nrmse": 0.02834862284362316}, {"layer": 57, "field": "conv", "exact": false, "max_abs": 0.375, "nrmse": 0.02874755673110485}, {"layer": 57, "field": "recurrent", "exact": false, "max_abs": 0.04297995567321777, "nrmse": 0.029417790472507477}, {"layer": 58, "field": "conv", "exact": false, "max_abs": 0.3125, "nrmse": 0.032507408410310745}, {"layer": 58, "field": "recurrent", "exact": false, "max_abs": 0.03727492690086365, "nrmse": 0.026456860825419426}, {"layer": 59, "field": "key", "exact": false, "max_abs": 0.4453125, "nrmse": 0.016698600724339485}, {"layer": 59, "field": "value", "exact": false, "max_abs": 1.25, "nrmse": 0.018926065415143967}, {"layer": 60, "field": "conv", "exact": false, "max_abs": 0.5, "nrmse": 0.02549414336681366}, {"layer": 60, "field": "recurrent", "exact": false, "max_abs": 0.04253384470939636, "nrmse": 0.032147426158189774}, {"layer": 61, "field": "conv", "exact": false, "max_abs": 0.4013671875, "nrmse": 0.03053426183760166}, {"layer": 61, "field": "recurrent", "exact": false, "max_abs": 0.02396983653306961, "nrmse": 0.03130072355270386}, {"layer": 62, "field": "conv", "exact": false, "max_abs": 0.5, "nrmse": 0.026290709152817726}, {"layer": 62, "field": "recurrent", "exact": false, "max_abs": 0.03833132982254028, "nrmse": 0.023686330765485764}, {"layer": 63, "field": "key", "exact": false, "max_abs": 0.41796875, "nrmse": 0.01783461682498455}, {"layer": 63, "field": "value", "exact": false, "max_abs": 1.921875, "nrmse": 0.010746504180133343}], "buffered_graph_state_diff": []} -{"phase": "complete", "verdict": "INCONCLUSIVE", "compact_history_matches": true, "graph_original_prefix_matches": true, "compact_history_mismatches": [], "graph_prefix_mismatches": [], "seconds": 57.72265838296153, "scope": "numerical_diagnostic_not_task_quality_or_service_acceptance"} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/graph-forced96-impact.md b/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/graph-forced96-impact.md deleted file mode 100644 index 32bc11217..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/graph-forced96-impact.md +++ /dev/null @@ -1,40 +0,0 @@ -# Residual Graph divergence: fixed-history observations - -Refs [Issue #67](https://github.com/InferMatrixLab/InferRSI/issues/67). -**Campaign INCONCLUSIVE. No engine change or new E2E gain.** - -Job1957 completed with exit0 in65 allocated GPU seconds. All96 predicted -positions have exactly equal full logits and valid KV/conv/recurrent state -between capacity1024 eager and Graph. Compact logits first differ from those -arms at index23; only index81 changes greedy argmax within this fixed history. -The96 compact choices reproduce the original eager history, and Graph matches -the original Graph prefix through index81. The complete records and observer -are retained beside this report and under validator/diagnostics. - -At index81 compact logits for tokens220 and264 are both25.625; argmax chooses220. -Capacity eager and Graph both give token26425.75 and token22025.625, choosing264. -This isolates the observed divergence to the compact-versus-capacity numerical -path in this trace. It provides no evidence of an additional Graph replay or -live-state corruption at these96 positions. It does not identify the exact -internal operator, prove all contexts/buckets correct, establish task accuracy, -or require future candidates to reproduce every greedy token bitwise. - -The prior HTTP comparison remains3/5 full matching pairs; job1951 failure and -1955 repair outcomes are unchanged. Issue67 task-quality gates, including the -separate long-context retrieval gate, remain necessary. Historical Issue57 -numerical failures are not rescored. No service timing is inferred from this -instrumented in-process diagnostic. - -Readiness allocation total904/7200 GPU seconds (0.251111 GPU-hours), with6296 -seconds remaining. The initial147-second diagnosis is separate. All relevant -allocations are released. Runtime/environment dependencies remain unchanged. - -Independent reviewer Codex review_fp8_graph checked the exact observer, -all12 runtime source digests, tree equivalence to merged main, history inputs, -96 comparisons and index81. No blocking finding for retaining this diagnostic. -The reviewer confirmed the bounded numerical-path conclusion and explicitly -excluded general correctness, task-quality or performance acceptance. - -KnowledgeStore candidate `case.qwen38-fixed-capacity-numerics-cand`, revision -`sha256:22d93a266b3eaec5a839f10a774dd88c81388be6df6d5a15f560750a88ee1331`, -was created and read back byte-for-byte; no protected library promotion. diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/graph-long-1951.json b/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/graph-long-1951.json deleted file mode 100644 index 7bcf53225..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/graph-long-1951.json +++ /dev/null @@ -1,238 +0,0 @@ -{ - "campaign_verdict": "INCONCLUSIVE", - "execution": "failed", - "job": "1951", - "allocated_gpu_seconds": 226, - "budget_stage": "readiness", - "approved_total_gpu_hours": 44, - "readiness_used_gpu_seconds": 370, - "readiness_remaining_gpu_seconds": 6830, - "initial_separate_allowance_used_gpu_seconds": 147, - "exit_code": 1, - "slurm_state": "FAILED", - "allocation_released": true, - "raw_observations_sha256": "2fefd58069ba62dd5d670f9efbe5ce684b0865435f99c8e7571e6b76175be1f0", - "observations": [ - { - "phase": "protocol", - "cases": [ - [ - "gsm8k/test/783", - 2048, - false - ], - [ - "gsm8k/test/1", - 2048, - false - ], - [ - "gsm8k/test/424", - 2048, - false - ], - [ - "gsm8k/test/783", - 1024, - true - ] - ], - "client_sha256": "f31fae8c573ecc4bf8e31dae03823502e71404333e8111f4b535432758302b29", - "observer_sha256": "32818d06d2263e90fa35fcf12e47ca920530edd28d2347246d07e134221b5e0e", - "fixture_sha256": "f359c66f92b69d9a92a66564d8a087809dbc45b7e8b9e916229065806c4c188c", - "timing_scope": "readiness_only_official_function_with_response_chunk_recording", - "source_hashes": { - "nanovllm/qwen38/__init__.py": "1f07e0cfd12bbf370c1f465c6a4bd73e72a5c826af1f29e60ccc7d6f45562cfc", - "nanovllm/qwen38/__main__.py": "9e2a0881eedf8fa4abc6037511b42fd3a44171d121a7b2cb096023f485673ba2", - "nanovllm/qwen38/_triton_fp8.py": "9f0cb68be017cb3858aebe8ba275039084e305df92cb4567d6e7da54d119f19e", - "nanovllm/qwen38/_triton_rms.py": "ba99fa3ba82bfcab4cd602fd0db907acad0eee7c7a63ab23d560a17fd2f72272", - "nanovllm/qwen38/config.py": "002ea25bbf9c8d17e5f1d3fb2b76c400e4291d7d2d06d5dd13fb5c3ac15c3d79", - "nanovllm/qwen38/engine.py": "c47e1c72fcc24bc8eeb5497887da6be11c2de567abcc3efb275b061c061fd755", - "nanovllm/qwen38/graph.py": "d4213aa68599564a01ced3b990993c9b17482615c154120bc4180f78a5da6098", - "nanovllm/qwen38/loader.py": "1aa6b3321bc135fa28d1eebf6223bbe3bc5f646630193e292fb58a9f0ca2763f", - "nanovllm/qwen38/model.py": "ad8386f156ff5315dfd73de313ab766fedd9ef892c7b94cec95303f11fd4e198", - "nanovllm/qwen38/quant.py": "5590e4769a0fbbcaf5afeee1a93db5c4bccacf09f6f3b34e42d52ec31e9f50c3", - "nanovllm/qwen38/runner.py": "3b99c999c887f6acc2f2b6aa957c01680246fe424fb3da961322c06dacf857a0", - "nanovllm/qwen38/server.py": "d2ccb1ccb61dbf06b22f7f199854d58fc7fbb246c6d8e8ec9d5f6344c3016494" - } - }, - { - "phase": "ready", - "arm": "graph", - "startup_s": 40.337891348055564, - "versions": { - "torch": "2.14.1+cu130", - "transformers": "5.20.2", - "triton": "3.8.0" - }, - "device": "NVIDIA H20-3e", - "backend": "triton_block_fp8", - "graph_prewarm": { - "buckets": { - "256": { - "capture_seconds": 1.9853857660200447, - "resident_MiB": 229.5 - }, - "1024": { - "capture_seconds": 0.17588893498759717, - "resident_MiB": 215.224609375 - }, - "2048": { - "capture_seconds": 0.17547687701880932, - "resident_MiB": 276.224609375 - }, - "4096": { - "capture_seconds": 0.1883964140433818, - "resident_MiB": 415.224609375 - }, - "8448": { - "capture_seconds": 0.1814323510043323, - "resident_MiB": 685.224609375 - } - }, - "started_unix": 1792688056.6019392, - "total_seconds": 2.7065803430741653, - "resident_MiB_total": 1821.3984375, - "completed_unix": 1792688059.3100214 - } - }, - { - "phase": "request", - "arm": "graph", - "case": "gsm8k/test/783", - "index": 0, - "cap": 2048, - "ignore_eos": false, - "prompt_tokens": 149, - "prompt_sha256": "f2233be4ebb7b122d80433fe4081fdabeda64f735b7bc24f9706c4c4339b824f", - "tokens": 529, - "terminal": [ - "stop" - ], - "usage": [ - { - "prompt_tokens": 149, - "completion_tokens": 529, - "total_tokens": 678 - } - ], - "output_text_sha256": "da51da33ffc217b8e774f28fe0fe3488b2710e0a9f9cb7ce4f315746a9b53d4f", - "wire_sha256": "4672b1858f18c6d5183f911306fb54f14ba30ee0cd85e2021c4e2296d11ff22d", - "client_success": true, - "client_error": false, - "valid": true, - "ttft_s": 2.690360520966351, - "latency_s": 49.139186647953466, - "tpot_s": 0.08797126160414226, - "active_requests_after": 0, - "runner_state_count_after": 0, - "itl_count": 528 - }, - { - "phase": "request", - "arm": "graph", - "case": "gsm8k/test/1", - "index": 1, - "cap": 2048, - "ignore_eos": false, - "prompt_tokens": 70, - "prompt_sha256": "78d335abe81363d4bb48e5aa256c120344c2ceadbcf8739afe4ec3923c747238", - "tokens": 169, - "terminal": [ - "stop" - ], - "usage": [ - { - "prompt_tokens": 70, - "completion_tokens": 169, - "total_tokens": 239 - } - ], - "output_text_sha256": "53ef84987096a7daa32212812c15c7fce023a5b06f50ea52a2c0e026d0a1f1e6", - "wire_sha256": "6a28e5186f8c971d36bcc2380c149c3f85999bb2cd81f72bc669ae85bab2c32a", - "client_success": true, - "client_error": false, - "valid": true, - "ttft_s": 1.3289078939706087, - "latency_s": 16.037696316023357, - "tpot_s": 0.08755231203602827, - "active_requests_after": 0, - "runner_state_count_after": 0, - "itl_count": 168 - }, - { - "phase": "request", - "arm": "graph", - "case": "gsm8k/test/424", - "index": 2, - "cap": 2048, - "ignore_eos": false, - "prompt_tokens": 106, - "prompt_sha256": "03cbdf5804bab62d4efbc0ae89e191bd557840c6d9a1a5ada9669de953a0b783", - "tokens": 314, - "terminal": [ - "stop" - ], - "usage": [ - { - "prompt_tokens": 106, - "completion_tokens": 314, - "total_tokens": 420 - } - ], - "output_text_sha256": "bb797c1732c29c467b45fd24a2a27f80a1a5d8ce8dc38799eb28b29a5c056bb2", - "wire_sha256": "2cbf91dc0352f4c06b658abadcef5faa663131e601b50398b30ca88c2ef4032b", - "client_success": true, - "client_error": false, - "valid": true, - "ttft_s": 1.931810028036125, - "latency_s": 29.341104445047677, - "tpot_s": 0.08756963072527653, - "active_requests_after": 0, - "runner_state_count_after": 0, - "itl_count": 313 - }, - { - "phase": "request", - "arm": "graph", - "case": "gsm8k/test/783", - "index": 3, - "cap": 1024, - "ignore_eos": true, - "prompt_tokens": 149, - "prompt_sha256": "f2233be4ebb7b122d80433fe4081fdabeda64f735b7bc24f9706c4c4339b824f", - "tokens": 876, - "terminal": [ - "error" - ], - "usage": [], - "output_text_sha256": "3b403d820505dda9a0f8558c997d4c97d2e67976b752d6d90d12d6a315c48477", - "wire_sha256": "f7c7fc174a0ccb05a660f825815ef04ac9e7d01b00e1564666e8b8486c73a3c9", - "client_success": true, - "client_error": false, - "valid": false, - "ttft_s": 2.548390226904303, - "latency_s": 79.12946258590091, - "tpot_s": 0.08752122555313897, - "active_requests_after": 0, - "runner_state_count_after": 0, - "itl_count": 876 - }, - { - "phase": "error", - "arm": "graph", - "exception": "RuntimeError" - }, - { - "phase": "cleanup", - "arm": "graph", - "exit_code": 0, - "forced": false, - "closed": true, - "active_requests": 0, - "runner_state_count": 0 - } - ], - "scope": "Three natural-EOS Graph requests healthy; fourth ignore-EOS request errors after876 output tokens at1024 capacity. Eager arm not run; comparison incomplete. No performance or quality verdict.", - "client_limitation": "Official client reports success=true for error terminal; strict observer rejected missing usage/unhealthy terminal/token shortfall." -} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/graph-long-1951.jsonl b/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/graph-long-1951.jsonl deleted file mode 100644 index 20cc7357d..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/graph-long-1951.jsonl +++ /dev/null @@ -1,8 +0,0 @@ -{"phase": "protocol", "cases": [["gsm8k/test/783", 2048, false], ["gsm8k/test/1", 2048, false], ["gsm8k/test/424", 2048, false], ["gsm8k/test/783", 1024, true]], "client_sha256": "f31fae8c573ecc4bf8e31dae03823502e71404333e8111f4b535432758302b29", "observer_sha256": "32818d06d2263e90fa35fcf12e47ca920530edd28d2347246d07e134221b5e0e", "fixture_sha256": "f359c66f92b69d9a92a66564d8a087809dbc45b7e8b9e916229065806c4c188c", "timing_scope": "readiness_only_official_function_with_response_chunk_recording", "source_hashes": {"nanovllm/qwen38/__init__.py": "1f07e0cfd12bbf370c1f465c6a4bd73e72a5c826af1f29e60ccc7d6f45562cfc", "nanovllm/qwen38/__main__.py": "9e2a0881eedf8fa4abc6037511b42fd3a44171d121a7b2cb096023f485673ba2", "nanovllm/qwen38/_triton_fp8.py": "9f0cb68be017cb3858aebe8ba275039084e305df92cb4567d6e7da54d119f19e", "nanovllm/qwen38/_triton_rms.py": "ba99fa3ba82bfcab4cd602fd0db907acad0eee7c7a63ab23d560a17fd2f72272", "nanovllm/qwen38/config.py": "002ea25bbf9c8d17e5f1d3fb2b76c400e4291d7d2d06d5dd13fb5c3ac15c3d79", "nanovllm/qwen38/engine.py": "c47e1c72fcc24bc8eeb5497887da6be11c2de567abcc3efb275b061c061fd755", "nanovllm/qwen38/graph.py": "d4213aa68599564a01ced3b990993c9b17482615c154120bc4180f78a5da6098", "nanovllm/qwen38/loader.py": "1aa6b3321bc135fa28d1eebf6223bbe3bc5f646630193e292fb58a9f0ca2763f", "nanovllm/qwen38/model.py": "ad8386f156ff5315dfd73de313ab766fedd9ef892c7b94cec95303f11fd4e198", "nanovllm/qwen38/quant.py": "5590e4769a0fbbcaf5afeee1a93db5c4bccacf09f6f3b34e42d52ec31e9f50c3", "nanovllm/qwen38/runner.py": "3b99c999c887f6acc2f2b6aa957c01680246fe424fb3da961322c06dacf857a0", "nanovllm/qwen38/server.py": "d2ccb1ccb61dbf06b22f7f199854d58fc7fbb246c6d8e8ec9d5f6344c3016494"}} -{"phase": "ready", "arm": "graph", "startup_s": 40.337891348055564, "versions": {"torch": "2.14.1+cu130", "transformers": "5.20.2", "triton": "3.8.0"}, "device": "NVIDIA H20-3e", "backend": "triton_block_fp8", "graph_prewarm": {"buckets": {"256": {"capture_seconds": 1.9853857660200447, "resident_MiB": 229.5}, "1024": {"capture_seconds": 0.17588893498759717, "resident_MiB": 215.224609375}, "2048": {"capture_seconds": 0.17547687701880932, "resident_MiB": 276.224609375}, "4096": {"capture_seconds": 0.1883964140433818, "resident_MiB": 415.224609375}, "8448": {"capture_seconds": 0.1814323510043323, "resident_MiB": 685.224609375}}, "started_unix": 1792688056.6019392, "total_seconds": 2.7065803430741653, "resident_MiB_total": 1821.3984375, "completed_unix": 1792688059.3100214}} -{"phase": "request", "arm": "graph", "case": "gsm8k/test/783", "index": 0, "cap": 2048, "ignore_eos": false, "prompt_tokens": 149, "prompt_sha256": "f2233be4ebb7b122d80433fe4081fdabeda64f735b7bc24f9706c4c4339b824f", "token_ids": [1206, 11290, 411, 3377, 11, 567, 1144, 310, 7995, 279, 1324, 314, 4022, 7397, 13853, 10471, 3018, 383, 279, 1534, 11013, 314, 18016, 310, 4022, 7397, 303, 780, 34393, 13, 271, 332, 16, 13, 36189, 2891, 279, 2763, 1528, 64700, 198, 9, 256, 3991, 18016, 7397, 25, 220, 18, 198, 9, 256, 3991, 4022, 7397, 25, 220, 19, 198, 9, 256, 10342, 2763, 7397, 25, 220, 22, 271, 332, 17, 13, 36189, 2891, 279, 4203, 64700, 198, 9, 256, 13853, 10471, 264, 2702, 314, 220, 16, 22, 7397, 13, 198, 9, 256, 6558, 393, 86, 3, 381, 279, 1324, 314, 4022, 7397, 10471, 13, 198, 9, 256, 6558, 393, 78, 3, 381, 279, 1324, 314, 18016, 7397, 10471, 13, 198, 9, 256, 1165, 1366, 421, 393, 86, 478, 296, 283, 220, 16, 22, 53031, 864, 23197, 393, 78, 283, 220, 16, 22, 471, 288, 12576, 271, 332, 18, 13, 36189, 2891, 279, 1534, 1528, 64700, 198, 9, 256, 12650, 1324, 314, 4022, 7397, 283, 3991, 4022, 478, 74522, 4022, 283, 393, 19, 478, 288, 3, 198, 9, 256, 12650, 1324, 314, 18016, 7397, 283, 3991, 18016, 478, 74522, 18016, 283, 393, 18, 478, 296, 3, 198, 9, 256, 3593, 3570, 10292, 393, 78, 283, 220, 16, 22, 471, 288, 3, 1083, 279, 18016, 22868, 25, 198, 262, 12650, 18016, 7397, 283, 393, 18, 478, 318, 16, 22, 471, 288, 8, 283, 220, 17, 15, 471, 288, 3, 271, 332, 19, 13, 19927, 279, 2876, 64700, 198, 760, 3377, 5134, 421, 303, 279, 809, 11, 13853, 995, 10598, 430, 1599, 18016, 7397, 430, 4022, 7397, 13, 198, 13682, 1088, 1272, 90, 18770, 21281, 92, 283, 220, 17, 1088, 14695, 1088, 1272, 90, 18770, 5626, 92, 25273, 271, 3036, 7417, 279, 22666, 494, 2923, 220, 18, 1083, 411, 22868, 25, 198, 13682, 220, 17, 15, 471, 288, 283, 220, 17, 1088, 14695, 318, 19, 478, 288, 8, 25273, 271, 332, 20, 13, 61148, 364, 393, 86, 60439, 332, 198, 13682, 220, 17, 15, 471, 288, 283, 220, 23, 478, 220, 17, 86, 25273, 198, 2143, 393, 86, 3, 310, 2107, 10744, 25, 198, 13682, 220, 17, 15, 283, 220, 23, 478, 220, 18, 86, 25273, 198, 3036, 2076, 220, 23, 494, 2107, 10744, 25, 198, 13682, 220, 16, 17, 283, 220, 18, 86, 25273, 198, 12151, 562, 539, 220, 18, 25, 198, 13682, 288, 283, 220, 19, 25273, 271, 332, 21, 13, 53611, 64700, 198, 9, 256, 1368, 13853, 10471, 220, 19, 4022, 7397, 11, 551, 10471, 393, 16, 22, 471, 220, 19, 283, 220, 16, 18, 3, 18016, 7397, 13, 198, 9, 256, 12650, 4022, 7397, 25, 393, 19, 19185, 1272, 90, 9147, 5242, 478, 220, 19, 19185, 1272, 90, 65, 2370, 5242, 283, 220, 23, 3, 198, 9, 256, 12650, 18016, 7397, 25, 393, 18, 19185, 1272, 90, 9147, 5242, 478, 220, 16, 18, 19185, 1272, 90, 65, 2370, 5242, 283, 220, 16, 21, 3, 198, 9, 256, 2091, 220, 16, 21, 10598, 430, 1599, 430, 220, 23, 30, 7179, 11, 393, 16, 21, 283, 220, 17, 1088, 14695, 220, 23, 12576, 271, 760, 1324, 314, 4022, 7397, 13853, 10471, 369, 220, 19, 13, 271, 794, 220, 19, 248046], "tokens": 529, "terminal": ["stop"], "usage": [{"prompt_tokens": 149, "completion_tokens": 529, "total_tokens": 678}], "output_text_sha256": "da51da33ffc217b8e774f28fe0fe3488b2710e0a9f9cb7ce4f315746a9b53d4f", "wire_sha256": "4672b1858f18c6d5183f911306fb54f14ba30ee0cd85e2021c4e2296d11ff22d", "client_success": true, "client_error": false, "valid": true, "ttft_s": 2.690360520966351, "latency_s": 49.139186647953466, "itl": [0.08828017103951424, 0.08809124492108822, 0.08786057506222278, 0.08812916895840317, 0.08800284506287426, 0.08806974592152983, 0.08801666600629687, 0.08795209298841655, 0.08811199001502246, 0.08792819804511964, 0.08807785098906606, 0.08797882101498544, 0.08801014593336731, 0.08798253105487674, 0.08789491094648838, 0.0880560390651226, 0.08797172899357975, 0.08804826997220516, 0.08788154099602252, 0.0880324940662831, 0.08794474299065769, 0.08799407200422138, 0.08799969300162047, 0.08793131390120834, 0.08793955307919532, 0.08785209793131799, 0.08816265198402107, 0.08792737103067338, 0.0883295179810375, 0.08795585506595671, 0.08796196093317121, 0.08801188401412219, 0.08805316500365734, 0.08798041997943074, 0.08783893508370966, 0.0879934870172292, 0.08796404395252466, 0.08805119793396443, 0.0879486370831728, 0.0880199879175052, 0.08814470504876226, 0.08789761702064425, 0.08811382995918393, 0.08780613099224865, 0.08805279701482505, 0.08802338503301144, 0.08797309396322817, 0.0879514740081504, 0.0879380990518257, 0.08801457891240716, 0.0879423760343343, 0.08815899898763746, 0.0879694310715422, 0.08801328798290342, 0.0879583009518683, 0.08794510504230857, 0.08800178999081254, 0.08787688496522605, 0.08819118305109441, 0.0877711100038141, 0.08798393292818218, 0.08797892509028316, 0.08799012890085578, 0.08802640903741121, 0.08799416397232562, 0.08806483808439225, 0.08780694392044097, 0.08807362709194422, 0.08784458495210856, 0.0880249059991911, 0.0879117080476135, 0.08796808891929686, 0.08801195002160966, 0.08787553897127509, 0.08806232898496091, 0.08788610307965428, 0.08800887002144009, 0.08796021190937608, 0.08805131609551609, 0.08793872792739421, 0.0879554720595479, 0.08800788596272469, 0.08797558001242578, 0.08812238893005997, 0.08776475500781089, 0.08803325507324189, 0.0879618190228939, 0.08793149900157005, 0.08799483394250274, 0.0879201500210911, 0.08801584597676992, 0.08795894996728748, 0.08799211599398404, 0.08789665705990046, 0.08809945196844637, 0.08795512304641306, 0.0879838119726628, 0.08806351304519922, 0.08799889590591192, 0.08794345101341605, 0.08781056001316756, 0.08803831902332604, 0.08790275896899402, 0.08790979499462992, 0.08795870700851083, 0.08790456503629684, 0.08794420293997973, 0.08794050104916096, 0.088166982983239, 0.08787841396406293, 0.08796933304984123, 0.08791789400856942, 0.08795617800205946, 0.08796137303579599, 0.08790929289534688, 0.08826343307737261, 0.08783917897380888, 0.08806658303365111, 0.08782596001401544, 0.0880296080140397, 0.08797159895766526, 0.08793147001415491, 0.08788930100854486, 0.08796957100275904, 0.08814591390546411, 0.08790974004659802, 0.08810059702955186, 0.08806129393633455, 0.08803743706084788, 0.0879062790190801, 0.08793674898333848, 0.08803163096308708, 0.08781595097389072, 0.08806189801543951, 0.08783909003250301, 0.08805357001256198, 0.08798322093207389, 0.08797594509087503, 0.08805549400858581, 0.08788126893341541, 0.08809632307384163, 0.08793545688968152, 0.0880109251011163, 0.0879008979536593, 0.08796710497699678, 0.08811504603363574, 0.08805327699519694, 0.08803670201450586, 0.08797137998044491, 0.08809134794864804, 0.08788202109280974, 0.08800825790967792, 0.08789664704818279, 0.08800336497370154, 0.08807163999881595, 0.08793654898181558, 0.08799423300661147, 0.08790220902301371, 0.08803790505044162, 0.08782249595969915, 0.08797729597426951, 0.08784001204185188, 0.08801501302514225, 0.08795548498164862, 0.08791135298088193, 0.08799619798082858, 0.08800269197672606, 0.08808771905023605, 0.08809364098124206, 0.08801929303444922, 0.08800675196107477, 0.08788272796664387, 0.08800052001606673, 0.08784272999037057, 0.08802998508326709, 0.08779808890540153, 0.08802799205295742, 0.08793492498807609, 0.08794183004647493, 0.08793789800256491, 0.08794342493638396, 0.08804759208578616, 0.08781643491238356, 0.08804659603629261, 0.08792947302572429, 0.08799725992139429, 0.08791049709543586, 0.08798834192566574, 0.08798692200798541, 0.08790100796613842, 0.08796277502551675, 0.08790536702144891, 0.08801585296168923, 0.08793401799630374, 0.08800794999115169, 0.08796666108537465, 0.08792514202650636, 0.08826518093701452, 0.08786939398851246, 0.0879936550045386, 0.08789897104725242, 0.08796387899201363, 0.08782884001266211, 0.08796951093245298, 0.08795036701485515, 0.08792912203352898, 0.08798308693803847, 0.08787993609439582, 0.08806124795228243, 0.08806665998417884, 0.0881988420151174, 0.08799402904696763, 0.08794899098575115, 0.08798665599897504, 0.08795547194313258, 0.08800296403933316, 0.08784382801968604, 0.08807110390625894, 0.08788775908760726, 0.08800931298173964, 0.08786718698684126, 0.08794368000235409, 0.08793544094078243, 0.08786891505587846, 0.08805514604318887, 0.08794792590197176, 0.08815118903294206, 0.08806521003134549, 0.08805395301897079, 0.08792598498985171, 0.08808471693191677, 0.08799646806437522, 0.087922333041206, 0.0879596199374646, 0.08787671697791666, 0.08802374906372279, 0.08800334902480245, 0.08789454190991819, 0.0880167280556634, 0.08786562492605299, 0.08793800207786262, 0.0879240520298481, 0.08803452691063285, 0.08791421097703278, 0.08798134105745703, 0.08801582700107247, 0.08799994899891317, 0.0879508430371061, 0.08793306292500347, 0.08809535298496485, 0.08795162406750023, 0.08806827198714018, 0.08795729500707239, 0.08796737797092646, 0.08797634206712246, 0.08803136693313718, 0.08796208596322685, 0.08789775799959898, 0.08799459808506072, 0.08796366292517632, 0.0880718210246414, 0.08791814802680165, 0.08799516898579895, 0.08796792896464467, 0.08795797603670508, 0.08799314801581204, 0.08785217604599893, 0.0881408239947632, 0.08785208489280194, 0.0880361630115658, 0.08803485799580812, 0.08803897106554359, 0.08810382604133338, 0.08790769299957901, 0.08808481297455728, 0.08794227801263332, 0.08799447095952928, 0.08793939801398665, 0.08802523801568896, 0.08800064597744495, 0.08790872700046748, 0.08802718599326909, 0.08784596202895045, 0.08804380591027439, 0.08798979909624904, 0.08803378092125058, 0.08790076209697872, 0.08794458291959018, 0.08801786007825285, 0.08779293694533408, 0.0879146889783442, 0.08796309307217598, 0.08801312500145286, 0.08826213690917939, 0.08802778203971684, 0.08791884302627295, 0.08794709900394082, 0.08800141094252467, 0.08785489399451762, 0.08802026405464858, 0.08778470696415752, 0.08797826897352934, 0.08795189100783318, 0.08797150698956102, 0.0879717260831967, 0.08791563392151147, 0.0879839340923354, 0.0879071889212355, 0.08795046806335449, 0.08803806290961802, 0.08811870100907981, 0.08786609605886042, 0.08800062700174749, 0.08806978503707796, 0.08791951497551054, 0.0880608520237729, 0.08778795890975744, 0.08807335805613548, 0.08789732400327921, 0.08800698700360954, 0.08791718096472323, 0.08796152495779097, 0.08801570208743215, 0.08798566390760243, 0.0880609730957076, 0.08789373200852424, 0.08797567198053002, 0.08790912199765444, 0.08794379595201463, 0.08797908306587487, 0.08797307591885328, 0.08812829398084432, 0.08786421408876777, 0.0880226269364357, 0.08785159408580512, 0.08794508292339742, 0.0880601640092209, 0.08791079197544605, 0.08795084105804563, 0.0879215159220621, 0.08800528105348349, 0.08787215000484139, 0.08798070903867483, 0.0878135790117085, 0.08795489091426134, 0.08791713998652995, 0.08797505707480013, 0.08794285799376667, 0.08792846498545259, 0.08805647597182542, 0.08786241000052541, 0.08803784800693393, 0.08800821902696043, 0.08792964403983206, 0.0879504659678787, 0.08791385998483747, 0.08806380804162472, 0.08790368493646383, 0.08803036203607917, 0.0879206609679386, 0.08803582598920912, 0.08784064603969455, 0.08790307503659278, 0.08798762795049697, 0.08787116396706551, 0.0880231880582869, 0.08789315295871347, 0.08802260097581893, 0.0879799909889698, 0.08789461804553866, 0.0879895519465208, 0.08804618299473077, 0.08796620403882116, 0.08797303703613579, 0.08806215797085315, 0.08792639698367566, 0.08803154597990215, 0.0879637940088287, 0.08796377503313124, 0.08801865298300982, 0.08787465805653483, 0.08794741600286216, 0.08901292691007257, 0.08684701600577682, 0.08784674399066716, 0.08797541807871312, 0.08795811596792191, 0.08782037405762821, 0.0880290229106322, 0.08789927908219397, 0.08808944991324097, 0.08786391199100763, 0.08794165099970996, 0.08791259105782956, 0.08798556996043772, 0.08782381704077125, 0.08799106196966022, 0.08797815907746553, 0.08785367000382394, 0.08809701597783715, 0.08787377597764134, 0.08799483894836158, 0.08789882808923721, 0.08797361992765218, 0.08799506106879562, 0.08796779694966972, 0.08802283403929323, 0.08782176894601434, 0.08792578207794577, 0.08786285691894591, 0.08796102402266115, 0.08791211398784071, 0.08802247000858188, 0.08798321604263037, 0.08794199593830854, 0.08806033700238913, 0.08780980401206762, 0.08809780003502965, 0.08821539091877639, 0.08805758610833436, 0.08792033500503749, 0.0879282399546355, 0.0879301360109821, 0.08795986394397914, 0.08814619504846632, 0.08780536497943103, 0.08793854794930667, 0.08792911900673062, 0.08796791103668511, 0.08789729897398502, 0.08794095600023866, 0.08815718500409275, 0.08787726808805019, 0.08799192192964256, 0.08785645896568894, 0.08803783904295415, 0.08795422606635839, 0.08791337697766721, 0.08818787301424891, 0.08799898892175406, 0.0880440870532766, 0.08791097393259406, 0.08803179999813437, 0.08788086101412773, 0.08803546498529613, 0.08796248899307102, 0.08788402110803872, 0.08800154097843915, 0.08794239000417292, 0.0879765700083226, 0.08787767495959997, 0.08796222100500017, 0.08788950403686613, 0.08797394100110978, 0.08798597997520119, 0.08802434091921896, 0.08808942406903952, 0.08778758998960257, 0.08804942900314927, 0.08781652001198381, 0.08801532094366848, 0.08792977500706911, 0.08791234600357711, 0.08797707501798868, 0.08809179195668548, 0.0880713970400393, 0.08787363802548498, 0.08803444600198418, 0.08786197192966938, 0.08798229100648314, 0.08795548102352768, 0.08798900805413723, 0.08802146092057228, 0.0879030420910567, 0.08805348700843751, 0.08786811493337154, 0.08808164100628346, 0.08789854601491243, 0.08797507104463875, 0.08797429595142603, 0.0878542079590261, 0.08801749907433987, 0.08785532100591809, 0.08805188292171806, 0.08800783508922905, 0.08798164199106395, 0.0878831809386611, 0.08799645397812128, 0.08796333207283169, 0.08819280692841858, 0.08780945604667068, 0.0879020489519462, 0.08803506509866565, 0.08788898098282516, 0.08795327390544116, 0.08789542911108583, 0.08800666790921241, 0.08802360401023179, 0.08795555797405541, 0.08805095299612731, 0.08788263308815658, 0.08803009998518974, 0.08795370301231742, 0.08790010400116444, 0.08790956495795399, 0.08787716005463153, 0.08802221796941012, 0.08781051496043801, 0.08805741497781128, 0.0879585430957377, 0.08804971491917968, 0.08792199799790978, 0.08792628208175302, 0.08794342295732349, 0.08786735404282808, 0.08805714396294206, 0.08792794495820999, 0.0879494909895584, 0.08785738702863455, 0.08797536906786263, 0.08794704894535244, 0.08787263499107212, 0.08802787004970014, 0.0878853719914332, 0.08799334801733494, 0.08782195800449699, 0.08803222689311951, 0.08801882504485548], "tpot_s": 0.08797126160414226, "active_requests_after": 0, "runner_state_count_after": 0} -{"phase": "request", "arm": "graph", "case": "gsm8k/test/1", "index": 1, "cap": 2048, "ignore_eos": false, "prompt_tokens": 70, "prompt_sha256": "78d335abe81363d4bb48e5aa256c120344c2ceadbcf8739afe4ec3923c747238", "token_ids": [1206, 7995, 279, 2702, 1324, 314, 47205, 2483, 364, 279, 60507, 11, 567, 1144, 310, 10724, 279, 3202, 314, 4022, 23043, 321, 884, 424, 310, 279, 3202, 314, 6105, 23043, 13, 271, 16, 13, 220, 2972, 10025, 23043, 92217, 561, 3377, 5134, 421, 279, 60507, 4829, 220, 17, 47205, 314, 6105, 23043, 13, 198, 17, 13, 220, 2972, 13672, 23043, 92217, 561, 3377, 5134, 421, 424, 4829, 328, 35778, 421, 1691, 1, 4022, 23043, 13, 328, 4639, 1691, 1, 18675, 310, 279, 3202, 314, 6105, 23043, 13, 198, 262, 348, 256, 71690, 25, 393, 17, 1088, 1272, 90, 47205, 92, 593, 220, 17, 283, 220, 16, 1088, 1272, 90, 30797, 92, 12576, 198, 18, 13, 220, 2972, 7354, 47205, 92217, 2604, 279, 6105, 23043, 321, 279, 4022, 23043, 3658, 13, 198, 262, 348, 256, 71690, 25, 393, 17, 1088, 1272, 90, 47205, 318, 11855, 8866, 478, 220, 16, 1088, 1272, 90, 30797, 318, 5603, 8866, 283, 220, 18, 1088, 1272, 90, 47205, 92, 12576, 271, 794, 220, 18, 248046], "tokens": 169, "terminal": ["stop"], "usage": [{"prompt_tokens": 70, "completion_tokens": 169, "total_tokens": 239}], "output_text_sha256": "53ef84987096a7daa32212812c15c7fce023a5b06f50ea52a2c0e026d0a1f1e6", "wire_sha256": "6a28e5186f8c971d36bcc2380c149c3f85999bb2cd81f72bc669ae85bab2c32a", "client_success": true, "client_error": false, "valid": true, "ttft_s": 1.3289078939706087, "latency_s": 16.037696316023357, "itl": [0.08746136503759772, 0.08738634292967618, 0.08741781604476273, 0.08763664797879755, 0.08749722607899457, 0.08754246693570167, 0.08756379503756762, 0.08748531702440232, 0.08757838991004974, 0.08742591901682317, 0.08771446300670505, 0.08744006103370339, 0.08758591196965426, 0.08749093499500304, 0.08751952997408807, 0.08754635509103537, 0.087493936996907, 0.08771555800922215, 0.0874135559424758, 0.08747148199472576, 0.08743572304956615, 0.08784963202197105, 0.08757376298308372, 0.08753520192112774, 0.0875154590466991, 0.08753165893722326, 0.08754607301671058, 0.08742738398723304, 0.08755286608356982, 0.08750450401566923, 0.08751311199739575, 0.08762059698347002, 0.08757382701151073, 0.08780651795677841, 0.0874656590167433, 0.08766055502928793, 0.08753427397459745, 0.08753929601516575, 0.08758443593978882, 0.08741519704926759, 0.08765182399656624, 0.08739496592897922, 0.08760515204630792, 0.08758835901971906, 0.08763891598209739, 0.08747772395145148, 0.08756359003018588, 0.08752822608221322, 0.08756687794812024, 0.08758686704095453, 0.08743455191142857, 0.08758457307703793, 0.08752718393225223, 0.08757820702157915, 0.08752218796871603, 0.08753121306654066, 0.08766698301769793, 0.08749862597323954, 0.08768548793159425, 0.08760785008780658, 0.08755048096645623, 0.08746409695595503, 0.08762970706447959, 0.08745927398558706, 0.08750580693595111, 0.0876527150394395, 0.08750447304919362, 0.087648308952339, 0.08736877900082618, 0.08755202207248658, 0.08752528892364353, 0.08753009501378983, 0.08760444901417941, 0.08754880295600742, 0.08757406403310597, 0.08754908596165478, 0.08766294503584504, 0.08751001697964966, 0.08758799696806818, 0.08757022710051388, 0.08751341991592199, 0.08789832901675254, 0.08717332605738193, 0.087658685981296, 0.08748479397036135, 0.08754106506239623, 0.08765223692171276, 0.08752995007671416, 0.08758459892123938, 0.08751542901154608, 0.08760190906468779, 0.08745880692731589, 0.08764446107670665, 0.08757331594824791, 0.08751079405192286, 0.08749243698548526, 0.08746852399781346, 0.08761671395041049, 0.08740574901457876, 0.08761503000278026, 0.08753705804701895, 0.08758072694763541, 0.08766418404411525, 0.0875375309260562, 0.08752718602772802, 0.08750021306332201, 0.08766128995921463, 0.08748268603812903, 0.08757963799871504, 0.08746576099656522, 0.08761246199719608, 0.0876180490013212, 0.08747450693044811, 0.08762642205692828, 0.08743121998850256, 0.087653027032502, 0.0875444388948381, 0.08764385501854122, 0.0876898739952594, 0.08781841606833041, 0.08750678901560605, 0.08741701091639698, 0.08759780402760953, 0.087501444038935, 0.08753684302791953, 0.08752322196960449, 0.08751612098421901, 0.08751036995090544, 0.0875179140130058, 0.08755991701036692, 0.08746612002141774, 0.0876938080182299, 0.08740810700692236, 0.0877114050090313, 0.08768942800816149, 0.08761884097475559, 0.08763185993302613, 0.08746590209193528, 0.08767193695530295, 0.08743597695138305, 0.08758606703486294, 0.08753761101979762, 0.08754693693481386, 0.08759862510487437, 0.08754294691607356, 0.08759996306616813, 0.08734382700640708, 0.0875506759621203, 0.08749935496598482, 0.08758112008217722, 0.08748688199557364, 0.08754523599054664, 0.08775012590922415, 0.08758751000277698, 0.08767337200697511, 0.08745771099347621, 0.08761129004415125, 0.08744710695464164, 0.087513005011715, 0.08760833600535989, 0.08748421398922801, 0.08771226601675153, 0.08747126604430377, 0.0876374610234052, 0.08739765791688114, 0.08756615500897169, 0.08747110900003463, 0.08760083606466651], "tpot_s": 0.08755231203602827, "active_requests_after": 0, "runner_state_count_after": 0} -{"phase": "request", "arm": "graph", "case": "gsm8k/test/424", "index": 2, "cap": 2048, "ignore_eos": false, "prompt_tokens": 106, "prompt_sha256": "03cbdf5804bab62d4efbc0ae89e191bd557840c6d9a1a5ada9669de953a0b783", "token_ids": [1206, 7995, 279, 2702, 1324, 314, 7998, 421, 628, 4973, 279, 1834, 8151, 11, 567, 1144, 310, 10724, 279, 2702, 8390, 314, 660, 279, 47224, 11145, 13, 271, 16, 13, 220, 2972, 46270, 279, 8390, 314, 279, 32275, 64700, 198, 262, 2532, 513, 220, 19, 32275, 11, 321, 1754, 9687, 220, 21, 15, 7998, 13, 198, 262, 25273, 19, 1088, 14695, 220, 21, 15, 283, 220, 17, 19, 15, 1088, 1272, 90, 7998, 92, 13682, 271, 17, 13, 220, 2972, 46270, 279, 8390, 314, 279, 73320, 4623, 64700, 198, 262, 2532, 513, 220, 21, 73320, 4623, 11, 321, 1754, 9687, 220, 18, 15, 7998, 13, 198, 262, 25273, 21, 1088, 14695, 220, 18, 15, 283, 220, 16, 23, 15, 1088, 1272, 90, 7998, 92, 13682, 271, 18, 13, 220, 2972, 46270, 279, 8390, 314, 279, 1262, 340, 581, 64700, 198, 262, 2532, 513, 220, 16, 15, 1262, 340, 581, 11, 321, 1754, 9687, 220, 16, 20, 7998, 13, 198, 262, 25273, 16, 15, 1088, 14695, 220, 16, 20, 283, 220, 16, 20, 15, 1088, 1272, 90, 7998, 92, 13682, 271, 19, 13, 220, 2972, 8917, 279, 56468, 64700, 198, 262, 10342, 7998, 283, 318, 28432, 314, 32275, 8, 478, 318, 28432, 314, 73320, 4623, 8, 478, 318, 28432, 314, 1262, 340, 581, 8, 198, 262, 25273, 17, 19, 15, 478, 220, 16, 23, 15, 478, 220, 16, 20, 15, 13682, 271, 262, 5339, 11, 884, 220, 17, 19, 15, 321, 220, 16, 23, 15, 25, 198, 262, 25273, 17, 19, 15, 478, 220, 16, 23, 15, 283, 220, 19, 17, 15, 13682, 271, 262, 4844, 11, 884, 220, 16, 20, 15, 25, 198, 262, 25273, 19, 17, 15, 478, 220, 16, 20, 15, 283, 220, 20, 22, 15, 13682, 271, 760, 2702, 1324, 314, 7998, 421, 628, 4973, 279, 1834, 8151, 369, 220, 20, 22, 15, 13, 271, 794, 220, 20, 22, 15, 248046], "tokens": 314, "terminal": ["stop"], "usage": [{"prompt_tokens": 106, "completion_tokens": 314, "total_tokens": 420}], "output_text_sha256": "bb797c1732c29c467b45fd24a2a27f80a1a5d8ce8dc38799eb28b29a5c056bb2", "wire_sha256": "2cbf91dc0352f4c06b658abadcef5faa663131e601b50398b30ca88c2ef4032b", "client_success": true, "client_error": false, "valid": true, "ttft_s": 1.931810028036125, "latency_s": 29.341104445047677, "itl": [0.08746430103201419, 0.08759792894124985, 0.08752246503718197, 0.08759009500499815, 0.08763674600049853, 0.08759550901595503, 0.08765526593197137, 0.08752834110055119, 0.08767661894671619, 0.08752920804545283, 0.08767034497577697, 0.08751130802556872, 0.08755277597811073, 0.08760833798442036, 0.08752068202011287, 0.08766098995693028, 0.08743052405770868, 0.08765717095229775, 0.08745621400885284, 0.08760716195683926, 0.08752890909090638, 0.08744939090684056, 0.08752988104242831, 0.08744029002264142, 0.08767079096287489, 0.08754267205949873, 0.08799133193679154, 0.08758357807528228, 0.0875873229233548, 0.08761382300872356, 0.087437286041677, 0.0876974209677428, 0.08742336696013808, 0.08767768205143511, 0.08749148098286241, 0.08757319406140596, 0.0875870588934049, 0.08765527710784227, 0.08762352494522929, 0.08758310205303133, 0.08770907390862703, 0.08746697602327913, 0.08756553498096764, 0.08745801006443799, 0.08758012496400625, 0.08752818696666509, 0.08743764599785209, 0.08760070602875203, 0.08747167000547051, 0.08771591901313514, 0.0875137880211696, 0.08751902100630105, 0.08753339992836118, 0.08747585501987487, 0.08752160705626011, 0.08751297392882407, 0.08762687398120761, 0.08744375500828028, 0.08755708707030863, 0.08757057099137455, 0.08751251094508916, 0.08752271009143442, 0.08759960089810193, 0.08776054705958813, 0.08745224494487047, 0.08760349301155657, 0.08750389597844332, 0.08762286009732634, 0.08745138894300908, 0.08747339597903192, 0.08762216498143971, 0.08736505010165274, 0.08766249497421086, 0.08741638401988894, 0.08755669894162565, 0.08748799096792936, 0.08763707103207707, 0.08754437405150384, 0.08756419899873435, 0.08760062290821224, 0.08751204900909215, 0.08773773803841323, 0.08763450803235173, 0.08752691291738302, 0.0874434660654515, 0.08755596994888037, 0.08760572108440101, 0.08746998698916286, 0.08768601599149406, 0.08745923300739378, 0.08754982298705727, 0.08750493393745273, 0.08759188605472445, 0.08754009299445897, 0.08754859701730311, 0.08763129299040884, 0.08746776101179421, 0.08757674600929022, 0.0874938489869237, 0.08762254694011062, 0.08751168299932033, 0.08751176006626338, 0.08761111402418464, 0.08745667897164822, 0.08767810394056141, 0.08744924806524068, 0.08749039494432509, 0.0875094870571047, 0.0884425740223378, 0.0875525459414348, 0.08752164198085666, 0.08850708696991205, 0.08742932509630919, 0.08771862799767405, 0.08824173000175506, 0.08751785301137716, 0.08751959598157555, 0.0883436209987849, 0.0875914809294045, 0.08747575304005295, 0.08771081897430122, 0.08746674400754273, 0.08768239698838443, 0.08864444005303085, 0.08744130202103406, 0.08751287590712309, 0.08751010207924992, 0.08758099493570626, 0.08843712008092552, 0.08749878092203289, 0.0873912509996444, 0.08756499900482595, 0.08748474100138992, 0.08835313201416284, 0.08755360601935536, 0.08742551994509995, 0.08770412509329617, 0.08753883093595505, 0.08775024197530001, 0.08754042501095682, 0.0875973739894107, 0.08756497805006802, 0.08749690500553697, 0.08758381998632103, 0.08754743204917759, 0.08759505290072411, 0.08741253905463964, 0.08761802001390606, 0.08751721703447402, 0.08757005596999079, 0.08758731803391129, 0.08747388597112149, 0.08754783892072737, 0.08744537306483835, 0.08765434101223946, 0.08750134299043566, 0.08766998304054141, 0.08761378889903426, 0.08759967109654099, 0.08756781194824725, 0.08748707501217723, 0.08758516702800989, 0.08742132398765534, 0.08760432794224471, 0.08763524505775422, 0.08744649798609316, 0.08757144003175199, 0.08742283296305686, 0.08762623299844563, 0.08739021304063499, 0.08761404594406486, 0.08746036002412438, 0.08767305396031588, 0.08750206197146326, 0.08756420307327062, 0.08759529399685562, 0.0876973420381546, 0.08764351496938616, 0.08741001295857131, 0.08754648303147405, 0.087519351975061, 0.08760644402354956, 0.08747567003592849, 0.08749525295570493, 0.08766066597308964, 0.08740463701542467, 0.08761234697885811, 0.08750701404642314, 0.08750869100913405, 0.08754541899543256, 0.08755945903249085, 0.08754552691243589, 0.08745854301378131, 0.08762620505876839, 0.08746606693603098, 0.08771597302984446, 0.08749352197628468, 0.08751055202446878, 0.0875581200234592, 0.08752267202362418, 0.08762221492361277, 0.08745169104076922, 0.08757782401517034, 0.08746236993465573, 0.08753028302453458, 0.08750777505338192, 0.08754884696099907, 0.08768166904337704, 0.08741888497024775, 0.08756430400535464, 0.08745897293556482, 0.08770789706613868, 0.08745211793575436, 0.0875393400201574, 0.08748856198508292, 0.08746692002750933, 0.08758645702619106, 0.08744252601172775, 0.0876715750200674, 0.08738626295235008, 0.08762586093507707, 0.0875180991133675, 0.08744151692371815, 0.08756049198564142, 0.08743433200288564, 0.08765346405562013, 0.0874202559934929, 0.08763494598679245, 0.08741684805136174, 0.08746089495252818, 0.08759231399744749, 0.08745566895231605, 0.08767168899066746, 0.08736245799809694, 0.08767533907666802, 0.08778601395897567, 0.08758120599668473, 0.08749086107127368, 0.08748395298607647, 0.08762758097145706, 0.08746200997848064, 0.08764324698131531, 0.08743604202754796, 0.08754442096687853, 0.0874931620201096, 0.08769840106833726, 0.08768476196564734, 0.08744490495882928, 0.08766128297429532, 0.08739652403164655, 0.08761330996640027, 0.08746541105210781, 0.08761152799706906, 0.0877237890381366, 0.08771554194390774, 0.08754536206834018, 0.08740242093335837, 0.08767121203709394, 0.08743679395411164, 0.08754190499894321, 0.08749631699174643, 0.08748901006765664, 0.08753946400247514, 0.08749945892486721, 0.08778833399992436, 0.08739241701550782, 0.08750668202992529, 0.08762110804673284, 0.08760339696891606, 0.08754757093265653, 0.08751339500304312, 0.08751828805543482, 0.08748472295701504, 0.08762675197795033, 0.08746006002184004, 0.08761445106938481, 0.08746979502029717, 0.08756500389426947, 0.08774488302879035, 0.08750797098036855, 0.0876209270209074, 0.08746831596363336, 0.08758828905411065, 0.08745820203330368, 0.08758974098600447, 0.08744926797226071, 0.08756922604516149, 0.0875156429829076, 0.0873842400033027, 0.08763691794592887, 0.08749733201693743, 0.08763144200202078, 0.08752420998644084, 0.0875540190609172, 0.08751183992717415, 0.08758191205561161, 0.08757094503380358, 0.0874448559479788, 0.08757409104146063, 0.0874863649951294, 0.08764493593480438, 0.08755358401685953, 0.08753801696002483, 0.08763013407588005, 0.08744988101534545, 0.08762733195908368, 0.08744653302710503, 0.0875846209237352, 0.08746248204261065, 0.08751542295794934, 0.08758546703029424, 0.08747806400060654, 0.08766541501972824], "tpot_s": 0.08756963072527653, "active_requests_after": 0, "runner_state_count_after": 0} -{"phase": "request", "arm": "graph", "case": "gsm8k/test/783", "index": 3, "cap": 1024, "ignore_eos": true, "prompt_tokens": 149, "prompt_sha256": "f2233be4ebb7b122d80433fe4081fdabeda64f735b7bc24f9706c4c4339b824f", "token_ids": [1206, 11290, 411, 3377, 11, 567, 1144, 310, 7995, 279, 1324, 314, 4022, 7397, 13853, 10471, 3018, 383, 279, 1534, 11013, 314, 18016, 310, 4022, 7397, 303, 780, 34393, 13, 271, 332, 16, 13, 36189, 2891, 279, 2763, 1528, 64700, 198, 9, 256, 3991, 18016, 7397, 25, 220, 18, 198, 9, 256, 3991, 4022, 7397, 25, 220, 19, 198, 9, 256, 10342, 2763, 7397, 25, 220, 22, 271, 332, 17, 13, 36189, 2891, 279, 4203, 64700, 198, 9, 256, 13853, 10471, 264, 2702, 314, 220, 16, 22, 7397, 13, 198, 9, 256, 6558, 393, 86, 3, 381, 279, 1324, 314, 4022, 7397, 10471, 13, 198, 9, 256, 6558, 393, 78, 3, 381, 279, 1324, 314, 18016, 7397, 10471, 13, 198, 9, 256, 1165, 1366, 421, 393, 86, 478, 296, 283, 220, 16, 22, 53031, 864, 23197, 393, 78, 283, 220, 16, 22, 471, 288, 12576, 271, 332, 18, 13, 36189, 2891, 279, 1534, 1528, 64700, 198, 9, 256, 12650, 1324, 314, 4022, 7397, 283, 3991, 4022, 478, 74522, 4022, 283, 393, 19, 478, 288, 3, 198, 9, 256, 12650, 1324, 314, 18016, 7397, 283, 3991, 18016, 478, 74522, 18016, 283, 393, 18, 478, 296, 3, 198, 9, 256, 3593, 3570, 10292, 393, 78, 283, 220, 16, 22, 471, 288, 3, 1083, 279, 18016, 22868, 25, 198, 262, 12650, 18016, 7397, 283, 393, 18, 478, 318, 16, 22, 471, 288, 8, 283, 220, 17, 15, 471, 288, 3, 271, 332, 19, 13, 19927, 279, 2876, 64700, 198, 760, 3377, 5134, 421, 303, 279, 809, 11, 13853, 995, 10598, 430, 1599, 18016, 7397, 430, 4022, 7397, 13, 198, 13682, 1088, 1272, 90, 18770, 21281, 92, 283, 220, 17, 1088, 14695, 1088, 1272, 90, 18770, 5626, 92, 25273, 271, 3036, 7417, 279, 22666, 494, 2923, 220, 18, 1083, 411, 22868, 25, 198, 13682, 220, 17, 15, 471, 288, 283, 220, 17, 1088, 14695, 318, 19, 478, 288, 8, 25273, 271, 332, 20, 13, 61148, 364, 393, 86, 60439, 332, 198, 13682, 220, 17, 15, 471, 288, 283, 220, 23, 478, 220, 17, 86, 25273, 198, 2143, 393, 86, 3, 310, 2107, 10744, 25, 198, 13682, 220, 17, 15, 283, 220, 23, 478, 220, 18, 86, 25273, 198, 3036, 2076, 220, 23, 494, 2107, 10744, 25, 198, 13682, 220, 16, 17, 283, 220, 18, 86, 25273, 198, 12151, 562, 539, 220, 18, 25, 198, 13682, 288, 283, 220, 19, 25273, 271, 332, 21, 13, 53611, 64700, 198, 9, 256, 1368, 13853, 10471, 220, 19, 4022, 7397, 11, 551, 10471, 393, 16, 22, 471, 220, 19, 283, 220, 16, 18, 3, 18016, 7397, 13, 198, 9, 256, 12650, 4022, 7397, 25, 393, 19, 19185, 1272, 90, 9147, 5242, 478, 220, 19, 19185, 1272, 90, 65, 2370, 5242, 283, 220, 23, 3, 198, 9, 256, 12650, 18016, 7397, 25, 393, 18, 19185, 1272, 90, 9147, 5242, 478, 220, 16, 18, 19185, 1272, 90, 65, 2370, 5242, 283, 220, 16, 21, 3, 198, 9, 256, 2091, 220, 16, 21, 10598, 430, 1599, 430, 220, 23, 30, 7179, 11, 393, 16, 21, 283, 220, 17, 1088, 14695, 220, 23, 12576, 271, 760, 1324, 314, 4022, 7397, 13853, 10471, 369, 220, 19, 13, 271, 794, 220, 19, 248046, 198, 248044, 248045, 248045, 248045, 248045, 248046, 198, 248044, 248045, 248045, 248045, 248046, 198, 248045, 846, 198, 2523, 513, 264, 16097, 321, 16908, 28016, 13, 248046, 198, 248045, 248068, 198, 760, 1156, 6587, 728, 310, 11290, 279, 6673, 3377, 883, 13853, 579, 7397, 34393, 13, 6558, 728, 944, 1472, 424, 15060, 13, 271, 6140, 1528, 25, 198, 12, 220, 18, 18016, 7397, 198, 12, 220, 19, 4022, 7397, 198, 12, 10342, 25, 220, 22, 7397, 271, 31318, 47866, 220, 16, 22, 7397, 494, 264, 9238, 12526, 13, 6558, 579, 1910, 551, 47866, 288, 4022, 7397, 321, 296, 18016, 7397, 11, 1332, 288, 478, 296, 283, 220, 16, 22, 13, 271, 5836, 7594, 1070, 310, 780, 34393, 25, 198, 12, 10342, 4022, 7397, 25, 220, 19, 478, 288, 198, 12, 10342, 18016, 7397, 25, 220, 18, 478, 296, 283, 220, 18, 478, 318, 16, 22, 471, 288, 8, 283, 220, 17, 15, 471, 288, 271, 10234, 25, 1216, 1381, 682, 10598, 430, 1599, 18016, 7397, 430, 4022, 7397, 13, 198, 4272, 25, 220, 17, 15, 471, 288, 283, 220, 17, 7, 19, 478, 288, 8, 198, 17, 15, 471, 288, 283, 220, 23, 478, 220, 17, 86, 198, 16, 17, 283, 220, 18, 86, 198, 86, 283, 220, 19, 271, 9764, 728, 9845, 25, 198, 12, 5626, 7397, 10471, 25, 220, 19, 198, 12, 21281, 7397, 10471, 25, 220, 16, 18, 198, 12, 12650, 4022, 25, 220, 19, 478, 220, 19, 283, 220, 23, 198, 12, 12650, 18016, 25, 220, 18, 478, 220, 16, 18, 283, 220, 16, 21, 198, 12, 220, 16, 21, 283, 220, 17, 23985, 220, 23, 50615, 271, 760, 4087, 369, 220, 19, 13, 198, 248069, 271, 2, 11144, 4261, 13853, 579, 16120, 21397, 271, 550, 19427, 3038, 279, 21397, 271, 332, 6140, 34393, 64700, 198, 12, 21281, 7397, 25, 220, 18, 198, 12, 5626, 7397, 25, 220, 19, 271, 332, 62360, 14307, 64700, 220, 16, 22, 2702, 318, 64, 6311, 314, 4022, 321, 18016, 8, 271, 9764, 728, 6762, 25, 198, 12, 393, 86, 3, 283, 1324, 314, 4022, 7397, 10471, 198, 12], "tokens": 876, "terminal": ["error"], "usage": [], "output_text_sha256": "3b403d820505dda9a0f8558c997d4c97d2e67976b752d6d90d12d6a315c48477", "wire_sha256": "f7c7fc174a0ccb05a660f825815ef04ac9e7d01b00e1564666e8b8486c73a3c9", "client_success": true, "client_error": false, "valid": false, "ttft_s": 2.548390226904303, "latency_s": 79.12946258590091, "itl": [0.08732773910742253, 0.08747876388952136, 0.08764748100657016, 0.08747163799125701, 0.08757982810493559, 0.08763381198514253, 0.08779059594962746, 0.08771897701080889, 0.08732472395058721, 0.08753265510313213, 0.08744183590169996, 0.0875676569994539, 0.08760834101121873, 0.08747871208470315, 0.08756826992612332, 0.08742380701005459, 0.08775191800668836, 0.08751101302914321, 0.0875256760045886, 0.08746595296543092, 0.08753192098811269, 0.08745995303615928, 0.0875094230286777, 0.08770611591171473, 0.08742739306762815, 0.08762387197930366, 0.08753805002197623, 0.08751656499225646, 0.0875699199968949, 0.08760620001703501, 0.08756005601026118, 0.08746427297592163, 0.08762127498630434, 0.08744117303285748, 0.08759432297665626, 0.08751948701683432, 0.08751240698620677, 0.08766530000139028, 0.0874172099865973, 0.08766257204115391, 0.08739562891423702, 0.08761859103105962, 0.08742888399865478, 0.08767536596860737, 0.0876235991017893, 0.08757944300305098, 0.08766956091858447, 0.08746526599861681, 0.08764784003142267, 0.08747005299665034, 0.08758250798564404, 0.08750280400272459, 0.087577361962758, 0.08751707000192255, 0.08748762810137123, 0.08760629198513925, 0.08748441294301301, 0.08763870305847377, 0.08755057991947979, 0.08756432705558836, 0.0875278579769656, 0.08753784594591707, 0.08756007300689816, 0.08750013110693544, 0.08759167895186692, 0.08746181498281658, 0.087534453952685, 0.08750484301708639, 0.08751249907072634, 0.08757503901142627, 0.08743152895476669, 0.08770805597305298, 0.08735046803485602, 0.08759504894260317, 0.08748094108887017, 0.087580312974751, 0.08751373004633933, 0.08753822790458798, 0.08758024906273931, 0.08744277199730277, 0.08753861696459353, 0.08754907397087663, 0.08761531603522599, 0.08753471705131233, 0.08759488491341472, 0.08752037107478827, 0.08751329500228167, 0.08763268392067403, 0.08740715100429952, 0.08762353798374534, 0.08745517104398459, 0.08763247798196971, 0.08751725405454636, 0.08755871292669326, 0.08756482100579888, 0.08745490806177258, 0.08762971393298358, 0.0873995030997321, 0.08751270198263228, 0.08750984200742096, 0.08745217998512089, 0.08752789197023958, 0.08745319000445306, 0.08762103295885026, 0.0878894260386005, 0.08763997303321958, 0.08745095191989094, 0.08756910008378327, 0.08746584900654852, 0.08750209398567677, 0.08751291397493333, 0.08745775301940739, 0.08765066997148097, 0.0874451290583238, 0.0875955019146204, 0.08747797505930066, 0.08755200693849474, 0.08768694999162108, 0.08762309607118368, 0.08760995301418006, 0.08742237801197916, 0.0876352199120447, 0.08749161497689784, 0.08761114603839815, 0.08748395997099578, 0.08753343508578837, 0.08758417400531471, 0.08751168590970337, 0.08756949205417186, 0.08738889603409916, 0.08757659001275897, 0.0874584469711408, 0.08749531500507146, 0.08759259595535696, 0.08747327607125044, 0.08753524196799845, 0.0874661459820345, 0.08770770998671651, 0.08753524697385728, 0.08751262002624571, 0.08761907799635082, 0.08751250896602869, 0.08759802300482988, 0.08742304099723697, 0.08766971901059151, 0.08731480897404253, 0.08755756402388215, 0.08741929801180959, 0.0875296910526231, 0.08748397696763277, 0.08747877494897693, 0.08758855005726218, 0.08745579200331122, 0.08768608898390085, 0.08744601102080196, 0.08761049900203943, 0.08744757296517491, 0.0874620369868353, 0.08766603900585324, 0.0874058500630781, 0.08771026693284512, 0.08754632400814444, 0.08757967897690833, 0.08746844704728574, 0.08750767994206399, 0.08750456909183413, 0.08743875997606665, 0.0876312890322879, 0.08743368496652693, 0.08762387000024319, 0.0874694639351219, 0.08770218200515956, 0.0874890040140599, 0.08753837808035314, 0.08757177297957242, 0.08745705091860145, 0.08753725606948137, 0.08745630702469498, 0.08758844900876284, 0.08748692693188787, 0.08758898603264242, 0.08753403695300221, 0.08747973700519651, 0.08760035107843578, 0.08736163296271116, 0.08766610093880445, 0.08744912105612457, 0.08762623299844563, 0.0874045769451186, 0.08753178711049259, 0.08753658889327198, 0.08744528703391552, 0.08761111903004348, 0.08748297300189734, 0.08765836292877793, 0.08741944108624011, 0.08754388592205942, 0.08759359002579004, 0.0875013810582459, 0.08756216301117092, 0.08744660497177392, 0.08761217992287129, 0.0874956970801577, 0.08758117991965264, 0.08751279604621232, 0.08754495799075812, 0.08758942503482103, 0.08743605203926563, 0.08761712990235537, 0.0874642290873453, 0.08753150794655085, 0.08752511802595109, 0.08748116402421147, 0.0875255250139162, 0.08751301595475525, 0.08796889195218682, 0.0874624770367518, 0.08763952401932329, 0.0875167689519003, 0.08751565706916153, 0.08746575401164591, 0.08752023591659963, 0.08753556408919394, 0.08750060899183154, 0.0876706229755655, 0.08740764099638909, 0.08762156800366938, 0.08745549398008734, 0.08755945705343038, 0.08758189890068024, 0.08742020407225937, 0.08761984494049102, 0.0874185289721936, 0.08771275810431689, 0.08758896600920707, 0.08770205196924508, 0.08750073402188718, 0.08756377396639436, 0.08755136898253113, 0.08746446506120265, 0.08762957295402884, 0.08746066596359015, 0.08754464902449399, 0.08749074896331877, 0.08752744505181909, 0.08753733593039215, 0.08747575699817389, 0.08757473202422261, 0.08745023200754076, 0.08768004307057709, 0.08745950798038393, 0.08757709001656622, 0.08752500300761312, 0.08747689193114638, 0.08769388601649553, 0.08744592405855656, 0.08771507593337446, 0.0874346459750086, 0.08764863607939333, 0.08752116898540407, 0.08752331393770874, 0.08748545206617564, 0.08750693302135915, 0.08756890892982483, 0.08748852601274848, 0.08763198798988014, 0.0874566079583019, 0.08757846604567021, 0.08744120795745403, 0.08757515100296587, 0.0875232300022617, 0.08743035001680255, 0.08760483097285032, 0.0873921950114891, 0.08753730403259397, 0.08747098303865641, 0.087501117028296, 0.08758417493663728, 0.08740325400140136, 0.08760089497081935, 0.0874368310905993, 0.08757469791453332, 0.0874950319994241, 0.08758330706041306, 0.08743307692930102, 0.08747706306166947, 0.08751541492529213, 0.0874766040360555, 0.08754146401770413, 0.08745152701158077, 0.08756039501167834, 0.08743140997830778, 0.08759990695398301, 0.08749519009143114, 0.08751298696734011, 0.08762406301684678, 0.08746317902114242, 0.08757487498223782, 0.08743287995457649, 0.08755684702191502, 0.08745588804595172, 0.08753510389942676, 0.08747181599028409, 0.08747224602848291, 0.08762658899649978, 0.08746751304715872, 0.08762893592938781, 0.08743517403490841, 0.08762236696202308, 0.08747656911145896, 0.08750524499919266, 0.08761713095009327, 0.08736663404852152, 0.08766140497755259, 0.08746172697283328, 0.08762489503715187, 0.08747654792387038, 0.08753304497804493, 0.08752498403191566, 0.08743841701652855, 0.08765277697239071, 0.08749278599862009, 0.08762852102518082, 0.08747739403042942, 0.08753815502859652, 0.08750889799557626, 0.08747829589992762, 0.08759619307238609, 0.08742825395893306, 0.08762804605066776, 0.08738562592770904, 0.08758366503752768, 0.08746373699977994, 0.0875505639705807, 0.08744998299516737, 0.08748495404142886, 0.0875735420268029, 0.08741955598816276, 0.087580855935812, 0.08744611009024084, 0.08760071301367134, 0.08747456595301628, 0.08745803195051849, 0.08758074208162725, 0.08743840898387134, 0.08760337193962187, 0.08737995498813689, 0.08753349410835654, 0.08749234396964312, 0.08749411394819617, 0.08761297003366053, 0.08769092394504696, 0.08756987599190325, 0.08733059302903712, 0.08754122699610889, 0.08758362801745534, 0.0874730380019173, 0.08750312798656523, 0.0875294579891488, 0.0876094059785828, 0.08729537308681756, 0.08760881901253015, 0.0874560319352895, 0.08757411304395646, 0.08749877393711358, 0.08743600407615304, 0.08752255991566926, 0.08753134310245514, 0.08753343392163515, 0.08742280502337962, 0.08758602803573012, 0.08741417899727821, 0.08751649793703109, 0.08748312608804554, 0.08766359789296985, 0.08773391309659928, 0.08750709891319275, 0.08764812909066677, 0.08744923793710768, 0.08753043506294489, 0.08740704890806228, 0.08757755998522043, 0.08755036804359406, 0.08746543095912784, 0.08758326701354235, 0.08741533104330301, 0.08756477700080723, 0.08747807296458632, 0.0875795140163973, 0.08752664504572749, 0.08755851094610989, 0.08757180301472545, 0.08736713498365134, 0.08757790899835527, 0.08742615696974099, 0.08752057002857327, 0.08750835608225316, 0.08751915988977998, 0.08761019702069461, 0.08743243908975273, 0.08757852599956095, 0.08741339296102524, 0.08767674793489277, 0.08756814210209996, 0.08763315691612661, 0.08743693900760263, 0.08748563006520271, 0.08753414498642087, 0.08740708197001368, 0.08755571802612394, 0.08752369997091591, 0.08754958095960319, 0.08747528109233826, 0.08753274194896221, 0.08754959900397807, 0.08750716899521649, 0.08768043702002615, 0.08724412601441145, 0.08758145698811859, 0.08746188203804195, 0.0876331179169938, 0.0874747020425275, 0.08754473400767893, 0.08759767701849341, 0.0874775139382109, 0.0875926730223, 0.08744935598224401, 0.08757500897627324, 0.08746275398880243, 0.08753680402878672, 0.08758786099497229, 0.08752415399067104, 0.08761736506130546, 0.08741942897904664, 0.08760497905313969, 0.08746571990195662, 0.0875545940361917, 0.08750770904589444, 0.08748753392137587, 0.08754286204930395, 0.08746733493171632, 0.08756412100046873, 0.08740115200635046, 0.0875468150479719, 0.08748873195145279, 0.08749103406444192, 0.08746682002674788, 0.08746151300147176, 0.08762199396733195, 0.08737671398557723, 0.08748057205229998, 0.08744475001003593, 0.08755273988936096, 0.0875398360658437, 0.08751972601749003, 0.08760412794072181, 0.08741077606100589, 0.08768388791941106, 0.08748888701666147, 0.08763098297640681, 0.08737753110472113, 0.08756801893468946, 0.08757776103448123, 0.08745719795115292, 0.08758684201166034, 0.08741298597306013, 0.0876254450995475, 0.08737748791463673, 0.08751049707643688, 0.08748139790259302, 0.08756080700550228, 0.08751195506192744, 0.0873695770278573, 0.08760671596974134, 0.08745036204345524, 0.08765834791120142, 0.08746202499605715, 0.08754559699445963, 0.08756270504090935, 0.08738266699947417, 0.08760483004152775, 0.08742135600186884, 0.08767173194792122, 0.08747480495367199, 0.08750217000488192, 0.0874750770162791, 0.08742304507177323, 0.08756907400675118, 0.08747079595923424, 0.08765053597744554, 0.08742665301542729, 0.08755539206322283, 0.0875328138936311, 0.08751698408741504, 0.08757969795260578, 0.08740776195190847, 0.08757824206259102, 0.08744809194467962, 0.08753169304691255, 0.08748103503603488, 0.08755805995315313, 0.08749306504614651, 0.08746937592513859, 0.08754070906434208, 0.08744109096005559, 0.08766249404288828, 0.08738035394344479, 0.08759821706917137, 0.08747343590948731, 0.08750594209413975, 0.08751353691332042, 0.08746262709610164, 0.0876053039683029, 0.08749217202421278, 0.08796962199267, 0.08738468599040061, 0.08759698597714305, 0.08750603697262704, 0.08745128102600574, 0.08758946997113526, 0.08741046604700387, 0.08759072900284082, 0.08738962793722749, 0.087581129046157, 0.08742915198672563, 0.08765672997105867, 0.08748330199159682, 0.08742513100150973, 0.08759353100322187, 0.08735376200638711, 0.08753694407641888, 0.08740461501292884, 0.08756034693215042, 0.0874818570446223, 0.08746240602340549, 0.08761888090521097, 0.08738822001032531, 0.08761731698177755, 0.08742945699486881, 0.08763877407182008, 0.08738992793951184, 0.08772586099803448, 0.08766810700763017, 0.08748388604726642, 0.08755620301235467, 0.08737685799133033, 0.08760300895664841, 0.08748700900468975, 0.08752903004642576, 0.08746278996113688, 0.08751420397311449, 0.0875410430599004, 0.08745140000246465, 0.08757268392946571, 0.08743480499833822, 0.0876424090238288, 0.08743827103171498, 0.08759554394055158, 0.08754310104995966, 0.08749567903578281, 0.08757223293650895, 0.08741919999010861, 0.08758931909687817, 0.08737003791611642, 0.08750412601511925, 0.08749065699521452, 0.08747503999620676, 0.08752989408094436, 0.0874110528966412, 0.08765069406945258, 0.08739090303424746, 0.08758712199050933, 0.08748048893176019, 0.08761010400485247, 0.08745765907224268, 0.08760295889806002, 0.0876688821008429, 0.08738448598887771, 0.08753300993703306, 0.08750851906370372, 0.08750898693688214, 0.08747676596976817, 0.08749813900794834, 0.08757401909679174, 0.0873581669293344, 0.08759335696231574, 0.08741817099507898, 0.08756573009304702, 0.0874631810002029, 0.08745913091115654, 0.08750883908942342, 0.0874649939360097, 0.08760240499395877, 0.08739917608909309, 0.0875848289579153, 0.08741630497388542, 0.08754277706611902, 0.08746708196122199, 0.08748712402302772, 0.08752524096053094, 0.08744144404772669, 0.08767484198324382, 0.08743641001638025, 0.08760364691261202, 0.08747931802645326, 0.08748867805115879, 0.08757878094911575, 0.08748534799087793, 0.08761393907479942, 0.08734676497988403, 0.08760853495914489, 0.08747147605754435, 0.08761023392435163, 0.0874163550324738, 0.08749072893988341, 0.08759044809266925, 0.08759360597468913, 0.08758975996170193, 0.08735290705226362, 0.08763234794605523, 0.08747728099115193, 0.08747986704111099, 0.08750424801837653, 0.08742913196329027, 0.08759042900055647, 0.08740806102287024, 0.08757531200535595, 0.08745985198765993, 0.08754867594689131, 0.08751461107749492, 0.08760573400650173, 0.0875042729312554, 0.08740355307236314, 0.08755851292517036, 0.08744486304931343, 0.08758204698096961, 0.08746780699584633, 0.08757512597367167, 0.08754287601914257, 0.08750033006072044, 0.08752823190297931, 0.08744185999967158, 0.08763625007122755, 0.08738602593075484, 0.08762338408268988, 0.08753615897148848, 0.08744665700942278, 0.0875903150299564, 0.08744223392568529, 0.08766544400714338, 0.08736555697396398, 0.08758884109556675, 0.08747239690274, 0.08750760601833463, 0.0874941700603813, 0.0874574159970507, 0.0875924009596929, 0.08743030799087137, 0.0876182250212878, 0.08743506600148976, 0.08751119405496866, 0.08753842196892947, 0.08753860392607749, 0.087527348077856, 0.08739836898166686, 0.08762343705166131, 0.08750018698628992, 0.0875574549427256, 0.08746851503383368, 0.08762177894823253, 0.08751375298015773, 0.08746737707406282, 0.08746595296543092, 0.0873570820549503, 0.08757120091468096, 0.08740517299156636, 0.08759740903042257, 0.0875232300022617, 0.08748225204180926, 0.08753293799236417, 0.08740806893911213, 0.08765531203243881, 0.08743974403478205, 0.08758674201089889, 0.08744215394835919, 0.08749086607713252, 0.08760031894780695, 0.08744211099110544, 0.08751896501053125, 0.08735142694786191, 0.0875851270975545, 0.08741769893094897, 0.08756966004148126, 0.08741669892333448, 0.08748945407569408, 0.0875570309581235, 0.08739371201954782, 0.08761081099510193, 0.08738716598600149, 0.08766286296304315, 0.08737577509600669, 0.08755882293917239, 0.08748532400932163, 0.08749148098286241, 0.08764204604085535, 0.08741720695979893, 0.08760052197612822, 0.08745025901589543, 0.08755953307263553, 0.0875664500053972, 0.08783256192691624, 0.08761075499933213, 0.08737370499875396, 0.08750922908075154, 0.08741088293027133, 0.08748488896526396, 0.08745197800453752, 0.08746370603330433, 0.08754166204016656, 0.08741383999586105, 0.08757823903579265, 0.08740315400063992, 0.08763927593827248, 0.08736791706178337, 0.08753534790594131, 0.08747926505748183, 0.08755246200598776, 0.08750510693062097, 0.08742774999700487, 0.08761919499374926, 0.0874146850546822, 0.08760806196369231, 0.0874616940272972, 0.08757918502669781, 0.08747019001748413, 0.08749921293929219, 0.08754957199562341, 0.08740248600952327, 0.08757245203014463, 0.08736208395566791, 0.08755202800966799, 0.08745819004252553, 0.08768622495699674, 0.08766896999441087, 0.08740835904609412, 0.08767739299219102, 0.08740607998333871, 0.08751865697558969, 0.08752963203005493, 0.08752544596791267, 0.08745768805965781, 0.08738361194264144, 0.08763069903943688, 0.08741653396282345, 0.08756752801127732, 0.087492706021294, 0.08762395603116602, 0.08746503700967878, 0.0874015640001744, 0.08756297791842371, 0.08754015003796667, 0.08750430995132774, 0.08747616701293737, 0.08751098206266761, 0.0874874759465456, 0.08748979005031288, 0.0874722070293501, 0.08749438589438796, 0.08744788100011647, 0.08733739308081567, 0.08755603898316622, 0.08745791204273701, 0.0875488129677251, 0.08751385193318129, 0.08746371604502201, 0.08752815204206854, 0.0874081589281559, 0.08760821807663888, 0.08742739295121282, 0.08763231500051916, 0.08743667998351157, 0.08749312197323889, 0.0875465200515464, 0.08757431199774146, 0.0877562389941886, 0.0874239329714328, 0.08772440406028181, 0.0874499250203371, 0.08755683898925781, 0.08750130597036332, 0.08751848700921983, 0.08765458094421774, 0.08737967407796532, 0.08767891896422952, 0.08740745694376528, 0.08764240110758692, 0.08744098292663693, 0.08746528206393123, 0.08752515690866858, 0.08746574399992824, 0.08756958309095353, 0.08747333090286702, 0.08751018904149532, 0.08742950204759836, 0.08758181799203157, 0.08755167096387595, 0.0875827840063721, 0.08755622897297144, 0.08739910903386772, 0.08752168598584831, 0.08740499604027718, 0.08750022097956389, 0.08752194594126195, 0.08752211602404714, 0.08753697597421706, 0.08742143609561026, 0.0875701199984178, 0.08745941496454179, 0.0876587339444086, 0.08739894209429622, 0.08761257398873568, 0.08748401899356395, 0.08750157593749464, 0.08759811997879297, 0.08753414801321924, 0.08753680507652462, 0.08737205993384123, 0.0875741810305044, 0.08743362105451524, 0.08750716294161975, 0.0874416190199554, 0.08741929498501122, 0.08757185796275735, 0.08737900399137288, 0.08762981800828129, 0.0874585930723697, 0.08752539195120335, 0.0875884119886905, 0.08742583205457777, 0.08749787393026054, 0.08742311201058328, 0.08760456705931574, 0.08742121898103505, 0.08763563900720328, 0.08748077997006476, 0.08749242697376758, 0.08750560402404517, 0.08752100798301399, 0.08767573104705662, 0.08739739493466914, 0.0875491350889206, 0.08742179395630956, 0.08754254598170519, 0.08743080298881978, 0.08745911600999534, 0.08756615803577006, 0.08740783296525478, 0.08755439007654786, 0.08739482890814543, 0.08749993401579559, 0.08745553507469594, 0.08752509497571737, 0.08755394502077252, 0.08746329392306507, 0.08752454700879753, 0.08738055499270558, 0.08761618903372437, 0.08740890095941722, 0.08753287908621132, 0.08757308393251151, 0.08741622499655932, 0.08756101096514612, 0.08736338303424418, 0.08753550297114998, 0.08749105199240148, 6.824999582022429e-05], "tpot_s": 0.08752122555313897, "active_requests_after": 0, "runner_state_count_after": 0} -{"phase": "error", "arm": "graph", "exception": "RuntimeError"} -{"phase": "cleanup", "arm": "graph", "exit_code": 0, "forced": false, "closed": true, "active_requests": 0, "runner_state_count": 0} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/graph-prefill-repair.jsonl b/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/graph-prefill-repair.jsonl deleted file mode 100644 index 4cd8db962..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/graph-prefill-repair.jsonl +++ /dev/null @@ -1,57 +0,0 @@ -{"phase": "identity", "python": "3.12.13 (main, May 10 2026, 19:30:01) [Clang 22.1.3 ]", "torch": "2.14.1+cu130", "cuda": "13.0", "device": "NVIDIA H20-3e", "capability": [9, 0], "dependencies": {"triton": "3.8.0", "transformers": "5.20.2", "flashinfer-python": null, "safetensors": "0.8.0"}} -{"phase": "loaded", "seconds": 15.517316329060122, "report": {"loaded_parameters": 1251, "excluded_tensors": {"vision": 333, "mtp": 22}, "scope": "text_only", "dtype": "fp8", "activation_dtype": "bfloat16", "weight_dtype": "float8_e4m3fn", "scale_dtype": "float32", "checkpoint_scale_dtype": "bfloat16", "quantization": {"activation_scheme": "dynamic", "fmt": "e4m3", "quant_method": "fp8", "weight_block_size": [128, 128]}, "execution": "native_fp8", "fp8_linear_modules": 400, "dequantized": false, "backend": "triton_block_fp8", "kernel_name": "qwen38_block_fp8"}} -{"phase": "prefill", "case": "gsm8k/test/783", "prompt_tokens": 149, "capacity": 1024, "prompt_sha256": "f2233be4ebb7b122d80433fe4081fdabeda64f735b7bc24f9706c4c4339b824f", "compact_buffered": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "buffered_repeated": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [1206, 1206, 1206], "layer_diff": [], "state_diff": []} -{"phase": "capture_restore", "case": "gsm8k/test/783", "state_diff": []} -{"phase": "decode", "case": "gsm8k/test/783", "step": 0, "teacher_token": 1206, "compact_buffered": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [11290, 11290, 11290], "layer_diff": [], "state_diff": []} -{"phase": "decode", "case": "gsm8k/test/783", "step": 1, "teacher_token": 11290, "compact_buffered": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [411, 411, 411], "layer_diff": [], "state_diff": []} -{"phase": "decode", "case": "gsm8k/test/783", "step": 2, "teacher_token": 411, "compact_buffered": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [3377, 3377, 3377], "layer_diff": [], "state_diff": []} -{"phase": "decode", "case": "gsm8k/test/783", "step": 3, "teacher_token": 3377, "compact_buffered": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [11, 11, 11], "layer_diff": [], "state_diff": []} -{"phase": "decode", "case": "gsm8k/test/783", "step": 4, "teacher_token": 11, "compact_buffered": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [567, 567, 567], "layer_diff": [], "state_diff": []} -{"phase": "decode", "case": "gsm8k/test/783", "step": 5, "teacher_token": 567, "compact_buffered": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [1144, 1144, 1144], "layer_diff": [], "state_diff": []} -{"phase": "decode", "case": "gsm8k/test/783", "step": 6, "teacher_token": 1144, "compact_buffered": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [310, 310, 310], "layer_diff": [], "state_diff": []} -{"phase": "decode", "case": "gsm8k/test/783", "step": 7, "teacher_token": 310, "compact_buffered": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [7995, 7995, 7995], "layer_diff": [], "state_diff": []} -{"phase": "decode", "case": "gsm8k/test/783", "step": 8, "teacher_token": 7995, "compact_buffered": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [279, 279, 279], "layer_diff": [], "state_diff": []} -{"phase": "decode", "case": "gsm8k/test/783", "step": 9, "teacher_token": 279, "compact_buffered": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [1324, 1324, 1324], "layer_diff": [], "state_diff": []} -{"phase": "decode", "case": "gsm8k/test/783", "step": 10, "teacher_token": 1324, "compact_buffered": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [314, 314, 314], "layer_diff": [], "state_diff": []} -{"phase": "decode", "case": "gsm8k/test/783", "step": 11, "teacher_token": 314, "compact_buffered": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [4022, 4022, 4022], "layer_diff": [], "state_diff": []} -{"phase": "decode", "case": "gsm8k/test/783", "step": 12, "teacher_token": 4022, "compact_buffered": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [7397, 7397, 7397], "layer_diff": [], "state_diff": []} -{"phase": "decode", "case": "gsm8k/test/783", "step": 13, "teacher_token": 7397, "compact_buffered": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [13853, 13853, 13853], "layer_diff": [], "state_diff": []} -{"phase": "decode", "case": "gsm8k/test/783", "step": 14, "teacher_token": 13853, "compact_buffered": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [10471, 10471, 10471], "layer_diff": [], "state_diff": []} -{"phase": "decode", "case": "gsm8k/test/783", "step": 15, "teacher_token": 10471, "compact_buffered": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [3018, 3018, 3018], "layer_diff": [], "state_diff": []} -{"phase": "prefill", "case": "gsm8k/test/1", "prompt_tokens": 70, "capacity": 1024, "prompt_sha256": "78d335abe81363d4bb48e5aa256c120344c2ceadbcf8739afe4ec3923c747238", "compact_buffered": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "buffered_repeated": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [1206, 1206, 1206], "layer_diff": [], "state_diff": []} -{"phase": "capture_restore", "case": "gsm8k/test/1", "state_diff": []} -{"phase": "decode", "case": "gsm8k/test/1", "step": 0, "teacher_token": 1206, "compact_buffered": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [7995, 7995, 7995], "layer_diff": [], "state_diff": []} -{"phase": "decode", "case": "gsm8k/test/1", "step": 1, "teacher_token": 7995, "compact_buffered": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [279, 279, 279], "layer_diff": [], "state_diff": []} -{"phase": "decode", "case": "gsm8k/test/1", "step": 2, "teacher_token": 279, "compact_buffered": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [2702, 2702, 2702], "layer_diff": [], "state_diff": []} -{"phase": "decode", "case": "gsm8k/test/1", "step": 3, "teacher_token": 2702, "compact_buffered": {"exact": false, "max_abs": 0.265625, "nrmse": 0.016974342986941338}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [1324, 1324, 1324], "layer_diff": [{"layer": 51, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.004192069172859192}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 52, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.010037201456725597}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 53, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.014402993954718113}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 54, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.016260989010334015}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 55, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.018922777846455574}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 56, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.02012086659669876}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 57, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.021113229915499687}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 58, "compact_buffered": {"exact": false, "max_abs": 0.34375, "nrmse": 0.022059498354792595}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 59, "compact_buffered": {"exact": false, "max_abs": 0.40625, "nrmse": 0.023631153628230095}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 60, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.022782977670431137}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 61, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.023218752816319466}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 62, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.022562820464372635}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 63, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.026348838582634926}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}], "state_diff": []} -{"phase": "decode", "case": "gsm8k/test/1", "step": 4, "teacher_token": 1324, "compact_buffered": {"exact": false, "max_abs": 0.32421875, "nrmse": 0.013844876550137997}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [314, 314, 314], "layer_diff": [{"layer": 52, "compact_buffered": {"exact": false, "max_abs": 0.0703125, "nrmse": 0.007360011339187622}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 53, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.012917333282530308}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 54, "compact_buffered": {"exact": false, "max_abs": 0.1875, "nrmse": 0.01536696869879961}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 55, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.018100781366229057}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 56, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.019300080835819244}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 57, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.021108075976371765}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 58, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.02060718648135662}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 59, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.022297820076346397}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 60, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.0231693796813488}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 61, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.022321078926324844}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 62, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.025107813999056816}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 63, "compact_buffered": {"exact": false, "max_abs": 6.0, "nrmse": 0.02903517335653305}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}], "state_diff": []} -{"phase": "decode", "case": "gsm8k/test/1", "step": 5, "teacher_token": 314, "compact_buffered": {"exact": false, "max_abs": 0.34375, "nrmse": 0.017295168712735176}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [47205, 47205, 47205], "layer_diff": [{"layer": 52, "compact_buffered": {"exact": false, "max_abs": 0.125, "nrmse": 0.008268028497695923}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 53, "compact_buffered": {"exact": false, "max_abs": 0.125, "nrmse": 0.013479012064635754}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 54, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.017548559233546257}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 55, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.020484568551182747}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 56, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.02205175720155239}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 57, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.02284286357462406}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 58, "compact_buffered": {"exact": false, "max_abs": 0.375, "nrmse": 0.0230034738779068}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 59, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.024906473234295845}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 60, "compact_buffered": {"exact": false, "max_abs": 2.0, "nrmse": 0.02467332035303116}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 61, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.024097029119729996}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 62, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.025757284834980965}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 63, "compact_buffered": {"exact": false, "max_abs": 5.0, "nrmse": 0.03440675511956215}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}], "state_diff": []} -{"phase": "decode", "case": "gsm8k/test/1", "step": 6, "teacher_token": 47205, "compact_buffered": {"exact": false, "max_abs": 0.265625, "nrmse": 0.01655212603509426}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [2483, 2483, 2483], "layer_diff": [{"layer": 52, "compact_buffered": {"exact": false, "max_abs": 0.125, "nrmse": 0.006808439269661903}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 53, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.01196344569325447}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 54, "compact_buffered": {"exact": false, "max_abs": 0.1875, "nrmse": 0.014859169721603394}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 55, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.017450500279664993}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 56, "compact_buffered": {"exact": false, "max_abs": 0.375, "nrmse": 0.01859382353723049}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 57, "compact_buffered": {"exact": false, "max_abs": 0.28125, "nrmse": 0.019558457657694817}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 58, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.021031705662608147}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 59, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.02225661091506481}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 60, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.021312648430466652}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 61, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.021911388263106346}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 62, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.022254616022109985}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 63, "compact_buffered": {"exact": false, "max_abs": 2.0, "nrmse": 0.025984283536672592}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}], "state_diff": []} -{"phase": "decode", "case": "gsm8k/test/1", "step": 7, "teacher_token": 2483, "compact_buffered": {"exact": false, "max_abs": 0.21875, "nrmse": 0.015436816029250622}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [364, 364, 364], "layer_diff": [{"layer": 52, "compact_buffered": {"exact": false, "max_abs": 0.125, "nrmse": 0.0071892449632287025}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 53, "compact_buffered": {"exact": false, "max_abs": 0.1875, "nrmse": 0.011414185166358948}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 54, "compact_buffered": {"exact": false, "max_abs": 0.75, "nrmse": 0.014215651899576187}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 55, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.016584554687142372}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 56, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.01789895072579384}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 57, "compact_buffered": {"exact": false, "max_abs": 0.3125, "nrmse": 0.01861792430281639}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 58, "compact_buffered": {"exact": false, "max_abs": 1.5, "nrmse": 0.019392333924770355}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 59, "compact_buffered": {"exact": false, "max_abs": 2.0, "nrmse": 0.02123606577515602}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 60, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.020399296656250954}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 61, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.020752865821123123}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 62, "compact_buffered": {"exact": false, "max_abs": 0.75, "nrmse": 0.02124437503516674}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 63, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.022680403664708138}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}], "state_diff": []} -{"phase": "decode", "case": "gsm8k/test/1", "step": 8, "teacher_token": 364, "compact_buffered": {"exact": false, "max_abs": 0.20703125, "nrmse": 0.014042757451534271}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [279, 279, 279], "layer_diff": [{"layer": 52, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.006374783348292112}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 53, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.01039191149175167}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 54, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.01265820674598217}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 55, "compact_buffered": {"exact": false, "max_abs": 1.5, "nrmse": 0.016201341524720192}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 56, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.01660863682627678}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 57, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.017436636611819267}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 58, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.017304876819252968}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 59, "compact_buffered": {"exact": false, "max_abs": 0.375, "nrmse": 0.01859012246131897}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 60, "compact_buffered": {"exact": false, "max_abs": 2.0, "nrmse": 0.01936807483434677}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 61, "compact_buffered": {"exact": false, "max_abs": 3.0, "nrmse": 0.020450066775083542}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 62, "compact_buffered": {"exact": false, "max_abs": 3.0, "nrmse": 0.020603898912668228}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 63, "compact_buffered": {"exact": false, "max_abs": 0.75, "nrmse": 0.01912855915725231}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}], "state_diff": []} -{"phase": "decode", "case": "gsm8k/test/1", "step": 9, "teacher_token": 279, "compact_buffered": {"exact": false, "max_abs": 0.40625, "nrmse": 0.0230456180870533}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [60507, 60507, 60507], "layer_diff": [{"layer": 52, "compact_buffered": {"exact": false, "max_abs": 0.125, "nrmse": 0.00691695511341095}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 53, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.011661535128951073}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 54, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.015562513843178749}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 55, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.01798592507839203}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 56, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.019519701600074768}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 57, "compact_buffered": {"exact": false, "max_abs": 0.375, "nrmse": 0.019780775532126427}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 58, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.01883254013955593}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 59, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.02334459498524666}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 60, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.02176261693239212}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 61, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.02199896238744259}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 62, "compact_buffered": {"exact": false, "max_abs": 0.546875, "nrmse": 0.022038675844669342}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 63, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.02503371424973011}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}], "state_diff": []} -{"phase": "decode", "case": "gsm8k/test/1", "step": 10, "teacher_token": 60507, "compact_buffered": {"exact": false, "max_abs": 0.265625, "nrmse": 0.016168558970093727}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [11, 11, 11], "layer_diff": [{"layer": 52, "compact_buffered": {"exact": false, "max_abs": 0.125, "nrmse": 0.007084327284246683}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 53, "compact_buffered": {"exact": false, "max_abs": 0.1875, "nrmse": 0.012760188430547714}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 54, "compact_buffered": {"exact": false, "max_abs": 0.1875, "nrmse": 0.015312744304537773}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 55, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.01736706681549549}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 56, "compact_buffered": {"exact": false, "max_abs": 0.375, "nrmse": 0.019020091742277145}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 57, "compact_buffered": {"exact": false, "max_abs": 0.2890625, "nrmse": 0.019861312583088875}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 58, "compact_buffered": {"exact": false, "max_abs": 0.75, "nrmse": 0.021342990919947624}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 59, "compact_buffered": {"exact": false, "max_abs": 1.5, "nrmse": 0.020473739132285118}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 60, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.021089166402816772}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 61, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.020610515028238297}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 62, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.023008685559034348}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 63, "compact_buffered": {"exact": false, "max_abs": 2.0, "nrmse": 0.02467254549264908}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}], "state_diff": []} -{"phase": "decode", "case": "gsm8k/test/1", "step": 11, "teacher_token": 11, "compact_buffered": {"exact": false, "max_abs": 0.34765625, "nrmse": 0.0336960107088089}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [567, 567, 567], "layer_diff": [{"layer": 52, "compact_buffered": {"exact": false, "max_abs": 0.125, "nrmse": 0.007425264455378056}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 53, "compact_buffered": {"exact": false, "max_abs": 0.1875, "nrmse": 0.013792325742542744}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 54, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.016844570636749268}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 55, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.019499080255627632}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 56, "compact_buffered": {"exact": false, "max_abs": 0.28125, "nrmse": 0.021456947550177574}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 57, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.023110872134566307}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 58, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.022726301103830338}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 59, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.02432378940284252}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 60, "compact_buffered": {"exact": false, "max_abs": 0.75, "nrmse": 0.023974517360329628}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 61, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.024591434746980667}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 62, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.02398037165403366}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 63, "compact_buffered": {"exact": false, "max_abs": 4.0, "nrmse": 0.03175168111920357}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}], "state_diff": []} -{"phase": "decode", "case": "gsm8k/test/1", "step": 12, "teacher_token": 567, "compact_buffered": {"exact": false, "max_abs": 0.3125, "nrmse": 0.025021051988005638}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [1144, 1144, 1144], "layer_diff": [{"layer": 52, "compact_buffered": {"exact": false, "max_abs": 0.06640625, "nrmse": 0.007481100969016552}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 53, "compact_buffered": {"exact": false, "max_abs": 0.75, "nrmse": 0.01473709475249052}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 54, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.017887933179736137}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 55, "compact_buffered": {"exact": false, "max_abs": 0.23046875, "nrmse": 0.020596858114004135}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 56, "compact_buffered": {"exact": false, "max_abs": 0.375, "nrmse": 0.022472470998764038}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 57, "compact_buffered": {"exact": false, "max_abs": 0.3125, "nrmse": 0.023566467687487602}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 58, "compact_buffered": {"exact": false, "max_abs": 0.375, "nrmse": 0.02452041208744049}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 59, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.025468094274401665}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 60, "compact_buffered": {"exact": false, "max_abs": 0.375, "nrmse": 0.025508418679237366}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 61, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.026130542159080505}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 62, "compact_buffered": {"exact": false, "max_abs": 2.5, "nrmse": 0.02776006981730461}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 63, "compact_buffered": {"exact": false, "max_abs": 5.0, "nrmse": 0.032615285366773605}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}], "state_diff": []} -{"phase": "decode", "case": "gsm8k/test/1", "step": 13, "teacher_token": 1144, "compact_buffered": {"exact": false, "max_abs": 0.375, "nrmse": 0.017284702509641647}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [310, 310, 310], "layer_diff": [{"layer": 52, "compact_buffered": {"exact": false, "max_abs": 0.0625, "nrmse": 0.005684087052941322}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 53, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.011953089386224747}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 54, "compact_buffered": {"exact": false, "max_abs": 0.203125, "nrmse": 0.01461456436663866}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 55, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.017888935282826424}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 56, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.019925637170672417}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 57, "compact_buffered": {"exact": false, "max_abs": 0.26171875, "nrmse": 0.021050218492746353}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 58, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.02201216295361519}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 59, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.021176602691411972}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 60, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.022189626470208168}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 61, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.022641409188508987}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 62, "compact_buffered": {"exact": false, "max_abs": 2.0, "nrmse": 0.025658436119556427}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 63, "compact_buffered": {"exact": false, "max_abs": 2.0, "nrmse": 0.02625197544693947}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}], "state_diff": []} -{"phase": "decode", "case": "gsm8k/test/1", "step": 14, "teacher_token": 310, "compact_buffered": {"exact": false, "max_abs": 0.2734375, "nrmse": 0.022608842700719833}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [10724, 10724, 10724], "layer_diff": [{"layer": 52, "compact_buffered": {"exact": false, "max_abs": 0.125, "nrmse": 0.006451616995036602}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 53, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.012466785497963428}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 54, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.015781251713633537}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 55, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.017663585022091866}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 56, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.019764920696616173}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 57, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.020468920469284058}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 58, "compact_buffered": {"exact": false, "max_abs": 0.3203125, "nrmse": 0.02147844433784485}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 59, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.02334902063012123}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 60, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.023049885407090187}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 61, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.02460392750799656}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 62, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.026219181716442108}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 63, "compact_buffered": {"exact": false, "max_abs": 2.0, "nrmse": 0.025661209598183632}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}], "state_diff": []} -{"phase": "decode", "case": "gsm8k/test/1", "step": 15, "teacher_token": 10724, "compact_buffered": {"exact": false, "max_abs": 0.28125, "nrmse": 0.013467720709741116}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [279, 279, 279], "layer_diff": [{"layer": 52, "compact_buffered": {"exact": false, "max_abs": 0.25, "nrmse": 0.005542002618312836}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 53, "compact_buffered": {"exact": false, "max_abs": 0.125, "nrmse": 0.010796396993100643}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 54, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.015289975330233574}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 55, "compact_buffered": {"exact": false, "max_abs": 0.75, "nrmse": 0.017505381256341934}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 56, "compact_buffered": {"exact": false, "max_abs": 0.75, "nrmse": 0.018648486584424973}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 57, "compact_buffered": {"exact": false, "max_abs": 0.75, "nrmse": 0.019693171605467796}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 58, "compact_buffered": {"exact": false, "max_abs": 3.0, "nrmse": 0.02352668158710003}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 59, "compact_buffered": {"exact": false, "max_abs": 0.5, "nrmse": 0.019833454862236977}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 60, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.021107729524374008}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 61, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.021489834412932396}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 62, "compact_buffered": {"exact": false, "max_abs": 1.0, "nrmse": 0.023015763610601425}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}, {"layer": 63, "compact_buffered": {"exact": false, "max_abs": 4.0, "nrmse": 0.026564551517367363}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}}], "state_diff": []} -{"phase": "prefill", "case": "gsm8k/test/424", "prompt_tokens": 106, "capacity": 1024, "prompt_sha256": "03cbdf5804bab62d4efbc0ae89e191bd557840c6d9a1a5ada9669de953a0b783", "compact_buffered": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "buffered_repeated": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [1206, 1206, 1206], "layer_diff": [], "state_diff": []} -{"phase": "capture_restore", "case": "gsm8k/test/424", "state_diff": []} -{"phase": "decode", "case": "gsm8k/test/424", "step": 0, "teacher_token": 1206, "compact_buffered": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [7995, 7995, 7995], "layer_diff": [], "state_diff": []} -{"phase": "decode", "case": "gsm8k/test/424", "step": 1, "teacher_token": 7995, "compact_buffered": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [279, 279, 279], "layer_diff": [], "state_diff": []} -{"phase": "decode", "case": "gsm8k/test/424", "step": 2, "teacher_token": 279, "compact_buffered": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [2702, 2702, 2702], "layer_diff": [], "state_diff": []} -{"phase": "decode", "case": "gsm8k/test/424", "step": 3, "teacher_token": 2702, "compact_buffered": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [1324, 1324, 1324], "layer_diff": [], "state_diff": []} -{"phase": "decode", "case": "gsm8k/test/424", "step": 4, "teacher_token": 1324, "compact_buffered": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [314, 314, 314], "layer_diff": [], "state_diff": []} -{"phase": "decode", "case": "gsm8k/test/424", "step": 5, "teacher_token": 314, "compact_buffered": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [7998, 7998, 7998], "layer_diff": [], "state_diff": []} -{"phase": "decode", "case": "gsm8k/test/424", "step": 6, "teacher_token": 7998, "compact_buffered": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [421, 421, 421], "layer_diff": [], "state_diff": []} -{"phase": "decode", "case": "gsm8k/test/424", "step": 7, "teacher_token": 421, "compact_buffered": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [628, 628, 628], "layer_diff": [], "state_diff": []} -{"phase": "decode", "case": "gsm8k/test/424", "step": 8, "teacher_token": 628, "compact_buffered": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [4973, 4973, 4973], "layer_diff": [], "state_diff": []} -{"phase": "decode", "case": "gsm8k/test/424", "step": 9, "teacher_token": 4973, "compact_buffered": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [279, 279, 279], "layer_diff": [], "state_diff": []} -{"phase": "decode", "case": "gsm8k/test/424", "step": 10, "teacher_token": 279, "compact_buffered": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [1834, 1834, 1834], "layer_diff": [], "state_diff": []} -{"phase": "decode", "case": "gsm8k/test/424", "step": 11, "teacher_token": 1834, "compact_buffered": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [8151, 8151, 8151], "layer_diff": [], "state_diff": []} -{"phase": "decode", "case": "gsm8k/test/424", "step": 12, "teacher_token": 8151, "compact_buffered": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [11, 11, 11], "layer_diff": [], "state_diff": []} -{"phase": "decode", "case": "gsm8k/test/424", "step": 13, "teacher_token": 11, "compact_buffered": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [567, 567, 567], "layer_diff": [], "state_diff": []} -{"phase": "decode", "case": "gsm8k/test/424", "step": 14, "teacher_token": 567, "compact_buffered": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [1144, 1144, 1144], "layer_diff": [], "state_diff": []} -{"phase": "decode", "case": "gsm8k/test/424", "step": 15, "teacher_token": 1144, "compact_buffered": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "buffered_graph": {"exact": true, "max_abs": 0.0, "nrmse": 0.0}, "top1": [310, 310, 310], "layer_diff": [], "state_diff": []} -{"phase": "complete", "scope": "diagnostic_only", "verdict": "INCONCLUSIVE", "seconds": 53.73372380202636} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/http-impact.md b/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/http-impact.md deleted file mode 100644 index 9de908786..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/http-impact.md +++ /dev/null @@ -1,70 +0,0 @@ -# HTTP impact of the bounded publication repair - -Acceptance: https://github.com/InferMatrixLab/InferRSI/issues/67 -Campaign verdict: **INCONCLUSIVE**. This is a completed readiness comparison, -not the final benchmark or task-quality evaluation. - -Baseline engine: `83b567c1b07d9a230f6a8ca8bff979144d719241`. -Publication candidate: `3e8636f04378ef7d316272f2051315515a9ac6b9`. -Commit `84eaffd` validates FP8 selection and discloses actual dispatch/counter -scope; commit `3e8636f` preserves initial-prefill operands with reserved KV. -These commits were tested together. The measurement does not estimate an -independent incremental speed effect for the provenance-only change. - -The publication branch was reconstructed from main to exclude legacy HTTP API -changes and tolerance adjustments. Model, quant, graph, runner and engine bytes -match the previously reviewed implementation; server and model-test definitions -differ. Earlier HTTP/CPU totals therefore cannot substitute for this run. - -## Observations - -Job 1943 completed with exit 0 in 144 allocated GPU seconds (0.04 GPU-hours). -Each of three service arms returned 16 tokens for all six requests, with valid -terminal framing, no active request/state left after each request, and graceful -process exit 0 without forced kills. All GPU resources were released. - -Original-main Graph differs from repaired eager in both repetitions of case783 -(4/6 request outputs equal). Repaired Graph matches repaired eager exactly in -all six 16-token sequences, including pooled Graph reuse. This repairs the -observed short-output mismatch. It does not prove general Graph correctness; -the earlier diagnostic still records 13/48 nonidentical compact/fixed decode -logits, and long generation, bucket transition and task accuracy remain untested. - -All 18 raw request samples are retained in http-readiness.jsonl and exported -without exclusions in http-timings.csv. Descriptive per-request ranges for -original Graph -> repaired Graph are: - -| Measurement | Original Graph | Repaired Graph | Paired candidate/base range | -| --- | --- | --- | --- | -| Token-event TTFT | 1.2381–2.6922 s | 1.2305–2.6411 s | 0.9575–1.0166 | -| Token-event TPOT | 87.984–88.228 ms | 87.438–87.670 ms | 0.9931–0.9959 | -| Complete HTTP latency | 2.5616–4.0132 s | 2.5457–3.9535 s | See all six pairs in http-summary.json | - -These small changes are **not established acceleration**: fixed engine order, -only two repetitions, short prompts/caps, cache and first-use effects, and a -different output sequence on the failing base case limit attribution. Startup -was 41.70 / 23.86 / 13.44 seconds for original Graph / repaired Graph / eager; -shared compilation caches make those startup figures noncomparable. -The observer's token-ID event clock differs from official benchmark semantics. -No vLLM service ran in this probe. The main HTTP server still rejects -ignore_eos, return_token_ids and stream_options; that prerequisite is tracked -separately and is not hidden by the passing short-response check. - -## Checks and retained failures - -Exact publication-head full Qwen CPU job 1944: **155 passed, 1 skipped, 1 failed**. -The failed production-head BF16 recurrent-state assertion reports 12/49,152 -elements, maximum absolute difference 1.5277801139745861e-6 against atol1e-6. -Unmodified-main targeted job 1945 reproduces exactly the same failure and has -one passing FP32 variant. No tolerance was relaxed and the suite is not green. -New prefill/backend tests passed. The skipped test needs the separately gated -GPU contract environment. Local runner/service/entry tests: 58 passed. - -Raw CPU logs remain in controlled storage; their digests and exact counts are -in http-summary.json. Independent reviewer review_fp8_graph found no introduced -blocker and confirmed this inherited failure does not alone block the bounded -repair; actual required repository checks must still be satisfied. - -Budget: the new 44 GPU-hour ceiling has consumed 0.04 GPU-hours in readiness; -1.96 of the 2 readiness GPU-hours remain. Earlier 147 GPU seconds belong to the -separately approved diagnostic allowance. Search and finalization are unused. diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/http-readiness.jsonl b/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/http-readiness.jsonl deleted file mode 100644 index 044129ffe..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/http-readiness.jsonl +++ /dev/null @@ -1,29 +0,0 @@ -{"phase": "protocol", "cases": ["gsm8k/test/783", "gsm8k/test/1", "gsm8k/test/424"], "repetitions": 2, "max_tokens": 16, "natural_eos": true, "fixture_sha256": "f359c66f92b69d9a92a66564d8a087809dbc45b7e8b9e916229065806c4c188c", "client_sha256": "e7c5d58dcdfbfd40ceaa595fc0799bcb68f16823534d595f49a1ae354de0e069", "scope": "readiness_only_custom_token_event_HTTP_timer_not_vLLM_acceptance"} -{"phase": "source", "arm": "main_graph", "graph": true, "hashes": {"nanovllm/qwen38/__init__.py": "1f07e0cfd12bbf370c1f465c6a4bd73e72a5c826af1f29e60ccc7d6f45562cfc", "nanovllm/qwen38/__main__.py": "9e2a0881eedf8fa4abc6037511b42fd3a44171d121a7b2cb096023f485673ba2", "nanovllm/qwen38/_triton_fp8.py": "9f0cb68be017cb3858aebe8ba275039084e305df92cb4567d6e7da54d119f19e", "nanovllm/qwen38/_triton_rms.py": "ba99fa3ba82bfcab4cd602fd0db907acad0eee7c7a63ab23d560a17fd2f72272", "nanovllm/qwen38/config.py": "002ea25bbf9c8d17e5f1d3fb2b76c400e4291d7d2d06d5dd13fb5c3ac15c3d79", "nanovllm/qwen38/engine.py": "1b58947ca00b3aaa6e97f1050abe50b0afa45193da11558d5d3c073db5a7d0c1", "nanovllm/qwen38/graph.py": "d4213aa68599564a01ced3b990993c9b17482615c154120bc4180f78a5da6098", "nanovllm/qwen38/loader.py": "1aa6b3321bc135fa28d1eebf6223bbe3bc5f646630193e292fb58a9f0ca2763f", "nanovllm/qwen38/model.py": "ca8730ba7f4760657d404e7f858d64e82efbc40078cc4b08ee5e2d417287a2aa", "nanovllm/qwen38/quant.py": "c13367e1993f85529c555a9c02de268bb169c52d7fa46626c4aa8a86fc2e3fd1", "nanovllm/qwen38/runner.py": "3b99c999c887f6acc2f2b6aa957c01680246fe424fb3da961322c06dacf857a0", "nanovllm/qwen38/server.py": "fdbe05d82598b299eb777c99d8f8cf7f2bab685d85a073428279e6665d590543"}} -{"phase": "ready", "arm": "main_graph", "startup_s": 41.70116449799389, "versions": {"torch": "2.14.1+cu130", "transformers": "5.20.2", "triton": "3.8.0"}, "device": "NVIDIA H20-3e", "graph_prewarm": {"buckets": {"256": {"capture_seconds": 2.0316781350411475, "resident_MiB": 229.5}, "1024": {"capture_seconds": 0.17580005491618067, "resident_MiB": 215.224609375}, "2048": {"capture_seconds": 0.17862341401632875, "resident_MiB": 276.224609375}, "4096": {"capture_seconds": 0.1829959739698097, "resident_MiB": 415.224609375}, "8448": {"capture_seconds": 0.1828454890055582, "resident_MiB": 685.224609375}}, "started_unix": 1792686496.4724932, "total_seconds": 2.751943066949025, "resident_MiB_total": 1821.3984375, "completed_unix": 1792686499.2259169}, "backend": "triton_block_fp8", "fp8_execution": {"modules": 400, "total_calls": 4000, "executed_modules": 400, "per_module_calls": {"model.layers.0.linear_attn.in_proj_qkv": 10, "model.layers.0.linear_attn.in_proj_z": 10, "model.layers.0.linear_attn.out_proj": 10, "model.layers.0.mlp.gate_proj": 10, "model.layers.0.mlp.up_proj": 10, "model.layers.0.mlp.down_proj": 10, "model.layers.1.linear_attn.in_proj_qkv": 10, "model.layers.1.linear_attn.in_proj_z": 10, "model.layers.1.linear_attn.out_proj": 10, "model.layers.1.mlp.gate_proj": 10, "model.layers.1.mlp.up_proj": 10, "model.layers.1.mlp.down_proj": 10, "model.layers.2.linear_attn.in_proj_qkv": 10, "model.layers.2.linear_attn.in_proj_z": 10, "model.layers.2.linear_attn.out_proj": 10, "model.layers.2.mlp.gate_proj": 10, "model.layers.2.mlp.up_proj": 10, "model.layers.2.mlp.down_proj": 10, "model.layers.3.self_attn.q_proj": 10, "model.layers.3.self_attn.k_proj": 10, "model.layers.3.self_attn.v_proj": 10, "model.layers.3.self_attn.o_proj": 10, "model.layers.3.mlp.gate_proj": 10, "model.layers.3.mlp.up_proj": 10, "model.layers.3.mlp.down_proj": 10, "model.layers.4.linear_attn.in_proj_qkv": 10, "model.layers.4.linear_attn.in_proj_z": 10, "model.layers.4.linear_attn.out_proj": 10, "model.layers.4.mlp.gate_proj": 10, "model.layers.4.mlp.up_proj": 10, "model.layers.4.mlp.down_proj": 10, "model.layers.5.linear_attn.in_proj_qkv": 10, "model.layers.5.linear_attn.in_proj_z": 10, "model.layers.5.linear_attn.out_proj": 10, "model.layers.5.mlp.gate_proj": 10, "model.layers.5.mlp.up_proj": 10, "model.layers.5.mlp.down_proj": 10, "model.layers.6.linear_attn.in_proj_qkv": 10, "model.layers.6.linear_attn.in_proj_z": 10, "model.layers.6.linear_attn.out_proj": 10, "model.layers.6.mlp.gate_proj": 10, "model.layers.6.mlp.up_proj": 10, "model.layers.6.mlp.down_proj": 10, "model.layers.7.self_attn.q_proj": 10, "model.layers.7.self_attn.k_proj": 10, "model.layers.7.self_attn.v_proj": 10, "model.layers.7.self_attn.o_proj": 10, "model.layers.7.mlp.gate_proj": 10, "model.layers.7.mlp.up_proj": 10, "model.layers.7.mlp.down_proj": 10, "model.layers.8.linear_attn.in_proj_qkv": 10, "model.layers.8.linear_attn.in_proj_z": 10, "model.layers.8.linear_attn.out_proj": 10, "model.layers.8.mlp.gate_proj": 10, "model.layers.8.mlp.up_proj": 10, "model.layers.8.mlp.down_proj": 10, "model.layers.9.linear_attn.in_proj_qkv": 10, "model.layers.9.linear_attn.in_proj_z": 10, "model.layers.9.linear_attn.out_proj": 10, "model.layers.9.mlp.gate_proj": 10, "model.layers.9.mlp.up_proj": 10, "model.layers.9.mlp.down_proj": 10, "model.layers.10.linear_attn.in_proj_qkv": 10, "model.layers.10.linear_attn.in_proj_z": 10, "model.layers.10.linear_attn.out_proj": 10, "model.layers.10.mlp.gate_proj": 10, "model.layers.10.mlp.up_proj": 10, "model.layers.10.mlp.down_proj": 10, "model.layers.11.self_attn.q_proj": 10, "model.layers.11.self_attn.k_proj": 10, "model.layers.11.self_attn.v_proj": 10, "model.layers.11.self_attn.o_proj": 10, "model.layers.11.mlp.gate_proj": 10, "model.layers.11.mlp.up_proj": 10, "model.layers.11.mlp.down_proj": 10, "model.layers.12.linear_attn.in_proj_qkv": 10, "model.layers.12.linear_attn.in_proj_z": 10, "model.layers.12.linear_attn.out_proj": 10, "model.layers.12.mlp.gate_proj": 10, "model.layers.12.mlp.up_proj": 10, "model.layers.12.mlp.down_proj": 10, "model.layers.13.linear_attn.in_proj_qkv": 10, "model.layers.13.linear_attn.in_proj_z": 10, "model.layers.13.linear_attn.out_proj": 10, "model.layers.13.mlp.gate_proj": 10, "model.layers.13.mlp.up_proj": 10, "model.layers.13.mlp.down_proj": 10, "model.layers.14.linear_attn.in_proj_qkv": 10, "model.layers.14.linear_attn.in_proj_z": 10, "model.layers.14.linear_attn.out_proj": 10, "model.layers.14.mlp.gate_proj": 10, "model.layers.14.mlp.up_proj": 10, "model.layers.14.mlp.down_proj": 10, "model.layers.15.self_attn.q_proj": 10, "model.layers.15.self_attn.k_proj": 10, "model.layers.15.self_attn.v_proj": 10, "model.layers.15.self_attn.o_proj": 10, "model.layers.15.mlp.gate_proj": 10, "model.layers.15.mlp.up_proj": 10, "model.layers.15.mlp.down_proj": 10, "model.layers.16.linear_attn.in_proj_qkv": 10, "model.layers.16.linear_attn.in_proj_z": 10, "model.layers.16.linear_attn.out_proj": 10, "model.layers.16.mlp.gate_proj": 10, "model.layers.16.mlp.up_proj": 10, "model.layers.16.mlp.down_proj": 10, "model.layers.17.linear_attn.in_proj_qkv": 10, "model.layers.17.linear_attn.in_proj_z": 10, "model.layers.17.linear_attn.out_proj": 10, "model.layers.17.mlp.gate_proj": 10, "model.layers.17.mlp.up_proj": 10, "model.layers.17.mlp.down_proj": 10, "model.layers.18.linear_attn.in_proj_qkv": 10, "model.layers.18.linear_attn.in_proj_z": 10, "model.layers.18.linear_attn.out_proj": 10, "model.layers.18.mlp.gate_proj": 10, "model.layers.18.mlp.up_proj": 10, "model.layers.18.mlp.down_proj": 10, "model.layers.19.self_attn.q_proj": 10, "model.layers.19.self_attn.k_proj": 10, "model.layers.19.self_attn.v_proj": 10, "model.layers.19.self_attn.o_proj": 10, "model.layers.19.mlp.gate_proj": 10, "model.layers.19.mlp.up_proj": 10, "model.layers.19.mlp.down_proj": 10, "model.layers.20.linear_attn.in_proj_qkv": 10, "model.layers.20.linear_attn.in_proj_z": 10, "model.layers.20.linear_attn.out_proj": 10, "model.layers.20.mlp.gate_proj": 10, "model.layers.20.mlp.up_proj": 10, "model.layers.20.mlp.down_proj": 10, "model.layers.21.linear_attn.in_proj_qkv": 10, "model.layers.21.linear_attn.in_proj_z": 10, "model.layers.21.linear_attn.out_proj": 10, "model.layers.21.mlp.gate_proj": 10, "model.layers.21.mlp.up_proj": 10, "model.layers.21.mlp.down_proj": 10, "model.layers.22.linear_attn.in_proj_qkv": 10, "model.layers.22.linear_attn.in_proj_z": 10, "model.layers.22.linear_attn.out_proj": 10, "model.layers.22.mlp.gate_proj": 10, "model.layers.22.mlp.up_proj": 10, "model.layers.22.mlp.down_proj": 10, "model.layers.23.self_attn.q_proj": 10, "model.layers.23.self_attn.k_proj": 10, "model.layers.23.self_attn.v_proj": 10, "model.layers.23.self_attn.o_proj": 10, "model.layers.23.mlp.gate_proj": 10, "model.layers.23.mlp.up_proj": 10, "model.layers.23.mlp.down_proj": 10, "model.layers.24.linear_attn.in_proj_qkv": 10, "model.layers.24.linear_attn.in_proj_z": 10, "model.layers.24.linear_attn.out_proj": 10, "model.layers.24.mlp.gate_proj": 10, "model.layers.24.mlp.up_proj": 10, "model.layers.24.mlp.down_proj": 10, "model.layers.25.linear_attn.in_proj_qkv": 10, "model.layers.25.linear_attn.in_proj_z": 10, "model.layers.25.linear_attn.out_proj": 10, "model.layers.25.mlp.gate_proj": 10, "model.layers.25.mlp.up_proj": 10, "model.layers.25.mlp.down_proj": 10, "model.layers.26.linear_attn.in_proj_qkv": 10, "model.layers.26.linear_attn.in_proj_z": 10, "model.layers.26.linear_attn.out_proj": 10, "model.layers.26.mlp.gate_proj": 10, "model.layers.26.mlp.up_proj": 10, "model.layers.26.mlp.down_proj": 10, "model.layers.27.self_attn.q_proj": 10, "model.layers.27.self_attn.k_proj": 10, "model.layers.27.self_attn.v_proj": 10, "model.layers.27.self_attn.o_proj": 10, "model.layers.27.mlp.gate_proj": 10, "model.layers.27.mlp.up_proj": 10, "model.layers.27.mlp.down_proj": 10, "model.layers.28.linear_attn.in_proj_qkv": 10, "model.layers.28.linear_attn.in_proj_z": 10, "model.layers.28.linear_attn.out_proj": 10, "model.layers.28.mlp.gate_proj": 10, "model.layers.28.mlp.up_proj": 10, "model.layers.28.mlp.down_proj": 10, "model.layers.29.linear_attn.in_proj_qkv": 10, "model.layers.29.linear_attn.in_proj_z": 10, "model.layers.29.linear_attn.out_proj": 10, "model.layers.29.mlp.gate_proj": 10, "model.layers.29.mlp.up_proj": 10, "model.layers.29.mlp.down_proj": 10, "model.layers.30.linear_attn.in_proj_qkv": 10, "model.layers.30.linear_attn.in_proj_z": 10, "model.layers.30.linear_attn.out_proj": 10, "model.layers.30.mlp.gate_proj": 10, "model.layers.30.mlp.up_proj": 10, "model.layers.30.mlp.down_proj": 10, "model.layers.31.self_attn.q_proj": 10, "model.layers.31.self_attn.k_proj": 10, "model.layers.31.self_attn.v_proj": 10, "model.layers.31.self_attn.o_proj": 10, "model.layers.31.mlp.gate_proj": 10, "model.layers.31.mlp.up_proj": 10, "model.layers.31.mlp.down_proj": 10, "model.layers.32.linear_attn.in_proj_qkv": 10, "model.layers.32.linear_attn.in_proj_z": 10, "model.layers.32.linear_attn.out_proj": 10, "model.layers.32.mlp.gate_proj": 10, "model.layers.32.mlp.up_proj": 10, "model.layers.32.mlp.down_proj": 10, "model.layers.33.linear_attn.in_proj_qkv": 10, "model.layers.33.linear_attn.in_proj_z": 10, "model.layers.33.linear_attn.out_proj": 10, "model.layers.33.mlp.gate_proj": 10, "model.layers.33.mlp.up_proj": 10, "model.layers.33.mlp.down_proj": 10, "model.layers.34.linear_attn.in_proj_qkv": 10, "model.layers.34.linear_attn.in_proj_z": 10, "model.layers.34.linear_attn.out_proj": 10, "model.layers.34.mlp.gate_proj": 10, "model.layers.34.mlp.up_proj": 10, "model.layers.34.mlp.down_proj": 10, "model.layers.35.self_attn.q_proj": 10, "model.layers.35.self_attn.k_proj": 10, "model.layers.35.self_attn.v_proj": 10, "model.layers.35.self_attn.o_proj": 10, "model.layers.35.mlp.gate_proj": 10, "model.layers.35.mlp.up_proj": 10, "model.layers.35.mlp.down_proj": 10, "model.layers.36.linear_attn.in_proj_qkv": 10, "model.layers.36.linear_attn.in_proj_z": 10, "model.layers.36.linear_attn.out_proj": 10, "model.layers.36.mlp.gate_proj": 10, "model.layers.36.mlp.up_proj": 10, "model.layers.36.mlp.down_proj": 10, "model.layers.37.linear_attn.in_proj_qkv": 10, "model.layers.37.linear_attn.in_proj_z": 10, "model.layers.37.linear_attn.out_proj": 10, "model.layers.37.mlp.gate_proj": 10, "model.layers.37.mlp.up_proj": 10, "model.layers.37.mlp.down_proj": 10, "model.layers.38.linear_attn.in_proj_qkv": 10, "model.layers.38.linear_attn.in_proj_z": 10, "model.layers.38.linear_attn.out_proj": 10, "model.layers.38.mlp.gate_proj": 10, "model.layers.38.mlp.up_proj": 10, "model.layers.38.mlp.down_proj": 10, "model.layers.39.self_attn.q_proj": 10, "model.layers.39.self_attn.k_proj": 10, "model.layers.39.self_attn.v_proj": 10, "model.layers.39.self_attn.o_proj": 10, "model.layers.39.mlp.gate_proj": 10, "model.layers.39.mlp.up_proj": 10, "model.layers.39.mlp.down_proj": 10, "model.layers.40.linear_attn.in_proj_qkv": 10, "model.layers.40.linear_attn.in_proj_z": 10, "model.layers.40.linear_attn.out_proj": 10, "model.layers.40.mlp.gate_proj": 10, "model.layers.40.mlp.up_proj": 10, "model.layers.40.mlp.down_proj": 10, "model.layers.41.linear_attn.in_proj_qkv": 10, "model.layers.41.linear_attn.in_proj_z": 10, "model.layers.41.linear_attn.out_proj": 10, "model.layers.41.mlp.gate_proj": 10, "model.layers.41.mlp.up_proj": 10, "model.layers.41.mlp.down_proj": 10, "model.layers.42.linear_attn.in_proj_qkv": 10, "model.layers.42.linear_attn.in_proj_z": 10, "model.layers.42.linear_attn.out_proj": 10, "model.layers.42.mlp.gate_proj": 10, "model.layers.42.mlp.up_proj": 10, "model.layers.42.mlp.down_proj": 10, "model.layers.43.self_attn.q_proj": 10, "model.layers.43.self_attn.k_proj": 10, "model.layers.43.self_attn.v_proj": 10, "model.layers.43.self_attn.o_proj": 10, "model.layers.43.mlp.gate_proj": 10, "model.layers.43.mlp.up_proj": 10, "model.layers.43.mlp.down_proj": 10, "model.layers.44.linear_attn.in_proj_qkv": 10, "model.layers.44.linear_attn.in_proj_z": 10, "model.layers.44.linear_attn.out_proj": 10, "model.layers.44.mlp.gate_proj": 10, "model.layers.44.mlp.up_proj": 10, "model.layers.44.mlp.down_proj": 10, "model.layers.45.linear_attn.in_proj_qkv": 10, "model.layers.45.linear_attn.in_proj_z": 10, "model.layers.45.linear_attn.out_proj": 10, "model.layers.45.mlp.gate_proj": 10, "model.layers.45.mlp.up_proj": 10, "model.layers.45.mlp.down_proj": 10, "model.layers.46.linear_attn.in_proj_qkv": 10, "model.layers.46.linear_attn.in_proj_z": 10, "model.layers.46.linear_attn.out_proj": 10, "model.layers.46.mlp.gate_proj": 10, "model.layers.46.mlp.up_proj": 10, "model.layers.46.mlp.down_proj": 10, "model.layers.47.self_attn.q_proj": 10, "model.layers.47.self_attn.k_proj": 10, "model.layers.47.self_attn.v_proj": 10, "model.layers.47.self_attn.o_proj": 10, "model.layers.47.mlp.gate_proj": 10, "model.layers.47.mlp.up_proj": 10, "model.layers.47.mlp.down_proj": 10, "model.layers.48.linear_attn.in_proj_qkv": 10, "model.layers.48.linear_attn.in_proj_z": 10, "model.layers.48.linear_attn.out_proj": 10, "model.layers.48.mlp.gate_proj": 10, "model.layers.48.mlp.up_proj": 10, "model.layers.48.mlp.down_proj": 10, "model.layers.49.linear_attn.in_proj_qkv": 10, "model.layers.49.linear_attn.in_proj_z": 10, "model.layers.49.linear_attn.out_proj": 10, "model.layers.49.mlp.gate_proj": 10, "model.layers.49.mlp.up_proj": 10, "model.layers.49.mlp.down_proj": 10, "model.layers.50.linear_attn.in_proj_qkv": 10, "model.layers.50.linear_attn.in_proj_z": 10, "model.layers.50.linear_attn.out_proj": 10, "model.layers.50.mlp.gate_proj": 10, "model.layers.50.mlp.up_proj": 10, "model.layers.50.mlp.down_proj": 10, "model.layers.51.self_attn.q_proj": 10, "model.layers.51.self_attn.k_proj": 10, "model.layers.51.self_attn.v_proj": 10, "model.layers.51.self_attn.o_proj": 10, "model.layers.51.mlp.gate_proj": 10, "model.layers.51.mlp.up_proj": 10, "model.layers.51.mlp.down_proj": 10, "model.layers.52.linear_attn.in_proj_qkv": 10, "model.layers.52.linear_attn.in_proj_z": 10, "model.layers.52.linear_attn.out_proj": 10, "model.layers.52.mlp.gate_proj": 10, "model.layers.52.mlp.up_proj": 10, "model.layers.52.mlp.down_proj": 10, "model.layers.53.linear_attn.in_proj_qkv": 10, "model.layers.53.linear_attn.in_proj_z": 10, "model.layers.53.linear_attn.out_proj": 10, "model.layers.53.mlp.gate_proj": 10, "model.layers.53.mlp.up_proj": 10, "model.layers.53.mlp.down_proj": 10, "model.layers.54.linear_attn.in_proj_qkv": 10, "model.layers.54.linear_attn.in_proj_z": 10, "model.layers.54.linear_attn.out_proj": 10, "model.layers.54.mlp.gate_proj": 10, "model.layers.54.mlp.up_proj": 10, "model.layers.54.mlp.down_proj": 10, "model.layers.55.self_attn.q_proj": 10, "model.layers.55.self_attn.k_proj": 10, "model.layers.55.self_attn.v_proj": 10, "model.layers.55.self_attn.o_proj": 10, "model.layers.55.mlp.gate_proj": 10, "model.layers.55.mlp.up_proj": 10, "model.layers.55.mlp.down_proj": 10, "model.layers.56.linear_attn.in_proj_qkv": 10, "model.layers.56.linear_attn.in_proj_z": 10, "model.layers.56.linear_attn.out_proj": 10, "model.layers.56.mlp.gate_proj": 10, "model.layers.56.mlp.up_proj": 10, "model.layers.56.mlp.down_proj": 10, "model.layers.57.linear_attn.in_proj_qkv": 10, "model.layers.57.linear_attn.in_proj_z": 10, "model.layers.57.linear_attn.out_proj": 10, "model.layers.57.mlp.gate_proj": 10, "model.layers.57.mlp.up_proj": 10, "model.layers.57.mlp.down_proj": 10, "model.layers.58.linear_attn.in_proj_qkv": 10, "model.layers.58.linear_attn.in_proj_z": 10, "model.layers.58.linear_attn.out_proj": 10, "model.layers.58.mlp.gate_proj": 10, "model.layers.58.mlp.up_proj": 10, "model.layers.58.mlp.down_proj": 10, "model.layers.59.self_attn.q_proj": 10, "model.layers.59.self_attn.k_proj": 10, "model.layers.59.self_attn.v_proj": 10, "model.layers.59.self_attn.o_proj": 10, "model.layers.59.mlp.gate_proj": 10, "model.layers.59.mlp.up_proj": 10, "model.layers.59.mlp.down_proj": 10, "model.layers.60.linear_attn.in_proj_qkv": 10, "model.layers.60.linear_attn.in_proj_z": 10, "model.layers.60.linear_attn.out_proj": 10, "model.layers.60.mlp.gate_proj": 10, "model.layers.60.mlp.up_proj": 10, "model.layers.60.mlp.down_proj": 10, "model.layers.61.linear_attn.in_proj_qkv": 10, "model.layers.61.linear_attn.in_proj_z": 10, "model.layers.61.linear_attn.out_proj": 10, "model.layers.61.mlp.gate_proj": 10, "model.layers.61.mlp.up_proj": 10, "model.layers.61.mlp.down_proj": 10, "model.layers.62.linear_attn.in_proj_qkv": 10, "model.layers.62.linear_attn.in_proj_z": 10, "model.layers.62.linear_attn.out_proj": 10, "model.layers.62.mlp.gate_proj": 10, "model.layers.62.mlp.up_proj": 10, "model.layers.62.mlp.down_proj": 10, "model.layers.63.self_attn.q_proj": 10, "model.layers.63.self_attn.k_proj": 10, "model.layers.63.self_attn.v_proj": 10, "model.layers.63.self_attn.o_proj": 10, "model.layers.63.mlp.gate_proj": 10, "model.layers.63.mlp.up_proj": 10, "model.layers.63.mlp.down_proj": 10}, "weight_dtype": "float8_e4m3fn", "scale_dtype": "float32", "actual_dtypes_valid": true}} -{"phase": "request", "arm": "main_graph", "case": "gsm8k/test/783", "repetition": 0, "prompt_tokens": 149, "prompt_sha256": "f2233be4ebb7b122d80433fe4081fdabeda64f735b7bc24f9706c4c4339b824f", "active_requests_after": 0, "runner_state_count_after": 0, "token_ids": [1206, 1423, 680, 1204, 1599, 4022, 7397, 13853, 10471, 11, 567, 628, 716, 685, 449, 45309], "token_times_s": [2.692203041049652, 2.78136371797882, 2.8693681079894304, 2.9573882670374587, 3.045341977966018, 3.1334574830252677, 3.221387909958139, 3.309419407043606, 3.3973573959665373, 3.485344114014879, 3.5733625000575557, 3.661294150049798, 3.7492523080436513, 3.8371689680498093, 3.9251978950342163, 4.013172678067349], "text_sha256": "605c111382f5aa3badacba632d1cff554bedc8c1a6637defa7ee98e8a19cdcd0", "tokens": 16, "terminal": "length", "done": true, "ttft_token_s": 2.692203041049652, "tpot_token_s": 0.08806464246784647, "latency_s": 4.013243875000626} -{"phase": "request", "arm": "main_graph", "case": "gsm8k/test/1", "repetition": 0, "prompt_tokens": 70, "prompt_sha256": "78d335abe81363d4bb48e5aa256c120344c2ceadbcf8739afe4ec3923c747238", "active_requests_after": 0, "runner_state_count_after": 0, "token_ids": [1206, 7995, 279, 2702, 1324, 314, 47205, 2483, 364, 279, 60507, 11, 567, 1144, 310, 10724], "token_times_s": [1.3219671979313716, 1.4098825979745016, 1.4978378819068894, 1.585882316925563, 1.6738443949725479, 1.7618563869036734, 1.849850415950641, 1.9377737250179052, 2.0257913309615105, 2.1136269359849393, 2.201620928943157, 2.289883268997073, 2.3778872549301013, 2.465784002910368, 2.5537270120112225, 2.6417272799881175], "text_sha256": "63454f086bab80397da9b2b530254b6dfaa499fcda59fdee055d99530ff45fec", "tokens": 16, "terminal": "length", "done": true, "ttft_token_s": 1.3219671979313716, "tpot_token_s": 0.08798400547044972, "latency_s": 2.641791876987554} -{"phase": "request", "arm": "main_graph", "case": "gsm8k/test/424", "repetition": 0, "prompt_tokens": 106, "prompt_sha256": "03cbdf5804bab62d4efbc0ae89e191bd557840c6d9a1a5ada9669de953a0b783", "active_requests_after": 0, "runner_state_count_after": 0, "token_ids": [1206, 7995, 279, 2702, 1324, 314, 7998, 421, 628, 4973, 279, 1834, 8151, 11, 567, 1144], "token_times_s": [1.9445371369365603, 2.0324338750215247, 2.1203689669491723, 2.2084475939627737, 2.2965175149729475, 2.3844939219998196, 2.4725375160342082, 2.560554572963156, 2.648514178930782, 2.736575498012826, 2.824569332995452, 2.912572260014713, 3.000587148941122, 3.088529338943772, 3.176542268949561, 3.2644837979460135], "text_sha256": "c8bdc8989c61acbc782bbeff0e2cffd37f1b1ffe9270709e6c2cf58d34c5537f", "tokens": 16, "terminal": "length", "done": true, "ttft_token_s": 1.9445371369365603, "tpot_token_s": 0.08799644406729688, "latency_s": 3.2645760159939528} -{"phase": "request", "arm": "main_graph", "case": "gsm8k/test/783", "repetition": 1, "prompt_tokens": 149, "prompt_sha256": "f2233be4ebb7b122d80433fe4081fdabeda64f735b7bc24f9706c4c4339b824f", "active_requests_after": 0, "runner_state_count_after": 0, "token_ids": [1206, 1423, 680, 1204, 1599, 4022, 7397, 13853, 10471, 11, 567, 628, 716, 685, 449, 45309], "token_times_s": [2.5663553439080715, 2.6543150959769264, 2.742237602942623, 2.830241900985129, 2.918242731015198, 3.006187547929585, 3.094302783953026, 3.1824889529962093, 3.270642119925469, 3.3586434919852763, 3.446638716966845, 3.5347343100002035, 3.622780286008492, 3.7109024769160897, 3.798861905001104, 3.8869768199510872], "text_sha256": "605c111382f5aa3badacba632d1cff554bedc8c1a6637defa7ee98e8a19cdcd0", "tokens": 16, "terminal": "length", "done": true, "ttft_token_s": 2.5663553439080715, "tpot_token_s": 0.08804143173620105, "latency_s": 3.887056217994541} -{"phase": "request", "arm": "main_graph", "case": "gsm8k/test/1", "repetition": 1, "prompt_tokens": 70, "prompt_sha256": "78d335abe81363d4bb48e5aa256c120344c2ceadbcf8739afe4ec3923c747238", "active_requests_after": 0, "runner_state_count_after": 0, "token_ids": [1206, 7995, 279, 2702, 1324, 314, 47205, 2483, 364, 279, 60507, 11, 567, 1144, 310, 10724], "token_times_s": [1.2381265569711104, 1.3261253238888457, 1.4141487349988893, 1.5028744009323418, 1.590900159906596, 1.6796635629143566, 1.767542102956213, 1.8554247659631073, 1.944246292929165, 2.0323217269033194, 2.1205726159969345, 2.2086279299110174, 2.2965450999327004, 2.3854157499736175, 2.473423025920056, 2.561542744981125], "text_sha256": "63454f086bab80397da9b2b530254b6dfaa499fcda59fdee055d99530ff45fec", "tokens": 16, "terminal": "length", "done": true, "ttft_token_s": 1.2381265569711104, "tpot_token_s": 0.08822774586733431, "latency_s": 2.5616036219289526} -{"phase": "request", "arm": "main_graph", "case": "gsm8k/test/424", "repetition": 1, "prompt_tokens": 106, "prompt_sha256": "03cbdf5804bab62d4efbc0ae89e191bd557840c6d9a1a5ada9669de953a0b783", "active_requests_after": 0, "runner_state_count_after": 0, "token_ids": [1206, 7995, 279, 2702, 1324, 314, 7998, 421, 628, 4973, 279, 1834, 8151, 11, 567, 1144], "token_times_s": [1.8674182330723852, 1.9553972850553691, 2.043524938984774, 2.1315956340404227, 2.2196454730583355, 2.307545730029233, 2.395534547045827, 2.4834720770595595, 2.571397255989723, 2.6594193240161985, 2.7474687510402873, 2.835666997008957, 2.9237499659648165, 3.0118105279980227, 3.0996940539916977, 3.187767955008894], "text_sha256": "c8bdc8989c61acbc782bbeff0e2cffd37f1b1ffe9270709e6c2cf58d34c5537f", "tokens": 16, "terminal": "length", "done": true, "ttft_token_s": 1.8674182330723852, "tpot_token_s": 0.08802331479576726, "latency_s": 3.1878462790045887} -{"phase": "cleanup", "arm": "main_graph", "exit_code": 0, "forced": false, "closed": true, "active_requests": 0, "runner_state_count": 0} -{"phase": "source", "arm": "repair_graph", "graph": true, "hashes": {"nanovllm/qwen38/__init__.py": "1f07e0cfd12bbf370c1f465c6a4bd73e72a5c826af1f29e60ccc7d6f45562cfc", "nanovllm/qwen38/__main__.py": "9e2a0881eedf8fa4abc6037511b42fd3a44171d121a7b2cb096023f485673ba2", "nanovllm/qwen38/_triton_fp8.py": "9f0cb68be017cb3858aebe8ba275039084e305df92cb4567d6e7da54d119f19e", "nanovllm/qwen38/_triton_rms.py": "ba99fa3ba82bfcab4cd602fd0db907acad0eee7c7a63ab23d560a17fd2f72272", "nanovllm/qwen38/config.py": "002ea25bbf9c8d17e5f1d3fb2b76c400e4291d7d2d06d5dd13fb5c3ac15c3d79", "nanovllm/qwen38/engine.py": "c47e1c72fcc24bc8eeb5497887da6be11c2de567abcc3efb275b061c061fd755", "nanovllm/qwen38/graph.py": "d4213aa68599564a01ced3b990993c9b17482615c154120bc4180f78a5da6098", "nanovllm/qwen38/loader.py": "1aa6b3321bc135fa28d1eebf6223bbe3bc5f646630193e292fb58a9f0ca2763f", "nanovllm/qwen38/model.py": "ad8386f156ff5315dfd73de313ab766fedd9ef892c7b94cec95303f11fd4e198", "nanovllm/qwen38/quant.py": "5590e4769a0fbbcaf5afeee1a93db5c4bccacf09f6f3b34e42d52ec31e9f50c3", "nanovllm/qwen38/runner.py": "3b99c999c887f6acc2f2b6aa957c01680246fe424fb3da961322c06dacf857a0", "nanovllm/qwen38/server.py": "fdbe05d82598b299eb777c99d8f8cf7f2bab685d85a073428279e6665d590543"}} -{"phase": "ready", "arm": "repair_graph", "startup_s": 23.85664731799625, "versions": {"torch": "2.14.1+cu130", "transformers": "5.20.2", "triton": "3.8.0"}, "device": "NVIDIA H20-3e", "graph_prewarm": {"buckets": {"256": {"capture_seconds": 3.3120902059599757, "resident_MiB": 229.5}, "1024": {"capture_seconds": 0.21492932399269193, "resident_MiB": 215.224609375}, "2048": {"capture_seconds": 0.4119006529217586, "resident_MiB": 276.224609375}, "4096": {"capture_seconds": 0.26738236600067466, "resident_MiB": 415.224609375}, "8448": {"capture_seconds": 0.20350945694372058, "resident_MiB": 685.224609375}}, "started_unix": 1792686539.3018615, "total_seconds": 4.4098120058188215, "resident_MiB_total": 1821.3984375, "completed_unix": 1792686543.7132533}, "backend": "triton_block_fp8", "fp8_execution": {"modules": 400, "total_calls": 4000, "call_counter_scope": "python_forward_only_includes_warmup_and_capture_excludes_graph_replay", "backend": "triton_block_fp8", "kernel_name": "qwen38_block_fp8", "executed_modules": 400, "per_module_calls": {"model.layers.0.linear_attn.in_proj_qkv": 10, "model.layers.0.linear_attn.in_proj_z": 10, "model.layers.0.linear_attn.out_proj": 10, "model.layers.0.mlp.gate_proj": 10, "model.layers.0.mlp.up_proj": 10, "model.layers.0.mlp.down_proj": 10, "model.layers.1.linear_attn.in_proj_qkv": 10, "model.layers.1.linear_attn.in_proj_z": 10, "model.layers.1.linear_attn.out_proj": 10, "model.layers.1.mlp.gate_proj": 10, "model.layers.1.mlp.up_proj": 10, "model.layers.1.mlp.down_proj": 10, "model.layers.2.linear_attn.in_proj_qkv": 10, "model.layers.2.linear_attn.in_proj_z": 10, "model.layers.2.linear_attn.out_proj": 10, "model.layers.2.mlp.gate_proj": 10, "model.layers.2.mlp.up_proj": 10, "model.layers.2.mlp.down_proj": 10, "model.layers.3.self_attn.q_proj": 10, "model.layers.3.self_attn.k_proj": 10, "model.layers.3.self_attn.v_proj": 10, "model.layers.3.self_attn.o_proj": 10, "model.layers.3.mlp.gate_proj": 10, "model.layers.3.mlp.up_proj": 10, "model.layers.3.mlp.down_proj": 10, "model.layers.4.linear_attn.in_proj_qkv": 10, "model.layers.4.linear_attn.in_proj_z": 10, "model.layers.4.linear_attn.out_proj": 10, "model.layers.4.mlp.gate_proj": 10, "model.layers.4.mlp.up_proj": 10, "model.layers.4.mlp.down_proj": 10, "model.layers.5.linear_attn.in_proj_qkv": 10, "model.layers.5.linear_attn.in_proj_z": 10, "model.layers.5.linear_attn.out_proj": 10, "model.layers.5.mlp.gate_proj": 10, "model.layers.5.mlp.up_proj": 10, "model.layers.5.mlp.down_proj": 10, "model.layers.6.linear_attn.in_proj_qkv": 10, "model.layers.6.linear_attn.in_proj_z": 10, "model.layers.6.linear_attn.out_proj": 10, "model.layers.6.mlp.gate_proj": 10, "model.layers.6.mlp.up_proj": 10, "model.layers.6.mlp.down_proj": 10, "model.layers.7.self_attn.q_proj": 10, "model.layers.7.self_attn.k_proj": 10, "model.layers.7.self_attn.v_proj": 10, "model.layers.7.self_attn.o_proj": 10, "model.layers.7.mlp.gate_proj": 10, "model.layers.7.mlp.up_proj": 10, "model.layers.7.mlp.down_proj": 10, "model.layers.8.linear_attn.in_proj_qkv": 10, "model.layers.8.linear_attn.in_proj_z": 10, "model.layers.8.linear_attn.out_proj": 10, "model.layers.8.mlp.gate_proj": 10, "model.layers.8.mlp.up_proj": 10, "model.layers.8.mlp.down_proj": 10, "model.layers.9.linear_attn.in_proj_qkv": 10, "model.layers.9.linear_attn.in_proj_z": 10, "model.layers.9.linear_attn.out_proj": 10, "model.layers.9.mlp.gate_proj": 10, "model.layers.9.mlp.up_proj": 10, "model.layers.9.mlp.down_proj": 10, "model.layers.10.linear_attn.in_proj_qkv": 10, "model.layers.10.linear_attn.in_proj_z": 10, "model.layers.10.linear_attn.out_proj": 10, "model.layers.10.mlp.gate_proj": 10, "model.layers.10.mlp.up_proj": 10, "model.layers.10.mlp.down_proj": 10, "model.layers.11.self_attn.q_proj": 10, "model.layers.11.self_attn.k_proj": 10, "model.layers.11.self_attn.v_proj": 10, "model.layers.11.self_attn.o_proj": 10, "model.layers.11.mlp.gate_proj": 10, "model.layers.11.mlp.up_proj": 10, "model.layers.11.mlp.down_proj": 10, "model.layers.12.linear_attn.in_proj_qkv": 10, "model.layers.12.linear_attn.in_proj_z": 10, "model.layers.12.linear_attn.out_proj": 10, "model.layers.12.mlp.gate_proj": 10, "model.layers.12.mlp.up_proj": 10, "model.layers.12.mlp.down_proj": 10, "model.layers.13.linear_attn.in_proj_qkv": 10, "model.layers.13.linear_attn.in_proj_z": 10, "model.layers.13.linear_attn.out_proj": 10, "model.layers.13.mlp.gate_proj": 10, "model.layers.13.mlp.up_proj": 10, "model.layers.13.mlp.down_proj": 10, "model.layers.14.linear_attn.in_proj_qkv": 10, "model.layers.14.linear_attn.in_proj_z": 10, "model.layers.14.linear_attn.out_proj": 10, "model.layers.14.mlp.gate_proj": 10, "model.layers.14.mlp.up_proj": 10, "model.layers.14.mlp.down_proj": 10, "model.layers.15.self_attn.q_proj": 10, "model.layers.15.self_attn.k_proj": 10, "model.layers.15.self_attn.v_proj": 10, "model.layers.15.self_attn.o_proj": 10, "model.layers.15.mlp.gate_proj": 10, "model.layers.15.mlp.up_proj": 10, "model.layers.15.mlp.down_proj": 10, "model.layers.16.linear_attn.in_proj_qkv": 10, "model.layers.16.linear_attn.in_proj_z": 10, "model.layers.16.linear_attn.out_proj": 10, "model.layers.16.mlp.gate_proj": 10, "model.layers.16.mlp.up_proj": 10, "model.layers.16.mlp.down_proj": 10, "model.layers.17.linear_attn.in_proj_qkv": 10, "model.layers.17.linear_attn.in_proj_z": 10, "model.layers.17.linear_attn.out_proj": 10, "model.layers.17.mlp.gate_proj": 10, "model.layers.17.mlp.up_proj": 10, "model.layers.17.mlp.down_proj": 10, "model.layers.18.linear_attn.in_proj_qkv": 10, "model.layers.18.linear_attn.in_proj_z": 10, "model.layers.18.linear_attn.out_proj": 10, "model.layers.18.mlp.gate_proj": 10, "model.layers.18.mlp.up_proj": 10, "model.layers.18.mlp.down_proj": 10, "model.layers.19.self_attn.q_proj": 10, "model.layers.19.self_attn.k_proj": 10, "model.layers.19.self_attn.v_proj": 10, "model.layers.19.self_attn.o_proj": 10, "model.layers.19.mlp.gate_proj": 10, "model.layers.19.mlp.up_proj": 10, "model.layers.19.mlp.down_proj": 10, "model.layers.20.linear_attn.in_proj_qkv": 10, "model.layers.20.linear_attn.in_proj_z": 10, "model.layers.20.linear_attn.out_proj": 10, "model.layers.20.mlp.gate_proj": 10, "model.layers.20.mlp.up_proj": 10, "model.layers.20.mlp.down_proj": 10, "model.layers.21.linear_attn.in_proj_qkv": 10, "model.layers.21.linear_attn.in_proj_z": 10, "model.layers.21.linear_attn.out_proj": 10, "model.layers.21.mlp.gate_proj": 10, "model.layers.21.mlp.up_proj": 10, "model.layers.21.mlp.down_proj": 10, "model.layers.22.linear_attn.in_proj_qkv": 10, "model.layers.22.linear_attn.in_proj_z": 10, "model.layers.22.linear_attn.out_proj": 10, "model.layers.22.mlp.gate_proj": 10, "model.layers.22.mlp.up_proj": 10, "model.layers.22.mlp.down_proj": 10, "model.layers.23.self_attn.q_proj": 10, "model.layers.23.self_attn.k_proj": 10, "model.layers.23.self_attn.v_proj": 10, "model.layers.23.self_attn.o_proj": 10, "model.layers.23.mlp.gate_proj": 10, "model.layers.23.mlp.up_proj": 10, "model.layers.23.mlp.down_proj": 10, "model.layers.24.linear_attn.in_proj_qkv": 10, "model.layers.24.linear_attn.in_proj_z": 10, "model.layers.24.linear_attn.out_proj": 10, "model.layers.24.mlp.gate_proj": 10, "model.layers.24.mlp.up_proj": 10, "model.layers.24.mlp.down_proj": 10, "model.layers.25.linear_attn.in_proj_qkv": 10, "model.layers.25.linear_attn.in_proj_z": 10, "model.layers.25.linear_attn.out_proj": 10, "model.layers.25.mlp.gate_proj": 10, "model.layers.25.mlp.up_proj": 10, "model.layers.25.mlp.down_proj": 10, "model.layers.26.linear_attn.in_proj_qkv": 10, "model.layers.26.linear_attn.in_proj_z": 10, "model.layers.26.linear_attn.out_proj": 10, "model.layers.26.mlp.gate_proj": 10, "model.layers.26.mlp.up_proj": 10, "model.layers.26.mlp.down_proj": 10, "model.layers.27.self_attn.q_proj": 10, "model.layers.27.self_attn.k_proj": 10, "model.layers.27.self_attn.v_proj": 10, "model.layers.27.self_attn.o_proj": 10, "model.layers.27.mlp.gate_proj": 10, "model.layers.27.mlp.up_proj": 10, "model.layers.27.mlp.down_proj": 10, "model.layers.28.linear_attn.in_proj_qkv": 10, "model.layers.28.linear_attn.in_proj_z": 10, "model.layers.28.linear_attn.out_proj": 10, "model.layers.28.mlp.gate_proj": 10, "model.layers.28.mlp.up_proj": 10, "model.layers.28.mlp.down_proj": 10, "model.layers.29.linear_attn.in_proj_qkv": 10, "model.layers.29.linear_attn.in_proj_z": 10, "model.layers.29.linear_attn.out_proj": 10, "model.layers.29.mlp.gate_proj": 10, "model.layers.29.mlp.up_proj": 10, "model.layers.29.mlp.down_proj": 10, "model.layers.30.linear_attn.in_proj_qkv": 10, "model.layers.30.linear_attn.in_proj_z": 10, "model.layers.30.linear_attn.out_proj": 10, "model.layers.30.mlp.gate_proj": 10, "model.layers.30.mlp.up_proj": 10, "model.layers.30.mlp.down_proj": 10, "model.layers.31.self_attn.q_proj": 10, "model.layers.31.self_attn.k_proj": 10, "model.layers.31.self_attn.v_proj": 10, "model.layers.31.self_attn.o_proj": 10, "model.layers.31.mlp.gate_proj": 10, "model.layers.31.mlp.up_proj": 10, "model.layers.31.mlp.down_proj": 10, "model.layers.32.linear_attn.in_proj_qkv": 10, "model.layers.32.linear_attn.in_proj_z": 10, "model.layers.32.linear_attn.out_proj": 10, "model.layers.32.mlp.gate_proj": 10, "model.layers.32.mlp.up_proj": 10, "model.layers.32.mlp.down_proj": 10, "model.layers.33.linear_attn.in_proj_qkv": 10, "model.layers.33.linear_attn.in_proj_z": 10, "model.layers.33.linear_attn.out_proj": 10, "model.layers.33.mlp.gate_proj": 10, "model.layers.33.mlp.up_proj": 10, "model.layers.33.mlp.down_proj": 10, "model.layers.34.linear_attn.in_proj_qkv": 10, "model.layers.34.linear_attn.in_proj_z": 10, "model.layers.34.linear_attn.out_proj": 10, "model.layers.34.mlp.gate_proj": 10, "model.layers.34.mlp.up_proj": 10, "model.layers.34.mlp.down_proj": 10, "model.layers.35.self_attn.q_proj": 10, "model.layers.35.self_attn.k_proj": 10, "model.layers.35.self_attn.v_proj": 10, "model.layers.35.self_attn.o_proj": 10, "model.layers.35.mlp.gate_proj": 10, "model.layers.35.mlp.up_proj": 10, "model.layers.35.mlp.down_proj": 10, "model.layers.36.linear_attn.in_proj_qkv": 10, "model.layers.36.linear_attn.in_proj_z": 10, "model.layers.36.linear_attn.out_proj": 10, "model.layers.36.mlp.gate_proj": 10, "model.layers.36.mlp.up_proj": 10, "model.layers.36.mlp.down_proj": 10, "model.layers.37.linear_attn.in_proj_qkv": 10, "model.layers.37.linear_attn.in_proj_z": 10, "model.layers.37.linear_attn.out_proj": 10, "model.layers.37.mlp.gate_proj": 10, "model.layers.37.mlp.up_proj": 10, "model.layers.37.mlp.down_proj": 10, "model.layers.38.linear_attn.in_proj_qkv": 10, "model.layers.38.linear_attn.in_proj_z": 10, "model.layers.38.linear_attn.out_proj": 10, "model.layers.38.mlp.gate_proj": 10, "model.layers.38.mlp.up_proj": 10, "model.layers.38.mlp.down_proj": 10, "model.layers.39.self_attn.q_proj": 10, "model.layers.39.self_attn.k_proj": 10, "model.layers.39.self_attn.v_proj": 10, "model.layers.39.self_attn.o_proj": 10, "model.layers.39.mlp.gate_proj": 10, "model.layers.39.mlp.up_proj": 10, "model.layers.39.mlp.down_proj": 10, "model.layers.40.linear_attn.in_proj_qkv": 10, "model.layers.40.linear_attn.in_proj_z": 10, "model.layers.40.linear_attn.out_proj": 10, "model.layers.40.mlp.gate_proj": 10, "model.layers.40.mlp.up_proj": 10, "model.layers.40.mlp.down_proj": 10, "model.layers.41.linear_attn.in_proj_qkv": 10, "model.layers.41.linear_attn.in_proj_z": 10, "model.layers.41.linear_attn.out_proj": 10, "model.layers.41.mlp.gate_proj": 10, "model.layers.41.mlp.up_proj": 10, "model.layers.41.mlp.down_proj": 10, "model.layers.42.linear_attn.in_proj_qkv": 10, "model.layers.42.linear_attn.in_proj_z": 10, "model.layers.42.linear_attn.out_proj": 10, "model.layers.42.mlp.gate_proj": 10, "model.layers.42.mlp.up_proj": 10, "model.layers.42.mlp.down_proj": 10, "model.layers.43.self_attn.q_proj": 10, "model.layers.43.self_attn.k_proj": 10, "model.layers.43.self_attn.v_proj": 10, "model.layers.43.self_attn.o_proj": 10, "model.layers.43.mlp.gate_proj": 10, "model.layers.43.mlp.up_proj": 10, "model.layers.43.mlp.down_proj": 10, "model.layers.44.linear_attn.in_proj_qkv": 10, "model.layers.44.linear_attn.in_proj_z": 10, "model.layers.44.linear_attn.out_proj": 10, "model.layers.44.mlp.gate_proj": 10, "model.layers.44.mlp.up_proj": 10, "model.layers.44.mlp.down_proj": 10, "model.layers.45.linear_attn.in_proj_qkv": 10, "model.layers.45.linear_attn.in_proj_z": 10, "model.layers.45.linear_attn.out_proj": 10, "model.layers.45.mlp.gate_proj": 10, "model.layers.45.mlp.up_proj": 10, "model.layers.45.mlp.down_proj": 10, "model.layers.46.linear_attn.in_proj_qkv": 10, "model.layers.46.linear_attn.in_proj_z": 10, "model.layers.46.linear_attn.out_proj": 10, "model.layers.46.mlp.gate_proj": 10, "model.layers.46.mlp.up_proj": 10, "model.layers.46.mlp.down_proj": 10, "model.layers.47.self_attn.q_proj": 10, "model.layers.47.self_attn.k_proj": 10, "model.layers.47.self_attn.v_proj": 10, "model.layers.47.self_attn.o_proj": 10, "model.layers.47.mlp.gate_proj": 10, "model.layers.47.mlp.up_proj": 10, "model.layers.47.mlp.down_proj": 10, "model.layers.48.linear_attn.in_proj_qkv": 10, "model.layers.48.linear_attn.in_proj_z": 10, "model.layers.48.linear_attn.out_proj": 10, "model.layers.48.mlp.gate_proj": 10, "model.layers.48.mlp.up_proj": 10, "model.layers.48.mlp.down_proj": 10, "model.layers.49.linear_attn.in_proj_qkv": 10, "model.layers.49.linear_attn.in_proj_z": 10, "model.layers.49.linear_attn.out_proj": 10, "model.layers.49.mlp.gate_proj": 10, "model.layers.49.mlp.up_proj": 10, "model.layers.49.mlp.down_proj": 10, "model.layers.50.linear_attn.in_proj_qkv": 10, "model.layers.50.linear_attn.in_proj_z": 10, "model.layers.50.linear_attn.out_proj": 10, "model.layers.50.mlp.gate_proj": 10, "model.layers.50.mlp.up_proj": 10, "model.layers.50.mlp.down_proj": 10, "model.layers.51.self_attn.q_proj": 10, "model.layers.51.self_attn.k_proj": 10, "model.layers.51.self_attn.v_proj": 10, "model.layers.51.self_attn.o_proj": 10, "model.layers.51.mlp.gate_proj": 10, "model.layers.51.mlp.up_proj": 10, "model.layers.51.mlp.down_proj": 10, "model.layers.52.linear_attn.in_proj_qkv": 10, "model.layers.52.linear_attn.in_proj_z": 10, "model.layers.52.linear_attn.out_proj": 10, "model.layers.52.mlp.gate_proj": 10, "model.layers.52.mlp.up_proj": 10, "model.layers.52.mlp.down_proj": 10, "model.layers.53.linear_attn.in_proj_qkv": 10, "model.layers.53.linear_attn.in_proj_z": 10, "model.layers.53.linear_attn.out_proj": 10, "model.layers.53.mlp.gate_proj": 10, "model.layers.53.mlp.up_proj": 10, "model.layers.53.mlp.down_proj": 10, "model.layers.54.linear_attn.in_proj_qkv": 10, "model.layers.54.linear_attn.in_proj_z": 10, "model.layers.54.linear_attn.out_proj": 10, "model.layers.54.mlp.gate_proj": 10, "model.layers.54.mlp.up_proj": 10, "model.layers.54.mlp.down_proj": 10, "model.layers.55.self_attn.q_proj": 10, "model.layers.55.self_attn.k_proj": 10, "model.layers.55.self_attn.v_proj": 10, "model.layers.55.self_attn.o_proj": 10, "model.layers.55.mlp.gate_proj": 10, "model.layers.55.mlp.up_proj": 10, "model.layers.55.mlp.down_proj": 10, "model.layers.56.linear_attn.in_proj_qkv": 10, "model.layers.56.linear_attn.in_proj_z": 10, "model.layers.56.linear_attn.out_proj": 10, "model.layers.56.mlp.gate_proj": 10, "model.layers.56.mlp.up_proj": 10, "model.layers.56.mlp.down_proj": 10, "model.layers.57.linear_attn.in_proj_qkv": 10, "model.layers.57.linear_attn.in_proj_z": 10, "model.layers.57.linear_attn.out_proj": 10, "model.layers.57.mlp.gate_proj": 10, "model.layers.57.mlp.up_proj": 10, "model.layers.57.mlp.down_proj": 10, "model.layers.58.linear_attn.in_proj_qkv": 10, "model.layers.58.linear_attn.in_proj_z": 10, "model.layers.58.linear_attn.out_proj": 10, "model.layers.58.mlp.gate_proj": 10, "model.layers.58.mlp.up_proj": 10, "model.layers.58.mlp.down_proj": 10, "model.layers.59.self_attn.q_proj": 10, "model.layers.59.self_attn.k_proj": 10, "model.layers.59.self_attn.v_proj": 10, "model.layers.59.self_attn.o_proj": 10, "model.layers.59.mlp.gate_proj": 10, "model.layers.59.mlp.up_proj": 10, "model.layers.59.mlp.down_proj": 10, "model.layers.60.linear_attn.in_proj_qkv": 10, "model.layers.60.linear_attn.in_proj_z": 10, "model.layers.60.linear_attn.out_proj": 10, "model.layers.60.mlp.gate_proj": 10, "model.layers.60.mlp.up_proj": 10, "model.layers.60.mlp.down_proj": 10, "model.layers.61.linear_attn.in_proj_qkv": 10, "model.layers.61.linear_attn.in_proj_z": 10, "model.layers.61.linear_attn.out_proj": 10, "model.layers.61.mlp.gate_proj": 10, "model.layers.61.mlp.up_proj": 10, "model.layers.61.mlp.down_proj": 10, "model.layers.62.linear_attn.in_proj_qkv": 10, "model.layers.62.linear_attn.in_proj_z": 10, "model.layers.62.linear_attn.out_proj": 10, "model.layers.62.mlp.gate_proj": 10, "model.layers.62.mlp.up_proj": 10, "model.layers.62.mlp.down_proj": 10, "model.layers.63.self_attn.q_proj": 10, "model.layers.63.self_attn.k_proj": 10, "model.layers.63.self_attn.v_proj": 10, "model.layers.63.self_attn.o_proj": 10, "model.layers.63.mlp.gate_proj": 10, "model.layers.63.mlp.up_proj": 10, "model.layers.63.mlp.down_proj": 10}, "weight_dtype": "float8_e4m3fn", "scale_dtype": "float32", "actual_dtypes_valid": true}} -{"phase": "request", "arm": "repair_graph", "case": "gsm8k/test/783", "repetition": 0, "prompt_tokens": 149, "prompt_sha256": "f2233be4ebb7b122d80433fe4081fdabeda64f735b7bc24f9706c4c4339b824f", "active_requests_after": 0, "runner_state_count_after": 0, "token_ids": [1206, 11290, 411, 3377, 11, 567, 1144, 310, 7995, 279, 1324, 314, 4022, 7397, 13853, 10471], "token_times_s": [2.641130000934936, 2.72888184490148, 2.816403752891347, 2.903841823921539, 2.9912921759532765, 3.07879762200173, 3.166266349959187, 3.2537541519850492, 3.3412110588978976, 3.4286989689571783, 3.5161262879846618, 3.603614641004242, 3.691052032983862, 3.7784803649410605, 3.8659949869615957, 3.953400812926702], "text_sha256": "4b5d6370a617c539db1e6ee0df2c726c7165c1e9395eab1b873ff700a997510a", "tokens": 16, "terminal": "length", "done": true, "ttft_token_s": 2.641130000934936, "tpot_token_s": 0.08748472079945108, "latency_s": 3.9534696829505265} -{"phase": "request", "arm": "repair_graph", "case": "gsm8k/test/1", "repetition": 0, "prompt_tokens": 70, "prompt_sha256": "78d335abe81363d4bb48e5aa256c120344c2ceadbcf8739afe4ec3923c747238", "active_requests_after": 0, "runner_state_count_after": 0, "token_ids": [1206, 7995, 279, 2702, 1324, 314, 47205, 2483, 364, 279, 60507, 11, 567, 1144, 310, 10724], "token_times_s": [1.3439365949016064, 1.4313265070086345, 1.5187029369408265, 1.6062005349667743, 1.6935259809251875, 1.7809020939748734, 1.8691657269373536, 1.9566818949533626, 2.0441846039611846, 2.131596675957553, 2.219991465914063, 2.30744721298106, 2.3950159879168496, 2.4823907589307055, 2.570741964969784, 2.658268114901148], "text_sha256": "63454f086bab80397da9b2b530254b6dfaa499fcda59fdee055d99530ff45fec", "tokens": 16, "terminal": "length", "done": true, "ttft_token_s": 1.3439365949016064, "tpot_token_s": 0.08762210133330275, "latency_s": 2.6583360099466518} -{"phase": "request", "arm": "repair_graph", "case": "gsm8k/test/424", "repetition": 0, "prompt_tokens": 106, "prompt_sha256": "03cbdf5804bab62d4efbc0ae89e191bd557840c6d9a1a5ada9669de953a0b783", "active_requests_after": 0, "runner_state_count_after": 0, "token_ids": [1206, 7995, 279, 2702, 1324, 314, 7998, 421, 628, 4973, 279, 1834, 8151, 11, 567, 1144], "token_times_s": [1.8619363659527153, 1.9497902329312637, 2.0371719649992883, 2.1247237229254097, 2.2122637999709696, 2.299665567930788, 2.387103370972909, 2.4746336039388552, 2.5620403619250283, 2.6495344799477607, 2.7369566130219027, 2.8244487019255757, 2.9127775439992547, 3.00026863894891, 3.0876826340099797, 3.175309109967202], "text_sha256": "c8bdc8989c61acbc782bbeff0e2cffd37f1b1ffe9270709e6c2cf58d34c5537f", "tokens": 16, "terminal": "length", "done": true, "ttft_token_s": 1.8619363659527153, "tpot_token_s": 0.08755818293429911, "latency_s": 3.1754319419851527} -{"phase": "request", "arm": "repair_graph", "case": "gsm8k/test/783", "repetition": 1, "prompt_tokens": 149, "prompt_sha256": "f2233be4ebb7b122d80433fe4081fdabeda64f735b7bc24f9706c4c4339b824f", "active_requests_after": 0, "runner_state_count_after": 0, "token_ids": [1206, 11290, 411, 3377, 11, 567, 1144, 310, 7995, 279, 1324, 314, 4022, 7397, 13853, 10471], "token_times_s": [2.549849965958856, 2.638156895060092, 2.7255205219844356, 2.8128517500590533, 2.900282092974521, 2.9876500560203567, 3.0750051010400057, 3.16240310005378, 3.2497186650289223, 3.3371625910513103, 3.4244724459713325, 3.5117604229599237, 3.5990703069837764, 3.6864508980652317, 3.7739032370736822, 3.8614154329989105], "text_sha256": "4b5d6370a617c539db1e6ee0df2c726c7165c1e9395eab1b873ff700a997510a", "tokens": 16, "terminal": "length", "done": true, "ttft_token_s": 2.549849965958856, "tpot_token_s": 0.0874376978026703, "latency_s": 3.8614948170725256} -{"phase": "request", "arm": "repair_graph", "case": "gsm8k/test/1", "repetition": 1, "prompt_tokens": 70, "prompt_sha256": "78d335abe81363d4bb48e5aa256c120344c2ceadbcf8739afe4ec3923c747238", "active_requests_after": 0, "runner_state_count_after": 0, "token_ids": [1206, 7995, 279, 2702, 1324, 314, 47205, 2483, 364, 279, 60507, 11, 567, 1144, 310, 10724], "token_times_s": [1.2305328489746898, 1.3179330399725586, 1.406281939940527, 1.493851792998612, 1.5813292680541053, 1.6689069319982082, 1.7563999160192907, 1.8448323199991137, 1.9323865559417754, 2.0199524289928377, 2.107789649046026, 2.19534716301132, 2.2829498739447445, 2.3704859090503305, 2.4580126570072025, 2.5455865210387856], "text_sha256": "63454f086bab80397da9b2b530254b6dfaa499fcda59fdee055d99530ff45fec", "tokens": 16, "terminal": "length", "done": true, "ttft_token_s": 1.2305328489746898, "tpot_token_s": 0.08767024480427305, "latency_s": 2.545653288019821} -{"phase": "request", "arm": "repair_graph", "case": "gsm8k/test/424", "repetition": 1, "prompt_tokens": 106, "prompt_sha256": "03cbdf5804bab62d4efbc0ae89e191bd557840c6d9a1a5ada9669de953a0b783", "active_requests_after": 0, "runner_state_count_after": 0, "token_ids": [1206, 7995, 279, 2702, 1324, 314, 7998, 421, 628, 4973, 279, 1834, 8151, 11, 567, 1144], "token_times_s": [1.851010213023983, 1.9382869970286265, 2.0256367550464347, 2.1130571749527007, 2.2003178029553965, 2.287644697003998, 2.374968877993524, 2.4622687089722604, 2.549625267041847, 2.637876559048891, 2.7254201730247587, 2.8136576620163396, 2.9011383899487555, 2.9885407460387796, 3.0760136919561774, 3.1643124859547243], "text_sha256": "c8bdc8989c61acbc782bbeff0e2cffd37f1b1ffe9270709e6c2cf58d34c5537f", "tokens": 16, "terminal": "length", "done": true, "ttft_token_s": 1.851010213023983, "tpot_token_s": 0.08755348486204942, "latency_s": 3.1644248709781095} -{"phase": "cleanup", "arm": "repair_graph", "exit_code": 0, "forced": false, "closed": true, "active_requests": 0, "runner_state_count": 0} -{"phase": "source", "arm": "repair_eager", "graph": false, "hashes": {"nanovllm/qwen38/__init__.py": "1f07e0cfd12bbf370c1f465c6a4bd73e72a5c826af1f29e60ccc7d6f45562cfc", "nanovllm/qwen38/__main__.py": "9e2a0881eedf8fa4abc6037511b42fd3a44171d121a7b2cb096023f485673ba2", "nanovllm/qwen38/_triton_fp8.py": "9f0cb68be017cb3858aebe8ba275039084e305df92cb4567d6e7da54d119f19e", "nanovllm/qwen38/_triton_rms.py": "ba99fa3ba82bfcab4cd602fd0db907acad0eee7c7a63ab23d560a17fd2f72272", "nanovllm/qwen38/config.py": "002ea25bbf9c8d17e5f1d3fb2b76c400e4291d7d2d06d5dd13fb5c3ac15c3d79", "nanovllm/qwen38/engine.py": "c47e1c72fcc24bc8eeb5497887da6be11c2de567abcc3efb275b061c061fd755", "nanovllm/qwen38/graph.py": "d4213aa68599564a01ced3b990993c9b17482615c154120bc4180f78a5da6098", "nanovllm/qwen38/loader.py": "1aa6b3321bc135fa28d1eebf6223bbe3bc5f646630193e292fb58a9f0ca2763f", "nanovllm/qwen38/model.py": "ad8386f156ff5315dfd73de313ab766fedd9ef892c7b94cec95303f11fd4e198", "nanovllm/qwen38/quant.py": "5590e4769a0fbbcaf5afeee1a93db5c4bccacf09f6f3b34e42d52ec31e9f50c3", "nanovllm/qwen38/runner.py": "3b99c999c887f6acc2f2b6aa957c01680246fe424fb3da961322c06dacf857a0", "nanovllm/qwen38/server.py": "fdbe05d82598b299eb777c99d8f8cf7f2bab685d85a073428279e6665d590543"}} -{"phase": "ready", "arm": "repair_eager", "startup_s": 13.441448123077862, "versions": {"torch": "2.14.1+cu130", "transformers": "5.20.2", "triton": "3.8.0"}, "device": "NVIDIA H20-3e", "graph_prewarm": {}, "backend": "triton_block_fp8", "fp8_execution": {"modules": 400, "total_calls": 0, "call_counter_scope": "python_forward_only_includes_warmup_and_capture_excludes_graph_replay", "backend": "triton_block_fp8", "kernel_name": "qwen38_block_fp8", "executed_modules": 0, "per_module_calls": {"model.layers.0.linear_attn.in_proj_qkv": 0, "model.layers.0.linear_attn.in_proj_z": 0, "model.layers.0.linear_attn.out_proj": 0, "model.layers.0.mlp.gate_proj": 0, "model.layers.0.mlp.up_proj": 0, "model.layers.0.mlp.down_proj": 0, "model.layers.1.linear_attn.in_proj_qkv": 0, "model.layers.1.linear_attn.in_proj_z": 0, "model.layers.1.linear_attn.out_proj": 0, "model.layers.1.mlp.gate_proj": 0, "model.layers.1.mlp.up_proj": 0, "model.layers.1.mlp.down_proj": 0, "model.layers.2.linear_attn.in_proj_qkv": 0, "model.layers.2.linear_attn.in_proj_z": 0, "model.layers.2.linear_attn.out_proj": 0, "model.layers.2.mlp.gate_proj": 0, "model.layers.2.mlp.up_proj": 0, "model.layers.2.mlp.down_proj": 0, "model.layers.3.self_attn.q_proj": 0, "model.layers.3.self_attn.k_proj": 0, "model.layers.3.self_attn.v_proj": 0, "model.layers.3.self_attn.o_proj": 0, "model.layers.3.mlp.gate_proj": 0, "model.layers.3.mlp.up_proj": 0, "model.layers.3.mlp.down_proj": 0, "model.layers.4.linear_attn.in_proj_qkv": 0, "model.layers.4.linear_attn.in_proj_z": 0, "model.layers.4.linear_attn.out_proj": 0, "model.layers.4.mlp.gate_proj": 0, "model.layers.4.mlp.up_proj": 0, "model.layers.4.mlp.down_proj": 0, "model.layers.5.linear_attn.in_proj_qkv": 0, "model.layers.5.linear_attn.in_proj_z": 0, "model.layers.5.linear_attn.out_proj": 0, "model.layers.5.mlp.gate_proj": 0, "model.layers.5.mlp.up_proj": 0, "model.layers.5.mlp.down_proj": 0, "model.layers.6.linear_attn.in_proj_qkv": 0, "model.layers.6.linear_attn.in_proj_z": 0, "model.layers.6.linear_attn.out_proj": 0, "model.layers.6.mlp.gate_proj": 0, "model.layers.6.mlp.up_proj": 0, "model.layers.6.mlp.down_proj": 0, "model.layers.7.self_attn.q_proj": 0, "model.layers.7.self_attn.k_proj": 0, "model.layers.7.self_attn.v_proj": 0, "model.layers.7.self_attn.o_proj": 0, "model.layers.7.mlp.gate_proj": 0, "model.layers.7.mlp.up_proj": 0, "model.layers.7.mlp.down_proj": 0, "model.layers.8.linear_attn.in_proj_qkv": 0, "model.layers.8.linear_attn.in_proj_z": 0, "model.layers.8.linear_attn.out_proj": 0, "model.layers.8.mlp.gate_proj": 0, "model.layers.8.mlp.up_proj": 0, "model.layers.8.mlp.down_proj": 0, "model.layers.9.linear_attn.in_proj_qkv": 0, "model.layers.9.linear_attn.in_proj_z": 0, "model.layers.9.linear_attn.out_proj": 0, "model.layers.9.mlp.gate_proj": 0, "model.layers.9.mlp.up_proj": 0, "model.layers.9.mlp.down_proj": 0, "model.layers.10.linear_attn.in_proj_qkv": 0, "model.layers.10.linear_attn.in_proj_z": 0, "model.layers.10.linear_attn.out_proj": 0, "model.layers.10.mlp.gate_proj": 0, "model.layers.10.mlp.up_proj": 0, "model.layers.10.mlp.down_proj": 0, "model.layers.11.self_attn.q_proj": 0, "model.layers.11.self_attn.k_proj": 0, "model.layers.11.self_attn.v_proj": 0, "model.layers.11.self_attn.o_proj": 0, "model.layers.11.mlp.gate_proj": 0, "model.layers.11.mlp.up_proj": 0, "model.layers.11.mlp.down_proj": 0, "model.layers.12.linear_attn.in_proj_qkv": 0, "model.layers.12.linear_attn.in_proj_z": 0, "model.layers.12.linear_attn.out_proj": 0, "model.layers.12.mlp.gate_proj": 0, "model.layers.12.mlp.up_proj": 0, "model.layers.12.mlp.down_proj": 0, "model.layers.13.linear_attn.in_proj_qkv": 0, "model.layers.13.linear_attn.in_proj_z": 0, "model.layers.13.linear_attn.out_proj": 0, "model.layers.13.mlp.gate_proj": 0, "model.layers.13.mlp.up_proj": 0, "model.layers.13.mlp.down_proj": 0, "model.layers.14.linear_attn.in_proj_qkv": 0, "model.layers.14.linear_attn.in_proj_z": 0, "model.layers.14.linear_attn.out_proj": 0, "model.layers.14.mlp.gate_proj": 0, "model.layers.14.mlp.up_proj": 0, "model.layers.14.mlp.down_proj": 0, "model.layers.15.self_attn.q_proj": 0, "model.layers.15.self_attn.k_proj": 0, "model.layers.15.self_attn.v_proj": 0, "model.layers.15.self_attn.o_proj": 0, "model.layers.15.mlp.gate_proj": 0, "model.layers.15.mlp.up_proj": 0, "model.layers.15.mlp.down_proj": 0, "model.layers.16.linear_attn.in_proj_qkv": 0, "model.layers.16.linear_attn.in_proj_z": 0, "model.layers.16.linear_attn.out_proj": 0, "model.layers.16.mlp.gate_proj": 0, "model.layers.16.mlp.up_proj": 0, "model.layers.16.mlp.down_proj": 0, "model.layers.17.linear_attn.in_proj_qkv": 0, "model.layers.17.linear_attn.in_proj_z": 0, "model.layers.17.linear_attn.out_proj": 0, "model.layers.17.mlp.gate_proj": 0, "model.layers.17.mlp.up_proj": 0, "model.layers.17.mlp.down_proj": 0, "model.layers.18.linear_attn.in_proj_qkv": 0, "model.layers.18.linear_attn.in_proj_z": 0, "model.layers.18.linear_attn.out_proj": 0, "model.layers.18.mlp.gate_proj": 0, "model.layers.18.mlp.up_proj": 0, "model.layers.18.mlp.down_proj": 0, "model.layers.19.self_attn.q_proj": 0, "model.layers.19.self_attn.k_proj": 0, "model.layers.19.self_attn.v_proj": 0, "model.layers.19.self_attn.o_proj": 0, "model.layers.19.mlp.gate_proj": 0, "model.layers.19.mlp.up_proj": 0, "model.layers.19.mlp.down_proj": 0, "model.layers.20.linear_attn.in_proj_qkv": 0, "model.layers.20.linear_attn.in_proj_z": 0, "model.layers.20.linear_attn.out_proj": 0, "model.layers.20.mlp.gate_proj": 0, "model.layers.20.mlp.up_proj": 0, "model.layers.20.mlp.down_proj": 0, "model.layers.21.linear_attn.in_proj_qkv": 0, "model.layers.21.linear_attn.in_proj_z": 0, "model.layers.21.linear_attn.out_proj": 0, "model.layers.21.mlp.gate_proj": 0, "model.layers.21.mlp.up_proj": 0, "model.layers.21.mlp.down_proj": 0, "model.layers.22.linear_attn.in_proj_qkv": 0, "model.layers.22.linear_attn.in_proj_z": 0, "model.layers.22.linear_attn.out_proj": 0, "model.layers.22.mlp.gate_proj": 0, "model.layers.22.mlp.up_proj": 0, "model.layers.22.mlp.down_proj": 0, "model.layers.23.self_attn.q_proj": 0, "model.layers.23.self_attn.k_proj": 0, "model.layers.23.self_attn.v_proj": 0, "model.layers.23.self_attn.o_proj": 0, "model.layers.23.mlp.gate_proj": 0, "model.layers.23.mlp.up_proj": 0, "model.layers.23.mlp.down_proj": 0, "model.layers.24.linear_attn.in_proj_qkv": 0, "model.layers.24.linear_attn.in_proj_z": 0, "model.layers.24.linear_attn.out_proj": 0, "model.layers.24.mlp.gate_proj": 0, "model.layers.24.mlp.up_proj": 0, "model.layers.24.mlp.down_proj": 0, "model.layers.25.linear_attn.in_proj_qkv": 0, "model.layers.25.linear_attn.in_proj_z": 0, "model.layers.25.linear_attn.out_proj": 0, "model.layers.25.mlp.gate_proj": 0, "model.layers.25.mlp.up_proj": 0, "model.layers.25.mlp.down_proj": 0, "model.layers.26.linear_attn.in_proj_qkv": 0, "model.layers.26.linear_attn.in_proj_z": 0, "model.layers.26.linear_attn.out_proj": 0, "model.layers.26.mlp.gate_proj": 0, "model.layers.26.mlp.up_proj": 0, "model.layers.26.mlp.down_proj": 0, "model.layers.27.self_attn.q_proj": 0, "model.layers.27.self_attn.k_proj": 0, "model.layers.27.self_attn.v_proj": 0, "model.layers.27.self_attn.o_proj": 0, "model.layers.27.mlp.gate_proj": 0, "model.layers.27.mlp.up_proj": 0, "model.layers.27.mlp.down_proj": 0, "model.layers.28.linear_attn.in_proj_qkv": 0, "model.layers.28.linear_attn.in_proj_z": 0, "model.layers.28.linear_attn.out_proj": 0, "model.layers.28.mlp.gate_proj": 0, "model.layers.28.mlp.up_proj": 0, "model.layers.28.mlp.down_proj": 0, "model.layers.29.linear_attn.in_proj_qkv": 0, "model.layers.29.linear_attn.in_proj_z": 0, "model.layers.29.linear_attn.out_proj": 0, "model.layers.29.mlp.gate_proj": 0, "model.layers.29.mlp.up_proj": 0, "model.layers.29.mlp.down_proj": 0, "model.layers.30.linear_attn.in_proj_qkv": 0, "model.layers.30.linear_attn.in_proj_z": 0, "model.layers.30.linear_attn.out_proj": 0, "model.layers.30.mlp.gate_proj": 0, "model.layers.30.mlp.up_proj": 0, "model.layers.30.mlp.down_proj": 0, "model.layers.31.self_attn.q_proj": 0, "model.layers.31.self_attn.k_proj": 0, "model.layers.31.self_attn.v_proj": 0, "model.layers.31.self_attn.o_proj": 0, "model.layers.31.mlp.gate_proj": 0, "model.layers.31.mlp.up_proj": 0, "model.layers.31.mlp.down_proj": 0, "model.layers.32.linear_attn.in_proj_qkv": 0, "model.layers.32.linear_attn.in_proj_z": 0, "model.layers.32.linear_attn.out_proj": 0, "model.layers.32.mlp.gate_proj": 0, "model.layers.32.mlp.up_proj": 0, "model.layers.32.mlp.down_proj": 0, "model.layers.33.linear_attn.in_proj_qkv": 0, "model.layers.33.linear_attn.in_proj_z": 0, "model.layers.33.linear_attn.out_proj": 0, "model.layers.33.mlp.gate_proj": 0, "model.layers.33.mlp.up_proj": 0, "model.layers.33.mlp.down_proj": 0, "model.layers.34.linear_attn.in_proj_qkv": 0, "model.layers.34.linear_attn.in_proj_z": 0, "model.layers.34.linear_attn.out_proj": 0, "model.layers.34.mlp.gate_proj": 0, "model.layers.34.mlp.up_proj": 0, "model.layers.34.mlp.down_proj": 0, "model.layers.35.self_attn.q_proj": 0, "model.layers.35.self_attn.k_proj": 0, "model.layers.35.self_attn.v_proj": 0, "model.layers.35.self_attn.o_proj": 0, "model.layers.35.mlp.gate_proj": 0, "model.layers.35.mlp.up_proj": 0, "model.layers.35.mlp.down_proj": 0, "model.layers.36.linear_attn.in_proj_qkv": 0, "model.layers.36.linear_attn.in_proj_z": 0, "model.layers.36.linear_attn.out_proj": 0, "model.layers.36.mlp.gate_proj": 0, "model.layers.36.mlp.up_proj": 0, "model.layers.36.mlp.down_proj": 0, "model.layers.37.linear_attn.in_proj_qkv": 0, "model.layers.37.linear_attn.in_proj_z": 0, "model.layers.37.linear_attn.out_proj": 0, "model.layers.37.mlp.gate_proj": 0, "model.layers.37.mlp.up_proj": 0, "model.layers.37.mlp.down_proj": 0, "model.layers.38.linear_attn.in_proj_qkv": 0, "model.layers.38.linear_attn.in_proj_z": 0, "model.layers.38.linear_attn.out_proj": 0, "model.layers.38.mlp.gate_proj": 0, "model.layers.38.mlp.up_proj": 0, "model.layers.38.mlp.down_proj": 0, "model.layers.39.self_attn.q_proj": 0, "model.layers.39.self_attn.k_proj": 0, "model.layers.39.self_attn.v_proj": 0, "model.layers.39.self_attn.o_proj": 0, "model.layers.39.mlp.gate_proj": 0, "model.layers.39.mlp.up_proj": 0, "model.layers.39.mlp.down_proj": 0, "model.layers.40.linear_attn.in_proj_qkv": 0, "model.layers.40.linear_attn.in_proj_z": 0, "model.layers.40.linear_attn.out_proj": 0, "model.layers.40.mlp.gate_proj": 0, "model.layers.40.mlp.up_proj": 0, "model.layers.40.mlp.down_proj": 0, "model.layers.41.linear_attn.in_proj_qkv": 0, "model.layers.41.linear_attn.in_proj_z": 0, "model.layers.41.linear_attn.out_proj": 0, "model.layers.41.mlp.gate_proj": 0, "model.layers.41.mlp.up_proj": 0, "model.layers.41.mlp.down_proj": 0, "model.layers.42.linear_attn.in_proj_qkv": 0, "model.layers.42.linear_attn.in_proj_z": 0, "model.layers.42.linear_attn.out_proj": 0, "model.layers.42.mlp.gate_proj": 0, "model.layers.42.mlp.up_proj": 0, "model.layers.42.mlp.down_proj": 0, "model.layers.43.self_attn.q_proj": 0, "model.layers.43.self_attn.k_proj": 0, "model.layers.43.self_attn.v_proj": 0, "model.layers.43.self_attn.o_proj": 0, "model.layers.43.mlp.gate_proj": 0, "model.layers.43.mlp.up_proj": 0, "model.layers.43.mlp.down_proj": 0, "model.layers.44.linear_attn.in_proj_qkv": 0, "model.layers.44.linear_attn.in_proj_z": 0, "model.layers.44.linear_attn.out_proj": 0, "model.layers.44.mlp.gate_proj": 0, "model.layers.44.mlp.up_proj": 0, "model.layers.44.mlp.down_proj": 0, "model.layers.45.linear_attn.in_proj_qkv": 0, "model.layers.45.linear_attn.in_proj_z": 0, "model.layers.45.linear_attn.out_proj": 0, "model.layers.45.mlp.gate_proj": 0, "model.layers.45.mlp.up_proj": 0, "model.layers.45.mlp.down_proj": 0, "model.layers.46.linear_attn.in_proj_qkv": 0, "model.layers.46.linear_attn.in_proj_z": 0, "model.layers.46.linear_attn.out_proj": 0, "model.layers.46.mlp.gate_proj": 0, "model.layers.46.mlp.up_proj": 0, "model.layers.46.mlp.down_proj": 0, "model.layers.47.self_attn.q_proj": 0, "model.layers.47.self_attn.k_proj": 0, "model.layers.47.self_attn.v_proj": 0, "model.layers.47.self_attn.o_proj": 0, "model.layers.47.mlp.gate_proj": 0, "model.layers.47.mlp.up_proj": 0, "model.layers.47.mlp.down_proj": 0, "model.layers.48.linear_attn.in_proj_qkv": 0, "model.layers.48.linear_attn.in_proj_z": 0, "model.layers.48.linear_attn.out_proj": 0, "model.layers.48.mlp.gate_proj": 0, "model.layers.48.mlp.up_proj": 0, "model.layers.48.mlp.down_proj": 0, "model.layers.49.linear_attn.in_proj_qkv": 0, "model.layers.49.linear_attn.in_proj_z": 0, "model.layers.49.linear_attn.out_proj": 0, "model.layers.49.mlp.gate_proj": 0, "model.layers.49.mlp.up_proj": 0, "model.layers.49.mlp.down_proj": 0, "model.layers.50.linear_attn.in_proj_qkv": 0, "model.layers.50.linear_attn.in_proj_z": 0, "model.layers.50.linear_attn.out_proj": 0, "model.layers.50.mlp.gate_proj": 0, "model.layers.50.mlp.up_proj": 0, "model.layers.50.mlp.down_proj": 0, "model.layers.51.self_attn.q_proj": 0, "model.layers.51.self_attn.k_proj": 0, "model.layers.51.self_attn.v_proj": 0, "model.layers.51.self_attn.o_proj": 0, "model.layers.51.mlp.gate_proj": 0, "model.layers.51.mlp.up_proj": 0, "model.layers.51.mlp.down_proj": 0, "model.layers.52.linear_attn.in_proj_qkv": 0, "model.layers.52.linear_attn.in_proj_z": 0, "model.layers.52.linear_attn.out_proj": 0, "model.layers.52.mlp.gate_proj": 0, "model.layers.52.mlp.up_proj": 0, "model.layers.52.mlp.down_proj": 0, "model.layers.53.linear_attn.in_proj_qkv": 0, "model.layers.53.linear_attn.in_proj_z": 0, "model.layers.53.linear_attn.out_proj": 0, "model.layers.53.mlp.gate_proj": 0, "model.layers.53.mlp.up_proj": 0, "model.layers.53.mlp.down_proj": 0, "model.layers.54.linear_attn.in_proj_qkv": 0, "model.layers.54.linear_attn.in_proj_z": 0, "model.layers.54.linear_attn.out_proj": 0, "model.layers.54.mlp.gate_proj": 0, "model.layers.54.mlp.up_proj": 0, "model.layers.54.mlp.down_proj": 0, "model.layers.55.self_attn.q_proj": 0, "model.layers.55.self_attn.k_proj": 0, "model.layers.55.self_attn.v_proj": 0, "model.layers.55.self_attn.o_proj": 0, "model.layers.55.mlp.gate_proj": 0, "model.layers.55.mlp.up_proj": 0, "model.layers.55.mlp.down_proj": 0, "model.layers.56.linear_attn.in_proj_qkv": 0, "model.layers.56.linear_attn.in_proj_z": 0, "model.layers.56.linear_attn.out_proj": 0, "model.layers.56.mlp.gate_proj": 0, "model.layers.56.mlp.up_proj": 0, "model.layers.56.mlp.down_proj": 0, "model.layers.57.linear_attn.in_proj_qkv": 0, "model.layers.57.linear_attn.in_proj_z": 0, "model.layers.57.linear_attn.out_proj": 0, "model.layers.57.mlp.gate_proj": 0, "model.layers.57.mlp.up_proj": 0, "model.layers.57.mlp.down_proj": 0, "model.layers.58.linear_attn.in_proj_qkv": 0, "model.layers.58.linear_attn.in_proj_z": 0, "model.layers.58.linear_attn.out_proj": 0, "model.layers.58.mlp.gate_proj": 0, "model.layers.58.mlp.up_proj": 0, "model.layers.58.mlp.down_proj": 0, "model.layers.59.self_attn.q_proj": 0, "model.layers.59.self_attn.k_proj": 0, "model.layers.59.self_attn.v_proj": 0, "model.layers.59.self_attn.o_proj": 0, "model.layers.59.mlp.gate_proj": 0, "model.layers.59.mlp.up_proj": 0, "model.layers.59.mlp.down_proj": 0, "model.layers.60.linear_attn.in_proj_qkv": 0, "model.layers.60.linear_attn.in_proj_z": 0, "model.layers.60.linear_attn.out_proj": 0, "model.layers.60.mlp.gate_proj": 0, "model.layers.60.mlp.up_proj": 0, "model.layers.60.mlp.down_proj": 0, "model.layers.61.linear_attn.in_proj_qkv": 0, "model.layers.61.linear_attn.in_proj_z": 0, "model.layers.61.linear_attn.out_proj": 0, "model.layers.61.mlp.gate_proj": 0, "model.layers.61.mlp.up_proj": 0, "model.layers.61.mlp.down_proj": 0, "model.layers.62.linear_attn.in_proj_qkv": 0, "model.layers.62.linear_attn.in_proj_z": 0, "model.layers.62.linear_attn.out_proj": 0, "model.layers.62.mlp.gate_proj": 0, "model.layers.62.mlp.up_proj": 0, "model.layers.62.mlp.down_proj": 0, "model.layers.63.self_attn.q_proj": 0, "model.layers.63.self_attn.k_proj": 0, "model.layers.63.self_attn.v_proj": 0, "model.layers.63.self_attn.o_proj": 0, "model.layers.63.mlp.gate_proj": 0, "model.layers.63.mlp.up_proj": 0, "model.layers.63.mlp.down_proj": 0}, "weight_dtype": "float8_e4m3fn", "scale_dtype": "float32", "actual_dtypes_valid": true}} -{"phase": "request", "arm": "repair_eager", "case": "gsm8k/test/783", "repetition": 0, "prompt_tokens": 149, "prompt_sha256": "f2233be4ebb7b122d80433fe4081fdabeda64f735b7bc24f9706c4c4339b824f", "active_requests_after": 0, "runner_state_count_after": 0, "token_ids": [1206, 11290, 411, 3377, 11, 567, 1144, 310, 7995, 279, 1324, 314, 4022, 7397, 13853, 10471], "token_times_s": [3.506790488027036, 3.6458767079748213, 3.7376974370563403, 3.831228194059804, 3.922709250007756, 4.014338386012241, 4.105827176012099, 4.197326855966821, 4.288825029972941, 4.38031348597724, 4.471941646072082, 4.563823075965047, 4.655415912042372, 4.746840746025555, 4.838389193988405, 4.929952791077085], "text_sha256": "4b5d6370a617c539db1e6ee0df2c726c7165c1e9395eab1b873ff700a997510a", "tokens": 16, "terminal": "length", "done": true, "ttft_token_s": 3.506790488027036, "tpot_token_s": 0.09487748687000325, "latency_s": 4.930077199009247} -{"phase": "request", "arm": "repair_eager", "case": "gsm8k/test/1", "repetition": 0, "prompt_tokens": 70, "prompt_sha256": "78d335abe81363d4bb48e5aa256c120344c2ceadbcf8739afe4ec3923c747238", "active_requests_after": 0, "runner_state_count_after": 0, "token_ids": [1206, 7995, 279, 2702, 1324, 314, 47205, 2483, 364, 279, 60507, 11, 567, 1144, 310, 10724], "token_times_s": [1.2539250300033018, 1.345473185996525, 1.4372812759829685, 1.5287497979588807, 1.6201732900226489, 1.711729564005509, 1.8031629989854991, 1.8947403749916703, 1.9861155389808118, 2.0776933550368994, 2.169111907016486, 2.2606806219555438, 2.3521588830044493, 2.443608472007327, 2.5349347019800916, 2.626617430942133], "text_sha256": "63454f086bab80397da9b2b530254b6dfaa499fcda59fdee055d99530ff45fec", "tokens": 16, "terminal": "length", "done": true, "ttft_token_s": 1.2539250300033018, "tpot_token_s": 0.09151282672925541, "latency_s": 2.626695593004115} -{"phase": "request", "arm": "repair_eager", "case": "gsm8k/test/424", "repetition": 0, "prompt_tokens": 106, "prompt_sha256": "03cbdf5804bab62d4efbc0ae89e191bd557840c6d9a1a5ada9669de953a0b783", "active_requests_after": 0, "runner_state_count_after": 0, "token_ids": [1206, 7995, 279, 2702, 1324, 314, 7998, 421, 628, 4973, 279, 1834, 8151, 11, 567, 1144], "token_times_s": [1.8730802100617439, 1.964716664981097, 2.0563046030001715, 2.1478036290500313, 2.239479333977215, 2.330947438022122, 2.4227813329780474, 2.514341543079354, 2.605788634973578, 2.697387801017612, 2.788881944026798, 2.8804332750150934, 2.971950498991646, 3.0635281620780006, 3.15502143104095, 3.246827915078029], "text_sha256": "c8bdc8989c61acbc782bbeff0e2cffd37f1b1ffe9270709e6c2cf58d34c5537f", "tokens": 16, "terminal": "length", "done": true, "ttft_token_s": 1.8730802100617439, "tpot_token_s": 0.091583180334419, "latency_s": 3.2469369469908997} -{"phase": "request", "arm": "repair_eager", "case": "gsm8k/test/783", "repetition": 1, "prompt_tokens": 149, "prompt_sha256": "f2233be4ebb7b122d80433fe4081fdabeda64f735b7bc24f9706c4c4339b824f", "active_requests_after": 0, "runner_state_count_after": 0, "token_ids": [1206, 11290, 411, 3377, 11, 567, 1144, 310, 7995, 279, 1324, 314, 4022, 7397, 13853, 10471], "token_times_s": [2.5452123020077124, 2.6369570560054854, 2.728505673003383, 2.8199319270206615, 2.911390113993548, 3.002880922984332, 3.0941876879660413, 3.1856958859134465, 3.2771183169679716, 3.3685744130052626, 3.460168404970318, 3.551574549986981, 3.6431491979165003, 3.7345546989236027, 3.826009591924958, 3.917477910988964], "text_sha256": "4b5d6370a617c539db1e6ee0df2c726c7165c1e9395eab1b873ff700a997510a", "tokens": 16, "terminal": "length", "done": true, "ttft_token_s": 2.5452123020077124, "tpot_token_s": 0.09148437393208345, "latency_s": 3.917557892971672} -{"phase": "request", "arm": "repair_eager", "case": "gsm8k/test/1", "repetition": 1, "prompt_tokens": 70, "prompt_sha256": "78d335abe81363d4bb48e5aa256c120344c2ceadbcf8739afe4ec3923c747238", "active_requests_after": 0, "runner_state_count_after": 0, "token_ids": [1206, 7995, 279, 2702, 1324, 314, 47205, 2483, 364, 279, 60507, 11, 567, 1144, 310, 10724], "token_times_s": [1.2274330699583516, 1.318991250009276, 1.4102913519600406, 1.5016648749588057, 1.593066724948585, 1.6846574869705364, 1.7759370219428092, 1.8674776930129156, 1.9587591419694945, 2.0502618580358103, 2.1416087199468166, 2.233056063996628, 2.3245767110493034, 2.415939082042314, 2.5072422199882567, 2.598776143975556], "text_sha256": "63454f086bab80397da9b2b530254b6dfaa499fcda59fdee055d99530ff45fec", "tokens": 16, "terminal": "length", "done": true, "ttft_token_s": 1.2274330699583516, "tpot_token_s": 0.09142287160114695, "latency_s": 2.5988404089584947} -{"phase": "request", "arm": "repair_eager", "case": "gsm8k/test/424", "repetition": 1, "prompt_tokens": 106, "prompt_sha256": "03cbdf5804bab62d4efbc0ae89e191bd557840c6d9a1a5ada9669de953a0b783", "active_requests_after": 0, "runner_state_count_after": 0, "token_ids": [1206, 7995, 279, 2702, 1324, 314, 7998, 421, 628, 4973, 279, 1834, 8151, 11, 567, 1144], "token_times_s": [1.8544134719995782, 1.9460022939601913, 2.037392426049337, 2.1290493060369045, 2.2204010209534317, 2.31183753896039, 2.4031987589551136, 2.4946893060114235, 2.5861230220180005, 2.6775193649809808, 2.768937823944725, 2.8603260229574516, 2.951878665946424, 3.043289183988236, 3.1346634769579396, 3.226205014041625], "text_sha256": "c8bdc8989c61acbc782bbeff0e2cffd37f1b1ffe9270709e6c2cf58d34c5537f", "tokens": 16, "terminal": "length", "done": true, "ttft_token_s": 1.8544134719995782, "tpot_token_s": 0.09145276946946979, "latency_s": 3.226272381027229} -{"phase": "cleanup", "arm": "repair_eager", "exit_code": 0, "forced": false, "closed": true, "active_requests": 0, "runner_state_count": 0} -{"phase": "complete", "verdict": "INCONCLUSIVE", "scope": "readiness_only"} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/http-summary.json b/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/http-summary.json deleted file mode 100644 index 9b290bea4..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/http-summary.json +++ /dev/null @@ -1,80 +0,0 @@ -{ - "verdict": "INCONCLUSIVE", - "job": "1943", - "allocated_gpu_seconds": 144, - "budget_stage": "readiness", - "budget_total_gpu_hours": 44, - "base": "83b567c1b07d9a230f6a8ca8bff979144d719241", - "candidate": "3e8636f04378ef7d316272f2051315515a9ac6b9", - "raw_sha256": "75cafaf44d76b7cb382986901f8dc6e9fce56e8bf66c86fa91e14de8b1e56cbe", - "pairs": [ - { - "case": "gsm8k/test/783", - "repetition": 0, - "main_graph_equals_eager": false, - "repair_graph_equals_eager": true, - "repair_main_ttft_ratio": 0.9810292762707812, - "repair_main_tpot_ratio": 0.9934148183408894, - "repair_main_latency_ratio": 0.985105766329715 - }, - { - "case": "gsm8k/test/783", - "repetition": 1, - "main_graph_equals_eager": false, - "repair_graph_equals_eager": true, - "repair_main_ttft_ratio": 0.9935685531668889, - "repair_main_tpot_ratio": 0.99314261567963, - "repair_main_latency_ratio": 0.9934239693257632 - }, - { - "case": "gsm8k/test/1", - "repetition": 0, - "main_graph_equals_eager": true, - "repair_graph_equals_eager": true, - "repair_main_ttft_ratio": 1.0166187156569488, - "repair_main_tpot_ratio": 0.9958867053709152, - "repair_main_latency_ratio": 1.0062624664354571 - }, - { - "case": "gsm8k/test/1", - "repetition": 1, - "main_graph_equals_eager": true, - "repair_graph_equals_eager": true, - "repair_main_ttft_ratio": 0.9938667756105665, - "repair_main_tpot_ratio": 0.993681114057934, - "repair_main_latency_ratio": 0.9937733013130577 - }, - { - "case": "gsm8k/test/424", - "repetition": 0, - "main_graph_equals_eager": true, - "repair_graph_equals_eager": true, - "repair_main_ttft_ratio": 0.9575216284560268, - "repair_main_tpot_ratio": 0.9950195586009974, - "repair_main_latency_ratio": 0.9726935217400172 - }, - { - "case": "gsm8k/test/424", - "repetition": 1, - "main_graph_equals_eager": true, - "repair_graph_equals_eager": true, - "repair_main_ttft_ratio": 0.9912135269122831, - "repair_main_tpot_ratio": 0.9946624376188519, - "repair_main_latency_ratio": 0.9926529054488183 - } - ], - "cpu": { - "1944": { - "passed": 155, - "skipped": 1, - "failed": 1, - "log_sha256": "05374f1ed45332cf788b273e93a08085b3f38955ad93a6ee1fc36d0475237069" - }, - "1945": { - "passed": 1, - "failed": 1, - "log_sha256": "5227bab4cf9d429e03c3c2388bcc7d9350e54e73256c1d3a39a091a7418f3f92" - }, - "baseline_failure_identical": true - } -} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/http-timings.csv b/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/http-timings.csv deleted file mode 100644 index bcd9bc42e..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/http-timings.csv +++ /dev/null @@ -1,19 +0,0 @@ -arm,case,repetition,prompt_tokens,tokens,ttft_token_s,tpot_token_s,latency_s,terminal,done -main_graph,gsm8k/test/783,0,149,16,2.692203041049652,0.08806464246784647,4.013243875000626,length,True -main_graph,gsm8k/test/1,0,70,16,1.3219671979313716,0.08798400547044972,2.641791876987554,length,True -main_graph,gsm8k/test/424,0,106,16,1.9445371369365603,0.08799644406729688,3.2645760159939528,length,True -main_graph,gsm8k/test/783,1,149,16,2.5663553439080715,0.08804143173620105,3.887056217994541,length,True -main_graph,gsm8k/test/1,1,70,16,1.2381265569711104,0.08822774586733431,2.5616036219289526,length,True -main_graph,gsm8k/test/424,1,106,16,1.8674182330723852,0.08802331479576726,3.1878462790045887,length,True -repair_graph,gsm8k/test/783,0,149,16,2.641130000934936,0.08748472079945108,3.9534696829505265,length,True -repair_graph,gsm8k/test/1,0,70,16,1.3439365949016064,0.08762210133330275,2.6583360099466518,length,True -repair_graph,gsm8k/test/424,0,106,16,1.8619363659527153,0.08755818293429911,3.1754319419851527,length,True -repair_graph,gsm8k/test/783,1,149,16,2.549849965958856,0.0874376978026703,3.8614948170725256,length,True -repair_graph,gsm8k/test/1,1,70,16,1.2305328489746898,0.08767024480427305,2.545653288019821,length,True -repair_graph,gsm8k/test/424,1,106,16,1.851010213023983,0.08755348486204942,3.1644248709781095,length,True -repair_eager,gsm8k/test/783,0,149,16,3.506790488027036,0.09487748687000325,4.930077199009247,length,True -repair_eager,gsm8k/test/1,0,70,16,1.2539250300033018,0.09151282672925541,2.626695593004115,length,True -repair_eager,gsm8k/test/424,0,106,16,1.8730802100617439,0.091583180334419,3.2469369469908997,length,True -repair_eager,gsm8k/test/783,1,149,16,2.5452123020077124,0.09148437393208345,3.917557892971672,length,True -repair_eager,gsm8k/test/1,1,70,16,1.2274330699583516,0.09142287160114695,2.5988404089584947,length,True -repair_eager,gsm8k/test/424,1,106,16,1.8544134719995782,0.09145276946946979,3.226272381027229,length,True diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/ipc-preflight-1960.json b/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/ipc-preflight-1960.json deleted file mode 100644 index beb9336b5..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/ipc-preflight-1960.json +++ /dev/null @@ -1,29 +0,0 @@ -{ - "scope": "CPU_only_transport_preflight", - "job": 1960, - "exit_code": 0, - "elapsed_seconds": 1, - "gpu_seconds": 0, - "outcomes": [ - { - "kind": "artifact_directory", - "ok": false, - "errno": 95, - "endpoint_bytes": 92 - }, - { - "kind": "short_task_directory", - "ok": false, - "errno": 95, - "endpoint_bytes": 70 - }, - { - "kind": "linux_abstract", - "ok": true, - "endpoint_bytes": 59 - } - ], - "ninja_on_corrected_path": true, - "nvcc_exists": true, - "allocations_released": true -} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/quality-activation-review.md b/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/quality-activation-review.md deleted file mode 100644 index 60fe9ee04..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/quality-activation-review.md +++ /dev/null @@ -1,41 +0,0 @@ -# Quality preparation activation review - -Refs [Issue67](https://github.com/InferMatrixLab/InferRSI/issues/67). -**INCONCLUSIVE. No model quality score or generated-code execution.** - -Independent read-only audit verified all eight archived v2 helper hashes against -the preparation manifest. Dataset/cohort preparation counts are1319 GSM8K, -164 HumanEval+ and1000 MMLU; CPU preparation remains distinct from evaluation. -No final-set contents or outputs were inspected during this review. - -Three blockers remain before activation: - -1. The archived SSE parser treats `prompt_token_ids:null` on later choices as a - repeated echo. Replay of both ordinary and coalesced raw readiness streams - fails. Removing only these null optional fields in an in-memory diagnosis - lets the128K8-token/7-choice example parse. Preserve archived v2; fix a new - version that requires exactly one non-null first-choice echo. Coalescing - itself is already supported by this quality parser. -2. Arbitrary non-EOS output IDs with finish_reason=stop pass the current parser. - Bind effective EOS/stop configuration and independently authenticate natural - EOS versus capped termination before claiming workload integrity. -3. HumanEval+ still has no qualified generated-code executor. Namespace - availability and a proposed benign probe are insufficient. The scorer's - image_digest declaration cannot stand in for the proposed bubblewrap - readonly-bind rootfs_digest. Qualify actual containment, resource/descendant - cleanup and exact full base+Plus invocation, then use a truthful versioned - provenance contract and bind samples, dataset and results. - -Missing/duplicate responses, raw/summary mismatch, invalid counts and missing or -repeated DONE fail closed in the inspected helpers. Exact per-task comparison -implements one percentage point as at most13/1/10 lost correct items for these -three final denominators. No bar is changed by identifying these gaps. - -The original source archive and manifest are retained privately. Archive SHA256: -9c9cddd3242c00b6c39e4a6342a4ab7d17ab3e74f07bbd7c628c6952f4517099; -manifest SHA256:4ac94ae332467ab7d3221906c571ada1836ec45382159d8677cf9f30e36c8c9f. -This is failed compatibility/unfinished qualification evidence, to be followed -by prospective repairs; no old artifact is rewritten or reclassified as PASS. - -KnowledgeStore candidate `case.qwen38-quality-activation-gaps-cand` revision -`sha256:eaf0aee4c2a45c339daf99c70ef9a92c24dfc1118963ed67fd30f623eeeabcd3` was written and read back exactly; no library promotion. diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/quality-binding-gap.json b/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/quality-binding-gap.json deleted file mode 100644 index f9930d914..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/quality-binding-gap.json +++ /dev/null @@ -1,22 +0,0 @@ -{ - "scope": "synthetic_contract_diagnostic", - "verdict": "INCONCLUSIVE", - "fabricated_summaries_only": true, - "real_quality_evaluation": false, - "score_sha256": "79e93cf0cd73750821a438dc17380eead9883c1c6b84c75dff99c29c3cf96005", - "observations": [ - { - "fixture": "absent", - "comparator_returned": "PASS" - }, - { - "fixture": "runtime-mismatch", - "comparator_returned": "PASS" - }, - { - "fixture": "qualification-mismatch", - "comparator_returned": "PASS" - } - ], - "limitation": "This directly calls compare with synthetic summaries; it does not establish that score accepted real mismatched sandbox executions. Score currently omits sandbox identity from its summary, while compare does not require or compare it." -} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/quality-binding-impact.md b/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/quality-binding-impact.md deleted file mode 100644 index 1fc22e4d9..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/quality-binding-impact.md +++ /dev/null @@ -1,57 +0,0 @@ -# Prospective evaluator provenance binding - -Refs [Issue67](https://github.com/InferMatrixLab/InferRSI/issues/67). -**INCONCLUSIVE — reviewed harness repair; no acceptance activation or E2E gain.** - -The previous comparator returned task-quality PASS for three fabricated count -summaries with absent or different runtime/qualification identities. These were -direct comparator fixtures, not model answers or real sandbox results; they do -not establish that the complete scorer accepted a mismatched real execution. -The previous scorer also omitted sandbox identity from its summary. - -Version5 requires a separately frozen evaluator protocol and qualification -artifact for completed HumanEval+ scoring and comparison. The protocol identifies -the readonly bind set (not an invented OCI image), evaluator revision, executor, -driver, sandbox binary, policy and qualification. Each side retains its own -execution and result hashes; samples/results may differ. Both sides must match -the independently supplied expected protocol. Pairwise equality alone is -insufficient. Export without evaluation remains possible and incomplete. - -Protocol, qualification, execution record, HumanEval results and comparison -summaries are parsed and authenticated from the same byte snapshot. Duplicate -provenance keys are rejected. The comparator requires seven correctly formed -per-arm evidence digests. The source preparation inventory and validator identity -also authenticate the new provenance module. Prompt construction, natural EOS, -task counts and accepted quality thresholds are unchanged. - -Independent review rejected the initial version for three P2 issues: the new -module was missing from the preparation manifest, authentication and parsing -reread the protocol/qualification files, and comparison allowed absent execution -and result digests. All three were corrected and the exact four changed source -files reviewed again. Rejected source identities and control results are retained; -older validators and historical evidence remain unchanged. - -Local validation:117 provenance controls and6 full synthetic export/score controls -passed under optimized Python;35 existing protocol,7 preparation guard and3 new -manifest tests passed under pytest. The19 EOS controls and30 retained retrieval -streams also passed. The new repeated-read control initially reused an exclusive -output pathname and failed with FileExistsError; a distinct output fixed the -fixture. Targeted source/control Ruff checks passed after import/format cleanup. -No full-directory lint claim is made for unchanged legacy diagnostics. - -The control scripts are retained under patch/diagnostics. Set QUALITY_SOURCE_DIR -to the reconstructed v5 directory for both standalone check scripts and guard -pytest. For the guard test, set QUALITY_REFERENCE_TEST_DIR to the authenticated -original test_prepare_guard.py directory; its digest is recorded in the inventory. -The controls use fabricated data and never execute generated code. Qualification -fixture PASS values are not qualification artifacts. - -This repair authenticates selected identities and report scope; it does not by -itself establish host isolation, anti-cheating, aggregate resource limits or -cleanup. No actual qualified generated-code executor report exists. It must be -independently reviewed before its bytes are accepted into the eventual freeze. -The complete runtime, commands and validator bundle still need freeze, alongside -the owner's pending statistical/retrieval decisions. No GPU work, held-out task -reading, evaluator execution or measurable service performance change occurred. - -KnowledgeStore candidate `case.qwen38-quality-activation-gaps-cand`: `sha256:bb6879aead15810a26d83fa05ff4f6332011bd9c911a4c85962e00bfc114da9f`; readback verified, previous attempts retained. diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/quality-binding-v5-controls.json b/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/quality-binding-v5-controls.json deleted file mode 100644 index 55f594feb..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/quality-binding-v5-controls.json +++ /dev/null @@ -1,126 +0,0 @@ -{ - "scope": "fabricated_provenance_fixtures_only", - "verdict": "INCONCLUSIVE", - "checks_passed": 117, - "checks": [ - "valid-synthetic-binding", - "missing-independent-protocol", - "wrong-protocol-file-hash", - "missing-qualification", - "changed-qualification-bytes", - "benign-scope", - "unqualified", - "failed-qualification", - "wrong-qualified-runtime", - "bad-protocol-schema_version", - "bad-protocol-runtime_kind", - "bad-protocol-runtime_digest", - "bad-protocol-evaluator_revision", - "duplicate-protocol-key", - "bad-execution-protocol-{'schema_ver", - "bad-execution-network-host", - "bad-execution-exit_code-False", - "bad-execution-exit_code-1", - "bad-execution-timed_out-True", - "bad-execution-cleanup_complete-False", - "bad-execution-qualified_generated_code_executor-False", - "bad-execution-results_sha256-000000000000", - "bad-execution-samples_sha256-None", - "duplicate-execution-key", - "comparison-missing", - "comparison-different", - "comparison-same-wrong", - "comparison-wrong-freeze-hash", - "comparison-bool-version", - "baseline-run_manifest_sha256-missing", - "baseline-run_manifest_sha256-None", - "baseline-run_manifest_sha256-", - "baseline-run_manifest_sha256-GGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGG", - "baseline-run_manifest_sha256-aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", - "baseline-run_manifest_sha256-0", - "baseline-responses_sha256-missing", - "baseline-responses_sha256-None", - "baseline-responses_sha256-", - "baseline-responses_sha256-GGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGG", - "baseline-responses_sha256-aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", - "baseline-responses_sha256-0", - "baseline-prepared_manifest_sha256-missing", - "baseline-prepared_manifest_sha256-None", - "baseline-prepared_manifest_sha256-", - "baseline-prepared_manifest_sha256-GGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGG", - "baseline-prepared_manifest_sha256-aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", - "baseline-prepared_manifest_sha256-0", - "baseline-tokenized_manifest_sha256-missing", - "baseline-tokenized_manifest_sha256-None", - "baseline-tokenized_manifest_sha256-", - "baseline-tokenized_manifest_sha256-GGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGG", - "baseline-tokenized_manifest_sha256-aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", - "baseline-tokenized_manifest_sha256-0", - "baseline-sandbox_record_sha256-missing", - "baseline-sandbox_record_sha256-None", - "baseline-sandbox_record_sha256-", - "baseline-sandbox_record_sha256-GGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGG", - "baseline-sandbox_record_sha256-aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", - "baseline-sandbox_record_sha256-0", - "baseline-human_results_sha256-missing", - "baseline-human_results_sha256-None", - "baseline-human_results_sha256-", - "baseline-human_results_sha256-GGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGG", - "baseline-human_results_sha256-aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", - "baseline-human_results_sha256-0", - "baseline-sandbox_protocol_sha256-missing", - "baseline-sandbox_protocol_sha256-None", - "baseline-sandbox_protocol_sha256-", - "baseline-sandbox_protocol_sha256-GGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGG", - "baseline-sandbox_protocol_sha256-aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", - "baseline-sandbox_protocol_sha256-0", - "candidate-run_manifest_sha256-missing", - "candidate-run_manifest_sha256-None", - "candidate-run_manifest_sha256-", - "candidate-run_manifest_sha256-GGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGG", - "candidate-run_manifest_sha256-aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", - "candidate-run_manifest_sha256-0", - "candidate-responses_sha256-missing", - "candidate-responses_sha256-None", - "candidate-responses_sha256-", - "candidate-responses_sha256-GGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGG", - "candidate-responses_sha256-aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", - "candidate-responses_sha256-0", - "candidate-prepared_manifest_sha256-missing", - "candidate-prepared_manifest_sha256-None", - "candidate-prepared_manifest_sha256-", - "candidate-prepared_manifest_sha256-GGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGG", - "candidate-prepared_manifest_sha256-aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", - "candidate-prepared_manifest_sha256-0", - "candidate-tokenized_manifest_sha256-missing", - "candidate-tokenized_manifest_sha256-None", - "candidate-tokenized_manifest_sha256-", - "candidate-tokenized_manifest_sha256-GGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGG", - "candidate-tokenized_manifest_sha256-aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", - "candidate-tokenized_manifest_sha256-0", - "candidate-sandbox_record_sha256-missing", - "candidate-sandbox_record_sha256-None", - "candidate-sandbox_record_sha256-", - "candidate-sandbox_record_sha256-GGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGG", - "candidate-sandbox_record_sha256-aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", - "candidate-sandbox_record_sha256-0", - "candidate-human_results_sha256-missing", - "candidate-human_results_sha256-None", - "candidate-human_results_sha256-", - "candidate-human_results_sha256-GGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGG", - "candidate-human_results_sha256-aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", - "candidate-human_results_sha256-0", - "candidate-sandbox_protocol_sha256-missing", - "candidate-sandbox_protocol_sha256-None", - "candidate-sandbox_protocol_sha256-", - "candidate-sandbox_protocol_sha256-GGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGG", - "candidate-sandbox_protocol_sha256-aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", - "candidate-sandbox_protocol_sha256-0", - "duplicate-summary-key", - "comparison-matched-frozen-identity-distinct-arm-digests", - "protocol-and-qualification-single-read", - "summary-digests-bind-parsed-snapshot" - ], - "real_evaluator_executed": false, - "qualification_established": false -} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/quality-eos-impact.md b/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/quality-eos-impact.md deleted file mode 100644 index a0a503a7e..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/quality-eos-impact.md +++ /dev/null @@ -1,46 +0,0 @@ -# Prospective natural-EOS authentication - -Refs [Issue67](https://github.com/InferMatrixLab/InferRSI/issues/67). -**INCONCLUSIVE — a CPU-validated protocol successor, not quality acceptance.** - -The previous quality parser accepted finish_reason=stop with arbitrary non-EOS -IDs. Version4 binds the authenticated generation configuration in both the -collector and scorer. Formal CLI calls require the configuration and its expected -SHA256. The run records the termination policy; scoring rereads/authenticates the -configuration, rejects policy drift and rechecks raw SSE with the exact EOS IDs. -Matched task summaries must have identical termination policy. Legacy low-level -parse calls without an EOS list remain available only for compatibility tests; -formal collection/scoring pass the authenticated list. - -For greedy natural EOS without user stop strings/tokens, output may end on either -configured EOS, with no earlier EOS followed by more tokens. A stop terminal must -have an EOS final token and absent/null or matching integer stop_reason. A length -terminal must reach the cap, have no EOS token, and no stop_reason. EOS takes -precedence at the cap, matching the pinned upstream scheduler and native engine. -This changes no prompt, scoring extraction, task threshold or model semantics. - -Parent checks:19 targeted cases,30 retained retrieval SSE streams and35 -existing synthetic protocol tests pass. The35-test rerun explicitly imported and -authenticated the version4 run/score modules before pytest collection to avoid -path shadowing. Independent review reran19+30 and added5 end-to-end synthetic -configuration/policy binding checks. Only run.py and score.py change from v3; -no blocking finding. No held-out task contents, model execution, generated code -execution, package changes or GPU work occurred. - -The preserved invalid non-EOS-stop example motivates this prospective tightening. -Old validators and historical outcomes remain unchanged; all30 existing retrieval -streams also satisfy the new checks, without relabeling the original run as a -frozen acceptance result. There is no measured E2E performance effect. - -Source identities and reconstruction steps are in plan/quality-v4-source-identities.json. -The configuration SHA, runtime commands and complete validator bundle still need -final freeze binding before evaluation. HumanEval+ sandbox qualification and owner -statistical/retrieval confirmation remain open; this is not search activation. - -Primary source comparison: vLLM0.32.0 v1/core/sched/utils.py check_stop and -sampling_params.py update_from_generation_config at the source revision recorded -in Issue67; candidate engine.py reads the same generation configuration and checks -EOS before the length cap. Public model provenance remains in Issue67. - -KnowledgeStore candidate `case.qwen38-quality-activation-gaps-cand` advanced to -`sha256:82466b2ab5dfb771d9c86d15a36848d6de2d0ff97e384e83e2b13796b54aea07` with verified readback; prior failures retained. diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/quality-eos-v4-check.json b/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/quality-eos-v4-check.json deleted file mode 100644 index ad0102e14..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/quality-eos-v4-check.json +++ /dev/null @@ -1,42 +0,0 @@ -{ - "verdict": "INCONCLUSIVE", - "scope": "prospective_natural_eos_validator_CPU_checks", - "source_sha256": { - "run.py": "c0102c31c713758c0a35379e79cd162189ad44511d8ba90e1e5f85ab0d3f4af9", - "score.py": "79e93cf0cd73750821a438dc17380eead9883c1c6b84c75dff99c29c3cf96005" - }, - "policy": { - "mode": "natural_eos_no_user_stop", - "generation_config_sha256": "e70c136c1b78ddc1fb0905bac8e733a4dc448d4f852a5dd75143fffc70be550e", - "eos_token_ids": [ - 248044, - 248046 - ], - "ignore_eos": false - }, - "synthetic_checks_passed": 19, - "checks": [ - "primary_eos", - "secondary_eos_null", - "secondary_eos_token_reason", - "eos_at_cap_precedes_length", - "full_cap_without_eos", - "non_eos_stop", - "continuation_after_eos", - "early_eos_followed_by_length", - "eos_final_marked_length", - "foreign_integer_stop_reason", - "string_stop_reason", - "boolean_stop_reason", - "length_has_stop_reason", - "short_length", - "empty_allowlist", - "bool_allowlist", - "missing_output_ids", - "changed_config_rejected", - "malformed_config_rejected" - ], - "retained_retrieval_raw_replays_passed": 30, - "gpu_work_performed": false, - "holdout_data_read": false -} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/quality-eos-v5-check.json b/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/quality-eos-v5-check.json deleted file mode 100644 index 741aefb6b..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/quality-eos-v5-check.json +++ /dev/null @@ -1,42 +0,0 @@ -{ - "verdict": "INCONCLUSIVE", - "scope": "prospective_natural_eos_validator_CPU_checks", - "source_sha256": { - "run.py": "c0102c31c713758c0a35379e79cd162189ad44511d8ba90e1e5f85ab0d3f4af9", - "score.py": "1e5d454c50c8ec742f74281938a5b28d49f1e815cb058274085e90a1d82bfaaa" - }, - "policy": { - "mode": "natural_eos_no_user_stop", - "generation_config_sha256": "e70c136c1b78ddc1fb0905bac8e733a4dc448d4f852a5dd75143fffc70be550e", - "eos_token_ids": [ - 248044, - 248046 - ], - "ignore_eos": false - }, - "synthetic_checks_passed": 19, - "checks": [ - "primary_eos", - "secondary_eos_null", - "secondary_eos_token_reason", - "eos_at_cap_precedes_length", - "full_cap_without_eos", - "non_eos_stop", - "continuation_after_eos", - "early_eos_followed_by_length", - "eos_final_marked_length", - "foreign_integer_stop_reason", - "string_stop_reason", - "boolean_stop_reason", - "length_has_stop_reason", - "short_length", - "empty_allowlist", - "bool_allowlist", - "missing_output_ids", - "changed_config_rejected", - "malformed_config_rejected" - ], - "retained_retrieval_raw_replays_passed": 30, - "gpu_work_performed": false, - "holdout_data_read": false -} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/quality-null-echo-check.json b/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/quality-null-echo-check.json deleted file mode 100644 index 6b1905990..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/quality-null-echo-check.json +++ /dev/null @@ -1,50 +0,0 @@ -{ - "scope": "prospective_quality_transport_fix_only", - "verdict": "INCONCLUSIVE", - "original_run_sha256": "3a5685f5d5728e6b28521fb2385e407c9b21e777ff96a15612f0b9832def8baf", - "candidate_run_sha256": "5255b0fe1aa6896f57e9dc7cfcf61aab5b03104186cafd55a1358321bea2b7dc", - "source_archive_sha256": "9c9cddd3242c00b6c39e4a6342a4ab7d17ab3e74f07bbd7c628c6952f4517099", - "cases": [ - { - "index": 0, - "input_tokens": 32, - "output_tokens": 8, - "wire_sha256": "b0a324d872cadaaa510d8ea834429ce02150d1f115f7ab6af6f130fb4335cacc", - "old_error": "prompt IDs must occur once in the first choice", - "candidate_parse_valid": true - }, - { - "index": 1, - "input_tokens": 32, - "output_tokens": 8, - "wire_sha256": "f908a8271c0b9a61997d3d48f75fcb14a76e227e36d7d1901ce6d32c685d31d5", - "old_error": "prompt IDs must occur once in the first choice", - "candidate_parse_valid": true - }, - { - "index": 2, - "input_tokens": 8192, - "output_tokens": 32, - "wire_sha256": "2e1b811c1713c7216f530cc453ee306da3bf315c9a017227d007757b1f62991d", - "old_error": "prompt IDs must occur once in the first choice", - "candidate_parse_valid": true - }, - { - "index": 3, - "input_tokens": 65536, - "output_tokens": 8, - "wire_sha256": "576105c92deb99d4e1af0ca2b5562d20515ca9f1e43141d8a0ae689904703c68", - "old_error": "prompt IDs must occur once in the first choice", - "candidate_parse_valid": true - }, - { - "index": 4, - "input_tokens": 131072, - "output_tokens": 8, - "wire_sha256": "72b8195b0af091869382a5efb61897a3648dd5bbef5372160a6bc2e135d431a1", - "old_error": "prompt IDs must occur once in the first choice", - "candidate_parse_valid": true - } - ], - "duplicate_nonnull_rejected": true -} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/quality-null-echo-impact.md b/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/quality-null-echo-impact.md deleted file mode 100644 index 732efd1dc..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/quality-null-echo-impact.md +++ /dev/null @@ -1,31 +0,0 @@ -# Prospective quality SSE null-echo repair - -Refs [Issue67](https://github.com/InferMatrixLab/InferRSI/issues/67). -**INCONCLUSIVE — compatibility repair, no task quality or performance verdict.** - -The archived v2 parser interprets a present but null prompt_token_ids field as -another prompt echo. All five retained baseline1961 response streams reproduce -that error. A one-line versioned successor treats null/absent as no echo while -retaining the exact, once-only, non-null first-choice identity requirement. -Five of five original streams parse in the successor; genuine duplicate echoes -still fail. These are CPU replays of completed service responses, not a new GPU -run or end-to-end performance improvement. Original1961 observer failure and v2 -quality source/results are unchanged. - -Independent review verified the patch, new run.py identity and unchanged seven -other archive members. Eleven focused checks covered the five real responses, -null/absent fields and rejection of duplicate, late, missing and wrong echoes. -No blocking finding; the narrow successor is suitable for publication. - -New run.py SHA2565255b0fe1aa6896f57e9dc7cfcf61aab5b03104186cafd55a1358321bea2b7dc. -Patch SHA2565411013bdec21cab01b70b303cf05e3c2fa753897062eb75bda6792cb5d2efbf. -The full original archive and versioned successor stay in controlled storage; -apply the committed patch to the archive bound in quality-activation-review.md -for reproduction. No package, model, dataset, scorer or gate changed. - -A new source manifest and full freeze must bind the successor before execution. -Natural-EOS/stop authentication and HumanEval+ sandbox qualification remain open. -This repair does not remove those activation blockers or authorize search. - -KnowledgeStore candidate `case.qwen38-quality-activation-gaps-cand` advanced to -`sha256:8bc1297472e32767d457a37c6f56179e4832c45b332e593784e63fe66859ecd7` with exact readback; prior failure history retained. diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/quality-score-v5-controls.json b/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/quality-score-v5-controls.json deleted file mode 100644 index cf46d3237..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/quality-score-v5-controls.json +++ /dev/null @@ -1,15 +0,0 @@ -{ - "scope": "handwritten_synthetic_scoring_fixture", - "verdict": "INCONCLUSIVE", - "checks": [ - "export-without-evaluation", - "full-score-binds-frozen-protocol", - "full-score-reads-provenance-once", - "full-score-rejects-protocol", - "full-score-rejects-samples_sha256", - "full-score-rejects-dataset_sha256" - ], - "checks_passed": 6, - "real_evaluator_executed": false, - "real_quality_result": false -} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/repair-summary.json b/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/repair-summary.json deleted file mode 100644 index 511ff064e..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/repair-summary.json +++ /dev/null @@ -1,13 +0,0 @@ -{ - "verdict": "INCONCLUSIVE", - "scope": "bounded_graph_correctness_diagnostic", - "prefill_cases": 3, - "prefill_compact_buffered_exact": 3, - "steps": 48, - "decode_compact_buffered_exact": 35, - "decode_buffered_graph_exact": 48, - "graph_state_exact": 48, - "top1_all_equal": 48, - "complete": true, - "raw_sha256": "5f8c16f6899d8f8f22426a1466a3afe7a7685a349d0bf2fd1fb26aae542404d6" -} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/retrieval-baseline-1968.json b/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/retrieval-baseline-1968.json deleted file mode 100644 index 85784829d..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/retrieval-baseline-1968.json +++ /dev/null @@ -1,541 +0,0 @@ -{ - "verdict": "INCONCLUSIVE", - "scope": "baseline_retrieval_qualification_only_no_candidate", - "job_id": 1968, - "observer_sha256": "10f7d81bd971337f237ed9a20ff704ee7bcd84cfbd769a9e94b049bf6d21d671", - "observations_sha256": "b8d1e8fff097fb82696a02351b18eda3999e28b51e5d6f992c9704ae5c6ef8b2", - "requests_verified": 30, - "by_length": { - "8192": { - "complete": 10, - "expected": 10, - "position_matches": [ - 10, - 10, - 10, - 10 - ], - "generated_tokens": 1289, - "requests_with_coalescing": 0, - "descriptive_median_ttft_s": 1.6782136439578608, - "descriptive_median_latency_s": 3.0286843659123406 - }, - "65536": { - "complete": 10, - "expected": 10, - "position_matches": [ - 10, - 10, - 10, - 10 - ], - "generated_tokens": 1236, - "requests_with_coalescing": 0, - "descriptive_median_ttft_s": 18.607111204997636, - "descriptive_median_latency_s": 20.278450252022594 - }, - "131072": { - "complete": 10, - "expected": 10, - "position_matches": [ - 10, - 10, - 10, - 10 - ], - "generated_tokens": 1241, - "requests_with_coalescing": 9, - "descriptive_median_ttft_s": 49.16869743494317, - "descriptive_median_latency_s": 51.34115820302395 - } - }, - "raw_records": [ - { - "case": "retrieval-n8192-s0", - "input_tokens": 8192, - "wire_sha256": "2b5874a141f018516e5118c6a4d0f3f16d96c5794debcaf60020e82fe79728ac", - "client_sha256": "36ce0b1f37a91910bcbaa996a245cc91d15b882d06410755230fd22a4ed74956", - "complete": true, - "position_matches": [ - true, - true, - true, - true - ], - "output_tokens": 136, - "stream_choices": 136 - }, - { - "case": "retrieval-n8192-s1", - "input_tokens": 8192, - "wire_sha256": "81fae0cb633e0e612aaba709d83abe827c6aca4d46de37cb7752306c1bb52470", - "client_sha256": "dede2bf95de7cfbb1338a625d4e9bf1acc3c6c78584ac071a7a2c430b96709ca", - "complete": true, - "position_matches": [ - true, - true, - true, - true - ], - "output_tokens": 126, - "stream_choices": 126 - }, - { - "case": "retrieval-n8192-s2", - "input_tokens": 8192, - "wire_sha256": "2bfd62f113634a68d372de397dde093cf029ff16756c7881e784a63c2b28b7cc", - "client_sha256": "912f881c740237f6851c3ca3cebd3582dde11a1fe85e04ded2a9f5fefa005b24", - "complete": true, - "position_matches": [ - true, - true, - true, - true - ], - "output_tokens": 130, - "stream_choices": 130 - }, - { - "case": "retrieval-n8192-s3", - "input_tokens": 8192, - "wire_sha256": "a4503a9c1aaa169db4fea7df6f36f9425817eb08237b16de5a46587ca2a3d50d", - "client_sha256": "1bb3ac80b74ab8e6e17451329db65818d79420a06599db1dd70fda85ca6a47c4", - "complete": true, - "position_matches": [ - true, - true, - true, - true - ], - "output_tokens": 126, - "stream_choices": 126 - }, - { - "case": "retrieval-n8192-s4", - "input_tokens": 8192, - "wire_sha256": "8507aacde77b46bc25755ccbbf5ac0cdaf1384cba97a83fcd2d2c27f95d60b2b", - "client_sha256": "b2fe1c4806ea3a0a841d9c2cc5aebcbc4393904c00cf7e46da6413770744ff65", - "complete": true, - "position_matches": [ - true, - true, - true, - true - ], - "output_tokens": 128, - "stream_choices": 128 - }, - { - "case": "retrieval-n8192-s5", - "input_tokens": 8192, - "wire_sha256": "0b3a92b354b404ff719bd560e0fc4be5e0a3dda80630569e1c43db72fc075a13", - "client_sha256": "9414e2a04eda4bc4ff32b1d5cc6aed104f0848b857181a773d14f9a089fccf1a", - "complete": true, - "position_matches": [ - true, - true, - true, - true - ], - "output_tokens": 132, - "stream_choices": 132 - }, - { - "case": "retrieval-n8192-s6", - "input_tokens": 8192, - "wire_sha256": "ec18e4835f92a184778d7d61e2749471036dc10472422dae1024af461e414348", - "client_sha256": "4c4531a09a8b8b11ce5d42b733ce73417fb99451f92a3bf7363e308c8ba9ca31", - "complete": true, - "position_matches": [ - true, - true, - true, - true - ], - "output_tokens": 130, - "stream_choices": 130 - }, - { - "case": "retrieval-n8192-s7", - "input_tokens": 8192, - "wire_sha256": "a4e3e723f1bdefebca3ef493cf2a448745855b7c16e0accf4bab4d3a3e603875", - "client_sha256": "ff80f22ef7b4d3a049f6cd46d6144fed5f067b1036da288870ae5b589398c216", - "complete": true, - "position_matches": [ - true, - true, - true, - true - ], - "output_tokens": 127, - "stream_choices": 127 - }, - { - "case": "retrieval-n8192-s8", - "input_tokens": 8192, - "wire_sha256": "0c5518a15e072a69a54b94046327c4be6dae10eabb8a1bd286df786df09a950c", - "client_sha256": "d9e618ef2c8d5ca5b8bf8b28f6bbedd5f668ad47d33eef830ec6c15f0d84c956", - "complete": true, - "position_matches": [ - true, - true, - true, - true - ], - "output_tokens": 126, - "stream_choices": 126 - }, - { - "case": "retrieval-n8192-s9", - "input_tokens": 8192, - "wire_sha256": "84848552128f510eaa89904677fe292ed27cb6b9a5883a2aaa3bcbad6d4d5df9", - "client_sha256": "8c9863a593e82e0c050124ef3d3c290583895e836178825aa3f0810a23ad4df0", - "complete": true, - "position_matches": [ - true, - true, - true, - true - ], - "output_tokens": 128, - "stream_choices": 128 - }, - { - "case": "retrieval-n65536-s0", - "input_tokens": 65536, - "wire_sha256": "523ce285f75123cedeaab4c6b7531db37a3f6104d973d9bc4e0c9ddd0c0235ae", - "client_sha256": "bfe83ff17b1116ee6b182a3138fa171542600bc9d9650725a9e7b09294987b08", - "complete": true, - "position_matches": [ - true, - true, - true, - true - ], - "output_tokens": 122, - "stream_choices": 122 - }, - { - "case": "retrieval-n65536-s1", - "input_tokens": 65536, - "wire_sha256": "40563d1c4e8879a85800e9858fc5b8f90a8fd6f3f142fb44f10a71f33e3d47e5", - "client_sha256": "6d923b208b3d681911dc5972dc3bfe0da30c3ab3f5b2d9a456e226519a6fea3c", - "complete": true, - "position_matches": [ - true, - true, - true, - true - ], - "output_tokens": 115, - "stream_choices": 115 - }, - { - "case": "retrieval-n65536-s2", - "input_tokens": 65536, - "wire_sha256": "24e056dacc2ffd4db0e44618188cb9c1fe1b7dfa793d6c2640754574e7c402fd", - "client_sha256": "32e6f424db89c22dab0a8d9dbf60144dd35ecc3ca10fe932c301244d09de9176", - "complete": true, - "position_matches": [ - true, - true, - true, - true - ], - "output_tokens": 134, - "stream_choices": 134 - }, - { - "case": "retrieval-n65536-s3", - "input_tokens": 65536, - "wire_sha256": "b77729065e8833e3f2ebe408396bab89b742deed154a7ae698da0cbc79d56f5c", - "client_sha256": "5bbe97397eab9158684d5f72283eb1c7946f59b028c7a8d951df95f20a71dd70", - "complete": true, - "position_matches": [ - true, - true, - true, - true - ], - "output_tokens": 122, - "stream_choices": 122 - }, - { - "case": "retrieval-n65536-s4", - "input_tokens": 65536, - "wire_sha256": "22e6316b1bf89f984112cace1fc16d8dcc1afbd666d1a3afb39656028e30b5cf", - "client_sha256": "61aa3b9ce42369444a4627464ef74105d39093d8fe1f2ba35bb1092af18f3ae6", - "complete": true, - "position_matches": [ - true, - true, - true, - true - ], - "output_tokens": 126, - "stream_choices": 126 - }, - { - "case": "retrieval-n65536-s5", - "input_tokens": 65536, - "wire_sha256": "cd2723e611927176f0a795879fe977af3d459040d21b7ac42bf2236ae7e9b63d", - "client_sha256": "946ca3da8803ea27ecee456b6656280f70e4c0714dfa024542135d1f2efb3e76", - "complete": true, - "position_matches": [ - true, - true, - true, - true - ], - "output_tokens": 126, - "stream_choices": 126 - }, - { - "case": "retrieval-n65536-s6", - "input_tokens": 65536, - "wire_sha256": "de9292e31939eea5178157901cd4c3d168b09b84f4951738a069b55a14c82a36", - "client_sha256": "d4bf7d9440582fb292b5fb08114d28c9e483e6f217da3b99eceb2896ffa8d705", - "complete": true, - "position_matches": [ - true, - true, - true, - true - ], - "output_tokens": 123, - "stream_choices": 123 - }, - { - "case": "retrieval-n65536-s7", - "input_tokens": 65536, - "wire_sha256": "ef5e0d89d140482f4d853bfff6c8d2b8545478cae2ea0c0dce9008c49a3fef7a", - "client_sha256": "01343b158ad2b97c3e1f56e90741ae665405a1c79b7ce0fa06470414a7be8491", - "complete": true, - "position_matches": [ - true, - true, - true, - true - ], - "output_tokens": 124, - "stream_choices": 124 - }, - { - "case": "retrieval-n65536-s8", - "input_tokens": 65536, - "wire_sha256": "738f38a292a7174e07afebbd3e1fd9ba250ee742d4ac7e2d3cd2bdb9ccff43c3", - "client_sha256": "c386b1198d9649192f13ae3c6d70f6e2cfd717c93329fc274f5701b9fe1368f9", - "complete": true, - "position_matches": [ - true, - true, - true, - true - ], - "output_tokens": 122, - "stream_choices": 122 - }, - { - "case": "retrieval-n65536-s9", - "input_tokens": 65536, - "wire_sha256": "0054e9fe347477c996d718ae633ce8985d183797fa203ab6d2031e29a0416e88", - "client_sha256": "9d58112b1bf9f27336f96da62cf389645e7d0aad46bc52e33347886c4990e5b7", - "complete": true, - "position_matches": [ - true, - true, - true, - true - ], - "output_tokens": 122, - "stream_choices": 122 - }, - { - "case": "retrieval-n131072-s0", - "input_tokens": 131072, - "wire_sha256": "9078f34f54154e7cd18eae0ff172df68ad704feaa41110e3234d182232a1aadd", - "client_sha256": "333591cf552b910a2641aa27304581a9ae0db63c1c224448068749cc72c31bf6", - "complete": true, - "position_matches": [ - true, - true, - true, - true - ], - "output_tokens": 124, - "stream_choices": 123 - }, - { - "case": "retrieval-n131072-s1", - "input_tokens": 131072, - "wire_sha256": "0386d7f5cfac2f2599cb7c37d644dedda1af9dfc2f6ee1436ac3e8e5fd2538b5", - "client_sha256": "b3b7271b4be3e24f85c435b24708ad10aaf40ee7ee6572bbfcd944898c0e55af", - "complete": true, - "position_matches": [ - true, - true, - true, - true - ], - "output_tokens": 126, - "stream_choices": 125 - }, - { - "case": "retrieval-n131072-s2", - "input_tokens": 131072, - "wire_sha256": "bca8cd633c56bd1245367f13ff646ce62eb7899d82b34104d955f16988f04155", - "client_sha256": "032a2524cd9fab9cf8dac72e447e0ae5fc75eb6094ef3d746092e16c819d8dcd", - "complete": true, - "position_matches": [ - true, - true, - true, - true - ], - "output_tokens": 122, - "stream_choices": 121 - }, - { - "case": "retrieval-n131072-s3", - "input_tokens": 131072, - "wire_sha256": "3ad4b6268d8b6ddfed2159c13814c2bde874b7da14284ef6ce03ae80e8bb0073", - "client_sha256": "1c161a09c43804025f65a740fc943be6ce0c21eea02e6e2c225bc4f75a79b68a", - "complete": true, - "position_matches": [ - true, - true, - true, - true - ], - "output_tokens": 124, - "stream_choices": 123 - }, - { - "case": "retrieval-n131072-s4", - "input_tokens": 131072, - "wire_sha256": "623562d6ecbe2e16369af4bef8f0669c8eac9772ee0c7f77cdc6bbeb263a5cfa", - "client_sha256": "23f28d5b3fa1045cec879ad4452f57facd2283b0d1a1210e46baf37e944fb39f", - "complete": true, - "position_matches": [ - true, - true, - true, - true - ], - "output_tokens": 120, - "stream_choices": 119 - }, - { - "case": "retrieval-n131072-s5", - "input_tokens": 131072, - "wire_sha256": "d62f3e8b343d4055ecc97e29d2fce7c0723d6d30f29363146f74df3037746e14", - "client_sha256": "966ea68f91f18371caf87d7779b2a1a24ea3350138316bfc04c4e9da646d278b", - "complete": true, - "position_matches": [ - true, - true, - true, - true - ], - "output_tokens": 125, - "stream_choices": 124 - }, - { - "case": "retrieval-n131072-s6", - "input_tokens": 131072, - "wire_sha256": "9e05f3980d11e8d86b8095e968860b1c169de37a3d3541973ac7daf6b2199286", - "client_sha256": "fb57cbd37115305e98c76f88247fb890fcf5454e5cd5d8e8c94764918ff1783d", - "complete": true, - "position_matches": [ - true, - true, - true, - true - ], - "output_tokens": 128, - "stream_choices": 127 - }, - { - "case": "retrieval-n131072-s7", - "input_tokens": 131072, - "wire_sha256": "2dd86fa4465e312453b1e1f6b2a35f2dce8ce04f483d3cec19062569ed07db0d", - "client_sha256": "5fe493fc9ba1025e6edcb0c6b627a8f3471acc611997f84c74a8f96924cbb348", - "complete": true, - "position_matches": [ - true, - true, - true, - true - ], - "output_tokens": 116, - "stream_choices": 115 - }, - { - "case": "retrieval-n131072-s8", - "input_tokens": 131072, - "wire_sha256": "117a029b0275d1734c18d735740c575dd5f87730d3eb1438ec3209fae3d104fd", - "client_sha256": "de0a7237296665ec67e71e5c047b2ab2d499bda19ad6f28dd33a030353c879fb", - "complete": true, - "position_matches": [ - true, - true, - true, - true - ], - "output_tokens": 128, - "stream_choices": 128 - }, - { - "case": "retrieval-n131072-s9", - "input_tokens": 131072, - "wire_sha256": "357d083f84dc48374e93a6430728f6d9283f93cc9f89847e1c5998f9ecb4fb3f", - "client_sha256": "6c2754ac782ef79b019bd860b364e68f45ee5940a9cb4117954ee9a7a5d435b7", - "complete": true, - "position_matches": [ - true, - true, - true, - true - ], - "output_tokens": 128, - "stream_choices": 127 - } - ], - "startup_s": 119.13848478195723, - "cleanup": [ - { - "phase": "cleanup", - "exit_code": 0, - "forced": false, - "owned_group_remaining": 0 - } - ], - "scheduler": { - "state": "COMPLETED", - "exit_code": "0:0", - "allocated_gpus": 1, - "elapsed_seconds": 879, - "allocation_released": true - }, - "budget": { - "new_readiness_used_seconds": 2388, - "new_readiness_cap_seconds": 7200, - "remaining_readiness_seconds": 4812, - "separate_initial_seconds": 147, - "total_new_approved_gpu_hours": 44 - }, - "cleanup_limitations": [ - "One multiprocessing resource_tracker semaphore-cleanup warning remains; do not describe as warning-free teardown." - ], - "termination_supplement": { - "generation_config_sha256": "e70c136c1b78ddc1fb0905bac8e733a4dc448d4f852a5dd75143fffc70be550e", - "declared_eos_token_ids": [ - 248046, - 248044 - ], - "all_30_last_token_ids": 248046, - "all_30_finish_reasons": "stop", - "all_30_stop_reasons": null, - "scope": "post_run_raw_audit_not_a_frozen_termination_validator" - } -} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/retrieval-baseline-1968.jsonl b/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/retrieval-baseline-1968.jsonl deleted file mode 100644 index 371a167c0..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/retrieval-baseline-1968.jsonl +++ /dev/null @@ -1,34 +0,0 @@ -{"phase": "protocol", "observer_sha256": "10f7d81bd971337f237ed9a20ff704ee7bcd84cfbd769a9e94b049bf6d21d671", "fixture_sha256": "935b24411e897b0a1909fe94587ded8b809d7c6777cceac4e2decce8d239f75e", "client_sha256": "f31fae8c573ecc4bf8e31dae03823502e71404333e8111f4b535432758302b29", "cases": ["retrieval-n8192-s0", "retrieval-n8192-s1", "retrieval-n8192-s2", "retrieval-n8192-s3", "retrieval-n8192-s4", "retrieval-n8192-s5", "retrieval-n8192-s6", "retrieval-n8192-s7", "retrieval-n8192-s8", "retrieval-n8192-s9", "retrieval-n65536-s0", "retrieval-n65536-s1", "retrieval-n65536-s2", "retrieval-n65536-s3", "retrieval-n65536-s4", "retrieval-n65536-s5", "retrieval-n65536-s6", "retrieval-n65536-s7", "retrieval-n65536-s8", "retrieval-n65536-s9", "retrieval-n131072-s0", "retrieval-n131072-s1", "retrieval-n131072-s2", "retrieval-n131072-s3", "retrieval-n131072-s4", "retrieval-n131072-s5", "retrieval-n131072-s6", "retrieval-n131072-s7", "retrieval-n131072-s8", "retrieval-n131072-s9"], "verdict": "INCONCLUSIVE", "scope": "retrieval_baseline_qualification_no_candidate_or_activated_gates", "versions": {"vllm": "0.32.0", "torch": "2.13.0", "transformers": "5.19.0", "flashinfer-python": "0.7.0.post1", "triton": "3.7.1", "aiohttp": "3.14.5"}, "server_settings": {"max_model_len": 133120, "prefix_caching": false, "language_model_only": true, "shutdown_timeout": 30, "other_optimization_flags": "defaults", "cpu_thread_policy": "vLLM_default_no_OMP_override", "omp_num_threads_env": null}} -{"phase": "ready", "startup_s": 119.13848478195723} -{"phase": "request", "index": 0, "case": "retrieval-n8192-s0", "score": {"shape_valid": true, "position_matches": [true, true, true, true], "complete": true}, "generated_text": "```json\n{\n \"KEY_48D0482FB109\": \"VAL_00C19710A484\",\n \"KEY_9625E218DE62\": \"VAL_D10F531B5345\",\n \"KEY_B4D07F699C59\": \"VAL_3D11490EF091\",\n \"KEY_84088D6AA1FE\": \"VAL_148ABC144720\"\n}\n```", "input_tokens": 8192, "requested_output_tokens": 2048, "output_tokens": 136, "token_ids": [71093, 2164, 198, 90, 198, 220, 328, 4631, 62, 19, 23, 35, 15, 19, 23, 17, 15731, 16, 15, 24, 763, 328, 5419, 62, 15, 15, 34, 16, 24, 22, 16, 15, 32, 19, 23, 19, 487, 198, 220, 328, 4631, 62, 24, 21, 17, 20, 36, 17, 16, 23, 1113, 21, 17, 763, 328, 5419, 1502, 16, 15, 37, 20, 18, 16, 33, 20, 18, 19, 20, 487, 198, 220, 328, 4631, 1609, 19, 35, 15, 22, 37, 21, 24, 24, 34, 20, 24, 763, 328, 5419, 62, 18, 35, 16, 16, 19, 24, 15, 14453, 15, 24, 16, 487, 198, 220, 328, 4631, 62, 23, 19, 15, 23, 23, 35, 21, 5840, 16, 11091, 763, 328, 5419, 62, 16, 19, 23, 24606, 16, 19, 19, 22, 17, 15, 1, 198, 92, 198, 71093, 248046], "terminal": ["stop"], "usage": [{"prompt_tokens": 8192, "total_tokens": 8328, "completion_tokens": 136}], "valid": true, "token_counts_per_choice": [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], "done_unique_last": true, "client_success": true, "client_error": false, "wire_sha256": "2b5874a141f018516e5118c6a4d0f3f16d96c5794debcaf60020e82fe79728ac", "prompt_sha256": "a16cff586ba6bbed823d511c7ccbfa89b97f956ff84ed4695240ddd679f85f24", "ttft_s": 1.71112799202092, "latency_s": 3.143356279004365, "itl": [0.007023670943453908, 0.010413069045171142, 0.010642222943715751, 0.010518798022530973, 0.01077679602894932, 0.010564330965280533, 0.010640647960826755, 0.01059802109375596, 0.010649267002008855, 0.01061559293884784, 0.010647149058058858, 0.010563308955170214, 0.010766003979369998, 0.010457850061357021, 0.01068607799243182, 0.010581803973764181, 0.010717684985138476, 0.010645695962011814, 0.01059872005134821, 0.01051848498173058, 0.010645971051417291, 0.010675648925825953, 0.010575658991001546, 0.010644199093803763, 0.010642918990924954, 0.010640449007041752, 0.010637826984748244, 0.010626820963807404, 0.010633119964040816, 0.010622662026435137, 0.010637709987349808, 0.010653791017830372, 0.010639203013852239, 0.010619245003908873, 0.010691632982343435, 0.01062522002030164, 0.010920954984612763, 0.01049807295203209, 0.010581871028989553, 0.010618063039146364, 0.010660558007657528, 0.01056870399042964, 0.010622203932143748, 0.010666829068213701, 0.010622403933666646, 0.010603115078993142, 0.010642203968018293, 0.010615442995913327, 0.010639764950610697, 0.01067529700230807, 0.010640765074640512, 0.01064537395723164, 0.010624887072481215, 0.010675734956748784, 0.010632875026203692, 0.01060145499650389, 0.010647473973222077, 0.01063276594504714, 0.010647645103745162, 0.010632499004714191, 0.010665895999409258, 0.010610318975523114, 0.010657154023647308, 0.010651220916770399, 0.010611199075356126, 0.010650547919794917, 0.010617763036862016, 0.010636158054694533, 0.010687064961530268, 0.010603163973428309, 0.010644349968060851, 0.010637764004059136, 0.010649053030647337, 0.010628661955706775, 0.010667573078535497, 0.010628122952766716, 0.010617605992592871, 0.010644969996064901, 0.010638041072525084, 0.010653591947630048, 0.010656635044142604, 0.01063013100065291, 0.010655552963726223, 0.01060614304151386, 0.010650211945176125, 0.01065319706685841, 0.010635999962687492, 0.0106599279679358, 0.010618999018333852, 0.010654442943632603, 0.010615414008498192, 0.010821739095263183, 0.010593791957944632, 0.010621392983011901, 0.010607877979055047, 0.010803519980981946, 0.010672039003111422, 0.010464462102390826, 0.010663754888810217, 0.010661337058991194, 0.010728196939453483, 0.0104466910706833, 0.010792727931402624, 0.010539924027398229, 0.010620919987559319, 0.010595794999971986, 0.010670199990272522, 0.01093531702645123, 0.010442128055728972, 0.010686923982575536, 0.01055889599956572, 0.010735907941125333, 0.010523422039113939, 0.010558812995441258, 0.01060255104675889, 0.010602536960504949, 0.010640228050760925, 0.010640261927619576, 0.010617841966450214, 0.010772481095045805, 0.01060641196090728, 0.010642157983966172, 0.010554334032349288, 0.010623989976011217, 0.01060601498465985, 0.010666514979675412, 0.010610279045067728, 0.010629888041876256, 0.011159928981214762, 0.01036879897583276, 0.010559284943155944, 0.010610848083160818, 0.010603555012494326, 0.0106121099088341, 0.010761380079202354], "tpot_s": 0.010609098422099594} -{"phase": "request", "index": 1, "case": "retrieval-n8192-s1", "score": {"shape_valid": true, "position_matches": [true, true, true, true], "complete": true}, "generated_text": "{\n \"KEY_658357C7B44B\": \"VAL_0A075A1112A6\",\n \"KEY_BF4A1A24CE68\": \"VAL_B3E0DCAC2698\",\n \"KEY_B2D170FC8F40\": \"VAL_3B54EC2AD5EC\",\n \"KEY_ACCA18675051\": \"VAL_F62C097883A9\"\n}", "input_tokens": 8192, "requested_output_tokens": 2048, "output_tokens": 126, "token_ids": [90, 198, 220, 328, 4631, 62, 21, 20, 23, 18, 20, 22, 34, 22, 33, 19, 19, 33, 763, 328, 5419, 62, 15, 32, 15, 22, 20, 32, 16, 16, 16, 17, 32, 21, 487, 198, 220, 328, 4631, 1609, 37, 19, 32, 16, 32, 17, 19, 2036, 21, 23, 763, 328, 5419, 1609, 18, 36, 15, 5449, 1646, 17, 21, 24, 23, 487, 198, 220, 328, 4631, 1609, 17, 35, 16, 22, 15, 6542, 23, 37, 19, 15, 763, 328, 5419, 62, 18, 33, 20, 19, 7263, 17, 1745, 20, 7263, 487, 198, 220, 328, 4631, 24848, 32, 16, 23, 21, 22, 20, 15, 20, 16, 763, 328, 5419, 1352, 21, 17, 34, 15, 24, 22, 23, 23, 18, 32, 24, 1, 198, 92, 248046], "terminal": ["stop"], "usage": [{"prompt_tokens": 8192, "total_tokens": 8318, "completion_tokens": 126}], "valid": true, "token_counts_per_choice": [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], "done_unique_last": true, "client_success": true, "client_error": false, "wire_sha256": "81fae0cb633e0e612aaba709d83abe827c6aca4d46de37cb7752306c1bb52470", "prompt_sha256": "d90e85ab8e5af236b4d4d223dfb3444ea228b4963f5cfc8f8d3e8e1c8485443b", "ttft_s": 1.6779797839699313, "latency_s": 3.0040628779679537, "itl": [0.0075319219613447785, 0.010536086047068238, 0.010632924037054181, 0.010769831947982311, 0.010711107053793967, 0.01042318600229919, 0.010528023936785758, 0.010622938047163188, 0.010589084937237203, 0.010626303032040596, 0.01064389105886221, 0.010615132981911302, 0.01075606094673276, 0.010506210965104401, 0.010674553108401597, 0.01058590889442712, 0.010607019066810608, 0.010674897930584848, 0.010607543052174151, 0.0106570910429582, 0.010641225962899625, 0.01056201197206974, 0.010828019003383815, 0.010521618998609483, 0.010627240058965981, 0.010633873986080289, 0.010637015919201076, 0.010636079008691013, 0.010617234045639634, 0.010643359972164035, 0.010587873053736985, 0.010730720940046012, 0.010597842978313565, 0.01059360010549426, 0.010604746988974512, 0.01064085797406733, 0.010690244031138718, 0.01059079694095999, 0.010635585058480501, 0.01063912594690919, 0.010625766008161008, 0.010658066952601075, 0.010612942045554519, 0.010618222993798554, 0.010670696035958827, 0.010639242944307625, 0.010624379036016762, 0.010637040017172694, 0.010595415951684117, 0.010632753022946417, 0.010708027053624392, 0.010592060978524387, 0.010615382925607264, 0.01063146407250315, 0.010648287017829716, 0.010621530935168266, 0.010631527053192258, 0.010630321921780705, 0.010639768093824387, 0.010650998912751675, 0.010631407028995454, 0.010619431035593152, 0.010634229984134436, 0.010649760020896792, 0.010653681005351245, 0.010610771947540343, 0.010631309007294476, 0.010654742014594376, 0.010609330958686769, 0.010652840021066368, 0.010594110004603863, 0.010624313028529286, 0.010651630000211298, 0.010625701979734004, 0.010596806998364627, 0.010636160033755004, 0.01065091195050627, 0.010652960045263171, 0.010637599974870682, 0.010622813017107546, 0.010674507939256728, 0.01061455998569727, 0.010639027110300958, 0.010706727975048125, 0.010563781019300222, 0.010636135935783386, 0.010639818967320025, 0.010620479006320238, 0.010654545039869845, 0.010659193038009107, 0.010955257923342288, 0.01040692103561014, 0.010517352959141135, 0.01079246704466641, 0.010592891951091588, 0.010559985996223986, 0.010778823052532971, 0.010488644009456038, 0.01066884002648294, 0.01058756501879543, 0.010738765937276185, 0.010584907024167478, 0.010631226003170013, 0.010555697954259813, 0.010615422972477973, 0.010947933071292937, 0.010579872992821038, 0.010636872961185873, 0.010597633081488311, 0.010626834002323449, 0.010656213969923556, 0.01060714793857187, 0.010642111068591475, 0.01064586394932121, 0.010632551042363048, 0.010601136018522084, 0.01065626391209662, 0.010610138066112995, 0.010654164012521505, 0.010633677011355758, 0.010629419004544616, 0.010747352964244783, 0.010513651999644935, 0.010640535037964582, 0.01065698394086212], "tpot_s": 0.010608664751984178} -{"phase": "request", "index": 2, "case": "retrieval-n8192-s2", "score": {"shape_valid": true, "position_matches": [true, true, true, true], "complete": true}, "generated_text": "```json\n{\n \"KEY_8CACAB1A5C8D\": \"VAL_6036D54D7373\",\n \"KEY_7EB1C16A83A4\": \"VAL_BE12EF59BE37\",\n \"KEY_5EAE4B0C7C6E\": \"VAL_23C6CDC883BA\",\n \"KEY_09A2CE135CD8\": \"VAL_80F4CF7086F9\"\n}\n```", "input_tokens": 8192, "requested_output_tokens": 2048, "output_tokens": 130, "token_ids": [71093, 2164, 198, 90, 198, 220, 328, 4631, 62, 23, 89635, 1803, 16, 32, 20, 34, 23, 35, 763, 328, 5419, 62, 21, 15, 18, 21, 35, 20, 19, 35, 22, 18, 22, 18, 487, 198, 220, 328, 4631, 62, 22, 8006, 16, 34, 16, 21, 32, 23, 18, 32, 19, 763, 328, 5419, 26294, 16, 17, 14453, 20, 24, 11261, 18, 22, 487, 198, 220, 328, 4631, 62, 20, 36, 13276, 19, 33, 15, 34, 22, 34, 21, 36, 763, 328, 5419, 62, 17, 18, 34, 21, 70960, 23, 23, 18, 6844, 487, 198, 220, 328, 4631, 62, 15, 24, 32, 17, 2036, 16, 18, 20, 6281, 23, 763, 328, 5419, 62, 23, 15, 37, 19, 9362, 22, 15, 23, 21, 37, 24, 1, 198, 92, 198, 71093, 248046], "terminal": ["stop"], "usage": [{"prompt_tokens": 8192, "total_tokens": 8322, "completion_tokens": 130}], "valid": true, "token_counts_per_choice": [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], "done_unique_last": true, "client_success": true, "client_error": false, "wire_sha256": "2bfd62f113634a68d372de397dde093cf029ff16756c7881e784a63c2b28b7cc", "prompt_sha256": "8b73ef634d270b00294acf9c0efdf4384ba70a95863273ee63b719d435be8ae6", "ttft_s": 1.6784475039457902, "latency_s": 3.046813153894618, "itl": [0.0069735279539600015, 0.010461200028657913, 0.01058631599880755, 0.010576589033007622, 0.01060969103127718, 0.010540855000726879, 0.010633594007231295, 0.010636469000019133, 0.010659699910320342, 0.010681857005693018, 0.010538030066527426, 0.010640016989782453, 0.0106456020148471, 0.010616921004839242, 0.010640218970365822, 0.010609700926579535, 0.010652860044501722, 0.010653281002305448, 0.01065980305429548, 0.010615651961416006, 0.01074279798194766, 0.010595318977721035, 0.010598183027468622, 0.010657728998921812, 0.01063947204966098, 0.010618047905154526, 0.010661308071576059, 0.010601631947793067, 0.010624425020068884, 0.010633278987370431, 0.01065515202935785, 0.010614454979076982, 0.010655213962309062, 0.010608622105792165, 0.010637586936354637, 0.01062997302506119, 0.010642254026606679, 0.010641418979503214, 0.010617945925332606, 0.010673317010514438, 0.010631471988745034, 0.01065207808278501, 0.010623864945955575, 0.010615534032694995, 0.010658583021722734, 0.010666533024050295, 0.010634439997375011, 0.010627324925735593, 0.010658077080734074, 0.010638525942340493, 0.010635519050993025, 0.010657520964741707, 0.010627347975969315, 0.010631793993525207, 0.01062211999669671, 0.010634929058142006, 0.010657319915480912, 0.01067127799615264, 0.010629543103277683, 0.01063746795989573, 0.010642076027579606, 0.010652565979398787, 0.01063747494481504, 0.010619978071190417, 0.010677176993340254, 0.01064716198015958, 0.010618951986543834, 0.010644812020473182, 0.010623330948874354, 0.010719986981712282, 0.010600905050523579, 0.010616280022077262, 0.010674591991119087, 0.010686084046028554, 0.010585881886072457, 0.01069245405960828, 0.01063520205207169, 0.010628350893966854, 0.010613105027005076, 0.010635319980792701, 0.010682380059733987, 0.010706603992730379, 0.010703773936256766, 0.010693235090002418, 0.01060878299176693, 0.010810983018018305, 0.010437464923597872, 0.010607955045998096, 0.010785107035189867, 0.010480919969268143, 0.01059557602275163, 0.010659531923010945, 0.010806290083564818, 0.010468420921824872, 0.010568013996817172, 0.010726772015914321, 0.01064551598392427, 0.01068954006768763, 0.010498184012249112, 0.010608333977870643, 0.010617917985655367, 0.010698813945055008, 0.010578721994534135, 0.010626124101690948, 0.011204450973309577, 0.010156237985938787, 0.010589660028927028, 0.010625211987644434, 0.010657593957148492, 0.010613215970806777, 0.010641450993716717, 0.010676086996681988, 0.010630080010741949, 0.010626110015437007, 0.010657286969944835, 0.010635872022248805, 0.010640098014846444, 0.01060969103127718, 0.010657418984919786, 0.010655042016878724, 0.01069125800859183, 0.010552164982073009, 0.010608272976242006, 0.010670949006453156, 0.010590562014840543, 0.010639008018188179, 0.010650516953319311, 0.01066273998003453, 0.010676726000383496], "tpot_s": 0.010607485658518045} -{"phase": "request", "index": 3, "case": "retrieval-n8192-s3", "score": {"shape_valid": true, "position_matches": [true, true, true, true], "complete": true}, "generated_text": "{\n \"KEY_7C17A7C224BD\": \"VAL_975AF3765BE1\",\n \"KEY_9A0B8D3D5847\": \"VAL_E1384C5C0A7C\",\n \"KEY_2919BD59FEDC\": \"VAL_1A6A4BBC3A34\",\n \"KEY_C4CCAE40EC9B\": \"VAL_F4E762A0BCE7\"\n}", "input_tokens": 8192, "requested_output_tokens": 2048, "output_tokens": 126, "token_ids": [90, 198, 220, 328, 4631, 62, 22, 34, 16, 22, 32, 22, 34, 17, 17, 19, 9265, 763, 328, 5419, 62, 24, 22, 20, 8018, 18, 22, 21, 20, 11261, 16, 487, 198, 220, 328, 4631, 62, 24, 32, 15, 33, 23, 35, 18, 35, 20, 23, 19, 22, 763, 328, 5419, 2021, 16, 18, 23, 19, 34, 20, 34, 15, 32, 22, 34, 487, 198, 220, 328, 4631, 62, 17, 24, 16, 24, 9265, 20, 24, 37, 1425, 34, 763, 328, 5419, 62, 16, 32, 21, 32, 19, 64500, 18, 32, 18, 19, 487, 198, 220, 328, 4631, 892, 19, 3580, 13276, 19, 15, 7263, 24, 33, 763, 328, 5419, 1352, 19, 36, 22, 21, 17, 32, 15, 33, 2036, 22, 1, 198, 92, 248046], "terminal": ["stop"], "usage": [{"prompt_tokens": 8192, "total_tokens": 8318, "completion_tokens": 126}], "valid": true, "token_counts_per_choice": [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], "done_unique_last": true, "client_success": true, "client_error": false, "wire_sha256": "a4503a9c1aaa169db4fea7df6f36f9425817eb08237b16de5a46587ca2a3d50d", "prompt_sha256": "c2396e748d38efc579bc745cc2e6ce5fde2d5457d516f58c6c8a854b43cf3d8d", "ttft_s": 1.6769163639983162, "latency_s": 3.0035807009553537, "itl": [0.0075732029508799314, 0.010488737025298178, 0.010624947026371956, 0.010608331998810172, 0.01067652297206223, 0.010740794008597732, 0.010515540954656899, 0.010638582054525614, 0.010609654942527413, 0.010615609004162252, 0.01067084597889334, 0.010626328061334789, 0.01063995691947639, 0.01064106507692486, 0.010734742972999811, 0.010703152045607567, 0.010553714004345238, 0.010607681004330516, 0.01063509692903608, 0.010669645038433373, 0.010621165973134339, 0.010680153034627438, 0.010629363940097392, 0.011310023022815585, 0.010037153959274292, 0.010642056004144251, 0.010594588005915284, 0.010640119086019695, 0.010623491951264441, 0.010635321028530598, 0.01062752294819802, 0.010649812989868224, 0.010640588006936014, 0.010657227016054094, 0.010600459994748235, 0.010649175033904612, 0.010649089002981782, 0.010646681999787688, 0.010634609032422304, 0.010613402002491057, 0.01062044594436884, 0.010642269044183195, 0.010654844925738871, 0.0106687240768224, 0.010634893900714815, 0.010618828004226089, 0.010642118053510785, 0.01068331697024405, 0.010608357028104365, 0.010645736008882523, 0.010628098039887846, 0.010785933001898229, 0.01055539189837873, 0.010630824021063745, 0.010578493005596101, 0.010667487047612667, 0.010638464009389281, 0.010708392015658319, 0.010542348958551884, 0.010633575031533837, 0.01077059400267899, 0.010675848927348852, 0.010486604063771665, 0.010616917978040874, 0.01081722800154239, 0.010463543003425002, 0.010613634949550033, 0.01060478703584522, 0.010629271040670574, 0.010663074906915426, 0.010582713061012328, 0.010661912965588272, 0.010596016072668135, 0.010685065994039178, 0.010751030989922583, 0.010552424006164074, 0.01081643090583384, 0.010511577012948692, 0.01058131200261414, 0.010681041050702333, 0.010629822965711355, 0.010573397972621024, 0.010655852034687996, 0.010605792049318552, 0.010641942964866757, 0.010845093056559563, 0.010502263903617859, 0.010643774061463773, 0.010631287936121225, 0.010719676036387682, 0.010740746045485139, 0.010481719975359738, 0.010567088960669935, 0.010642367997206748, 0.010652319062501192, 0.01062129600904882, 0.010623738984577358, 0.010631926939822733, 0.010680372011847794, 0.010615371051244438, 0.010659849969670177, 0.010644886991940439, 0.010619409033097327, 0.010623174952343106, 0.010649100062437356, 0.010634201928041875, 0.010634166072122753, 0.010674668941646814, 0.010594823979772627, 0.010681112995371222, 0.010648795054294169, 0.010606528026983142, 0.010671425960026681, 0.010633129044435918, 0.01063806691672653, 0.010608576005324721, 0.010653820005245507, 0.010620251065120101, 0.010648249997757375, 0.010636516963131726, 0.01063375105150044, 0.01071096200030297, 0.01059214398264885, 0.010645402944646776, 0.01068974903319031], "tpot_s": 0.0106133146956563} -{"phase": "request", "index": 4, "case": "retrieval-n8192-s4", "score": {"shape_valid": true, "position_matches": [true, true, true, true], "complete": true}, "generated_text": "{\n \"KEY_34E16D590568\": \"VAL_4DF45120C441\",\n \"KEY_D844BF781F00\": \"VAL_0E815DD3A3FE\",\n \"KEY_34F0672C565F\": \"VAL_A5480F57AC37\",\n \"KEY_7CA1CBFFCFE2\": \"VAL_83AF30750C36\"\n}", "input_tokens": 8192, "requested_output_tokens": 2048, "output_tokens": 128, "token_ids": [90, 198, 220, 328, 4631, 62, 18, 19, 36, 16, 21, 35, 20, 24, 15, 20, 21, 23, 763, 328, 5419, 62, 19, 5095, 19, 20, 16, 17, 15, 34, 19, 19, 16, 487, 198, 220, 328, 4631, 1502, 23, 19, 19, 19278, 22, 23, 16, 37, 15, 15, 763, 328, 5419, 62, 15, 36, 23, 16, 20, 3967, 18, 32, 18, 11091, 487, 198, 220, 328, 4631, 62, 18, 19, 37, 15, 21, 22, 17, 34, 20, 21, 20, 37, 763, 328, 5419, 1511, 20, 19, 23, 15, 37, 20, 22, 1646, 18, 22, 487, 198, 220, 328, 4631, 62, 22, 4887, 16, 12607, 1686, 9362, 36, 17, 763, 328, 5419, 62, 23, 18, 8018, 18, 15, 22, 20, 15, 34, 18, 21, 1, 198, 92, 248046], "terminal": ["stop"], "usage": [{"prompt_tokens": 8192, "total_tokens": 8320, "completion_tokens": 128}], "valid": true, "token_counts_per_choice": [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], "done_unique_last": true, "client_success": true, "client_error": false, "wire_sha256": "8507aacde77b46bc25755ccbbf5ac0cdaf1384cba97a83fcd2d2c27f95d60b2b", "prompt_sha256": "1b10359c718100228e60b87fe7af0a1e450d30b56f25f05fca28f61a79f1eb8a", "ttft_s": 1.676698304945603, "latency_s": 3.024260980891995, "itl": [0.0074664599960669875, 0.010480843018740416, 0.010601583984680474, 0.010641315020620823, 0.01057011994998902, 0.010626201052218676, 0.010622303001582623, 0.010686491965316236, 0.010602039052173495, 0.010589086916297674, 0.010635680984705687, 0.010649432078935206, 0.010689995950087905, 0.010617346037179232, 0.01084380701649934, 0.010436702985316515, 0.010609744931571186, 0.010623262031003833, 0.010622669011354446, 0.010627684998326004, 0.010628500021994114, 0.010612800950184464, 0.010650524985976517, 0.010630859062075615, 0.01063983200583607, 0.010663707042112947, 0.010626801988109946, 0.010640513966791332, 0.01062045304570347, 0.010638135951012373, 0.010671476949937642, 0.010632531018927693, 0.010622916044667363, 0.010632488992996514, 0.010640320018865168, 0.010633496916852891, 0.010650052106939256, 0.010614414000883698, 0.010708468966186047, 0.010560164926573634, 0.010618535103276372, 0.01068355597089976, 0.010617443942464888, 0.010613991995342076, 0.01063902908936143, 0.010630876990035176, 0.010649684933014214, 0.010630032978951931, 0.010638012085109949, 0.010652119992300868, 0.010625044000335038, 0.010635963990353048, 0.010654652025550604, 0.01061840693000704, 0.010625137016177177, 0.010635012993589044, 0.010628851014189422, 0.010653942008502781, 0.010635379934683442, 0.01062718010507524, 0.010626983945257962, 0.010633197030983865, 0.010651240008883178, 0.010655156918801367, 0.010650900076143444, 0.010662326007150114, 0.01062465994618833, 0.010677297017537057, 0.010589031968265772, 0.010712982038967311, 0.01061222399584949, 0.010603268980048597, 0.010711102047935128, 0.010724471998400986, 0.010818724986165762, 0.010365283000282943, 0.010636560968123376, 0.010764703038148582, 0.010561864008195698, 0.010595428990200162, 0.010683289030566812, 0.010574987973086536, 0.010665960027836263, 0.010621654917486012, 0.010693658026866615, 0.010738856974057853, 0.010532155982218683, 0.010842494084499776, 0.01051901001483202, 0.010628266958519816, 0.010701647028326988, 0.010560518014244735, 0.0106167959747836, 0.01059961598366499, 0.01059849001467228, 0.010622671921737492, 0.010663510067388415, 0.010611605015583336, 0.010715816984884441, 0.010590502992272377, 0.010622577043250203, 0.010618647909723222, 0.010637895087711513, 0.01064804894849658, 0.010711297043599188, 0.010598713997751474, 0.010623711976222694, 0.01068667194340378, 0.010722520062699914, 0.01056104397866875, 0.010667986003682017, 0.010592199047096074, 0.010636094957590103, 0.010591835947707295, 0.010690012015402317, 0.010602053022012115, 0.01068641699384898, 0.010581076960079372, 0.010619604028761387, 0.010628557996824384, 0.010612925980240107, 0.010623213020153344, 0.010616980027407408, 0.010665907990187407, 0.01068981597200036, 0.010591141995973885, 0.010702728992328048], "tpot_s": 0.010610729731861356} -{"phase": "request", "index": 5, "case": "retrieval-n8192-s5", "score": {"shape_valid": true, "position_matches": [true, true, true, true], "complete": true}, "generated_text": "{\n \"KEY_C957D9C07257\": \"VAL_2A575856A1BE\",\n \"KEY_143ECD9B470B\": \"VAL_F5C7F636E406\",\n \"KEY_9E1A8D57F08A\": \"VAL_3B35C77D0E43\",\n \"KEY_837DD3E1EA3D\": \"VAL_24F0FCE9E515\"\n}", "input_tokens": 8192, "requested_output_tokens": 2048, "output_tokens": 132, "token_ids": [90, 198, 220, 328, 4631, 892, 24, 20, 22, 35, 24, 34, 15, 22, 17, 20, 22, 763, 328, 5419, 62, 17, 32, 20, 22, 20, 23, 20, 21, 32, 16, 11261, 487, 198, 220, 328, 4631, 62, 16, 19, 18, 235181, 24, 33, 19, 22, 15, 33, 763, 328, 5419, 1352, 20, 34, 22, 37, 21, 18, 21, 36, 19, 15, 21, 487, 198, 220, 328, 4631, 62, 24, 36, 16, 32, 23, 35, 20, 22, 37, 15, 23, 32, 763, 328, 5419, 62, 18, 33, 18, 20, 34, 22, 22, 35, 15, 36, 19, 18, 487, 198, 220, 328, 4631, 62, 23, 18, 22, 3967, 18, 36, 16, 18535, 18, 35, 763, 328, 5419, 62, 17, 19, 37, 15, 37, 2036, 24, 36, 20, 16, 20, 1, 198, 92, 248046], "terminal": ["stop"], "usage": [{"prompt_tokens": 8192, "total_tokens": 8324, "completion_tokens": 132}], "valid": true, "token_counts_per_choice": [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], "done_unique_last": true, "client_success": true, "client_error": false, "wire_sha256": "0b3a92b354b404ff719bd560e0fc4be5e0a3dda80630569e1c43db72fc075a13", "prompt_sha256": "cfb959e0d5cf1d9665b6aaca31dc926cd317f6ce82d8525f34220c86e3d1c896", "ttft_s": 1.6767129219369963, "latency_s": 3.0685174269601703, "itl": [0.007610151078552008, 0.01058023190125823, 0.01051424304023385, 0.010625001974403858, 0.010648197028785944, 0.010584327974356711, 0.010626001982018352, 0.010636060032993555, 0.010654350044205785, 0.010673983953893185, 0.010632339981384575, 0.010651918011717498, 0.01063025405164808, 0.010700889979489148, 0.010571692953817546, 0.01062726799864322, 0.010622038040310144, 0.010625527007505298, 0.010693892021663487, 0.010627390001900494, 0.01069583895150572, 0.010608570999465883, 0.010614295955747366, 0.010632442077621818, 0.010662807035259902, 0.010640977998264134, 0.010668891947716475, 0.010604294016957283, 0.010630142991431057, 0.010687939007766545, 0.010609035030938685, 0.010655066929757595, 0.010659766965545714, 0.010614248109050095, 0.010629810974933207, 0.010650345007888973, 0.010657735983841121, 0.010652965982444584, 0.01065704703796655, 0.010597797925584018, 0.010667321039363742, 0.010644063004292548, 0.010634311940521002, 0.010632933001033962, 0.010616345098242164, 0.010703093954361975, 0.010623657959513366, 0.010608879034407437, 0.010658701998181641, 0.01063918904401362, 0.010659151012077928, 0.010626495000906289, 0.01065955392550677, 0.01064817898441106, 0.01063631009310484, 0.010618949891068041, 0.01063582906499505, 0.010674061952158809, 0.010616432060487568, 0.010609083925373852, 0.010651073069311678, 0.010659110965207219, 0.010675300960429013, 0.010633481084369123, 0.010630688979290426, 0.010649907984770834, 0.010616568033583462, 0.010716927936300635, 0.0106014619814232, 0.010643330053426325, 0.010623845038935542, 0.0108214559732005, 0.010522895958274603, 0.010602351045235991, 0.010659966967068613, 0.01061280001886189, 0.010644527967087924, 0.010700692073442042, 0.010608326992951334, 0.010641306987963617, 0.010646329028531909, 0.010661548003554344, 0.010631099925376475, 0.01062604901380837, 0.010678093996830285, 0.010773987043648958, 0.01050452294293791, 0.010632315999828279, 0.010631808079779148, 0.01062077993992716, 0.010642800945788622, 0.0106585311004892, 0.0106267569353804, 0.010677676065824926, 0.010670224903151393, 0.010666805086657405, 0.0106123419245705, 0.010612009093165398, 0.010624098940752447, 0.010664595058187842, 0.010629002004861832, 0.010653684963472188, 0.010626837029121816, 0.010670226998627186, 0.010678338003344834, 0.010578387998975813, 0.010629973956383765, 0.01065166702028364, 0.010638752020895481, 0.010664372006431222, 0.010617551975883543, 0.01062025292776525, 0.010662969085387886, 0.010623183916322887, 0.010645419009961188, 0.010709831025451422, 0.010673151002265513, 0.010732391965575516, 0.010735185001976788, 0.010723176994360983, 0.010740605997852981, 0.010725355008617043, 0.010669377981685102, 0.010694852098822594, 0.010657474980689585, 0.010705373017117381, 0.010679636965505779, 0.010698521044105291, 0.010676187928766012, 0.010697741992771626, 0.010738307028077543], "tpot_s": 0.010624461870405909} -{"phase": "request", "index": 6, "case": "retrieval-n8192-s6", "score": {"shape_valid": true, "position_matches": [true, true, true, true], "complete": true}, "generated_text": "{\n \"KEY_51C062623F76\": \"VAL_00C8B1F0EE3B\",\n \"KEY_5860EDCDBA1F\": \"VAL_D0AC821B1C73\",\n \"KEY_D1F424F3850C\": \"VAL_EBE14AF8B77A\",\n \"KEY_A757B77E7E51\": \"VAL_94C0C64D7904\"\n}", "input_tokens": 8192, "requested_output_tokens": 2048, "output_tokens": 130, "token_ids": [90, 198, 220, 328, 4631, 62, 20, 16, 34, 15, 21, 17, 21, 17, 18, 37, 22, 21, 763, 328, 5419, 62, 15, 15, 34, 23, 33, 16, 37, 15, 6878, 18, 33, 487, 198, 220, 328, 4631, 62, 20, 23, 21, 15, 1425, 34, 3392, 32, 16, 37, 763, 328, 5419, 1502, 15, 1646, 23, 17, 16, 33, 16, 34, 22, 18, 487, 198, 220, 328, 4631, 1502, 16, 37, 19, 17, 19, 37, 18, 23, 20, 15, 34, 763, 328, 5419, 2021, 11261, 16, 19, 8018, 23, 33, 22, 22, 32, 487, 198, 220, 328, 4631, 1511, 22, 20, 22, 33, 22, 22, 36, 22, 36, 20, 16, 763, 328, 5419, 62, 24, 19, 34, 15, 34, 21, 19, 35, 22, 24, 15, 19, 1, 198, 92, 248046], "terminal": ["stop"], "usage": [{"prompt_tokens": 8192, "total_tokens": 8322, "completion_tokens": 130}], "valid": true, "token_counts_per_choice": [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], "done_unique_last": true, "client_success": true, "client_error": false, "wire_sha256": "ec18e4835f92a184778d7d61e2749471036dc10472422dae1024af461e414348", "prompt_sha256": "f95b5ea57e093c239d9e366ebc4cda3429f85425aac558a8055a0f2760488d4f", "ttft_s": 1.6798088889336213, "latency_s": 3.048854212043807, "itl": [0.007574116112664342, 0.010601889924146235, 0.010556592023931444, 0.010493356036022305, 0.010614336933940649, 0.010621786001138389, 0.01062132406514138, 0.010623061913065612, 0.01061955199111253, 0.010624248068779707, 0.01061950996518135, 0.010628910036757588, 0.010662331013008952, 0.010643980000168085, 0.010631809942424297, 0.010644784080795944, 0.010615125996991992, 0.010648889932781458, 0.010656779981218278, 0.010624193004332483, 0.01063316105864942, 0.010641322005540133, 0.010622201953083277, 0.010647414019331336, 0.010662475018762052, 0.010634389007464051, 0.010635903920046985, 0.010623657028190792, 0.010650382027961314, 0.010638667969033122, 0.01062775100581348, 0.010635736980475485, 0.010671054013073444, 0.010627123061567545, 0.010631216922774911, 0.010648169089108706, 0.010625457973219454, 0.010665409034118056, 0.010620455956086516, 0.010649468982592225, 0.010673720971681178, 0.01062513398937881, 0.010631326003931463, 0.01068226306233555, 0.010582179995253682, 0.010645475937053561, 0.010663678054697812, 0.010617429041303694, 0.010620237910188735, 0.010695717995986342, 0.010601727059110999, 0.010647027986124158, 0.01063485094346106, 0.010677012032829225, 0.01092551602050662, 0.010471913032233715, 0.010634226026013494, 0.010945875896140933, 0.010293338098563254, 0.01055537594947964, 0.010643217014148831, 0.010645980015397072, 0.010678439983166754, 0.010570451966486871, 0.010649836040101945, 0.01069292495958507, 0.010717728990130126, 0.010522541007958353, 0.01070056005846709, 0.010585426003672183, 0.01062062697019428, 0.010757423005998135, 0.0105368240037933, 0.01060093892738223, 0.010634903097525239, 0.010806723963469267, 0.010685803019441664, 0.010519028990529478, 0.01055598200764507, 0.010633904952555895, 0.01065782702062279, 0.010641828994266689, 0.01064840005710721, 0.01061642193235457, 0.010612264974042773, 0.010773034999147058, 0.010530870058573782, 0.010628573945723474, 0.010683714062906802, 0.010634173988364637, 0.010590165969915688, 0.01062260998878628, 0.010652004973962903, 0.010681296000257134, 0.010605182033032179, 0.01070969202555716, 0.010612840997055173, 0.01060504000633955, 0.010709066991694272, 0.010576291009783745, 0.010627927957102656, 0.01060935202986002, 0.01061849296092987, 0.01061230304185301, 0.01064020802732557, 0.010612537967972457, 0.010642503970302641, 0.010661420994438231, 0.010677231010049582, 0.010619906010106206, 0.01061613007914275, 0.010640164953656495, 0.010673159034922719, 0.01065313199069351, 0.010602044989354908, 0.010630599921569228, 0.010613079066388309, 0.010680368985049427, 0.010630317032337189, 0.010599260916933417, 0.010660169064067304, 0.010615297011099756, 0.010641937027685344, 0.01069413498044014, 0.010607725940644741, 0.01062774599995464, 0.010622792993672192, 0.010674729011952877, 0.010748065076768398], "tpot_s": 0.010612754442714617} -{"phase": "request", "index": 7, "case": "retrieval-n8192-s7", "score": {"shape_valid": true, "position_matches": [true, true, true, true], "complete": true}, "generated_text": "{\n \"KEY_C3830AF22CA3\": \"VAL_F96D78A77312\",\n \"KEY_7AF82659EEDD\": \"VAL_CE1C0DA20FF8\",\n \"KEY_5E3C3F3158D5\": \"VAL_2CC4F1C5B0E4\",\n \"KEY_5A99CD921008\": \"VAL_61DA90A62470\"\n}", "input_tokens": 8192, "requested_output_tokens": 2048, "output_tokens": 127, "token_ids": [90, 198, 220, 328, 4631, 892, 18, 23, 18, 15, 8018, 17, 17, 4887, 18, 763, 328, 5419, 1352, 24, 21, 35, 22, 23, 32, 22, 22, 18, 16, 17, 487, 198, 220, 328, 4631, 62, 22, 8018, 23, 17, 21, 20, 24, 36, 1425, 35, 763, 328, 5419, 91531, 16, 34, 15, 6151, 17, 15, 1686, 23, 487, 198, 220, 328, 4631, 62, 20, 36, 18, 34, 18, 37, 18, 16, 20, 23, 35, 20, 763, 328, 5419, 62, 17, 3580, 19, 37, 16, 34, 20, 33, 15, 36, 19, 487, 198, 220, 328, 4631, 62, 20, 32, 24, 24, 6281, 24, 17, 16, 15, 15, 23, 763, 328, 5419, 62, 21, 16, 6151, 24, 15, 32, 21, 17, 19, 22, 15, 1, 198, 92, 248046], "terminal": ["stop"], "usage": [{"prompt_tokens": 8192, "total_tokens": 8319, "completion_tokens": 127}], "valid": true, "token_counts_per_choice": [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], "done_unique_last": true, "client_success": true, "client_error": false, "wire_sha256": "a4e3e723f1bdefebca3ef493cf2a448745855b7c16e0accf4bab4d3a3e603875", "prompt_sha256": "b058a3d5ea8d20503575898c27e429821e82c8def7b59babfe7ef73f33a1c8e7", "ttft_s": 1.6795212079305202, "latency_s": 3.0200262329308316, "itl": [0.007463349029421806, 0.010501308017410338, 0.010564266005530953, 0.010767617030069232, 0.010550479986704886, 0.010638064006343484, 0.010612723999656737, 0.010630332981236279, 0.0105935069732368, 0.010639643995091319, 0.010627712006680667, 0.010637880070134997, 0.010624077985994518, 0.010639973916113377, 0.010629437048919499, 0.010639042942784727, 0.010630018077790737, 0.0106638289289549, 0.010655515012331307, 0.010617438005283475, 0.010644429014064372, 0.010642345063388348, 0.010649623931385577, 0.010631570010446012, 0.010662149055860937, 0.010644954978488386, 0.010638835025019944, 0.010629980941303074, 0.010639567975886166, 0.010667648050002754, 0.010624042013660073, 0.010637354920618236, 0.010597160086035728, 0.010649436968378723, 0.01067424495704472, 0.010611931094899774, 0.010667035938240588, 0.010666187037713826, 0.010590960970148444, 0.010662129032425582, 0.010702820960432291, 0.010573563049547374, 0.010656149010173976, 0.010645374888554215, 0.010670869029127061, 0.010670467047020793, 0.010622340952977538, 0.010653243982233107, 0.010623492067679763, 0.010597563930787146, 0.010656893020495772, 0.010687148082070053, 0.010616149986162782, 0.010624349000863731, 0.01065239799208939, 0.010672120959497988, 0.01062096597161144, 0.010639755055308342, 0.010628785938024521, 0.010690958006307483, 0.01062622107565403, 0.010618765954859555, 0.010666675050742924, 0.01063278189394623, 0.010631078039295971, 0.010680072009563446, 0.010733162052929401, 0.010645723901689053, 0.010614171042107046, 0.010569651029072702, 0.01067888701800257, 0.0106372149894014, 0.01064289198257029, 0.010640854015946388, 0.010654652025550604, 0.010592952952720225, 0.010655045043677092, 0.010606142925098538, 0.010636252001859248, 0.010682738036848605, 0.010691481991671026, 0.0107779570389539, 0.01080650498624891, 0.010940016945824027, 0.010919393971562386, 0.010830855113454163, 0.010757022886537015, 0.01076624309644103, 0.010790196945890784, 0.01071821607183665, 0.010711757931858301, 0.010739471996203065, 0.010677459998987615, 0.010810032021254301, 0.010577011969871819, 0.010660203988663852, 0.010688611073419452, 0.010686049005016685, 0.010665294947102666, 0.010690281051211059, 0.010667089954949915, 0.01066335802897811, 0.010680357925593853, 0.010672475094906986, 0.01069127197843045, 0.010712648974731565, 0.010644676978699863, 0.01067321898881346, 0.010677226004190743, 0.01067189103923738, 0.010697540012188256, 0.010673232027329504, 0.01065259799361229, 0.01066827296745032, 0.010711387963965535, 0.010714045027270913, 0.010657883016392589, 0.010642925975844264, 0.010662273038178682, 0.010713637922890484, 0.010664385044947267, 0.010674200020730495, 0.010701001971028745, 0.010705764056183398, 0.01066711894236505, 0.010731335962191224], "tpot_s": 0.010638928769843741} -{"phase": "request", "index": 8, "case": "retrieval-n8192-s8", "score": {"shape_valid": true, "position_matches": [true, true, true, true], "complete": true}, "generated_text": "{\n \"KEY_EF7F95C8C310\": \"VAL_5DE51EB3B32C\",\n \"KEY_68CC3411901D\": \"VAL_2273CB504F70\",\n \"KEY_9D1AA5AF4ADF\": \"VAL_8059F047F934\",\n \"KEY_E0AA0BBC9D57\": \"VAL_E1A082136237\"\n}", "input_tokens": 8192, "requested_output_tokens": 2048, "output_tokens": 126, "token_ids": [90, 198, 220, 328, 4631, 2021, 37, 22, 37, 24, 20, 34, 23, 34, 18, 16, 15, 763, 328, 5419, 62, 20, 1113, 20, 16, 8006, 18, 33, 18, 17, 34, 487, 198, 220, 328, 4631, 62, 21, 23, 3580, 18, 19, 16, 16, 24, 15, 16, 35, 763, 328, 5419, 62, 17, 17, 22, 18, 12607, 20, 15, 19, 37, 22, 15, 487, 198, 220, 328, 4631, 62, 24, 35, 16, 5840, 20, 8018, 19, 88121, 763, 328, 5419, 62, 23, 15, 20, 24, 37, 15, 19, 22, 37, 24, 18, 19, 487, 198, 220, 328, 4631, 2021, 15, 5840, 15, 64500, 24, 35, 20, 22, 763, 328, 5419, 2021, 16, 32, 15, 23, 17, 16, 18, 21, 17, 18, 22, 1, 198, 92, 248046], "terminal": ["stop"], "usage": [{"prompt_tokens": 8192, "total_tokens": 8318, "completion_tokens": 126}], "valid": true, "token_counts_per_choice": [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], "done_unique_last": true, "client_success": true, "client_error": false, "wire_sha256": "0c5518a15e072a69a54b94046327c4be6dae10eabb8a1bd286df786df09a950c", "prompt_sha256": "193b5b7024d00e3b768bc83facb989f6ff28f9b647b0227fecf99f07113ba55c", "ttft_s": 1.6771616539917886, "latency_s": 3.0050365349743515, "itl": [0.007498297025449574, 0.010442425962537527, 0.010859127971343696, 0.010465797036886215, 0.01061687502078712, 0.010578765999525785, 0.010622861911542714, 0.010642510023899376, 0.010708636022172868, 0.010554642067290843, 0.010652201948687434, 0.010598883964121342, 0.010640193009749055, 0.010655608028173447, 0.010634204023517668, 0.01062843797262758, 0.010653426987119019, 0.010638370993547142, 0.010626445058733225, 0.010655327001586556, 0.01062768790870905, 0.010674972087144852, 0.010608037002384663, 0.010625620954670012, 0.010639471001923084, 0.010662268963642418, 0.010652434080839157, 0.010621233959682286, 0.010657098959200084, 0.010621492052450776, 0.010661424021236598, 0.010615482926368713, 0.010681354091502726, 0.01065218192525208, 0.01060154102742672, 0.01062471594195813, 0.010639365995302796, 0.010640033055096865, 0.010655662976205349, 0.010660829022526741, 0.010624220944009721, 0.010666428017430007, 0.010617230087518692, 0.01072792592458427, 0.010656850063242018, 0.010641076019965112, 0.010626027942635119, 0.010649655945599079, 0.01061386801302433, 0.010575241059996188, 0.010618689935654402, 0.010823137010447681, 0.010655143996700644, 0.010615329025313258, 0.010720627033151686, 0.010664222994819283, 0.010778157040476799, 0.010503827943466604, 0.010508888983167708, 0.01078788097947836, 0.010469485074281693, 0.010612176964059472, 0.010648197028785944, 0.010606986936181784, 0.010674237040802836, 0.011272315052337945, 0.01020721800159663, 0.010603247908875346, 0.010519187082536519, 0.010591289959847927, 0.010634033009409904, 0.010662322980351746, 0.010619648033753037, 0.01062167901545763, 0.010760752949863672, 0.010604645009152591, 0.010569507954642177, 0.010632554069161415, 0.010620765970088542, 0.010661927983164787, 0.01060669997241348, 0.010814023087732494, 0.01048012892715633, 0.010594118037261069, 0.0106167399790138, 0.01069596002344042, 0.010593938990496099, 0.010704756947234273, 0.010652716038748622, 0.010573970968835056, 0.010624176007695496, 0.010676275007426739, 0.010578833054751158, 0.01063789299223572, 0.01070055493619293, 0.01053560699801892, 0.010668758070096374, 0.010612513986416161, 0.010627022944390774, 0.010671337018720806, 0.010593002079986036, 0.01064926793333143, 0.010665918001905084, 0.010607199976220727, 0.010648087016306818, 0.010626147035509348, 0.01069383998401463, 0.010615905048325658, 0.010631031938828528, 0.010602946043945849, 0.01068750093691051, 0.010611891048029065, 0.010641756001859903, 0.010746683925390244, 0.01077520102262497, 0.010750113055109978, 0.010781278018839657, 0.010772926965728402, 0.010808486957103014, 0.01073740900028497, 0.01071347901597619, 0.010726446053013206, 0.01070489699486643, 0.010681379004381597, 0.010873510967940092], "tpot_s": 0.010622999047860503} -{"phase": "request", "index": 9, "case": "retrieval-n8192-s9", "score": {"shape_valid": true, "position_matches": [true, true, true, true], "complete": true}, "generated_text": "{\n \"KEY_EF1EBFE0AAB8\": \"VAL_A57016B6636B\",\n \"KEY_8CD68D503195\": \"VAL_D13978547AC6\",\n \"KEY_B0BF1CE73D7E\": \"VAL_2E936A05D1A9\",\n \"KEY_260766402CE1\": \"VAL_7350F5E3B561\"\n}", "input_tokens": 8192, "requested_output_tokens": 2048, "output_tokens": 128, "token_ids": [90, 198, 220, 328, 4631, 2021, 37, 16, 8006, 11091, 15, 32, 1803, 23, 763, 328, 5419, 1511, 20, 22, 15, 16, 21, 33, 21, 21, 18, 21, 33, 487, 198, 220, 328, 4631, 62, 23, 6281, 21, 23, 35, 20, 15, 18, 16, 24, 20, 763, 328, 5419, 1502, 16, 18, 24, 22, 23, 20, 19, 22, 1646, 21, 487, 198, 220, 328, 4631, 1609, 15, 19278, 16, 2036, 22, 18, 35, 22, 36, 763, 328, 5419, 62, 17, 36, 24, 18, 21, 32, 15, 20, 35, 16, 32, 24, 487, 198, 220, 328, 4631, 62, 17, 21, 15, 22, 21, 21, 19, 15, 17, 2036, 16, 763, 328, 5419, 62, 22, 18, 20, 15, 37, 20, 36, 18, 33, 20, 21, 16, 1, 198, 92, 248046], "terminal": ["stop"], "usage": [{"prompt_tokens": 8192, "total_tokens": 8320, "completion_tokens": 128}], "valid": true, "token_counts_per_choice": [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], "done_unique_last": true, "client_success": true, "client_error": false, "wire_sha256": "84848552128f510eaa89904677fe292ed27cb6b9a5883a2aaa3bcbad6d4d5df9", "prompt_sha256": "746334e725720e1dd85d2c06116c10857b3def1efb1fd72400be6245bb03fdaa", "ttft_s": 1.6809620839776471, "latency_s": 3.033107750932686, "itl": [0.007530778995715082, 0.010463924030773342, 0.010612097918055952, 0.01063172600697726, 0.010645522037521005, 0.01060817705001682, 0.010650906944647431, 0.01070772495586425, 0.010585004114545882, 0.010575096937827766, 0.010662734974175692, 0.010641996981576085, 0.010618321015499532, 0.010670094983652234, 0.010649003088474274, 0.010624983930028975, 0.010644602007232606, 0.010630102013237774, 0.010657606064341962, 0.010655346908606589, 0.010628194082528353, 0.010656993952579796, 0.010618934058584273, 0.010636126971803606, 0.01066638098563999, 0.010625664028339088, 0.01074441394302994, 0.010582849034108222, 0.010601515998132527, 0.010684740962460637, 0.010594444000162184, 0.010635682032443583, 0.010660934029147029, 0.010617744992487133, 0.010650859912857413, 0.010864511015824974, 0.010413498035632074, 0.01063399005215615, 0.010647519957274199, 0.010634082951582968, 0.010667502996511757, 0.010642434004694223, 0.010641694068908691, 0.010635957005433738, 0.010658668004907668, 0.010651297983713448, 0.0106375030009076, 0.010620328946970403, 0.010657096980139613, 0.010712640010751784, 0.010562369017861784, 0.010643917019478977, 0.010659483959898353, 0.010620131040923297, 0.01064903603401035, 0.01067120092920959, 0.01061952905729413, 0.010669979965314269, 0.010633538011461496, 0.010640984983183444, 0.010631208075210452, 0.010614449973218143, 0.010625128983519971, 0.01087721495423466, 0.010479037999175489, 0.010793966008350253, 0.010500574018806219, 0.010609838995151222, 0.010655525955371559, 0.010602801106870174, 0.010614536935463548, 0.010678300983272493, 0.010637929080985487, 0.010616159997880459, 0.010678389924578369, 0.010612184065394104, 0.010671875905245543, 0.010644432040862739, 0.010586422053165734, 0.010677307960577309, 0.010692531010136008, 0.010726061998866498, 0.010913530015386641, 0.010975562036037445, 0.010996657889336348, 0.010958733037114143, 0.010970545001327991, 0.010790216969326138, 0.010802890988998115, 0.01071846706327051, 0.010754753951914608, 0.010736675001680851, 0.010722764069214463, 0.010659063002094626, 0.010685016983188689, 0.010722165927290916, 0.010672456002794206, 0.0106647580396384, 0.010718089062720537, 0.010705504915677011, 0.010736508993431926, 0.01074868009891361, 0.010672358912415802, 0.010685587069019675, 0.010661497944965959, 0.01072960498277098, 0.010638625011779368, 0.010688715032301843, 0.010691900039091706, 0.010702817933633924, 0.010682160034775734, 0.010667701018974185, 0.010688376962207258, 0.010692910989746451, 0.01069465302862227, 0.010672400007024407, 0.010666438029147685, 0.010679079918190837, 0.01072242297232151, 0.010658204089850187, 0.010701949009671807, 0.010674231918528676, 0.010708070010878146, 0.01066712907049805, 0.010665387962944806, 0.010686725028790534, 0.010707677924074233], "tpot_s": 0.010646816275236526} -{"phase": "request", "index": 10, "case": "retrieval-n65536-s0", "score": {"shape_valid": true, "position_matches": [true, true, true, true], "complete": true}, "generated_text": "{\n\"KEY_F199ADFDE5EC\": \"VAL_4DC9E697A944\",\n\"KEY_145B316B246A\": \"VAL_054F40195969\",\n\"KEY_D504F8218514\": \"VAL_BD2095C7EF7C\",\n\"KEY_1FFEA3476750\": \"VAL_CD7AB7F38418\"\n}", "input_tokens": 65536, "requested_output_tokens": 2048, "output_tokens": 122, "token_ids": [90, 198, 1, 4631, 1352, 16, 24, 24, 88121, 1113, 20, 7263, 763, 328, 5419, 62, 19, 5449, 24, 36, 21, 24, 22, 32, 24, 19, 19, 487, 198, 1, 4631, 62, 16, 19, 20, 33, 18, 16, 21, 33, 17, 19, 21, 32, 763, 328, 5419, 62, 15, 20, 19, 37, 19, 15, 16, 24, 20, 24, 21, 24, 487, 198, 1, 4631, 1502, 20, 15, 19, 37, 23, 17, 16, 23, 20, 16, 19, 763, 328, 5419, 1609, 35, 17, 15, 24, 20, 34, 22, 14453, 22, 34, 487, 198, 1, 4631, 62, 16, 1686, 18535, 18, 19, 22, 21, 22, 20, 15, 763, 328, 5419, 53346, 22, 1803, 22, 37, 18, 23, 19, 16, 23, 1, 198, 92, 248046], "terminal": ["stop"], "usage": [{"prompt_tokens": 65536, "total_tokens": 65658, "completion_tokens": 122}], "valid": true, "token_counts_per_choice": [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], "done_unique_last": true, "client_success": true, "client_error": false, "wire_sha256": "523ce285f75123cedeaab4c6b7531db37a3f6104d973d9bc4e0c9ddd0c0235ae", "prompt_sha256": "0b3a378a35b3c06ee6cbf7398b3eec2028084d3a690cbe1e11faba5bde5cc51e", "ttft_s": 18.636450113961473, "latency_s": 20.292327199946158, "itl": [0.00026285601779818535, 0.0022526219254359603, 0.013731652987189591, 0.013844240107573569, 0.013832357944920659, 0.013860921026207507, 0.013933922979049385, 0.013856812962330878, 0.014852186082862318, 0.013884329935535789, 0.013896301039494574, 0.013874139986000955, 0.013881291961297393, 0.013912893016822636, 0.013909362955018878, 0.013908851076848805, 0.01391129195690155, 0.013861831976100802, 0.01389865402597934, 0.013862480060197413, 0.013845497975125909, 0.013905431958846748, 0.014481819001957774, 0.013330060988664627, 0.013833331991918385, 0.013881693012081087, 0.013901237049140036, 0.013862207997590303, 0.013888738001696765, 0.01384960999712348, 0.013890085043385625, 0.013866216992028058, 0.013894047006033361, 0.013828564900904894, 0.013873442076146603, 0.013872475945390761, 0.01392215397208929, 0.013845740002579987, 0.014752854011021554, 0.013929711072705686, 0.01389991701580584, 0.013858354999683797, 0.013947646948508918, 0.013820769032463431, 0.013920133002102375, 0.013826309004798532, 0.0138844900066033, 0.013888061977922916, 0.013860448030754924, 0.013867426896467805, 0.013923058053478599, 0.014188172994181514, 0.01375677902251482, 0.013852396979928017, 0.013923685997724533, 0.013837901991792023, 0.013882283004932106, 0.013900969992391765, 0.0138600820209831, 0.013854427030310035, 0.013878805912099779, 0.013848278089426458, 0.013874106924049556, 0.013862631050869823, 0.013934755930677056, 0.013655686052516103, 0.013885880005545914, 0.013842287939041853, 0.01387000409886241, 0.01389082195237279, 0.013904188992455602, 0.013883250067010522, 0.013899952988140285, 0.01383353490382433, 0.013858688063919544, 0.013921482022851706, 0.013841575011610985, 0.013849409995600581, 0.013894887990318239, 0.013963470933958888, 0.013794098980724812, 0.013866864028386772, 0.013858232065103948, 0.01386510394513607, 0.013864703010767698, 0.013874531025066972, 0.013912622933275998, 0.01386171905323863, 0.01385739294346422, 0.013853334006853402, 0.013889747089706361, 0.01388318999670446, 0.013867789995856583, 0.013834910001605749, 0.013868122012354434, 0.013924042927101254, 0.013883095001801848, 0.013867154018953443, 0.01386709394864738, 0.01391302002593875, 0.013852460077032447, 0.013904043938964605, 0.013884932966902852, 0.013965169084258378, 0.013857570011168718, 0.013859023922123015, 0.013951339991763234, 0.0139491050504148, 0.013786599040031433, 0.013869209913536906, 0.013860014034435153, 0.013931966037489474, 0.013922190992161632, 0.013792448909953237, 0.014240396092645824, 0.013579090009443462, 0.013966881902888417, 0.013791488017886877, 0.01385775103699416, 0.013988581951707602, 0.013829124043695629], "tpot_s": 0.013684934594914749} -{"phase": "request", "index": 11, "case": "retrieval-n65536-s1", "score": {"shape_valid": true, "position_matches": [true, true, true, true], "complete": true}, "generated_text": "{\n\"KEY_D149BAB85691\": \"VAL_AA85D60A28A8\",\n\"KEY_EEF286EF5EDC\": \"VAL_34DEFF8B16EC\",\n\"KEY_411578C5828D\": \"VAL_128D4D609ACA\",\n\"KEY_67DE09AC6DCE\": \"VAL_AECE17ACD968\"\n}", "input_tokens": 65536, "requested_output_tokens": 2048, "output_tokens": 115, "token_ids": [90, 198, 1, 4631, 1502, 16, 19, 24, 92771, 23, 20, 21, 24, 16, 763, 328, 5419, 79115, 23, 20, 35, 21, 15, 32, 17, 23, 32, 23, 487, 198, 1, 4631, 2021, 14453, 17, 23, 21, 14453, 20, 1425, 34, 763, 328, 5419, 62, 18, 19, 1113, 1686, 23, 33, 16, 21, 7263, 487, 198, 1, 4631, 62, 19, 16, 16, 20, 22, 23, 34, 20, 23, 17, 23, 35, 763, 328, 5419, 62, 16, 17, 23, 35, 19, 35, 21, 15, 24, 60431, 487, 198, 1, 4631, 62, 21, 22, 1113, 15, 24, 1646, 21, 35, 2036, 763, 328, 5419, 1511, 233737, 16, 22, 1646, 35, 24, 21, 23, 1, 198, 92, 248046], "terminal": ["stop"], "usage": [{"prompt_tokens": 65536, "total_tokens": 65651, "completion_tokens": 115}], "valid": true, "token_counts_per_choice": [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], "done_unique_last": true, "client_success": true, "client_error": false, "wire_sha256": "40563d1c4e8879a85800e9858fc5b8f90a8fd6f3f142fb44f10a71f33e3d47e5", "prompt_sha256": "b25fa9f7e93b04220dbb232db729ab7f67db8122798f1fe3d087b7b131978393", "ttft_s": 18.624157981947064, "latency_s": 20.17839317698963, "itl": [0.00035761098843067884, 5.674082785844803e-06, 0.012874734937213361, 0.013862660038284957, 0.013878693920560181, 0.013837294070981443, 0.013895928976126015, 0.013906499021686614, 0.013850703951902688, 0.013933609006926417, 0.013873112038709223, 0.013825910980813205, 0.01388247695285827, 0.013869399088434875, 0.0139544130070135, 0.013846980989910662, 0.013873084913939238, 0.01387959101703018, 0.013913831091485918, 0.013949908898212016, 0.013790867058560252, 0.013859385042451322, 0.013872718904167414, 0.013885683030821383, 0.013858256046660244, 0.013872277922928333, 0.01387885503936559, 0.013864269014447927, 0.013871419010683894, 0.01390774897299707, 0.013933810056187212, 0.013820845982991159, 0.013873629970476031, 0.013904714025557041, 0.013856750912964344, 0.013890270027332008, 0.01390603103209287, 0.013896450051106513, 0.013905134983360767, 0.013906841981224716, 0.013884456013329327, 0.013959049945697188, 0.01383367006201297, 0.01389964995905757, 0.013891258975490928, 0.013891934067942202, 0.013872046954929829, 0.013871446019038558, 0.013896035961806774, 0.013872331008315086, 0.013884364045225084, 0.013880647951737046, 0.013846282963640988, 0.013887400040403008, 0.013896633987314999, 0.013882943079806864, 0.013896141899749637, 0.013899106997996569, 0.01386738708242774, 0.013876069919206202, 0.013856002013199031, 0.013877006014809012, 0.013863209984265268, 0.013857395038940012, 0.01386775600258261, 0.013886169996112585, 0.013869376038201153, 0.013924827915616333, 0.013874148018658161, 0.013925757026299834, 0.01388416695408523, 0.013917563017457724, 0.013888549990952015, 0.013881447026506066, 0.013897691969759762, 0.013931649038568139, 0.013895147014409304, 0.013881211983971298, 0.013899547979235649, 0.013895853073336184, 0.01389843295328319, 0.01387698296457529, 0.0138843460008502, 0.013871707022190094, 0.013902292004786432, 0.013884786982089281, 0.013891151989810169, 0.01387109700590372, 0.013876458979211748, 0.013889776077121496, 0.013873366988264024, 0.013906224980019033, 0.013921799953095615, 0.013700940064154565, 0.013932702946476638, 0.013792828074656427, 0.01385725091677159, 0.013897289987653494, 0.014039095025509596, 0.01370591705199331, 0.013950853957794607, 0.013796985964290798, 0.014035158092156053, 0.013834844925440848, 0.013795083039440215, 0.013869963004253805, 0.013908799970522523, 0.013888595043681562, 0.013827254995703697, 0.013878958998247981, 0.013894181931391358, 0.013894216041080654, 0.013964176992885768, 0.013930264045484364], "tpot_s": 0.01363364206177689} -{"phase": "request", "index": 12, "case": "retrieval-n65536-s2", "score": {"shape_valid": true, "position_matches": [true, true, true, true], "complete": true}, "generated_text": "{\n \"KEY_171981E46F20\": \"VAL_E0A822C2E874\",\n \"KEY_0AE521510615\": \"VAL_94709834762E\",\n \"KEY_443C031EBE71\": \"VAL_1920A538E912\",\n \"KEY_4559964CF82F\": \"VAL_E7BCCF21D6E4\"\n}", "input_tokens": 65536, "requested_output_tokens": 2048, "output_tokens": 134, "token_ids": [90, 198, 220, 328, 4631, 62, 16, 22, 16, 24, 23, 16, 36, 19, 21, 37, 17, 15, 763, 328, 5419, 2021, 15, 32, 23, 17, 17, 34, 17, 36, 23, 22, 19, 487, 198, 220, 328, 4631, 62, 15, 13276, 20, 17, 16, 20, 16, 15, 21, 16, 20, 763, 328, 5419, 62, 24, 19, 22, 15, 24, 23, 18, 19, 22, 21, 17, 36, 487, 198, 220, 328, 4631, 62, 19, 19, 18, 34, 15, 18, 16, 8006, 36, 22, 16, 763, 328, 5419, 62, 16, 24, 17, 15, 32, 20, 18, 23, 36, 24, 16, 17, 487, 198, 220, 328, 4631, 62, 19, 20, 20, 24, 24, 21, 19, 9362, 23, 17, 37, 763, 328, 5419, 2021, 22, 33, 3580, 37, 17, 16, 35, 21, 36, 19, 1, 198, 92, 248046], "terminal": ["stop"], "usage": [{"prompt_tokens": 65536, "total_tokens": 65670, "completion_tokens": 134}], "valid": true, "token_counts_per_choice": [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], "done_unique_last": true, "client_success": true, "client_error": false, "wire_sha256": "24e056dacc2ffd4db0e44618188cb9c1fe1b7dfa793d6c2640754574e7c402fd", "prompt_sha256": "ad32fc9e6db4f3770d2a5922f38b1d317d6d09dfeb03c7fac5de4174c7b01489", "ttft_s": 18.59972815902438, "latency_s": 20.419999910052866, "itl": [0.00033864204306155443, 0.0011273639975115657, 0.013771819998510182, 0.013889973983168602, 0.013874512980692089, 0.013869247981347144, 0.01407175196800381, 0.013739349087700248, 0.01396239991299808, 0.013958521070890129, 0.01378751895390451, 0.013927767053246498, 0.013843556982465088, 0.013816969003528357, 0.013959259958937764, 0.01401102205272764, 0.013774910010397434, 0.014015038963407278, 0.013784224982373416, 0.013833304983563721, 0.013827232061885297, 0.013940087985247374, 0.01390465593431145, 0.013876040000468493, 0.013778224005363882, 0.013854925986379385, 0.013968156999908388, 0.01388472004327923, 0.013955457019619644, 0.013690753025002778, 0.0138383669545874, 0.013872107956558466, 0.013931752066127956, 0.013902244041673839, 0.01388610596768558, 0.013942581019364297, 0.013935560011304915, 0.014045365969650447, 0.013786999974399805, 0.01379745895974338, 0.013857964077033103, 0.013872428913600743, 0.013875550008378923, 0.013884028070606291, 0.013917690026573837, 0.013889914960600436, 0.013833475997671485, 0.013882305938750505, 0.013941237004473805, 0.0138656219933182, 0.013820250052958727, 0.01389803399797529, 0.013892709976062179, 0.013881981023587286, 0.013845104956999421, 0.013855259050615132, 0.013856063946150243, 0.013889961992390454, 0.01388240500818938, 0.01386517402715981, 0.013893760973587632, 0.013863106025382876, 0.013963685021735728, 0.01383800501935184, 0.013948190957307816, 0.013900420046411455, 0.013866701978258789, 0.013979139039292932, 0.013826462905853987, 0.014039459056220949, 0.013841095962561667, 0.013939711032435298, 0.013726477976888418, 0.01386011007707566, 0.013873082934878767, 0.013914852985180914, 0.013841759064234793, 0.014065940980799496, 0.013737902976572514, 0.014087129035033286, 0.013827919028699398, 0.01376844197511673, 0.013856534962542355, 0.013980875024572015, 0.013820238993503153, 0.013811256038025022, 0.01385361491702497, 0.013878982048481703, 0.013861179002560675, 0.013968223938718438, 0.01406671607401222, 0.013770265970379114, 0.013798673055134714, 0.013903346960432827, 0.013927798951044679, 0.014053563005290926, 0.013686864986084402, 0.013882574043236673, 0.013855693046934903, 0.013895227923057973, 0.013886503991670907, 0.013942082994617522, 0.013877278077416122, 0.0138808679766953, 0.01389196899253875, 0.013873665011487901, 0.013933341018855572, 0.013862245017662644, 0.013928087893873453, 0.013848879025317729, 0.013895613024942577, 0.013856134028173983, 0.013880261918529868, 0.013875456061214209, 0.013856863952241838, 0.01393636001739651, 0.013845370034687221, 0.013869617017917335, 0.013869129936210811, 0.013889253023080528, 0.013929379056207836, 0.013818862964399159, 0.01384115603286773, 0.01388770795892924, 0.013867805944755673, 0.013910685083828866, 0.013887077919207513, 0.013884503045119345, 0.013914986979216337, 0.013898160075768828, 0.013873616000637412, 0.013948429957963526, 0.013952304027043283], "tpot_s": 0.01368625376713147} -{"phase": "request", "index": 13, "case": "retrieval-n65536-s3", "score": {"shape_valid": true, "position_matches": [true, true, true, true], "complete": true}, "generated_text": "{\n\"KEY_B2115793251B\": \"VAL_616A8972FFEF\",\n\"KEY_32C0D8CD2496\": \"VAL_3E80F953CA33\",\n\"KEY_7C45203F80D3\": \"VAL_D993F1B00109\",\n\"KEY_EEEDBD9EDA93\": \"VAL_AB6D4F5362AC\"\n}", "input_tokens": 65536, "requested_output_tokens": 2048, "output_tokens": 122, "token_ids": [90, 198, 1, 4631, 1609, 17, 16, 16, 20, 22, 24, 18, 17, 20, 16, 33, 763, 328, 5419, 62, 21, 16, 21, 32, 23, 24, 22, 17, 1686, 14453, 487, 198, 1, 4631, 62, 18, 17, 34, 15, 35, 23, 6281, 17, 19, 24, 21, 763, 328, 5419, 62, 18, 36, 23, 15, 37, 24, 20, 18, 4887, 18, 18, 487, 198, 1, 4631, 62, 22, 34, 19, 20, 17, 15, 18, 37, 23, 15, 35, 18, 763, 328, 5419, 1502, 24, 24, 18, 37, 16, 33, 15, 15, 16, 15, 24, 487, 198, 1, 4631, 2021, 36, 1425, 9265, 24, 83765, 24, 18, 763, 328, 5419, 31561, 21, 35, 19, 37, 20, 18, 21, 17, 1646, 1, 198, 92, 248046], "terminal": ["stop"], "usage": [{"prompt_tokens": 65536, "total_tokens": 65658, "completion_tokens": 122}], "valid": true, "token_counts_per_choice": [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], "done_unique_last": true, "client_success": true, "client_error": false, "wire_sha256": "b77729065e8833e3f2ebe408396bab89b742deed154a7ae698da0cbc79d56f5c", "prompt_sha256": "77e868b0bc709d1f955df4a2ca499fa3463bb84719e6d9fc90e69d432a37d81f", "ttft_s": 18.608797868015245, "latency_s": 20.263611733098514, "itl": [0.0003350540064275265, 0.0025623140390962362, 0.013775930041447282, 0.013987686950713396, 0.013785655028186738, 0.013868863927200437, 0.013856647070497274, 0.013828922994434834, 0.013941586017608643, 0.013830212992615998, 0.01390625792555511, 0.013901776052080095, 0.013817970990203321, 0.013848806032910943, 0.013932009926065803, 0.013888010988011956, 0.013968005077913404, 0.01379766792524606, 0.013933629030361772, 0.01385240803938359, 0.013831085991114378, 0.013912194990552962, 0.013778461026959121, 0.013848520000465214, 0.013926739920862019, 0.013872700044885278, 0.01386817405000329, 0.013902179896831512, 0.013885891996324062, 0.01386546203866601, 0.014036940061487257, 0.013773370999842882, 0.013921523932367563, 0.013873376999981701, 0.013840888976119459, 0.013891295064240694, 0.013932590023614466, 0.013860990991815925, 0.01386445399839431, 0.013909657951444387, 0.014125813031569123, 0.013680250034667552, 0.013869303977116942, 0.013844594941474497, 0.013902208069339395, 0.013834711979143322, 0.013912874041125178, 0.013912782887928188, 0.01383102301042527, 0.013922757003456354, 0.013847565045580268, 0.013951653963886201, 0.013806939008645713, 0.013870803988538682, 0.014101858017966151, 0.013736972003243864, 0.01386223896406591, 0.013895831070840359, 0.013798004947602749, 0.013858822057954967, 0.013872272917069495, 0.013958160998299718, 0.013880533049814403, 0.013802394038066268, 0.013888170942664146, 0.013907768065109849, 0.013894196948967874, 0.013987059006467462, 0.013862789026461542, 0.013884249958209693, 0.01397379906848073, 0.013801580993458629, 0.013838632963597775, 0.013866959023289382, 0.013873480958864093, 0.01389144198037684, 0.013873878051526845, 0.014006873941980302, 0.013726372038945556, 0.013897177996113896, 0.01392122800461948, 0.013836674974299967, 0.013875955017283559, 0.013879592064768076, 0.013919775956310332, 0.013858607038855553, 0.013858608901500702, 0.013888132059946656, 0.013874497963115573, 0.01389962004031986, 0.013822401990182698, 0.013880174956284463, 0.013891178998164833, 0.013879343983717263, 0.013881449005566537, 0.013906224048696458, 0.013902681996114552, 0.013857653015293181, 0.01391954196151346, 0.013880770071409643, 0.013869826914742589, 0.013906373060308397, 0.013870629016309977, 0.014058989938348532, 0.013708790997043252, 0.01386489300057292, 0.013885176973417401, 0.013905503088608384, 0.013860838953405619, 0.013872527983039618, 0.013891877024434507, 0.01389220601413399, 0.013909650966525078, 0.013868084992282093, 0.013924209983088076, 0.013879652018658817, 0.013807825976982713, 0.013860478065907955, 0.013923843973316252, 0.013854952063411474, 0.014092708006501198], "tpot_s": 0.01367614764531628} -{"phase": "request", "index": 14, "case": "retrieval-n65536-s4", "score": {"shape_valid": true, "position_matches": [true, true, true, true], "complete": true}, "generated_text": "{\n\"KEY_8699EF52D026\": \"VAL_84F827532C15\",\n\"KEY_9C2DDEC0F196\": \"VAL_069ACF9EA29C\",\n\"KEY_A4ACB8658F04\": \"VAL_7644D34AE0F9\",\n\"KEY_5123C1DD6BE8\": \"VAL_81578A2A4507\"\n}", "input_tokens": 65536, "requested_output_tokens": 2048, "output_tokens": 126, "token_ids": [90, 198, 1, 4631, 62, 23, 21, 24, 24, 14453, 20, 17, 35, 15, 17, 21, 763, 328, 5419, 62, 23, 19, 37, 23, 17, 22, 20, 18, 17, 34, 16, 20, 487, 198, 1, 4631, 62, 24, 34, 17, 35, 29723, 15, 37, 16, 24, 21, 763, 328, 5419, 62, 15, 21, 24, 1646, 37, 24, 18535, 17, 24, 34, 487, 198, 1, 4631, 1511, 19, 1646, 33, 23, 21, 20, 23, 37, 15, 19, 763, 328, 5419, 62, 22, 21, 19, 19, 35, 18, 19, 13276, 15, 37, 24, 487, 198, 1, 4631, 62, 20, 16, 17, 18, 34, 16, 3967, 21, 11261, 23, 763, 328, 5419, 62, 23, 16, 20, 22, 23, 32, 17, 32, 19, 20, 15, 22, 1, 198, 92, 248046], "terminal": ["stop"], "usage": [{"prompt_tokens": 65536, "total_tokens": 65662, "completion_tokens": 126}], "valid": true, "token_counts_per_choice": [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], "done_unique_last": true, "client_success": true, "client_error": false, "wire_sha256": "22e6316b1bf89f984112cace1fc16d8dcc1afbd666d1a3afb39656028e30b5cf", "prompt_sha256": "78020a13451217dfba221a3e718cdc0c52bb42764156ffdc603280f22c8404f8", "ttft_s": 18.60291354509536, "latency_s": 20.313377944054082, "itl": [0.0003277449868619442, 0.0026806569658219814, 0.0136505919508636, 0.013856814010068774, 0.014023707015439868, 0.01384406303986907, 0.013750563957728446, 0.013865070068277419, 0.013855080935172737, 0.013878936995752156, 0.013881320017389953, 0.013853225973434746, 0.014065881026908755, 0.013763051014393568, 0.013821662054397166, 0.013878565980121493, 0.013908884953707457, 0.01386160496622324, 0.013948572101071477, 0.014098717947490513, 0.013717881054617465, 0.01381028990726918, 0.01384261203929782, 0.013977317954413593, 0.01378679508343339, 0.01384234894067049, 0.013917303062044084, 0.013821644010022283, 0.013876986922696233, 0.01403468707576394, 0.013875891920179129, 0.013787353062070906, 0.01388453901745379, 0.013872678973712027, 0.013958162977360189, 0.01414528104942292, 0.013666550978086889, 0.01391376496758312, 0.013899681041948497, 0.01378198096062988, 0.013885005959309638, 0.014041349990293384, 0.013768730103038251, 0.013866844936273992, 0.013817094033583999, 0.0139354239217937, 0.013862410094588995, 0.013840222964063287, 0.013963905978016555, 0.013868159032426775, 0.013878430938348174, 0.013867789995856583, 0.013896345044486225, 0.013903515064157546, 0.013863234897144139, 0.013883044011890888, 0.013891158043406904, 0.013854093966074288, 0.01382333307992667, 0.013913389993831515, 0.013855423894710839, 0.013864625012502074, 0.014017085079103708, 0.013764702016487718, 0.013858362915925682, 0.013872347073629498, 0.013908102991990745, 0.013867930974811316, 0.013915583956986666, 0.013874794007278979, 0.013907231041230261, 0.01389062195084989, 0.013903929037041962, 0.013863528030924499, 0.013897780911065638, 0.013875031028874218, 0.013878782046958804, 0.014420085004530847, 0.013468561926856637, 0.013825278030708432, 0.013867871020920575, 0.01385442295577377, 0.013886579079553485, 0.01456158293876797, 0.0133237469708547, 0.013809027033858001, 0.013859997037798166, 0.013855325989425182, 0.01388970494735986, 0.013860222068615258, 0.013856324017979205, 0.013915700954385102, 0.01384571404196322, 0.013883192907087505, 0.013876601005904377, 0.013961595017462969, 0.013844506000168622, 0.013927718973718584, 0.013973143068142235, 0.01401059702038765, 0.013806193950586021, 0.013902080012485385, 0.013855222961865366, 0.01380733703263104, 0.013859739061444998, 0.013913106988184154, 0.013867671019397676, 0.014050978934392333, 0.013718339032493532, 0.014047796954400837, 0.013764905976131558, 0.013848557020537555, 0.01384479901753366, 0.013889631954953074, 0.01387651008553803, 0.013864449923858047, 0.013903114013373852, 0.013935131020843983, 0.01385868398938328, 0.013839218998327851, 0.013902463018894196, 0.013881672057323158, 0.01387246698141098, 0.013855939963832498, 0.014053079998120666], "tpot_s": 0.013683715191669762} -{"phase": "request", "index": 15, "case": "retrieval-n65536-s5", "score": {"shape_valid": true, "position_matches": [true, true, true, true], "complete": true}, "generated_text": "{\n\"KEY_38FD70C43096\": \"VAL_EE1BDA4688E9\",\n\"KEY_BF77F05BE6E4\": \"VAL_667B69AC1A10\",\n\"KEY_0C61812A34A9\": \"VAL_9D82AE136DEE\",\n\"KEY_D4DE64056B19\": \"VAL_8A6F6053080D\"\n}", "input_tokens": 65536, "requested_output_tokens": 2048, "output_tokens": 126, "token_ids": [90, 198, 1, 4631, 62, 18, 23, 14176, 22, 15, 34, 19, 18, 15, 24, 21, 763, 328, 5419, 2021, 36, 16, 33, 6151, 19, 21, 23, 23, 36, 24, 487, 198, 1, 4631, 1609, 37, 22, 22, 37, 15, 20, 11261, 21, 36, 19, 763, 328, 5419, 62, 21, 21, 22, 33, 21, 24, 1646, 16, 32, 16, 15, 487, 198, 1, 4631, 62, 15, 34, 21, 16, 23, 16, 17, 32, 18, 19, 32, 24, 763, 328, 5419, 62, 24, 35, 23, 17, 13276, 16, 18, 21, 1113, 36, 487, 198, 1, 4631, 1502, 19, 1113, 21, 19, 15, 20, 21, 33, 16, 24, 763, 328, 5419, 62, 23, 32, 21, 37, 21, 15, 20, 18, 15, 23, 15, 35, 1, 198, 92, 248046], "terminal": ["stop"], "usage": [{"prompt_tokens": 65536, "total_tokens": 65662, "completion_tokens": 126}], "valid": true, "token_counts_per_choice": [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], "done_unique_last": true, "client_success": true, "client_error": false, "wire_sha256": "cd2723e611927176f0a795879fe977af3d459040d21b7ac42bf2236ae7e9b63d", "prompt_sha256": "f0b6b92c65eb9c509288888d450ef0a5092daa68881435783128a9d9971def09", "ttft_s": 18.60937272803858, "latency_s": 20.317997936042957, "itl": [0.00034873594995588064, 0.0006299880333244801, 0.013726362958550453, 0.013879265054129064, 0.014095256919972599, 0.013770877034403384, 0.013733942992985249, 0.013861011015251279, 0.013879294041544199, 0.013834822922945023, 0.01394719909876585, 0.013841590960510075, 0.013894901028834283, 0.013864025007933378, 0.013895761920139194, 0.013880255981348455, 0.014001464005559683, 0.013759399997070432, 0.01386793504934758, 0.013888287940062582, 0.013861946063116193, 0.013878074008971453, 0.013879328034818172, 0.013836694997735322, 0.013862044899724424, 0.013932191068306565, 0.01387322903610766, 0.013934178976342082, 0.013879374018870294, 0.013905069907195866, 0.013897225027903914, 0.013901600032113492, 0.01391528092790395, 0.013840496074408293, 0.013863274944014847, 0.013866541092284024, 0.013923724996857345, 0.013905093888752162, 0.013867664034478366, 0.013875424047000706, 0.013888770015910268, 0.0138938749441877, 0.013869995018467307, 0.013889765017665923, 0.01386164699215442, 0.01388727396260947, 0.01390443101990968, 0.013868295005522668, 0.013888188055716455, 0.013894337927922606, 0.013855771045200527, 0.013888956978917122, 0.013867269037291408, 0.013899042969569564, 0.013872549985535443, 0.013893713941797614, 0.013854194083251059, 0.013874204945750535, 0.013930062064900994, 0.01381978893186897, 0.01387887797318399, 0.013869261019863188, 0.013917093048803508, 0.013882266939617693, 0.01393025100696832, 0.013838779996149242, 0.013895092066377401, 0.013904297025874257, 0.013881554012186825, 0.013936142902821302, 0.01386032800655812, 0.014033826068043709, 0.013761418987996876, 0.013955354923382401, 0.013803894049488008, 0.013892355957068503, 0.013872898067347705, 0.01388866500928998, 0.0138830830110237, 0.013874275959096849, 0.01392203604336828, 0.013825512956827879, 0.013964651967398822, 0.013820616994053125, 0.013894106028601527, 0.014107731054536998, 0.013710084953345358, 0.013817830011248589, 0.013917995966039598, 0.01383250905200839, 0.013871548930183053, 0.01387052598875016, 0.013848836999386549, 0.01389025105163455, 0.01383864798117429, 0.013914576033130288, 0.013834033044986427, 0.013884753920137882, 0.013914418988861144, 0.0138846900081262, 0.013888854067772627, 0.013894998002797365, 0.013907412998378277, 0.013897252967581153, 0.013885580003261566, 0.013900018995627761, 0.013876265031285584, 0.013858851976692677, 0.013872766983695328, 0.013857670011930168, 0.013912043999880552, 0.01385218498762697, 0.013913566945120692, 0.013866367051377892, 0.013900978025048971, 0.01388139696791768, 0.01388172700535506, 0.013869091053493321, 0.013873949996195734, 0.013930339948274195, 0.013877916033379734, 0.013856068952009082, 0.014217494986951351, 0.013666338985785842, 0.014060032088309526], "tpot_s": 0.013669001664035021} -{"phase": "request", "index": 16, "case": "retrieval-n65536-s6", "score": {"shape_valid": true, "position_matches": [true, true, true, true], "complete": true}, "generated_text": "{\n\"KEY_880F23B3AEB1\": \"VAL_19009431F219\",\n\"KEY_D404B6DFC7DF\": \"VAL_F6238EB3EF83\",\n\"KEY_39B55FA9E793\": \"VAL_B6C1FF528106\",\n\"KEY_C0B0373FD4E3\": \"VAL_8F7050CDB233\"\n}", "input_tokens": 65536, "requested_output_tokens": 2048, "output_tokens": 123, "token_ids": [90, 198, 1, 4631, 62, 23, 23, 15, 37, 17, 18, 33, 18, 32, 8006, 16, 763, 328, 5419, 62, 16, 24, 15, 15, 24, 19, 18, 16, 37, 17, 16, 24, 487, 198, 1, 4631, 1502, 19, 15, 19, 33, 21, 5095, 34, 22, 5095, 763, 328, 5419, 1352, 21, 17, 18, 23, 8006, 18, 14453, 23, 18, 487, 198, 1, 4631, 62, 18, 24, 33, 20, 20, 3505, 24, 36, 22, 24, 18, 763, 328, 5419, 1609, 21, 34, 16, 1686, 20, 17, 23, 16, 15, 21, 487, 198, 1, 4631, 892, 15, 33, 15, 18, 22, 18, 14176, 19, 36, 18, 763, 328, 5419, 62, 23, 37, 22, 15, 20, 15, 34, 3392, 17, 18, 18, 1, 198, 92, 248046], "terminal": ["stop"], "usage": [{"prompt_tokens": 65536, "total_tokens": 65659, "completion_tokens": 123}], "valid": true, "token_counts_per_choice": [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], "done_unique_last": true, "client_success": true, "client_error": false, "wire_sha256": "de9292e31939eea5178157901cd4c3d168b09b84f4951738a069b55a14c82a36", "prompt_sha256": "288e7801790710bdcef491199a51536eb50ebb2f01a2861d30a019d12def8aab", "ttft_s": 18.607708627008833, "latency_s": 20.276684029959142, "itl": [0.00034801394212991, 0.00293334003072232, 0.013725548051297665, 0.01385048299562186, 0.013868222944438457, 0.013893490075133741, 0.013849124894477427, 0.01388628501445055, 0.014267847058363259, 0.01361132797319442, 0.013819182058796287, 0.013871167902834713, 0.013824837049469352, 0.013916119001805782, 0.013818138046190143, 0.013825580943375826, 0.014034830033779144, 0.013723794952966273, 0.013934359070844948, 0.01377659197896719, 0.013850439921952784, 0.0139162220293656, 0.013832597993314266, 0.01390871696639806, 0.013944131089374423, 0.013845309964381158, 0.013887810055166483, 0.014267435995861888, 0.01359867793507874, 0.01383833005093038, 0.01387959998100996, 0.013855210971087217, 0.013907875050790608, 0.013858643011189997, 0.013874845928512514, 0.013879691017791629, 0.013864513952285051, 0.013868205016478896, 0.013880069018341601, 0.013867311994545162, 0.01387899206019938, 0.013990322011522949, 0.013810119940899312, 0.013878623954951763, 0.013862046063877642, 0.013861002982594073, 0.013902936014346778, 0.013899561949074268, 0.013830003095790744, 0.013898384990170598, 0.013859833939932287, 0.013878592057153583, 0.013897351920604706, 0.01389773003757, 0.013869634014554322, 0.013879857957363129, 0.013962493045255542, 0.013852391042746603, 0.013865336892195046, 0.013879522099159658, 0.013889239984564483, 0.013909543980844319, 0.013932695961557329, 0.013850616989657283, 0.01390236709266901, 0.013906457927078009, 0.01386557100340724, 0.01388513797428459, 0.013862573076039553, 0.013902909005992115, 0.01386849896516651, 0.0138588190311566, 0.013922916958108544, 0.013887273031286895, 0.013857202953658998, 0.013886107015423477, 0.013907775050029159, 0.013906945940107107, 0.013872412033379078, 0.013980526011437178, 0.013792295940220356, 0.013925885083153844, 0.013781221932731569, 0.013902259059250355, 0.013789037009701133, 0.013849765993654728, 0.013912983937188983, 0.013869091984815896, 0.013894998002797365, 0.013921468984335661, 0.01385832007508725, 0.013888141955249012, 0.013880776008591056, 0.013874675962142646, 0.013884034007787704, 0.013895358075387776, 0.013869246002286673, 0.013914104900322855, 0.013863746076822281, 0.013865481014363468, 0.013945480925031006, 0.013921646983362734, 0.013818040024489164, 0.013889434980228543, 0.01389675703831017, 0.013860495062544942, 0.013899547979235649, 0.013871480943635106, 0.013900700025260448, 0.013862257008440793, 0.013931820983998477, 0.013893214985728264, 0.013879314996302128, 0.013860329054296017, 0.014732440002262592, 0.01298578700516373, 0.013884162995964289, 0.013911711983382702, 0.013854683027602732, 0.013847659924067557, 0.014966665068641305, 0.01301463192794472], "tpot_s": 0.013680126253691059} -{"phase": "request", "index": 17, "case": "retrieval-n65536-s7", "score": {"shape_valid": true, "position_matches": [true, true, true, true], "complete": true}, "generated_text": "{\n\"KEY_EF3AAA7B95B4\": \"VAL_459DC3229BBD\",\n\"KEY_8EE66A56A687\": \"VAL_45B85352F16D\",\n\"KEY_F9DB04A30085\": \"VAL_E6FCAA87F646\",\n\"KEY_58D5FBD6608A\": \"VAL_49A2E1F7A24E\"\n}", "input_tokens": 65536, "requested_output_tokens": 2048, "output_tokens": 124, "token_ids": [90, 198, 1, 4631, 2021, 37, 18, 48421, 22, 33, 24, 20, 33, 19, 763, 328, 5419, 62, 19, 20, 24, 5449, 18, 17, 17, 24, 33, 9265, 487, 198, 1, 4631, 62, 23, 6878, 21, 21, 32, 20, 21, 32, 21, 23, 22, 763, 328, 5419, 62, 19, 20, 33, 23, 20, 18, 20, 17, 37, 16, 21, 35, 487, 198, 1, 4631, 1352, 24, 3392, 15, 19, 32, 18, 15, 15, 23, 20, 763, 328, 5419, 2021, 21, 37, 27698, 23, 22, 37, 21, 19, 21, 487, 198, 1, 4631, 62, 20, 23, 35, 20, 37, 9265, 21, 21, 15, 23, 32, 763, 328, 5419, 62, 19, 24, 32, 17, 36, 16, 37, 22, 32, 17, 19, 36, 1, 198, 92, 248046], "terminal": ["stop"], "usage": [{"prompt_tokens": 65536, "total_tokens": 65660, "completion_tokens": 124}], "valid": true, "token_counts_per_choice": [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], "done_unique_last": true, "client_success": true, "client_error": false, "wire_sha256": "ef5e0d89d140482f4d853bfff6c8d2b8545478cae2ea0c0dce9008c49a3fef7a", "prompt_sha256": "e2d8a9d40ca2020577d647021a3da60b11e48bbb018efdf6b2414ca8dc86b451", "ttft_s": 18.597550817998126, "latency_s": 20.280216474086046, "itl": [0.0003418700071051717, 0.0027535120025277138, 0.013766143005341291, 0.013807550072669983, 0.013863049913197756, 0.013853131094947457, 0.013868324924260378, 0.013884335989132524, 0.013860246050171554, 0.013899330981075764, 0.013876446057111025, 0.013889280962757766, 0.013908713008277118, 0.013827089918777347, 0.013871047995053232, 0.013894745032303035, 0.013861639075912535, 0.013937088893726468, 0.013837547041475773, 0.013852408970706165, 0.01390971103683114, 0.01385680795647204, 0.01387196104042232, 0.013881533057428896, 0.013872846961021423, 0.013901671976782382, 0.01381895795930177, 0.013861272018402815, 0.013871631002984941, 0.013891846989281476, 0.013921773992478848, 0.014125167042948306, 0.013735578046180308, 0.013863389962352812, 0.013885256950743496, 0.013833746081218123, 0.013886073022149503, 0.013862609979696572, 0.01413318794220686, 0.013765671057626605, 0.013859207974746823, 0.01392802398186177, 0.013881091959774494, 0.013933715061284602, 0.013744628988206387, 0.013856832985766232, 0.01386306097265333, 0.013874617055989802, 0.013902380014769733, 0.01387616596184671, 0.013856256031431258, 0.013970886007882655, 0.013817609986290336, 0.013833979959599674, 0.013890531030483544, 0.013883386040106416, 0.013952804962173104, 0.013891677954234183, 0.013862087042070925, 0.01387402496766299, 0.013833528035320342, 0.013863997999578714, 0.013834920013323426, 0.01387990196235478, 0.013855691999197006, 0.01388482702895999, 0.01387158001307398, 0.013920666999183595, 0.013878282974474132, 0.013867928995750844, 0.013892243965528905, 0.013903610059060156, 0.013900642981752753, 0.013867081026546657, 0.013888748013414443, 0.013883769977837801, 0.013868951937183738, 0.013909827102907002, 0.013888097950257361, 0.01390758901834488, 0.013845456996932626, 0.01387387199793011, 0.013879903010092676, 0.013900165911763906, 0.013857341022230685, 0.013915276038460433, 0.01385237998329103, 0.013883662060834467, 0.013880426995456219, 0.014039431931450963, 0.013775681029073894, 0.013864275999367237, 0.013983938959427178, 0.013862351072020829, 0.013804469956085086, 0.013831280055455863, 0.013857043930329382, 0.01388087507802993, 0.013852102914825082, 0.01390258502215147, 0.01388066599611193, 0.013863099040463567, 0.01384151796810329, 0.013892209040932357, 0.014022610965184867, 0.013798248954117298, 0.013879170990549028, 0.013948220061138272, 0.013862933963537216, 0.013850750052370131, 0.013869342976249754, 0.013882884988561273, 0.013897145981900394, 0.01416087697725743, 0.013723228010348976, 0.013917692005634308, 0.013828841038048267, 0.013864960055798292, 0.013853473938070238, 0.013904916006140411, 0.013835435034707189, 0.013951985980384052, 0.014033874031156301], "tpot_s": 0.01368020858608065} -{"phase": "request", "index": 18, "case": "retrieval-n65536-s8", "score": {"shape_valid": true, "position_matches": [true, true, true, true], "complete": true}, "generated_text": "{\n\"KEY_FE3D0A329FBA\": \"VAL_12CF84845CD1\",\n\"KEY_8CC64479CDE7\": \"VAL_1E65FA0E02C8\",\n\"KEY_FB9F29729FA0\": \"VAL_33EBEE672177\",\n\"KEY_EB4F30C29F54\": \"VAL_4A3D76474E4E\"\n}", "input_tokens": 65536, "requested_output_tokens": 2048, "output_tokens": 122, "token_ids": [90, 198, 1, 4631, 41877, 18, 35, 15, 32, 18, 17, 24, 37, 6844, 763, 328, 5419, 62, 16, 17, 9362, 23, 19, 23, 19, 20, 6281, 16, 487, 198, 1, 4631, 62, 23, 3580, 21, 19, 19, 22, 24, 34, 1113, 22, 763, 328, 5419, 62, 16, 36, 21, 20, 3505, 15, 36, 15, 17, 34, 23, 487, 198, 1, 4631, 62519, 24, 37, 17, 24, 22, 17, 24, 3505, 15, 763, 328, 5419, 62, 18, 18, 8006, 6878, 21, 22, 17, 16, 22, 22, 487, 198, 1, 4631, 2021, 33, 19, 37, 18, 15, 34, 17, 24, 37, 20, 19, 763, 328, 5419, 62, 19, 32, 18, 35, 22, 21, 19, 22, 19, 36, 19, 36, 1, 198, 92, 248046], "terminal": ["stop"], "usage": [{"prompt_tokens": 65536, "total_tokens": 65658, "completion_tokens": 122}], "valid": true, "token_counts_per_choice": [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], "done_unique_last": true, "client_success": true, "client_error": false, "wire_sha256": "738f38a292a7174e07afebbd3e1fd9ba250ee742d4ac7e2d3cd2bdb9ccff43c3", "prompt_sha256": "e349ba8052cedbd8feebe1a3045a712cc0fb98b19761a3027c836a03c8e5d36a", "ttft_s": 18.60608541604597, "latency_s": 20.25684067606926, "itl": [0.00036289694253355265, 5.345093086361885e-06, 0.012023492949083447, 0.013830782962031662, 0.013943042024038732, 0.013859911006875336, 0.013965941034257412, 0.013765780022367835, 0.013920304947532713, 0.013788404990918934, 0.013859480968676507, 0.013891502982005477, 0.01394220907241106, 0.01388722495175898, 0.01378908904734999, 0.01383597997482866, 0.013874944997951388, 0.01394746801815927, 0.013826187001541257, 0.014060631045140326, 0.013928093947470188, 0.013900603051297367, 0.01372678391635418, 0.013863843050785363, 0.01391978794708848, 0.013865841086953878, 0.013889255002140999, 0.013895250973291695, 0.013905821950174868, 0.013870535069145262, 0.013907651999033988, 0.013879488920792937, 0.014005608041770756, 0.013794707017950714, 0.013940397999249399, 0.013865072978660464, 0.01401977997738868, 0.01383408298715949, 0.013842725078575313, 0.013843260938301682, 0.013852389995008707, 0.013852413976565003, 0.013879163074307144, 0.013844413915649056, 0.013877809047698975, 0.013898132019676268, 0.013868120964616537, 0.013883684994652867, 0.013855846016667783, 0.014007486053742468, 0.013754211948253214, 0.013861997984349728, 0.013916705036535859, 0.013896442949771881, 0.013890559086576104, 0.013889545924030244, 0.013950710068456829, 0.013833449920639396, 0.013915974064730108, 0.013876682962290943, 0.01389205502346158, 0.013915795949287713, 0.01389375003054738, 0.013897172990255058, 0.013932525063864887, 0.013881159946322441, 0.013849511044099927, 0.014082985930144787, 0.013907602988183498, 0.013836788013577461, 0.013943618978373706, 0.013740537106059492, 0.013876432902179658, 0.01383310800883919, 0.013881977065466344, 0.013953162007965147, 0.013880947954021394, 0.013816684018820524, 0.013921395991928875, 0.013881891965866089, 0.013920021010562778, 0.013860746985301375, 0.014029002049937844, 0.013719239039346576, 0.013823830988258123, 0.013854743912816048, 0.013853291049599648, 0.013893287046812475, 0.01409796392545104, 0.01377521501854062, 0.01390978298150003, 0.01386579405516386, 0.0139014539308846, 0.013862443040125072, 0.013875962002202868, 0.013937473995611072, 0.013840834959410131, 0.013904157094657421, 0.013943452970124781, 0.01387056102976203, 0.013886253000237048, 0.013862927909940481, 0.013882445986382663, 0.013872975017875433, 0.01389160705730319, 0.013864599983207881, 0.013921809964813292, 0.014027766999788582, 0.013744559022597969, 0.013874043012037873, 0.013872712035663426, 0.013869104906916618, 0.013853308046236634, 0.013904129969887435, 0.013862255029380322, 0.013854773016646504, 0.013885234016925097, 0.013878608937375247, 0.013892651069909334, 0.013866723980754614, 0.013943459023721516], "tpot_s": 0.013642605454737921} -{"phase": "request", "index": 19, "case": "retrieval-n65536-s9", "score": {"shape_valid": true, "position_matches": [true, true, true, true], "complete": true}, "generated_text": "{\n \"KEY_033985BF7C13\": \"VAL_ACDDDD7004B6\",\n \"KEY_AD798726BF07\": \"VAL_9F1D28803ABD\",\n \"KEY_CA7212A0DA15\": \"VAL_C71860C5B090\",\n \"KEY_90FCD15E8307\": \"VAL_D81DD45CCEAE\"\n}", "input_tokens": 65536, "requested_output_tokens": 2048, "output_tokens": 122, "token_ids": [90, 198, 220, 328, 4631, 62, 15, 18, 18, 24, 23, 20, 19278, 22, 34, 16, 18, 763, 328, 5419, 15045, 3967, 3967, 22, 15, 15, 19, 33, 21, 487, 198, 220, 328, 4631, 21811, 22, 24, 23, 22, 17, 21, 19278, 15, 22, 763, 328, 5419, 62, 24, 37, 16, 35, 17, 23, 23, 15, 18, 1803, 35, 487, 198, 220, 328, 4631, 53045, 22, 17, 16, 17, 32, 15, 6151, 16, 20, 763, 328, 5419, 892, 22, 16, 23, 21, 15, 34, 20, 33, 15, 24, 15, 487, 198, 220, 328, 4631, 62, 24, 15, 37, 6281, 16, 20, 36, 23, 18, 15, 22, 763, 328, 5419, 1502, 23, 16, 3967, 19, 20, 34, 2036, 13276, 1, 198, 92, 248046], "terminal": ["stop"], "usage": [{"prompt_tokens": 65536, "total_tokens": 65658, "completion_tokens": 122}], "valid": true, "token_counts_per_choice": [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], "done_unique_last": true, "client_success": true, "client_error": false, "wire_sha256": "0054e9fe347477c996d718ae633ce8985d183797fa203ab6d2031e29a0416e88", "prompt_sha256": "9c79220b1e295e03a3ea11f5ffe443de49b18b104d3672dd832c6346e6338256", "ttft_s": 18.60651378298644, "latency_s": 20.26053787698038, "itl": [0.00032656406983733177, 0.0015168929239735007, 0.013791740988381207, 0.013937324983999133, 0.013851994066499174, 0.013907001935876906, 0.013882922008633614, 0.013834746088832617, 0.013890383997932076, 0.01389796391595155, 0.013914720038883388, 0.0139035809552297, 0.013923896010965109, 0.013826939975842834, 0.013941291021183133, 0.013877323013730347, 0.013934095040895045, 0.013910111971199512, 0.01384619704913348, 0.01381044799927622, 0.013891779002733529, 0.013852668926119804, 0.013860171078704298, 0.013898356002755463, 0.01387175393756479, 0.013966100988909602, 0.013809044030494988, 0.013881349936127663, 0.013873218093067408, 0.013897040975280106, 0.013890928938053548, 0.013818636070936918, 0.013853952987119555, 0.013898727018386126, 0.013886751024983823, 0.013840573956258595, 0.013864818960428238, 0.014002726064063609, 0.013836897909641266, 0.013830209034495056, 0.013884167070500553, 0.013902668957598507, 0.014025236945599318, 0.013787390082143247, 0.01389666402246803, 0.01387855492066592, 0.013868657988496125, 0.014032634091563523, 0.013869015965610743, 0.01398897694889456, 0.013737734057940543, 0.013824492925778031, 0.013890849077142775, 0.013862524996511638, 0.013901102938689291, 0.013968840008601546, 0.013893979019485414, 0.013867645058780909, 0.013827012968249619, 0.013889461988583207, 0.01392518193461001, 0.01388636406045407, 0.013851654948666692, 0.013865847024135292, 0.013841481995768845, 0.014089509029872715, 0.013758040964603424, 0.013814817066304386, 0.013884298969060183, 0.013892777031287551, 0.013930337969213724, 0.01405984302982688, 0.013861737912520766, 0.013817355036735535, 0.01382076705340296, 0.013903575949370861, 0.013836732017807662, 0.013837152975611389, 0.013901039026677608, 0.01392710697837174, 0.013851794064976275, 0.01386596797965467, 0.013886878965422511, 0.013881130958907306, 0.013957902090623975, 0.013855241937562823, 0.013882186962291598, 0.014166154083795846, 0.013669903972186148, 0.013994910987094045, 0.013810766045935452, 0.01385737198870629, 0.013813611003570259, 0.013945912010967731, 0.013812551973387599, 0.01386533200275153, 0.013844616012647748, 0.013910260982811451, 0.013985243975184858, 0.013776903040707111, 0.013856997014954686, 0.013865824905224144, 0.013876717071980238, 0.013857641024515033, 0.013892306946218014, 0.013990730047225952, 0.01384183601476252, 0.013867347966879606, 0.013913615955971181, 0.013928660075180233, 0.013850719900801778, 0.01386097609065473, 0.013877327903173864, 0.013862099032849073, 0.013912740047089756, 0.013903404003940523, 0.013867111993022263, 0.013855653000064194, 0.013883337960578501, 0.01386204396840185, 0.01400728803128004], "tpot_s": 0.013669620611520161} -{"phase": "request", "index": 20, "case": "retrieval-n131072-s0", "score": {"shape_valid": true, "position_matches": [true, true, true, true], "complete": true}, "generated_text": "{\n\"KEY_1D7DE61CB245\": \"VAL_15E5B80A5BB9\",\n\"KEY_3376BD5C4999\": \"VAL_6058CFFAB853\",\n\"KEY_FB52141FEAA6\": \"VAL_A378E0DEB51A\",\n\"KEY_3B931512F097\": \"VAL_6B7264F3B5F9\"\n}", "input_tokens": 131072, "requested_output_tokens": 2048, "output_tokens": 124, "token_ids": [90, 198, 1, 4631, 62, 16, 35, 22, 1113, 21, 16, 12607, 17, 19, 20, 763, 328, 5419, 62, 16, 20, 36, 20, 33, 23, 15, 32, 20, 9798, 24, 487, 198, 1, 4631, 62, 18, 18, 22, 21, 9265, 20, 34, 19, 24, 24, 24, 763, 328, 5419, 62, 21, 15, 20, 23, 34, 1686, 1803, 23, 20, 18, 487, 198, 1, 4631, 62519, 20, 17, 16, 19, 16, 11091, 5840, 21, 763, 328, 5419, 1511, 18, 22, 23, 36, 15, 1113, 33, 20, 16, 32, 487, 198, 1, 4631, 62, 18, 33, 24, 18, 16, 20, 16, 17, 37, 15, 24, 22, 763, 328, 5419, 62, 21, 33, 22, 17, 21, 19, 37, 18, 33, 20, 37, 24, 1, 198, 92, 248046], "terminal": ["stop"], "usage": [{"prompt_tokens": 131072, "total_tokens": 131196, "completion_tokens": 124}], "valid": true, "token_counts_per_choice": [1, 2, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], "done_unique_last": true, "client_success": true, "client_error": false, "wire_sha256": "9078f34f54154e7cd18eae0ff172df68ad704feaa41110e3234d182232a1aadd", "prompt_sha256": "b6692faefa0614a176face1bc026d52a96ad7524cb9ed685f3018bc19c561fd3", "ttft_s": 49.15679336001631, "latency_s": 51.33746368403081, "itl": [0.00050704600289464, 0.006516280001960695, 0.01789011002983898, 0.018079578992910683, 0.01826370193157345, 0.017994801979511976, 0.017981872078962624, 0.01804001396521926, 0.018088088021613657, 0.018129496020264924, 0.018114696023985744, 0.01802288694307208, 0.018080833018757403, 0.018115508020855486, 0.01811717892996967, 0.01817627204582095, 0.018047209945507348, 0.018082757014781237, 0.018134891986846924, 0.018126690993085504, 0.01809593301732093, 0.018126768060028553, 0.018195345997810364, 0.01807325694244355, 0.01802462199702859, 0.018175504985265434, 0.018008276005275548, 0.018094005063176155, 0.018156355945393443, 0.018133220029994845, 0.018158445018343627, 0.01822013803757727, 0.018104255897924304, 0.018092273036018014, 0.018108322052285075, 0.01806980895344168, 0.018068235949613154, 0.018250292050652206, 0.01799463201314211, 0.01812612998764962, 0.01815999497193843, 0.018147199065424502, 0.01816841692198068, 0.018115970073267817, 0.01808906195219606, 0.01807824498973787, 0.018072791048325598, 0.01808915298897773, 0.01807459199335426, 0.018116772989742458, 0.018201095052063465, 0.018156134989112616, 0.018140709027647972, 0.018261506920680404, 0.018075590021908283, 0.018104871967807412, 0.01806473301257938, 0.018092704005539417, 0.0181244210107252, 0.018062088987790048, 0.018179835053160787, 0.018075937987305224, 0.0181990850251168, 0.018065901938825846, 0.018153388053178787, 0.018359657959081233, 0.01794377702753991, 0.01804936199914664, 0.018145209993235767, 0.018017148016951978, 0.018076543929055333, 0.0180731340078637, 0.018178857979364693, 0.018195271026343107, 0.01810830505564809, 0.018157552927732468, 0.018142599030397832, 0.018099096952937543, 0.01810275809839368, 0.018140900996513665, 0.01821042795199901, 0.017959055956453085, 0.018044208991341293, 0.018134961021132767, 0.018154671997763216, 0.01813377602957189, 0.01813548302743584, 0.018131247023120522, 0.018116285908035934, 0.018096173997037113, 0.018086733063682914, 0.018059839960187674, 0.018120076972991228, 0.018087794072926044, 0.018164700944907963, 0.018119959044270217, 0.018159747938625515, 0.018101667053997517, 0.018172390991821885, 0.018158965976908803, 0.01808238006196916, 0.018148130970075727, 0.01831354305613786, 0.017938069999217987, 0.017919295001775026, 0.018115335959009826, 0.01822405296843499, 0.018065207987092435, 0.018224845989607275, 0.018120602006092668, 0.018077008076943457, 0.018071269965730608, 0.018085593008436263, 0.01811606192495674, 0.018152910051867366, 0.018026871024630964, 0.018278779927641153, 0.018047091085463762, 0.018187933950684965, 0.01816217298619449, 0.018195925978943706, 0.018122964072972536], "tpot_s": 0.01772902702450813} -{"phase": "request", "index": 21, "case": "retrieval-n131072-s1", "score": {"shape_valid": true, "position_matches": [true, true, true, true], "complete": true}, "generated_text": "{\n\"KEY_BA5A6B7C3728\": \"VAL_37FF2EB72750\",\n\"KEY_62C6A258F632\": \"VAL_4DC8403FE68F\",\n\"KEY_07C7FF8DBC99\": \"VAL_9F6931691352\",\n\"KEY_6D13C79B4107\": \"VAL_4BAB1C50E88F\"\n}", "input_tokens": 131072, "requested_output_tokens": 2048, "output_tokens": 126, "token_ids": [90, 198, 1, 4631, 1609, 32, 20, 32, 21, 33, 22, 34, 18, 22, 17, 23, 763, 328, 5419, 62, 18, 22, 1686, 17, 8006, 22, 17, 22, 20, 15, 487, 198, 1, 4631, 62, 21, 17, 34, 21, 32, 17, 20, 23, 37, 21, 18, 17, 763, 328, 5419, 62, 19, 5449, 23, 19, 15, 18, 11091, 21, 23, 37, 487, 198, 1, 4631, 62, 15, 22, 34, 22, 1686, 23, 43548, 24, 24, 763, 328, 5419, 62, 24, 37, 21, 24, 18, 16, 21, 24, 16, 18, 20, 17, 487, 198, 1, 4631, 62, 21, 35, 16, 18, 34, 22, 24, 33, 19, 16, 15, 22, 763, 328, 5419, 62, 19, 92771, 16, 34, 20, 15, 36, 23, 23, 37, 1, 198, 92, 248046], "terminal": ["stop"], "usage": [{"prompt_tokens": 131072, "total_tokens": 131198, "completion_tokens": 126}], "valid": true, "token_counts_per_choice": [1, 2, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], "done_unique_last": true, "client_success": true, "client_error": false, "wire_sha256": "0386d7f5cfac2f2599cb7c37d644dedda1af9dfc2f6ee1436ac3e8e5fd2538b5", "prompt_sha256": "13cb483781aaef368d7b8c077a4fbbb8e685c6f74f09a239ddb3582fd051a8ce", "ttft_s": 49.14600812399294, "latency_s": 51.36137570999563, "itl": [0.0005061900010332465, 0.0055315259378403425, 0.017908338108099997, 0.01820027595385909, 0.018071277998387814, 0.01813330000732094, 0.01794784003868699, 0.018067869008518755, 0.018019996932707727, 0.01802566705737263, 0.0180398509837687, 0.018076580949127674, 0.018062578048557043, 0.018114393926225603, 0.01812233601231128, 0.018113458063453436, 0.018107187002897263, 0.018114288919605315, 0.018136086990125477, 0.018077514017932117, 0.01808593701571226, 0.018151614000089467, 0.01801287103444338, 0.018120032036677003, 0.018103879992850125, 0.018179983948357403, 0.0181686399737373, 0.018157379003241658, 0.018205319996923208, 0.01807767804712057, 0.018082635942846537, 0.018106834031641483, 0.018090241006575525, 0.01811404398176819, 0.018120686057955027, 0.01805853098630905, 0.01807878795079887, 0.018057818058878183, 0.018107641953974962, 0.018110736971721053, 0.018023494048975408, 0.018109300988726318, 0.018114170990884304, 0.018104289076291025, 0.01818689692299813, 0.018066021031700075, 0.018039962043985724, 0.018128089956007898, 0.018061597947962582, 0.018060727044939995, 0.0181272909976542, 0.018026103964075446, 0.018176959012635052, 0.018124392023310065, 0.01814991794526577, 0.018145793001167476, 0.01816010707989335, 0.01810820202808827, 0.018138849991373718, 0.018108056974597275, 0.018139421939849854, 0.018147119088098407, 0.01809984096325934, 0.018059651949442923, 0.0181440319865942, 0.018160770065151155, 0.01811384898610413, 0.018154336023144424, 0.018166834022849798, 0.01799119298812002, 0.01810094795655459, 0.01819099998101592, 0.018056721077300608, 0.018051687977276742, 0.018126523005776107, 0.018136492930352688, 0.018179574981331825, 0.018188765039667487, 0.01815664500463754, 0.018078987952321768, 0.01808054803404957, 0.018059310968965292, 0.01809747703373432, 0.018171261064708233, 0.01809938799124211, 0.01806004100944847, 0.018135129008442163, 0.01814719696994871, 0.018135014921426773, 0.01815424009691924, 0.018145008943974972, 0.018076064996421337, 0.018087042961269617, 0.018084669020026922, 0.018243614002130926, 0.018035231041722, 0.018157785991206765, 0.018163952976465225, 0.018144519068300724, 0.0181009880034253, 0.018259272910654545, 0.01813228998798877, 0.018041653092950583, 0.0180795940104872, 0.018082614988088608, 0.018062195973470807, 0.01805566204711795, 0.018212603987194598, 0.018116827006451786, 0.018161683925427496, 0.01816022302955389, 0.018116574035957456, 0.01815702300518751, 0.018044191994704306, 0.018276115995831788, 0.018109682947397232, 0.018106047995388508, 0.01808200404047966, 0.017974499030970037, 0.018064991920255125, 0.01806918904185295, 0.01810655801091343, 0.018075426924042404, 0.018269711057655513], "tpot_s": 0.01772294068802148} -{"phase": "request", "index": 22, "case": "retrieval-n131072-s2", "score": {"shape_valid": true, "position_matches": [true, true, true, true], "complete": true}, "generated_text": "{\n\"KEY_B0E1F87FB6D0\": \"VAL_7CAF6CF37C7F\",\n\"KEY_44A1F0EABBD6\": \"VAL_D392021D4836\",\n\"KEY_2B5041437D6B\": \"VAL_5767CA6BC6B2\",\n\"KEY_8F0CFD23CB09\": \"VAL_B372C4F1AEF7\"\n}", "input_tokens": 131072, "requested_output_tokens": 2048, "output_tokens": 122, "token_ids": [90, 198, 1, 4631, 1609, 15, 36, 16, 37, 23, 22, 15731, 21, 35, 15, 763, 328, 5419, 62, 22, 78515, 21, 9362, 18, 22, 34, 22, 37, 487, 198, 1, 4631, 62, 19, 19, 32, 16, 37, 15, 36, 1803, 9265, 21, 763, 328, 5419, 1502, 18, 24, 17, 15, 17, 16, 35, 19, 23, 18, 21, 487, 198, 1, 4631, 62, 17, 33, 20, 15, 19, 16, 19, 18, 22, 35, 21, 33, 763, 328, 5419, 62, 20, 22, 21, 22, 4887, 21, 4738, 21, 33, 17, 487, 198, 1, 4631, 62, 23, 37, 15, 9362, 35, 17, 18, 12607, 15, 24, 763, 328, 5419, 1609, 18, 22, 17, 34, 19, 37, 16, 13276, 37, 22, 1, 198, 92, 248046], "terminal": ["stop"], "usage": [{"prompt_tokens": 131072, "total_tokens": 131194, "completion_tokens": 122}], "valid": true, "token_counts_per_choice": [1, 2, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], "done_unique_last": true, "client_success": true, "client_error": false, "wire_sha256": "bca8cd633c56bd1245367f13ff646ce62eb7899d82b34104d955f16988f04155", "prompt_sha256": "6f33a95979e5e3379187e29505e2b72c7aba7c81e477c4607303677e8495a415", "ttft_s": 49.159475074964575, "latency_s": 51.301845506066456, "itl": [0.0005243730265647173, 0.005656808032654226, 0.017909554997459054, 0.018068158999085426, 0.01809842698276043, 0.018076829961501062, 0.018147959024645388, 0.01802563399542123, 0.018106406088918447, 0.018054111977107823, 0.01808475994039327, 0.01813839003443718, 0.018051630002446473, 0.01815840497147292, 0.01812300202436745, 0.018153136945329607, 0.018229839042760432, 0.018097024061717093, 0.018306347890757024, 0.0180158520815894, 0.017970711924135685, 0.01814324001315981, 0.018016027053818107, 0.018069330020807683, 0.01809625595342368, 0.01811701199039817, 0.0180801700334996, 0.018072327016852796, 0.018093845923431218, 0.018087162054143846, 0.018024818040430546, 0.018064693897031248, 0.018046331009827554, 0.018071984057314694, 0.018059710040688515, 0.01809386699460447, 0.018142183893360198, 0.018100084038451314, 0.01814174698665738, 0.018091830075718462, 0.018079296918585896, 0.01808470208197832, 0.018093774910084903, 0.018080217065289617, 0.01808707299642265, 0.018082128022797406, 0.0181411299854517, 0.018155243014916778, 0.018151932978071272, 0.018191667972132564, 0.018102362053468823, 0.01811859291046858, 0.018115739105269313, 0.018030439969152212, 0.01805899292230606, 0.01808753504883498, 0.018096107058227062, 0.018121022963896394, 0.018242013989947736, 0.018147296039387584, 0.01818878692574799, 0.018276253016665578, 0.017919962061569095, 0.01806246698834002, 0.018074248917400837, 0.018075142987072468, 0.018197240075096488, 0.018098096014000475, 0.018081130925565958, 0.018135869060643017, 0.01811154594179243, 0.01837475399952382, 0.017999163013882935, 0.018078165012411773, 0.01822191395331174, 0.018007734091952443, 0.018066224991343915, 0.018143185996450484, 0.018028997932560742, 0.01809345104265958, 0.01817290298640728, 0.018125785049051046, 0.018203429994173348, 0.018117752973921597, 0.018077522981911898, 0.018110556062310934, 0.018033979926258326, 0.018079619971103966, 0.018078146036714315, 0.018064187024720013, 0.018157848971895874, 0.01848744007293135, 0.01784931996371597, 0.018142428947612643, 0.0181272680638358, 0.018075904925353825, 0.01805523701477796, 0.01807710900902748, 0.01808569405693561, 0.0180939199635759, 0.018101906054653227, 0.01807514892425388, 0.01809740497265011, 0.01808985508978367, 0.018064185976982117, 0.018137302016839385, 0.018077711923979223, 0.018066103104501963, 0.018077172921039164, 0.018127231975086033, 0.01810891309287399, 0.018113958998583257, 0.0181049439124763, 0.01810853509232402, 0.01809276093263179, 0.018157007987610996, 0.018031208077445626, 0.018105325987562537, 0.0180863639106974, 0.018219754099845886], "tpot_s": 0.017705540752908106} -{"phase": "request", "index": 23, "case": "retrieval-n131072-s3", "score": {"shape_valid": true, "position_matches": [true, true, true, true], "complete": true}, "generated_text": "{\n\"KEY_11ECFE992D7D\": \"VAL_67B0199E1814\",\n\"KEY_46B8F2CAE751\": \"VAL_178F5396AD53\",\n\"KEY_401CFE21BBD2\": \"VAL_2689382834E7\",\n\"KEY_CBFC39ABE091\": \"VAL_37B51CFC6EB9\"\n}", "input_tokens": 131072, "requested_output_tokens": 2048, "output_tokens": 124, "token_ids": [90, 198, 1, 4631, 62, 16, 16, 7263, 11091, 24, 24, 17, 35, 22, 35, 763, 328, 5419, 62, 21, 22, 33, 15, 16, 24, 24, 36, 16, 23, 16, 19, 487, 198, 1, 4631, 62, 19, 21, 33, 23, 37, 17, 4887, 36, 22, 20, 16, 763, 328, 5419, 62, 16, 22, 23, 37, 20, 18, 24, 21, 1745, 20, 18, 487, 198, 1, 4631, 62, 19, 15, 16, 9362, 36, 17, 16, 33, 9265, 17, 763, 328, 5419, 62, 17, 21, 23, 24, 18, 23, 17, 23, 18, 19, 36, 22, 487, 198, 1, 4631, 47294, 6542, 18, 24, 1803, 36, 15, 24, 16, 763, 328, 5419, 62, 18, 22, 33, 20, 16, 34, 6542, 21, 8006, 24, 1, 198, 92, 248046], "terminal": ["stop"], "usage": [{"prompt_tokens": 131072, "total_tokens": 131196, "completion_tokens": 124}], "valid": true, "token_counts_per_choice": [1, 2, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], "done_unique_last": true, "client_success": true, "client_error": false, "wire_sha256": "3ad4b6268d8b6ddfed2159c13814c2bde874b7da14284ef6ce03ae80e8bb0073", "prompt_sha256": "ff945bc7b9eda5b1f1859db1ba1ec49b6a5e937aea6ec7901af7fb1f9527864a", "ttft_s": 49.17208435398061, "latency_s": 51.344852722017094, "itl": [0.0005440739914774895, 0.00033409101888537407, 0.01780669402796775, 0.018183549982495606, 0.018015805049799383, 0.018100461922585964, 0.018341655028052628, 0.017844639951363206, 0.01826411101501435, 0.018061871989630163, 0.01808740699198097, 0.018110595061443746, 0.01808599999640137, 0.018144248984754086, 0.018155887024477124, 0.018132188939489424, 0.018074127030558884, 0.018101350986398757, 0.018083631061017513, 0.018105212948285043, 0.018088344018906355, 0.018066648044623435, 0.018063040915876627, 0.018055611988529563, 0.01805441698525101, 0.018092941027134657, 0.018058718997053802, 0.018107895040884614, 0.018088110024109483, 0.0181025950005278, 0.01811092102434486, 0.01811443199403584, 0.018065954907797277, 0.01808831002563238, 0.01804716303013265, 0.01806378399487585, 0.018120534950867295, 0.01806332205887884, 0.01813958294223994, 0.018162553082220256, 0.01815679995343089, 0.01814129203557968, 0.018141979933716357, 0.018105194089002907, 0.01806475396733731, 0.018084042938426137, 0.018115729093551636, 0.01809795794542879, 0.018131489981897175, 0.018011220963671803, 0.018091815058141947, 0.01804354996420443, 0.018076458014547825, 0.01803448307327926, 0.018068785895593464, 0.018083692993968725, 0.01810725301038474, 0.018091964069753885, 0.018123777932487428, 0.018097924068570137, 0.018085347022861242, 0.018103873007930815, 0.018042361945845187, 0.018142670043744147, 0.01810857397504151, 0.018065460957586765, 0.018105230061337352, 0.018148353905417025, 0.01814211707096547, 0.018138348939828575, 0.018170612049289048, 0.01812422799412161, 0.01806687598582357, 0.018099455046467483, 0.01805685891304165, 0.018091346020810306, 0.01811360102146864, 0.01817222498357296, 0.01814343500882387, 0.018127770046703517, 0.018130052951164544, 0.018123234040103853, 0.01810706197284162, 0.01807453203946352, 0.018045268952846527, 0.018069982063025236, 0.018108729971572757, 0.018101210007444024, 0.018160121981054544, 0.01812908099964261, 0.0181357329711318, 0.018144623027183115, 0.01814242801629007, 0.01811443897895515, 0.018064907984808087, 0.01812483905814588, 0.018103665905073285, 0.018072731094434857, 0.018044324009679258, 0.018156125908717513, 0.018134057987481356, 0.018125976086594164, 0.018183836014941335, 0.01812934991903603, 0.01810947002377361, 0.01808525004889816, 0.018087230972014368, 0.018100901041179895, 0.018091554986312985, 0.018045175937004387, 0.01803961698897183, 0.018110409029759467, 0.01806514896452427, 0.018111671088263392, 0.01811728999018669, 0.018052898929454386, 0.01808282604906708, 0.01804771099705249, 0.018049256992526352, 0.018193696974776685, 0.018013995024375618, 0.018186560017056763], "tpot_s": 0.017664783479971434} -{"phase": "request", "index": 24, "case": "retrieval-n131072-s4", "score": {"shape_valid": true, "position_matches": [true, true, true, true], "complete": true}, "generated_text": "{\n\"KEY_FA7DF2E2511E\": \"VAL_D0A643F8B6F5\",\n\"KEY_B171AACE1ED2\": \"VAL_6A6DCDE46388\",\n\"KEY_2AD587AE30D4\": \"VAL_8BD555A16B79\",\n\"KEY_5D6A51DBAC61\": \"VAL_2916C63FA7A0\"\n}", "input_tokens": 131072, "requested_output_tokens": 2048, "output_tokens": 120, "token_ids": [90, 198, 1, 4631, 18485, 22, 5095, 17, 36, 17, 20, 16, 16, 36, 763, 328, 5419, 1502, 15, 32, 21, 19, 18, 37, 23, 33, 21, 37, 20, 487, 198, 1, 4631, 1609, 16, 22, 16, 32, 5401, 16, 1425, 17, 763, 328, 5419, 62, 21, 32, 21, 5449, 1113, 19, 21, 18, 23, 23, 487, 198, 1, 4631, 62, 17, 1745, 20, 23, 22, 13276, 18, 15, 35, 19, 763, 328, 5419, 62, 23, 9265, 20, 20, 20, 32, 16, 21, 33, 22, 24, 487, 198, 1, 4631, 62, 20, 35, 21, 32, 20, 16, 3392, 1646, 21, 16, 763, 328, 5419, 62, 17, 24, 16, 21, 34, 21, 18, 3505, 22, 32, 15, 1, 198, 92, 248046], "terminal": ["stop"], "usage": [{"prompt_tokens": 131072, "total_tokens": 131192, "completion_tokens": 120}], "valid": true, "token_counts_per_choice": [1, 2, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], "done_unique_last": true, "client_success": true, "client_error": false, "wire_sha256": "623562d6ecbe2e16369af4bef8f0669c8eac9772ee0c7f77cdc6bbeb263a5cfa", "prompt_sha256": "c3b2aa980714d9b40734874ca907db8598316d8bf233e414f24c94e4c97cd428", "ttft_s": 49.17177932697814, "latency_s": 51.27785956102889, "itl": [0.0005137290572747588, 0.005972426966764033, 0.018060634029097855, 0.01806168502662331, 0.018030629958957434, 0.017925838008522987, 0.018102261004969478, 0.01803321798797697, 0.018075944972224534, 0.018086164025589824, 0.01804023899603635, 0.01807103503961116, 0.018089213990606368, 0.018052691011689603, 0.01804608595557511, 0.01808064605575055, 0.01804069895297289, 0.01811541395727545, 0.018358960980549455, 0.017996764043346047, 0.01805061602499336, 0.018091522972099483, 0.018029971979558468, 0.01807764999102801, 0.018082860042341053, 0.018094882951118052, 0.018142993096262217, 0.018075364991091192, 0.018119022017344832, 0.018112035002559423, 0.018149546929635108, 0.018214593990705907, 0.018239618046209216, 0.018019745941273868, 0.018019362003542483, 0.018068699981085956, 0.018337207031436265, 0.01797033299226314, 0.01802679302636534, 0.018117061001248658, 0.0181203109677881, 0.018131894059479237, 0.018150018993765116, 0.0181455499259755, 0.01829520706087351, 0.017911450937390327, 0.01805362303275615, 0.018063355004414916, 0.01809302601031959, 0.018084330018609762, 0.018094058032147586, 0.018182836938649416, 0.01810579199809581, 0.018127662013284862, 0.018151127034798265, 0.018127845018170774, 0.018063718918710947, 0.018110778066329658, 0.018050326965749264, 0.01807270897552371, 0.018071666010655463, 0.018147791968658566, 0.018151939031668007, 0.018130274023860693, 0.018239725031889975, 0.01822797697968781, 0.017988063977099955, 0.018058871966786683, 0.01806250400841236, 0.018055922002531588, 0.018069623038172722, 0.018089210032485425, 0.01815081899985671, 0.018130602897144854, 0.01811212406028062, 0.018153468961827457, 0.01815789402462542, 0.018119271961040795, 0.018114041071385145, 0.01807883393485099, 0.01805313304066658, 0.018073374056257308, 0.018099615001119673, 0.018091658945195377, 0.01807465497404337, 0.018067994969896972, 0.01807855407241732, 0.01806670695077628, 0.018066811026073992, 0.018224274972453713, 0.018044297001324594, 0.01813187508378178, 0.018067843979224563, 0.01821273798123002, 0.018033421016298234, 0.0180503879673779, 0.01814092497806996, 0.018040118040516973, 0.01822957501281053, 0.01824090292211622, 0.01794530905317515, 0.018007799051702023, 0.018097264925017953, 0.018010128056630492, 0.018054444924928248, 0.018108364078216255, 0.01817927695810795, 0.018113838974386454, 0.018127255025319755, 0.018260829034261405, 0.018065879005007446, 0.018153880955651402, 0.01803001796361059, 0.01829377398826182, 0.01796005805954337, 0.017978823045268655, 0.018156594946049154, 0.01820743002463132], "tpot_s": 0.017698153227317232} -{"phase": "request", "index": 25, "case": "retrieval-n131072-s5", "score": {"shape_valid": true, "position_matches": [true, true, true, true], "complete": true}, "generated_text": "{\n\"KEY_B6E9C81511D5\": \"VAL_B7CC31A093A4\",\n\"KEY_28DC460BC080\": \"VAL_3F9B063DB035\",\n\"KEY_009AB49B10D8\": \"VAL_FA9AAA83C638\",\n\"KEY_41259044966F\": \"VAL_5A7C37B697D1\"\n}", "input_tokens": 131072, "requested_output_tokens": 2048, "output_tokens": 125, "token_ids": [90, 198, 1, 4631, 1609, 21, 36, 24, 34, 23, 16, 20, 16, 16, 35, 20, 763, 328, 5419, 1609, 22, 3580, 18, 16, 32, 15, 24, 18, 32, 19, 487, 198, 1, 4631, 62, 17, 23, 5449, 19, 21, 15, 4738, 15, 23, 15, 763, 328, 5419, 62, 18, 37, 24, 33, 15, 21, 18, 3392, 15, 18, 20, 487, 198, 1, 4631, 62, 15, 15, 24, 1803, 19, 24, 33, 16, 15, 35, 23, 763, 328, 5419, 18485, 24, 48421, 23, 18, 34, 21, 18, 23, 487, 198, 1, 4631, 62, 19, 16, 17, 20, 24, 15, 19, 19, 24, 21, 21, 37, 763, 328, 5419, 62, 20, 32, 22, 34, 18, 22, 33, 21, 24, 22, 35, 16, 1, 198, 92, 248046], "terminal": ["stop"], "usage": [{"prompt_tokens": 131072, "total_tokens": 131197, "completion_tokens": 125}], "valid": true, "token_counts_per_choice": [1, 2, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], "done_unique_last": true, "client_success": true, "client_error": false, "wire_sha256": "d62f3e8b343d4055ecc97e29d2fce7c0723d6d30f29363146f74df3037746e14", "prompt_sha256": "4b961ab78b1a5c2746638247ababec4ece02541caacc90b52d0ac1bf6683b5d6", "ttft_s": 49.09869424905628, "latency_s": 51.29532700101845, "itl": [0.0005141280125826597, 0.006076160934753716, 0.01786729204468429, 0.018038382986560464, 0.01805364398751408, 0.018105463008396327, 0.01810095796827227, 0.01798286708071828, 0.01806789101101458, 0.018064294010400772, 0.01812177593819797, 0.018085210002027452, 0.018095046980306506, 0.018109394004568458, 0.018121615052223206, 0.018113548983819783, 0.018090751953423023, 0.018084043986164033, 0.018131337012164295, 0.01812082703690976, 0.01808607904240489, 0.018126411945559084, 0.018172616022638977, 0.0180981740122661, 0.01808410999365151, 0.018162859953008592, 0.018438489991240203, 0.01785883808042854, 0.018160856910981238, 0.018044798052869737, 0.017974804970435798, 0.018125778064131737, 0.018438495928421617, 0.01791368902195245, 0.018124107969924808, 0.017953403992578387, 0.018129031057469547, 0.018107088981196284, 0.01806304499041289, 0.018096875050105155, 0.018108766991645098, 0.01808827801141888, 0.018081776914186776, 0.018132192082703114, 0.018143152934499085, 0.01816348801366985, 0.018278234056197107, 0.018122553010471165, 0.017983150901272893, 0.018057623994536698, 0.01806591311469674, 0.01808693096973002, 0.018041927949525416, 0.018097040010616183, 0.01808648696169257, 0.01809680904261768, 0.01813937001861632, 0.018074115039780736, 0.01811653294134885, 0.018055305001325905, 0.018175226054154336, 0.018115420010872185, 0.01928696990944445, 0.016974845086224377, 0.01820004894398153, 0.017907700035721064, 0.01810925896279514, 0.018312950036488473, 0.018079977948218584, 0.0180223670322448, 0.018116749939508736, 0.01801676000468433, 0.018071819096803665, 0.018081092974171042, 0.018169260933063924, 0.018022419069893658, 0.018110952922143042, 0.018099187058396637, 0.01806456793565303, 0.01818176405504346, 0.018119717948138714, 0.018236944102682173, 0.017918467987328768, 0.018124911934137344, 0.018113760044798255, 0.018145228968933225, 0.01803944306448102, 0.01808398892171681, 0.018329633050598204, 0.017948481952771544, 0.01821643707808107, 0.018046741955913603, 0.018071688013151288, 0.01807420898694545, 0.01808557298500091, 0.018068577046506107, 0.018123721005395055, 0.01802336797118187, 0.0180995479458943, 0.018110849079675972, 0.018124800990335643, 0.01823471998795867, 0.018074096995405853, 0.01802204200066626, 0.018303407006897032, 0.017898901947773993, 0.01824255706742406, 0.01802992494776845, 0.01800133695360273, 0.018096330109983683, 0.01810527394991368, 0.018143872963264585, 0.01809631404466927, 0.018138817977160215, 0.018102637957781553, 0.018066320102661848, 0.01806958590168506, 0.018074529012665153, 0.018070454010739923, 0.01810297998599708, 0.018148575094528496, 0.018048680969513953, 0.01821212994400412], "tpot_s": 0.017714780257759435} -{"phase": "request", "index": 26, "case": "retrieval-n131072-s6", "score": {"shape_valid": true, "position_matches": [true, true, true, true], "complete": true}, "generated_text": "{\n \"KEY_2DEC79C0A476\": \"VAL_B8D0DA50F844\",\n \"KEY_4E6B89164AC9\": \"VAL_B31777B11B6C\",\n \"KEY_9424061BDD36\": \"VAL_778404A0DF8E\",\n \"KEY_AD95C05D361F\": \"VAL_DCE753B1D85B\"\n}", "input_tokens": 131072, "requested_output_tokens": 2048, "output_tokens": 128, "token_ids": [90, 198, 220, 328, 4631, 62, 17, 29723, 22, 24, 34, 15, 32, 19, 22, 21, 763, 328, 5419, 1609, 23, 35, 15, 6151, 20, 15, 37, 23, 19, 19, 487, 198, 220, 328, 4631, 62, 19, 36, 21, 33, 23, 24, 16, 21, 19, 1646, 24, 763, 328, 5419, 1609, 18, 16, 22, 22, 22, 33, 16, 16, 33, 21, 34, 487, 198, 220, 328, 4631, 62, 24, 19, 17, 19, 15, 21, 16, 33, 3967, 18, 21, 763, 328, 5419, 62, 22, 22, 23, 19, 15, 19, 32, 15, 5095, 23, 36, 487, 198, 220, 328, 4631, 21811, 24, 20, 34, 15, 20, 35, 18, 21, 16, 37, 763, 328, 5419, 1502, 2036, 22, 20, 18, 33, 16, 35, 23, 20, 33, 1, 198, 92, 248046], "terminal": ["stop"], "usage": [{"prompt_tokens": 131072, "total_tokens": 131200, "completion_tokens": 128}], "valid": true, "token_counts_per_choice": [1, 2, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], "done_unique_last": true, "client_success": true, "client_error": false, "wire_sha256": "9e05f3980d11e8d86b8095e968860b1c169de37a3d3541973ac7daf6b2199286", "prompt_sha256": "34e9d7f506bb331d9c49af8dae4f02468a41bf63de6a0475c478a47475d63e2d", "ttft_s": 49.1656155429082, "latency_s": 51.41327736398671, "itl": [0.0005376969929784536, 0.0003698921063914895, 0.01799020590260625, 0.018007720005698502, 0.01806402998045087, 0.018113607075065374, 0.01820043392945081, 0.01808374107349664, 0.018077702959999442, 0.018091748002916574, 0.018140380969271064, 0.018094252096489072, 0.0182112290058285, 0.018021422903984785, 0.01820705598220229, 0.018060674075968564, 0.018135474994778633, 0.018151914002373815, 0.01813922100700438, 0.018135569989681244, 0.018182824016548693, 0.018081713002175093, 0.018130006967112422, 0.018204019987024367, 0.01796291396021843, 0.018096005078405142, 0.018124748021364212, 0.018148569972254336, 0.018090497003868222, 0.018128750030882657, 0.018119554966688156, 0.01812393101863563, 0.018170733936131, 0.0180637160083279, 0.018136320053599775, 0.018065450945869088, 0.01815338002052158, 0.018134391983039677, 0.018139941967092454, 0.018169261049479246, 0.018169188988395035, 0.018143855035305023, 0.018139987951144576, 0.018103617010638118, 0.01809442404191941, 0.018087573000229895, 0.01815246802289039, 0.01805148390121758, 0.01809114799834788, 0.018111490993760526, 0.01811683503910899, 0.018145369016565382, 0.018065050011500716, 0.018115911982022226, 0.01812839601188898, 0.018149481969885528, 0.01810879004187882, 0.0180986609775573, 0.0181373170344159, 0.018124850932508707, 0.01817862200550735, 0.018143179011531174, 0.018255179980769753, 0.018089916091412306, 0.018161245970986784, 0.018098396947607398, 0.018209076020866632, 0.017994533060118556, 0.018139722989872098, 0.018007864942774177, 0.018094136961735785, 0.018111678073182702, 0.018098989967256784, 0.01811500603798777, 0.018077087006531656, 0.018122570938430727, 0.01812374102883041, 0.018172056996263564, 0.018053777050226927, 0.018131763907149434, 0.018123760004527867, 0.018148652045056224, 0.018179174046963453, 0.01810382097028196, 0.018173284945078194, 0.018160840030759573, 0.01816927397157997, 0.018106817035004497, 0.018092427984811366, 0.018115270067937672, 0.018102556001394987, 0.018086936906911433, 0.018094453029334545, 0.018132862052880228, 0.018102409900166094, 0.018126354087144136, 0.018140270956791937, 0.01812108897138387, 0.018078222987242043, 0.01815802010241896, 0.018062399001792073, 0.018121986999176443, 0.01815524895209819, 0.018084843992255628, 0.018149410956539214, 0.018166891066357493, 0.01815599703695625, 0.01822179788723588, 0.018162086023949087, 0.01808397308923304, 0.018082624999806285, 0.018107743002474308, 0.018149131909012794, 0.01802334899548441, 0.01818633102811873, 0.018029776983894408, 0.0181498039746657, 0.01825753308366984, 0.01799353095702827, 0.01813137298449874, 0.018068999983370304, 0.018072761013172567, 0.01814087003003806, 0.018117118976078928, 0.018090698053129017, 0.01820389600470662], "tpot_s": 0.01769812457542133} -{"phase": "request", "index": 27, "case": "retrieval-n131072-s7", "score": {"shape_valid": true, "position_matches": [true, true, true, true], "complete": true}, "generated_text": "{\n\"KEY_5EE1BEA30178\": \"VAL_A91589D23FF0\",\n\"KEY_5FCAB7E42EDC\": \"VAL_F0E6FC236486\",\n\"KEY_169EC8E3E10F\": \"VAL_ABEAF259FBCC\",\n\"KEY_91FAFFBC08C1\": \"VAL_3323D4E2EEE8\"\n}", "input_tokens": 131072, "requested_output_tokens": 2048, "output_tokens": 116, "token_ids": [90, 198, 1, 4631, 62, 20, 6878, 16, 11261, 32, 18, 15, 16, 22, 23, 763, 328, 5419, 1511, 24, 16, 20, 23, 24, 35, 17, 18, 1686, 15, 487, 198, 1, 4631, 62, 20, 6542, 1803, 22, 36, 19, 17, 1425, 34, 763, 328, 5419, 1352, 15, 36, 21, 6542, 17, 18, 21, 19, 23, 21, 487, 198, 1, 4631, 62, 16, 21, 24, 7263, 23, 36, 18, 36, 16, 15, 37, 763, 328, 5419, 1511, 11261, 8018, 17, 20, 24, 15731, 3580, 487, 198, 1, 4631, 62, 24, 16, 3505, 1686, 4738, 15, 23, 34, 16, 763, 328, 5419, 62, 18, 18, 17, 18, 35, 19, 36, 17, 31749, 23, 1, 198, 92, 248046], "terminal": ["stop"], "usage": [{"prompt_tokens": 131072, "total_tokens": 131188, "completion_tokens": 116}], "valid": true, "token_counts_per_choice": [1, 2, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], "done_unique_last": true, "client_success": true, "client_error": false, "wire_sha256": "2dd86fa4465e312453b1e1f6b2a35f2dce8ce04f483d3cec19062569ed07db0d", "prompt_sha256": "52f7ef11474d1512fb195ac6659cc2a02abc2584aac43d3d0304614888397f02", "ttft_s": 49.19754179206211, "latency_s": 51.229562125052325, "itl": [0.0005267629167065024, 0.0021109021035954356, 0.01795967889484018, 0.018118301057256758, 0.018090644967742264, 0.01811502198688686, 0.01811253000050783, 0.018099912093020976, 0.018136715982109308, 0.018123426008969545, 0.018111850949935615, 0.018136724014766514, 0.01819446694571525, 0.018210873007774353, 0.018277595983818173, 0.017998358001932502, 0.01806594501249492, 0.01808118703775108, 0.018057544948533177, 0.01811523106880486, 0.018129058997146785, 0.018125297967344522, 0.018093000049702823, 0.018112016026861966, 0.0180968219647184, 0.018148657982237637, 0.018127487041056156, 0.018079911940731108, 0.01816682005301118, 0.018048944999463856, 0.018116337014362216, 0.018226001993753016, 0.018064440926536918, 0.018245685030706227, 0.01803179003763944, 0.018168082926422358, 0.018142148037441075, 0.018165578017942607, 0.01814563397783786, 0.018409971962682903, 0.0177833610214293, 0.01809397805482149, 0.018007310922257602, 0.01810706709511578, 0.018086322932504117, 0.018148913979530334, 0.018064185976982117, 0.01812026707921177, 0.018104761955328286, 0.01813655497971922, 0.018124644993804395, 0.018116641091182828, 0.01810795394703746, 0.018107481068000197, 0.0181166889378801, 0.018145026988349855, 0.01817446597851813, 0.018133901990950108, 0.018124604015611112, 0.018316143075935543, 0.018011601991020143, 0.018083300907164812, 0.018088088021613657, 0.01810112106613815, 0.018084676004946232, 0.018100408953614533, 0.01814022404141724, 0.018118005944415927, 0.018098077969625592, 0.01809231110382825, 0.01817126094829291, 0.018146060057915747, 0.018044240889139473, 0.018423393019475043, 0.01801281701773405, 0.01810969307553023, 0.018056156928651035, 0.018110285978764296, 0.018095382023602724, 0.01811817300040275, 0.018139394000172615, 0.018161469022743404, 0.01817277993541211, 0.018067795084789395, 0.01807847898453474, 0.018136493978090584, 0.018055993015877903, 0.0181000690208748, 0.018436856917105615, 0.018099043052643538, 0.017958211014047265, 0.018238025018945336, 0.018051074934192002, 0.017973167006857693, 0.018086618976667523, 0.018096404033713043, 0.018158413004130125, 0.01815057103522122, 0.018049431033432484, 0.018127114977687597, 0.018159415922127664, 0.018180075101554394, 0.018215834978036582, 0.01809852698352188, 0.018161755986511707, 0.018112045945599675, 0.01808205700945109, 0.018088274053297937, 0.0180771560408175, 0.0180913939839229, 0.018115633982233703, 0.018108626012690365, 0.018088826909661293, 0.01829039107542485], "tpot_s": 0.017669742026001863} -{"phase": "request", "index": 28, "case": "retrieval-n131072-s8", "score": {"shape_valid": true, "position_matches": [true, true, true, true], "complete": true}, "generated_text": "{\n \"KEY_6AB0243D81A0\": \"VAL_310AE8CD38F3\",\n \"KEY_FB2B98D7F0FF\": \"VAL_AC78F7C75A1B\",\n \"KEY_BC9200996E46\": \"VAL_03FC2E983876\",\n \"KEY_0F4197453005\": \"VAL_3284D482A2AD\"\n}", "input_tokens": 131072, "requested_output_tokens": 2048, "output_tokens": 128, "token_ids": [90, 198, 220, 328, 4631, 62, 21, 1803, 15, 17, 19, 18, 35, 23, 16, 32, 15, 763, 328, 5419, 62, 18, 16, 15, 13276, 23, 6281, 18, 23, 37, 18, 487, 198, 220, 328, 4631, 62519, 17, 33, 24, 23, 35, 22, 37, 15, 1686, 763, 328, 5419, 15045, 22, 23, 37, 22, 34, 22, 20, 32, 16, 33, 487, 198, 220, 328, 4631, 69792, 24, 17, 15, 15, 24, 24, 21, 36, 19, 21, 763, 328, 5419, 62, 15, 18, 6542, 17, 36, 24, 23, 18, 23, 22, 21, 487, 198, 220, 328, 4631, 62, 15, 37, 19, 16, 24, 22, 19, 20, 18, 15, 15, 20, 763, 328, 5419, 62, 18, 17, 23, 19, 35, 19, 23, 17, 32, 17, 1745, 1, 198, 92, 248046], "terminal": ["stop"], "usage": [{"prompt_tokens": 131072, "total_tokens": 131200, "completion_tokens": 128}], "valid": true, "token_counts_per_choice": [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], "done_unique_last": true, "client_success": true, "client_error": false, "wire_sha256": "117a029b0275d1734c18d735740c575dd5f87730d3eb1438ec3209fae3d104fd", "prompt_sha256": "aee9994267afb34e94693203427be77c9e5149e92f7701ba296681517474b02f", "ttft_s": 49.184093618998304, "latency_s": 51.43968084396329, "itl": [0.0005339269991964102, 8.765025995671749e-06, 0.00771765096578747, 0.018033480970188975, 0.018167803063988686, 0.01812109095044434, 0.01812462997622788, 0.018166758003644645, 0.01818105299025774, 0.018112882040441036, 0.018197005963884294, 0.017996233073063195, 0.018096589948982, 0.018076819949783385, 0.01818812603596598, 0.0180612460244447, 0.0180939199635759, 0.018123934045433998, 0.018122854991815984, 0.01828522398136556, 0.018042066949419677, 0.01828716800082475, 0.018083180068060756, 0.018092509941197932, 0.01805003301706165, 0.018057462060824037, 0.01816529396455735, 0.01814808405470103, 0.018117318977601826, 0.01836200593970716, 0.017900260048918426, 0.018356947926804423, 0.017976853996515274, 0.018019833019934595, 0.018048203084617853, 0.018090655910782516, 0.01811717904638499, 0.018156634992919862, 0.018080321024172008, 0.018073023995384574, 0.018099774955771863, 0.018183685024268925, 0.018075083033181727, 0.018085581017658114, 0.018126440001651645, 0.018120383960194886, 0.018164678942412138, 0.01803782710339874, 0.018162947962991893, 0.01817911199759692, 0.018228586995974183, 0.018105035996995866, 0.01815614802762866, 0.018136024940758944, 0.018089089076966047, 0.01810676499735564, 0.018241561949253082, 0.018109582946635783, 0.017995171016082168, 0.018086340045556426, 0.01809292600955814, 0.01818561996333301, 0.018149869050830603, 0.01828633202239871, 0.018064973992295563, 0.018003639997914433, 0.018102804897353053, 0.018095616018399596, 0.018507031025364995, 0.01780261704698205, 0.018113662023097277, 0.018262829980812967, 0.01804612693376839, 0.018272140063345432, 0.018209390924312174, 0.018076189095154405, 0.01814308692701161, 0.0182017560582608, 0.017940484918653965, 0.01811549998819828, 0.017964569036848843, 0.01809376198798418, 0.0181204320397228, 0.01817377598490566, 0.018108795979060233, 0.018121775006875396, 0.01823330798652023, 0.01804196706507355, 0.01805372501257807, 0.018085001967847347, 0.018090737983584404, 0.018107020994648337, 0.018127594026736915, 0.018165549961850047, 0.018130006035789847, 0.01819128100760281, 0.01814807194750756, 0.018144826986826956, 0.018096037092618644, 0.0181323429569602, 0.018117396044544876, 0.018562275916337967, 0.01776664098724723, 0.018039221060462296, 0.018111538025550544, 0.01811446901410818, 0.01812442089430988, 0.01811109308619052, 0.01811467600055039, 0.01812661299481988, 0.018144420930184424, 0.018122642068192363, 0.018079608911648393, 0.0181223270483315, 0.018163940985687077, 0.018178623053245246, 0.018119529937393963, 0.018154671997763216, 0.01814893400296569, 0.01815535605419427, 0.018111104960553348, 0.018097336986102164, 0.018080363050103188, 0.0181945109507069, 0.018077890039421618, 0.018078482011333108, 0.018241108977235854], "tpot_s": 0.017760529330432942} -{"phase": "request", "index": 29, "case": "retrieval-n131072-s9", "score": {"shape_valid": true, "position_matches": [true, true, true, true], "complete": true}, "generated_text": "{\n\"KEY_085A834BA79A\": \"VAL_59D4EBF50164\",\n\"KEY_8627554907F8\": \"VAL_6E20721FEE7F\",\n\"KEY_05871F38A4E4\": \"VAL_A77A20C2564E\",\n\"KEY_00F87ADA09A5\": \"VAL_B3E46FBE4032\"\n}", "input_tokens": 131072, "requested_output_tokens": 2048, "output_tokens": 128, "token_ids": [90, 198, 1, 4631, 62, 15, 23, 20, 32, 23, 18, 19, 6844, 22, 24, 32, 763, 328, 5419, 62, 20, 24, 35, 19, 8006, 37, 20, 15, 16, 21, 19, 487, 198, 1, 4631, 62, 23, 21, 17, 22, 20, 20, 19, 24, 15, 22, 37, 23, 763, 328, 5419, 62, 21, 36, 17, 15, 22, 17, 16, 37, 6878, 22, 37, 487, 198, 1, 4631, 62, 15, 20, 23, 22, 16, 37, 18, 23, 32, 19, 36, 19, 763, 328, 5419, 1511, 22, 22, 32, 17, 15, 34, 17, 20, 21, 19, 36, 487, 198, 1, 4631, 62, 15, 15, 37, 23, 22, 31228, 15, 24, 32, 20, 763, 328, 5419, 1609, 18, 36, 19, 21, 37, 11261, 19, 15, 18, 17, 1, 198, 92, 248046], "terminal": ["stop"], "usage": [{"prompt_tokens": 131072, "total_tokens": 131200, "completion_tokens": 128}], "valid": true, "token_counts_per_choice": [1, 2, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], "done_unique_last": true, "client_success": true, "client_error": false, "wire_sha256": "357d083f84dc48374e93a6430728f6d9283f93cc9f89847e1c5998f9ecb4fb3f", "prompt_sha256": "c7bfcabe355ce2ef26e293403979601332f80f19a857473621dc1258ba35fe0c", "ttft_s": 49.175294371903874, "latency_s": 51.42824721895158, "itl": [0.0004955090116709471, 0.005268463981337845, 0.017877549049444497, 0.0181708550080657, 0.01801314693875611, 0.018128017080016434, 0.018152061966247857, 0.018097226042300463, 0.018166363006457686, 0.018042153911665082, 0.018130476004444063, 0.018128175055608153, 0.018088910961523652, 0.01811278099194169, 0.01814415503758937, 0.018102200934663415, 0.018265227088704705, 0.01811734400689602, 0.018133425968699157, 0.018174501950852573, 0.01823538402095437, 0.018051426974125206, 0.018090062076225877, 0.018076710985042155, 0.01808178296778351, 0.018117500003427267, 0.01811672397889197, 0.01809081109240651, 0.01810130395460874, 0.018144620000384748, 0.018125621019862592, 0.01810285891406238, 0.018126493087038398, 0.01814467494841665, 0.01805124804377556, 0.01810600096359849, 0.0181216299533844, 0.018076047068461776, 0.018172631040215492, 0.018157293903641403, 0.018185308086685836, 0.018162879976443946, 0.018143306020647287, 0.018140004016458988, 0.0180665769148618, 0.018097782973200083, 0.018097585067152977, 0.018145684036426246, 0.01806460099760443, 0.01809824991505593, 0.0180930980714038, 0.018112762947566807, 0.018127318006008863, 0.018145306035876274, 0.01812928798608482, 0.01811914995778352, 0.01809998508542776, 0.018192929914221168, 0.018220940022729337, 0.018200329039245844, 0.017989941989071667, 0.018108341959305108, 0.01818171003833413, 0.018163128988817334, 0.01819267903920263, 0.018123334972187877, 0.018093978986144066, 0.018112163990736008, 0.018062851973809302, 0.018090815981850028, 0.018105798051692545, 0.018080277950502932, 0.018119971035048366, 0.01815560506656766, 0.018090152996592224, 0.018112662946805358, 0.0181125330273062, 0.01813218300230801, 0.018117205007001758, 0.018176177982240915, 0.018079094938002527, 0.018107957090251148, 0.01816222001798451, 0.01813270489219576, 0.018188206013292074, 0.018141515087336302, 0.018195264972746372, 0.018113738973625004, 0.018087967997416854, 0.018141619977541268, 0.018073706072755158, 0.018119264976121485, 0.01807541702874005, 0.0181316229281947, 0.01814124407246709, 0.018099428969435394, 0.01813464192673564, 0.01815870706923306, 0.01816098904237151, 0.018159090890549123, 0.01800579600967467, 0.01810041698627174, 0.01820240099914372, 0.018106332048773766, 0.018145801033824682, 0.01827548793517053, 0.018026860081590712, 0.018189589958637953, 0.018232161994092166, 0.0180682489881292, 0.018189401016570628, 0.01819999294821173, 0.017930099042132497, 0.018119064974598587, 0.01808778103441, 0.01803371903952211, 0.018126365961506963, 0.018143438966944814, 0.018149271025322378, 0.018175791017711163, 0.018014027969911695, 0.01810710900463164, 0.018121695960871875, 0.018110143020749092, 0.018198566045612097, 0.01822491199709475], "tpot_s": 0.017739786197226055} -{"phase": "cleanup", "exit_code": 0, "forced": false, "owned_group_remaining": 0} -{"phase": "complete", "verdict": "INCONCLUSIVE", "scope": "retrieval_baseline_qualification_only", "elapsed_s": 868.1522934909444} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/retrieval-baseline-impact.md b/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/retrieval-baseline-impact.md deleted file mode 100644 index 4b02c32cd..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/retrieval-baseline-impact.md +++ /dev/null @@ -1,56 +0,0 @@ -# Baseline retrieval qualification outcome - -Refs [Issue67](https://github.com/InferMatrixLab/InferRSI/issues/67#issuecomment-6347838911). -**INCONCLUSIVE — baseline qualification only, no candidate or activated gates.** - -| Input tokens | Complete cases | Exact matches at each of four positions | Generated tokens | -| ---: | ---: | --- | ---: | -| 8192 | 10/10 | 10/10, 10/10, 10/10, 10/10 | 1289 | -| 65536 | 10/10 | 10/10, 10/10, 10/10, 10/10 | 1236 | -| 131072 | 10/10 | 10/10, 10/10, 10/10, 10/10 | 1241 | - -Job1968 completed0:0 in879 allocated GPU-seconds, one GPU, allocation released. -The v2 homogeneous/nonrepeating inputs and v3 observer were bound prospectively. -The run used unchanged vLLM0.32.0 default optimization choices, no OMP override, -greedy natural EOS with maximum2048, and the authenticated official client. -All30 raw streams were independently re-read locally: exact prompt echoes, -output IDs/usage, unique final DONE, final stop, official-client text/timing and -fixed scoring agree. Nine128K requests coalesced multiple tokens into one choice; -no requests were dropped, retried or regenerated. Full raw SSE/client/server -records remain in controlled storage; their identities are in the summary. -The shared observation JSONL is byte-identical to the original and contains no -private operational paths or device identities. - -All30 final IDs are248046; authenticated generation_config declares EOS248046 -and248044, and every final choice has stop_reason:null. This post-run audit -supports observed EOS termination; a prospective full termination validator -is still required before acceptance. No historical validator or score changed. - -Startup was119.138485 seconds. Descriptive median TTFT/whole latency were -1.678214/3.028684 seconds at8K,18.607111/20.278450 at64K and49.168697/51.341158 -at128K. Natural-EOS outputs totaled3766 tokens. These are retrieval costs, -not the required2048-token performance matrix, a paired speedup or parity claim. -No production engine patch was made by this attempt. - -Observer cleanup reports server0, forced=false and no owned group remaining. -Server logs show30-second drain and manager shutdown, without observed kill -escalation, but one multiprocessing semaphore cleanup warning remains. Preserve -that limitation; do not label teardown entirely warning-free. - -Readiness total is2388/7200 GPU-seconds (0.663333/2GPU-hours), with4812 seconds -remaining. The separate initial147-second authorization is unchanged. Baseline4, -search8 and finalization30GPU-hour stages remain unused and reserved. - -During independent local audit, an initial check incorrectly equated the -observer's default-spaced JSON prompt digest with the fixture's compact JSON -digest. Exact echoed IDs were already equal. The corrected audit verifies both -serialization-specific digests separately and compares actual ID arrays; no -source inputs, responses, raw hashes, valid flags or scoring were rewritten. - -The proposed owner policy remains baseline>=9/10 complete per length and -candidate no regression against matched baseline complete/per-position counts. -This observed baseline is10/10, so that policy would require candidate10/10 at -each length. Owner confirmation and complete validator freeze remain pending. - -KnowledgeStore candidate `case.qwen38-retrieval-construction-cand` revision -`sha256:3e1c05db461c9002bdf1af8065adc39b6eb30a662ace24d44ea81735be89f62d` was written and read back exactly; no library promotion. diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/retrieval-preparation-impact.md b/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/retrieval-preparation-impact.md deleted file mode 100644 index 06b7b5171..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/retrieval-preparation-impact.md +++ /dev/null @@ -1,38 +0,0 @@ -# Retrieval construction and review attempts - -Refs [Issue67](https://github.com/InferMatrixLab/InferRSI/issues/67). -**INCONCLUSIVE. Preparation only; no model answers or activated gates.** - -CPU1964 prepared v1:30 exact-size inputs totaling2,048,000 tokens;28 seconds, -exit0, no GPU. Local hashes and positions verified. Independent review identified -a meaningful construct-validity defect: decoys had KEY_D/VAL_D prefixes and an -extra sentence, while targets did not; a128-record filler pool repeated. This -version is retained and rejected for freezing, not mislabeled a model failure. -The first scorer also conditioned every position on a fully valid JSON schema. -Its behavior was internally consistent but did not represent independent recall. - -Before any model answers, v2 made identifier lengths/templates homogeneous and -used nonrepeating decoys. CPU1965 prepared the same30-case length/position matrix; -all generator assertions and local readback passed, with no GPU use. V2 extraction -separates exact whole-object success from independent available-key matches; -duplicate/invalid JSON still fails every position. Eight targeted local scorer -checks passed. These facts qualify input preparation only; independent v2 review, -baseline qualification and owner gate confirmation remain pending. - -Both versions retain compressed inputs, manifest, generator and observer sources -in the campaign evidence tree. V1 observer was never GPU-executed. V2 observer -also corrects the baseline1961 one-token-per-choice assumption prospectively; -the old failed observer/records are unchanged. No production engine or gates -were changed, and no candidate was optimized against these final-set targets. - -KnowledgeStore candidate `case.qwen38-retrieval-construction-cand`, revision -`sha256:f061eeb3c8c18d57eec16253fb990e42880483b3a9fad3f5beca2c9cf5b7a4fb`, -was created and read back byte-for-byte; no library promotion. - -Subsequent independent v2 and v3 review found no blocking issue. V2 removes the -construct shortcut; ten independent scorer edge cases passed. V3 asserts and -records absent OMP override and requests30-second server shutdown with45-second -observer wait. Scorer/inputs/client/kernels unchanged. Five raw baseline streams -and five synthetic invalid mutations confirm the prospective wire expression; -this does not rescore1961. A separately declared30-case baseline qualification -is submitted under Issue67 comment6347838911, max1800 GPU-seconds; outcome pending. diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/review.md b/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/review.md deleted file mode 100644 index c5af00e55..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/review.md +++ /dev/null @@ -1,25 +0,0 @@ -# Independent review of bounded repairs - -Reviewer: Codex reviewer subagent `review_fp8_graph` (read-only; no edits or GPU work). -Base: `eb56fbb05ef7d56756d98b8bac546e8d0264b2cf`. -Configured-backend commit: `4cdfd11f3f1602c27c94c2a7f7e13a5ba7a9d2c8`. -Reviewed head: `0f8ab73a3cc0d3f21099bc37f5a800e0633f7c32`. - -The reviewer inspected both commits and consumers, plus the diagnostic observer -and job 1935 raw artifact/hash. No introduced blocking correctness defects were -found. The reviewer independently verified matching teacher-token sequences -between jobs 1925 and 1935, initial prefill equality in all three cases, -fixed-capacity eager/replay equality in 48/48 steps, compact/buffered equality -in 35/48 steps, and all-arm greedy equality in 48/48 steps. - -Recommendation: accept the bounded repairs after the pending CPU suite passes, -while retaining campaign INCONCLUSIVE. The observer uses hooks, all three -prompts use bucket1024, and it bypasses HTTP and RequestRunner prewarm/reuse. -Production lifecycle, capacity transition and remaining decode differences -must be assessed before broader Graph qualification. No performance or service -acceptance claim is supported. - -Post-review evidence: CPU job 1937 completed with 156 passed / 1 skipped; -the skipped test requires the separately gated GPU contract environment. No -source changes were made after review. This closes the review's CPU-suite -condition, not the remaining service/quality gaps or campaign acceptance. diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-benign-retry-2002.json b/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-benign-retry-2002.json deleted file mode 100644 index 17bee28fb..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-benign-retry-2002.json +++ /dev/null @@ -1,40 +0,0 @@ -{ - "scope": "installed_runtime_inventory_before_trusted_benign_probe", - "verdict": "INCONCLUSIVE", - "job": 2002, - "slurm_state": "FAILED", - "slurm_exit_code": "1:0", - "allocated_seconds": 4, - "allocated_cpus": 4, - "allocated_gpus": 0, - "requested_memory_gib": 16, - "walltime_limit_seconds": 900, - "allocation_released": true, - "terminal": { - "completed": false, - "signal": "none", - "exit_code": 1, - "job_id": "2002" - }, - "reason": "Frozen ELF inventory scanner found unresolved dependency text; exact file/library not recorded by this scanner.", - "package_audit_before_sha256": "1ae0edd0e0c05b44b73a6b7f58ab6202f37722c6707cdb0f53aff7e02dd2148d", - "package_installation_performed": false, - "bwrap_sha256": "04683adbb7f45376ba56d6936a02203535ff9451b1592299ac4ba69838c33627", - "complete_rootfs_produced": false, - "probe_ran": false, - "postflight_ran": false, - "installed_evaluator_source_byte_comparison_ran": false, - "evaluator_or_model_executed": false, - "measured_e2e_effect": null, - "script_sha256": "a2b9d88462c0066fdc58095f9a961a5b7bcef3ce712743c7093520296173e1c1", - "private_evidence_sha256": { - "submission.private.txt": "90d11685fa518e69652802000eb067af76170409da6d8dca3f6d76cf491c9735", - "scontrol-final.private.txt": "9c69e6c75d1580169d9404b175191c8ec2833dd1b12ea454a8193c165be4bd7d", - "squeue-final.private.txt": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", - "probe.private.log": "02325dfae03f6eb64711ddf84c09c34ad606d6f9107350e44f1692b32607e859", - "output.private.tar": "14e15739bfbd439c77e48f0ca01df851d006623bb9cb0d1c5c9a402f0a7351b7" - }, - "readiness_gpu_seconds_used": 2388, - "readiness_gpu_seconds_cap": 7200, - "earlier_diagnostic_gpu_seconds_separate": 147 -} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-benign-retry-2006.json b/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-benign-retry-2006.json deleted file mode 100644 index 01872ac25..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-benign-retry-2006.json +++ /dev/null @@ -1,49 +0,0 @@ -{ - "scope": "trusted_benign_isolation_probe", - "verdict": "INCONCLUSIVE", - "job": 2006, - "slurm_state": "COMPLETED", - "slurm_exit_code": "0:0", - "allocated_seconds": 68, - "allocated_cpus": 4, - "allocated_gpus": 0, - "requested_memory_gib": 16, - "walltime_limit_seconds": 900, - "allocation_released": true, - "terminal": { - "completed": true, - "signal": "none", - "exit_code": 0, - "job_id": "2006" - }, - "rootfs_digest": "sha256:83ffef463b512690cf420fb809798525e8c26f5dda6d861622e3c43420786c0e", - "rootfs_json_sha256": "4bd529089780175cc234db209a71ec12dc6cc9b842dfd0cbf089a7b6088e4f89", - "readonly_bind_entry_count": 25241, - "rootfs_before_after_equal": true, - "checked_evaluator_python_sources": 43, - "package_count": 97, - "package_audits_before_after_equal": true, - "benign_checks_passed": 12, - "postflight_checks_passed": 12, - "probe_stdout_sha256": "95b04d5b1776fe8359c6dd4e2597aafa9050c9454c48c7167a2e34a4933c0806", - "probe_execution_sha256": "8fdda7600618f49b2e9d020d7b720ede439c11e4cbd4f0750028aa4047654d89", - "loader_policy": { - "base_library_directory": "/opt/python/lib", - "sandbox_ld_library_path": "/opt/python/lib", - "scanner_inherits_environment": false - }, - "generated_code_executed": false, - "full_evaluator_executed": false, - "qualified_generated_code_executor": false, - "measured_e2e_effect": null, - "private_evidence_sha256": { - "submission.private.txt": "8fb318176f488a408ab85613f5734e5b094726c61bcf3ff2ef038daac5a38433", - "scontrol-final.private.txt": "5cc3aa2b872d63a74efb15af3d0717d61a9f2172509e894faed6e959f999165f", - "squeue-final.private.txt": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", - "diagnostic.private.log": "4db89a71042f1ef606ff105af33a09a0e4827960c04eeafbc9664b1719f75421", - "output.private.tar": "79941ba251352ec7e51fc0bb3537185d9db081882a69d66fd3febb3b847f709f" - }, - "readiness_gpu_seconds_used": 2388, - "readiness_gpu_seconds_cap": 7200, - "earlier_diagnostic_gpu_seconds_separate": 147 -} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-benign-retry-impact.md b/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-benign-retry-impact.md deleted file mode 100644 index a9a9e8893..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-benign-retry-impact.md +++ /dev/null @@ -1,24 +0,0 @@ -# Benign retry stopped at unresolved ELF inventory - -Refs [Issue #67](https://github.com/InferMatrixLab/InferRSI/issues/67). -**INCONCLUSIVE: the trusted isolation probe did not start.** - -CPU2002 reused the existing evaluator environment without package installation -and authenticated the completed2001bubblewrap binary. The97-distribution audit -and original source/freeze checks completed. Inventory passed the eight required -bubblewrap option checks, then the unchanged scanner raised ELF runtime dependency -missing. Its error does not record the selected ELF or library, so this result -does not establish which dependency is absent or whether independent shared-library -scanning reproduces the actual importer/loader context. - -Slurm ended FAILED1:0/4allocated seconds,4CPU/zeroGPU,released; completed=false -matches. No complete rootfs, installed-source Python-byte comparison, benign probe -or postflight was reached. The prior build remains successful only in its limited -construction scope; environment/evaluator qualification is incomplete. Next use a -bounded read-only first-failing-ELF diagnostic retaining full private ldd output; -do not skip missing-dependency checks or add guessed libraries. - -No measured E2E effect or newGPUusage. Readiness remains2388/7200seconds with -initial147separate. Original locks, runtime helper and probe bytes are unchanged. - -KnowledgeStore candidate `case.qwen38-eval-sandbox-preparation-cand` revision `sha256:9289dd074a86d10dd9d42b6c09b7ebe6a6fb0a1be7eb9eb291c9de58b04b3835`; readback verified. diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-benign-retry-packages-before.json b/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-benign-retry-packages-before.json deleted file mode 100644 index 026dead7f..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-benign-retry-packages-before.json +++ /dev/null @@ -1 +0,0 @@ -{"actual": {"aiohappyeyeballs": "2.7.1", "aiohttp": "3.14.5", "aiosignal": "1.4.0", "annotated-doc": "0.0.5", "annotated-types": "0.8.0", "anthropic": "2.1.0", "anyio": "4.15.1", "appdirs": "1.4.4", "attrs": "26.1.0", "boto3": "1.43.117", "botocore": "1.43.117", "certifi": "2026.10.19", "cffi": "2.1.1", "charset-normalizer": "3.5.2", "click": "8.5.0", "cryptography": "50.0.2", "datasets": "5.1.0", "dill": "0.4.1", "docstring-parser": "0.18.0", "evalplus": "0.1.0.dev1", "filelock": "4.2.4", "fire": "0.7.1", "frozenlist": "1.8.0", "fsspec": "2026.7.0", "google-ai-generativelanguage": "0.6.15", "google-api-core": "2.33.0", "google-api-python-client": "2.201.0", "google-auth": "2.62.0", "google-auth-httplib2": "0.4.4", "google-generativeai": "0.8.6", "googleapis-common-protos": "1.75.0", "grpcio": "1.84.1", "grpcio-status": "1.71.2", "h11": "0.16.0", "hf-xet": "1.7.0", "httpcore": "1.0.9", "httpcore2": "2.16.0", "httplib2": "0.32.0", "httpx": "0.28.1", "httpx2": "2.16.0", "huggingface-hub": "2.2.0", "idna": "3.20", "jiter": "0.17.0", "jmespath": "1.1.0", "markdown-it-py": "4.2.0", "mdurl": "0.1.2", "multidict": "7.1.0", "multipledispatch": "1.0.0", "multiprocess": "0.70.19", "numpy": "2.5.4", "ollama": "0.6.3", "openai": "3.36.1", "opentelemetry-api": "1.45.1", "packaging": "26.3", "pandas": "3.1.0", "propcache": "0.5.4", "proto-plus": "1.28.2", "protobuf": "5.29.6", "psutil": "7.2.2", "pyarrow": "26.0.0", "pyasn1": "0.6.4", "pyasn1-modules": "0.4.2", "pycparser": "3.12", "pydantic": "2.14.1", "pydantic-core": "2.50.1", "pygments": "2.21.0", "pyparsing": "3.3.3", "python-dateutil": "2.9.0.post0", "pyyaml": "6.0.3", "regex": "2026.9.29", "requests": "2.34.2", "rich": "15.0.0", "s3transfer": "0.19.3", "safetensors": "0.8.0", "setuptools": "84.2.1", "setuptools-scm": "10.3.4", "shellingham": "1.5.4", "six": "1.17.0", "sniffio": "1.3.1", "tempdir": "0.7.1", "termcolor": "3.3.0", "tokenizers": "0.23.3", "tqdm": "4.70.1", "transformers": "5.20.2", "tree-sitter": "0.26.0", "tree-sitter-python": "0.25.0", "truststore": "0.10.5", "typer": "0.27.3", "typing-extensions": "4.16.0", "typing-inspection": "0.4.4", "uritemplate": "4.2.0", "urllib3": "2.8.0", "vcs-versioning": "2.6.0", "wget": "3.2", "wheel": "0.48.0", "xxhash": "4.0.2", "yarl": "1.25.1"}, "expected": {"aiohappyeyeballs": "2.7.1", "aiohttp": "3.14.5", "aiosignal": "1.4.0", "annotated-doc": "0.0.5", "annotated-types": "0.8.0", "anthropic": "2.1.0", "anyio": "4.15.1", "appdirs": "1.4.4", "attrs": "26.1.0", "boto3": "1.43.117", "botocore": "1.43.117", "certifi": "2026.10.19", "cffi": "2.1.1", "charset-normalizer": "3.5.2", "click": "8.5.0", "cryptography": "50.0.2", "datasets": "5.1.0", "dill": "0.4.1", "docstring-parser": "0.18.0", "filelock": "4.2.4", "fire": "0.7.1", "frozenlist": "1.8.0", "fsspec": "2026.7.0", "google-ai-generativelanguage": "0.6.15", "google-api-core": "2.33.0", "google-api-python-client": "2.201.0", "google-auth": "2.62.0", "google-auth-httplib2": "0.4.4", "google-generativeai": "0.8.6", "googleapis-common-protos": "1.75.0", "grpcio": "1.84.1", "grpcio-status": "1.71.2", "h11": "0.16.0", "hf-xet": "1.7.0", "httpcore": "1.0.9", "httpcore2": "2.16.0", "httplib2": "0.32.0", "httpx": "0.28.1", "httpx2": "2.16.0", "huggingface-hub": "2.2.0", "idna": "3.20", "jiter": "0.17.0", "jmespath": "1.1.0", "markdown-it-py": "4.2.0", "mdurl": "0.1.2", "multidict": "7.1.0", "multipledispatch": "1.0.0", "multiprocess": "0.70.19", "numpy": "2.5.4", "ollama": "0.6.3", "openai": "3.36.1", "opentelemetry-api": "1.45.1", "packaging": "26.3", "pandas": "3.1.0", "propcache": "0.5.4", "proto-plus": "1.28.2", "protobuf": "5.29.6", "psutil": "7.2.2", "pyarrow": "26.0.0", "pyasn1": "0.6.4", "pyasn1-modules": "0.4.2", "pycparser": "3.12", "pydantic": "2.14.1", "pydantic-core": "2.50.1", "pygments": "2.21.0", "pyparsing": "3.3.3", "python-dateutil": "2.9.0.post0", "pyyaml": "6.0.3", "regex": "2026.9.29", "requests": "2.34.2", "rich": "15.0.0", "s3transfer": "0.19.3", "safetensors": "0.8.0", "setuptools": "84.2.1", "setuptools-scm": "10.3.4", "shellingham": "1.5.4", "six": "1.17.0", "sniffio": "1.3.1", "tempdir": "0.7.1", "termcolor": "3.3.0", "tokenizers": "0.23.3", "tqdm": "4.70.1", "transformers": "5.20.2", "tree-sitter": "0.26.0", "tree-sitter-python": "0.25.0", "truststore": "0.10.5", "typer": "0.27.3", "typing-extensions": "4.16.0", "typing-inspection": "0.4.4", "uritemplate": "4.2.0", "urllib3": "2.8.0", "vcs-versioning": "2.6.0", "wget": "3.2", "wheel": "0.48.0", "xxhash": "4.0.2", "yarl": "1.25.1"}, "phase": "evaluator"} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-benign-v2-impact.md b/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-benign-v2-impact.md deleted file mode 100644 index db9bb9cab..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-benign-v2-impact.md +++ /dev/null @@ -1,43 +0,0 @@ -# Trusted benign probe succeeds with explicit loader policy - -Refs [Issue #67](https://github.com/InferMatrixLab/InferRSI/issues/67). -**INCONCLUSIVE for generated-code execution, task quality and E2E.** - -CPU2006 completed0:0 in68seconds on4CPUs/zeroGPUs and released its allocation. -The separately versioned scanner uses a cleared subprocess environment with the -registered base lib directory. The sandbox deliberately sets its matching -relocated readonly directory after clearenv. This policy was independently -reviewed; CPU2005 did not establish that Python imports require that setting. -Missing dependencies, unexpected external paths and now nonzero ldd returns -remain fatal. The original frozen bundle, trusted probe and installed packages -were reused without changing their bytes. - -The actual trusted probe exits0 without timeout or stderr. All12 checks pass: -new namespaces, absent host paths, readonly runtime/root, blocked network with -only loopback, no-new-privileges, zero effective capabilities, namespace UID, -cleared synthetic inherited variable, relocated Python, single-CPU affinity and -the specified per-process rlimits. EvalPlus0.1.0.dev1, NumPy2.5.4 and psutil7.2.2 -import successfully; the small trusted NumPy sum returns5. - -Before/after rootfs JSON is byte-identical:25,241 entries with a recomputed -readonly-bind-set digest. All43 tracked EvalPlus Python files match the pinned -source. The97-distribution package audits are identical; all12 postflight checks -succeed, including old/new freezes, script/binary identities and source checks. -The full inventories and private mount locators stay in controlled storage, -bound by the report's hashes. Public probe output and executed source bytes are -retained. These observations cover this trusted probe and its tested invariants; -they do not certify every dependency, adversarial cleanup, aggregate resource -enforcement, answer secrecy or arbitrary generated-code containment. - -The five local mocked loader controls were independently rerun before execution. -Their initial fixture failed because a local temporary lexical path was not -resolved; that source is retained. Correcting the fixture to use resolved roots -matches the existing inventory entry point, without a runtime change for that -fixture failure. - -No full evaluator, benchmark data, generated model code or model ran. No measured -E2E effect. Next separately bind the synthetic evaluator trial to this completed -reference and its loader policy; the older conditional launcher is not activated -by this result. Readiness remains2388/7200GPUseconds with initial147separate. - -KnowledgeStore candidate `case.qwen38-eval-sandbox-preparation-cand` revision `sha256:095a2b664b624afb2f613cb7ec19b124d1b3668546e61d5abdb81ef7d53cdeea`; readback verified. diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-benign-v2-packages-verified.json b/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-benign-v2-packages-verified.json deleted file mode 100644 index 026dead7f..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-benign-v2-packages-verified.json +++ /dev/null @@ -1 +0,0 @@ -{"actual": {"aiohappyeyeballs": "2.7.1", "aiohttp": "3.14.5", "aiosignal": "1.4.0", "annotated-doc": "0.0.5", "annotated-types": "0.8.0", "anthropic": "2.1.0", "anyio": "4.15.1", "appdirs": "1.4.4", "attrs": "26.1.0", "boto3": "1.43.117", "botocore": "1.43.117", "certifi": "2026.10.19", "cffi": "2.1.1", "charset-normalizer": "3.5.2", "click": "8.5.0", "cryptography": "50.0.2", "datasets": "5.1.0", "dill": "0.4.1", "docstring-parser": "0.18.0", "evalplus": "0.1.0.dev1", "filelock": "4.2.4", "fire": "0.7.1", "frozenlist": "1.8.0", "fsspec": "2026.7.0", "google-ai-generativelanguage": "0.6.15", "google-api-core": "2.33.0", "google-api-python-client": "2.201.0", "google-auth": "2.62.0", "google-auth-httplib2": "0.4.4", "google-generativeai": "0.8.6", "googleapis-common-protos": "1.75.0", "grpcio": "1.84.1", "grpcio-status": "1.71.2", "h11": "0.16.0", "hf-xet": "1.7.0", "httpcore": "1.0.9", "httpcore2": "2.16.0", "httplib2": "0.32.0", "httpx": "0.28.1", "httpx2": "2.16.0", "huggingface-hub": "2.2.0", "idna": "3.20", "jiter": "0.17.0", "jmespath": "1.1.0", "markdown-it-py": "4.2.0", "mdurl": "0.1.2", "multidict": "7.1.0", "multipledispatch": "1.0.0", "multiprocess": "0.70.19", "numpy": "2.5.4", "ollama": "0.6.3", "openai": "3.36.1", "opentelemetry-api": "1.45.1", "packaging": "26.3", "pandas": "3.1.0", "propcache": "0.5.4", "proto-plus": "1.28.2", "protobuf": "5.29.6", "psutil": "7.2.2", "pyarrow": "26.0.0", "pyasn1": "0.6.4", "pyasn1-modules": "0.4.2", "pycparser": "3.12", "pydantic": "2.14.1", "pydantic-core": "2.50.1", "pygments": "2.21.0", "pyparsing": "3.3.3", "python-dateutil": "2.9.0.post0", "pyyaml": "6.0.3", "regex": "2026.9.29", "requests": "2.34.2", "rich": "15.0.0", "s3transfer": "0.19.3", "safetensors": "0.8.0", "setuptools": "84.2.1", "setuptools-scm": "10.3.4", "shellingham": "1.5.4", "six": "1.17.0", "sniffio": "1.3.1", "tempdir": "0.7.1", "termcolor": "3.3.0", "tokenizers": "0.23.3", "tqdm": "4.70.1", "transformers": "5.20.2", "tree-sitter": "0.26.0", "tree-sitter-python": "0.25.0", "truststore": "0.10.5", "typer": "0.27.3", "typing-extensions": "4.16.0", "typing-inspection": "0.4.4", "uritemplate": "4.2.0", "urllib3": "2.8.0", "vcs-versioning": "2.6.0", "wget": "3.2", "wheel": "0.48.0", "xxhash": "4.0.2", "yarl": "1.25.1"}, "expected": {"aiohappyeyeballs": "2.7.1", "aiohttp": "3.14.5", "aiosignal": "1.4.0", "annotated-doc": "0.0.5", "annotated-types": "0.8.0", "anthropic": "2.1.0", "anyio": "4.15.1", "appdirs": "1.4.4", "attrs": "26.1.0", "boto3": "1.43.117", "botocore": "1.43.117", "certifi": "2026.10.19", "cffi": "2.1.1", "charset-normalizer": "3.5.2", "click": "8.5.0", "cryptography": "50.0.2", "datasets": "5.1.0", "dill": "0.4.1", "docstring-parser": "0.18.0", "filelock": "4.2.4", "fire": "0.7.1", "frozenlist": "1.8.0", "fsspec": "2026.7.0", "google-ai-generativelanguage": "0.6.15", "google-api-core": "2.33.0", "google-api-python-client": "2.201.0", "google-auth": "2.62.0", "google-auth-httplib2": "0.4.4", "google-generativeai": "0.8.6", "googleapis-common-protos": "1.75.0", "grpcio": "1.84.1", "grpcio-status": "1.71.2", "h11": "0.16.0", "hf-xet": "1.7.0", "httpcore": "1.0.9", "httpcore2": "2.16.0", "httplib2": "0.32.0", "httpx": "0.28.1", "httpx2": "2.16.0", "huggingface-hub": "2.2.0", "idna": "3.20", "jiter": "0.17.0", "jmespath": "1.1.0", "markdown-it-py": "4.2.0", "mdurl": "0.1.2", "multidict": "7.1.0", "multipledispatch": "1.0.0", "multiprocess": "0.70.19", "numpy": "2.5.4", "ollama": "0.6.3", "openai": "3.36.1", "opentelemetry-api": "1.45.1", "packaging": "26.3", "pandas": "3.1.0", "propcache": "0.5.4", "proto-plus": "1.28.2", "protobuf": "5.29.6", "psutil": "7.2.2", "pyarrow": "26.0.0", "pyasn1": "0.6.4", "pyasn1-modules": "0.4.2", "pycparser": "3.12", "pydantic": "2.14.1", "pydantic-core": "2.50.1", "pygments": "2.21.0", "pyparsing": "3.3.3", "python-dateutil": "2.9.0.post0", "pyyaml": "6.0.3", "regex": "2026.9.29", "requests": "2.34.2", "rich": "15.0.0", "s3transfer": "0.19.3", "safetensors": "0.8.0", "setuptools": "84.2.1", "setuptools-scm": "10.3.4", "shellingham": "1.5.4", "six": "1.17.0", "sniffio": "1.3.1", "tempdir": "0.7.1", "termcolor": "3.3.0", "tokenizers": "0.23.3", "tqdm": "4.70.1", "transformers": "5.20.2", "tree-sitter": "0.26.0", "tree-sitter-python": "0.25.0", "truststore": "0.10.5", "typer": "0.27.3", "typing-extensions": "4.16.0", "typing-inspection": "0.4.4", "uritemplate": "4.2.0", "urllib3": "2.8.0", "vcs-versioning": "2.6.0", "wget": "3.2", "wheel": "0.48.0", "xxhash": "4.0.2", "yarl": "1.25.1"}, "phase": "evaluator"} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-benign-v2-postflight-status.tsv b/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-benign-v2-postflight-status.tsv deleted file mode 100644 index b22c1a709..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-benign-v2-postflight-status.tsv +++ /dev/null @@ -1,12 +0,0 @@ -inventory 0 -runtime-byte-comparison 0 -source 0 -package-set 0 -batch-script 0 -audit-script 0 -bwrap-binary 0 -python-binary 0 -freeze-manifest 0 -freeze-files 0 -source-freeze-manifest 0 -source-freeze-files 0 diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-benign-v2-probe-execution.json b/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-benign-v2-probe-execution.json deleted file mode 100644 index ca9cbe645..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-benign-v2-probe-execution.json +++ /dev/null @@ -1 +0,0 @@ -{"executor":{"binary_sha256":"04683adbb7f45376ba56d6936a02203535ff9451b1592299ac4ba69838c33627","kind":"bubblewrap","version":"bubblewrap 0.13.1"},"exit_code":0,"full_evaluator_executed":false,"generated_code_executed":false,"phase":"trusted_benign_probe","policy_script_sha256":"dc315ea40742ec3d27c7466e962864f4023996943e00776460d8be075c3174bf","probe_script_sha256":"8593cf53ca863b497ccb5ceced2b81d1c846522add2c9acd68b8a03975469d5c","qualified_generated_code_executor":false,"rootfs_digest":"sha256:83ffef463b512690cf420fb809798525e8c26f5dda6d861622e3c43420786c0e","stdout_sha256":"95b04d5b1776fe8359c6dd4e2597aafa9050c9454c48c7167a2e34a4933c0806","timed_out":false,"verdict":"INCONCLUSIVE"} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-benign-v2-probe-stdout.json b/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-benign-v2-probe-stdout.json deleted file mode 100644 index 5cabd9c54..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-benign-v2-probe-stdout.json +++ /dev/null @@ -1 +0,0 @@ -{"benchmark_data_mounted": false, "checks": {"environment_cleared": true, "fixed_rlimits": true, "host_paths_absent": true, "limited_cpu_affinity": true, "network_blocked": true, "new_namespaces": true, "no_new_privileges": true, "nonroot_namespace_uid": true, "only_loopback": true, "readonly_runtime_and_root": true, "relocated_python": true, "zero_effective_capabilities": true}, "evalplus_imported": true, "full_evaluator_executed": false, "generated_code_executed": false, "namespaces_changed": {"ipc": true, "net": true, "pid": true, "user": true, "uts": true}, "numpy_benign_result": 5, "packages": {"evalplus": "0.1.0.dev1", "numpy": "2.5.4", "psutil": "7.2.2"}, "passed": true, "psutil_imported": true, "qualified_generated_code_executor": false} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-bounds-2012.json b/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-bounds-2012.json deleted file mode 100644 index d649725fb..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-bounds-2012.json +++ /dev/null @@ -1,149 +0,0 @@ -{ - "scope": "trusted_resource_cleanup_readiness", - "verdict": "INCONCLUSIVE", - "diagnostic_verdict": "FAIL", - "job": 2012, - "slurm_state": "FAILED", - "slurm_exit_code": "2:0", - "allocated_seconds": 81, - "allocated_cpus": 4, - "allocated_gpus": 0, - "allocation_released": true, - "terminal": { - "completed": false, - "signal": "none", - "exit_code": 2, - "job_id": "2012" - }, - "failure": "Missing Python pidfd API, EOF startup-gate release during cleanup, and stale output digests.", - "kernel_pidfd_support_tested": false, - "late_bounds_checks_reported_true": 17, - "late_bounds_payload_accepted_as_qualification": false, - "cases": [ - { - "mode": "bounds", - "error": { - "message": "module 'os' has no attribute 'pidfd_open'", - "type": "AttributeError" - }, - "launcher_exit_code": -9, - "forced_launcher_cleanup": true, - "namespace_init_exit_observed": false, - "payload_start_token_written": false, - "later_payload_execution_observed": true, - "stdout_bytes": 1510, - "stderr_bytes": 0, - "reported_stdout_sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", - "actual_stdout_sha256": "5bd0021582e0eb7d8c3c994fad48feb144c3a613ca3ede420d8b87be669bf439", - "reported_stdout_digest_matches_final_bytes": false, - "stderr_sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", - "status_sha256": "931e1f5b31a06f8b069547952be4904c9e463ae5b0c7e6c11ed60216d0e5b289" - }, - { - "mode": "synthetic", - "error": { - "message": "module 'os' has no attribute 'pidfd_open'", - "type": "AttributeError" - }, - "launcher_exit_code": -9, - "forced_launcher_cleanup": true, - "namespace_init_exit_observed": false, - "payload_start_token_written": false, - "later_payload_execution_observed": true, - "stdout_bytes": 0, - "stderr_bytes": 1639, - "reported_stdout_sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", - "actual_stdout_sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", - "reported_stdout_digest_matches_final_bytes": true, - "stderr_sha256": "186dd2a1c020de95d505f029b4171ef9c62531366fdeda5120fee55557b7e0a7", - "status_sha256": "2e35c2b301ca390f92cf8ae99b261321bf10398854b521905177596695bd17bf" - }, - { - "mode": "normal", - "error": { - "message": "module 'os' has no attribute 'pidfd_open'", - "type": "AttributeError" - }, - "launcher_exit_code": -9, - "forced_launcher_cleanup": true, - "namespace_init_exit_observed": false, - "payload_start_token_written": false, - "later_payload_execution_observed": true, - "stdout_bytes": 170, - "stderr_bytes": 0, - "reported_stdout_sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", - "actual_stdout_sha256": "142004c4c4cad294e63b516d541091a0b3a4d361335c9bcfc9d843e3e034d940", - "reported_stdout_digest_matches_final_bytes": false, - "stderr_sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", - "status_sha256": "d97554e81eb422d060794b744e013a9933e5bc156d8af72d5a3f0511eee730f1" - }, - { - "mode": "timeout", - "error": { - "message": "module 'os' has no attribute 'pidfd_open'", - "type": "AttributeError" - }, - "launcher_exit_code": -9, - "forced_launcher_cleanup": true, - "namespace_init_exit_observed": false, - "payload_start_token_written": false, - "later_payload_execution_observed": true, - "stdout_bytes": 171, - "stderr_bytes": 0, - "reported_stdout_sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", - "actual_stdout_sha256": "133b2978b838ef3fc68b86360d3f5b339d3dcd7735791b445c97b9295effffff", - "reported_stdout_digest_matches_final_bytes": false, - "stderr_sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", - "status_sha256": "a060830715844e8a0cbf0af28af6f725c10991604cb033539907efd175cd43a7" - }, - { - "mode": "launcher_kill", - "error": { - "message": "module 'os' has no attribute 'pidfd_open'", - "type": "AttributeError" - }, - "launcher_exit_code": -9, - "forced_launcher_cleanup": true, - "namespace_init_exit_observed": false, - "payload_start_token_written": false, - "later_payload_execution_observed": true, - "stdout_bytes": 174, - "stderr_bytes": 390, - "reported_stdout_sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", - "actual_stdout_sha256": "563f95551a6393daf82519d3873f526fccdfe6391a93bc9852ae40a42b765aaa", - "reported_stdout_digest_matches_final_bytes": false, - "stderr_sha256": "b232cce8febd3f6b3e2370264207176e31a86d357324783687093e0feec213c5", - "status_sha256": "09d660a16e767e67c168b0c2006b452c0297205d1e2226fb83d17dd33152f69c" - } - ], - "rootfs_digest": "sha256:8035176c31d635366f5d215917c90c5f3f76bbc8d7ebfea41c1dafc3e17a93be", - "rootfs_json_sha256": "8f42ad8148ad5c15b50454e8cc66dbc6c92bc82593f3a886232ef42a8957d741", - "rootfs_before_after_equal": true, - "readonly_entry_count": 25241, - "package_count": 97, - "checked_evaluator_python_sources": 43, - "postflight_checks_passed": 10, - "source_sha256": { - "runtime.py": "997cbfed8398e8ad62d1beb4f6b4bc6e61a0f4572b9156cfd0c6f3fb52b03233", - "bounds_driver.py": "86e5706deaec92b3b1a976382856062cb8d4d305def38d05df050af001664428", - "synthetic_probe.py": "e8492c07878e12f17be272a2b5dc873bae8413de1ebf7ab3f2fce1a3d9e9dbed", - "run-synthetic.sbatch": "8d0f4405931e32ca5c9cbc3a602acdde8e71e999dea379d791f58c859870a46b" - }, - "summary_sha256": "a05976b5c69449e4e7b53c19cd0efeeec4c00cf212f84982a18b5c7294980716", - "generated_code_executed": false, - "benchmark_data_mounted": false, - "inner_probe_executed": true, - "full_evaluator_completed": false, - "qualified_generated_code_executor": false, - "measured_e2e_effect": null, - "private_evidence_sha256": { - "submission.private.txt": "5681cc87ec34ece52c0eb6855a9a2e0de1f8b77721306628e638e9793df934bd", - "scontrol-final.private.txt": "df8543ee639b95e2e357cb132c59de7292050796de22e795a182911ddaba3142", - "squeue-final.private.txt": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", - "diagnostic.private.log": "97f21712ab2953a8014d61d92c54f45ef18822f24900ed4aff9dece1d2313369", - "output.private.tar": "2b1bd3562a5af466699ae6f0751a08e7a4d804e7eb7afa43eb0d6faff12c9e64" - }, - "readiness_gpu_seconds_used": 2388, - "readiness_gpu_seconds_cap": 7200, - "earlier_diagnostic_gpu_seconds_separate": 147 -} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-bounds-2014.json b/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-bounds-2014.json deleted file mode 100644 index 0aef89b18..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-bounds-2014.json +++ /dev/null @@ -1,390 +0,0 @@ -{ - "scope": "trusted_startup_resource_and_cleanup_readiness", - "verdict": "INCONCLUSIVE", - "diagnostic_checks_passed": true, - "job": 2014, - "slurm_state": "COMPLETED", - "slurm_exit_code": "0:0", - "allocated_seconds": 114, - "allocated_cpus": 4, - "allocated_gpus": 0, - "allocation_released": true, - "terminal": { - "completed": true, - "signal": "none", - "exit_code": 0, - "job_id": "2014" - }, - "cases": [ - { - "authorization_sent": true, - "cleanup_complete": true, - "cleanup_or_evidence_errors": [], - "cleanup_registered_child_exits": [], - "exit_code": 0, - "final_init_exit_observed": true, - "final_launcher_exit_code": 0, - "forced_cleanup": false, - "generated_code_executed": false, - "init_exit_observed": true, - "mode": "gate_valid", - "outputs_stable": true, - "passed": true, - "probe_passed": true, - "registered_child_exit_observed": [], - "rootfs_digest": "sha256:a48b5c59b7ca4794e8a966ccb935040fb64920fb6c76ad50639e7d3ce0c99ac2", - "status_eof": true, - "status_sha256": "0efa5073e75bea5c443932a0f8ad8aea7cd329656b04b80bee74ae48cd2fa4d2", - "stderr_sha256": "3df969ed24948f490ec11adea8475939c24c5964ef12f5834eac38c4ba3cebd4", - "stdout_sha256": "6f8933157f82a7816e16df15631dff4dba873c73c2bced39f5e7f27e235586e1", - "stdout_bytes": 39, - "stderr_bytes": 25, - "private_report_sha256": "7e453b930e09a8c80fe4c7c0c1109e36e7271f0df82b5896097c8a4f1d137feb" - }, - { - "authorization_sent": false, - "cleanup_complete": true, - "cleanup_or_evidence_errors": [], - "cleanup_registered_child_exits": [], - "exit_code": 23, - "final_init_exit_observed": true, - "final_launcher_exit_code": 23, - "forced_cleanup": false, - "generated_code_executed": false, - "init_exit_observed": true, - "mode": "gate_eof", - "outputs_stable": true, - "passed": true, - "probe_passed": true, - "registered_child_exit_observed": [], - "rootfs_digest": "sha256:a48b5c59b7ca4794e8a966ccb935040fb64920fb6c76ad50639e7d3ce0c99ac2", - "status_eof": true, - "status_sha256": "ba8237cb6e5fac4df117c9c3a8c90e860c2c721bb932780d27c5db892c0e56a6", - "stderr_sha256": "5d527751db402beb0e72674bf9b4482c2b8d4806e7f35d4964a3870153bed26f", - "stdout_sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", - "stdout_bytes": 0, - "stderr_bytes": 23, - "private_report_sha256": "f5d9242611a91ec2e934a747d2c11a9fda5c3f5c5b22d2755871f033f3e10ae5" - }, - { - "authorization_sent": false, - "cleanup_complete": true, - "cleanup_or_evidence_errors": [], - "cleanup_registered_child_exits": [], - "exit_code": 23, - "final_init_exit_observed": true, - "final_launcher_exit_code": 23, - "forced_cleanup": false, - "generated_code_executed": false, - "init_exit_observed": true, - "mode": "gate_short", - "outputs_stable": true, - "passed": true, - "probe_passed": true, - "registered_child_exit_observed": [], - "rootfs_digest": "sha256:a48b5c59b7ca4794e8a966ccb935040fb64920fb6c76ad50639e7d3ce0c99ac2", - "status_eof": true, - "status_sha256": "216853e8324102617b31d84eec83a0c85064cfaab34a3a0fb99426d43a2a256b", - "stderr_sha256": "5d527751db402beb0e72674bf9b4482c2b8d4806e7f35d4964a3870153bed26f", - "stdout_sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", - "stdout_bytes": 0, - "stderr_bytes": 23, - "private_report_sha256": "1941ebf47713495200904ed589f817a5303009ce7f5f1582ee931bac8089561d" - }, - { - "authorization_sent": false, - "cleanup_complete": true, - "cleanup_or_evidence_errors": [], - "cleanup_registered_child_exits": [], - "exit_code": 23, - "final_init_exit_observed": true, - "final_launcher_exit_code": 23, - "forced_cleanup": false, - "generated_code_executed": false, - "init_exit_observed": true, - "mode": "gate_wrong", - "outputs_stable": true, - "passed": true, - "probe_passed": true, - "registered_child_exit_observed": [], - "rootfs_digest": "sha256:a48b5c59b7ca4794e8a966ccb935040fb64920fb6c76ad50639e7d3ce0c99ac2", - "status_eof": true, - "status_sha256": "1994b35c837a9d58cb0b6869ae194926405f2150abe589cda0fe2bda31f0fff3", - "stderr_sha256": "5d527751db402beb0e72674bf9b4482c2b8d4806e7f35d4964a3870153bed26f", - "stdout_sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", - "stdout_bytes": 0, - "stderr_bytes": 23, - "private_report_sha256": "6c87bb228fd8045f92d793f367c77785e8fa83c74f704fe602db9f2623e0eb1e" - }, - { - "authorization_sent": false, - "cleanup_complete": true, - "cleanup_or_evidence_errors": [], - "cleanup_registered_child_exits": [], - "exit_code": 23, - "final_init_exit_observed": true, - "final_launcher_exit_code": 23, - "forced_cleanup": false, - "generated_code_executed": false, - "init_exit_observed": true, - "mode": "gate_timeout", - "outputs_stable": true, - "passed": true, - "probe_passed": true, - "registered_child_exit_observed": [], - "rootfs_digest": "sha256:a48b5c59b7ca4794e8a966ccb935040fb64920fb6c76ad50639e7d3ce0c99ac2", - "status_eof": true, - "status_sha256": "61a385367ea80e39d29fe4df482c645f0a90259dcd8fec7da526a7fed6e71b2b", - "stderr_sha256": "5d527751db402beb0e72674bf9b4482c2b8d4806e7f35d4964a3870153bed26f", - "stdout_sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", - "stdout_bytes": 0, - "stderr_bytes": 23, - "private_report_sha256": "b58ad1deb0d1ee3f69e9a0769a2ec214dd6913989856116b9dbb34e71af84899" - }, - { - "authorization_sent": false, - "cleanup_complete": true, - "cleanup_or_evidence_errors": [], - "cleanup_registered_child_exits": [], - "exit_code": -9, - "final_init_exit_observed": true, - "final_launcher_exit_code": -9, - "forced_cleanup": false, - "generated_code_executed": false, - "init_exit_observed": true, - "mode": "gate_early_launcher_death", - "outputs_stable": true, - "passed": true, - "probe_passed": true, - "registered_child_exit_observed": [], - "rootfs_digest": "sha256:a48b5c59b7ca4794e8a966ccb935040fb64920fb6c76ad50639e7d3ce0c99ac2", - "status_eof": true, - "status_sha256": "0c5fc7182a02643564dc2d3b75e02a7794fde3a69278e3b3ced78e68bfdc5293", - "stderr_sha256": "5d527751db402beb0e72674bf9b4482c2b8d4806e7f35d4964a3870153bed26f", - "stdout_sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", - "stdout_bytes": 0, - "stderr_bytes": 23, - "private_report_sha256": "dd3a2a83e7cf30774e004f0827523dc847178ded941c40c9210fd936411ec770" - }, - { - "authorization_sent": true, - "cleanup_complete": true, - "cleanup_or_evidence_errors": [], - "cleanup_registered_child_exits": [], - "exit_code": 0, - "final_init_exit_observed": true, - "final_launcher_exit_code": 0, - "forced_cleanup": false, - "generated_code_executed": false, - "init_exit_observed": true, - "mode": "bounds", - "outputs_stable": true, - "passed": true, - "probe_passed": true, - "registered_child_exit_observed": [], - "rootfs_digest": "sha256:a48b5c59b7ca4794e8a966ccb935040fb64920fb6c76ad50639e7d3ce0c99ac2", - "status_eof": true, - "status_sha256": "861543a3935c257cf434bbf63db68665e5cf8a9bd3cf5ab3f8d93f3a7c6d6f6d", - "stderr_sha256": "3df969ed24948f490ec11adea8475939c24c5964ef12f5834eac38c4ba3cebd4", - "stdout_sha256": "5bd0021582e0eb7d8c3c994fad48feb144c3a613ca3ede420d8b87be669bf439", - "stdout_bytes": 1510, - "stderr_bytes": 25, - "private_report_sha256": "aae41a495bb62d04ce5b2ab9b14a88a6832fa15d80830ca4c4493c9479769f2e" - }, - { - "authorization_sent": true, - "cleanup_complete": true, - "cleanup_or_evidence_errors": [], - "cleanup_registered_child_exits": [], - "exit_code": 0, - "final_init_exit_observed": true, - "final_launcher_exit_code": 0, - "forced_cleanup": false, - "generated_code_executed": false, - "init_exit_observed": true, - "mode": "synthetic", - "outputs_stable": true, - "passed": true, - "probe_passed": true, - "registered_child_exit_observed": [], - "rootfs_digest": "sha256:a48b5c59b7ca4794e8a966ccb935040fb64920fb6c76ad50639e7d3ce0c99ac2", - "status_eof": true, - "status_sha256": "5f1eaef936ec3022ad1806ae66bbec1c2fe7a74ac25e21e185ad5490418c6fac", - "stderr_sha256": "3df969ed24948f490ec11adea8475939c24c5964ef12f5834eac38c4ba3cebd4", - "stdout_sha256": "09ab71530d00650abdebf228ee99106e18481dd00e73eb35839186f8fd11a481", - "stdout_bytes": 3710, - "stderr_bytes": 25, - "private_report_sha256": "f2e1cb7a0525aaecd701d56bce99e0489391d27f8636e1ca65876eda06e504a6" - }, - { - "authorization_sent": true, - "cleanup_complete": true, - "cleanup_or_evidence_errors": [], - "cleanup_registered_child_exits": [ - true, - true - ], - "exit_code": 0, - "final_init_exit_observed": true, - "final_launcher_exit_code": 0, - "forced_cleanup": false, - "generated_code_executed": false, - "init_exit_observed": true, - "mode": "normal", - "outputs_stable": true, - "passed": true, - "probe_passed": true, - "registered_child_exit_observed": [ - true, - true - ], - "rootfs_digest": "sha256:a48b5c59b7ca4794e8a966ccb935040fb64920fb6c76ad50639e7d3ce0c99ac2", - "status_eof": true, - "status_sha256": "0e82d4f546b05034d263bc6c1fbc8dd394d804c4271af0e4c32702719074dea9", - "stderr_sha256": "3df969ed24948f490ec11adea8475939c24c5964ef12f5834eac38c4ba3cebd4", - "stdout_sha256": "37be20ce6e8e3955fca705a806074cfbf681f9160986bd0c0bdb697df2ed1c4f", - "stdout_bytes": 170, - "stderr_bytes": 25, - "private_report_sha256": "01550d88abe3d0eee4050049b4ed1a1d07a72cc8898431fed91aefd1236f8f8f" - }, - { - "authorization_sent": true, - "cleanup_complete": true, - "cleanup_or_evidence_errors": [], - "cleanup_registered_child_exits": [ - true, - true - ], - "exit_code": 137, - "final_init_exit_observed": true, - "final_launcher_exit_code": 137, - "forced_cleanup": false, - "generated_code_executed": false, - "init_exit_observed": true, - "mode": "timeout", - "outputs_stable": true, - "passed": true, - "probe_passed": true, - "registered_child_exit_observed": [ - true, - true - ], - "rootfs_digest": "sha256:a48b5c59b7ca4794e8a966ccb935040fb64920fb6c76ad50639e7d3ce0c99ac2", - "status_eof": true, - "status_sha256": "d2812b886ec2b48dc73e573dcf0014ea9223f9d8b6e48a9f4e585e2ecdb9a7bf", - "stderr_sha256": "3df969ed24948f490ec11adea8475939c24c5964ef12f5834eac38c4ba3cebd4", - "stdout_sha256": "b7cce746eb320438693ac4b25b78e0737d81df3a0642996a2da9b38673119007", - "timeout_observed": true, - "stdout_bytes": 171, - "stderr_bytes": 25, - "private_report_sha256": "b4c4919af18896ff3216786116c777d1a048cdfee9b28725332af07097e8af69" - }, - { - "authorization_sent": true, - "cleanup_complete": true, - "cleanup_or_evidence_errors": [], - "cleanup_registered_child_exits": [ - true, - true - ], - "exit_code": -9, - "final_init_exit_observed": true, - "final_launcher_exit_code": -9, - "forced_cleanup": false, - "generated_code_executed": false, - "init_exit_observed": true, - "mode": "launcher_kill", - "outputs_stable": true, - "passed": true, - "probe_passed": true, - "registered_child_exit_observed": [ - true, - true - ], - "rootfs_digest": "sha256:a48b5c59b7ca4794e8a966ccb935040fb64920fb6c76ad50639e7d3ce0c99ac2", - "status_eof": true, - "status_sha256": "8a64c2c67caf277a051126c3803e8368dc425f326cdabfd3bdedc6c810cad894", - "stderr_sha256": "3df969ed24948f490ec11adea8475939c24c5964ef12f5834eac38c4ba3cebd4", - "stdout_sha256": "9f5f89cb1c2c064b9a1e3424a0cb59d9bea9d0e498c32d32a7315b63c0811272", - "stdout_bytes": 177, - "stderr_bytes": 25, - "private_report_sha256": "c1d4d083254ba057b20960e1ed7bcd6be34ace96627c8004815a793721c825da" - } - ], - "source_sha256": { - "runtime.py": "997cbfed8398e8ad62d1beb4f6b4bc6e61a0f4572b9156cfd0c6f3fb52b03233", - "bounds_driver.py": "bfeeb6ea794c0ebf24e72a0127458ff74951253471daee080b3a3b1d82a062ba", - "synthetic_probe.py": "236920f27573447b9f3833afb72739033c965e22a9e406b09f28da815529ca64", - "run-synthetic.sbatch": "8c0a3b5699578fe854a3b8f3c9e225d999f5f313a78151310e409bd31791c0f5" - }, - "actual_pidfd_capability": { - "child_exit_code": -9, - "child_pidfd_exit_observed": true, - "cleanup_errors": [], - "driver_sha256": "bfeeb6ea794c0ebf24e72a0127458ff74951253471daee080b3a3b1d82a062ba", - "final_child_exit_code": -9, - "generated_code_executed": false, - "implementation": "linux-x86_64-lp64-raw-syscalls-434-and-424", - "job_id": "2014", - "kernel_release": "5.10.134-16.3.al8.x86_64", - "own_fd_not_exited": true, - "passed": true, - "python_binary_sha256": "202c17d1671602a4ef1d43e9b2fdbef0769443f37bf5e51f6b603e0b2c27d9d8", - "python_exposes_pidfd_open": false, - "python_exposes_pidfd_send_signal": false, - "schema": 1 - }, - "capability_raw_sha256": "25687bed9a54effddfb90a492f2b23c43e2bb525bede7e03af0e47a57227262e", - "private_summary_sha256": "cc01340e0e280f56f8e03a21ea9ab8819ab4e8f873ea6eec2ffd2f0c2616ac19", - "rootfs_digest": "sha256:a48b5c59b7ca4794e8a966ccb935040fb64920fb6c76ad50639e7d3ce0c99ac2", - "rootfs_json_sha256": "8be9a74499696b17e77a27af30df9e92b459801fe716f9ffc8f69e0c67a02b4f", - "rootfs_before_after_equal": true, - "readonly_entry_count": 25241, - "package_count": 97, - "checked_evaluator_python_sources": 43, - "postflight_checks_passed": 10, - "bounds_checks_passed": 17, - "synthetic_statuses": { - "Synthetic/correct": [ - [ - "pass", - "pass" - ] - ], - "Synthetic/plus_wrong": [ - [ - "pass", - "fail" - ] - ], - "Synthetic/exception": [ - [ - "fail", - "fail" - ] - ], - "Synthetic/input_timeout": [ - [ - "fail", - "fail" - ] - ] - }, - "synthetic_evaluator_seconds": 11.390425316989422, - "serialized_results_sha256": "f544bfd3ff3d5cb8e25d4540b6c677ca2ed9a09af1462ed66b7cba2f85313718", - "generated_code_executed": false, - "benchmark_data_mounted": false, - "full_evaluator_completed": true, - "qualified_generated_code_executor": false, - "measured_e2e_effect": null, - "private_evidence_sha256": { - "submission.private.txt": "ca3a4a32f6060d1a9ddd731b8447242fb73bb59a0b160a62bea3e83f394d3c03", - "scontrol-final.private.txt": "0555457be7b3322e8bdfbd365bfb3e3291edbe51cc29ccb1955293f21e06efe7", - "squeue-final.private.txt": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", - "diagnostic.private.log": "848014c5c9675801865308a4f81ce3b37b9ca04bdd74c8e0fdcd1bf57b6d540f", - "output.private.tar": "82fe92b12abd7700d8b14273ad11aede3d7b515dcb3120cc548f80e5ce8d82e5" - }, - "readiness_gpu_seconds_used": 2388, - "readiness_gpu_seconds_cap": 7200, - "earlier_diagnostic_gpu_seconds_separate": 147 -} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-bounds-v1-impact.md b/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-bounds-v1-impact.md deleted file mode 100644 index f8564fe73..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-bounds-v1-impact.md +++ /dev/null @@ -1,44 +0,0 @@ -# Missing pidfd API exposes a startup-gate and evidence race - -Refs [Issue #67](https://github.com/InferMatrixLab/InferRSI/issues/67). -**Diagnostic FAIL; campaign, task quality and E2E INCONCLUSIVE.** - -CPU2012 failed2:0 in81seconds on4CPUs/zeroGPUs; allocation released. Each of -five supervisor cases encountered AttributeError because the registered Python -does not expose os.pidfd_open. This establishes an absent Python API; kernel -pidfd syscall availability remains untested. The owned launchers were killed and -reaped with code-9, but namespace PID1 was never bound or observed exiting. - -Full raw collection then disproved the initial live-report inference that no -payload ran. Closing the block pipe during failure cleanup permits bubblewrap's -ignored read result to reach EOF and continue setup. Actual late payload output -exists after supervisor failure: the bounds case reports17 true checks, normal -and timeout announce two children, and launcher-kill announces one child plus a -fork EAGAIN traceback. The synthetic evaluator also reaches worker creation and -fails fork with EAGAIN; no completed four-case score is retained. These late -observations are not accepted as a correctly supervised policy or cleanup test. -Thread-creation RuntimeError and EAGAIN are retained without assigning a sole -resource cause; concurrent late payloads may affect UID-scoped task accounting. - -All five supervisor reports hashed stdout as empty. Four final stdout files are -nonempty and disagree with those digests; synthetic stdout remains empty with a -nonempty error log. Original reports, final output bytes and both identities are -preserved. The first local recording assertion caught the mismatch before any -incorrect public result or KnowledgeStore update. The live commentary was -corrected once the complete archive was inspected. This is a failed startup and -output-stability protocol, not merely a Python compatibility failure. - -All10 postflight checks pass; before/after25,241-entry readonly inventories are -identical, with97packages and43 checked evaluator source files. Executed sources -match the reviewed proposal. No environment change, heldout task, model-generated -program or measured E2E effect occurred. Preparation failures and rejected drafts -remain retained, including the local-control capture conflict. - -A subsequent proposal must check actual pidfd capability before launching the -sandbox and use a payload handshake that requires an explicit valid token; EOF -must reject execution. It must observe namespace exit before hashing final output. -Direct Linux bindings, if used, must preserve stable pidfd identity and explicit -ABI checks. No qualification or candidate-search activation follows this failure. -Readiness GPU usage remains2388/7200seconds; initial147seconds remain separate. - -KnowledgeStore candidate `case.qwen38-eval-sandbox-preparation-cand` revision `sha256:acd39f49c345c3162d4160fbb0f780fb8f0682b6cc15d0a42dc63f1b8cbfd691`; readback verified. diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-bounds-v1-late-bounds-stdout.json b/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-bounds-v1-late-bounds-stdout.json deleted file mode 100644 index dc69b63da..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-bounds-v1-late-bounds-stdout.json +++ /dev/null @@ -1 +0,0 @@ -{"as_errno": 12, "checks": {"address_space_denied": true, "bounded_fork_denied": true, "bounded_thread_denied": true, "cpu_limit_kills": true, "descriptor_limit_denied": true, "device_mount_readonly": true, "device_read_available": true, "exact_limits": true, "file_size_denied": true, "fork_exec_filter_inheritance": true, "fork_exec_inheritance": true, "no_effective_capabilities": true, "no_new_privileges": true, "nonroot": true, "raise_hard_limit_denied": true, "seccomp_filter_present": true, "sysv_creation_denied": true}, "cpu_exit_code": -9, "descriptor_errno": 24, "device_creation_errno": 30, "file_size_errno": 27, "forks": {"attempt_cap": 17, "created": 11, "rejection_errno": 11}, "generated_code_executed": false, "hard_limit_raise_results": {"AS": "ValueError", "CORE": "ValueError", "CPU": "ValueError", "FSIZE": "ValueError", "MEMLOCK": "ValueError", "MSGQUEUE": "ValueError", "NOFILE": "ValueError", "NPROC": "ValueError"}, "inherited_exit_code": 0, "ipc_attempts": [{"errno": 1, "return": -1, "syscall": 29}, {"errno": 1, "return": -1, "syscall": 64}, {"errno": 1, "return": -1, "syscall": 68}, {"errno": 1, "return": -1, "syscall": 240}], "limits": {"AS": [2147483648, 2147483648], "CORE": [0, 0], "CPU": [30, 30], "FSIZE": [1048576, 1048576], "MEMLOCK": [0, 0], "MSGQUEUE": [0, 0], "NOFILE": [64, 64], "NPROC": [16, 16]}, "mode": "bounds", "opened_descriptors": 61, "passed": true, "threads": {"all_joined": true, "attempt_cap": 17, "created": 11, "rejection": "can't start new thread"}} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-bounds-v1-late-launcher-stderr.txt b/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-bounds-v1-late-launcher-stderr.txt deleted file mode 100644 index bc6c9bb03..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-bounds-v1-late-launcher-stderr.txt +++ /dev/null @@ -1,11 +0,0 @@ -Traceback (most recent call last): - File "/opt/probe.py", line 450, in - raise SystemExit(main()) - ^^^^^^ - File "/opt/probe.py", line 445, in main - return cleanup(mode) - ^^^^^^^^^^^^^ - File "/opt/probe.py", line 289, in cleanup - grandchild = os.fork() - ^^^^^^^^^ -BlockingIOError: [Errno 11] Resource temporarily unavailable diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-bounds-v1-late-synthetic-stderr.txt b/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-bounds-v1-late-synthetic-stderr.txt deleted file mode 100644 index 16d0ffda9..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-bounds-v1-late-synthetic-stderr.txt +++ /dev/null @@ -1,32 +0,0 @@ -Traceback (most recent call last): - File "/opt/probe.py", line 450, in - raise SystemExit(main()) - ^^^^^^ - File "/opt/probe.py", line 441, in main - return synthetic_evaluator() - ^^^^^^^^^^^^^^^^^^^^^ - File "/opt/probe.py", line 373, in synthetic_evaluator - evaluator.evaluate( - File "/opt/site-packages/evalplus/evaluate.py", line 238, in evaluate - futures.append(executor.submit(check_correctness, *args)) - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - File "/opt/python/lib/python3.12/concurrent/futures/process.py", line 823, in submit - self._start_executor_manager_thread() - File "/opt/python/lib/python3.12/concurrent/futures/process.py", line 762, in _start_executor_manager_thread - self._launch_processes() - File "/opt/python/lib/python3.12/concurrent/futures/process.py", line 789, in _launch_processes - self._spawn_process() - File "/opt/python/lib/python3.12/concurrent/futures/process.py", line 799, in _spawn_process - p.start() - File "/opt/python/lib/python3.12/multiprocessing/process.py", line 121, in start - self._popen = self._Popen(self) - ^^^^^^^^^^^^^^^^^ - File "/opt/python/lib/python3.12/multiprocessing/context.py", line 282, in _Popen - return Popen(process_obj) - ^^^^^^^^^^^^^^^^^^ - File "/opt/python/lib/python3.12/multiprocessing/popen_fork.py", line 19, in __init__ - self._launch(process_obj) - File "/opt/python/lib/python3.12/multiprocessing/popen_fork.py", line 66, in _launch - self.pid = os.fork() - ^^^^^^^^^ -BlockingIOError: [Errno 11] Resource temporarily unavailable diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-bounds-v1-postflight-status.tsv b/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-bounds-v1-postflight-status.tsv deleted file mode 100644 index 04d262973..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-bounds-v1-postflight-status.tsv +++ /dev/null @@ -1,10 +0,0 @@ -inventory 0 -runtime-byte-comparison 0 -runtime-script 0 -bounds-driver 0 -synthetic-probe 0 -batch-script 0 -freeze-manifest 0 -freeze-files 0 -python-binary 0 -bwrap-binary 0 diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-bounds-v1-summary.json b/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-bounds-v1-summary.json deleted file mode 100644 index 48047465d..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-bounds-v1-summary.json +++ /dev/null @@ -1 +0,0 @@ -{"cases":[{"cleanup_or_evidence_errors":[],"cleanup_registered_child_exits":[],"error":{"message":"module 'os' has no attribute 'pidfd_open'","type":"AttributeError"},"final_launcher_exit_code":-9,"forced_cleanup":true,"generated_code_executed":false,"mode":"bounds","passed":false,"rootfs_digest":"sha256:8035176c31d635366f5d215917c90c5f3f76bbc8d7ebfea41c1dafc3e17a93be","status_sha256":"931e1f5b31a06f8b069547952be4904c9e463ae5b0c7e6c11ed60216d0e5b289","stdout_sha256":"e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"},{"cleanup_or_evidence_errors":[],"cleanup_registered_child_exits":[],"error":{"message":"module 'os' has no attribute 'pidfd_open'","type":"AttributeError"},"final_launcher_exit_code":-9,"forced_cleanup":true,"generated_code_executed":false,"mode":"synthetic","passed":false,"rootfs_digest":"sha256:8035176c31d635366f5d215917c90c5f3f76bbc8d7ebfea41c1dafc3e17a93be","status_sha256":"2e35c2b301ca390f92cf8ae99b261321bf10398854b521905177596695bd17bf","stdout_sha256":"e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"},{"cleanup_or_evidence_errors":[],"cleanup_registered_child_exits":[],"error":{"message":"module 'os' has no attribute 'pidfd_open'","type":"AttributeError"},"final_launcher_exit_code":-9,"forced_cleanup":true,"generated_code_executed":false,"mode":"normal","passed":false,"rootfs_digest":"sha256:8035176c31d635366f5d215917c90c5f3f76bbc8d7ebfea41c1dafc3e17a93be","status_sha256":"d97554e81eb422d060794b744e013a9933e5bc156d8af72d5a3f0511eee730f1","stdout_sha256":"e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"},{"cleanup_or_evidence_errors":[],"cleanup_registered_child_exits":[],"error":{"message":"module 'os' has no attribute 'pidfd_open'","type":"AttributeError"},"final_launcher_exit_code":-9,"forced_cleanup":true,"generated_code_executed":false,"mode":"timeout","passed":false,"rootfs_digest":"sha256:8035176c31d635366f5d215917c90c5f3f76bbc8d7ebfea41c1dafc3e17a93be","status_sha256":"a060830715844e8a0cbf0af28af6f725c10991604cb033539907efd175cd43a7","stdout_sha256":"e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"},{"cleanup_or_evidence_errors":[],"cleanup_registered_child_exits":[],"error":{"message":"module 'os' has no attribute 'pidfd_open'","type":"AttributeError"},"final_launcher_exit_code":-9,"forced_cleanup":true,"generated_code_executed":false,"mode":"launcher_kill","passed":false,"rootfs_digest":"sha256:8035176c31d635366f5d215917c90c5f3f76bbc8d7ebfea41c1dafc3e17a93be","status_sha256":"09d660a16e767e67c168b0c2006b452c0297205d1e2226fb83d17dd33152f69c","stdout_sha256":"e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"}],"driver_sha256":"86e5706deaec92b3b1a976382856062cb8d4d305def38d05df050af001664428","generated_code_executed":false,"passed":false,"probe_sha256":"e8492c07878e12f17be272a2b5dc873bae8413de1ebf7ab3f2fce1a3d9e9dbed","qualified_generated_code_executor":false,"runtime_sha256":"997cbfed8398e8ad62d1beb4f6b4bc6e61a0f4572b9156cfd0c6f3fb52b03233","scope":"trusted_kernel_boundary_and_cleanup_probes","verdict":"INCONCLUSIVE"} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-bounds-v2-actual-review.json b/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-bounds-v2-actual-review.json deleted file mode 100644 index c5e95c6f2..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-bounds-v2-actual-review.json +++ /dev/null @@ -1,21 +0,0 @@ -{ - "reviewer": "Codex independent review agent /root/review_fp8_graph", - "archive_sha256": "82fe92b12abd7700d8b14273ad11aede3d7b515dcb3120cc548f80e5ce8d82e5", - "archive_bytes": 8888320, - "blocking_findings": [], - "recommendation": "Publish scoped successful trusted diagnostic; no generated-code qualification, task-quality acceptance or E2E PASS.", - "checked": [ - "Slurm COMPLETED0:0/114seconds/4CPU/0GPU and released allocation", - "four executed source hashes match reviewed versions", - "all11 case report/output/status hashes and init/launcher/registered-descendant exit evidence", - "rejected startup cases have no payload output", - "1011serialized evaluator bytes match SHA and parsed four-case result and input hashes", - "25241entry before/after inventory,97packages,43EvalPlus Python sources and old frozen files", - "all10postflight checks zero" - ], - "limitations": [ - "Two killed-launcher modes have no final bubblewrap JSON exit record; exit conclusions rely on pidfd and owned-launcher reaping.", - "Thread RuntimeError kernel cause remains unclassified.", - "Fixed trusted diagnostics do not qualify general generated-code execution or model quality." - ] -} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-bounds-v2-bounds-stdout.json b/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-bounds-v2-bounds-stdout.json deleted file mode 100644 index dc69b63da..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-bounds-v2-bounds-stdout.json +++ /dev/null @@ -1 +0,0 @@ -{"as_errno": 12, "checks": {"address_space_denied": true, "bounded_fork_denied": true, "bounded_thread_denied": true, "cpu_limit_kills": true, "descriptor_limit_denied": true, "device_mount_readonly": true, "device_read_available": true, "exact_limits": true, "file_size_denied": true, "fork_exec_filter_inheritance": true, "fork_exec_inheritance": true, "no_effective_capabilities": true, "no_new_privileges": true, "nonroot": true, "raise_hard_limit_denied": true, "seccomp_filter_present": true, "sysv_creation_denied": true}, "cpu_exit_code": -9, "descriptor_errno": 24, "device_creation_errno": 30, "file_size_errno": 27, "forks": {"attempt_cap": 17, "created": 11, "rejection_errno": 11}, "generated_code_executed": false, "hard_limit_raise_results": {"AS": "ValueError", "CORE": "ValueError", "CPU": "ValueError", "FSIZE": "ValueError", "MEMLOCK": "ValueError", "MSGQUEUE": "ValueError", "NOFILE": "ValueError", "NPROC": "ValueError"}, "inherited_exit_code": 0, "ipc_attempts": [{"errno": 1, "return": -1, "syscall": 29}, {"errno": 1, "return": -1, "syscall": 64}, {"errno": 1, "return": -1, "syscall": 68}, {"errno": 1, "return": -1, "syscall": 240}], "limits": {"AS": [2147483648, 2147483648], "CORE": [0, 0], "CPU": [30, 30], "FSIZE": [1048576, 1048576], "MEMLOCK": [0, 0], "MSGQUEUE": [0, 0], "NOFILE": [64, 64], "NPROC": [16, 16]}, "mode": "bounds", "opened_descriptors": 61, "passed": true, "threads": {"all_joined": true, "attempt_cap": 17, "created": 11, "rejection": "can't start new thread"}} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-bounds-v2-impact.md b/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-bounds-v2-impact.md deleted file mode 100644 index 826b744ab..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-bounds-v2-impact.md +++ /dev/null @@ -1,51 +0,0 @@ -# Explicit startup authorization and stable output collection verified - -Refs [Issue #67](https://github.com/InferMatrixLab/InferRSI/issues/67). -**Fixed trusted diagnostic checks passed; campaign, model quality and E2E remain INCONCLUSIVE.** - -CPU2014 completed0:0 in114seconds on4CPUs/zeroGPUs; allocation released. -The actual registered Python exposes neither pidfd API, but the ABI-checked raw -Linux bindings opened stable descriptors, sent the trusted child SIGKILL, and -observed that child's exit. Capability evidence was bound to the same job, -source, Python binary and kernel before any bubblewrap launch. - -All11 fixed modes completed. The valid token admitted only the gate check; -EOF, short, wrong and held-open inputs returned23 with no stdout and the rejection -marker. Killing the launcher before setup release also produced no stdout and -the rejection marker, closing CPU2012's observed EOF path. Resource checks and -synthetic evaluation then ran with explicit authorization. Normal exit, timeout -and launcher death each confirmed namespace PID1 and two registered descendants -exited. Every case confirmed launcher exit and stable final output hashes that -match the collected archive, with no forced fallback cleanup or cleanup errors. - -All17 finite resource checks passed with the existing UID task limit16:11 forks -were created before EAGAIN, and11 threads before RuntimeError. The latter's -kernel cause remains unclassified. Limits and the narrow IPC filter survived a -fresh interpreter; readonly /dev remained readable, and file/address-space/CPU -rejections behaved as expected. These observations do not establish an aggregate -16GiB hard memory cap, exhaustive syscall confinement or production qualification. - -The pinned full EvalPlus API produced the four expected handwritten pairs: -correct pass/pass, plus-only error pass/fail, exception fail/fail, and input timeout -fail/fail. The evaluator call took11.390425316989422seconds and reported zero live -children afterward. Exact serialized result bytes are retained and hash-bound to -the parsed result. These are synthetic control scores, not HumanEval+ model scores. - -All10 postflight checks passed. The25,241-entry readonly inventory is identical -before/after, with97packages and43 checked evaluator Python sources; relative to -the prior benign reference, only the explicitly changed probe differs. Actual -sources match the independently reviewed proposal. The raw archive and private -PID/namespace identities remain retained; public records omit those identifiers. - -The failed local exclusive-file fixture and the review-rejected stability logic -remain preserved. Independent review identified that a live launcher could still -write after namespace init exit; the repaired implementation now requires both -writers to exit, and stops further cases on incomplete cleanup. CPU2012's actual -failure and late payload evidence remain unchanged in their original records. - -No heldout task, model-generated program, GPU or measured E2E change occurred. -Production executor integration and qualification, complete protocol freeze, -owner-confirmed statistics/retrieval gates and formal paired runs remain pending. -Readiness usage remains2388/7200GPUseconds; initial147seconds remain separate. - -KnowledgeStore candidate `case.qwen38-eval-sandbox-preparation-cand` revision `sha256:c57741dffc71bca7dd018bf3fa17dd760e4b52acb9665a7ea61cbc710eb9faf4`; readback verified. diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-bounds-v2-postflight-status.tsv b/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-bounds-v2-postflight-status.tsv deleted file mode 100644 index 04d262973..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-bounds-v2-postflight-status.tsv +++ /dev/null @@ -1,10 +0,0 @@ -inventory 0 -runtime-byte-comparison 0 -runtime-script 0 -bounds-driver 0 -synthetic-probe 0 -batch-script 0 -freeze-manifest 0 -freeze-files 0 -python-binary 0 -bwrap-binary 0 diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-bounds-v2-serialized-evaluator-results.json b/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-bounds-v2-serialized-evaluator-results.json deleted file mode 100644 index 96d0bf429..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-bounds-v2-serialized-evaluator-results.json +++ /dev/null @@ -1 +0,0 @@ -{"date": "2026-10-22 23:41", "hash": "34b23f353a9c3f4df6c2bb1cda4b3f35", "eval": {"Synthetic/correct": [{"task_id": "Synthetic/correct", "solution": "def f(x):\n return x + 1\n", "base_status": "pass", "plus_status": "pass", "base_fail_tests": [], "plus_fail_tests": []}], "Synthetic/plus_wrong": [{"task_id": "Synthetic/plus_wrong", "solution": "def f(x):\n return 1\n", "base_status": "pass", "plus_status": "fail", "base_fail_tests": [], "plus_fail_tests": [[1]]}], "Synthetic/exception": [{"task_id": "Synthetic/exception", "solution": "def f(x):\n raise ArithmeticError('synthetic')\n", "base_status": "fail", "plus_status": "fail", "base_fail_tests": [[0]], "plus_fail_tests": [[1]]}], "Synthetic/input_timeout": [{"task_id": "Synthetic/input_timeout", "solution": "import time\ndef f(x):\n time.sleep(5)\n return x + 1\n", "base_status": "fail", "plus_status": "fail", "base_fail_tests": [[0]], "plus_fail_tests": [[1]]}]}, "pass_at_k": {"base": {"pass@1": 0.5}, "plus": {"pass@1": 0.25}}} \ No newline at end of file diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-bounds-v2-synthetic-stdout.json b/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-bounds-v2-synthetic-stdout.json deleted file mode 100644 index e4df69fc0..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-bounds-v2-synthetic-stdout.json +++ /dev/null @@ -1 +0,0 @@ -{"checks": {"bounded_output": true, "dataset_hash": true, "four_exact_status_pairs": true, "timeout_override_absent": true}, "dataset_sha256": "8098a24b8e5bde365d9d5ae8c2cf0e61d7936f2e99a4411faa284a488c51d430", "elapsed_seconds": 11.390425316989422, "evaluator_module_sha256": "47f766df2dbb2280f7dd2c1040bfff0396c2b414a8d44b639424aa941bee7eab", "evaluator_stderr": "\r0it [00:00, ?it/s]\r4it [00:00, 1177.35it/s]\n\r 0%| | 0/4 [00:00=3.10", - "tag_rank": 866 - }, - { - "name": "aiohttp", - "version": "3.14.5", - "filename": "aiohttp-3.14.5-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", - "sha256": "20f085697d7e911f1f73c43ed03fafbed1e7121797e2eb5428efa80398060584", - "size": 1919630, - "source": "https://files.pythonhosted.org/packages/da/e4/aa172eb534b7f02f1f8ff1c3213347eaf3cf218db91a727c6863c22f1035/aiohttp-3.14.5-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", - "metadata_url": "https://pypi.org/pypi/aiohttp/3.14.5/json", - "metadata_sha256": "7deb2f0d6dc95c63a2b576236bbe3051f3c0b1d402bd5f59e7619c05684f499a", - "requires_python": ">=3.10", - "tag_rank": 5 - }, - { - "name": "aiosignal", - "version": "1.4.0", - "filename": "aiosignal-1.4.0-py3-none-any.whl", - "sha256": "053243f8b92b990551949e63930a839ff0cf0b0ebbe0597b0f3fb19e1a0fe82e", - "size": 7490, - "source": "https://files.pythonhosted.org/packages/fb/76/641ae371508676492379f16e2fa48f4e2c11741bd63c48be4b12a6b09cba/aiosignal-1.4.0-py3-none-any.whl", - "metadata_url": "https://pypi.org/pypi/aiosignal/1.4.0/json", - "metadata_sha256": "443bdf542505c93e62117e868b9984b0c5784d50ab5b40ad0ae36d600b2d75ae", - "requires_python": ">=3.9", - "tag_rank": 866 - }, - { - "name": "annotated-doc", - "version": "0.0.5", - "filename": "annotated_doc-0.0.5-py3-none-any.whl", - "sha256": "117bac03a25ede5df5440e855b32d556049ca169ead221505badf432fed4b101", - "size": 5302, - "source": "https://files.pythonhosted.org/packages/3e/30/e900b21425a860e195f32e37657aa1f7c7f2b1bfb26f03ca209b90933c06/annotated_doc-0.0.5-py3-none-any.whl", - "metadata_url": "https://pypi.org/pypi/annotated-doc/0.0.5/json", - "metadata_sha256": "1aab691a5f37dd292e6d93ab53f7da5d839f56ba605345f017022ec75eb643af", - "requires_python": ">=3.9", - "tag_rank": 866 - }, - { - "name": "annotated-types", - "version": "0.8.0", - "filename": "annotated_types-0.8.0-py3-none-any.whl", - "sha256": "f072f4d804ea359e4eaf198b1af7a8b0943881a87f31bb764f8bf219bb9419e0", - "size": 13427, - "source": "https://files.pythonhosted.org/packages/99/91/8acff4f5e50511b911bbccb72b8628a49c68ce14148cd9f6431094859a90/annotated_types-0.8.0-py3-none-any.whl", - "metadata_url": "https://pypi.org/pypi/annotated-types/0.8.0/json", - "metadata_sha256": "a6e8e5b10c8fd2f596bba53e015cd37f6279fb5cec26ce0c5e9968625be4fefa", - "requires_python": ">=3.10", - "tag_rank": 866 - }, - { - "name": "anthropic", - "version": "2.1.0", - "filename": "anthropic-2.1.0-py3-none-any.whl", - "sha256": "d776fa2ea9a07e20336a0668d1ef3a12357fdef75c895f54a050b3746b8e4de8", - "size": 2105866, - "source": "https://files.pythonhosted.org/packages/02/f6/6c205e7e7fd2e6087692422b286fac4a2557ca60996b797276f23e03b431/anthropic-2.1.0-py3-none-any.whl", - "metadata_url": "https://pypi.org/pypi/anthropic/2.1.0/json", - "metadata_sha256": "5fdded51faff270e460ba9ae2a9b586672f132366f27278ac5abd404d381bcde", - "requires_python": ">=3.10", - "tag_rank": 866 - }, - { - "name": "anyio", - "version": "4.15.1", - "filename": "anyio-4.15.1-py3-none-any.whl", - "sha256": "6152fdbbf9a77fdec97731721bebf7c4c44f7c29b424b0065826173efc7ed101", - "size": 132079, - "source": "https://files.pythonhosted.org/packages/12/b8/4bd346e22b28902df4d651910f5242c28d84e4a5c2435ca5c3f797ed7e2e/anyio-4.15.1-py3-none-any.whl", - "metadata_url": "https://pypi.org/pypi/anyio/4.15.1/json", - "metadata_sha256": "e2ee22fad2fab3e66be503cba9b41945607d3076c9b7aa49b6e2f95ac323efab", - "requires_python": ">=3.10", - "tag_rank": 866 - }, - { - "name": "appdirs", - "version": "1.4.4", - "filename": "appdirs-1.4.4-py2.py3-none-any.whl", - "sha256": "a841dacd6b99318a741b166adb07e19ee71a274450e68237b4650ca1055ab128", - "size": 9566, - "source": "https://files.pythonhosted.org/packages/3b/00/2344469e2084fb287c2e0b57b72910309874c3245463acd6cf5e3db69324/appdirs-1.4.4-py2.py3-none-any.whl", - "metadata_url": "https://pypi.org/pypi/appdirs/1.4.4/json", - "metadata_sha256": "04a6aa996829cd8daeb8fa992c5ee482afbea4a8d74dd4917add59c73b10cac3", - "requires_python": null, - "tag_rank": 866 - }, - { - "name": "attrs", - "version": "26.1.0", - "filename": "attrs-26.1.0-py3-none-any.whl", - "sha256": "c647aa4a12dfbad9333ca4e71fe62ddc36f4e63b2d260a37a8b83d2f043ac309", - "size": 67548, - "source": "https://files.pythonhosted.org/packages/64/b4/17d4b0b2a2dc85a6df63d1157e028ed19f90d4cd97c36717afef2bc2f395/attrs-26.1.0-py3-none-any.whl", - "metadata_url": "https://pypi.org/pypi/attrs/26.1.0/json", - "metadata_sha256": "a882bf941fb4a0bb748d0194df544121cc130f5fb15a710ab6b804c17d2899b7", - "requires_python": ">=3.9", - "tag_rank": 866 - }, - { - "name": "boto3", - "version": "1.43.117", - "filename": "boto3-1.43.117-py3-none-any.whl", - "sha256": "a5b6cd7e67653987fcf0e212187c16f7b58af2ae192e368f7212ddab7c05b25d", - "size": 140043, - "source": "https://files.pythonhosted.org/packages/f3/3b/22b05a545fb95983733d84681364c6b0a36afb0710d832f77732d1fc4c0b/boto3-1.43.117-py3-none-any.whl", - "metadata_url": "https://pypi.org/pypi/boto3/1.43.117/json", - "metadata_sha256": "fa73b26e117eeb6306c2d26b8fff58f02570423cdd6b3c24581c085b5f8bd497", - "requires_python": ">=3.10", - "tag_rank": 866 - }, - { - "name": "botocore", - "version": "1.43.117", - "filename": "botocore-1.43.117-py3-none-any.whl", - "sha256": "5920ef9b3002e1361846549078549d9154a6e0004c22853c01ce7b279d05d495", - "size": 16061765, - "source": "https://files.pythonhosted.org/packages/10/94/050e35c9febf6dd2fce4a76be587ae4a323964865700467d9d7b082f7133/botocore-1.43.117-py3-none-any.whl", - "metadata_url": "https://pypi.org/pypi/botocore/1.43.117/json", - "metadata_sha256": "b77f9b42b68f9cf114f6d97bfa2e16a46ce48a166b9d8b8c7c0e5fd11fcf5e87", - "requires_python": ">=3.10", - "tag_rank": 866 - }, - { - "name": "certifi", - "version": "2026.10.19", - "filename": "certifi-2026.10.19-py3-none-any.whl", - "sha256": "42d35340cc5b9ba78e818ddbff0c5b1c2b779de448a340fd1b989f5fa8670e1b", - "size": 136464, - "source": "https://files.pythonhosted.org/packages/c4/51/29046e48f693c54e5b7412216d447a1d8362c9653148d98c02cdf65fa733/certifi-2026.10.19-py3-none-any.whl", - "metadata_url": "https://pypi.org/pypi/certifi/2026.10.19/json", - "metadata_sha256": "5b6ba21c20c09f5e4ca52bf8251de25fafbdb56909c60e3ba44d4f2b6ef5d319", - "requires_python": ">=3.7", - "tag_rank": 866 - }, - { - "name": "cffi", - "version": "2.1.1", - "filename": "cffi-2.1.1-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", - "sha256": "c1453022f490d2459a11819d83ad1d586e9ff65a12ac3e705ffebd46d3685dcf", - "size": 221822, - "source": "https://files.pythonhosted.org/packages/b1/db/dceb9dd5b231e1da801793f8acc9f3c52a7e1afe40bb1aae37e02b0faad5/cffi-2.1.1-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", - "metadata_url": "https://pypi.org/pypi/cffi/2.1.1/json", - "metadata_sha256": "2255e530e3f80593a354e3d84a29966924d0e10309d1e3a6d50f845ab7d8df66", - "requires_python": ">=3.10", - "tag_rank": 16 - }, - { - "name": "charset-normalizer", - "version": "3.5.2", - "filename": "charset_normalizer-3.5.2-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", - "sha256": "3d31298449090ab8d47b7b1b2a555ff73cac7ed438a08b7ac160980c7ebed649", - "size": 257601, - "source": "https://files.pythonhosted.org/packages/7f/c5/38806a25ab5e65fc178f39affeda20858efafede2fce1ffc2556cfc9fe73/charset_normalizer-3.5.2-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", - "metadata_url": "https://pypi.org/pypi/charset-normalizer/3.5.2/json", - "metadata_sha256": "9fda300857ae6a0486b4cd2d2fec7817ed2d45657249a6dfc0c2bbd1a44267b6", - "requires_python": ">=3.7", - "tag_rank": 5 - }, - { - "name": "click", - "version": "8.5.0", - "filename": "click-8.5.0-py3-none-any.whl", - "sha256": "255bc9599cf7748b4b1a446ccc735421bd08a2ae529a8b88597d3de5664ee360", - "size": 125251, - "source": "https://files.pythonhosted.org/packages/58/50/6c0d534c5f134586a8e1ba4e330569e32f057e33372ae556463212fb4cd3/click-8.5.0-py3-none-any.whl", - "metadata_url": "https://pypi.org/pypi/click/8.5.0/json", - "metadata_sha256": "4e36adc9b46f837e10ce801b7a478f9146fefdfb287292b7668fc64b1474e41d", - "requires_python": ">=3.10", - "tag_rank": 866 - }, - { - "name": "cryptography", - "version": "50.0.2", - "filename": "cryptography-50.0.2-cp311-abi3-manylinux_2_28_x86_64.whl", - "sha256": "4061c0079120205fb760c58acab6443e217307dcf05e3702cf970e0689972856", - "size": 4753050, - "source": "https://files.pythonhosted.org/packages/1a/f1/b474e930c4d910328780e3940da76f5aa5cbc48ce1fc14e44d239d9ea9db/cryptography-50.0.2-cp311-abi3-manylinux_2_28_x86_64.whl", - "metadata_url": "https://pypi.org/pypi/cryptography/50.0.2/json", - "metadata_sha256": "7423990cc0948676ef47a71ea62a476ece4c76245e6bebd1c3a71f03c4ffc0f4", - "requires_python": "!=3.9.0,!=3.9.1,>=3.9", - "tag_rank": 101 - }, - { - "name": "datasets", - "version": "5.1.0", - "filename": "datasets-5.1.0-py3-none-any.whl", - "sha256": "86fd08b280485aab82894d2c81842c4151d93b575efea0054315104cb7bc3cec", - "size": 604432, - "source": "https://files.pythonhosted.org/packages/24/cc/7f624629846c709ef1c68aa61bfb8d0ae2638cbe265104a1c2f7a630949e/datasets-5.1.0-py3-none-any.whl", - "metadata_url": "https://pypi.org/pypi/datasets/5.1.0/json", - "metadata_sha256": "96dead343a196d7af06c6436f368ee61f460ed7f3caca2cfcfe98ebeaeec8dd2", - "requires_python": ">=3.10.0", - "tag_rank": 866 - }, - { - "name": "dill", - "version": "0.4.1", - "filename": "dill-0.4.1-py3-none-any.whl", - "sha256": "1e1ce33e978ae97fcfcff5638477032b801c46c7c65cf717f95fbc2248f79a9d", - "size": 120019, - "source": "https://files.pythonhosted.org/packages/1e/77/dc8c558f7593132cf8fefec57c4f60c83b16941c574ac5f619abb3ae7933/dill-0.4.1-py3-none-any.whl", - "metadata_url": "https://pypi.org/pypi/dill/0.4.1/json", - "metadata_sha256": "3f326608b5939dfc66291b30ac368c5667f082dbd307a6434428e02b9639ee02", - "requires_python": ">=3.9", - "tag_rank": 866 - }, - { - "name": "docstring-parser", - "version": "0.18.0", - "filename": "docstring_parser-0.18.0-py3-none-any.whl", - "sha256": "b3fcbed555c47d8479be0796ef7e19c2670d428d72e96da63f3a40122860374b", - "size": 22484, - "source": "https://files.pythonhosted.org/packages/a7/5f/ed01f9a3cdffbd5a008556fc7b2a08ddb1cc6ace7effa7340604b1d16699/docstring_parser-0.18.0-py3-none-any.whl", - "metadata_url": "https://pypi.org/pypi/docstring-parser/0.18.0/json", - "metadata_sha256": "d327fa6f57f9d666866df08051263cf35541797d2b615c489d23d26a937931bb", - "requires_python": ">=3.8", - "tag_rank": 866 - }, - { - "name": "filelock", - "version": "4.2.4", - "filename": "filelock-4.2.4-py3-none-any.whl", - "sha256": "301241438c57936ef93f4243a94f2559022268bfcf9b375a9a72eb0f8db8358e", - "size": 162475, - "source": "https://files.pythonhosted.org/packages/e0/1b/5e74289a1878e666bb4b65fe5805a04cafcac2aa5f519ef1ead67ef91c56/filelock-4.2.4-py3-none-any.whl", - "metadata_url": "https://pypi.org/pypi/filelock/4.2.4/json", - "metadata_sha256": "15f44ff22fbab63cb4a5d2a9e03df833b06e70f11fef3cb3eb2cbaaac8652617", - "requires_python": ">=3.11", - "tag_rank": 866 - }, - { - "name": "fire", - "version": "0.7.1", - "filename": "fire-0.7.1-py3-none-any.whl", - "sha256": "e43fd8a5033a9001e7e2973bab96070694b9f12f2e0ecf96d4683971b5ab1882", - "size": 115945, - "source": "https://files.pythonhosted.org/packages/e5/4c/93d0f85318da65923e4b91c1c2ff03d8a458cbefebe3bc612a6693c7906d/fire-0.7.1-py3-none-any.whl", - "metadata_url": "https://pypi.org/pypi/fire/0.7.1/json", - "metadata_sha256": "dbd772570436bbfa29560e16e2031d2e063880750837ece8f084af384c3c3ac8", - "requires_python": ">=3.7", - "tag_rank": 866 - }, - { - "name": "frozenlist", - "version": "1.8.0", - "filename": "frozenlist-1.8.0-cp312-cp312-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", - "sha256": "494a5952b1c597ba44e0e78113a7266e656b9794eec897b19ead706bd7074383", - "size": 242411, - "source": "https://files.pythonhosted.org/packages/6a/bd/d91c5e39f490a49df14320f4e8c80161cfcce09f1e2cde1edd16a551abb3/frozenlist-1.8.0-cp312-cp312-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", - "metadata_url": "https://pypi.org/pypi/frozenlist/1.8.0/json", - "metadata_sha256": "b12a7f8b816bfb2a70f2eb97140890efb08a234ab3b75841b0e89e197effa5ad", - "requires_python": ">=3.9", - "tag_rank": 5 - }, - { - "name": "fsspec", - "version": "2026.7.0", - "filename": "fsspec-2026.7.0-py3-none-any.whl", - "sha256": "b57ddbafedfaef7018c1ecab32aa200a9d7ca26b77965f64e48b70061249d279", - "size": 206583, - "source": "https://files.pythonhosted.org/packages/fd/3c/6a2bf344106328fd04963664a60b9bb6496fc25df8e962fcdc1367285fb9/fsspec-2026.7.0-py3-none-any.whl", - "metadata_url": "https://pypi.org/pypi/fsspec/2026.7.0/json", - "metadata_sha256": "55eeed4d2aa6c53010b8dc6ac67ce83f2620ef17203c5922f5a642f479b484b0", - "requires_python": ">=3.10", - "tag_rank": 866 - }, - { - "name": "google-ai-generativelanguage", - "version": "0.6.15", - "filename": "google_ai_generativelanguage-0.6.15-py3-none-any.whl", - "sha256": "5a03ef86377aa184ffef3662ca28f19eeee158733e45d7947982eb953c6ebb6c", - "size": 1327356, - "source": "https://files.pythonhosted.org/packages/7c/a3/67b8a6ff5001a1d8864922f2d6488dc2a14367ceb651bc3f09a947f2f306/google_ai_generativelanguage-0.6.15-py3-none-any.whl", - "metadata_url": "https://pypi.org/pypi/google-ai-generativelanguage/0.6.15/json", - "metadata_sha256": "851b4a96ed5be42c4cc07be604f4fb31471814ae206e6d86c15a117aaef1b1ca", - "requires_python": ">=3.7", - "tag_rank": 866 - }, - { - "name": "google-api-core", - "version": "2.33.0", - "filename": "google_api_core-2.33.0-py3-none-any.whl", - "sha256": "a2e22a0c1d0f03eafff1858b38cf46f832d5902b0c052235bf0ab8402929fbdc", - "size": 176462, - "source": "https://files.pythonhosted.org/packages/89/31/5056a347bb934ea04583c8b27916ef1501729c72638629545bce26ff4223/google_api_core-2.33.0-py3-none-any.whl", - "metadata_url": "https://pypi.org/pypi/google-api-core/2.33.0/json", - "metadata_sha256": "a71af8df6e57267ff9d52e599c006ca6c427d65d370fe2a41b05aac910c04b2b", - "requires_python": ">=3.10", - "tag_rank": 866 - }, - { - "name": "google-api-python-client", - "version": "2.201.0", - "filename": "google_api_python_client-2.201.0-py3-none-any.whl", - "sha256": "2d9bf1ba3f12eee8ed3d0f1791ce0605d163432f496baa72d3677faa2cf097d6", - "size": 16812656, - "source": "https://files.pythonhosted.org/packages/8b/c0/221313acdee33b8e5d3fe8a06ba398fc18cfd14627816764bb00318a6065/google_api_python_client-2.201.0-py3-none-any.whl", - "metadata_url": "https://pypi.org/pypi/google-api-python-client/2.201.0/json", - "metadata_sha256": "c9f55d48f7e85f8f089cb5fa1fcb8ce60cd393c0bbeb7179564f23ebaa5a2fc1", - "requires_python": ">=3.10", - "tag_rank": 866 - }, - { - "name": "google-auth", - "version": "2.62.0", - "filename": "google_auth-2.62.0-py3-none-any.whl", - "sha256": "4ff4319aeb4ad128409759d397a9fcafad126d0031d241cc0dd6b9a00b43e3f3", - "size": 267911, - "source": "https://files.pythonhosted.org/packages/eb/a3/87fe9b7203ec2e56ea6a57c75e6d632eaf9487c7f7978cdad1d801bee0af/google_auth-2.62.0-py3-none-any.whl", - "metadata_url": "https://pypi.org/pypi/google-auth/2.62.0/json", - "metadata_sha256": "1fa3d7f366e108c7bdabb0e0cd5bce66724edf377662fba762c13ae731e72eed", - "requires_python": ">=3.10", - "tag_rank": 866 - }, - { - "name": "google-auth-httplib2", - "version": "0.4.4", - "filename": "google_auth_httplib2-0.4.4-py3-none-any.whl", - "sha256": "bbe5d7b2401bb3a4017f4720e1e91bd273ab9a2bb60b84e65edbc0de127852da", - "size": 9542, - "source": "https://files.pythonhosted.org/packages/de/ea/27cf5d623efdc1cdb2632b4c872ac5092cd3895671ba1d5004e73c3f1a7c/google_auth_httplib2-0.4.4-py3-none-any.whl", - "metadata_url": "https://pypi.org/pypi/google-auth-httplib2/0.4.4/json", - "metadata_sha256": "89fb79d34f3795ecb483f4a1769fbca7f2051cd5796b72ec93490398dcea832d", - "requires_python": ">=3.10", - "tag_rank": 866 - }, - { - "name": "google-generativeai", - "version": "0.8.6", - "filename": "google_generativeai-0.8.6-py3-none-any.whl", - "sha256": "37a0eaaa95e5bbf888828e20a4a1b2c196cc9527d194706e58a68ff388aeb0fa", - "size": 155098, - "source": "https://files.pythonhosted.org/packages/97/0f/ef33b5bb71437966590c6297104c81051feae95d54b11ece08533ef937d3/google_generativeai-0.8.6-py3-none-any.whl", - "metadata_url": "https://pypi.org/pypi/google-generativeai/0.8.6/json", - "metadata_sha256": "7e51495df385a2964c01b45f2768bec24b98a119fb92c9d8ad3395ada5d09383", - "requires_python": ">=3.9", - "tag_rank": 866 - }, - { - "name": "googleapis-common-protos", - "version": "1.75.0", - "filename": "googleapis_common_protos-1.75.0-py3-none-any.whl", - "sha256": "961ed60399c457ceb0ee8f285a84c870aabc9c6a832b9d37bb281b5bebde43ed", - "size": 300631, - "source": "https://files.pythonhosted.org/packages/e7/c8/e2645aa8ed02fd4c7a2f59d68783b65b1f3cbdfe39a6308e156509d1fee8/googleapis_common_protos-1.75.0-py3-none-any.whl", - "metadata_url": "https://pypi.org/pypi/googleapis-common-protos/1.75.0/json", - "metadata_sha256": "f5dbafcdaffc42af0490ed434b803f583ac0abd4780e92f1ea78ee42b9635479", - "requires_python": ">=3.9", - "tag_rank": 866 - }, - { - "name": "grpcio", - "version": "1.84.1", - "filename": "grpcio-1.84.1-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", - "sha256": "474408c92b27c823ea6aab2315e2ea52d5a4e0040b37ce1e65e89d9149c48205", - "size": 7121552, - "source": "https://files.pythonhosted.org/packages/00/98/456ef643a796ba04ad737e39a820c27291c238db87f7d168355dbdaf7b4b/grpcio-1.84.1-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", - "metadata_url": "https://pypi.org/pypi/grpcio/1.84.1/json", - "metadata_sha256": "b530dd44e9c50116cd6e07349a91f2e2d004e29ebf010a758b3fd301424a0ef3", - "requires_python": ">=3.10", - "tag_rank": 16 - }, - { - "name": "grpcio-status", - "version": "1.71.2", - "filename": "grpcio_status-1.71.2-py3-none-any.whl", - "sha256": "803c98cb6a8b7dc6dbb785b1111aed739f241ab5e9da0bba96888aa74704cfd3", - "size": 14424, - "source": "https://files.pythonhosted.org/packages/67/58/317b0134129b556a93a3b0afe00ee675b5657f0155509e22fcb853bafe2d/grpcio_status-1.71.2-py3-none-any.whl", - "metadata_url": "https://pypi.org/pypi/grpcio-status/1.71.2/json", - "metadata_sha256": "c1afe60232c47426f0eaaa3d394b6eeebfaa7e888de9a6bf9655df494693eacc", - "requires_python": ">=3.9", - "tag_rank": 866 - }, - { - "name": "h11", - "version": "0.16.0", - "filename": "h11-0.16.0-py3-none-any.whl", - "sha256": "63cf8bbe7522de3bf65932fda1d9c2772064ffb3dae62d55932da54b31cb6c86", - "size": 37515, - "source": "https://files.pythonhosted.org/packages/04/4b/29cac41a4d98d144bf5f6d33995617b185d14b22401f75ca86f384e87ff1/h11-0.16.0-py3-none-any.whl", - "metadata_url": "https://pypi.org/pypi/h11/0.16.0/json", - "metadata_sha256": "97f4efb9cc00826f0408076983adc1a1e48e8f2cabf403aaf250f724e081440d", - "requires_python": ">=3.8", - "tag_rank": 866 - }, - { - "name": "hf-xet", - "version": "1.7.0", - "filename": "hf_xet-1.7.0-cp38-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", - "sha256": "2814a6e999d13464c4d679b788cc5d784eb5a4edfc638a31f10e9a11ab531ef8", - "size": 4212180, - "source": "https://files.pythonhosted.org/packages/a3/4c/2bf3b66c215d409655f28de1622393dde04c9461280d48c7924bb3b2decd/hf_xet-1.7.0-cp38-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", - "metadata_url": "https://pypi.org/pypi/hf-xet/1.7.0/json", - "metadata_sha256": "769c976e22c0d9561ef6fc650517dfd76d345377f980e9b1fc3dd4e4973d84b6", - "requires_python": ">=3.8", - "tag_rank": 208 - }, - { - "name": "httpcore", - "version": "1.0.9", - "filename": "httpcore-1.0.9-py3-none-any.whl", - "sha256": "2d400746a40668fc9dec9810239072b40b4484b640a8c38fd654a024c7a1bf55", - "size": 78784, - "source": "https://files.pythonhosted.org/packages/7e/f5/f66802a942d491edb555dd61e3a9961140fd64c90bce1eafd741609d334d/httpcore-1.0.9-py3-none-any.whl", - "metadata_url": "https://pypi.org/pypi/httpcore/1.0.9/json", - "metadata_sha256": "e26eca5c3d288fe9e6446af9bcbca1038ba688629d5c8a2886c47d74718d87a2", - "requires_python": ">=3.8", - "tag_rank": 866 - }, - { - "name": "httpcore2", - "version": "2.16.0", - "filename": "httpcore2-2.16.0-py3-none-any.whl", - "sha256": "5c4358319f2602d984ad93182c9fee5160f22e9da7babdda870a5e7c4e2dc34f", - "size": 85070, - "source": "https://files.pythonhosted.org/packages/af/c2/ce417c397afa5181f0e3c6ff01a4d01639414e1adef08730d97571297151/httpcore2-2.16.0-py3-none-any.whl", - "metadata_url": "https://pypi.org/pypi/httpcore2/2.16.0/json", - "metadata_sha256": "194dc38b07c5836c3793de7ce6a56f998798304de165a4857c52f6525a6dd9ab", - "requires_python": ">=3.11", - "tag_rank": 866 - }, - { - "name": "httplib2", - "version": "0.32.0", - "filename": "httplib2-0.32.0-py3-none-any.whl", - "sha256": "dc6705cacdf3fb0a2aba7629fa33c90fd93e30035db0c157325826be177e4816", - "size": 93148, - "source": "https://files.pythonhosted.org/packages/33/a0/550eec327e5f5c7b732531c489f5307efec41f047b0d703bd4ca1e5ad2db/httplib2-0.32.0-py3-none-any.whl", - "metadata_url": "https://pypi.org/pypi/httplib2/0.32.0/json", - "metadata_sha256": "c17bb7cf199943348c01829c67663cc1fbbcac04aad0e06eb9f7a963393d8c03", - "requires_python": ">=3.8", - "tag_rank": 866 - }, - { - "name": "httpx", - "version": "0.28.1", - "filename": "httpx-0.28.1-py3-none-any.whl", - "sha256": "d909fcccc110f8c7faf814ca82a9a4d816bc5a6dbfea25d6591d6985b8ba59ad", - "size": 73517, - "source": "https://files.pythonhosted.org/packages/2a/39/e50c7c3a983047577ee07d2a9e53faf5a69493943ec3f6a384bdc792deb2/httpx-0.28.1-py3-none-any.whl", - "metadata_url": "https://pypi.org/pypi/httpx/0.28.1/json", - "metadata_sha256": "293bfab7bf587fcd0825f1bd8eed8e6303285ba14801b15659853827cbe30796", - "requires_python": ">=3.8", - "tag_rank": 866 - }, - { - "name": "httpx2", - "version": "2.16.0", - "filename": "httpx2-2.16.0-py3-none-any.whl", - "sha256": "fd2c5d7909e8d86fe92d53020c310ca0e4e431b838c207c5754b9bbf0308d49a", - "size": 97880, - "source": "https://files.pythonhosted.org/packages/7b/3d/2e4bc4fd1ef9770871669adb348d612a6e4786b303a23f9935b81c48dd60/httpx2-2.16.0-py3-none-any.whl", - "metadata_url": "https://pypi.org/pypi/httpx2/2.16.0/json", - "metadata_sha256": "95a397422b57e91a6e9ee2dc062b31229cbed63a2e863f2c1a58ec5ec2555cf0", - "requires_python": ">=3.11", - "tag_rank": 866 - }, - { - "name": "huggingface-hub", - "version": "2.2.0", - "filename": "huggingface_hub-2.2.0-py3-none-any.whl", - "sha256": "1667f145dc56dc210d60966069397df9ecfca9607a5d43db88b308c89dae56b3", - "size": 839884, - "source": "https://files.pythonhosted.org/packages/eb/b0/0f7b430fd100b3a3b037fdbb314878200241082e607b3383c63d91a13a72/huggingface_hub-2.2.0-py3-none-any.whl", - "metadata_url": "https://pypi.org/pypi/huggingface-hub/2.2.0/json", - "metadata_sha256": "575a5634e27bb1bb964be8a436abf3bdce1cdde049ece448911be810d364fffb", - "requires_python": ">=3.10.0", - "tag_rank": 866 - }, - { - "name": "idna", - "version": "3.20", - "filename": "idna-3.20-py3-none-any.whl", - "sha256": "ab7ae7122974553370f0bdb919e1a960b2cd1bc1ef0276416d896db81c14582c", - "size": 69583, - "source": "https://files.pythonhosted.org/packages/58/a2/bb081bab032533a855d44de1d56f8e8426114ff1ba5d1f07a438a0a654f8/idna-3.20-py3-none-any.whl", - "metadata_url": "https://pypi.org/pypi/idna/3.20/json", - "metadata_sha256": "e7cbd26e23ce60f6cb3a151d2adf3b1b087bb8f10ac2d9d6db70fdccbf3bb041", - "requires_python": ">=3.9", - "tag_rank": 866 - }, - { - "name": "jiter", - "version": "0.17.0", - "filename": "jiter-0.17.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", - "sha256": "8c21265b251d99bbb40080d178a8953e35601d3a1564e05c4de4c0d2ca616797", - "size": 345025, - "source": "https://files.pythonhosted.org/packages/52/f8/b6e513ecbdf3b3cebe587c2279281ecf775b729a58cf4cc7bdf898ded029/jiter-0.17.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", - "metadata_url": "https://pypi.org/pypi/jiter/0.17.0/json", - "metadata_sha256": "1cb61cf33ea8e4b6dd80de66d275030013a45b5961e54bb41a3b101deffd6928", - "requires_python": ">=3.10", - "tag_rank": 16 - }, - { - "name": "jmespath", - "version": "1.1.0", - "filename": "jmespath-1.1.0-py3-none-any.whl", - "sha256": "a5663118de4908c91729bea0acadca56526eb2698e83de10cd116ae0f4e97c64", - "size": 20419, - "source": "https://files.pythonhosted.org/packages/14/2f/967ba146e6d58cf6a652da73885f52fc68001525b4197effc174321d70b4/jmespath-1.1.0-py3-none-any.whl", - "metadata_url": "https://pypi.org/pypi/jmespath/1.1.0/json", - "metadata_sha256": "de5bfed61841dea8dd10c05206276a42f526f10cd96a57717e24fa8d94f98111", - "requires_python": ">=3.9", - "tag_rank": 866 - }, - { - "name": "markdown-it-py", - "version": "4.2.0", - "filename": "markdown_it_py-4.2.0-py3-none-any.whl", - "sha256": "9f7ebbcd14fe59494226453aed97c1070d83f8d24b6fc3a3bcf9a38092641c4a", - "size": 91687, - "source": "https://files.pythonhosted.org/packages/b3/81/4da04ced5a082363ecfa159c010d200ecbd959ae410c10c0264a38cac0f5/markdown_it_py-4.2.0-py3-none-any.whl", - "metadata_url": "https://pypi.org/pypi/markdown-it-py/4.2.0/json", - "metadata_sha256": "62affba459c80b1279b5759735f7da3e9c7d24966086ff81ec0303c94512b966", - "requires_python": ">=3.10", - "tag_rank": 866 - }, - { - "name": "mdurl", - "version": "0.1.2", - "filename": "mdurl-0.1.2-py3-none-any.whl", - "sha256": "84008a41e51615a49fc9966191ff91509e3c40b939176e643fd50a5c2196b8f8", - "size": 9979, - "source": "https://files.pythonhosted.org/packages/b3/38/89ba8ad64ae25be8de66a6d463314cf1eb366222074cfda9ee839c56a4b4/mdurl-0.1.2-py3-none-any.whl", - "metadata_url": "https://pypi.org/pypi/mdurl/0.1.2/json", - "metadata_sha256": "c7baac0b363d92cf55f1f3a313f4bb0564bff27343e6a0a4e9d7ff03afc8a6c7", - "requires_python": ">=3.7", - "tag_rank": 866 - }, - { - "name": "multidict", - "version": "7.1.0", - "filename": "multidict-7.1.0-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", - "sha256": "fed6b7705d49dd07e5e0dd5f5c873fc44047e92d714299b13245b5fecac49d01", - "size": 483000, - "source": "https://files.pythonhosted.org/packages/7a/f3/374c0ab122bb98b1a62a8742b1e3f59563e4d609942005a4041861a0df67/multidict-7.1.0-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", - "metadata_url": "https://pypi.org/pypi/multidict/7.1.0/json", - "metadata_sha256": "88c3e0e2b3739fdd6797eed7b4b47cf7d49f494dd5aac9fd775f23fdafaf6f09", - "requires_python": ">=3.10", - "tag_rank": 5 - }, - { - "name": "multipledispatch", - "version": "1.0.0", - "filename": "multipledispatch-1.0.0-py3-none-any.whl", - "sha256": "0c53cd8b077546da4e48869f49b13164bebafd0c2a5afceb6bb6a316e7fb46e4", - "size": 12818, - "source": "https://files.pythonhosted.org/packages/51/c0/00c9809d8b9346eb238a6bbd5f83e846a4ce4503da94a4c08cb7284c325b/multipledispatch-1.0.0-py3-none-any.whl", - "metadata_url": "https://pypi.org/pypi/multipledispatch/1.0.0/json", - "metadata_sha256": "625ddf81c2e7152072e7338ce054e4d1a7a8f46a412fd4be30379471c3a87e82", - "requires_python": null, - "tag_rank": 866 - }, - { - "name": "multiprocess", - "version": "0.70.19", - "filename": "multiprocess-0.70.19-py312-none-any.whl", - "sha256": "3a56c0e85dd5025161bac5ce138dcac1e49174c7d8e74596537e729fd5c53c28", - "size": 150281, - "source": "https://files.pythonhosted.org/packages/71/70/38998b950a97ea279e6bd657575d22d1a2047256caf707d9a10fbce4f065/multiprocess-0.70.19-py312-none-any.whl", - "metadata_url": "https://pypi.org/pypi/multiprocess/0.70.19/json", - "metadata_sha256": "0543301c4a1c4760dc2d488d6f668e5d7de468545329103df853049a67c25e30", - "requires_python": ">=3.9", - "tag_rank": 865 - }, - { - "name": "numpy", - "version": "2.5.4", - "filename": "numpy-2.5.4-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", - "sha256": "fbde6962867ee75b48b0ee29b2b9372ec5d617799dbaf38e82dc0596f2f7738a", - "size": 16727283, - "source": "https://files.pythonhosted.org/packages/43/a3/c799c62e19c337e6d3770b08e475887fb30ce8477d3c09efca6b2f0228a6/numpy-2.5.4-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", - "metadata_url": "https://pypi.org/pypi/numpy/2.5.4/json", - "metadata_sha256": "a42b40d53fac5388e974f21aa1f01d3b47440ad8181fc827b08023bf8ca3cc96", - "requires_python": ">=3.12", - "tag_rank": 5 - }, - { - "name": "ollama", - "version": "0.6.3", - "filename": "ollama-0.6.3-py3-none-any.whl", - "sha256": "6a20bc42c1a5f889295d7ec490d35e5132fc31f339561530f43a8abd4dbfe508", - "size": 16603, - "source": "https://files.pythonhosted.org/packages/4d/64/87505d9e006461233c21c8e66dc1ecee49c996090584b216abd0dd4a8322/ollama-0.6.3-py3-none-any.whl", - "metadata_url": "https://pypi.org/pypi/ollama/0.6.3/json", - "metadata_sha256": "6a5eee469f483b98227fd7a69f8d3c409bf049fe9fe8067e90bb6be0dd2a4378", - "requires_python": ">=3.8", - "tag_rank": 866 - }, - { - "name": "openai", - "version": "3.36.1", - "filename": "openai-3.36.1-py3-none-any.whl", - "sha256": "4c15b6765fe1792c5ad753556f44c7f82c2920c5e9e40219c0fca97369e923d3", - "size": 2296265, - "source": "https://files.pythonhosted.org/packages/be/e4/83f139714d10d75af2b6891a20c973945dd9b942b11cc407ba4536fb40b9/openai-3.36.1-py3-none-any.whl", - "metadata_url": "https://pypi.org/pypi/openai/3.36.1/json", - "metadata_sha256": "bea64993df255209f88f779874a72216db076f258d6308970eeaf2c44aecaaf0", - "requires_python": ">=3.10", - "tag_rank": 866 - }, - { - "name": "opentelemetry-api", - "version": "1.45.1", - "filename": "opentelemetry_api-1.45.1-py3-none-any.whl", - "sha256": "b31553efa588ae44bc306f863c785c5333a9ecc091248c6ee68b4b6c87fdedfb", - "size": 60256, - "source": "https://files.pythonhosted.org/packages/1e/41/f7dcf80b81ee8e71c1a2b59f14208bc723edbd89ed027a73b175abf6348e/opentelemetry_api-1.45.1-py3-none-any.whl", - "metadata_url": "https://pypi.org/pypi/opentelemetry-api/1.45.1/json", - "metadata_sha256": "da574eb54158846f06ac0a51034378feb22c47be67856f001da7337f3a116d2c", - "requires_python": ">=3.10", - "tag_rank": 866 - }, - { - "name": "packaging", - "version": "26.3", - "filename": "packaging-26.3-py3-none-any.whl", - "sha256": "d7193f7c8e4e93f444fde0262bf90af30e16fa0ad0ad44cb553c87339b23cd1c", - "size": 129956, - "source": "https://files.pythonhosted.org/packages/63/34/ba1c580383c9eada3711951fef0795c80b829a078d72188184bcab9dd527/packaging-26.3-py3-none-any.whl", - "metadata_url": "https://pypi.org/pypi/packaging/26.3/json", - "metadata_sha256": "4f190d2f0b312b3086058cee64ab4333bf3ce67fb78b4c2bdb43899049b29f4d", - "requires_python": ">=3.9", - "tag_rank": 866 - }, - { - "name": "pandas", - "version": "3.1.0", - "filename": "pandas-3.1.0-cp312-cp312-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", - "sha256": "9433b5dc2f26ef9d9b56daaf4d41d078bf4372c5d88c42ed5b397a6eea80de16", - "size": 12019688, - "source": "https://files.pythonhosted.org/packages/c5/a6/dc89cdd95e12093bc813e9b814a09f1a056be3e1304edaef9bf9dbee108a/pandas-3.1.0-cp312-cp312-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", - "metadata_url": "https://pypi.org/pypi/pandas/3.1.0/json", - "metadata_sha256": "22fdd3bd46dfe287d02a70c640837ce39d81f95c453fb613f374c698b5d48129", - "requires_python": ">=3.11", - "tag_rank": 5 - }, - { - "name": "propcache", - "version": "0.5.4", - "filename": "propcache-0.5.4-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", - "sha256": "2814ecd8e818f487bee4b0f921bc4d1c176cc5fc71ac0f072d0fa67eda4ac14b", - "size": 250424, - "source": "https://files.pythonhosted.org/packages/85/9f/83a07b6ec0e043c050cfdd35fb0cf1b7897b91d554d6eea293740309afe7/propcache-0.5.4-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", - "metadata_url": "https://pypi.org/pypi/propcache/0.5.4/json", - "metadata_sha256": "253a90067f5c44ffec37f3f1081934c91fb60edda0dc1e9bdf805edc8cecf693", - "requires_python": ">=3.10", - "tag_rank": 5 - }, - { - "name": "proto-plus", - "version": "1.28.2", - "filename": "proto_plus-1.28.2-py3-none-any.whl", - "sha256": "b874236fcac2358f601e4330bcb76cb8b89c851303ccf4078408b3d4774d1c52", - "size": 50693, - "source": "https://files.pythonhosted.org/packages/9d/84/4e9a53a062d4073c74897a6bd20fff74d55307341b3e85c081002462b3ef/proto_plus-1.28.2-py3-none-any.whl", - "metadata_url": "https://pypi.org/pypi/proto-plus/1.28.2/json", - "metadata_sha256": "7072e7982033e7d77974d393d89997047ca2bf02dddf79eaee6648da6d7d9b8e", - "requires_python": ">=3.10", - "tag_rank": 866 - }, - { - "name": "protobuf", - "version": "5.29.6", - "filename": "protobuf-5.29.6-cp38-abi3-manylinux2014_x86_64.whl", - "sha256": "e3387f44798ac1106af0233c04fb8abf543772ff241169946f698b3a9a3d3ab9", - "size": 320478, - "source": "https://files.pythonhosted.org/packages/e3/dd/cadd6ec43069247d91f6345fa7a0d2858bef6af366dbd7ba8f05d2c77d3b/protobuf-5.29.6-cp38-abi3-manylinux2014_x86_64.whl", - "metadata_url": "https://pypi.org/pypi/protobuf/5.29.6/json", - "metadata_sha256": "4f849176afc9df638bee2b019d59b99c1ddcf19808b0ddfe610b6ea6aef78d67", - "requires_python": ">=3.8", - "tag_rank": 209 - }, - { - "name": "psutil", - "version": "7.2.2", - "filename": "psutil-7.2.2-cp36-abi3-manylinux2010_x86_64.manylinux_2_12_x86_64.manylinux_2_28_x86_64.whl", - "sha256": "076a2d2f923fd4821644f5ba89f059523da90dc9014e85f8e45a5774ca5bc6f9", - "size": 155560, - "source": "https://files.pythonhosted.org/packages/b5/70/5d8df3b09e25bce090399cf48e452d25c935ab72dad19406c77f4e828045/psutil-7.2.2-cp36-abi3-manylinux2010_x86_64.manylinux_2_12_x86_64.manylinux_2_28_x86_64.whl", - "metadata_url": "https://pypi.org/pypi/psutil/7.2.2/json", - "metadata_sha256": "84f9d2597c3c15fe19d1ce069bfdbf25951dc423dc51d67c3979468c029b6415", - "requires_python": ">=3.6", - "tag_rank": 261 - }, - { - "name": "pyarrow", - "version": "26.0.0", - "filename": "pyarrow-26.0.0-cp312-cp312-manylinux_2_28_x86_64.whl", - "sha256": "6a628922ba20705fa964ca73e4ef959c2fb2f14b9bbec5589a6a1e68e6257c85", - "size": 53931932, - "source": "https://files.pythonhosted.org/packages/88/e4/9cd356d906e71bd79b0c3fc5c9a54e01a0020dcf14c152ccfbcb503c7298/pyarrow-26.0.0-cp312-cp312-manylinux_2_28_x86_64.whl", - "metadata_url": "https://pypi.org/pypi/pyarrow/26.0.0/json", - "metadata_sha256": "820e0086a99265aeb261219e687d72eae106dba6290a66683315b4fe85276be5", - "requires_python": ">=3.11", - "tag_rank": 5 - }, - { - "name": "pyasn1", - "version": "0.6.4", - "filename": "pyasn1-0.6.4-py3-none-any.whl", - "sha256": "deda9277cfd454080ec40b207fb6df82206a3a2688735233cdcd8d3d565f088b", - "size": 84410, - "source": "https://files.pythonhosted.org/packages/9a/3b/6163796d69c3977d1e4287bea4a6979161cbbdd170ebb430511e8e1999ce/pyasn1-0.6.4-py3-none-any.whl", - "metadata_url": "https://pypi.org/pypi/pyasn1/0.6.4/json", - "metadata_sha256": "cce3a342bc3f2027bcf079357de8886d17f1c058c5233b3f348b749a9ed2c38b", - "requires_python": ">=3.8", - "tag_rank": 866 - }, - { - "name": "pyasn1-modules", - "version": "0.4.2", - "filename": "pyasn1_modules-0.4.2-py3-none-any.whl", - "sha256": "29253a9207ce32b64c3ac6600edc75368f98473906e8fd1043bd6b5b1de2c14a", - "size": 181259, - "source": "https://files.pythonhosted.org/packages/47/8d/d529b5d697919ba8c11ad626e835d4039be708a35b0d22de83a269a6682c/pyasn1_modules-0.4.2-py3-none-any.whl", - "metadata_url": "https://pypi.org/pypi/pyasn1-modules/0.4.2/json", - "metadata_sha256": "5176a4663fb2345b3a0cd53e6d72d541fab642ed1ad815b1224c124f8287cc8e", - "requires_python": ">=3.8", - "tag_rank": 866 - }, - { - "name": "pycparser", - "version": "3.12", - "filename": "pycparser-3.12-py3-none-any.whl", - "sha256": "a076f3f489b39b43ed13563a1dafe42447445bec2e294154ec5c57fcdda4c85e", - "size": 62600, - "source": "https://files.pythonhosted.org/packages/b6/ae/315bbeee126a769cbde43cf4908acb4b40408d8bf8bbf334dd7368d8dadf/pycparser-3.12-py3-none-any.whl", - "metadata_url": "https://pypi.org/pypi/pycparser/3.12/json", - "metadata_sha256": "e919b291615d54a2328a0da6128cd9d95601ad04d27cec48eeb75e0d57c3b6ad", - "requires_python": ">=3.11", - "tag_rank": 866 - }, - { - "name": "pydantic", - "version": "2.14.1", - "filename": "pydantic-2.14.1-py3-none-any.whl", - "sha256": "9195d967ec791692a04438115466764fb8b9a27b31f14a760437694f40d6b454", - "size": 488106, - "source": "https://files.pythonhosted.org/packages/35/ea/a56b9fe5066f3537b7882f77e9c5dfb26d8c2eefdaed9b5fc73d57b4dc22/pydantic-2.14.1-py3-none-any.whl", - "metadata_url": "https://pypi.org/pypi/pydantic/2.14.1/json", - "metadata_sha256": "7413703dd60983a01d25e54730f9153c4d7a83fc788c77955b5237269cf9f071", - "requires_python": ">=3.10", - "tag_rank": 866 - }, - { - "name": "pydantic-core", - "version": "2.50.1", - "filename": "pydantic_core-2.50.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", - "sha256": "93ba4e9d8210d941c200431a56b2c0400b131865947903937ed3ec5404307d2e", - "size": 2160157, - "source": "https://files.pythonhosted.org/packages/90/24/48cd98388e4ac2c08d0800af36d15413db15e20387f1038123a2c86cb2f6/pydantic_core-2.50.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", - "metadata_url": "https://pypi.org/pypi/pydantic-core/2.50.1/json", - "metadata_sha256": "e93d77eaf72dda287409887550bda890a4067311ea072bf9b274df9ce6d96804", - "requires_python": ">=3.10", - "tag_rank": 16 - }, - { - "name": "pygments", - "version": "2.21.0", - "filename": "pygments-2.21.0-py3-none-any.whl", - "sha256": "2363c69b61c4a97c838da3b130dcd6468f4848992b21a82f2a63ec34377137d9", - "size": 1250147, - "source": "https://files.pythonhosted.org/packages/71/46/17f022dd3e953bf20a04a028a21ec746d942f8d2af30fa0f124fa0e6a684/pygments-2.21.0-py3-none-any.whl", - "metadata_url": "https://pypi.org/pypi/pygments/2.21.0/json", - "metadata_sha256": "330104a10adc8a587e3f98f4e207557125ca0c1d489e129334da5acb2b813a03", - "requires_python": ">=3.9", - "tag_rank": 866 - }, - { - "name": "pyparsing", - "version": "3.3.3", - "filename": "pyparsing-3.3.3-py3-none-any.whl", - "sha256": "ece8c00a69cf01b45d0b1dedabb469c90d8caf996d4fda40f147627a122849a4", - "size": 126420, - "source": "https://files.pythonhosted.org/packages/38/bb/d215ee7c73b61497b28a5503f9f53523f294fcc936762b7caf90e0c1c2b5/pyparsing-3.3.3-py3-none-any.whl", - "metadata_url": "https://pypi.org/pypi/pyparsing/3.3.3/json", - "metadata_sha256": "b851d5414eca50b38263f56c00a1fbec80383a26a3ad8e41a204b72cdeee5751", - "requires_python": ">=3.9", - "tag_rank": 866 - }, - { - "name": "python-dateutil", - "version": "2.9.0.post0", - "filename": "python_dateutil-2.9.0.post0-py2.py3-none-any.whl", - "sha256": "a8b2bc7bffae282281c8140a97d3aa9c14da0b136dfe83f850eea9a5f7470427", - "size": 229892, - "source": "https://files.pythonhosted.org/packages/ec/57/56b9bcc3c9c6a792fcbaf139543cee77261f3651ca9da0c93f5c1221264b/python_dateutil-2.9.0.post0-py2.py3-none-any.whl", - "metadata_url": "https://pypi.org/pypi/python-dateutil/2.9.0.post0/json", - "metadata_sha256": "f3a0d4d44ef5e71ff77e4dd013b39b1d1bd173fb04f8b0aecc83e5502c44d858", - "requires_python": "!=3.0.*,!=3.1.*,!=3.2.*,>=2.7", - "tag_rank": 866 - }, - { - "name": "pyyaml", - "version": "6.0.3", - "filename": "pyyaml-6.0.3-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", - "sha256": "ba1cc08a7ccde2d2ec775841541641e4548226580ab850948cbfda66a1befcdc", - "size": 807870, - "source": "https://files.pythonhosted.org/packages/8b/9d/b3589d3877982d4f2329302ef98a8026e7f4443c765c46cfecc8858c6b4b/pyyaml-6.0.3-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", - "metadata_url": "https://pypi.org/pypi/pyyaml/6.0.3/json", - "metadata_sha256": "c3f35597bc2f08cc990c2a5fe57bef6687b3a3d7c61d8b0ba4cc067777eb1def", - "requires_python": ">=3.8", - "tag_rank": 5 - }, - { - "name": "regex", - "version": "2026.9.29", - "filename": "regex-2026.9.29-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", - "sha256": "39ab5894d971f9ac68baa6eca5c50387db579cfcacf36ae8df3feceb1815e6d0", - "size": 807783, - "source": "https://files.pythonhosted.org/packages/26/83/d2fbd2e4e3afb1167daa825187d196f313cbaa1a4768f311fb041bb0e3d2/regex-2026.9.29-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", - "metadata_url": "https://pypi.org/pypi/regex/2026.9.29/json", - "metadata_sha256": "08fc1fa7daa8db0404cbbaea4eed71acc7798421386d3de50f92302068fe58c5", - "requires_python": ">=3.10", - "tag_rank": 5 - }, - { - "name": "requests", - "version": "2.34.2", - "filename": "requests-2.34.2-py3-none-any.whl", - "sha256": "2a0d60c172f83ac6ab31e4554906c0f3b3588d37b5cb939b1c061f4907e278e0", - "size": 73075, - "source": "https://files.pythonhosted.org/packages/a0/f4/c67b0b3f1b9245e8d266f0f112c500d50e5b4e83cb6f3b71b6528104182a/requests-2.34.2-py3-none-any.whl", - "metadata_url": "https://pypi.org/pypi/requests/2.34.2/json", - "metadata_sha256": "8622e50ffd295ed759ec793f2a009172a4ed0204bbc5e948de23111bc3d2aeae", - "requires_python": ">=3.10", - "tag_rank": 866 - }, - { - "name": "rich", - "version": "15.0.0", - "filename": "rich-15.0.0-py3-none-any.whl", - "sha256": "33bd4ef74232fb73fe9279a257718407f169c09b78a87ad3d296f548e27de0bb", - "size": 310654, - "source": "https://files.pythonhosted.org/packages/82/3b/64d4899d73f91ba49a8c18a8ff3f0ea8f1c1d75481760df8c68ef5235bf5/rich-15.0.0-py3-none-any.whl", - "metadata_url": "https://pypi.org/pypi/rich/15.0.0/json", - "metadata_sha256": "455015cb0b237a24819d91236152ad0e71b2be39a83cbe6afa0e29ea8aafa550", - "requires_python": ">=3.9.0", - "tag_rank": 866 - }, - { - "name": "s3transfer", - "version": "0.19.3", - "filename": "s3transfer-0.19.3-py3-none-any.whl", - "sha256": "24eb7535804306355e31c952a90022bf5b2f9d760f9a11062c7493a217c08553", - "size": 90252, - "source": "https://files.pythonhosted.org/packages/e2/a4/88f9e10498759121060b8050a1554753dfc04df39bbbcc7aad9ecc8d9440/s3transfer-0.19.3-py3-none-any.whl", - "metadata_url": "https://pypi.org/pypi/s3transfer/0.19.3/json", - "metadata_sha256": "05e5e0fe25ff2a54e775257f557bd60e2648803eb487e23945f1583820cb62e5", - "requires_python": ">=3.10", - "tag_rank": 866 - }, - { - "name": "safetensors", - "version": "0.8.0", - "filename": "safetensors-0.8.0-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", - "sha256": "fd6f3f93c9a0a7cc2788ee63fb763353d4bd2e89b0751bc78fcf7dda00bea774", - "size": 516040, - "source": "https://files.pythonhosted.org/packages/28/50/f203ff3a3ddfe19308efc83c5a3a29ed02bf786732ec35e68bf9162f3365/safetensors-0.8.0-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", - "metadata_url": "https://pypi.org/pypi/safetensors/0.8.0/json", - "metadata_sha256": "b8d395b433d0319498c9192ed323b86a84d82b5f40b4adf1b97b3bf8963fafad", - "requires_python": ">=3.10", - "tag_rank": 144 - }, - { - "name": "setuptools", - "version": "84.2.1", - "filename": "setuptools-84.2.1-py3-none-any.whl", - "sha256": "ca140b45d5325a9478f880e351484f62fef15f336462e449ca0c03c873d589c2", - "size": 876263, - "source": "https://files.pythonhosted.org/packages/ef/55/6dc6d6bb953bdfc52519a0059592c52b6e3b8ad51cf8256dffceb55b0610/setuptools-84.2.1-py3-none-any.whl", - "metadata_url": "https://pypi.org/pypi/setuptools/84.2.1/json", - "metadata_sha256": "d75de8ba0d3304a3d74484fb9f340edb87ef02712da4fa02a2ed92e71b0572dd", - "requires_python": ">=3.10", - "tag_rank": 866 - }, - { - "name": "setuptools-scm", - "version": "10.3.4", - "filename": "setuptools_scm-10.3.4-py3-none-any.whl", - "sha256": "82f34c3e3084fc2b57d397200637cc13f2338004052d2fee47baa5f0e902464e", - "size": 33366, - "source": "https://files.pythonhosted.org/packages/da/f5/54538a1f17ea753c42b0928dc2e62986d463f8c393340b9a1e602f2d4dd9/setuptools_scm-10.3.4-py3-none-any.whl", - "metadata_url": "https://pypi.org/pypi/setuptools-scm/10.3.4/json", - "metadata_sha256": "545356d9f16294b42b5eec3671fdab29c2b25af270f92da42ae0ce31d6ba4875", - "requires_python": ">=3.8", - "tag_rank": 866 - }, - { - "name": "shellingham", - "version": "1.5.4", - "filename": "shellingham-1.5.4-py2.py3-none-any.whl", - "sha256": "7ecfff8f2fd72616f7481040475a65b2bf8af90a56c89140852d1120324e8686", - "size": 9755, - "source": "https://files.pythonhosted.org/packages/e0/f9/0595336914c5619e5f28a1fb793285925a8cd4b432c9da0a987836c7f822/shellingham-1.5.4-py2.py3-none-any.whl", - "metadata_url": "https://pypi.org/pypi/shellingham/1.5.4/json", - "metadata_sha256": "9381443cd6295d50fc7f7bd34a457243b0f3a54732cde45b0fc2d0994b61f3af", - "requires_python": ">=3.7", - "tag_rank": 866 - }, - { - "name": "six", - "version": "1.17.0", - "filename": "six-1.17.0-py2.py3-none-any.whl", - "sha256": "4721f391ed90541fddacab5acf947aa0d3dc7d27b2e1e8eda2be8970586c3274", - "size": 11050, - "source": "https://files.pythonhosted.org/packages/b7/ce/149a00dd41f10bc29e5921b496af8b574d8413afcd5e30dfa0ed46c2cc5e/six-1.17.0-py2.py3-none-any.whl", - "metadata_url": "https://pypi.org/pypi/six/1.17.0/json", - "metadata_sha256": "d8de39dd52b73048d81a8fd78022ee2bc72f8b568bcf0d88e5d5069c302b1208", - "requires_python": "!=3.0.*,!=3.1.*,!=3.2.*,>=2.7", - "tag_rank": 866 - }, - { - "name": "sniffio", - "version": "1.3.1", - "filename": "sniffio-1.3.1-py3-none-any.whl", - "sha256": "2f6da418d1f1e0fddd844478f41680e794e6051915791a034ff65e5f100525a2", - "size": 10235, - "source": "https://files.pythonhosted.org/packages/e9/44/75a9c9421471a6c4805dbf2356f7c181a29c1879239abab1ea2cc8f38b40/sniffio-1.3.1-py3-none-any.whl", - "metadata_url": "https://pypi.org/pypi/sniffio/1.3.1/json", - "metadata_sha256": "1217ad6ed84734e8f77ecfd151d91786d5a633b0482af1cde28185de0159bd37", - "requires_python": ">=3.7", - "tag_rank": 866 - }, - { - "name": "tempdir", - "version": "0.7.1", - "filename": "tempdir-0.7.1-py3-none-any.whl", - "sha256": "b70640b48d5379585b85ff7dc46a989a8a82ef1292dc22c057a79cf47842256c", - "size": 2243, - "source": "reviewed_cpu1986_build", - "tag_rank": 866 - }, - { - "name": "termcolor", - "version": "3.3.0", - "filename": "termcolor-3.3.0-py3-none-any.whl", - "sha256": "cf642efadaf0a8ebbbf4bc7a31cec2f9b5f21a9f726f4ccbb08192c9c26f43a5", - "size": 7734, - "source": "https://files.pythonhosted.org/packages/33/d1/8bb87d21e9aeb323cc03034f5eaf2c8f69841e40e4853c2627edf8111ed3/termcolor-3.3.0-py3-none-any.whl", - "metadata_url": "https://pypi.org/pypi/termcolor/3.3.0/json", - "metadata_sha256": "4bf3a3f04fccdadaa5f0a513a867baa4361521f4edfa785d2eb8a0ef79862ce8", - "requires_python": ">=3.10", - "tag_rank": 866 - }, - { - "name": "tokenizers", - "version": "0.23.3", - "filename": "tokenizers-0.23.3-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", - "sha256": "376851d22bcf9d650a5c3090bb83e6cf9e895fbf0595369fa4cd43c1f69b5f87", - "size": 3396193, - "source": "https://files.pythonhosted.org/packages/3a/40/6706b82693715581457c6d5423eaa7faae576bb0526c5738a57085eb4449/tokenizers-0.23.3-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", - "metadata_url": "https://pypi.org/pypi/tokenizers/0.23.3/json", - "metadata_sha256": "c08480d45857e1c4d1174f000a98e74f542a875d07130d19b9c500090214487e", - "requires_python": ">=3.10", - "tag_rank": 144 - }, - { - "name": "tqdm", - "version": "4.70.1", - "filename": "tqdm-4.70.1-py3-none-any.whl", - "sha256": "c293e525e6fef9c20e8728fd4612df02a0aa31bb5fe91ecd93e123b1b7bffa73", - "size": 80199, - "source": "https://files.pythonhosted.org/packages/a7/03/921a3d3c75785aca9ebfbfcabfbc3a1be12e2ab5265deb026d55a5a3f83e/tqdm-4.70.1-py3-none-any.whl", - "metadata_url": "https://pypi.org/pypi/tqdm/4.70.1/json", - "metadata_sha256": "0589819a1bc3437c8f215d95f3f64ff24d602dc6a62598eb8d2e420c8e6a0b25", - "requires_python": ">=3.8", - "tag_rank": 866 - }, - { - "name": "transformers", - "version": "5.20.2", - "filename": "transformers-5.20.2-py3-none-any.whl", - "sha256": "7ae75dbc1fb3922aff1b6c97b9e5479d6cf26fd86a8866bfb71324190479f807", - "size": 12844921, - "source": "https://files.pythonhosted.org/packages/96/b6/724faefed06913dd3e42e0476fe2c42448675c42f687a12bf810aa098005/transformers-5.20.2-py3-none-any.whl", - "metadata_url": "https://pypi.org/pypi/transformers/5.20.2/json", - "metadata_sha256": "20047075e9e109298e86411a439af990e4557e8af8fd73209e5d3585dd2280a3", - "requires_python": ">=3.10.0", - "tag_rank": 866 - }, - { - "name": "tree-sitter", - "version": "0.26.0", - "filename": "tree_sitter-0.26.0-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", - "sha256": "5a6b333b0282d8bb0af741f9b018bd2523d4eecb2686bf6717066a625fecfaa4", - "size": 667487, - "source": "https://files.pythonhosted.org/packages/8a/2f/6e6781b31677231366cb3cf27bc8269157f6d4b03c9032865a4f5f2bbe7e/tree_sitter-0.26.0-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", - "metadata_url": "https://pypi.org/pypi/tree-sitter/0.26.0/json", - "metadata_sha256": "9092685a2f892a458da47b612f842f80aff303f6fadb82261414dc644af2e699", - "requires_python": ">=3.10", - "tag_rank": 5 - }, - { - "name": "tree-sitter-python", - "version": "0.25.0", - "filename": "tree_sitter_python-0.25.0-cp310-abi3-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", - "sha256": "86f118e5eecad616ecdb81d171a36dde9bef5a0b21ed71ea9c3e390813c3baf5", - "size": 108133, - "source": "https://files.pythonhosted.org/packages/aa/cb/d9b0b67d037922d60cbe0359e0c86457c2da721bc714381a63e2c8e35eba/tree_sitter_python-0.25.0-cp310-abi3-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", - "metadata_url": "https://pypi.org/pypi/tree-sitter-python/0.25.0/json", - "metadata_sha256": "06e87483c3cd9c63fd6d9de77936fbb9ab237229849faed87384347c70ef1153", - "requires_python": ">=3.10", - "tag_rank": 133 - }, - { - "name": "truststore", - "version": "0.10.5", - "filename": "truststore-0.10.5-py3-none-any.whl", - "sha256": "9aaaedaefaf06d8b206278cf8b5012bc897f485a874503501e12d776df78951c", - "size": 19017, - "source": "https://files.pythonhosted.org/packages/51/e9/3a7820be2bb0fe53b6bc9c3be26d3d1158004e4c3ab953aa6840b955b1e9/truststore-0.10.5-py3-none-any.whl", - "metadata_url": "https://pypi.org/pypi/truststore/0.10.5/json", - "metadata_sha256": "3aa1222ab6468e1e01b751aeda8db44afca8117297c93cb497c8d27e8f37e17b", - "requires_python": ">=3.10", - "tag_rank": 866 - }, - { - "name": "typer", - "version": "0.27.3", - "filename": "typer-0.27.3-py3-none-any.whl", - "sha256": "e50022f28b82a86313e54501317a1db64bf8f8d036ff8cfe5ca7e47675454aff", - "size": 123312, - "source": "https://files.pythonhosted.org/packages/07/ea/2e31b67051e91a133189e9c000c222502ddc6969856416de0d095de4c0b0/typer-0.27.3-py3-none-any.whl", - "metadata_url": "https://pypi.org/pypi/typer/0.27.3/json", - "metadata_sha256": "61552fe268e5dd109efec5a3542ed135afaf4094fae84d648a13c6e18004ad62", - "requires_python": ">=3.10", - "tag_rank": 866 - }, - { - "name": "typing-extensions", - "version": "4.16.0", - "filename": "typing_extensions-4.16.0-py3-none-any.whl", - "sha256": "481caa481374e813c1b176ada14e97f1f67a4539ce9cfeb3f350d78d6370c2e8", - "size": 45571, - "source": "https://files.pythonhosted.org/packages/49/d3/b8441a820a491ddfc024b0b0cf0393375b75ea13866d9c66727e54c2fc80/typing_extensions-4.16.0-py3-none-any.whl", - "metadata_url": "https://pypi.org/pypi/typing-extensions/4.16.0/json", - "metadata_sha256": "2140da00d62dde967f52d0be0ae28444f1d6a3fe603f52c83c23988c32122d5a", - "requires_python": ">=3.9", - "tag_rank": 866 - }, - { - "name": "typing-inspection", - "version": "0.4.4", - "filename": "typing_inspection-0.4.4-py3-none-any.whl", - "sha256": "65b8397ba37ccbce054456aaccddfc91e6e3083c92824df348d96ca832f3f147", - "size": 14750, - "source": "https://files.pythonhosted.org/packages/67/81/4add07e5172b7ac40d8ed5ff580409a7801a4fe26d529bdd915401dabfbe/typing_inspection-0.4.4-py3-none-any.whl", - "metadata_url": "https://pypi.org/pypi/typing-inspection/0.4.4/json", - "metadata_sha256": "0605416f2f37f2df04e45360cf5ce4ca5eeca6d414bb6ddb1ec5a9893c62ac30", - "requires_python": ">=3.10", - "tag_rank": 866 - }, - { - "name": "uritemplate", - "version": "4.2.0", - "filename": "uritemplate-4.2.0-py3-none-any.whl", - "sha256": "962201ba1c4edcab02e60f9a0d3821e82dfc5d2d6662a21abd533879bdb8a686", - "size": 11488, - "source": "https://files.pythonhosted.org/packages/a9/99/3ae339466c9183ea5b8ae87b34c0b897eda475d2aec2307cae60e5cd4f29/uritemplate-4.2.0-py3-none-any.whl", - "metadata_url": "https://pypi.org/pypi/uritemplate/4.2.0/json", - "metadata_sha256": "c1008c57f5e5ea9a0e02ff7beb439af6761884701477a7c258d15196b81d5fa2", - "requires_python": ">=3.9", - "tag_rank": 866 - }, - { - "name": "urllib3", - "version": "2.8.0", - "filename": "urllib3-2.8.0-py3-none-any.whl", - "sha256": "0cf3cae568d36aa9576b28dfb35f11328f1cb974ca7647d9475ebb86c75ac6e3", - "size": 135717, - "source": "https://files.pythonhosted.org/packages/92/9d/c4e665119135114480843e7ab388fa94d8480650450e6f8e26b70d323a4c/urllib3-2.8.0-py3-none-any.whl", - "metadata_url": "https://pypi.org/pypi/urllib3/2.8.0/json", - "metadata_sha256": "5315e7401db3cacccd0c2ce5daf81f752a9cc3614b18c37f2b79930b7159ce2f", - "requires_python": ">=3.10", - "tag_rank": 866 - }, - { - "name": "vcs-versioning", - "version": "2.6.0", - "filename": "vcs_versioning-2.6.0-py3-none-any.whl", - "sha256": "f8bebd1302ebc12e67ceb30fc7dd698ec969414b6fa43b30c3f14ac9b1e72c41", - "size": 123154, - "source": "https://files.pythonhosted.org/packages/73/1d/ad4064bf12f4503bf61e65d74c93c523672776d25ee0a06aaf42ae09ef3f/vcs_versioning-2.6.0-py3-none-any.whl", - "metadata_url": "https://pypi.org/pypi/vcs-versioning/2.6.0/json", - "metadata_sha256": "e00a63b71c4cf966f3fcb3f2c7abcf8382223539fdf45c7f8fb0e89c441f88dc", - "requires_python": ">=3.8", - "tag_rank": 866 - }, - { - "name": "wget", - "version": "3.2", - "filename": "wget-3.2-py3-none-any.whl", - "sha256": "c00a5fd7032ccf1c4af99f26155f5a379892d2e9d94520b36859d3f010922553", - "size": 9686, - "source": "reviewed_cpu1986_build", - "tag_rank": 866 - }, - { - "name": "wheel", - "version": "0.48.0", - "filename": "wheel-0.48.0-py3-none-any.whl", - "sha256": "3217dcc807155e45db462d7ef2431f5ddda0d7273b700d05a67b271ceb1287ab", - "size": 33320, - "source": "https://files.pythonhosted.org/packages/2e/29/69cfbb602cd91690c55d38ba9fe53e6a7e76a6fa647bf38f19c138d25449/wheel-0.48.0-py3-none-any.whl", - "metadata_url": "https://pypi.org/pypi/wheel/0.48.0/json", - "metadata_sha256": "03f655da3a6dd010d9ac41d4771d5d973109cb662a1e10eb9d967535ffd145f8", - "requires_python": ">=3.9", - "tag_rank": 866 - }, - { - "name": "xxhash", - "version": "4.0.2", - "filename": "xxhash-4.0.2-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", - "sha256": "7c7c95dba490be3b4e5a2fa5595b4d4f088fe7b8726e5998b8c1d83b1f9eb3e4", - "size": 261797, - "source": "https://files.pythonhosted.org/packages/b2/c6/3a85fc0b821ed2dfcf6fb7cfdf07e0b261ada40374583a50abb3480f2863/xxhash-4.0.2-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", - "metadata_url": "https://pypi.org/pypi/xxhash/4.0.2/json", - "metadata_sha256": "444449ad30cce1f870a2b9cd992ff657ce18f2e98f4cd8cb6ec41c2d4127762a", - "requires_python": ">=3.9", - "tag_rank": 5 - }, - { - "name": "yarl", - "version": "1.25.1", - "filename": "yarl-1.25.1-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", - "sha256": "c6f117789d22dce188e5754e8bc65b7e6ebf8cb73963b9fa761f672a5883769d", - "size": 117050, - "source": "https://files.pythonhosted.org/packages/bc/7b/ca212cbe170ac8b96e45317ecbcf9c3c3ecf0cdec98d5b088a9c4088929b/yarl-1.25.1-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", - "metadata_url": "https://pypi.org/pypi/yarl/1.25.1/json", - "metadata_sha256": "a07cf3b564a15e3be66faacf0c81a23497ebfbf1ec849a3a77b494ba4b60cf0f", - "requires_python": ">=3.10", - "tag_rank": 5 - } - ], - "errors": [] -} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-platform-fs-1997.json b/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-platform-fs-1997.json deleted file mode 100644 index 9c7013b2f..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-platform-fs-1997.json +++ /dev/null @@ -1,43 +0,0 @@ -{ - "scope": "compute_platform_and_filesystem_readiness", - "verdict": "INCONCLUSIVE", - "job": 1997, - "slurm_state": "COMPLETED", - "slurm_exit_code": "0:0", - "allocated_seconds": 11, - "requested_cpus": 1, - "allocated_cpus": 2, - "allocated_gpus": 0, - "allocation_released": true, - "requested_memory_gib": 2, - "memory_request_is_verified_hard_limit": false, - "requested_walltime_seconds": 180, - "helper_exit_code": 0, - "platform_complete": true, - "filesystem_complete": true, - "observations_sha256": "9d13f8c349b340b3a697c4892794557f17d0cac10a87a28f15afe15aa5a674db", - "synthetic_file_bytes_verified": true, - "private_evidence_sha256": { - "submission.private.txt": "b1058d4b5ff4f75af7bb5c9215cb2d36ceae960f742f0cc9cfbe9395abfc319d", - "scontrol-final.private.txt": "02f8b5a5ffcf1fa01908b0440dd8ea5337079b16bf8c4ba85b1e88dad9956f0e", - "squeue-final.private.txt": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", - "diagnostic.private.log": "db564efce693d79c6fac3a0054fb99ec188e498f60da6a9e1bfe73749a5b117e", - "output.private.tar": "d4964ba9a03a66f60b1ccbdc8b57fce5bca66ba7230d0b34fcc2929cfd909839" - }, - "source_files": { - "platform_fs.py": "cf73bf4b580b76bacca228b7e1a498ceb4ecd42e2188421c520baf14272b81d8", - "probe.py": "12bbacf986425c5fa5fea035203de7599902fa9a7f98124e6e8eccb3f4f40d5d", - "run.sbatch": "110dd758de6b249264cfb747ff296616b6fb73d52ec2bcec9017b7b5d15f4342", - "check_driver.py": "f87ccc50e112b1b32ac87842e7261348eecfa1019aa996c8115bc14818ca490a", - "controls.json": "1ad4a1d2a69a5624243744980b76610f85a5f6595e2f7e0428ee228e054c61fc" - }, - "network_performed": false, - "installation_performed": false, - "evaluator_executed": false, - "model_executed": false, - "sandbox_qualified": false, - "measured_e2e_effect": null, - "readiness_gpu_seconds_used": 2388, - "readiness_gpu_seconds_cap": 7200, - "earlier_diagnostic_gpu_seconds_separate": 147 -} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-platform-fs-controls.json b/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-platform-fs-controls.json deleted file mode 100644 index a9dcbaebc..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-platform-fs-controls.json +++ /dev/null @@ -1,11 +0,0 @@ -{ - "checks_passed": 5, - "checks": [ - "complete", - "metadata-failure", - "filesystem-failure", - "late-timeout", - "deadline" - ], - "real_network_or_filesystem_probe": false -} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-platform-fs-impact.md b/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-platform-fs-impact.md deleted file mode 100644 index 9256c6f03..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-platform-fs-impact.md +++ /dev/null @@ -1,42 +0,0 @@ -# Compute platform and bounded filesystem readiness - -Refs [Issue #67](https://github.com/InferMatrixLab/InferRSI/issues/67). -**INCONCLUSIVE for sandbox and campaign acceptance; both diagnostic phases completed.** - -The previous CPU1996 transport probe timed out before storage work. CPU1997 -isolated the missing platform/storage observations without network requests, -package changes, evaluator or model execution. Slurm reports COMPLETED0:0 in -11seconds with2CPUs actually allocated against1requested, zeroGPUs; the allocation -is released. The3-minute job bound includes startup; the120-second helper deadline -begins only after imports and output setup. The2GiB request is not a verified hard -memory limit. - -The actual compute interpreter reports CPython3.12.13/Linuxx86_64/glibc2.32 and -packaging26.3. Its879 ordered supported wheel tags and hashes of packaging's -top-level Python sources are retained in the observation. This supplies a target -for wheel selection; it does not prove package installation or evaluator behavior. - -The filesystem phase completed in3.898181969seconds:32 writes of256KiB followed -by fsync (8MiB total), plus128 separately opened/written/closed4KiB files. -Small files were not fsynced. Every produced synthetic byte and the exact file -inventory were checked against the reviewed generator.420 timed operations are -retained. This is one diagnostic sample, not physical storage throughput, a -repeatable extraction benchmark, or proof that all earlier installer delay came -from the network. The earlier network timeout and its unmeasured-storage scope -remain unchanged. - -The executed wrapper/helper/probe match reviewed bytes in the private output -archive. Five optimized-Python mock controls cover complete execution, metadata -and filesystem failure, a late timeout and skipped work after the deadline. -Ruff and shell syntax checks passed. Raw scheduler/log/archive identities are -retained by hash; the complete public observation has no private paths or hosts. - -The next proposed action is local raw-wheel prefetch using these actual tags, -the unchanged5-build/92-runtime locked package sets (96unique packages), and -the two previously reviewed local wheels. Full wheel metadata/size/digest checks -must precede any offline installation. No runtime version or package set change -is implied by a different transport route. CPU1997 itself has no measured E2E -effect and does not activate quality or performance gates. Readiness GPU usage -remains2388/7200seconds; the earlier147seconds stay separate. - -KnowledgeStore candidate `case.qwen38-eval-sandbox-preparation-cand`: `sha256:3ff1c617e4b6d279706e4abce9083b02636536bb2c6f12e6bc4461e3d6f97a88`; readback verified. Previous attempts retained. diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-platform-fs-observations.json b/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-platform-fs-observations.json deleted file mode 100644 index ad7390fab..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-platform-fs-observations.json +++ /dev/null @@ -1 +0,0 @@ -{"completed": true, "evaluator_executed": false, "filesystem": {"completed": true, "elapsed_seconds": 3.898181969067082, "operations": [{"completed": true, "operation": "stream_open", "seconds": 0.0015088069485500455}, {"completed": true, "operation": "stream_write", "seconds": 5.5146985687315464e-05}, {"completed": true, "operation": "stream_write", "seconds": 7.308193016797304e-05}, {"completed": true, "operation": "stream_write", "seconds": 7.299601566046476e-05}, {"completed": true, "operation": "stream_write", "seconds": 7.329101208597422e-05}, {"completed": true, "operation": "stream_write", "seconds": 7.68100144341588e-05}, {"completed": true, "operation": "stream_write", "seconds": 7.060496136546135e-05}, {"completed": true, "operation": "stream_write", "seconds": 7.58440000936389e-05}, {"completed": true, "operation": "stream_write", "seconds": 7.191300392150879e-05}, {"completed": true, "operation": "stream_write", "seconds": 6.642204243689775e-05}, {"completed": true, "operation": "stream_write", "seconds": 6.874697282910347e-05}, {"completed": true, "operation": "stream_write", "seconds": 6.481597665697336e-05}, {"completed": true, "operation": "stream_write", "seconds": 6.701494567096233e-05}, {"completed": true, "operation": "stream_write", "seconds": 6.489607039839029e-05}, {"completed": true, "operation": "stream_write", "seconds": 6.326497532427311e-05}, {"completed": true, "operation": "stream_write", "seconds": 6.27090921625495e-05}, {"completed": true, "operation": "stream_write", "seconds": 7.359799928963184e-05}, {"completed": true, "operation": "stream_write", "seconds": 6.356195081025362e-05}, {"completed": true, "operation": "stream_write", "seconds": 6.28279522061348e-05}, {"completed": true, "operation": "stream_write", "seconds": 6.337405648082495e-05}, {"completed": true, "operation": "stream_write", "seconds": 6.444798782467842e-05}, {"completed": true, "operation": "stream_write", "seconds": 6.520398892462254e-05}, {"completed": true, "operation": "stream_write", "seconds": 6.247300188988447e-05}, {"completed": true, "operation": "stream_write", "seconds": 6.28120033070445e-05}, {"completed": true, "operation": "stream_write", "seconds": 6.641598884016275e-05}, {"completed": true, "operation": "stream_write", "seconds": 6.687105633318424e-05}, {"completed": true, "operation": "stream_write", "seconds": 6.596406456083059e-05}, {"completed": true, "operation": "stream_write", "seconds": 6.741110701113939e-05}, {"completed": true, "operation": "stream_write", "seconds": 6.412598304450512e-05}, {"completed": true, "operation": "stream_write", "seconds": 6.442295853048563e-05}, {"completed": true, "operation": "stream_write", "seconds": 6.481306627392769e-05}, {"completed": true, "operation": "stream_write", "seconds": 7.502594962716103e-05}, {"completed": true, "operation": "stream_write", "seconds": 6.461795419454575e-05}, {"completed": true, "operation": "stream_fsync", "seconds": 0.012135811033658683}, {"completed": true, "operation": "stream_close", "seconds": 8.987030014395714e-06}, {"completed": true, "operation": "small_mkdir", "seconds": 0.0015270620351657271}, {"completed": true, "operation": "small_open", "seconds": 0.0018158439779654145}, {"completed": true, "operation": "small_write", "seconds": 7.3069240897893906e-06}, {"completed": true, "operation": "small_close", "seconds": 0.004399048979394138}, {"completed": true, "operation": "small_open", "seconds": 0.0016796760028228164}, {"completed": true, "operation": "small_write", "seconds": 4.9299560487270355e-06}, {"completed": true, "operation": "small_close", "seconds": 0.004598944913595915}, {"completed": true, "operation": "small_open", "seconds": 0.0015064119361341}, {"completed": true, "operation": "small_write", "seconds": 4.325993359088898e-06}, {"completed": true, "operation": "small_close", "seconds": 0.004043470951728523}, {"completed": true, "operation": "small_open", "seconds": 0.0016244110884144902}, {"completed": true, "operation": "small_write", "seconds": 9.331037290394306e-06}, {"completed": true, "operation": "small_close", "seconds": 0.7545928789768368}, {"completed": true, "operation": "small_open", "seconds": 0.002025185967795551}, {"completed": true, "operation": "small_write", "seconds": 1.0072952136397362e-05}, {"completed": true, "operation": "small_close", "seconds": 0.1686382710468024}, {"completed": true, "operation": "small_open", "seconds": 0.0019687520107254386}, {"completed": true, "operation": "small_write", "seconds": 7.467926479876041e-06}, {"completed": true, "operation": "small_close", "seconds": 0.004989497945643961}, {"completed": true, "operation": "small_open", "seconds": 0.0015491510275751352}, {"completed": true, "operation": "small_write", "seconds": 5.615991540253162e-06}, {"completed": true, "operation": "small_close", "seconds": 0.004852919024415314}, {"completed": true, "operation": "small_open", "seconds": 0.0017879799706861377}, {"completed": true, "operation": "small_write", "seconds": 6.94196205586195e-06}, {"completed": true, "operation": "small_close", "seconds": 0.005734669044613838}, {"completed": true, "operation": "small_open", "seconds": 0.0019027719972655177}, {"completed": true, "operation": "small_write", "seconds": 7.653026841580868e-06}, {"completed": true, "operation": "small_close", "seconds": 0.00488870800472796}, {"completed": true, "operation": "small_open", "seconds": 0.0018116809660568833}, {"completed": true, "operation": "small_write", "seconds": 9.319046512246132e-06}, {"completed": true, "operation": "small_close", "seconds": 0.00500826898496598}, {"completed": true, "operation": "small_open", "seconds": 0.0017420660005882382}, {"completed": true, "operation": "small_write", "seconds": 7.044989615678787e-06}, {"completed": true, "operation": "small_close", "seconds": 0.27889553108252585}, {"completed": true, "operation": "small_open", "seconds": 0.0015179039910435677}, {"completed": true, "operation": "small_write", "seconds": 1.1580996215343475e-05}, {"completed": true, "operation": "small_close", "seconds": 0.0036746219266206026}, {"completed": true, "operation": "small_open", "seconds": 0.0015972449909895658}, {"completed": true, "operation": "small_write", "seconds": 8.406001143157482e-06}, {"completed": true, "operation": "small_close", "seconds": 0.00386696495115757}, {"completed": true, "operation": "small_open", "seconds": 0.0017040630336850882}, {"completed": true, "operation": "small_write", "seconds": 1.1180061846971512e-05}, {"completed": true, "operation": "small_close", "seconds": 0.003871744032949209}, {"completed": true, "operation": "small_open", "seconds": 0.0014648709911853075}, {"completed": true, "operation": "small_write", "seconds": 6.493995897471905e-06}, {"completed": true, "operation": "small_close", "seconds": 0.005536823999136686}, {"completed": true, "operation": "small_open", "seconds": 0.0018326229183003306}, {"completed": true, "operation": "small_write", "seconds": 8.365022949874401e-06}, {"completed": true, "operation": "small_close", "seconds": 0.0045378359500318766}, {"completed": true, "operation": "small_open", "seconds": 0.001748149050399661}, {"completed": true, "operation": "small_write", "seconds": 6.925081834197044e-06}, {"completed": true, "operation": "small_close", "seconds": 0.0042326010297983885}, {"completed": true, "operation": "small_open", "seconds": 0.0017336300807073712}, {"completed": true, "operation": "small_write", "seconds": 4.032975994050503e-06}, {"completed": true, "operation": "small_close", "seconds": 0.005439384956844151}, {"completed": true, "operation": "small_open", "seconds": 0.0016356309643015265}, {"completed": true, "operation": "small_write", "seconds": 3.1419331207871437e-06}, {"completed": true, "operation": "small_close", "seconds": 0.005152784986421466}, {"completed": true, "operation": "small_open", "seconds": 0.0014099590480327606}, {"completed": true, "operation": "small_write", "seconds": 6.567919626832008e-06}, {"completed": true, "operation": "small_close", "seconds": 0.9359303360106423}, {"completed": true, "operation": "small_open", "seconds": 0.002089775982312858}, {"completed": true, "operation": "small_write", "seconds": 1.214293297380209e-05}, {"completed": true, "operation": "small_close", "seconds": 0.004745141952298582}, {"completed": true, "operation": "small_open", "seconds": 0.0017201490700244904}, {"completed": true, "operation": "small_write", "seconds": 4.2719766497612e-06}, {"completed": true, "operation": "small_close", "seconds": 0.004183664917945862}, {"completed": true, "operation": "small_open", "seconds": 0.0019420619355514646}, {"completed": true, "operation": "small_write", "seconds": 2.424931153655052e-06}, {"completed": true, "operation": "small_close", "seconds": 0.005211842013522983}, {"completed": true, "operation": "small_open", "seconds": 0.0017178739653900266}, {"completed": true, "operation": "small_write", "seconds": 2.6479829102754593e-06}, {"completed": true, "operation": "small_close", "seconds": 0.005050398991443217}, {"completed": true, "operation": "small_open", "seconds": 0.0015446090837940574}, {"completed": true, "operation": "small_write", "seconds": 2.851942554116249e-06}, {"completed": true, "operation": "small_close", "seconds": 0.0021725089754909277}, {"completed": true, "operation": "small_open", "seconds": 0.001396968960762024}, {"completed": true, "operation": "small_write", "seconds": 2.305954694747925e-06}, {"completed": true, "operation": "small_close", "seconds": 0.005227255984209478}, {"completed": true, "operation": "small_open", "seconds": 0.001617775997146964}, {"completed": true, "operation": "small_write", "seconds": 2.5799963623285294e-06}, {"completed": true, "operation": "small_close", "seconds": 0.004377263016067445}, {"completed": true, "operation": "small_open", "seconds": 0.0015535190468654037}, {"completed": true, "operation": "small_write", "seconds": 2.7979258447885513e-06}, {"completed": true, "operation": "small_close", "seconds": 0.00442350993398577}, {"completed": true, "operation": "small_open", "seconds": 0.0015318209771066904}, {"completed": true, "operation": "small_write", "seconds": 2.361019141972065e-06}, {"completed": true, "operation": "small_close", "seconds": 0.004102983977645636}, {"completed": true, "operation": "small_open", "seconds": 0.0014025421114638448}, {"completed": true, "operation": "small_write", "seconds": 2.361019141972065e-06}, {"completed": true, "operation": "small_close", "seconds": 0.004628853057511151}, {"completed": true, "operation": "small_open", "seconds": 0.0019494059961289167}, {"completed": true, "operation": "small_write", "seconds": 3.5510165616869926e-06}, {"completed": true, "operation": "small_close", "seconds": 0.0056404180359095335}, {"completed": true, "operation": "small_open", "seconds": 0.0016110159922391176}, {"completed": true, "operation": "small_write", "seconds": 3.051944077014923e-06}, {"completed": true, "operation": "small_close", "seconds": 0.0048162140883505344}, {"completed": true, "operation": "small_open", "seconds": 0.0016085311071947217}, {"completed": true, "operation": "small_write", "seconds": 3.1850067898631096e-06}, {"completed": true, "operation": "small_close", "seconds": 0.004118037992157042}, {"completed": true, "operation": "small_open", "seconds": 0.0015652950387448072}, {"completed": true, "operation": "small_write", "seconds": 4.674890078604221e-06}, {"completed": true, "operation": "small_close", "seconds": 0.005609332001768053}, {"completed": true, "operation": "small_open", "seconds": 0.0015330020105466247}, {"completed": true, "operation": "small_write", "seconds": 3.5170232877135277e-06}, {"completed": true, "operation": "small_close", "seconds": 0.0017599500715732574}, {"completed": true, "operation": "small_open", "seconds": 0.001518349046818912}, {"completed": true, "operation": "small_write", "seconds": 2.308981493115425e-06}, {"completed": true, "operation": "small_close", "seconds": 0.004251659964211285}, {"completed": true, "operation": "small_open", "seconds": 0.0013834589626640081}, {"completed": true, "operation": "small_write", "seconds": 2.6170164346694946e-06}, {"completed": true, "operation": "small_close", "seconds": 0.0048752749571576715}, {"completed": true, "operation": "small_open", "seconds": 0.0017539480468258262}, {"completed": true, "operation": "small_write", "seconds": 2.2579915821552277e-06}, {"completed": true, "operation": "small_close", "seconds": 0.0038518570363521576}, {"completed": true, "operation": "small_open", "seconds": 0.0013321120059117675}, {"completed": true, "operation": "small_write", "seconds": 2.9419315978884697e-06}, {"completed": true, "operation": "small_close", "seconds": 0.005076927016489208}, {"completed": true, "operation": "small_open", "seconds": 0.0016655710060149431}, {"completed": true, "operation": "small_write", "seconds": 4.542060196399689e-06}, {"completed": true, "operation": "small_close", "seconds": 0.001926001044921577}, {"completed": true, "operation": "small_open", "seconds": 0.001534895971417427}, {"completed": true, "operation": "small_write", "seconds": 7.658032700419426e-06}, {"completed": true, "operation": "small_close", "seconds": 0.005287925014272332}, {"completed": true, "operation": "small_open", "seconds": 0.0016009350074455142}, {"completed": true, "operation": "small_write", "seconds": 7.109949365258217e-06}, {"completed": true, "operation": "small_close", "seconds": 0.004758529015816748}, {"completed": true, "operation": "small_open", "seconds": 0.001600532908923924}, {"completed": true, "operation": "small_write", "seconds": 1.085805706679821e-05}, {"completed": true, "operation": "small_close", "seconds": 0.003898808965459466}, {"completed": true, "operation": "small_open", "seconds": 0.001873205997981131}, {"completed": true, "operation": "small_write", "seconds": 3.0070077627897263e-06}, {"completed": true, "operation": "small_close", "seconds": 0.002552516059949994}, {"completed": true, "operation": "small_open", "seconds": 0.001459310995414853}, {"completed": true, "operation": "small_write", "seconds": 7.699942216277122e-06}, {"completed": true, "operation": "small_close", "seconds": 0.003991367993876338}, {"completed": true, "operation": "small_open", "seconds": 0.001498177065514028}, {"completed": true, "operation": "small_write", "seconds": 4.322035238146782e-06}, {"completed": true, "operation": "small_close", "seconds": 0.0036036279052495956}, {"completed": true, "operation": "small_open", "seconds": 0.0018020229181274772}, {"completed": true, "operation": "small_write", "seconds": 3.338092938065529e-06}, {"completed": true, "operation": "small_close", "seconds": 0.0017902060644701123}, {"completed": true, "operation": "small_open", "seconds": 0.0016042269999161363}, {"completed": true, "operation": "small_write", "seconds": 2.622022293508053e-06}, {"completed": true, "operation": "small_close", "seconds": 0.0045435960637405515}, {"completed": true, "operation": "small_open", "seconds": 0.0013483620714396238}, {"completed": true, "operation": "small_write", "seconds": 3.771972842514515e-06}, {"completed": true, "operation": "small_close", "seconds": 0.0018389979377388954}, {"completed": true, "operation": "small_open", "seconds": 0.0016648370074108243}, {"completed": true, "operation": "small_write", "seconds": 3.213062882423401e-06}, {"completed": true, "operation": "small_close", "seconds": 0.003981570014730096}, {"completed": true, "operation": "small_open", "seconds": 0.0015412919456139207}, {"completed": true, "operation": "small_write", "seconds": 2.4549663066864014e-06}, {"completed": true, "operation": "small_close", "seconds": 0.0040418829303234816}, {"completed": true, "operation": "small_open", "seconds": 0.0016414730343967676}, {"completed": true, "operation": "small_write", "seconds": 2.1579908207058907e-06}, {"completed": true, "operation": "small_close", "seconds": 0.004945946973748505}, {"completed": true, "operation": "small_open", "seconds": 0.0015499379951506853}, {"completed": true, "operation": "small_write", "seconds": 2.6599736884236336e-06}, {"completed": true, "operation": "small_close", "seconds": 0.0044277680572122335}, {"completed": true, "operation": "small_open", "seconds": 0.0013093329034745693}, {"completed": true, "operation": "small_write", "seconds": 2.3550819605588913e-06}, {"completed": true, "operation": "small_close", "seconds": 0.004171008011326194}, {"completed": true, "operation": "small_open", "seconds": 0.0014119179686531425}, {"completed": true, "operation": "small_write", "seconds": 2.4850014597177505e-06}, {"completed": true, "operation": "small_close", "seconds": 0.0050644660368561745}, {"completed": true, "operation": "small_open", "seconds": 0.0014943790156394243}, {"completed": true, "operation": "small_write", "seconds": 2.58604995906353e-06}, {"completed": true, "operation": "small_close", "seconds": 0.003916065092198551}, {"completed": true, "operation": "small_open", "seconds": 0.0016386340139433742}, {"completed": true, "operation": "small_write", "seconds": 3.797002136707306e-06}, {"completed": true, "operation": "small_close", "seconds": 0.0055520409950986505}, {"completed": true, "operation": "small_open", "seconds": 0.0017570999916642904}, {"completed": true, "operation": "small_write", "seconds": 2.5919871404767036e-06}, {"completed": true, "operation": "small_close", "seconds": 0.004640536033548415}, {"completed": true, "operation": "small_open", "seconds": 0.0024135219864547253}, {"completed": true, "operation": "small_write", "seconds": 2.7710339054465294e-06}, {"completed": true, "operation": "small_close", "seconds": 0.0036387928994372487}, {"completed": true, "operation": "small_open", "seconds": 0.0012993490090593696}, {"completed": true, "operation": "small_write", "seconds": 2.491055056452751e-06}, {"completed": true, "operation": "small_close", "seconds": 0.0016204339917749166}, {"completed": true, "operation": "small_open", "seconds": 0.0014482379192486405}, {"completed": true, "operation": "small_write", "seconds": 1.9780127331614494e-06}, {"completed": true, "operation": "small_close", "seconds": 0.004932415089569986}, {"completed": true, "operation": "small_open", "seconds": 0.0019079409539699554}, {"completed": true, "operation": "small_write", "seconds": 2.4029286578297615e-06}, {"completed": true, "operation": "small_close", "seconds": 0.00539989594835788}, {"completed": true, "operation": "small_open", "seconds": 0.0015360648976638913}, {"completed": true, "operation": "small_write", "seconds": 5.843932740390301e-06}, {"completed": true, "operation": "small_close", "seconds": 0.00434725999366492}, {"completed": true, "operation": "small_open", "seconds": 0.0014203309547156096}, {"completed": true, "operation": "small_write", "seconds": 7.439055480062962e-06}, {"completed": true, "operation": "small_close", "seconds": 0.004508116981014609}, {"completed": true, "operation": "small_open", "seconds": 0.002010565949603915}, {"completed": true, "operation": "small_write", "seconds": 7.138936780393124e-06}, {"completed": true, "operation": "small_close", "seconds": 0.004034304991364479}, {"completed": true, "operation": "small_open", "seconds": 0.0015458150301128626}, {"completed": true, "operation": "small_write", "seconds": 6.369082257151604e-06}, {"completed": true, "operation": "small_close", "seconds": 0.0038642550352960825}, {"completed": true, "operation": "small_open", "seconds": 0.0014500219840556383}, {"completed": true, "operation": "small_write", "seconds": 6.003072485327721e-06}, {"completed": true, "operation": "small_close", "seconds": 0.0038405819796025753}, {"completed": true, "operation": "small_open", "seconds": 0.001699264976195991}, {"completed": true, "operation": "small_write", "seconds": 6.752088665962219e-06}, {"completed": true, "operation": "small_close", "seconds": 0.003956201020628214}, {"completed": true, "operation": "small_open", "seconds": 0.0016982390079647303}, {"completed": true, "operation": "small_write", "seconds": 6.137997843325138e-06}, {"completed": true, "operation": "small_close", "seconds": 0.005305150989443064}, {"completed": true, "operation": "small_open", "seconds": 0.0017099370015785098}, {"completed": true, "operation": "small_write", "seconds": 6.736954674124718e-06}, {"completed": true, "operation": "small_close", "seconds": 0.00405328709166497}, {"completed": true, "operation": "small_open", "seconds": 0.0015855919336900115}, {"completed": true, "operation": "small_write", "seconds": 6.299000233411789e-06}, {"completed": true, "operation": "small_close", "seconds": 0.003984790062531829}, {"completed": true, "operation": "small_open", "seconds": 0.0014074109494686127}, {"completed": true, "operation": "small_write", "seconds": 7.648952305316925e-06}, {"completed": true, "operation": "small_close", "seconds": 0.003994031925685704}, {"completed": true, "operation": "small_open", "seconds": 0.0017321330960839987}, {"completed": true, "operation": "small_write", "seconds": 6.6569773480296135e-06}, {"completed": true, "operation": "small_close", "seconds": 0.005219813087023795}, {"completed": true, "operation": "small_open", "seconds": 0.0013613260816782713}, {"completed": true, "operation": "small_write", "seconds": 6.379908882081509e-06}, {"completed": true, "operation": "small_close", "seconds": 0.001966579002328217}, {"completed": true, "operation": "small_open", "seconds": 0.0014716190053150058}, {"completed": true, "operation": "small_write", "seconds": 4.802015610039234e-06}, {"completed": true, "operation": "small_close", "seconds": 0.9841437970753759}, {"completed": true, "operation": "small_open", "seconds": 0.002407004009000957}, {"completed": true, "operation": "small_write", "seconds": 1.1240015737712383e-05}, {"completed": true, "operation": "small_close", "seconds": 0.0017653449904173613}, {"completed": true, "operation": "small_open", "seconds": 0.0016871049301698804}, {"completed": true, "operation": "small_write", "seconds": 3.1599774956703186e-06}, {"completed": true, "operation": "small_close", "seconds": 0.004886626964434981}, {"completed": true, "operation": "small_open", "seconds": 0.0016966380644589663}, {"completed": true, "operation": "small_write", "seconds": 3.816094249486923e-06}, {"completed": true, "operation": "small_close", "seconds": 0.004913331009447575}, {"completed": true, "operation": "small_open", "seconds": 0.0015560840256512165}, {"completed": true, "operation": "small_write", "seconds": 2.1280720829963684e-06}, {"completed": true, "operation": "small_close", "seconds": 0.004928236943669617}, {"completed": true, "operation": "small_open", "seconds": 0.0014241900062188506}, {"completed": true, "operation": "small_write", "seconds": 4.415982402861118e-06}, {"completed": true, "operation": "small_close", "seconds": 0.0019991559674963355}, {"completed": true, "operation": "small_open", "seconds": 0.00158736202865839}, {"completed": true, "operation": "small_write", "seconds": 2.196989953517914e-06}, {"completed": true, "operation": "small_close", "seconds": 0.005025552003644407}, {"completed": true, "operation": "small_open", "seconds": 0.0016432279953733087}, {"completed": true, "operation": "small_write", "seconds": 7.955008186399937e-06}, {"completed": true, "operation": "small_close", "seconds": 0.0018798260716721416}, {"completed": true, "operation": "small_open", "seconds": 0.0017888789298012853}, {"completed": true, "operation": "small_write", "seconds": 3.6780256778001785e-06}, {"completed": true, "operation": "small_close", "seconds": 0.00503114506136626}, {"completed": true, "operation": "small_open", "seconds": 0.0015028170309960842}, {"completed": true, "operation": "small_write", "seconds": 2.678949385881424e-06}, {"completed": true, "operation": "small_close", "seconds": 0.0046317450469359756}, {"completed": true, "operation": "small_open", "seconds": 0.001418199040926993}, {"completed": true, "operation": "small_write", "seconds": 2.4440232664346695e-06}, {"completed": true, "operation": "small_close", "seconds": 0.006389423971995711}, {"completed": true, "operation": "small_open", "seconds": 0.0018511050147935748}, {"completed": true, "operation": "small_write", "seconds": 2.6259804144501686e-06}, {"completed": true, "operation": "small_close", "seconds": 0.004212351981550455}, {"completed": true, "operation": "small_open", "seconds": 0.0015403339639306068}, {"completed": true, "operation": "small_write", "seconds": 2.0720763131976128e-06}, {"completed": true, "operation": "small_close", "seconds": 0.005719412001781166}, {"completed": true, "operation": "small_open", "seconds": 0.0018945069750770926}, {"completed": true, "operation": "small_write", "seconds": 5.002948455512524e-06}, {"completed": true, "operation": "small_close", "seconds": 0.004767840960994363}, {"completed": true, "operation": "small_open", "seconds": 0.0016155249904841185}, {"completed": true, "operation": "small_write", "seconds": 7.294933311641216e-06}, {"completed": true, "operation": "small_close", "seconds": 0.004258217988535762}, {"completed": true, "operation": "small_open", "seconds": 0.0016352549428120255}, {"completed": true, "operation": "small_write", "seconds": 4.73496038466692e-06}, {"completed": true, "operation": "small_close", "seconds": 0.004217148991301656}, {"completed": true, "operation": "small_open", "seconds": 0.0017686899518594146}, {"completed": true, "operation": "small_write", "seconds": 2.310960553586483e-06}, {"completed": true, "operation": "small_close", "seconds": 0.004378353012725711}, {"completed": true, "operation": "small_open", "seconds": 0.0014620019355788827}, {"completed": true, "operation": "small_write", "seconds": 2.704095095396042e-06}, {"completed": true, "operation": "small_close", "seconds": 0.005104369018226862}, {"completed": true, "operation": "small_open", "seconds": 0.0018707190174609423}, {"completed": true, "operation": "small_write", "seconds": 4.469999112188816e-06}, {"completed": true, "operation": "small_close", "seconds": 0.005970339057967067}, {"completed": true, "operation": "small_open", "seconds": 0.0017638630233705044}, {"completed": true, "operation": "small_write", "seconds": 2.4779001250863075e-06}, {"completed": true, "operation": "small_close", "seconds": 0.0055986689403653145}, {"completed": true, "operation": "small_open", "seconds": 0.001873902976512909}, {"completed": true, "operation": "small_write", "seconds": 2.4659093469381332e-06}, {"completed": true, "operation": "small_close", "seconds": 0.004462311044335365}, {"completed": true, "operation": "small_open", "seconds": 0.0017067710869014263}, {"completed": true, "operation": "small_write", "seconds": 1.945067197084427e-06}, {"completed": true, "operation": "small_close", "seconds": 0.004289075033739209}, {"completed": true, "operation": "small_open", "seconds": 0.001919349073432386}, {"completed": true, "operation": "small_write", "seconds": 5.605979822576046e-06}, {"completed": true, "operation": "small_close", "seconds": 0.004505074000917375}, {"completed": true, "operation": "small_open", "seconds": 0.0018334449268877506}, {"completed": true, "operation": "small_write", "seconds": 2.9939692467451096e-06}, {"completed": true, "operation": "small_close", "seconds": 0.005025733029469848}, {"completed": true, "operation": "small_open", "seconds": 0.0022810769733041525}, {"completed": true, "operation": "small_write", "seconds": 8.514034561812878e-06}, {"completed": true, "operation": "small_close", "seconds": 0.005555251031182706}, {"completed": true, "operation": "small_open", "seconds": 0.0017274910351261497}, {"completed": true, "operation": "small_write", "seconds": 3.3190008252859116e-06}, {"completed": true, "operation": "small_close", "seconds": 0.005238811951130629}, {"completed": true, "operation": "small_open", "seconds": 0.0016876300796866417}, {"completed": true, "operation": "small_write", "seconds": 2.678949385881424e-06}, {"completed": true, "operation": "small_close", "seconds": 0.00429946300573647}, {"completed": true, "operation": "small_open", "seconds": 0.001778327045030892}, {"completed": true, "operation": "small_write", "seconds": 2.076965756714344e-06}, {"completed": true, "operation": "small_close", "seconds": 0.0048387410352006555}, {"completed": true, "operation": "small_open", "seconds": 0.0018126630457118154}, {"completed": true, "operation": "small_write", "seconds": 3.7350691854953766e-06}, {"completed": true, "operation": "small_close", "seconds": 0.004469699924811721}, {"completed": true, "operation": "small_open", "seconds": 0.001658126013353467}, {"completed": true, "operation": "small_write", "seconds": 2.18802597373724e-06}, {"completed": true, "operation": "small_close", "seconds": 0.005533887073397636}, {"completed": true, "operation": "small_open", "seconds": 0.0018169779796153307}, {"completed": true, "operation": "small_write", "seconds": 6.3659390434622765e-06}, {"completed": true, "operation": "small_close", "seconds": 0.005409709992818534}, {"completed": true, "operation": "small_open", "seconds": 0.0016754179960116744}, {"completed": true, "operation": "small_write", "seconds": 2.7921050786972046e-06}, {"completed": true, "operation": "small_close", "seconds": 0.004171081003732979}, {"completed": true, "operation": "small_open", "seconds": 0.001749748014844954}, {"completed": true, "operation": "small_write", "seconds": 2.418062649667263e-06}, {"completed": true, "operation": "small_close", "seconds": 0.0041345469653606415}, {"completed": true, "operation": "small_open", "seconds": 0.0018231930444017053}, {"completed": true, "operation": "small_write", "seconds": 2.1760351955890656e-06}, {"completed": true, "operation": "small_close", "seconds": 0.004355465993285179}, {"completed": true, "operation": "small_open", "seconds": 0.0016863549826666713}, {"completed": true, "operation": "small_write", "seconds": 4.230067133903503e-06}, {"completed": true, "operation": "small_close", "seconds": 0.005472016055136919}, {"completed": true, "operation": "small_open", "seconds": 0.001598978997208178}, {"completed": true, "operation": "small_write", "seconds": 2.0900042727589607e-06}, {"completed": true, "operation": "small_close", "seconds": 0.004291830002330244}, {"completed": true, "operation": "small_open", "seconds": 0.001849098945967853}, {"completed": true, "operation": "small_write", "seconds": 2.6030465960502625e-06}, {"completed": true, "operation": "small_close", "seconds": 0.005900277057662606}, {"completed": true, "operation": "small_open", "seconds": 0.0018606879748404026}, {"completed": true, "operation": "small_write", "seconds": 2.6209745556116104e-06}, {"completed": true, "operation": "small_close", "seconds": 0.004675440024584532}, {"completed": true, "operation": "small_open", "seconds": 0.001678113010711968}, {"completed": true, "operation": "small_write", "seconds": 2.1760351955890656e-06}, {"completed": true, "operation": "small_close", "seconds": 0.004918457008898258}, {"completed": true, "operation": "small_open", "seconds": 0.0017203819006681442}, {"completed": true, "operation": "small_write", "seconds": 2.388027496635914e-06}, {"completed": true, "operation": "small_close", "seconds": 0.002119165030308068}, {"completed": true, "operation": "small_open", "seconds": 0.002676783944480121}, {"completed": true, "operation": "small_write", "seconds": 2.1819723770022392e-06}, {"completed": true, "operation": "small_close", "seconds": 0.0020668100332841277}, {"completed": true, "operation": "small_open", "seconds": 0.0020712700206786394}, {"completed": true, "operation": "small_write", "seconds": 3.334018401801586e-06}, {"completed": true, "operation": "small_close", "seconds": 0.004465605947189033}, {"completed": true, "operation": "small_open", "seconds": 0.0022156710037961602}, {"completed": true, "operation": "small_write", "seconds": 2.3100292310118675e-06}, {"completed": true, "operation": "small_close", "seconds": 0.005007017985917628}, {"completed": true, "operation": "small_open", "seconds": 0.0022287709871307015}, {"completed": true, "operation": "small_write", "seconds": 3.0909432098269463e-06}, {"completed": true, "operation": "small_close", "seconds": 0.005010043038055301}, {"completed": true, "operation": "small_open", "seconds": 0.002084015985019505}, {"completed": true, "operation": "small_write", "seconds": 3.247056156396866e-06}, {"completed": true, "operation": "small_close", "seconds": 0.003969440935179591}, {"completed": true, "operation": "small_open", "seconds": 0.0019318748963996768}, {"completed": true, "operation": "small_write", "seconds": 2.691056579351425e-06}, {"completed": true, "operation": "small_close", "seconds": 0.0019072500290349126}, {"completed": true, "operation": "small_open", "seconds": 0.002234815969131887}, {"completed": true, "operation": "small_write", "seconds": 2.8669601306319237e-06}, {"completed": true, "operation": "small_close", "seconds": 0.005308396997861564}, {"completed": true, "operation": "small_open", "seconds": 0.002077293931506574}, {"completed": true, "operation": "small_write", "seconds": 2.547050826251507e-06}, {"completed": true, "operation": "small_close", "seconds": 0.004609500057995319}, {"completed": true, "operation": "small_open", "seconds": 0.0019488800317049026}, {"completed": true, "operation": "small_write", "seconds": 4.14892565459013e-06}, {"completed": true, "operation": "small_close", "seconds": 0.0019428189843893051}, {"completed": true, "operation": "small_open", "seconds": 0.002434068010188639}, {"completed": true, "operation": "small_write", "seconds": 2.585933543741703e-06}, {"completed": true, "operation": "small_close", "seconds": 0.004488741979002953}, {"completed": true, "operation": "small_open", "seconds": 0.002440782031044364}, {"completed": true, "operation": "small_write", "seconds": 3.637978807091713e-06}, {"completed": true, "operation": "small_close", "seconds": 0.004342464031651616}, {"completed": true, "operation": "small_open", "seconds": 0.0018962270114570856}, {"completed": true, "operation": "small_write", "seconds": 2.825050614774227e-06}, {"completed": true, "operation": "small_close", "seconds": 0.0049504219787195325}, {"completed": true, "operation": "small_open", "seconds": 0.0022532149450853467}, {"completed": true, "operation": "small_write", "seconds": 2.374988980591297e-06}, {"completed": true, "operation": "small_close", "seconds": 0.005791024072095752}, {"completed": true, "operation": "small_open", "seconds": 0.002158199902623892}, {"completed": true, "operation": "small_write", "seconds": 2.674991264939308e-06}, {"completed": true, "operation": "small_close", "seconds": 0.004841197049245238}], "physical_storage_throughput_claimed": false, "small_file_bytes": 524288, "small_file_count": 128, "small_files_fsynced": false, "stream_bytes": 8388608}, "gpu_used": false, "job_id": "1997", "network_requested": false, "package_changes": false, "platform": {"completed": true, "implementation": "cpython", "libc": ["glibc", "2.32"], "machine": "x86_64", "packaging_sources": {"__init__.py": "6e758cdd0028b2cad74c739b9adc9c9f06f054932306280b878bfe59d39219b5", "_elffile.py": "1f025f54c573f1a85740c4fbd908cc1d1fb0c177c14377ebf5f09d6cd5368780", "_manylinux.py": "5b6e84324c42538ab4cdb148e2430fdba36f04427b0efda55bea083a35fa6136", "_musllinux.py": "0328fb827b1170c77df4c107d49be8df8d37262103490a61e8acad8a2f9cadff", "_parser.py": "88a348115209327497e453b635e5db099fd592f9335d710c7313b29cdb30c50c", "_ranges.py": "b7b595f2e692aeb3697c545b85c24cbbcb4552f2adc6d9059aa82d0877e816b7", "_structures.py": "eb48d16c5efca7ccf930a9cd77a700a6b80e69d0891d5d03966526441a8565cb", "_tokenizer.py": "cc25e57c0d564c89e15073471d6e1c264299ca946fb17c0e1e4af54ef7713ee6", "dependency_groups.py": "5309cc564ca3022dfb594816e9d593bb09cdf34208c1fb8c5ec9afb6c670278c", "direct_url.py": "843baa95d04cbb49288de9d9fd77ee946e862cb603123e7e46a69e171fa7d06b", "errors.py": "ea17c461701ff2ffc817ae7e94569d24e30889e04fdb120ab721178c6d67188b", "markers.py": "6cbe860d35d98d2b599e7f0ec73feab124e015711b30be9045e678c92940d9cc", "metadata.py": "934149f420a5449e6f376aa61d6f26a0823eb82cc7c6a1c8cde5aa7be43d0f63", "pylock.py": "1bce0a9fe63e42b87e8ced05da272baf16a68aa704b3718405e2735a6112467c", "ranges.py": "b4781ce913e5dff9e07906f20585c54c3028e1f7ff56341828e79d522ddbfab6", "requirements.py": "459476eb2107fe30e6b1310051a5bdb44c7d3b7f78e0e87bada268282cba62f5", "specifiers.py": "661ad31ddf45ffb861d2626734f4e3d66966029b8fdb79586bb105825d8d9d59", "tags.py": "0e717a082a79620d79cf0321b420df15593c4916fcda03c27cb9d82e07ef5d6e", "utils.py": "a13742cd9994eeca5adfb0249d126408c478026eb1f48cb97afbf2fe9f28ecc6", "version.py": "558b2fb50fd198c660a4c082f96b8919c565911b6b4b6e07f34b6b60c074be97"}, "packaging_version": "26.3", "python": "3.12.13", "soabi": "cpython-312-x86_64-linux-gnu", "supported_tags": ["cp312-cp312-linux_x86_64", "cp312-cp312-manylinux_2_32_x86_64", "cp312-cp312-manylinux_2_31_x86_64", "cp312-cp312-manylinux_2_30_x86_64", "cp312-cp312-manylinux_2_29_x86_64", "cp312-cp312-manylinux_2_28_x86_64", "cp312-cp312-manylinux_2_27_x86_64", "cp312-cp312-manylinux_2_26_x86_64", "cp312-cp312-manylinux_2_25_x86_64", "cp312-cp312-manylinux_2_24_x86_64", "cp312-cp312-manylinux_2_23_x86_64", "cp312-cp312-manylinux_2_22_x86_64", "cp312-cp312-manylinux_2_21_x86_64", "cp312-cp312-manylinux_2_20_x86_64", "cp312-cp312-manylinux_2_19_x86_64", "cp312-cp312-manylinux_2_18_x86_64", "cp312-cp312-manylinux_2_17_x86_64", "cp312-cp312-manylinux2014_x86_64", "cp312-cp312-manylinux_2_16_x86_64", "cp312-cp312-manylinux_2_15_x86_64", "cp312-cp312-manylinux_2_14_x86_64", "cp312-cp312-manylinux_2_13_x86_64", "cp312-cp312-manylinux_2_12_x86_64", "cp312-cp312-manylinux2010_x86_64", "cp312-cp312-manylinux_2_11_x86_64", "cp312-cp312-manylinux_2_10_x86_64", "cp312-cp312-manylinux_2_9_x86_64", "cp312-cp312-manylinux_2_8_x86_64", "cp312-cp312-manylinux_2_7_x86_64", "cp312-cp312-manylinux_2_6_x86_64", "cp312-cp312-manylinux_2_5_x86_64", "cp312-cp312-manylinux1_x86_64", "cp312-abi3-linux_x86_64", "cp312-abi3-manylinux_2_32_x86_64", "cp312-abi3-manylinux_2_31_x86_64", "cp312-abi3-manylinux_2_30_x86_64", "cp312-abi3-manylinux_2_29_x86_64", "cp312-abi3-manylinux_2_28_x86_64", "cp312-abi3-manylinux_2_27_x86_64", "cp312-abi3-manylinux_2_26_x86_64", "cp312-abi3-manylinux_2_25_x86_64", "cp312-abi3-manylinux_2_24_x86_64", "cp312-abi3-manylinux_2_23_x86_64", "cp312-abi3-manylinux_2_22_x86_64", "cp312-abi3-manylinux_2_21_x86_64", "cp312-abi3-manylinux_2_20_x86_64", "cp312-abi3-manylinux_2_19_x86_64", "cp312-abi3-manylinux_2_18_x86_64", "cp312-abi3-manylinux_2_17_x86_64", "cp312-abi3-manylinux2014_x86_64", "cp312-abi3-manylinux_2_16_x86_64", "cp312-abi3-manylinux_2_15_x86_64", "cp312-abi3-manylinux_2_14_x86_64", "cp312-abi3-manylinux_2_13_x86_64", "cp312-abi3-manylinux_2_12_x86_64", "cp312-abi3-manylinux2010_x86_64", "cp312-abi3-manylinux_2_11_x86_64", "cp312-abi3-manylinux_2_10_x86_64", "cp312-abi3-manylinux_2_9_x86_64", "cp312-abi3-manylinux_2_8_x86_64", "cp312-abi3-manylinux_2_7_x86_64", "cp312-abi3-manylinux_2_6_x86_64", "cp312-abi3-manylinux_2_5_x86_64", "cp312-abi3-manylinux1_x86_64", "cp312-none-linux_x86_64", "cp312-none-manylinux_2_32_x86_64", "cp312-none-manylinux_2_31_x86_64", "cp312-none-manylinux_2_30_x86_64", "cp312-none-manylinux_2_29_x86_64", "cp312-none-manylinux_2_28_x86_64", "cp312-none-manylinux_2_27_x86_64", "cp312-none-manylinux_2_26_x86_64", "cp312-none-manylinux_2_25_x86_64", "cp312-none-manylinux_2_24_x86_64", "cp312-none-manylinux_2_23_x86_64", "cp312-none-manylinux_2_22_x86_64", "cp312-none-manylinux_2_21_x86_64", "cp312-none-manylinux_2_20_x86_64", "cp312-none-manylinux_2_19_x86_64", "cp312-none-manylinux_2_18_x86_64", "cp312-none-manylinux_2_17_x86_64", "cp312-none-manylinux2014_x86_64", "cp312-none-manylinux_2_16_x86_64", "cp312-none-manylinux_2_15_x86_64", "cp312-none-manylinux_2_14_x86_64", "cp312-none-manylinux_2_13_x86_64", "cp312-none-manylinux_2_12_x86_64", "cp312-none-manylinux2010_x86_64", "cp312-none-manylinux_2_11_x86_64", "cp312-none-manylinux_2_10_x86_64", "cp312-none-manylinux_2_9_x86_64", "cp312-none-manylinux_2_8_x86_64", "cp312-none-manylinux_2_7_x86_64", "cp312-none-manylinux_2_6_x86_64", "cp312-none-manylinux_2_5_x86_64", "cp312-none-manylinux1_x86_64", "cp311-abi3-linux_x86_64", "cp311-abi3-manylinux_2_32_x86_64", "cp311-abi3-manylinux_2_31_x86_64", "cp311-abi3-manylinux_2_30_x86_64", "cp311-abi3-manylinux_2_29_x86_64", "cp311-abi3-manylinux_2_28_x86_64", "cp311-abi3-manylinux_2_27_x86_64", "cp311-abi3-manylinux_2_26_x86_64", "cp311-abi3-manylinux_2_25_x86_64", "cp311-abi3-manylinux_2_24_x86_64", "cp311-abi3-manylinux_2_23_x86_64", "cp311-abi3-manylinux_2_22_x86_64", "cp311-abi3-manylinux_2_21_x86_64", "cp311-abi3-manylinux_2_20_x86_64", "cp311-abi3-manylinux_2_19_x86_64", "cp311-abi3-manylinux_2_18_x86_64", "cp311-abi3-manylinux_2_17_x86_64", "cp311-abi3-manylinux2014_x86_64", "cp311-abi3-manylinux_2_16_x86_64", "cp311-abi3-manylinux_2_15_x86_64", "cp311-abi3-manylinux_2_14_x86_64", "cp311-abi3-manylinux_2_13_x86_64", "cp311-abi3-manylinux_2_12_x86_64", "cp311-abi3-manylinux2010_x86_64", "cp311-abi3-manylinux_2_11_x86_64", "cp311-abi3-manylinux_2_10_x86_64", "cp311-abi3-manylinux_2_9_x86_64", "cp311-abi3-manylinux_2_8_x86_64", "cp311-abi3-manylinux_2_7_x86_64", "cp311-abi3-manylinux_2_6_x86_64", "cp311-abi3-manylinux_2_5_x86_64", "cp311-abi3-manylinux1_x86_64", "cp310-abi3-linux_x86_64", "cp310-abi3-manylinux_2_32_x86_64", "cp310-abi3-manylinux_2_31_x86_64", "cp310-abi3-manylinux_2_30_x86_64", "cp310-abi3-manylinux_2_29_x86_64", "cp310-abi3-manylinux_2_28_x86_64", "cp310-abi3-manylinux_2_27_x86_64", "cp310-abi3-manylinux_2_26_x86_64", "cp310-abi3-manylinux_2_25_x86_64", "cp310-abi3-manylinux_2_24_x86_64", "cp310-abi3-manylinux_2_23_x86_64", "cp310-abi3-manylinux_2_22_x86_64", "cp310-abi3-manylinux_2_21_x86_64", "cp310-abi3-manylinux_2_20_x86_64", "cp310-abi3-manylinux_2_19_x86_64", "cp310-abi3-manylinux_2_18_x86_64", "cp310-abi3-manylinux_2_17_x86_64", "cp310-abi3-manylinux2014_x86_64", "cp310-abi3-manylinux_2_16_x86_64", "cp310-abi3-manylinux_2_15_x86_64", "cp310-abi3-manylinux_2_14_x86_64", "cp310-abi3-manylinux_2_13_x86_64", "cp310-abi3-manylinux_2_12_x86_64", "cp310-abi3-manylinux2010_x86_64", "cp310-abi3-manylinux_2_11_x86_64", "cp310-abi3-manylinux_2_10_x86_64", "cp310-abi3-manylinux_2_9_x86_64", "cp310-abi3-manylinux_2_8_x86_64", "cp310-abi3-manylinux_2_7_x86_64", "cp310-abi3-manylinux_2_6_x86_64", "cp310-abi3-manylinux_2_5_x86_64", "cp310-abi3-manylinux1_x86_64", "cp39-abi3-linux_x86_64", "cp39-abi3-manylinux_2_32_x86_64", "cp39-abi3-manylinux_2_31_x86_64", "cp39-abi3-manylinux_2_30_x86_64", "cp39-abi3-manylinux_2_29_x86_64", "cp39-abi3-manylinux_2_28_x86_64", "cp39-abi3-manylinux_2_27_x86_64", "cp39-abi3-manylinux_2_26_x86_64", "cp39-abi3-manylinux_2_25_x86_64", "cp39-abi3-manylinux_2_24_x86_64", "cp39-abi3-manylinux_2_23_x86_64", "cp39-abi3-manylinux_2_22_x86_64", "cp39-abi3-manylinux_2_21_x86_64", "cp39-abi3-manylinux_2_20_x86_64", "cp39-abi3-manylinux_2_19_x86_64", "cp39-abi3-manylinux_2_18_x86_64", "cp39-abi3-manylinux_2_17_x86_64", "cp39-abi3-manylinux2014_x86_64", "cp39-abi3-manylinux_2_16_x86_64", "cp39-abi3-manylinux_2_15_x86_64", "cp39-abi3-manylinux_2_14_x86_64", "cp39-abi3-manylinux_2_13_x86_64", "cp39-abi3-manylinux_2_12_x86_64", "cp39-abi3-manylinux2010_x86_64", "cp39-abi3-manylinux_2_11_x86_64", "cp39-abi3-manylinux_2_10_x86_64", "cp39-abi3-manylinux_2_9_x86_64", "cp39-abi3-manylinux_2_8_x86_64", "cp39-abi3-manylinux_2_7_x86_64", "cp39-abi3-manylinux_2_6_x86_64", "cp39-abi3-manylinux_2_5_x86_64", "cp39-abi3-manylinux1_x86_64", "cp38-abi3-linux_x86_64", "cp38-abi3-manylinux_2_32_x86_64", "cp38-abi3-manylinux_2_31_x86_64", "cp38-abi3-manylinux_2_30_x86_64", "cp38-abi3-manylinux_2_29_x86_64", "cp38-abi3-manylinux_2_28_x86_64", "cp38-abi3-manylinux_2_27_x86_64", "cp38-abi3-manylinux_2_26_x86_64", "cp38-abi3-manylinux_2_25_x86_64", "cp38-abi3-manylinux_2_24_x86_64", "cp38-abi3-manylinux_2_23_x86_64", "cp38-abi3-manylinux_2_22_x86_64", "cp38-abi3-manylinux_2_21_x86_64", "cp38-abi3-manylinux_2_20_x86_64", "cp38-abi3-manylinux_2_19_x86_64", "cp38-abi3-manylinux_2_18_x86_64", "cp38-abi3-manylinux_2_17_x86_64", "cp38-abi3-manylinux2014_x86_64", "cp38-abi3-manylinux_2_16_x86_64", "cp38-abi3-manylinux_2_15_x86_64", "cp38-abi3-manylinux_2_14_x86_64", "cp38-abi3-manylinux_2_13_x86_64", "cp38-abi3-manylinux_2_12_x86_64", "cp38-abi3-manylinux2010_x86_64", "cp38-abi3-manylinux_2_11_x86_64", "cp38-abi3-manylinux_2_10_x86_64", "cp38-abi3-manylinux_2_9_x86_64", "cp38-abi3-manylinux_2_8_x86_64", "cp38-abi3-manylinux_2_7_x86_64", "cp38-abi3-manylinux_2_6_x86_64", "cp38-abi3-manylinux_2_5_x86_64", "cp38-abi3-manylinux1_x86_64", "cp37-abi3-linux_x86_64", "cp37-abi3-manylinux_2_32_x86_64", "cp37-abi3-manylinux_2_31_x86_64", "cp37-abi3-manylinux_2_30_x86_64", "cp37-abi3-manylinux_2_29_x86_64", "cp37-abi3-manylinux_2_28_x86_64", "cp37-abi3-manylinux_2_27_x86_64", "cp37-abi3-manylinux_2_26_x86_64", "cp37-abi3-manylinux_2_25_x86_64", "cp37-abi3-manylinux_2_24_x86_64", "cp37-abi3-manylinux_2_23_x86_64", "cp37-abi3-manylinux_2_22_x86_64", "cp37-abi3-manylinux_2_21_x86_64", "cp37-abi3-manylinux_2_20_x86_64", "cp37-abi3-manylinux_2_19_x86_64", "cp37-abi3-manylinux_2_18_x86_64", "cp37-abi3-manylinux_2_17_x86_64", "cp37-abi3-manylinux2014_x86_64", "cp37-abi3-manylinux_2_16_x86_64", "cp37-abi3-manylinux_2_15_x86_64", "cp37-abi3-manylinux_2_14_x86_64", "cp37-abi3-manylinux_2_13_x86_64", "cp37-abi3-manylinux_2_12_x86_64", "cp37-abi3-manylinux2010_x86_64", "cp37-abi3-manylinux_2_11_x86_64", "cp37-abi3-manylinux_2_10_x86_64", "cp37-abi3-manylinux_2_9_x86_64", "cp37-abi3-manylinux_2_8_x86_64", "cp37-abi3-manylinux_2_7_x86_64", "cp37-abi3-manylinux_2_6_x86_64", "cp37-abi3-manylinux_2_5_x86_64", "cp37-abi3-manylinux1_x86_64", "cp36-abi3-linux_x86_64", "cp36-abi3-manylinux_2_32_x86_64", "cp36-abi3-manylinux_2_31_x86_64", "cp36-abi3-manylinux_2_30_x86_64", "cp36-abi3-manylinux_2_29_x86_64", "cp36-abi3-manylinux_2_28_x86_64", "cp36-abi3-manylinux_2_27_x86_64", "cp36-abi3-manylinux_2_26_x86_64", "cp36-abi3-manylinux_2_25_x86_64", "cp36-abi3-manylinux_2_24_x86_64", "cp36-abi3-manylinux_2_23_x86_64", "cp36-abi3-manylinux_2_22_x86_64", "cp36-abi3-manylinux_2_21_x86_64", "cp36-abi3-manylinux_2_20_x86_64", "cp36-abi3-manylinux_2_19_x86_64", "cp36-abi3-manylinux_2_18_x86_64", "cp36-abi3-manylinux_2_17_x86_64", "cp36-abi3-manylinux2014_x86_64", "cp36-abi3-manylinux_2_16_x86_64", "cp36-abi3-manylinux_2_15_x86_64", "cp36-abi3-manylinux_2_14_x86_64", "cp36-abi3-manylinux_2_13_x86_64", "cp36-abi3-manylinux_2_12_x86_64", "cp36-abi3-manylinux2010_x86_64", "cp36-abi3-manylinux_2_11_x86_64", "cp36-abi3-manylinux_2_10_x86_64", "cp36-abi3-manylinux_2_9_x86_64", "cp36-abi3-manylinux_2_8_x86_64", "cp36-abi3-manylinux_2_7_x86_64", "cp36-abi3-manylinux_2_6_x86_64", "cp36-abi3-manylinux_2_5_x86_64", "cp36-abi3-manylinux1_x86_64", "cp35-abi3-linux_x86_64", "cp35-abi3-manylinux_2_32_x86_64", "cp35-abi3-manylinux_2_31_x86_64", "cp35-abi3-manylinux_2_30_x86_64", "cp35-abi3-manylinux_2_29_x86_64", "cp35-abi3-manylinux_2_28_x86_64", "cp35-abi3-manylinux_2_27_x86_64", "cp35-abi3-manylinux_2_26_x86_64", "cp35-abi3-manylinux_2_25_x86_64", "cp35-abi3-manylinux_2_24_x86_64", "cp35-abi3-manylinux_2_23_x86_64", "cp35-abi3-manylinux_2_22_x86_64", "cp35-abi3-manylinux_2_21_x86_64", "cp35-abi3-manylinux_2_20_x86_64", "cp35-abi3-manylinux_2_19_x86_64", "cp35-abi3-manylinux_2_18_x86_64", "cp35-abi3-manylinux_2_17_x86_64", "cp35-abi3-manylinux2014_x86_64", "cp35-abi3-manylinux_2_16_x86_64", "cp35-abi3-manylinux_2_15_x86_64", "cp35-abi3-manylinux_2_14_x86_64", "cp35-abi3-manylinux_2_13_x86_64", "cp35-abi3-manylinux_2_12_x86_64", "cp35-abi3-manylinux2010_x86_64", "cp35-abi3-manylinux_2_11_x86_64", "cp35-abi3-manylinux_2_10_x86_64", "cp35-abi3-manylinux_2_9_x86_64", "cp35-abi3-manylinux_2_8_x86_64", "cp35-abi3-manylinux_2_7_x86_64", "cp35-abi3-manylinux_2_6_x86_64", "cp35-abi3-manylinux_2_5_x86_64", "cp35-abi3-manylinux1_x86_64", "cp34-abi3-linux_x86_64", "cp34-abi3-manylinux_2_32_x86_64", "cp34-abi3-manylinux_2_31_x86_64", "cp34-abi3-manylinux_2_30_x86_64", "cp34-abi3-manylinux_2_29_x86_64", "cp34-abi3-manylinux_2_28_x86_64", "cp34-abi3-manylinux_2_27_x86_64", "cp34-abi3-manylinux_2_26_x86_64", "cp34-abi3-manylinux_2_25_x86_64", "cp34-abi3-manylinux_2_24_x86_64", "cp34-abi3-manylinux_2_23_x86_64", "cp34-abi3-manylinux_2_22_x86_64", "cp34-abi3-manylinux_2_21_x86_64", "cp34-abi3-manylinux_2_20_x86_64", "cp34-abi3-manylinux_2_19_x86_64", "cp34-abi3-manylinux_2_18_x86_64", "cp34-abi3-manylinux_2_17_x86_64", "cp34-abi3-manylinux2014_x86_64", "cp34-abi3-manylinux_2_16_x86_64", "cp34-abi3-manylinux_2_15_x86_64", "cp34-abi3-manylinux_2_14_x86_64", "cp34-abi3-manylinux_2_13_x86_64", "cp34-abi3-manylinux_2_12_x86_64", "cp34-abi3-manylinux2010_x86_64", "cp34-abi3-manylinux_2_11_x86_64", "cp34-abi3-manylinux_2_10_x86_64", "cp34-abi3-manylinux_2_9_x86_64", "cp34-abi3-manylinux_2_8_x86_64", "cp34-abi3-manylinux_2_7_x86_64", "cp34-abi3-manylinux_2_6_x86_64", "cp34-abi3-manylinux_2_5_x86_64", "cp34-abi3-manylinux1_x86_64", "cp33-abi3-linux_x86_64", "cp33-abi3-manylinux_2_32_x86_64", "cp33-abi3-manylinux_2_31_x86_64", "cp33-abi3-manylinux_2_30_x86_64", "cp33-abi3-manylinux_2_29_x86_64", "cp33-abi3-manylinux_2_28_x86_64", "cp33-abi3-manylinux_2_27_x86_64", "cp33-abi3-manylinux_2_26_x86_64", "cp33-abi3-manylinux_2_25_x86_64", "cp33-abi3-manylinux_2_24_x86_64", "cp33-abi3-manylinux_2_23_x86_64", "cp33-abi3-manylinux_2_22_x86_64", "cp33-abi3-manylinux_2_21_x86_64", "cp33-abi3-manylinux_2_20_x86_64", "cp33-abi3-manylinux_2_19_x86_64", "cp33-abi3-manylinux_2_18_x86_64", "cp33-abi3-manylinux_2_17_x86_64", "cp33-abi3-manylinux2014_x86_64", "cp33-abi3-manylinux_2_16_x86_64", "cp33-abi3-manylinux_2_15_x86_64", "cp33-abi3-manylinux_2_14_x86_64", "cp33-abi3-manylinux_2_13_x86_64", "cp33-abi3-manylinux_2_12_x86_64", "cp33-abi3-manylinux2010_x86_64", "cp33-abi3-manylinux_2_11_x86_64", "cp33-abi3-manylinux_2_10_x86_64", "cp33-abi3-manylinux_2_9_x86_64", "cp33-abi3-manylinux_2_8_x86_64", "cp33-abi3-manylinux_2_7_x86_64", "cp33-abi3-manylinux_2_6_x86_64", "cp33-abi3-manylinux_2_5_x86_64", "cp33-abi3-manylinux1_x86_64", "cp32-abi3-linux_x86_64", "cp32-abi3-manylinux_2_32_x86_64", "cp32-abi3-manylinux_2_31_x86_64", "cp32-abi3-manylinux_2_30_x86_64", "cp32-abi3-manylinux_2_29_x86_64", "cp32-abi3-manylinux_2_28_x86_64", "cp32-abi3-manylinux_2_27_x86_64", "cp32-abi3-manylinux_2_26_x86_64", "cp32-abi3-manylinux_2_25_x86_64", "cp32-abi3-manylinux_2_24_x86_64", "cp32-abi3-manylinux_2_23_x86_64", "cp32-abi3-manylinux_2_22_x86_64", "cp32-abi3-manylinux_2_21_x86_64", "cp32-abi3-manylinux_2_20_x86_64", "cp32-abi3-manylinux_2_19_x86_64", "cp32-abi3-manylinux_2_18_x86_64", "cp32-abi3-manylinux_2_17_x86_64", "cp32-abi3-manylinux2014_x86_64", "cp32-abi3-manylinux_2_16_x86_64", "cp32-abi3-manylinux_2_15_x86_64", "cp32-abi3-manylinux_2_14_x86_64", "cp32-abi3-manylinux_2_13_x86_64", "cp32-abi3-manylinux_2_12_x86_64", "cp32-abi3-manylinux2010_x86_64", "cp32-abi3-manylinux_2_11_x86_64", "cp32-abi3-manylinux_2_10_x86_64", "cp32-abi3-manylinux_2_9_x86_64", "cp32-abi3-manylinux_2_8_x86_64", "cp32-abi3-manylinux_2_7_x86_64", "cp32-abi3-manylinux_2_6_x86_64", "cp32-abi3-manylinux_2_5_x86_64", "cp32-abi3-manylinux1_x86_64", "py312-none-linux_x86_64", "py312-none-manylinux_2_32_x86_64", "py312-none-manylinux_2_31_x86_64", "py312-none-manylinux_2_30_x86_64", "py312-none-manylinux_2_29_x86_64", "py312-none-manylinux_2_28_x86_64", "py312-none-manylinux_2_27_x86_64", "py312-none-manylinux_2_26_x86_64", "py312-none-manylinux_2_25_x86_64", "py312-none-manylinux_2_24_x86_64", "py312-none-manylinux_2_23_x86_64", "py312-none-manylinux_2_22_x86_64", "py312-none-manylinux_2_21_x86_64", "py312-none-manylinux_2_20_x86_64", "py312-none-manylinux_2_19_x86_64", "py312-none-manylinux_2_18_x86_64", "py312-none-manylinux_2_17_x86_64", "py312-none-manylinux2014_x86_64", "py312-none-manylinux_2_16_x86_64", "py312-none-manylinux_2_15_x86_64", "py312-none-manylinux_2_14_x86_64", "py312-none-manylinux_2_13_x86_64", "py312-none-manylinux_2_12_x86_64", "py312-none-manylinux2010_x86_64", "py312-none-manylinux_2_11_x86_64", "py312-none-manylinux_2_10_x86_64", "py312-none-manylinux_2_9_x86_64", "py312-none-manylinux_2_8_x86_64", "py312-none-manylinux_2_7_x86_64", "py312-none-manylinux_2_6_x86_64", "py312-none-manylinux_2_5_x86_64", "py312-none-manylinux1_x86_64", "py3-none-linux_x86_64", "py3-none-manylinux_2_32_x86_64", "py3-none-manylinux_2_31_x86_64", "py3-none-manylinux_2_30_x86_64", "py3-none-manylinux_2_29_x86_64", "py3-none-manylinux_2_28_x86_64", "py3-none-manylinux_2_27_x86_64", "py3-none-manylinux_2_26_x86_64", "py3-none-manylinux_2_25_x86_64", "py3-none-manylinux_2_24_x86_64", "py3-none-manylinux_2_23_x86_64", "py3-none-manylinux_2_22_x86_64", "py3-none-manylinux_2_21_x86_64", "py3-none-manylinux_2_20_x86_64", "py3-none-manylinux_2_19_x86_64", "py3-none-manylinux_2_18_x86_64", "py3-none-manylinux_2_17_x86_64", "py3-none-manylinux2014_x86_64", "py3-none-manylinux_2_16_x86_64", "py3-none-manylinux_2_15_x86_64", "py3-none-manylinux_2_14_x86_64", "py3-none-manylinux_2_13_x86_64", "py3-none-manylinux_2_12_x86_64", "py3-none-manylinux2010_x86_64", "py3-none-manylinux_2_11_x86_64", "py3-none-manylinux_2_10_x86_64", "py3-none-manylinux_2_9_x86_64", "py3-none-manylinux_2_8_x86_64", "py3-none-manylinux_2_7_x86_64", "py3-none-manylinux_2_6_x86_64", "py3-none-manylinux_2_5_x86_64", "py3-none-manylinux1_x86_64", "py311-none-linux_x86_64", "py311-none-manylinux_2_32_x86_64", "py311-none-manylinux_2_31_x86_64", "py311-none-manylinux_2_30_x86_64", "py311-none-manylinux_2_29_x86_64", "py311-none-manylinux_2_28_x86_64", "py311-none-manylinux_2_27_x86_64", "py311-none-manylinux_2_26_x86_64", "py311-none-manylinux_2_25_x86_64", "py311-none-manylinux_2_24_x86_64", "py311-none-manylinux_2_23_x86_64", "py311-none-manylinux_2_22_x86_64", "py311-none-manylinux_2_21_x86_64", "py311-none-manylinux_2_20_x86_64", "py311-none-manylinux_2_19_x86_64", "py311-none-manylinux_2_18_x86_64", "py311-none-manylinux_2_17_x86_64", "py311-none-manylinux2014_x86_64", "py311-none-manylinux_2_16_x86_64", "py311-none-manylinux_2_15_x86_64", "py311-none-manylinux_2_14_x86_64", "py311-none-manylinux_2_13_x86_64", "py311-none-manylinux_2_12_x86_64", "py311-none-manylinux2010_x86_64", "py311-none-manylinux_2_11_x86_64", "py311-none-manylinux_2_10_x86_64", "py311-none-manylinux_2_9_x86_64", "py311-none-manylinux_2_8_x86_64", "py311-none-manylinux_2_7_x86_64", "py311-none-manylinux_2_6_x86_64", "py311-none-manylinux_2_5_x86_64", "py311-none-manylinux1_x86_64", "py310-none-linux_x86_64", "py310-none-manylinux_2_32_x86_64", "py310-none-manylinux_2_31_x86_64", "py310-none-manylinux_2_30_x86_64", "py310-none-manylinux_2_29_x86_64", "py310-none-manylinux_2_28_x86_64", "py310-none-manylinux_2_27_x86_64", "py310-none-manylinux_2_26_x86_64", "py310-none-manylinux_2_25_x86_64", "py310-none-manylinux_2_24_x86_64", "py310-none-manylinux_2_23_x86_64", "py310-none-manylinux_2_22_x86_64", "py310-none-manylinux_2_21_x86_64", "py310-none-manylinux_2_20_x86_64", "py310-none-manylinux_2_19_x86_64", "py310-none-manylinux_2_18_x86_64", "py310-none-manylinux_2_17_x86_64", "py310-none-manylinux2014_x86_64", "py310-none-manylinux_2_16_x86_64", "py310-none-manylinux_2_15_x86_64", "py310-none-manylinux_2_14_x86_64", "py310-none-manylinux_2_13_x86_64", "py310-none-manylinux_2_12_x86_64", "py310-none-manylinux2010_x86_64", "py310-none-manylinux_2_11_x86_64", "py310-none-manylinux_2_10_x86_64", "py310-none-manylinux_2_9_x86_64", "py310-none-manylinux_2_8_x86_64", "py310-none-manylinux_2_7_x86_64", "py310-none-manylinux_2_6_x86_64", "py310-none-manylinux_2_5_x86_64", "py310-none-manylinux1_x86_64", "py39-none-linux_x86_64", "py39-none-manylinux_2_32_x86_64", "py39-none-manylinux_2_31_x86_64", "py39-none-manylinux_2_30_x86_64", "py39-none-manylinux_2_29_x86_64", "py39-none-manylinux_2_28_x86_64", "py39-none-manylinux_2_27_x86_64", "py39-none-manylinux_2_26_x86_64", "py39-none-manylinux_2_25_x86_64", "py39-none-manylinux_2_24_x86_64", "py39-none-manylinux_2_23_x86_64", "py39-none-manylinux_2_22_x86_64", "py39-none-manylinux_2_21_x86_64", "py39-none-manylinux_2_20_x86_64", "py39-none-manylinux_2_19_x86_64", "py39-none-manylinux_2_18_x86_64", "py39-none-manylinux_2_17_x86_64", "py39-none-manylinux2014_x86_64", "py39-none-manylinux_2_16_x86_64", "py39-none-manylinux_2_15_x86_64", "py39-none-manylinux_2_14_x86_64", "py39-none-manylinux_2_13_x86_64", "py39-none-manylinux_2_12_x86_64", "py39-none-manylinux2010_x86_64", "py39-none-manylinux_2_11_x86_64", "py39-none-manylinux_2_10_x86_64", "py39-none-manylinux_2_9_x86_64", "py39-none-manylinux_2_8_x86_64", "py39-none-manylinux_2_7_x86_64", "py39-none-manylinux_2_6_x86_64", "py39-none-manylinux_2_5_x86_64", "py39-none-manylinux1_x86_64", "py38-none-linux_x86_64", "py38-none-manylinux_2_32_x86_64", "py38-none-manylinux_2_31_x86_64", "py38-none-manylinux_2_30_x86_64", "py38-none-manylinux_2_29_x86_64", "py38-none-manylinux_2_28_x86_64", "py38-none-manylinux_2_27_x86_64", "py38-none-manylinux_2_26_x86_64", "py38-none-manylinux_2_25_x86_64", "py38-none-manylinux_2_24_x86_64", "py38-none-manylinux_2_23_x86_64", "py38-none-manylinux_2_22_x86_64", "py38-none-manylinux_2_21_x86_64", "py38-none-manylinux_2_20_x86_64", "py38-none-manylinux_2_19_x86_64", "py38-none-manylinux_2_18_x86_64", "py38-none-manylinux_2_17_x86_64", "py38-none-manylinux2014_x86_64", "py38-none-manylinux_2_16_x86_64", "py38-none-manylinux_2_15_x86_64", "py38-none-manylinux_2_14_x86_64", "py38-none-manylinux_2_13_x86_64", "py38-none-manylinux_2_12_x86_64", "py38-none-manylinux2010_x86_64", "py38-none-manylinux_2_11_x86_64", "py38-none-manylinux_2_10_x86_64", "py38-none-manylinux_2_9_x86_64", "py38-none-manylinux_2_8_x86_64", "py38-none-manylinux_2_7_x86_64", "py38-none-manylinux_2_6_x86_64", "py38-none-manylinux_2_5_x86_64", "py38-none-manylinux1_x86_64", "py37-none-linux_x86_64", "py37-none-manylinux_2_32_x86_64", "py37-none-manylinux_2_31_x86_64", "py37-none-manylinux_2_30_x86_64", "py37-none-manylinux_2_29_x86_64", "py37-none-manylinux_2_28_x86_64", "py37-none-manylinux_2_27_x86_64", "py37-none-manylinux_2_26_x86_64", "py37-none-manylinux_2_25_x86_64", "py37-none-manylinux_2_24_x86_64", "py37-none-manylinux_2_23_x86_64", "py37-none-manylinux_2_22_x86_64", "py37-none-manylinux_2_21_x86_64", "py37-none-manylinux_2_20_x86_64", "py37-none-manylinux_2_19_x86_64", "py37-none-manylinux_2_18_x86_64", "py37-none-manylinux_2_17_x86_64", "py37-none-manylinux2014_x86_64", "py37-none-manylinux_2_16_x86_64", "py37-none-manylinux_2_15_x86_64", "py37-none-manylinux_2_14_x86_64", "py37-none-manylinux_2_13_x86_64", "py37-none-manylinux_2_12_x86_64", "py37-none-manylinux2010_x86_64", "py37-none-manylinux_2_11_x86_64", "py37-none-manylinux_2_10_x86_64", "py37-none-manylinux_2_9_x86_64", "py37-none-manylinux_2_8_x86_64", "py37-none-manylinux_2_7_x86_64", "py37-none-manylinux_2_6_x86_64", "py37-none-manylinux_2_5_x86_64", "py37-none-manylinux1_x86_64", "py36-none-linux_x86_64", "py36-none-manylinux_2_32_x86_64", "py36-none-manylinux_2_31_x86_64", "py36-none-manylinux_2_30_x86_64", "py36-none-manylinux_2_29_x86_64", "py36-none-manylinux_2_28_x86_64", "py36-none-manylinux_2_27_x86_64", "py36-none-manylinux_2_26_x86_64", "py36-none-manylinux_2_25_x86_64", "py36-none-manylinux_2_24_x86_64", "py36-none-manylinux_2_23_x86_64", "py36-none-manylinux_2_22_x86_64", "py36-none-manylinux_2_21_x86_64", "py36-none-manylinux_2_20_x86_64", "py36-none-manylinux_2_19_x86_64", "py36-none-manylinux_2_18_x86_64", "py36-none-manylinux_2_17_x86_64", "py36-none-manylinux2014_x86_64", "py36-none-manylinux_2_16_x86_64", "py36-none-manylinux_2_15_x86_64", "py36-none-manylinux_2_14_x86_64", "py36-none-manylinux_2_13_x86_64", "py36-none-manylinux_2_12_x86_64", "py36-none-manylinux2010_x86_64", "py36-none-manylinux_2_11_x86_64", "py36-none-manylinux_2_10_x86_64", "py36-none-manylinux_2_9_x86_64", "py36-none-manylinux_2_8_x86_64", "py36-none-manylinux_2_7_x86_64", "py36-none-manylinux_2_6_x86_64", "py36-none-manylinux_2_5_x86_64", "py36-none-manylinux1_x86_64", "py35-none-linux_x86_64", "py35-none-manylinux_2_32_x86_64", "py35-none-manylinux_2_31_x86_64", "py35-none-manylinux_2_30_x86_64", "py35-none-manylinux_2_29_x86_64", "py35-none-manylinux_2_28_x86_64", "py35-none-manylinux_2_27_x86_64", "py35-none-manylinux_2_26_x86_64", "py35-none-manylinux_2_25_x86_64", "py35-none-manylinux_2_24_x86_64", "py35-none-manylinux_2_23_x86_64", "py35-none-manylinux_2_22_x86_64", "py35-none-manylinux_2_21_x86_64", "py35-none-manylinux_2_20_x86_64", "py35-none-manylinux_2_19_x86_64", "py35-none-manylinux_2_18_x86_64", "py35-none-manylinux_2_17_x86_64", "py35-none-manylinux2014_x86_64", "py35-none-manylinux_2_16_x86_64", "py35-none-manylinux_2_15_x86_64", "py35-none-manylinux_2_14_x86_64", "py35-none-manylinux_2_13_x86_64", "py35-none-manylinux_2_12_x86_64", "py35-none-manylinux2010_x86_64", "py35-none-manylinux_2_11_x86_64", "py35-none-manylinux_2_10_x86_64", "py35-none-manylinux_2_9_x86_64", "py35-none-manylinux_2_8_x86_64", "py35-none-manylinux_2_7_x86_64", "py35-none-manylinux_2_6_x86_64", "py35-none-manylinux_2_5_x86_64", "py35-none-manylinux1_x86_64", "py34-none-linux_x86_64", "py34-none-manylinux_2_32_x86_64", "py34-none-manylinux_2_31_x86_64", "py34-none-manylinux_2_30_x86_64", "py34-none-manylinux_2_29_x86_64", "py34-none-manylinux_2_28_x86_64", "py34-none-manylinux_2_27_x86_64", "py34-none-manylinux_2_26_x86_64", "py34-none-manylinux_2_25_x86_64", "py34-none-manylinux_2_24_x86_64", "py34-none-manylinux_2_23_x86_64", "py34-none-manylinux_2_22_x86_64", "py34-none-manylinux_2_21_x86_64", "py34-none-manylinux_2_20_x86_64", "py34-none-manylinux_2_19_x86_64", "py34-none-manylinux_2_18_x86_64", "py34-none-manylinux_2_17_x86_64", "py34-none-manylinux2014_x86_64", "py34-none-manylinux_2_16_x86_64", "py34-none-manylinux_2_15_x86_64", "py34-none-manylinux_2_14_x86_64", "py34-none-manylinux_2_13_x86_64", "py34-none-manylinux_2_12_x86_64", "py34-none-manylinux2010_x86_64", "py34-none-manylinux_2_11_x86_64", "py34-none-manylinux_2_10_x86_64", "py34-none-manylinux_2_9_x86_64", "py34-none-manylinux_2_8_x86_64", "py34-none-manylinux_2_7_x86_64", "py34-none-manylinux_2_6_x86_64", "py34-none-manylinux_2_5_x86_64", "py34-none-manylinux1_x86_64", "py33-none-linux_x86_64", "py33-none-manylinux_2_32_x86_64", "py33-none-manylinux_2_31_x86_64", "py33-none-manylinux_2_30_x86_64", "py33-none-manylinux_2_29_x86_64", "py33-none-manylinux_2_28_x86_64", "py33-none-manylinux_2_27_x86_64", "py33-none-manylinux_2_26_x86_64", "py33-none-manylinux_2_25_x86_64", "py33-none-manylinux_2_24_x86_64", "py33-none-manylinux_2_23_x86_64", "py33-none-manylinux_2_22_x86_64", "py33-none-manylinux_2_21_x86_64", "py33-none-manylinux_2_20_x86_64", "py33-none-manylinux_2_19_x86_64", "py33-none-manylinux_2_18_x86_64", "py33-none-manylinux_2_17_x86_64", "py33-none-manylinux2014_x86_64", "py33-none-manylinux_2_16_x86_64", "py33-none-manylinux_2_15_x86_64", "py33-none-manylinux_2_14_x86_64", "py33-none-manylinux_2_13_x86_64", "py33-none-manylinux_2_12_x86_64", "py33-none-manylinux2010_x86_64", "py33-none-manylinux_2_11_x86_64", "py33-none-manylinux_2_10_x86_64", "py33-none-manylinux_2_9_x86_64", "py33-none-manylinux_2_8_x86_64", "py33-none-manylinux_2_7_x86_64", "py33-none-manylinux_2_6_x86_64", "py33-none-manylinux_2_5_x86_64", "py33-none-manylinux1_x86_64", "py32-none-linux_x86_64", "py32-none-manylinux_2_32_x86_64", "py32-none-manylinux_2_31_x86_64", "py32-none-manylinux_2_30_x86_64", "py32-none-manylinux_2_29_x86_64", "py32-none-manylinux_2_28_x86_64", "py32-none-manylinux_2_27_x86_64", "py32-none-manylinux_2_26_x86_64", "py32-none-manylinux_2_25_x86_64", "py32-none-manylinux_2_24_x86_64", "py32-none-manylinux_2_23_x86_64", "py32-none-manylinux_2_22_x86_64", "py32-none-manylinux_2_21_x86_64", "py32-none-manylinux_2_20_x86_64", "py32-none-manylinux_2_19_x86_64", "py32-none-manylinux_2_18_x86_64", "py32-none-manylinux_2_17_x86_64", "py32-none-manylinux2014_x86_64", "py32-none-manylinux_2_16_x86_64", "py32-none-manylinux_2_15_x86_64", "py32-none-manylinux_2_14_x86_64", "py32-none-manylinux_2_13_x86_64", "py32-none-manylinux_2_12_x86_64", "py32-none-manylinux2010_x86_64", "py32-none-manylinux_2_11_x86_64", "py32-none-manylinux_2_10_x86_64", "py32-none-manylinux_2_9_x86_64", "py32-none-manylinux_2_8_x86_64", "py32-none-manylinux_2_7_x86_64", "py32-none-manylinux_2_6_x86_64", "py32-none-manylinux_2_5_x86_64", "py32-none-manylinux1_x86_64", "py31-none-linux_x86_64", "py31-none-manylinux_2_32_x86_64", "py31-none-manylinux_2_31_x86_64", "py31-none-manylinux_2_30_x86_64", "py31-none-manylinux_2_29_x86_64", "py31-none-manylinux_2_28_x86_64", "py31-none-manylinux_2_27_x86_64", "py31-none-manylinux_2_26_x86_64", "py31-none-manylinux_2_25_x86_64", "py31-none-manylinux_2_24_x86_64", "py31-none-manylinux_2_23_x86_64", "py31-none-manylinux_2_22_x86_64", "py31-none-manylinux_2_21_x86_64", "py31-none-manylinux_2_20_x86_64", "py31-none-manylinux_2_19_x86_64", "py31-none-manylinux_2_18_x86_64", "py31-none-manylinux_2_17_x86_64", "py31-none-manylinux2014_x86_64", "py31-none-manylinux_2_16_x86_64", "py31-none-manylinux_2_15_x86_64", "py31-none-manylinux_2_14_x86_64", "py31-none-manylinux_2_13_x86_64", "py31-none-manylinux_2_12_x86_64", "py31-none-manylinux2010_x86_64", "py31-none-manylinux_2_11_x86_64", "py31-none-manylinux_2_10_x86_64", "py31-none-manylinux_2_9_x86_64", "py31-none-manylinux_2_8_x86_64", "py31-none-manylinux_2_7_x86_64", "py31-none-manylinux_2_6_x86_64", "py31-none-manylinux_2_5_x86_64", "py31-none-manylinux1_x86_64", "py30-none-linux_x86_64", "py30-none-manylinux_2_32_x86_64", "py30-none-manylinux_2_31_x86_64", "py30-none-manylinux_2_30_x86_64", "py30-none-manylinux_2_29_x86_64", "py30-none-manylinux_2_28_x86_64", "py30-none-manylinux_2_27_x86_64", "py30-none-manylinux_2_26_x86_64", "py30-none-manylinux_2_25_x86_64", "py30-none-manylinux_2_24_x86_64", "py30-none-manylinux_2_23_x86_64", "py30-none-manylinux_2_22_x86_64", "py30-none-manylinux_2_21_x86_64", "py30-none-manylinux_2_20_x86_64", "py30-none-manylinux_2_19_x86_64", "py30-none-manylinux_2_18_x86_64", "py30-none-manylinux_2_17_x86_64", "py30-none-manylinux2014_x86_64", "py30-none-manylinux_2_16_x86_64", "py30-none-manylinux_2_15_x86_64", "py30-none-manylinux_2_14_x86_64", "py30-none-manylinux_2_13_x86_64", "py30-none-manylinux_2_12_x86_64", "py30-none-manylinux2010_x86_64", "py30-none-manylinux_2_11_x86_64", "py30-none-manylinux_2_10_x86_64", "py30-none-manylinux_2_9_x86_64", "py30-none-manylinux_2_8_x86_64", "py30-none-manylinux_2_7_x86_64", "py30-none-manylinux_2_6_x86_64", "py30-none-manylinux_2_5_x86_64", "py30-none-manylinux1_x86_64", "cp312-none-any", "py312-none-any", "py3-none-any", "py311-none-any", "py310-none-any", "py39-none-any", "py38-none-any", "py37-none-any", "py36-none-any", "py35-none-any", "py34-none-any", "py33-none-any", "py32-none-any", "py31-none-any", "py30-none-any"], "system": "Linux"}, "scope": "compute_platform_and_filesystem_readiness", "verdict": "INCONCLUSIVE"} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-resolve-1985.json b/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-resolve-1985.json deleted file mode 100644 index 6bfeb6552..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-resolve-1985.json +++ /dev/null @@ -1,54 +0,0 @@ -{ - "verdict": "INCONCLUSIVE", - "scope": "sandbox_dependency_resolution_readiness", - "job": 1985, - "state": "FAILED", - "exit_code": "1:0", - "allocated_seconds": 166, - "allocated_cpus": 4, - "allocated_gpu_count": 0, - "allocated_gpu_seconds": 0, - "requested_memory_gib": 16, - "allocation_released": true, - "source_revision": "26d6d00bb1fd0fa37f39c99d5290da67891d1c5e", - "files_sha256": { - "runtime.py": "82837b968f28ee731b936b25f331e7c3111ce9314bf258a3eb527463a9d3ad26", - "benign_probe.py": "8593cf53ca863b497ccb5ceced2b81d1c846522add2c9acd68b8a03975469d5c", - "prepare-sandbox.sbatch": "c1fa2e5737d602c1c8c0c9fa7e76711dc3bbd55af1436aaa32ab30f34ea2e96c", - "resolver-policy.json": "16cfcc1abc4f835bc99cca87171895976e82357377d4e9789a571df741c6ee84", - "source.json": "63efb30f8ab4005cc8f7ea6bec35cbcaee92a35e62ff571a2260a44c8d2a3308", - "build.in": "f200daa93e8d24937050a3e2f91e705a408f8da5a5ce7a864f1c03d7d1765f60", - "dependencies.in": "4ab24ff00dbfb3bb02ffc52720054433c978213bb1628cb1c5a2bffd99295ab3", - "build.lock": "8f033a8ba24ddabb64ac12e4e5e701d7c52986902a7943cd3c2aee2dc46c0bd1" - }, - "build_lock_packages": [ - [ - "packaging", - "26.3" - ], - [ - "setuptools", - "84.2.1" - ], - [ - "setuptools-scm", - "10.3.4" - ], - [ - "vcs-versioning", - "2.6.0" - ], - [ - "wheel", - "0.48.0" - ] - ], - "runtime_dependency_lock_created": false, - "freeze_manifest_created": false, - "environment_created": false, - "dependencies_installed": false, - "model_executed": false, - "generated_code_executed": false, - "reason": "Official wget>=3.2 dependency has no usable wheel under the reviewed no-build resolver policy.", - "followup": "Review explicit source-only distributions and locked build tools before any separate controlled build; keep unrestricted resolver builds disabled." -} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-resolve-1991.json b/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-resolve-1991.json deleted file mode 100644 index e5569939f..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-resolve-1991.json +++ /dev/null @@ -1,139 +0,0 @@ -{ - "scope": "sandbox_dependency_resolution_readiness", - "verdict": "INCONCLUSIVE", - "job": 1991, - "state": "COMPLETED", - "exit_code": "0:0", - "allocated_seconds": 681, - "allocated_cpus": 4, - "allocated_gpu_count": 0, - "allocation_released": true, - "requested_memory_gib": 16, - "freeze_manifest_sha256": "ba7fe57fbb62274e08c14cb839c47db9dbbafc22e6ce83d2e3afffd18c8da833", - "files_sha256": { - "runtime.py": "82837b968f28ee731b936b25f331e7c3111ce9314bf258a3eb527463a9d3ad26", - "benign_probe.py": "8593cf53ca863b497ccb5ceced2b81d1c846522add2c9acd68b8a03975469d5c", - "prepare-sandbox.sbatch": "6067d950b91d200ce9bd95e8efbebd908ec92c1fce20006ea616da1f97a0dcb0", - "resolver-policy.json": "00a5d86b5540bccc681bf3b0022594c0621cba2ed6661eebbd32cba4f08335ac", - "source.json": "63efb30f8ab4005cc8f7ea6bec35cbcaee92a35e62ff571a2260a44c8d2a3308", - "build.in": "f200daa93e8d24937050a3e2f91e705a408f8da5a5ce7a864f1c03d7d1765f60", - "dependencies.in": "4ab24ff00dbfb3bb02ffc52720054433c978213bb1628cb1c5a2bffd99295ab3", - "build.lock": "8f033a8ba24ddabb64ac12e4e5e701d7c52986902a7943cd3c2aee2dc46c0bd1", - "dependencies.lock": "2c60d833d1a705050a152c767da55c90d9f30a4b831d140e115dbffe4d6bb823", - "wheel-build-output.sha256": "020c2e2f96dc5ac4b86adb897ea72acde2770124aa296031ffe4bf93da1c173a", - "wheelhouse/tempdir-0.7.1-py3-none-any.whl": "b70640b48d5379585b85ff7dc46a989a8a82ef1292dc22c057a79cf47842256c", - "wheelhouse/wget-3.2-py3-none-any.whl": "c00a5fd7032ccf1c4af99f26155f5a379892d2e9d94520b36859d3f010922553", - "uv-version.txt": "2a8efc3897d42d0a573eb7441c38bf9558000512691ace60cccf946322857da3", - "uv-sha256.txt": "aca6700fbf8490898187af39cfbd3f600dd95fcf825b1b4184e55d1bfcbd0713", - "python-sha256.txt": "2e7848b702b8dae1c0b3eee5a4864627e96f526ed2bc8f2f4049b6412c0e130d" - }, - "runtime_package_count": 92, - "build_package_count": 5, - "direct_requirements_checked": 19, - "all_pins_have_unique_sha256_hashes": true, - "direct_requirements_satisfied": true, - "build_runtime_shared_versions_match": true, - "reviewed_local_wheels_bound_exactly": true, - "source_and_declarations_unchanged": true, - "environment_created": false, - "dependencies_installed": false, - "model_executed": false, - "generated_code_executed": false, - "full_evaluator_executed": false, - "sandbox_qualified": false, - "runtime_packages": { - "aiohappyeyeballs": "2.7.1", - "aiohttp": "3.14.5", - "aiosignal": "1.4.0", - "annotated-doc": "0.0.5", - "annotated-types": "0.8.0", - "anthropic": "2.1.0", - "anyio": "4.15.1", - "appdirs": "1.4.4", - "attrs": "26.1.0", - "boto3": "1.43.117", - "botocore": "1.43.117", - "certifi": "2026.10.19", - "cffi": "2.1.1", - "charset-normalizer": "3.5.2", - "click": "8.5.0", - "cryptography": "50.0.2", - "datasets": "5.1.0", - "dill": "0.4.1", - "docstring-parser": "0.18.0", - "filelock": "4.2.4", - "fire": "0.7.1", - "frozenlist": "1.8.0", - "fsspec": "2026.7.0", - "google-ai-generativelanguage": "0.6.15", - "google-api-core": "2.33.0", - "google-api-python-client": "2.201.0", - "google-auth": "2.62.0", - "google-auth-httplib2": "0.4.4", - "google-generativeai": "0.8.6", - "googleapis-common-protos": "1.75.0", - "grpcio": "1.84.1", - "grpcio-status": "1.71.2", - "h11": "0.16.0", - "hf-xet": "1.7.0", - "httpcore": "1.0.9", - "httpcore2": "2.16.0", - "httplib2": "0.32.0", - "httpx": "0.28.1", - "httpx2": "2.16.0", - "huggingface-hub": "2.2.0", - "idna": "3.20", - "jiter": "0.17.0", - "jmespath": "1.1.0", - "markdown-it-py": "4.2.0", - "mdurl": "0.1.2", - "multidict": "7.1.0", - "multipledispatch": "1.0.0", - "multiprocess": "0.70.19", - "numpy": "2.5.4", - "ollama": "0.6.3", - "openai": "3.36.1", - "opentelemetry-api": "1.45.1", - "packaging": "26.3", - "pandas": "3.1.0", - "propcache": "0.5.4", - "proto-plus": "1.28.2", - "protobuf": "5.29.6", - "psutil": "7.2.2", - "pyarrow": "26.0.0", - "pyasn1": "0.6.4", - "pyasn1-modules": "0.4.2", - "pycparser": "3.12", - "pydantic": "2.14.1", - "pydantic-core": "2.50.1", - "pygments": "2.21.0", - "pyparsing": "3.3.3", - "python-dateutil": "2.9.0.post0", - "pyyaml": "6.0.3", - "regex": "2026.9.29", - "requests": "2.34.2", - "rich": "15.0.0", - "s3transfer": "0.19.3", - "safetensors": "0.8.0", - "shellingham": "1.5.4", - "six": "1.17.0", - "sniffio": "1.3.1", - "tempdir": "0.7.1", - "termcolor": "3.3.0", - "tokenizers": "0.23.3", - "tqdm": "4.70.1", - "transformers": "5.20.2", - "tree-sitter": "0.26.0", - "tree-sitter-python": "0.25.0", - "truststore": "0.10.5", - "typer": "0.27.3", - "typing-extensions": "4.16.0", - "typing-inspection": "0.4.4", - "uritemplate": "4.2.0", - "urllib3": "2.8.0", - "wget": "3.2", - "xxhash": "4.0.2", - "yarl": "1.25.1" - }, - "limitation": "Static identity/direct-requirement checks only; installed dependency closure, imports and containment still need validation." -} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-resolve-impact.md b/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-resolve-impact.md deleted file mode 100644 index 11d735029..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-resolve-impact.md +++ /dev/null @@ -1,34 +0,0 @@ -# Sandbox dependency resolution: preserved no-wheel failure - -Refs [Issue67](https://github.com/InferMatrixLab/InferRSI/issues/67). -**INCONCLUSIVE — no quality or performance qualification.** - -The initial submission draft was rejected before execution: dependency compilation -could execute source build backends before lock review, and inherited UV/PIP -settings could alter package sources. Version2 explicitly disables builds, -clears inherited resolver settings, fixes the PyPI index and records that policy. -Independent review found no blocker for the bounded resolve-only step. - -CPU job1985 failed1:0 after166 allocated seconds on4CPUs and zeroGPUs. Source -checkout and passive declaration parsing completed. The build-tool lock resolved -five packages, but the official wget>=3.2 dependency has no usable wheel under -the no-build policy. Runtime dependencies and the complete freeze manifest did -not resolve. The proposed environment was not created; nothing was installed, -and no dataset, model, generated solution or full evaluator was executed. -Allocation release was checked. Failed output and the original scripts remain -in controlled storage, with sanitized identities in sandbox-resolve-1985.json. - -This is a readiness failure, not model failure. Do not silently allow unrestricted -source builds or remove official dependencies. The next proposed step is static -review of explicitly pinned source-only package archives and exact build tools, -followed by a separately reviewed bounded build if justified. The resulting -wheel bytes and their origin must be authenticated before retrying resolution. -No successor build, install or sandbox probe is claimed here. - -No GPU budget was spent; readiness remains2388/7200GPU-seconds, with the earlier -147seconds accounted separately. There is no measured E2E performance effect. -The actual generated-code executor and full quality freeze remain unqualified. - -Primary package reference: [wget3.2 release files](https://pypi.org/project/wget/3.2/#files). - -KnowledgeStore candidate `case.qwen38-eval-sandbox-preparation-cand`: `sha256:97f1b1cb31bd88aca212725cc15c0cdd75574bf31ea09071e81fa7831d2a9972`; readback verified and rejected draft retained. diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-resolve-v3-impact.md b/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-resolve-v3-impact.md deleted file mode 100644 index 242b5ef51..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-resolve-v3-impact.md +++ /dev/null @@ -1,38 +0,0 @@ -# Reviewed-wheel dependency resolution completed - -Refs [Issue67](https://github.com/InferMatrixLab/InferRSI/issues/67). -**INCONCLUSIVE — a dependency freeze proposal, not campaign activation.** - -The v3 resolver uses the two authenticated wheels from CPU1986 while keeping -source builds disabled. It clears inherited resolver settings and uses only -explicit PyPI and the reviewed local wheelhouse. The fixed EvalPlus source and -official declarations are unchanged. The original draft rejection and CPU1985 -no-wheel failure remain retained; neither was overwritten or relabeled. - -CPU1991 completed0:0 after681 allocated seconds,4CPUs/zeroGPUs, and allocation -release was checked. During the long resolution, a read-only job step observed -new package metadata cache entries; the live job was retained instead of being -resubmitted. Resolution eventually produced92 pinned runtime packages and the -complete dependency freeze proposal. The evaluator environment was not created, -and no package installation, task data, model, generated solution or full -evaluator execution occurred. - -Parent static audit authenticates all15 freeze entries, the unchanged source and -declarations, all19 direct requirement specifiers and shared build/runtime package -versions. Each lock entry has a unique exact pin and SHA256 hashes; there are no -direct URL requirements or hidden index directives. Both local wheel entries bind -only their previously reviewed exact hashes. No torch, vLLM or FlashInfer package -is selected. The dependency lock and reviewed resolver script are retained under -plan/ and patch/diagnostics/; sandbox-resolve-1991.json records complete identities. - -These checks authenticate the proposed inputs, not an installed runtime. Actual -dependency closure, pinned evaluator installation, imports, readonly runtime -identity, isolation and cleanup remain to be checked. The exact source/locks and -separate install/probe script require independent review before that next step. -All generated-code executor and full quality qualification remains open. - -No measured E2E performance effect or GPU budget change: readiness remains -2388/7200GPU-seconds, with the earlier147seconds separate. Statistical/retrieval -policy confirmation and complete campaign freeze are still pending. - -KnowledgeStore candidate `case.qwen38-eval-sandbox-preparation-cand`: `sha256:677ac8f26a6c55a4c3979f7c5a0423929ec7331bc6ebb79e35f9db1a6aed9926`; readback verified and earlier attempts retained. diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-synthetic-2009.json b/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-synthetic-2009.json deleted file mode 100644 index a7c0b8ab3..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-synthetic-2009.json +++ /dev/null @@ -1,88 +0,0 @@ -{ - "scope": "trusted_four_case_synthetic_evaluator", - "verdict": "INCONCLUSIVE", - "job": 2009, - "slurm_state": "COMPLETED", - "slurm_exit_code": "0:0", - "allocated_seconds": 116, - "allocated_cpus": 4, - "allocated_gpus": 0, - "requested_memory_gib": 16, - "walltime_limit_seconds": 900, - "allocation_released": true, - "terminal": { - "completed": true, - "signal": "none", - "exit_code": 0, - "job_id": "2009" - }, - "rootfs_digest": "sha256:2d6fff81631384f7025f222c31115cbddf8153552571a92dcec61a68e15498f5", - "rootfs_json_sha256": "1b514a1d39c33dd273c9b989075dfc7778c1d98cdab7e7b92fc23153fbc0e168", - "readonly_bind_entry_count": 25241, - "rootfs_before_after_equal": true, - "reference_2006_difference_only_probe": true, - "checked_evaluator_python_sources": 43, - "package_count": 97, - "probe_checks_passed": 4, - "postflight_checks_passed": 9, - "probe_stdout_sha256": "b5fd8773611054c083cf9de73ef9a1d28caffad61356fbd95f6cecc39ac0ab5a", - "probe_execution_sha256": "afb4645ce50d5ca267c72368a7325c3deb59f1c51ea61eafe462ef29e09fd347", - "statuses": { - "Synthetic/correct": [ - [ - "pass", - "pass" - ] - ], - "Synthetic/exception": [ - [ - "fail", - "fail" - ] - ], - "Synthetic/input_timeout": [ - [ - "fail", - "fail" - ] - ], - "Synthetic/plus_wrong": [ - [ - "pass", - "fail" - ] - ] - }, - "evaluator_elapsed_seconds": 11.31006149307359, - "evaluator_module_sha256": "47f766df2dbb2280f7dd2c1040bfff0396c2b414a8d44b639424aa941bee7eab", - "sampled_peaks": { - "errors": 1, - "processes": 5, - "samples": 857, - "sum_fds": 58, - "sum_rss_bytes": 412422144, - "sum_vms_bytes": 4910592000, - "threads": 10 - }, - "sampling_failures": [], - "sampling_thread_joined": true, - "known_live_descendants_after_exit": [], - "inside_live_children_after_evaluate": 0, - "sampling_error_reason_not_recorded": true, - "aggregate_hard_bounds_or_exhaustive_cleanup_proven": false, - "generated_code_executed": false, - "real_benchmark_data_mounted": false, - "full_evaluator_executed": true, - "qualified_generated_code_executor": false, - "measured_e2e_effect": null, - "private_evidence_sha256": { - "submission.private.txt": "f57ecfc1e13bf5bc575effdd8785723b8a0aadef6fb0c109bb16fd7af77ac612", - "scontrol-final.private.txt": "41caa1c94fad7d5876ee16a9df9429fe39c2e156db33bf17d08f6a89ba17e63f", - "squeue-final.private.txt": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", - "diagnostic.private.log": "b1a149ec1937495c613cc3af3e507708025f9483609cae586d3e3da7f1e49528", - "output.private.tar": "fb3ed76accb4d569537cb0fc370357684f0426a4d7ba7d4a3278588e6928a0e5" - }, - "readiness_gpu_seconds_used": 2388, - "readiness_gpu_seconds_cap": 7200, - "earlier_diagnostic_gpu_seconds_separate": 147 -} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-synthetic-v3-impact.md b/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-synthetic-v3-impact.md deleted file mode 100644 index 2905cd7c4..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-synthetic-v3-impact.md +++ /dev/null @@ -1,42 +0,0 @@ -# Pinned EvalPlus executes four trusted cases correctly - -Refs [Issue #67](https://github.com/InferMatrixLab/InferRSI/issues/67). -**INCONCLUSIVE for generated-code executor qualification, task quality and E2E.** - -CPU2009 completed0:0 in116seconds on4CPUs/zeroGPUs; its allocation is released. -The actual pinned evaluator API processed four handwritten cases inside the -isolated runtime: correct gives pass/pass, plus-only wrong gives pass/fail, -exception gives fail/fail, and per-input timeout gives fail/fail. The timeout -classification follows the pinned evaluator's exception handler, not its outer -process timeout. These deliberately mixed synthetic cases are not a model -accuracy benchmark. The evaluator call took11.31006149307359seconds; outer probe -exit0, no outer timeout, empty outer stderr. Captured evaluator progress and -results remain intact. - -The trial uses parallel1, explicit2GiB per-process memory and test_details=True. -It inherits CPU2006's loader policy and authenticates that actual successful -reference. The before/after25,241-entry readonly inventories are byte-identical; -only the probe entry differs from the2006reference. All43 pinned EvalPlus Python -source checks and the same97-package metadata inventory remain present. All nine -postflight checks return0. Actual wrapper/runtime/probe bytes are retained. - -The supervisor collected857samples. Observed maxima are5processes,10threads, -412,422,144summed RSS bytes,4,910,592,000summed VMS bytes and58summed descriptors; -these maxima need not occur in the same sample. There was one per-process -observation error, whose reason was not recorded. The failure list is empty, -the observer joined, no known live descendant was observed after exit, and the -inner parent reported zero live children. Polling can miss transient processes: -these observations do not establish aggregate hard limits or exhaustive cleanup. - -Independent pre-run review reran18 completion and13 benign-reference controls. -Nine inherited snapshot Ruff findings are disclosed; the newly authored control -and shell syntax checks pass. The prospective plan remains a historical -pre-execution record. Failed and rejected earlier attempts remain retained. - -No heldout benchmark task or model-generated program ran. No service benchmark -or measured E2E effect. Full generated-code qualification still needs a bounded -execution policy and cleanup evidence before entering the frozen quality -protocol; this trial does not activate that protocol or performance search. -Readiness GPU consumption remains2388/7200seconds, initial147separate. - -KnowledgeStore candidate `case.qwen38-eval-sandbox-preparation-cand` revision `sha256:6d008608b3221cf8d5e9874913ed16a898389c3bf4ae74f2d817eed980caf15d`; readback verified. diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-synthetic-v3-postflight-status.tsv b/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-synthetic-v3-postflight-status.tsv deleted file mode 100644 index e0d514281..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-synthetic-v3-postflight-status.tsv +++ /dev/null @@ -1,9 +0,0 @@ -inventory 0 -runtime-byte-comparison 0 -runtime-script 0 -synthetic-probe 0 -batch-script 0 -freeze-manifest 0 -freeze-files 0 -python-binary 0 -bwrap-binary 0 diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-synthetic-v3-probe-execution.json b/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-synthetic-v3-probe-execution.json deleted file mode 100644 index e51a056dc..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-synthetic-v3-probe-execution.json +++ /dev/null @@ -1 +0,0 @@ -{"executor":{"binary_sha256":"04683adbb7f45376ba56d6936a02203535ff9451b1592299ac4ba69838c33627","kind":"bubblewrap","version":"bubblewrap 0.13.1"},"exit_code":0,"full_evaluator_attempt_requested":true,"generated_code_executed":false,"known_live_descendants_after_exit":[],"phase":"trusted_synthetic_evaluator","policy_script_sha256":"997cbfed8398e8ad62d1beb4f6b4bc6e61a0f4572b9156cfd0c6f3fb52b03233","probe_script_sha256":"04f2f5f7aba97fec1d4bab54c818cbe46f73356d46477afefcd815c33fe5a7ef","qualified_generated_code_executor":false,"rootfs_digest":"sha256:2d6fff81631384f7025f222c31115cbddf8153552571a92dcec61a68e15498f5","sampled_peaks":{"errors":1,"processes":5,"samples":857,"sum_fds":58,"sum_rss_bytes":412422144,"sum_vms_bytes":4910592000,"threads":10},"sampling_failures":[],"sampling_limitation":"Polling may miss transient processes; sampled peaks are not hard bounds or exhaustive cleanup proof.","sampling_thread_joined":true,"stdout_sha256":"b5fd8773611054c083cf9de73ef9a1d28caffad61356fbd95f6cecc39ac0ab5a","timed_out":false,"verdict":"INCONCLUSIVE"} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-synthetic-v3-probe-stdout.json b/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-synthetic-v3-probe-stdout.json deleted file mode 100644 index ac7a0a132..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-synthetic-v3-probe-stdout.json +++ /dev/null @@ -1 +0,0 @@ -{"checks": {"bounded_output": true, "dataset_hash": true, "four_exact_status_pairs": true, "timeout_override_absent": true}, "dataset_sha256": "8098a24b8e5bde365d9d5ae8c2cf0e61d7936f2e99a4411faa284a488c51d430", "elapsed_seconds": 11.31006149307359, "evaluator_module_sha256": "47f766df2dbb2280f7dd2c1040bfff0396c2b414a8d44b639424aa941bee7eab", "evaluator_stderr": "\r0it [00:00, ?it/s]\r4it [00:00, 795.81it/s]\n\r 0%| | 0/4 [00:00=32. This is an upstream different-model observation, not a measured Qwen3.8failure; nevertheless it is a concrete reason to prioritize the originalDeepGEMM plan before spendingGPUtime here. - -Do not extrapolate2076'ssmall-Mmicrobench: itsM32row actuallyusedTriton. No all-M GPUoutput or modelquality result exists. Parentdisposition: analyticallydefer; retainpatch/CPUevidence and sourcecounter-signal. Revisitonlywithprospectivelybounded numerical and taskquality controls. ThisdoesnotweakenIssue67qualitygates or claimFIlargeMisuniversallyinvalid. diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/flashinfer-small-m/actual-2076.json b/evidence/qwen38-h20-fp8-single-request-vllm032/results/flashinfer-small-m/actual-2076.json deleted file mode 100644 index 893112f6e..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/flashinfer-small-m/actual-2076.json +++ /dev/null @@ -1,79 +0,0 @@ -{ - "verdict": "INCONCLUSIVE", - "scope": "single fixed input descriptive engineering E2E; not model quality or vLLM parity", - "job": 2076, - "overall_exit": 124, - "allocated_gpu_seconds": 1320, - "vllm_arm": "startup incomplete at outer HTTP1300s timeout; no request or explicit cleanup result", - "native_output_tokens_identical": true, - "requests": [ - { - "arm": "triton", - "purpose": "warmup", - "valid": true, - "input_tokens": 8192, - "output_tokens": 16, - "ttft_s": 153.77971190097742, - "tpot_s": 0.09321315833367408, - "latency_s": 155.17790927598253, - "wire_sha256": "09d370f3a0e2feb313f588b69a981fd7cd3a3e49e9d436311608ffbb2e50da49" - }, - { - "arm": "triton", - "purpose": "measurement", - "valid": true, - "input_tokens": 8192, - "output_tokens": 2048, - "ttft_s": 151.34831038408447, - "tpot_s": 0.09886639672152936, - "latency_s": 353.7278244730551, - "wire_sha256": "6acf75a785df72415f4cded77423ebdd6961383e99c30de43e8075b35b71219c" - }, - { - "arm": "flashinfer", - "purpose": "warmup", - "valid": true, - "input_tokens": 8192, - "output_tokens": 16, - "ttft_s": 151.57424227509182, - "tpot_s": 0.016324430261738597, - "latency_s": 151.8191087290179, - "wire_sha256": "3a9a7a185fdb3ab8a271af7e7b48e3ad1857441a2bffabb21bc23dd29dd7243e" - }, - { - "arm": "flashinfer", - "purpose": "measurement", - "valid": true, - "input_tokens": 8192, - "output_tokens": 2048, - "ttft_s": 151.3486660369672, - "tpot_s": 0.050836547537371025, - "latency_s": 255.41107884596568, - "wire_sha256": "b3559857c5fd9d6e9415b9991d92fc8816606e18a5e55a646de2c00ce8eafada" - } - ], - "candidate_over_control": { - "ttft_s": 1.000002349896618, - "tpot_s": 0.5141943999492473, - "latency_s": 0.7220553803661024 - }, - "native_cleanup": [ - { - "phase": "cleanup", - "arm": "triton", - "exit_code": 0, - "forced": false, - "owned_group_remaining": 0 - }, - { - "phase": "cleanup", - "arm": "flashinfer", - "exit_code": 0, - "forced": false, - "owned_group_remaining": 0 - } - ], - "source_commit": "8859ef319486eb5ed4a444fc66c1018cbebbab33", - "raw_archive_sha256": "f5398a9465acaa27c0c9885d7b725c9a9eb1bffbdb8f44233e9ad88e2812b958", - "evidence_checks": "rawSSE/client/metadata/timing/count/prompt/terminal/cleanup independently re-read by parent" -} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/flashinfer-small-m/e2e-observation-2076.md b/evidence/qwen38-h20-fp8-single-request-vllm032/results/flashinfer-small-m/e2e-observation-2076.md deleted file mode 100644 index 136995f4f..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/flashinfer-small-m/e2e-observation-2076.md +++ /dev/null @@ -1,20 +0,0 @@ -# First measured native E2E effect - -Governing Issue67; single fixed input diagnostic; **INCONCLUSIVE** for formal service acceptance, task quality and vLLM parity. - -Engine runtime8859ef319486eb5ed4a444fc66c1018cbebbab33 was tested on the same physical H20 using the same installedruntime, pinnedofficialHTTPclient and n8192-s0 input. Bothnativearms completed8192input/2048output,ignoreEOS,concurrency1,after8192/16warmup. The complete2048outputtoken sequences match for this synthetic input. Counts/prompt echoes/terminal/usage/client timings and rawSSE hashes were rechecked. This is not general task-quality evidence. - -| Arm | TTFTseconds | TPOTmilliseconds | Requestseconds | -| --- | ---: | ---: | ---: | -| OriginalTriton |151.348310384|98.866396722|353.727824473| -| FlashInfer smallM |151.348666037|50.836547537|255.411078846| - -Descriptivecandidate/controlratios:TTFT1.000002350,TPOT0.514194400,request0.722055380. Onecase/one repetition perarm; no variability or formal10case/3repeat inference. Nativepostflight active/state counts werezero and bothownedprocessgroups stoppedwithoutforce. Defaultbackend remains unchanged. - -The fulljob FAILED124:0 after1320GPU-seconds: the outer1300-secondHTTPtimeout interrupted the vLLM arm during startup/Graphcapture. No vLLMrequest or explicitobservercleanuprecord was produced. Slurmreports the allocationreleased; do not claim gracefulbaselinecleanup or a completedparitycomparison. Keep this partialresult and timeout unchanged. - -Sixoperatorcontrols hadfiniteoutput and exact changed-inputGraph/eager results. M32usedTriton; onlyM1/M31 qualify actualFlashInferdispatch in this run. Realprojectionmicrobench observations are not general modelquality evidence. Prefill remainedTriton and dominatesTTFT. Thegraphbucket ceiling also causesdemotion during2048-token output; this request measures that deliveredbehavior, not a fullGraphdecode claim. - -Nano-vllmPR11 head a23bfc61f7c946879c8b64c67d5dade2e530ed5f differs from testedruntimeonlyin Markdownheading/whitespace. CPU 108 checks remain applicable. Independent review rehashed the raw archive and all 27 extracted files, matched tested runtime source, and verified all four SSE/client records and cleanup. PR11 merged at 7f5e64d52dc91a66b4a68aadb56641f543d063c8 after review; no repository Actions/status checks were configured for its head. Issue completion remains open. - -Budget after2076:2388earlier+1800conservative2041reservation+11actual2074+1320actual2076=5519/7200readinessGPU-seconds. Initial147seconds remainseparate. The44GPU-hourtotal is unchanged. diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/flashinfer-small-m/knowledge-2076.json b/evidence/qwen38-h20-fp8-single-request-vllm032/results/flashinfer-small-m/knowledge-2076.json deleted file mode 100644 index 3cedc312b..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/flashinfer-small-m/knowledge-2076.json +++ /dev/null @@ -1,5 +0,0 @@ -{ - "ref": "store:case.qwen38-flashinfer-small-m-integration-cand", - "revision": "sha256:0a0cd071051bb6c308fae576b95326c6f037f2f195272ed979b2810cd7a6e51b", - "readback_verified": true -} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/flashinfer-small-m/launch-failures.md b/evidence/qwen38-h20-fp8-single-request-vllm032/results/flashinfer-small-m/launch-failures.md deleted file mode 100644 index ad63cb750..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/flashinfer-small-m/launch-failures.md +++ /dev/null @@ -1,13 +0,0 @@ -# FlashInfer engineering attempts: actual outcomes - -Historical launch snapshot before GPU2076. See e2e-observation-2076.md for the subsequent actual result and integration decision. - -Candidate 8859ef319486eb5ed4a444fc66c1018cbebbab33; Issue67; 2026-10-23. INCONCLUSIVE for model quality and service performance. - -Job2041 (v3) exited1 before the first operator result: selected runtime had ninja installed but wrapper PATH omitted its bin directory. The preceding pending/estimated-start observation is superseded. Raw log starts02:43:41UTC; exit-file mtime02:43:55UTC. The scheduler record expired before final capture, so this is an approximately14-second log window, not certified allocated duration. Reserve its full1800-second limit in the budget. No HTTP requests or measured performance results exist. - -Job2074 (v4) FAILED1:0 with scheduler RunTime11seconds, oneGPU, released. Adding runtimebin resolved ninja; the installed FlashInfer C++ path then invoked bare nvcc, absent from PATH. It also selected a default TensorRT-LLM cache outside the required task root. No GEMM output/model/HTTP result was produced. The fallback-directory violation is preserved; no later run is allowed to reuse it. This is an environment failure, not measured kernel regression. - -Prospectivev5 supplies CUDA_HOME, explicit TRTLLM_DG_NVCC_COMPILER and root-contained TRTLLM_DG_CACHE_DIR/DG_JIT_CACHE_DIR; it restores the already verified abstract VLLM IPC endpoint. Engine/operator/HTTP bytes are unchanged. Independent review caught a v4 EXIT trap that could replace the original exit on scheduler-query failure; v5 uses best-effort scheduler capture and explicit original exit. A trap-time RUNNING snapshot is not final completion evidence. v5 is not yet executed in this record. - -Budget: prior2388 + conservative2041reserve1800 + actual2074elapsed11 =4199 of7200 readiness GPU-seconds committed. A prospective1800-second retry remains within this phase cap. Initial147seconds remain separately recorded. No service gain, final quality, or parity is claimed. diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/flashinfer-small-m/operator-2076.json b/evidence/qwen38-h20-fp8-single-request-vllm032/results/flashinfer-small-m/operator-2076.json deleted file mode 100644 index b899b0e0d..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/flashinfer-small-m/operator-2076.json +++ /dev/null @@ -1,131 +0,0 @@ -{ - "scope": "operator_integration_not_model_quality_or_service_acceptance", - "rows": [ - { - "shape": [ - 1, - 128, - 64 - ], - "diagnostic_nrmse": 0.0008510621846653521, - "graph_changed_inputs_exact": [ - true, - true - ], - "kernel_calls": { - "flashinfer": 30, - "triton": 0 - }, - "milliseconds": { - "triton": 0.034441599249839784, - "flashinfer": 0.03458400070667267 - } - }, - { - "shape": [ - 31, - 128, - 64 - ], - "diagnostic_nrmse": 0.0019299185369163752, - "graph_changed_inputs_exact": [ - true, - true - ], - "kernel_calls": { - "flashinfer": 30, - "triton": 0 - }, - "milliseconds": { - "triton": 0.032743999361991884, - "flashinfer": 0.03487200140953064 - } - }, - { - "shape": [ - 32, - 128, - 64 - ], - "diagnostic_nrmse": 0.0, - "graph_changed_inputs_exact": [ - true, - true - ], - "kernel_calls": { - "flashinfer": 0, - "triton": 30 - }, - "milliseconds": { - "triton": 0.03271040022373199, - "flashinfer": 0.031436800956726074 - } - }, - { - "shape": [ - 1, - 5120, - 5120 - ], - "diagnostic_nrmse": 0.0038501042872667313, - "graph_changed_inputs_exact": [ - true, - true - ], - "kernel_calls": { - "flashinfer": 30, - "triton": 0 - }, - "milliseconds": { - "triton": 0.12773439884185792, - "flashinfer": 0.03402880132198334 - } - }, - { - "shape": [ - 1, - 5120, - 17408 - ], - "diagnostic_nrmse": 0.002684346865862608, - "graph_changed_inputs_exact": [ - true, - true - ], - "kernel_calls": { - "flashinfer": 30, - "triton": 0 - }, - "milliseconds": { - "triton": 0.1725200057029724, - "flashinfer": 0.03452320098876953 - } - }, - { - "shape": [ - 1, - 17408, - 5120 - ], - "diagnostic_nrmse": 0.0036326672416180372, - "graph_changed_inputs_exact": [ - true, - true - ], - "kernel_calls": { - "flashinfer": 30, - "triton": 0 - }, - "milliseconds": { - "triton": 0.48456478118896484, - "flashinfer": 0.03461920022964478 - } - } - ], - "versions": { - "torch": "2.13.0", - "triton": "3.7.1", - "flashinfer-python": "0.7.0.post1" - }, - "flashinfer_source_sha256": "6ab2f4f8496b8fff54319a299565dbb41858012edf54f86c721578504a34108c" -} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/flashinfer-small-m/progress.md b/evidence/qwen38-h20-fp8-single-request-vllm032/results/flashinfer-small-m/progress.md deleted file mode 100644 index feeb2318f..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/flashinfer-small-m/progress.md +++ /dev/null @@ -1,23 +0,0 @@ -# FlashInfer small-M FP8 engine integration - -Governing acceptance: InferRSI Issue #67. Date: 2026-10-23. -Status: candidate; service performance and model quality INCONCLUSIVE. - -The owner explicitly prioritized starting engine performance optimization and parking further evaluator diagnosis. Implement one model-plane hypothesis: opt-in FlashInfer SM90 GEMM for M<32, preserving the existing Triton prefill path. The prior observation that a FlashInfer environment label did not change dispatch motivated binding selection to module construction and reporting actual dispatch counters. Python counters cover warmup/capture and exclude Graph replay. - -Pinned vLLM 0.32.0 source uses BF16 activations, official E4M3 weights and FP32 block multipliers in its FlashInfer small-M call. Its larger-M path uses DeepGEMM; that is not implemented in this slice. Installed FlashInfer 0.7.0.post1 source confirms K multiple128, N multiple64 and ceil(N/128) by K/128 per-block scales. Its workspace allocation and cold JIT require actual warmed CUDA Graph verification; code inspection alone does not qualify this path. - -Engine commit 8859ef319486eb5ed4a444fc66c1018cbebbab33, base 3e918df2dd7b478d3d2985350bb5efb523ea4981, is pushed as nano-vllm PR11. CPU Slurm2037 completed exit0 in14s, 108 tests passed in11.05s, 8CPU/0GPU, allocation released. Tests cover shape boundary, unchanged scale objects, missing dependency failure, constructed-backend reporting and loader/service/runner regressions. Mocked dispatch is not CUDA evidence. - -Attempts retained: -- GPU2038: frozen v1 observer submitted but stayed pending for resources; cancelled before allocation,0GPU. Independent review identified an observer regression: it required one token per SSE choice, although vLLM may merge tokens. This was not an engine failure or measured performance result. -- GPU2040: v2 observer had an unmatched parenthesis; AST/Ruff rejected it locally. The caller shell failed to stop and still submitted. It was cancelled while pending,0GPU. Preserve this tooling failure; future submission follows separate successful syntax checks. -- Prospective v3 corrects the observer, requires one final DONE, counts exact output token IDs/usage and accepts multiple tokens per choice, with ITL counted by choice event. OMP override is removed. AST/Ruff/bash syntax checks passed. Slurm2041 was submitted after successful syntax checks and is pending resources/priority; actual GPU results remain pending. - -The bounded experiment requests1H20/30min from authorized readiness, leaving prior usage2388/7200GPU-seconds unchanged until allocation. All arms use the existing vLLM runtime environment without dependency mutation; historical native-runtime results are not a matched control. Six operator shapes check finite outputs and changed-input Graph/eager equality. Then each native Triton/native FlashInfer/vLLM arm uses the same official client, fixed n8192-s0,16-output warmup and2048-output measured request, ignoringEOS, concurrency1. Retain all output IDs, raw SSE, timings, source identities and cleanup. - -A single case is an engineering feasibility observation, not the formal10-case/3-repeat matrix, a speedup claim, task-quality qualification or final parity. Full evaluator qualification and statistical approval remain pending. Next decision: inspect actual operator and HTTP results before promoting or rejecting this implementation; preserve any failure with its own scope. No reusable knowledge promotion occurs. - -Independent reviewer inspected exact engine head/base and installed FlashInfer source, confirmed the CPU log, verified all five v3 freeze hashes, and exercised four observer controls (merged tokens accepted; duplicate DONE, post-DONE data and incorrect ITL count rejected). Recommendation: proceed with the bounded GPU diagnostic, without quality/performance qualification. - -Knowledge candidate: `store:case.qwen38-flashinfer-small-m-integration-cand`, initial retained revision `sha256:481253f455d97aed6f15db825cbaf2f4c71a6f18d80c421f53bf442495405c77`; this public status supplements the initial pending proposal. diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/flashinfer-small-m/status.json b/evidence/qwen38-h20-fp8-single-request-vllm032/results/flashinfer-small-m/status.json deleted file mode 100644 index 8b70fcc6a..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/flashinfer-small-m/status.json +++ /dev/null @@ -1,62 +0,0 @@ -{ - "verdict": "INCONCLUSIVE", - "scope": "implementation and CPU tests; GPU queued", - "engine_pr": "https://github.com/jiangkuaixue123/nano-vllm/pull/11", - "candidate": "8859ef319486eb5ed4a444fc66c1018cbebbab33", - "cpu": { - "job": 2037, - "state": "COMPLETED", - "exit_code": "0:0", - "elapsed_seconds": 14, - "gpus": 0, - "tests_passed": 108, - "pytest_seconds": 11.05, - "log_sha256": "a33665bf666708d40b29b7271b79e716c5e0b3004758b3b28cf1e0b53d42a7d9" - }, - "gpu_attempts": [ - { - "job": 2038, - "state": "CANCELLED", - "allocated": false, - "gpu_seconds": 0, - "reason": "observer review correction while pending" - }, - { - "job": 2040, - "state": "CANCELLED", - "allocated": false, - "gpu_seconds": 0, - "reason": "local syntax failure; submission caller did not stop; cancelled while pending" - }, - { - "job": 2041, - "state": "PENDING", - "allocated": false, - "reason": "scheduler resources/priority", - "max_seconds": 1800, - "requested_gpus": 1 - } - ], - "readiness_gpu_seconds_used": 2388, - "gpu_seconds_cap": 7200, - "independent_review": { - "reviewer": "Codex /root/review_fp8_graph", - "base": "3e918df2dd7b478d3d2985350bb5efb523ea4981", - "head": "8859ef319486eb5ed4a444fc66c1018cbebbab33", - "patch_sha256": "1f8d55bc9f694aeaf2c59040c1ec7493bcaaacddcf8bfc2279208901f38de54f", - "implementation_blockers": [], - "recommendation": "proceed with bounded GPU integration; actual GPU results required", - "observer_controls": [ - "merged two-token choice accepted", - "duplicate DONE rejected", - "data after DONE rejected", - "token-count-based incorrect ITL rejected" - ] - }, - "source_artifacts": { - "source.tar": "2f431b13e97bc0a26fe4d685be1b7896a53d9a0b93d582e692dc2ea53976fc3d", - "operator_probe.py": "fa45568dcc07a227b513d03dd2606e000b96dc68b7b099a65ce220c62423f8e2", - "http_probe.py": "e20a54b81e8f5a1b6d6dcfd86c41fd1bc98b7e910763ae3445637cce69dc7512", - "gpu.sbatch": "d48c728806168fa862b9afa4e6f099e8450d9b17b03f2008b69a046f4800a703" - } -} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/formal-8k-baseline/independent-review.json b/evidence/qwen38-h20-fp8-single-request-vllm032/results/formal-8k-baseline/independent-review.json deleted file mode 100644 index b1bb2e8e5..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/formal-8k-baseline/independent-review.json +++ /dev/null @@ -1,86 +0,0 @@ -{ - "scope": "independent baseline-only raw accounting and descriptive statistics; no paired verdict", - "verdict": "INCONCLUSIVE", - "source_audit_sha256": "c1ddee3ccbc5815ab428a2ec36ea279fbb8c232a6f07d246302610042b925ff7", - "raw_audit_recomputed_equal": true, - "requests": 33, - "warmups": 3, - "measurements": 30, - "case_counts": { - "n8192-s0": 3, - "n8192-s1": 3, - "n8192-s2": 3, - "n8192-s3": 3, - "n8192-s4": 3, - "n8192-s5": 3, - "n8192-s6": 3, - "n8192-s7": 3, - "n8192-s8": 3, - "n8192-s9": 3 - }, - "restart_count": 3, - "cleanup_count": 3, - "all_clean_exit0_unforced": true, - "guard_equal": true, - "guard_model_files": 74, - "guard_release_file_count": 5446, - "guard_package_count": 214, - "metrics": { - "ttft_s": { - "n": 30, - "median": 1.681210256938357, - "mean": 1.6816727065403636, - "min": 1.6798835719237104, - "max": 1.687585426028818, - "per_repetition_median": { - "0": 1.6813473429647274, - "1": 1.6821416944731027, - "2": 1.68089741357835 - } - }, - "tpot_s": { - "n": 30, - "median": 0.010452101139955015, - "mean": 0.010469657965684623, - "min": 0.010357489934055527, - "max": 0.0107228563663754, - "per_repetition_median": { - "0": 0.010444566901812113, - "1": 0.01045334976255547, - "2": 0.010455227820450667 - } - }, - "latency_s": { - "n": 30, - "median": 23.077676588494796, - "mean": 23.113062562296786, - "min": 22.883453769958578, - "max": 23.632067498983815, - "per_repetition_median": { - "0": 23.06043024198152, - "1": 23.079509996518027, - "2": 23.08356784749776 - } - } - }, - "artifacts": { - "raw-audit.json": "9b7ee8644b05ae2bb59a99fe879c798f0bdcd5bdc2fa40a0534ed1b85d8fa2c2", - "integrity-before.json": "5077d874c429e86df9099f78d17dc123f71803c86c838b138af9202547af65ae", - "integrity-after.json": "5077d874c429e86df9099f78d17dc123f71803c86c838b138af9202547af65ae", - "device.private.json": "3c991e29211cc6de3f0386e899eb13c799cd3fd68aa28159938a0895219b7c63" - }, - "client_sha256": "f31fae8c573ecc4bf8e31dae03823502e71404333e8111f4b535432758302b29", - "collector_sha256": "2bc77b7203d712e97e6f05167bdc3ae368b4665e91f0d17360b319c8256bef64", - "actual_gpu_seconds": 1214, - "scheduler_receipt_sha256": "ebd2627ea220b0d9859b5c01215018525408877080efd3db2cc221f8c2cf7fff", - "squeue_empty": true, - "server_log_caveats": { - "all_three_starts": [ - "TritonBundler missing-cubin warnings during startup", - "vLLM default abort/timeout=0 internal EngineCore force-kill during shutdown", - "one resource_tracker semaphore warning" - ], - "outer_cleanup_only": "all owned groups empty, server exit0, observer forced=false; does not imply all internal processes exited gracefully" - }, - "baseline_phase_used_seconds_including_2139": 1221 -} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/formal-8k-baseline/observation.md b/evidence/qwen38-h20-fp8-single-request-vllm032/results/formal-8k-baseline/observation.md deleted file mode 100644 index 7c48cce39..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/formal-8k-baseline/observation.md +++ /dev/null @@ -1,31 +0,0 @@ -# Completed formal 8K baseline observations - -Governing acceptance Issue67; harness plane; baseline-only observation, not a -paired candidate result or campaign PASS. Original2139 failed attempt remains -in observation-preparation.md and its immutable raw artifacts. - -GPU2140 completed0:0 in1214allocatedGPU-seconds, one H20 and8CPU. Three restarts -produced3discarded warmups and30measured requests (10fixed8Kinputs×3), each with -2048actual generated tokens ignoringEOS. Independent audit re-parsed every SSE -and official client result: prompt/output identity, usage, text, DONE, choice -counts and ITL/timing all match. The saved raw audit is byte-value equivalent. -Full before/after guard receipts are identical:74model/tokenizer/config files, -5446installed vLLM members and214packages. All3outer owned-group cleanups are -exit0,unforced,remaining0,errors[]. Baseline logs retain missing-cubin warnings, -internal EngineCore force-kill after abort timeout0, and a semaphore warning; -outer cleanup success does not mean every internal worker stopped gracefully. - -TTFT median1.681210257s; TPOT median10.452101140ms. Observed min/max TTFT -1.679883572/1.687585426s; TPOT10.357489934/10.722856366ms. Request latency median -23.077676588s is descriptive only, not an acceptance gate. Later candidate -statistics must use their contemporaneous paired baseline, not substitute -these30observations. - -Complete private raw archive SHA256 -68a755285f13dbde2183b60c1a7720d77eaaa8c06980e47784765c31c1c70b69. -Independent review SHA256 -95310bf77a74853660c81273b218584422d8e931f69d1c46783a02c5f8e727f9. -Frozen component792cc20248c7657e0ac7f837ff864a58bab05f00bf950fcb4bd7027579285aa3. -Baseline phase cumulative1221/14400GPU-s including2139's7seconds. Readiness -7119/7200; search and final phases unused at this result. No environment changes. -Verdict INCONCLUSIVE: baseline work is complete; candidate/quality gates remain. diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/formal-8k-baseline/raw-audit.json b/evidence/qwen38-h20-fp8-single-request-vllm032/results/formal-8k-baseline/raw-audit.json deleted file mode 100644 index 24af20a0a..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/formal-8k-baseline/raw-audit.json +++ /dev/null @@ -1,370 +0,0 @@ -{ - "schema": 1, - "scope": "raw HTTP/client accounting only", - "verdict": "INCONCLUSIVE", - "runtime_scheduler_quality_bindings_required": true, - "raw_accounting_valid": true, - "mode": "baseline_only", - "measurements": [ - { - "arm": "baseline", - "repetition": 0, - "case": "n8192-s0", - "input_tokens": 8192, - "output_tokens": 2048, - "ttft_s": 1.6816718749469146, - "tpot_s": 0.010357489934055527, - "latency_s": 22.883453769958578, - "output_tokens_per_second": 89.49697980855566, - "wire_sha256": "0f080909c6775074f8a4d6d540e8323a066ab4dd820252bd8ce731b3c6896f30" - }, - { - "arm": "baseline", - "repetition": 0, - "case": "n8192-s1", - "input_tokens": 8192, - "output_tokens": 2048, - "ttft_s": 1.687585426028818, - "tpot_s": 0.010389891410377067, - "latency_s": 22.955693143070675, - "output_tokens_per_second": 89.21534136372624, - "wire_sha256": "fbfb1741c988f835e2dc91e57e016e17572765b76087168f2869fd993cd2f10f" - }, - { - "arm": "baseline", - "repetition": 0, - "case": "n8192-s2", - "input_tokens": 8192, - "output_tokens": 2048, - "ttft_s": 1.68507449910976, - "tpot_s": 0.010387061959922833, - "latency_s": 22.9473903310718, - "output_tokens_per_second": 89.2476212088882, - "wire_sha256": "30a571c583e2d87ed897138bd90500c9e97532cdef52f8d45b9fe4f6aff7c273" - }, - { - "arm": "baseline", - "repetition": 0, - "case": "n8192-s3", - "input_tokens": 8192, - "output_tokens": 2048, - "ttft_s": 1.6810228109825402, - "tpot_s": 0.010442285488542072, - "latency_s": 23.056381206028163, - "output_tokens_per_second": 88.82573469354956, - "wire_sha256": "f99352ca1b95ee892a5d48cb049a20f64fefa5bae299b221fe36c21a68060283" - }, - { - "arm": "baseline", - "repetition": 0, - "case": "n8192-s4", - "input_tokens": 8192, - "output_tokens": 2048, - "ttft_s": 1.6798835719237104, - "tpot_s": 0.010444255724481504, - "latency_s": 23.059275039937347, - "output_tokens_per_second": 88.81458746872921, - "wire_sha256": "8ea98afd0da0723a0791489aa82261cf75da149edd3cb0bdefda6ac6872a1273" - }, - { - "arm": "baseline", - "repetition": 0, - "case": "n8192-s5", - "input_tokens": 8192, - "output_tokens": 2048, - "ttft_s": 1.6809200160205364, - "tpot_s": 0.010444878079142725, - "latency_s": 23.061585444025695, - "output_tokens_per_second": 88.80568965957856, - "wire_sha256": "cedcf72f85e27645ffd937c0f3b51de6517abac0fea2d9ace5399830ec675034" - }, - { - "arm": "baseline", - "repetition": 0, - "case": "n8192-s6", - "input_tokens": 8192, - "output_tokens": 2048, - "ttft_s": 1.6808278849348426, - "tpot_s": 0.010454091873009925, - "latency_s": 23.080353948986158, - "output_tokens_per_second": 88.73347456137958, - "wire_sha256": "55c82a1780bf8a8b69971fc81ab12456407ac5210e9b27e331c53da6aefae9f0" - }, - { - "arm": "baseline", - "repetition": 0, - "case": "n8192-s7", - "input_tokens": 8192, - "output_tokens": 2048, - "ttft_s": 1.682304915972054, - "tpot_s": 0.010452978026395292, - "latency_s": 23.079550936003216, - "output_tokens_per_second": 88.73656188887101, - "wire_sha256": "ee606cf8df859c49049a76f3a8bba54edb39aa45f2ba1b0897379c6f2e63976d" - }, - { - "arm": "baseline", - "repetition": 0, - "case": "n8192-s8", - "input_tokens": 8192, - "output_tokens": 2048, - "ttft_s": 1.6806061220122501, - "tpot_s": 0.010457302661954767, - "latency_s": 23.086704671033658, - "output_tokens_per_second": 88.70906563679385, - "wire_sha256": "78682d38ad6a10065a7798dc40786a32317ec7ec068458ca88c99d82fa0938f8" - }, - { - "arm": "baseline", - "repetition": 0, - "case": "n8192-s9", - "input_tokens": 8192, - "output_tokens": 2048, - "ttft_s": 1.6820766689488664, - "tpot_s": 0.010451404364965219, - "latency_s": 23.07610140403267, - "output_tokens_per_second": 88.7498266774864, - "wire_sha256": "000e0f9398473168390f63e50afe1546a414c368cb214087d59adf63d05f3860" - }, - { - "arm": "baseline", - "repetition": 1, - "case": "n8192-s0", - "input_tokens": 8192, - "output_tokens": 2048, - "ttft_s": 1.6830165620194748, - "tpot_s": 0.010722545645805735, - "latency_s": 23.632067498983815, - "output_tokens_per_second": 86.66190548448901, - "wire_sha256": "7af352fb4f7337d97e6c50ef2448bcaedd77bf33d3c01df6f6b27204b7993985" - }, - { - "arm": "baseline", - "repetition": 1, - "case": "n8192-s1", - "input_tokens": 8192, - "output_tokens": 2048, - "ttft_s": 1.682199697010219, - "tpot_s": 0.010518424083038793, - "latency_s": 23.21341379499063, - "output_tokens_per_second": 88.22485215173096, - "wire_sha256": "00e13b0d4d9657f1a65011e6727e679156a81086100810098347da23831e4b47" - }, - { - "arm": "baseline", - "repetition": 1, - "case": "n8192-s2", - "input_tokens": 8192, - "output_tokens": 2048, - "ttft_s": 1.6823768810136244, - "tpot_s": 0.010432243590576653, - "latency_s": 23.037179510924034, - "output_tokens_per_second": 88.89977173763202, - "wire_sha256": "ad322687217bf88fe0d0bc0338ca27238cbdb20d8b4eee16f66e0f646e4bd9b1" - }, - { - "arm": "baseline", - "repetition": 1, - "case": "n8192-s3", - "input_tokens": 8192, - "output_tokens": 2048, - "ttft_s": 1.6811685889260843, - "tpot_s": 0.01044223565413991, - "latency_s": 23.05642497295048, - "output_tokens_per_second": 88.82556607985362, - "wire_sha256": "18266826830c723cec548c43fa08898767dde1cac5cb3aa0e40d706d052f22de" - }, - { - "arm": "baseline", - "repetition": 1, - "case": "n8192-s4", - "input_tokens": 8192, - "output_tokens": 2048, - "ttft_s": 1.6808728029718623, - "tpot_s": 0.010445916706388593, - "latency_s": 23.063664300949313, - "output_tokens_per_second": 88.79768510659875, - "wire_sha256": "64be4201c70f8afbc3887e8188735d5bbd055405bc9fea3bf72d518bc0bbf4ad" - }, - { - "arm": "baseline", - "repetition": 1, - "case": "n8192-s5", - "input_tokens": 8192, - "output_tokens": 2048, - "ttft_s": 1.6823674180777743, - "tpot_s": 0.010452315309683238, - "latency_s": 23.07825685699936, - "output_tokens_per_second": 88.74153765988899, - "wire_sha256": "3cc2571bacb3300c7c2f132b8dd8e0355bfce86ca5add633e523891ef88f52e7" - }, - { - "arm": "baseline", - "repetition": 1, - "case": "n8192-s6", - "input_tokens": 8192, - "output_tokens": 2048, - "ttft_s": 1.6806386470561847, - "tpot_s": 0.010454384215427704, - "latency_s": 23.080763136036694, - "output_tokens_per_second": 88.73190145097047, - "wire_sha256": "8f0a59fb60f671291c6607052049ef9ac3a6c3448c148cfb5f1d538b1645f697" - }, - { - "arm": "baseline", - "repetition": 1, - "case": "n8192-s7", - "input_tokens": 8192, - "output_tokens": 2048, - "ttft_s": 1.6825329139828682, - "tpot_s": 0.010457252639934555, - "latency_s": 23.088529067928903, - "output_tokens_per_second": 88.70205607185137, - "wire_sha256": "3108a1a096188c15951f59a31d8d0673b32baa4bf2bdd9941c99a289d05ad320" - }, - { - "arm": "baseline", - "repetition": 1, - "case": "n8192-s8", - "input_tokens": 8192, - "output_tokens": 2048, - "ttft_s": 1.6804305909899995, - "tpot_s": 0.010456559666840549, - "latency_s": 23.0850082290126, - "output_tokens_per_second": 88.71558457692599, - "wire_sha256": "dcc5b279311afa027697e56b223cf18b885e82b4344c4e80531b46e994c4260e" - }, - { - "arm": "baseline", - "repetition": 1, - "case": "n8192-s9", - "input_tokens": 8192, - "output_tokens": 2048, - "ttft_s": 1.6820836919359863, - "tpot_s": 0.010451886970226794, - "latency_s": 23.077096319990233, - "output_tokens_per_second": 88.74600043272979, - "wire_sha256": "e3bcc8a2694e1aaa544f6995ca33e434c6ce63a3f411afebb5691b70117d5260" - }, - { - "arm": "baseline", - "repetition": 2, - "case": "n8192-s0", - "input_tokens": 8192, - "output_tokens": 2048, - "ttft_s": 1.6816927720792592, - "tpot_s": 0.0107228563663754, - "latency_s": 23.631379754049703, - "output_tokens_per_second": 86.66442760918498, - "wire_sha256": "12c930b7ea5e18810add55bc5c61da71cf7c2884383e9b86a9fcf8cdc76bd836" - }, - { - "arm": "baseline", - "repetition": 2, - "case": "n8192-s1", - "input_tokens": 8192, - "output_tokens": 2048, - "ttft_s": 1.6812519249506295, - "tpot_s": 0.010648662520767726, - "latency_s": 23.479064104962163, - "output_tokens_per_second": 87.22664544227584, - "wire_sha256": "8cd642c44c2160700d22b4699eb7e71c1b082b0b800dd7c69e8269aac026e5e7" - }, - { - "arm": "baseline", - "repetition": 2, - "case": "n8192-s2", - "input_tokens": 8192, - "output_tokens": 2048, - "ttft_s": 1.6809311080724, - "tpot_s": 0.010433060178771145, - "latency_s": 23.037405294016935, - "output_tokens_per_second": 88.89890045611551, - "wire_sha256": "67d253d199b0d034081af5fb17722eb2ce0151b4ee8bbafc0346670b36656617" - }, - { - "arm": "baseline", - "repetition": 2, - "case": "n8192-s3", - "input_tokens": 8192, - "output_tokens": 2048, - "ttft_s": 1.6800238110590726, - "tpot_s": 0.010439216788961909, - "latency_s": 23.0491005780641, - "output_tokens_per_second": 88.8537924967488, - "wire_sha256": "fbf7b73d463a336a886a97edf3a4eb10616a0f28d1f531586d63c10b7ad03bd1" - }, - { - "arm": "baseline", - "repetition": 2, - "case": "n8192-s4", - "input_tokens": 8192, - "output_tokens": 2048, - "ttft_s": 1.6803534020436928, - "tpot_s": 0.010450707150929852, - "latency_s": 23.0729509399971, - "output_tokens_per_second": 88.7619448992881, - "wire_sha256": "857178ee33c44b56888ff04c764a1eb6ec584afb75aeb64ddd1feea98040a8d6" - }, - { - "arm": "baseline", - "repetition": 2, - "case": "n8192-s5", - "input_tokens": 8192, - "output_tokens": 2048, - "ttft_s": 1.681646759971045, - "tpot_s": 0.010449858450874027, - "latency_s": 23.07250700891018, - "output_tokens_per_second": 88.76365274086166, - "wire_sha256": "afbf3f4290adcaec4d0fe8a075710d51c2e5c6af9b73e73f913af60a7009e661" - }, - { - "arm": "baseline", - "repetition": 2, - "case": "n8192-s6", - "input_tokens": 8192, - "output_tokens": 2048, - "ttft_s": 1.6808637190843, - "tpot_s": 0.010456654935976027, - "latency_s": 23.085636373027228, - "output_tokens_per_second": 88.71317068793651, - "wire_sha256": "05b3879536ecd111fb96a6acd40f145d0928f45d992fcaf1837ec8cfcb4ddde8" - }, - { - "arm": "baseline", - "repetition": 2, - "case": "n8192-s7", - "input_tokens": 8192, - "output_tokens": 2048, - "ttft_s": 1.6806062490213662, - "tpot_s": 0.010460440746431649, - "latency_s": 23.09312845696695, - "output_tokens_per_second": 88.68438954974678, - "wire_sha256": "1bd18d463ac6b9b19e54933a80fca8191868a3deab21d2b12bba240e0ef62cbc" - }, - { - "arm": "baseline", - "repetition": 2, - "case": "n8192-s8", - "input_tokens": 8192, - "output_tokens": 2048, - "ttft_s": 1.6805805860785767, - "tpot_s": 0.01045907712161621, - "latency_s": 23.09031145402696, - "output_tokens_per_second": 88.69520898743997, - "wire_sha256": "315011871c3ac4347a7b58a9a48ff252528d5882f94e764ba841b6ac501c0c2d" - }, - { - "arm": "baseline", - "repetition": 2, - "case": "n8192-s9", - "input_tokens": 8192, - "output_tokens": 2048, - "ttft_s": 1.6825692789861932, - "tpot_s": 0.010453800704925307, - "latency_s": 23.081499321968295, - "output_tokens_per_second": 88.72907134116602, - "wire_sha256": "9b17a4f65c81dceb3c39e055a99ce56a8424ec471dc25ac6d9cd55a4ad2c5f43" - } - ] -} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/fp8-projections-search-v1/bound-result.json b/evidence/qwen38-h20-fp8-single-request-vllm032/results/fp8-projections-search-v1/bound-result.json deleted file mode 100644 index 1acc09511..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/fp8-projections-search-v1/bound-result.json +++ /dev/null @@ -1,964 +0,0 @@ -{ - "campaign_verdict": "INCONCLUSIVE", - "scope": "authenticated 8K paired performance only", - "quality_and_retrieval_required": true, - "candidate": "2d676dfa615d0c773970e0124183458a9101748b", - "baseline": "5df63a7203cdb9328d824a27b682e77c7b9928be", - "component_sha256": "a274225b0d4c746326b364e02a37fcc8e6962bcac3aaf3af56ca08537e476fdf", - "terminal_sha256": "1c0527583a5d2a082507bbec7c997c7cdb088d841ac3de4b5bd5c2d4a20461c5", - "scheduler": { - "JobId": "2163", - "JobState": "COMPLETED", - "ExitCode": "0:0", - "RunTime": "00:37:24", - "AllocTRES": "cpu=8,node=1,billing=8,gres/gpu=1" - }, - "raw_audit_sha256": "e4af16c0d5a0f17259b516b9b25d66425af8e6872f23974db6c4788920541d1e", - "model_runtime_sha256": "5077d874c429e86df9099f78d17dc123f71803c86c838b138af9202547af65ae", - "device_receipt_sha256": "59d777d3274214599697a8d7beeacf8412776364f068246bae07bf0b8980c46a", - "rows": [ - { - "arm": "baseline", - "repetition": 0, - "case": "n8192-s0", - "input_tokens": 8192, - "output_tokens": 2048, - "ttft_s": 1.6832628219854087, - "tpot_s": 0.010785954115326773, - "latency_s": 23.762110896059312, - "output_tokens_per_second": 86.18762907716412, - "wire_sha256": "c4763a1538ed0de5f8ec939e3fffb672ed4a1b5a58d727928e83992a2305b397", - "valid": true - }, - { - "arm": "baseline", - "repetition": 0, - "case": "n8192-s1", - "input_tokens": 8192, - "output_tokens": 2048, - "ttft_s": 1.6818121579708531, - "tpot_s": 0.010789498406927351, - "latency_s": 23.76791539695114, - "output_tokens_per_second": 86.16658069486017, - "wire_sha256": "4a94583a6ff11c597ec6741c2055b3269ffd81c8d62452e024e481a4a0c55883", - "valid": true - }, - { - "arm": "baseline", - "repetition": 0, - "case": "n8192-s2", - "input_tokens": 8192, - "output_tokens": 2048, - "ttft_s": 1.6806817230535671, - "tpot_s": 0.010825176634070517, - "latency_s": 23.839818292995915, - "output_tokens_per_second": 85.90669504396759, - "wire_sha256": "d9f09723eb750d72e07aec229806903884b69f4efcef28b21441a82922b468aa", - "valid": true - }, - { - "arm": "baseline", - "repetition": 0, - "case": "n8192-s3", - "input_tokens": 8192, - "output_tokens": 2048, - "ttft_s": 1.6797754659783095, - "tpot_s": 0.010700449406926424, - "latency_s": 23.5835954019567, - "output_tokens_per_second": 86.8400243938242, - "wire_sha256": "de1727f2a4bc3e03246db3894172f6cd21c5fb43bbdc7dd47c964f90ab7f5c95", - "valid": true - }, - { - "arm": "baseline", - "repetition": 0, - "case": "n8192-s4", - "input_tokens": 8192, - "output_tokens": 2048, - "ttft_s": 1.6814116280293092, - "tpot_s": 0.010554844753304428, - "latency_s": 23.287178838043474, - "output_tokens_per_second": 87.94538893024912, - "wire_sha256": "3d13e23b2e2b5b2780358b09f2ae882ae144a67fef06646c27092d30a2e71b9e", - "valid": true - }, - { - "arm": "baseline", - "repetition": 0, - "case": "n8192-s5", - "input_tokens": 8192, - "output_tokens": 2048, - "ttft_s": 1.6818553659832105, - "tpot_s": 0.010565818184665957, - "latency_s": 23.310085189994425, - "output_tokens_per_second": 87.85896676512704, - "wire_sha256": "3927e146de3edb2537aaf8c9aa6b8c281f66c1e047a268e6df81d4881eebd82b", - "valid": true - }, - { - "arm": "baseline", - "repetition": 0, - "case": "n8192-s6", - "input_tokens": 8192, - "output_tokens": 2048, - "ttft_s": 1.6814635730115697, - "tpot_s": 0.010570122000934674, - "latency_s": 23.318503308924846, - "output_tokens_per_second": 87.82724915351473, - "wire_sha256": "39e1023d1ae84f8e339111f3b3b41fa795cce7038eeb9159392df666e259469b", - "valid": true - }, - { - "arm": "baseline", - "repetition": 0, - "case": "n8192-s7", - "input_tokens": 8192, - "output_tokens": 2048, - "ttft_s": 1.6811402529710904, - "tpot_s": 0.010566554917466182, - "latency_s": 23.310878169024363, - "output_tokens_per_second": 87.85597801808234, - "wire_sha256": "d3cd45645416fba41f36966776c1571a0b6915493cb76e3f4899b18b54e15b19", - "valid": true - }, - { - "arm": "baseline", - "repetition": 0, - "case": "n8192-s8", - "input_tokens": 8192, - "output_tokens": 2048, - "ttft_s": 1.6815629521152005, - "tpot_s": 0.010574152320418662, - "latency_s": 23.3268527520122, - "output_tokens_per_second": 87.7958129102237, - "wire_sha256": "fba9584a2991e8c949f8840b85080f06d057409d5bc4543b61c156c68ef72d7d", - "valid": true - }, - { - "arm": "baseline", - "repetition": 0, - "case": "n8192-s9", - "input_tokens": 8192, - "output_tokens": 2048, - "ttft_s": 1.6824272719677538, - "tpot_s": 0.010567818869610054, - "latency_s": 23.314752498059534, - "output_tokens_per_second": 87.84137855078896, - "wire_sha256": "68da9dc191d6216f383309a9ff4d7fef912578954f80a0050fea4a4809092e18", - "valid": true - }, - { - "arm": "candidate", - "repetition": 0, - "case": "n8192-s0", - "input_tokens": 8192, - "output_tokens": 2048, - "ttft_s": 2.8654061800334603, - "tpot_s": 0.012192451242793353, - "latency_s": 27.823353874031454, - "output_tokens_per_second": 73.60722971329034, - "wire_sha256": "40096c0dccf8fa531ebcc2c03c99ea60887cc88270ba44317c3e671b272d8dbc", - "valid": true - }, - { - "arm": "candidate", - "repetition": 0, - "case": "n8192-s1", - "input_tokens": 8192, - "output_tokens": 2048, - "ttft_s": 2.85970177303534, - "tpot_s": 0.012178798054702558, - "latency_s": 27.789701391011477, - "output_tokens_per_second": 73.69636582933639, - "wire_sha256": "d00cc76815f437853a3e599e22985881eb042ca91f69b20255a78799f49930fd", - "valid": true - }, - { - "arm": "candidate", - "repetition": 0, - "case": "n8192-s2", - "input_tokens": 8192, - "output_tokens": 2048, - "ttft_s": 2.8579154579201713, - "tpot_s": 0.012187681619931444, - "latency_s": 27.806099733919837, - "output_tokens_per_second": 73.65290420438598, - "wire_sha256": "24765ce3b9fbaacc28677b7f2042b0a1dc16744ae0e9e21bb1f325240a7b47ce", - "valid": true - }, - { - "arm": "candidate", - "repetition": 0, - "case": "n8192-s3", - "input_tokens": 8192, - "output_tokens": 2048, - "ttft_s": 2.8576242129784077, - "tpot_s": 0.01217501766924643, - "latency_s": 27.77988538192585, - "output_tokens_per_second": 73.72240640461641, - "wire_sha256": "4a461e05f16ba4578ccd1d2d20c756e1e323384ce3e8c7169300d78e14745b7d", - "valid": true - }, - { - "arm": "candidate", - "repetition": 0, - "case": "n8192-s4", - "input_tokens": 8192, - "output_tokens": 2048, - "ttft_s": 2.8592277240240946, - "tpot_s": 0.012172944966781453, - "latency_s": 27.77724607102573, - "output_tokens_per_second": 73.72941128732901, - "wire_sha256": "c26cb9140a502cc33de10c86b55953b44d194b440ca2874b65b2dcb96f4209d1", - "valid": true - }, - { - "arm": "candidate", - "repetition": 0, - "case": "n8192-s5", - "input_tokens": 8192, - "output_tokens": 2048, - "ttft_s": 2.85803726001177, - "tpot_s": 0.012187504831919448, - "latency_s": 27.80585965095088, - "output_tokens_per_second": 73.65354014257079, - "wire_sha256": "d6000e6ddb00213eeb46f7bfa9b8496e080db76664f6e5acf8cc9b10ea6535eb", - "valid": true - }, - { - "arm": "candidate", - "repetition": 0, - "case": "n8192-s6", - "input_tokens": 8192, - "output_tokens": 2048, - "ttft_s": 2.858293110039085, - "tpot_s": 0.012180044174885435, - "latency_s": 27.79084353602957, - "output_tokens_per_second": 73.69333706423343, - "wire_sha256": "fae73ff5d64d6b6174afcdb070f77656a941bcd3b657edddf4bce8e128998c43", - "valid": true - }, - { - "arm": "candidate", - "repetition": 0, - "case": "n8192-s7", - "input_tokens": 8192, - "output_tokens": 2048, - "ttft_s": 2.8658795250812545, - "tpot_s": 0.012190183126490471, - "latency_s": 27.819184385007247, - "output_tokens_per_second": 73.61826183170706, - "wire_sha256": "5054daf3ef555a6d80d62e746830c2dfd3d1aa8cde5d2202d9b13ed3c150af37", - "valid": true - }, - { - "arm": "candidate", - "repetition": 0, - "case": "n8192-s8", - "input_tokens": 8192, - "output_tokens": 2048, - "ttft_s": 2.860816820990294, - "tpot_s": 0.012185469346812963, - "latency_s": 27.804472573916428, - "output_tokens_per_second": 73.65721448430722, - "wire_sha256": "fdcd6b244f47a3b63406413e4782ee8c8f18b343b51b1b4f104396560faf7135", - "valid": true - }, - { - "arm": "candidate", - "repetition": 0, - "case": "n8192-s9", - "input_tokens": 8192, - "output_tokens": 2048, - "ttft_s": 2.8604780230671167, - "tpot_s": 0.012183059501708733, - "latency_s": 27.799200823064893, - "output_tokens_per_second": 73.67118260107614, - "wire_sha256": "6c945a14ebb1affba437202b1f1cadff1e7f94e2286db8e5c035f78dd7b76646", - "valid": true - }, - { - "arm": "candidate", - "repetition": 1, - "case": "n8192-s0", - "input_tokens": 8192, - "output_tokens": 2048, - "ttft_s": 2.864647479960695, - "tpot_s": 0.01221309133613565, - "latency_s": 27.86484544503037, - "output_tokens_per_second": 73.49762639237089, - "wire_sha256": "f7f280097cff449954cc2fa46cd77e9b86c2e66b4f7320db3708b10a314faa90", - "valid": true - }, - { - "arm": "candidate", - "repetition": 1, - "case": "n8192-s1", - "input_tokens": 8192, - "output_tokens": 2048, - "ttft_s": 2.863577203010209, - "tpot_s": 0.012215168506571769, - "latency_s": 27.86802713596262, - "output_tokens_per_second": 73.48923517291738, - "wire_sha256": "0409428b610c2313f570406cdcdf802caf728e55ca30e6f760f3b7123ecf531a", - "valid": true - }, - { - "arm": "candidate", - "repetition": 1, - "case": "n8192-s2", - "input_tokens": 8192, - "output_tokens": 2048, - "ttft_s": 2.862603401998058, - "tpot_s": 0.012205258329272904, - "latency_s": 27.84676720201969, - "output_tokens_per_second": 73.54534137274868, - "wire_sha256": "63b556b8174f5e7f117c07b7d972663f3a0673707814ec909075a237c6b2dfc7", - "valid": true - }, - { - "arm": "candidate", - "repetition": 1, - "case": "n8192-s3", - "input_tokens": 8192, - "output_tokens": 2048, - "ttft_s": 2.8612983430502936, - "tpot_s": 0.012202976980466353, - "latency_s": 27.84079222206492, - "output_tokens_per_second": 73.56112511686645, - "wire_sha256": "1fc6702b09861b723845fc52dd0eecc849796dbc09c7cf5c1854c19e787fdb45", - "valid": true - }, - { - "arm": "candidate", - "repetition": 1, - "case": "n8192-s4", - "input_tokens": 8192, - "output_tokens": 2048, - "ttft_s": 2.8589519520755857, - "tpot_s": 0.012202900211528329, - "latency_s": 27.838288685074076, - "output_tokens_per_second": 73.56774057372523, - "wire_sha256": "9babe82777eedbbfa68fe6045916e3b61f232879e477f512df439accb99113c3", - "valid": true - }, - { - "arm": "candidate", - "repetition": 1, - "case": "n8192-s5", - "input_tokens": 8192, - "output_tokens": 2048, - "ttft_s": 2.8603304059943184, - "tpot_s": 0.012215038408895179, - "latency_s": 27.86451402900275, - "output_tokens_per_second": 73.49850056126374, - "wire_sha256": "2a276ceae090f7068f09dbee2128262ee57bd40ee62aa69bc6b8ee8736f109b0", - "valid": true - }, - { - "arm": "candidate", - "repetition": 1, - "case": "n8192-s6", - "input_tokens": 8192, - "output_tokens": 2048, - "ttft_s": 2.8604786589276046, - "tpot_s": 0.01221474497409562, - "latency_s": 27.86406162090134, - "output_tokens_per_second": 73.49969390190259, - "wire_sha256": "58abcdd12afe77008e50b1116f3131fce9089a0be01310758006c72afde42b46", - "valid": true - }, - { - "arm": "candidate", - "repetition": 1, - "case": "n8192-s7", - "input_tokens": 8192, - "output_tokens": 2048, - "ttft_s": 2.8595704520121217, - "tpot_s": 0.012204545358108092, - "latency_s": 27.842274800059386, - "output_tokens_per_second": 73.55720804808779, - "wire_sha256": "f02517acb06bca895ce84108de8620c80640d1b066c57a230114f970f1d24514", - "valid": true - }, - { - "arm": "candidate", - "repetition": 1, - "case": "n8192-s8", - "input_tokens": 8192, - "output_tokens": 2048, - "ttft_s": 2.8586549350293353, - "tpot_s": 0.012205623885667616, - "latency_s": 27.843567028990947, - "output_tokens_per_second": 73.5537942343237, - "wire_sha256": "45c02910b29b74c29d7d21dca8cffaa0c52be17cd4f1a5c630b8553751d246fb", - "valid": true - }, - { - "arm": "candidate", - "repetition": 1, - "case": "n8192-s9", - "input_tokens": 8192, - "output_tokens": 2048, - "ttft_s": 2.8608115300303325, - "tpot_s": 0.012203409744046254, - "latency_s": 27.841191276093014, - "output_tokens_per_second": 73.56007074879011, - "wire_sha256": "fe0f801ddfc098226be2c1a89fdc2469d806929fdc83d4a481e5ae5c12d517ed", - "valid": true - }, - { - "arm": "baseline", - "repetition": 1, - "case": "n8192-s0", - "input_tokens": 8192, - "output_tokens": 2048, - "ttft_s": 1.6857226910069585, - "tpot_s": 0.010484485398126292, - "latency_s": 23.14746430097148, - "output_tokens_per_second": 88.47621378182868, - "wire_sha256": "f029ec458927562d7b9f38eaa1ad21c53f9cd7bcd1a42b679a79e8419070ad84", - "valid": true - }, - { - "arm": "baseline", - "repetition": 1, - "case": "n8192-s1", - "input_tokens": 8192, - "output_tokens": 2048, - "ttft_s": 1.6850971460808069, - "tpot_s": 0.010499448300444834, - "latency_s": 23.177467817091383, - "output_tokens_per_second": 88.36168023884717, - "wire_sha256": "69a3a5a13bb22fa65f91a83cba4ae5cbd811d2b0ee9747261e205e7e70dbb1f7", - "valid": true - }, - { - "arm": "baseline", - "repetition": 1, - "case": "n8192-s2", - "input_tokens": 8192, - "output_tokens": 2048, - "ttft_s": 1.6826470530359074, - "tpot_s": 0.010540509264737378, - "latency_s": 23.25906951795332, - "output_tokens_per_second": 88.05167371029954, - "wire_sha256": "607fe49be5c59f258db7f5e95dc5d13a0e701e4f39db3ab29f883722518873a4", - "valid": true - }, - { - "arm": "baseline", - "repetition": 1, - "case": "n8192-s3", - "input_tokens": 8192, - "output_tokens": 2048, - "ttft_s": 1.681117522995919, - "tpot_s": 0.010550894088914384, - "latency_s": 23.278797723003663, - "output_tokens_per_second": 87.97705209561599, - "wire_sha256": "307be0da1a39fad74d2e47ea6842ec4fad7c2b1f142235929e850632639316e7", - "valid": true - }, - { - "arm": "baseline", - "repetition": 1, - "case": "n8192-s4", - "input_tokens": 8192, - "output_tokens": 2048, - "ttft_s": 1.6845659039681777, - "tpot_s": 0.010571897186613704, - "latency_s": 23.325239444966428, - "output_tokens_per_second": 87.80188537108275, - "wire_sha256": "f063a979b75eeee3dd2c6358c5f8ddbb5c142348429f81ec5cc564bd21c37023", - "valid": true - }, - { - "arm": "baseline", - "repetition": 1, - "case": "n8192-s5", - "input_tokens": 8192, - "output_tokens": 2048, - "ttft_s": 1.6851165640400723, - "tpot_s": 0.010571550469937437, - "latency_s": 23.325080376002006, - "output_tokens_per_second": 87.80248414951159, - "wire_sha256": "600e5bfa50c2e402fae7db112c70bd14395f998133c3a8f25ebdb749319ed7bb", - "valid": true - }, - { - "arm": "baseline", - "repetition": 1, - "case": "n8192-s6", - "input_tokens": 8192, - "output_tokens": 2048, - "ttft_s": 1.6820782279828563, - "tpot_s": 0.01057214868101635, - "latency_s": 23.323266578023322, - "output_tokens_per_second": 87.809312351202, - "wire_sha256": "7b2857fc96364a155101870b2498d3aaa29ed11de3bf5744306713295d9e438a", - "valid": true - }, - { - "arm": "baseline", - "repetition": 1, - "case": "n8192-s7", - "input_tokens": 8192, - "output_tokens": 2048, - "ttft_s": 1.6815011579310521, - "tpot_s": 0.010569683617002749, - "latency_s": 23.31764352193568, - "output_tokens_per_second": 87.830487590797, - "wire_sha256": "70d011fc8c628ec642b69de557c7f21f8b0ef1724f665d711dce6a84d01cab5b", - "valid": true - }, - { - "arm": "baseline", - "repetition": 1, - "case": "n8192-s8", - "input_tokens": 8192, - "output_tokens": 2048, - "ttft_s": 1.681562370969914, - "tpot_s": 0.010579107008780238, - "latency_s": 23.33699441794306, - "output_tokens_per_second": 87.75765907649868, - "wire_sha256": "8307868023a447436de8ca9a13a0f17ed5cd181874da5c5a0624309fd1d0dfac", - "valid": true - }, - { - "arm": "baseline", - "repetition": 1, - "case": "n8192-s9", - "input_tokens": 8192, - "output_tokens": 2048, - "ttft_s": 1.6823157770559192, - "tpot_s": 0.010573584782587511, - "latency_s": 23.326443827012554, - "output_tokens_per_second": 87.79735201764314, - "wire_sha256": "1abac3429246b0359f7112244e141ede24d997c2d5cac0034985cbe141702fbc", - "valid": true - }, - { - "arm": "baseline", - "repetition": 2, - "case": "n8192-s0", - "input_tokens": 8192, - "output_tokens": 2048, - "ttft_s": 1.6841326649300754, - "tpot_s": 0.01048652616366571, - "latency_s": 23.150051721953787, - "output_tokens_per_second": 88.46632502586718, - "wire_sha256": "e1f9b6de382b074bb95b7a8548f376f6b4691684da4fcf6c966b6b19dd89c744", - "valid": true - }, - { - "arm": "baseline", - "repetition": 2, - "case": "n8192-s1", - "input_tokens": 8192, - "output_tokens": 2048, - "ttft_s": 1.6856599890161306, - "tpot_s": 0.010499924487053518, - "latency_s": 23.179005414014682, - "output_tokens_per_second": 88.35581869969802, - "wire_sha256": "c4b2b5808ce317dff9ab330ffd7ac1afd2287c7bc7da550cb6ffcb54d6fd7a9c", - "valid": true - }, - { - "arm": "baseline", - "repetition": 2, - "case": "n8192-s2", - "input_tokens": 8192, - "output_tokens": 2048, - "ttft_s": 1.6826174970483407, - "tpot_s": 0.01054540865166949, - "latency_s": 23.269069007015787, - "output_tokens_per_second": 88.01383499195923, - "wire_sha256": "d3cc2c0ee84bf086b85ac66b3412a37823d38c70d446a43767babe45dd6900ea", - "valid": true - }, - { - "arm": "baseline", - "repetition": 2, - "case": "n8192-s3", - "input_tokens": 8192, - "output_tokens": 2048, - "ttft_s": 1.6815673090750352, - "tpot_s": 0.010544778034193365, - "latency_s": 23.26672794506885, - "output_tokens_per_second": 88.02269080702656, - "wire_sha256": "9ac6319a3e665cfb55ab796b0014d38c63be346657598f813194f7e934121b34", - "valid": true - }, - { - "arm": "baseline", - "repetition": 2, - "case": "n8192-s4", - "input_tokens": 8192, - "output_tokens": 2048, - "ttft_s": 1.6820982119534165, - "tpot_s": 0.010555806312185468, - "latency_s": 23.289833732997067, - "output_tokens_per_second": 87.93536370757302, - "wire_sha256": "bd1a9841a55cdf1584b2be3ee29aee513d392af588074ec8a0e2d5fd5c152963", - "valid": true - }, - { - "arm": "baseline", - "repetition": 2, - "case": "n8192-s5", - "input_tokens": 8192, - "output_tokens": 2048, - "ttft_s": 1.6819882810814306, - "tpot_s": 0.010559279786986555, - "latency_s": 23.29683400504291, - "output_tokens_per_second": 87.90894074090424, - "wire_sha256": "ec0defdd523c7ed705156d9bcd336ad6782ced667f46df6d38f8ab399b3b3484", - "valid": true - }, - { - "arm": "baseline", - "repetition": 2, - "case": "n8192-s6", - "input_tokens": 8192, - "output_tokens": 2048, - "ttft_s": 1.6811246409779415, - "tpot_s": 0.010564157074741823, - "latency_s": 23.305954172974452, - "output_tokens_per_second": 87.8745399051225, - "wire_sha256": "7aa3be1742d0a914f674cad2299a742a0320cace28bad0d8f6bf1107d0166ca0", - "valid": true - }, - { - "arm": "baseline", - "repetition": 2, - "case": "n8192-s7", - "input_tokens": 8192, - "output_tokens": 2048, - "ttft_s": 1.682546842028387, - "tpot_s": 0.01056860319932929, - "latency_s": 23.316477591055445, - "output_tokens_per_second": 87.83487951823581, - "wire_sha256": "0916ab5e9a53ccf9d0a04e98a273a021375db53ec81c685021f3944bbb5810e2", - "valid": true - }, - { - "arm": "baseline", - "repetition": 2, - "case": "n8192-s8", - "input_tokens": 8192, - "output_tokens": 2048, - "ttft_s": 1.6811733349459246, - "tpot_s": 0.010573832049811624, - "latency_s": 23.32580754091032, - "output_tokens_per_second": 87.79974697159292, - "wire_sha256": "dc0c1209d8a98b446a579619a093a2e305b15837a8cf7d1e00761eb2c73e4061", - "valid": true - }, - { - "arm": "baseline", - "repetition": 2, - "case": "n8192-s9", - "input_tokens": 8192, - "output_tokens": 2048, - "ttft_s": 1.6821187750902027, - "tpot_s": 0.010566601684407076, - "latency_s": 23.31195242307149, - "output_tokens_per_second": 87.85192946658235, - "wire_sha256": "0b16f35b516d5e664ab2415c1ffb17a67c8c777e393c276339b56b0399cf4aae", - "valid": true - }, - { - "arm": "candidate", - "repetition": 2, - "case": "n8192-s0", - "input_tokens": 8192, - "output_tokens": 2048, - "ttft_s": 2.864158966927789, - "tpot_s": 0.012190465644410524, - "latency_s": 27.81804214103613, - "output_tokens_per_second": 73.62128469058817, - "wire_sha256": "a30b4b7e60ca36f5ddcbc28263c77c2ffe84c19b95e50cd179ab90f098ee2ce5", - "valid": true - }, - { - "arm": "candidate", - "repetition": 2, - "case": "n8192-s1", - "input_tokens": 8192, - "output_tokens": 2048, - "ttft_s": 2.8642801280366257, - "tpot_s": 0.012185862813880607, - "latency_s": 27.80874130805023, - "output_tokens_per_second": 73.64590785729426, - "wire_sha256": "27bce7cf746b738e9877cc2a74733390de42bf0f576a17d39656f82bcb0c0a32", - "valid": true - }, - { - "arm": "candidate", - "repetition": 2, - "case": "n8192-s2", - "input_tokens": 8192, - "output_tokens": 2048, - "ttft_s": 2.8639679399784654, - "tpot_s": 0.012183560423530696, - "latency_s": 27.8037161269458, - "output_tokens_per_second": 73.6592184530036, - "wire_sha256": "e3ba197d830e5287dafeda53b22d2281acacf51e7c031fb96028a8b7b21b4d53", - "valid": true - }, - { - "arm": "candidate", - "repetition": 2, - "case": "n8192-s3", - "input_tokens": 8192, - "output_tokens": 2048, - "ttft_s": 2.8680209269514307, - "tpot_s": 0.01219302122375387, - "latency_s": 27.8271353719756, - "output_tokens_per_second": 73.59722704560234, - "wire_sha256": "ef76c03a5091a9adf1433067339383a440b36f00b0b8b367110b522860dd15fa", - "valid": true - }, - { - "arm": "candidate", - "repetition": 2, - "case": "n8192-s4", - "input_tokens": 8192, - "output_tokens": 2048, - "ttft_s": 2.8611791260773316, - "tpot_s": 0.012181969009759291, - "latency_s": 27.7976696890546, - "output_tokens_per_second": 73.67524051148808, - "wire_sha256": "a06c219daa7d8b4fb4dc8d961f1a9bb51f646b4b2775de7aa55ad8ae8c848f7a", - "valid": true - }, - { - "arm": "candidate", - "repetition": 2, - "case": "n8192-s5", - "input_tokens": 8192, - "output_tokens": 2048, - "ttft_s": 2.8650790579849854, - "tpot_s": 0.012180685601868264, - "latency_s": 27.79894248500932, - "output_tokens_per_second": 73.67186723395652, - "wire_sha256": "f7712fc87db4c7dda4aa73ac335a0cad03444d50b9c14a263637a45ae398e4d5", - "valid": true - }, - { - "arm": "candidate", - "repetition": 2, - "case": "n8192-s6", - "input_tokens": 8192, - "output_tokens": 2048, - "ttft_s": 2.8637148349080235, - "tpot_s": 0.01218898042848331, - "latency_s": 27.81455777201336, - "output_tokens_per_second": 73.63050733313008, - "wire_sha256": "150ede4b6cb3b2ccad6d366dbf030302bc9665ee736e0a4f87f02cacadcea060", - "valid": true - }, - { - "arm": "candidate", - "repetition": 2, - "case": "n8192-s7", - "input_tokens": 8192, - "output_tokens": 2048, - "ttft_s": 2.8647522090468556, - "tpot_s": 0.012184348541780993, - "latency_s": 27.80611367407255, - "output_tokens_per_second": 73.65286727967423, - "wire_sha256": "cfa3ee17b92bd5fe6ae37573cd2a678cf1c60a22c3a98857261d62dd77ed719f", - "valid": true - }, - { - "arm": "candidate", - "repetition": 2, - "case": "n8192-s8", - "input_tokens": 8192, - "output_tokens": 2048, - "ttft_s": 2.8642269329866394, - "tpot_s": 0.012185240383466963, - "latency_s": 27.807413997943513, - "output_tokens_per_second": 73.64942314130538, - "wire_sha256": "d949d47c7105d477ba3dea9d00f4466d5f3393ab2954c1ab9556d6e097f9db5e", - "valid": true - }, - { - "arm": "candidate", - "repetition": 2, - "case": "n8192-s9", - "input_tokens": 8192, - "output_tokens": 2048, - "ttft_s": 2.86069319001399, - "tpot_s": 0.01218681400537156, - "latency_s": 27.807101459009573, - "output_tokens_per_second": 73.65025092669062, - "wire_sha256": "48483e569fcc37d726739e7f914400e11683a6eec71d396aa6a78fc1ca9ec088", - "valid": true - } - ], - "statistics": { - "campaign_verdict": "INCONCLUSIVE", - "statistics_approved": true, - "scope": "approved_8k_statistics_only", - "metrics": [ - { - "input_tokens": 8192, - "metric": "ttft_s", - "paired_ratios_by_input": [ - [ - 1.702292798609853, - 1.699358675803024, - 1.700673009062922 - ], - [ - 1.700369306692157, - 1.6993543723400795, - 1.6992039596955852 - ], - [ - 1.7004501320617282, - 1.7012500612253891, - 1.702090905985738 - ], - [ - 1.7011941600862193, - 1.7020216040287146, - 1.7055641552219623 - ], - [ - 1.7004924174190703, - 1.6971446147289424, - 1.7009584254623584 - ], - [ - 1.6993359344791013, - 1.6974080411011256, - 1.7033882401028912 - ], - [ - 1.699884050963867, - 1.7005622041478314, - 1.7034518233235505 - ], - [ - 1.7047236362441305, - 1.7006056989758997, - 1.7026285019163367 - ], - [ - 1.701284401747634, - 1.6999993484514537, - 1.7037070916181098 - ], - [ - 1.700209019865402, - 1.7005199434298837, - 1.7006487486952797 - ] - ], - "median_paired_ratio": 1.7006272238355897, - "paired_ratio_p10": 1.6993227370007498, - "paired_ratio_p90": 1.7034773501530065, - "upper_one_sided_95": 1.7012392809169268, - "limit": 1.03, - "metric_verdict": "FAIL", - "arms": { - "baseline": { - "median": 1.6820332545321435, - "p10": 1.6811239291797393, - "p90": 1.6850990878767333 - }, - "candidate": { - "median": 2.860997973533813, - "p10": 2.8582675250363536, - "p90": 2.865111770189833 - } - } - }, - { - "input_tokens": 8192, - "metric": "tpot_s", - "paired_ratios_by_input": [ - [ - 1.1304008076084764, - 1.1648727498173903, - 1.162488459395516 - ], - [ - 1.1287640625520843, - 1.1634105104412242, - 1.1605667096849948 - ], - [ - 1.125864457635976, - 1.1579382003965255, - 1.155342654416893 - ], - [ - 1.1378043301027634, - 1.156582264747381, - 1.156308950668831 - ], - [ - 1.1533040277992193, - 1.1542772310517573, - 1.1540538590307976 - ], - [ - 1.1534842469281767, - 1.1554632826690245, - 1.1535526899173518 - ], - [ - 1.1523087598996873, - 1.1553701468490283, - 1.153805300531391 - ], - [ - 1.1536572914924696, - 1.154674614713675, - 1.1528816355366849 - ], - [ - 1.1523826191989737, - 1.1537480314300095, - 1.152395869923435 - ], - [ - 1.1528452230330744, - 1.1541411919392488, - 1.1533333392660574 - ] - ], - "median_paired_ratio": 1.1537766659807003, - "paired_ratio_p10": 1.1370639778533347, - "paired_ratio_p90": 1.1607588846560468, - "upper_one_sided_95": 1.154741923178071, - "limit": 1.03, - "metric_verdict": "FAIL", - "arms": { - "baseline": { - "median": 0.010567210277008566, - "p10": 0.01049987686839265, - "p90": 0.010708999877766459 - }, - "candidate": { - "median": 0.012188331024207376, - "p10": 0.012179919562867148, - "p90": 0.012213256699931646 - } - } - } - ], - "errors": [], - "bootstrap": { - "draws": 10000, - "seed": 670032, - "cluster": "input_id_all_three_repetitions", - "quantile": "linear", - "sample_indices_sha256": "f3dc88ecceb93a2bd68db9ada5acd635fec76425dcc68f7c5b0a2f8fa513ccce" - } - } -} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/fp8-projections-search-v1/knowledge.json b/evidence/qwen38-h20-fp8-single-request-vllm032/results/fp8-projections-search-v1/knowledge.json deleted file mode 100644 index f544841e5..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/fp8-projections-search-v1/knowledge.json +++ /dev/null @@ -1 +0,0 @@ -{"id": "case.qwen38-merged-fp8-projections-cand", "revision": "sha256:17c5daaa7ea8a59f6dc76027211b465d93cdf4b957bed90242ebc1eb5a53b747", "readback_verified": true} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/fp8-projections-search-v1/observation.md b/evidence/qwen38-h20-fp8-single-request-vllm032/results/fp8-projections-search-v1/observation.md deleted file mode 100644 index 634460bcf..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/fp8-projections-search-v1/observation.md +++ /dev/null @@ -1,16 +0,0 @@ -# Merged FP8 projections: first complete 8K paired search - -Issue #67's approved statistics produce **FAIL for both performance metrics**. Campaign status remains **INCONCLUSIVE** because task quality and retrieval are not complete. This is the cumulative candidate versus vLLM 0.32.0, not an isolated estimate of the projection-merge speedup. - -Job 2163 completed with scheduler exit0:0 in2244seconds on one H20 (2244GPU-seconds). Six owned service starts generated66valid8192-input/2048-output requests: six discarded warmups and60measurements, yielding30matched pairs. Frozen source/runtime/model/overlay guards, seven actual FP8 operator cases,16same-candidate eager/Graph predictions and final live state, raw SSE accounting, and external terminal binding all passed their scoped checks. No validator or threshold was changed after observation. - -| Metric | Baseline median | Candidate median | Median paired ratio | One-sided95% upper | Frozen metric verdict | -| --- | ---: | ---: | ---: | ---: | --- | -| TTFT |1.682033255s|2.860997974s|1.700627224|1.701239281|FAIL| -| TPOT |10.567210277ms|12.188331024ms|1.153776666|1.154741923|FAIL| - -Both median and upper bound must be at most1.03. Statistics use10fixed inputs×3pairs and10,000input-cluster bootstrap draws, seed670032, linearquantile. Parent independently recomputed arm medians, paired medians, sample identity and upper bounds from retained rows. The delegated receipt reviewer ran the unchanged frozen binder; no errors were reported. Successful local integration is not model/task quality acceptance. Cross-revision output identity or incremental speedup is not established by this batch. - -Baseline startup/shutdown warnings remain retained: cached cubin reload RuntimeError traces, missing precompiled kernels/JIT notices, deprecated entrypoint, generation/chat-template warnings, forced internal EngineCore shutdown and leaked semaphore notices. The six outer owned processes exited0 with zero remaining groups; that does not imply all internal workers shut down gracefully. - -Raw archive SHA2567a2774618dee78bf60ba0def9fc8e605d22f80475c4648fe29db3235a076bcb1; bound result SHA256818046ab9571161fb631ff07d17e9ac267c95d1e1f50ce2b741fdb14cac93137. Candidate and baseline revisions are bound in the Issue and result. Raw private runtime paths, model tokens and logs remain outside publication. Follow-up hypotheses are causal FlashAttention prefill and separate BF16 b/a projection merge, each independently scoped and measured. This unsuccessful parity attempt remains retained; no candidate is silently relabeled PASS. diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/fp8-projections-search-v1/parent-review.json b/evidence/qwen38-h20-fp8-single-request-vllm032/results/fp8-projections-search-v1/parent-review.json deleted file mode 100644 index d9210de5a..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/fp8-projections-search-v1/parent-review.json +++ /dev/null @@ -1,17 +0,0 @@ -{ - "reviewer": "root", - "scope": "independent review of delegated receipt collection plus direct recomputation of frozen statistics", - "checks": [ - "66raw requests and60measurements", - "arm medians and30paired medians", - "10000input-cluster draws seed670032 hash andlinear95%upper", - "unchanged source/model/runtime/overlay receipt andcleanup binding", - "failure wording and scoped quality limits" - ], - "result_sha256": "818046ab9571161fb631ff07d17e9ac267c95d1e1f50ce2b741fdb14cac93137", - "metric_verdicts": { - "ttft_s": "FAIL", - "tpot_s": "FAIL" - }, - "campaign_verdict": "INCONCLUSIVE" -} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/fp8-projections-search-v1/receipt-review.json b/evidence/qwen38-h20-fp8-single-request-vllm032/results/fp8-projections-search-v1/receipt-review.json deleted file mode 100644 index 5cf9e3899..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/fp8-projections-search-v1/receipt-review.json +++ /dev/null @@ -1,63 +0,0 @@ -{ - "job_id": 2163, - "scheduler_state": "COMPLETED", - "scheduler_exit": "0:0", - "scheduler_seconds": 2244, - "allocated_gpu_seconds": 2244, - "allocated_gpus": 1, - "allocated_cpus": 8, - "sacct_agrees": true, - "archive_sha256": "7a2774618dee78bf60ba0def9fc8e605d22f80475c4648fe29db3235a076bcb1", - "safe_extraction": true, - "frozen_files_overwritten": false, - "binder_exit": 0, - "binder_output_sha256": "818046ab9571161fb631ff07d17e9ac267c95d1e1f50ce2b741fdb14cac93137", - "binder_stdout_stderr_bytes": 0, - "validated_measurement_rows": 60, - "postflight_all_zero": true, - "campaign_verdict": "INCONCLUSIVE", - "metric_verdicts": { - "ttft_s": "FAIL", - "tpot_s": "FAIL" - }, - "warnings": [ - { - "category": "Deprecated vLLM API server entrypoint/module command", - "count": 6 - }, - { - "category": "Missing or mismatched FlashInfer precompiled kernels; startup compilation warning", - "count": 3 - }, - { - "category": "FlashInfer GDN prefill JIT compilation warning", - "count": 3 - }, - { - "category": "TritonBundler cached cubin reload failures, missing saved cubin; RuntimeError traces during baseline startup", - "count": 30 - }, - { - "category": "Model generation_config overrides default sampling parameters", - "count": 3 - }, - { - "category": "Chat template system-first ordering warning", - "count": 3 - }, - { - "category": "Forced remaining EngineCore process shutdown", - "count": 3, - "log_lines": 6 - }, - { - "category": "One leaked semaphore reported by resource tracker at shutdown", - "count": 3 - } - ], - "limits": [ - "Warnings retained; successful scheduler exit is not a claim of warning-free execution.", - "Authenticated 8K paired performance only; quality and retrieval remain required.", - "Validator unchanged; no new scheduled jobs or model executions during receipt review." - ] -} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/full-context-graph/actual-2091.json b/evidence/qwen38-h20-fp8-single-request-vllm032/results/full-context-graph/actual-2091.json deleted file mode 100644 index 1cf7f888f..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/full-context-graph/actual-2091.json +++ /dev/null @@ -1,207 +0,0 @@ -{ - "issue": 67, - "scope": "single-case engineering diagnostic, not formal acceptance", - "verdict": "INCONCLUSIVE", - "job": 2091, - "execution": "COMPLETED 0:0", - "allocated_gpu_seconds": 265, - "engine_tested": "2061bbcda4bb886fd1bb0468c5f1aa587aaa0300", - "raw_archive_sha256": "b95c6c7a8c8258ec0cb489614c56523519c612b5bcf6d621871b9e3e28e87bc7", - "requests_verified": 4, - "metrics": { - "buckets": { - "ttft_s": 3.529094479046762, - "tpot_s": 0.05229901888813372, - "latency_s": 110.58518614305649 - }, - "full_context": { - "ttft_s": 3.5279052620753646, - "tpot_s": 0.016347277947677857, - "latency_s": 36.990783220971934 - } - }, - "full_context_ratios": { - "buckets": { - "ttft_s": 0.9996630248981834, - "tpot_s": 0.31257331963806567, - "latency_s": 0.3345003477510947 - } - }, - "token_comparisons": { - "buckets": { - "matching_positions": 2048, - "total": 2048, - "first_mismatch": null - } - }, - "cleanup": [ - { - "phase": "cleanup", - "arm": "buckets", - "exit_code": 0, - "forced": false, - "owned_group_remaining": 0, - "cleanup_errors": [], - "cleanup_complete": true - }, - { - "phase": "cleanup", - "arm": "full_context", - "exit_code": 0, - "forced": false, - "owned_group_remaining": 0, - "cleanup_errors": [], - "cleanup_complete": true - } - ], - "graph_check": { - "complete": true, - "scope": "fixed_capacity_eager_graph_integration_not_task_quality", - "capacity": 10240, - "predictions": 16, - "graph_replays": 15, - "history": "shared eager argmax chosen online", - "rows": [ - { - "step": 0, - "exact": true, - "finite": true, - "max_abs": 0.0 - }, - { - "step": 1, - "exact": true, - "finite": true, - "max_abs": 0.0 - }, - { - "step": 2, - "exact": true, - "finite": true, - "max_abs": 0.0 - }, - { - "step": 3, - "exact": true, - "finite": true, - "max_abs": 0.0 - }, - { - "step": 4, - "exact": true, - "finite": true, - "max_abs": 0.0 - }, - { - "step": 5, - "exact": true, - "finite": true, - "max_abs": 0.0 - }, - { - "step": 6, - "exact": true, - "finite": true, - "max_abs": 0.0 - }, - { - "step": 7, - "exact": true, - "finite": true, - "max_abs": 0.0 - }, - { - "step": 8, - "exact": true, - "finite": true, - "max_abs": 0.0 - }, - { - "step": 9, - "exact": true, - "finite": true, - "max_abs": 0.0 - }, - { - "step": 10, - "exact": true, - "finite": true, - "max_abs": 0.0 - }, - { - "step": 11, - "exact": true, - "finite": true, - "max_abs": 0.0 - }, - { - "step": 12, - "exact": true, - "finite": true, - "max_abs": 0.0 - }, - { - "step": 13, - "exact": true, - "finite": true, - "max_abs": 0.0 - }, - { - "step": 14, - "exact": true, - "finite": true, - "max_abs": 0.0 - }, - { - "step": 15, - "exact": true, - "finite": true, - "max_abs": 0.0 - } - ], - "state_exact": true, - "prewarm": { - "policy": "full_context", - "buckets": { - "10240": { - "capture_seconds": 5.89499322604388, - "resident_MiB": 853.5009765625 - } - }, - "started_unix": 1792729415.244322, - "total_seconds": 5.89499322604388, - "resident_MiB_total": 853.5009765625, - "completed_unix": 1792729421.139933 - }, - "loader": { - "loaded_parameters": 1251, - "excluded_tensors": { - "vision": 333, - "mtp": 22 - }, - "scope": "text_only", - "dtype": "fp8", - "activation_dtype": "bfloat16", - "weight_dtype": "float8_e4m3fn", - "scale_dtype": "float32", - "checkpoint_scale_dtype": "bfloat16", - "quantization": { - "activation_scheme": "dynamic", - "fmt": "e4m3", - "quant_method": "fp8", - "weight_block_size": [ - 128, - 128 - ] - }, - "execution": "native_fp8", - "fp8_linear_modules": 400, - "dequantized": false, - "backend": "flashinfer_small_m_deepgemm_prefill", - "kernel_name": "fp8_blockscale_gemm_sm90+deep_gemm.fp8_gemm_nt" - } - }, - "overlay_prepost_exact": true, - "readiness_used_gpu_seconds": 6740, - "readiness_remaining_gpu_seconds": 460 -} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/full-context-graph/e2e-observation-2091.md b/evidence/qwen38-h20-fp8-single-request-vllm032/results/full-context-graph/e2e-observation-2091.md deleted file mode 100644 index 7afb29905..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/full-context-graph/e2e-observation-2091.md +++ /dev/null @@ -1,24 +0,0 @@ -# Full-context Graph: actual bounded E2E result - -Issue67; execution COMPLETED0:0, one H20 for265 GPU-seconds, released. Formal quality/retrieval/performance acceptance remains INCONCLUSIVE. Readiness usage6740/7200GPU-seconds;460 remain. Initial147 seconds are separately authorized;44GPU-hour phased cap unchanged. - -Tested source2061bbcda4bb886fd1bb0468c5f1aa587aaa0300. Final candidate cc651a5bc0696676ff4d214c4a13a7d586a5084c has the exact same tree6a4b128996625a3359ab0d4d9d0e32e112e7417e after merging current main; documentation conflict resolution retains the tested bytes. Both arms use FlashInfer small-M and DeepGEMM prefill with unchanged official FP8 weights/scales, same runtime and pinned official HTTP client. Only the Graph capacity policy differs. Default buckets remain the default; full_context is opt-in and requires one active request. - -Before HTTP, one whole-model check used capacity10240 for both eager and Graph, the fixed8192-token input and shared history chosen online from eager argmax. It produced16 predictions including initial prefill and15 Graph replays. Every logit was finite and exactly equal; final live KV through8207 tokens and full convolution/recurrent states were also exactly equal. This is bounded same-model Graph correctness, not an externally frozen history or general task-quality gate. - -Each HTTP arm ran one8192/16 warmup followed by one8192/2048 measured request, ignoreEOS, concurrency1. Order: buckets then full_context. All2048 outputtoken IDs match; raw prompt echoes, counts, usage, terminal reason, final DONE, client ITL and timing computations were verified for all four requests. - -| Graph policy | TTFT s | TPOT ms | Request s | -| --- | ---: | ---: | ---: | -| Default buckets |3.529094479|52.299018888|110.585186143| -| Full context |3.527905262|16.347277948|36.990783221| - -Full-context/control ratios are0.999663025 TTFT,0.312573320 TPOT and0.334500348 request latency. This single-case E2E improvement follows removal of the mid-generation eager transition; it does not establish formal10-input/3-repeat parity or quality. The vLLM observation from2084 belongs to a separate run and is not a same-job baseline for2091. - -Actual loader/provenance dtypes are valid. Full-context Python forward counters are1600 total:800 FlashInfer and800 DeepGEMM, representing one graph's warmup/capture plus two prefills, excluding replay. Buckets report721200 total (720400 FlashInfer,800 DeepGEMM), consistent with the later eager decode. No Triton GEMM was selected. These Python counters are not hardware kernel launch counts. - -Reported graph-prewarm allocated-memory delta was1821.403MiB for all five default buckets and853.501MiB for one full-context graph. This is neither peak model memory nor a general memory-reduction claim. The configured full context can reserve more per-request KV than a shorter bucket for other workloads. - -Both services completed with zero active requests/state, exit0, no observer force-kill, no remaining owned group and no cleanup errors. Overlay pre/post authentication matched all899 files and licenses; imported operator origin was checked. Package and runtime dependencies were unchanged. Startup/JIT/warmup are excluded from timed requests. - -Archive SHAb95c6c7a8c8258ec0cb489614c56523519c612b5bcf6d621871b9e3e28e87bc7;27 authenticated payloads plus their hash list. Private raw logs, paths and device identity remain outside the repository. The initial CPU-review lazy-capture boundary defect and fix are retained in the preparation case;23 CPU runner tests passed. Independent actual review verified all source/result hashes, four SSE/client records, exact Graph states, operator origins and cleanup; no remaining blocker. PR13 merged at77f4ab13ddce1b4b0ba6f1ae5e612608959fbd1b from reviewed headcc651a5bc0696676ff4d214c4a13a7d586a5084c against base83d82067e72f500fd81163fc7ccfda979a499200. Current-head Actions/status lists were empty. This opt-in integration does not close Issue67 or establish formal parity. diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/full-context-graph/knowledge-2091.json b/evidence/qwen38-h20-fp8-single-request-vllm032/results/full-context-graph/knowledge-2091.json deleted file mode 100644 index 947c88095..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/full-context-graph/knowledge-2091.json +++ /dev/null @@ -1,5 +0,0 @@ -{ - "ref": "store:case.qwen38-full-context-graph-integration-cand", - "revision": "sha256:f77484b1dbe4758f72af43f6afc82afcce6f49712cf1c174bd787f7a993982fb", - "readback_verified": true -} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/full-context-graph/knowledge.json b/evidence/qwen38-h20-fp8-single-request-vllm032/results/full-context-graph/knowledge.json deleted file mode 100644 index a7bc99c90..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/full-context-graph/knowledge.json +++ /dev/null @@ -1,5 +0,0 @@ -{ - "ref": "store:case.qwen38-full-context-graph-integration-cand", - "revision": "sha256:3de024fc3465ac332f43313022ea0784abd65d3184f21c9001b4a723d5158a35", - "readback_verified": true -} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/full-context-graph/preparation.md b/evidence/qwen38-h20-fp8-single-request-vllm032/results/full-context-graph/preparation.md deleted file mode 100644 index 394511b7a..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/full-context-graph/preparation.md +++ /dev/null @@ -1,13 +0,0 @@ -# Full-context Graph capacity preparation - -Issue67; model-plane implementation, candidate state; formal performance and quality INCONCLUSIVE. Runtime proposal4f835312bfa368281cca102e0637a6e5c8f99159 derives from engine main7f5e64d52dc91a66b4a68aadb56641f543d063c8. No GPU measurement exists for this slice. - -Observed mechanism: the prior bucket ceiling8448 and prompt+256 headroom cause an8192-input/2048-output request to demote to eager after its small reserved decode window. The completed2076 HTTP observation includes this delivered behavior. Hypothesis: keeping one prewarmed graph at configured max_context_tokens will remove that transition and retain replay across all admitted output positions. - -The opt-in full_context policy requires one active request. Default buckets are preserved. Expected tradeoff: more resident KV memory and masked-attention work for shorter inputs. Same-capacity graph reuse resets request state; this still needs GPU numerical checks because a new capacity can change floating-point behavior. - -Initial CPU implementation passed23 runner checks. Independent review found that unprimed direct prefill at exactly the context limit would lazy-capture at an out-of-range position and retain the failed request. The revision now skips lazy capture when no decode position remains. A regression verifies full-capacity prefill and normal over-limit append rejection/cleanup; independent rerun of23 CPU checks passed. Tests also cover all2048 positions after8192 inputs, fresh reuse, unchanged default selection and rejected concurrent admission. The initial review rejection is retained here; no GPU failure is implied. - -Disposition: commit and draft review only. No service gain, GPU correctness, full task quality or formal parity conclusion is made. A subsequent bounded comparison must use matched GEMM implementations and the same fixed HTTP client/input/output workload, preserve both arms, and retain actual cleanup and token evidence. - -The prepared branch later integrated the reviewed DeepGEMM code at2061bbcda4bb886fd1bb0468c5f1aa587aaa0300. Its Graph runtime/test slice is byte-identical to the reviewed4f83531 implementation. A subsequent GPU diagnostic remains prospective. diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/merged-flash-prefill-v1/knowledge.json b/evidence/qwen38-h20-fp8-single-request-vllm032/results/merged-flash-prefill-v1/knowledge.json deleted file mode 100644 index fb89bf788..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/merged-flash-prefill-v1/knowledge.json +++ /dev/null @@ -1 +0,0 @@ -{"id": "case.qwen38-merged-flash-prefill-cand", "revision": "sha256:811ad9bf7620d9ee591e924c0271629e47570e7719aabd9d629ec7dea59df87c", "readback_verified": true} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/merged-flash-prefill-v1/observation.md b/evidence/qwen38-h20-fp8-single-request-vllm032/results/merged-flash-prefill-v1/observation.md deleted file mode 100644 index d4d950a31..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/merged-flash-prefill-v1/observation.md +++ /dev/null @@ -1,19 +0,0 @@ -# Merged FP8 plus FlashAttention: actual engineering result - -Under InferRSI Issue #67, job 2175 completed with scheduler exit `0:0` in 278 seconds on one H20. This is **INCONCLUSIVE** for quality and parity acceptance. It is a single original 8K input / 2048-output comparison, ineligible for the formal 10-input × 3-pair reducer. - -| HTTP measurement | Native prefill | SDPA Flash prefill | -| --- | ---: | ---: | -| TTFT | 2.864008470 s | 2.321464958 s | -| TPOT | 12.212332628 ms | 12.188997569 ms | -| Request latency | 27.862653359 s | 27.272342981 s | - -Observed TTFT is 18.943502% lower in this one comparison. Each arm had a discarded 2048-token warmup and one measured request; all four original raw SSE responses pass the unchanged official-client accounting audit. The measured output token sequences **differ**, first at zero-based output position 5. Therefore no quality-preserving gain is established. Quality screening remains required before promotion; attention arithmetic differs despite unchanged weights and valid same-candidate Graph behavior. - -Synthetic tests used production 24Q/4KV heads, dimension 256, and merged-V token stride 14336 at lengths 129 and 8192. Six changed-input eager/Graph replays were exact, causal prefixes were preserved, and two prefill-to-decode outputs plus full cache updates were exact. The short native-rounding observation was max_abs 0.015625, NRMSE 0.002592582 with no acceptance tolerance assigned. This synthetic probe used injected projections and is not checkpoint FP8 evidence. The separate actual-checkpoint probe passed all 16 same-candidate eager/Graph predictions, 15 replays and final live state at capacity 10240. - -All 16 component and 60 source hashes, complete before/after model/runtime and overlay checks, postflight statuses and external terminal evidence were verified. Replayed audit exactly matches the original. Both owned service processes exited 0 without forced termination or remaining groups. No warning/error/traceback marker was found in archived run logs. Attention selection is evidenced by launch configuration plus frozen constructor source; no independent call counter was added, and no equivalence to vLLM FA3 is claimed. - -Preparation negatives remain: initial operator import-order lint failure, three collector/auditor line-length findings, one retained control-script line-length finding, and unsupported initial wrapper CLI flags, all fixed before submission. A premature remote hash check ran before asynchronous source transfer completed and failed with a missing manifest; it submitted no job. After transfer completion all 16 hashes passed and the sole GPU job was submitted. The earlier unrelated job 2113 startup timeout remains retained in its original case. Nineteen persistent CPU controls distinguish 14 raw/synthetic-health audits and 5 actual-main mocks; none substitute for GPU validation. - -Raw archive SHA256 `a64663a8b0867d41150dd53dcb34fa5f303c02bd0791aa2074d4866c1f274f4e`; raw-audit SHA256 `c672817356064e27f95b513b7e991572f7b407da1eae42d0c7d8880fc6107184`. Component freeze `e3795574ca2e44725af4440f3b30dbf4c32012146a24cac572ef2a14b2705721` was published prospectively in Issue #67. Source identity and previous preparation are retained. Search allocation spent through this run is 2613 / 28800 GPU-seconds; other phase caps are unchanged. No full quality, retrieval, formal parity, or merge readiness is asserted. diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/merged-flash-prefill-v1/parent-review.json b/evidence/qwen38-h20-fp8-single-request-vllm032/results/merged-flash-prefill-v1/parent-review.json deleted file mode 100644 index 015effbcf..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/merged-flash-prefill-v1/parent-review.json +++ /dev/null @@ -1,15 +0,0 @@ -{ - "reviewer": "root", - "scope": "independent interpretation of delegated raw-receipt review and unchanged auditor replay", - "requests": 4, - "measured_requests": 2, - "ttft_ratio": 0.8105649764179438, - "ttft_reduction_percent": 18.94350235820562, - "tpot_ratio": 0.9980892217854955, - "output_token_ids_equal": false, - "first_different_output_token_zero_based": 5, - "full_quality_required": true, - "formal_statistics_eligible": false, - "verdict": "INCONCLUSIVE", - "receipt_sha256": "3c730644ac349ff858b10aa7ac10d621ff2eaa1cf71fbb175f98c6e08db6d3ca" -} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/merged-flash-prefill-v1/raw-audit.json b/evidence/qwen38-h20-fp8-single-request-vllm032/results/merged-flash-prefill-v1/raw-audit.json deleted file mode 100644 index 7283efc43..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/merged-flash-prefill-v1/raw-audit.json +++ /dev/null @@ -1,79 +0,0 @@ -{ - "schema": 1, - "scope": "single-case engineering HTTP comparison only", - "verdict": "INCONCLUSIVE", - "formal_statistics_eligible": false, - "runtime_scheduler_quality_bindings_required": true, - "raw_accounting_valid": true, - "requests_audited": 4, - "warmups_discarded": 2, - "measurements": [ - { - "arm": "native", - "case": "n8192-s0", - "input_tokens": 8192, - "output_tokens": 2048, - "ttft_s": 2.864008470089175, - "tpot_s": 0.012212332627739672, - "latency_s": 27.862653359072283, - "output_tokens_per_second": 73.50340879624648, - "wire_sha256": "d3493ef42b60f1e912734d3f20a81ecd648598deb967250d5d972898d64ca5ef" - }, - { - "arm": "sdpa_flash", - "case": "n8192-s0", - "input_tokens": 8192, - "output_tokens": 2048, - "ttft_s": 2.3214649580186233, - "tpot_s": 0.012188997568606304, - "latency_s": 27.272342980955727, - "output_tokens_per_second": 75.09439146574675, - "wire_sha256": "64efde7c644737d28b30d11c3e4d8df990bee56e56f9f8072e7b40f90b814e85" - } - ], - "measurement_token_ids_equal": false, - "attention_mode_evidence": "launch options plus bound constructor; no attention counter", - "source_identities_sha256": "5a8672bfc8982581b304d4a6137926aadf2b7a174bcff328770c9dae22541b9b", - "collector_sha256": "389c26a2d11978125fb4e06b4c1f5fc983f3758a1b048998cb39bf46d11fda69", - "auditor_sha256": "f252787270e1f4331b1ff18c29d95d353dfe793e8f75b2b6dcb7e305e93cbef3", - "receipts": [ - { - "arm": "native", - "purpose": "warmup", - "wire_sha256": "0fad85f1c60b93198166b0f535bb956660628bc845cb9c6a2ead29b0880416a5", - "client_sha256": "c7c07bec49b811be9e8fe0a7f83cf4926e97710816ffd5dc04450bc0374d5ec7" - }, - { - "arm": "native", - "purpose": "measurement", - "wire_sha256": "d3493ef42b60f1e912734d3f20a81ecd648598deb967250d5d972898d64ca5ef", - "client_sha256": "8c841d5fad535296427f434a2ab42536cd8296a4324da3d13bc1d4188dc45775" - }, - { - "arm": "native", - "ready_health_sha256": "0d164321ef0d38008e65f2c2454966c65bf129cd79500eacf348dea89397911c", - "health_sha256": "11ba85ffac373d4ee7c5d3f703d633ac2ab810c6fb3ff3f4b3ee5d96cf6ab230", - "config_sha256": "f2946e0e21de6066aa5505075a882610b2951e253a78865be32f5bd994811e9f", - "shutdown_sha256": "7e33d01f9def10f57fab56f508af7a0a707e0edfb517dac5cdf7ac32ed6c15c9" - }, - { - "arm": "sdpa_flash", - "purpose": "warmup", - "wire_sha256": "351242a1b81d3fe861453e872d38e248e14cab034fe34b95bcc3f566c15827bb", - "client_sha256": "1eabe84479d42a064979d61799c26557017938ff8df490310f028a065021c5ff" - }, - { - "arm": "sdpa_flash", - "purpose": "measurement", - "wire_sha256": "64efde7c644737d28b30d11c3e4d8df990bee56e56f9f8072e7b40f90b814e85", - "client_sha256": "ee0023ae9765ceae889e0fdab23f08d0c4089da3ca5d3335d0a1b943f16ae643" - }, - { - "arm": "sdpa_flash", - "ready_health_sha256": "5da7d129c8e00538afa10ca41935607a80c7f662baae8fe0ba24e22d8544d6c1", - "health_sha256": "720ba1fcc7027bccc1a9e07d1c7405bc701a8d851029d6d20e6f1ab85eadab41", - "config_sha256": "ea81d7077bc3321de9884d5f7f9241b45d6a3628c9238e3d1f71f87e50a575aa", - "shutdown_sha256": "97bc349f290872d4e9828373a86414c84126b9863db30dd0b1f023d0c5749122" - } - ] -} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/merged-flash-prefill-v1/receipt-review.json b/evidence/qwen38-h20-fp8-single-request-vllm032/results/merged-flash-prefill-v1/receipt-review.json deleted file mode 100644 index 1a9775dc9..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/merged-flash-prefill-v1/receipt-review.json +++ /dev/null @@ -1,38 +0,0 @@ -{ - "job_id": 2175, - "scheduler_state": "COMPLETED", - "scheduler_exit": "0:0", - "duration_seconds": 278, - "allocated_gpu_seconds": 278, - "archive_sha256": "a64663a8b0867d41150dd53dcb34fa5f303c02bd0791aa2074d4866c1f274f4e", - "component_files_verified": 16, - "source_files_verified": 60, - "pre_post_model_runtime_equal": true, - "pre_post_overlay_equal": true, - "postflight_status_all_zero": true, - "audit_exit": 0, - "fresh_audit_equals_original": true, - "fresh_audit_sha256": "c672817356064e27f95b513b7e991572f7b407da1eae42d0c7d8880fc6107184", - "operators_complete": true, - "operator_lengths": [ - 129, - 8192 - ], - "synthetic_graph_replays_exact": 6, - "synthetic_boundary_decode_steps_exact": 2, - "graph_complete": true, - "graph_predictions_exact": 16, - "graph_replays": 15, - "graph_final_state_exact": true, - "capacity": 10240, - "native_rounding_observation": { - "max_abs": 0.015625, - "nrmse": 0.002592581557109952, - "acceptance_tolerance": null - }, - "measurement_token_ids_equal": false, - "scope": "single-case engineering comparison only; not formal statistics or quality acceptance", - "verdict": "INCONCLUSIVE", - "warnings": {}, - "attention_mode_evidence": "Launch options plus bound constructor; no attention counter." -} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/profile-merged-v1/knowledge.json b/evidence/qwen38-h20-fp8-single-request-vllm032/results/profile-merged-v1/knowledge.json deleted file mode 100644 index 534611d8c..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/profile-merged-v1/knowledge.json +++ /dev/null @@ -1 +0,0 @@ -{"id": "case.qwen38-merged-decode-profile-cand", "revision": "sha256:fe29579f307020bef30df65196519c64b08c7a531ea4e4cb154f9ec95c1ab69f", "readback_verified": true} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/profile-merged-v1/observation.md b/evidence/qwen38-h20-fp8-single-request-vllm032/results/profile-merged-v1/observation.md deleted file mode 100644 index 5f1bdb857..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/profile-merged-v1/observation.md +++ /dev/null @@ -1,22 +0,0 @@ -# Merged FP8 graph decode: profiler observation - -Parent-reviewed diagnostic under InferRSI Issue #67. **INCONCLUSIVE** for service performance and full model quality. This single instrumented profile has no matched baseline profile. It supports candidate selection only; it does not establish an HTTP gain, formal regression cause, TPOT, or quality acceptance. - -Job 2169 completed with scheduler exit `0:0` in 91 seconds using one GPU and eight CPUs: **91 allocated GPU-seconds**. Completion records `complete=true` and `state_released=true`. All four postflight statuses are zero. Before/after model-runtime and overlay JSON are identical; all 21 component and 59 source checks are successful and agree with locally verified manifests. Private originals remain outside the knowledge record; `receipt-review.json` contains sanitized hashes and counts. - -The observation covers one 8192-token prefill and eight pooled CUDA Graph decode steps after one prefill and two decode warmups, with graph capacity 10240. The backend uses merged native FP8 projections, FlashInfer small-M decode GEMMs, DeepGEMM prefill, FlashInfer GDN, and native full attention. GPU kernels are attributed inside synchronized profiler intervals. Source-region attribution is inferred from repeated graph-node ordering, not module NVTX annotations. - -| Instrumented phase | Host interval | GPU kernel sum | Kernel count | -| --- | ---: | ---: | ---: | -| Prefill | 2918.167 ms | 2861.021 ms | 3470 | -| Eight decode steps | 103.192 ms | 97.940 ms | 17328 | - -Decode averages 12.243 ms of summed device kernel duration per token. Each replay contains 2163 graph kernel nodes, averaging 12.232 ms; graph sums range from 12.223 to 12.240 ms. Twenty-four additional kernels cover sampling and input/position fills. Timestamp overlap gives a 97.775 ms interval union, slightly below the sum. These instrumented sums must not be compared directly with HTTP latency or interpreted as host launch counts. - -FP8 projections consume 7.128 ms/token; their activation quantization adds 0.396 ms/token. Separate BF16 GDN b/a projections plus split-K reductions consume 0.598 ms/token (96 GEMMs and 96 reductions). The convolution-window/copy/convolution/SiLU region consumes 0.503 ms/token, while FlashInfer GDN recurrence itself consumes 0.238 ms/token. The `deep_gemm::fp8_gemm_kernel_swapAB` decode symbols originate from the intended FlashInfer small-M API; they do not demonstrate accidental large-M DeepGEMM dispatch. The 256 FP8 module and activation-quantization counts agree. - -The next bounded model hypothesis is to merge each GDN layer's two BF16 `[48,5120]` b/a projections into one `[96,5120]` projection ordered `[b(48), a(48)]`. The pinned vLLM source already uses this structure. Preserve BF16 exclusions, original checkpoint mappings, 256 FP8 modules, scales, and immutable model bytes. Keep the decode bridge's contiguous conversions until its stride contract is verified. A new GEMM shape may change reductions and BF16 results; reduced kernel count or time remains a hypothesis. Required checks include loader structure, actual-weight operator comparison, fixed-history eager/graph continuation, reset/reuse, and the existing frozen quality protocol before a separately budgeted matched service comparison. - -Defer causal-convolution update fusion to a separate step. Its state mutation and rounding risks exceed b/a projection fusion: preserve the request-owned window, chronological update, and current BF16 convolution-output round-trip before SiLU. The pinned vLLM implementation applies SiLU to an FP32 accumulator and is not a drop-in arithmetic equivalent. Combining these changes would obscure attribution. - -Public source provenance is preserved in `source-provenance.json`, including immutable URLs and byte hashes. The primary references are vLLM's `qwen_gdn_linear_attn.py` (`create_ba_proj`, `split_ba`, and decode flow), `flashinfer.py` (small-M dispatch), and `causal_conv1d.py` (deferred update semantics). The diagnostic changes no engine or validator. Its findings and limitations are recorded as a store-owned candidate; this is not a promoted library entry. diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/profile-merged-v1/parent-review.json b/evidence/qwen38-h20-fp8-single-request-vllm032/results/profile-merged-v1/parent-review.json deleted file mode 100644 index 029c25020..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/profile-merged-v1/parent-review.json +++ /dev/null @@ -1,13 +0,0 @@ -{ - "reviewer": "root", - "attribution": "Independent parent review of Astra xhigh trace analysis and Astra low receipt/draft preparation", - "checked": [ - "Trace SHA and synchronized phase kernel counts/sums independently recomputed", - "Six pinned vLLM readset files verified against local git objects", - "Parser/source grouping reviewed as inference; no NVTX attribution claim", - "Receipt review and model/overlay/component evidence checked", - "No formal TPOT or speedup claim from instrumented profile" - ], - "verdict": "INCONCLUSIVE", - "recommendation": "Record diagnostic and pursue BF16 b/a single-slice hypothesis" -} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/profile-merged-v1/phase-summary.json b/evidence/qwen38-h20-fp8-single-request-vllm032/results/profile-merged-v1/phase-summary.json deleted file mode 100644 index 758fa6a7b..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/profile-merged-v1/phase-summary.json +++ /dev/null @@ -1,179 +0,0 @@ -{ - "source_trace_sha256": "1b8acff4a74a52f4583331f2eace080c2304701f86a88dd58697481ce05cb5c6", - "method": "GPU kernel start/end within CPU record_function boundaries; both phases end with CUDA synchronize; one device and stream verified. Device duration is instrumented and not a service latency claim.", - "phases": { - "profile_prefill_8192": { - "host_phase_us": 2918166.624, - "kernel_count": 3470, - "kernel_sum_us": 2861020.825, - "kernel_union_us": 2861020.813720703, - "first_to_last_kernel_us": 2905205.6950683594, - "memcpy_count": 116, - "memcpy_us": 26983.22, - "memset_count": 34, - "memset_us": 33.246 - }, - "profile_graph_decode_8": { - "host_phase_us": 103191.981, - "kernel_count": 17328, - "kernel_sum_us": 97940.197, - "kernel_union_us": 97775.2578125, - "first_to_last_kernel_us": 103063.44580078125, - "memcpy_count": 8, - "memcpy_us": 20.512, - "memset_count": 144, - "memset_us": 124.962 - } - }, - "decode_structure": { - "method": "Repeated graph-node ordering checked for all eight replays; each of 64 layer blocks anchored by hidden input RMSNorm [grid=1], then checked against model.py's 30-node GDN or 44-node full-attention sequence. Attribution to source phases is inference from these matched sequences, not NVTX module annotations.", - "graph_nodes_per_replay": 2163, - "non_graph_kernel_count": 24, - "non_graph_kernel_us": 82.534, - "groups": { - "fp8_mlp_gate_up": { - "kernel_count": 512, - "device_us": 24650.932, - "per_token_us": 3081.3665, - "share_of_all_decode_kernels_pct": 25.169371468591187 - }, - "fp8_mlp_down": { - "kernel_count": 512, - "device_us": 14005.476999999999, - "per_token_us": 1750.6846249999999, - "share_of_all_decode_kernels_pct": 14.300029435309385 - }, - "fp8_gdn_qkvz": { - "kernel_count": 384, - "device_us": 9370.256999999998, - "per_token_us": 1171.2821249999997, - "share_of_all_decode_kernels_pct": 9.56732504836599 - }, - "fp8_attention_output": { - "kernel_count": 512, - "device_us": 6104.394999999997, - "per_token_us": 763.0493749999996, - "share_of_all_decode_kernels_pct": 6.232777947138493 - }, - "full_scores_mask_softmax_value_bmm": { - "kernel_count": 1408, - "device_us": 5577.405, - "per_token_us": 697.175625, - "share_of_all_decode_kernels_pct": 5.694704698214973 - }, - "bf16_gdn_b_a_projection_and_splitK_reduce": { - "kernel_count": 1536, - "device_us": 4785.7339999999995, - "per_token_us": 598.2167499999999, - "share_of_all_decode_kernels_pct": 4.886383881788598 - }, - "bf16_lm_head": { - "kernel_count": 8, - "device_us": 4541.804, - "per_token_us": 567.7255, - "share_of_all_decode_kernels_pct": 4.637323733379871 - }, - "gdn_conv_window_copy_conv_silu": { - "kernel_count": 1920, - "device_us": 4021.128000000001, - "per_token_us": 502.64100000000013, - "share_of_all_decode_kernels_pct": 4.105697275654858 - }, - "hidden_input_rmsnorm": { - "kernel_count": 512, - "device_us": 3582.819, - "per_token_us": 447.852375, - "share_of_all_decode_kernels_pct": 3.658170097411587 - }, - "full_q_gate_layout_qk_norm_partial_rope": { - "kernel_count": 2048, - "device_us": 3496.767999999999, - "per_token_us": 437.0959999999999, - "share_of_all_decode_kernels_pct": 3.5703093388713514 - }, - "fp8_activation_quant": { - "kernel_count": 2048, - "device_us": 3167.651999999999, - "per_token_us": 395.9564999999999, - "share_of_all_decode_kernels_pct": 3.234271623938023 - }, - "fp8_full_qkv": { - "kernel_count": 128, - "device_us": 2896.261, - "per_token_us": 362.032625, - "share_of_all_decode_kernels_pct": 2.9571729368688118 - }, - "gdn_qk_repeat_Alog_cast_output_zero": { - "kernel_count": 1536, - "device_us": 2219.371, - "per_token_us": 277.421375, - "share_of_all_decode_kernels_pct": 2.2660471062764964 - }, - "post_attention_add_rmsnorm": { - "kernel_count": 512, - "device_us": 2050.9760000000006, - "per_token_us": 256.37200000000007, - "share_of_all_decode_kernels_pct": 2.0941105519728542 - }, - "flashinfer_gdn_recurrence": { - "kernel_count": 384, - "device_us": 1905.0459999999991, - "per_token_us": 238.1307499999999, - "share_of_all_decode_kernels_pct": 1.9451114642948892 - }, - "gdn_norm_silu_gate": { - "kernel_count": 1152, - "device_us": 1738.3990000000006, - "per_token_us": 217.29987500000007, - "share_of_all_decode_kernels_pct": 1.7749596725846901 - }, - "mlp_silu_multiply": { - "kernel_count": 1024, - "device_us": 1693.3380000000006, - "per_token_us": 211.66725000000008, - "share_of_all_decode_kernels_pct": 1.728950984241946 - }, - "mlp_output_residual_add": { - "kernel_count": 512, - "device_us": 780.4710000000003, - "per_token_us": 97.55887500000004, - "share_of_all_decode_kernels_pct": 0.7968852666285737 - }, - "full_kv_cache_index_copy": { - "kernel_count": 256, - "device_us": 651.3229999999999, - "per_token_us": 81.41537499999998, - "share_of_all_decode_kernels_pct": 0.6650211250851373 - }, - "full_sigmoid_output_gate": { - "kernel_count": 256, - "device_us": 379.388, - "per_token_us": 47.4235, - "share_of_all_decode_kernels_pct": 0.3873669970257462 - }, - "embedding_and_position_tables": { - "kernel_count": 136, - "device_us": 183.008, - "per_token_us": 22.876, - "share_of_all_decode_kernels_pct": 0.18685688369607834 - }, - "final_hidden_rmsnorm": { - "kernel_count": 8, - "device_us": 55.711, - "per_token_us": 6.963875, - "share_of_all_decode_kernels_pct": 0.056882670962975494 - } - }, - "per_replay_kernel_us": [ - 12233.914, - 12224.673, - 12236.986, - 12240.038, - 12234.154, - 12237.394, - 12227.691, - 12222.813 - ] - }, - "publication_note": "Compact derived copy; long per-kernel symbol lists omitted. Original trace and full report remain private; original trace digest retained. Source-region attribution is inference." -} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/profile-merged-v1/receipt-review.json b/evidence/qwen38-h20-fp8-single-request-vllm032/results/profile-merged-v1/receipt-review.json deleted file mode 100644 index 1e540b5e4..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/profile-merged-v1/receipt-review.json +++ /dev/null @@ -1,84 +0,0 @@ -{ - "job_id": 2169, - "campaign_verdict": "INCONCLUSIVE", - "scheduler": { - "state": "COMPLETED", - "exit_code": "0:0", - "duration_seconds": 91, - "allocated_gpus": 1, - "allocated_cpus": 8, - "gpu_seconds": 91, - "external_terminal_sha256": "950997d69cea4261148b4313433a25330725e611ac87bbcea7bb86a1ec2ee0a8" - }, - "completion": { - "complete": true, - "state_released": true, - "sha256": "87cae17245dce4c7a998fd90c16f9682a8dd640809c30d7987468771040e67cc", - "process_exit_code": 0 - }, - "postflight_status": { - "model_runtime": 0, - "overlay": 0, - "engine_source": 0, - "component": 0 - }, - "integrity_checks": { - "model_runtime": { - "before_after_equal": true, - "before_sha256": "5077d874c429e86df9099f78d17dc123f71803c86c838b138af9202547af65ae", - "after_sha256": "5077d874c429e86df9099f78d17dc123f71803c86c838b138af9202547af65ae" - }, - "overlay": { - "before_after_equal": true, - "before_sha256": "d20a0b32aea71f311c83a1a11348a90149c1b41c4a6ec557464f3895de9372a9", - "after_sha256": "d20a0b32aea71f311c83a1a11348a90149c1b41c4a6ec557464f3895de9372a9" - }, - "component": { - "before_after_equal": true, - "all_ok": true, - "file_count": 21, - "before_sha256": "dbac07005cb8a2c324bc8785584b1007db2a23d414160812604b76a4768aa3c0", - "after_sha256": "dbac07005cb8a2c324bc8785584b1007db2a23d414160812604b76a4768aa3c0" - }, - "source": { - "before_after_equal": true, - "all_ok": true, - "file_count": 59, - "before_sha256": "dd30e4b2d36e9e02f419744cfeb592fc3b172fd45fe7d2b7d701404843f7ea1a", - "after_sha256": "dd30e4b2d36e9e02f419744cfeb592fc3b172fd45fe7d2b7d701404843f7ea1a" - }, - "component.sha256": { - "file_count": 21, - "sha256": "76c36671b4f2365535a08630b6106cf741b8dc819e3253b8442d230344946bbe", - "local_hashes_match": true - }, - "engine-files.sha256": { - "file_count": 59, - "sha256": "d6be54685c86cd3fe3c835893bdbbce2019b1bbbb3748652c5136bd889e70529", - "local_hashes_match": true - } - }, - "archive": { - "sha256": "dace1707e052f2a12cca746c33d1b3fd89de92296a8d64ef4ff4322564574b27", - "bytes": 2957075, - "file_count": 79, - "member_count": 92, - "safe_data_filter_extraction": true, - "frozen_files_overwritten": false, - "excluded_directories": [ - "tmp", - "cache", - ".cache" - ] - }, - "profile_trace": { - "sha256": "1b8acff4a74a52f4583331f2eace080c2304701f86a88dd58697481ce05cb5c6", - "bytes": 36125711 - }, - "scope_caveats": [ - "Profiler-overhead kernel diagnosis only.", - "No HTTP service, end-to-end latency, throughput, quality, or performance-gain claim.", - "Original runtime paths and scheduler details retained only in private raw artifacts.", - "No new scheduled jobs, installations, or model executions were performed for this receipt review." - ] -} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/profile-merged-v1/source-provenance.json b/evidence/qwen38-h20-fp8-single-request-vllm032/results/profile-merged-v1/source-provenance.json deleted file mode 100644 index 26d5a709d..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/profile-merged-v1/source-provenance.json +++ /dev/null @@ -1,81 +0,0 @@ -{ - "status": "INCONCLUSIVE", - "purpose": "Sanitized profile observation and candidate-selection provenance. Parent review and KnowledgeStore recording are complete as retained in parent-review.json and knowledge.json; campaign remains INCONCLUSIVE.", - "public_sources": [ - { - "repository": "https://github.com/vllm-project/vllm", - "revision": "5df63a7203cdb9328d824a27b682e77c7b9928be", - "path": "vllm/model_executor/kernels/linear/scaled_mm/flashinfer.py", - "sha256": "8ee5dd1883336d909e62b35c76e1f0f9a159a25533cc2d4c9b50c0aab0b5c21b", - "local_relative_path": "source-readset/vllm/model_executor/kernels/linear/scaled_mm/flashinfer.py", - "url": "https://github.com/vllm-project/vllm/blob/5df63a7203cdb9328d824a27b682e77c7b9928be/vllm/model_executor/kernels/linear/scaled_mm/flashinfer.py" - }, - { - "repository": "https://github.com/vllm-project/vllm", - "revision": "5df63a7203cdb9328d824a27b682e77c7b9928be", - "path": "vllm/model_executor/layers/mamba/gdn/qwen_gdn_linear_attn.py", - "sha256": "3fbf257fef7f303b0082fdbedce4394df74b78ad9a594157079c89632b5c221e", - "local_relative_path": "source-readset/vllm/model_executor/layers/mamba/gdn/qwen_gdn_linear_attn.py", - "url": "https://github.com/vllm-project/vllm/blob/5df63a7203cdb9328d824a27b682e77c7b9928be/vllm/model_executor/layers/mamba/gdn/qwen_gdn_linear_attn.py" - }, - { - "repository": "https://github.com/vllm-project/vllm", - "revision": "5df63a7203cdb9328d824a27b682e77c7b9928be", - "path": "vllm/model_executor/layers/mamba/ops/causal_conv1d.py", - "sha256": "cb16cc9250c4195c09d6d83b43bba607c2d749b621d8d135a920443d1577265f", - "local_relative_path": "source-readset/vllm/model_executor/layers/mamba/ops/causal_conv1d.py", - "url": "https://github.com/vllm-project/vllm/blob/5df63a7203cdb9328d824a27b682e77c7b9928be/vllm/model_executor/layers/mamba/ops/causal_conv1d.py" - }, - { - "repository": "https://github.com/vllm-project/vllm", - "revision": "5df63a7203cdb9328d824a27b682e77c7b9928be", - "path": "vllm/model_executor/layers/layernorm.py", - "sha256": "7be031e88921fb21661f3b9ee28b4ccb7746405e0ff32ed5895d75b43d3357ed", - "local_relative_path": "source-readset/vllm/model_executor/layers/layernorm.py", - "url": "https://github.com/vllm-project/vllm/blob/5df63a7203cdb9328d824a27b682e77c7b9928be/vllm/model_executor/layers/layernorm.py" - }, - { - "repository": "https://github.com/vllm-project/vllm", - "revision": "5df63a7203cdb9328d824a27b682e77c7b9928be", - "path": "vllm/model_executor/layers/fused_qk_norm_rope.py", - "sha256": "c201d88fd5e810a819cf671e8883037f53f5801211b8141e4801df1a4e5d558d", - "local_relative_path": "source-readset/vllm/model_executor/layers/fused_qk_norm_rope.py", - "url": "https://github.com/vllm-project/vllm/blob/5df63a7203cdb9328d824a27b682e77c7b9928be/vllm/model_executor/layers/fused_qk_norm_rope.py" - }, - { - "repository": "https://github.com/vllm-project/vllm", - "revision": "5df63a7203cdb9328d824a27b682e77c7b9928be", - "path": "vllm/model_executor/models/qwen3_next.py", - "sha256": "09ea2a385259edc9feadc79e304e86a31582b9770f80b8003d2d641782405818", - "local_relative_path": "source-readset/vllm/model_executor/models/qwen3_next.py", - "url": "https://github.com/vllm-project/vllm/blob/5df63a7203cdb9328d824a27b682e77c7b9928be/vllm/model_executor/models/qwen3_next.py" - } - ], - "evidence": [ - { - "artifact": "receipt-review.json", - "sha256": "0fa866171cef72a32f162a1d7ff7d5ed2a0824d43347c481640f0d759ae9d289" - }, - { - "artifact": "analysis.md", - "sha256": "46a7e6442a29dc48958fcd9b3331f22ae6375746582d1f640c59b4e9ca4ad7c1" - }, - { - "artifact": "phase-summary.json", - "sha256": "31f883855ea4b4220c45e7b6a0a86848566ca65bd6c05cdd86b4dcaf744c3539" - }, - { - "artifact": "source-readset.json", - "sha256": "bb701f8b9fca692cbe05dc34193f79a9ee259c19a302e0306c0a50b6eb791d36" - } - ], - "private_exclusions": [ - "Raw trace and raw archive", - "Scheduler receipts and logs", - "Device records", - "Unredacted runtime manifests and source snapshots" - ], - "publication_note": "Evidence artifact names and source-readset relative paths refer to controlled original storage. Public URLs and byte hashes preserve source attribution; originals are not bundled here.", - "original_preparation_purpose": "Sanitized profile observation and candidate-selection provenance; parent review required before KB recording.", - "original_source_provenance_sha256": "6b0297f6aff8f9c82dab909b7b0dda71c8647f24e46285ca5dce84fc97e5157e" -} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/quality-executor/production-binding-observation.md b/evidence/qwen38-h20-fp8-single-request-vllm032/results/quality-executor/production-binding-observation.md deleted file mode 100644 index fc4fd3391..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/quality-executor/production-binding-observation.md +++ /dev/null @@ -1,32 +0,0 @@ -# Production quality binding and runtime inventory - -Issue67 harness-only preparation. The delegated adapter now accepts the original -32/164 cohorts and authenticates actual quality-v5 export, dataset, protocol, -qualification and stable result bytes. It preserves official EvalPlus defaults, -source data, answers, tolerances and per-case timing. A pending record is never -scoreable; successful terminal/scheduler and twelve postflight checks are -required before metadata-only sealing. No all-canonical-pass gate was added. - -The parent independently reviewed the14-source manifest -5911860f01e7bc6bc54d6f2da582ac4cba4e9d59b691613c8cdc33d176039bb6 -and reran17pureCPUcontrols, including the unchanged v5 scorer consumer. Three -isolation/supervision helpers and the v5 provenance reader are byte-identical -to earlier reviewed snapshots. Source preparation style failures were retained. - -CPU2137 failed1:0/1s,4CPU/0GPU before inventory: the new metadata-only wrapper -assigned TASK_ROOT without exporting it. The Python inventory therefore raised -KeyError. Original source/output retained. Wrapper v2 adds export, with no change -to the production adapter, dependencies or resource policy. - -CPU2138 completed0:0/45s,4CPU/0GPU. Runtime inventory has25253entries and digest -sha256:b0ec2f42b1e415e04dbe89c1d2aae50b2a33cefaf5e8e75a41f7280bd81c34a2. -All25241non-executor entries exactly match the actual2024runtime inventory. -Only12declared executor files form the new source identity. Resource/evaluator -policy differs from the full164pilot only in scope/qualification metadata. - -A qualification/protocol proposal is being reviewed using the actual2014 bounds, -2018 official external-export,2024 full164 capacity and2117 failure-classification -evidence plus this source/consumer review. No new production entry or model code -has executed yet, and the proposal is not activated. Original2024 FAILED and its -seven official HumanEval32 assertions remain preserved. This record does not -claim new runtime execution, model quality, performance parity or campaign PASS. diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/quality-executor/production-binding-review.json b/evidence/qwen38-h20-fp8-single-request-vllm032/results/quality-executor/production-binding-review.json deleted file mode 100644 index eaf0c566b..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/quality-executor/production-binding-review.json +++ /dev/null @@ -1,11 +0,0 @@ -{ - "scope": "production export binding source/CPU controls", - "reviewer": "parent Codex, independent of delegated implementer", - "source_manifest_sha256": "5911860f01e7bc6bc54d6f2da582ac4cba4e9d59b691613c8cdc33d176039bb6", - "controls": 17, - "controls_passed": true, - "unchanged_helpers_verified": true, - "finding_count": 0, - "qualification_created": false, - "next": "Capture actual new runtime inventory and bind reviewed diagnostic evidence to the exact production identity before model-output evaluation." -} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/search-observations-2163-2175-integrity.json b/evidence/qwen38-h20-fp8-single-request-vllm032/results/search-observations-2163-2175-integrity.json deleted file mode 100644 index 7d286d039..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/results/search-observations-2163-2175-integrity.json +++ /dev/null @@ -1,18 +0,0 @@ -{ - "results/fp8-projections-search-v1/bound-result.json": "818046ab9571161fb631ff07d17e9ac267c95d1e1f50ce2b741fdb14cac93137", - "results/fp8-projections-search-v1/knowledge.json": "a320b154ed8b839dbe50ee7424124a79360649ea6ba974a0d8392ba9fcc806be", - "results/fp8-projections-search-v1/observation.md": "d11c4bbbe05cc4685955d4764f38bda12ef99e9cdc78184d49e6859d2f960371", - "results/fp8-projections-search-v1/parent-review.json": "f5fc1788edbc69a2bc099ed5cd05e2ece3901913e023910baf8f14747ba33eda", - "results/fp8-projections-search-v1/receipt-review.json": "51a28dcf3fcbb3099dcd633f7f60038fe5c44b832517f370809473dbe89d4620", - "results/merged-flash-prefill-v1/knowledge.json": "e990fa0ae9872b8e324df4a996f92e903c854c31bbf5e6086834eee82cda0cc5", - "results/merged-flash-prefill-v1/observation.md": "42342e366d39415bb1f2ea42e1b23cceb3537c87a14cb9174b0ad2aeb822a5ac", - "results/merged-flash-prefill-v1/parent-review.json": "058ddcab3d3445828c5fb9068802490439dbc9ade450d8dd71a017340bb06a4e", - "results/merged-flash-prefill-v1/raw-audit.json": "c672817356064e27f95b513b7e991572f7b407da1eae42d0c7d8880fc6107184", - "results/merged-flash-prefill-v1/receipt-review.json": "3c730644ac349ff858b10aa7ac10d621ff2eaa1cf71fbb175f98c6e08db6d3ca", - "results/profile-merged-v1/knowledge.json": "87de11494e763186daaabac573ba15dea8fc2d844a69a88f1fdd3860eb836db3", - "results/profile-merged-v1/observation.md": "b853259137023260e717eec329cb20de2b2d1e4b8e30e00c0e266bb1f93029f8", - "results/profile-merged-v1/parent-review.json": "446a6411f9b69f9c70b7d6ad1e09bf9397faaa453de452874baaea2f4bd9c3b8", - "results/profile-merged-v1/phase-summary.json": "b8523c0ee2c1e1a81872ed9b2d71f65ca6c3b8b4678fae3d21b6491056e9764b", - "results/profile-merged-v1/receipt-review.json": "0fa866171cef72a32f162a1d7ff7d5ed2a0824d43347c481640f0d759ae9d289", - "results/profile-merged-v1/source-provenance.json": "4a910772cfbef214067cffc260feb7c971ba9f15d1bf56d1bab12ac5620d3e83" -} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/summary.json b/evidence/qwen38-h20-fp8-single-request-vllm032/results/summary.json new file mode 100644 index 000000000..ce34c3a11 --- /dev/null +++ b/evidence/qwen38-h20-fp8-single-request-vllm032/results/summary.json @@ -0,0 +1,605 @@ +{ + "schema_version": 2, + "source_revision": "6c7e237935a22cc5e90e954d4f397376e753e7d8", + "artifact_id": "qwen38-h20-fp8-single-request-vllm032-6c7e237-v1", + "campaign_verdict": "INCONCLUSIVE", + "coverage": { + "original_files": 413, + "original_bytes": 5205664, + "indexed_result_documents": [ + "results/deepgemm-prefill/actual-2084.json", + "results/deepgemm-prefill/extraction-metadata.json", + "results/deepgemm-prefill/knowledge-2084.json", + "results/diagnostics/baseline-long-1961.json", + "results/diagnostics/benchmark-client-ledger.json", + "results/diagnostics/client-base-1950.json", + "results/diagnostics/client-repair-1950.json", + "results/diagnostics/demotion-preflight.json", + "results/diagnostics/graph-demotion-1955.json", + "results/diagnostics/graph-forced96-1957.json", + "results/diagnostics/graph-long-1951.json", + "results/diagnostics/http-summary.json", + "results/diagnostics/ipc-preflight-1960.json", + "results/diagnostics/quality-binding-gap.json", + "results/diagnostics/quality-binding-v5-controls.json", + "results/diagnostics/quality-eos-v4-check.json", + "results/diagnostics/quality-eos-v5-check.json", + "results/diagnostics/quality-null-echo-check.json", + "results/diagnostics/quality-score-v5-controls.json", + "results/diagnostics/repair-summary.json", + "results/diagnostics/retrieval-baseline-1968.json", + "results/diagnostics/sandbox-benign-retry-2002.json", + "results/diagnostics/sandbox-benign-retry-2006.json", + "results/diagnostics/sandbox-benign-retry-packages-before.json", + "results/diagnostics/sandbox-benign-v2-packages-verified.json", + "results/diagnostics/sandbox-benign-v2-probe-execution.json", + "results/diagnostics/sandbox-benign-v2-probe-stdout.json", + "results/diagnostics/sandbox-bounds-2012.json", + "results/diagnostics/sandbox-bounds-2014.json", + "results/diagnostics/sandbox-bounds-v1-late-bounds-stdout.json", + "results/diagnostics/sandbox-bounds-v1-summary.json", + "results/diagnostics/sandbox-bounds-v2-actual-review.json", + "results/diagnostics/sandbox-bounds-v2-bounds-stdout.json", + "results/diagnostics/sandbox-bounds-v2-serialized-evaluator-results.json", + "results/diagnostics/sandbox-bounds-v2-synthetic-stdout.json", + "results/diagnostics/sandbox-build-1986.json", + "results/diagnostics/sandbox-bwrap-build-2000.json", + "results/diagnostics/sandbox-bwrap-build-2001.json", + "results/diagnostics/sandbox-bwrap-build-destdir-control.json", + "results/diagnostics/sandbox-bwrap-build-extraction-control.json", + "results/diagnostics/sandbox-bwrap-build-make-control.json", + "results/diagnostics/sandbox-bwrap-build-ninja-control.json", + "results/diagnostics/sandbox-bwrap-capability-1999.json", + "results/diagnostics/sandbox-elf-context-2004.json", + "results/diagnostics/sandbox-elf-context-2005.json", + "results/diagnostics/sandbox-elf-context-v1-observations.json", + "results/diagnostics/sandbox-elf-context-v2-observations.json", + "results/diagnostics/sandbox-elf-dependency-2003.json", + "results/diagnostics/sandbox-elf-dependency-observations.json", + "results/diagnostics/sandbox-executor-2018.json", + "results/diagnostics/sandbox-executor-v1-actual-review.json", + "results/diagnostics/sandbox-executor-v1-execution-record.json", + "results/diagnostics/sandbox-executor-v1-summary.json", + "results/diagnostics/sandbox-full164-2024.json", + "results/diagnostics/sandbox-full164-v1-actual-review.json", + "results/diagnostics/sandbox-full164-v1-failed-status-metadata.json", + "results/diagnostics/sandbox-full164-v1-summary.json", + "results/diagnostics/sandbox-install-1992.json", + "results/diagnostics/sandbox-install-1994.json", + "results/diagnostics/sandbox-offline-install-1998.json", + "results/diagnostics/sandbox-offline-install-dependencies-verified.json", + "results/diagnostics/sandbox-offline-install-evaluator-packages-verified.json", + "results/diagnostics/sandbox-offline-install-wheels-before.json", + "results/diagnostics/sandbox-offline-prefetch-controls.json", + "results/diagnostics/sandbox-offline-prefetch-report.json", + "results/diagnostics/sandbox-offline-prefetch-wheel-records.json", + "results/diagnostics/sandbox-platform-fs-1997.json", + "results/diagnostics/sandbox-platform-fs-controls.json", + "results/diagnostics/sandbox-platform-fs-observations.json", + "results/diagnostics/sandbox-resolve-1985.json", + "results/diagnostics/sandbox-resolve-1991.json", + "results/diagnostics/sandbox-synthetic-2009.json", + "results/diagnostics/sandbox-synthetic-v3-probe-execution.json", + "results/diagnostics/sandbox-synthetic-v3-probe-stdout.json", + "results/diagnostics/sandbox-transport-1996.json", + "results/diagnostics/sandbox-transport-controls.json", + "results/diagnostics/sandbox-transport-local-control.json", + "results/diagnostics/sandbox-transport-observations.json", + "results/diagnostics/sandbox-wheel-prefetch.json", + "results/diagnostics/statistics-proposal-check.json", + "results/flash-attention-prefill/result-2113.json", + "results/flashinfer-gdn/actual-2097.json", + "results/flashinfer-gdn/knowledge-2095.json", + "results/flashinfer-gdn/knowledge-2097.json", + "results/flashinfer-gdn/knowledge-preparation.json", + "results/flashinfer-small-m/actual-2076.json", + "results/flashinfer-small-m/knowledge-2076.json", + "results/flashinfer-small-m/operator-2076.json", + "results/flashinfer-small-m/status.json", + "results/formal-8k-baseline/independent-review.json", + "results/formal-8k-baseline/raw-audit.json", + "results/fp8-projections-search-v1/bound-result.json", + "results/fp8-projections-search-v1/knowledge.json", + "results/fp8-projections-search-v1/parent-review.json", + "results/fp8-projections-search-v1/receipt-review.json", + "results/full-context-graph/actual-2091.json", + "results/full-context-graph/knowledge-2091.json", + "results/full-context-graph/knowledge.json", + "results/merged-flash-prefill-v1/knowledge.json", + "results/merged-flash-prefill-v1/parent-review.json", + "results/merged-flash-prefill-v1/raw-audit.json", + "results/merged-flash-prefill-v1/receipt-review.json", + "results/profile-merged-v1/knowledge.json", + "results/profile-merged-v1/parent-review.json", + "results/profile-merged-v1/phase-summary.json", + "results/profile-merged-v1/receipt-review.json", + "results/profile-merged-v1/source-provenance.json", + "results/quality-executor/production-binding-review.json" + ], + "index_entries": 130, + "named_attempt_identities": 59, + "record_occurrences": 130, + "status_counts": { + "CANCELLED": 2, + "COMPLETED": 15, + "COMPLETED 0:0": 3, + "FAILED": 9, + "FAILED124:0": 1, + "INCONCLUSIVE": 1, + "PENDING": 1, + "completed": 2, + "failed": 1, + "recorded": 95 + }, + "unit": "published record/attempt observation; not a count of independent trials" + }, + "recomputed_metrics": { + "baseline-ttft_s": { + "source": { + "name": "baseline-ttft_s", + "path": "results/fp8-projections-search-v1/bound-result.json", + "pointer": "/rows", + "field": "ttft_s", + "where": { + "arm": "baseline" + }, + "unit": "s", + "scope": "Published paired 8K search: 10 inputs x 3 repetitions per arm; pooled descriptive distribution" + }, + "distribution": { + "count": 30, + "min": 1.6797754659783095, + "max": 1.6857226910069585, + "median": 1.6820332545321435, + "mean": 1.6824048391426913, + "stdev": 0.0015193378245608993 + } + }, + "baseline-tpot_s": { + "source": { + "name": "baseline-tpot_s", + "path": "results/fp8-projections-search-v1/bound-result.json", + "pointer": "/rows", + "field": "tpot_s", + "where": { + "arm": "baseline" + }, + "unit": "s", + "scope": "Published paired 8K search: 10 inputs x 3 repetitions per arm; pooled descriptive distribution" + }, + "distribution": { + "count": 30, + "min": 0.010484485398126292, + "max": 0.010825176634070517, + "median": 0.010567210277008566, + "mean": 0.010582620528395193, + "stdev": 8.29582774749074e-05 + } + }, + "candidate-ttft_s": { + "source": { + "name": "candidate-ttft_s", + "path": "results/fp8-projections-search-v1/bound-result.json", + "pointer": "/rows", + "field": "ttft_s", + "where": { + "arm": "candidate" + }, + "unit": "s", + "scope": "Published paired 8K search: 10 inputs x 3 repetitions per arm; pooled descriptive distribution" + }, + "distribution": { + "count": 30, + "min": 2.8576242129784077, + "max": 2.8680209269514307, + "median": 2.860997973533813, + "mean": 2.861812592072723, + "stdev": 0.0028047357042848713 + } + }, + "candidate-tpot_s": { + "source": { + "name": "candidate-tpot_s", + "path": "results/fp8-projections-search-v1/bound-result.json", + "pointer": "/rows", + "field": "tpot_s", + "where": { + "arm": "candidate" + }, + "unit": "s", + "scope": "Published paired 8K search: 10 inputs x 3 repetitions per arm; pooled descriptive distribution" + }, + "distribution": { + "count": 30, + "min": 0.012172944966781453, + "max": 0.012215168506571769, + "median": 0.012188331024207376, + "mean": 0.012192562011545538, + "stdev": 1.2489118313243294e-05 + } + } + }, + "recorded_results": [ + { + "source": { + "path": "results/fp8-projections-search-v1/bound-result.json", + "pointer": "/statistics" + }, + "value": { + "campaign_verdict": "INCONCLUSIVE", + "statistics_approved": true, + "scope": "approved_8k_statistics_only", + "metrics": [ + { + "input_tokens": 8192, + "metric": "ttft_s", + "paired_ratios_by_input": [ + [ + 1.702292798609853, + 1.699358675803024, + 1.700673009062922 + ], + [ + 1.700369306692157, + 1.6993543723400795, + 1.6992039596955852 + ], + [ + 1.7004501320617282, + 1.7012500612253891, + 1.702090905985738 + ], + [ + 1.7011941600862193, + 1.7020216040287146, + 1.7055641552219623 + ], + [ + 1.7004924174190703, + 1.6971446147289424, + 1.7009584254623584 + ], + [ + 1.6993359344791013, + 1.6974080411011256, + 1.7033882401028912 + ], + [ + 1.699884050963867, + 1.7005622041478314, + 1.7034518233235505 + ], + [ + 1.7047236362441305, + 1.7006056989758997, + 1.7026285019163367 + ], + [ + 1.701284401747634, + 1.6999993484514537, + 1.7037070916181098 + ], + [ + 1.700209019865402, + 1.7005199434298837, + 1.7006487486952797 + ] + ], + "median_paired_ratio": 1.7006272238355897, + "paired_ratio_p10": 1.6993227370007498, + "paired_ratio_p90": 1.7034773501530065, + "upper_one_sided_95": 1.7012392809169268, + "limit": 1.03, + "metric_verdict": "FAIL", + "arms": { + "baseline": { + "median": 1.6820332545321435, + "p10": 1.6811239291797393, + "p90": 1.6850990878767333 + }, + "candidate": { + "median": 2.860997973533813, + "p10": 2.8582675250363536, + "p90": 2.865111770189833 + } + } + }, + { + "input_tokens": 8192, + "metric": "tpot_s", + "paired_ratios_by_input": [ + [ + 1.1304008076084764, + 1.1648727498173903, + 1.162488459395516 + ], + [ + 1.1287640625520843, + 1.1634105104412242, + 1.1605667096849948 + ], + [ + 1.125864457635976, + 1.1579382003965255, + 1.155342654416893 + ], + [ + 1.1378043301027634, + 1.156582264747381, + 1.156308950668831 + ], + [ + 1.1533040277992193, + 1.1542772310517573, + 1.1540538590307976 + ], + [ + 1.1534842469281767, + 1.1554632826690245, + 1.1535526899173518 + ], + [ + 1.1523087598996873, + 1.1553701468490283, + 1.153805300531391 + ], + [ + 1.1536572914924696, + 1.154674614713675, + 1.1528816355366849 + ], + [ + 1.1523826191989737, + 1.1537480314300095, + 1.152395869923435 + ], + [ + 1.1528452230330744, + 1.1541411919392488, + 1.1533333392660574 + ] + ], + "median_paired_ratio": 1.1537766659807003, + "paired_ratio_p10": 1.1370639778533347, + "paired_ratio_p90": 1.1607588846560468, + "upper_one_sided_95": 1.154741923178071, + "limit": 1.03, + "metric_verdict": "FAIL", + "arms": { + "baseline": { + "median": 0.010567210277008566, + "p10": 0.01049987686839265, + "p90": 0.010708999877766459 + }, + "candidate": { + "median": 0.012188331024207376, + "p10": 0.012179919562867148, + "p90": 0.012213256699931646 + } + } + } + ], + "errors": [], + "bootstrap": { + "draws": 10000, + "seed": 670032, + "cluster": "input_id_all_three_repetitions", + "quantile": "linear", + "sample_indices_sha256": "f3dc88ecceb93a2bd68db9ada5acd635fec76425dcc68f7c5b0a2f8fa513ccce" + } + } + }, + { + "source": { + "path": "results/merged-flash-prefill-v1/parent-review.json" + }, + "value": { + "reviewer": "root", + "scope": "independent interpretation of delegated raw-receipt review and unchanged auditor replay", + "requests": 4, + "measured_requests": 2, + "ttft_ratio": 0.8105649764179438, + "ttft_reduction_percent": 18.94350235820562, + "tpot_ratio": 0.9980892217854955, + "output_token_ids_equal": false, + "first_different_output_token_zero_based": 5, + "full_quality_required": true, + "formal_statistics_eligible": false, + "verdict": "INCONCLUSIVE", + "receipt_sha256": "3c730644ac349ff858b10aa7ac10d621ff2eaa1cf71fbb175f98c6e08db6d3ca" + } + }, + { + "source": { + "path": "results/profile-merged-v1/phase-summary.json" + }, + "value": { + "source_trace_sha256": "1b8acff4a74a52f4583331f2eace080c2304701f86a88dd58697481ce05cb5c6", + "method": "GPU kernel start/end within CPU record_function boundaries; both phases end with CUDA synchronize; one device and stream verified. Device duration is instrumented and not a service latency claim.", + "phases": { + "profile_prefill_8192": { + "host_phase_us": 2918166.624, + "kernel_count": 3470, + "kernel_sum_us": 2861020.825, + "kernel_union_us": 2861020.813720703, + "first_to_last_kernel_us": 2905205.6950683594, + "memcpy_count": 116, + "memcpy_us": 26983.22, + "memset_count": 34, + "memset_us": 33.246 + }, + "profile_graph_decode_8": { + "host_phase_us": 103191.981, + "kernel_count": 17328, + "kernel_sum_us": 97940.197, + "kernel_union_us": 97775.2578125, + "first_to_last_kernel_us": 103063.44580078125, + "memcpy_count": 8, + "memcpy_us": 20.512, + "memset_count": 144, + "memset_us": 124.962 + } + }, + "decode_structure": { + "method": "Repeated graph-node ordering checked for all eight replays; each of 64 layer blocks anchored by hidden input RMSNorm [grid=1], then checked against model.py's 30-node GDN or 44-node full-attention sequence. Attribution to source phases is inference from these matched sequences, not NVTX module annotations.", + "graph_nodes_per_replay": 2163, + "non_graph_kernel_count": 24, + "non_graph_kernel_us": 82.534, + "groups": { + "fp8_mlp_gate_up": { + "kernel_count": 512, + "device_us": 24650.932, + "per_token_us": 3081.3665, + "share_of_all_decode_kernels_pct": 25.169371468591187 + }, + "fp8_mlp_down": { + "kernel_count": 512, + "device_us": 14005.476999999999, + "per_token_us": 1750.6846249999999, + "share_of_all_decode_kernels_pct": 14.300029435309385 + }, + "fp8_gdn_qkvz": { + "kernel_count": 384, + "device_us": 9370.256999999998, + "per_token_us": 1171.2821249999997, + "share_of_all_decode_kernels_pct": 9.56732504836599 + }, + "fp8_attention_output": { + "kernel_count": 512, + "device_us": 6104.394999999997, + "per_token_us": 763.0493749999996, + "share_of_all_decode_kernels_pct": 6.232777947138493 + }, + "full_scores_mask_softmax_value_bmm": { + "kernel_count": 1408, + "device_us": 5577.405, + "per_token_us": 697.175625, + "share_of_all_decode_kernels_pct": 5.694704698214973 + }, + "bf16_gdn_b_a_projection_and_splitK_reduce": { + "kernel_count": 1536, + "device_us": 4785.7339999999995, + "per_token_us": 598.2167499999999, + "share_of_all_decode_kernels_pct": 4.886383881788598 + }, + "bf16_lm_head": { + "kernel_count": 8, + "device_us": 4541.804, + "per_token_us": 567.7255, + "share_of_all_decode_kernels_pct": 4.637323733379871 + }, + "gdn_conv_window_copy_conv_silu": { + "kernel_count": 1920, + "device_us": 4021.128000000001, + "per_token_us": 502.64100000000013, + "share_of_all_decode_kernels_pct": 4.105697275654858 + }, + "hidden_input_rmsnorm": { + "kernel_count": 512, + "device_us": 3582.819, + "per_token_us": 447.852375, + "share_of_all_decode_kernels_pct": 3.658170097411587 + }, + "full_q_gate_layout_qk_norm_partial_rope": { + "kernel_count": 2048, + "device_us": 3496.767999999999, + "per_token_us": 437.0959999999999, + "share_of_all_decode_kernels_pct": 3.5703093388713514 + }, + "fp8_activation_quant": { + "kernel_count": 2048, + "device_us": 3167.651999999999, + "per_token_us": 395.9564999999999, + "share_of_all_decode_kernels_pct": 3.234271623938023 + }, + "fp8_full_qkv": { + "kernel_count": 128, + "device_us": 2896.261, + "per_token_us": 362.032625, + "share_of_all_decode_kernels_pct": 2.9571729368688118 + }, + "gdn_qk_repeat_Alog_cast_output_zero": { + "kernel_count": 1536, + "device_us": 2219.371, + "per_token_us": 277.421375, + "share_of_all_decode_kernels_pct": 2.2660471062764964 + }, + "post_attention_add_rmsnorm": { + "kernel_count": 512, + "device_us": 2050.9760000000006, + "per_token_us": 256.37200000000007, + "share_of_all_decode_kernels_pct": 2.0941105519728542 + }, + "flashinfer_gdn_recurrence": { + "kernel_count": 384, + "device_us": 1905.0459999999991, + "per_token_us": 238.1307499999999, + "share_of_all_decode_kernels_pct": 1.9451114642948892 + }, + "gdn_norm_silu_gate": { + "kernel_count": 1152, + "device_us": 1738.3990000000006, + "per_token_us": 217.29987500000007, + "share_of_all_decode_kernels_pct": 1.7749596725846901 + }, + "mlp_silu_multiply": { + "kernel_count": 1024, + "device_us": 1693.3380000000006, + "per_token_us": 211.66725000000008, + "share_of_all_decode_kernels_pct": 1.728950984241946 + }, + "mlp_output_residual_add": { + "kernel_count": 512, + "device_us": 780.4710000000003, + "per_token_us": 97.55887500000004, + "share_of_all_decode_kernels_pct": 0.7968852666285737 + }, + "full_kv_cache_index_copy": { + "kernel_count": 256, + "device_us": 651.3229999999999, + "per_token_us": 81.41537499999998, + "share_of_all_decode_kernels_pct": 0.6650211250851373 + }, + "full_sigmoid_output_gate": { + "kernel_count": 256, + "device_us": 379.388, + "per_token_us": 47.4235, + "share_of_all_decode_kernels_pct": 0.3873669970257462 + }, + "embedding_and_position_tables": { + "kernel_count": 136, + "device_us": 183.008, + "per_token_us": 22.876, + "share_of_all_decode_kernels_pct": 0.18685688369607834 + }, + "final_hidden_rmsnorm": { + "kernel_count": 8, + "device_us": 55.711, + "per_token_us": 6.963875, + "share_of_all_decode_kernels_pct": 0.056882670962975494 + } + }, + "per_replay_kernel_us": [ + 12233.914, + 12224.673, + 12236.986, + 12240.038, + 12234.154, + 12237.394, + 12227.691, + 12222.813 + ] + }, + "publication_note": "Compact derived copy; long per-kernel symbol lists omitted. Original trace and full report remain private; original trace digest retained. Source-region attribution is inference." + } + } + ], + "limitations": [ + "Campaign acceptance remains INCONCLUSIVE; this is a storage-only projection.", + "The index covers all published result JSON documents and their named/nested attempt observations, not unpublished controller history.", + "Repeated identical observations of one named attempt are merged with every source pointer retained. Changed observations and anonymous records remain distinct.", + "Record/index entry counts are not independent trial counts. Text-only history and per-request traces remain in the byte-preserved original archive.", + "Only declared sample distributions are recomputed. Other retained results are explicitly reported frozen values, not independently requalified quality or statistics.", + "Original private models, media, raw receipts and runtime records were not published; archiving does not restore them or complete outstanding acceptance." + ] +} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/results/trials.jsonl b/evidence/qwen38-h20-fp8-single-request-vllm032/results/trials.jsonl new file mode 100644 index 000000000..b662af5d5 --- /dev/null +++ b/evidence/qwen38-h20-fp8-single-request-vllm032/results/trials.jsonl @@ -0,0 +1,130 @@ +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"{\"scope\":\"controlled_elf_loader_context_diagnostic\",\"verdict\":\"INCONCLUSIVE\"}","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"49e27df252e6e8f0ab73199f92a9aa9fd8e658924b721ff48766d390564e0c25"},"id":"002c565d0e190a2ad72980f1bf91012f133547fd706aba773cfcf2db24b582f3","identity":null,"kind":"published_record","source_record":"qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-elf-context-v1-observations.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-elf-context-v1-observations.json#"],"status":"recorded"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"INCONCLUSIVE: no standalone verdict recorded","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"ad8ce617e4fe7438adb52d94ab84ac6cc82c264b9dd3ed77d6a344d8b2939d5a"},"id":"008ec228782709f43b83877edf606fed092b50f09554d42b6305ced849fb835c","identity":"job_id:2003","kind":"attempt_observation","source_record":"qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-elf-dependency-2003.json#/terminal","source_records":["qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-elf-dependency-2003.json#/terminal"],"status":"recorded"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"INCONCLUSIVE: no standalone verdict recorded","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"16d33d5ae85696343aa3a4a50ea5bd3dd87c63156f0d5a2aea0594425bf64057"},"id":"00d037cedf9ac2227cd62ebef24a35a6277bcc460e5526ebaabe9d902239010d","identity":"job_id:2012","kind":"attempt_observation","source_record":"qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-bounds-2012.json#/terminal","source_records":["qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-bounds-2012.json#/terminal"],"status":"recorded"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"{\"scope\":\"selected_bubblewrap_version_help_and_bytes\",\"verdict\":\"INCONCLUSIVE\"}","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"fe6db0d2d88e2819d3b46eb8515c31170b82c1cf6ac6b115dff1d76d6495d6ff"},"id":"01560119fda203683ffab3eaf9181b8cae0884d5f64a69c75a53b24a4415241c","identity":"job:1999","kind":"attempt_observation","source_record":"qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-bwrap-capability-1999.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-bwrap-capability-1999.json#"],"status":"COMPLETED"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"{\"scope\":\"prospective_natural_eos_validator_CPU_checks\",\"verdict\":\"INCONCLUSIVE\"}","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"3f22714e62005999141e6977433ab93d8e221138767578cb9367fdadec17343c"},"id":"0169bb782504d268211a20f9dde81a9567db40b9c2215c7b114ef4f3c6cd1d6f","identity":null,"kind":"published_record","source_record":"qwen38-h20-fp8-single-request-vllm032/results/diagnostics/quality-eos-v5-check.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/diagnostics/quality-eos-v5-check.json#"],"status":"recorded"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"INCONCLUSIVE: no standalone verdict recorded","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"51d088714d1061aab7eff10bbc0463c5da7d7d5cb1611223f5d6b19792f76931"},"id":"02bbb84928780e8b240a0d95fb92d973f026fde9f2cc8bbc0a250ee55998df87","identity":null,"kind":"published_record","source_record":"qwen38-h20-fp8-single-request-vllm032/results/full-context-graph/knowledge.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/full-context-graph/knowledge.json#"],"status":"recorded"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"INCONCLUSIVE: no standalone verdict recorded","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"3d5a128ecb5eca5307c74d99d4cc35351bf96b72b01bc26a7f9fb1af109fb0ba"},"id":"03c17e1fdcad452b3fbf44bb5eccfd7b38f7836b04c0ba7e347902aa0869a6bf","identity":null,"kind":"published_record","source_record":"qwen38-h20-fp8-single-request-vllm032/results/full-context-graph/knowledge-2091.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/full-context-graph/knowledge-2091.json#"],"status":"recorded"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"INCONCLUSIVE: no standalone verdict recorded","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"7eb064eb433670596dba389a9e9f94e63fe92d0ed77d8791f022b89490cf41c8"},"id":"092f9aee5f58ffaa72e907f28eeafaca93b0617de31c7ccb834f337f81fdd17a","identity":null,"kind":"published_record","source_record":"qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-transport-controls.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-transport-controls.json#"],"status":"recorded"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"{\"scope\":\"reviewed_dependency_wheel_bytes\",\"verdict\":\"INCONCLUSIVE\"}","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"677bc3c2f62f6b94ccadbc91e44aba28dc08484310904184cbff8a944da7a37e"},"id":"0c787bb7fed76034ee87364b78fee972add920999af972532f29f28104a21960","identity":"job:1986","kind":"attempt_observation","source_record":"qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-build-1986.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-build-1986.json#"],"status":"COMPLETED"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"{\"scope\":\"compute_platform_and_filesystem_readiness\",\"verdict\":\"INCONCLUSIVE\"}","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"f54e81fb1bea37cadaeb5ca2f1d7b1ec449d875d51edd79a87d82c68a3472767"},"id":"0d28a05afd9ceff8a6f8c003467c7905e6823ea7932d0579dcac25b300371ae4","identity":"job_id:1997","kind":"attempt_observation","source_record":"qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-platform-fs-observations.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-platform-fs-observations.json#"],"status":"recorded"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"{\"scope\":\"first_missing_elf_inventory_diagnostic\",\"verdict\":\"INCONCLUSIVE\"}","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"b83ff014c628e8aaa4dd92993513127ae543d8b29b4856fed9c84ca8cae95bec"},"id":"0df2a53fa2e9d30d4afc705581a08affcad73cea9b9f5061260e7efb8c03fdd2","identity":null,"kind":"published_record","source_record":"qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-elf-dependency-observations.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-elf-dependency-observations.json#"],"status":"recorded"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"{\"scope\":\"trusted_startup_resource_and_cleanup_readiness\",\"verdict\":\"INCONCLUSIVE\"}","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"62cf6c77c53e29761913116541e29de5d518a93a6f3a3ded8e877ea3b3f24422"},"id":"0df5054cb6c39d8b63420f9d7058264a09696304c600536f0c5f4aecdb1491b4","identity":"job:2014","kind":"attempt_observation","source_record":"qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-bounds-2014.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-bounds-2014.json#"],"status":"COMPLETED"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"{\"scope\":\"synthetic_proposal_checks_only\",\"verdict\":\"INCONCLUSIVE\"}","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"260613e691050124cd13255904d715a92ce6652b3ff7f8ea3786d9f0cba93b8f"},"id":"0f0101ed0a572b8d25d17a9f9930a936e3f82f1f3f4c269aec938a3c7b9e98df","identity":null,"kind":"published_record","source_record":"qwen38-h20-fp8-single-request-vllm032/results/diagnostics/statistics-proposal-check.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/diagnostics/statistics-proposal-check.json#"],"status":"recorded"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"{\"scope\":\"prospective_quality_transport_fix_only\",\"verdict\":\"INCONCLUSIVE\"}","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"524df18888a2d086a9e8ba3d9dac9f17434d23d01559c34d0f68182aea887456"},"id":"11970955a4060b43b8acca59307a4b9345d212f6015ff3a9a54313d47aeaef13","identity":null,"kind":"published_record","source_record":"qwen38-h20-fp8-single-request-vllm032/results/diagnostics/quality-null-echo-check.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/diagnostics/quality-null-echo-check.json#"],"status":"recorded"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"INCONCLUSIVE: no standalone verdict recorded","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"268a04f11fa8faa8153e64a04161be2cfc88670588b92e282de8a953e388c305"},"id":"13f82e7f425847a1a6ccc991bd9dc14551299b5f3aed42e14e6ef568c9b670bc","identity":null,"kind":"published_record","source_record":"qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-benign-v2-probe-stdout.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-benign-v2-probe-stdout.json#"],"status":"recorded"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"{\"scope\":\"four-handwritten-fixtures-readiness\",\"verdict\":\"INCONCLUSIVE\"}","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"d3677729732cc84c5656c14db3fd23441865ae1534ed62494b88cd265e37e8da"},"id":"15b8c7a4ec2a31a85bff970529a705eadcb631dd28a28da5e64a49c9e8446ae9","identity":null,"kind":"published_record","source_record":"qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-executor-v1-execution-record.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-executor-v1-execution-record.json#"],"status":"recorded"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"{\"scope\":\"single-case engineering HTTP comparison only\",\"verdict\":\"INCONCLUSIVE\"}","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"8c287b706b83bb12a232ee7950fab4418dc0fc6675e3a67917f148c75064d286"},"id":"1c88cc3bdc4f974c6de4581ce649575aed07221caac369292a6e3eebbb4045f3","identity":null,"kind":"published_record","source_record":"qwen38-h20-fp8-single-request-vllm032/results/merged-flash-prefill-v1/raw-audit.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/merged-flash-prefill-v1/raw-audit.json#"],"status":"recorded"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"{\"scope\":\"external_export_evaluator_trusted_readiness\",\"verdict\":\"INCONCLUSIVE\"}","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"43e05470cccccb908abdebcb9a4de9dae0a474c18f6d4f630cf9f4180a7d53c3"},"id":"1d42fd8bc1420f412cb9e05603cc31537b3709007cbe5b64fcd5c9495eb6f409","identity":"job:2018","kind":"attempt_observation","source_record":"qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-executor-2018.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-executor-2018.json#"],"status":"COMPLETED"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"INCONCLUSIVE: no standalone verdict recorded","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"6289e39d2440d46f5a62e6d0136f07c3374e52de153580d9fed8b80fc6deb9d3"},"id":"1e22a7b77cd969e73de1b893e30cb9259e64fad75894b136686834ed2c39ea6b","identity":null,"kind":"published_record","source_record":"qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-bounds-v2-serialized-evaluator-results.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-bounds-v2-serialized-evaluator-results.json#"],"status":"recorded"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"INCONCLUSIVE: no standalone verdict recorded","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"b46984ad72074538eb5d7e5512f6f67d8c803b6ccce31ca36248153b9561f253"},"id":"201915cd9109d5838cc6bd3376863156b9db661094c8073aaaa4f1883dc3375d","identity":null,"kind":"published_record","source_record":"qwen38-h20-fp8-single-request-vllm032/results/flashinfer-gdn/knowledge-2097.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/flashinfer-gdn/knowledge-2097.json#"],"status":"recorded"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"{\"scope\":\"full164_canonical_resource_pilot\",\"verdict\":\"INCONCLUSIVE\"}","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"934aad1fe22e081f19f9b2bcc3dbbd810b3c47622e77936d4fcf867c0eceaf7f"},"id":"22704a56f5f65ae35d9bf156f91983af2c7129720f05d0a86387f3a869fad6a9","identity":"job:2024","kind":"attempt_observation","source_record":"qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-full164-2024.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-full164-2024.json#"],"status":"FAILED"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"{\"scope\":\"local_read_only_network_control\"}","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"f7273c32939fdb845428bf756265dcfdc374b9f269d3ee5acbb36679911b9327"},"id":"23409f370a8a2691fdb1b0e9d8fac28aedb7592f271c80fc266437a14b2d46c6","identity":null,"kind":"published_record","source_record":"qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-transport-local-control.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-transport-local-control.json#"],"status":"recorded"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"{\"scope\":\"handwritten_synthetic_scoring_fixture\",\"verdict\":\"INCONCLUSIVE\"}","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"7c1be3ca0a0ad7c0a439311d335f6c1a94126e84b125ad57e0e70c092cb1318a"},"id":"23eb4a2c7edffe85833cfe8b8b3917cd0b4b6b7d9580707bbcf343ffceacbbed","identity":null,"kind":"published_record","source_record":"qwen38-h20-fp8-single-request-vllm032/results/diagnostics/quality-score-v5-controls.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/diagnostics/quality-score-v5-controls.json#"],"status":"recorded"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"INCONCLUSIVE: no standalone verdict recorded","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"871191ce046daadcb96a2a46c86bd440d60bcacf5092bceb47ee76864487a0ea"},"id":"246dc69d20b4c5383713402fc6b875cd134fbb7507230906c265e1c895dcca28","identity":"job:2041","kind":"attempt_observation","source_record":"qwen38-h20-fp8-single-request-vllm032/results/flashinfer-small-m/status.json#/gpu_attempts/2","source_records":["qwen38-h20-fp8-single-request-vllm032/results/flashinfer-small-m/status.json#/gpu_attempts/2"],"status":"PENDING"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"INCONCLUSIVE: no standalone verdict recorded","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"a7b7269ea413880a3b8c08cfaab1e25cc775dc7b0aaaf6bee94f99cbd6bfad1b"},"id":"27c2ce6a674aff8f0811a9c7c32d375ffe8af42194c44cd932070711dfb3df06","identity":"job:2038","kind":"attempt_observation","source_record":"qwen38-h20-fp8-single-request-vllm032/results/flashinfer-small-m/status.json#/gpu_attempts/0","source_records":["qwen38-h20-fp8-single-request-vllm032/results/flashinfer-small-m/status.json#/gpu_attempts/0"],"status":"CANCELLED"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"INCONCLUSIVE: no standalone verdict recorded","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"7730720fa86629606d65d4bdfff7a9c3cd6f109155395e03caba22a996cd69dd"},"id":"2ba5cef3d07a7c6f032a77632124711c37bbeb8489e5b8fb305359f25355ece1","identity":null,"kind":"published_record","source_record":"qwen38-h20-fp8-single-request-vllm032/results/profile-merged-v1/phase-summary.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/profile-merged-v1/phase-summary.json#"],"status":"recorded"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"{\"campaign_verdict\":\"INCONCLUSIVE\"}","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"97203987f555afff973e97400aa02ccada669bd583b0c90e3404d970ffee73b9"},"id":"2bd30b27162ac41b4eb63c988bd5fb335679031638e707ba33c679af4cd4fbb7","identity":null,"kind":"published_record","source_record":"qwen38-h20-fp8-single-request-vllm032/results/diagnostics/demotion-preflight.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/diagnostics/demotion-preflight.json#"],"status":"recorded"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"{\"verdict\":\"INCONCLUSIVE\"}","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"d8f877af2425a84b396be0b4935a4b7cbe5c8b4275efcdcdf797e8aba677a0c6"},"id":"2d0d829e6232720d9857c513eb89cd13fd069c24b4afdd50439cca42acd44235","identity":null,"kind":"published_record","source_record":"qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-benign-v2-probe-execution.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-benign-v2-probe-execution.json#"],"status":"recorded"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"INCONCLUSIVE: no standalone verdict recorded","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"ede83d3d12dac3d587dad805e7b0ed5556c3e0197b71aa412ed7616c0cc135a2"},"id":"2d180b75a32a207b283236d77f7277731701279d37dbfca9b613573638406745","identity":null,"kind":"published_record","source_record":"qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-bounds-v2-bounds-stdout.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-bounds-v2-bounds-stdout.json#"],"status":"recorded"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"{\"campaign_verdict\":\"INCONCLUSIVE\"}","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"2326287308e3c7c84259693e01b2e06ef6e2decc2abf91693374d4da63467ce8"},"id":"315f4f0da9e5bdb9f96d1eaab6cb95d24692c13d34da2c392a684022d9fe3f1c","identity":"job_id:2163","kind":"attempt_observation","source_record":"qwen38-h20-fp8-single-request-vllm032/results/fp8-projections-search-v1/receipt-review.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/fp8-projections-search-v1/receipt-review.json#"],"status":"COMPLETED"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"{\"scope\":\"sandbox_dependency_resolution_readiness\",\"verdict\":\"INCONCLUSIVE\"}","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"691f1c9f998679a779fc59689e9388e875081f464d5e777bd76a2ddc952b6d7d"},"id":"345e0da900b32fa884dd9360a06b4fccf2a17eb9c2e99dde0678b34c4aa2cf3c","identity":"job:1985","kind":"attempt_observation","source_record":"qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-resolve-1985.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-resolve-1985.json#"],"status":"FAILED"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"{\"scope\":\"local_authenticated_wheel_prefetch_only\"}","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"1b974b1130a2b0094847aa4ee8da39a9cf79f84279573794fe75f1791569da8e"},"id":"34b44f792097a24f1729b10152431c797e9a9869851fe4567018890cefc39cbd","identity":null,"kind":"published_record","source_record":"qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-wheel-prefetch.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-wheel-prefetch.json#"],"status":"recorded"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"{\"campaign_verdict\":\"INCONCLUSIVE\",\"scope\":\"authenticated 8K paired performance only\"}","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"05e1c50d88b032357eb8b23284737db6abf6c42db95590279b3d5f3317853b33"},"id":"36b853e9027f80e4a2ae662084d3fa0345fb3e134c6bdd5ffd8ae9ed09cdebcb","identity":null,"kind":"published_record","source_record":"qwen38-h20-fp8-single-request-vllm032/results/fp8-projections-search-v1/bound-result.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/fp8-projections-search-v1/bound-result.json#"],"status":"COMPLETED"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"INCONCLUSIVE: no standalone verdict recorded","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"89dea26133879e1652e0bb4d1dc3e169e4ed8ceb77c46cc52ae603854aefaee4"},"id":"38bfbad15932d25301554db047b50dbea8543c9514a02182659cb6f4b002904c","identity":null,"kind":"published_record","source_record":"qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-offline-prefetch-controls.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-offline-prefetch-controls.json#"],"status":"recorded"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"{\"scope\":\"single-case engineering diagnostic, not formal acceptance\",\"verdict\":\"INCONCLUSIVE\"}","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"bf2d4b66622323d188838bab5152f935bdaa25e8d8babb7045cd7849ebe93cd8"},"id":"3a40ad67225b167b9f03268822980a5c0f7d31778a53624ee67fe3ea0f3c466f","identity":"job:2084","kind":"attempt_observation","source_record":"qwen38-h20-fp8-single-request-vllm032/results/deepgemm-prefill/actual-2084.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/deepgemm-prefill/actual-2084.json#"],"status":"COMPLETED 0:0"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"{\"campaign_verdict\":\"INCONCLUSIVE\",\"scope\":\"independent review of delegated receipt collection plus direct recomputation of frozen statistics\"}","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"3ef8001016c3a86b25c6b11adc25dca6c0ed01374451cfa28467fe137894e106"},"id":"3b35a1bd697e65d53921e9d651906ee6ecef5f257a37c2127e5cc65bb156350d","identity":null,"kind":"published_record","source_record":"qwen38-h20-fp8-single-request-vllm032/results/fp8-projections-search-v1/parent-review.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/fp8-projections-search-v1/parent-review.json#"],"status":"recorded"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"{\"verdict\":\"INCONCLUSIVE\"}","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"c5dec1c358198c55b47b9235c0c687aa69599b8da380a41ac6702c8ec8aa50da"},"id":"3be282b0f8c29c2c01ca3494193f5cdd1b9bf7687129707c2a05dfcfad1372b0","identity":null,"kind":"published_record","source_record":"qwen38-h20-fp8-single-request-vllm032/results/profile-merged-v1/parent-review.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/profile-merged-v1/parent-review.json#"],"status":"recorded"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"INCONCLUSIVE: no standalone verdict recorded","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"bd612d4051172cd903bdd6e8a2b14f4cd33e7ccdc17fea79d0853ea291757599"},"id":"3dc36b128240787ea91d57c7c3bebd0915dfc58da2666e242139cc02a274b261","identity":null,"kind":"published_record","source_record":"qwen38-h20-fp8-single-request-vllm032/results/fp8-projections-search-v1/knowledge.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/fp8-projections-search-v1/knowledge.json#"],"status":"recorded"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"INCONCLUSIVE: no standalone verdict recorded","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"bf1a00b2085c48b8bbc42fc98592c953ff72c645b0ba6bd21410431b15741e63"},"id":"3e973e88f27101f849a00f87e686d71098727bde7c30007deb97b03783601f5c","identity":null,"kind":"published_record","source_record":"qwen38-h20-fp8-single-request-vllm032/results/deepgemm-prefill/extraction-metadata.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/deepgemm-prefill/extraction-metadata.json#"],"status":"recorded"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"{\"scope\":\"local_offline_wheel_prefetch\",\"verdict\":\"INCONCLUSIVE\"}","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"ca6197c54183630f73f0ec72de63bb41beebbda0c0ada8e74f3d5971c342489f"},"id":"4154a3e26370cc775d5253fbcba0248ce143311ecac53f614056a27671b58475","identity":null,"kind":"published_record","source_record":"qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-offline-prefetch-wheel-records.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-offline-prefetch-wheel-records.json#"],"status":"recorded"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"INCONCLUSIVE: no standalone verdict recorded","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"ede83d3d12dac3d587dad805e7b0ed5556c3e0197b71aa412ed7616c0cc135a2"},"id":"42f72f7a18db3586e4bd34f712952cbd62dd017a4f882702c5d5d4f9029aadfc","identity":null,"kind":"published_record","source_record":"qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-bounds-v1-late-bounds-stdout.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-bounds-v1-late-bounds-stdout.json#"],"status":"recorded"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"INCONCLUSIVE: no standalone verdict recorded","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"c6121f98d118274f986773c719d565653c03e765b7fc861f84807728e3a59687"},"id":"4470f92108bb7fbc7d940c0d83e035d85d2b44e9903930cff7dbe363ca6c4c32","identity":"job:1952","kind":"attempt_observation","source_record":"qwen38-h20-fp8-single-request-vllm032/results/diagnostics/demotion-preflight.json#/attempts/0","source_records":["qwen38-h20-fp8-single-request-vllm032/results/diagnostics/demotion-preflight.json#/attempts/0"],"status":"recorded"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"{\"scope\":\"local_offline_wheel_prefetch\",\"verdict\":\"INCONCLUSIVE\"}","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"9e05c0cb839b62a6b1eb548a1a27343825e743b0f3f4abbe02bd45fb787d0885"},"id":"448b5d97f0922b51b72d5438c901d9784c19f9f7a2bf48e043489165bc8bdc22","identity":null,"kind":"published_record","source_record":"qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-offline-prefetch-report.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-offline-prefetch-report.json#"],"status":"recorded"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"INCONCLUSIVE: no standalone verdict recorded","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"eb8ff8fddb5203ccc552aa6022d9d7144bf442244dfc24df60776aaac6726c22"},"id":"453d92bdd9040f0a0c633cc1d52bcaf58f52fb9fd5088e64a8de0dc9d662a558","identity":"job_id:2006","kind":"attempt_observation","source_record":"qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-benign-retry-2006.json#/terminal","source_records":["qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-benign-retry-2006.json#/terminal"],"status":"recorded"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"INCONCLUSIVE: no standalone verdict recorded","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"6d748ad850aec76d50d7a3a8f040f98771eeceee10b2fb40f0c66cf833eb3ce5"},"id":"46105ea1232360dac42de9ac88df449f0d1c391821ede7b881b15842e5906ca7","identity":null,"kind":"published_record","source_record":"qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-full164-v1-actual-review.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-full164-v1-actual-review.json#"],"status":"recorded"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"{\"scope\":\"independent interpretation of delegated raw-receipt review and unchanged auditor replay\",\"verdict\":\"INCONCLUSIVE\"}","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"dff2d1e074aeef8d620f4c029b8db88afa2e4ec5d93ee551d6298e815e0a79dc"},"id":"4a787c85186060f204ac1705fa0c6cd1ea0607dab4d278449629aa8c7065a5c5","identity":null,"kind":"published_record","source_record":"qwen38-h20-fp8-single-request-vllm032/results/merged-flash-prefill-v1/parent-review.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/merged-flash-prefill-v1/parent-review.json#"],"status":"recorded"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"INCONCLUSIVE: no standalone verdict recorded","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"636e530a541c92fdc468b6ada0315ac183cd7fadfdecd056df7462c5968c4b67"},"id":"536781e91aae6ae643952760d3984c892834fc1aeedb74e4f8c1c935e2482d86","identity":"job_id:2014","kind":"attempt_observation","source_record":"qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-bounds-2014.json#/terminal","source_records":["qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-bounds-2014.json#/terminal"],"status":"recorded"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"INCONCLUSIVE: no standalone verdict recorded","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"903d6357f742513d081b91c6e44a186da240e244730eb6a32c6ea55b2f7313a7"},"id":"5a440f3ceb5c8b9efeadac544b437d27cd748f495604c5916a86ae65629a07de","identity":null,"kind":"published_record","source_record":"qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-synthetic-v3-probe-stdout.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-synthetic-v3-probe-stdout.json#"],"status":"recorded"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"INCONCLUSIVE: no standalone verdict recorded","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"448f232e4438b01815c74f2de4ad6041c7b7a23fe9450232f3bde5eeb6a28fc1"},"id":"5d493c73d3b3d2307f637a598e1f98e54b64fd6aa0af551e233b2b6fbb37555e","identity":"job:2037","kind":"attempt_observation","source_record":"qwen38-h20-fp8-single-request-vllm032/results/flashinfer-small-m/status.json#/cpu","source_records":["qwen38-h20-fp8-single-request-vllm032/results/flashinfer-small-m/status.json#/cpu"],"status":"COMPLETED"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"{\"scope\":\"exact_env_i_policy_with_observation_shell\"}","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"4a056f20e064364de0b49f68e740f8b0255735d0aa05f01a93d6e9425d6a36fa"},"id":"5e30fe060bd680dcb0c086f309cca9edf7620472cf47fabbd61709bce9b495ee","identity":null,"kind":"published_record","source_record":"qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-bwrap-build-make-control.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-bwrap-build-make-control.json#"],"status":"recorded"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"{\"campaign_verdict\":\"INCONCLUSIVE\",\"scope\":\"Three natural-EOS Graph requests healthy; fourth ignore-EOS request errors after876 output tokens at1024 capacity. Eager arm not run; comparison incomplete. No performance or quality verdict.\"}","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"240f80571d04a13658b01fdb5ba27153588853aca9e162b443763c2f7143e3a0"},"id":"5e47dd1e00d37ec0d9533243f782c7049fc787b15f49c0e49b602b3e1e135ad1","identity":"job:1951","kind":"attempt_observation","source_record":"qwen38-h20-fp8-single-request-vllm032/results/diagnostics/graph-long-1951.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/diagnostics/graph-long-1951.json#"],"status":"FAILED"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"{\"scope\":\"CPU synthetic protocol, not model performance acceptance\"}","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"abbb12f40aaa737c9c08a49e5a623066cfc8b946c3e3e8fd64d4346a9a8c5809"},"id":"5ebfb03daa76d094be05b6e19ad7318e83e97e9a4a1df0d64ea495f5774b4dea","identity":null,"kind":"published_record","source_record":"qwen38-h20-fp8-single-request-vllm032/results/diagnostics/client-repair-1950.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/diagnostics/client-repair-1950.json#"],"status":"recorded"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"INCONCLUSIVE: no standalone verdict recorded","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"ae6b839d4053a8ac4c861cc0fb26cf9d74bb050075ce4cdb6b893ab85b4152ee"},"id":"5f0166d2b2a9f33b06ad33249ed21767cc8f02cf57a040d02b6010cb796b231e","identity":"job_id:2005","kind":"attempt_observation","source_record":"qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-elf-context-2005.json#/terminal","source_records":["qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-elf-context-2005.json#/terminal"],"status":"recorded"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"{\"scope\":\"installed_runtime_inventory_before_trusted_benign_probe\",\"verdict\":\"INCONCLUSIVE\"}","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"ed9083bca0270500326a9aa0fb74e1251350e3ee5fb0f9381b16c908a6c33a91"},"id":"6133ef6b0915b5b02045a3859fc497955c05fea32287d19572c67b0bf6aab3c3","identity":"job:2002","kind":"attempt_observation","source_record":"qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-benign-retry-2002.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-benign-retry-2002.json#"],"status":"FAILED"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"{\"scope\":\"full164-canonical-resource-pilot\",\"verdict\":\"INCONCLUSIVE\"}","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"13d112d80b458a2c0ff94e16cb4e9591cad5e4743ad1edc1bf8ed74b451e534e"},"id":"62a6d28b306806b66d2a82a5d8cbce4fe2d4e606a013db9e7d58d0496c9eb4d9","identity":null,"kind":"published_record","source_record":"qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-full164-v1-summary.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-full164-v1-summary.json#"],"status":"recorded"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"INCONCLUSIVE: no standalone verdict recorded","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"0c63892a4f72e7b40292484d01cf455ca554b62115454e17eafa235baf7c4b9b"},"id":"63d8273b95e72f7f8148ed8b89b4087a4ec13aae9e9d5d789045a2b190fdf222","identity":"job_id:2004","kind":"attempt_observation","source_record":"qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-elf-context-2004.json#/terminal","source_records":["qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-elf-context-2004.json#/terminal"],"status":"recorded"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"{\"scope\":\"fabricated_provenance_fixtures_only\",\"verdict\":\"INCONCLUSIVE\"}","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"8e2fa8789304bfbee9a39bb8d8365c6013e2f6c6a3dbf9937b5213892584d513"},"id":"6440322bf74d4f9410e255ecd7ff281042828d66811b27e7b18359aeea89ee2d","identity":null,"kind":"published_record","source_record":"qwen38-h20-fp8-single-request-vllm032/results/diagnostics/quality-binding-v5-controls.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/diagnostics/quality-binding-v5-controls.json#"],"status":"recorded"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"{\"scope\":\"CPU_only_transport_preflight\"}","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"72fd44faac811bd86fff6bb84b5f017b2e95d6f38cc32026ac04c7c28f7b95a8"},"id":"671d7d8658ec96aaca004a00a48ccd7ae2b2ee2c141744dc9612298aa3d42054","identity":"job:1960","kind":"attempt_observation","source_record":"qwen38-h20-fp8-single-request-vllm032/results/diagnostics/ipc-preflight-1960.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/diagnostics/ipc-preflight-1960.json#"],"status":"recorded"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"{\"scope\":\"task_local_bubblewrap_build_dependency_preflight\",\"verdict\":\"INCONCLUSIVE\"}","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"363ff188c3e42161fd7b7644f33d2b21c784fbe0c0b4626d0f49a0c1c8d2ebcb"},"id":"677989e86a768fbe37abfa9f690462f03c2029843649635af0e80677a386ee91","identity":"job:2000","kind":"attempt_observation","source_record":"qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-bwrap-build-2000.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-bwrap-build-2000.json#"],"status":"FAILED"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"INCONCLUSIVE: no standalone verdict recorded","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"d5b5f910dbb63faf3a0c8c0356cd1f9c80107f5dc6b7dd187ed8dfb9eb2482d2"},"id":"6897f042afc87443b62f9bedc30885b7e76e14b5385ba8a4f2f043b2540787a6","identity":"job_id:2014","kind":"attempt_observation","source_record":"qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-bounds-2014.json#/actual_pidfd_capability","source_records":["qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-bounds-2014.json#/actual_pidfd_capability"],"status":"recorded"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"{\"scope\":\"single fixed input descriptive engineering E2E; not model quality or vLLM parity\",\"verdict\":\"INCONCLUSIVE\"}","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"6f3981d9a7f6d378f12acd0facaabb0b5bca8b75a7216d2f8f76b5f0b35ac0db"},"id":"69743c68cccb9b51957c0bf8f1b8887d22a7be1ae702e83182609198d724bd8f","identity":"job:2076","kind":"attempt_observation","source_record":"qwen38-h20-fp8-single-request-vllm032/results/flashinfer-small-m/actual-2076.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/flashinfer-small-m/actual-2076.json#"],"status":"recorded"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"{\"campaign_verdict\":\"INCONCLUSIVE\"}","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"c369cd87a27f47c8796f5cca24317450b879fb430274a2553c3df2be84962aae"},"id":"6aba2d432a7289117702f79d0856f3f555606b9960b5d6907c01526907af059d","identity":"job_id:2169","kind":"attempt_observation","source_record":"qwen38-h20-fp8-single-request-vllm032/results/profile-merged-v1/receipt-review.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/profile-merged-v1/receipt-review.json#"],"status":"recorded"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"INCONCLUSIVE: no standalone verdict recorded","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"c390176c402e0102e9f833a3598acd5ec61470d134d06327a466105bc5b71612"},"id":"6c9854984aba9978b0083b937f2b0e034dd820dbb692086b1ed07e58a8b57b74","identity":null,"kind":"published_record","source_record":"qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-bounds-v2-actual-review.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-bounds-v2-actual-review.json#"],"status":"recorded"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"{\"verdict\":\"INCONCLUSIVE\"}","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"efbe495b9a01e9e8c47a7f8feeb6da470e3e8186c481f9081b2129268376ff49"},"id":"71f86058ef2cc0c77595daab919ad4c3c98d59c6626cf7bfd5b487bc6b00d2ff","identity":"job:1943","kind":"attempt_observation","source_record":"qwen38-h20-fp8-single-request-vllm032/results/diagnostics/http-summary.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/diagnostics/http-summary.json#"],"status":"recorded"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"{\"scope\":\"trusted readiness only; qualified false, protocol null, campaign INCONCLUSIVE\"}","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"fdff4b27385b30e8df75ea9a054dd0744d828f4ef8f23579e7e67a647b2d52cb"},"id":"73bdd7c205d0daa0266a2670255c0cf19e3c0f9326014a5fb9ef83a976cb80f1","identity":null,"kind":"published_record","source_record":"qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-executor-v1-actual-review.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-executor-v1-actual-review.json#"],"status":"recorded"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"{\"scope\":\"trusted_kernel_boundary_and_cleanup_probes\",\"verdict\":\"INCONCLUSIVE\"}","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"49b90b390e9f13e207be6d63cc1e287cf71720f1baaaa59a05b5a8c627804634"},"id":"75928980299efa52eb922f0a0eb659cb1d477bb4cd379db4d6757428362f2b8e","identity":null,"kind":"published_record","source_record":"qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-bounds-v1-summary.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-bounds-v1-summary.json#"],"status":"recorded"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"{\"scope\":\"task_local_static_libcap_and_bubblewrap_build\",\"verdict\":\"INCONCLUSIVE\"}","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"f0f97fb436fd4d6df181239cb5e60c9497b1f085b29e3b7b7e507452dee6f007"},"id":"77f14db4c813c07ba6f7f9bfaa97b827bde1d10b800ba78c05b2a42c43dfbe70","identity":"job:2001","kind":"attempt_observation","source_record":"qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-bwrap-build-2001.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-bwrap-build-2001.json#"],"status":"COMPLETED"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"{\"scope\":\"diagnostic_lexical_alias_failure\",\"verdict\":\"INCONCLUSIVE\"}","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"00136e639794e0b764c80eaa409952973151ce2f1fdab90c19f210428872297f"},"id":"7c41ddd82a321671553f3eaea3244b1d8f955fda645ab3dae69172060143d8b0","identity":"job:2004","kind":"attempt_observation","source_record":"qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-elf-context-2004.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-elf-context-2004.json#"],"status":"FAILED"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"{\"scope\":\"CPU synthetic protocol, not model performance acceptance\"}","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"8883a584f58d558ffaf36f4c00b8a972e3fb79b707c5c3a1e5a63b234221f967"},"id":"7e3de8a0e625cced21ba2932348c449fdd477cc0b0eb5006ed0d757e990fb12a","identity":null,"kind":"published_record","source_record":"qwen38-h20-fp8-single-request-vllm032/results/diagnostics/client-base-1950.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/diagnostics/client-base-1950.json#"],"status":"recorded"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"{\"scope\":\"trusted_resource_cleanup_readiness\",\"verdict\":\"INCONCLUSIVE\"}","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"5310effd13228dfb26bf8bf50d181c8d59178d2e362e1a9deebc75b7fe5ea5b5"},"id":"7e88710fdd8a315cfff8ee90e3e9877b97b025291c4d9e63120fc8713ecc2b76","identity":"job:2012","kind":"attempt_observation","source_record":"qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-bounds-2012.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-bounds-2012.json#"],"status":"FAILED"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"{\"scope\":\"incomplete single-case prefill feasibility\",\"verdict\":\"INCONCLUSIVE\"}","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"7453cc60fceaa835767095189afc5245f5dfb962c58042223f2f274d3c421a05"},"id":"7ef8970df1e3ed2f59cf6d7f764cbeefdf095597072ee862d54003a2dbaa6910","identity":"job:2113","kind":"attempt_observation","source_record":"qwen38-h20-fp8-single-request-vllm032/results/flash-attention-prefill/result-2113.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/flash-attention-prefill/result-2113.json#"],"status":"FAILED124:0"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"INCONCLUSIVE: no standalone verdict recorded","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"de829a0e953ec1c705d3232bcc30162224cccf216db96b348f9c55f778bf56a3"},"id":"7f6a767e704311ac572ce3e4109c4e426f6e6973fe2c5e137647f75f3447b350","identity":"job_id:2002","kind":"attempt_observation","source_record":"qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-benign-retry-2002.json#/terminal","source_records":["qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-benign-retry-2002.json#/terminal"],"status":"recorded"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"{\"scope\":\"offline_evaluator_installation_readiness\",\"verdict\":\"INCONCLUSIVE\"}","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"22035b23f65be55521076bc295f711e3bd49a72b774d328c3cefb9e8046e7a8d"},"id":"7f957eb6dee2cbc634266b9650b21a2647b93e0e0f0d94a681cc2c24fa06b737","identity":"job:1998","kind":"attempt_observation","source_record":"qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-offline-install-1998.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-offline-install-1998.json#"],"status":"FAILED"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"INCONCLUSIVE: no standalone verdict recorded","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"b7ed5be15c6c3cfc51489ddfac4a0f6f5b927e8c6c388dd91e0dbfad5638ce81"},"id":"84a31a4a9d2b3df5a609a3fdd6b74761a3c9a36f903857301a7d0f977ace9193","identity":"job_id:2000","kind":"attempt_observation","source_record":"qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-bwrap-build-2000.json#/terminal","source_records":["qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-bwrap-build-2000.json#/terminal"],"status":"recorded"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"{\"scope\":\"explicit_elf_loader_context_diagnostic\",\"verdict\":\"INCONCLUSIVE\"}","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"09e9e894d5bed8d5a83fbdd317abcd45d13d90d8ab9f17604c440180367426d2"},"id":"8eef0102b2b85bcf3d358fead23bc15b409be6f35006cbafe88759b4935aa8ae","identity":"job:2005","kind":"attempt_observation","source_record":"qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-elf-context-2005.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-elf-context-2005.json#"],"status":"COMPLETED"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"INCONCLUSIVE: no standalone verdict recorded","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"061218f8db4b88a3b3a813795dd131a6e773438e82f3736741ac4edd13a1cdce"},"id":"8ffc2d4cb6b5dcd5bd0332c0584a4f814ce386090484d41936ad6fbd18f9ba0b","identity":null,"kind":"published_record","source_record":"qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-offline-install-wheels-before.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-offline-install-wheels-before.json#"],"status":"recorded"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"INCONCLUSIVE: no standalone verdict recorded","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"0e5e6eb3280b3587995425afba65e2cf7c01e7149e6f870906c3e850c7e1d280"},"id":"9139ad620313852782bf1685ebb69f223fc943e38dfa7379b73f811e651a0ca2","identity":null,"kind":"published_record","source_record":"qwen38-h20-fp8-single-request-vllm032/results/profile-merged-v1/source-provenance.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/profile-merged-v1/source-provenance.json#"],"status":"INCONCLUSIVE"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"{\"scope\":\"controlled_elf_loader_context_diagnostic\",\"verdict\":\"INCONCLUSIVE\"}","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"cc13a46da5b8d8f88e2bcbe483142c11c73e55323b76ccf516730ce1055568f5"},"id":"957346bba8662c8a4615804e754b8793737bcc3cdefc20a2412bc50c053366a0","identity":null,"kind":"published_record","source_record":"qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-elf-context-v2-observations.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-elf-context-v2-observations.json#"],"status":"recorded"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"{\"scope\":\"sandbox_dependency_resolution_readiness\",\"verdict\":\"INCONCLUSIVE\"}","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"4c18c45203e33f70912ac7a17ad6e2464bc9294a9e9883980602c2c23202d9af"},"id":"9881aee946c9e46115fd715be69dacdd5077794c7c8738d1ca09d969e834e949","identity":"job:1991","kind":"attempt_observation","source_record":"qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-resolve-1991.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-resolve-1991.json#"],"status":"COMPLETED"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"INCONCLUSIVE: no standalone verdict recorded","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"da03f4456227efabc6bc34c6c21fcfd684ef96adcd874900f88212bc05f576b3"},"id":"98ef8523084fa2dd28b8df9fccc3414cc1c6a13c4c5ddaf4e0505d493d2537fc","identity":null,"kind":"published_record","source_record":"qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-benign-retry-packages-before.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-benign-retry-packages-before.json#"],"status":"recorded"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"{\"verdict\":\"INCONCLUSIVE\"}","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"7bc3be244db5a4806aed830b269cff2ef3bae53509b0f5824ec87e62491d0819"},"id":"a0e1736ed8ba999f3053bb22cd7d55ac4de5914207659c75c7f24e4ba73f507e","identity":"job:1961","kind":"attempt_observation","source_record":"qwen38-h20-fp8-single-request-vllm032/results/diagnostics/baseline-long-1961.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/diagnostics/baseline-long-1961.json#"],"status":"recorded"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"{\"scope\":\"trusted_four_case_synthetic_evaluator\",\"verdict\":\"INCONCLUSIVE\"}","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"58abf403133093793a8db37268ef22f13765e0b9f4144c844f4c69434b84f55d"},"id":"a2838c9f9a4d422345f93538cf7e27446731db1541164e134b10048d70bd55f1","identity":"job:2009","kind":"attempt_observation","source_record":"qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-synthetic-2009.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-synthetic-2009.json#"],"status":"COMPLETED"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"{\"scope\":\"synthetic_contract_diagnostic\",\"verdict\":\"INCONCLUSIVE\"}","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"d05059bc8c48b8a4da400e3a29b44c94d667884b8ea4df784bf563dfb9425a95"},"id":"a4acbfa6c1303aa6c6e669633258f547016cbfb720c2b71417fa243f2b7558dc","identity":null,"kind":"published_record","source_record":"qwen38-h20-fp8-single-request-vllm032/results/diagnostics/quality-binding-gap.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/diagnostics/quality-binding-gap.json#"],"status":"recorded"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"{\"scope\":\"first_missing_elf_inventory_diagnostic\",\"verdict\":\"INCONCLUSIVE\"}","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"1610404120ef75d3385c65963c3a5434d9c928ee2fcda14271ecd1b5f523d599"},"id":"a614363c6624779d7bdbcc8feff7bcc9dce59b8cb5b9c8e51ee81d93124f6e87","identity":"job:2003","kind":"attempt_observation","source_record":"qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-elf-dependency-2003.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-elf-dependency-2003.json#"],"status":"COMPLETED"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"INCONCLUSIVE: no standalone verdict recorded","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"f3fad8d2fb5a30401e179705d46f9d1f3c1700647d32858f460626c684fd448b"},"id":"a7bd1fda370453ccaa93e80ababbed47b86becec8d68a4b479ba7a1a75409480","identity":"job_id:2018","kind":"attempt_observation","source_record":"qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-executor-2018.json#/terminal","source_records":["qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-executor-2018.json#/terminal"],"status":"recorded"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"INCONCLUSIVE: no standalone verdict recorded","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"466c272385aad8dd4c8b8e82ea77cf93865bfecd67aaba7d22a8add7a7a5eb6c"},"id":"a8e5ab6b2391c7b2e612cbcc2cc6121a224baa033e93cf41c2fdabec3000ebdb","identity":null,"kind":"published_record","source_record":"qwen38-h20-fp8-single-request-vllm032/results/merged-flash-prefill-v1/knowledge.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/merged-flash-prefill-v1/knowledge.json#"],"status":"recorded"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"{\"scope\":\"single-case engineering comparison only; not formal statistics or quality acceptance\",\"verdict\":\"INCONCLUSIVE\"}","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"8290bc13550b2c7ff5ea7f7d1cc319bb92d6f7a963e813546f942359e6d50513"},"id":"abd290090f2bb4fb0904b705d8e7c8c1dae112c6a5c97db859996fa130214a49","identity":"job_id:2175","kind":"attempt_observation","source_record":"qwen38-h20-fp8-single-request-vllm032/results/merged-flash-prefill-v1/receipt-review.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/merged-flash-prefill-v1/receipt-review.json#"],"status":"COMPLETED"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"{\"scope\":\"bounded_transport_and_filesystem_diagnostic\",\"verdict\":\"INCONCLUSIVE\"}","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"096d761140bb3004fa022402d907e89f9dc07610adf00c190bdacdc29443aa1b"},"id":"ac5732c2f270e21838e1718f4343b650bc8aec150303ad09291a7a77c99006b7","identity":"job_id:1996","kind":"attempt_observation","source_record":"qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-transport-observations.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-transport-observations.json#"],"status":"recorded"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"{\"scope\":\"independent baseline-only raw accounting and descriptive statistics; no paired verdict\",\"verdict\":\"INCONCLUSIVE\"}","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"2626c49e4ff7078e655e233c035d93460dd964b5f19c06c336d442c90b21c7f4"},"id":"ac73607318a4fc796bfc12faa27ec9e5aecd591f3692d2ddc6a90c5c52223d71","identity":null,"kind":"published_record","source_record":"qwen38-h20-fp8-single-request-vllm032/results/formal-8k-baseline/independent-review.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/formal-8k-baseline/independent-review.json#"],"status":"recorded"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"INCONCLUSIVE: no standalone verdict recorded","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"91fc4032ab6d57c77aeefe170817909d8d9bebdbdaa940ec68f7786b8c9f3ef2"},"id":"ae29e91da8e5ffab32b968eb37abdf6f2d2677b296e1efd4d64ad7b1f5f2c2ba","identity":null,"kind":"published_record","source_record":"qwen38-h20-fp8-single-request-vllm032/results/flashinfer-small-m/knowledge-2076.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/flashinfer-small-m/knowledge-2076.json#"],"status":"recorded"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"INCONCLUSIVE: no standalone verdict recorded","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"8e16baa6abbd533895506471669b5100f05c6ad1eea852ef1b9d959b9206f56f"},"id":"b1f66d11967819f2f32fb710a8742097b2c5fe4d311839bf8dd27946fabd1701","identity":null,"kind":"published_record","source_record":"qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-offline-install-dependencies-verified.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-offline-install-dependencies-verified.json#"],"status":"recorded"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"INCONCLUSIVE: no standalone verdict recorded","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"2f1a11686c46a992552c05dee1b71591c499711afdafa61098860f8b3f92d53b"},"id":"b3cc3e0d22b38bb0e792f5593276e08904535e10b2683dd70df71e5d7d4b0dee","identity":null,"kind":"published_record","source_record":"qwen38-h20-fp8-single-request-vllm032/results/profile-merged-v1/knowledge.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/profile-merged-v1/knowledge.json#"],"status":"recorded"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"INCONCLUSIVE: no standalone verdict recorded","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"12f9ac77bfdc23a271504e4133c425d1640a1435631e681d6589cd4455d53a35"},"id":"b444b20ec5af7a4ba93ee9e42848d595e737c50478d5030256e72f78e247568b","identity":null,"kind":"published_record","source_record":"qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-full164-v1-failed-status-metadata.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-full164-v1-failed-status-metadata.json#"],"status":"recorded"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"{\"scope\":\"single-case engineering diagnostic, not formal acceptance\",\"verdict\":\"INCONCLUSIVE\"}","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"9a0d91596efd7d9afa35039682819bb48b8329c3c83c59c94a621d26512e8400"},"id":"b8c675f432248501a47b0b6fd34ad3a5606d3262634606d31dff4845f0c62309","identity":"job:2091","kind":"attempt_observation","source_record":"qwen38-h20-fp8-single-request-vllm032/results/full-context-graph/actual-2091.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/full-context-graph/actual-2091.json#"],"status":"COMPLETED 0:0"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"{\"scope\":\"sandbox_installation_readiness\",\"verdict\":\"INCONCLUSIVE\"}","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"b81a672d6359ce83f967853bb109748095fb8e3d9a8a282a4c17f8ec27938653"},"id":"bd512b59d9ca544149c853c0b2d09c8388f22a22884d5584d243cdcaf973b70a","identity":"job:1994","kind":"attempt_observation","source_record":"qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-install-1994.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-install-1994.json#"],"status":"recorded"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"INCONCLUSIVE: no standalone verdict recorded","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"c8d0c7f85eef30fab4bbe47288f6759f28cb21bc320552e8aa3141ccd88ce364"},"id":"bdf28e3c1e6e8d60119f7bcfe1f44af92d327669319f9912532254f34cceb0ce","identity":null,"kind":"published_record","source_record":"qwen38-h20-fp8-single-request-vllm032/results/deepgemm-prefill/knowledge-2084.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/deepgemm-prefill/knowledge-2084.json#"],"status":"recorded"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"INCONCLUSIVE: no standalone verdict recorded","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"9ad317ecf32a78710c9cfc7bcce3c15ab9b99a2a3c5f7936412310cb2aab49e0"},"id":"be01f2fec487d812f0e7ff268c9771457b85d7060d050e27089981165d80dd5a","identity":"job:2040","kind":"attempt_observation","source_record":"qwen38-h20-fp8-single-request-vllm032/results/flashinfer-small-m/status.json#/gpu_attempts/1","source_records":["qwen38-h20-fp8-single-request-vllm032/results/flashinfer-small-m/status.json#/gpu_attempts/1"],"status":"CANCELLED"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"INCONCLUSIVE: no standalone verdict recorded","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"32edeb13544006ac1b725b61746d2fc18b9512ba4f648917fc2aa5721755750e"},"id":"bf7088b145bb8f1be72fc945954a76ad8d0d6775dea4d9f97b34488be5ca20c0","identity":"job_id:2001","kind":"attempt_observation","source_record":"qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-bwrap-build-2001.json#/terminal","source_records":["qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-bwrap-build-2001.json#/terminal"],"status":"recorded"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"INCONCLUSIVE: no standalone verdict recorded","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"f851c39f44ce4dc94e617c48b57bd9cdd07b01d8de5c054688e31cd1f75af764"},"id":"c0a217c38536bb95fedc9849674cc011aab4f92a4b9b1d5ceaa925597b3cb87b","identity":null,"kind":"published_record","source_record":"qwen38-h20-fp8-single-request-vllm032/results/flashinfer-gdn/knowledge-2095.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/flashinfer-gdn/knowledge-2095.json#"],"status":"recorded"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"INCONCLUSIVE: no standalone verdict recorded","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"929bae6ed3f8baad56b3936984b4f59213210be0ecdb6ec08fcc1bb20170a2e1"},"id":"c35b1761f47614361e44459cc566d08f2b0982a5aaaa999076fe4a3f44daa2d2","identity":"job_id:2009","kind":"attempt_observation","source_record":"qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-synthetic-2009.json#/terminal","source_records":["qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-synthetic-2009.json#/terminal"],"status":"recorded"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"{\"scope\":\"implementation and CPU tests; GPU queued\",\"verdict\":\"INCONCLUSIVE\"}","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"feff5c78693357a7172e1e9eaea96456255cfaa807ba89e66f35854657f682ea"},"id":"c4eb17e7b0a1843c9e409b1fc8ef0e10695eff9c6717faeba776ab2d3151c0a4","identity":null,"kind":"published_record","source_record":"qwen38-h20-fp8-single-request-vllm032/results/flashinfer-small-m/status.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/flashinfer-small-m/status.json#"],"status":"recorded"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"INCONCLUSIVE: no standalone verdict recorded","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"8d5343f1552f18a68b6c89b25ad1cd76661a277b9e2c18eb00c3a9913a1b3d19"},"id":"c5c697facf016723771b1ae3947ced03650327a8c5cd883c280d6d467c8e2dc9","identity":null,"kind":"published_record","source_record":"qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-bwrap-build-extraction-control.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-bwrap-build-extraction-control.json#"],"status":"recorded"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"{\"scope\":\"fixed_history_numerical_diagnostic_not_quality_or_performance_acceptance\",\"verdict\":\"INCONCLUSIVE\"}","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"3a5e08b4919f7688d70efa0cf29b6d0004cce2f469cd0e704c54266ac6d4ae46"},"id":"c6549379bde8fce6fece83bed823c64bf837bdd0f7604bf31e738c5f45e4fd7a","identity":"job:1957","kind":"attempt_observation","source_record":"qwen38-h20-fp8-single-request-vllm032/results/diagnostics/graph-forced96-1957.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/diagnostics/graph-forced96-1957.json#"],"status":"recorded"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"INCONCLUSIVE: no standalone verdict recorded","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"da03f4456227efabc6bc34c6c21fcfd684ef96adcd874900f88212bc05f576b3"},"id":"c74f5177784c64b5cca6ca577655ab4c66d04117e2e801ccd20854e9fdecea49","identity":null,"kind":"published_record","source_record":"qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-benign-v2-packages-verified.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-benign-v2-packages-verified.json#"],"status":"recorded"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"{\"scope\":\"sandbox_installation_readiness\",\"verdict\":\"INCONCLUSIVE\"}","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"dabd6a8bc88469186b131b60015a862a33d2e591fdd4d9e98fc82fce4039b0f6"},"id":"c7bdc6f742b34b6ce04d9870dda1978c5b4916e27a9f1f7d9b2cb93e837b1a45","identity":"job:1992","kind":"attempt_observation","source_record":"qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-install-1992.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-install-1992.json#"],"status":"recorded"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"{\"verdict\":\"INCONCLUSIVE\"}","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"1ce70465074bb841dc6aa8d1faf59dc4f6afb5e81eee853c112540054ea4b970"},"id":"c8b94ce2965c89dd45fd3b5b27bc8172c03c40987557ab7a011e691891cb6d36","identity":null,"kind":"published_record","source_record":"qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-synthetic-v3-probe-execution.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-synthetic-v3-probe-execution.json#"],"status":"recorded"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"INCONCLUSIVE: no standalone verdict recorded","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"da03f4456227efabc6bc34c6c21fcfd684ef96adcd874900f88212bc05f576b3"},"id":"cda18fc0d8f386119364cccdfa9313bea1fd10322c2237bcf1f2592ac8db37d8","identity":null,"kind":"published_record","source_record":"qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-offline-install-evaluator-packages-verified.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-offline-install-evaluator-packages-verified.json#"],"status":"recorded"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"INCONCLUSIVE: no standalone verdict recorded","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"4eb3de0750218c0af3370e120311fb9c4698840f8ccc3f67174b36ce9acde713"},"id":"ce51503f28a396c6bbcd89f594ccd1e28255dafe1d3552e1155d2353d9cfa348","identity":"job:1954","kind":"attempt_observation","source_record":"qwen38-h20-fp8-single-request-vllm032/results/diagnostics/demotion-preflight.json#/attempts/2","source_records":["qwen38-h20-fp8-single-request-vllm032/results/diagnostics/demotion-preflight.json#/attempts/2"],"status":"recorded"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"{\"scope\":\"single-case engineering diagnostic, not formal acceptance\",\"verdict\":\"INCONCLUSIVE\"}","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"c5291c60f3e3e4c7511a9bee2860fda2e66c067387e43c8b8d666bbadda11213"},"id":"d0f7f6b439e40022f2ac615578939e5b9cde78f0814284be9a964f0083591081","identity":"job:2097","kind":"attempt_observation","source_record":"qwen38-h20-fp8-single-request-vllm032/results/flashinfer-gdn/actual-2097.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/flashinfer-gdn/actual-2097.json#"],"status":"COMPLETED 0:0"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"{\"campaign_verdict\":\"INCONCLUSIVE\"}","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"ecba41d964840c9e0b456efb61e0baf93fc915448fbc26d92690b8a3213f9e2b"},"id":"d1595f60fd3b813de0016d1057a97ef4c81122531ffe7d38d297688b5ac3f7cd","identity":null,"kind":"published_record","source_record":"qwen38-h20-fp8-single-request-vllm032/results/diagnostics/benchmark-client-ledger.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/diagnostics/benchmark-client-ledger.json#"],"status":"recorded"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"{\"scope\":\"operator_integration_not_model_quality_or_service_acceptance\"}","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"3e0c619a346fc3de5d7c01ed1d10fc21e13d3986453f6a14086810505ae0a653"},"id":"d26241b7a7b1384653958310487d3db287956647c21a7e7cee799ca0bba6800c","identity":null,"kind":"published_record","source_record":"qwen38-h20-fp8-single-request-vllm032/results/flashinfer-small-m/operator-2076.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/flashinfer-small-m/operator-2076.json#"],"status":"recorded"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"{\"scope\":\"production export binding source/CPU controls\"}","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"78279f183caaacafd9c22756a470abf217b3d10fda75b8967175eb5b87e64360"},"id":"d65c3f37667f8afb147b4de5629630e8f6094e581883e53904b48715e1b85918","identity":null,"kind":"published_record","source_record":"qwen38-h20-fp8-single-request-vllm032/results/quality-executor/production-binding-review.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/quality-executor/production-binding-review.json#"],"status":"recorded"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"INCONCLUSIVE: no standalone verdict recorded","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"5e50076c13f96cf4196f0689484a19d42c4cfa0cffbac489f08c6bb270d9d9f5"},"id":"d706a4aec208f2e72070ede04072274ef4f6d292a9bdb9a5aa26a20e06753864","identity":"job:1950","kind":"attempt_observation","source_record":"qwen38-h20-fp8-single-request-vllm032/results/diagnostics/benchmark-client-ledger.json#/cpu_attempts/1","source_records":["qwen38-h20-fp8-single-request-vllm032/results/diagnostics/benchmark-client-ledger.json#/cpu_attempts/1"],"status":"completed"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"{\"scope\":\"four-handwritten-fixtures-through-external-export-interface\",\"verdict\":\"INCONCLUSIVE\"}","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"f9754ebad573195b5239757654b23a0aa34b8d3e34073641542d384c89170161"},"id":"d8688fbd4d8336f788903f01d4c9dcb9eb415f8cd3996607c77a5a51b8abe36f","identity":null,"kind":"published_record","source_record":"qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-executor-v1-summary.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-executor-v1-summary.json#"],"status":"recorded"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"INCONCLUSIVE: no standalone verdict recorded","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"b21a8b5c4b27ee2b97e85bdb46721881dbdda89d66d2a2d0a896389f8358af93"},"id":"d9e846359815780c5ab59ad50b1695a502223fa87349dbd065d49d2420ff9f5e","identity":null,"kind":"published_record","source_record":"qwen38-h20-fp8-single-request-vllm032/results/flashinfer-gdn/knowledge-preparation.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/flashinfer-gdn/knowledge-preparation.json#"],"status":"recorded"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"{\"scope\":\"exact_wrapper_boundary_lines_with_mock_meson\"}","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"4a2dda280b87b746713278e10206daafd2a80ea409a3ec72ffa82f8ac3add3fd"},"id":"db8a0ad881d40f9378e21afeea54968a263ac411758583edb3b37c8064bdd16b","identity":null,"kind":"published_record","source_record":"qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-bwrap-build-destdir-control.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-bwrap-build-destdir-control.json#"],"status":"recorded"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"INCONCLUSIVE: no standalone verdict recorded","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"bc2d6df29c83c802b72c0756a05aa3a1c3aa8929e6ef8b7fe7af32efef32c54b"},"id":"dc52b03096125438f3fcb9a0e391b00395451ef2964137b397f140c80ececf54","identity":null,"kind":"published_record","source_record":"qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-bounds-v2-synthetic-stdout.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-bounds-v2-synthetic-stdout.json#"],"status":"recorded"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"{\"scope\":\"trusted_benign_isolation_probe\",\"verdict\":\"INCONCLUSIVE\"}","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"d18e144fe457132eb77258222e3926b330ab97c20631c26bfb167b6520a3982e"},"id":"de7455b5200b0fb443b45804a2c22da69f13e0bfbc0b106731bcfce8875af2b6","identity":"job:2006","kind":"attempt_observation","source_record":"qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-benign-retry-2006.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-benign-retry-2006.json#"],"status":"COMPLETED"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"{\"scope\":\"compute_platform_and_filesystem_readiness\",\"verdict\":\"INCONCLUSIVE\"}","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"e113b252aad2ca40300ce910e64d6a5725f71bbaee421604754feeff8f9715d9"},"id":"df4d3fac033f985c4df1d812fa7ae926de1fb2dd574d1e960a31a9f80a1229db","identity":"job:1997","kind":"attempt_observation","source_record":"qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-platform-fs-1997.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-platform-fs-1997.json#"],"status":"COMPLETED"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"{\"campaign_verdict\":\"INCONCLUSIVE\"}","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"58e37d2eca6f2ea0437b09bb04b4d779c11597c31b3e150497bfe1543e9e26b1"},"id":"e0e71c4f4ea8b75059e22fd20676f9fba598172aba320318004bc38d5c6455e2","identity":"job:1955","kind":"attempt_observation","source_record":"qwen38-h20-fp8-single-request-vllm032/results/diagnostics/graph-demotion-1955.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/diagnostics/graph-demotion-1955.json#"],"status":"completed"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"{\"scope\":\"raw HTTP/client accounting only\",\"verdict\":\"INCONCLUSIVE\"}","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"69a522de277a2173c7307e06afdb761def8e90ceda3c2be75f344c59d0248719"},"id":"e1856026cb36b9ed56e03f8c066beb72464dca4b61c905123770bcf5e23c2ce2","identity":null,"kind":"published_record","source_record":"qwen38-h20-fp8-single-request-vllm032/results/formal-8k-baseline/raw-audit.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/formal-8k-baseline/raw-audit.json#"],"status":"recorded"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"INCONCLUSIVE: no standalone verdict recorded","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"7d39fcabf8d6cca7046c8816f25151ac802aa9bf18d663063d7b33e2478a2c13"},"id":"e5f69328c570614eebd56f7fb1f7fa3c6c18a02230d5eb77ba38de1a45027d45","identity":"job:1947","kind":"attempt_observation","source_record":"qwen38-h20-fp8-single-request-vllm032/results/diagnostics/benchmark-client-ledger.json#/cpu_attempts/0","source_records":["qwen38-h20-fp8-single-request-vllm032/results/diagnostics/benchmark-client-ledger.json#/cpu_attempts/0"],"status":"failed"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"INCONCLUSIVE: no standalone verdict recorded","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"62b07c24988ec8a8478aabf2ed4da07ac97c428c42185e1cbdf86aec52a0f899"},"id":"e71e1f5b1876e10eed15a3d6b326dac0c6103e53a89f571d58883797cb8df246","identity":"job:1953","kind":"attempt_observation","source_record":"qwen38-h20-fp8-single-request-vllm032/results/diagnostics/demotion-preflight.json#/attempts/1","source_records":["qwen38-h20-fp8-single-request-vllm032/results/diagnostics/demotion-preflight.json#/attempts/1"],"status":"recorded"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"{\"scope\":\"dependency_transport_readiness\",\"verdict\":\"INCONCLUSIVE\"}","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"b736bc441d3904b85e5e95772c3a58508027d6a636cf545cef9f2b44dd53cfeb"},"id":"e84d55d26c62898b55478922b89d849bda8ba7a49a76ffe3b4c841226ddfd6b0","identity":"job:1996","kind":"attempt_observation","source_record":"qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-transport-1996.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-transport-1996.json#"],"status":"FAILED"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"{\"scope\":\"baseline_retrieval_qualification_only_no_candidate\",\"verdict\":\"INCONCLUSIVE\"}","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"3c55580f60d4f6850f5050f58e435101c7c358ebd4685fd83a6061b0a24583b9"},"id":"eedc17560107b2d18877d025e5ba5cd6622a404f65d82044fba492564c0687dc","identity":"job_id:1968","kind":"attempt_observation","source_record":"qwen38-h20-fp8-single-request-vllm032/results/diagnostics/retrieval-baseline-1968.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/diagnostics/retrieval-baseline-1968.json#"],"status":"recorded"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"INCONCLUSIVE: no standalone verdict recorded","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"7d8392068f74266826c370c2ca01730404d37f24ad6102cb232247c9fcf6f770"},"id":"f38e2c223fce0a5908a7dc2685ad94bd795152f1db881223ff864de98dbe032a","identity":"job_id:1998","kind":"attempt_observation","source_record":"qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-offline-install-1998.json#/terminal_record","source_records":["qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-offline-install-1998.json#/terminal_record"],"status":"recorded"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"{\"scope\":\"prospective_natural_eos_validator_CPU_checks\",\"verdict\":\"INCONCLUSIVE\"}","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"9a3a143dc8f9ab14d0b1e327fc221ad042f2f0e1afcb75fdf6356dc2f7b85f28"},"id":"f697558cb2f0f476e2bef93f38a939d6af9883c6d7f55603633c174b6374ecae","identity":null,"kind":"published_record","source_record":"qwen38-h20-fp8-single-request-vllm032/results/diagnostics/quality-eos-v4-check.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/diagnostics/quality-eos-v4-check.json#"],"status":"recorded"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"INCONCLUSIVE: no standalone verdict recorded","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"d7bbee6738eca453c10d19e65a6bdeb4fa9408e1d11ab8cb8c594a41cbb440d1"},"id":"f723ec01543d8dd8c6a70c71bdc0fd4320400f0e29f24ccf6f86b8f7a3aa0307","identity":null,"kind":"published_record","source_record":"qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-platform-fs-controls.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-platform-fs-controls.json#"],"status":"recorded"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"{\"scope\":\"bounded_graph_correctness_diagnostic\",\"verdict\":\"INCONCLUSIVE\"}","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"12b78fba1529390823b95b6b7c05f450a3f4b82fabd29189a1a600bd1605ea45"},"id":"fba10b4dbddf81e5f7b272101b26b3d30c60a61aa929f244bd658ce69fa466e3","identity":null,"kind":"published_record","source_record":"qwen38-h20-fp8-single-request-vllm032/results/diagnostics/repair-summary.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/diagnostics/repair-summary.json#"],"status":"recorded"} +{"artifact_id":"qwen38-h20-fp8-single-request-vllm032-6c7e237-v1","conclusion":"{\"scope\":\"exact_wrapper_environment_line\"}","configuration":{"campaign_records":["qwen38-h20-fp8-single-request-vllm032/plan/search-8k-v1/activation.json","qwen38-h20-fp8-single-request-vllm032/plan/formal-8k-baseline-identities.json","qwen38-h20-fp8-single-request-vllm032/plan/quality-v5-source-identities.json"],"record_sha256":"1474f48a774510cf7057f6c6d2fa0db000e8430b969787e8f8739f8d4815ab04"},"id":"ffa65bd8388febe76c09d171d7b21cf3c1bb9b4ab72ee24aa904b785778f178f","identity":null,"kind":"published_record","source_record":"qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-bwrap-build-ninja-control.json#","source_records":["qwen38-h20-fp8-single-request-vllm032/results/diagnostics/sandbox-bwrap-build-ninja-control.json#"],"status":"recorded"} diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/validator/README.md b/evidence/qwen38-h20-fp8-single-request-vllm032/validator/README.md new file mode 100644 index 000000000..f6aa2c323 --- /dev/null +++ b/evidence/qwen38-h20-fp8-single-request-vllm032/validator/README.md @@ -0,0 +1,28 @@ +# Archive replay and frozen validation + +From the repository root, with Python 3.10+ and the standard library: + +```bash +python tools/verify_evidence_archive.py evidence/qwen38-h20-fp8-single-request-vllm032/results/artifacts.json --download +# Download the pinned asset separately to ; never run archived code. +python tools/rebuild_model_evidence.py evidence/qwen38-h20-fp8-single-request-vllm032/results/artifacts.json \ + --archive --output +python tools/verify_evidence_archive.py evidence/qwen38-h20-fp8-single-request-vllm032/results/artifacts.json \ + --archive --recomputed +``` + +Replay authenticates compressed bytes, bounds archive reads, checks every +original file and original integrity manifest, and checks that the local replay +implementation equals the pinned archived implementation. It neither extracts +nor executes archived code, and refuses an existing output directory. It scans +all published result JSON documents, including nested failed/cancelled attempts, +then regenerates the index and summary without reading retained views. +Structure, IDs, source pointers and integers compare exactly. Finite floats use +predeclared absolute and relative tolerances of 1e-12 across Python versions. + +Only the sample distributions declared by `plan/archive-projection.json` are +recomputed. Frozen statistical/quality verdicts are labelled recorded results; +unavailable raw inputs are not requalified. The index is complete for published +structured observations, not unavailable private history or narrative-only +attempts. The original validators remain byte-preserved in the archive and +require their original model/device/input dependencies for model acceptance. diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/validator/diagnostics/baseline_long_readiness.py b/evidence/qwen38-h20-fp8-single-request-vllm032/validator/diagnostics/baseline_long_readiness.py deleted file mode 100644 index 4f4ffe57d..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/validator/diagnostics/baseline_long_readiness.py +++ /dev/null @@ -1,179 +0,0 @@ -"""Bounded baseline feasibility; no performance acceptance or candidate search.""" -import argparse -import asyncio -import dataclasses -import hashlib -import importlib.metadata -import json -import os -from pathlib import Path -import signal -import socket -import subprocess -import sys -import time -import urllib.request - -import aiohttp -import vllm.benchmarks.lib.endpoint_request_func as client - - -class RecordingResponse: - def __init__(self, response, chunks): - self.response, self.chunks = response, chunks - self.status, self.reason, self.content = response.status, response.reason, self - - async def iter_any(self): - async for chunk in self.response.content.iter_any(): - self.chunks.append(chunk) - yield chunk - - -class RecordingContext: - def __init__(self, context, chunks): - self.context, self.chunks = context, chunks - - async def __aenter__(self): - return RecordingResponse(await self.context.__aenter__(), self.chunks) - - async def __aexit__(self, *args): - return await self.context.__aexit__(*args) - - -class RecordingSession: - def __init__(self, session, chunks): - self.session, self.chunks = session, chunks - - def post(self, *args, **kwargs): - return RecordingContext(self.session.post(*args, **kwargs), self.chunks) - - -def digest(path): - return hashlib.sha256(Path(path).read_bytes()).hexdigest() - - -def group_members(pgid): - rows = subprocess.check_output(['ps', '-eo', 'pid=,pgid='], text=True) - return [int(pid) for pid, group in (r.split() for r in rows.splitlines()) - if int(group) == pgid] - - -def terminate_owned(process): - forced = False - try: - os.killpg(process.pid, signal.SIGTERM) - except ProcessLookupError: - pass - try: - process.wait(timeout=30) - except subprocess.TimeoutExpired: - forced = True - if group_members(process.pid): - forced = True - try: - os.killpg(process.pid, signal.SIGKILL) - except ProcessLookupError: - pass - process.wait(timeout=10) - return {'exit_code': process.returncode, 'forced': forced, - 'owned_group_remaining': len(group_members(process.pid))} - - -async def main(): - p = argparse.ArgumentParser() - for name in ('model', 'fixture', 'output'): - p.add_argument('--' + name, required=True) - args = p.parse_args() - fixture_digest = '4f4256cc4edc9a23f55f66151738a1ecef07eb752c695e7b7fbdfa17e01d6aa9' - client_digest = 'f31fae8c573ecc4bf8e31dae03823502e71404333e8111f4b535432758302b29' - assert digest(args.fixture) == fixture_digest - assert digest(client.__file__) == client_digest - assert digest(Path(args.model)/'config.json') == '74227dd615bf1ea975aa676bdf355a0379858c12f394b5365cd9dfa5fc2c70bc' - assert importlib.metadata.version('vllm') == '0.32.0' - data = json.loads(Path(args.fixture).read_text()) - samples = {r['id']: r['input_ids'] for r in data['samples']} - cases = [('warmup', 32, 8), ('framing', 32, 8), ('feasibility', 8192, 32), - ('feasibility', 65536, 8), ('feasibility', 131072, 8)] - for _, length, _ in cases: - assert len(samples[f'n{length}-s0']) == length - out = Path(args.output); out.mkdir(parents=True, exist_ok=False) - with (out/'observations.jsonl').open('x') as stream: - def emit(**row): - stream.write(json.dumps(row)+'\n'); stream.flush() - print(json.dumps({k: v for k, v in row.items() if k not in ('token_ids', 'itl')}), flush=True) - emit(phase='protocol', observer_sha256=digest(__file__), fixture_sha256=fixture_digest, - client_sha256=client_digest, cases=cases, verdict='INCONCLUSIVE', - scope='single_baseline_feasibility_no_candidate_no_formal_matrix', - versions={k: importlib.metadata.version(k) for k in - ('vllm', 'torch', 'transformers', 'flashinfer-python', 'triton', 'aiohttp')}, - server_settings={'max_model_len': 133120, 'prefix_caching': False, - 'language_model_only': True, 'other_optimization_flags': 'defaults'}) - with socket.socket() as sock: - sock.bind(('127.0.0.1', 0)); port = sock.getsockname()[1] - url = f'http://127.0.0.1:{port}' - command = [sys.executable, '-u', '-m', 'vllm.entrypoints.openai.api_server', - '--model', args.model, '--host', '127.0.0.1', '--port', str(port), - '--served-model-name', 'qwen38-fp8', '--max-model-len', '133120', - '--no-enable-prefix-caching', '--language-model-only'] - (out/'command.private.json').write_text(json.dumps(command)+'\n') - with (out/'server.private.log').open('x') as log: - start = time.monotonic() - process = subprocess.Popen(command, stdout=log, stderr=log, start_new_session=True) - try: - while True: - if process.poll() is not None: - raise RuntimeError('server_exited_before_readiness') - try: - with urllib.request.urlopen(url+'/health', timeout=2) as response: - assert response.status == 200 - break - except (OSError, AssertionError): - if time.monotonic()-start > 1200: - raise TimeoutError('startup_timeout') - await asyncio.sleep(1) - emit(phase='ready', startup_s=time.monotonic()-start) - async with aiohttp.ClientSession(timeout=aiohttp.ClientTimeout(total=180)) as session: - for index, (purpose, length, cap) in enumerate(cases): - prompt = samples[f'n{length}-s0']; chunks = [] - request = client.RequestFuncInput( - prompt=prompt, api_url=url+'/v1/completions', prompt_len=length, - output_len=cap, model='qwen38-fp8', ignore_eos=True, - extra_body={'temperature': 0, 'return_token_ids': True}) - result = await client.async_request_openai_completions(request, RecordingSession(session, chunks)) - wire = b''.join(chunks) - (out/f'{index}.sse').write_bytes(wire) - (out/f'{index}-client.json').write_text(json.dumps(dataclasses.asdict(result), indent=2)+'\n') - events = [json.loads(line[5:].strip()) for line in wire.splitlines() - if line.startswith(b'data:') and line[5:].strip() != b'[DONE]'] - choices = [c for e in events for c in e.get('choices', [])] - tokens = [t for c in choices for t in c.get('token_ids', [])] - usage = [e['usage'] for e in events if e.get('usage')] - echoes = [c['prompt_token_ids'] for c in choices if c.get('prompt_token_ids') is not None] - terminal = [c['finish_reason'] for c in choices if c.get('finish_reason')] - errors = [e for e in events if 'error' in e] - valid = (result.success and not result.error and not errors and echoes == [prompt] - and terminal == ['length'] and b'data: [DONE]' in wire - and len(usage) == 1 and len(tokens) == cap == result.output_tokens - and usage[0]['completion_tokens'] == cap - and usage[0]['prompt_tokens'] == length == result.prompt_len - and len(result.itl) == cap-1 and len(choices) == cap - and all(len(c.get('token_ids', [])) == 1 for c in choices)) - emit(phase='request', index=index, purpose=purpose, input_tokens=length, - requested_output_tokens=cap, output_tokens=len(tokens), token_ids=tokens, - terminal=terminal, usage=usage, valid=valid, - client_success=result.success, client_error=bool(result.error), - wire_sha256=hashlib.sha256(wire).hexdigest(), - prompt_sha256=hashlib.sha256(json.dumps(prompt).encode()).hexdigest(), - ttft_s=result.ttft, latency_s=result.latency, itl=result.itl, - tpot_s=(result.latency-result.ttft)/(cap-1)) - if not valid: - raise RuntimeError('baseline_framing_or_accounting_invalid') - except BaseException as exc: - emit(phase='error', exception=type(exc).__name__, reason=str(exc) if isinstance(exc, (RuntimeError, TimeoutError)) else 'see_private_log') - raise - finally: - emit(phase='cleanup', **terminate_owned(process)) - emit(phase='complete', verdict='INCONCLUSIVE', scope='feasibility_only', elapsed_s=time.monotonic()-start) - - -asyncio.run(main()) diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/validator/diagnostics/graph_forced96.py b/evidence/qwen38-h20-fp8-single-request-vllm032/validator/diagnostics/graph_forced96.py deleted file mode 100644 index ee69f6695..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/validator/diagnostics/graph_forced96.py +++ /dev/null @@ -1,127 +0,0 @@ -"""Fixed-history, same-model numerical diagnosis; never an acceptance oracle.""" -import argparse -import hashlib -import importlib.metadata -import json -import os -from pathlib import Path -import time - -import torch - -from nanovllm.qwen38.loader import load_checkpoint -from nanovllm.qwen38.runner import RequestRunner - - -def digest(path): - return hashlib.sha256(Path(path).read_bytes()).hexdigest() - - -def compare(a, b): - if torch.equal(a, b): - return {"exact": True, "max_abs": 0., "nrmse": 0.} - a, b = a.float(), b.float() - return {"exact": False, "max_abs": (a-b).abs().max().item(), - "nrmse": ((a-b).square().mean().sqrt() / - b.square().mean().sqrt().clamp_min(1e-30)).item()} - - -def state_differences(a, b, length): - diffs = [] - for i, (x, y) in enumerate(zip(a.layers, b.layers)): - for field in ("key", "value", "conv", "recurrent"): - left, right = getattr(x, field), getattr(y, field) - if left is None or right is None: - assert left is right - continue - if field in ("key", "value"): - left, right = left[:, :, :length], right[:, :, :length] - if not torch.equal(left, right): - diffs.append({"layer": i, "field": field, **compare(left, right)}) - return diffs - - -def top(logits): - values, indices = logits.float().topk(5) - return {"ids": indices.tolist(), "logits": values.tolist(), - "margin": (values[0]-values[1]).item(), - "argmax": int(logits.argmax().item())} - - -def main(): - p = argparse.ArgumentParser() - for name in ("model", "fixtures", "history", "source", "output"): - p.add_argument("--"+name, required=True) - args = p.parse_args() - assert digest(args.fixtures) == 'f359c66f92b69d9a92a66564d8a087809dbc45b7e8b9e916229065806c4c188c' - assert digest(args.history) == '7fcf761c8d664f40ecf4a5f08a83fcf4b362e4127ecc4c932fd9d5c4ceb6cb54' - assert digest(Path(args.model)/'config.json') == '74227dd615bf1ea975aa676bdf355a0379858c12f394b5365cd9dfa5fc2c70bc' - case = 'gsm8k/test/783' - fixture = {r['id']:r for r in map(json.loads,Path(args.fixtures).read_text().splitlines())}[case] - records = [r for r in map(json.loads,Path(args.history).read_text().splitlines()) - if r.get('phase')=='request' and r['index']==0] - history = next(r['token_ids'] for r in records if r['arm']=='eager') - graph_history = next(r['token_ids'] for r in records if r['arm']=='graph') - steps, capacity = 96, 1024 - prompt = fixture['prompt'] - assert len(history)>steps and len(prompt)==149 - os.environ['NANOVLLM_CUDA_GRAPH']='1' - out=Path(args.output);out.parent.mkdir(parents=True,exist_ok=True) - with out.open('x') as stream: - def emit(**row): - stream.write(json.dumps(row)+'\n');stream.flush() - print(json.dumps({k:v for k,v in row.items() if k not in - ('compact_buffered_state_diff','buffered_graph_state_diff')}),flush=True) - emit(phase='protocol',case=case,steps=steps,capacity=capacity, - fixture_sha256=digest(args.fixtures),history_sha256=digest(args.history), - observer_sha256=digest(__file__), - source_hashes={str(f.relative_to(args.source)):digest(f) - for f in sorted(Path(args.source).glob('nanovllm/qwen38/*.py'))}, - torch=torch.__version__,device=torch.cuda.get_device_name(0), - triton=importlib.metadata.version('triton'), - scope='fixed_eager_history_same_model_no_quality_or_performance_claim') - started=time.monotonic() - model,report=load_checkpoint(args.model) - emit(phase='loaded',elapsed_s=time.monotonic()-started,report=report) - runner=RequestRunner(model,max_context_tokens=4096) - runner.prewarm_graphs() - graph=runner._acquire_graph_state(capacity) - assert graph is not None - captured=graph.reset_request_state() - compact,buffered=model.new_state(),model.new_state(capacity) - compact_match,graph_prefix_match=[],[] - with torch.inference_mode(): - inputs=torch.tensor([prompt],dtype=torch.long,device='cuda') - a=model(inputs,state=compact)[0,-1].clone() - b=model(inputs,state=buffered)[0,-1].clone() - c=model(inputs,state=captured)[0,-1].clone() - for index in range(steps): - if index: - token=history[index-1] - inp=torch.tensor([[token]],dtype=torch.long,device='cuda') - a=model(inp,state=compact)[0,-1].clone() - b=model(inp,state=buffered)[0,-1].clone() - c=graph.step(token,len(prompt)+index-1)[0,-1].clone() - length=len(prompt)+index - aa,bb,cc=top(a),top(b),top(c) - compact_match.append(aa['argmax']==history[index]) - if index<=81: - graph_prefix_match.append(cc['argmax']==graph_history[index]) - emit(phase='step',index=index,position=length-1, - teacher_token=None if index==0 else history[index-1], - compact=aa,buffered=bb,graph=cc, - compact_buffered=compare(a,b),buffered_graph=compare(b,c), - compact_buffered_state_diff=state_differences(compact,buffered,length), - buffered_graph_state_diff=state_differences(buffered,captured,length)) - runner._graph_busy.discard(capacity) - emit(phase='complete',verdict='INCONCLUSIVE', - compact_history_matches=all(compact_match), - graph_original_prefix_matches=all(graph_prefix_match), - compact_history_mismatches=[i for i,x in enumerate(compact_match) if not x], - graph_prefix_mismatches=[i for i,x in enumerate(graph_prefix_match) if not x], - seconds=time.monotonic()-started, - scope='numerical_diagnostic_not_task_quality_or_service_acceptance') - - -if __name__=='__main__': - main() diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/validator/diagnostics/graph_long_http.py b/evidence/qwen38-h20-fp8-single-request-vllm032/validator/diagnostics/graph_long_http.py deleted file mode 100644 index 01893bdec..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/validator/diagnostics/graph_long_http.py +++ /dev/null @@ -1,175 +0,0 @@ -"""Bounded Graph/eager lifecycle diagnostic with the pinned official client.""" -import argparse -import asyncio -import dataclasses -import hashlib -import json -import os -from pathlib import Path -import subprocess -import time -import urllib.request - -import aiohttp -import vllm.benchmarks.lib.endpoint_request_func as client - - -class RecordingResponse: - def __init__(self, response, chunks): - self.response, self.chunks = response, chunks - self.status, self.reason, self.content = response.status, response.reason, self - - async def iter_any(self): - async for chunk in self.response.content.iter_any(): - self.chunks.append(chunk) - yield chunk - - -class RecordingContext: - def __init__(self, context, chunks): - self.context, self.chunks = context, chunks - - async def __aenter__(self): - return RecordingResponse(await self.context.__aenter__(), self.chunks) - - async def __aexit__(self, *args): - return await self.context.__aexit__(*args) - - -class RecordingSession: - def __init__(self, session, chunks): - self.session, self.chunks = session, chunks - - def post(self, *args, **kwargs): - return RecordingContext(self.session.post(*args, **kwargs), self.chunks) - - -def get_json(url): - with urllib.request.urlopen(url, timeout=10) as response: - return json.load(response) - - -async def main(): - parser = argparse.ArgumentParser() - for name in ("source", "server-python", "model", "fixtures", "output", "client-sha256"): - parser.add_argument("--" + name, required=True) - args = parser.parse_args() - assert hashlib.sha256(Path(client.__file__).read_bytes()).hexdigest() == args.client_sha256 - out = Path(args.output) - out.mkdir(parents=True, exist_ok=False) - fixture = Path(args.fixtures) - rows = {r["id"]: r for r in map(json.loads, fixture.read_text().splitlines())} - cases = [("gsm8k/test/783", 2048, False), ("gsm8k/test/1", 2048, False), - ("gsm8k/test/424", 2048, False), ("gsm8k/test/783", 1024, True)] - observed = {} - with (out / "observations.jsonl").open("x") as stream: - def emit(**row): - stream.write(json.dumps(row) + "\n") - stream.flush() - print(json.dumps({k: v for k, v in row.items() if k not in ("token_ids", "itl")}), flush=True) - - emit(phase="protocol", cases=cases, client_sha256=args.client_sha256, - observer_sha256=hashlib.sha256(Path(__file__).read_bytes()).hexdigest(), - fixture_sha256=hashlib.sha256(fixture.read_bytes()).hexdigest(), - timing_scope="readiness_only_official_function_with_response_chunk_recording", - source_hashes={str(p.relative_to(args.source)): hashlib.sha256(p.read_bytes()).hexdigest() - for p in sorted(Path(args.source).glob("nanovllm/qwen38/*.py"))}) - for arm in ("graph", "eager"): - ready, shutdown = out / (arm + "-ready.json"), out / (arm + "-shutdown.json") - env = dict(os.environ, PYTHONPATH=args.source, NANOVLLM_FP8_GEMM="triton", - NANOVLLM_CUDA_GRAPH=str(int(arm == "graph"))) - command = [args.server_python, "-u", "-m", "nanovllm.qwen38.server", - "--model", args.model, "--port", "0", "--max-context-tokens", "4096", - "--max-active-requests", "1", "--ready-file", str(ready), - "--shutdown-file", str(shutdown)] - with (out / (arm + "-server.log")).open("x") as log: - start = time.monotonic() - process = subprocess.Popen(command, cwd=args.source, env=env, stdout=log, stderr=log) - try: - while True: - if process.poll() is not None: - raise RuntimeError("server_exited_before_readiness") - try: - info = json.loads(ready.read_text()) - url = "http://127.0.0.1:" + str(info["address"][1]) - health = get_json(url + "/health") - break - except (FileNotFoundError, json.JSONDecodeError, OSError): - if time.monotonic() - start > 180: - raise TimeoutError("startup_timeout") - await asyncio.sleep(0.2) - emit(phase="ready", arm=arm, startup_s=time.monotonic() - start, - versions=health["versions"], device=health["device_model"], - backend=health["loader_report"].get("backend"), - graph_prewarm=health.get("graph_prewarm")) - async with aiohttp.ClientSession(timeout=aiohttp.ClientTimeout(total=300)) as session: - for index, (case, cap, ignore_eos) in enumerate(cases): - prompt = rows[case]["prompt"] - chunks = [] - request = client.RequestFuncInput( - prompt=prompt, api_url=url + "/v1/completions", prompt_len=len(prompt), - output_len=cap, model=health["model_id"], ignore_eos=ignore_eos, - extra_body={"temperature": 0, "return_token_ids": True}) - output = await client.async_request_openai_completions( - request, RecordingSession(session, chunks)) - # Originals contain the model identifier and remain private. - wire = b"".join(chunks) - (out / f"{arm}-{index}.sse").write_bytes(wire) - (out / f"{arm}-{index}-client.json").write_text( - json.dumps(dataclasses.asdict(output), indent=2) + "\n") - events = [json.loads(line[5:].strip()) for line in wire.splitlines() - if line.startswith(b"data:") and line[5:].strip() != b"[DONE]"] - choices = [choice for event in events for choice in event.get("choices", [])] - tokens = [token for choice in choices for token in choice.get("token_ids", [])] - usage = [event["usage"] for event in events if event.get("usage")] - echoes = [choice["prompt_token_ids"] for choice in choices - if "prompt_token_ids" in choice] - terminal = [choice["finish_reason"] for choice in choices if choice.get("finish_reason")] - stats = get_json(url + "/stats") - valid = (output.success and len(usage) == 1 and echoes == [prompt] - and terminal in (["length"], ["stop"]) and b"data: [DONE]" in wire - and len(tokens) == output.output_tokens == usage[0]["completion_tokens"] - and output.prompt_len == usage[0]["prompt_tokens"] == len(prompt) - and len(output.itl) == len(tokens) - 1 - and not stats["active_requests"] and not stats["runner_state_count"] - and (not ignore_eos or len(tokens) == cap)) - observed[(arm, index)] = tokens - emit(phase="request", arm=arm, case=case, index=index, cap=cap, - ignore_eos=ignore_eos, prompt_tokens=len(prompt), - prompt_sha256=hashlib.sha256(json.dumps(prompt).encode()).hexdigest(), - token_ids=tokens, tokens=len(tokens), terminal=terminal, usage=usage, - output_text_sha256=hashlib.sha256(output.generated_text.encode()).hexdigest(), - wire_sha256=hashlib.sha256(wire).hexdigest(), - client_success=output.success, client_error=bool(output.error), valid=valid, - ttft_s=output.ttft, latency_s=output.latency, itl=output.itl, - tpot_s=(output.latency-output.ttft)/(len(tokens)-1) if len(tokens)>1 else None, - active_requests_after=stats["active_requests"], - runner_state_count_after=stats["runner_state_count"]) - if not valid: - raise RuntimeError("HTTP_accounting_or_lifecycle_failed") - except BaseException as exc: - emit(phase="error", arm=arm, exception=type(exc).__name__) - raise - finally: - process.terminate() - forced = False - try: - process.wait(timeout=20) - except subprocess.TimeoutExpired: - process.kill() - forced = True - process.wait(timeout=10) - final = json.loads(shutdown.read_text()) if shutdown.exists() else {} - emit(phase="cleanup", arm=arm, exit_code=process.returncode, forced=forced, - closed=final.get("closed"), active_requests=final.get("active_requests"), - runner_state_count=final.get("runner_state_count")) - for index, (case, cap, ignore_eos) in enumerate(cases): - a, b = observed[("graph", index)], observed[("eager", index)] - first = next((i for i, (x, y) in enumerate(zip(a, b)) if x != y), - None if len(a) == len(b) else min(len(a), len(b))) - emit(phase="comparison", case=case, cap=cap, ignore_eos=ignore_eos, - graph_tokens=len(a), eager_tokens=len(b), exact=a == b, first_difference=first) - emit(phase="complete", verdict="INCONCLUSIVE", scope="bounded_readiness") - - -asyncio.run(main()) diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/validator/diagnostics/graph_long_http_v2.py b/evidence/qwen38-h20-fp8-single-request-vllm032/validator/diagnostics/graph_long_http_v2.py deleted file mode 100644 index 682adbf49..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/validator/diagnostics/graph_long_http_v2.py +++ /dev/null @@ -1,175 +0,0 @@ -"""Bounded Graph/eager lifecycle diagnostic with the pinned official client.""" -import argparse -import asyncio -import dataclasses -import hashlib -import json -import os -from pathlib import Path -import subprocess -import time -import urllib.request - -import aiohttp -import vllm.benchmarks.lib.endpoint_request_func as client - - -class RecordingResponse: - def __init__(self, response, chunks): - self.response, self.chunks = response, chunks - self.status, self.reason, self.content = response.status, response.reason, self - - async def iter_any(self): - async for chunk in self.response.content.iter_any(): - self.chunks.append(chunk) - yield chunk - - -class RecordingContext: - def __init__(self, context, chunks): - self.context, self.chunks = context, chunks - - async def __aenter__(self): - return RecordingResponse(await self.context.__aenter__(), self.chunks) - - async def __aexit__(self, *args): - return await self.context.__aexit__(*args) - - -class RecordingSession: - def __init__(self, session, chunks): - self.session, self.chunks = session, chunks - - def post(self, *args, **kwargs): - return RecordingContext(self.session.post(*args, **kwargs), self.chunks) - - -def get_json(url): - with urllib.request.urlopen(url, timeout=10) as response: - return json.load(response) - - -async def main(): - parser = argparse.ArgumentParser() - for name in ("source", "server-python", "model", "fixtures", "output", "client-sha256"): - parser.add_argument("--" + name, required=True) - args = parser.parse_args() - assert hashlib.sha256(Path(client.__file__).read_bytes()).hexdigest() == args.client_sha256 - out = Path(args.output) - out.mkdir(parents=True, exist_ok=False) - fixture = Path(args.fixtures) - rows = {r["id"]: r for r in map(json.loads, fixture.read_text().splitlines())} - cases = [("gsm8k/test/783", 2048, False), ("gsm8k/test/1", 2048, False), - ("gsm8k/test/424", 2048, False), ("gsm8k/test/783", 1024, True), ("gsm8k/test/783", 16, True)] - observed = {} - with (out / "observations.jsonl").open("x") as stream: - def emit(**row): - stream.write(json.dumps(row) + "\n") - stream.flush() - print(json.dumps({k: v for k, v in row.items() if k not in ("token_ids", "itl")}), flush=True) - - emit(phase="protocol", cases=cases, client_sha256=args.client_sha256, - observer_sha256=hashlib.sha256(Path(__file__).read_bytes()).hexdigest(), - fixture_sha256=hashlib.sha256(fixture.read_bytes()).hexdigest(), - timing_scope="readiness_only_official_function_with_response_chunk_recording", - source_hashes={str(p.relative_to(args.source)): hashlib.sha256(p.read_bytes()).hexdigest() - for p in sorted(Path(args.source).glob("nanovllm/qwen38/*.py"))}) - for arm in ("graph", "eager"): - ready, shutdown = out / (arm + "-ready.json"), out / (arm + "-shutdown.json") - env = dict(os.environ, PYTHONPATH=args.source, NANOVLLM_FP8_GEMM="triton", - NANOVLLM_CUDA_GRAPH=str(int(arm == "graph"))) - command = [args.server_python, "-u", "-m", "nanovllm.qwen38.server", - "--model", args.model, "--port", "0", "--max-context-tokens", "4096", - "--max-active-requests", "1", "--ready-file", str(ready), - "--shutdown-file", str(shutdown)] - with (out / (arm + "-server.log")).open("x") as log: - start = time.monotonic() - process = subprocess.Popen(command, cwd=args.source, env=env, stdout=log, stderr=log) - try: - while True: - if process.poll() is not None: - raise RuntimeError("server_exited_before_readiness") - try: - info = json.loads(ready.read_text()) - url = "http://127.0.0.1:" + str(info["address"][1]) - health = get_json(url + "/health") - break - except (FileNotFoundError, json.JSONDecodeError, OSError): - if time.monotonic() - start > 180: - raise TimeoutError("startup_timeout") - await asyncio.sleep(0.2) - emit(phase="ready", arm=arm, startup_s=time.monotonic() - start, - versions=health["versions"], device=health["device_model"], - backend=health["loader_report"].get("backend"), - graph_prewarm=health.get("graph_prewarm")) - async with aiohttp.ClientSession(timeout=aiohttp.ClientTimeout(total=300)) as session: - for index, (case, cap, ignore_eos) in enumerate(cases): - prompt = rows[case]["prompt"] - chunks = [] - request = client.RequestFuncInput( - prompt=prompt, api_url=url + "/v1/completions", prompt_len=len(prompt), - output_len=cap, model=health["model_id"], ignore_eos=ignore_eos, - extra_body={"temperature": 0, "return_token_ids": True}) - output = await client.async_request_openai_completions( - request, RecordingSession(session, chunks)) - # Originals contain the model identifier and remain private. - wire = b"".join(chunks) - (out / f"{arm}-{index}.sse").write_bytes(wire) - (out / f"{arm}-{index}-client.json").write_text( - json.dumps(dataclasses.asdict(output), indent=2) + "\n") - events = [json.loads(line[5:].strip()) for line in wire.splitlines() - if line.startswith(b"data:") and line[5:].strip() != b"[DONE]"] - choices = [choice for event in events for choice in event.get("choices", [])] - tokens = [token for choice in choices for token in choice.get("token_ids", [])] - usage = [event["usage"] for event in events if event.get("usage")] - echoes = [choice["prompt_token_ids"] for choice in choices - if "prompt_token_ids" in choice] - terminal = [choice["finish_reason"] for choice in choices if choice.get("finish_reason")] - stats = get_json(url + "/stats") - valid = (output.success and len(usage) == 1 and echoes == [prompt] - and terminal in (["length"], ["stop"]) and b"data: [DONE]" in wire - and len(tokens) == output.output_tokens == usage[0]["completion_tokens"] - and output.prompt_len == usage[0]["prompt_tokens"] == len(prompt) - and len(output.itl) == len(tokens) - 1 - and not stats["active_requests"] and not stats["runner_state_count"] - and (not ignore_eos or len(tokens) == cap)) - observed[(arm, index)] = tokens - emit(phase="request", arm=arm, case=case, index=index, cap=cap, - ignore_eos=ignore_eos, prompt_tokens=len(prompt), - prompt_sha256=hashlib.sha256(json.dumps(prompt).encode()).hexdigest(), - token_ids=tokens, tokens=len(tokens), terminal=terminal, usage=usage, - output_text_sha256=hashlib.sha256(output.generated_text.encode()).hexdigest(), - wire_sha256=hashlib.sha256(wire).hexdigest(), - client_success=output.success, client_error=bool(output.error), valid=valid, - ttft_s=output.ttft, latency_s=output.latency, itl=output.itl, - tpot_s=(output.latency-output.ttft)/(len(tokens)-1) if len(tokens)>1 else None, - active_requests_after=stats["active_requests"], - runner_state_count_after=stats["runner_state_count"]) - if not valid: - raise RuntimeError("HTTP_accounting_or_lifecycle_failed") - except BaseException as exc: - emit(phase="error", arm=arm, exception=type(exc).__name__) - raise - finally: - process.terminate() - forced = False - try: - process.wait(timeout=20) - except subprocess.TimeoutExpired: - process.kill() - forced = True - process.wait(timeout=10) - final = json.loads(shutdown.read_text()) if shutdown.exists() else {} - emit(phase="cleanup", arm=arm, exit_code=process.returncode, forced=forced, - closed=final.get("closed"), active_requests=final.get("active_requests"), - runner_state_count=final.get("runner_state_count")) - for index, (case, cap, ignore_eos) in enumerate(cases): - a, b = observed[("graph", index)], observed[("eager", index)] - first = next((i for i, (x, y) in enumerate(zip(a, b)) if x != y), - None if len(a) == len(b) else min(len(a), len(b))) - emit(phase="comparison", case=case, cap=cap, ignore_eos=ignore_eos, - graph_tokens=len(a), eager_tokens=len(b), exact=a == b, first_difference=first) - emit(phase="complete", verdict="INCONCLUSIVE", scope="bounded_readiness") - - -asyncio.run(main()) diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/validator/diagnostics/graph_probe.py b/evidence/qwen38-h20-fp8-single-request-vllm032/validator/diagnostics/graph_probe.py deleted file mode 100644 index 0d8c1689b..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/validator/diagnostics/graph_probe.py +++ /dev/null @@ -1,132 +0,0 @@ -"""Bounded same-model diagnostic: compact eager / buffered eager / replay. - -No benchmark or acceptance verdict. Teacher-forced inputs isolate numerical -differences from autoregressive divergence. Writes one durable row per phase. -""" -import argparse -import gc -import hashlib -import importlib.metadata -import json -import sys -import time -from pathlib import Path - -import torch - -from nanovllm.qwen38.graph import DecodeGraph -from nanovllm.qwen38.loader import load_checkpoint - - -def dependency_version(name): - try: - return importlib.metadata.version(name) - except importlib.metadata.PackageNotFoundError: - return None - - -def compare(a, b): - a, b = a.detach().float(), b.detach().float() - return {"exact": torch.equal(a, b), "max_abs": (a - b).abs().max().item(), - "nrmse": ((a-b).square().mean().sqrt() / - b.square().mean().sqrt().clamp_min(1e-30)).item()} - - -def state_summary(a, b, length): - result = [] - for i, (x, y) in enumerate(zip(a.layers, b.layers)): - for key in ("key", "value", "conv", "recurrent"): - left, right = getattr(x, key), getattr(y, key) - if left is None or right is None: - assert left is right - continue - if key in ("key", "value"): - left, right = left[:, :, :length], right[:, :, :length] - item = compare(left, right) - if not item["exact"]: - result.append({"layer": i, "state": key, **item}) - return result - - -def main(): - parser = argparse.ArgumentParser() - parser.add_argument("--model", required=True) - parser.add_argument("--fixtures", required=True) - parser.add_argument("--output", required=True) - parser.add_argument("--steps", type=int, default=16) - args = parser.parse_args() - assert 1 <= args.steps <= 64 - out = Path(args.output) - out.parent.mkdir(parents=True, exist_ok=True) - with out.open("x") as stream: - def emit(**row): - stream.write(json.dumps(row) + "\n") - stream.flush() - print(json.dumps({k: v for k, v in row.items() - if k not in ("state_diff", "layer_diff")}), flush=True) - - emit(phase="identity", python=sys.version, torch=torch.__version__, - cuda=torch.version.cuda, device=torch.cuda.get_device_name(0), - capability=torch.cuda.get_device_capability(0), - dependencies={n: dependency_version(n) for n in - ("triton", "transformers", "flashinfer-python", "safetensors")}) - start = time.monotonic() - model, report = load_checkpoint(args.model) - emit(phase="loaded", seconds=time.monotonic()-start, report=report) - fixtures = [json.loads(line) for line in Path(args.fixtures).read_text().splitlines()] - ids = ("gsm8k/test/783", "gsm8k/test/1", "gsm8k/test/424") - by_id = {row["id"]: row for row in fixtures} - traces = {} - handles = [layer.register_forward_hook( - lambda _m, _a, y, i=i: traces.__setitem__(i, y.detach().clone())) - for i, layer in enumerate(model.model.layers)] - for case in ids: - prompt = by_id[case]["prompt"] - tokens = torch.tensor([prompt], device="cuda", dtype=torch.long) - capacity = DecodeGraph.bucket_for(len(prompt)+256) - compact, buffered, captured = model.new_state(), model.new_state(capacity), model.new_state(capacity) - with torch.inference_mode(): - a = model(tokens, state=compact).clone() - trace_a = dict(traces) - b = model(tokens, state=buffered).clone() - trace_b = dict(traces) - c = model(tokens, state=captured).clone() - emit(phase="prefill", case=case, prompt_tokens=len(prompt), capacity=capacity, - prompt_sha256=hashlib.sha256(json.dumps(prompt).encode()).hexdigest(), - compact_buffered=compare(a, b), buffered_repeated=compare(b, c), - top1=[int(x[0,-1].argmax()) for x in (a,b,c)], - layer_diff=[{"layer":i, **compare(trace_a[i], trace_b[i])} - for i in trace_a if not torch.equal(trace_a[i], trace_b[i])], - state_diff=state_summary(compact, buffered, len(prompt))) - graph = DecodeGraph(model, captured, len(prompt), capacity).capture() - captured_traces = dict(traces) - emit(phase="capture_restore", case=case, - state_diff=state_summary(buffered, captured, len(prompt))) - for step in range(args.steps): - token = int(a[0,-1].argmax()) - inp = torch.tensor([[token]], dtype=torch.long, device="cuda") - a = model(inp, state=compact).clone() - trace_a = dict(traces) - b = model(inp, state=buffered).clone() - trace_b = dict(traces) - c = graph.step(token, len(prompt)+step).clone() - emit(phase="decode", case=case, step=step, teacher_token=token, - compact_buffered=compare(a,b), buffered_graph=compare(b,c), - top1=[int(x[0,-1].argmax()) for x in (a,b,c)], - layer_diff=[{"layer":i, "compact_buffered":compare(trace_a[i], trace_b[i]), - "buffered_graph":compare(trace_b[i], captured_traces[i])} - for i in trace_a if not torch.equal(trace_a[i], trace_b[i]) - or not torch.equal(trace_b[i], captured_traces[i])], - state_diff=state_summary(buffered,captured,len(prompt)+step+1)) - del graph, compact, buffered, captured, a, b, c, trace_a, trace_b, captured_traces - traces.clear() - gc.collect() - torch.cuda.empty_cache() - for handle in handles: - handle.remove() - emit(phase="complete", scope="diagnostic_only", verdict="INCONCLUSIVE", - seconds=time.monotonic()-start) - - -if __name__ == "__main__": - main() diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/validator/diagnostics/http_readiness.py b/evidence/qwen38-h20-fp8-single-request-vllm032/validator/diagnostics/http_readiness.py deleted file mode 100644 index 2492fa412..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/validator/diagnostics/http_readiness.py +++ /dev/null @@ -1,144 +0,0 @@ -"""Bounded HTTP correctness/impact probe, not the campaign benchmark client.""" -import argparse -import hashlib -import json -import os -from pathlib import Path -import subprocess -import sys -import time -import urllib.request - - -def digest(value): - return hashlib.sha256(json.dumps(value).encode()).hexdigest() - - -def get_json(url): - with urllib.request.urlopen(url, timeout=10) as response: - return json.load(response) - - -def completion(url, prompt, request_id): - payload = {"prompt": prompt, "temperature": 0, "max_tokens": 16, - "stream": True, "request_id": request_id} - request = urllib.request.Request(url + "/v1/completions", - json.dumps(payload).encode(), - {"Content-Type": "application/json"}) - ids, stamps, texts, terminal, done = [], [], [], None, False - started = time.perf_counter() - with urllib.request.urlopen(request, timeout=120) as response: - for line in response: - if not line.startswith(b"data:"): - continue - data = line[5:].strip() - if data == b"[DONE]": - done = True - continue - event = json.loads(data) - if "error" in event: - raise RuntimeError("stream_error") - for choice in event.get("choices", []): - tokens = choice.get("token_ids", []) - ids.extend(tokens) - stamps.extend([time.perf_counter() - started] * len(tokens)) - texts.append(choice.get("text", "")) - terminal = choice.get("finish_reason") or terminal - elapsed = time.perf_counter() - started - return {"token_ids": ids, "token_times_s": stamps, - "text_sha256": hashlib.sha256("".join(texts).encode()).hexdigest(), - "tokens": len(ids), "terminal": terminal, "done": done, - "ttft_token_s": stamps[0] if stamps else None, - "tpot_token_s": (stamps[-1] - stamps[0]) / (len(stamps) - 1) - if len(stamps) > 1 else None, "latency_s": elapsed} - - -def main(): - parser = argparse.ArgumentParser() - for name in ("baseline", "candidate", "model", "fixtures", "output"): - parser.add_argument("--" + name, required=True) - args = parser.parse_args() - out = Path(args.output) - out.mkdir(parents=True, exist_ok=False) - fixture = Path(args.fixtures) - rows = {row["id"]: row for row in map(json.loads, fixture.read_text().splitlines())} - cases = ("gsm8k/test/783", "gsm8k/test/1", "gsm8k/test/424") - with (out / "observations.jsonl").open("x") as stream: - def emit(**row): - stream.write(json.dumps(row) + "\n") - stream.flush() - print(json.dumps(row), flush=True) - - emit(phase="protocol", cases=list(cases), repetitions=2, max_tokens=16, - natural_eos=True, fixture_sha256=hashlib.sha256(fixture.read_bytes()).hexdigest(), - client_sha256=hashlib.sha256(Path(__file__).read_bytes()).hexdigest(), - scope="readiness_only_custom_token_event_HTTP_timer_not_vLLM_acceptance") - arms = (("main_graph", args.baseline, True), - ("repair_graph", args.candidate, True), - ("repair_eager", args.candidate, False)) - for arm, source, graph in arms: - env = dict(os.environ, PYTHONPATH=source, NANOVLLM_CUDA_GRAPH=str(int(graph)), - NANOVLLM_FP8_GEMM="triton") - ready, shutdown = out / (arm + "-ready.json"), out / (arm + "-shutdown.json") - command = [sys.executable, "-u", "-m", "nanovllm.qwen38.server", - "--model", args.model, "--port", "0", - "--max-context-tokens", "2048", "--max-active-requests", "1", - "--ready-file", str(ready), "--shutdown-file", str(shutdown)] - emit(phase="source", arm=arm, graph=graph, - hashes={str(p.relative_to(source)): hashlib.sha256(p.read_bytes()).hexdigest() - for p in sorted(Path(source).glob("nanovllm/qwen38/*.py"))}) - with (out / (arm + "-server.log")).open("x") as log: - start = time.monotonic() - process = subprocess.Popen(command, cwd=source, env=env, stdout=log, stderr=log) - try: - while True: - if process.poll() is not None: - raise RuntimeError("server_exited_before_readiness") - try: - info = json.loads(ready.read_text()) - url = "http://127.0.0.1:" + str(info["address"][1]) - health = get_json(url + "/health") - break - except (FileNotFoundError, json.JSONDecodeError, OSError): - if time.monotonic() - start > 180: - raise TimeoutError("startup_timeout") - time.sleep(0.2) - emit(phase="ready", arm=arm, startup_s=time.monotonic() - start, - versions=health["versions"], device=health["device_model"], - graph_prewarm=health.get("graph_prewarm"), - backend=health["loader_report"].get("backend"), - fp8_execution=health.get("fp8_execution")) - for repetition in range(2): - for case in cases: - prompt = rows[case]["prompt"] - result = completion(url, prompt, f"{arm}-{repetition}-{case.split('/')[-1]}") - stats = get_json(url + "/stats") - emit(phase="request", arm=arm, case=case, repetition=repetition, - prompt_tokens=len(prompt), prompt_sha256=digest(prompt), - active_requests_after=stats["active_requests"], - runner_state_count_after=stats["runner_state_count"], **result) - if not result["done"] or result["terminal"] not in ("stop", "length"): - raise RuntimeError("incomplete_response") - if stats["active_requests"] or stats["runner_state_count"]: - raise RuntimeError("request_state_not_released") - except BaseException as exc: - emit(phase="error", arm=arm, exception=type(exc).__name__) - raise - finally: - process.terminate() - forced = False - try: - process.wait(timeout=20) - except subprocess.TimeoutExpired: - process.kill() - forced = True - process.wait(timeout=10) - state = json.loads(shutdown.read_text()) if shutdown.exists() else {} - emit(phase="cleanup", arm=arm, exit_code=process.returncode, forced=forced, - closed=state.get("closed"), active_requests=state.get("active_requests"), - runner_state_count=state.get("runner_state_count")) - emit(phase="complete", verdict="INCONCLUSIVE", scope="readiness_only") - - -if __name__ == "__main__": - main() diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/validator/diagnostics/official_client_probe.py b/evidence/qwen38-h20-fp8-single-request-vllm032/validator/diagnostics/official_client_probe.py deleted file mode 100644 index 76ce0297a..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/validator/diagnostics/official_client_probe.py +++ /dev/null @@ -1,50 +0,0 @@ -"""Exercise the unmodified official request function against a synthetic service.""" -import argparse -import asyncio -import dataclasses -import hashlib -import json -from pathlib import Path - -import aiohttp -import vllm.benchmarks.lib.endpoint_request_func as client -from test_qwen38_service import engine, serving - - -async def main(): - parser = argparse.ArgumentParser() - parser.add_argument("--output", required=True) - parser.add_argument("--client-sha256", required=True) - parser.add_argument("--expect", choices=("supported", "rejected"), required=True) - args = parser.parse_args() - actual = hashlib.sha256(Path(client.__file__).read_bytes()).hexdigest() - assert actual == args.client_sha256, (actual, args.client_sha256) - rows = [] - with serving(engine(eos=(1,))) as (model, address): - async with aiohttp.ClientSession(timeout=aiohttp.ClientTimeout(total=20)) as session: - for ignore_eos in (False, True): - request = client.RequestFuncInput( - prompt=[1], api_url=f"http://{address[0]}:{address[1]}/v1/completions", - prompt_len=999, output_len=4, model=model.model_id, - ignore_eos=ignore_eos, - extra_body={"temperature": 0, "return_token_ids": True}) - output = await client.async_request_openai_completions(request, session) - row = dataclasses.asdict(output) - row.update(ignore_eos=ignore_eos, requests_left=model.stats()["active_requests"]) - rows.append(row) - if args.expect == "supported": - expected = 4 if ignore_eos else 1 - assert output.success and output.output_tokens == expected - assert output.prompt_len == 1 # Actual usage replaces caller estimate. - assert len(output.itl) == expected - 1 - assert not output.error and row["requests_left"] == 0 - else: - assert not output.success and output.error == "Bad Request" - assert not model.runner.model.calls - result = {"client_sha256": actual, "expected": args.expect, "rows": rows, - "scope": "CPU synthetic protocol, not model performance acceptance"} - Path(args.output).write_text(json.dumps(result, indent=2) + "\n") - print(json.dumps(result), flush=True) - - -asyncio.run(main()) diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/validator/diagnostics/official_client_probe_v2.py b/evidence/qwen38-h20-fp8-single-request-vllm032/validator/diagnostics/official_client_probe_v2.py deleted file mode 100644 index 02b0d7f8a..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/validator/diagnostics/official_client_probe_v2.py +++ /dev/null @@ -1,88 +0,0 @@ -"""Exercise the unmodified official request function against a synthetic service.""" -import argparse -import asyncio -import dataclasses -import hashlib -import json -import os -from pathlib import Path -import subprocess -import time - -import aiohttp -import vllm.benchmarks.lib.endpoint_request_func as client - - -async def main(): - parser = argparse.ArgumentParser() - parser.add_argument("--output", required=True) - parser.add_argument("--client-sha256", required=True) - parser.add_argument("--expect", choices=("supported", "rejected"), required=True) - parser.add_argument("--source", required=True) - parser.add_argument("--server-python", required=True) - args = parser.parse_args() - actual = hashlib.sha256(Path(client.__file__).read_bytes()).hexdigest() - assert actual == args.client_sha256, (actual, args.client_sha256) - rows = [] - ready = Path(args.output + ".ready") - code = ''' -import json, sys -from pathlib import Path -from test_qwen38_service import engine, serving -with serving(engine(eos=(1,))) as (model, address): - Path(sys.argv[1]).write_text(json.dumps({"address":address,"model_id":model.model_id})) - sys.stdin.read() -''' - env = dict(os.environ, PYTHONPATH=args.source + os.pathsep + args.source + "/tests") - server = subprocess.Popen([args.server_python, "-c", code, str(ready)], env=env, - stdin=subprocess.PIPE) - try: - deadline = time.monotonic() + 30 - while True: - if server.poll() is not None: - raise RuntimeError("synthetic_server_exited") - try: - info = json.loads(ready.read_text()) - address = info["address"] - break - except (FileNotFoundError, json.JSONDecodeError): - if time.monotonic() > deadline: - raise TimeoutError("synthetic_server_timeout") - await asyncio.sleep(0.05) - async with aiohttp.ClientSession(timeout=aiohttp.ClientTimeout(total=20)) as session: - for ignore_eos in (False, True): - request = client.RequestFuncInput( - prompt=[1], api_url=f"http://{address[0]}:{address[1]}/v1/completions", - prompt_len=999, output_len=4, model=info["model_id"], - ignore_eos=ignore_eos, - extra_body={"temperature": 0, "return_token_ids": True}) - output = await client.async_request_openai_completions(request, session) - row = dataclasses.asdict(output) - async with session.get(f"http://{address[0]}:{address[1]}/stats") as response: - stats = await response.json() - row.update(ignore_eos=ignore_eos, requests_left=stats["active_requests"]) - rows.append(row) - if args.expect == "supported": - expected = 4 if ignore_eos else 1 - assert output.success and output.output_tokens == expected - assert output.prompt_len == 1 # Actual usage replaces caller estimate. - assert len(output.itl) == expected - 1 - assert not output.error and row["requests_left"] == 0 - else: - assert not output.success and output.error == "Bad Request" - assert stats["counters"] == {} - finally: - server.stdin.close() - try: - server.wait(timeout=10) - except subprocess.TimeoutExpired: - server.kill() - server.wait(timeout=10) - result = {"client_sha256": actual, "expected": args.expect, "rows": rows, - "server_exit": server.returncode, - "scope": "CPU synthetic protocol, not model performance acceptance"} - Path(args.output).write_text(json.dumps(result, indent=2) + "\n") - print(json.dumps(result), flush=True) - - -asyncio.run(main()) diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/validator/diagnostics/prepare_retrieval.py b/evidence/qwen38-h20-fp8-single-request-vllm032/validator/diagnostics/prepare_retrieval.py deleted file mode 100644 index 2cebfc2d3..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/validator/diagnostics/prepare_retrieval.py +++ /dev/null @@ -1,72 +0,0 @@ -"""Deterministic proposed retrieval inputs; preparation does not activate gates.""" -import argparse -import gzip -import hashlib -import importlib.metadata -import json -from pathlib import Path -from transformers import AutoTokenizer - - -def digest(path): - return hashlib.sha256(Path(path).read_bytes()).hexdigest() - - -def ident(text): - return hashlib.sha256(('inferrsi-retrieval-v1:'+text).encode()).hexdigest()[:12].upper() - - -def main(): - p=argparse.ArgumentParser() - p.add_argument('--model', required=True);p.add_argument('--output', required=True) - args=p.parse_args() - model=Path(args.model);out=Path(args.output);out.mkdir(parents=True,exist_ok=False) - assert digest(model/'config.json') == '74227dd615bf1ea975aa676bdf355a0379858c12f394b5365cd9dfa5fc2c70bc' - tokenizer=AutoTokenizer.from_pretrained(model,local_files_only=True) - encode=lambda text:tokenizer.encode(text,add_special_tokens=False) - cases=[];manifest=[] - for length in (8192,65536,131072): - for index in range(10): - case=f'retrieval-n{length}-s{index}' - keys=['KEY_'+ident(case+':key:'+str(i)) for i in range(4)] - values=['VAL_'+ident(case+':value:'+str(i)) for i in range(4)] - query='Return one JSON object and nothing else. For each requested key, copy its recorded value exactly. Requested keys: '+', '.join(keys)+'.' - message='Read this archive of independent records. Each record associates a key with a value. Ignore unrelated records.\nARCHIVE_START\n__ARCHIVE_BODY__\nARCHIVE_END\n'+query - template=tokenizer.apply_chat_template([{'role':'user','content':message}],tokenize=False,add_generation_prompt=True,enable_thinking=False) - assert template.count('__ARCHIVE_BODY__')==1 - prefix,suffix=map(encode,template.split('__ARCHIVE_BODY__')) - filler=''.join('\nRecord KEY_D'+ident(case+':decoy-key:'+str(i))+' has value VAL_D'+ident(case+':decoy-value:'+str(i))+'. This record is independent of all other records.\n' for i in range(128)) - filler_ids=encode(filler); tokens=list(prefix);placements=[];cursor=0 - def pad(count): - nonlocal cursor - assert count>=0 - for _ in range(count): - tokens.append(filler_ids[cursor%len(filler_ids)]);cursor+=1 - for i,percent in enumerate((10,35,65,90)): - position=length*percent//100 - pad(position-len(tokens)) - needle='\nRecord '+keys[i]+' has value '+values[i]+'.\n' - needle_ids=encode(needle);tokens.extend(needle_ids) - placements.append({'index':i,'target_percent':percent,'record_start_token':position,'record_tokens':len(needle_ids),'key':keys[i],'value':values[i]}) - pad(length-len(suffix)-len(tokens));tokens.extend(suffix) - assert len(tokens)==length - decoded=tokenizer.decode(tokens,skip_special_tokens=False) - assert all(decoded.count(k)==2 for k in keys) - assert all(decoded.count(v)==1 for v in values) - for row in placements: - start=row['record_start_token'];end=start+row['record_tokens'] - assert tokenizer.decode(tokens[start:end])=='\nRecord '+row['key']+' has value '+row['value']+'.\n' - token_sha=hashlib.sha256(json.dumps(tokens,separators=(',',':')).encode()).hexdigest() - row={'id':case,'input_tokens':length,'input_ids':tokens,'input_ids_sha256':token_sha,'expected':dict(zip(keys,values)),'placements':placements,'max_output_tokens':2048,'ignore_eos':False,'temperature':0,'thinking':False} - cases.append(row);manifest.append({k:v for k,v in row.items() if k!='input_ids'}) - with (out/'inputs.jsonl').open('x') as f: - for row in cases:f.write(json.dumps(row,separators=(',',':'))+'\n') - with (out/'inputs.jsonl.gz').open('xb') as f: - with gzip.GzipFile(filename='',fileobj=f,mode='wb',mtime=0) as z:z.write((out/'inputs.jsonl').read_bytes()) - report={'schema':'retrieval-proposal-v1','status':'PROPOSED_NOT_ACTIVATED','cases':manifest,'generator_sha256':digest(__file__),'inputs_sha256':digest(out/'inputs.jsonl'),'compressed_sha256':digest(out/'inputs.jsonl.gz'),'tokenizer_files':{p.name:digest(p) for p in sorted(model.iterdir()) if p.name in ('tokenizer.json','tokenizer_config.json','chat_template.jinja','special_tokens_map.json','config.json')},'transformers':importlib.metadata.version('transformers'),'extraction':'Strip whitespace; optionally unwrap one enclosing json code fence; json.loads entire remaining response; require exactly the four expected keys, all string values. Duplicate JSON keys are rejected. Exact string equality; no case folding or substring matching.','gate_status':'baseline qualification and owner approval pending'} - (out/'manifest.json').write_text(json.dumps(report,indent=2)+'\n') - print(json.dumps({k:v for k,v in report.items() if k!='cases'}),flush=True) - print(json.dumps({'prepared_cases':len(cases),'input_tokens':sum(len(r['input_ids']) for r in cases),'all_exact_lengths_and_placements':True}),flush=True) - - -if __name__=='__main__':main() diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/validator/diagnostics/prepare_retrieval_v2.py b/evidence/qwen38-h20-fp8-single-request-vllm032/validator/diagnostics/prepare_retrieval_v2.py deleted file mode 100644 index 3ecc24e0e..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/validator/diagnostics/prepare_retrieval_v2.py +++ /dev/null @@ -1,72 +0,0 @@ -"""Deterministic proposed retrieval inputs; preparation does not activate gates.""" -import argparse -import gzip -import hashlib -import importlib.metadata -import json -from pathlib import Path -from transformers import AutoTokenizer - - -def digest(path): - return hashlib.sha256(Path(path).read_bytes()).hexdigest() - - -def ident(text): - return hashlib.sha256(('inferrsi-retrieval-v2:'+text).encode()).hexdigest()[:12].upper() - - -def main(): - p=argparse.ArgumentParser() - p.add_argument('--model', required=True);p.add_argument('--output', required=True) - args=p.parse_args() - model=Path(args.model);out=Path(args.output);out.mkdir(parents=True,exist_ok=False) - assert digest(model/'config.json') == '74227dd615bf1ea975aa676bdf355a0379858c12f394b5365cd9dfa5fc2c70bc' - tokenizer=AutoTokenizer.from_pretrained(model,local_files_only=True) - encode=lambda text:tokenizer.encode(text,add_special_tokens=False) - cases=[];manifest=[] - for length in (8192,65536,131072): - for index in range(10): - case=f'retrieval-n{length}-s{index}' - keys=['KEY_'+ident(case+':key:'+str(i)) for i in range(4)] - values=['VAL_'+ident(case+':value:'+str(i)) for i in range(4)] - query='Return one JSON object and nothing else. For each requested key, copy its recorded value exactly. Requested keys: '+', '.join(keys)+'.' - message='Read this archive of independent records. Each record associates a key with a value. Ignore unrelated records.\nARCHIVE_START\n__ARCHIVE_BODY__\nARCHIVE_END\n'+query - template=tokenizer.apply_chat_template([{'role':'user','content':message}],tokenize=False,add_generation_prompt=True,enable_thinking=False) - assert template.count('__ARCHIVE_BODY__')==1 - prefix,suffix=map(encode,template.split('__ARCHIVE_BODY__')) - filler=''.join('\nRecord KEY_'+ident(case+':decoy-key:'+str(i))+' has value VAL_'+ident(case+':decoy-value:'+str(i))+'.\n' for i in range((length+15)//16+64)) - filler_ids=encode(filler); assert len(filler_ids)>=length; tokens=list(prefix);placements=[];cursor=0 - def pad(count): - nonlocal cursor - assert count>=0 - assert cursor+count<=len(filler_ids) - tokens.extend(filler_ids[cursor:cursor+count]);cursor+=count - for i,percent in enumerate((10,35,65,90)): - position=length*percent//100 - pad(position-len(tokens)) - needle='\nRecord '+keys[i]+' has value '+values[i]+'.\n' - needle_ids=encode(needle);tokens.extend(needle_ids) - placements.append({'index':i,'target_percent':percent,'record_start_token':position,'record_tokens':len(needle_ids),'key':keys[i],'value':values[i]}) - pad(length-len(suffix)-len(tokens));tokens.extend(suffix) - assert len(tokens)==length - decoded=tokenizer.decode(tokens,skip_special_tokens=False) - assert all(decoded.count(k)==2 for k in keys) - assert all(decoded.count(v)==1 for v in values) - for row in placements: - start=row['record_start_token'];end=start+row['record_tokens'] - assert tokenizer.decode(tokens[start:end])=='\nRecord '+row['key']+' has value '+row['value']+'.\n' - token_sha=hashlib.sha256(json.dumps(tokens,separators=(',',':')).encode()).hexdigest() - row={'id':case,'input_tokens':length,'input_ids':tokens,'input_ids_sha256':token_sha,'expected':dict(zip(keys,values)),'placements':placements,'max_output_tokens':2048,'ignore_eos':False,'temperature':0,'thinking':False} - cases.append(row);manifest.append({k:v for k,v in row.items() if k!='input_ids'}) - with (out/'inputs.jsonl').open('x') as f: - for row in cases:f.write(json.dumps(row,separators=(',',':'))+'\n') - with (out/'inputs.jsonl.gz').open('xb') as f: - with gzip.GzipFile(filename='',fileobj=f,mode='wb',mtime=0) as z:z.write((out/'inputs.jsonl').read_bytes()) - report={'schema':'retrieval-proposal-v2','status':'PROPOSED_NOT_ACTIVATED','cases':manifest,'generator_sha256':digest(__file__),'inputs_sha256':digest(out/'inputs.jsonl'),'compressed_sha256':digest(out/'inputs.jsonl.gz'),'tokenizer_files':{p.name:digest(p) for p in sorted(model.iterdir()) if p.name in ('tokenizer.json','tokenizer_config.json','chat_template.jinja','special_tokens_map.json','config.json')},'transformers':importlib.metadata.version('transformers'),'extraction':'Strip whitespace; optionally unwrap one enclosing json code fence; json.loads entire remaining response; require exactly the four expected keys, all string values. Duplicate JSON keys are rejected. Exact string equality; no case folding or substring matching. Complete-case success additionally requires the exact four-key schema; per-position counts use available parsed key values even if another key is absent or extra. Invalid JSON or duplicate keys score all positions false.','gate_status':'baseline qualification and owner approval pending'} - (out/'manifest.json').write_text(json.dumps(report,indent=2)+'\n') - print(json.dumps({k:v for k,v in report.items() if k!='cases'}),flush=True) - print(json.dumps({'prepared_cases':len(cases),'input_tokens':sum(len(r['input_ids']) for r in cases),'all_exact_lengths_and_placements':True}),flush=True) - - -if __name__=='__main__':main() diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/validator/diagnostics/retrieval_baseline.py b/evidence/qwen38-h20-fp8-single-request-vllm032/validator/diagnostics/retrieval_baseline.py deleted file mode 100644 index cbbe542e1..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/validator/diagnostics/retrieval_baseline.py +++ /dev/null @@ -1,200 +0,0 @@ -"""Fixed retrieval baseline qualification; proposed gates remain unactivated.""" -import argparse -import asyncio -import dataclasses -import hashlib -import importlib.metadata -import json -import os -from pathlib import Path -import signal -import socket -import subprocess -import sys -import time -import urllib.request - -import aiohttp -import vllm.benchmarks.lib.endpoint_request_func as client - - -class RecordingResponse: - def __init__(self, response, chunks): - self.response, self.chunks = response, chunks - self.status, self.reason, self.content = response.status, response.reason, self - - async def iter_any(self): - async for chunk in self.response.content.iter_any(): - self.chunks.append(chunk) - yield chunk - - -class RecordingContext: - def __init__(self, context, chunks): - self.context, self.chunks = context, chunks - - async def __aenter__(self): - return RecordingResponse(await self.context.__aenter__(), self.chunks) - - async def __aexit__(self, *args): - return await self.context.__aexit__(*args) - - -class RecordingSession: - def __init__(self, session, chunks): - self.session, self.chunks = session, chunks - - def post(self, *args, **kwargs): - return RecordingContext(self.session.post(*args, **kwargs), self.chunks) - - -def digest(path): - return hashlib.sha256(Path(path).read_bytes()).hexdigest() - - -def group_members(pgid): - rows = subprocess.check_output(['ps', '-eo', 'pid=,pgid='], text=True) - return [int(pid) for pid, group in (r.split() for r in rows.splitlines()) - if int(group) == pgid] - - -def terminate_owned(process): - forced = False - try: - os.killpg(process.pid, signal.SIGTERM) - except ProcessLookupError: - pass - try: - process.wait(timeout=30) - except subprocess.TimeoutExpired: - forced = True - if group_members(process.pid): - forced = True - try: - os.killpg(process.pid, signal.SIGKILL) - except ProcessLookupError: - pass - process.wait(timeout=10) - return {'exit_code': process.returncode, 'forced': forced, - 'owned_group_remaining': len(group_members(process.pid))} - - -def score_response(text, expected): - def unique(pairs): - result={} - for key,value in pairs: - if key in result: raise ValueError('duplicate_key') - result[key]=value - return result - body=text.strip() - if body.startswith('```json\n') and body.endswith('\n```'): - body=body[8:-4].strip() - try: - parsed=json.loads(body,object_pairs_hook=unique) - shape_valid=isinstance(parsed,dict) and set(parsed)==set(expected) and all(isinstance(v,str) for v in parsed.values()) - except (ValueError,TypeError): - parsed={};shape_valid=False - matches=[shape_valid and parsed.get(key)==value for key,value in expected.items()] - return {'shape_valid':shape_valid,'position_matches':matches,'complete':all(matches)} - - -async def main(): - p = argparse.ArgumentParser() - for name in ('model', 'fixture', 'output'): - p.add_argument('--' + name, required=True) - args = p.parse_args() - fixture_digest = '285ddc88444b082236aa11cc0469f9112b93225c660ed9cdc4a079316ca35db1' - client_digest = 'f31fae8c573ecc4bf8e31dae03823502e71404333e8111f4b535432758302b29' - assert digest(args.fixture) == fixture_digest - assert digest(client.__file__) == client_digest - assert digest(Path(args.model)/'config.json') == '74227dd615bf1ea975aa676bdf355a0379858c12f394b5365cd9dfa5fc2c70bc' - assert importlib.metadata.version('vllm') == '0.32.0' - cases = [json.loads(line) for line in Path(args.fixture).read_text().splitlines()] - assert len(cases) == 30 - assert [r['input_tokens'] for r in cases] == [8192]*10+[65536]*10+[131072]*10 - for row in cases: - assert len(row['input_ids']) == row['input_tokens'] - assert hashlib.sha256(json.dumps(row['input_ids'],separators=(',',':')).encode()).hexdigest() == row['input_ids_sha256'] - out = Path(args.output); out.mkdir(parents=True, exist_ok=False) - with (out/'observations.jsonl').open('x') as stream: - def emit(**row): - stream.write(json.dumps(row)+'\n'); stream.flush() - print(json.dumps({k: v for k, v in row.items() if k not in ('token_ids', 'itl')}), flush=True) - emit(phase='protocol', observer_sha256=digest(__file__), fixture_sha256=fixture_digest, - client_sha256=client_digest, cases=[r['id'] for r in cases], verdict='INCONCLUSIVE', - scope='retrieval_baseline_qualification_no_candidate_or_activated_gates', - versions={k: importlib.metadata.version(k) for k in - ('vllm', 'torch', 'transformers', 'flashinfer-python', 'triton', 'aiohttp')}, - server_settings={'max_model_len': 133120, 'prefix_caching': False, - 'language_model_only': True, 'other_optimization_flags': 'defaults'}) - with socket.socket() as sock: - sock.bind(('127.0.0.1', 0)); port = sock.getsockname()[1] - url = f'http://127.0.0.1:{port}' - command = [sys.executable, '-u', '-m', 'vllm.entrypoints.openai.api_server', - '--model', args.model, '--host', '127.0.0.1', '--port', str(port), - '--served-model-name', 'qwen38-fp8', '--max-model-len', '133120', - '--no-enable-prefix-caching', '--language-model-only'] - (out/'command.private.json').write_text(json.dumps(command)+'\n') - with (out/'server.private.log').open('x') as log: - start = time.monotonic() - process = subprocess.Popen(command, stdout=log, stderr=log, start_new_session=True) - try: - while True: - if process.poll() is not None: - raise RuntimeError('server_exited_before_readiness') - try: - with urllib.request.urlopen(url+'/health', timeout=2) as response: - assert response.status == 200 - break - except (OSError, AssertionError): - if time.monotonic()-start > 1200: - raise TimeoutError('startup_timeout') - await asyncio.sleep(1) - emit(phase='ready', startup_s=time.monotonic()-start) - async with aiohttp.ClientSession(timeout=aiohttp.ClientTimeout(total=180)) as session: - for index, case in enumerate(cases): - prompt = case['input_ids']; length=case['input_tokens']; cap=case['max_output_tokens'];chunks=[] - request = client.RequestFuncInput( - prompt=prompt, api_url=url+'/v1/completions', prompt_len=length, - output_len=cap, model='qwen38-fp8', ignore_eos=False, - extra_body={'temperature': 0, 'return_token_ids': True}) - result = await client.async_request_openai_completions(request, RecordingSession(session, chunks)) - wire = b''.join(chunks) - (out/f'{index}.sse').write_bytes(wire) - (out/f'{index}-client.json').write_text(json.dumps(dataclasses.asdict(result), indent=2)+'\n') - events = [json.loads(line[5:].strip()) for line in wire.splitlines() - if line.startswith(b'data:') and line[5:].strip() != b'[DONE]'] - choices = [c for e in events for c in e.get('choices', [])] - tokens = [t for c in choices for t in c.get('token_ids', [])] - usage = [e['usage'] for e in events if e.get('usage')] - echoes = [c['prompt_token_ids'] for c in choices if c.get('prompt_token_ids') is not None] - terminal = [c['finish_reason'] for c in choices if c.get('finish_reason')] - errors = [e for e in events if 'error' in e] - valid = (result.success and not result.error and not errors and echoes == [prompt] - and terminal in (['length'], ['stop']) and b'data: [DONE]' in wire - and len(usage) == 1 and 0 < len(tokens) <= cap and len(tokens) == result.output_tokens - and usage[0]['completion_tokens'] == len(tokens) - and usage[0]['prompt_tokens'] == length == result.prompt_len - and len(result.itl) == len(tokens)-1 and len(choices) == len(tokens) - and all(len(c.get('token_ids', [])) == 1 for c in choices)) - score = score_response(result.generated_text, case['expected']) - emit(phase='request', index=index, case=case['id'], score=score, generated_text=result.generated_text, input_tokens=length, - requested_output_tokens=cap, output_tokens=len(tokens), token_ids=tokens, - terminal=terminal, usage=usage, valid=valid, - client_success=result.success, client_error=bool(result.error), - wire_sha256=hashlib.sha256(wire).hexdigest(), - prompt_sha256=hashlib.sha256(json.dumps(prompt).encode()).hexdigest(), - ttft_s=result.ttft, latency_s=result.latency, itl=result.itl, - tpot_s=(result.latency-result.ttft)/(len(tokens)-1) if len(tokens)>1 else None) - if not valid: - raise RuntimeError('baseline_framing_or_accounting_invalid') - except BaseException as exc: - emit(phase='error', exception=type(exc).__name__, reason=str(exc) if isinstance(exc, (RuntimeError, TimeoutError)) else 'see_private_log') - raise - finally: - emit(phase='cleanup', **terminate_owned(process)) - emit(phase='complete', verdict='INCONCLUSIVE', scope='retrieval_baseline_qualification_only', elapsed_s=time.monotonic()-start) - - -if __name__=='__main__': - asyncio.run(main()) diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/validator/diagnostics/retrieval_baseline_v2.py b/evidence/qwen38-h20-fp8-single-request-vllm032/validator/diagnostics/retrieval_baseline_v2.py deleted file mode 100644 index f1eba5616..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/validator/diagnostics/retrieval_baseline_v2.py +++ /dev/null @@ -1,205 +0,0 @@ -"""Fixed retrieval baseline qualification; proposed gates remain unactivated.""" -import argparse -import asyncio -import dataclasses -import hashlib -import importlib.metadata -import json -import os -from pathlib import Path -import signal -import socket -import subprocess -import sys -import time -import urllib.request - -import aiohttp -import vllm.benchmarks.lib.endpoint_request_func as client - - -class RecordingResponse: - def __init__(self, response, chunks): - self.response, self.chunks = response, chunks - self.status, self.reason, self.content = response.status, response.reason, self - - async def iter_any(self): - async for chunk in self.response.content.iter_any(): - self.chunks.append(chunk) - yield chunk - - -class RecordingContext: - def __init__(self, context, chunks): - self.context, self.chunks = context, chunks - - async def __aenter__(self): - return RecordingResponse(await self.context.__aenter__(), self.chunks) - - async def __aexit__(self, *args): - return await self.context.__aexit__(*args) - - -class RecordingSession: - def __init__(self, session, chunks): - self.session, self.chunks = session, chunks - - def post(self, *args, **kwargs): - return RecordingContext(self.session.post(*args, **kwargs), self.chunks) - - -def digest(path): - return hashlib.sha256(Path(path).read_bytes()).hexdigest() - - -def group_members(pgid): - rows = subprocess.check_output(['ps', '-eo', 'pid=,pgid='], text=True) - return [int(pid) for pid, group in (r.split() for r in rows.splitlines()) - if int(group) == pgid] - - -def terminate_owned(process): - forced = False - try: - os.killpg(process.pid, signal.SIGTERM) - except ProcessLookupError: - pass - try: - process.wait(timeout=30) - except subprocess.TimeoutExpired: - forced = True - if group_members(process.pid): - forced = True - try: - os.killpg(process.pid, signal.SIGKILL) - except ProcessLookupError: - pass - process.wait(timeout=10) - return {'exit_code': process.returncode, 'forced': forced, - 'owned_group_remaining': len(group_members(process.pid))} - - -def score_response(text, expected): - def unique(pairs): - result={} - for key,value in pairs: - if key in result: raise ValueError('duplicate_key') - result[key]=value - return result - body=text.strip() - if body.startswith('```json\n') and body.endswith('\n```'): - body=body[8:-4].strip() - try: - parsed=json.loads(body,object_pairs_hook=unique) - shape_valid=isinstance(parsed,dict) and set(parsed)==set(expected) and all(isinstance(v,str) for v in parsed.values()) - except (ValueError,TypeError): - parsed={};shape_valid=False - matches=[isinstance(parsed,dict) and isinstance(parsed.get(key),str) and parsed.get(key)==value for key,value in expected.items()] - return {'shape_valid':shape_valid,'position_matches':matches,'complete':shape_valid and all(matches)} - - -async def main(): - p = argparse.ArgumentParser() - for name in ('model', 'fixture', 'output'): - p.add_argument('--' + name, required=True) - args = p.parse_args() - fixture_digest = '935b24411e897b0a1909fe94587ded8b809d7c6777cceac4e2decce8d239f75e' - client_digest = 'f31fae8c573ecc4bf8e31dae03823502e71404333e8111f4b535432758302b29' - assert digest(args.fixture) == fixture_digest - assert digest(client.__file__) == client_digest - assert digest(Path(args.model)/'config.json') == '74227dd615bf1ea975aa676bdf355a0379858c12f394b5365cd9dfa5fc2c70bc' - assert importlib.metadata.version('vllm') == '0.32.0' - cases = [json.loads(line) for line in Path(args.fixture).read_text().splitlines()] - assert len(cases) == 30 - assert [r['input_tokens'] for r in cases] == [8192]*10+[65536]*10+[131072]*10 - for row in cases: - assert len(row['input_ids']) == row['input_tokens'] - assert hashlib.sha256(json.dumps(row['input_ids'],separators=(',',':')).encode()).hexdigest() == row['input_ids_sha256'] - out = Path(args.output); out.mkdir(parents=True, exist_ok=False) - with (out/'observations.jsonl').open('x') as stream: - def emit(**row): - stream.write(json.dumps(row)+'\n'); stream.flush() - print(json.dumps({k: v for k, v in row.items() if k not in ('token_ids', 'itl')}), flush=True) - emit(phase='protocol', observer_sha256=digest(__file__), fixture_sha256=fixture_digest, - client_sha256=client_digest, cases=[r['id'] for r in cases], verdict='INCONCLUSIVE', - scope='retrieval_baseline_qualification_no_candidate_or_activated_gates', - versions={k: importlib.metadata.version(k) for k in - ('vllm', 'torch', 'transformers', 'flashinfer-python', 'triton', 'aiohttp')}, - server_settings={'max_model_len': 133120, 'prefix_caching': False, - 'language_model_only': True, 'other_optimization_flags': 'defaults', 'cpu_threads': 'vLLM_default_no_OMP_override'}) - with socket.socket() as sock: - sock.bind(('127.0.0.1', 0)); port = sock.getsockname()[1] - url = f'http://127.0.0.1:{port}' - command = [sys.executable, '-u', '-m', 'vllm.entrypoints.openai.api_server', - '--model', args.model, '--host', '127.0.0.1', '--port', str(port), - '--served-model-name', 'qwen38-fp8', '--max-model-len', '133120', - '--no-enable-prefix-caching', '--language-model-only'] - (out/'command.private.json').write_text(json.dumps(command)+'\n') - with (out/'server.private.log').open('x') as log: - start = time.monotonic() - process = subprocess.Popen(command, stdout=log, stderr=log, start_new_session=True) - try: - while True: - if process.poll() is not None: - raise RuntimeError('server_exited_before_readiness') - try: - with urllib.request.urlopen(url+'/health', timeout=2) as response: - assert response.status == 200 - break - except (OSError, AssertionError): - if time.monotonic()-start > 1200: - raise TimeoutError('startup_timeout') - await asyncio.sleep(1) - emit(phase='ready', startup_s=time.monotonic()-start) - async with aiohttp.ClientSession(timeout=aiohttp.ClientTimeout(total=180)) as session: - for index, case in enumerate(cases): - prompt = case['input_ids']; length=case['input_tokens']; cap=case['max_output_tokens'];chunks=[] - request = client.RequestFuncInput( - prompt=prompt, api_url=url+'/v1/completions', prompt_len=length, - output_len=cap, model='qwen38-fp8', ignore_eos=False, - extra_body={'temperature': 0, 'return_token_ids': True}) - result = await client.async_request_openai_completions(request, RecordingSession(session, chunks)) - wire = b''.join(chunks) - (out/f'{index}.sse').write_bytes(wire) - (out/f'{index}-client.json').write_text(json.dumps(dataclasses.asdict(result), indent=2)+'\n') - data_lines = [line[5:].strip() for line in wire.splitlines() if line.startswith(b'data:')] - done_unique_last = bool(data_lines) and data_lines.count(b'[DONE]') == 1 and data_lines[-1] == b'[DONE]' - events = [json.loads(line[5:].strip()) for line in wire.splitlines() - if line.startswith(b'data:') and line[5:].strip() != b'[DONE]'] - choices = [c for e in events for c in e.get('choices', [])] - tokens = [t for c in choices for t in c.get('token_ids', [])] - usage = [e['usage'] for e in events if e.get('usage')] - echoes = [c['prompt_token_ids'] for c in choices if c.get('prompt_token_ids') is not None] - terminal = [c['finish_reason'] for c in choices if c.get('finish_reason')] - errors = [e for e in events if 'error' in e] - valid = (result.success and not result.error and not errors and echoes == [prompt] - and terminal in (['length'], ['stop']) and done_unique_last - and len(usage) == 1 and 0 < len(tokens) <= cap and len(tokens) == result.output_tokens - and usage[0]['completion_tokens'] == len(tokens) - and usage[0]['prompt_tokens'] == length == result.prompt_len - and len(result.itl) == len(choices)-1 and len(choices)>0 - and all(len(c.get('token_ids', [])) >= 1 for c in choices) - and usage[0]['total_tokens'] == length + len(tokens) - and choices[-1].get('finish_reason') in ('stop','length')) - score = score_response(result.generated_text, case['expected']) - emit(phase='request', index=index, case=case['id'], score=score, generated_text=result.generated_text, input_tokens=length, - requested_output_tokens=cap, output_tokens=len(tokens), token_ids=tokens, - terminal=terminal, usage=usage, valid=valid, - token_counts_per_choice=[len(c.get('token_ids',[])) for c in choices], done_unique_last=done_unique_last, - client_success=result.success, client_error=bool(result.error), - wire_sha256=hashlib.sha256(wire).hexdigest(), - prompt_sha256=hashlib.sha256(json.dumps(prompt).encode()).hexdigest(), - ttft_s=result.ttft, latency_s=result.latency, itl=result.itl, - tpot_s=(result.latency-result.ttft)/(len(tokens)-1) if len(tokens)>1 else None) - if not valid: - raise RuntimeError('baseline_framing_or_accounting_invalid') - except BaseException as exc: - emit(phase='error', exception=type(exc).__name__, reason=str(exc) if isinstance(exc, (RuntimeError, TimeoutError)) else 'see_private_log') - raise - finally: - emit(phase='cleanup', **terminate_owned(process)) - emit(phase='complete', verdict='INCONCLUSIVE', scope='retrieval_baseline_qualification_only', elapsed_s=time.monotonic()-start) - - -if __name__=='__main__': - asyncio.run(main()) diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/validator/diagnostics/retrieval_baseline_v3.py b/evidence/qwen38-h20-fp8-single-request-vllm032/validator/diagnostics/retrieval_baseline_v3.py deleted file mode 100644 index d91b3317c..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/validator/diagnostics/retrieval_baseline_v3.py +++ /dev/null @@ -1,207 +0,0 @@ -"""Fixed retrieval baseline qualification; proposed gates remain unactivated.""" -import argparse -import asyncio -import dataclasses -import hashlib -import importlib.metadata -import json -import os -from pathlib import Path -import signal -import socket -import subprocess -import sys -import time -import urllib.request - -import aiohttp -import vllm.benchmarks.lib.endpoint_request_func as client - - -class RecordingResponse: - def __init__(self, response, chunks): - self.response, self.chunks = response, chunks - self.status, self.reason, self.content = response.status, response.reason, self - - async def iter_any(self): - async for chunk in self.response.content.iter_any(): - self.chunks.append(chunk) - yield chunk - - -class RecordingContext: - def __init__(self, context, chunks): - self.context, self.chunks = context, chunks - - async def __aenter__(self): - return RecordingResponse(await self.context.__aenter__(), self.chunks) - - async def __aexit__(self, *args): - return await self.context.__aexit__(*args) - - -class RecordingSession: - def __init__(self, session, chunks): - self.session, self.chunks = session, chunks - - def post(self, *args, **kwargs): - return RecordingContext(self.session.post(*args, **kwargs), self.chunks) - - -def digest(path): - return hashlib.sha256(Path(path).read_bytes()).hexdigest() - - -def group_members(pgid): - rows = subprocess.check_output(['ps', '-eo', 'pid=,pgid='], text=True) - return [int(pid) for pid, group in (r.split() for r in rows.splitlines()) - if int(group) == pgid] - - -def terminate_owned(process): - forced = False - try: - os.killpg(process.pid, signal.SIGTERM) - except ProcessLookupError: - pass - try: - process.wait(timeout=45) - except subprocess.TimeoutExpired: - forced = True - if group_members(process.pid): - forced = True - try: - os.killpg(process.pid, signal.SIGKILL) - except ProcessLookupError: - pass - process.wait(timeout=10) - return {'exit_code': process.returncode, 'forced': forced, - 'owned_group_remaining': len(group_members(process.pid))} - - -def score_response(text, expected): - def unique(pairs): - result={} - for key,value in pairs: - if key in result: raise ValueError('duplicate_key') - result[key]=value - return result - body=text.strip() - if body.startswith('```json\n') and body.endswith('\n```'): - body=body[8:-4].strip() - try: - parsed=json.loads(body,object_pairs_hook=unique) - shape_valid=isinstance(parsed,dict) and set(parsed)==set(expected) and all(isinstance(v,str) for v in parsed.values()) - except (ValueError,TypeError): - parsed={};shape_valid=False - matches=[isinstance(parsed,dict) and isinstance(parsed.get(key),str) and parsed.get(key)==value for key,value in expected.items()] - return {'shape_valid':shape_valid,'position_matches':matches,'complete':shape_valid and all(matches)} - - -async def main(): - p = argparse.ArgumentParser() - for name in ('model', 'fixture', 'output'): - p.add_argument('--' + name, required=True) - args = p.parse_args() - assert 'OMP_NUM_THREADS' not in os.environ, 'explicit_OMP_override_not_allowed_in_this_probe' - fixture_digest = '935b24411e897b0a1909fe94587ded8b809d7c6777cceac4e2decce8d239f75e' - client_digest = 'f31fae8c573ecc4bf8e31dae03823502e71404333e8111f4b535432758302b29' - assert digest(args.fixture) == fixture_digest - assert digest(client.__file__) == client_digest - assert digest(Path(args.model)/'config.json') == '74227dd615bf1ea975aa676bdf355a0379858c12f394b5365cd9dfa5fc2c70bc' - assert importlib.metadata.version('vllm') == '0.32.0' - cases = [json.loads(line) for line in Path(args.fixture).read_text().splitlines()] - assert len(cases) == 30 - assert [r['input_tokens'] for r in cases] == [8192]*10+[65536]*10+[131072]*10 - for row in cases: - assert len(row['input_ids']) == row['input_tokens'] - assert hashlib.sha256(json.dumps(row['input_ids'],separators=(',',':')).encode()).hexdigest() == row['input_ids_sha256'] - out = Path(args.output); out.mkdir(parents=True, exist_ok=False) - with (out/'observations.jsonl').open('x') as stream: - def emit(**row): - stream.write(json.dumps(row)+'\n'); stream.flush() - print(json.dumps({k: v for k, v in row.items() if k not in ('token_ids', 'itl')}), flush=True) - emit(phase='protocol', observer_sha256=digest(__file__), fixture_sha256=fixture_digest, - client_sha256=client_digest, cases=[r['id'] for r in cases], verdict='INCONCLUSIVE', - scope='retrieval_baseline_qualification_no_candidate_or_activated_gates', - versions={k: importlib.metadata.version(k) for k in - ('vllm', 'torch', 'transformers', 'flashinfer-python', 'triton', 'aiohttp')}, - server_settings={'max_model_len': 133120, 'prefix_caching': False, - 'language_model_only': True, 'shutdown_timeout': 30, 'other_optimization_flags': 'defaults', 'cpu_thread_policy': 'vLLM_default_no_OMP_override', - 'omp_num_threads_env': os.environ.get('OMP_NUM_THREADS')}) - with socket.socket() as sock: - sock.bind(('127.0.0.1', 0)); port = sock.getsockname()[1] - url = f'http://127.0.0.1:{port}' - command = [sys.executable, '-u', '-m', 'vllm.entrypoints.openai.api_server', - '--model', args.model, '--host', '127.0.0.1', '--port', str(port), - '--served-model-name', 'qwen38-fp8', '--max-model-len', '133120', - '--no-enable-prefix-caching', '--language-model-only', '--shutdown-timeout', '30'] - (out/'command.private.json').write_text(json.dumps(command)+'\n') - with (out/'server.private.log').open('x') as log: - start = time.monotonic() - process = subprocess.Popen(command, stdout=log, stderr=log, start_new_session=True) - try: - while True: - if process.poll() is not None: - raise RuntimeError('server_exited_before_readiness') - try: - with urllib.request.urlopen(url+'/health', timeout=2) as response: - assert response.status == 200 - break - except (OSError, AssertionError): - if time.monotonic()-start > 1200: - raise TimeoutError('startup_timeout') - await asyncio.sleep(1) - emit(phase='ready', startup_s=time.monotonic()-start) - async with aiohttp.ClientSession(timeout=aiohttp.ClientTimeout(total=180)) as session: - for index, case in enumerate(cases): - prompt = case['input_ids']; length=case['input_tokens']; cap=case['max_output_tokens'];chunks=[] - request = client.RequestFuncInput( - prompt=prompt, api_url=url+'/v1/completions', prompt_len=length, - output_len=cap, model='qwen38-fp8', ignore_eos=False, - extra_body={'temperature': 0, 'return_token_ids': True}) - result = await client.async_request_openai_completions(request, RecordingSession(session, chunks)) - wire = b''.join(chunks) - (out/f'{index}.sse').write_bytes(wire) - (out/f'{index}-client.json').write_text(json.dumps(dataclasses.asdict(result), indent=2)+'\n') - data_lines = [line[5:].strip() for line in wire.splitlines() if line.startswith(b'data:')] - done_unique_last = bool(data_lines) and data_lines.count(b'[DONE]') == 1 and data_lines[-1] == b'[DONE]' - events = [json.loads(line[5:].strip()) for line in wire.splitlines() - if line.startswith(b'data:') and line[5:].strip() != b'[DONE]'] - choices = [c for e in events for c in e.get('choices', [])] - tokens = [t for c in choices for t in c.get('token_ids', [])] - usage = [e['usage'] for e in events if e.get('usage')] - echoes = [c['prompt_token_ids'] for c in choices if c.get('prompt_token_ids') is not None] - terminal = [c['finish_reason'] for c in choices if c.get('finish_reason')] - errors = [e for e in events if 'error' in e] - valid = (result.success and not result.error and not errors and echoes == [prompt] - and terminal in (['length'], ['stop']) and done_unique_last - and len(usage) == 1 and 0 < len(tokens) <= cap and len(tokens) == result.output_tokens - and usage[0]['completion_tokens'] == len(tokens) - and usage[0]['prompt_tokens'] == length == result.prompt_len - and len(result.itl) == len(choices)-1 and len(choices)>0 - and all(len(c.get('token_ids', [])) >= 1 for c in choices) - and usage[0]['total_tokens'] == length + len(tokens) - and choices[-1].get('finish_reason') in ('stop','length')) - score = score_response(result.generated_text, case['expected']) - emit(phase='request', index=index, case=case['id'], score=score, generated_text=result.generated_text, input_tokens=length, - requested_output_tokens=cap, output_tokens=len(tokens), token_ids=tokens, - terminal=terminal, usage=usage, valid=valid, - token_counts_per_choice=[len(c.get('token_ids',[])) for c in choices], done_unique_last=done_unique_last, - client_success=result.success, client_error=bool(result.error), - wire_sha256=hashlib.sha256(wire).hexdigest(), - prompt_sha256=hashlib.sha256(json.dumps(prompt).encode()).hexdigest(), - ttft_s=result.ttft, latency_s=result.latency, itl=result.itl, - tpot_s=(result.latency-result.ttft)/(len(tokens)-1) if len(tokens)>1 else None) - if not valid: - raise RuntimeError('baseline_framing_or_accounting_invalid') - except BaseException as exc: - emit(phase='error', exception=type(exc).__name__, reason=str(exc) if isinstance(exc, (RuntimeError, TimeoutError)) else 'see_private_log') - raise - finally: - emit(phase='cleanup', **terminate_owned(process)) - emit(phase='complete', verdict='INCONCLUSIVE', scope='retrieval_baseline_qualification_only', elapsed_s=time.monotonic()-start) - - -if __name__=='__main__': - asyncio.run(main()) diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/validator/proposals/check_paired_statistics_v1.py b/evidence/qwen38-h20-fp8-single-request-vllm032/validator/proposals/check_paired_statistics_v1.py deleted file mode 100644 index f2e433036..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/validator/proposals/check_paired_statistics_v1.py +++ /dev/null @@ -1,124 +0,0 @@ -import copy -import hashlib -import importlib.util -import json -from pathlib import Path - -p = Path(__file__).with_name("paired_statistics_v1.py") -spec = importlib.util.spec_from_file_location("proposal", p) -m = importlib.util.module_from_spec(spec) -spec.loader.exec_module(m) - - -def fixture(ratios): - return [ - { - "input_tokens": n, - "case": f"n{n}-s{i}", - "repetition": r, - "arm": a, - "valid": True, - "output_tokens": 2048, - "ttft_s": (ratios[i] if a == "candidate" else 1), - "latency_s": 20 * (ratios[i] if a == "candidate" else 1), - } - for n in m.LENGTHS - for i in range(10) - for r in m.REPETITIONS - for a in m.ARMS - ] - - -checks = [] - - -def check(name, condition): - assert condition, name - checks.append(name) - - -rows = fixture([1] * 10) -report = m.summarize(rows) -check( - "identity_proposed_pass_only", - len(report["metrics"]) == 6 - and all( - r["proposed_metric_verdict"] == "PASS" - and r["median_paired_ratio"] == 1 - and r["upper_one_sided_95"] == 1 - for r in report["metrics"] - ) - and report["campaign_verdict"] == "INCONCLUSIVE" - and report["activated"] is False, -) -check("bootstrap_repeatability", report == m.summarize(rows)) -check("row_order_invariant", report == m.summarize(list(reversed(rows)))) -report = m.summarize(fixture([1.04] * 10)) -check( - "slower_than_margin_fails", - all(r["proposed_metric_verdict"] == "FAIL" for r in report["metrics"]), -) -report = m.summarize(fixture([0.99] * 5 + [1.06] * 5)) -check( - "uncertainty_inconclusive", - all( - r["proposed_metric_verdict"] == "INCONCLUSIVE" - and abs(r["median_paired_ratio"] - 1.025) < 1e-12 - and r["upper_one_sided_95"] > 1.03 - for r in report["metrics"] - ), -) -mutations = { - "missing": rows[:-1], - "duplicate": rows + [rows[0]], - "wrong_top_level": {}, - "bad_record": rows + [[]], -} -for label, field, value in [ - ("huge_int", "ttft_s", 10**500), - ("nan", "ttft_s", float("nan")), - ("negative", "ttft_s", -1), - ("bad_length", "output_tokens", 2047), - ("bool_length", "output_tokens", True), - ("false_valid", "valid", False), - ("unexpected_case", "case", "n8192-s10"), - ("bad_identity_type", "repetition", True), - ("unhashable", "case", []), - ("zero_decode", "latency_s", 1), - ("over_timeout", "latency_s", 1801), -]: - altered = copy.deepcopy(rows) - altered[0][field] = value - mutations[label] = altered -altered = copy.deepcopy(rows) -altered[0]["ttft_s"] = 5e-324 -mutations["ratio_overflow"] = altered -for label, altered in mutations.items(): - result = m.summarize(altered) - check( - label, - bool(result["errors"]) - and not result["metrics"] - and result["campaign_verdict"] == "INCONCLUSIVE", - ) -check("stable_large_median", m.median([1e308, 1e308]) == 1e308) -try: - m.parse_records('[{"valid":false,"valid":true}]') -except ValueError: - check("duplicate_json_key_rejected", True) -else: - check("duplicate_json_key_rejected", False) -check("linear_quantile", m.quantile([0, 10], 0.95) == 9.5) -print( - json.dumps( - { - "verdict": "INCONCLUSIVE", - "scope": "synthetic_proposal_checks_only", - "validator_sha256": hashlib.sha256(p.read_bytes()).hexdigest(), - "checks_passed": len(checks), - "checks": checks, - "bootstrap_sample_sha256": m.summarize(rows)["bootstrap"]["sample_indices_sha256"], - }, - indent=2, - ) -) diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/validator/proposals/paired_statistics_v1.py b/evidence/qwen38-h20-fp8-single-request-vllm032/validator/proposals/paired_statistics_v1.py deleted file mode 100644 index 9ac4f4787..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/validator/proposals/paired_statistics_v1.py +++ /dev/null @@ -1,191 +0,0 @@ -"""Issue-scoped proposed metric reducer; cannot award campaign acceptance. - -Input is a JSON list of independently authenticated per-request records. -This reducer validates completeness and numeric fields only. Source, device, -protocol, timing integrity and quality require separate frozen validators. -""" - -import argparse -import hashlib -import json -import math -import random -from pathlib import Path - -LENGTHS = (8192, 65536, 131072) -REPETITIONS = (0, 1, 2) -ARMS = ("baseline", "candidate") -DRAWS = 10000 -SEED = 670032 -LIMIT = 1.03 - - -def quantile(values, fraction): - ordered = sorted(values) - position = (len(ordered) - 1) * fraction - left = math.floor(position) - right = math.ceil(position) - return ordered[left] + (ordered[right] - ordered[left]) * (position - left) - - -def median(values): - return quantile(values, 0.5) - - -def positive(value): - try: - return type(value) in (int, float) and math.isfinite(value) and value > 0 - except OverflowError: - return False - - -def parse_records(text): - def unique(pairs): - result = {} - for key, value in pairs: - if key in result: - raise ValueError("duplicate_json_key:" + key) - result[key] = value - return result - - return json.loads(text, object_pairs_hook=unique) - - -def summarize(rows): - output = { - "campaign_verdict": "INCONCLUSIVE", - "activated": False, - "scope": "proposed_statistics_only", - "metrics": [], - "errors": [], - } - expected = { - (length, f"n{length}-s{case}", rep, arm) - for length in LENGTHS - for case in range(10) - for rep in REPETITIONS - for arm in ARMS - } - indexed = {} - if not isinstance(rows, list): - output["errors"].append("records_must_be_list") - return output - for index, row in enumerate(rows): - if not isinstance(row, dict): - output["errors"].append(f"invalid_record:{index}") - continue - try: - key = (row["input_tokens"], row["case"], row["repetition"], row["arm"]) - identity_valid = ( - type(key[0]) is int - and type(key[2]) is int - and isinstance(key[1], str) - and isinstance(key[3], str) - ) - if not identity_valid or key not in expected or key in indexed: - raise ValueError("identity") - if ( - row.get("valid") is not True - or type(row.get("output_tokens")) is not int - or row["output_tokens"] != 2048 - or not positive(row.get("ttft_s")) - or not positive(row.get("latency_s")) - or row["latency_s"] <= row["ttft_s"] - or row["latency_s"] > 1800 - or not positive((row["latency_s"] - row["ttft_s"]) / 2047) - ): - raise ValueError("measurement") - indexed[key] = row - except (KeyError, TypeError, ValueError): - output["errors"].append(f"invalid_or_duplicate_record:{index}") - if set(indexed) != expected: - output["errors"].append(f"incomplete_matrix:{len(indexed)}/180") - if output["errors"]: - return output - rng = random.Random(SEED) - samples = [[rng.randrange(10) for _ in range(10)] for _ in range(DRAWS)] - output["bootstrap"] = { - "draws": DRAWS, - "seed": SEED, - "cluster": "input_id_all_three_repetitions", - "quantile": "linear", - "sample_indices_sha256": hashlib.sha256( - json.dumps(samples, separators=(",", ":")).encode() - ).hexdigest(), - } - for length in LENGTHS: - for metric in ("ttft_s", "tpot_s"): - arms = {arm: [] for arm in ARMS} - clusters = [] - - def value(row): - if metric == "ttft_s": - return row["ttft_s"] - return (row["latency_s"] - row["ttft_s"]) / (row["output_tokens"] - 1) - - for case in range(10): - ratios = [] - for rep in REPETITIONS: - measured = { - arm: value(indexed[(length, f"n{length}-s{case}", rep, arm)]) - for arm in ARMS - } - for arm in ARMS: - arms[arm].append(measured[arm]) - ratio = measured["candidate"] / measured["baseline"] - if not positive(ratio): - output["errors"].append("nonfinite_or_underflowed_ratio") - output["metrics"] = [] - return output - ratios.append(ratio) - clusters.append(ratios) - ratios = [ratio for cluster in clusters for ratio in cluster] - primary = median(ratios) - upper = quantile( - [median([ratio for i in sample for ratio in clusters[i]]) for sample in samples], - 0.95, - ) - verdict = "FAIL" if primary > LIMIT else ("INCONCLUSIVE" if upper > LIMIT else "PASS") - output["metrics"].append( - { - "input_tokens": length, - "metric": metric, - "paired_ratios_by_input": clusters, - "median_paired_ratio": primary, - "paired_ratio_p10": quantile(ratios, 0.1), - "paired_ratio_p90": quantile(ratios, 0.9), - "upper_one_sided_95": upper, - "limit": LIMIT, - "proposed_metric_verdict": verdict, - "arms": { - arm: { - "median": median(values), - "p10": quantile(values, 0.1), - "p90": quantile(values, 0.9), - } - for arm, values in arms.items() - }, - } - ) - return output - - -def main(): - parser = argparse.ArgumentParser(description=__doc__) - parser.add_argument("records", type=Path) - args = parser.parse_args() - try: - result = summarize(parse_records(args.records.read_text())) - except (ValueError, OSError) as error: - result = { - "campaign_verdict": "INCONCLUSIVE", - "activated": False, - "metrics": [], - "errors": [type(error).__name__ + ":invalid_input"], - } - print(json.dumps(result, indent=2, allow_nan=False)) - return int(bool(result["errors"])) - - -if __name__ == "__main__": - raise SystemExit(main()) diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/validator/statistics-8k-v1/check_paired_statistics_8k_v1.py b/evidence/qwen38-h20-fp8-single-request-vllm032/validator/statistics-8k-v1/check_paired_statistics_8k_v1.py deleted file mode 100644 index cb4e4cf8b..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/validator/statistics-8k-v1/check_paired_statistics_8k_v1.py +++ /dev/null @@ -1,142 +0,0 @@ -import copy -import hashlib -import importlib.util -import json -from pathlib import Path - -p = Path(__file__).with_name("paired_statistics_8k_v1.py") -spec = importlib.util.spec_from_file_location("proposal", p) -m = importlib.util.module_from_spec(spec) -spec.loader.exec_module(m) - - -def fixture(ratios): - return [ - { - "input_tokens": n, - "case": f"n{n}-s{i}", - "repetition": r, - "arm": a, - "valid": True, - "output_tokens": 2048, - "ttft_s": (ratios[i] if a == "candidate" else 1), - "latency_s": 20 * (ratios[i] if a == "candidate" else 1), - } - for n in m.LENGTHS - for i in range(10) - for r in m.REPETITIONS - for a in m.ARMS - ] - - -checks = [] - - -def check(name, condition): - assert condition, name - checks.append(name) - - -rows = fixture([1] * 10) -report = m.summarize(rows) -check( - "identity_metric_pass_only", - len(report["metrics"]) == 2 - and all( - r["metric_verdict"] == "PASS" - and r["median_paired_ratio"] == 1 - and r["upper_one_sided_95"] == 1 - for r in report["metrics"] - ) - and report["campaign_verdict"] == "INCONCLUSIVE" - and report["statistics_approved"] is True, -) -check("bootstrap_repeatability", report == m.summarize(rows)) -check("row_order_invariant", report == m.summarize(list(reversed(rows)))) -report = m.summarize(fixture([1.04] * 10)) -check( - "slower_than_margin_fails", - all(r["metric_verdict"] == "FAIL" for r in report["metrics"]), -) -report = m.summarize(fixture([0.99] * 5 + [1.06] * 5)) -check( - "uncertainty_inconclusive", - all( - r["metric_verdict"] == "INCONCLUSIVE" - and abs(r["median_paired_ratio"] - 1.025) < 1e-12 - and r["upper_one_sided_95"] > 1.03 - for r in report["metrics"] - ), -) - -check("matrix_size_8k_only", len(rows) == 60 and m.LENGTHS == (8192,)) -varied = copy.deepcopy(rows) -for row in varied: - if row["arm"] == "candidate": - ratio = 0.9 + int(row["case"].split("s")[1]) * 0.01 + row["repetition"] * 0.001 - row["ttft_s"] = ratio - row["latency_s"] = ratio * 20 -cluster_report = m.summarize(varied) -check("retains_three_repetitions_per_input", all( - len(clusters) == 10 and all(len(values) == 3 for values in clusters) - and all(abs(values[1]-values[0]-0.001) < 1e-12 - and abs(values[2]-values[1]-0.001) < 1e-12 for values in clusters) - for clusters in [metric["paired_ratios_by_input"] for metric in cluster_report["metrics"]] -)) -boundary = m.summarize(fixture([1.03] * 10)) -check("inclusive_margin", all(row["metric_verdict"] == "PASS" for row in boundary["metrics"])) - -mutations = { - "missing": rows[:-1], - "duplicate": rows + [rows[0]], - "wrong_top_level": {}, - "bad_record": rows + [[]], -} -for label, field, value in [ - ("huge_int", "ttft_s", 10**500), - ("nan", "ttft_s", float("nan")), - ("negative", "ttft_s", -1), - ("bad_length", "output_tokens", 2047), - ("bool_length", "output_tokens", True), - ("false_valid", "valid", False), - ("unexpected_case", "case", "n8192-s10"), - ("bad_identity_type", "repetition", True), - ("unhashable", "case", []), - ("zero_decode", "latency_s", 1), - ("unsupported_64k", "input_tokens", 65536), -]: - altered = copy.deepcopy(rows) - altered[0][field] = value - mutations[label] = altered -altered = copy.deepcopy(rows) -altered[0]["ttft_s"] = 5e-324 -mutations["ratio_overflow"] = altered -for label, altered in mutations.items(): - result = m.summarize(altered) - check( - label, - bool(result["errors"]) - and not result["metrics"] - and result["campaign_verdict"] == "INCONCLUSIVE", - ) -check("stable_large_median", m.median([1e308, 1e308]) == 1e308) -try: - m.parse_records('[{"valid":false,"valid":true}]') -except ValueError: - check("duplicate_json_key_rejected", True) -else: - check("duplicate_json_key_rejected", False) -check("linear_quantile", m.quantile([0, 10], 0.95) == 9.5) -print( - json.dumps( - { - "verdict": "INCONCLUSIVE", - "scope": "synthetic_approved_8k_reducer_checks_only", - "validator_sha256": hashlib.sha256(p.read_bytes()).hexdigest(), - "checks_passed": len(checks), - "checks": checks, - "bootstrap_sample_sha256": m.summarize(rows)["bootstrap"]["sample_indices_sha256"], - }, - indent=2, - ) -) diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/validator/statistics-8k-v1/component.sha256 b/evidence/qwen38-h20-fp8-single-request-vllm032/validator/statistics-8k-v1/component.sha256 deleted file mode 100644 index 3e2f5b519..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/validator/statistics-8k-v1/component.sha256 +++ /dev/null @@ -1,4 +0,0 @@ -aed136820cb121ce06cafc34e3dfdf6ed8d19d739bd01caf2449189bfdd8ab4b plan/approved-statistics-8k-v1.md -328571c178612273dcf5a6176973e659c4d1f93a5208db5b0a356f78b1e29269 validator/statistics-8k-v1/paired_statistics_8k_v1.py -a6cd987b8ad5935f31f782c0d15674214132f07078b56f2d9349acbd7d1b015e validator/statistics-8k-v1/check_paired_statistics_8k_v1.py -aa061fd32859ce78fc5b9de869e6d772545fc51225725b3904a44cee9eb520f6 validator/statistics-8k-v1/synthetic-checks.json diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/validator/statistics-8k-v1/paired_statistics_8k_v1.py b/evidence/qwen38-h20-fp8-single-request-vllm032/validator/statistics-8k-v1/paired_statistics_8k_v1.py deleted file mode 100644 index f43752fda..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/validator/statistics-8k-v1/paired_statistics_8k_v1.py +++ /dev/null @@ -1,190 +0,0 @@ -"""Owner-approved 8K metric reducer; cannot award campaign acceptance. - -Input is a JSON list of independently authenticated per-request records. -This reducer validates completeness and numeric fields only. Source, device, -protocol, timing integrity and quality require separate frozen validators. -""" - -import argparse -import hashlib -import json -import math -import random -from pathlib import Path - -LENGTHS = (8192,) -REPETITIONS = (0, 1, 2) -ARMS = ("baseline", "candidate") -DRAWS = 10000 -SEED = 670032 -LIMIT = 1.03 - - -def quantile(values, fraction): - ordered = sorted(values) - position = (len(ordered) - 1) * fraction - left = math.floor(position) - right = math.ceil(position) - return ordered[left] + (ordered[right] - ordered[left]) * (position - left) - - -def median(values): - return quantile(values, 0.5) - - -def positive(value): - try: - return type(value) in (int, float) and math.isfinite(value) and value > 0 - except OverflowError: - return False - - -def parse_records(text): - def unique(pairs): - result = {} - for key, value in pairs: - if key in result: - raise ValueError("duplicate_json_key:" + key) - result[key] = value - return result - - return json.loads(text, object_pairs_hook=unique) - - -def summarize(rows): - output = { - "campaign_verdict": "INCONCLUSIVE", - "statistics_approved": True, - "scope": "approved_8k_statistics_only", - "metrics": [], - "errors": [], - } - expected = { - (length, f"n{length}-s{case}", rep, arm) - for length in LENGTHS - for case in range(10) - for rep in REPETITIONS - for arm in ARMS - } - indexed = {} - if not isinstance(rows, list): - output["errors"].append("records_must_be_list") - return output - for index, row in enumerate(rows): - if not isinstance(row, dict): - output["errors"].append(f"invalid_record:{index}") - continue - try: - key = (row["input_tokens"], row["case"], row["repetition"], row["arm"]) - identity_valid = ( - type(key[0]) is int - and type(key[2]) is int - and isinstance(key[1], str) - and isinstance(key[3], str) - ) - if not identity_valid or key not in expected or key in indexed: - raise ValueError("identity") - if ( - row.get("valid") is not True - or type(row.get("output_tokens")) is not int - or row["output_tokens"] != 2048 - or not positive(row.get("ttft_s")) - or not positive(row.get("latency_s")) - or row["latency_s"] <= row["ttft_s"] - or not positive((row["latency_s"] - row["ttft_s"]) / 2047) - ): - raise ValueError("measurement") - indexed[key] = row - except (KeyError, TypeError, ValueError): - output["errors"].append(f"invalid_or_duplicate_record:{index}") - if set(indexed) != expected: - output["errors"].append(f"incomplete_matrix:{len(indexed)}/60") - if output["errors"]: - return output - rng = random.Random(SEED) - samples = [[rng.randrange(10) for _ in range(10)] for _ in range(DRAWS)] - output["bootstrap"] = { - "draws": DRAWS, - "seed": SEED, - "cluster": "input_id_all_three_repetitions", - "quantile": "linear", - "sample_indices_sha256": hashlib.sha256( - json.dumps(samples, separators=(",", ":")).encode() - ).hexdigest(), - } - for length in LENGTHS: - for metric in ("ttft_s", "tpot_s"): - arms = {arm: [] for arm in ARMS} - clusters = [] - - def value(row): - if metric == "ttft_s": - return row["ttft_s"] - return (row["latency_s"] - row["ttft_s"]) / (row["output_tokens"] - 1) - - for case in range(10): - ratios = [] - for rep in REPETITIONS: - measured = { - arm: value(indexed[(length, f"n{length}-s{case}", rep, arm)]) - for arm in ARMS - } - for arm in ARMS: - arms[arm].append(measured[arm]) - ratio = measured["candidate"] / measured["baseline"] - if not positive(ratio): - output["errors"].append("nonfinite_or_underflowed_ratio") - output["metrics"] = [] - return output - ratios.append(ratio) - clusters.append(ratios) - ratios = [ratio for cluster in clusters for ratio in cluster] - primary = median(ratios) - upper = quantile( - [median([ratio for i in sample for ratio in clusters[i]]) for sample in samples], - 0.95, - ) - verdict = "FAIL" if primary > LIMIT else ("INCONCLUSIVE" if upper > LIMIT else "PASS") - output["metrics"].append( - { - "input_tokens": length, - "metric": metric, - "paired_ratios_by_input": clusters, - "median_paired_ratio": primary, - "paired_ratio_p10": quantile(ratios, 0.1), - "paired_ratio_p90": quantile(ratios, 0.9), - "upper_one_sided_95": upper, - "limit": LIMIT, - "metric_verdict": verdict, - "arms": { - arm: { - "median": median(values), - "p10": quantile(values, 0.1), - "p90": quantile(values, 0.9), - } - for arm, values in arms.items() - }, - } - ) - return output - - -def main(): - parser = argparse.ArgumentParser(description=__doc__) - parser.add_argument("records", type=Path) - args = parser.parse_args() - try: - result = summarize(parse_records(args.records.read_text())) - except (ValueError, OSError) as error: - result = { - "campaign_verdict": "INCONCLUSIVE", - "statistics_approved": True, - "metrics": [], - "errors": [type(error).__name__ + ":invalid_input"], - } - print(json.dumps(result, indent=2, allow_nan=False)) - return int(bool(result["errors"])) - - -if __name__ == "__main__": - raise SystemExit(main()) diff --git a/evidence/qwen38-h20-fp8-single-request-vllm032/validator/statistics-8k-v1/synthetic-checks.json b/evidence/qwen38-h20-fp8-single-request-vllm032/validator/statistics-8k-v1/synthetic-checks.json deleted file mode 100644 index df51a6b05..000000000 --- a/evidence/qwen38-h20-fp8-single-request-vllm032/validator/statistics-8k-v1/synthetic-checks.json +++ /dev/null @@ -1,36 +0,0 @@ -{ - "verdict": "INCONCLUSIVE", - "scope": "synthetic_approved_8k_reducer_checks_only", - "validator_sha256": "328571c178612273dcf5a6176973e659c4d1f93a5208db5b0a356f78b1e29269", - "checks_passed": 27, - "checks": [ - "identity_metric_pass_only", - "bootstrap_repeatability", - "row_order_invariant", - "slower_than_margin_fails", - "uncertainty_inconclusive", - "matrix_size_8k_only", - "retains_three_repetitions_per_input", - "inclusive_margin", - "missing", - "duplicate", - "wrong_top_level", - "bad_record", - "huge_int", - "nan", - "negative", - "bad_length", - "bool_length", - "false_valid", - "unexpected_case", - "bad_identity_type", - "unhashable", - "zero_decode", - "unsupported_64k", - "ratio_overflow", - "stable_large_median", - "duplicate_json_key_rejected", - "linear_quantile" - ], - "bootstrap_sample_sha256": "f3dc88ecceb93a2bd68db9ada5acd635fec76425dcc68f7c5b0a2f8fa513ccce" -} diff --git a/tests/check_installed.py b/tests/check_installed.py index 93102e98b..9ac409b0c 100644 --- a/tests/check_installed.py +++ b/tests/check_installed.py @@ -540,7 +540,7 @@ def _inventory(root, prefix): **_inventory(files("inferrsi").joinpath("knowledge", "data"), "knowledge/data"), **_inventory(files("inferrsi").joinpath("harness", "data"), "harness/data"), } -assert len(inventory) == 927, f"installed resource count is {len(inventory)}, expected 927" +assert len(inventory) == 928, f"installed resource count is {len(inventory)}, expected 928" assert sum(name.startswith("harness/data/") for name in inventory) == len(REPO_TYPES) # Every pinned record path and every registered skill body resolves inside that # same enumeration, so the inventory and the records describe one resource set. diff --git a/tests/test_evidence_archives.py b/tests/test_evidence_archives.py new file mode 100644 index 000000000..09254df78 --- /dev/null +++ b/tests/test_evidence_archives.py @@ -0,0 +1,450 @@ +"""Offline adversarial archives and compatibility with both frozen pilots.""" + +import copy +import gzip +import hashlib +import io +import json +import tarfile +import urllib.error +from pathlib import Path + +import pytest + +from tools import evidence_archives as archive +from tools.validate_evidence import budget_assets, validate_inventory +from tools.verify_evidence_archive import main + +ROOT = Path(__file__).resolve().parents[1] + + +def make_tar(path, entries): + with tarfile.open(path, "w:gz") as target: + for name, payload, kind in entries: + member = tarfile.TarInfo(name) + member.type = kind + member.size = len(payload) if kind == tarfile.REGTYPE else 0 + member.linkname = "../escape" if kind != tarfile.REGTYPE else "" + target.addfile(member, io.BytesIO(payload)) + + +def identity(path, manifest): + manifest["archive"]["bytes"] = path.stat().st_size + manifest["archive"]["sha256"] = archive.sha256(path) + + +@pytest.fixture +def bundle(tmp_path): + files = { + "sample/control.jsonl": b'{"id":"failed","status":"failed"}\n', + "sample/config.json": b'{"samples":3}\n', + "sample/recompute.py": b'raise RuntimeError("must never execute")\n', + } + path = tmp_path / "sample-v1.tar.gz" + make_tar(path, [(name, data, tarfile.REGTYPE) for name, data in files.items()]) + manifest = { + "schema_version": 2, + "archive": { + "id": "sample-v1", + "file": path.name, + "url": "https://github.com/example/example/releases/download/v1/" + path.name, + "sha256": "0" * 64, + "bytes": 1, + "format": "tar+gzip", + "retention_owner": "fixture-owner", + "source_revision": "a" * 40, + "root": "sample", + "files": len(files), + "uncompressed_bytes": sum(map(len, files.values())), + "key_files": {"control": "sample/control.jsonl", "code": "sample/recompute.py"}, + }, + "verification": { + "trials": "results/trials.jsonl", + "summary": "results/summary.json", + "command": ["python", "validator/recompute.py", "--output", ""], + "absolute_tolerance": 1e-12, + "relative_tolerance": 0, + "inputs": [ + {"role": role, "path": name, "sha256": hashlib.sha256(files[name]).hexdigest()} + for role, name in zip(("control", "configuration", "validator"), files) + ], + }, + } + identity(path, manifest) + return path, manifest + + +@pytest.mark.parametrize("campaign,files", [("minicpm-vllm-seedtts", 56), ("h3-image-ascend", 156)]) +def test_same_reader_normalizes_frozen_pilots(campaign, files): + path = ROOT / "evidence" / campaign / "results/artifacts.json" + original = path.read_bytes() + normalized = archive.normalize_manifest(archive.read_json(path)) + assert normalized["archive"]["files"] == files + assert normalized["archive"]["format"] == "tar+gzip" + assert normalized["archive"]["root"] == campaign + assert normalized["verification"] is None # Frozen replay is campaign-specific. + assert path.read_bytes() == original + + +def test_conflicting_legacy_aliases_are_rejected(): + manifest = archive.read_json(ROOT / "evidence/h3-image-ascend/results/artifacts.json") + manifest["archive"]["files"] = manifest["archive"]["original_files"] + 1 + with pytest.raises(archive.VerificationError, match="conflicting files"): + archive.normalize_manifest(manifest) + + +@pytest.mark.parametrize( + "key,value", + [ + ("bytes", True), + ("bytes", 0), + ("sha256", "z" * 64), + ("retention_owner", " "), + ("format", "zip"), + ("file", "../archive.tar.gz"), + ("source_revision", "main"), + ("url", "https://user:fake@example.invalid/sample-v1.tar.gz"), + ("url", "https://example.invalid/sample-v1.tar.gz?fake=secret"), + ("url", "file:///sample-v1.tar.gz"), + ("key_files", {}), + ], +) +def test_bad_metadata_is_rejected(bundle, key, value): + _, manifest = bundle + manifest["archive"][key] = value + with pytest.raises(archive.VerificationError): + archive.normalize_manifest(manifest) + + +def test_unknown_version_and_missing_pins_fail(bundle): + _, manifest = bundle + manifest["schema_version"] = 3 + with pytest.raises(archive.VerificationError): + archive.normalize_manifest(manifest) + manifest["schema_version"] = 2 + manifest["verification"]["inputs"].pop() + with pytest.raises(archive.VerificationError, match="Pin control"): + archive.normalize_manifest(manifest) + + +def test_duplicate_json_keys_are_rejected(tmp_path): + path = tmp_path / "manifest.json" + path.write_text('{"schema_version":1,"schema_version":2}') + with pytest.raises(archive.VerificationError, match="Duplicate JSON"): + archive.read_json(path) + + +def test_malformed_gzip_is_a_named_failure(bundle): + path, manifest = bundle + path.write_bytes(b"not a gzip stream") + identity(path, manifest) + with pytest.raises(archive.VerificationError) as caught: + archive.verify_archive(path, archive.normalize_manifest(manifest)) + assert caught.value.code == "invalid_archive" + + +def test_verified_content_never_executes_or_extracts(bundle, tmp_path): + path, manifest = bundle + before = list(tmp_path.iterdir()) + result = archive.verify_archive(path, archive.normalize_manifest(manifest)) + assert result["content"] == "verified" + assert result["recomputation"] == "not_checked" + assert result["campaign_verdict"] == "unchanged" + assert list(tmp_path.iterdir()) == before + + +@pytest.mark.parametrize( + "key,value,code", [("bytes", 1, "bad_bytes"), ("sha256", "0" * 64, "bad_hash")] +) +def test_authenticates_before_parsing(bundle, monkeypatch, key, value, code): + path, manifest = bundle + manifest["archive"][key] = value + monkeypatch.setattr(archive.tarfile, "open", lambda **kw: pytest.fail("parsed before hash")) + with pytest.raises(archive.VerificationError) as caught: + archive.verify_archive(path, archive.normalize_manifest(manifest)) + assert caught.value.code == code + + +@pytest.mark.parametrize("name", ["/outside", "../outside", "a/../b", "a//b", "a\\b", "C:/file"]) +def test_unsafe_paths_rejected(bundle, name): + path, manifest = bundle + make_tar(path, [(name, b"x", tarfile.REGTYPE)]) + identity(path, manifest) + with pytest.raises(archive.VerificationError) as caught: + archive.verify_archive(path, archive.normalize_manifest(manifest)) + assert caught.value.code == "unsafe_path" + + +@pytest.mark.parametrize( + "kind", [tarfile.SYMTYPE, tarfile.LNKTYPE, tarfile.CHRTYPE, tarfile.FIFOTYPE] +) +def test_nonregular_members_rejected(bundle, kind): + path, manifest = bundle + make_tar(path, [("sample/link", b"", kind)]) + identity(path, manifest) + with pytest.raises(archive.VerificationError) as caught: + archive.verify_archive(path, archive.normalize_manifest(manifest)) + assert caught.value.code == "unsafe_member" + + +def test_duplicate_members_rejected(bundle): + path, manifest = bundle + make_tar(path, [("sample/a", b"x", tarfile.REGTYPE)] * 2) + identity(path, manifest) + with pytest.raises(archive.VerificationError, match="Duplicate archive"): + archive.verify_archive(path, archive.normalize_manifest(manifest)) + + +@pytest.mark.parametrize( + "limits", + [ + archive.Limits(compressed_bytes=1), + archive.Limits(expanded_bytes=64), + archive.Limits(members=1), + archive.Limits(seconds=-1), + ], +) +def test_resource_limits(bundle, limits): + path, manifest = bundle + with pytest.raises(archive.VerificationError) as caught: + archive.verify_archive(path, archive.normalize_manifest(manifest), limits) + assert caught.value.code == "resource_limit" + + +def test_hidden_tar_metadata_is_bounded(bundle): + path, manifest = bundle + with tarfile.open(path, "w:gz", format=tarfile.PAX_FORMAT) as target: + member = tarfile.TarInfo("sample/a") + member.pax_headers = {"comment": "x" * 100000} + target.addfile(member, io.BytesIO(b"")) + identity(path, manifest) + with pytest.raises(archive.VerificationError, match="Expanded archive limit"): + archive.verify_archive( + path, archive.normalize_manifest(manifest), archive.Limits(expanded_bytes=2048) + ) + + +def test_trailing_compressed_padding_is_bounded(bundle): + path, manifest = bundle + path.write_bytes(path.read_bytes() + gzip.compress(b"\0" * 100000)) + identity(path, manifest) + with pytest.raises(archive.VerificationError, match="Expanded archive limit"): + archive.verify_archive( + path, archive.normalize_manifest(manifest), archive.Limits(expanded_bytes=20000) + ) + + +def test_missing_key_file_and_wrong_input_are_distinct(bundle): + path, manifest = bundle + manifest["archive"]["key_files"]["missing"] = "sample/missing" + with pytest.raises(archive.VerificationError) as caught: + archive.verify_archive(path, archive.normalize_manifest(manifest)) + assert caught.value.code == "missing_file" + del manifest["archive"]["key_files"]["missing"] + manifest["verification"]["inputs"][0]["sha256"] = "0" * 64 + with pytest.raises(archive.VerificationError) as caught: + archive.verify_archive(path, archive.normalize_manifest(manifest)) + assert caught.value.code == "bad_hash" + + +def test_local_missing_archive_is_unavailable(bundle): + path, manifest = bundle + path.unlink() + with pytest.raises(archive.VerificationError) as caught: + archive.verify_archive(path, archive.normalize_manifest(manifest)) + assert caught.value.code == "unavailable" + + +@pytest.mark.parametrize( + "url", + [ + "http://github.com/a", + "https://example.invalid/a", + "https://user:fake@github.com/a", + "https://github.com:123/a", + ], +) +def test_redirects_reject_unsupported_destinations(url): + with pytest.raises(archive.VerificationError) as caught: + archive.ReleaseRedirect().redirect_request(None, None, 302, "", {}, url) + assert caught.value.code == "unsafe_redirect" + + +@pytest.mark.parametrize( + "status,code", [(401, "restricted_access"), (403, "restricted_access"), (404, "unavailable")] +) +def test_download_failure_classes(bundle, monkeypatch, tmp_path, status, code): + _, manifest = bundle + + class Opener: + def open(self, *args, **kwargs): + raise urllib.error.HTTPError("fixture", status, "fake private details", {}, None) + + monkeypatch.setattr(archive.urllib.request, "build_opener", lambda *args: Opener()) + with pytest.raises(archive.VerificationError) as caught: + archive.download_archive(archive.normalize_manifest(manifest), tmp_path / "download") + assert caught.value.code == code + assert "private details" not in str(caught.value) + + +def test_controlled_storage_metadata_is_valid_but_never_auto_fetched(bundle, monkeypatch): + path, manifest = bundle + manifest["archive"]["url"] = "https://storage.example.invalid/" + path.name + normalized = archive.normalize_manifest(manifest) + monkeypatch.setattr( + archive.urllib.request, "build_opener", lambda *args: pytest.fail("network") + ) + assert archive.verify_archive(path, normalized)["content"] == "verified" + with pytest.raises(archive.VerificationError) as caught: + archive.download_archive(normalized, path.parent / "download") + assert caught.value.code == "restricted_access" + + +def trial_rows(): + return [ + { + "id": str(i), + "configuration": {"batch": 1}, + "status": status, + "conclusion": "INCONCLUSIVE", + "artifact_id": "sample-v1", + "source_record": "control.jsonl#" + str(i), + } + for i, status in enumerate(("failed", "interrupted", "measured")) + ] + + +def results(root, rows=None, summary=None): + (root / "results").mkdir(parents=True) + (root / "results/trials.jsonl").write_text( + "\n".join(json.dumps(row) for row in (rows if rows is not None else trial_rows())) + "\n" + ) + (root / "results/summary.json").write_text(json.dumps(summary or {"n": 3, "median": 1.0})) + + +@pytest.mark.parametrize( + "mutation,code", + [ + ("omit", "trial_mismatch"), + ("duplicate", "trial_mismatch"), + ("status", "trial_mismatch"), + ("summary", "recompute_mismatch"), + ], +) +def test_replay_catches_missing_failed_trials_and_wrong_summary(bundle, tmp_path, mutation, code): + _, manifest = bundle + retained, generated = tmp_path / "retained", tmp_path / "generated" + rows = trial_rows() + if mutation == "omit": + rows.pop(0) + elif mutation == "duplicate": + rows.append(rows[0]) + elif mutation == "status": + rows[0]["status"] = "measured" + results(retained) + results(generated, rows, {"n": 3, "median": 2.0} if mutation == "summary" else None) + with pytest.raises(archive.VerificationError) as caught: + archive.compare_results(archive.normalize_manifest(manifest), retained, generated) + assert caught.value.code == code + + +def test_replay_ignores_row_order_and_allows_only_frozen_float_tolerance(bundle, tmp_path): + _, manifest = bundle + retained, generated = tmp_path / "retained", tmp_path / "generated" + results(retained) + results(generated, list(reversed(trial_rows())), {"n": 3, "median": 1.0 + 1e-13}) + result = archive.compare_results(archive.normalize_manifest(manifest), retained, generated) + assert result == {"recomputation": "outputs_match", "trials": 3} + for a, b in [(1, 1.0), (True, 1), (float("nan"), float("nan")), (1, 2)]: + assert not archive.compare_values(a, b, 10, 10) + + +def test_cli_default_is_offline_and_local_verification_reports_scope(bundle, monkeypatch, capsys): + path, manifest = bundle + source = path.parent / "artifacts.json" + source.write_text(json.dumps(manifest)) + monkeypatch.setattr( + archive.urllib.request, "build_opener", lambda *args: pytest.fail("network") + ) + assert main([str(source)]) == 0 + assert json.loads(capsys.readouterr().out)["content"] == "not_checked" + assert main([str(source), "--archive", str(path)]) == 0 + assert json.loads(capsys.readouterr().out)["recomputation"] == "not_checked" + path.write_bytes(b"tampered") + assert main([str(source), "--archive", str(path)]) == 1 + assert json.loads(capsys.readouterr().out)["failure"] == "bad_bytes" + + +def test_cli_compares_replay_and_classifies_missing_output(bundle, tmp_path, capsys): + path, manifest = bundle + retained, generated = tmp_path / "retained", tmp_path / "generated" + results(retained) + results(generated) + source = retained / "results/artifacts.json" + source.write_text(json.dumps(manifest)) + args = [str(source), "--archive", str(path), "--recomputed", str(generated)] + assert main(args) == 0 + assert json.loads(capsys.readouterr().out)["recomputation"] == "outputs_match" + (generated / "results/summary.json").unlink() + assert main(args) == 1 + assert json.loads(capsys.readouterr().out)["failure"] == "missing_file" + + +def test_replay_rejects_output_symlink_escape(bundle, tmp_path): + _, manifest = bundle + retained, generated = tmp_path / "retained", tmp_path / "generated" + results(retained) + results(generated) + target = generated / "results/summary.json" + target.unlink() + try: + target.symlink_to(retained / "results/summary.json") + except OSError: + pytest.skip("symlinks unavailable") + with pytest.raises(archive.VerificationError) as caught: + archive.compare_results(archive.normalize_manifest(manifest), retained, generated) + assert caught.value.code == "unsafe_path" + + +def test_inventory_format_validation_and_budget_projection_are_offline(monkeypatch): + monkeypatch.setattr( + archive.urllib.request, "build_opener", lambda *args: pytest.fail("network") + ) + report = validate_inventory(ROOT) + assert not report["errors"] + assert report["unavailable"] + view = budget_assets(ROOT) + assert set(view["bundles"]) == { + "minicpm-vllm-seedtts", "h3-image-ascend", + "h3-ref2va-910c-4logical-20261101-297", "qwen38-h20-fp8-single-request-vllm032", + } + for campaign, value in view["bundles"].items(): + original = archive.read_json(ROOT / "evidence" / campaign / "results/artifacts.json") + (asset,) = value["external_assets"] + assert asset == {key: original["archive"][key] for key in asset} + + +def test_inventory_rejects_duplicate_archive_authority(bundle, tmp_path): + _, manifest = bundle + campaign = tmp_path / "evidence/synthetic" + for segment in ("plan", "results", "patch", "validator"): + (campaign / segment).mkdir(parents=True) + (campaign / "results/artifacts.json").write_text(json.dumps(manifest)) + entry = { + "kind": "campaign", + "manifests": [ + { + "path": "results/artifacts.json", + "schema": "archive-sha256", + "field": "archive", + "artifact": "results/sample-v1.tar.gz", + } + ], + } + inventory = tmp_path / "tools/evidence_inventory.json" + inventory.parent.mkdir() + inventory.write_text(json.dumps({"version": 1, "bundles": {"synthetic": entry}})) + assert not validate_inventory(tmp_path)["errors"] + entry["external_assets"] = [copy.deepcopy(manifest["archive"])] + inventory.write_text(json.dumps({"version": 1, "bundles": {"synthetic": entry}})) + assert "duplicates authoritative" in str(validate_inventory(tmp_path)["errors"]) diff --git a/tests/test_evidence_inventory.py b/tests/test_evidence_inventory.py index 463026272..824424ef8 100644 --- a/tests/test_evidence_inventory.py +++ b/tests/test_evidence_inventory.py @@ -146,5 +146,5 @@ def test_checked_in_inventory(): result = validate_inventory(repo) assert result["errors"] == [] assert len(result["bundles"]) == 26 - assert result["checked_hashes"] == 560 + assert result["checked_hashes"] == 461 assert result["unavailable"] # Historical raw artifacts are not in this checkout. diff --git a/tests/test_model_evidence_replay.py b/tests/test_model_evidence_replay.py new file mode 100644 index 000000000..5cd1b56a8 --- /dev/null +++ b/tests/test_model_evidence_replay.py @@ -0,0 +1,159 @@ +"""Record-preserving compaction: failures, revisions, source identity and replay.""" + +import hashlib +import io +import json +import tarfile +from pathlib import Path + +import pytest + +from tools import evidence_archives as archive +from tools import rebuild_model_evidence as replay + + +def encoded(value): + return json.dumps(value).encode() + + +def test_repeated_observations_merge_without_erasing_revisions_or_retries(): + failed = {"trial": "first-attempt", "status": "failed"} + changed = {**failed, "reason": "missing output"} + retry = {"trial": "second-attempt", "status": "completed"} + originals = { + "results/early.json": encoded({"runs": [failed]}), + "results/later.json": encoded({"runs": [failed, changed, retry]}), + "results/anonymous-a.json": encoded({"status": "interrupted"}), + "results/anonymous-b.json": encoded({"status": "interrupted"}), + } + rows, sources = replay.build_index( + originals, {"campaign": "synthetic", "configuration": [], "integrity_manifests": []}, + "synthetic-v1", + ) + attempts = [row for row in rows if row["kind"] == "attempt_observation"] + assert len(attempts) == 3 + repeated = next(row for row in attempts if len(row["source_records"]) == 2) + assert repeated["status"] == "failed" + assert repeated["source_records"] == [ + "synthetic/results/early.json#/runs/0", "synthetic/results/later.json#/runs/0", + ] + assert sum(row["status"] == "interrupted" for row in rows) == 2 + assert set(sources) == set(originals) + + +def test_nested_cancelled_and_pending_jobs_remain_explicit(): + data = {"gpu_attempts": [{"job": 1, "state": "CANCELLED"}, + {"job": 2, "state": "PENDING"}, + {"status": "failed", "reason": "submission failed"}]} + rows, _ = replay.build_index( + {"results/jobs.json": encoded(data)}, + {"campaign": "synthetic", "configuration": [], "integrity_manifests": []}, "v1", + ) + assert {row["status"] for row in rows} == {"recorded", "CANCELLED", "PENDING", "failed"} + assert all(row["conclusion"].startswith("INCONCLUSIVE") for row in rows) + + +@pytest.fixture +def frozen(tmp_path): + originals = { + "plan/config.json": encoded({"repetitions": 3}), + "results/control.json": encoded({"trial": "attempt", "status": "failed", + "campaign_verdict": "INCONCLUSIVE", "samples": [1, 2, 9]}), + "validator/check.py": b'raise RuntimeError("never execute")\n', + } + originals["plan/frozen.json"] = encoded({ + "validator/check.py": hashlib.sha256(originals["validator/check.py"]).hexdigest(), + }) + projection = { + "campaign": "synthetic", "configuration": ["plan/config.json"], + "integrity_manifests": ["plan/frozen.json"], + "verdict": {"path": "results/control.json", "pointer": "/campaign_verdict"}, + "metrics": [{"name": "latency", "path": "results/control.json", "pointer": "/samples"}], + "recorded_results": [], "limitations": ["synthetic recorded observations only"], + } + snapshot = {"source_revision": "a" * 40, + "files": {p: hashlib.sha256(data).hexdigest() for p, data in originals.items()}} + members = {"synthetic/" + p: data for p, data in originals.items()} + members.update({"snapshot.json": encoded(snapshot), + "replay/projection.json": encoded(projection)}) + for module in (archive, replay): + path = Path(module.__file__) + members["replay/" + path.name] = path.read_bytes() + + def pack(): + path = tmp_path / "synthetic-v1.tar.gz" + with tarfile.open(path, "w:gz") as target: + for name, payload in members.items(): + member = tarfile.TarInfo(name) + member.size = len(payload) + target.addfile(member, io.BytesIO(payload)) + original_names = [name for name in members if name.startswith("synthetic/")] + manifest = { + "schema_version": 2, + "archive": { + "id": "synthetic-v1", "file": path.name, + "url": "https://example.invalid/" + path.name, + "sha256": hashlib.sha256(path.read_bytes()).hexdigest(), + "bytes": path.stat().st_size, "format": "tar+gzip", + "retention_owner": "fixture", "source_revision": "a" * 40, + "root": "synthetic", "files": len(original_names), + "uncompressed_bytes": sum(len(members[n]) for n in original_names), + "key_files": {"snapshot": "snapshot.json", "projection": "replay/projection.json"}, + }, + "verification": { + "command": ["python", "rebuild_model_evidence.py"], + "trials": "results/trials.jsonl", "summary": "results/summary.json", + "absolute_tolerance": 0, "relative_tolerance": 0, + "inputs": [{"role": role, "path": name, + "sha256": hashlib.sha256(members[name]).hexdigest()} + for role, name in [("control", "synthetic/results/control.json"), + ("configuration", "synthetic/plan/config.json"), + ("validator", "synthetic/validator/check.py")]], + }, + } + return path, manifest + + return members, pack + + +def test_replay_recomputes_from_verified_originals_without_executing_them(frozen, tmp_path): + members, pack = frozen + path, manifest = pack() + capture = {} + archive.verify_archive(path, archive.normalize_manifest(manifest), capture=capture) + assert capture == members + summary, rows = replay.replay(archive.normalize_manifest(manifest), path) + assert summary["campaign_verdict"] == "INCONCLUSIVE" + assert summary["recomputed_metrics"]["latency"]["distribution"]["median"] == 2 + assert rows[0]["status"] == "failed" + source = tmp_path / "artifacts.json" + source.write_text(json.dumps(manifest)) + output = tmp_path / "generated" + assert replay.main([str(source), "--archive", str(path), "--output", str(output)]) == 0 + original = (output / "results/summary.json").read_bytes() + assert replay.main([str(source), "--archive", str(path), "--output", str(output)]) == 1 + assert (output / "results/summary.json").read_bytes() == original + + +@pytest.mark.parametrize("mutation", ["changed_original", "missing_original", "changed_code"]) +def test_archive_identity_does_not_replace_original_or_replay_identity(frozen, mutation): + members, pack = frozen + if mutation == "changed_original": + members["synthetic/validator/check.py"] = b"changed" + elif mutation == "missing_original": + del members["synthetic/plan/frozen.json"] + else: + members["replay/rebuild_model_evidence.py"] = b"changed" + path, manifest = pack() + with pytest.raises(archive.VerificationError): + replay.replay(archive.normalize_manifest(manifest), path) + + +def test_failed_verification_never_exposes_partially_captured_members(frozen): + _, pack = frozen + path, manifest = pack() + manifest["verification"]["inputs"][0]["sha256"] = "0" * 64 + captured = {} + with pytest.raises(archive.VerificationError, match="SHA-256"): + archive.verify_archive(path, archive.normalize_manifest(manifest), capture=captured) + assert not captured diff --git a/tests/test_vllm_ascend_rc_skill.py b/tests/test_vllm_ascend_rc_skill.py index 53b4494a9..5df28b190 100644 --- a/tests/test_vllm_ascend_rc_skill.py +++ b/tests/test_vllm_ascend_rc_skill.py @@ -68,34 +68,28 @@ def git(*args): source.write_text("def fused_operation(): pass\n") git("add", ".") git("commit", "-m", "Introduce an operator") + first = git("rev-parse", "HEAD").strip() + (tmp_path / "README.md").write_text("Synthetic project\n") + git("add", ".") + git("commit", "-m", "Document the project") + source.write_text("def fused_operation(): return 'DeepSeek V4'\n") + git("add", ".") + git("commit", "-m", "Support DeepSeek V4 fusion") pin = git("rev-parse", "HEAD").strip() source.write_text("uncommitted work\n") before = git("status", "--porcelain") result = history_module().inventory(tmp_path, pin) - assert result["reachable_commits"] == result["candidate_commits"] == 1 - record, = result["commits"] - assert record["commit"] == pin - assert record["runtime_patch_match"] is True - assert record["topics"] == ["fusion"] + assert result["reachable_commits"] == 3 + assert result["candidate_commits"] == 2 + assert {row["commit"] for row in result["commits"]} == {first, pin} + for topic, count in result["topic_counts"].items(): + assert count == sum(topic in row["topics"] for row in result["commits"]) + assert result["topic_counts"]["fusion"] == 2 + assert result["topic_counts"]["deepseek-v4-dspark"] == 1 + assert all(row["runtime_patch_match"] for row in result["commits"]) assert git("status", "--porcelain") == before -def test_archived_history_preserves_separate_screening_and_semantic_levels(): - archive = Path(__file__).resolve().parents[1] / ( - "docs/reviews/vllm-ascend-0271rc1/history-inventory.json" - ) - inventory = json.loads(archive.read_text()) - commits = inventory["commits"] - assert len({row["commit"] for row in commits}) == inventory["candidate_commits"] - assert inventory["candidate_commits"] <= inventory["reachable_commits"] - for topic, count in inventory["topic_counts"].items(): - assert count == sum(topic in row["topics"] for row in commits) - reviewed = skill_resource(LIBRARY, NAME, "references/history-reviewed.json").read_text() - inspected = set(re.findall(r'"commit": "([a-f0-9]{40})"', reviewed)) - assert inspected <= {row["commit"] for row in commits} - assert inventory["commit"] == skill_metadata(LIBRARY, NAME)["sources"][0]["commit"] - - def test_history_screen_rejects_incomplete_history(monkeypatch, tmp_path): module = history_module() monkeypatch.setitem( diff --git a/tools/evidence_archives.py b/tools/evidence_archives.py new file mode 100644 index 000000000..c86b14a03 --- /dev/null +++ b/tools/evidence_archives.py @@ -0,0 +1,507 @@ +"""Shared offline manifest reader and bounded archive verification. + +Accept the two original v1 pilot layouts without rewriting frozen evidence. +New publications use v2. Nothing in this module executes archive content or +adjudicates a campaign verdict. +""" + +from __future__ import annotations + +import gzip +import hashlib +import json +import math +import re +import tarfile +import time +import urllib.error +import urllib.parse +import urllib.request +import zlib +from dataclasses import dataclass +from pathlib import Path + + +class VerificationError(ValueError): + def __init__(self, code, message): + self.code = code + super().__init__(message) + + +def require(condition, code, message): + if not condition: + raise VerificationError(code, message) + + +def safe_name(name): + require( + isinstance(name, str) + and bool(name) + and "\\" not in name + and ":" not in name + and not any(ord(char) < 32 for char in name) + and all(part not in ("", ".", "..") for part in name.split("/")), + "unsafe_path", + "Expected a safe relative POSIX path", + ) + return name + + +def parse_json(payload): + def pairs(items): + result = {} + for key, value in items: + require(key not in result, "metadata", "Duplicate JSON key") + result[key] = value + return result + + try: + return json.loads(payload, object_pairs_hook=pairs) + except ValueError as exc: + if isinstance(exc, VerificationError): + raise + raise VerificationError("metadata", "Unreadable or invalid JSON") from exc + + +def read_json(path): + try: + return parse_json(Path(path).read_text(encoding="utf-8")) + except FileNotFoundError as exc: + raise VerificationError("missing_file", "JSON file is unavailable") from exc + + +def sha256(path): + result = hashlib.sha256() + with Path(path).open("rb") as stream: + for block in iter(lambda: stream.read(1024 * 1024), b""): + result.update(block) + return result.hexdigest() + + +def digest(value): + require( + isinstance(value, str) and re.fullmatch(r"[0-9a-f]{64}", value), + "metadata", + "Expected a lowercase SHA-256 digest", + ) + return value + + +def positive_int(value): + require(type(value) is int and value > 0, "metadata", "Expected a positive integer") + return value + + +def text(value): + require(isinstance(value, str) and bool(value.strip()), "metadata", "Expected nonempty text") + return value + + +def alias(values, label): + found = [value for value in values if value is not None] + require( + bool(found) and all(value == found[0] for value in found), + "metadata", + f"Missing or conflicting {label}", + ) + return found[0] + + +def https_url(url): + try: + parsed = urllib.parse.urlsplit(text(url)) + require( + parsed.scheme == "https" + and bool(parsed.hostname) + and parsed.username is None + and parsed.password is None + and parsed.port in (None, 443) + and not parsed.query + and not parsed.fragment, + "metadata", + "Expected a credential-free HTTPS URL without query or fragment", + ) + except ValueError as exc: + raise VerificationError("metadata", "Invalid HTTPS URL") from exc + return parsed + + +def normalize_manifest(data): + """Return one v2-shaped view; never mutate the authoritative source document.""" + try: + require( + isinstance(data, dict) + and type(data.get("schema_version")) is int + and data["schema_version"] in (1, 2), + "metadata", + "Unsupported manifest version", + ) + version = data["schema_version"] + source = data["archive"] + require(isinstance(source, dict), "metadata", "Expected archive object") + fmt = source["format"] + require(fmt in ("tar.gz", "tar+gzip"), "metadata", "Unsupported archive format") + revision = alias( + [ + source.get("source_revision"), + source.get("snapshot_revision"), + data.get("source_revision"), + ], + "source revision", + ) + require( + isinstance(revision, str) and re.fullmatch(r"[0-9a-f]{40,64}", revision), + "metadata", + "Expected an immutable source revision", + ) + files = positive_int(alias([source.get("files"), source.get("original_files")], "files")) + size = positive_int( + alias( + [source.get("uncompressed_bytes"), source.get("original_bytes")], + "uncompressed bytes", + ) + ) + keys = alias([source.get("key_files"), data.get("key_files")], "key files") + require(isinstance(keys, dict) and bool(keys), "metadata", "Expected named key files") + # H3 v1 counts original files beneath its narrative directory; snapshot.json + # sits outside that root and is independently listed as a key file. + root = source.get("root") + require(root is None or isinstance(root, str), "metadata", "Invalid archive root") + if root is None and version == 1 and "original_files" in source: + root = safe_name(keys["historical_narrative"]).rsplit("/", 1)[0] + root = safe_name(root) if root else "" + normalized_keys = {} + for label, name in keys.items(): + text(label) + if version == 1 and "key_files" in data: + name = root + "/" + name if root else name + directory = isinstance(name, str) and name.endswith("/") + normalized_keys[label] = safe_name(name.rstrip("/") if directory else name) + if directory: + normalized_keys[label] += "/" + filename = safe_name(source["file"]) + require("/" not in filename, "metadata", "Archive file must be a basename") + url = text(source["url"]) + parsed = https_url(url) + require( + urllib.parse.unquote(parsed.path.rsplit("/", 1)[-1]) == filename, + "metadata", + "URL and filename disagree", + ) + archive = { + "id": text(source.get("id", filename)), + "file": filename, + "url": url, + "sha256": digest(source["sha256"]), + "bytes": positive_int(source["bytes"]), + "format": "tar+gzip", + "retention_owner": text(source["retention_owner"]), + "source_revision": revision, + "root": root, + "files": files, + "uncompressed_bytes": size, + "key_files": normalized_keys, + } + verification = data.get("verification") if version == 2 else None + if version == 2: + require( + fmt == "tar+gzip" and all(key in source for key in archive), + "metadata", + "v2 requires canonical archive fields", + ) + require(isinstance(verification, dict), "metadata", "v2 requires verification contract") + for name in ("trials", "summary"): + safe_name(verification[name]) + command = verification["command"] + require( + isinstance(command, list) and bool(command), "metadata", "Expected command argv" + ) + for part in command: + text(part) + inputs = verification["inputs"] + require( + isinstance(inputs, list) and bool(inputs), "metadata", "Expected input identities" + ) + require( + {item["role"] for item in inputs} >= {"control", "configuration", "validator"}, + "metadata", + "Pin control, configuration and validator inputs", + ) + names = [] + for item in inputs: + names.append(safe_name(item["path"])) + digest(item["sha256"]) + require(len(names) == len(set(names)), "metadata", "Duplicate verification input") + for name in ("absolute_tolerance", "relative_tolerance"): + value = verification[name] + require( + type(value) in (int, float) and math.isfinite(value) and value >= 0, + "metadata", + "Expected finite nonnegative comparison tolerance", + ) + return { + "schema_version": 2, + "source_schema_version": version, + "archive": archive, + "verification": verification, + } + except (KeyError, TypeError, AttributeError) as exc: + raise VerificationError("metadata", "Missing or malformed archive metadata") from exc + + +@dataclass(frozen=True) +class Limits: + compressed_bytes: int = 256 * 1024 * 1024 + expanded_bytes: int = 512 * 1024 * 1024 + members: int = 10000 + seconds: float = 120 + + +def deadline_check(deadline): + require(time.monotonic() <= deadline, "resource_limit", "Verification time limit exceeded") + + +class LimitedStream: + """Count tar headers, padding and PAX payloads as well as file contents.""" + + def __init__(self, source, limit, deadline): + self.source, self.limit, self.deadline, self.count = source, limit, deadline, 0 + + def read(self, size): + deadline_check(self.deadline) + block = self.source.read(min(size, self.limit - self.count + 1)) + self.count += len(block) + require(self.count <= self.limit, "resource_limit", "Expanded archive limit exceeded") + deadline_check(self.deadline) + return block + + +def verify_archive(path, manifest, limits=Limits(), *, capture=None): + """Hash before parsing; inspect member bytes without writing or executing them.""" + spec = manifest["archive"] + require(spec["bytes"] <= limits.compressed_bytes, "resource_limit", "Compressed size limit") + deadline = time.monotonic() + limits.seconds + try: + # One open handle prevents path replacement between authentication and parsing. + with Path(path).open("rb") as stream: + identity, size = hashlib.sha256(), 0 + while block := stream.read(1024 * 1024): + size += len(block) + require(size <= spec["bytes"], "bad_bytes", "Archive byte count mismatch") + deadline_check(deadline) + identity.update(block) + require(size == spec["bytes"], "bad_bytes", "Archive byte count mismatch") + require(identity.hexdigest() == spec["sha256"], "bad_hash", "Archive SHA-256 mismatch") + stream.seek(0) + names, identities, total, originals, original_bytes = set(), {}, 0, 0, 0 + captured = {} + with gzip.GzipFile(fileobj=stream) as uncompressed: + bounded = LimitedStream(uncompressed, limits.expanded_bytes, deadline) + with tarfile.open(fileobj=bounded, mode="r|") as archive: + for member in archive: + deadline_check(deadline) + name = safe_name(member.name) + require( + member.isfile() and not member.issparse(), + "unsafe_member", + "Only regular nonsparse files are supported", + ) + require(name not in names, "unsafe_member", "Duplicate archive member") + names.add(name) + total += member.size + require( + member.size >= 0 + and len(names) <= limits.members + and total <= limits.expanded_bytes, + "resource_limit", + "Member limit", + ) + member_hash, received = hashlib.sha256(), 0 + chunks = [] + with archive.extractfile(member) as content: + while block := content.read(1024 * 1024): + deadline_check(deadline) + received += len(block) + member_hash.update(block) + if capture is not None: + chunks.append(block) + require(received == member.size, "invalid_archive", "Truncated member") + identities[name] = member_hash.hexdigest() + if capture is not None: + captured[name] = b"".join(chunks) + if not spec["root"] or name.startswith(spec["root"] + "/"): + originals += 1 + original_bytes += received + # Validate gzip CRC and bound padding/trailing data after tar's end marker. + while bounded.read(1024 * 1024): + pass + except FileNotFoundError as exc: + raise VerificationError("unavailable", "Local archive unavailable") from exc + except (tarfile.TarError, EOFError, gzip.BadGzipFile, zlib.error) as exc: + raise VerificationError("invalid_archive", "Malformed archive") from exc + for name in spec["key_files"].values(): + present = ( + any(item.startswith(name) for item in names) if name.endswith("/") else name in names + ) + require(present, "missing_file", "Archive is missing a declared key file") + require( + originals == spec["files"] and original_bytes == spec["uncompressed_bytes"], + "bad_contents", + "Original file count or expanded byte count mismatch", + ) + if manifest["verification"]: + for item in manifest["verification"]["inputs"]: + require(item["path"] in identities, "missing_file", "Missing verification input") + require( + identities[item["path"]] == item["sha256"], + "bad_hash", + "Verification input SHA-256 mismatch", + ) + # Replay callers receive the exact authenticated bytes, only after all checks + # succeed. No second path open, extraction, or archived-code execution. + if capture is not None: + capture.update(captured) + return { + "content": "verified", + "members": len(names), + "original_files": originals, + "original_bytes": original_bytes, + "sha256": spec["sha256"], + "recomputation": "not_checked", + "campaign_verdict": "unchanged", + } + + +DOWNLOAD_HOSTS = { + "github.com", + "release-assets.githubusercontent.com", + "objects.githubusercontent.com", +} + + +def allowed_download(url): + try: + parsed = urllib.parse.urlsplit(url) + return ( + parsed.scheme == "https" + and parsed.hostname in DOWNLOAD_HOSTS + and parsed.username is None + and parsed.password is None + and parsed.port in (None, 443) + ) + except ValueError: + return False + + +class ReleaseRedirect(urllib.request.HTTPRedirectHandler): + def redirect_request(self, req, fp, code, msg, headers, newurl): + require(allowed_download(newurl), "unsafe_redirect", "Unsupported archive redirect") + return super().redirect_request(req, fp, code, msg, headers, newurl) + + +def download_archive(manifest, destination, limits=Limits()): + spec = manifest["archive"] + require( + re.fullmatch(r"https://github\.com/[^/]+/[^/]+/releases/download/[^/]+/[^/]+", spec["url"]), + "restricted_access", + "Use --archive for controlled storage; automatic download supports Releases", + ) + require(spec["bytes"] <= limits.compressed_bytes, "resource_limit", "Compressed size limit") + deadline = time.monotonic() + limits.seconds + try: + opener = urllib.request.build_opener(ReleaseRedirect()) + with opener.open(spec["url"], timeout=min(10, limits.seconds)) as response: + require( + allowed_download(response.url), "unsafe_redirect", "Unsupported download location" + ) + with Path(destination).open("xb") as sink: + received = 0 + while block := response.read(1024 * 1024): + deadline_check(deadline) + received += len(block) + require( + received <= spec["bytes"], "bad_bytes", "Download exceeds declared size" + ) + sink.write(block) + except urllib.error.HTTPError as exc: + code = "restricted_access" if exc.code in (401, 403) else "unavailable" + raise VerificationError(code, "Archive download failed") from exc + except (urllib.error.URLError, TimeoutError) as exc: + raise VerificationError("unavailable", "Archive download unavailable") from exc + + +def compare_values(left, right, absolute, relative): + """Exact structure/integers; finite floats use the contract's declared tolerance.""" + if type(left) is float and type(right) is float: + return ( + math.isfinite(left) + and math.isfinite(right) + and math.isclose(left, right, abs_tol=absolute, rel_tol=relative) + ) + if type(left) is not type(right): + return False + if isinstance(left, dict): + return left.keys() == right.keys() and all( + compare_values(left[key], right[key], absolute, relative) for key in left + ) + if isinstance(left, list): + return len(left) == len(right) and all( + compare_values(a, b, absolute, relative) for a, b in zip(left, right) + ) + return left == right + + +def compare_results(manifest, retained, regenerated): + """Compare outputs of an explicitly reviewed, separately run replay command.""" + contract = manifest["verification"] + require(contract is not None, "not_supported", "Use the frozen v1 campaign replay checker") + + def local(root, name): + path = Path(root).joinpath(safe_name(name)).resolve() + require( + path.is_relative_to(Path(root).resolve()), "unsafe_path", "Result path escapes root" + ) + return path + + def trials(path): + try: + rows = [parse_json(line) for line in path.read_text(encoding="utf-8").splitlines()] + ids = [text(row["id"]) for row in rows] + require(len(ids) == len(set(ids)), "trial_mismatch", "Duplicate trial ID") + for row in rows: + for key in ("status", "conclusion", "artifact_id", "source_record"): + text(row[key]) + require( + isinstance(row["configuration"], dict), "trial_mismatch", "Trial configuration" + ) + require( + row["artifact_id"] == manifest["archive"]["id"], + "trial_mismatch", + "Trial references a different archive", + ) + return dict(zip(ids, rows)) + except (KeyError, TypeError, ValueError) as exc: + if isinstance(exc, VerificationError): + raise + raise VerificationError("trial_mismatch", "Invalid trial index") from exc + + for key, reader, code in ( + ("trials", trials, "trial_mismatch"), + ("summary", read_json, "recompute_mismatch"), + ): + try: + expected = reader(local(retained, contract[key])) + actual = reader(local(regenerated, contract[key])) + except FileNotFoundError as exc: + raise VerificationError("missing_file", "Missing replay output") from exc + require( + compare_values( + expected, actual, contract["absolute_tolerance"], contract["relative_tolerance"] + ), + code, + f"Regenerated {key} differs", + ) + return { + "recomputation": "outputs_match", + "trials": len(trials(local(retained, contract["trials"]))), + } diff --git a/tools/evidence_inventory.json b/tools/evidence_inventory.json index 670362afa..b6e021fc0 100644 --- a/tools/evidence_inventory.json +++ b/tools/evidence_inventory.json @@ -46,9 +46,15 @@ "kind": "campaign", "manifests": [ { - "path": "results/publication-integrity.json", + "path": "results/core-integrity.json", "schema": "bundle-sha256" }, + { + "path": "results/artifacts.json", + "schema": "archive-sha256", + "field": "archive", + "artifact": "results/h3-ref2va-910c-4logical-20261101-297-6c7e237-v1.tar.gz" + }, { "path": "validator/frozen-snapshots.json", "schema": "bundle-sha256" @@ -154,13 +160,22 @@ }, "_template": {"kind": "template", "manifests": []}, "qwen38-h20-fp8-single-request-vllm032": { - "kind": "campaign", "manifests": [ - {"path": "plan/retrieval-integrity.json", "schema": "bundle-sha256"}, - {"path": "plan/search-8k-v1-integrity.json", "schema": "bundle-sha256"}, - {"path": "results/search-observations-2163-2175-integrity.json", "schema": "bundle-sha256"}, - {"path": "results/diagnostics/sandbox-offline-prefetch-integrity.json", "schema": "bundle-sha256"} + "kind": "campaign", + "manifests": [ + { + "path": "results/core-integrity.json", + "schema": "bundle-sha256" + }, + { + "path": "results/artifacts.json", + "schema": "archive-sha256", + "field": "archive", + "artifact": "results/qwen38-h20-fp8-single-request-vllm032-6c7e237-v1.tar.gz" + } ], - "external": ["Owner-approved 8K search protocol activated prospectively in Issue67; campaign remains INCONCLUSIVE. Private source wrappers, operational logs, model outputs and runtime identities remain in controlled task storage."] + "external": [ + "Owner-approved 8K search protocol activated prospectively in Issue67; campaign remains INCONCLUSIVE. Private source wrappers, operational logs, model outputs and runtime identities remain in controlled task storage." + ] }, "drama3-ascend-20261023": { "kind": "campaign", diff --git a/tools/rebuild_model_evidence.py b/tools/rebuild_model_evidence.py new file mode 100644 index 000000000..fcc4b8094 --- /dev/null +++ b/tools/rebuild_model_evidence.py @@ -0,0 +1,223 @@ +"""Rebuild compact published-record views from an explicitly verified archive. + +This indexes published observations, including revisions of the same attempt. +It cannot reconstruct unpublished controller records or qualify model behavior. +Projection rules and original files are authenticated inputs, never executed. +""" + +from __future__ import annotations + +import argparse +import hashlib +import json +import math +import statistics +from collections import Counter +from pathlib import Path + +if __package__: + from . import evidence_archives as archive +else: + import evidence_archives as archive + + +def canonical(value): + return json.dumps(value, sort_keys=True, separators=(",", ":"), ensure_ascii=False) + + +def pointer(value, path): + for part in path.removeprefix("/").split("/") if path else (): + part = part.replace("~1", "/").replace("~0", "~") + value = value[int(part)] if isinstance(value, list) else value[part] + return value + + +def walk(value, path=""): + if isinstance(value, dict): + yield path, value + for key, child in value.items(): + escaped = key.replace("~", "~0").replace("/", "~1") + yield from walk(child, path + "/" + escaped) + elif isinstance(value, list): + for index, child in enumerate(value): + yield from walk(child, path + "/" + str(index)) + + +def attempt_identity(value): + for key in ("trial", "native_run_id", "run", "job_id", "job"): + item = value.get(key) + if type(item) in (str, int) and str(item).strip(): + return f"{key}:{item}" + return None + + +def status(value): + for path in ( + "/status", "/run_state", "/scheduler_state", "/slurm_state", "/state", + "/execution", "/model_controller/status", "/scheduler/JobState", + ): + try: + result = pointer(value, path) + except (KeyError, TypeError): + continue + if isinstance(result, str) and result: + return result + return "recorded" # Missing execution status never implies completion. + + +def conclusion(value): + fields = ("campaign_verdict", "verdict", "quality_verdict", "duration_verdict", + "native_verdict", "decision", "scope") + result = {key: value[key] for key in fields if isinstance(value.get(key), str)} + return canonical(result) if result else "INCONCLUSIVE: no standalone verdict recorded" + + +def build_index(originals, projection, artifact_id): + rows, sources = {}, [] + for name, payload in sorted(originals.items()): + if not name.startswith("results/") or not name.endswith(".json"): + continue + if name in projection["integrity_manifests"]: + continue + document = archive.parse_json(payload) + sources.append(name) + for offset, value in walk(document): + identity = attempt_identity(value) + # Whole published reports remain discoverable even without an attempt + # ID. Explicit attempt collections also retain anonymous failures. + collection = offset.rsplit("/", 1)[0].rsplit("/", 1)[-1] + if (offset and identity is None + and collection not in ("runs", "attempts", "gpu_attempts")): + continue + ref = projection["campaign"] + "/" + name + "#" + offset + digest = hashlib.sha256(canonical(value).encode()).hexdigest() + # Only repeated observations of the SAME named attempt collapse. + # Revisions with different bytes, retries, and anonymous records stay. + key = (identity or ref, digest) + if key in rows: + rows[key]["source_records"].append(ref) + continue + row_id = hashlib.sha256(canonical(key).encode()).hexdigest() + rows[key] = { + "id": row_id, + "kind": "attempt_observation" if identity else "published_record", + "identity": identity, + "configuration": { + "record_sha256": digest, + "campaign_records": [projection["campaign"] + "/" + name + for name in projection["configuration"]], + }, + "status": status(value), + "conclusion": conclusion(value), + "artifact_id": artifact_id, + "source_record": ref, + "source_records": [ref], + } + return sorted(rows.values(), key=lambda row: row["id"]), sources + + +def distribution(samples): + archive.require( + isinstance(samples, list) and bool(samples) + and all(type(n) in (int, float) and math.isfinite(n) for n in samples), + "recompute_mismatch", "Expected a nonempty finite measurement group", + ) + return {"count": len(samples), "min": min(samples), "max": max(samples), + "median": statistics.median(samples), "mean": statistics.mean(samples), + "stdev": statistics.stdev(samples) if len(samples) > 1 else None} + + +def build_views(originals, projection, manifest): + def read(spec): + return pointer(archive.parse_json(originals[spec["path"]]), spec.get("pointer", "")) + + rows, sources = build_index(originals, projection, manifest["archive"]["id"]) + metrics = {} + for spec in projection["metrics"]: + samples = read(spec) + if "field" in spec: + samples = [row[spec["field"]] for row in samples + if all(row[key] == value for key, value in spec.get("where", {}).items())] + metrics[spec["name"]] = {"source": spec, "distribution": distribution(samples)} + summary = { + "schema_version": 2, + "source_revision": manifest["archive"]["source_revision"], + "artifact_id": manifest["archive"]["id"], + "campaign_verdict": read(projection["verdict"]), + "coverage": { + "original_files": len(originals), + "original_bytes": sum(map(len, originals.values())), + "indexed_result_documents": sources, + "index_entries": len(rows), + "named_attempt_identities": len({r["identity"] for r in rows if r["identity"]}), + "record_occurrences": sum(len(r["source_records"]) for r in rows), + "status_counts": dict(sorted(Counter(r["status"] for r in rows).items())), + "unit": "published record/attempt observation; not a count of independent trials", + }, + "recomputed_metrics": metrics, + "recorded_results": [ + {"source": spec, "value": read(spec)} for spec in projection["recorded_results"] + ], + "limitations": projection["limitations"], + } + return summary, rows + + +def replay(manifest, source): + contents = {} + archive.verify_archive(source, manifest, capture=contents) + keys = manifest["archive"]["key_files"] + snapshot = archive.parse_json(contents[keys["snapshot"]]) + projection = archive.parse_json(contents[keys["projection"]]) + root = manifest["archive"]["root"] + archive.require(projection["campaign"] == root, "metadata", "Projection campaign mismatch") + archive.require(snapshot["source_revision"] == manifest["archive"]["source_revision"], + "metadata", "Snapshot source mismatch") + originals = {name[len(root) + 1:]: data for name, data in contents.items() + if name.startswith(root + "/")} + archive.require(set(originals) == set(snapshot["files"]), "missing_file", + "Original snapshot membership differs") + for name, data in originals.items(): + archive.require(hashlib.sha256(data).hexdigest() == snapshot["files"][name], + "bad_hash", "Original file differs from frozen snapshot") + # Bind the code actually used for regeneration, without executing a script + # retrieved from the archive. The manifest pins these same archive members. + for script in (Path(__file__), Path(archive.__file__)): + archive.require(contents["replay/" + script.name] == script.read_bytes(), + "bad_hash", "Use the replay implementation pinned by this archive") + for name in projection["integrity_manifests"]: + mapping = archive.parse_json(originals[name]) + for path, expected in mapping.items(): + archive.require(path in originals, "missing_file", "Frozen manifest target missing") + archive.require(hashlib.sha256(originals[path]).hexdigest() == expected, + "bad_hash", "Frozen integrity check failed") + return build_views(originals, projection, manifest) + + +def main(argv=None): + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("manifest", type=Path) + parser.add_argument("--archive", required=True, type=Path) + parser.add_argument("--output", required=True, type=Path, + help="New directory, separate from retained results") + args = parser.parse_args(argv) + try: + manifest = archive.normalize_manifest(archive.read_json(args.manifest)) + summary, rows = replay(manifest, args.archive) + # Refuse any existing destination, including the retained evidence tree. + args.output.mkdir(parents=True, exist_ok=False) + results = args.output / "results" + results.mkdir() + (results / "summary.json").write_text(json.dumps(summary, indent=2) + "\n") + (results / "trials.jsonl").write_text("".join(canonical(row) + "\n" for row in rows)) + print(json.dumps({"recomputed_index_entries": len(rows), + "campaign_verdict": summary["campaign_verdict"]})) + return 0 + except (archive.VerificationError, OSError, KeyError, TypeError, ValueError) as exc: + print(json.dumps({"verification": "failed", "failure": getattr(exc, "code", "replay"), + "campaign_verdict": "unchanged"})) + return 1 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/tools/validate_evidence.py b/tools/validate_evidence.py index 067a93000..1271a4c2b 100644 --- a/tools/validate_evidence.py +++ b/tools/validate_evidence.py @@ -6,6 +6,13 @@ import re from pathlib import Path +if __package__: + from .evidence_archives import normalize_manifest + from .evidence_archives import read_json as read_archive_json +else: + from evidence_archives import normalize_manifest + from evidence_archives import read_json as read_archive_json + SEGMENTS = ("plan", "results", "patch", "validator") REQUIRED = { "template": [f"{part}/README.md" for part in SEGMENTS], @@ -78,6 +85,13 @@ def validate_manifest(repo, bundle, entry, unavailable): verify_file(local, expected) checked += 1 elif schema == "archive-sha256": + # The campaign document owns the metadata. Inventory entries remain + # pointers; validation never downloads or implies remote availability. + if "schema_version" in data and "archive" in data: + normalized = normalize_manifest(read_archive_json(path)) + if entry.get("field") != "archive": + raise ValueError("versioned archive inventory must reference field archive") + data = normalized if "field" in entry: data = data[entry["field"]] if not isinstance(data, dict): @@ -142,11 +156,20 @@ def validate_inventory(repo, inventory=None): json.loads(path.read_text(encoding="utf-8")) if ( "manifest" in path.name - or path.name in ("freeze.json", "frozen-compatibility.json") + or path.name in ( + "freeze.json", "frozen-compatibility.json", "artifacts.json" + ) ) and path.relative_to(bundle).as_posix() not in declared: errors.append(f"{name}: unclassified manifest {path.relative_to(bundle)}") for manifest in manifests: try: + if ( + manifest["path"] == "results/artifacts.json" + and entry.get("external_assets") + ): + raise ValueError( + "external_assets duplicates authoritative artifacts.json" + ) report["checked_hashes"] += validate_manifest( repo, bundle, manifest, unavailable ) @@ -168,10 +191,40 @@ def validate_inventory(repo, inventory=None): return report +def budget_assets(repo): + """Generate #328's legacy input shape from authority; do not persist a second index.""" + inventory = read_object(repo / "tools/evidence_inventory.json") + bundles = {} + for name, entry in inventory["bundles"].items(): + assets = [] + for manifest in entry["manifests"]: + if manifest["path"] != "results/artifacts.json": + continue + path = safe_path(safe_path(repo / "evidence", name), manifest["path"]) + record = normalize_manifest(read_archive_json(path))["archive"] + assets.append({ + key: record[key] for key in ("url", "sha256", "bytes", "retention_owner") + }) + if assets: + if entry.get("external_assets"): + raise ValueError("external_assets duplicates authoritative artifacts.json") + bundles[name] = {"external_assets": assets} + return {"bundles": bundles} + + def main(): parser = argparse.ArgumentParser(description=__doc__) parser.add_argument("--root", type=Path, default=Path(__file__).resolve().parents[1]) + parser.add_argument("--budget-assets", action="store_true", + help="Emit a derived, disposable external-assets view for the R6 checker") args = parser.parse_args() + if args.budget_assets: + try: + print(json.dumps(budget_assets(args.root.resolve()), indent=2)) + return 0 + except (OSError, ValueError, KeyError, TypeError) as exc: + print(json.dumps({"errors": [str(exc)]})) + return 1 report = validate_inventory(args.root) print(json.dumps(report, indent=2)) return int(bool(report["errors"])) diff --git a/tools/verify_evidence_archive.py b/tools/verify_evidence_archive.py new file mode 100644 index 000000000..e3629cc93 --- /dev/null +++ b/tools/verify_evidence_archive.py @@ -0,0 +1,89 @@ +"""Inspect metadata offline, or explicitly verify a local/downloaded archive.""" + +import argparse +import json +import tempfile +from pathlib import Path + +if __package__: + from .evidence_archives import ( + VerificationError, + compare_results, + download_archive, + normalize_manifest, + read_json, + verify_archive, + ) +else: + from evidence_archives import ( + VerificationError, + compare_results, + download_archive, + normalize_manifest, + read_json, + verify_archive, + ) + + +def main(argv=None): + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("manifest", type=Path, help="Authoritative results/artifacts.json") + source = parser.add_mutually_exclusive_group() + source.add_argument("--archive", type=Path, help="Already retrieved archive; no network access") + source.add_argument("--download", action="store_true", help="Explicitly fetch a public Release") + parser.add_argument( + "--recomputed", type=Path, help="Outputs from the reviewed v2 replay command" + ) + parser.add_argument( + "--bundle", type=Path, help="Retained bundle; default: manifest's parent/.." + ) + args = parser.parse_args(argv) + if args.recomputed and not (args.archive or args.download): + parser.error("--recomputed requires --archive or --download") + try: + manifest = normalize_manifest(read_json(args.manifest)) + result = { + "metadata": "valid", + "content": "not_checked", + "recomputation": "not_checked", + "campaign_verdict": "unchanged", + } + if args.archive or args.download: + with tempfile.TemporaryDirectory(prefix="evidence-verify-") as scratch: + path = args.archive + if args.download: + path = Path(scratch) / manifest["archive"]["file"] + download_archive(manifest, path) + result.update(verify_archive(path, manifest)) + if args.recomputed: + result.update( + compare_results( + manifest, + args.bundle or args.manifest.resolve().parents[1], + args.recomputed, + ) + ) + else: + result["normalized_manifest"] = manifest + print(json.dumps(result, indent=2)) + return 0 + except VerificationError as exc: + result = { + "verification": "failed", + "failure": exc.code, + "error": str(exc), + "campaign_verdict": "unchanged", + } + except OSError: + result = { + "verification": "failed", + "failure": "local_io", + "error": "Local file access failed", + "campaign_verdict": "unchanged", + } + print(json.dumps(result, indent=2)) + return 1 + + +if __name__ == "__main__": + raise SystemExit(main())