Skip to content

[Campaign] MiniMax-H3 Ref2VA Ascend NPU latency optimization with video/audio quality gates #293

Closed
1 of 1 issue completed

Description

@amy-why-3459

Current state: PR #294 merged on 2026-11-02 (919ed42c) with historical five-second evidence: 407.27 → 108.12 s HTTP end-to-end latency (73.45% lower) on eight 910B1 NPUs, all 14 parity comparisons passing. Amy now reports a requester-narrowed lossless 5 s / 15 s follow-up (scope record); Draft #318 (f325d3c8) publishes validator repairs and author-reported initial screening: one 5 s baseline completed, but the candidate failed before inference due to conflicting offload settings. The prior narrative describes corrected matched screening as queued; the new revision supplies no fresh hardware outcome, so no paired lossless speedup or campaign acceptance is established. The author's inventory repair merges the then-main 663f3b3 snapshot, preserves the union of its bundles with h3-ref2va-lossless-293, and replaces the shared inventory-census assertion with this bundle's two manifests / 13 digests plus full-tree validation. It reports 22 bundles / 443 checked hashes without importing unmerged #317/#321 bundles. Current-head draft smoke and declarations pass. The later Ready run at f325d3c8 also completed: pr-references, full-check on 3.10/3.11 and installed-package on 3.10/3.11 pass; kv-task-qualification is skipped. The original reviewer's scoped follow-up now clears the fixed shared-census defect at f325d3c: the full-tree validator remains required while this bundle's two manifests / 13 digests replace repository-wide fixed totals. The review records tested merge 2bc1209 combining this head with main 9dded7a, although PR base metadata still names 663f3b3. The combined tree with open #317/#321 was not tested; their registrations and the resulting tree still need preservation and validation during integration. This scoped software clearance establishes no new device, losslessness or final-combination acceptance.
Decision needed: none
Next action: Amy's existing follow-up tracks expected-duration audio validation, LPIPS weight identity, baseline repeatability and independent final-combination evidence at both durations. Review and qualification remain pending; the historical five-second result does not establish the new lossless outcome. Preserve the frozen scope/evidence below; the reported scope change applies to this follow-up and does not alter the separate 910C campaign. Final acceptance stays with the requester.

Goal and authority

User-requested campaign: minimize MiniMax-H3 Ref2VA delivered-video latency on Ascend NPU, retaining quality acceptance. The requester explicitly allows quantization and caches with quality gates. Codex acts as the implementation agent for the authenticated contributor. This authorizes this campaign only, not closure of other research issues or paid model evaluation.

Deliver accepted vLLM-Omni improvements in one PR, with each optimization's source and individually measured effect. Deliver reusable RSI experience and rejected experiments to InferRSI (the canonical successor of OmniRSI). No global optimality claim: report the best validated configuration within the declared search space.

Frozen initial scope

  • Baseline source: vllm-project/vllm-omni c7f56f0155953c7d4a77276a2f4fcbd844ab0da2.
  • Hardware: one node, 8 Ascend 910B1, 64 GiB/device, fixed physical allocation inside npu run; no unreserved workload.
  • Environment image: quay.io/ascend/vllm-omni:nightly-20261022-6b35bca, local image SHA256 7fc6c73bd19a6ef4ee8d6048009972234796a1fc1bd3067168a425343cc476f7. Runtime compatibility is currently being checked; a changed environment requires a new frozen RunSpec before comparisons.
  • Checkpoint: original MiniMaxAI/MiniMax-H3 Ref2VA BF16; local content identity to be recorded before formal measurements.
  • Inputs: official Ref2VA test prompt plus video SHA256 d6edc20ac82aa980c9cbd0b05b371f60d12dc8d018492e571b42de0086819c41 and audio SHA256 2a48d447b343b38a2c504d5281a0c97c688db0ba87931ed9282c598bd6752454. Downloaded content matches the pinned upstream accuracy test.
  • Workload: 1344x768, 24 FPS, output durations 5 s and 15 s, seed 0, 50 configured inference steps, flow shifts 12/3, concurrency 1. Seeds/reference variations are held-out checks before broader claims. Short-step bringup is diagnostic only.
  • Base serving: USP8/ring1, text TP8, VAE tile8, DLO, dense LaserAttention, original BF16. Startup, warmup, request upload-to-full-MP4 delivery, and stage timings remain separate.
  • Quality gates: decoded-video SSIM >=0.97 and PSNR >=34 dB (from upstream H3 reference test), LPIPS <=0.20, audio multi-resolution spectral cosine >=0.80 and RMS ratio in [0.50,2.00] (from upstream H3 quantization test). Apply to matched baseline/candidate outputs; these are campaign adoption of existing thresholds, not evidence that the original CUDA tests qualify NPU. Require matching frame count/dimensions/FPS, audio rate/channels/sample count, finite values and successful decoding. Record baseline rerun variance. Temporal/perceptual generalization remains a separate limitation.
  • Formal comparisons: alternating AB/BA, independent server processes, a warmup per workload/arm, at least three measured requests per workload/arm; fixed inputs/device/config except the named candidate. No estimated p95 from means; retain raw samples and report small sample limits.

Search and attribution

Profiling first, then one hypothesis per step: residency/transfer, fused operators/packing, reference-KV reuse, TeaCache/Cache-DiT, INT8 and RainFusion. Existing main features are configuration baselines, not newly authored code. Do not use T2VA-only FastH3 distillation for Ref2VA. Test individual effects and the final combination directly; do not add percentages from separate cohorts.

Sources: task benchmark selection #60, engineering benchmark scope #252, pinned vLLM-Omni NPU recipe, H3 accuracy/quantization tests, and InferRSI source-attributed H3 records. Imported CUDA/older NPU results remain unverified on this campaign.

Acceptance and delivery

  • Runnable baseline, source/model/environment manifests and lifecycle evidence.
  • Profiling-backed candidates with exact source attribution and rejected outcomes retained.
  • Independent video/audio quality checks and matched repeated E2E for every accepted candidate.
  • Direct original-baseline versus final-combination test at both durations.
  • One vLLM-Omni PR with individual effects and exact evidence scope.
  • InferRSI contribution using existing evidence/knowledge contracts; source and optimizer improvement claims separated.

Current status: INCONCLUSIVE, bringup underway, no optimization result claimed. Stop candidate promotion on failed quality, request failures, resource interference or incomplete process cleanup. Implementation readiness and campaign acceptance are separate; final acceptance remains with the requester. Any unavailable measurement is reported explicitly.

Activity

  1. added
    area:harnessHarness module: profile, A/B loop, quality gates, evidence reports
    kind:acceptanceCampaign acceptance issue: frozen gates + evidence tree; PASS/FAIL/INCONCLUSIVE
    on Oct 31, 2026
  2. Dong1017 commented on Oct 31, 2026

    @Dong1017
    Collaborator

    同步一下:我们这边也会做 4×910C 的 H3 Ref2VA campaign。

    当前这个 Issue 和 amy 的关联 PR #294 针对 单机 8×910B1(64 GiB/卡)。4×910C 的 baseline 目前还不清楚,我们会先做 preflight,确认环境、模型/代码版本、四卡运行配置与 baseline 的可运行条件,再讨论性能与质量对照。现阶段是计划,尚无 910C 实测结果。

    两边的环境、性能和质量证据会分开记录,不直接沿用 910B1 的结论;amy 现有 campaign 的归属和范围保持不变。4×910C 拟按完整 campaign 跟踪:baseline 建立、瓶颈分析、优化筛选、性能与质量对照、最终组合验证及交付沉淀;preflight/baseline 只是第一阶段,不是整个任务。独立子 Issue、具体工作负载矩阵与验收指标待进一步讨论。

  3. amy-why-3459 commented on Nov 2, 2026

    @amy-why-3459
    CollaboratorAuthor

    The requester has explicitly narrowed the next delivery to lossless Ref2VA acceleration with no audio-quality degradation, and requires both 5-second and 15-second outputs. This supersedes the earlier permission to select approximate caches/quantization for this follow-up; the historical five-second lossy evidence remains unchanged.

    The existing engine delivery remains vLLM-Omni #9698. Reusable validation and new evidence will be a follow-up InferRSI contribution, because #294 is already merged. InferRSI is the OmniRSI successor.

    Initial control: engine 922f1e9f1cb85b9f2b66d86e1eb500e6872b2dcb, native BF16, dense attention, 50 steps, no Cache-DiT, no reference-KV reuse, no quantization or sparse attention. Hardware remains eight 910B1 devices reserved together. Preserve the original source baseline c7f56f0155953c7d4a77276a2f4fcbd844ab0da2 for the final direct comparison; matched shutdown budgets must be disclosed.

    First search slice: exact weight placement/transfer, using the existing DLO rank-local and resident_layers implementation, with unchanged arithmetic and weights. Screen the current AllGather control against rank-local transfer and a bounded resident prefix before selecting a candidate. Configuration reuse is attributed to the existing implementation, not claimed as a new kernel. Any final candidate needs independent AB/BA runs at both durations, warmups, raw HTTP E2E samples, separate stage profiling, every-output quality and actual worker exits. Do not add isolated percentage gains.

    Quality work precedes promotion: add a new validator version without changing the old frozen bundle. Check audio/video coverage and timestamps against expected output length; authenticate actual LPIPS weight bytes before loading; preserve upstream media metrics. The old 0.80 audio cosine gate alone cannot demonstrate no degradation. Retained dense baseline repetitions already have different decoded PCM hashes, so bitwise media equality cannot be presumed even for baseline-vs-baseline. Establish current control repeatability, retain exact-equality diagnostics, and compare candidate deviations against matched control variability. If evidence cannot establish preservation, retain INCONCLUSIVE instead of describing a perceptual threshold as losslessness.

    Status: INCONCLUSIVE, preparation only. No new lossless speedup measured. This update does not request campaign closure or alter the independent 910C campaign.

    Agent-assisted-by: Codex

  4. zengchuang-hw commented on Nov 17, 2026

    @zengchuang-hw
    Contributor

    Independent requirements-to-code review of #293 at main 61303fa8150472dfbda62ec8d77908bc05c73f3b. Scope: separable requirements, actual validator code and CPU evidence-handling checks. This does not reconstruct or accept a later lossless contract from the original permissive proposal, and does not resume stopped work.

    Verified boundaries and gaps

    1. The retained comparator is a thresholded historical validator, not proof of a current lossless contract. evidence/h3-ref2va-ascend-293/validator/compare.py:152-168 combines SSIM, PSNR, LPIPS and audio spectral/RMS thresholds. It does not require decoded video/audio equality. Therefore its PASS cannot silently be promoted to later lossless acceptance. Resolve and freeze the currently authoritative search space, quality/equivalence definition, validator identity and stop/resume authority first. Do not rewrite historical validator bytes or treat their old thresholds as the governing follow-up criteria.

    2. Audio timeline/coverage needs an explicit check. The retained probe() at lines 24-54 checks video dimensions, average FPS, counted frames, audio sample rate and channel count, but does not inspect audio start time or duration. The subsequent comparator at lines 129-132 checks the two decoded audio arrays have equal shapes; equality between arms is not an independent expected-duration or synchronization check. I isolated the actual probe() function with mocked ffprobe data: the matched case passed and six geometry/format mismatches were rejected, while an altered audio start/duration was not rejected. This is a narrow probe-boundary result, not a claim that a real malformed clip passed the complete comparator. Before a new delivered-media qualification, freeze expected audio/video coverage and timestamp/alignment behavior and test shifted/truncated audio, rather than relying on spectral/RMS scores alone.

    3. The packaged smoke checks are narrower than campaign acceptance. validator/smoke.py generates synthetic small media and checks identity, silence rejection and wrong frame count. Those checks do not establish real H3 control repeatability, lossless preservation, both requested durations, or independent-server AB/BA behavior on eight 910B1 devices. Nor does any comparator result certify HTTP upload-to-complete-MP4 timing, clean worker exits, fixed allocation or absence of interference.

    What would resolve the acceptance gap

    A versioned authoritative contract and hash-bound inputs/runtime/validator; separately retained baseline repeatability and clean timing; individual source-attributed candidates and a directly measured final combination at both durations; raw successful/failed/rejected attempts plus worker-cleanup and allocation evidence. Preserve the original versus later-contract distinction. Do not add component percentage gains, infer p95 from means, generalize seed-0 results to other references, or substitute another device-count campaign. No new numeric target or scope acceptance is proposed here.

    Checks actually run

    Isolated pinned-main checkout:
    pytest -o addopts='' -q tests/test_model_evidence_replay.py tests/test_h3_evidence_archive.py tests/test_evidence_inventory.py
    → 45 passed, 0 skipped.

    Additional isolated production-probe() test: 8 assertions completed, using mocked ffprobe output: matched input accepted; six mismatches rejected; audio start/duration not checked. Source comments/docstrings were excluded from the additional code inspection.

    These are CPU archive/evidence and narrow validator checks, not model-quality or performance evidence. I did not run the packaged LPIPS smoke, real media replay, upstream inference, full suite, target devices, deployment or paid work. No code/frozen evidence changed. Current lossless qualification and E2E speed remain INCONCLUSIVE; preserve any existing stop and do not close the Issue on these checks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:harnessHarness module: profile, A/B loop, quality gates, evidence reportskind:acceptanceCampaign acceptance issue: frozen gates + evidence tree; PASS/FAIL/INCONCLUSIVE

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions