Skip to content

[Roadmap] vLLM-Omni Model Optimization #416

Open

Description

@amy-why-3459

Current state: vLLM-Omni model optimization roadmap, tracking RSI deliverables, their explicitly linked vLLM-Omni outputs, requested upstream references, and campaign acceptance separately. Status and PR-reported performance checked on 2026-11-17 (Asia/Shanghai) against current PR bodies/states and campaign issues/comments.
Decision needed: Confirm owners, hardware, and budgets for new acceptance campaigns and extensions. Existing tasks retain their recorded scope and decisions.
Next action: Follow the RSI evidence/knowledge PRs, their downstream engine deliveries, and acceptance gaps below; establish new baselines on revisions that already include merged optimizations.

Goal and scope

Optimize vLLM-Omni across image generation/editing, video/audio generation, TTS, and multimodal/Omni serving. Organize work by model × scenario × hardware. Deliver changes to vllm-project/vllm-omni, reproducible end-to-end measurements, quality validation, and deployment guidance. InferRSI supplies experiment tracking, evaluation, and reusable lessons; #316 / #61 remain the platform roadmaps.

Official implementations, Diffusers, SGLang-Omni, and standalone engines may provide references or comparisons. Their improvements count toward this roadmap when the corresponding work is delivered and qualified in vLLM-Omni. Pure-text Qwen3.8 (#53 / #54 / #67) and DeepSeek V4.1 Flash (#353) remain related InferRSI work outside this model table; their existing states and goals are unchanged.

Model status and optimization roadmap

Each row tracks a bounded optimization, experiment, or evidence/knowledge deliverable. Issue links the governing InferRSI task and RSI PR links only InferMatrixLab/InferRSI pull requests. vLLM-Omni PR links engine deliveries; explicitly labeled upstream references are included when requested without asserting an RSI campaign attribution.

Attribution as an RSI output requires an RSI task/evidence record naming the engine PR, a reciprocal campaign link, or an explicit upstream statement that InferRSI was used. A shared model name, PR author, imported source, or an existing upstream optimization is not enough. For the two-RTX-3090 H3-Image row, the attribution comes from the engine author's InferRSI-use declaration; its RSI link is clearly marked as a summary rather than a campaign export.

Legend: ✅️ Completed deliverable · 🚧 In progress · ⏳ Awaiting acceptance · 📋 Planned · 🧪 Baseline preparation · ⏸️ Stopped · ❌ Target unmet · ⚪ Closed / inconclusive.

Status describes the named RSI deliverable/campaign. Where RSI and engine PR states differ, both are stated in the status/notes. ✅️ on an evidence import or bounded engine change does not mean the whole model campaign passed. Now / Next / Later remains priority. TBD means a child issue is needed; — means no corresponding link was identified, not that the model is unsupported. Existing engine support can be checked in the supported-model list.

Performance column: Values below are author-reported measurements from the linked PRs, not new benchmarks run for this update. Each result retains its hardware, concurrency, metric and comparison scope. Negative latency/RTF percentages mean reductions; positive throughput percentages mean increases. Component-only, imported, lossy, exploratory and regressed results are labeled; independent comparisons are not additive. A PR merge or issue closure does not establish campaign PASS, and historical measurements do not automatically qualify a later merged revision. — means no corresponding PR-reported gain was identified.

Now — Image and video priorities

Model Optimization / deliverable Status Issue RSI PR vLLM-Omni PR Performance improvement reported in PR Notes / next step
Qwen-Image-2.1-Turbo B300 campaign evidence and BF16 optimization delivery ✅️ Completed #3 #2 #8739 4×B300, warm HTTP E2E: default→default 381.91→279.83 ms (−26.73%); symmem→symmem 374.65→264.19 ms (−29.48%). Default→opt-in symmem: −30.82%, including configuration change. Reported in engine #8739 at d57480f5; later heads were not rebenchmarked. Both PRs merged. The RSI campaign and downstream PR explicitly link to each other; acceptance disposition remains separate below.
Qwen-Image-2.1-Turbo Formal B300 campaign acceptance ⏳ Awaiting acceptance #3 #2 #8739 Historical RSI #2 cohort: 384.16→266.60 ms (−30.60%), 4×B300, native 8-step RGBA, median of five trial means. Separate revisions/protocol from the engine #8739 comparison above; do not count the two rows as separate cumulative gains. Performance evidence is published; the acceptance issue still has no recorded final acceptance decision.
Qwen-Image-2.1-Turbo Ascend conditioning/modulation and kernel optimization ✅️ Engine merged; ⚪ lifecycle inconclusive — #2 (historical knowledge import only) #8729 (upstream reference) 1×910B1, final merged PR: warm offline generation 2921.23→2272.83 ms (−22.20%), BF16, 1280×704, native 8-step, C1; A/B/B/A with two fresh processes per arm. Timer covers Omni.generate until PIL outputs return, excluding PNG encoding/writing; not HTTP latency. Separate historical 910B3 cohort: −12.96% repeated prompts / −12.58% new prompts at an earlier revision. Merged 2026-10-17; measured immediate parent 69d298af46 → squash merge b164fea797. 18/18 candidate RGBA images were pixel-identical to baseline. All four processes later exited 134, including both unchanged-parent runs, so clean lifecycle acceptance remains INCONCLUSIVE. Per-feature ablation effects are not additive. Earlier Ascend import evidence in RSI #2 is separate from this final-merge measurement.
Qwen-Image-2.1 standard Dedicated RSI E2E/quality optimization campaign 📋 Planned TBD — — — No corresponding PR-reported gain identified. Freeze the standard checkpoint/scheduler baseline; profile attention, prefix KV and VAE; qualify text rendering, edits and RGBA. No corresponding RSI delivery is identified yet.
MiniMax-H3-Image 4×910B1 VAE execution optimization and evidence ✅️ Completed delivery; ⚪ campaign closed #390 #394 #10184 Engine #10184, matched-runtime W1/W2/W3 HTTP medians: four-stream single-tile graphs −33.1% / −30.9% / −19.2%; opt-in batch32 decode + FP16 batch8 encode −35.7% / −38.2% / −23.4%. W1=4736×3552 generation; W2=1024² generation request; W3=1024² edit. Sequential screens at 5e579b70d, not measurements of the merged single-tile/FP32 defaults. RSI #394 and engine #10184 are merged; issue #390 is closed. Campaign evidence remains INCONCLUSIVE with all latency targets unmet. The latest exact-prompt-cache screen was not adopted; no fresh full-model gain is established for the merged defaults.
MiniMax-H3-Image Two-RTX-3090 offload/residency/text-reuse optimization 🚧 Upstream open; ✅️ RSI summary merged TBD — no dedicated RSI campaign issue identified #415 (summary only) #10243 2×RTX 3090, engine #10243: warm mean HTTP latency 36.624→14.589 s (−60.16%; 2.51×), six formal images in one paired deployment comparison; nine RGB pairs identical including warmups. Includes code and residency configuration; not isolated per-feature gains. Engine #10243 remains open and explicitly declares InferRSI use. RSI #415 is merged documentation, not a campaign export. The PR also reports a same-head cumulative ablation; independent repeated qualification remains incomplete.
MiniMax-H3-Image Same-card two-die 910C baseline/profiling ⏸️ Closed / not planned #396 — — — No controlled PR-reported optimization gain identified. Issue #396 is closed as not planned. No linked evidence/output PR or independent controlled baseline has been identified; historical shared-host observations are not an accepted optimization result.
MiniMax-H3 Ref2VA Historical eight-910B1 five-second evidence delivery ✅️ RSI evidence; 🚧 upstream draft #293 #294 #9698 8×910B1, 5 s Ref2VA: HTTP E2E 407.27→108.12 s (−73.45%; 3.77×) in RSI #294 / engine #9698. Existing lossy Cache-DiT profile, A/B/B/A; sampled peak HBM +3.10%. Not lossless or a 15 s result. RSI #294 is merged; engine #9698 remains draft/open; issue #293 is closed. Historical frozen quality gates passed for the lossy 5 s profile. The later lossless 5 s/15 s follow-up has a separate acceptance scope.
MiniMax-H3 Ref2VA Eight-910B1 lossless 5s/15s follow-up ⚪ RSI closed / inconclusive; 🚧 upstream draft #293 #318 #9698 No measured lossless speedup in RSI #318: 5 s baseline completed, candidate failed before inference. The historical −73.45% lossy result does not apply to this scope. RSI #318 is closed without merge; issue #293 is closed; engine #9698 remains draft/open. Lossless audio coverage, repeatability and 5 s/15 s E2E qualification remain incomplete. Closure does not establish acceptance.
MiniMax-H3 Ref2VA Four-910B1 lossless execution/offload campaign ⚪ RSI closed / inconclusive; 🚧 upstream draft #320 #321 #9698 No qualified gain in RSI #321 / engine #9698. Resident-layer diagnostic 173.211→365.884 s regressed and was rejected; planned ABBA incomplete. Shutdown and RoPE-placement fixes establish completion/lifecycle behavior, not faster inference. RSI #321 is closed without merge; issue #320 is closed; engine #9698 remains draft/open. No qualified lossless gain; keep this cohort separate from eight-device work and retain failed/rejected screens.
MiniMax-H3 Ref2VA Four-logical-device 910C historical campaign ⏸️ Stopped #297 #317 #9745 4 logical 910C devices, 5 s: 291.2478→286.6668 s (−1.5729%) in RSI #317 / engine #9745. One clean screening pair combining migration + C1 prefix attention; 50% target unmet. No new-base full-model timing or 15 s speedup. RSI evidence merged; its explicitly linked BF16 prefix-attention engine PR is draft/open. Campaign closed as not planned, target unmet, final verdict INCONCLUSIVE.
MiniMax-H3 T2VA Four-L20X lossless T2VA campaign 📋 Proposed / inconclusive #438 — — — No linked PR or qualified candidate gain. The ≥10% latency reduction is an acceptance target, not a result. New child of #416: BF16, 49 denoising evaluations, C1, 20 matched requests, decoded video/audio byte identity. Issue is open/proposed; ownership and baseline gate feasibility still need resolution before candidate acceptance.

Next — Speech and Omni optimization

Model Optimization / deliverable Status Issue RSI PR vLLM-Omni PR Performance improvement reported in PR Notes / next step
Qwen3-TTS Ascend optimization knowledge/evidence import ✅️ Completed #35 #36 — — RSI #36 reports a source-grounded knowledge import, with no new matched service A/B or vLLM-Omni speedup. Completed source-grounded import from the standalone Qwen3-TTS engine. It does not deliver a vLLM-Omni runtime port or establish a downstream gain.
Qwen3-TTS 1.7B Ascend codec/predictor runtime optimization (upstream reference) ✅️ Engine merged; ⚪ quality unresolved — No governing InferRSI campaign identified — #9068 (upstream reference) 1×910B1, 1.7B CustomVoice, C1/C16/C32: throughput 2.58× / 2.35× / 2.35× versus the historical no-PR baseline; mean RTF 0.538→0.211 / 0.880→0.392 / 1.239→0.545. Fresh same-card no-PR→seven-path throughput: 1.869→4.822 / 16.463→38.755 / 21.245→52.091 audio s/s. Two timed passes per cell; historical and fresh baselines are distinct. Merged 2026-11-16. Seven NPU optimization paths are enabled by default; measured at 7b0b24e01, not the final merged revision. Mandarin speaker similarity and some high-concurrency ASR results regressed; no quality threshold or listening acceptance was established. Mechanisms are adapted from the independent engine; no linked InferRSI campaign/export was identified. This runtime delivery is separate from the #35/#36 knowledge import and the RTX-4090 campaign.
Qwen3-TTS 0.6B Single-RTX-4090 optimization round ❌ Target unmet #75 — — — No output PR or qualified PR-reported gain. Issue-only rejected screens are not promoted to delivered improvements. Recorded FAIL; no validated source change or downstream PR was produced. Owner acceptance/closure remains pending.
Qwen3-TTS (0.6B / 1.7B) New RSI TTFA/RTF/concurrency optimization campaign 📋 Planned TBD — — — No corresponding PR-reported gain identified. Freeze a new scope before applying imported techniques. General upstream TTS optimizations are baseline/reference material, not counted as RSI output.
MiniCPM-o 4.5 Single-L20X search, quality evidence and CPU-reference speaker extraction ⚪ RSI closed / FAIL; ✅️ upstream merged #49 #51 #9188 1×L20X, engine #9188: TensorRT speaker + MPS throughput +20.5% at C16 / +25.4% at C32; C32 TTFA p50/p95 −21.8% / −17.6%, mean RTF −20.3%. Combined-profile gain at an earlier tested revision; speech-quality verdict FAIL. RSI #51 is closed without merge; engine #9188 is merged; issue #49 remains closed. Speech-quality acceptance is FAIL. The earlier automatic issue closure has an unresolved owner-disposition request; merged engine code does not resolve the campaign verdict.
MiniCPM-o 4.5 Single-L20X packed conditioning and opt-in CFM/speaker graphs ✅️ Engine merged; ❌ campaign target unmet #49 #51 (closed, unmerged) #9218 1×L20X, C32, same 50-frame window: throughput +15.55%, TTFA p50 −27.69%, mean RTF −15.34%. Incremental packed-conditioning screen on an already tuned FP16/speaker-graph parent, not total PR gain over main. Exploratory WER margins were exceeded. Engine #9218 explicitly cites #49/#51 and is merged. One measured tuning pass; current-head repeated service and held-out quality qualification were not established. Independent-engine throughput target remained unmet.
MiniCPM-o 4.5 Single-L20X Seed-TTS runtime and default-deployment optimization ✅️ Deliveries merged; ⚪ acceptance inconclusive #431 #435 #10331 1×L20X, C1/C16/C32: mean RTF −18.84% / −42.42% / −50.72% (0.07783→0.06316 / 0.34287→0.19744 / 0.65548→0.32300); throughput 2.94→3.66 / 10.62→18.51 / 11.46→22.99 requests/s; TTFA p95 229.0→189.7 / 1118.1→611.6 / 2253.0→789.5 ms. Five fresh-server repetitions; 12,800 successful measured requests; complete code + default-config comparison versus upstream main. Engine #10331 merged 2026-11-16; RSI #435 is merged and issue #431 is closed. Baseline e25c9682a7 → measured head 5a6c5377e1; later review fixes were not rebenchmarked. Formal speech-quality non-inferiority is not established and rare overlong outputs remain. The independent-engine target was not rerun; RSI #435 retains earlier development evidence separately.
MiniCPM-o 4.5 Two-910B3 Seed-TTS optimization against H100 reference ❌ Target unmet; issue open #417 — — — No linked PR-reported gain identified. Issue-only single-sweep improvements are not counted as a delivered PR result. New child of #416. Selected A2 RTF is 0.21128 / 0.31078 / 0.46732 at C1/C4/C8; every tier misses the frozen H100-relative ceiling. Single-sweep performance and a separate quality pair remain distinct; C4 WER regressed.
MiMo-Audio-7B-Instruct Dedicated RSI model/tokenizer and serving optimization campaign 📋 Planned TBD — — — No corresponding PR-reported gain identified. No corresponding RSI campaign/evidence PR or RSI-produced downstream PR is identified in this audit. Profile patch encoder, LLM, patch decoder and codec; general upstream graph/sampling PRs are not credited here.
Qwen3-Omni-30B-A3B Two-GPU streaming-speech optimization and evidence ⚪ RSI closed / inconclusive; 🚧 upstream draft #112 #113 #9283 2×L20X, engine #9283, C64: codec-autotuning throughput +7.79%, mean RTF −6.06%; TTFA p50 +7.12% (regression). One sequential A/B pilot; C1/C64 targets unmet. Both arms include fused staging, so its benefit is not isolated. RSI #113 is closed without merge; issue #112 is closed; engine #9283 remains draft/open. Full campaign acceptance remains INCONCLUSIVE with targets unmet; preserve the explicit throughput/first-audio tradeoff.
Qwen3-Omni-30B-A3B Campaign measurement prerequisites: audio timing, concurrency and JIT accounting ⚪ Closed without merge #112 #113 #9278, #9279, #9281 — Measurement/correctness fixes; no standalone inference-speedup claim. Engine #9278, #9279 and #9281 are closed without merge, as is RSI #113. Engine #9283 explicitly includes the #9278 audio-timing and #9281 JIT prerequisites; do not infer that closing the standalone PRs delivered every prerequisite.
Drama 3 RSI NPU readiness and validation-contract supplement ✅️ Completed #27 #28 — — RSI #28 reports readiness/validation guidance, not a runtime performance gain. Readiness/knowledge delivery merged; no NPU runtime port or measured performance gain is established by this PR.
Drama 3 Ascend full-decode graph optimization and evidence ✅️ Completed — (no separate acceptance issue identified) #132 #9308 2×910B1, C1, engine #9308 / RSI #132: throughput 17.31→22.40–22.91 frames/s (about +30%); mean RTF 1.237→0.947–0.962 (about −23%); TTFA p50 0.91→0.78–0.80 s. Model-level RTF targets remain unmet. Both PRs merged. RSI PR explicitly states it documents the same tuning pass behind this engine PR. Model-level verdict remains INCONCLUSIVE and RTF targets unmet; raw artifacts/revision pins are incomplete.
Drama 3 NPU normalization investigation/follow-up ⚪ Closed / inconclusive #166 — — — No linked implementation PR or measured E2E improvement. Issue #166 is closed. No linked RSI delivery or engine optimization PR is identified; runtime dispatch, numerical equivalence and E2E benefit remain unqualified.
Drama 3 Historical standalone-engine comparison ⏸️ Stopped #10 #9, #22 — RSI #22, standalone engine C64: throughput 4674.81→4534.13 frames/s (regression); the 1.5× target was not met. Not a vLLM-Omni improvement. RSI evidence/lessons merged; the recorded optimization was paused and the target remains unmet / INCONCLUSIVE. The standalone runtime is not a vLLM-Omni output PR.

Later — Cross-model optimization and transfer

Model Optimization / deliverable Status Issue RSI PR vLLM-Omni PR Performance improvement reported in PR Notes / next step
HunyuanImage-3.0 Historical DiT latency campaign ⚪ Closed / inconclusive #125 — — — No linked delivery PR or qualified PR-reported gain. Closed with INCONCLUSIVE recorded. No RSI delivery PR or downstream output PR is identified; local screens are not a completed vLLM-Omni optimization.
HunyuanImage-3.0 New RSI vLLM-Omni optimization scope 📋 Planned TBD — — — No corresponding PR-reported gain identified. Scope AR/DiT, experts, attention and VAE explicitly; establish fresh E2E/quality acceptance. Unrelated upstream scheduling/graph PRs are not credited as RSI output.
Wan2.2 Dedicated RSI video-optimization campaign 📋 Planned TBD — — — No corresponding PR-reported gain identified. No corresponding RSI campaign/evidence PR or downstream output PR identified. Assess expert switching, attention, SP, VAE tiling and offload with fixed geometry and quality gates.
LTX-2.5 Dedicated RSI audio/video optimization campaign 📋 Planned TBD — — — No corresponding PR-reported gain identified. Legacy protocol fixtures do not establish a current campaign or downstream delivery. Freeze Full/SFT or distilled pipeline and qualify denoising, VAE, upsampling, memory and synchronization.
Z-Image / Z-Image-Turbo Transfer Qwen-Image lessons; separate standard/Turbo baselines 📋 Planned #416 (candidate; child TBD) — — — No corresponding PR-reported gain identified. Verify standard-checkpoint readiness; evaluate Chinese/English text, aspect ratios, fixed overhead, quality and memory. No RSI-generated engine PR is identified yet.

The tables retain the existing model tracks and add the linked MiniCPM-o #417/#431 and H3 T2VA #438 work, plus the explicitly attributed MiniCPM-o engine #9218 delivery. The requested Qwen3-TTS #9068 runtime delivery is tracked separately as an upstream reference. Models without a traceable RSI delivery remain in the planning queue; general upstream progress is not used as their RSI completion status. Published negative results and stopped campaigns retain their original verdicts.

Initial focus: Qwen-Image-2.1

The standard variant supports generation, editing, and RGBA. Turbo uses a separate accelerated checkpoint with an official 8-step recommendation and its own sampling configuration. Pin versions and sampling semantics separately to evaluate execution improvements within each model. Sources: standard model card, Turbo model card.

  • Baseline coverage: Reproduce the existing Turbo 1280×704 scenario, then establish a standard-variant baseline. Gradually add square / landscape / portrait images, short and long prompts, single-image / multi-image editing, and RGBA. Record additional resolutions, reference-image counts, concurrency levels, and sample sizes in the subtasks.
  • Stage profiling: Record text / image encoding, conditioning, each DiT step, VAE, response encoding, and complete HTTP latency. Also capture peak device memory, communication, and host/device synchronization.
  • Cache correctness: Distinguish native prefix KV caching from added cross-request conditioning caches. Measure cold caches, new prompts, and repeated prompts with new seeds separately. Verify invalidation rules and request isolation when prompts, reference images, dimensions, or configurations change.
  • Optimization candidates: Use the profile to select attention backends, RoPE / Norm / projection fusion, graph execution, layouts, and parallel configurations. Check whether earlier optimizations are already in the baseline to avoid attributing their gains twice.
  • Quality and generalization: Validate complete RGBA outputs for lossless tracks. For tracks allowing numerical changes, independently freeze perceptual quality, text correctness, edit preservation, and alpha-edge criteria, then evaluate on samples withheld from tuning.
  • GPU / NPU transfer: Reuse existing B300 / 910B1 experience first. Create separate combinations for 910C or other devices after confirming capacity, backend readiness, and budget. Report results independently rather than carrying over historical improvement percentages.

Register FP8 / INT8, approximate caching, sparse attention, reduced sampling steps, and distilled-checkpoint substitutions as separate experiment tracks. When quality, sampling semantics, or model identity changes, evaluate the costs and benefits separately.

Common deliverables and evaluation

Workload Performance metrics Quality and functional checks
Images / editing Complete request latency p50 / p95, images/s, peak device memory; separate cold start from steady state Text correctness, prompt adherence, preservation of reference subjects and unedited regions, RGBA; lossless tracks additionally require pixel identity
Video / audio-video Complete generation latency, generated video seconds per wall-clock second, peak device memory; fixed resolution, frame count, and FPS Visual clarity, motion and temporal consistency, prompt adherence, audio completeness, and audio/video synchronization
TTS / Omni TTFA, complete synthesis RTF, audio seconds per wall-clock second, playback stalls, and failure rate; report each concurrency level separately WER / CER, speaker similarity where applicable, audio quality, complete audio, and correct multi-turn state

Freeze quality metric implementations, versions, and thresholds per scenario. Incorporate the video-quality concern in #364: PSNR / SSIM alone cannot establish perceptual quality. Judge lossless tracks and tracks allowing quality changes separately.

Each campaign delivers:

  1. Reproducible baseline and targets: Model / weight identities, engine and dependency versions, device topology, input set, timing boundaries, warmup / cache policies, budget, and stop conditions.
  2. Profile and candidate rationale: E2E stage proportions, kernel / communication analysis at actual shapes, performance-ceiling estimates, and one attributable hypothesis per round.
  3. Formal comparison: Repeated paired A/B measurements, quality acceptance, and held-out checks after the protocol is frozen. Preserve failed, regressed, and interrupted runs. Keep profiling separate from formal timing.
  4. Reviewable changes and evidence: InferRSI evidence/knowledge PR, explicitly linked vLLM-Omni output patch / PR, and plan/, results/, patch/, and validator/ artifacts. Report PASS / FAIL / INCONCLUSIVE and the applicable scope.
  5. Reusable lessons: Record applicable models / shapes / backends, sources of gains, unsuccessful attempts, fallback conditions, and evidence links. Add lessons to the knowledge base after review.

Retain existing acceptance targets when they are already defined. For new tasks without performance targets, derive proposals from the baseline, profiling, and performance-ceiling analysis, then freeze them before candidate search. Do not impose a universal target such as “30% faster / 2× for every model.” See #399, #143, #303, and #385. Model quality constraints require independent justification; they cannot be derived from hardware peak performance.

Milestones and tracking

Milestone Scope Exit criteria
M0: Task readiness Refresh Qwen-Image-2.1 baselines after merged work; organize existing H3 campaigns Owners, devices, and budgets confirmed; reference outputs and repeated baselines reproducible; targets and validators frozen
M1: Initial image / video optimization Standard / Turbo variants and authorized H3 tracks Every combination actually started has complete E2E and quality verdicts; useful changes are reviewable; unmet targets retain their reasons and next steps
M2: Speech / Omni expansion Select vLLM-Omni speech / Omni work from Next according to owners and resources Reuse the common experiment workflow to qualify new modalities, including streaming boundaries, concurrency, and state correctness
M3: Cross-model transfer and service readiness Later models, a second hardware backend, longer inputs, and higher concurrency Independently remeasure new combinations and deliver evidence for sustained operation, failure recovery, and resource release; identify what transfers and what does not

Maintain one row per bounded optimization or acceptance item. Keep Status, Issue, RSI PR, vLLM-Omni PR, and Performance improvement reported in PR separate; record hardware, owner, verdict/evidence, next action, and last checked in the row or linked task. Update on PR merge, campaign verdict, or an explicit stop/resume decision. Check the latest issue comments as well as the body; a newer closure/disposition can supersede an older status block. Mark stale or missing execution information as unknown rather than inferring active experiments from an open issue. Keep tracks without confirmed owners or hardware in the candidate queue. This issue is a dated tracking snapshot; it does not set up automatic polling.

Platform dependencies: reuse task and evaluation work from #336 / #337; reference performance ceilings in #303 / #385; connect knowledge retrieval and adoption attribution to #379 / #344, and process visualization to #345. Integrate these dependencies according to their actual delivery status.

Proposal and authority: This issue publishes the requested vLLM-Omni model optimization roadmap and current status audit. Execution owners, resource budgets, and final acceptance owners remain to be defined in the corresponding subtasks. Existing campaigns retain their authorization and historical verdicts. Update roadmap progress from subtask evidence; an issue's open/closed state does not establish successful model optimization.

较大性能收益汇总(2026-11-17)

选取延迟/RTF 降低约 20% 以上,或吞吐提升约 20% 以上的代表性结果。同一模型在不同硬件上分开列出,各项收益不叠加。不同指标和测试条件不构成统一排名。

模型 硬件 / 场景 主要性能收益 PR / 状态 结果边界
MiniMax-H3 Ref2VA 8×910B1,5 秒视频 延迟 407.27→108.12 s,降低 73.45%,约 3.77× 加速 #9698 · Draft 使用有损 Cache-DiT;不代表无损或 15 秒视频结果
MiniMax-H3-Image 2×RTX 3090,图像生成 延迟 36.624→14.589 s,降低 60.16%,约 2.51× 加速 #10243 · Open 代码与部署配置的组合收益;单组 A/B,9 对图像 RGB 一致
Qwen3-TTS 1.7B 1×910B1,C1/C16/C32 吞吐达到历史基线的 2.58× / 2.35× / 2.35× #9068 · 已合并 存在中文音色相似度及部分 ASR 指标回退;非无损结论
MiniCPM-o 4.5 1×L20X,Seed-TTS C16/C32 RTF 分别降低 42.42% / 50.72%;C32 吞吐 11.46→22.99 请求/s #10331 · 已合并 五轮独立服务比较;正式语音质量非劣效尚未确立
MiniMax-H3-Image 4×910B1,生成 / 编辑 两种生成场景延迟降低 35.7% / 38.2%;编辑降低 23.4% #10184 · 已合并 来自显式开启的批量 VAE+FP16 编码配置,不是合并后默认配置的实测收益
Qwen-Image-2.1-Turbo 4×B300,图像生成 同配置比较,HTTP 延迟降低 26.73%–29.48% #8739 · 已合并 同时改变通信配置时可达 30.82%;不同口径需区分
Drama 3 2×910B1,C1 吞吐约 提升 30%;RTF 约 降低 23% #9308 · 已合并 FULL decode graph 收益;整体 RTF 目标仍未达到
Qwen-Image-2.1-Turbo 1×910B1,图像生成 离线延迟 2921.23→2272.83 ms,降低 22.20% #8729 · 已合并 18/18 图像 RGBA 一致;测量不含 HTTP,进程退出异常仍未解决

以上是 PR 报告的性能收益;“已合并”与“整个模型实验通过验收”是两个不同状态。各结果的测量版本、验收结论和来源归属以正文对应条目及关联 PR 为准。

Activity

  1. added
    kind:roadmapProgram-level roadmap anchor (Now / Next / Later)
    area:harnessHarness module: profile, A/B loop, quality gates, evidence reports
    area:knowledgeKnowledge line: packs, skill library, knowledge admission
    status:proposedWaiting for the decision named in the status block
    on Nov 12, 2026
  2. amy-why-3459 commented on Nov 12, 2026

    @amy-why-3459
    CollaboratorAuthor

    Add a Later track: Z-Image / Z-Image-Turbo to evaluate whether Qwen-Image-2.1 optimization lessons transfer to another image model family.

    Model / scenario Optimization focus and first deliverable Acceptance and tracking
    Z-Image / Z-Image-Turbo: text-to-image, emphasizing Chinese / English text and different aspect ratios Establish separate complete-service baselines for the standard and Turbo variants. Profile text encoding, iterative denoising, VAE, image encoding, and HTTP responses. Use measured stage proportions to select attention, compilation / graph execution, caching, and memory-layout optimizations, paying particular attention to fixed overhead in few-step workloads. New campaign; owner and hardware to be assigned. Report complete-image E2E p50/p95, images/s, peak device memory, and quality. Compare the standard and Turbo variants separately, retaining checks for prompt adherence, text correctness, and perceptual quality on independent samples.

    Source: official code and model links. This is a candidate track with no new experimental results under this roadmap. Its first deliverable is a reproducible baseline with a pinned checkpoint, target engine, and hardware configuration. Record both transferable optimizations and unsuccessful attempts in this issue, following the M0–M3 progression and acceptance process in the body.

    Also retain two historical states explicitly in the tracking table: #3 contains Turbo performance evidence but has no recorded formal acceptance verdict; #49 is closed, but its last recorded overall acceptance verdict is FAIL. Both can inform subsequent experiments. Neither performance figures alone nor issue closure establishes completed model optimization.

  3. changed the title [-][Roadmap] 多模型推理优化:Qwen-Image-2.1、视频生成、语音/Omni 与 LLM/MoE[/-] [+][Roadmap] Multi-Model Inference Optimization[/+] on Nov 12, 2026
  4. changed the title [-][Roadmap] Multi-Model Inference Optimization[/-] [+][Roadmap] vLLM-Omni Model Optimization[/+] on Nov 12, 2026
  5. Dong1017 commented on Nov 17, 2026

    @Dong1017
    Collaborator

    Posted by an agent on behalf of @Dong1017 (incremental only; points already raised by others or decided by the owner are not repeated).

    Incremental note: three acceptance issues that name this roadmap as their parent or related roadmap have no row in the table yet, and one of them already has merged evidence. The table was checked on 2026-11-12.

    Scope: the issue body (no mention of #417, #431, #438 or #435), the three issue bodies, and the #435 merge state and file list. I did not re-check the other rows.

  6. zengchuang-hw commented on Nov 17, 2026

    @zengchuang-hw
    Contributor

    独立源码检视(普通 Issue 技术反馈,不是 PR approval,也不是 campaign PASS)。

    检查基线:InferRSI 61303fa8150472dfbda62ec8d77908bc05c73f3b。针对图像生成/编辑、独立质量门禁和 E2E 测量需求,有三个具体的现有实现复用边界:

    1. 不能把现有摘要当作请求尾延迟或并发吞吐证据。 采样器 L69–85逐条串行请求,保留 samples_ms,但 groups_ms 是每组请求的算术均值;B300 示例 L202再以组均值生成 latency_ms。CPU 回调探针输入 [1,1,100] ms,实际得到 groups_ms=[34]。这是聚合语义验证,不是性能测量。后续 campaign 如报告 p50/p95,应从冻结场景中的逐请求样本计算并声明样本量/分位数方法;并发吞吐须有真实负载和整体观测窗口,不能从该串行均值推出。

    2. 冻结 timing 边界时要区分 HTTP 完成和最终可用媒体。 HTTP 实现 L8–16在请求编码之后启动计时,读完 response body 后停止;ImageProtocol L43–56随后才 JSON/base64 解码并写 PNG。这个值可作为已声明的 HTTP 指标,不能无说明地称为包含客户端解码/保存的完整完成耗时。该适配器使用 /v1/images/generations;它本身也不是编辑参考图输入路径已验证的证据。生成和编辑应各自固定请求协议、输入资产及完成边界。

    3. 历史 Turbo 验证器不是所有模型/形状的默认质量门禁。 历史示例 L38–98固定 1280×704 RGBA、RGB/alpha 分别检查、SSIM≥0.99/PSNR≥35,并要求 canonical reference 的三项 artifact 覆盖。这些约束不应直接移植为 standard、编辑、视频或其他设备的通用验收。新的图像 campaign 可参考 显式 ImageQualityGate 与逐通道校验,但仍需保留已有目标、冻结自身形状/模式/质量门槛及独立 holdout;像素相似度本身不证明编辑语义、视频时序或语音质量。

    实际检查:读取当前 AGENTS、contributing/privacy 规则、review skill,以及 vLLM-Omni serving/validation 指南;追踪上述源码并运行 CPU 聚焦检查:
    PYTHONPATH=src python -m pytest -o addopts= -q -ra tests/test_image_modules.py tests/test_qwen_quality_compatibility.py tests/test_qwen21_artifact_ownership.py
    结果 11 passed、7 skipped;7 项因测试环境缺少 numpy 跳过,故不声称像素比较测试已通过。另执行了上述采样聚合探针。

    边界:未运行真实模型、GPU/NPU、并发服务、缓存失效、holdout、视频/语音质量或跨设备实验,也未验证任何加速数字。此次仅识别可复用实现及其证据边界;性能/质量仍为 INCONCLUSIVE,不是接受新 scope、承诺实施或启动昂贵实验。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:harnessHarness module: profile, A/B loop, quality gates, evidence reportsarea:knowledgeKnowledge line: packs, skill library, knowledge admissionkind:roadmapProgram-level roadmap anchor (Now / Next / Later)status:proposedWaiting for the decision named in the status block

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions