Skip to content

feat: support exact 17k+5 frame durations for vLLM-Omni - #334

Merged
FayeSpica merged 7 commits into
ThinkFlowLab:mainfrom
FayeSpica:feat/vllm-omni-exact-duration
Nov 16, 2026
Merged

FayeSpica merged 7 commits into
ThinkFlowLab:mainfrom
FayeSpica:feat/vllm-omni-exact-duration

Conversation

@FayeSpica

@FayeSpica FayeSpica commented Nov 15, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Support fractional video durations for vLLM-Omni H3 and FastH3 adapters, with integer slider steps and decimal typing for users and agents. Show the aligned frame count and actual expected duration beside the input.

  • Send num_frames and fps for H3/FastH3, without seconds or a duration alias. The server owns 17k+5 alignment: 107/24 seconds requests exactly 107 frames; 15 seconds requests 360 frames and aligns to 362.
  • Generic vLLM-Omni video adapters accept integer seconds and send seconds. Explicit H3/FastH3 adapter aliases retain fractional support; other protocols keep their declared rules.
  • Preserve the original returned video bytes, frames and audio during initial download and recovery. Remove automatic trimming, re-encoding and soundtrack replacement. Exact clip ranges belong to explicit timeline edits; derived media retains the original and its provenance.
  • Update CLI help, both UI languages and the architecture constraints. No frame-count input mode.

Test plan

  • python -m pytest tests/test_video_resolution.py tests/test_creation_parameters.py tests/test_vllm_omni_h3.py tests/test_fasth3_inputs.py tests/test_self_hosted.py tests/test_self_hosted_queue.py tests/test_video_capabilities.py -q
  • Run the byte-preservation regressions against the previous implementation and this change.

Test result

At 4561e40d:

  • 167 tests passed. Updated the stale FastH3 wire assertions that caused four CI failures: requests use num_frames and fps, without seconds or a duration alias. Added 4.33-second and 107/24-second cases; recovery still makes no second submission. Coverage also includes adapter gating, duration persistence and original-byte preservation.
  • Both byte-preservation cases fail against 4b1f5dce and pass with this change: a 4.33-second request retains the returned video longer than 4.4 seconds during normal download and recovery.

Evidence that it works

A real FastH3 generation at 4b1f5dce used the current canvas: a 4.33-second request sent num_frames=104 and fps=24, without seconds or a duration alias. The server returned 107 frames at 24 fps (4.458333 seconds); the original file decoded successfully. That run exposed the previous automatic trim to 104 frames, which this change removes. Original-byte preservation at the new head is verified with local-media regression tests; no new real-provider generation was run after removal.

Earlier UI validation

At 4b1f5dce, 820 frontend tests, build, i18n and contract checks passed. Browser checks covered integer dragging, decimal typing (including 8.), persistence, both UI languages and generic video choices. The current change modifies backend result handling and architecture documentation.

Earlier input/persistence screenshots from 31ff8a16 (before the aligned-output preview was added):

中文 English
Fractional duration input in Chinese Fractional duration input in English

Signed-off-by: Weiming Liao <liaowm5@gmail.com>
@FayeSpica FayeSpica added type:feat New capability or behaviour area:canvas Canvas, nodes and direct manipulation area:provider Model providers and generation protocols modality:video Video generation, editing or rendering labels Nov 15, 2026
Signed-off-by: Weiming Liao <liaowm5@gmail.com>
@FayeSpica
FayeSpica marked this pull request as ready for review November 15, 2026 06:01
Signed-off-by: Weiming Liao <liaowm5@gmail.com>
@github-actions github-actions Bot added the area:docs Documentation / 文档 label Nov 15, 2026
@github-actions
github-actions Bot requested a review from wuzhf9 November 15, 2026 06:14

@hsliuustc0106 hsliuustc0106 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review: Approve with follow-ups

整体设计正确、CI 全绿(head 31ff8a1):duration_matches 单一判定前后端同步、InputRules.fractionalDuration 经 exclude_if 序列化、H3 走 h3_frames 出 num_frames。以下按 file:symbol 逐条标注,从「先确认」到 Nit 排列:

  • P1 先确认范围:fractionalDuration 对所有 vllm_omni 视频置真(非仅 H3)。
  • P2 需验证:FastH3 小数路径未端到端验证,form['seconds'] 对 fast 是死代码。
  • P3 边界:接受区间与 H3 可达帧不对齐。
  • P3/Nit:时长控件 UI 与全精度序列化小问题。

Comment thread src/canvas/DurationControl.tsx Resolved
Signed-off-by: Weiming Liao <liaowm5@gmail.com>
Signed-off-by: Weiming Liao <liaowm5@gmail.com>
Signed-off-by: Weiming Liao <liaowm5@gmail.com>
Signed-off-by: Weiming Liao <liaowm5@gmail.com>
@github-actions github-actions Bot added the provider:minimax MiniMax H3 / speech / music services label Nov 16, 2026
@FayeSpica
FayeSpica merged commit ee7c92e into ThinkFlowLab:main Nov 16, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:canvas Canvas, nodes and direct manipulation area:docs Documentation / 文档 area:provider Model providers and generation protocols modality:video Video generation, editing or rendering provider:minimax MiniMax H3 / speech / music services type:feat New capability or behaviour

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants