Repository navigation
[feat] deploy: add hardware-aware vLLM-Omni setup - #259
WEIXILIN0307 wants to merge 17 commits into
Conversation
| return dict(status='stopped') | ||
|
|
||
|
|
||
| def main(): |
There was a problem hiding this comment.
Could we expose deployment and verification through the installed opendirector CLI, with --help and packaged profiles, so users and Agents can use them without a source checkout?
There was a problem hiding this comment.
Thanks! Deployment and verification are now available through opendirector vllm-omni deploy and verify, with --help, packaged profiles, and the range downloader included. I verified the wheel installation outside the source checkout as well.
| This record separates resource eligibility, actual inference, project delivery | ||
| and creative review. 本记录分别记录资源预检、真实推理、工程交付与创作品质审看,不能互相代替。 | ||
|
|
||
| ## Environment / 环境 |
There was a problem hiding this comment.
Could we move this validation record and docs/development/assets/vllm-omni-validation.png to the PR description as text and attachments, keeping reusable instructions in the guides?
There was a problem hiding this comment.
Done, thanks! I moved the validation results and screenshot to the PR description and removed the one-off files from the repository. The reusable English and Chinese guides remain in the repo.
…ment # Conflicts: # packages/core/src/opendirector/cli.py
Summary
Local model setup requires a separate deployment workflow. This PR brings hardware-aware setup and verification into the installed OpenDirector CLI, helping users and Agents connect a local model and produce project assets without a source checkout.
Implementation
Start with the CLI adapter, then review
plan,start/stopandverify.omni-envand caches; pin top-level runtime/compiler packages, probe CUDA/FlashInfer and record the environment.--infersubmits once, saves a receipt, decodes PNG pixels, probes audio metadata or compares ASR text. No automatic inference retry.opendirector vllm-omni deploy/verify, with--help; package shared tool sources, profiles and downloader without adding GPU runtimes to application dependencies.canvas add-materialso imported transcripts remain editable.Review process ownership, receipt provenance and package boundaries first. Project delivery uses existing application operations; deployment helpers never write project files. The material-title fix was discovered during ASR acceptance and could be reviewed separately.
Validation
Scope:
fe50e27. Audit reruns are marked below; other results come from reviewed local implementation logs/receipts, not uploaded CI artifacts. No new full-suite or GPU run was performed; model checks do not establish production readiness.test_vllm_omni_cli.py,test_vllm_omni_deployment.py,test_cli.py; covers discovery, preflight, lifecycle boundaries, unknown submissions and output validation.test_native_runtime.py::test_rejects_removed_runtime_and_invalid_selection_before_acceptancestill returned 503 instead of 409. Earlier isolated passes do not resolve this; a probe reported invalid OpenCode skill-catalog JSON.0.1.4sdist/Linux x86_64 wheel; tool/profile bytes match this head. Audit reran verify_install.py outside the checkout: help, profiles, downloader plan, application/web startup and project persistence passed;uv pip checkpassed.action_required, with no jobs started.Reproduction and troubleshooting: the English / Chinese guides.
Real-model Evidence
November 9–10, 2026: Ubuntu 22.04.5, Python 3.12.7, driver 580.126.09; one RTX PRO 6000 Blackwell Server Edition (SM120, 97,887 MiB), not an Ampere A6000. Recorded software: vLLM/vLLM-Omni 0.32.0, Torch 2.13.0+cu130, CUDA 13.0. CUDA, FlashInfer and dependency checks passed.
f332072aa78be7aecdf3ee76d5c247082da564a685e237c12c027371202489a0ec509ded67b5e4b5vivianMP3; 24 kHz mono, 8.304 s of audio; recorded elapsed 6.158 s.5eb144179a02acc5e5ba31e748d22b0cf3e303b0Verifier elapsed time ends when the response is received; decoding/hashing follows, and ASR also includes input probing. These single requests are not throughput or end-to-end deployment benchmarks.
Participant report: assisted cached setup, no help requests/retries, SSH recovery and assets retained after reopening. First success was estimated retrospectively at 5–10 minutes; exact timing, total manual actions and recovery duration were not recorded.
The existing screenshot shows the reopened project before the later application-process restart:
Limitations
stopreturningstoppedonce the leader exited while a SIGTERM-ignoring descendant remained alive. stop currently checks the leader identity; complete descendant shutdown is not established.root/omni-env/bin/python; incomplete state can raise unstructured exceptions; the wheel includes unused FastH3 helper modules. No code was changed by this audit.Related Issue / Review Notes