Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
8effee4
feat(deploy): add hardware-aware vLLM-Omni setup and verification
WEIXILIN0307 Nov 9, 2026
b7d4ef6
fix(deploy): reuse resumable ranges and align the image probe with pr…
WEIXILIN0307 Nov 9, 2026
862d06e
fix(deploy): align CUDA JIT tooling and decode verification images
WEIXILIN0307 Nov 9, 2026
916a8f6
fix(deploy): hash the exact frozen environment bytes
WEIXILIN0307 Nov 9, 2026
180d5dd
docs(deploy): add bilingual skill and real GPU acceptance evidence
WEIXILIN0307 Nov 9, 2026
b96d73d
fix(cli): name displayed materials from their registered assets
WEIXILIN0307 Nov 10, 2026
fd58e09
feat(deploy): validate native ASR and preserve participant acceptance…
WEIXILIN0307 Nov 10, 2026
ce3e528
docs(deploy): record successful participant first-use trial
WEIXILIN0307 Nov 10, 2026
4ba9494
docs(deploy): corroborate participant trial with terminal receipts
WEIXILIN0307 Nov 10, 2026
4bef626
docs(deploy): verify participant generation and audio artifact
WEIXILIN0307 Nov 10, 2026
a0cef6e
docs(deploy): record participant recovery and help feedback
WEIXILIN0307 Nov 10, 2026
1312d34
docs(deploy): record participant timing estimate and retries
WEIXILIN0307 Nov 10, 2026
20ec50c
fix(deploy): honor GPU selection for existing services
WEIXILIN0307 Nov 10, 2026
707440b
docs(deploy): include real project validation screenshot
WEIXILIN0307 Nov 10, 2026
2b3d577
docs(deploy): clarify screenshot capture point
WEIXILIN0307 Nov 10, 2026
74fc8a0
feat(deploy): expose packaged vllm-omni CLI
WEIXILIN0307 Nov 17, 2026
fe50e27
Merge remote-tracking branch 'origin/main' into feat/vllm-omni-deploy…
WEIXILIN0307 Nov 17, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
82 changes: 82 additions & 0 deletions .agents/skills/deploy-vllm-omni/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
---
name: deploy-vllm-omni
description: Inspect a Linux GPU host, select a compatible vLLM-Omni model recipe, deploy an isolated service, connect OpenDirector and verify real generated assets. Use for self-hosted model setup or deployment repair; ordinary application installation uses install-opendirector.
---

# Deploy vLLM-Omni for OpenDirector

Deliver a reproducible working model connection and actual output. Reply in the
user's language. Follow the user's chosen host, assigned GPU, directory and model.
A discussion or plan alone does not authorize installation or inference; an
authorized deployment-and-validation request does not require confirmation at
each routine step.

Use the installed `opendirector vllm-omni` CLI on the target host. Discover its
options with `opendirector vllm-omni deploy --help` and
`opendirector vllm-omni verify --help`. The application package includes the
profiles and deployment tools; model dependencies belong to an isolated environment.
Read the matching [English guide](../../../docs/guides/vllm-omni.md) or
[Chinese guide](../../../docs/guides/vllm-omni.zh-CN.md) when available.

## Inspect before changing anything

Run `opendirector vllm-omni deploy --root DATA_DIRECTORY doctor` with a Python 3.12 model interpreter on the target host.
The helper is read-only. Resolve SSH aliases using actual SSH configuration, and
record the detected hardware rather than inferring it from an alias. Respect
assigned resources, inherited device visibility and other workloads.

Run `plan` for an explicit physical index or UUID and model profile. Inspect OS,
GPU architecture, driver, free VRAM, available host memory and the relevant data
disk. Multiple cards do not form a larger single GPU. Unknown measurements do not
establish readiness. In particular, the repository's FastH3 NVFP4/SM120 recipe
does not work on an Ampere RTX A6000. Eligibility and measured qualification are
different facts; do not claim support for an untested combination.

## Deploy and observe the original operation

Use `install`, `download`, `start`, `status` and `stop` from the guide. Reuse the
fixed profile and mature uv/Hugging Face mechanisms. Keep runtime data, caches,
logs and outputs under the selected data directory. Use the profile's isolated
environment, preserve existing application configuration and do not change
global drivers or package registries. Choose a reachable trusted mirror only
when needed; keep it scoped to the deployment.
If observed long-transfer failures warrant it, use the guide's range backend,
which reuses the repository's existing pinned downloader. Preserve original
operations and partial files; do not invent another downloader or run duplicate
transports for the same model. Reuse existing weights only after manifest checks.

Observe exit status, readiness and the reported log. A started PID is not ready
inference. `startup_pending` refers to the original possibly loading process;
query it before another action. Preserve conflicting listeners and shared
workloads. After a transient failure retry the same operation once only when
evidence supports it; continued failure requires diagnosis or changed conditions.

## Prove delivery through OpenDirector

Use `opendirector vllm-omni verify` for read-only health/model identity and `opendirector vllm-omni verify --infer` for
one authorized real request. Inspect the PNG or listen to the audio, and report
technical validity separately from creative review. Preserve verification
receipts, exact versions, parameters, artifacts and resource sampling scope.
Inference timeouts are unknown submissions; do not automatically resubmit them.

Connect through Settings → Model access or existing model configuration APIs.
Choose the URL reachable from the OpenDirector backend, which may be on a
different machine from the browser. Keep remote model services on loopback and
use an SSH tunnel for the selected topology.

Finish in an isolated real OpenDirector project using existing business
commands: create generation intent, submit, wait by returned job identity, inspect
the registered asset and adoption state, reopen and continue. Use CLI discovery
and returned next commands; never edit project files or credential storage to
bypass business rules. The `qwen3-asr-06b` companion uses native vLLM in the same
environment; use `opendirector vllm-omni verify --infer --audio-file` for transcription. Reference
text is only a local comparison, never a model hint. OpenDirector has no automatic
ASR tool: register transcripts through existing text-asset commands and describe
this handoff explicitly. Do not treat music-to-score as ASR or invent capabilities
from the provider name. Record unavailable integrations explicitly. Follow the
guide's first-use protocol for a real participant; keep unobserved human feedback
pending and distinguish cached setup from clean installation.

Hand off copyable restart/status/stop commands for the actual host and
data directory, the model connection, measured results and remaining limitations.
Stop test processes you created unless the user requested a running service.
3 changes: 3 additions & 0 deletions .agents/skills/deploy-vllm-omni/agents/openai.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
interface:
display_name: "Deploy vLLM-Omni"
short_description: "Deploy GPU models and verify OpenDirector integration"
2 changes: 2 additions & 0 deletions MANIFEST.in
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@ recursive-include src *
recursive-include packages/core/src *.py
recursive-include apps/api/src *.py
recursive-include packages/agent/skills *
recursive-include tools/vllm_omni *.py *.json
include tools/fasth3/__init__.py tools/fasth3/download_models.py
prune packages/core/src/opendirector/_bundled
prune node_modules
prune dist
Expand Down
1 change: 1 addition & 0 deletions docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ This directory is the entry point for the repository's shared documentation. The
| [Business CLI](guides/cli.md) | Querying, editing, and generating through a terminal or Agent Shell | Commands, authorization, or receipt semantics change |
| [DeepSeek Harness plugin](../plugins/deepseek-harness/README.md) | Installing the DSH bundle and editing an OpenDirector project from DSH | Plugin setup, tools, or creation workflows change |
| [Video model capabilities](guides/video-model-capabilities.md) | Declaring output specifications supported by a service | Configuration fields or capability integration changes |
| [Local vLLM-Omni deployment](guides/vllm-omni.md) ([中文](guides/vllm-omni.zh-CN.md)) | Hardware inspection, fixed model recipes, service lifecycle and real inference verification | Deployment tooling, recipes or operational guidance change |
| [OpenAI Images](guides/openai-images.md) | Wiring gpt-image as an optional image service, request shape and failures | Integration fields, capability scope, or the OpenAI interface change |
| [Kling video](guides/kling-video.md) ([中文](guides/kling-video.zh-CN.md)) | Optional Kling 4.0 and Flash setup, model-specific inputs, and recovery limits | Kling inputs, model declarations, or recovery behavior change |
| [BigModel video](guides/bigmodel-video.md) ([中文](guides/bigmodel-video.zh-CN.md)) | CogVideoX-3 text, single-image and first/last-frame generation, task recovery and adoption | BigModel protocol, inputs, output sizes or recovery behavior change |
Expand Down
2 changes: 1 addition & 1 deletion docs/development/packaging.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,4 +42,4 @@ python -m pip check
python scripts/verify_install.py --python /absolute/path/to/venv/bin/python
```

Windows 对应传入 `venv/Scripts/python.exe`。验证脚本在临时工作目录运行,移除 `PYTHONPATH` 并收窄 PATH,检查内置 OpenCode、技能、正式 serve 入口、健康接口、首页与静态资源、CLI 创建和重读工程,最后清理自己启动的进程。它不调用真实模型,也不覆盖媒体处理或所有平台验收。
Windows 对应传入 `venv/Scripts/python.exe`。验证脚本在临时工作目录运行,移除 `PYTHONPATH` 并收窄 PATH,检查部署命令帮助、全部配方、分段下载器计划、内置 OpenCode、技能、正式 serve 入口、健康接口、首页与静态资源、CLI 创建和重读工程,最后清理自己启动的进程。它不调用真实模型,也不覆盖媒体处理或所有平台验收。
2 changes: 2 additions & 0 deletions docs/guides/installation.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,8 @@

Installing dependencies and calling remote models require a network connection. Prepare self-hosted model services separately; starting this application does not install or deploy models.

For a Linux NVIDIA model host, see [local vLLM-Omni deployment](vllm-omni.md).

You can install and open the application, upload local media, and edit projects before connecting a model. AI chat and generation require a configured service; running the application does not require a local GPU when using a remote service.

#### macOS prerequisites
Expand Down
2 changes: 2 additions & 0 deletions docs/guides/installation.zh-CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,8 @@

安装依赖和调用远程模型需要网络。自部署模型服务需另外准备,启动本应用不会自动安装或部署模型。

Linux NVIDIA 模型主机的配置与验证参见 [vLLM-Omni 本地部署](vllm-omni.zh-CN.md)。

尚未接入模型也可以安装、打开应用、上传本地素材和编辑工程。AI 对话和生成需要配置模型服务;使用远程服务时,运行本应用的 Mac 无需本地 GPU 推理环境。

#### macOS 环境准备
Expand Down
Loading
Loading