Any embodiment. Any policy. Any inference strategy.
ManiMux is an extensible real-world manipulation harness that standardizes deployment and experiment workflows across embodiments, policies, and inference strategies. Built around RoboGUI, it brings together live digital twin visualization and a runtime designed for 100–200 Hz command execution.
Policy × Runtime × Embodiment
Quick start · Architecture · Integrate · Documentation · Citation
Read the ManiMux Guide → · HTML source Installation, station setup, deployment recipes, RoboGUI workflows and integration protocols. For agent-led development, start with the development skill.
-
2026-10-13 — Added execution-qualified evaluation: chunk handoff jumps, 10/20/30-step overlap and segmented reference smoothness, with versioned offline HTML reports.
-
2026-10-12 — Added task and method switching from a config list, template-driven partial scores and explicit warmup controls.
-
2026-10-02 — Added ALOHA-AgileX and PiPER RoboGUI demos, plus experimental ARX X5 / PiPER controllers. Hardware validation is pending.
-
2026-10-01 — The ManiMux Guide is live, covering setup, deployment, RoboGUI and integration protocols.
-
2026-10-01 — Updated research workflows and agent integration guides for free exploration, study templates and component development.
Prepare Run / Prepare Experiment → Activate Warmup → Start → Finish → Score.
Warmup is shown when configured. Free runs need no scoring; experiments read their
score options from the task YAML’s inline evaluation rules. Recorded trajectories replay
in a separate, hardware-free view. Research workflow →
- Policies with robot deployment recipes (9): Pi05, DP, SAPolicy, ABC-DiT, GR00T N1.7, LingBot-VLA2, Xiaomi XR-1, UMI DP and OpenWAM.
- Policies with offline recipes (5): Isaac 0.5 and StarVLA's QwenOFT, QwenPI-v3, QwenGR00T and QwenFast.
- Base inference modes (8), plus opt-in combinations: Serial, asynchronous chunking, RTC, ACT temporal ensembling, AAC, PAINT, AutoHorizon and BID backward-only.
- Hardware integrations (4): YAM, Tianji–TacCap, and experimental ARX X5 / PiPER. Executors: Direct, Smooth and MPC.
ALOHA-AgileX follower-arm assets and standard PiPER assets are available for offline RoboGUI preview and replay. Experimental ARX X5 / PiPER SDK adapters include station templates and offline validation; physical deployment remains unvalidated. The SDK installation guide covers their external dependencies.
Policy serving uses XPolicyLab or StarVLA. Counts describe included integrations, not all model × method × robot combinations or completed hardware validation. See the support catalog for scope and recipes. Evaluation is optional; choose your own research protocol and metrics.
Choose a base algorithm, a request schedule, and optional ACT fusion. The eight base modes remain available; combinations reuse their existing samplers and do not require changes to XPolicyLab or model weights.
| Purpose | Modes | Behavior |
|---|---|---|
| Execution scheduling | Serial, asynchronous chunking | Infer after the selected prefix finishes, or overlap inference with execution |
| Consistency between chunks | RTC, PAINT, ACT temporal ensembling, BID backward-only | Guide sampling, adjust initial noise, average aligned predictions, or select a coherent candidate |
| Adaptive execution length | AAC, AutoHorizon | Select a source prefix using candidate uncertainty/motion or action attention |
The current compatibility matrix is:
| Base algorithm | Serial | Async single_inflight |
Async multi_inflight |
Optional ACT fusion |
|---|---|---|---|---|
Ordinary chunking (serial / async) |
Yes | Yes | Yes | Select act_temporal_ensemble |
| ACT temporal ensembling | New | Yes | Yes | Already enabled |
| RTC | — | Yes | Yes | New |
| PAINT | — | Yes | New | New |
| BID backward-only | Yes | Yes | Yes | New |
| AAC | Yes | New | New | New |
| AutoHorizon | Yes | New | New | New |
BID also supports AAC adaptive horizons, with optional ACT fusion. For deadline aliases, decoding requirements and YAML examples, use the inference guide and supported combinations.
Model frameworks own inference. ManiMux owns observation/action adaptation, scheduling, execution and experiment operation. Strategies decide when to request and hand off chunks; executors turn timeline references into commands.
%%{init: {"flowchart": {"wrappingWidth": 180, "curve": "basis", "nodeSpacing": 24, "rankSpacing": 28, "padding": 14}}}%%
flowchart LR
OBS["<b>OBSERVE</b><br/>Cameras · robot state<br/>Build policy inputs"]:::stage
XPOLICY["<b>PREDICT · XPolicyLab</b><br/>Pi05 · XR-1 · GR00T<br/>LingBot · OpenWAM"]:::xpolicy
STARVLA["<b>PREDICT · StarVLA</b><br/>OFT · PI-v3 · GR00T · FAST"]:::xpolicy
PLAN["<b>ADAPT & SCHEDULE</b><br/>Async · RTC · PAINT<br/>Serial · adaptive<br/><br/>Adapter → Timeline"]:::handoff
ACT["<b>EXECUTE</b><br/>Direct · Smooth · MPC<br/><br/>Executor + Safety<br/>Control profile"]:::stage
ROBOT(["<b>ROBOT</b><br/>RobotBase<br/>Hardware"]):::robot
REVIEW(["<b>OPERATE & REVIEW</b><br/>RoboGUI · records · replay<br/>Optional evaluation"]):::side
OBS --> XPOLICY --> PLAN --> ACT --> ROBOT
OBS --> STARVLA --> PLAN
ACT -.-> REVIEW
classDef stage fill:#F6F8FA,stroke:#8C959F,stroke-width:1px,color:#1F2328
classDef handoff fill:#FFF4E5,stroke:#E36209,stroke-width:2.5px,color:#1F2328
classDef side fill:#FFFFFF,stroke:#8C959F,stroke-dasharray:4 3,color:#57606A
classDef robot fill:#1F2328,stroke:#1F2328,color:#FFFFFF
classDef xpolicy fill:#8957E5,stroke:#6633B8,color:#FFFFFF
XPolicyLab and StarVLA have independent serving environments and ManiMux clients. A new framework can implement the same client interface. A new robot implements component and assembly protocols. Extension map →
Python 3.11 or 3.12 and uv are required for these commands:
git clone https://github.com/SII-LiuLab/manimux.git
cd manimux
uv sync --dev
uv run manimux-robogui --robot yam --demo --host 127.0.0.1 --port 8086Open http://127.0.0.1:8086. This demo displays synthetic data and the bundled YAM model; it does not connect to a robot. Model-framework submodules and checkpoints are only needed for the deployment path you choose.
Try the new arms in RoboGUI using the same environment:
uv run manimux-robogui --robot piper --demo --host 127.0.0.1 --port 8087Open http://127.0.0.1:8087. Use --robot aloha for ALOHA-AgileX.
Both presets animate arms and grippers with synthetic data; no device SDK is needed.
ARX X5 currently has a kinematic model and experimental controller, without a
bundled RoboGUI mesh preset.
Upgrading an existing checkout? See the RoboGUI interface migration.
- Select a model/robot runbook and prepare its hardware and model environments.
- Bind devices, service addresses and checkpoint paths in your private station file.
- Start the camera, RoboGUI, model server and runtime using the complete Pi05/YAM example or the selected model's runbook.
- Continue in RoboGUI: select the configured task/method, prepare and run trials, then review records. For your own task/study, start with the experiment skill.
manimux serve keeps the service available for repeated GUI-driven rollouts.
manimux run executes one rollout. Preparation can connect and move the selected robot
as configured; use the runbook matching your actual setup.
Configuration explained · Current study and launch commands
A collection organizes tasks, methods, trial counts and a recording root. RoboGUI exposes Task → Method / Config → Layout; compatible entries switch while idle. Task files define instructions, layouts and scoring. Each trial records its resolved settings, trajectories and optional human assessment for reproducible comparisons.
The published Pi05 study illustrates five tasks and six inference methods, with additional starting-pose variants. Use it as a concrete example; choose task rules, checkpoints and comparison settings for your own study. Offline reports combine human scores, motion metrics and optional PRM assessments. Runtime completion and task success are distinct.
Experiment workflow · Launch commands · Recording protocol · Evaluation and reports
| Entry | Purpose |
|---|---|
| This README | Project scope, architecture, supported integrations and getting started |
| Directory READMEs, such as configs | Local file organization, module responsibilities and links to detailed guidance |
| User guide | Setup, parameter explanations, commands and complete experiment workflows |
| Agent skills | What an agent should inspect, ask, implement and verify for a specific task |
For users: configure an existing combination and use RoboGUI.
For your coding agent: use the development skill
or the matching task skill below. If your tool does not discover repository skills,
ask it to read the linked SKILL.md explicitly.
| Task | Entry point |
|---|---|
| Add a robot, gripper or camera | Component protocols |
| Add a model, framework or action adapter | Policy protocols |
| Add scheduling, execution or a GUI feature | Runtime protocols |
| Connect supported hardware at another lab | Station setup skill |
| Build a custom study, collect trials or evaluate recordings | Experiment skill |
Each integration documents its input/output semantics, owning files, YAML selection and validation.
Build with us. Contributions are welcome—from new embodiments, policies, and inference strategies to documentation and bug fixes. Start with the integration guide, follow the shared protocols, and see Contributing for the expected handoff.
See the support catalog for model, hardware and method runbooks. Capabilities vary by checkpoint and backend; an integration does not imply that every model × robot × algorithm combination has been tested on hardware. ManiMux is under active development. Each deployment guide states its validation scope.
If ManiMux supports your research, please cite:
ManiMux: An Extensible Real-World Manipulation Harness
@misc{manimux2026,
title = {{ManiMux}: An Extensible Real-World Manipulation Harness},
year = {2026},
howpublished = {GitHub repository},
url = {https://github.com/SII-LiuLab/manimux}
}Please also cite the components, models and methods used in your work: XPolicyLab, StarVLA and PRM-as-a-Judge.
Official BibTeX for related projects (cite those you use)
@article{community2026xpolicylab,
title={{XPolicyLab}: A Unified Standard and Open Ecosystem for Robot Policy Evaluation and Deployment},
author={Community, XPolicyLab and Chen, Tianxing and Chen, Yue and Nian, Tian and Cai, Zijian and Chen, Guangyu and Lin, Wenwei and Liang, Qiwei and Xiang, Peicheng and Su, Kailun and others},
journal={arXiv preprint arXiv:2608.09892},
year={2026}
}@article{community2026starvla,
title={StarVLA: A Lego-like Codebase for Vision-Language-Action Model Developing},
author={Community, StarVLA},
journal={arXiv preprint arXiv:2604.05014},
year={2026},
eprint={2604.05014},
archivePrefix={arXiv},
primaryClass={cs.RO}
}PRM-as-a-Judge 1.5 — toolkit report
@article{liu2026prmjudge15,
title = {PRM-as-a-Judge 1.5: A Toolkit for Robot Process Assessment},
author = {Liu, Yuyang and Shen, Yanqing and Chen, Ruike and Zhao, Jifan and Tian, Yuxuan and Zhang, Yichi and Long, Tianfeng and Yin, Zixuan and Wang, Yipu and Qin, Ziheng and Tan, Wenxing and Shi, Yang and Cao, Mingyu and Xiao, Runze and Wang, Ziqi and Yin, Zhixin and Chu, Shiwei and Zhang, Yi-Fan and Mu, Yao and Ji, Yuheng and Wang, Yihao and Yan, Jun and Wang, Zhongyuan and Wang, Pengwei and Zheng, Xiaolong},
journal = {arXiv preprint arXiv:2608.14284},
year = {2026},
url = {https://arxiv.org/pdf/2608.14284}
}PRM-as-a-Judge — original method
@article{ji2026prmjudge,
title = {PRM-as-a-Judge: A Dense Evaluation Paradigm for Fine-Grained Robotic Auditing},
author = {Ji, Yuheng and Liu, Yuyang and Tan, Huajie and Huang, Xuchuan and Huang, Fanding and Xu, Yijie and Chi, Cheng and Zhao, Yuting and Lyu, Huaihai and Co, Peterson and others},
journal = {arXiv preprint arXiv:2603.21669},
year = {2026}
}Third-party notices · Upstream licenses · Documentation
ManiMux is licensed under MIT. Third-party frameworks, SDKs and assets retain their own licenses; see third-party notices.