Repository navigation
Conversation
|
📝 Docs preview is not auto-deployed for fork PRs. A maintainer with write access to |
|
/preview-docs |
|
✅ Preview deployed: https://NVIDIA.github.io/IsaacCapture/preview/pr-1250/ |
|
@aristarkhovNV Can you please help review the core change part. |
ivany-nv
left a comment
There was a problem hiding this comment.
Summary
This adds a MOXI Receiver plugin that publishes AgileMaster poses as FullBodyPose, a body.jmex live tracker that reads them (same shape as the Noitom and Xsens readers), and an Isaac Lab G1 example. CI is green on every job. Built locally without the SDK, the jmex tests pass, and the plugin's own publisher, run against a CloudXR runtime, round-trips through body.jmex into a standard full-body MCAP with every joint value intact. Read independently, the axis change, slot mapping, quaternion order, timestamps and MCAP wiring all hold up. One suggestion, nothing blocking. The src/core/live_trackers reader and factory registration are with @aristarkhovNV; this review covers the plugin, example, tests and docs.
Legend: 🚫 BLOCKER = merge-blocking | 💡 SUGGESTION = meaningful improvement | 🧹 NIT = precise small correction
| Finding | |
|---|---|
| 🚫 | none |
| 💡 | src/plugins/jmex/app/main.cpp:126-129 - An unrecognized argument is only logged as a warning. So --channel 2 (space instead of =) or a typo like --chanel=2 leaves the plugin on channel 1, waiting for Player forever. That is the silent mismatch the strict --channel= parsing at L92-L93 exists to prevent. The Xsens plugin rejects unknown options; doing the same here makes the plugin fail fast with exit code 1. |
| 🧹 | none |
ivany-nv
left a comment
There was a problem hiding this comment.
Summary
A second pass, this time against this PR merged onto current main, which is 55 commits past your base. It merges cleanly and builds. ctest -L jmex, the vendor-validation cases, clang_format, pre-commit on the changed files and make current-docs all pass. The body.jmex reader is safe to add: the default vendor is unchanged, there is no schema or recording change, and it compiles against main's newer SchemaTracker. Before approval there are three things: two example fixes (the first two 💡 below) and a rebase onto main so CI reruns against it. The rest are questions and small items.
Legend: 🚫 BLOCKER = merge-blocking | 💡 SUGGESTION = meaningful improvement | ❓ QUESTION = needs your answer | 🧹 NIT = precise small correction
| Finding | |
|---|---|
| 🚫 | none |
| 💡 | jmex_g1_tasks.py:30-32 - Isaac Lab 3.0.0 EA, the release main's README names as current, moved this config to isaaclab_tasks.contrib.locomanip_pick_place.locomanipulation_g1_env_cfg (isaac-sim/IsaacLab#5891). The manager_based path no longer exists there, so --external_callback ...register_tasks fails with ModuleNotFoundError. Please import the new path with a fallback to the beta2 one, and name the tested Isaac Lab version in README prerequisite 4. |
| 💡 | jmex_g1_tasks.py:711-725 - Returning [] tells teleop_se3_agent.py to reject none of the leftover arguments: list_intersection(remaining_args, []) is empty, and with None it is all of them. So --cloudxr-env none (hyphen) is silently ignored and Isaac Lab starts its own auto-webrtc runtime. That is the failure README L97-99 warns about, now with no error. The callback parses no arguments of its own, so return None is right. |
| 💡 | main.cpp:59-75 - This extends my earlier comment. parse_port checks that the whole string is an integer but not its range. So --tcp-port=70000, --channel=0 or --channel=300 go straight to the SDK, and udp_start_port + 1 + channel (L151) can overflow. The noitom and xsens plugins reject out-of-range ports, and receiver_session.hpp:27-28 already states the valid channels. |
| ❓ | app/CMakeLists.txt:8-14 - Is Moxi::Receiver static or shared? If it is shared, the installed jmex_plugin has no RPATH to find it. The plugins that link a vendor shared library set INSTALL_RPATH; sharpa_avatar points it at an external SDK. readelf -d install/plugins/jmex/jmex_plugin on your machine answers this. |
| ❓ | receiver_session.cpp:136-153 - The joint table is re-read only after the channel drops. Can Player load a different model, as the README's troubleshooting tells operators to do, without the pairing dropping? If it can, slots keep the old joint indices and go out valid. Re-checking MxRGetBoneJointCount on each new sequence id would cover it. |
| ❓ | jmex_g1_tasks.py:769-777 - retargeting_execution is unset, so Isaac Lab runs the pipeline in pipelined mode, and the overlay's debug-draw calls run on the retarget worker thread. Which mode was the live run in? Setting mode="sync", or drawing from process_actions, settles it either way. |
| 🧹 | .gitignore:5 / ClangFormat.cmake:43 - Use MOXIReceiverSDK*/ and MOXIReceiverSDK[^/]*, so an archive unpacked under its versioned name is not picked up by git add -A or the format check. |
| 🧹 | full_body_publisher.hpp:42-48 - body.jmex accepts max_flatbuffer_size, but the plugin always creates a 16 KiB collection, so a smaller value makes the reader throw. Either drop the parameter or add --max-flatbuffer-size= as noitom and xsens do. |
| 🧹 | device_clock.cpp:52-60 - A Player timestamp of 2^63 or more turns negative, push_buffer throws, and publish() treats every such frame as a lost session. Falling back to the sample time there keeps push failures transport-only. |
| 🧹 | Naming, after the rebase: main now calls the library and the repository "Isaac Capture"; Isaac Teleop stays the teleop component. This affects the plugin README L8-9, the example README L11/29/31/49 and record_jmex_full_body.py:4. The devices.yaml row's URL should be NVIDIA/IsaacCapture. The example README's wheel steps (L22-43) can link the build-from-source page and keep only the Isaac-Lab-specific parts. |
| 🧹 | src/plugins/jmex/CMakeLists.txt - The README says Linux x86_64 only. A CMAKE_SYSTEM_NAME check with a "Linux only" skip message, as the xsens and noitom plugins have, reports the right reason on other platforms. |
Optional, not needed for approval:
- Build the SDK-free sources as an always-built core library, with the receiver in the SDK-gated part. The test could then link it, and those files would get the project warning set, as gamepad and spacemouse do.
- Split
JmexG1ActionSourceintoFullBodySourceplus a retargeter, to match theIDeviceIOSource"pure converter" contract.
Please rebase onto main, or we can update the branch, so CI reruns against it before approval.
Adds `jmex`, a plugin for the j-mex AgileMaster motion capture suit, and an example, in two parts, that consumes what it publishes. The plugin reads one MOXI Player channel through the MOXI Receiver SDK (0.8.0 or newer, obtained from j-mex) and runs on Linux only. It fills the 24 `FullBodyPose` slots from the standard human skeleton Player has loaded, one slot per joint, already in OpenXR axes, and pushes them under the `body.jmex` vendor on every frame. A slot with no joint, or with no usable value in a frame, goes out invalid; nothing is inferred from its neighbors. The raw device clock carries the time Player stamped on the frame (`--device-clock=local` uses the pickup time instead), and if the CloudXR runtime restarts underneath the plugin, the plugin reconnects rather than ending the run. An argument it cannot use stops it with exit code 1, naming the argument. Everything but the receiver is a library that builds without the SDK, so on Linux its unit tests run in every build that has tests enabled; without the SDK only the plugin executable is skipped. Isaac Capture learns `body.jmex` as a full-body vendor: a live tracker that reads the plugin's collection, with a `collection_id` parameter, its entry in the DeviceIO factory, the vendor validation cases, and a row in `devices.yaml`. An SDK unpacked into the plugin directory, under any versioned name, is kept out of git and the clang-format check. `isaaccapture_examples.jmex.jmex_g1_tasks` registers a task that inherits Isaac Lab's `LocomanipulationG1EnvCfg`, on Isaac Lab 3.0.0 EA or 3.0.0 beta2, and takes its action from the operator; nothing in Isaac Lab is modified. It is upper-body teleop. Each arm is rebuilt segment by segment onto the robot's own shoulder, so the operator's arm angles survive while the lengths stay the robot's, and the elbows and shoulders get IK targets of their own from that same chain. The wrist's orientation is the operator's hand, carried across through a forearm frame. The operator's torso turns the robot's shoulders about its waist, so leaning, bending sideways and twisting come through; their heading does not, so the operator can turn to face a screen without moving the robot. Only a frame that carries all 24 slots drives the robot, and slots that stay missing are named on the console. `isaaccapture_examples.jmex.record_jmex_full_body` writes the same stream to a standard full-body MCAP. Tests cover the plugin's argument parsing, slot mapping, change of axes, device clock choice and session rebuild after a failed push, the vendor's parameters, and the example's action source, including the geometry of the rebuilt arm and the orientation carried through the forearm frame. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: j-mex <jmex.dev@jmex.com.tw>
a91f3af to
8dcc95c
Compare
|
Thanks for the second pass. I rebased onto
The PR description has the updated local gate (both toolchains, six entries each) and the device |
|
Heads-up: #1394 just merged. It moves Could you switch [tool.hatch.build]
only-include = ["python/isaaccapture_examples/jmex"]
[tool.hatch.build.targets.wheel]
sources = ["python"]Nothing checks where |
main conflicted in tests/cpp/plugins/CMakeLists.txt and now asks examples to set only-include for the sdist too. On top of the merge: - tests/cpp/plugins/CMakeLists.txt registers the plugin's tests under if(TARGET jmex_plugin_core) at the end of the file instead of in the Linux-only block, so no existing line of the file changes. The core library exists on Linux, with or without the SDK. - examples/jmex/pyproject.toml sets only-include in [tool.hatch.build], so the sdist gets the same file list as the wheel. - The example's recording and replay steps and the plugin's runtime step start from a venv set up as in Running an example. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: j-mex <jmex.dev@jmex.com.tw>
|
Thanks for the heads-up. I merged
The PR description has the local gate rerun on this tree. The same commit passed the |
|
✅ Preview deployed: https://NVIDIA.github.io/IsaacCapture/preview/pr-1250/ Built from commit fa1b5e0. It may take a minute for the updated preview to appear. |
|
Review in progress. |
ivany-nv
left a comment
There was a problem hiding this comment.
Summary
Thanks. Every item from the last round is fixed. Merged onto main (75d567089), the jmex code builds without warnings, all jmex tests (23 Catch2, 91 pytest) pass, and upstream CI is green. What is left is inline: 1 🚫, 3 💡, 1 ❓, 8 🧹.
Legend: 🚫 BLOCKER = merge-blocking | 💡 SUGGESTION = meaningful improvement | ❓ QUESTION = needs your answer | 🧹 NIT = precise small correction
The CTS review has passed.
main conflicted in tests/cpp/plugins/CMakeLists.txt and tests/python/examples/CMakeLists.txt, where it added a subdirectory next to jmex's; both are kept. On top of the merge: - BUILD_PLUGIN_JMEX, default OFF, gates src/plugins/jmex and the example's Python tests; the C++ tests follow through if(TARGET jmex_plugin_core). With the option on and no SDK, configure warns and names it. The READMEs' build commands and the example's install hints pass -DBUILD_PLUGIN_JMEX=ON. - The plugin needs MOXI Receiver SDK 0.8.1, which advertises only from the TCP server's address when one is given. plugin.yaml binds the receiver to loopback, so a plugin TeleopSession launches pairs only with a Player on this machine; the README says what a Player on another machine needs, and that the channel's data port takes frames from any sender. The plugin no longer checks the data port's range itself: 0.8.1's MxROpenChannel fails on it, and the plugin says so. - --help or -h prints the flags and their defaults and exits 0. An empty --collection-id=, --bind-ip= or --broadcast-ip= stops the plugin. --channel=0, which Player uses for a suit on USB, is accepted. - The G1 task pins the root link on the Isaac Lab v3.0.0 release, where G1_29DOF_CFG's articulation_props is a list, and the example README lists v3.0.0 under Verified on. - Comments and wording: why the task retargets in sync, the base config named without a beta2-only path, the docs-site build link, MOXI Player for Robot, and Linux x86_64 in the device row. A test no longer claims its model is a G1 URDF, and the data-port case has a test of its own. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: j-mex <jmex.dev@jmex.com.tw>
|
Thanks for the review, and for the CTS result. The PR head is now
One change you did not ask for: The PR description has the local gate rerun with |
main merged without conflicts. On top of the merge: - The G1 example reads the suit through FullBodySource with the body.jmex vendor. JmexG1Retargeter, a BaseRetargeter on the optional full_body input, holds the state kept across frames and outputs the frame tasks' poses. LocomotionFixedRootCmdRetargeter gives the standing command and TensorReorderer lays out the action, with the Pink IK action term's hand_joint_dim fingers left at zero. JmexG1ActionSource, the module-level runtime and the example's own action packing are gone. - The example's plugin name, ids, defaults, search path and launcher live in one module, plugin.py, which looks only under install/plugins. Imports are at the top of each module, except Kit's two, which exist only inside a running Kit app, and Isaac Lab's Pink IK action class, which Isaac Lab's own config names by a string. The example's loggers are isaaccapture.examples.jmex.*. - The plugin takes its settings from flags only; JMEX_DEVICE_CLOCK is gone, and plugin.yaml no longer repeats the default channel and collection. parse_plugin_args returns std::optional<PluginArgs>, empty for --help. A scope object declared once the receiver is up logs the counters, however the run ends. - ReceiverSession keeps each pairing's state in one std::optional<Pairing>, and holds the SDK's system and channel as RAII members; the process-wide session claim is gone. BodyJointMapping is built by its constructor and takes core::Pose. Impossible inputs are asserted, and optional addresses and joint indices are std::optional. - CMake finds the SDK through CMAKE_PREFIX_PATH, MOXIReceiverSDK_DIR or the plugin directory; JMEX_SDK_ROOT is gone. On a target other than x86_64, configure warns and builds only the core library and its tests. jmex_plugin_core drops its PIC and visibility properties. - The docs no longer state the SDK version or the recovery time. The tests no longer sleep or check constant values, the Catch2 tests carry a jmex_ prefix, and a duplicate body.jmex vendor-validation section is gone. The core library's alias is jmex::jmex_plugin_core. A joint-mapping test feeds joints out of slot order. - A lower sequence id after UINT32_MAX no longer counts as dropped frames, the recording script times itself with time.monotonic(), and FrameTargetCfg refuses non-finite link lengths and quaternions. - Numeric flags are parsed with std::from_chars, so a leading space or plus sign is refused, and the plugin's CMakeLists.txt no longer repeats cmake_minimum_required. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: j-mex <jmex.dev@jmex.com.tw>
|
Thanks for the review. The PR head is now
On #33. Round 4 asked for the opposite. In round 2 an optional suggestion was to build
It also asked for Changes you did not ask for:
The PR description has the local gate rerun on this tree. The |
|
@jmex-sw Everything looks good, please address the last 2 left item. See comment under AristarkhovNV's comment. Thanks |
…t the option - A MOXIReceiverSDK_DIR that is set is the only place configure looks for the SDK. If it holds no MOXIReceiverSDK of the minimum version or newer, configure stops with a FATAL_ERROR that names the variable and its value, rather than building against an SDK found through CMAKE_PREFIX_PATH or in the plugin directory. find_package rewrites a value it cannot use to -NOTFOUND in the cache, so the value is written back first and the next configure stops too. Without the variable, the search and the warning are as before. - The plugin's SDK-free core library and its Catch2 tests build on Linux whether or not BUILD_PLUGIN_JMEX is on, and the example's Python tests are no longer gated on it. The option, still OFF by default, adds only the jmex_plugin executable and its install. - The plugin README's Building section and Troubleshooting table cover the new error, and the example README's offline-test section no longer asks for the option. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: j-mex <jmex.dev@jmex.com.tw>
|
Thanks. The PR head is now The PR description's Testing section has the local gate rerun on this tree. Every matrix entry left |
ivany-nv
left a comment
There was a problem hiding this comment.
Thanks. The MOXIReceiverSDK_DIR check and the ungated core and tests are fixed in fa1b5e067. CI is green, and the 25 jmex tests now run and pass on x64 and arm64. LGTM.
Description
Adds support for the j-mex AgileMaster motion capture suit: a device plugin that publishes the
operator's body pose as
FullBodyPoseunder a newbody.jmexvendor, the core registration thatlets Isaac Capture read it, and an Isaac Lab example that drives a Unitree G1 from it. None of this
is upstream yet; the plugin, the example and their tests are all new.
AgileMaster has three parts: the Mocap Suit the operator wears; MOXI Player, which solves the suit
onto a skeleton and streams it on a numbered channel; and the MOXI Receiver SDK, which the plugin
links against.
The plugin (
src/plugins/jmex/) receives one Player channel and fills everyBodyJointslotfrom the standard human skeleton loaded in Player, one slot per joint, in OpenXR axes. It pushes one
FullBodyPoseper frame, on thejmex_full_bodycollection by default. A slot with no joint, orwith no usable value in a frame, goes out with
is_validfalse; nothing is inferred fromneighboring joints. The raw device clock carries the frame's time from Player
(
--device-clock=localuses the pickup time instead). If the CloudXR runtime restarts underneathit, the plugin re-establishes its session and carries on. It is configured by flags only. An
argument it cannot use, such as a misspelt flag, an empty value or a channel outside 0–255, stops
it with exit code 1 and names the argument;
--helplists the flags and their defaults. Launched byTeleopSession, it listens and advertises on loopback only, so only a MOXI Player on the samemachine can pair; its README says what a Player on another machine needs. It builds only with
BUILD_PLUGIN_JMEXon (defaultOFF), runs on Linux x86_64 and needs the MOXI Receiver SDK, whichfind_packagelocates throughCMAKE_PREFIX_PATH,MOXIReceiverSDK_DIRor the plugin directory.A
MOXIReceiverSDK_DIRthat is set is the only place searched, and configure stops if it holds noSDK. With the option on and no SDK found, or on a target other than x86_64, configure warns and
skips the plugin executable. The plugin's SDK-free core library, which the tests link, builds on
Linux with or without the option; the rest of the tree builds as before.
Where the conversion happens. Player streams its own skeleton. The plugin maps it onto the
FullBodyPoseslots and into OpenXR axes (body_joint_mapping,openxr_convention); Player doesnot emit
FullBodyPose. There are two reasons for this. The change of axes was measured in thistree and is pinned by the plugin's tests, next to the code that consumes the slots. Done in Player,
those tests would live where this repository's CI never runs them. And Player does not have to
produce Isaac Capture's internal schema, so a change to
FullBodyPosestays in this tree instead ofwaiting on a Player release.
Registration in core. Outside the new directories, every change is an addition; no existing line
changes.
src/core/live_trackers/cpp/live_full_body_tracker_jmex_impl.{hpp,cpp}body.jmex. It reads the plugin's collection and takes one vendor parameter,collection_idsrc/core/live_trackers/cpp/CMakeLists.txtlive_deviceio_factory.{hpp,cpp}try_create_*function; one row ink_tracker_dispatch, after the existing full-body vendors, soFullBodyTracker's default vendor is unchanged; one branch invalidate_vendor_selectionstests/cpp/core/live_trackers/test_vendor_validation.cppSECTIONs forbody.jmex, beside the existing vendors'CMakeLists.txtBUILD_PLUGIN_JMEX, defaultOFF, after the other device plugins' optionssrc/plugins/CMakeLists.txtadd_subdirectory(jmex)with the plugins that are always entered; off Linux the plugin skips itself, and on Linux it builds its core library, and the plugin executable too whenBUILD_PLUGIN_JMEXis on and the SDK is found on x86_64tests/cpp/plugins/CMakeLists.txtadd_subdirectory(jmex)for the plugin's tests, when its core library is builttests/python/examples/CMakeLists.txtadd_subdirectory(jmex)for the example's testscmake/ClangFormat.cmakedocs/source/_data/devices.yamlThe example (
examples/jmex/, packageisaaccapture_examples.jmex) has two parts, and apluginmodule that holds the plugin's name, ids and defaults and launches it frominstall/plugins:jmex_g1_tasksregistersIsaac-PickPlace-Locomanipulation-G1-Jmex-Abs-v0, which inherits IsaacLab's
LocomanipulationG1EnvCfg(Isaac Lab 3.0.0, 3.0.0 EA or 3.0.0 beta2) and takes its action from theoperator; nothing in Isaac Lab is modified. Its pipeline:
This is upper-body teleop. Each arm is rebuilt segment by segment onto the G1's own shoulder, so
the operator's arm angles carry over at the robot's arm lengths, and the elbows and shoulders get
IK targets from that same chain. The wrist's orientation is the operator's hand, carried through
a forearm frame. The operator's torso turns the robot's shoulders about its waist, so leaning,
bending sideways and twisting come through; their heading does not, so the operator can turn to
face a screen without moving the robot. Only a frame that carries every slot drives the robot.
The state kept across frames is the retargeter's; the source node is upstream's.
record_jmex_full_bodyrecords the same stream to a standard full-body MCAP(
core.FullBodyPoseRecordon thefull_bodychannel), which the sharedreplay_full_bodyscriptplays back unmodified.
Type of change
Testing
The tests in this PR need neither the SDK nor
BUILD_PLUGIN_JMEX. A configure on Linux,CI's included, registers them without either, and
ctest -L jmexselects 25: the plugin's22 Catch2 cases (argument parsing and
--help, slot mapping, change of axes, device clock,one frame becoming one
FullBodyPose, session recovery after a failed push), which link theplugin's SDK-free core library; and the example's three Python tests, whose 98 pytest cases
cover the retargeter, including the geometry of the rebuilt arm and the orientation carried through
the forearm frame, the composed action's length and order, the plugin launcher, and the recording
script's
--helpwithout Isaac Lab. The Python tests also needBUILD_EXAMPLESandBUILD_VIZ. Thebody.jmexsections of the vendor validation test are in core and run underctest -R 'vendor validation'with or without the option.Local gate, Ubuntu x86_64. We ran the same Python and build-type matrix as the x64 entries of
build-ubuntu(Python 3.11, 3.12 and 3.13, each in Debug and Release), withBUILD_PLUGIN_JMEXoffas in CI and without the SDK, each from an empty tree, twice: once with CI's toolchain (GCC 11.4, CMake
3.31.6, uv 0.12.13, clang-format 14.0.0) and once with GCC 13.3. Local builds left
BUILD_OVXR_CAMERA_QUADSandBUILD_PLUGIN_OAK_CAMERAdisabled; CI enables both. In all twelve:jmextests and the 3 vendor validation cases passed;.cppfiles that build without the SDK compiled without warnings (theother two are covered below);
ctest, 710 passed, 9 skipped and1 failed: an upstream GPU test (
VkContextwith an explicit device index) that fails on thisworkstation's GPU setup and is unrelated to this change; CI's runners have no GPU and skip it.
With the option on and no SDK, configure warned and skipped the executable, and registered the same
25 tests; with SDK 0.8.1 it also built and installed the executable. With
MOXIReceiverSDK_DIRpointing at an empty directory, configure stopped with the error, also with a usable SDK on
CMAKE_PREFIX_PATHor in the plugin directory; it stopped likewise at SDK 0.8.0, and at an SDK movedaway after a configure had found it. Each stopped again on a second configure.
The two plugin sources that compile only with the SDK were built against SDK 0.8.1 with GCC 13 and
syntax-checked with GCC 11, without warnings. With CI's toolchain we also repeated the Release
install with
auditwheel repair, thepip-editable-installjob (its import check, and itswheel-content tests: 10 passed) and the configure of
test-viz-sanitizers.SKIP=check-copyright-year pre-commit run --all-files,make current-docs(-W),scripts/check_doc_refs.py --all, ruff, and clang-format 14 on the changed files all pass. Windowsand arm64 were not built locally.
Since this PR comes from a fork, its workflows wait for a maintainer's approval. The same commit has
already been through the
Build Ubuntuworkflow on our fork, where all twelvebuild-ubuntuentries(x64 and arm64) passed, along with DCO and pre-commit:
https://github.com/jmex-sw/IsaacTeleop/actions/runs/44007964864. Those entries build the core
library and run the 25
jmextests, on x64 and arm64 alike.With the device. With the current pipeline and
isaaccapture1.8 in Isaac Lab's Python, arecorded Player stream replayed over loopback drove the task headless on all three Isaac Lab
versions: each of 900 steps was computed from a full body pose, at about 12 steps/s on beta2, 11
on EA and 10 on 3.0.0, and the action-length check passed. On Isaac Lab 3.0.0 with Kit's window
open and MOXI Player for Robot on the same machine, the plugin the task launched paired over
loopback and the G1 followed. When Player disconnected, the G1 held its last pose, and it followed
again once Player paired again.
Ctrl-Cended the plugin on its counters line, with no framesdropped.
On 2026-11-06, with SDK 0.8.1 and MOXI Player for Robot on the same machine, the plugin with
plugin.yaml's loopback flags paired and published within 6 s in both of two runs, its TCP serverlistening only on 127.0.0.1. With Player on a second machine on the same subnet, the README's
settings for that case paired within 5 s in both of two runs, and
plugin.yaml's loopback flagskept that Player out: it did not connect in 40 s.
The runs below were made on earlier heads of this PR. Since then the skeleton mapping is unchanged,
and the example's retargeting moved onto
FullBodySourceand the upstream nodes with the samegeometry; its geometry tests changed only in how they feed the input. On 2026-10-30, driven live by
an operator wearing the upper-body part of the suit, with MOXI Player on the same machine (Isaac
Lab 3.0.0 beta2 on Isaac Sim 6.0.1, XR running, retargeting in sync mode), the G1's waist followed
lean, side bend and twist with a correlation of 0.97–0.99, 90–270 ms behind. The operator went past
the waist's limits in lean and twist, where the waist stops following. An earlier run in the full
suit, on 2026-09-23 with pipelined retargeting, gave 0.99 at the same latencies, and turning on the
spot through ±90° moved the G1's yaw less than 12°. Replayed headless from a recording (rerun on
2026-10-30), the waist followed all three with a correlation of 1.00, one or two physics steps
behind. On 2026-10-30, on Isaac Lab 3.0.0 EA (Isaac Sim 6.1.0), the G1 followed a replayed
recording with Kit's window open and the reference overlay drawn.
Running it yourself. Isaac Lab needs this repository's
isaaccaptureinstalled in its Python:no released package carries the
body.jmexvendor yet, and Isaac Lab pins the package under itsprevious name. The example README has the steps. The SDK and MOXI Player are not distributed with
this PR.
Checklist
SKIP=check-copyright-year pre-commit run --all-filesgit commit -s) per the DCO