Official code for MARIO: Motion-Augmented Real-Time Multi-Sensor Inertial Odometry (Yiquan Li*, Taeyoung Yeon*, Chenfeng Gao, Vasco Xu, Xuanyou Liu, Karan Ahuja).
- Paper: https://arxiv.org/abs/2606.02996
- Project page: https://spice-lab.org/projects/MARIO/
MARIO augments learning-based inertial odometry with (1) PoseNet, a frozen IMU-to-body-pose prior, and (2) a Multi-Sensor Fusion Module that adds the barometer, magnetometer and secondary IMU already available on AR glasses.
This repository is the minimal code needed to preprocess the datasets and to train and evaluate the AirIO-based MARIO models. It reproduces the AirIO rows of the paper's tables. The TLIO, EqNIO and RoNIN-LSTM backbones were trained in their own upstream repositories with the fusion features from this code plugged in.
The pretrained PoseNet prior ships with the repository at
experiments/posenet/ckpt/best_model.ckpt, so the _pose and _all models can be
trained and evaluated without training PoseNet first. The odometry checkpoints are not
distributed; every odometry model trains from scratch with the commands in section 3.
Retraining PoseNet yourself additionally needs two third-party components that are not
redistributed here; see section 3, step 0.
Left: the egocentric RGB stream of an Aria Everyday Activities recording
(loc1_script1_seq6_rec1, 128 s, 53 m of walking around an apartment). Right: the
top-down trajectory integrated from MARIO's velocity estimates, drawn against the ground
truth as the person moves. The model (AirIO + all sensors) was trained on Nymeria only and
is run here without fine-tuning; it ends the sequence with 0.51 m ATE and 2.9% drift. The
camera is shown for context only and is never an input to the model.
Full-length video: assets/demo/trajectory.mp4.
conda create -n mario python=3.10 -y
conda activate mario
pip install -r requirements.txt
export DATA_ROOT=/path/to/processed_data # every dataset config reads thisExpected layout under $DATA_ROOT (created by the preprocessing scripts in section 2):
processed_nymeria_2imu_both/{train,val,test}/ Nymeria, CPF frame, 200 Hz, dual IMU + baro + mag
aria_processed_cpf_both/{train,val,test,all}/ Aria Everyday Activities, same format
tlio_dataset_cpf/{train,val,test}/ TLIO converted to the CPF frame
processed_data_body_Ty/, processed_data_test_body_Ty/ Nymeria at 50 Hz with body pose (PoseNet only)
| Dataset | Command |
|---|---|
| Nymeria (200 Hz, dual IMU, baro, mag) | python preprocess_nymeria_mag_baro_2imu_both.py <nymeria_root> -o $DATA_ROOT/processed_nymeria_2imu_both --sample-hz 200 |
| Aria Everyday Activities | python preprocess_aria_mag_baro_2imu_both.py <aea_root> -o $DATA_ROOT/aria_processed_cpf_both --sample-hz 200 |
| TLIO | python tlio_to_cpf.py <tlio_root>/test --output $DATA_ROOT/tlio_dataset_cpf/test (same for train / val) |
| Nymeria body pose (PoseNet) | python preprocess_nymeria_body.py <nymeria_root> -o $DATA_ROOT/processed_data_body_Ty --imu-hz 50 --pose-hz 50 |
| train / val / test split | python tool_split_dataset.py $DATA_ROOT/processed_nymeria_2imu_both |
The Nymeria sequences used for training and testing are listed in
train_sequences.txt and test_sequences.txt. Coordinate conventions
(device -> CPF -> gravity-aligned CPF-world) follow the supplementary material.
Barometric altitude, vertical velocity and magnetometer yaw are derived at load
time in datasets/nymeriadataset.py.
Deriving PoseNet's labels needs SMPL kinematics and Meta's Nymeria loader. Both carry
licenses of their own and are therefore not shipped with MARIO (see THIRD_PARTY.md).
Fetch them into place with:
bash scripts/fetch_third_party.shThis clones the articulate toolkit from TransPose into core/articulate/ and the
Nymeria loader into nymeria/. Do not pip install nymeria: that name on PyPI belongs
to an unrelated package. Skip this step entirely if you only train and evaluate the
inertial-odometry models; nothing else in the repository imports these two packages.
Two things to know about the Nymeria loader. It expects the standard dataset layout,
with each sequence's body motion at <sequence>/body/xdata.npz; if you downloaded head
and body data into separate trees, preprocess_nymeria_body.py looks for the body tree
by replacing head with body_motion in the path. It also needs pymomentum, which
must be built against the same PyTorch you are running, or importing it after torch
fails with an undefined-symbol error.
Optional. The released checkpoint is already at experiments/posenet/ckpt/best_model.ckpt
(weights only, trained with configs/nymeria/posenet.conf, epoch 85). Skip to step 2
unless you want to retrain it; retraining overwrites that file.
PoseNet is trained once on Nymeria at 50 Hz with gravity removed, then frozen and reused as a prior. Its labels (SMPL joint positions and 6D joint rotations) are derived on the fly from the Xsens body pose with the SMPL body model.
- Download
basicmodel_m.pklfrom https://smpl.is.tue.mpg.de (license required) and place it atassets/smpl/basicmodel_m.pkl, or setSMPL_MODEL_PATH. - Preprocess the 50 Hz body-pose data (section 2).
- Train:
python train_motion.py --config configs/nymeria/posenet.confThis writes experiments/posenet/ckpt/best_model.ckpt, which is where the
prior.pretrained_path field of every _pose and _all config already points.
python train_motion.py --config configs/nymeria/airio.conf # single IMU (AirIO baseline)
python train_motion.py --config configs/nymeria/airio_pose.conf # + PoseNet prior
python train_motion.py --config configs/nymeria/airio_baro.conf # + barometer
python train_motion.py --config configs/nymeria/airio_mag.conf # + magnetometer
python train_motion.py --config configs/nymeria/airio_imu2.conf # + secondary IMU
python train_motion.py --config configs/nymeria/airio_all.conf # + all of the aboveCheckpoints go to experiments/<dataset>/<config name>/ckpt/. The same six configs
exist for the other datasets as configs/aria/train_airio*.conf and
configs/tlio/airio*.conf. Logging uses Weights & Biases when you are logged in and
falls back to stdout otherwise.
# 1) run the network over the test split (sliding window of 1000 samples)
python inference_motion.py --config configs/nymeria/airio_pose.conf --seqlen 1000 --stepsize 1
# 2) integrate velocities and report ATE, RTE-1s, RTE-5s and drift
python evaluation/evaluate_motion.py --dataconf configs/nymeria/airio_pose.conf \
--exp experiments/nymeria/airio_pose/ --seqlen 1000 --visInference loads <exp_dir>/ckpt/best_model.ckpt by default and writes
net_output.pickle next to it. Evaluation reads that file and writes
result/<exp>/result.json. RTE_200 and RTE_1000 are RTE-1s and RTE-5s at 200 Hz.
Cross-dataset evaluation. configs/aria/eval_airio*.conf run over all 143 Aria
Everyday Activities recordings. Use --ckpt to evaluate a Nymeria-trained model on
Aria without fine-tuning:
python inference_motion.py --config configs/aria/eval_airio_pose.conf \
--ckpt experiments/nymeria/airio_pose/ckpt/best_model.ckpt --seqlen 1000 --stepsize 1
python evaluation/evaluate_motion.py --dataconf configs/aria/eval_airio_pose.conf \
--exp experiments/aria/eval_airio_pose/ --seqlen 1000Note that the all split of Aria is the union of its train, val and test splits, so it
is only a valid evaluation set for a model that was not trained on Aria.
train_motion.py, inference_motion.py, evaluation/evaluate_motion.py train / infer / evaluate
preprocess_*.py, tlio_to_cpf.py, tool_split_dataset.py raw data -> $DATA_ROOT
model/code.py CodeNetMotionwithRot (AirIO backbone), CodeNetMotionwithPose_v4 (PoseNet),
CodeNetMotionwithRot_{Pose,Baro,Mag,LeftIMU,Combined} (fusion variants)
model/losses.py velocity, covariance and joint losses
datasets/ sequence loaders (nymeria, ariaCPF, tlioCPF), windowing, collate
utils/ integration, dual-IMU preprocessing, plotting
core/paths.py where to find the SMPL body model
scripts/ fetches the third-party components needed for PoseNet training
configs/datasets/ data locations and loader options
configs/{nymeria,aria,tlio}/ experiment configs
MARIO's own code is released under the MIT License (see LICENSE). Portions derived
from AirIO remain under BSD-3-Clause, and the components this repository does not
distribute carry licenses of their own; both are documented in THIRD_PARTY.md.
@article{li2026mario,
title = {MARIO: Motion-Augmented Real-Time Multi-Sensor Inertial Odometry},
author = {Li, Yiquan and Yeon, Taeyoung and Gao, Chenfeng and Xu, Vasco and Liu, Xuanyou and Ahuja, Karan},
journal = {arXiv preprint arXiv:2606.02996},
year = {2026}
}Built on AirIO and PyPose.
Data from Nymeria,
Aria Everyday Activities and
TLIO. Third-party components and their licenses are
listed in THIRD_PARTY.md.
