Hey, I'm Kaleb Coleman 👋

Digital twins of robotic systems | M.S. AI Engineering (Robotics) at ASU | Research with the RISE Lab

I build digital twins of robotic systems in simulation: a Unitree G1 humanoid, a UR5e arm doing contact-rich assembly, and quadrotors, in NVIDIA Isaac Sim, Isaac Lab, and MuJoCo. I also build the tools that let AI agents work in those simulators.

I am a master's student at Arizona State University in Artificial Intelligence Engineering with a focus on robotics, doing research with the RISE Lab. I graduated from Northern Arizona University with a B.S. in Data Science and a minor in Cybersecurity, so my work sits where data science meets robotics: machine learning and data pipelines on one side, simulation, control, and autonomous systems on the other.

I care about reproducible, honestly reported results: every claim should trace to a run, a log, or a saved artifact, and limitations get written down next to the numbers.

Each project below is the simulation side of a twin: a physics model of a real robot, with measured results and written limits. Long-term goal: data-informed digital twins that connect real robots and simulation to improve models and control. That sim-to-real loop is the direction of my research, not something already built. Everything shown below ran in simulation, and none of it is physically calibrated.

g1-locomotion-lab | Simulation demo complete; fall-recovery research ongoing

• Built: trained a simulated Unitree G1 walking policy using Isaac Lab's existing velocity task and PPO setup on Newton/MuJoCo-Warp, then wrote the evaluation and reporting around it: a scripted stand, walk, turn, and stop test with random pushes, a sweep across saved checkpoints, and a study of when and where robots fall. The task, rewards, and PPO are upstream components, not a new algorithm of mine.
• Demonstrated: 237 of 256 simulated robots stayed up through the 24-second pushed test in the published evaluation (two reruns gave 231 and 233), with turning the most common point of failure. This is one policy from one training seed, not a general reliability claim.
• Working toward: fall recovery. In simulation, the current get-up policy passes every validity check on 68% of 256 unseen lying poses, and 52% of 750 induced falls end with the robot walking again. The get-up still straddles and the hand-over to walking is the weak link, so I do not count that problem as solved.

_{Same scripted commands at training iteration 0 (left) and 1499 (right), 16 simulated robots per side. There are no pushes in this clip and robots that fall are reset in place, so it is not footage of the 256-robot test. G1 model by Unitree Robotics, rendered with Isaac Lab and Isaac Sim.}

contact-assembly-lab | Simulation testbed; real-robot comparison and Jenga research ongoing

• Built: a contact-rich robotic assembly testbed in Isaac Sim 6.2. A UR5e with an RG2 gripper assembles structures from interlocking parts while a joint-strength model decides, at every physics step, whether each connection holds. My work is the Isaac Sim 6.2 port, a weld-torque correction for PhysX 5.9's CPU TGS solver, the assembly controller, a library of 29 structures, reproducible sweeps, and a run dashboard. It builds on BrickSim by the Intelligent Control Lab (paper), whose connection and breakage physics it uses.
• Demonstrated: 21 of the 29 catalogued structures complete in simulation at their saved settings, and repeated runs gave identical results on the reference machine. The other 8, including two cantilever break tests designed to fail, do not, and the failure analysis says where and why. A completed run is one deterministic run, not evidence of robustness, and no real-robot trials have been run.
• Working toward: more ways to grip parts, building the same structures on the real UR5e to compare what fails, and carrying the evaluation methods over to Jenga manipulation, potentially with MPC or RL. The Jenga work is ongoing research in private repositories, and none of it is solved yet.

_{Left: the seven-part arch bridge at the end of its simulated build. Right: the run dashboard listing all 29 structures. Simulation only, rendered in Isaac Sim 6.2.}

twinsmith | Source version 0.3.0, installed from source; the PyPI and npm 0.0.1 releases only reserve the name

• Built: an MCP server an AI agent uses to build and drive robot simulations and digital twins: scenes, cameras and LiDAR, robot import and motion, and Isaac Lab training and evaluation, with Newton (MuJoCo-Warp) as the default engine for the scenes it builds, asset packs kept outside the repository, a RoboSimStudio real-to-sim bridge, and a UR5e Jenga work cell. Tools come in switchable toolsets with a status tool: a client sees only the 6 core tools (about 1,586 estimated tokens) until it turns more on, and opt-in toolsets add ROS 2, Graph-as-Policy skill graphs, the VABAR benchmark, desktop capture and computer use. The UR5e work cell began as a fork of Aldrin Inbaraj's isaac-sim-mcp; the bridge and client were first built on his omni-kit-mcp and rewritten here.
• Demonstrated: the example scenes and training runs in Examples are captures from live runs, with a record of how each was made. They illustrate the tools and are not benchmarks. The UR5e Jenga work cell is a candidate; runtime qualification is still open.
• Compatibility: Linux x86_64 (Ubuntu 24.04 or 22.04). Newton is the default engine for the scenes these tools build; PhysX is opt-in. The UR5e work cell builds on either engine; on Newton its cameras and real-robot mirroring are not qualified yet. The claimed builds are Isaac Sim 6.2.0.0 with Isaac Lab 3.0.0, and Isaac Sim 5.1.0.0 with Isaac Lab 2.3.2. Isaac Sim 5.1 has no Newton, so that pairing is PhysX only. See the compatibility matrix.

_{A warehouse built by one sim__quick_start environment=warehouse call. The walls, racks, and cargo are boxes authored by the tool; the wheeled robot is NVIDIA's public Carter model.}

Team research with John Lok, ongoing, simulation only

• What we are studying: nonlinear and prescribed-time quadrotor control, including runtime, parameter sensitivity, and modeled energy.
• My part: John led the controller development. I reproduced an archived MuJoCo NMPC baseline and am running a timing and validation study that is still in progress.
• Working toward: hardware validation at ASU Drone Studio. This is planned; the work so far is simulation only.
• Background: Robust Optimal Agile Flight of Quadrotors (IM-NMPO) and Prescribed-Time Trajectory Tracking Control for Quadrotor Without Velocity Measurement.

The collaborative code is in a private repository. I am happy to talk through the work on request.

Robotics

• g1-locomotion-lab: training and evaluating simulated Unitree G1 locomotion in Isaac Lab with Newton/MuJoCo-Warp; 237 of 256 robots stayed up through the published pushed test.
• contact-assembly-lab: contact-rich robotic assembly testbed in Isaac Sim 6.2, with a UR5e, joint-strength physics, and a catalog of 29 structures.
• twinsmith: MCP server for building robot simulations and digital twins in Isaac Sim and Isaac Lab, with Newton (MuJoCo-Warp) as the default engine, a RoboSimStudio real-to-sim bridge, switchable toolsets (6 tools listed by default), external asset packs, and a UR5e Jenga work cell.

Data science

• nba-modeling: leakage-aware NBA win modeling with time-ordered validation, a reproducible targets pipeline, and predictions logged before tip-off. On the one 2025-26 test season, the best model picked 68.6% of games correctly (95% interval 66.2% to 71.2%), against 55.4% for always picking the home team and 67.9% for Elo ratings alone. Site and paper.
• nau-capstone: NAU team capstone comparing shot quality and expected scoring across the NBA and NHL, with a Streamlit dashboard and research poster.

• Trained a simulated Unitree G1 walking policy and evaluated it with a scripted pushed test: 237 of 256 robots stayed up in the published run (231 and 233 in reruns), from one policy and one training seed.
• Ported a contact-rich assembly simulator to Isaac Sim 6.2 and built a UR5e controller, structure catalog, and failure analysis on it: 21 of 29 structures complete in simulation.
• My team received the Best Visualization Award at NAU's ASA DataFest 2026 for work on early signals of high healthcare utilization.
• Built a leakage-aware NBA win model and an ESPN data pipeline in R. Most of the signal is in the Elo ratings: the interval for the AUC gain from adding rolling features includes zero. With the 2025-26 test season now seen, 2026-27 predictions logged before each game and scored on the live page are the only untouched test.
• Worked with Kanaan Law and Drianna Walker on our NAU capstone's NBA/NHL shot-quality analysis (research poster) and Streamlit dashboards to communicate results.
• Trained models on university HPC clusters with Slurm and on other GPU resources.

• Programming languages: Python, R, C, SQL.
• Tools and libraries: PyTorch, scikit-learn, XGBoost, pandas and NumPy, NVIDIA Isaac Sim and Isaac Lab, Newton and MuJoCo-Warp, MuJoCo, Streamlit, PostgreSQL and SQLite, Linux, Git and GitHub.
• Robotics skills: digital twins in simulation, USD scene authoring, UR5e/RG2 manipulation and contact-rich assembly in simulation, inverse kinematics, PPO training and policy evaluation (rsl_rl), MCP-based simulator tooling, Linux/GPU workstation setup.
• Areas of interest: robotics and manipulation, control and optimization, digital twins and sim-to-real, deep learning, computer vision, data engineering, cybersecurity.

Now: doing research with the RISE Lab toward my M.S. thesis, a real-to-sim-to-real UR5e that plays Jenga with a person, starting with a higher-fidelity UR5e work cell and Jenga manipulation in simulation. I am also researching fall recovery for simulated humanoids and studying quadrotor control with John Lok ahead of planned hardware validation.

Long term: data-informed digital twins that connect real robots and simulation to improve models and control.

Next: completing my M.S. at ASU, aimed at sim-to-real transfer, learning-based and optimal control, drones, and space systems.

Outside of school and coding, I enjoy sports, gaming, technology, learning about AI, and working on projects that help me grow as a developer and researcher.

• LinkedIn: Kaleb Coleman
• GitHub: kalebcoleman
• Portfolio: kalebcoleman.github.io
• X: @kalebcoIeman