Repository navigation
rl-trainer-cold-start-cost: start phases, image manifest, lazy modelopt, sampler snapshot gate - #262
Draft
michaellchung wants to merge 10 commits into
Draft
michaellchung wants to merge 10 commits into
michaellchung wants to merge 10 commits into
Conversation
The ports image overlay layer now writes megatron/bridge/__init__.py from docker/miles-ports/megatron_bridge_init.py: the Megatron-Bridge 8cd3466 file without its top-level `import modelopt.torch`, which cost 21-30 s per trainer process. Bridge modules that use modelopt import it themselves. The image manifest records the patch. Takes effect on the next image build (rl-trainer-cold-start-cost L2.2). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> (cherry picked from commit 2838755)
…l prefetch - L0.1/L0.2 (rl-trainer-cold-start-cost): seven trainer_start_phase events (submitted, container_ready, setup_done, ray_ready, actor_import_done, actor_init_done, first_step_done) and a trainer_start_summary with total_s; the launcher copies the summaries into run_manifest.json (trainer_start) so the scheduler can read T_s of the last run. - L2.3: island setup checks peft and sglang against /opt/yeto/image-manifest.json without importing them; a missing entry falls back to the import check and records image_manifest_fallback; a mismatch fails setup before Ray starts. - L2.4 (code only, no image built): the ports image build puts the pinned peft wheel into site-packages and records it in the image manifest. - L2.5: the Hub prefetch uses snapshot_download (huggingface-cli exits 1 in the ports image) and records model_prefetch_failed with the exit code. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> (cherry picked from commit 4a39de3)
…tch and envs Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> (cherry picked from commit dc96be6)
--rl-sampler-snapshot (off by default) starts the fixed-partition rollout engines with enable_memory_saver + enable_weights_cpu_backup via an inline --sglang-config override. Launch is refused unless the shipped sampler_snapshot_compat.json records a pass for the pinned SGLang commit on the NCCL (distributed) weight-update path. The L8.1 tensor-path pass is recorded; the distributed-path record waits for the L8.2 GPU check. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> (cherry picked from commit b5e5328)
The launcher layer may not import adapters and reads placement only from island args; the island translation refuses before any Ray/GPU work. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> (cherry picked from commit 05c359c)
The image layer adds peft 0.20.0 and the lazy-modelopt Megatron-Bridge __init__.py; Miles s24-miles-import keeps light actors off megatron_utils and holds the master port until use. Regenerate the decoupling golden samples (backend identity and runtime fingerprint only) and record the build. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> (cherry picked from commit 47e3052)
Read the island tapes that the pre-teardown diagnostics fetched into ~/yeto-diag/<cluster>/ and merge their trainer_start_summary into the run manifest, so every backend gives the scheduler T_s. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> (cherry picked from commit 8c9ff33)
G-L8.2 (2x H100, SGLang 2fa880182): flags on vs off agree in text and full-model checksum across two update_weights_from_distributed updates, including a release/resume and a rebuilt NCCL group before the second one. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> (cherry picked from commit 0a5241f)
rl-trainer-cold-start-cost L2.2 did not pass on 17aeb5c-2fa8801: an import trace in the image showed that the trainer gets modelopt from Megatron-LM a84b105 (megatron/training/checkpointing.py and training.py import it at module top), not from Megatron-Bridge. - The image layer now applies docker/miles-ports/ megatron_lm_lazy_modelopt.patch to the base's /root/Megatron-LM files (make_layer.py patched_files, --megatron-lm-src). The modelopt helpers are imported where they are used. The checkpoint save/load helpers are skipped while modelopt is not imported: they are no-ops for a model that ModelOpt has not converted, and only ModelOpt converts a model. post_training/checkpointing.py imports modelopt per function, so get_model's has_modelopt_state file check no longer needs it. - Miles s24-miles-import2 (44d41efd0): the megatron_utils package __init__ stays light, so RayWorkerManager no longer loads megatron.bridge when it unpickles placement-group and worker-spec objects. - Regenerate the decoupling golden samples (backend identity and runtime fingerprint only), record the build, update hash-migration.md. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> (cherry picked from commit f013058)
Spec for openspec change rl-trainer-cold-start-cost, taken from s24-sampler-spec 7bd416e2 (PR #258). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This was referenced Oct 15, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements openspec change
rl-trainer-cold-start-cost(spec included as the last commit).yeto/rl/engine/start_phases.py, launcher events) andT_s, also for non-echo sky islands.__init__.pywithout the eagerimport modelopt.torch, and patch Megatron-LM to import modelopt lazily (docker/miles-ports/).44d41ef-2fa8801; build records underopenspec/changes/rl-infra-spec/evidence/ports-image/; golden samples andhash-migration.mdupdated.GPU evidence: L8.1, L8.2 and L2.6 runs (SESSION24 records). L2.2 was confirmed with the 44d41ef image.
Tests: full suite on a remote CPU box: 5673 passed / 0 failed.
🤖 Generated with Claude Code