Repository navigation
TB2/27B run support: NCCL stack dump, actor backup refresh, TB2 infra handling, qwen38 profile - #266
Draft
michaellchung wants to merge 12 commits into
Draft
michaellchung wants to merge 12 commits into
michaellchung wants to merge 12 commits into
Conversation
YETO_HANG_STACK_DUMP=1 makes the Modal island runner watch for the PyTorch NCCL flight-recorder file and, when it appears, run py-spy dump on every Python process in the container twice. A TP peer stuck on the host side prints nothing when the other rank times out; this captures where it is. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> (cherry picked from commit 341789cd5024b6d4995142372369a81d740ba87e)
The cut restore and the state import called weights_backuper.backup("actor")
unconditionally. Miles skips that pinned host copy when it is not needed
(fixed partition; colocated LoRA, where the memory saver's host copy of the
frozen base serves weight sync), so the call allocated a fresh pinned copy
of every weight per rank. On Qwen3.8-27B this added about 133 GB of host
memory after a resume and Ray OOM-killed the island on a 200 GB node.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 8690cc0ee2b16dffd7f2f83bbf3858bb8a4ce0d9)
- codex_infer: rank 0 serves the base model without LoRA; add SglangSessionLoader.start_base for runs that have no stored policy. - tb2_baseline: run every (task, trial) of a TB2 split through the eval island attempt path with bounded concurrency; append one trace per attempt and write a pass-rate summary (infra errors excluded, mixed pass/fail tasks counted, driver stop reasons tallied). - modal_tb2_baseline: one-GPU Modal shape with the eval-island image and mounts, a read-only models Volume, and periodic evidence sync to a results Volume so a timed-out container keeps its traces. - Add qwen38 (Qwen3.8-27B) to the Codex OpenEnv profile allowlist; without it the adapter silently fell back to the qwen35_08b identity. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> (cherry picked from commit 88675a432b0e8086c00f0a4153f175def0fc9b22)
…en38small - codex_harness_agent: a profile that declares allow_prose_with_tool_call keeps one line of prose next to its single tool call instead of failing the episode; every other profile keeps the strict rule. Only qwen38 opts in (Qwen3.8-27B TB2 baseline: 49% of episodes ended on that rule). - Re-pin CODEX_HARNESS_AGENT_SHA256 for the changed harness source. - qwen38 profile: tito_model is qwen38small, the Miles fork's Qwen3.8 family; "qwen38" is not a TITO type there and the launcher rejected --tito-model qwen38small for this profile. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> (cherry picked from commit 8dd93a0210f9ae14bb70371e52c0ec33c7f68469)
A sandbox that shuts down under the Codex worker or before the trusted verifier (Modal ConflictError / NotFoundError) now yields infrastructure metadata instead of an exception that fails the whole rollout step. Tests show such samples, and signed-boundary tool-output errors, get no reward and take their whole group out of the advantage via the TB2 group filter. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> (cherry picked from commit 698e11514605d5ed0f9e454743910b1ab70a22c1)
Miles does not mask ABORTED samples when it computes advantages, so an infrastructure TB2 sample only stays out of training if the TB2 group check runs as the dynamic sampling filter. It had no neutral name, so no spec could select it. Add tbench_group_check (bound to tbench_reward.check_group) and the agentic TB2 spec that uses it (TIS C=2.0, max policy staleness 4, over-sampling 8). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> (cherry picked from commit dc5ee0cbcfe2aa729e1fbdd334f22cd4576846ab)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> (cherry picked from commit af4600cc5c16a62013aaf0c62b562ec91482faff)
--rl-activation-recompute emits Megatron full uniform recompute (one layer per chunk); --rl-max-tokens-per-gpu enables Miles dynamic batching with a per-GPU token budget (refused below --seq-len). Both default off, so existing argv is unchanged. Needed by the 27B TB2 long run, whose step-0 training OOMed on 32.5K-token samples without recompute. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> (cherry picked from commit f5c8b89d47e65bd6e3805a88fcc428cda71d1e63)
Miles rejects --use-dynamic-batch-size with --qkv-format bshd at parse time on the island; fail at translation on the CPU instead. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> (cherry picked from commit 9af39b92646b908aa304914ab152489e246dd10c)
… LR check The cut's backend fingerprint hashes the full Miles argv, so resuming a cut into a longer run (--num-rollout, --lr-decay-iters) was refused even with --rl-resume-allow-config-change. With the flag, a differing backend fingerprint and a changed first-round learning rate are recorded in the rl_resume / rl_resume_check events instead of refusing the resume. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> (cherry picked from commit 84c7137a8841ae770201c7d7997111d80d38fbc5)
This was referenced Oct 15, 2026
requeue_untrained stopped only the Miles worker task; the group tasks it had submitted kept generating outside the in-flight budget while the next worker submitted a full budget again. On the 27B TB2 run the agentic in-flight count grew by about one budget per cut (29 -> 146 over four cuts) until a round took over two hours. Cancel and await this rollout function's _generate_group tasks on the rollout loop after the worker stops; their samples are requeued anyway. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The launcher mounts a cached HF token when one exists, which added an extra add_local_file step on machines that have one and broke the mount lists these tests pin. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on PR-3. Not part of the four openspec changes; run support for the TB2 and Qwen3.8-27B runs.
YETO_HANG_STACK_DUMP).tbench_group_checkdynamic sampling filter;codex_harness.jsongolden regenerated;hash-migration.mdentry.--rl-resume-allow-config-changealso covers the backend fingerprint and LR check (recorded in events; the semantic hash still refuses).🤖 Generated with Claude Code