Split yeto/launcher.py into the yeto/launch/ package

[michaellchung]

Summary

Structural refactor of yeto/launcher.py. No behavior change. The file was 8338 lines; it is now 959 lines plus a new package yeto/launch/ (25 modules, 7836 lines). The split was generated mechanically by a name-to-module map: code and comments are kept verbatim, only imports change.

Where responsibilities went

Module	Responsibility
launch/__init__.py	Package doc; REPO_ROOT (single definition)
launch/rl_backend.py	RL backend selection, Miles overlay, default RL image
launch/remote_setup.py	Remote shell snippets: WAN/NVMe/torch/HF token, pre-run block, Megatron/Protenix/Hunyuan installs, causal kernel install
launch/topology.py	Island topology and placement: multinode network preamble, island spec/count/layout, trainer shape, CPU floor, role env
launch/syncer.py	Syncer command, role syncers, schedule mode, HMAC, lease, port probe, syncer task
launch/loss.py	Loss resolution and pickled-loss staging
launch/credentials.py	Cloud credentials, secret env split, registry login
launch/storage.py	Remote storage and mounts: checkpoint store, model store, initial adapter path
launch/island_tasks.py	--rl-island-tasks plan, checks, distribution
launch/rl_ports.py	Ports backend algorithm/infra switch checks and island flags (elastic, eval, resume)
launch/rl_args.py	_prepare_rl_args, SAO streaming / dense full-sweep preprocessing
launch/args.py	prepare_launch_args, check_cloud_prerequisites
launch/images.py	Image selection: Blackwell AMI, GPU image override, parse_image_spec, Miles source install
launch/in_vm_docker.py	In-VM docker wrapper (Verda etc.)
launch/codex.py	Codex harness, sandbox env, compaction env
launch/rl_island_task.py	make_miles_island_task
launch/learner_task.py	make_learner_task (non-RL)
launch/diffusion_sample.py	Diffusion sample task and run_diffusion_sample
launch/sky_ops.py	SkySDKOps, cluster naming, sky log _tail
launch/tapes.py	Tape collection, recovery, leave decisions
launch/modal_islands.py	Modal island config, tape pull, Modal app stop check, container change polling
launch/fleet.py	FleetController, island state, exit codes and failure kinds
launch/local_syncer.py	LocalSyncer
launch/teardown.py	Shutdown verification and diagnostics collection
launch/verda.py	Verda island prep, candidates, rename hooks
launch/manifest.py	Run manifest and dashboard log
launcher.py	run(), dry_run_plan(), warn_if_model_wont_fit, GPU_MEM_GB, island task assembly helpers; re-exports

Dependency direction (acyclic, bottom to top)

1. __init__, rl_backend, remote_setup, island_tasks, loss, credentials, verda
2. topology, syncer, storage, in_vm_docker, images, codex, teardown
3. rl_ports
4. rl_args -> args
5. sky_ops -> fleet -> local_syncer, modal_islands, tapes
6. rl_island_task, learner_task, diffusion_sample, manifest
7. launcher (imports all)

yeto/launch/* never imports yeto.launcher. tests/test_launch_layout.py enforces this and checks the package import graph has no cycle.

Compatibility

• launcher.py re-exports, via explicit from .launch.x import (...), every name that in-repo callers and tests use (195 names). No duplicate implementations.
• External callers (yeto/cli.py, island_overrides, memory_estimate, shape/*, sky_patches, dashboard/sources, rl/ssh_harness, cloud/modal_eval_island) are unchanged.
• AST equivalence: each of the 321 top-level statements of the old file matches its moved copy; the only difference is REPO_ROOT now uses parents[2] (same value).

Mock path changes (tests)

Patches now target the module where the lookup happens, using string paths:

• REPO_ROOT -> yeto.launch.loss, yeto.launch.args, yeto.launch.diffusion_sample (test_provenance)
• tcp_probe, build_syncer_binary -> yeto.launch.syncer
• remote_sky_api_server -> yeto.launch.verda
• terminate_and_verify -> yeto.launch.teardown
• _list_modal_apps, MODAL_STOP_VERIFY_DELAY_S -> yeto.launch.modal_islands
• codex_harness_launch -> yeto.launch.codex
• _sky_docker_login_config -> yeto.launch.credentials
• _miles_overlay_manifest -> yeto.launch.manifest
• IN_VM_DOCKER_CLOUDS -> yeto.launch.rl_island_task

Patches whose lookup still happens in launcher.run() / dry_run_plan() or via cli.py's launcher.X stay on launcher (e.g. prepare_launch_args, make_syncer_task, FleetController, SkySDKOps, LocalSyncer, _tail, _write_run_manifest, prepare_verda_islands).

Source-scanning tests (test_rl_mixed_island_placement, test_run_flags) now scan launcher.py + yeto/launch/*.py with unchanged assertions; test_import_boundaries adds yeto/launch to the cloud-layer dirs.

Verification

• Full local suite on main before this PR (c6a636f): 5649 passed, 44 skipped, 0 failed.
• Full local suite on this branch: 5651 passed, 44 skipped, 0 failed (+2 = new layout tests). Failure set identical (empty).
• import yeto.launcher and python -m yeto.cli --help / launch --help / rl --help / sample-diffusion --help exit 0 on this branch.

Risks

• Out-of-repo scripts that read a private yeto.launcher.<name> not in the re-export list will get AttributeError.
• Class __module__ changes (e.g. yeto.launch.fleet.FleetController). Nothing in the repo pickles these by module path.
• Other branches that edit launcher.py will conflict heavily. Recommended: regenerate the split from their launcher.py with the split tooling instead of hand-merging.
• New monkeypatch.setattr(launcher, "X", ...) in other branches silently stops working if X's caller moved; check the lookup module when merging.
• _sky_docker_login_config has no in-repo caller (likely dead code); left as is.

🤖 Generated with Claude Code