Repository navigation
NV: lock GA10x's PCIe link at Gen1 (RTX A6000 behind Thunderbolt 5) [matrix_vector] - #247
Merged
Merged
Conversation
On an RTX A6000 (GA102, 10de:2230) in an Aoostar AG03 (Intel JHL9480, Thunderbolt 5), GSP-RM 570.144 leaves the link at Gen1 x4 by GSP_INIT_DONE (target still Gen4; the card trains at Gen4 x4 cold), and the link then dies under BAR1 traffic within seconds: the Mac's root port logs AER and a link timeout and disables the port, every read returns all ones, and the crash guard holds. Locking the link at load with RMPcieLinkSpeed=0x800002AA (gens 2-6 disabled; tinygrad PR #18215's value for Ada and Blackwell behind TB5 bridges) fixes it: the whole hardware matrix (16.4-16.11), the soaks, 20 warm boots, D1 SP and DP, kill-at-idle and the P4 recovery all passed on the A6000 with it. A Gen4 lock (0x80000295) broke the boot before INIT_DONE. - TinyGPUNVBoot.h: nv_ga10x_link_lock(); on architecture 0x17 the registry table gets the entry after tinygrad's two (the count no longer hard-coded at 2). BEAGLE_NV_GA10X_LINK_LOCK=0 sends tinygrad's table. - oracle/nv_init_helper.py: the same table for the goldens' tinygrad side. - golden_boot.py: GA102 cases from the A6000's own replies (early, NV_FLCN.init_sw on its VBIOS, the whole init_sw with and without the lock), each compared with the card's L0 recording when this Mac has it (as a prefix when the recording has no init_sw marker); a check that SET_REGISTRY carries the lock exactly on GA10x; perturbations of the value, the gate and the opt-out. Cases whose captured VBIOS is not on this Mac (the AD107 dumps) are reported SKIPPED instead of crashing the test. - TinyGPU.md: the Ampere row, the lock and its stickiness, the opt-out, and that TinyGPU.app c0d024f9 serves only display-class (0x0300) cards. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
On Ampere (GA10x, architecture 0x17) the NV boot now adds
RMPcieLinkSpeed=0x800002AAto the registry table it hands GSP-RM, which locks the PCIe link at Gen1 at load.BEAGLE_NV_GA10X_LINK_LOCK=0sends tinygrad's table instead. Ada and Blackwell are unchanged.Why
On an RTX A6000 (GA102,
10de:2230) in an Aoostar AG03 (Intel JHL9480, Thunderbolt 5), GSP-RM 570.144 leaves the link at Gen1 x4 by GSP_INIT_DONE (target still Gen4; the card trains at Gen4 x4 cold). The link then dies under BAR1 traffic within seconds: the Mac's root port logs AER and "Link timeout ... Disabling the port", every read returns all ones, and the crash guard holds. The value is the one tinygrad PR #18215 found for Ada and Blackwell behind TB5 bridges. A Gen4 lock (0x80000295) broke the boot before INIT_DONE.Changes
TinyGPUNVBoot.h:nv_ga10x_link_lock(); the registry entry count is no longer hard-coded at 2.oracle/nv_init_helper.py: the same table on the goldens' tinygrad side.golden_boot.py: GA102 cases built from the A6000's own replies (early boot,NV_FLCN.init_swon its VBIOS, the wholeinit_swwith and without the lock), compared with the card's L0 recording when the machine has it; a check thatSET_REGISTRYcarries the lock exactly on GA10x; three perturbations (the value, the gate, the opt-out). Cases whose captured VBIOS is not on the machine now printSKIPPEDinstead of crashing the test.TinyGPU.md: the Ampere row, the lock (it lasts until the enclosure is power-cycled), the opt-out, and that TinyGPU.appc0d024f9serves only display-class (0x0300) cards.Testing
On the RTX A6000 in the AG03:
usb2andmatrix_vector(logL -1498.89812 = CPU, clean teardown).golden_boot.pyall identical on both branches (GA102 cases equal to the card's recording).run_goldens.shfails only the goldens that need Docker or the author's private VBIOS and tables, the same set as before.Notes
~/.beagle/tinygpu. Those are NVIDIA firmware and stay out of git, so on other machines these cases printSKIPPED.a6000-fix→usb2andmv-a6000-fix→matrix_vector. Thematrix_vectorcherry-pick resolved a conflict ingolden_boot.py'smain()in that branch's style.🤖 Generated with Claude Code