[voltjia]

Summary

• Add a NineToothed-generated NVIDIA implementation for FusedAddRmsNorm at implementation index 9.
• Keep the implementation as a row fast path for rank-2 to rank-4 tensors with a normalized width of at most 1024.
• Preserve fp32 reduction semantics and optional weighted scaling.
• Include the operator in smoke generation and testing.
• Enable WITH_NINETOOTHED in NVIDIA CI with pinned NineToothed bf8d7c42f86cf5feafe4a3ad443f1df151016886 and ntops 82cba6cf revisions.
• Use complete, direct GitHub clones for CI dependencies; a partial clone produced an incomplete wheel missing emitters/ssa.py, while the proxy intermittently returned 401/403/522.
• Keep NVIDIA CI on NumPy 1.x after installing NineToothed, matching the NGC Torch ABI.
• Pass MetaX/Moore wrapper compilers in the initial CMake cache so backend and InfiniRT definitions survive CMake cache reconfiguration.
• Keep the Ascend workspace-pool C++ regression on Ascend devices; MetaX's compiler wrapper cannot compile that fixture.
• Mark Cast as an automatic-default operator so strided inputs can fall back from native slot 0 to Torch slot 8.

Motivation

The NineToothed row kernel is faster than the native CUDA implementation for the Qwen prefill residual-norm shape. This provides the reusable operator backend needed by InfiniLM without the temporary interposer used during the experiment.

InfiniOps' existing opt-in TuningManager can benchmark implementations and persist the selected index. A default-call validation with auto-tuning enabled benchmarked both implementations and selected index 9. Native InfiniLM runtimes need InfiniLM PR #605 to initialize this manager; Python bindings already initialize it on import.

Closes #.

Type of Change

• feat — new feature / new operator / new platform
• fix — bug fix
• build / ci — build system or CI configuration

Platforms Affected

• CPU (WITH_CPU)
• NVIDIA (WITH_NVIDIA)
• Iluvatar (WITH_ILUVATAR)
• MetaX (WITH_METAX)
• Cambricon (WITH_CAMBRICON)
• Moore (WITH_MOORE)
• Ascend (WITH_ASCEND)
• PyTorch C++ bindings (WITH_TORCH)
• Build system / CMake / CI
• Python bindings / user-facing API

Smoke Test Result

$ python3 -m pytest tests/test_fused_add_rms_norm.py --devices cuda -m smoke -q
Running 4 items in this shard
....                                                                     [100%]
4 passed, 228 deselected in 0.72s

Test Results on Supported Platforms

Platform	Affected	Result / Notes
NVIDIA	Yes	CUDA 12.6 clean smoke wheel build with WITH_NINETOOTHED=ON passed; focused suite passed; full native/Torch/Flash/Marlin install build passed
Iluvatar	No	CI smoke passed
MetaX	No	CI smoke passed after initial-cache compiler fix
Cambricon	No	CI smoke passed
Moore	No	Build passed; runner image fails importing Torch because /usr/lib/x86_64-linux-gnu/libmusa.so.4 is too short
Ascend	No	Runner repeatedly fails during Set up job before checkout/build

Focused FusedAddRmsNorm suite:

Running 232 items in this shard
........................................................................ [ 31%]
........................................................................ [ 62%]
........................................................................ [ 93%]
ssssssssssssssss                                                         [100%]
180 passed, 52 skipped in 2.00s

Strided Cast smoke previously selected native slot 0 and failed; slot 8 handles the case. kAutomaticDefault restores the supported-provider fallback path.

Benchmark / Performance Impact

A100, CUDA graph-free CUDA-event benchmark, (8192, 1024) bf16, 20 warmups and 100 timed launches:

Implementation	Average launch time
Native index 0	0.157911 ms
NineToothed index 9	0.093901 ms

With INFINI_OPS_ENABLE_AUTO_TUNING=1, the same default Python call benchmarked both active implementations and emitted:

[Tuning] FusedAddRmsNorm on nvidia: benchmarked 2 impls, chose index 9

The resulting tuning cache recorded best_implementation: 9; a subsequent trace selected implementation 9 without an explicit index.

Notes for Reviewers

The kernel intentionally handles at most 1024 elements per row. Wider rows remain eligible for the native implementation; explicit index-9 tests skip those cases.