Skip to content

chore: pin InfiniOps fused QK norm - #1580

Merged
voltjia merged 1 commit into
mainfrom
chore/pin-ninetoothed-fused-add-rms-norm
Nov 5, 2026
Merged

voltjia merged 1 commit into
mainfrom
chore/pin-ninetoothed-fused-add-rms-norm

Conversation

@voltjia

@voltjia voltjia commented Oct 30, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

  • Update the submodules/InfiniOps gitlink to the merged vLLM-prefix-aligned NineToothed fused Q/K normalization revision.
  • Keep this manifest PR to a single component revision change and introduce no component source copies or build logic.

Revision Update

Component Old revision New revision Upstream change
submodules/InfiniOps 640475a658de1f26bbf7c018fd0e0f8d5303a74f 1a68b261d19f0ed6f76bbabeab7a25f479f05e07 InfiniOps #1023

The new revision adds FusedQkNorm, including separate Q/K weights, packed-QKV in-place semantics, and an untouched V section. Its argument order is the direct prefix of vLLM's fused_qk_norm_rope API; RoPE remains a separate operation in this performance path.

Merge Status

InfiniOps #1023 was merged as master commit 1a68b261d19f0ed6f76bbabeab7a25f479f05e07. This manifest now pins that final master revision rather than the PR branch head.

Validation

Manifest validation on InfiniCore head 03db63ae:

git diff --check                 passed
git ls-files --stage submodules  passed
git submodule sync --recursive   passed
git submodule status --recursive passed

The recursive status confirms:

  • submodules/InfiniOps is a gitlink at 1a68b261d19f0ed6f76bbabeab7a25f479f05e07.
  • submodules/InfiniOps/.ci is initialized at b45d360c.
  • submodules/InfiniCCL and submodules/InfiniRT remain at their existing revisions.

Component validation for InfiniOps #1023:

  • Full and explicit-op Release builds/installs with CPU, NVIDIA, Torch, and NineToothed enabled.
  • A100-SXM4-80GB, CUDA 12.6.77, driver 580.105.08.
  • pytest -q tests/test_fused_qk_norm.py --devices cuda: 2 passed.
  • Ruff, clang-format, git diff --check, and all configured InfiniOps CI and CI v2 shadow matrix platforms passed on PR head 77cd5ea7.
  • GitHub merged the reviewed PR as final master commit 1a68b261.

Downstream InfiniLM validation:

  • Full InfiniLM Release build/install with C++11 ABI=1 passed on the same A100.
  • One-layer BF16 Qwen3 prefill plus decode smoke passed.
  • INFINI_OPS_TRACE_CALLS=1 recorded FusedQkNorm, NVIDIA, implementation 9 during Qwen3 forward.
  • Long-input prefill comparison against the prior merged-QK stack showed a 13.2% median TTFT reduction at 1024, approximately parity at 4096 and 8192, and no regression outside run-to-run variance.
  • The measurements used the reviewed issue/1022 - patch metax hpcc hrc include #1023 operator patch before its final squash merge. The operator patch was unchanged by rebases; the only later PR-head addition was the NVIDIA CI runtime PyPI mirror setting, which is preserved in the merge commit.

Checks Not Run

  • InfiniCore has no native build or test suite, so no repository-local unit tests are applicable.
  • No non-NVIDIA InfiniLM integration was run; the downstream fast path is NVIDIA-only and other configurations retain their existing path.

@voltjia
voltjia requested a review from a team October 30, 2026 05:24
@voltjia
voltjia force-pushed the chore/pin-ninetoothed-fused-add-rms-norm branch from f0309b2 to a6ba44f Compare October 30, 2026 10:09
@voltjia voltjia changed the title chore: pin NineToothed fused add RMSNorm chore: update NineToothed RMSNorm InfiniOps revision Oct 30, 2026
@voltjia
voltjia force-pushed the chore/pin-ninetoothed-fused-add-rms-norm branch from d8db74c to 0475caa Compare November 4, 2026 02:22
@voltjia voltjia changed the title chore: update NineToothed RMSNorm InfiniOps revision chore: update InfiniOps NineToothed revision Nov 4, 2026
@voltjia
voltjia marked this pull request as draft November 4, 2026 02:25
@voltjia
voltjia force-pushed the chore/pin-ninetoothed-fused-add-rms-norm branch 2 times, most recently from b444eb0 to d9159e9 Compare November 4, 2026 09:22
@voltjia voltjia closed this Nov 4, 2026
@voltjia
voltjia deleted the chore/pin-ninetoothed-fused-add-rms-norm branch November 4, 2026 11:30
@voltjia voltjia changed the title chore: update InfiniOps NineToothed revision chore: pin InfiniOps fused QK norm Nov 4, 2026
@voltjia
voltjia restored the chore/pin-ninetoothed-fused-add-rms-norm branch November 4, 2026 11:39
@voltjia voltjia reopened this Nov 4, 2026
@voltjia
voltjia force-pushed the chore/pin-ninetoothed-fused-add-rms-norm branch 2 times, most recently from 0869448 to 0c5ea52 Compare November 4, 2026 12:53
@voltjia
voltjia force-pushed the chore/pin-ninetoothed-fused-add-rms-norm branch from 0c5ea52 to 03db63a Compare November 5, 2026 02:00
@voltjia
voltjia marked this pull request as ready for review November 5, 2026 02:00
@voltjia
voltjia merged commit 82095ba into main Nov 5, 2026
@voltjia
voltjia deleted the chore/pin-ninetoothed-fused-add-rms-norm branch November 5, 2026 08:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants