All pull requests

Pull requests

• [shardformer]: fix llama causal mask with sequence parallelism and no flash attention

  #6452·LiRunGuo opened Sep 22, 2026·

  ·

• [shardformer]: fix attention mask of qwen2/qwen3 with pipeline + sequence parallelism

  #6451·LiRunGuo opened Sep 22, 2026·

  ·

• [shardformer]: support Qwen3-MoE with expert parallelism

  #6450·LiRunGuo opened Sep 22, 2026·

  ·

• [checkpoint]: unpad padded parameters when saving sharded MoE checkpoints

  #6449·LiRunGuo opened Sep 22, 2026·

  ·

• [tensor]: fix saving checkpoints of padded parameters with tp_size=1 (keep detach/clone of padded tensors unpaddable)

  #6448·LiRunGuo opened Sep 22, 2026·

  ·
  1

• [gemini]: fix loading checkpoints of models with tied parameters

  #6447·LiRunGuo opened Sep 21, 2026·

  ·
  1

• [shardformer]: fall back to native RMSNorm when apex is not installed

  #6446·LiRunGuo opened Sep 21, 2026·

  ·
  1

• [example]: fix Pytorch_DDP/Pytorch_ZeRO plans crashing in gpt gemini demo

  #6445·LiRunGuo opened Sep 21, 2026·

  ·

• [kernel]: fix Q/K cross-wiring and out-of-bounds accesses in fused_rotary_emb triton kernel

  #6444·LiRunGuo opened Sep 21, 2026·

  ·
  1

• docs: add 2D/2.5D/3D tensor parallelism to Shardformer roadmap (#6380)

  #6443·Suraj2105-1 opened Sep 20, 2026·

  ·
  1

• [fp8]: Read detected local world size environment variable

  #6438·tandede opened Aug 20, 2026·

  ·
  1

  1

• ci: add names to compatibility matrix steps for readability

  #6436·MGPOCKY opened Aug 1, 2026·

  ·

• Use HybridAdam for Gemini CPU optimizer offload

  #6435·sdjasj opened Jul 29, 2026·

  ·
  1

• Fix GPT-2 FlashAttention mask preparation without pipeline

  #6434·sdjasj opened Jul 29, 2026·

  ·
  1

• Preserve initialized optimizer state in low-level ZeRO

  #6433·sdjasj opened Jul 29, 2026·

  ·
  1

• Fix GaLore 8-bit updates with ZeRO CPU offload

  #6432·sdjasj opened Jul 29, 2026·

  ·
  1

• Fix LAMB norm reduction with ZeRO CPU offload

  #6431·sdjasj opened Jul 29, 2026·

  ·
  1

• fix: add path validation in profiling_utils.py...

  #6430·anupamme opened Jul 20, 2026·

  ·

• docs: fix documentation typos

  #6422·Zhao73 opened Jun 1, 2026·

  ·

• fix infinite recursion in ColoTracer on newer PyTorch versions

  #6420·ssam18 opened Apr 19, 2026·

  ·
  1

• Fix: Handle LowLevelZeroPlugin with use_fp8=True

  #6419·Truong5724 opened Apr 15, 2026·

  ·
  1

• Fix: Handle FusedLAMB Fails with LowLevelZeroPlugin When Using CPU Offload

  #6418·Truong5724 opened Apr 15, 2026·

  ·
  1

• Fix: Handle div_scale when using Gemini plugin with CPUAdam

  #6415·Truong5724 opened Apr 15, 2026·

  ·
  1

• docs: add Japanese README

  #6414·eltociear opened Apr 13, 2026Contributor·

  ·

• fix: correct typos in comments and docstrings

  #6404·thecaptain789 opened Feb 7, 2026·

  ·