Skip to content

fix(fp4): align MX matmul with logical operand dimensions - #1701

Closed
lwDavid wants to merge 4 commits into
hw-native-sys:masterfrom
lwDavid:fix/fp4-mx-logical-shapes
Closed

lwDavid wants to merge 4 commits into
hw-native-sys:masterfrom
lwDavid:fix/fp4-mx-logical-shapes

Conversation

@lwDavid

@lwDavid lwDavid commented Oct 28, 2026

Copy link
Copy Markdown
  • Validate packed FP4 MX operands in logical value units while retaining
    byte-carrier tile shapes and source offsets in the public IR.
  • Keep accumulator, scale, and scale-address geometry consistent, and
    convert packed-axis L1 extraction offsets for EmitC.
  • Expand VPTO MX matmul dimensions from the logical accumulator and
    update native FP4 fixtures for the carrier contract.

Fixes #1700

@lwDavid

lwDavid commented Oct 28, 2026

Copy link
Copy Markdown
Author

Validation against master 235f381f31ac26d0333f62f18362d4059c211dcd:

  • Built PTOAS with Python bindings and warnings as errors, using vpto-dev/llvm-project feature-vpto at fa2fd1f75d742fec91ecf40d98c9c7961647ec42.
  • All 80 selected lit tests passed (issue1700|tmatmul_mx|tgemv_mx|textract|tmov|tget_scale), including the three new regression files. Both modified native MX/MX-bias fixture modules also compile to VPTO at level3.
  • Full lit: candidate 2,219 passed, 39 failed, 1 unsupported; unchanged master on the same build and LLVM has 2,216 passed, the identical 39 failures, and 1 unsupported. The candidate adds three passing tests and introduces no new full-suite failures. Existing failures include fusion/yield and VPTO bridge/control-flow cases; this is not a claim that the full suite is green.
  • Downstream A5 hardware validation through the PyPTO tile API passed all four E2M1/E1M2 operand combinations, each with matmul and accumulation, logical M=64/K=128/N=128, seed 71, varied E8M0 scales, and exact comparison (rtol=atol=0) against an independent FP64 decoder/reference. Execution used the device queue; builds ran outside the allocation. This validates the EmitC path; the VPTO fixture updates have compile coverage only.
  • The downstream Indexer full/sparse score paths also passed exact comparisons at 1, 17, and 72 tokens, including negative weights, missing pages, short histories, sparse pools, and all packed nibble codes.

The IR continues to count packed byte carriers. For example, Left [16,128] and Right [128,32] describe logical [32,128] @ [128,64]; the FP32 accumulator is [32,64], with scales [32,4] and [4,64]. FP8 geometry and the existing GEMV operand-verification path are retained.

A PTOAS release containing this fix is needed before the downstream PyPTO toolchain pin can enable native FP4 with the default installed toolchain. PTOAS v0.70 does not contain it. Related integration: hw-native-sys/pypto#3094 and hw-native-sys/pypto-lib#1543.

- Validate packed FP4 MX operands in logical value units while retaining
  byte-carrier tile shapes and source offsets in the public IR.
- Keep accumulator, scale, and scale-address geometry consistent, and
  convert packed-axis L1 extraction offsets for EmitC.
- Expand VPTO MX matmul dimensions from the logical accumulator and
  update native FP4 fixtures for the carrier contract.

Fixes hw-native-sys#1700
@lwDavid
lwDavid force-pushed the fix/fp4-mx-logical-shapes branch from a1487c0 to a3c45a9 Compare October 28, 2026 05:23
@lwDavid

lwDavid commented Oct 28, 2026

Copy link
Copy Markdown
Author

Fixed the license check by removing the redundant encoding comment before the existing copyright header in lib/TileOps/a5/matmul/tmatmul_mx.py. The checker requires that header at the start of the file. The exact license checker and all pre-commit checks now pass locally; the copyright/license text is unchanged.

The build-and-test job stops before compiling this change: CMake 4.1.3 with -Werror=dev rejects FetchContent_Populate(cann-cmake) under CMP0169 at cmake/fetch_cann_cmake.cmake:74. The unchanged master commit 235f381f has the identical failure in its CI run, matching this PR's first run. This PR does not modify that CMake path. The independent build and baseline-controlled lit results are recorded above.

Install pytest in the isolated CI environment and remove the unused nanobind lookup from the pybind11-based legacy reader build.
Put convolution geometry on alloc_tile, preserve NC1HWC0 on the source views, and pass optional image-to-column offsets by keyword.
@lwDavid
lwDavid force-pushed the fix/fp4-mx-logical-shapes branch from 245c346 to 35ce848 Compare October 28, 2026 09:21
@lwDavid lwDavid closed this Oct 29, 2026
@lwDavid lwDavid moved this from In Progress to Done in pto project Oct 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request pypto

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

[Bug] Packed FP4 MX matmul emits operand dimensions inconsistent with accumulator and scales

1 participant