Planar NEON kernels, >= 2x faster on Apple Silicon & Raspberry Pi, bit-identical to a2_fast by rikkus · Pull Request #313 · sdatkinson/NeuralAmpModelerCore

rikkus · 2026-08-08T19:02:25Z

Planar NEON kernels for:

AArch64, tested and benchmarked on Apple Silicon M2 (Macbook Air) and Cortex-A76 (Raspberry Pi 500)
ARMv7, tested and benchmarked on ARM Cortex-A17 (RK3288, ASUS Tinker Board, same SoC as HeadRush Core & Prime)

Bit-identical to a2_fast as tested - 64 and 32 frame blocks

To give me a base to work on this, I made a project called NAMBench
That project is set up to build on various platforms and run some conformance tests (latest here).

Benchmark results

(Also published on Bencher as part of the build of NAMBench)

Macbook Air M2, MacOS 27 Beta 8

64-frame blocks

	`a2_fast`	planar NEON
A2 standard (8 ch)	418.3 ms	172.3 ms	2.43×
A2 nano (3 ch)	57.2 ms	28.4 ms	2.01×

32-frame blocks

	`a2_fast`	planar NEON
A2 standard (8 ch)	470.2 ms	177.6 ms	2.65×
A2 nano (3 ch)	61.4 ms	30.3 ms	2.03×

Cortex-A76 (Raspberry Pi 500, Ubuntu 24.04, GCC 13, 64-frame blocks)

	`a2_fast`	planar NEON
A2 standard (8 ch)	426.1 ms	198.5 ms	2.15×
A2 nano (3 ch)	107.7 ms	35.0 ms	3.08×

Cortex-A17 (RK3288 / ASUS Tinker Board, GCC 13, clock capped to 1416 MHz, 64-frame blocks)

	`a2_fast`	planar NEON
A2 standard (8 ch)	5855.0 ms	4340.6 ms	1.35×
A2 nano (3 ch)	974.1 ms	626.3 ms	1.55×

(clock capped because it overheats on long benchmark runs at its usual ~1800 Mhz)

Notes and caveats

Bit-identity is a property of a normal -O3 build. Under -ffast-math the compiler is free to contract across statements in either engine and the guarantee no longer applies - to a2_fast either.

honkkis · 2026-09-10T16:32:07Z

I run the measurements in M1 and they hold, similar >2x improvement.

I also backported to NeuralAmpModelerPlugin and looked at CPU usage with 10 plugins using activity monitor and there is a good improvement as well!

a2_fast keeps a frame's channels adjacent and vectorises across channels. These kernels keep each channel in its own plane and vectorise across frames instead, so one NEON lane runs a2_fast's per-frame scalar chain verbatim. Nothing is reassociated, so the output does not move by a bit. On an Apple M2, against a 10.9 s render at 64-frame blocks: A2 standard (8 ch) 417 ms -> 172 ms 2.43x A2 nano (3 ch) 57 ms -> 28 ms 2.01x and at 32-frame blocks, which is what a plugin actually runs, 2.65x and 2.03x -- a2_fast degrades at small blocks and these do not. The two channel counts reproduce two different orders of arithmetic, because a2_fast itself branches. C=3 reproduces its hand-written scalar 3x3 GEMV. C=8 reproduces what its Eigen expressions compute, including the per-tap partial that is summed into the running total only at the end of the tap, and the mixin's separate multiply and add -- folding the taps into one chain is the obvious thing to write and it is a different association. That order was established by comparing candidate orderings bit-for-bit against Eigen's own output, not assumed. Selection happens in A2FastConfig::create, and only on AArch64 with the A2 fast path already enabled; -DNAM_DISABLE_A2_PLANAR opts back out. On every other target the new file compiles to nothing and behaviour is unchanged. Verification ships with it: tools/test/test_a2_planar.cpp asserts memcmp equality against the reference over 14 block sizes per channel count, including 1, 3 and 7, which exercise the partial-tile and single-frame tails. tools/bench_a2_planar.cpp renders a whole signal through both engines, compares bit for bit, and only then reports speed. Built at -O3 rather than -Ofast on purpose: -ffast-math lets the compiler contract a multiply and an add across statements, which is the freedom the parity result is checking has not been taken. a2_fast.h gains create_a2_fast_reference_model so a test can get at the portable implementation directly rather than through the dispatcher, which now may hand back a specialised one.

…f it Two corrections, both about where this code is allowed to exist. bench_a2_planar.cpp was guarded on NAM_ENABLE_A2_FAST but called create_a2_planar_model unconditionally, so it failed to compile on any target without the planar kernels -- which is every non-AArch64 target, including the x86 Linux runners CI uses. Caught by cross-building for x86_64. It now builds everywhere and, where there is no planar kernel, prints that there is nothing to measure and exits 0. The target is still built on every platform on purpose: a tool that quietly disappears from some configurations is a tool nobody notices has stopped compiling. The activation gate was any AArch64 target. It is now Apple Silicon (__APPLE__ && __aarch64__). The kernels are very likely correct and faster on any AArch64 part, but they have only been built and measured on Apple Silicon, and two of the things they depend on are toolchain properties rather than architectural ones: the tile widths are M2 measurements, and bit-identity relies on the compiler contracting a*b+c into an FMA inside a2_fast's own 3-channel branch, which clang and gcc do by default and MSVC at /fp:precise does not. Claiming a target nobody has run is not worth the reach. Verified on x86_64 (cross-built on this machine): every target builds, a2_planar.cpp.o contains no symbols at all, and the full test suite passes. Off Apple Silicon the only thing that changes anywhere is that two lines of A2FastConfig::create now live in a named function.

The gate was __APPLE__ && __aarch64__ because Apple Silicon was the only place these kernels had been built and measured. It is now __aarch64__. Bit-identity was the thing worth checking off Apple, since it leans on the compiler contracting a*b+c into an FMA inside a2_fast's own 3-channel branch -- a toolchain behaviour rather than an architectural one. It holds: both submodels bit-identical to a2_fast, max|diff| exactly zero over a full render, on a Cortex-A76 (Raspberry Pi 500, Ubuntu 24.04) under GCC 13, and on Neoverse N2 under GCC 14 and Clang 18. The speed holds too, with a different shape. M2: 2.47x on A2 standard, 2.00x on A2 nano. Cortex-A76: 2.13x and 2.94x. Still __aarch64__ rather than a spelling that also catches MSVC's _M_ARM64. MSVC at /fp:precise does not contract into an FMA, so the reference branch it would be compared against computes something else and bit-identity would not hold. clang-cl on ARM64 defines __aarch64__ and is unaffected. The tile widths remain M2 measurements. They affect speed only, never output, and the Cortex-A76's different profile suggests re-tuning per part would be worth someone's time.

The kernels were AArch64-only because that is where they had been built and measured. They now build for ARMv7-A with NEON and VFPv4, gated on __arm__ && __ARM_NEON && __ARM_FEATURE_FMA -- the FMA half being load-bearing, since without it Eigen computes a2_fast's own C=8 path with non-fused vmlaq_f32 and the reference stops being the reference. Measured on a Rockchip RK3288 (quad Cortex-A17), GCC 13.3, clock-pinned: both submodels bit-identical to a2_fast, max|diff| exactly zero, with A2 standard at 78.5% -> 57.8% of one core and A2 nano 12.32% -> 8.87%. Three things this needed beyond a recompile: * ARMv7 has no by-element FMA at all, so vfmaq_laneq_f32 and friends do not exist. LaneWeights hides the difference: lane-addressed on AArch64, broadcast-from-memory (vld1q_dup_f32) on ARMv7, which is the right shape anyway on a machine with 16 Q registers rather than 32. * The tile widths do not transfer. The C=3 ladder peaks at 32 frames on an M2 and at 8 on a Cortex-A17, where 32 runs slower than a2_fast. Tile 8 is the last rung whose accumulators fit in 16 registers, and the measured spill counts turn over exactly there. * The C=8 mixin is a product and then a separate add -- two roundings, because that is what Eigen does -- and on ARMv7 -ffp-contract=fast folds the pair back into one vfma. round_now blocks that locally, at the point a2_fast rounds twice, rather than by turning contraction off globally, which the C=3 branch depends on. AArch64 is unchanged in output and in speed: bit-identical as before, identical fmla and fmul counts in the object, and 2.126x -> 2.146x (A2 standard) and 2.927x -> 2.921x (A2 nano) measured back to back on a Cortex-A76, which is noise. The header records the three caveats the ARMv7 claim carries: bit-identity rests on FPSCR.FZ being set, and both the C=8 parity and the C=3 speedup are GCC claims. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

The gate widened to 32-bit ARM in the previous commit, and the kernels were bit-identical there from the first build. They were also slow: 68.8% of one core on a Cortex-A17 against 57.4% for the lab kernel they were ported from, which is behind even a plain tile-8 kernel carrying none of the ring switches. The cause is the fold over generic lambdas that unrolls the C=8 conv's input and output channel loops. AArch64 needs that: each index has to be a compile-time value for vfmaq_laneq_f32. ARMv7 has no by-lane FMA at all, so it gains nothing from the fold and pays for it heavily -- with 16 Q registers the accumulators cannot stay resident at any useful tile width, and GCC schedules the spill traffic it cannot avoid far better for a plain loop nest than for a fold it must first decide to inline. Measured over the 523,808-frame render, fold form against loop form: 12.12 G instructions and 8.33 G memory accesses become 9.79 G and 6.34 G. Not stalls -- the fold form had the higher IPC of the two; it simply did a third more memory traffic. So ARMv7 gets the loop nest and AArch64 keeps the fold. The AArch64 object is byte-identical before and after this commit. On a Rockchip RK3288 at a pinned 1416 MHz, 32-frame blocks, both submodels bit-identical to a2_fast over the full render: A2 standard 78.79% -> 55.63% of one core (1.416x) A2 nano 12.29% -> 8.36% of one core (1.470x) Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GJ6wuGZ9ndKjCDh7Zkjui3

The header already justified every per-architecture decision, but only in prose, spread over two sections and a caveat list. Anyone arriving at this file to find out what it actually does on their target had to read all of it to learn that there is one engine here rather than four, and to assemble the speed figures from three different paragraphs. Two small tables up front: what varies per target and channel count -- tile width, weight delivery, and the C=8 conv loop shape -- and what each combination is worth against a2_fast. The speed table also records something the prose did not. The M2 rows are carried from the Apple Silicon campaign these kernels came out of and have not been re-measured since; every other row is a direct measurement of the code as it stands. And the Cortex-A76 rows are at a 32-frame block, which is why they read 2.40x/2.87x against the 2.13x/2.94x quoted further down -- same code, different block size. Both figures were already in the file with nothing to say they were not in conflict. Comment-only: the AArch64 object is byte-identical, and both conformance suites still pass. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GJ6wuGZ9ndKjCDh7Zkjui3

rikkus · 2026-09-11T14:41:36Z

I run the measurements in M1 and they hold, similar >2x improvement.

I also backported to NeuralAmpModelerPlugin and looked at CPU usage with 10 plugins using activity monitor and there is a good improvement as well!

Thanks for trying it out, @honkkis! I've updated the README to be a lot clearer, added ARMv7 kernels with their own possibly-worthwhile optimisations, put some public benchmarks up and shared the repo I've been running the optimisation work from.

Point NeuralAmpModelerCore at rikkus/OptimisationWorkOnNeuralAmpModelerCore (the armv7-a2-planar branch, head of sdatkinson/NeuralAmpModelerCore#313) and wire the planar kernels into plugin builds. This replaces an earlier attempt at enabling a different, now-retired engine ("fused"); that work is superseded by the planar kernels and dropped here. - macOS: swap wavenet/fused.{cpp,h} for wavenet/a2_planar.{cpp,h} in the Xcode project across all 8 native targets. NAM_ENABLE_A2_FAST was already defined here; no new build flag is needed, since a2_planar.h gates itself internally on NAM_ENABLE_A2_FAST plus the target being AArch64 or ARMv7 with NEON+FMA — the header declares nothing and the translation unit compiles to no symbols everywhere else. - iOS: same swap, plus a fix -- the previous attempt gave a2_fast.cpp and fused.cpp the same UUID for both their PBXFileReference and PBXBuildFile entries, a duplicate-key bug that meant only one of the two was ever actually wired into the build. Re-added both a2_fast.{cpp,h} (this PR's iOS enablement, kept) and a2_planar.{cpp,h} with distinct UUIDs across all 3 native targets. Also defines NAM_ENABLE_A2_FAST on iOS, which wasn't set before. - Windows: a2_fast.cpp/h was already registered in the three .vcxproj files; added a2_planar.cpp/h alongside it the same way. NAM_ENABLE_FUSED (which had been added to NeuralAmpModeler-win.props without ever registering fused.cpp in any .vcxproj -- a build that would not have linked) is removed; nothing else changes since a2_fast was already there. - All three platforms: also registered linear.{cpp,h}, nam_file.{cpp,h}, sequential.{cpp,h} and the newly-referenced headers (compiler.h, container.h, model_config.h, slimmable.h, wavenet/detail.h, wavenet/params.h, wavenet/slimmable.h), none of which this project's Core submodule pin had reached before. Without these the build fails to link on nam::validate_nam_file -- unrelated to fused/planar, just Core having grown new files since this project was last synced, surfaced now because the submodule bump needed to pick up the planar kernels also picks up everything else Core has added since. - .gitmodules points at the fork rather than upstream NeuralAmpModelerCore for now, since sdatkinson/NeuralAmpModelerCore#313 is not yet merged. Repoint to upstream once it lands. Verified on macOS (Apple Silicon): VST3 and AU targets both build clean (universal arm64+x86_64) from the official NeuralAmpModeler/scripts recipe's own toolchain steps, the planar kernel symbols (A2PlanarNano/A2PlanarFull) are present in the built binary, and `auval -v aufx 1YEo SDAa` passes in full, including real audio-render tests at multiple block sizes and sample rates. iOS and Windows are verified by project-file correctness only (no device/ Windows machine available here) -- same caveat as the PR this replaces. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

…tream Between this branch's original base and current main, upstream added a cached-prewarm optimisation to A2FastModel: Reset() restores a cached steady-state instead of re-running the legacy silence-processing prewarm, verified by a new test asserting zero allocations on that path (A2FastModel<N>::cached prewarm). The planar models derive from DSP directly, not from A2FastModel, so they simply inherited the old, allocating base-class prewarm() -- and that new test caught it after the rebase: 12 allocations where it expected zero. Same mechanism, mirrored for the planar ring layout: each ring's steady-state prewarm makes every column of every channel plane equal to the last one written, so caching that one column per plane is enough to rebuild the whole ring later without reprocessing silence. Added PrewarmFromCache / CacheStateAsPrewarmed to A2PlanarNano and A2PlanarFull, and an override of prewarm() that uses the cache once one exists, matching A2FastModel's own prewarm()/PrewarmFromCache()/CacheStateAsPrewarmed() shape. Full test suite passes after this, cross-built for AArch64, ARMv7 and x86_64. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

The rebase behind sdatkinson/NeuralAmpModelerCore#313 had picked up upstream's cached-prewarm optimisation to A2FastModel without adapting the planar models to it, so the previous pin (d3814ad) fails Core's own zero-allocation prewarm test. Follows that fix (d3814ad -> 552c5ab). Verified: VST3 target rebuilds clean (universal arm64+x86_64) with this pin. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

…hold test_a2_planar asserts memcmp equality against A2FastModel, but run_tests compiles every one of its sources at -O0 (upstream does this so the allocation tracking behaves). A2FastModel's 3-channel branch is a plain `a * b + c` chain, and its bit-identity premise is that the compiler contracts that into an FMA. GCC only contracts in its optimisers, so at -O0 the reference computes something the planar kernels -- intrinsics, so unconditionally fused -- cannot match: the test failed on the first sample of the first block size on every GCC target. Clang contracts during codegen, which is why the same test passed on Apple Silicon and hid this. Every shipping build is optimised, so -O0 was the one configuration in which the reference is not the code the claim is about. The kernels now build as an object library at -O3 (/O2 on MSVC) that only run_tests links, leaving every other test at the -O0 the allocation tracking wants. The allocation assertions that reach these kernels all require zero allocations, which optimisation cannot introduce. Verified: the full suite passes on an Apple M2 (clang), a Cortex-A76 and a Cortex-A17 (GCC 13). Before this, the latter two failed at channels=3, block=1. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

honkkis mentioned this pull request Sep 10, 2026

Experimental: Apple Silicon planar A2 integration for lower CPU usage tone-3000/tone3000-plugin#118

Open

rikkus and others added 6 commits September 11, 2026 12:53

rikkus force-pushed the apple-silicon-a2-planar branch from ca349f6 to d3814ad Compare September 11, 2026 12:28

rikkus changed the title ~~Planar NEON kernels for the A2 fast path on Apple Silicon: ~2.4x and ~2.0x, bit-identical~~ Planar NEON kernels for the A2 fast path: Apple Silicon, AArch64 and ARMv7, bit-identical Sep 11, 2026

rikkus changed the title ~~Planar NEON kernels for the A2 fast path: Apple Silicon, AArch64 and ARMv7, bit-identical~~ Planar NEON kernels, >= 2x faster on Apple Silicon & Raspberry Pi, bit-identical to a2_fast Sep 11, 2026

rikkus marked this pull request as ready for review September 11, 2026 14:36

This was referenced Sep 11, 2026

[FEATURE] Enable the planar NEON WaveNet engine (AArch64 / Apple Silicon / ARMv7) sdatkinson/NeuralAmpModelerPlugin#679

Open

[FEATURE] Enable the fused NEON WaveNet engine (AArch64 / Apple Silicon) sdatkinson/NeuralAmpModelerPlugin#656

Closed

rikkus mentioned this pull request Sep 11, 2026

Reduce CPU usage on ARM (AArch64 & ARMv7) sdatkinson/NeuralAmpModelerPlugin#680

Open

Navigation Menu

Planar NEON kernels, >= 2x faster on Apple Silicon & Raspberry Pi, bit-identical to a2_fast - #313

Planar NEON kernels, >= 2x faster on Apple Silicon & Raspberry Pi, bit-identical to a2_fast#313
rikkus wants to merge 8 commits into
sdatkinson:mainfrom
rikkus:apple-silicon-a2-planar

rikkus commented Aug 8, 2026 •

edited

Loading

Uh oh!

honkkis commented Sep 10, 2026

Uh oh!

rikkus commented Sep 11, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

rikkus commented Aug 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmark results

Macbook Air M2, MacOS 27 Beta 8

Cortex-A76 (Raspberry Pi 500, Ubuntu 24.04, GCC 13, 64-frame blocks)

Cortex-A17 (RK3288 / ASUS Tinker Board, GCC 13, clock capped to 1416 MHz, 64-frame blocks)

Notes and caveats

Uh oh!

honkkis commented Sep 10, 2026

Uh oh!

rikkus commented Sep 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

rikkus commented Aug 8, 2026 •

edited

Loading

rikkus commented Sep 11, 2026 •

edited

Loading