Skip to content

Remove native CPU cost cliffs at 2-3-5-smooth grid sizes - #24

Merged
jtravs merged 3 commits into
mainfrom
perf/native-size-cliffs
Oct 25, 2026
Merged

jtravs merged 3 commits into
mainfrom
perf/native-size-cliffs

Conversation

@jtravs

@jtravs jtravs commented Oct 25, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Some 2-3-5-smooth grid sizes cost 1.6–1.8× the neighbouring power of two per source evaluation on the native CPU backend. FFTW is not the cause; two size boundaries in our own code are:

  • Node-chunk tail (n_t 8640). The node-budget admissions take budget // per_node. At n_to 17280 the four-mode RDW's 33 nodes were cut to a chunk of 32, so every evaluation ran a 32-node pass and then a one-node pass on one thread. _balanced_chunk (translation) now spreads the nodes evenly over the same number of passes (17 + 16). Chunks never exceed the admitted size, planar chunks keep whole rows, and sub-row chunks are left as they are.
  • Power-of-two thread thresholds (n_t 4000). The stage threshold (8192 elements) and the modal FFT threshold (32,768 samples) sat just above the RDW's sizes at n_t 4000 (8004 and 32,000). Its stage entry and transforms therefore ran serially: stage entry 708 µs vs 59 µs, D2Z 104 vs 33 µs, Z2D 134 vs 53 µs (kernel timer, one Xeon Gold 6240 socket). On Linux the thresholds are now 4096 and 16,384, chosen from a sweep of stage {8192, 4096, 2048, 1024} × FFT {32768, 16384} over RDW, SSFS and reduced CtC. At 1024 the SSFS case regressed, so it stops at 4096.

Platform split. The lower thresholds apply on Linux only. macOS keeps 8192 and 32,768: on an Apple arm64 laptop the serial stage and transforms are the faster ones at these sizes, so the change made n_t 4000 17 % slower there (see the comments below). A roadmap item asks for a sweep of the Mac crossover. The node-chunk balancing applies on every platform.

Results

Xeon Gold 6240, 18 pinned cores, GCC 16, FFTW 3.3.11, ms per evaluation (native/benchmarks/size_cliffs.py, two rounds each):

n_t 4000 4096 4500 5000 6000 7200 8000 8192 8640
before 2.45–2.47 1.24–1.36 1.49–1.56 1.54–1.57 1.86–1.87 2.21–2.26 2.38–2.39 2.29–2.30 3.52–3.55
after 1.32 1.27–1.36 1.53–1.54 1.57–1.61 1.83–1.87 2.17–2.26 2.38–2.40 2.28–2.31 2.54–2.58

On a loaded 18-core arm64 laptop, RDW at n_t 4000 went from 0.82 to 0.95 ms, level with 4096 (0.98 ms). Thread wake-ups cost more there. SSFS and reduced CtC did not move.

Tests

  • GoogleTest CpuThreadPolicy.SmoothGridSizesKeepThePowerOfTwoWidths. The whole native GoogleTest binary passes locally (136); CI does not build it.
  • Python: test_budget_chunks_spread_the_nodes_evenly_over_their_passes, test_planar_chunks_stay_whole_rows_when_balanced, and the _even_chunk doctest.
  • Targeted runs pass: tests/translate, the held-plasma, memory and multimode backend tests (1821 passed), and native/tests (383 passed). The only failure is test_public_surface_is_exact on the local dirty-tree version suffix.

SEP 30 core A/B (same machine, 18 threads, 3 rounds, candidate/baseline)

case track ratio range
rdw adaptive-multistep 0.996 0.994–1.025
rdw adaptive-rk 0.986 0.985–0.995
ctc adaptive-multistep 1.004 1.001–1.006
ctc adaptive-rk 1.002 0.998–1.013
kerr_beam_384 fixed-rk4ip 1.005 1.003–1.006
radial_filament_256 adaptive-multistep 1.003 0.992–1.004
radial_filament_256 adaptive-rk 0.974 0.965–0.974

Geometric mean 0.996. Every case did the same work, and the fields agree to rounding (largest difference 5e-15).

🤖 Generated with Claude Code

Two size boundaries made some smooth sizes 1.6-1.8x slower per source
evaluation than the neighbouring power of two on the native CPU backend:

- The node-chunk admissions take budget // per_node, so the four-mode
  RDW's 33 nodes at n_t 8640 ran a 32-node pass and then a one-node tail
  on one thread. Translation now spreads the nodes evenly over the same
  number of passes (17 + 16), never above the admitted chunk.
- The stage threshold (8192 elements) and the modal FFT threshold
  (32768 samples) sat just above the RDW's sizes at n_t 4000 (8004 and
  32000), so its stage entry (708 us vs 59 us threaded on one Xeon Gold
  6240 socket) and transforms ran serially. They are now 4096 and 16384.

RDW per evaluation on 18 cores: n_t 4000 2.47 -> 1.32 ms (4096: 1.24-1.36),
8640 3.55 -> 2.54 ms (8192: 2.28-2.31); other sizes unchanged. Adds the
native/benchmarks/size_cliffs.py regression benchmark.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Copilot AI balanced review requested due to automatic review settings October 25, 2026 19:32

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota.

@read-the-docs-community

read-the-docs-community Bot commented Oct 25, 2026 •

Copy link
Copy Markdown

@jtravs

jtravs commented Oct 25, 2026

Copy link
Copy Markdown
Contributor Author

Mac A/B (18-core arm64 laptop, NEON FFTW, same commit, base/fix/fix/base interleaved, load 2.6–6; ms per evaluation, size_cliffs.py):

n_t 4000 4096 4500 5000 6000 7200 8000 8192 8640
before 0.88–0.89 1.07–1.08 1.07–1.10 1.12 1.19–1.21 1.29–1.32 1.41–1.48 1.50 2.32–2.37
after 1.04 1.04–1.07 0.96–1.06 1.10–1.11 1.03–1.21 1.30–1.31 1.38–1.41 1.48–1.51 1.73

The node-chunk balancing helps on both machines (8640: −26 % here, −28 % on the Xeon). The threshold change does not: here n_t 4000 is 17 % slower (now level with 4096). On this machine the serial stage and FFT were faster at that size; even 4096, threaded under the old thresholds, costs more than 4000 serial. So the stage and FFT crossovers differ between the two hosts by more than a factor of two, and one shared constant cannot suit both.

🤖 Generated with Claude Code

@codecov

codecov Bot commented Oct 25, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@jtravs

jtravs commented Oct 25, 2026

Copy link
Copy Markdown
Contributor Author

Zen 3 A/B (DMOG node12, dual EPYC 7643, idle node, GCC 16.2, FFTW 3.3.11, OMP_PLACES=cores OMP_PROC_BIND=close, base/fix/fix/base; ms per evaluation, size_cliffs.py):

threads build 4000 4096 4500 5000 6000 7200 8000 8192 8640
12 (one NUMA domain) before 2.36–2.42 1.94–1.98 2.05 2.18 2.57 3.02–3.08 3.25–3.36 3.62–3.76 3.75–3.77
12 after 1.74–1.75 2.01 2.02–2.03 2.18–2.19 2.53–2.55 3.01–3.04 3.23–3.27 3.54–3.61 3.60–3.64
24 (two domains) before 2.36–2.40 2.01–2.06 2.09–2.10 2.25–2.28 2.58–2.60 3.10–3.13 3.32–3.42 3.63–3.73 4.04–4.05
24 after 1.81–1.82 2.04–2.07 2.07–2.11 2.22–2.24 2.61 3.13–3.20 3.39–3.42 3.82–3.84 3.52–3.53

Both Linux x86 hosts gain, at n_t 4000 (Zen 3 −24 to −27 %, Cascade Lake −47 %) and at 8640 (−4 to −13 %, −28 %). The Apple arm64 laptop loses 17 % at 4000. The other sizes move by at most 5 %, within the spread as the node's load rose from 5 to 10 during the 24-thread run. A first attempt on a node shared with other jobs (load ~117 of 128, cores scattered over five NUMA domains) varied 2× between rounds of the same build and was discarded.

🤖 Generated with Claude Code

jtravs and others added 2 commits October 25, 2026 21:45
The lower starts (4096 elements, 16384 samples) gain on Linux: n_t 4000 RDW
is 47 % faster on Cascade Lake and 24-27 % on Zen 3. On an Apple arm64
laptop the serial stage and transforms are the faster ones at those
sizes (n_t 4000 went from 0.88 to 1.04 ms), so macOS keeps 8192 and
32768 until its own crossover is swept (roadmap item). The node-chunk
balancing is unchanged and gains on every host.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The roadmap linked a MyST label as a page anchor (docs -W), and the
native lane's strict basedpyright saw the execution mapping as partially
unknown.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@jtravs
jtravs merged commit 60e064f into main Oct 25, 2026
31 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants