Generate pawn moves set-wise and write the captures straight into the move list by aywrite · Pull Request #371 · aywrite/arche

aywrite · 2026-10-04T19:26:34Z

Two exact changes to move generation, one commit each. Every list holds the same moves in the same order, so the search visits the same tree: every per position line of the bench and the games suite is unchanged (5,355,551 and 9,409,127 nodes).

Generate pawn moves set-wise. Shifts of the pawn board give the pawns with a capture and the pawns with an open push, and the generator visits only those, in square order. In the captures door 95% of the pawns it used to visit gave nothing.
Write the captures straight into the caller's move list. The cursor and the count stay in registers. The quiet moves go to one unchecked buffer, and one copy puts them behind the captures. The captures door writes its list once, in place.

Instructions. Callgrind, the tree search against master e147526 (copy-make): the games suite 10,853,793,638 to 10,563,270,014 (2.677% fewer), the bench 7,593,352,355 to 7,330,283,073 (3.464% fewer). By commit: 1.424% and 1.271% on the games suite. The second alone, on the old pawn walk, saved 0.19% (measured on adfd966): the two pay mostly together.

Clock. scripts/speed.sh on the head against master e147526: +3.5% (95% interval +1.8% to +5.2%, 120 interleaved rounds over shuffled layouts). An interleaved clock of the same code on adfd966 over 100 layouts read the games suite at +3.16% (+2.46% to +3.85%) over 450 rounds at a 16 MB table, and +2.50% (+1.49% to +3.55%) at 256 MB over 120. The clock reads more than the instructions because the pawn loop's branches go with it: 9.4% fewer mispredicted branches over the games suite under callgrind's simulation.

Games. Four Strength runs of 500 games at 10+0.1 on 8moves_v3 against master, run before the rebase (the head b7d96b3 against 587f1f6, the same two changes), the decision fixed before the first game: the mean rate ratio of the four and its 95% interval. The runs (#264 to #267) read 1.0201, 1.0182, 1.0194 and 1.0179, so r = 1.0189 [1.0172, 1.0206], wholly above 1.000. Every game ended normally. CI's speed job read that head at +3.3% (+2.9% to +3.6%) on the bench.

Checked (on the rebased head unless said otherwise).

On adfd966: a build that runs master's generator beside the new one at every call asserted the same list (every field, in order), the same capture count and the same has_legal_move answer over the bench, the games suite and a 448 search UCI workload ending in a 1 MB table.
On adfd966: a million random positions went through every door, among them lists over 64 moves, en passant into and out of check, promotions with captures, and pawns on the back ranks. All 7,063,989 distinct positions in traced searches matched an independent reference generator.
Miri passes all three doors and both growth paths under Stacked Borrows and Tree Borrows.
The release, debug, machine-test and baseline target tests pass, as do the tactical and strategic suites. Master and the head search 1,800 suite positions to depth 7 and 150 play-outs of up to 60 plies identically.
New tests pin the pawns' order in both doors, and a captures list that outgrows the inline list.

Constraints this leaves:

The quiet buffer is written without a bound check. It holds because a side has at most 63 pieces besides its king and no piece more than 27 quiet moves (1,728 moves, 10,368 bytes). A move kind that gives a piece more quiet moves has to resize it. A debug build checks every push.
The full and evasions doors' frame is 10.6 KB, two probed pages where it was one. A 512 move buffer for a side of at most sixteen pieces keeps it under a page; it was exact and not shown faster, so it is left out.
The list's order is the old one, which the ordering's tie breaks read: in each part, the pawns in square order, each pawn's captures before its en passant capture and its single push before its double push.

🤖 Generated with Claude Code

https://claude.ai/code/session_01DtepPHhwporfDvjPSnX4ae

Generated by Claude Code

github-actions · 2026-10-04T19:31:33Z

Speed against `f2d249b9`

Measured on AMD EPYC 7763 64-Core Processor.
Both sides built with rustc 1.99.0 (b940084d7 2026-09-28).

Both sides built and run on this runner in this job, the way
scripts/speed.sh measures a perf commit. Each round runs
both sides on a layout of its own, the same compiled code with
its code and data shuffled and moved, so the 95% interval carries
where the code landed as well as the run. The verdict holds the
interval against a 1% threshold. The default layout, the one a
release ships, is measured after as a diagnostic. The node
counts are the search's: they move when the search does, and a
speed change leaves them alone.

round     base nps  candidate nps  change
    1      5027145        5196638   +3.4%
    2      5005500        5173211   +3.4%
    3      5029405        5189668   +3.2%
    4      4965334        5263984   +6.0%
    5      4988176        5236085   +5.0%
    6      4944156        5259976   +6.4%
    7      5004585        5200240   +3.9%
    8      5001777        5235726   +4.7%
    9      5002021        5224155   +4.4%
   10      5025177        5248832   +4.5%
   11      4971929        5207589   +4.7%
   12      4977303        5231589   +5.1%
   13      5009517        5233747   +4.5%
   14      4996805        5271900   +5.5%
   15      5047930        5226349   +3.5%
   16      4996672        5200340   +4.1%
   17      4972121        5285276   +6.3%
   18      5014081        5180706   +3.3%
   19      5060215        5177665   +2.3%
   20      4996884        5231749   +4.7%
   21      5024867        5242665   +4.3%
   22      5009913        5246520   +4.7%
   23      5049898        5255474   +4.1%
   24      4976429        5255130   +5.6%
   25      4990899        5251835   +5.2%
   26      5009727        5118538   +2.2%
   27      4973512        5241022   +5.4%
   28      4959638        5229087   +5.4%
   29      5037724        5239823   +4.0%
   30      5025833        5141908   +2.3%
   31      4984326        5258168   +5.5%
   32      5028162        5265744   +4.7%
   33      5012809        5187636   +3.5%
   34      4962785        5201897   +4.8%
   35      5004330        5221459   +4.3%
   36      5002084        5226455   +4.5%
   37      5018086        5174169   +3.1%
   38      4981471        5235605   +5.1%
   39      5059342        5233243   +3.4%
   40      4967979        5199519   +4.7%

              nodes    time  median nps  faster half
base       11088536  2.22 s     5003207      5025212
candidate  11088536  2.12 s     5231669      5249625
change                            +4.6%        +4.5%

paired change +4.4%, 95% interval +4.1% to +4.8%
diagnostic, on the default layout alone +5.6%, 95% interval +4.6% to +6.7%, 13 rounds

faster: the whole interval is above +1.0%

Speed: +4.4% (bench nps, 95% interval +4.1% to +4.8%, 40 interleaved rounds over shuffled layouts vs f2d249b9)

The instructions each side's bench executed, counted under
cachegrind. The count repeats to within a few hundred
instructions, so a small change here is a real one, but it
prices instructions only: cache misses, mispredicted branches
and where the code lands are the speed's to show.

instructions against f2d249b9, one cachegrind run a side, bench

             instructions     nodes  per node
base       15,683,747,166  11088536    1414.4
candidate  15,115,361,179  11088536    1363.2
change             -3.62%              -3.62%

fewer instructions, beyond the ±0.7% that edits not made for speed have moved the count

The generator visited every pawn of the side to move and asked each one its rank, its two captures, its pushes and en passant. In the captures door most of them gave nothing: over the games suite 5,405,615 of the 5,687,761 pawns it walked there emitted no move. The pawns are now walked as sets. Once a call, shifts of the pawn board give the pawns with a capture and the pawns whose single or double push is open (in check, each push masked by the evasion targets on its own square, as before), and the pawn attack table read at the en passant square gives the pawns with an en passant capture. The generator walks the pawns with a capture from the lowest square, giving each its captures and then its en passant capture, and then the pawns with a push, giving each its single push and then its double push. A pawn with nothing to give is never visited. The list is the one the old walk gave, move for move. A pawn's captures and en passant already went to the captures part of the list and its pushes to the other part, so splitting one walk into two keeps each part's order: the pawns in square order, and each pawn's moves in the old order. The shifts drop the steps off the board that the old range test dropped, so a pawn on the first or last rank (which `from_fen` accepts) gives what it gave before. A new test pins the pawns' order in both doors against the lists the one pawn at a time walk gave, over promotions with and without a capture, en passant for either side, single and double pushes, and pawns on the back ranks. Callgrind, the tree search (`search_root` inclusive) on a build with line tables, against this commit's parent: the games suite 10,853,793,638 instructions to 10,699,268,202 (-1.424%), the bench 7,593,352,355 to 7,412,325,240 (-2.384%). Every per position line of both suites is unchanged. Bench: 11088536 Speed: +1.7% (bench nps, 95% interval -1.1% to +4.6%, 60 interleaved rounds over shuffled layouts vs e147526) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DtepPHhwporfDvjPSnX4ae

The generator wrote every move twice: into one of two 512 move arrays in its frame, and then into the caller's list when it finished. Each array's count lived beside it in the same struct, so llvm kept the counts in memory, and every push loaded, incremented and stored its count and checked it against 512. The captures now go straight into the caller's list through a cursor, with the count and the capacity in registers. A push that finds the list full grows it on a cold path, so a list of any width is still built. The quiet moves go into one buffer that no position can fill (a side has at most 63 pieces besides its king and no piece more than 27 moves), so their push is not checked, and one copy puts them behind the captures. With the pawns walked as sets, the captures door gives its promoting pushes after every capture, so it writes its list once, in place, with no buffer and no copy. The list is unchanged move for move; only where a move waits before the list is built has changed. The quiet buffer is 10,368 bytes, so the frame of the full and evasions doors grows from about 6.2 KB to 10.6 KB and takes a second probed page. A 512 move buffer for a side of sixteen pieces or fewer, with the full one on a cold path, keeps the frame under a page; it was exact but not shown faster on the clock (+0.37% over the games suite at 16 MB, -0.61% at 256 MB), so the frame stays. The move list's writes into its buffer before `set_len` were one unsafe block in `board.rs`. The generator now has six: taking the cursor from the list, the cursor's write, the quiet buffer's write, the copy behind the captures with its `set_len`, the other `set_len`, and the cold growth of the list. docs/ROADMAP.md counts them, twenty in the crate. The cursor and the list are held as raw pointers: a list short of a spill keeps its moves inside itself, so a `&mut` to it reborrowed after the cursor was taken would invalidate the cursor. Miri passes a test of all three doors and both growth paths under Stacked Borrows and Tree Borrows, and reports undefined behaviour on the first form, which held the list as a `&mut`. A debug build checks every quiet push against the buffer's room, which is none in the captures door. A new test builds a captures list of more than 64 moves, so the cursor grows the list while it writes. Callgrind, against this commit's parent: the games suite 10,699,268,202 instructions to 10,563,270,014 (-1.271%; -2.677% for this commit and the last together), the bench 7,412,325,240 to 7,330,283,073 (-1.107%; -3.464% together). Writing the captures into the list with the pawns walked one at a time saved 0.19%, so the two changes pay mostly together. Every per position line of both suites is unchanged. Bench: 11088536 Speed: +2.0% (bench nps, 95% interval -0.8% to +4.5%, 60 interleaved rounds over shuffled layouts vs f764299) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DtepPHhwporfDvjPSnX4ae

aywrite force-pushed the claude/movegen-setwise-pawns branch 4 times, most recently from f68e432 to 58a0fcd Compare October 9, 2026 02:49

claude added 2 commits October 13, 2026 02:01

aywrite force-pushed the claude/movegen-setwise-pawns branch from 58a0fcd to 96681fd Compare October 13, 2026 02:01

aywrite merged commit f424675 into master Oct 13, 2026
21 checks passed

aywrite deleted the claude/movegen-setwise-pawns branch October 13, 2026 05:50

Navigation Menu

Repository navigation

Generate pawn moves set-wise and write the captures straight into the move list - #371

Generate pawn moves set-wise and write the captures straight into the move list#371
aywrite merged 2 commits into
masterfrom
claude/movegen-setwise-pawns

aywrite commented Oct 4, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Oct 4, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

aywrite commented Oct 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Oct 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Speed against f2d249b9

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

aywrite commented Oct 4, 2026 •

edited

Loading

github-actions Bot commented Oct 4, 2026 •

edited

Loading

Speed against `f2d249b9`