[ptgrogan]

Summary

Performance fixes from a profiling pass over the flood pipeline. Every result table is identical to main: values, dtypes and row order. The changes only remove overhead.

Step (synthetic regions)	main	this branch
GBS, debris, shelter and AAL: 5,000 blocks, 5 return periods	601 s	102 s
… of which AAL	408 s	4.6 s
Full run_scenario: 625 blocks, 5 depth grids, facilities	162 s	35 s
Facilities: 3,000 facilities, 4 return periods	14.6 s	0.6 s
Area weighting: 64M block-grid cells	4.4 s	1.9 s
prepare_depth_grid: 36M-cell UTM grid	14.8 s	3.9 s

Changes

• AAL (aal.py): exact integer arithmetic replaces the per-group Python loop over Fractions. The coefficients are C / L, so 2 L × AAL is an integer and rounding half away from zero is integer division. Python integers take over if int64 could overflow.
• GBS lookups (gbs.py): per-block × occupancy values come from an array lookup on factorized blocks. Before, each exposure table was melted and merged on two string keys, 18 times per run.
• pandas 3 strings (gbs.py, debris.py): isin(<large Series>) and set(series) handle Arrow-backed strings one element at a time, about 60× slower than unique(). They now use unique().
• Facilities (facility_analysis.py, hazard.py): default damage-function ids and restoration rows come from dicts built once. Each grid is sampled once for all facility kinds, with one read per 512-cell tile instead of one per point.
• Area weighting (exposure.py, scenario.py): depth bins and zones are computed once per depth cell, then resampled. Counts use bincount. The boundary is only rasterized for tiles it cuts. Block cell counts are shared between the grids of a scenario.
• Reprojection (hazard.py): GDAL warps and compresses on all CPUs, with a new threads= argument to limit that. merge_max also compresses in parallel.

Behaviour changes

• Exposure stored as nullable Int64 with <NA> now reads as NaN in the GBS lookups. Before, it raised.
• prepare_depth_grid uses all CPUs by default.

Testing

• New tests/test_vectorized_paths.py (19 tests). Each fast path is compared exactly with the code it replaced, on edge cases: duplicate rows, half-way rounding, int64 overflow, NoData, misaligned and partial grids, boundaries that cut tiles, points outside every grid, ambiguous defaults. Bugs injected into AAL rounding, boundary handling, point indexing and the xref filter all make it fail.
• Full suite: 73 passed, 98 skipped (SQL Server and Loss Library tests).
• End-to-end comparisons with main's output, all identical: area weighting (5 grid set-ups), facilities (10 tables), GBS, debris, shelter and AAL at 300 and 5,000 blocks, and a full run_scenario (24 tables).
• Not run: the SQL Server parity tests (tests/test_sql_parity*.py). Please run them before merging.

Not in this PR

• Memory. At 5,000 blocks flGBSFinal is 13M rows (about 2.7 GiB), and econ_loss briefly holds about 3.3× that because each merge copies the whole table. Selecting columns before those merges, or running GBS one return period at a time, would cut the peak.
• The 14-key groupby in _merge_calcs combines only about 1 % of rows but costs about 2 s at 1,000 blocks. A cheaper path has to keep pandas' Kahan-compensated float sums to stay exact.

🤖 Generated with Claude Code