Skip to content

Leave a valid destination when f throws inside map! or broadcast! - #1082

Merged
ViralBShah merged 2 commits into
mainfrom
vs/map-throw-valid-dest
Oct 13, 2026
Merged

ViralBShah merged 2 commits into
mainfrom
vs/map-throw-valid-dest

Conversation

@ViralBShah

@ViralBShah ViralBShah commented Oct 13, 2026 •

Copy link
Copy Markdown
Member

When f, or the conversion of its result to the destination's element type, throws part-way through an in-place map! or broadcast!, the sparse destination is left with a pattern that is half-written: column pointers of the new result next to those of the old one. The next use of it fails an internal assertion or reads the wrong entries.

Before (on #1071; the same on main and Julia 1.13)

julia> B = sparse([1.0 0 0 4; 0 2 0 0; 0 0 3 0; 1 1 0 0]);

julia> B .= ((x, y) -> x == 3 ? error("boom") : x + y).(B, sparse(ones(4, 4)))
ERROR: boom

julia> Array(B)
ERROR: AssertionError: _goodbuffers(S)

After

julia> B .= ((x, y) -> x == 3 ? error("boom") : x + y).(B, sparse(ones(4, 4)))
ERROR: boom

julia> Array(B)
4×4 Matrix{Float64}:
 0.0  0.0  0.0  0.0
 0.0  0.0  0.0  0.0
 0.0  0.0  0.0  0.0
 0.0  0.0  0.0  0.0

julia> nnz(B)
0

The kernels rebuild the pattern of the destination, or densify it, as they go, so a throw from f leaves it between two patterns. The dispatchers of map! and broadcast! now call the kernel inside a try (_guardpattern for the zero-preserving kernels, the existing _guardfill for those that fill, and the zero-argument C .= f.() path), and on a throw empty the destination before rethrowing. The kernels themselves are unchanged.

Behaviour to review: after a throw, a destination that is not fixed is in one of three states, never structurally invalid.

  • Untouched when the failure can be found first: the index type cannot address every entry, or the fill value f(0) does not convert. That is Validate before densifying the destination of map! and broadcast! #1071.
  • Partly written, as a dense destination is, for one argument whose pattern the destination already has (C .= f.(C)). This is what main does.
  • Empty otherwise, since the pattern, or the filled values, are then no longer the destination's own. It is not restored.

A fixed destination keeps its pattern throughout, as before. Truncating at the failing column instead of emptying needs loop state in the catch, which costs the hot loop.

The try is outside the kernels. Minimum of 15 runs, time in ms and allocated bytes, n = 10^5 square with about 10^6 stored entries per argument (map!(cos, C, A): a 1000×1000 destination, all 10^6 entries stored):

#1071 this PR
map(sin, A) 10.548, 16815672 10.458, 16815672
map(+, A, B) 15.268, 32839224 15.249, 32839224
A .* B 12.570, 32839320 12.599, 32839320
map!(+, C, A, B) 12.577, 48 12.800, 48
C .= A .+ B 12.563, 144 12.689, 144
map!(cos, C, A) 0.207, 48 0.211, 48

The out-of-place rows do not go through the new code. The inferred result types of all six are unchanged.

Split from #1022.

Backport: this fixes a corrupted destination, but it also changes what a destination holds after a throw (empty, where it used to be invalid but with some of its old entries readable), so it is a behaviour change as well as a fix.

Written by Claude Code.

🤖 Generated with Claude Code

@codecov

codecov Bot commented Oct 13, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 95.45455% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 94.75%. Comparing base (c85e991) to head (674de31).
⚠️ Report is 4 commits behind head on main.

Files with missing lines Patch % Lines
src/higherorderfns.jl 95.45% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1082      +/-   ##
==========================================
- Coverage   94.77%   94.75%   -0.02%     
==========================================
  Files          16       16              
  Lines        9917     9958      +41     
==========================================
+ Hits         9399     9436      +37     
- Misses        518      522       +4     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@ViralBShah
ViralBShah enabled auto-merge (squash) October 13, 2026 05:28

@ViralBShah ViralBShah left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed by Codex at bd3ad1c.

No actionable findings. The guards surround the pattern-building kernels, preserve the same-pattern unary case, and clear partial mutable structure after other failures; fixed destinations retain their existing path. The preflight fill conversion remains outside the mutation region.

@ViralBShah
ViralBShah force-pushed the vs/map-throw-valid-dest branch from bd3ad1c to 78ea5b2 Compare October 13, 2026 05:31
ViralBShah and others added 2 commits October 13, 2026 05:31
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@ViralBShah
ViralBShah force-pushed the vs/map-throw-valid-dest branch from 78ea5b2 to 674de31 Compare October 13, 2026 05:34
@ViralBShah
ViralBShah merged commit e8163d4 into main Oct 13, 2026
8 checks passed
@ViralBShah
ViralBShah deleted the vs/map-throw-valid-dest branch October 13, 2026 05:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant