Skip to content

perf(gif): decode a gif once across rayon workers - #239

Merged
LeadcodeDev merged 1 commit into
chantier/audit-2026-09from
perf/gif-decode-stampede
Sep 22, 2026
Merged

LeadcodeDev merged 1 commit into
chantier/audit-2026-09from
perf/gif-decode-stampede

Conversation

@LeadcodeDev

Copy link
Copy Markdown
Owner

Severity Medium, category quality. Location: crates/rustmotion-components/src/gif.rs:176

Impact

Triggering scenario: a scenario with one gif component, rendered to MP4. encode_with_ffmpeg (ffmpeg.rs:582) dispatches the FIRST chunk of batch_size = rayon::current_num_threads() * 2 frames — all frames 0..N of the same scene — to par_iter() at once. Every one of those workers calls Gif::paint_content, every one misses the still-empty gif_cache, and every one runs decode_composed_frames independently. Unlike icons and videos, GIFs are NOT preloaded (preload.rs only walks Component::Icon and Component::Video), so the first decode genuinely happens inside the parallel section. Each decode is itself unbounded: decode_composed_frames pushes one full-canvas RGBA copy per GIF frame with no cap (gif.rs:141). A 600-frame 800x600 GIF is 1.15 GB per decode; on a 16-core machine (batch_size 32) that is ~37 GB attempted concurrently, then 31 of the 32 results are thrown away when insert overwrites. The process OOM-kills before frame 33.

Fix

Decode GIFs in the preload pass alongside icons and videos (preload.rs), so the parallel section only ever hits a warm cache. If lazy decode must stay, gate it on a per-key single-flight primitive (e.g. DashMap<String, Arc<OnceLock<GifData>>> — insert the empty cell under the shard lock, then let exactly one worker fill it) and cap total decoded frames per source.

Evidence the audit read

let cached = if let Some(cached) = gcache.get(&self.src) {
    cached.clone()
} else {
    let Some(decoded) = decode_composed_frames(&self.src) else {
        return;
    };
    let cached = Arc::new(decoded);
    gcache.insert(self.src.clone(), cached.clone());
    cached
};
// decode_composed_frames, line 141:
frames.push((composed.clone(), canvas_w, canvas_h));

Based directly on the chantier branch.

Part of the September 2026 audit remediation chantier. Refs #220 (RM-20).

@LeadcodeDev LeadcodeDev added the bug Something isn't working label Sep 21, 2026
@LeadcodeDev LeadcodeDev self-assigned this Sep 21, 2026
@LeadcodeDev
LeadcodeDev force-pushed the perf/gif-decode-stampede branch 2 times, most recently from 4d15a91 to c89ddad Compare September 22, 2026 08:34
Triggering scenario: a scenario with one gif component, rendered to MP4.
encode_with_ffmpeg (ffmpeg.rs:582) dispatches the FIRST chunk of batch_size
= rayon::current_num_threads() * 2 frames — all frames 0..N of the same
scene — to par_iter() at once. Every one of those workers calls
Gif::paint_content, every one misses the still-empty gif_cache, and every
one runs decode_composed_frames independently. Unlike icons and videos, GIFs
are NOT preloaded (preload.rs only walks Component::Icon and
Component::Video), so the first decode genuinely happens inside the parallel
section. Each decode is itself unbounded: decode_composed_frames pushes one
full-canvas RGBA copy per GIF frame with no cap (gif.rs:141). A 600-frame
800x600 GIF is 1.15 GB per decode; on a 16-core machine (batch_size 32) that
is ~37 GB attempted concurrently, then 31 of the 32 results are thrown away
when insert overwrites. The process OOM-kills before frame 33.

Fix: Decode GIFs in the preload pass alongside icons and videos
(preload.rs), so the parallel section only ever hits a warm cache. If lazy
decode must stay, gate it on a per-key single-flight primitive (e.g.
DashMap<String, Arc<OnceLock<GifData>>> — insert the empty cell under the
shard lock, then let exactly one worker fill it) and cap total decoded
frames per source.

Refs #220
@LeadcodeDev
LeadcodeDev force-pushed the perf/gif-decode-stampede branch from c89ddad to 1c12b20 Compare September 22, 2026 08:44
@LeadcodeDev
LeadcodeDev merged commit 58c145e into chantier/audit-2026-09 Sep 22, 2026
3 checks passed
@LeadcodeDev
LeadcodeDev deleted the perf/gif-decode-stampede branch September 22, 2026 08:53
LeadcodeDev added a commit that referenced this pull request Sep 22, 2026
Triggering scenario: a scenario with one gif component, rendered to MP4.
encode_with_ffmpeg (ffmpeg.rs:582) dispatches the FIRST chunk of batch_size
= rayon::current_num_threads() * 2 frames — all frames 0..N of the same
scene — to par_iter() at once. Every one of those workers calls
Gif::paint_content, every one misses the still-empty gif_cache, and every
one runs decode_composed_frames independently. Unlike icons and videos, GIFs
are NOT preloaded (preload.rs only walks Component::Icon and
Component::Video), so the first decode genuinely happens inside the parallel
section. Each decode is itself unbounded: decode_composed_frames pushes one
full-canvas RGBA copy per GIF frame with no cap (gif.rs:141). A 600-frame
800x600 GIF is 1.15 GB per decode; on a 16-core machine (batch_size 32) that
is ~37 GB attempted concurrently, then 31 of the 32 results are thrown away
when insert overwrites. The process OOM-kills before frame 33.

Fix: Decode GIFs in the preload pass alongside icons and videos
(preload.rs), so the parallel section only ever hits a warm cache. If lazy
decode must stay, gate it on a per-key single-flight primitive (e.g.
DashMap<String, Arc<OnceLock<GifData>>> — insert the empty cell under the
shard lock, then let exactly one worker fill it) and cap total decoded
frames per source.

Refs #220
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant