Skip to content

perf(h264): shrink encoder tiles - #518

Merged
gunwooterry merged 1 commit into
mainfrom
gunwooterry/compact-encoder-tiles
Nov 8, 2026
Merged

gunwooterry merged 1 commit into
mainfrom
gunwooterry/compact-encoder-tiles

Conversation

@gunwooterry

@gunwooterry gunwooterry commented Nov 8, 2026 •

Copy link
Copy Markdown
Contributor

TL;DR

  • Reduce H.264 encoder tile storage without changing encoded bytes or reconstructed samples.

Background

Encoder tile copies retain more row padding than prediction needs. This adds initialization and copying work to every macroblock.

What changes

Library output and public APIs stay unchanged.

  • Covered: internal H.264 encoder tile storage.
  • Not covered: picture workspace reuse, prediction decisions, scheduling, or deblocking.

After merging: None.

Decisions

  • Keep 16 payload columns, four top-right prediction samples and the left border in each row. The existing 18-row allocation retains every required sample while reducing each tile from 720 to 378 bytes.

Evidence

Verification run: Automatic and forced-scalar release builds each pass all 91 library tests and Clippy with warnings denied. Across ten corpus cells and 286 generated fixture cells, all 4,736 verification encodes preserve complete streams and reconstructions at 1, 2, 3 and 8 requested workers. The pinned FFmpeg 9.0.2 oracle matches all 296 reconstructions. cargo fmt --all --check passes.

Exact-head similarity at 8d2d7daaf139c0e2ebe10bc4b67b9c1d2927489b: 110 spans, 0 not allowlisted, 0 problems. One existing HEVC allowlist entry is reported stale; this change touches no HEVC code.

Links: Refs #26.

  • Every commit is signed off (git commit -s, DCO).
  • I copied nothing from other implementations.

Open in Capy


Autofix with [code]smith
Open in [code]smith · Need help on this PR? Tag @codesmith-bot with what you need.

Summary by CodeRabbit

  • Improvements
    • Updated the tile-row layout used during H.264 video encoding. This is an internal encoding adjustment; no additional user-facing features or behavior changes were specified.

Signed-off-by: Gunwoo Kim <gunwooterry@gmail.com>
@coderabbitai

coderabbitai Bot commented Nov 8, 2026 •

Copy link
Copy Markdown

Explain Change →

No actionable comments were generated in the recent review.

Walkthrough

The STRIDE constant changes from 40 to 21. The constant is used for tile storage and indexing.

Changes: Tile-row stride

Layer / File(s) Summary
Update tile-row stride
crates/coreav-h264-encoder/src/codec.rs
STRIDE changes from 40 to 21.

Estimated code review effort: 2 (Simple) | ~5 minutes

Change: Refactor

Merge Risk: Minimal · up to 8d2d7

No actionable merge-blocking risk was identified for the tile-size reduction.

Pre-merge checks: Passed 5

Check name Status Explanation
Title check Passed The title "perf(h264): shrink encoder tiles" follows Conventional Commits and names the main change: reducing H.264 encoder tile storage. The raw summary confirms the change is a single STRIDE constan…
Description check Passed The description follows the template well. It has a TL;DR with one bullet, a Background that states the cause (extra row padding adds initialization and copy work), a What changes section that covers …
Docstring Coverage Passed No eligible functions found in the changed files. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentat…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Title check

Explanation

The title "perf(h264): shrink encoder tiles" follows Conventional Commits and names the main change: reducing H.264 encoder tile storage. The raw summary confirms the change is a single STRIDE constant change from 40 to 21 in codec.rs, and the objectives describe reducing tile storage from 720 to 378 bytes. The title is short, specific, and stays under 72 characters.

Full details: Description check

Explanation

The description follows the template well. It has a TL;DR with one bullet, a Background that states the cause (extra row padding adds initialization and copy work), a What changes section that covers scope and states no public API change, a Decisions bullet explaining the retained samples and the 720-to-378-byte reduction, and an Evidence section with a verification run and a DCO checkbox. Gaps: the template asks for a 'What must a person do after merging' answer (provided as 'None'), and the template's How it works section is omitted. The template treats this section as deletable when the diff stays in one layer and user-visible behavior does not change, which applies here. The 'Needs the maintainer' section is also omitted, which is acceptable. The Background does not name who hit the issue or how often, but the template allows an intent-based or problem-based start, and the description gives a clear cause. The 'Library output and public APIs stay unchanged' line covers the user-facing before/after requirement. The description meets the template's required content.

Full details: Docstring Coverage

Explanation

No eligible functions found in the changed files. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentation-only edits and deletions within surviving functions are included; unchanged diff context and fully deleted functions are excluded. Analyzed 0 functions across 1 files.

Finishing Touches

Generate docstrings

  • Commit to this branch
  • Create a new PR
Generate unit tests (beta)

  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
Recent review info

Run configuration

Commits

Reviewing files that changed from the base of the PR and between eb978f4 and 8d2d7da.

Files selected for processing: 1

  • crates/coreav-h264-encoder/src/codec.rs

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 4 remain after this review.

@capy-ai

capy-ai Bot commented Nov 8, 2026

Copy link
Copy Markdown

Review: no blocking issue at head 8d2d7daaf139c0e2ebe10bc4b67b9c1d2927489b

I checked every index into Tile against STRIDE = 21, ORIGIN = 22 and 378 bytes. Row r, column c is index 21r + c, so a row holds x = -1..19: the left border, 16 payload samples and 4 top-right samples. Payload rows are y = 0..15. The last payload sample at(15, 15) is 352, and row 17 (357..377) is unused.

  • Intra neighbours. IntraMut::new and BlockMut::new (coreav-kernels-h264 views.rs) check that the whole range fits the slice. For a 4x4 block at x = 12, the top-right samples are x = 16..19 on row y - 1, still inside that row. The dirty check in levels.rs:169-173 reads at most at - STRIDE + 7, so x = 19. left() and top_left() read only when avail allows, and the encoder runs intra prediction through Scalar, so no x86 intra kernel reads a whole row here.
  • Chroma. For n = 8, the top fill ORIGIN - STRIDE .. + 16 is 1..17, all inside the border row. at(7, 7) is 176.
  • SIMD. The encoder's AVX2 satd and sad load exactly width bytes per row inside a checked extent. The shared luma_mc and chroma_mc x86 stores are exact width (8/16-byte rows, row_mut::<W>, store_bytes(.., 8)), so no store reaches the next row's left border. The 4-wide luma path copies into a private 16-byte window. No kernel reads or writes 16 or 32 bytes past a 4- or 8-wide tile row.
  • Literals. STRIDE is used everywhere. The only 21 literal is the motion-compensation window in inter.rs:86-108 (16 + 5 taps). It indexes its own [0; 21 * 21] array and does not depend on the tile stride. No 40, 41 or 720 is left, and no assert or size check assumes the old layout.
  • Tests. At this head on arm64, cargo test --release -p coreav-h264-encoder --features full --lib and the same with scalar each pass 91/91.
  • PR text. pr_text_check.py reports the title and body clean against the private terms list. The title is 32 characters, the template is followed, there are no hard wraps, the commit is signed off and has no Co-authored-by trailer, and there are no FFmpeg speed figures. Greptile is paused and left no findings. CI and CodeRabbit were still running when I reviewed.

Non-blocking

  1. rows.rs:72-76 (and the test-only FramePlane::tiles, codec.rs:668) still copies 2n = 32 samples from ORIGIN - STRIDE for luma. With stride 21, the last 12 samples land on p[-1, 0] (index 21) and on payload row 0, x = 0..10. Today this is correct for four reasons. The left fill runs afterwards and restores index 21 when left exists. When the macroblock is at the picture's left edge, left() is gated by avail. Prediction writes the payload before anything reads it. The dirty check compares two tiles that come from the same base. Still, correctness now depends on the order of the fills, which nothing enforces. I suggest copying only the 4 top-right samples that are used (ORIGIN - STRIDE + n .. + n + 4 for luma) and adding const _: () = assert!(STRIDE >= 16 + 4 + 1); so the layout's constraint is stated in code. Either fits a follow-up.

@gunwooterry
gunwooterry added this pull request to the merge queue Nov 8, 2026
Merged via the queue into main with commit bcfab38 Nov 8, 2026
44 of 45 checks passed
@gunwooterry
gunwooterry deleted the gunwooterry/compact-encoder-tiles branch November 8, 2026 08:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant