Repository navigation
Photos alias when shrunk and turn blocky under camera zoom: image and video sample with nearest neighbour and no mipmaps #465
Description
image and video draw their bitmap with canvas.draw_image_rect(img, None, dst, &Paint::default()), which samples with Skia's default SamplingOptions: nearest neighbour, no mipmaps. A photo drawn smaller than its source picks one source pixel out of N and drops the rest, so it aliases (jagged hair, stair-stepped glasses, moiré on fabric). A photo drawn larger, for instance under a camera zoom, is enlarged pixel by pixel into visible blocks.
crates/rustmotion-components/src/image.rs:
let dst = Rect::from_xywh(offset_x, offset_y, draw_w, draw_h);
let paint = Paint::default();
// ...
canvas.draw_image_rect(img, None, dst, &paint);video.rs (draw_fitted) has the same call. The camera is applied as a canvas matrix (apply_camera_transform scales the canvas), so the scale the image is finally drawn at is the layout scale times the camera zoom, and both go through that nearest-neighbour sampling.
Reproduction
A 640x640 checkerboard of 1px black/white cells (checker.png), drawn at 64px and 120px on a #808080 background. Correct downsampling averages it to the background grey.
{
"version": "1.0",
"video": { "width": 320, "height": 160, "fps": 30, "background": "#808080" },
"scenes": [{
"duration": 1.0,
"layout": { "direction": "row", "align_items": "center", "justify_content": "center", "gap": 40 },
"children": [
{ "type": "image", "src": "checker.png", "fit": "cover", "style": { "width": 64, "height": 64 } },
{ "type": "image", "src": "checker.png", "fit": "cover", "style": { "width": 120, "height": 120 } }
]
}]
}Luma inside each image (4px inset), render --frame 0:
| drawn size | mean | min | max | near-black or near-white pixels |
|---|---|---|---|---|
| 64px (current) | 255 | 255 | 255 | 100% |
| 120px (current) | 124 | 0 | 255 | 100% |
| 64px (mipmapped) | 127 | 127 | 127 | 0% |
| 120px (mipmapped) | 127 | 127 | 127 | 0% |
At 64px the scale is exactly 1/10, so every sampled pixel lands on the same parity and the image renders solid white. At 120px it is black/white noise.
Seen in production on a video using portraits: 640px faces drawn as 52px avatars in a table had visibly jagged hair and glasses, and a wall of 400px portraits under a zoom: 3.4 camera move showed pixel blocks.
Proposed fix
Choose the sampling from the on-screen size, camera included:
- drawn smaller than the source:
SamplingOptions::new(FilterMode::Linear, MipmapMode::Linear); - drawn at or above source size:
SamplingOptions::from(CubicResampler::mitchell())(cubic does not use mipmaps, so it is only right when enlarging).
The on-screen width is dst.width() times the scale of canvas.local_to_device_as_3x3(), which already includes the camera zoom. A shared helper in rustmotion-core::engine::renderer (draw_photo(canvas, image, dst, paint)) can replace the draw_image_rect call in image.rs and video.rs. A local patch along those lines gives the "mipmapped" rows above. icon.rs already does something similar with CubicResampler::mitchell().
The other places that draw a bitmap with Paint::default() are not covered by that patch and likely have the same problem: avatar.rs, avatar_group.rs, badge.rs, list.rs, stat.rs, mockup.rs, gif.rs, svg.rs (raster fallbacks), lottie.rs, engine/transition.rs and the sheet command.
Activity
LeadcodeDev commentedon Oct 1, 2026
Working on this.
LeadcodeDev commentedon Oct 1, 2026
Mini-spec
Goal. One sampling decision, taken from the on-screen size the bitmap is finally drawn at, applied everywhere a raster bitmap is scaled to a destination rect.
Shape.
renderer::photo:photo_sampling(source, on_screen) -> SamplingOptions(pure, mipmapped linear when either axis minifies, Mitchell cubic otherwise) plusdraw_photo(canvas, image, dst, paint), which reads the on-screen size offcanvas.local_to_device_as_3x3()so the camera zoom is included. The axis scales come fromhypot(scale_x, skew_y)rather thanscale_xalone, so a rotated matrix does not report a shrink that is not there.Invariants.
Scope. 14 call sites:
image,video,avatar,avatar_group,badge,list,stat,mockup,gif,lottie,svgraster fallback,icon,transition::render_layer, and thesheetcommand.Out of scope, with the reason:
transition.rs:273andengine/render/scene.rs:207composite a full frame buffer at 1:1 through the point form ofdraw_image. Nothing is scaled, so the sampling is never consulted.paint_pass.rs:934draws a layer snapshot, also through the point form. It is under a scaling matrix, so it has the same defect, but a snapshot of already-rasterised content is a different question from a photo and the point form takes no dst rect. Left out deliberately.engine/text/cosmic.rs:196is the cosmic-text bridge, which CLAUDE.md records as not wired into real rendering.Acceptance.
main.main.Verify (exit).
cargo fmt --all --check;cargo clippy --workspace --all-targets --features rustmotion/studio -- -D warnings;cargo test --workspace --features rustmotion/studio.Risk. Mipmap generation costs memory and time per image on first draw. Measured before the PR goes up.
LeadcodeDev commentedon Oct 1, 2026
Fixed in #468.
Twenty call sites rather than the eleven listed:
video,lottieandsvgeach had more than one, andtransition::render_layeris reached throughsurface.canvas()so a grep forcanvas.draw_image_rectmisses it.iconwent through the helper too — it already asked for Mitchell, but unconditionally, so a shrunk icon had no mipmaps.Verified: the checkerboard reads mean 127 at both 64px and 120px where it read 255 solid and 0-to-255 noise before, measured through the
imagecomponent end to end and confirmed red before the fix. Two of the unit tests assert the old path's numbers so the fix cannot quietly become a no-op. A 640px rect on a canvas scaled to 0.1 confirms the camera zoom is read, not just the layout size. Full suite: 2153 passed, 0 failed.Measured cost, 300 frames at 1920x1080: twelve photos at 52px go 1.01s to 1.13s, one 640px source filling the frame goes 1.79s to 2.59s. The enlargement is where Mitchell is paid for.
Not done, and named in the PR:
paint_pass.rs:934draws a layer snapshot under a scaling matrix through the point form ofdraw_image, so it has the same defect. A snapshot of already-rasterised content is a different question from a photo and the point form takes no dst rect, so it is left out rather than guessed at. Say if you want an issue for it.