Skip to content

feat: render with ffmpeg alone, from and to URLs the caller presigns - #2

Open
rainhead wants to merge 1 commit into
mainfrom
ffmpeg-only
Open

rainhead wants to merge 1 commit into
mainfrom
ffmpeg-only

Conversation

@rainhead

Copy link
Copy Markdown
Collaborator

The renderer is now one ffmpeg command and the function no longer knows what S3 is. The image drops from 2.27 GB to 773 MB, there are no caches to seed, and a render takes 2 s at 1024 MB with a cold start of about a second. On a real production segment the output has production's width exactly (1877×512) and its luminance distribution to within 0.01.

Before and after

Production's image of rpi_orcasound_lab/hls/1790665213/live5238.ts (left) and this branch's (right), a 420-column crop from the middle of the segment. Production is a touch softer from its Lanczos resampling; the floor colour, feature colour and events are the same.

production on the left, ffmpeg render on the right

Luminance 10th/50th/90th percentiles: 0.193/0.199/0.323 here against 0.200/0.200/0.324 in production; block-averaged (32 px) correlation 0.97.

How it renders

app.py: showspectrumpic draws the spectrogram as intensity, then pseudocolor applies matplotlib's viridis (showspectrumpic's own viridis starts at black, so it cannot match the archive). The window size follows from the height (n_fft/2 rows) and the width is one column per hop_length samples, sized from the samples actually decoded rather than the container's duration, which undercounts an AAC stream by a frame or two (that cost 16 columns on the segment above before I caught it). The archive's dB window is relative to an amplitude of 0.01 in librosa's STFT; DB_OFFSET converts it to ffmpeg's full-scale reference and was fitted against the production image.

Contract

Request: audio_url (presigned GET), image_url (presigned PUT, or null to render and discard), and optional parameters with the archive's values as defaults. Response: image_size, sample_rate, width, height, the parameters applied, and the field names orcasite has stored since the first renderer. The previous audio_bucket/audio_key/image_bucket/image_key shape still works through the boto3 the base image ships, so this can deploy before orcasite changes; the S3 policy in template.yaml stays until then and goes with it.

The orcasite side (presign both URLs, pass parameters, find the function by config instead of listing every Lambda, delete server/audio_viz) is a separate PR there.

Verification

tests/smoke.py renders a synthetic AAC/MPEG-TS clip through the handler inside the built image as Lambda runs it (unprivileged, root-owned empty /tmp, 1024 MB tier); CI runs it. The comparison above was made the same way with the real segment through lambda_handler.

🤖 Generated with Claude Code

The Python stack (librosa, numba, OpenCV, matplotlib) is replaced by one
ffmpeg command: showspectrumpic draws the spectrogram as intensity and
pseudocolor applies matplotlib's viridis, which showspectrumpic's own
palette does not match. The image is sized from the decoded sample count
so tiles keep the archive's width, and the dB window is converted to
ffmpeg's full-scale reference by an offset fitted against a production
image. On the same segment the result has production's luminance
distribution to within 0.01 and the same width.

The request now carries presigned GET and PUT URLs and explicit render
parameters, echoed back in the response, so the function knows nothing
about S3. The previous bucket/key shape still works, via boto3 from the
base image, until orcasite has switched.

Image 2.27 GB -> 773 MB, no caches to seed, render 2 s at 1024 MB.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@rainhead

Copy link
Copy Markdown
Collaborator Author

@dbainj1 and @scottveirs, I broke this component out of the orcasite codebase. Claude then proposed to removing the Python-based orchestration around rendering, to slim it down to its ffmpeg bones. The result is a little different. Is the difference acceptable?

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant