Skip to content

Feature/video frame splitting - #101

Merged
KnifeLemon merged 27 commits into
KnifeLemon:mainfrom
pleguen:feature/video-frame-splitting
Oct 18, 2026
Merged

KnifeLemon merged 27 commits into
KnifeLemon:mainfrom
pleguen:feature/video-frame-splitting

Conversation

@pleguen

@pleguen pleguen commented Oct 17, 2026

Copy link
Copy Markdown
Contributor

What does this change?

Adds video frame extraction support to split video files into image sequences (PNG / JPG) within a dedicated, collision-safe subfolder (<video>_frames).

Following the feedback from the initial discussion, this PR implements:

  1. Preset Option: Added SaveEveryFrame (off by default) and FrameInterval ("Every Nth frame") settings under MediaOptions, exposed via PresetEditorView.
  2. Built-in Presets & Schema Migration (v10): Added extract-frames-png and extract-frames-jpg presets. They are deliberately excluded from the default donut profiles to prevent accidental high-volume exports.
  3. Preserved Still Frame Behavior: Kept PosterFrame intact with anamorphic pixel ratio handling (SquarePixels) and single still-frame tests so existing video-to-document routes remain unaffected.
  4. Collision Handling: Generates unique subfolder names (<video>_frames, <video>_frames (2), etc.) without overwriting existing directories, using a 6-digit naming pattern (frame_%06d.<ext>).
  5. Quality & Interval Mapping: Maps preset Image.Quality to FFmpeg's mjpeg -q:v range (1..31) and applies frame interval filtering (select='not(mod(n\,N))').

How was it tested?

  • dotnet format --verify-no-changes
  • dotnet test (all engine, schema migration, and preset parsing tests pass)
  • Tried it in the app (describe):
    • Converted a video file to PNG and JPG with "Save every frame as images" checked.
    • Verified that frames are extracted into a dedicated _frames subfolder with 6-digit numbering (frame_000001.png).
    • Tested running the conversion multiple times on the same file: properly created _frames (2) without collisions.
    • Tested "Every Nth frame" interval (e.g., set to 5): verified that only 1 in 5 frames was extracted.
    • Verified that standard video-to-image conversions without this option checked still yield a single still poster frame.

Screenshots (UI changes)

(Attached: Preset editor showing the "Save every frame as images" checkbox and "Every Nth frame" input)
image

Checklist

  • New UI strings are in all src/Filee.App/Assets/i18n/*.json files (en.json, ko.json, zh-CN.json)
  • New third-party code or binaries are listed in THIRD-PARTY-NOTICES.md (N/A: no new third-party dependencies added)

pleguen and others added 5 commits October 12, 2026 21:38
calibre's GitHub releases keep files only for the newest version: when 9.16.0
came out, the pinned 9.15.0 .txz and .dmg links there started returning 404,
so installing calibre failed on macOS and Linux. download.calibre-ebook.com
keeps every version; the files are the same (sizes and SHA-256 checked).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each PDF page is rendered with PDFium and becomes one JPEG picture on its own
slide, so the deck looks exactly like the PDF (not editable). The slide size is
the first page's own size (A4 portrait → portrait slides, 16:9 → 16:9); every
page is fitted and centred. Render DPI from the preset (150..600, pages over
25 megapixels render smaller), page range from the preset, JPEG quality from
the preset (90). The page text (PdfPig, reading order) goes into the speaker
notes. The package is written by hand and validated with the Open XML SDK in
the tests.

The edge is DirectOnly, a new ConversionEdge flag: the planner uses it only
for a PDF converted to PPTX, never as a step of a longer route, so no other
pair of formats changes (checked for every pair).

"PPTX (slides)" preset on the PDF donut after DOCX: settings schema 18 puts it
in place of TIFF on an unchanged default donut and offers it to a changed one
with room; the "pdf-slides" engine joins the priority list after "pdf-text".

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@pleguen
pleguen marked this pull request as draft October 17, 2026 19:32
Co-authored-by: Junie <junie@jetbrains.com>
@pleguen
pleguen marked this pull request as ready for review October 17, 2026 20:12
KnifeLemon and others added 12 commits October 18, 2026 12:52
A Filee running as administrator (started by an installer run elevated)
ignores drops from Explorer, can't read Explorer's selection, and the
Filee that Explorer's menu starts crashed on the elevated Filee's mutex
or waited 30 s on its pipe, all without a log line.

- Read Filee's and Explorer's integrity levels at start-up. When Filee
  runs elevated and Explorer doesn't: log a warning, show a banner on
  the Home page and a one-time notice card, both with "Restart without
  administrator rights" (relaunch through Explorer's shell, after
  releasing the single-instance lock; a failed restart says so).
- A later launch that can't open the lock or is refused by the pipe
  logs it and shows one message box (once per burst of launches); the
  pipe refusal no longer waits the whole 30 s.
- Log unhandled exceptions (any thread, unobserved tasks, UI thread)
  through a logger set up before anything else, and one start-up line
  with version, path, install kind, integrity levels, OS and argument
  count.
- UI watchdog: logs a UI thread blocked for more than 5 s and when it
  answers again.
- Rate-limited diagnostics: forwarded launches received and handled,
  drag gestures that never reached the donut, shortcuts that found no
  selection (with the foreground window), the first drag over the drop
  zone.
- FFmpeg -version / -encoders time out for real (background reads, kill
  on timeout); engine statuses are checked once in the background at
  start-up; unreachable network shares on PATH are skipped after 2 s.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A new preset option, off by default, encodes H.264 and HEVC on the GPU:
NVENC, Quick Sync and AMF on Windows, VideoToolbox on macOS (quality
mode on Apple Silicon, bitrate mode on Intel Macs), NVENC and VA-API on
Linux. Only encoders the bundled FFmpeg has are tried, best first, and
each must pass a tiny test encode with the real options (cached per
FFmpeg, encoder and bit depth) before it is used.

Quality levels map to each encoder's constant-quality control, one step
coarser than the x264/x265 CRF. HEVC keeps 10 bits, HDR colour tags and
the hvc1 tag; H.264 stays 8-bit 4:2:0 as with x264. Stream copy rules
are unchanged, and AV1, VP9 and the other codecs stay in software.

When a hardware encode fails anyway, the partial output is removed and
the file is encoded again with x264/x265; the failure goes to the log,
not to the user. Cancellation is never retried.

FFmpeg's version and encoder queries now read output asynchronously and
kill the process at their time limit, so a hung FFmpeg no longer blocks
engine detection.

The preset editor shows the option for H.264 and HEVC targets, with
English, Korean and Simplified Chinese text. ENGINES.md documents the
encoders, the probe, the quality mapping and the software retry.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…n#107)

Avalonia toggles the selected item of a ComboBox or ListBox off when it is
picked again with the command modifier (Ctrl+Space, Ctrl+Enter, Ctrl+click;
Cmd on macOS). A combo box was left empty and pushed null into settings that
always hold a value, which threw NullReferenceException in the view models
(e.g. TriggerGestureViewModel) and showed it under the combo box.

- SingleSelectionGuard (registered in App.Initialize) swallows that toggle on
  the selected item of combo boxes and single-selection list boxes; an open
  drop-down closes as after any pick. Multi-selection lists keep the toggle.
- Choices.Restore: every Choice<T> property (Triggers, Presets editor,
  General, Theme, Watch folders, rename rules) and the navigation selection
  ignore null and keep their previous value; the control then reads the bound
  value again.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The direct video → JPG edge and the MJPEG poster frame changed every video →
JPG conversion: rotate and flip presets failed with "can only turn images"
and resizing was ignored. Video → JPG is again one PNG still frame that the
image engines take on.

Saving every frame now has edges of its own (ConversionEdge.FrameSequence)
that the route planner takes only when asked for (Plan(..., frames: true)),
and then as the whole route. JobQueue and the donut's availability check ask
for it when Preset.SavesFramesOf the source: the preset saves every frame,
its target is PNG or JPG and the file is a video. The step says so
(ConversionStep.FrameSequence), so FFmpeg no longer guesses from extensions.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Whether a file's frames are saved is decided once, from the format registry
(a Video source, a PNG or JPG target by format id), instead of extension
lists and guesses; the old fallback to the preset's target made intermediate
steps of other routes (video → WEBP, PDF) write folders. The editor shows
the option for "png" and "jpg" (there is no "jpeg" id) and saves it, with
its interval, only while it is shown, so a box ticked for PNG and left behind
by another target does nothing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Without a frame rate filter the image2 muxer paced frames to a constant rate
and wrote the slow parts of a phone video several times (119 pictures for a
60-frame clip). Frames now pass through with their own timestamps
(-fps_mode passthrough; vfr with the interval's select filter), only the
first real video stream is read (-map 0:V:0, as for the still frame), and
the PNG / MJPEG encoder and image2 muxer are named.

The test records a clip at 30 then 10 fps with FFmpeg and checks there are as
many pictures as decoded frames.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The frames folder was named by hand next to a JPG/PNG path the allocator had
reserved: "Skip" and the parallel-file reservation didn't apply, and a
cancelled or failed run left a half-filled folder. It is now a folder output
like "Extract" (Allocate("", "_frames")): "clip_frames", "clip_frames (2)",
skipped or overwritten by the preset's rule. FrameSequenceWriter writes into
a hidden folder next to it and moves the frames into place only when FFmpeg
has finished; whatever happens, the hidden folder is removed.

FfmpegRunner now waits (up to 10 seconds) for a killed FFmpeg to exit, so the
partial output it still held open can be deleted on cancel. A % in the path
is escaped for FFmpeg's number pattern.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
SharpHook's libuiohook-xrecord.so and libuiohook-x11.so need GLIBC_2.38
(__isoc23_sscanf, built on Ubuntu 26.04) and rhwp's Linux release needs
GLIBC_2.39, so neither starts on Ubuntu 22.04 (KnifeLemon#102).

- portable.yml: a linux-native job builds both from pinned, unmodified
  source on ubuntu-22.04 and ubuntu-22.04-arm (build/build-linux-native.py:
  libuiohook at SharpHook 8.0.0's submodule commit with SharpHook's CMake
  flags and unit tests; rhwp v0.8.7 with cargo +1.93.1 --locked), cached.
  The Linux package jobs pass them to build-portable.ps1 (-RhwpOverride,
  -UiohookOverride); local builds keep using the downloads.
- build-linux-native.py check fails when Directory.Packages.props or
  engines.json move to another SharpHook or rhwp version than it builds.
- build/check-elf-glibc.py (standard library ELF reader) lists the newest
  GLIBC version of every ELF file and fails above 2.35, on the native builds
  and on every Linux package.
- The package smoke steps move to build/smoke-portable.ps1 and
  build/smoke-launch-app.py, add TXT -> HWPX -> HWP -> PDF through rhwp, load
  libuiohook-xrecord.so and fail on ErrorLinuxLoadBackend in the app log.
  A new job runs them on Ubuntu 22.04 (x64 and arm64) with its own packages.
- The Shortcuts page explains a Linux back end that doesn't load (X11
  session, libxtst6 and libxt6) instead of the raw error; the log keeps the
  libuiohook result.
- INSTALL.txt has the Ubuntu 22.04 and 24.04 dependency commands with
  libxt6; READMEs say Ubuntu 22.04+; notices and docs describe the builds.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…holds

Before anything is written, the frames are estimated (duration × frame rate ÷
interval). More than 100,000 pictures (an hour at 30 fps) fails with "Too
many frames … raise 'Every Nth frame'", and pictures that won't fit on the
output drive (a generous size guess per PNG or JPG) with "Not enough free
disk space", both in the user's language with the numbers in brackets. A
video that doesn't say its length or frame rate is not refused.

Engines can now refuse a file with such a reason: ConversionRefusedException
carries the error key and detail, which JobQueue shows instead of "The
engine reported an error". The editor says below the interval how many
pictures a minute of 30 fps video gives.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MediaOptions.FramesPerSecond had no control in the editor and no preset set
it: removed, with FFmpeg's fps branch. The preset names are in sentence case
like the others ("Extract frames (PNG)"), the hint and description keys no
screen showed are gone, the Korean texts end in 해요 and the Chinese ones
have no fullwidth colon. The progress test expects three reports between
start and end again, as before this branch.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Schema 18 is taken by PDF → PPTX (KnifeLemon#98), so "Extract frames (PNG / JPG)" come
with 19. They are inserted after "Split PDF", where a new library has them
(BuiltInData.FramePresets), instead of at the end, and stay on no donut.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ENGINES.md describes the frame route and when it is taken, the folder and
file names, the interval, variable frame rate, square pixels, HDR (not
tone-mapped) and 16-bit PNG, the JPG quality mapping, the 100,000-picture
and disk space limits, and that a cancelled run leaves nothing behind.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@KnifeLemon

Copy link
Copy Markdown
Owner

Thanks a lot for this, @pleguen!
Saving every frame is a great addition. I pushed a few commits on top of yours to get it ready to merge. Your commits are untouched.

  • Routing: frame extraction is now a separate route, used only when "Save every frame" is on. A normal MP4 → JPG still makes one poster frame (and can still be rotated), and frames are never an intermediate step to another format.
  • Variable frame rate: frames are written as decoded (-fps_mode passthrough, vfr with an interval), so VFR videos no longer get duplicated frames.
  • Output folder: uses the normal output naming (clip_frames, clip_frames (2), skip if it exists). Frames are written to a hidden staging folder and moved into place at the end, so a cancelled job leaves nothing behind. % in paths is escaped.
  • Limits: a job that would write more than 100,000 frames, or more than the free space on the drive, is refused up front. The preset editor shows an estimate for the chosen interval.
  • Cleanup: removed the unused FPS option and keys, renamed the preset to "Extract frames (PNG/JPG)", and moved the settings migration to schema 19.
  • Docs in docs/ENGINES.md, tests for each of the above.

I'll merge it once CI passes. Thanks again!

KnifeLemon and others added 7 commits October 18, 2026 20:20
SharpHook 8.0.1 builds libuiohook for older glibc again: all of its Linux
back ends (x64, arm64, arm) need GLIBC_2.34 at most. The Linux packages use
SharpHook's own libraries again, so the libuiohook source build, its CI
dependencies and -UiohookOverride are gone. rhwp is still built from source
on Ubuntu 22.04. The glibc audit and the Ubuntu 22.04 smoke job stay and fail
if a SharpHook or engine update raises the requirement again.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Generated SVGs often have "Q&A" in a <text> or "&nbsp;"/"&copy;", which
stopped Svg.Skia's XML parser ("An error occurred while parsing
EntityName"). When an SVG can't be read, Filee escapes ampersands that
don't start a reference and turns HTML entities into character
references, then reads it again. Declared DTD entities, the DOCTYPE,
comments and CDATA are left alone; valid SVGs are never changed.

Closes KnifeLemon#120.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…to HEAD

# Conflicts:
#	src/Filee.App/App.axaml.cs
… into HEAD

# Conflicts:
#	src/Filee.Engines/Media/FfmpegConverter.cs
# Conflicts:
#	src/Filee.App/ViewModels/PresetEditorViewModel.cs
#	src/Filee.Core/Conversion/IConverter.cs
#	src/Filee.Core/Conversion/RoutePlanner.cs
#	src/Filee.Core/Presets/BuiltInData.cs
#	src/Filee.Core/Presets/Preset.cs
#	src/Filee.Core/Settings/AppSettings.cs
#	src/Filee.Core/Settings/UserDataStore.cs
#	src/Filee.Engines/Media/FfmpegConverter.cs
#	tests/Filee.Core.Tests/RoutePlannerTests.cs
@KnifeLemon
KnifeLemon merged commit 3abc4bd into KnifeLemon:main Oct 18, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants