From a20c036d37f84f49f814f336e7a4f92f6e455941 Mon Sep 17 00:00:00 2001 From: Gunwoo Kim Date: Sun, 8 Nov 2026 12:40:21 +0900 Subject: [PATCH 1/6] docs(cli): design opt-in H.264 encoding Signed-off-by: Gunwoo Kim --- docs/design/cli-native-h264.md | 27 +++++++++++++++++++++++++++ 1 file changed, 27 insertions(+) create mode 100644 docs/design/cli-native-h264.md diff --git a/docs/design/cli-native-h264.md b/docs/design/cli-native-h264.md new file mode 100644 index 000000000..c9840264d --- /dev/null +++ b/docs/design/cli-native-h264.md @@ -0,0 +1,27 @@ +# Opt-in H.264 encoding through the CLI + +## Selection + +`coreav --native-h264-encoder ffmpeg -i in.mp4 -an -c:v libx264 -crf 23 out.mp4` prefers CoreAV's encoder. The `native-h264-encoder` Cargo feature enables the flag and the library wiring. Neither the feature nor its dependencies enter default, release-default, or release-full feature sets. A decoder backend remains a separate opt-in. + +The selector belongs to CoreAV, before the subcommand. A new codec name would require rewriting fallback arguments. A flag inside FFmpeg's arguments could collide with an option value or filename. An environment variable would make selection implicit. The outer flag avoids all three problems. Without the flag, the existing path remains unchanged, including in a feature-enabled build. + +## Admission and fallback + +The existing argument parser and transcode planner remain authoritative. Native selection requires explicit effective `-c:v libx264`, a file output in MP4 or Matroska, and no unsupported encoder options. Integer `-crf` values from 10 through 51 map to `RateMode::Crf`; integer `-qp` values in the same range map to `RateMode::FixedQp`. Neither option means CRF 23. Both together, fractional values, bitrate, VBV, profiles, B-frame controls, presets, filters, and other unmodeled controls keep the existing path. The original FFmpeg arguments are never rewritten for fallback. + +Decoded pictures must be progressive planar 8-bit 4:2:0, with even dimensions from 16 through 1920 horizontally and 16 through 1080 vertically. The source must also fit those dimensions. The library currently signals exactly 30/1 with square pixels, so other clocks, variable frame rates, and non-square pixels keep the external path. Unsupported color descriptions also keep it. The adapter maps absent color code points to unspecified, without converting samples. + +Native output uses Constrained Baseline, enabled deblocking, and an IDR interval of 250. The native encoder receives the requested thread count, but its output cannot depend on that count. Existing decoding policy, staging, spool limits, timestamps, audio handling, and muxing are reused. Strict-safe selection retains today's path and policy rather than changing decoder or process authorization. + +Native admission and library failures are transactional. Before publication, an unsupported frame or library refusal discards the native staging and spool files, then reruns the unchanged command through today's external encoder or FFmpeg fallback. I/O, malformed-input, and deadline failures remain terminal. A default command never enters this retry path. + +## Verification + +- Compare default-build command output with the pre-change binary, and compare feature-enabled commands without the flag with today's external path. +- Exercise every excluded option and input category, requiring unchanged fallback arguments and byte-identical output with and without the flag. +- Encode CRF and fixed-QP boundary values through the CLI, check emitted syntax and clocks, and decode the stored output with pinned FFmpeg 9.0.2 and CoreAV. +- Compare complete output bytes across thread counts, both output containers, and the existing CLI limit and timeout tests. +- Check default and release feature graphs, generated command documentation, public text, DCO, CI, and exact-head similarity. + +Speed and quality comparisons are outside this change. Configurable VUI timing, sample aspect, and additional native encoder capabilities require separate library work. From 87a1dcdb5d51bfd7de057e3388d5d77b34acd3f3 Mon Sep 17 00:00:00 2001 From: Gunwoo Kim Date: Sun, 8 Nov 2026 13:21:11 +0900 Subject: [PATCH 2/6] feat(cli): wire native H.264 encoder preference Signed-off-by: Gunwoo Kim --- .github/workflows/h264-encoder.yml | 85 +++++++ Cargo.lock | 1 + crates/coreav-cli/Cargo.toml | 1 + crates/coreav-cli/src/compat_cmd.rs | 31 +++ crates/coreav-cli/src/ffmpeg/args.rs | 32 ++- crates/coreav-cli/src/ffmpeg/encode.rs | 20 +- crates/coreav-cli/src/ffmpeg/mod.rs | 48 +++- crates/coreav-cli/src/ffmpeg/native_h264.rs | 205 ++++++++++++++++ crates/coreav-cli/src/main.rs | 20 ++ crates/coreav-cli/src/transcode.rs | 6 + crates/coreav-cli/tests/native_h264_cli.rs | 62 +++++ crates/coreav-transcode/Cargo.toml | 2 + crates/coreav-transcode/src/lib.rs | 2 + crates/coreav-transcode/src/native_h264.rs | 240 +++++++++++++++++++ crates/coreav-transcode/src/pipeline.rs | 252 +++++++++++++++++--- docs-site/commands/ffmpeg.mdx | 15 ++ docs/design/cli-native-h264.md | 8 +- docs/design/transcode.md | 2 + scripts/native_h264_cli.py | 233 ++++++++++++++++++ scripts/tests/test_native_h264_cli.py | 41 ++++ 20 files changed, 1268 insertions(+), 38 deletions(-) create mode 100644 crates/coreav-cli/src/ffmpeg/native_h264.rs create mode 100644 crates/coreav-cli/tests/native_h264_cli.rs create mode 100644 crates/coreav-transcode/src/native_h264.rs create mode 100644 scripts/native_h264_cli.py create mode 100644 scripts/tests/test_native_h264_cli.py diff --git a/.github/workflows/h264-encoder.yml b/.github/workflows/h264-encoder.yml index 05900f7c8..3be2985d2 100644 --- a/.github/workflows/h264-encoder.yml +++ b/.github/workflows/h264-encoder.yml @@ -7,6 +7,10 @@ on: branches: [main] paths: - crates/coreav-h264-encoder/** + - crates/coreav-cli/** + - crates/coreav-transcode/** + - crates/coreav-frame/** + - scripts/native_h264_cli.py - crates/coreav-kernels-h264/** - crates/coreav-h264/** - scripts/h264_g1* @@ -30,6 +34,10 @@ on: types: [opened, synchronize, reopened, ready_for_review] paths: - crates/coreav-h264-encoder/** + - crates/coreav-cli/** + - crates/coreav-transcode/** + - crates/coreav-frame/** + - scripts/native_h264_cli.py - crates/coreav-kernels-h264/** - crates/coreav-h264/** - scripts/h264_g1* @@ -59,6 +67,83 @@ concurrency: cancel-in-progress: ${{ github.event_name == 'pull_request' }} jobs: + native-h264-cli: + runs-on: ubuntu-24.04 + timeout-minutes: 30 + defaults: + run: + working-directory: candidate + steps: + - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 + with: + path: candidate + persist-credentials: false + - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 + with: + path: baseline + ref: ${{ github.event.pull_request.base.sha || github.sha }} + persist-credentials: false + - uses: dtolnay/rust-toolchain@686976e191b89faba57d3206551f0f330d8cb249 # stable + with: + toolchain: stable + components: clippy + - name: Preserve base-build byte references + run: | + cd ../baseline + cargo build --locked --release -p coreav-cli + cp target/release/coreav "$RUNNER_TEMP/default-base" + cargo build --locked --release -p coreav-cli --features native-h264 + cp target/release/coreav "$RUNNER_TEMP/external-base" + - name: Keep native encoding out of default and release builds + run: | + set -euo pipefail + for features in '' release-default release-full; do + cargo tree --locked -p coreav-cli --features "$features" -e normal > "$RUNNER_TEMP/native-tree" + if grep -q coreav-h264-encoder "$RUNNER_TEMP/native-tree"; then + echo 'native H.264 encoder entered a default or release feature set' >&2 + exit 1 + fi + done + cargo build --locked --release -p coreav-cli + cp target/release/coreav "$RUNNER_TEMP/default-candidate" + - name: Check opt-in adapters and existing CLI limits + run: | + python3 -I -m unittest discover -s scripts/tests -p 'test_native_h264_cli.py' + cargo test --locked --release -p coreav-cli --features native-h264-encoder,native-h264 --lib ffmpeg + cargo test --locked --release -p coreav-cli --features native-h264-encoder,native-h264 --test docs_site --test strict_safe_limit --test ffprobe_limits --test native_h264_cli + cargo test --locked --release -p coreav-transcode --features h264-native --lib + cargo clippy --locked -p coreav-cli --features native-h264-encoder,native-h264 --all-targets -- -D warnings + cargo build --locked --release -p coreav-cli --features native-h264-encoder,native-h264 + cargo build --locked --release -p coreav-h264-encoder --features full --example p + - name: Install pinned black-box tools and generate trusted fixtures + run: | + scripts/install-oracle-ffmpeg.sh "$RUNNER_TEMP/cli-oracles" + scripts/install-x264.sh "$RUNNER_TEMP/cli-oracles" + export PATH="$RUNNER_TEMP/cli-oracles:$PATH" + scripts/gen-fixtures.sh "$RUNNER_TEMP/cli-fixtures" + - name: Verify stored native output and unchanged external behavior + run: | + python3 -I scripts/native_h264_cli.py \ + --candidate target/release/coreav \ + --default-candidate "$RUNNER_TEMP/default-candidate" \ + --baseline-default "$RUNNER_TEMP/default-base" \ + --baseline-external "$RUNNER_TEMP/external-base" \ + --driver target/release/examples/p \ + --ffmpeg "$RUNNER_TEMP/cli-oracles/ffmpeg" \ + --output "$RUNNER_TEMP/native-cli" + - name: Retain encoder timeout and staging regressions + env: + COREAV_FFMPEG: ${{ runner.temp }}/cli-oracles/ffmpeg + COREAV_X264: ${{ runner.temp }}/cli-oracles/x264 + COREAV_FIXTURES: ${{ runner.temp }}/cli-fixtures + COREAV_REQUIRE_FFMPEG: '1' + run: cargo test --locked --release -p coreav-cli --features native-h264-encoder,native-h264 --test transcode hostile_encoders_end_in_an_error_and_leave_nothing -- --exact + - uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2 + if: always() + with: + name: native-h264-cli + path: ${{ runner.temp }}/native-cli/summary.json + if-no-files-found: ignore h264-encoder-conformance: runs-on: ubuntu-24.04 timeout-minutes: 30 diff --git a/Cargo.lock b/Cargo.lock index f718dbe05..113940231 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -574,6 +574,7 @@ dependencies = [ "coreav-decode", "coreav-ffv1", "coreav-frame", + "coreav-h264-encoder", "coreav-isobmff", "coreav-isobmff-write", "coreav-mkv", diff --git a/crates/coreav-cli/Cargo.toml b/crates/coreav-cli/Cargo.toml index de8b817cf..4cc65c4ae 100644 --- a/crates/coreav-cli/Cargo.toml +++ b/crates/coreav-cli/Cargo.toml @@ -40,6 +40,7 @@ ffmpeg-fallback = ["thumbnail", "transcode", "dep:coreav-worker", "coreav-worker # in this process. Opt-in and never in a default build: decoding H.264 may # need a patent licence where you are (docs/design/h264.md, "Packaging"). native-h264 = ["thumbnail", "transcode", "dep:coreav-backend-h264"] +native-h264-encoder = ["transcode", "coreav-transcode/h264-native"] # Experimental native profile-0 VP9, not in a default or release build. native-vp9 = ["thumbnail", "transcode", "dep:coreav-backend-vp9"] native-hevc = ["thumbnail", "transcode", "dep:coreav-backend-hevc", "coreav-backend-hevc/native-hevc"] diff --git a/crates/coreav-cli/src/compat_cmd.rs b/crates/coreav-cli/src/compat_cmd.rs index b8a0652bb..80912a71f 100644 --- a/crates/coreav-cli/src/compat_cmd.rs +++ b/crates/coreav-cli/src/compat_cmd.rs @@ -165,6 +165,37 @@ pub fn run_ffmpeg(args: &[OsString], strict_safe: StrictSafe) -> ExitCode { } } +#[cfg(feature = "native-h264-encoder")] +pub fn run_ffmpeg_native(args: &[OsString], strict_safe: StrictSafe) -> ExitCode { + let text = match strings("ffmpeg", args) { + Ok(text) => text, + Err(code) => return code, + }; + match ffmpeg::native_h264::run(&text, strict_safe) { + Ok(Some(_)) => ExitCode::SUCCESS, + Ok(None) => run_ffmpeg(args, strict_safe), + Err(Stop::Failed(failure)) => { + let json = failure.to_json(); + let text = json + .get("message") + .and_then(serde_json::Value::as_str) + .map_or_else(|| json.to_string(), str::to_owned); + message("ffmpeg", &text); + ExitCode::from(if failure.is_io() { + EXIT_IO + } else { + EXIT_REJECTED + }) + } + Err(Stop::Refused(Refusal::Usage(text))) => { + message("ffmpeg", &text); + ExitCode::from(EXIT_USAGE) + } + Err(Stop::Refused(Refusal::Help)) => print_usage(ffmpeg::USAGE), + Err(Stop::Refused(Refusal::NotNative(_))) => run_ffmpeg(args, strict_safe), + } +} + pub(super) struct EncodedInput { path: std::path::PathBuf, file: Option, diff --git a/crates/coreav-cli/src/ffmpeg/args.rs b/crates/coreav-cli/src/ffmpeg/args.rs index 76d4b77d4..6fe0bc174 100644 --- a/crates/coreav-cli/src/ffmpeg/args.rs +++ b/crates/coreav-cli/src/ffmpeg/args.rs @@ -683,6 +683,10 @@ struct Grouped<'a> { } fn grouped(args: &[String]) -> Result, Refusal> { + grouped_with_qp(args, false) +} + +fn grouped_with_qp(args: &[String], native_qp: bool) -> Result, Refusal> { let mut inputs: Vec<(&str, Group)> = Vec::new(); let mut outputs: Vec<(&str, Group)> = Vec::new(); let (mut yes, mut no) = (false, false); @@ -711,7 +715,10 @@ fn grouped(args: &[String]) -> Result, Refusal> { .chain(PER_FILE.iter()) .find(|(name, _)| *name == arg) .map(|(_, arity)| *arity) - .or_else(|| (arg == "-i" || codec_option(arg).is_some()).then_some(Arity::Value)); + .or_else(|| { + (arg == "-i" || codec_option(arg).is_some() || native_qp && arg == "-qp") + .then_some(Arity::Value) + }); let Some(arity) = arity else { let base = arg.split(':').next().unwrap_or(arg); let why = REFUSED.iter().find(|(name, _)| *name == base).map_or( @@ -807,13 +814,34 @@ pub fn summary(args: &[String]) -> Result Result { + parse_grouped(grouped(args)?) +} + +#[cfg(feature = "native-h264-encoder")] +pub(super) fn native_h264(args: &[String]) -> Result<(Command, Option), Refusal> { + let mut grouped = grouped_with_qp(args, true)?; + let mut qp = None; + if let [(_, options)] = grouped.outputs.as_mut_slice() { + options.retain(|(option, value)| { + if *option == "-qp" { + qp = value.map(str::to_owned); + false + } else { + true + } + }); + } + Ok((parse_grouped(grouped)?, qp)) +} + +fn parse_grouped(grouped: Grouped<'_>) -> Result { let Grouped { inputs, outputs, yes, no, .. - } = grouped(args)?; + } = grouped; let overwrite = match (yes, no) { (true, true) => return Err(usage("Error, both -y and -n supplied. Exiting.")), (true, false) => Overwrite::Yes, diff --git a/crates/coreav-cli/src/ffmpeg/encode.rs b/crates/coreav-cli/src/ffmpeg/encode.rs index 330b8ab32..c11e0e6d0 100644 --- a/crates/coreav-cli/src/ffmpeg/encode.rs +++ b/crates/coreav-cli/src/ffmpeg/encode.rs @@ -301,6 +301,8 @@ pub fn request( } }; Ok(Request { + #[cfg(feature = "native-h264-encoder")] + native_h264: None, matroska_output: command.muxer.matroska(), crf, bitrate, @@ -573,7 +575,15 @@ pub fn write( encoder: Encoder, strict_safe: StrictSafe, ) -> Result { - write_inner(command, job, encoder, strict_safe, None) + write_inner( + command, + job, + encoder, + strict_safe, + None, + #[cfg(feature = "native-h264-encoder")] + None, + ) } pub(super) fn write_inner( @@ -582,8 +592,16 @@ pub(super) fn write_inner( encoder: Encoder, strict_safe: StrictSafe, capture: Option<&Path>, + #[cfg(feature = "native-h264-encoder")] native: Option< + coreav_transcode::native_h264::NativeH264, + >, ) -> Result { let request = request(command, job, encoder, strict_safe)?; + #[cfg(feature = "native-h264-encoder")] + let request = Request { + native_h264: native, + ..request + }; #[cfg(feature = "input-capture")] let request = { let mut request = request; diff --git a/crates/coreav-cli/src/ffmpeg/mod.rs b/crates/coreav-cli/src/ffmpeg/mod.rs index 365c81bf2..286cab075 100644 --- a/crates/coreav-cli/src/ffmpeg/mod.rs +++ b/crates/coreav-cli/src/ffmpeg/mod.rs @@ -22,6 +22,8 @@ mod ffv1; mod frames; #[cfg(feature = "input-capture")] mod image_capture; +#[cfg(feature = "native-h264-encoder")] +pub mod native_h264; mod pcm; mod raw_audio; mod ts; @@ -372,7 +374,29 @@ fn run_with_input( image_capture: Option<&Path>, spooled: Option<&Path>, ) -> Result { - let mut command = args::parse(args)?; + let command = args::parse(args)?; + run_command( + command, + strict_safe, + capture, + image_capture, + spooled, + #[cfg(feature = "native-h264-encoder")] + None, + ) +} + +#[cfg_attr(not(feature = "transcode"), allow(unused_variables))] +fn run_command( + mut command: Command, + strict_safe: StrictSafe, + capture: Option<&Path>, + image_capture: Option<&Path>, + spooled: Option<&Path>, + #[cfg(feature = "native-h264-encoder")] native: Option< + coreav_transcode::native_h264::NativeH264, + >, +) -> Result { if let Some(input) = spooled { command.input = input.to_string_lossy().into_owned(); } @@ -441,6 +465,10 @@ fn run_with_input( return write(&command, &job); } let job = plan(&command, &report, list)?; + #[cfg(feature = "native-h264-encoder")] + if native.is_some() && !matches!(job, Job::Transcode(_)) { + return Err(Refusal::not_native("native H.264 requires video encoding").into()); + } if image_capture.is_some() && !matches!( job, @@ -473,11 +501,27 @@ fn run_with_input( } #[cfg(feature = "transcode")] if let Job::Transcode(transcode) = &job { + #[cfg(feature = "native-h264-encoder")] + if native.is_some() && !native_h264::source(&report, transcode.video) { + return Err(Refusal::not_native("the source is outside native H.264 limits").into()); + } if capture.is_some() && transcode.audio_encoder != Some(args::AudioEncoder::Aac) { return Err(Refusal::not_native("input capture without a native AAC encode").into()); } let encoder = encode::encoder(transcode, strict_safe)?; - return encode::write_inner(&command, transcode, encoder, strict_safe, capture); + return encode::write_inner( + &command, + transcode, + encoder, + strict_safe, + capture, + #[cfg(feature = "native-h264-encoder")] + native, + ); + } + #[cfg(feature = "native-h264-encoder")] + if native.is_some() { + return Err(Refusal::not_native("native H.264 requires video encoding").into()); } if capture.is_some() { return Err(Refusal::not_native("input capture without a native AAC encode").into()); diff --git a/crates/coreav-cli/src/ffmpeg/native_h264.rs b/crates/coreav-cli/src/ffmpeg/native_h264.rs new file mode 100644 index 000000000..fcb044f5b --- /dev/null +++ b/crates/coreav-cli/src/ffmpeg/native_h264.rs @@ -0,0 +1,205 @@ +//! Explicit native encoding preference with unchanged external fallback arguments. + +use coreav_transcode::native_h264::{NativeH264, RateMode}; + +use super::args::{Codec, Command, Muxer, Spec}; +use super::{Refusal, Stop}; +use crate::ffprobe::{Report, args::Kind2, model::Value}; +use crate::strict_safe::StrictSafe; + +fn candidate(args: &[String]) -> Option<(Command, NativeH264)> { + let (command, qp) = super::args::native_h264(args).ok()?; + let encoding = &command.encoding; + if !command.more_inputs.is_empty() + || command.muxer != Muxer::Mp4 + || !command.codecs.iter().any(|(spec, codec)| { + *codec == Codec::Libx264 && matches!(spec, Spec::Kind(Kind2::Video, None | Some(0))) + }) + || encoding.bitrate.is_some() + || encoding.preset.is_some() + || encoding.filter.is_some() + || encoding.size.is_some() + || encoding + .pixel_format + .as_deref() + .is_some_and(|s| s != "yuv420p") + { + return None; + } + let quality = |text: &str| text.parse::().ok().filter(|q| (10..=51).contains(q)); + let rate = match (encoding.crf.as_deref(), qp.as_deref()) { + (None, None) => RateMode::Crf(23), + (Some(text), None) => RateMode::Crf(quality(text)?), + (None, Some(text)) => RateMode::FixedQp(quality(text)?), + (Some(_), Some(_)) => return None, + }; + Some((command, NativeH264 { rate })) +} + +pub(super) fn source(report: &Report, index: usize) -> bool { + let Some(stream) = report.streams.get(index) else { + return false; + }; + let field = |name| { + stream + .entries + .iter() + .find_map(|(k, v)| (*k == name).then_some(v)) + }; + let dimension = |name, max| matches!(field(name), Some(Value::Int(n)) if (16..=max).contains(n) && n % 2 == 0); + matches!(field("field_order"), Some(Value::Str(s)) if s == "progressive") + && matches!(field("pix_fmt"), Some(Value::Str(s)) if s == "yuv420p") + && dimension("width", 1920) + && dimension("height", 1080) + && stream + .r_frame_rate() + .is_some_and(|(n, d)| d.checked_mul(30) == Some(n)) +} + +/// Tries the opt-in encoder without changing the command used on fallback. +pub fn run(args: &[String], strict_safe: StrictSafe) -> Result, Stop> { + if strict_safe.on() { + return Ok(None); + } + let Some((command, native)) = candidate(args) else { + return Ok(None); + }; + match super::run_command(command, strict_safe, None, None, None, Some(native)) { + Ok(value) => Ok(Some(value)), + Err(Stop::Refused(Refusal::NotNative(_))) => Ok(None), + Err(error) => Err(error), + } +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn source_admission_requires_known_progressive_geometry_and_clock() { + let mut report = Report::default(); + report.streams.push(crate::ffprobe::Stream { + kind: crate::ffprobe::StreamKind::Video, + attached_pic: false, + entries: vec![ + ("width", Value::Int(1920)), + ("height", Value::Int(1080)), + ("field_order", Value::Str("progressive".into())), + ("pix_fmt", Value::Str("yuv420p".into())), + ("r_frame_rate", Value::Str("30/1".into())), + ], + disposition: Vec::new(), + tags: Vec::new(), + side_data: Vec::new(), + }); + assert!(source(&report, 0)); + assert!(!source(&report, 1)); + for (name, value) in [ + ("width", Value::Int(1922)), + ("height", Value::Int(1082)), + ("width", Value::Int(14)), + ("width", Value::Int(17)), + ("field_order", Value::Str("tt".into())), + ("field_order", Value::Na), + ("pix_fmt", Value::Str("yuv420p10le".into())), + ("pix_fmt", Value::Str("yuv444p".into())), + ("r_frame_rate", Value::Str("30000/1001".into())), + ("r_frame_rate", Value::Str("3002/100".into())), + ] { + let mut changed = report.clone(); + if let Some(stream) = changed.streams.first_mut() { + for (key, field) in &mut stream.entries { + if *key == name { + *field = value.clone(); + } + } + } + assert!(!source(&changed, 0), "{name}: {value:?}"); + } + } + + fn args(extra: &str) -> Vec { + format!("-i in.mp4 -an -c:v libx264 {extra} out.mp4") + .split_whitespace() + .map(str::to_owned) + .collect() + } + + #[test] + fn integer_quality_modes_map_without_rounding() { + for q in 10..=51 { + assert_eq!( + candidate(&args(&format!("-crf {q}"))).map(|(_, n)| n.rate), + Some(RateMode::Crf(q)) + ); + assert_eq!( + candidate(&args(&format!("-qp {q}"))).map(|(_, n)| n.rate), + Some(RateMode::FixedQp(q)) + ); + } + assert_eq!( + candidate(&args("")).map(|(_, n)| n.rate), + Some(RateMode::Crf(23)) + ); + } + + #[test] + fn unsupported_controls_keep_the_existing_command() { + for option in [ + "-profile:v high", + "-profile:v baseline", + "-bf 1", + "-b:v 1M", + "-maxrate 1M", + "-bufsize 2M", + "-x264-params cabac=1", + "-x264opts bframes=1", + "-tune film", + "-g 12", + "-level 4.1", + "-r 25", + "-preset medium", + "-pix_fmt yuv420p10le", + "-pix_fmt yuv444p", + "-crf 9", + "-crf 52", + "-crf 23.5", + "-qp 0", + "-qp 52", + "-qp 23.5", + "-crf 23 -qp 24", + "-vf scale=32:32", + "-s 32x32", + "-pass 1", + "-b_strategy 1", + "-refs 2", + "-flags +ildct", + "-force_key_frames 0", + ] { + let original = args(option); + let before = original.clone(); + assert!(candidate(&original).is_none(), "{option}"); + assert_eq!(original, before); + } + } + + #[test] + fn qp_tokens_belong_to_their_option_or_filename() { + for line in [ + "-i in.mp4 -c:v -qp out.mp4", + "-qp 23 -i in.mp4 -an -c:v libx264 out.mp4", + "-i in.mp4 -an -c:v libx264 out.mp4 -qp 23", + ] { + let args: Vec = line.split_whitespace().map(str::to_owned).collect(); + assert!(candidate(&args).is_none(), "{line}"); + } + let named: Vec = "-i -qp -an -c:v libx264 out.mp4" + .split_whitespace() + .map(str::to_owned) + .collect(); + assert!(candidate(&named).is_some_and( + |(command, native)| command.input == "-qp" && native.rate == RateMode::Crf(23) + )); + assert!(super::super::args::parse(&args("-qp 23")).is_err()); + } +} diff --git a/crates/coreav-cli/src/main.rs b/crates/coreav-cli/src/main.rs index 7542a0529..9b18c1f34 100644 --- a/crates/coreav-cli/src/main.rs +++ b/crates/coreav-cli/src/main.rs @@ -107,6 +107,11 @@ struct Cli { /// refused (exit status 4). #[arg(long)] strict_safe: bool, + /// Prefer native Constrained Baseline H.264 for eligible ffmpeg commands; + /// other commands keep the external encoder. Requires a decoder backend. + #[cfg(feature = "native-h264-encoder")] + #[arg(long)] + native_h264_encoder: bool, #[cfg(feature = "input-capture")] #[arg(long, hide = true)] capture_encoder_input: Option, @@ -1014,6 +1019,19 @@ impl Write for Watched { /// Runs the command `cli` describes. #[cfg_attr(not(feature = "transcode"), allow(unused_variables))] fn run(cli: Cli, this: Option, host: CommandHost) -> ExitCode { + #[cfg(feature = "native-h264-encoder")] + if cli.native_h264_encoder { + #[cfg(feature = "input-capture")] + if cli.capture_encoder_input.is_some() { + return usage("native H.264 selection cannot capture an external encoder's input"); + } + return match &cli.command { + Command::Ffmpeg(args) => { + compat_cmd::run_ffmpeg_native(&args.args, StrictSafe::new(cli.strict_safe)) + } + _ => usage("--native-h264-encoder requires the ffmpeg command"), + }; + } #[cfg(feature = "input-capture")] if let Some(target) = cli.capture_encoder_input.as_ref() { return match &cli.command { @@ -1161,6 +1179,8 @@ fn run_transcode( } } let request = transcode::Request { + #[cfg(feature = "native-h264-encoder")] + native_h264: None, matroska_output: None, crf: args.crf, bitrate: args.bitrate, diff --git a/crates/coreav-cli/src/transcode.rs b/crates/coreav-cli/src/transcode.rs index e02dc592d..e197fb19a 100644 --- a/crates/coreav-cli/src/transcode.rs +++ b/crates/coreav-cli/src/transcode.rs @@ -76,6 +76,9 @@ fn diagnostic_backends(host: Option) -> Vec, /// Matroska output profile, or the default MP4 output. pub matroska_output: Option, /// `--crf`. @@ -182,6 +185,8 @@ pub fn make( }; let (frame_rates, timeout) = source_rates(&mut source, Duration::from_secs(request.timeout))?; let options = TranscodeOptions { + #[cfg(feature = "native-h264-encoder")] + native_h264: request.native_h264, matroska_output: request.matroska_output, rate: match (request.bitrate, request.crf) { (Some(kbps), _) => RateControl::Bitrate(kbps), @@ -499,6 +504,7 @@ fn document_common( pub(crate) fn failure(error: coreav_transcode::Error) -> WriteFailure { use coreav_transcode::Error as E; match error { + E::NativeIneligible(what) => WriteFailure::Rejected(Failure::Unsupported(what.into())), E::Isobmff(e) => WriteFailure::Rejected(Failure::Library(e)), E::Matroska(e) => WriteFailure::Rejected(Failure::Matroska(e)), E::Invalid(what) => WriteFailure::Rejected(Failure::Invalid(what)), diff --git a/crates/coreav-cli/tests/native_h264_cli.rs b/crates/coreav-cli/tests/native_h264_cli.rs new file mode 100644 index 000000000..6f5516064 --- /dev/null +++ b/crates/coreav-cli/tests/native_h264_cli.rs @@ -0,0 +1,62 @@ +//! Outer selector, unchanged help, and strict-safe behavior without an oracle. +#![cfg(feature = "native-h264-encoder")] +#![allow(clippy::unwrap_used)] + +use std::process::Command; + +fn run(args: &[&str]) -> std::process::Output { + Command::new(env!("CARGO_BIN_EXE_coreav")) + .args(args) + .env_remove("COREAV_STRICT_SAFE") + .env("COREAV_X264", "/nonexistent/native-h264-x264") + .env("COREAV_FFMPEG", "/nonexistent/native-h264-ffmpeg") + .env("COREAV_FALLBACK", "/nonexistent/native-h264-fallback") + .output() + .unwrap() +} + +#[test] +fn strict_safe_selection_keeps_the_existing_refusal_bytes() { + for extra in [["-crf", "23"], ["-qp", "23"], ["-bf", "2"]] { + let args = [ + "--strict-safe", + "ffmpeg", + "-i", + "/nonexistent/native-input.mp4", + "-an", + "-c:v", + "libx264", + extra[0], + extra[1], + "/nonexistent/native-output.mp4", + ]; + let expected = run(&args); + let mut native = vec!["--native-h264-encoder"]; + native.extend_from_slice(&args); + let actual = run(&native); + assert_eq!(actual.status.code(), expected.status.code()); + assert_eq!(actual.stdout, expected.stdout); + assert_eq!(actual.stderr, expected.stderr); + } +} + +#[test] +fn selection_is_outer_and_requires_the_ffmpeg_command() { + let output = run(&["--native-h264-encoder", "ffprobe", "--help"]); + assert_eq!(output.status.code(), Some(2)); + assert!( + String::from_utf8(output.stderr) + .unwrap() + .contains("--native-h264-encoder requires the ffmpeg command") + ); + assert!(output.stdout.is_empty()); +} + +#[test] +fn the_ffmpeg_help_bytes_do_not_depend_on_selection() { + let expected = run(&["ffmpeg", "-h"]); + let actual = run(&["--native-h264-encoder", "ffmpeg", "-h"]); + assert_eq!(actual.status.code(), Some(0)); + assert_eq!(actual.stdout, expected.stdout); + assert_eq!(actual.stderr, expected.stderr); +} diff --git a/crates/coreav-transcode/Cargo.toml b/crates/coreav-transcode/Cargo.toml index df1050c28..54aade4a3 100644 --- a/crates/coreav-transcode/Cargo.toml +++ b/crates/coreav-transcode/Cargo.toml @@ -9,6 +9,7 @@ repository.workspace = true publish = false [dependencies] +coreav-h264-encoder = { path = "../coreav-h264-encoder", optional = true, features = ["full"] } coreav-ffv1 = { path = "../coreav-ffv1" } coreav-audio = { path = "../coreav-audio" } coreav-decode = { path = "../coreav-decode" } @@ -23,6 +24,7 @@ sha2 = { version = "0.10", optional = true } serde = { workspace = true, optional = true } [features] +h264-native = ["dep:coreav-h264-encoder"] # AAC LC decoding in `coreav-audio`, including sources of audio encodes. aac = ["coreav-audio/aac"] input-capture = ["dep:sha2", "dep:serde"] diff --git a/crates/coreav-transcode/src/lib.rs b/crates/coreav-transcode/src/lib.rs index 8d7468f28..a5f02f297 100644 --- a/crates/coreav-transcode/src/lib.rs +++ b/crates/coreav-transcode/src/lib.rs @@ -34,6 +34,8 @@ pub mod contract; pub mod encoder; pub mod ffv1; pub mod mp3; +#[cfg(feature = "h264-native")] +pub mod native_h264; pub mod pipeline; pub mod rawmkv; pub mod scale; diff --git a/crates/coreav-transcode/src/native_h264.rs b/crates/coreav-transcode/src/native_h264.rs new file mode 100644 index 000000000..ff12da740 --- /dev/null +++ b/crates/coreav-transcode/src/native_h264.rs @@ -0,0 +1,240 @@ +//! Bounded native H.264 access units for the existing transcode spool and muxers. + +use coreav_frame::{ChromaLocation, ColorInfo, PixelFormat, Pixels}; +use coreav_h264_encoder::{Colour, Deblocking, Gop, Limits, Plane, Settings}; + +use crate::Error; +use crate::encoder::{EncoderSettings, Sink}; + +pub use coreav_h264_encoder::RateMode; + +/// Explicit preference for the opt-in native encoder. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub struct NativeH264 { + /// Integer CRF or fixed QP, from 10 through 51. + pub rate: RateMode, +} + +pub(crate) fn color(color: ColorInfo) -> ColorInfo { + ColorInfo { + primaries: Some(color.primaries.unwrap_or(2)), + transfer: Some(color.transfer.unwrap_or(2)), + matrix: Some(color.matrix.unwrap_or(2)), + full_range: Some(color.full_range.unwrap_or(false)), + chroma_location: Some(ChromaLocation::Left), + } +} + +pub(crate) struct Encoder { + encoder: coreav_h264_encoder::Encoder, + sink: S, + frames: u64, + size: (u32, u32), +} + +impl Encoder { + pub(crate) fn new( + settings: &EncoderSettings, + request: NativeH264, + max_bytes: u64, + sink: S, + ) -> Result { + if settings.format != PixelFormat::YUV420P + || settings.sample_aspect.is_some_and(|(n, d)| n != d) + { + return Err(Error::NativeIneligible( + "native H.264 needs square 8-bit 4:2:0 pixels", + )); + } + let c = settings.color; + let settings_native = Settings { + width: settings.width as usize, + height: settings.height as usize, + rate_mode: request.rate, + colour: Colour { + full_range: c.full_range.unwrap_or(false), + primaries: c.primaries.unwrap_or(2), + transfer: c.transfer.unwrap_or(2), + matrix: c.matrix.unwrap_or(2), + chroma_location: 0, + }, + deblocking: Deblocking::Enabled, + gop: Gop::Fixed { idr_interval: 250 }, + }; + let limits = Limits { + frames: u64::MAX, + total_bytes: max_bytes, + ..Limits::default() + }; + let encoder = match settings.threads.filter(|n| *n > 0) { + Some(n) => { + coreav_h264_encoder::Encoder::with_threads(settings_native, limits, n as usize) + } + None => coreav_h264_encoder::Encoder::new(settings_native, limits), + } + .map_err(ineligible)?; + Ok(Self { + encoder, + sink, + frames: 0, + size: (settings.width, settings.height), + }) + } + + pub(crate) fn send_frame(&mut self, pixels: Pixels) -> Result<(), Error> { + let picture = pixels + .into_cpu() + .map_err(|_| Error::NativeIneligible("native H.264 needs CPU planes"))?; + if picture.format() != PixelFormat::YUV420P + || (picture.width(), picture.height()) != self.size + { + return Err(Error::NativeIneligible( + "native H.264 needs fixed-size 4:2:0 pictures", + )); + } + let view = |index: usize, width, height| { + let plane = picture + .planes() + .get(index) + .ok_or(Error::NativeIneligible("native H.264 needs three planes"))?; + Plane::new(plane.data(), plane.stride(), width, height).map_err(ineligible) + }; + let (width, height) = (picture.width() as usize, picture.height() as usize); + let access = self + .encoder + .encode(coreav_h264_encoder::Picture { + y: view(0, width, height)?, + cb: view(1, width / 2, height / 2)?, + cr: view(2, width / 2, height / 2)?, + }) + .map_err(ineligible)?; + let idr = access + .nals + .iter() + .any(|nal| nal.first().is_some_and(|b| b & 31 == 5)); + for nal in &access.nals { + if !idr && nal.first().is_some_and(|b| matches!(b & 31, 7 | 8)) { + continue; + } + self.sink.write(&[0, 0, 0, 1]).map_err(Error::Encode)?; + self.sink.write(nal).map_err(Error::Encode)?; + } + self.frames += 1; + Ok(()) + } + + pub(crate) fn frames_sent(&self) -> u64 { + self.frames + } + + pub(crate) fn finish(self) -> S { + self.sink + } +} + +fn ineligible(_: coreav_h264_encoder::Error) -> Error { + Error::NativeIneligible("the picture exceeds native H.264 settings or coding limits") +} + +#[cfg(test)] +#[allow(clippy::unwrap_used, clippy::indexing_slicing)] +mod tests { + use super::*; + use crate::encoder::FrameRate; + + #[derive(Default)] + struct Bytes(Vec); + impl Sink for Bytes { + fn write(&mut self, data: &[u8]) -> Result<(), coreav_decode::Error> { + self.0.extend_from_slice(data); + Ok(()) + } + } + + fn picture() -> coreav_frame::Picture { + coreav_frame::Picture::new( + 18, + 34, + PixelFormat::YUV420P, + [(24, 34), (16, 17), (16, 17)] + .into_iter() + .map(|(stride, rows)| coreav_frame::Plane::new(vec![128; stride * rows], stride)) + .collect(), + ) + .unwrap() + } + + fn settings(threads: u32) -> EncoderSettings { + EncoderSettings { + threads: Some(threads), + color: color(ColorInfo::default()), + ..EncoderSettings::new(18, 34, PixelFormat::YUV420P, FrameRate { num: 30, den: 1 }) + } + } + + #[test] + fn padded_planes_and_every_quality_mode_are_thread_independent() { + for rate in [ + RateMode::Crf(10), + RateMode::Crf(23), + RateMode::Crf(51), + RateMode::FixedQp(10), + RateMode::FixedQp(51), + ] { + let mut expected = None; + for threads in [0, 1, 2, 8, 128] { + let mut encoder = Encoder::new( + &settings(threads), + NativeH264 { rate }, + 1_000_000, + Bytes::default(), + ) + .unwrap(); + for _ in 0..3 { + encoder.send_frame(Pixels::Cpu(picture())).unwrap(); + } + assert_eq!(encoder.frames_sent(), 3); + let bytes = encoder.finish().0; + assert!(!bytes.is_empty()); + if let Some(expected) = &expected { + assert_eq!(&bytes, expected); + } else { + expected = Some(bytes); + } + } + } + } + + #[test] + fn library_limits_are_retryable_without_counting_a_failed_picture() { + let request = NativeH264 { + rate: RateMode::Crf(23), + }; + let mut encoder = Encoder::new(&settings(1), request, 1, Bytes::default()).unwrap(); + assert!(matches!( + encoder.send_frame(Pixels::Cpu(picture())), + Err(Error::NativeIneligible(_)) + )); + assert_eq!(encoder.frames_sent(), 0); + assert!(encoder.finish().0.is_empty()); + for (width, height) in [(14, 16), (16, 14), (1922, 16), (16, 1082), (17, 16)] { + let settings = EncoderSettings { + width, + height, + ..settings(1) + }; + assert!(matches!( + Encoder::new(&settings, request, 1_000_000, Bytes::default()), + Err(Error::NativeIneligible(_)) + )); + } + let settings = EncoderSettings { + sample_aspect: Some((4, 3)), + ..settings(1) + }; + assert!(matches!( + Encoder::new(&settings, request, 1_000_000, Bytes::default()), + Err(Error::NativeIneligible(_)) + )); + } +} diff --git a/crates/coreav-transcode/src/pipeline.rs b/crates/coreav-transcode/src/pipeline.rs index 8bfdca05d..b7db60ece 100644 --- a/crates/coreav-transcode/src/pipeline.rs +++ b/crates/coreav-transcode/src/pipeline.rs @@ -117,6 +117,9 @@ pub enum SourceKind { /// What [`transcode`] does. #[derive(Clone, Debug)] pub struct TranscodeOptions { + /// Prefer native H.264 only within its supported envelope. + #[cfg(feature = "h264-native")] + pub native_h264: Option, /// Matroska/WebM output instead of the default MP4 output. pub matroska_output: Option, /// The encoder's rate control: CRF 23 by default. @@ -186,6 +189,8 @@ pub struct TranscodeOptions { impl Default for TranscodeOptions { fn default() -> Self { Self { + #[cfg(feature = "h264-native")] + native_h264: None, matroska_output: None, rate: RateControl::Crf(23.0), preset: "medium".into(), @@ -274,6 +279,8 @@ pub struct Summary { /// removed, and the caller discards what was written to its output. #[derive(Debug)] pub enum Error { + /// Native encoding cannot preserve the requested properties; no output is published. + NativeIneligible(&'static str), /// The MP4/MOV parser rejected the source or could not read it. Isobmff(coreav_isobmff::Error), /// The Matroska parser rejected the source or could not read it. @@ -308,6 +315,7 @@ pub enum Error { impl fmt::Display for Error { fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { match self { + Self::NativeIneligible(what) => write!(f, "native H.264 is ineligible: {what}"), Self::Isobmff(e) => write!(f, "{e}"), Self::Matroska(e) => write!(f, "{e}"), Self::Invalid(what) => write!(f, "invalid structure: {what}"), @@ -405,12 +413,28 @@ enum PreparedVideo { H264 { presented: Presented, frame_rate: FrameRate, - program: EncoderProgram, + program: H264Program, }, #[cfg(feature = "svt-av1")] Av1(av1_input::Plan), } +enum H264Program { + External(EncoderProgram), + #[cfg(feature = "h264-native")] + Native(crate::native_h264::NativeH264), +} + +impl H264Program { + fn name(&self) -> &'static str { + match self { + Self::External(program) => program.name(), + #[cfg(feature = "h264-native")] + Self::Native(_) => "coreav-h264", + } + } +} + struct CompletedJob { video: CompletedVideo, stream_index: usize, @@ -507,9 +531,7 @@ fn execute( VideoMode::H264 => { let presented = Presented::of(&video)?; let frame_rate = presented.frame_rate(video.time_base); - let program = find_encoder(options.encoder, options.x264.clone(), options.ffmpeg.clone()) - .ok_or_else(|| { - unsupported(match options.encoder { + let missing = match options.encoder { EncoderChoice::X264 => "no x264 program: install one, or name it with --x264", EncoderChoice::Ffmpeg => { "no ffmpeg program: install one with libx264, or name it with --ffmpeg" @@ -517,8 +539,29 @@ fn execute( EncoderChoice::Auto => { "no encoder program: install x264 or ffmpeg with libx264, or name one with --x264 or --ffmpeg" } - }) - })?; + }; + let external = || { + find_encoder( + options.encoder, + options.x264.clone(), + options.ffmpeg.clone(), + ) + .map(H264Program::External) + .ok_or_else(|| unsupported(missing)) + }; + #[cfg(feature = "h264-native")] + let program = match options.native_h264 { + Some(native) => { + if options.matroska_output.is_some() { + return Err(Error::NativeIneligible("native H.264 requires MP4 output")); + } + native_timing(&presented, &video)?; + H264Program::Native(native) + } + None => external()?, + }; + #[cfg(not(feature = "h264-native"))] + let program = external()?; PreparedVideo::H264 { presented, frame_rate, @@ -586,7 +629,7 @@ fn execute( // decoder leaves one out (docs/design/transcode.md). let timecodes = match &prepared { PreparedVideo::H264 { - program: EncoderProgram::X264(_), + program: H264Program::External(EncoderProgram::X264(_)), presented, .. } => { @@ -1351,20 +1394,52 @@ impl VideoRun<'_> { } } +enum H264Encoder { + External(Box>), + #[cfg(feature = "h264-native")] + Native(Box>), +} + +impl H264Encoder { + fn frames_sent(&self) -> u64 { + match self { + Self::External(encoder) => encoder.frames_sent(), + #[cfg(feature = "h264-native")] + Self::Native(encoder) => encoder.frames_sent(), + } + } + + fn finish(self) -> Result { + match self { + Self::External(encoder) => encoder.finish().map_err(Error::Encode), + #[cfg(feature = "h264-native")] + Self::Native(encoder) => Ok(encoder.finish()), + } + } + + fn send_frame_at(&mut self, pixels: Pixels, time: i64) -> Result<(), Error> { + match self { + Self::External(encoder) => encoder.send_frame_at(pixels, time).map_err(Error::Encode), + #[cfg(feature = "h264-native")] + Self::Native(encoder) => encoder.send_frame(pixels), + } + } +} + /// The transcode's state while frames come. struct Run<'a> { options: &'a TranscodeOptions, video: &'a VideoStream, presented: Presented, frame_rate: FrameRate, - program: EncoderProgram, + program: H264Program, launcher: Option<&'a ExecLauncher>, deadline: Instant, spool: &'a Spool, /// x264's timecode file. timecodes: Option, /// The encoder, once the first presented frame started it. - encoder: Option<(Encoder, EncoderSettings)>, + encoder: Option<(H264Encoder, EncoderSettings)>, /// The first presented frame's decoded size. first_size: Option<(u32, u32)>, /// Frames sent, for the reader's picture limit. @@ -1392,7 +1467,7 @@ impl Run<'_> { return Err(unsupported("the decoder returned no presented frame")); }; let frames = encoder.frames_sent(); - let sink = encoder.finish().map_err(Error::Encode)?; + let sink = encoder.finish()?; let encoded = sink.close(frames)?; let config = encoded.config; if config.width != settings.width || config.height != settings.height { @@ -1486,33 +1561,74 @@ impl Run<'_> { }, }; let decoded = Frame { pixels, ..frame }; + #[cfg(feature = "h264-native")] + if self.options.native_h264.is_some() { + let color = self.video.stream.color.or(decoded.color); + if decoded.format() != PixelFormat::YUV420P + || !native_size(decoded.width(), decoded.height()) + || color + .chroma_location + .is_some_and(|c| c != coreav_frame::ChromaLocation::Left) + || self.sent_pts.last().is_some_and(|last| { + !native_interval(i128::from(time) - i128::from(*last), self.video.time_base) + }) + { + return Err(Error::NativeIneligible( + "the decoded picture is outside native H.264 limits", + )); + } + } if self.encoder.is_none() { let settings = self.settings(&decoded)?; let sink = SpoolSink::new(self.spool, Arc::clone(&self.sent), self.options)?; - let limits = EncoderLimits::new(encoder_memory( - &settings, - self.options.decode_limits.max_memory_bytes, - host_memory(), - )); - let encoder = Encoder::spawn( - &self.program, - &settings, - self.launcher, - &limits, - self.deadline, - sink, - ) - .map_err(|e| match e.kind { - // A format or size the encoder does not take: refused before - // it starts. - ErrorKind::Unsupported => Error::Unsupported(e.message), - _ => Error::Encode(e), - })?; + let encoder = match &self.program { + H264Program::External(program) => { + let limits = EncoderLimits::new(encoder_memory( + &settings, + self.options.decode_limits.max_memory_bytes, + host_memory(), + )); + H264Encoder::External(Box::new( + Encoder::spawn( + program, + &settings, + self.launcher, + &limits, + self.deadline, + sink, + ) + .map_err(|e| match e.kind { + // A format or size the encoder does not take: refused before + // it starts. + ErrorKind::Unsupported => Error::Unsupported(e.message), + _ => Error::Encode(e), + })?, + )) + } + #[cfg(feature = "h264-native")] + H264Program::Native(request) => { + H264Encoder::Native(Box::new(crate::native_h264::Encoder::new( + &settings, + *request, + self.options.max_spool_bytes, + sink, + )?)) + } + }; self.encoder = Some((encoder, settings)); } let Some((encoder, settings)) = self.encoder.as_mut() else { return Err(encode_error("the encoder is gone".into())); }; + #[cfg(feature = "h264-native")] + if self.options.native_h264.is_some() + && crate::native_h264::color(self.video.stream.color.or(decoded.color)) + != settings.color + { + return Err(Error::NativeIneligible( + "the color description changes within the stream", + )); + } if decoded.format() != settings.format { return Err(unsupported(format!( "the pixel format changed within the stream, from {:?} to {:?}", @@ -1560,7 +1676,12 @@ impl Run<'_> { // Before the frame is queued: the encoder may write its picture as // soon as the writer thread has written the frame. self.sent.fetch_add(1, Ordering::SeqCst); - encoder.send_frame_at(pixels, time).map_err(Error::Encode)?; + let encoded = encoder.send_frame_at(pixels, time); + #[cfg(feature = "h264-native")] + if self.options.native_h264.is_some() && Instant::now() >= self.deadline { + return Err(timed_out()); + } + encoded?; self.sent_pts.push(time); self.last = Some((pts, time)); Ok(()) @@ -1612,6 +1733,12 @@ impl Run<'_> { }; let sample_aspect = sample_aspect(self.video.pixel_aspect, (width, height), scaled_to); let color = output_color(self.video.stream.color.or(frame.color)); + #[cfg(feature = "h264-native")] + let color = if self.options.native_h264.is_some() { + crate::native_h264::color(self.video.stream.color.or(frame.color)) + } else { + color + }; let (num, den) = time_base_terms(self.video.time_base)?; let settings = EncoderSettings { timing: Timing::Timestamps { @@ -1638,7 +1765,7 @@ impl Run<'_> { })?; // FFmpeg reads each frame from one Matroska block, which its reader // caps; x264 takes every size up to 16,384 a side (`MAX_SIZE`). - if let EncoderProgram::Ffmpeg(_) = self.program { + if let H264Program::External(EncoderProgram::Ffmpeg(_)) = self.program { let bytes = settings .frame_bytes() .map_err(|e| Error::Unsupported(e.message))?; @@ -1654,6 +1781,44 @@ impl Run<'_> { } } +#[cfg(feature = "h264-native")] +fn native_size(width: u32, height: u32) -> bool { + (16..=1920).contains(&width) + && (16..=1080).contains(&height) + && width % 2 == 0 + && height % 2 == 0 +} + +#[cfg(feature = "h264-native")] +fn native_interval(ticks: i128, time_base: Rational) -> bool { + ticks > 0 + && ticks + .checked_mul(time_base.num) + .and_then(|n| n.checked_mul(30)) + == Some(i128::from(time_base.den)) +} + +#[cfg(feature = "h264-native")] +fn native_timing(presented: &Presented, video: &VideoStream) -> Result<(), Error> { + let size = video.stream.width.zip(video.stream.height); + if !size.is_some_and(|(w, h)| native_size(w, h)) + || presented.times.len() != video.packets.iter().filter(|p| p.presented).count() + || presented.times.windows(2).any(|w| match w { + [a, b] => !native_interval(i128::from(*b) - i128::from(*a), video.time_base), + _ => true, + }) + || !native_interval( + i128::from(presented.last_duration(&presented.times, Some(presented.last))), + video.time_base, + ) + { + return Err(Error::NativeIneligible( + "native H.264 needs fixed-size constant 30 fps input", + )); + } + Ok(()) +} + /// The colour description the encoder signals: the frames', without the /// chroma location (not passed) and without colour primaries 22, which /// neither program can signal (`docs/design/transcode.md`, "The encoder @@ -1903,6 +2068,31 @@ impl Sink for SpoolSink { #[cfg(test)] mod tests { + #[cfg(feature = "h264-native")] + #[test] + fn native_h264_geometry_and_timing_are_exact_not_snapped() { + use super::{native_interval, native_size}; + use coreav_isobmff::Rational; + for (w, h) in [(16, 16), (1920, 1080), (18, 34)] { + assert!(native_size(w, h)); + } + for (w, h) in [ + (14, 16), + (16, 1082), + (1922, 1080), + (17, 16), + (16, 17), + (3840, 2160), + ] { + assert!(!native_size(w, h)); + } + let time_base = Rational::new(1, 15360).unwrap(); + assert!(native_interval(512, time_base)); + for ticks in [-1, 0, 511, 513, i128::MAX] { + assert!(!native_interval(ticks, time_base)); + } + assert!(!native_interval(1, Rational::new(1001, 30000).unwrap())); + } use super::*; #[test] diff --git a/docs-site/commands/ffmpeg.mdx b/docs-site/commands/ffmpeg.mdx index 625a72ba1..50d245a18 100644 --- a/docs-site/commands/ffmpeg.mdx +++ b/docs-site/commands/ffmpeg.mdx @@ -34,6 +34,21 @@ In every build with a decoder backend, such a command line can also encode one a These audio encoders also work without video: use `-vn` or select one audio stream with `-map a` or `-map 0:a:0`. AAC output can be MP4 or M4A; Opus output can be MP4. `-b:a` and the encoder defaults are unchanged. No video decoder or video encoder runs. Multiple audio streams, delayed source audio, chapters, source tags or dispositions the writer does not preserve, video encoder options and trim options are refused. On M4A, `-map_metadata -1` discards source tags. The standalone `coreav transcode` command still requires video. +### Opt-in native H.264 encoding + +A build with the `native-h264-encoder` Cargo feature accepts the outer flag `--native-h264-encoder` before `ffmpeg`. A decoder backend, such as `native-h264`, is also needed. This feature is absent from default and release feature sets. + +```sh +coreav --native-h264-encoder ffmpeg -i input.mp4 -an -c:v libx264 -crf 23 output.mp4 +coreav --native-h264-encoder ffmpeg -i input.mp4 -an -c:v libx264 -qp 28 fixed.mp4 +``` + +The flag prefers CoreAV's native encoder rather than changing what `libx264` means for other commands. Native output is Constrained Baseline, with enabled deblocking and an IDR interval of 250. Integer `-crf` and `-qp` values from 10 through 51 select constant quality and fixed quantization respectively. With neither, native encoding uses CRF 23. Thread count does not change output bytes. + +Native input must be progressive planar 8-bit 4:2:0 at exactly 30 frames per second, with square pixels and even dimensions from 16 through 1920 horizontally and 16 through 1080 vertically. Native output is limited to MP4 files; Matroska keeps the existing external path. Other clocks, variable frame rates, unsupported color descriptions, scaling, cropping, presets, profiles, B-frame controls, bitrate and VBV options also keep the existing path. So do fractional or out-of-range quality values and commands containing both `-crf` and `-qp`. + +The flag is a preference, not a promise to avoid external programs. If the native library cannot encode a picture within its coding limits, CoreAV discards the staged native output and retries the unchanged command through the existing path. Strict-safe mode also keeps its existing encoder policy. Without the flag, every command keeps its existing behavior. See the [native CLI design](https://github.com/coreav-dev/coreav/blob/main/docs/design/cli-native-h264.md). + ### MP3 output In a build with the `transcode` feature, a `.mp3` output or `-f mp3` defaults to MP3 encoding. `-c:a libmp3lame`, `-codec:a libmp3lame` and `-acodec libmp3lame` select it explicitly. CoreAV runs the `ffmpeg` program named by `COREAV_FFMPEG`, else the one on `PATH`, with libmp3lame as a separate resource-limited encoder. CoreAV supplies decoded PCM, checks its output, and writes the MP3 file itself, including ID3v2.4 and Xing/Info gapless metadata. It never links or bundles libmp3lame. diff --git a/docs/design/cli-native-h264.md b/docs/design/cli-native-h264.md index c9840264d..753497051 100644 --- a/docs/design/cli-native-h264.md +++ b/docs/design/cli-native-h264.md @@ -8,7 +8,9 @@ The selector belongs to CoreAV, before the subcommand. A new codec name would re ## Admission and fallback -The existing argument parser and transcode planner remain authoritative. Native selection requires explicit effective `-c:v libx264`, a file output in MP4 or Matroska, and no unsupported encoder options. Integer `-crf` values from 10 through 51 map to `RateMode::Crf`; integer `-qp` values in the same range map to `RateMode::FixedQp`. Neither option means CRF 23. Both together, fractional values, bitrate, VBV, profiles, B-frame controls, presets, filters, and other unmodeled controls keep the existing path. The original FFmpeg arguments are never rewritten for fallback. +The existing argument parser and transcode planner remain authoritative. Native selection requires explicit effective `-c:v libx264`, a file output in MP4, and no unsupported encoder options. Integer `-crf` values from 10 through 51 map to `RateMode::Crf`; integer `-qp` values in the same range map to `RateMode::FixedQp`. Neither option means CRF 23. Both together, fractional values, bitrate, VBV, profiles, B-frame controls, presets, filters, and other unmodeled controls keep the existing path. The original FFmpeg arguments are never rewritten for fallback. + +Verification narrowed the initial MP4 and Matroska design to MP4. The existing Matroska cadence calculation omits `DefaultDuration` when rounded nanosecond intervals differ. Pinned ffprobe then reports 60 fps for the native stream instead of 30. Matroska therefore keeps the existing external path; correcting its shared cadence calculation is separate work. Decoded pictures must be progressive planar 8-bit 4:2:0, with even dimensions from 16 through 1920 horizontally and 16 through 1080 vertically. The source must also fit those dimensions. The library currently signals exactly 30/1 with square pixels, so other clocks, variable frame rates, and non-square pixels keep the external path. Unsupported color descriptions also keep it. The adapter maps absent color code points to unspecified, without converting samples. @@ -16,12 +18,14 @@ Native output uses Constrained Baseline, enabled deblocking, and an IDR interval Native admission and library failures are transactional. Before publication, an unsupported frame or library refusal discards the native staging and spool files, then reruns the unchanged command through today's external encoder or FFmpeg fallback. I/O, malformed-input, and deadline failures remain terminal. A default command never enters this retry path. +The native path checks its deadline before and after each bounded picture. It does not interrupt a picture already executing. + ## Verification - Compare default-build command output with the pre-change binary, and compare feature-enabled commands without the flag with today's external path. - Exercise every excluded option and input category, requiring unchanged fallback arguments and byte-identical output with and without the flag. - Encode CRF and fixed-QP boundary values through the CLI, check emitted syntax and clocks, and decode the stored output with pinned FFmpeg 9.0.2 and CoreAV. -- Compare complete output bytes across thread counts, both output containers, and the existing CLI limit and timeout tests. +- Compare complete native MP4 bytes across thread counts, retain exact external Matroska output, and run the existing CLI limit and timeout tests. - Check default and release feature graphs, generated command documentation, public text, DCO, CI, and exact-head similarity. Speed and quality comparisons are outside this change. Configurable VUI timing, sample aspect, and additional native encoder capabilities require separate library work. diff --git a/docs/design/transcode.md b/docs/design/transcode.md index 8c547b0c8..6672547a8 100644 --- a/docs/design/transcode.md +++ b/docs/design/transcode.md @@ -1152,6 +1152,8 @@ that AAC source decoding is byte-identical to the oracle. ## Through `coreav ffmpeg` +The optional `native-h264-encoder` CLI feature and outer `--native-h264-encoder` flag prefer CoreAV's native H.264 encoder for eligible commands. The [native CLI design](cli-native-h264.md) defines its integer CRF and fixed-QP mapping, input limits, transactional retry, and unchanged default and release feature sets. The external encoder rules below remain the default and the fallback. + `coreav ffmpeg` ([`compat-cli.md`](compat-cli.md)) runs the common `ffmpeg` command lines of this job through this pipeline: one input, the video re-encoded with libx264, the audio copied, dropped, or one audio diff --git a/scripts/native_h264_cli.py b/scripts/native_h264_cli.py new file mode 100644 index 000000000..a6c75d6af --- /dev/null +++ b/scripts/native_h264_cli.py @@ -0,0 +1,233 @@ +#!/usr/bin/env python3 +"""Generated CLI encoding, unchanged defaults, and exact external fallback.""" +import argparse +from fractions import Fraction +import json +import os +from pathlib import Path +import re +import subprocess +import sys + +sys.path.insert(0, str(Path(__file__).resolve().parent)) +import h264_crf_conformance as crf +import h264_p_conformance as syntax +import test_oracle + + +def run(argv, *, env=None, success=True): + result = subprocess.run(list(map(str, argv)), capture_output=True, timeout=120, env=env) + if success and result.returncode: + raise ValueError(f'command failed: {argv[0]}: {result.stderr.decode(errors="replace")}') + return result + + +def same(actual, expected, label): + if actual != expected: + raise ValueError(f'{label} differs') + + +def annexb_units(data): + parts = re.split(rb'\x00\x00\x00?\x01', data) + if parts.pop(0) or not parts or any(not part for part in parts): + raise ValueError('invalid Annex B delimiters') + return [part.rstrip(b'\0') for part in parts] + + +def verify(args): + tools = test_oracle.require(args.ffmpeg, args.ffmpeg.with_name('ffprobe')) + root = args.output.resolve() + root.mkdir(parents=True, exist_ok=False) + environment = {**os.environ, 'COREAV_X264': '/nonexistent/native-cli-x264', + 'COREAV_FFMPEG': str(tools.ffmpeg), 'COREAV_FALLBACK': str(tools.ffmpeg), + 'COREAV_STRICT_SAFE': '0'} + native_only = {**environment, 'COREAV_FFMPEG': '/nonexistent/native-cli-ffmpeg', + 'COREAV_FALLBACK': '/nonexistent/native-cli-fallback'} + + def source(name, width=18, height=34, rate='30', extra=(), frames=4, raw=None): + path = root / f'{name}.mp4' + if raw is None: + raw = root / f'{name}.yuv' + raw.write_bytes(crf.pictures(width, height, frames)) + run([tools.ffmpeg, '-v', 'error', '-f', 'rawvideo', '-pix_fmt', 'yuv420p', + '-s', f'{width}x{height}', '-r', rate, '-i', raw, '-frames:v', frames, + '-an', '-c:v', 'libx264', '-preset', 'ultrafast', '-qp', 18, '-bf', 0, + '-color_primaries', 'bt709', '-color_trc', 'bt709', '-colorspace', 'bt709', + '-x264-params', 'colorprim=bt709:transfer=bt709:colormatrix=bt709', + *extra, path]) + return path + + source_path = source('source') + source_info = json.loads(run([tools.ffprobe, '-v', 'error', '-show_streams', '-of', 'json', source_path]).stdout)['streams'][0] + same([source_info.get(key) for key in ('color_space', 'color_transfer', 'color_primaries')], + ['bt709'] * 3, 'generated BT.709 source metadata') + raw = root / 'decoded-source.yuv' + run([tools.ffmpeg, '-v', 'error', '-i', source_path, '-an', '-pix_fmt', 'yuv420p', '-f', 'rawvideo', raw]) + rows = [] + for mode, quality in [('crf', 10), ('crf', 23), ('crf', 51), ('qp', 10), ('qp', 28), ('qp', 51)]: + reference = root / f'reference-{mode}-{quality}' + reference.mkdir() + expected = reference / 'reconstruction.yuv' + run([args.driver, 18, 34, f'crf:{quality}' if mode == 'crf' else quality, 4, + raw, reference / 'stream.h264', expected, reference / 'own.yuv', 'true', 250]) + container = 'mp4' + identity = None + for threads in ('1', '2', '8', '0', 'auto'): + output = root / f'native-{mode}-{quality}-{threads}.{container}' + command = ['-i', source_path, '-an', '-c:v', 'libx264', f'-{mode}', quality, + '-threads', threads, output] + run([args.candidate, '--native-h264-encoder', 'ffmpeg', *command], env=native_only) + if identity is None: + identity = output.read_bytes() + else: + same(output.read_bytes(), identity, 'thread-independent complete output') + if threads != '1': + continue + info = json.loads(run([tools.ffprobe, '-v', 'error', '-show_streams', + '-show_packets', '-of', 'json', output]).stdout) + stream, = info['streams'] + same((stream['profile'], stream['width'], stream['height'], stream['pix_fmt']), + ('Constrained Baseline', 18, 34, 'yuv420p'), 'native output properties') + same(stream['r_frame_rate'], '30/1', 'native output rate') + same(len(info['packets']), 4, 'stored picture count') + times = [Fraction(p['pts']) * Fraction(stream['time_base']) for p in info['packets']] + for index, time in enumerate(times): + if abs(time - Fraction(index, 30)) > Fraction(1, 1000): + raise ValueError('stored presentation time differs') + decoded = root / f'decoded-{mode}-{quality}.{container}.yuv' + run([tools.ffmpeg, '-v', 'error', '-xerror', '-i', output, '-an', '-f', 'rawvideo', decoded]) + same(decoded.read_bytes(), expected.read_bytes(), 'pinned decoder reconstruction') + own = root / f'own-{mode}-{quality}.{container}.yuv' + run([args.candidate, 'ffmpeg', '-i', output, '-an', '-c:v', 'rawvideo', + '-pix_fmt', 'yuv420p', '-f', 'rawvideo', own], env=native_only) + same(own.read_bytes(), expected.read_bytes(), 'CoreAV decoder reconstruction') + elementary = root / f'syntax-{mode}-{quality}.{container}.h264' + run([tools.ffmpeg, '-v', 'error', '-i', output, '-c:v', 'copy', '-an', + '-bsf:v', 'h264_mp4toannexb', '-f', 'h264', elementary]) + units = annexb_units(elementary.read_bytes()) + parameters = [n for n in units if n[0] & 31 in (7, 8)][:2] + pictures = [n for n in units if n[0] & 31 in (1, 5)] + repeated = b''.join(b'\0\0\0\1' + n for picture in pictures for n in [*parameters, picture]) + qps = crf.expected_qps(raw.read_bytes(), 18, 34, 4, quality, 250) if mode == 'crf' else None + syntax.inspect(repeated, 18, 34, quality, 4, True, 120, picture_qps=qps) + rows.append(dict(mode=mode, quality=quality, container=container, frames=4)) + + flat = root / 'gop.yuv' + flat.write_bytes(bytes([128]) * (384 * 251)) + gop_source = source('gop', width=16, height=16, frames=251, raw=flat) + gop_output = root / 'gop-native.mp4' + run([args.candidate, '--native-h264-encoder', 'ffmpeg', '-i', gop_source, + '-an', '-c:v', 'libx264', gop_output], env=native_only) + gop_packets = json.loads(run([tools.ffprobe, '-v', 'error', '-show_packets', '-of', 'json', gop_output]).stdout)['packets'] + same(len(gop_packets), 251, 'default-quality GOP picture count') + same([i for i,p in enumerate(gop_packets) if 'K' in p['flags']], [0, 250], 'native IDR interval') + gop_decoded = root / 'gop-decoded.yuv' + run([tools.ffmpeg, '-v', 'error', '-xerror', '-i', gop_output, '-an', '-f', 'rawvideo', gop_decoded]) + same(gop_decoded.stat().st_size, 251 * 384, 'complete GOP decode') + + fallback_cases = [] + + def compare(source_path, extra, label, container='mp4'): + outputs = [] + statuses = [] + for selected in (False, True): + output = root / f'fallback-{label}-{selected}.{container}' + output.write_bytes(b'preserved destination') + command = [args.candidate, *(['--native-h264-encoder'] if selected else []), + 'ffmpeg', '-y', '-i', source_path, '-an', '-c:v', 'libx264', *extra, output] + result = run(command, env=environment, success=False) + statuses.append(result.returncode) + outputs.append(output.read_bytes()) + same(statuses[1], statuses[0], 'fallback exit status') + same(outputs[1], outputs[0], f'complete fallback bytes ({label})') + fallback_cases.append(label) + + options = [('-profile:v', 'high'), ('-profile:v', 'baseline'), ('-bf', '2'), + ('-b:v', '100k'), ('-maxrate', '100k', '-bufsize', '200k'), + ('-preset', 'fast'), ('-pix_fmt', 'yuv420p10le'), ('-pix_fmt', 'yuv444p'), + ('-crf', '9'), ('-crf', '23.5'), ('-qp', '9'), ('-qp', '23.5'), + ('-crf', '23', '-qp', '24'), ('-tune', 'film'), ('-g', '12'), + ('-x264-params', 'cabac=1'), ('-vf', 'scale=32:32'), ('-s', '32x32')] + for index, extra in enumerate(options): + compare(source_path, extra, f'option-{index}') + compare(source_path, (), 'matroska', 'mkv') + for name, parameters in [('small', dict(width=14, height=16)), + ('wide', dict(width=1922, height=16, frames=1)), + ('tall', dict(width=16, height=1082, frames=1)), + ('4k', dict(width=3840, height=2160, frames=1)), + ('rate24', dict(rate='24')), ('rate2997', dict(rate='30000/1001')), + ('sar', dict(extra=('-vf', 'setsar=4/3'))), + ('depth10', dict(extra=('-pix_fmt', 'yuv420p10le'))), + ('chroma444', dict(extra=('-pix_fmt', 'yuv444p'))), + ('hdr', dict(extra=('-x264-params', 'colorprim=bt709:transfer=smpte2084:colormatrix=bt709'))), + ('interlace', dict(width=32, height=32, extra=('-flags', '+ildct+ilme', '-x264-params', 'tff=1')))]: + compare(source(name, **parameters), (), name) + + variable = source('variable-rate', extra=('-vf', 'setpts=if(gte(N\\,2)\\,N+1\\,N)/30/TB', + '-fps_mode', 'passthrough')) + compare(variable, ('-fps_mode', 'passthrough'), 'variable-rate') + + state = 17 + dense = bytearray() + for _ in range(384): + state ^= state << 13 & 0xffffffff + state ^= state >> 17 + state ^= state << 5 & 0xffffffff + dense.append(state & 255) + mixed = root / 'late-limit.yuv' + mixed.write_bytes(bytes([128]) * (384 * 250) + dense) + late = source('late-limit', width=16, height=16, frames=251, raw=mixed, extra=('-qp', '1')) + decoded_late = root / 'late-decoded.yuv' + run([tools.ffmpeg, '-v', 'error', '-i', late, '-f', 'rawvideo', decoded_late]) + late_reference = root / 'late-reconstruction.yuv' + rejected = run([args.driver, 16, 16, 10, 251, decoded_late, root / 'late.h264', late_reference, + root / 'late-own.yuv', 'true', 250], success=False) + if rejected.returncode == 0 or late_reference.stat().st_size != 384 * 250 or b'MacroblockLimit' not in rejected.stderr: + raise ValueError('late fixture did not hit the native coding limit after 250 pictures') + first_only = root / 'first-only.mp4' + run([tools.ffmpeg, '-v', 'error', '-i', late, '-frames:v', 1, '-c:v', 'copy', first_only]) + run([args.candidate, '--native-h264-encoder', 'ffmpeg', '-i', first_only, + '-an', '-c:v', 'libx264', '-qp', 10, root / 'first-native.mp4'], env=native_only) + destination = root / 'late-preserved.mp4' + destination.write_bytes(b'preserved destination') + before = {path.name for path in root.iterdir()} + rejected = run([args.candidate, '--native-h264-encoder', 'ffmpeg', '-y', '-i', late, + '-an', '-c:v', 'libx264', '-qp', 10, destination], env=native_only, success=False) + if rejected.returncode == 0: + raise ValueError('native-only late coding limit unexpectedly succeeded') + same(destination.read_bytes(), b'preserved destination', 'late-refusal destination preservation') + same({path.name for path in root.iterdir()}, before, 'late-refusal staging cleanup') + compare(late, ('-qp', '10'), 'late-coding-limit') + + for label, binary, command in [ + ('default-copy', args.baseline_default, ['-c', 'copy']), + ('external', args.baseline_external, ['-an', '-c:v', 'libx264', '-crf', '23']), + ]: + outputs = [] + for index, executable in enumerate((binary, args.candidate)): + output = root / f'{label}-{index}.mp4' + run([executable, 'ffmpeg', '-i', source_path, *command, output], env=environment) + outputs.append(output.read_bytes()) + same(*outputs, f'unchanged {label} bytes') + if label == 'default-copy': + output = root / 'default-current.mp4' + run([args.default_candidate, 'ffmpeg', '-i', source_path, *command, output], env=environment) + same(output.read_bytes(), outputs[0], 'default-build bytes against base') + + (root / 'summary.json').write_text(json.dumps(dict(native=rows, gop_frames=251, fallback_cases=fallback_cases, + default_bytes_unchanged=True), indent=2) + '\n') + print(f'native CLI: {len(rows)} conformance cases, thread identity, fallback and default byte checks passed') + + +def main(): + parser = argparse.ArgumentParser(description=__doc__) + for name in ('candidate', 'default-candidate', 'baseline-default', 'baseline-external', 'driver', 'ffmpeg', 'output'): + parser.add_argument('--' + name, type=Path, required=True) + args = parser.parse_args() + for name in ('candidate', 'default_candidate', 'baseline_default', 'baseline_external', 'driver', 'ffmpeg'): + setattr(args, name, getattr(args, name).resolve()) + verify(args) + + +if __name__ == '__main__': + main() diff --git a/scripts/tests/test_native_h264_cli.py b/scripts/tests/test_native_h264_cli.py new file mode 100644 index 000000000..ead29f216 --- /dev/null +++ b/scripts/tests/test_native_h264_cli.py @@ -0,0 +1,41 @@ +import pathlib +import sys +import unittest + +ROOT = pathlib.Path(__file__).resolve().parents[2] +sys.path.insert(0, str(ROOT / 'scripts')) +import native_h264_cli as cli + + +class NativeCli(unittest.TestCase): + def test_oracle_annex_b_accepts_both_standard_delimiter_lengths(self): + self.assertEqual(cli.annexb_units(b'\0\0\0\1\x67\x80\0\0\1\x68\x80\0\0\0\1\x65\x80'), + [b'\x67\x80', b'\x68\x80', b'\x65\x80']) + for data in (b'', b'no delimiter', b'\0\0\1', b'\0\0\1\0\0\1\x65\x80'): + with self.assertRaises(ValueError): + cli.annexb_units(data) + + def test_byte_comparator_rejects_a_changed_output(self): + cli.same(b'complete', b'complete', 'output') + with self.assertRaisesRegex(ValueError, 'output differs'): + cli.same(b'complete', b'changed', 'output') + + def test_workflow_builds_actual_base_and_candidate_modes(self): + workflow = (ROOT / '.github/workflows/h264-encoder.yml').read_text() + job = workflow.split(' native-h264-cli:', 1)[1].split(' h264-encoder-conformance:', 1)[0] + for required in ('github.event.pull_request.base.sha || github.sha', + 'cd ../baseline', 'default-base', 'external-base', 'default-candidate', + '--features native-h264-encoder,native-h264', '--features h264-native', + '--features full --example p', 'scripts/native_h264_cli.py', + 'hostile_encoders_end_in_an_error_and_leave_nothing -- --exact', + '--test docs_site --test strict_safe_limit --test ffprobe_limits', + "for features in '' release-default release-full"): + self.assertIn(required, job) + for event in ('push:', 'pull_request:'): + selected = workflow.split(' ' + event, 1)[1].split(' workflow_dispatch:', 1)[0] + for path in ('crates/coreav-cli/**', 'crates/coreav-transcode/**', 'scripts/native_h264_cli.py'): + self.assertIn(path, selected) + + +if __name__ == '__main__': + unittest.main() From 6c12ae060fef619dc5d5d17251494da0ec7ba3ec Mon Sep 17 00:00:00 2001 From: Gunwoo Kim Date: Sun, 8 Nov 2026 13:36:32 +0900 Subject: [PATCH 3/6] fix(python): recognize the CLI encoder cfg Signed-off-by: Gunwoo Kim --- .github/workflows/h264-encoder.yml | 2 +- crates/coreav-py/build.rs | 5 +++++ docs/agents/ci.md | 1 + 3 files changed, 7 insertions(+), 1 deletion(-) create mode 100644 crates/coreav-py/build.rs diff --git a/.github/workflows/h264-encoder.yml b/.github/workflows/h264-encoder.yml index 3be2985d2..dbd5390aa 100644 --- a/.github/workflows/h264-encoder.yml +++ b/.github/workflows/h264-encoder.yml @@ -108,7 +108,7 @@ jobs: cp target/release/coreav "$RUNNER_TEMP/default-candidate" - name: Check opt-in adapters and existing CLI limits run: | - python3 -I -m unittest discover -s scripts/tests -p 'test_native_h264_cli.py' + python3 -I -m unittest discover -s scripts/tests -p 'test_native_h264_cli.py' -v cargo test --locked --release -p coreav-cli --features native-h264-encoder,native-h264 --lib ffmpeg cargo test --locked --release -p coreav-cli --features native-h264-encoder,native-h264 --test docs_site --test strict_safe_limit --test ffprobe_limits --test native_h264_cli cargo test --locked --release -p coreav-transcode --features h264-native --lib diff --git a/crates/coreav-py/build.rs b/crates/coreav-py/build.rs new file mode 100644 index 000000000..6435f2ccf --- /dev/null +++ b/crates/coreav-py/build.rs @@ -0,0 +1,5 @@ +//! Recognized CLI-only configuration in the embedded command source. + +fn main() { + println!("cargo:rustc-check-cfg=cfg(feature, values(\"native-h264-encoder\"))"); +} diff --git a/docs/agents/ci.md b/docs/agents/ci.md index b73ee8a98..5de671440 100644 --- a/docs/agents/ci.md +++ b/docs/agents/ci.md @@ -39,6 +39,7 @@ List development caches with `docker volume ls --filter name=coreav-dev-`. After | Rule | Enforced by | |---|---| +| The embedded Python command recognizes CLI-only native encoder cfg values without exposing or enabling that encoder | `coreav-py/build.rs` declares only the exact known cfg value; `cargo clippy --workspace --all-targets -- -D warnings` rejects drift in the shared command source. The Python Cargo feature interface and default dependency graph remain unchanged | | MP3 byte parity executes real generated media in required Linux CI rather than passing with an unbuilt or unselected transcode test | The explicit `check` step enables `ffmpeg-fallback`, selects the exact MP3 test after oracle qualification and requires its success witness with pipeline failure propagation. `test_required_linux_mp3_comparison_rejects_zero_tests_and_failed_commands` rejects missing, failed and unrelated tests, and a failed Cargo command that printed the witness. Cargo features and test selection are runtime CI facts, not properties of a Rust test body | | A development image verifies its Rust bootstrap before executing it | `scripts/dev/Dockerfile` selects versioned rustup-init binaries and committed architecture-specific SHA-256 values; `test_corrupt_bootstrap_is_not_executed` executes its shell step with a corrupt download for both architectures and requires checksum failure without execution. The compiler toolchain remains owned by `rust-toolchain.toml` | | Local Linux development commands mount only their linked worktree and named Cargo caches, without writable Git pointers, host credentials or default runtime networking | `scripts/dev-container.py` owns the Docker argument list and explicit build inputs; `test_dev_container.py` checks the read-only `.git` overlay, primary-checkout refusal, mounts, non-root identity, limits, network opt-in, argument forwarding, build context, and refusal outside the worktree. Keep secrets outside the mounted worktree; arbitrary file contents cannot be classified by a path check | From 5f3508fbcf97c3b749a25a2bf7822cb918074838 Mon Sep 17 00:00:00 2001 From: Gunwoo Kim Date: Sun, 8 Nov 2026 13:55:48 +0900 Subject: [PATCH 4/6] fix(cli): preserve native retry deadline Signed-off-by: Gunwoo Kim --- .github/workflows/h264-encoder.yml | 10 ++- crates/coreav-cli/src/compat_cmd.rs | 78 +++++++++++++-------- crates/coreav-cli/src/ffmpeg/encode.rs | 9 +-- crates/coreav-cli/src/ffmpeg/mod.rs | 27 +++++-- crates/coreav-cli/src/ffmpeg/native_h264.rs | 62 +++++++++++++++- crates/coreav-cli/src/main.rs | 2 + crates/coreav-cli/src/transcode.rs | 50 +++++++++++-- crates/coreav-cli/tests/native_h264_cli.rs | 20 ++++++ docs-site/commands/ffmpeg.mdx | 2 + docs/agents/ci.md | 1 + docs/agents/transcode.md | 1 + docs/design/cli-native-h264.md | 2 +- scripts/tests/test_native_h264_cli.py | 9 +++ 13 files changed, 221 insertions(+), 52 deletions(-) diff --git a/.github/workflows/h264-encoder.yml b/.github/workflows/h264-encoder.yml index dbd5390aa..ee1057c3e 100644 --- a/.github/workflows/h264-encoder.yml +++ b/.github/workflows/h264-encoder.yml @@ -109,7 +109,7 @@ jobs: - name: Check opt-in adapters and existing CLI limits run: | python3 -I -m unittest discover -s scripts/tests -p 'test_native_h264_cli.py' -v - cargo test --locked --release -p coreav-cli --features native-h264-encoder,native-h264 --lib ffmpeg + cargo test --locked --release -p coreav-cli --features native-h264-encoder,native-h264 --lib ffmpeg -- --nocapture cargo test --locked --release -p coreav-cli --features native-h264-encoder,native-h264 --test docs_site --test strict_safe_limit --test ffprobe_limits --test native_h264_cli cargo test --locked --release -p coreav-transcode --features h264-native --lib cargo clippy --locked -p coreav-cli --features native-h264-encoder,native-h264 --all-targets -- -D warnings @@ -117,8 +117,12 @@ jobs: cargo build --locked --release -p coreav-h264-encoder --features full --example p - name: Install pinned black-box tools and generate trusted fixtures run: | - scripts/install-oracle-ffmpeg.sh "$RUNNER_TEMP/cli-oracles" - scripts/install-x264.sh "$RUNNER_TEMP/cli-oracles" + sudo apt-get update + sudo apt-get install -y --no-install-recommends nasm + echo 'Installing pinned FFmpeg' + scripts/install-oracle-ffmpeg.sh "$RUNNER_TEMP/cli-oracles" || { echo '::error::reason=pinned_ffmpeg_install_failed'; exit 1; } + echo 'Installing pinned x264' + scripts/install-x264.sh "$RUNNER_TEMP/cli-oracles" || { echo '::error::reason=pinned_x264_install_failed'; exit 1; } export PATH="$RUNNER_TEMP/cli-oracles:$PATH" scripts/gen-fixtures.sh "$RUNNER_TEMP/cli-fixtures" - name: Verify stored native output and unchanged external behavior diff --git a/crates/coreav-cli/src/compat_cmd.rs b/crates/coreav-cli/src/compat_cmd.rs index 80912a71f..49143f75e 100644 --- a/crates/coreav-cli/src/compat_cmd.rs +++ b/crates/coreav-cli/src/compat_cmd.rs @@ -74,11 +74,24 @@ pub fn run_ffmpeg(args: &[OsString], strict_safe: StrictSafe) -> ExitCode { Ok(args) => args, Err(code) => return code, }; - match ffmpeg::args::summary(&args) { - Ok(Some(options)) => return run_summary(Tool::Ffmpeg, &args, &options, strict_safe), + run_ffmpeg_with( + &args, + strict_safe, + #[cfg(feature = "native-h264-encoder")] + None, + ) +} + +fn run_ffmpeg_with( + args: &[String], + strict_safe: StrictSafe, + #[cfg(feature = "native-h264-encoder")] attempt: Option, +) -> ExitCode { + match ffmpeg::args::summary(args) { + Ok(Some(options)) => return run_summary(Tool::Ffmpeg, args, &options, strict_safe), Ok(None) => {} Err(Refusal::NotNative(refusal)) => { - return fallback_cmd::refuse(Tool::Ffmpeg, &args, refusal, strict_safe); + return fallback_cmd::refuse(Tool::Ffmpeg, args, refusal, strict_safe); } Err(Refusal::Usage(text)) => { message("ffmpeg", &text); @@ -87,7 +100,7 @@ pub fn run_ffmpeg(args: &[OsString], strict_safe: StrictSafe) -> ExitCode { Err(Refusal::Help) => return print_usage(ffmpeg::USAGE), } let mut spool = None; - if let Ok(command) = ffmpeg::args::parse(&args) + if let Ok(command) = ffmpeg::args::parse(args) && ffmpeg::args::stdin(&command.input) && command.more_inputs.is_empty() && matches!( @@ -108,9 +121,24 @@ pub fn run_ffmpeg(args: &[OsString], strict_safe: StrictSafe) -> ExitCode { } } let result = match &spool { - Some(input) => ffmpeg::run_spooled(&args, strict_safe, &input.path), - None => ffmpeg::run(&args, strict_safe), + Some(input) => ffmpeg::run_spooled(args, strict_safe, &input.path), + #[cfg(feature = "native-h264-encoder")] + None => match attempt { + Some(attempt) => ffmpeg::run_attempt(args, strict_safe, attempt), + None => ffmpeg::run(args, strict_safe), + }, + #[cfg(not(feature = "native-h264-encoder"))] + None => ffmpeg::run(args, strict_safe), }; + finish_ffmpeg(args, strict_safe, spool, result) +} + +fn finish_ffmpeg( + args: &[String], + strict_safe: StrictSafe, + mut spool: Option, + result: Result, +) -> ExitCode { match result { Ok(_) => ExitCode::SUCCESS, Err(Stop::Refused(Refusal::NotNative(refusal))) => { @@ -118,7 +146,7 @@ pub fn run_ffmpeg(args: &[OsString], strict_safe: StrictSafe) -> ExitCode { match input.into_replay() { Ok((file, input)) => fallback_cmd::refuse_with_input( Tool::Ffmpeg, - &args, + args, refusal, strict_safe, file, @@ -133,7 +161,7 @@ pub fn run_ffmpeg(args: &[OsString], strict_safe: StrictSafe) -> ExitCode { } } } else { - fallback_cmd::refuse(Tool::Ffmpeg, &args, refusal, strict_safe) + fallback_cmd::refuse(Tool::Ffmpeg, args, refusal, strict_safe) } } Err(Stop::Refused(Refusal::Usage(text))) => { @@ -155,7 +183,7 @@ pub fn run_ffmpeg(args: &[OsString], strict_safe: StrictSafe) -> ExitCode { .get("message") .and_then(serde_json::Value::as_str) .map_or_else(|| json.to_string(), str::to_owned); - message("ffmpeg", &format!("{}: {text}", args_input(&args))); + message("ffmpeg", &format!("{}: {text}", args_input(args))); ExitCode::from(if failure.is_io() { EXIT_IO } else { @@ -171,28 +199,18 @@ pub fn run_ffmpeg_native(args: &[OsString], strict_safe: StrictSafe) -> ExitCode Ok(text) => text, Err(code) => return code, }; - match ffmpeg::native_h264::run(&text, strict_safe) { + let deadline = std::time::Instant::now() + ffmpeg::ENCODE_LIMIT; + match ffmpeg::native_h264::run(&text, strict_safe, deadline) { Ok(Some(_)) => ExitCode::SUCCESS, - Ok(None) => run_ffmpeg(args, strict_safe), - Err(Stop::Failed(failure)) => { - let json = failure.to_json(); - let text = json - .get("message") - .and_then(serde_json::Value::as_str) - .map_or_else(|| json.to_string(), str::to_owned); - message("ffmpeg", &text); - ExitCode::from(if failure.is_io() { - EXIT_IO - } else { - EXIT_REJECTED - }) - } - Err(Stop::Refused(Refusal::Usage(text))) => { - message("ffmpeg", &text); - ExitCode::from(EXIT_USAGE) - } - Err(Stop::Refused(Refusal::Help)) => print_usage(ffmpeg::USAGE), - Err(Stop::Refused(Refusal::NotNative(_))) => run_ffmpeg(args, strict_safe), + Ok(None) | Err(Stop::Refused(Refusal::NotNative(_))) => run_ffmpeg_with( + &text, + strict_safe, + Some(ffmpeg::native_h264::EncodeAttempt { + native: None, + deadline, + }), + ), + Err(error) => finish_ffmpeg(&text, strict_safe, None, Err(error)), } } diff --git a/crates/coreav-cli/src/ffmpeg/encode.rs b/crates/coreav-cli/src/ffmpeg/encode.rs index c11e0e6d0..12667b243 100644 --- a/crates/coreav-cli/src/ffmpeg/encode.rs +++ b/crates/coreav-cli/src/ffmpeg/encode.rs @@ -303,6 +303,8 @@ pub fn request( Ok(Request { #[cfg(feature = "native-h264-encoder")] native_h264: None, + #[cfg(feature = "native-h264-encoder")] + deadline: None, matroska_output: command.muxer.matroska(), crf, bitrate, @@ -592,14 +594,13 @@ pub(super) fn write_inner( encoder: Encoder, strict_safe: StrictSafe, capture: Option<&Path>, - #[cfg(feature = "native-h264-encoder")] native: Option< - coreav_transcode::native_h264::NativeH264, - >, + #[cfg(feature = "native-h264-encoder")] attempt: Option, ) -> Result { let request = request(command, job, encoder, strict_safe)?; #[cfg(feature = "native-h264-encoder")] let request = Request { - native_h264: native, + native_h264: attempt.and_then(|attempt| attempt.native), + deadline: attempt.map(|attempt| attempt.deadline), ..request }; #[cfg(feature = "input-capture")] diff --git a/crates/coreav-cli/src/ffmpeg/mod.rs b/crates/coreav-cli/src/ffmpeg/mod.rs index 286cab075..bc13d90fe 100644 --- a/crates/coreav-cli/src/ffmpeg/mod.rs +++ b/crates/coreav-cli/src/ffmpeg/mod.rs @@ -32,7 +32,7 @@ pub(crate) const MUXER_ENCODER: &str = "Lavf63.1.102"; /// The wall-clock limit of one native encode command. #[cfg(feature = "transcode")] -const ENCODE_LIMIT: std::time::Duration = std::time::Duration::from_secs(3600); +pub const ENCODE_LIMIT: std::time::Duration = std::time::Duration::from_secs(3600); use crate::error::Failure; use crate::fallback::NotNative; @@ -306,6 +306,23 @@ pub fn run(args: &[String], strict_safe: StrictSafe) -> Result Result { + run_command( + args::parse(args)?, + strict_safe, + None, + None, + None, + Some(attempt), + ) +} + /// Runs a parsed pipe command from its privately spooled encoded input. pub fn run_spooled( args: &[String], @@ -393,10 +410,10 @@ fn run_command( capture: Option<&Path>, image_capture: Option<&Path>, spooled: Option<&Path>, - #[cfg(feature = "native-h264-encoder")] native: Option< - coreav_transcode::native_h264::NativeH264, - >, + #[cfg(feature = "native-h264-encoder")] attempt: Option, ) -> Result { + #[cfg(feature = "native-h264-encoder")] + let native = attempt.and_then(|attempt| attempt.native); if let Some(input) = spooled { command.input = input.to_string_lossy().into_owned(); } @@ -516,7 +533,7 @@ fn run_command( strict_safe, capture, #[cfg(feature = "native-h264-encoder")] - native, + attempt, ); } #[cfg(feature = "native-h264-encoder")] diff --git a/crates/coreav-cli/src/ffmpeg/native_h264.rs b/crates/coreav-cli/src/ffmpeg/native_h264.rs index fcb044f5b..2f4d668b5 100644 --- a/crates/coreav-cli/src/ffmpeg/native_h264.rs +++ b/crates/coreav-cli/src/ffmpeg/native_h264.rs @@ -1,12 +1,22 @@ //! Explicit native encoding preference with unchanged external fallback arguments. use coreav_transcode::native_h264::{NativeH264, RateMode}; +use std::time::Instant; use super::args::{Codec, Command, Muxer, Spec}; use super::{Refusal, Stop}; use crate::ffprobe::{Report, args::Kind2, model::Value}; use crate::strict_safe::StrictSafe; +#[derive(Clone, Copy)] +/// An encode attempt within the original opt-in command's time budget. +pub struct EncodeAttempt { + /// Native encoding, or the existing encoder on retry. + pub native: Option, + /// The same deadline for both attempts. + pub deadline: Instant, +} + fn candidate(args: &[String]) -> Option<(Command, NativeH264)> { let (command, qp) = super::args::native_h264(args).ok()?; let encoding = &command.encoding; @@ -57,14 +67,22 @@ pub(super) fn source(report: &Report, index: usize) -> bool { } /// Tries the opt-in encoder without changing the command used on fallback. -pub fn run(args: &[String], strict_safe: StrictSafe) -> Result, Stop> { +pub fn run( + args: &[String], + strict_safe: StrictSafe, + deadline: Instant, +) -> Result, Stop> { if strict_safe.on() { return Ok(None); } let Some((command, native)) = candidate(args) else { return Ok(None); }; - match super::run_command(command, strict_safe, None, None, None, Some(native)) { + let attempt = EncodeAttempt { + native: Some(native), + deadline, + }; + match super::run_command(command, strict_safe, None, None, None, Some(attempt)) { Ok(value) => Ok(Some(value)), Err(Stop::Refused(Refusal::NotNative(_))) => Ok(None), Err(error) => Err(error), @@ -73,8 +91,47 @@ pub fn run(args: &[String], strict_safe: StrictSafe) -> Result assert_eq!(failure.to_json()["limit"], "timeout"), + _ => panic!("the expired external retry must fail with timeout"), + } + assert_eq!(std::fs::read(&output).unwrap(), b"keep destination"); + assert_eq!(std::fs::read_dir(&dir).unwrap().count(), 2); + std::fs::remove_dir_all(dir).unwrap(); + } + #[test] fn source_admission_requires_known_progressive_geometry_and_clock() { let mut report = Report::default(); @@ -105,6 +162,7 @@ mod tests { ("pix_fmt", Value::Str("yuv444p".into())), ("r_frame_rate", Value::Str("30000/1001".into())), ("r_frame_rate", Value::Str("3002/100".into())), + ("r_frame_rate", Value::Str("0/0".into())), ] { let mut changed = report.clone(); if let Some(stream) = changed.streams.first_mut() { diff --git a/crates/coreav-cli/src/main.rs b/crates/coreav-cli/src/main.rs index 9b18c1f34..4fb13be04 100644 --- a/crates/coreav-cli/src/main.rs +++ b/crates/coreav-cli/src/main.rs @@ -1181,6 +1181,8 @@ fn run_transcode( let request = transcode::Request { #[cfg(feature = "native-h264-encoder")] native_h264: None, + #[cfg(feature = "native-h264-encoder")] + deadline: None, matroska_output: None, crf: args.crf, bitrate: args.bitrate, diff --git a/crates/coreav-cli/src/transcode.rs b/crates/coreav-cli/src/transcode.rs index e197fb19a..c74280d76 100644 --- a/crates/coreav-cli/src/transcode.rs +++ b/crates/coreav-cli/src/transcode.rs @@ -79,6 +79,9 @@ pub struct Request { /// Explicit opt-in native H.264 preference. #[cfg(feature = "native-h264-encoder")] pub native_h264: Option, + /// An inherited command deadline, rather than a new attempt timeout. + #[cfg(feature = "native-h264-encoder")] + pub deadline: Option, /// Matroska output profile, or the default MP4 output. pub matroska_output: Option, /// `--crf`. @@ -183,7 +186,12 @@ pub fn make( } else { SourceKind::Mp4 }; - let (frame_rates, timeout) = source_rates(&mut source, Duration::from_secs(request.timeout))?; + #[cfg(not(feature = "native-h264-encoder"))] + let deadline = None; + #[cfg(feature = "native-h264-encoder")] + let deadline = request.deadline; + let (frame_rates, timeout) = + source_rates(&mut source, Duration::from_secs(request.timeout), deadline)?; let options = TranscodeOptions { #[cfg(feature = "native-h264-encoder")] native_h264: request.native_h264, @@ -271,6 +279,7 @@ pub fn make( fn source_rates( source: &mut S, timeout: Duration, + deadline: Option, ) -> Result<(Vec<(usize, FrameRate)>, Duration), WriteFailure> { let expired = || { failure(coreav_transcode::Error::Decode(coreav_decode::Error::new( @@ -278,7 +287,9 @@ fn source_rates( "the transcode took longer than its timeout", ))) }; - let deadline = Instant::now().checked_add(timeout).ok_or_else(expired)?; + let deadline = deadline + .or_else(|| Instant::now().checked_add(timeout)) + .ok_or_else(expired)?; let mut source = RateSource { source, deadline }; let rates = match source.len() { Ok(_) => crate::ffprobe::source_frame_rates(&mut source) @@ -587,20 +598,45 @@ mod tests { reads: 0, delay: Duration::from_millis(50), }; - let error = source_rates(&mut source, Duration::ZERO).unwrap_err(); + let error = source_rates(&mut source, Duration::ZERO, None).unwrap_err(); assert_eq!(error.to_json()["limit"], "timeout"); assert_eq!((source.lengths, source.reads), (0, 0)); - let error = source_rates(&mut source, Duration::from_millis(1)).unwrap_err(); + let error = source_rates(&mut source, Duration::from_millis(1), None).unwrap_err(); assert_eq!(error.to_json()["limit"], "timeout"); assert!(source.reads <= 1); let budget = Duration::from_secs(10); - let (rates, remaining) = - source_rates(&mut source, budget).unwrap_or_else(|error| panic!("{}", error.to_json())); + let (rates, remaining) = source_rates(&mut source, budget, None) + .unwrap_or_else(|error| panic!("{}", error.to_json())); assert!(rates.is_empty()); assert!(remaining <= budget - source.delay); assert!(!remaining.is_zero()); } + #[test] + fn ffmpeg_native_retry_uses_only_the_original_deadline() { + let mut source = SlowSource { + lengths: 0, + reads: 0, + delay: Duration::ZERO, + }; + let limit = Duration::from_secs(3600); + let remaining = Duration::from_secs(2); + let deadline = Instant::now() + remaining; + let (_, left) = source_rates(&mut source, limit, Some(deadline)) + .unwrap_or_else(|error| panic!("{}", error.to_json())); + assert!(!left.is_zero()); + assert!(left <= remaining); + assert!(left < limit); + source.lengths = 0; + source.reads = 0; + let error = source_rates(&mut source, limit, Some(Instant::now())).unwrap_err(); + assert_eq!(error.to_json()["limit"], "timeout"); + assert_eq!((source.lengths, source.reads), (0, 0)); + let (_, left) = source_rates(&mut source, limit, None) + .unwrap_or_else(|error| panic!("{}", error.to_json())); + assert!(left > remaining); + } + #[test] fn rate_probe_stops_reads_after_the_deadline() { let mut source = SlowSource { @@ -671,7 +707,7 @@ mod tests { ); let file = [ftyp, bx(b"mdat", &vec![0; COUNT as usize]), moov].concat(); let mut source = coreav_isobmff::SliceSource::new(&file); - let (rates, _) = source_rates(&mut source, Duration::from_secs(1)) + let (rates, _) = source_rates(&mut source, Duration::from_secs(1), None) .unwrap_or_else(|error| panic!("{}", error.to_json())); assert_eq!(rates, vec![(0, FrameRate { num: 5, den: 1 })]); } diff --git a/crates/coreav-cli/tests/native_h264_cli.rs b/crates/coreav-cli/tests/native_h264_cli.rs index 6f5516064..e69aa4258 100644 --- a/crates/coreav-cli/tests/native_h264_cli.rs +++ b/crates/coreav-cli/tests/native_h264_cli.rs @@ -52,6 +52,26 @@ fn selection_is_outer_and_requires_the_ffmpeg_command() { assert!(output.stdout.is_empty()); } +#[test] +fn missing_input_keeps_the_existing_error_bytes() { + let args = [ + "ffmpeg", + "-i", + "/nonexistent/native-input.mp4", + "-an", + "-c:v", + "libx264", + "/nonexistent/native-output.mp4", + ]; + let expected = run(&args); + let mut native = vec!["--native-h264-encoder"]; + native.extend_from_slice(&args); + let actual = run(&native); + assert_eq!(actual.status.code(), expected.status.code()); + assert_eq!(actual.stdout, expected.stdout); + assert_eq!(actual.stderr, expected.stderr); +} + #[test] fn the_ffmpeg_help_bytes_do_not_depend_on_selection() { let expected = run(&["ffmpeg", "-h"]); diff --git a/docs-site/commands/ffmpeg.mdx b/docs-site/commands/ffmpeg.mdx index 50d245a18..f764200a8 100644 --- a/docs-site/commands/ffmpeg.mdx +++ b/docs-site/commands/ffmpeg.mdx @@ -49,6 +49,8 @@ Native input must be progressive planar 8-bit 4:2:0 at exactly 30 frames per sec The flag is a preference, not a promise to avoid external programs. If the native library cannot encode a picture within its coding limits, CoreAV discards the staged native output and retries the unchanged command through the existing path. Strict-safe mode also keeps its existing encoder policy. Without the flag, every command keeps its existing behavior. See the [native CLI design](https://github.com/coreav-dev/coreav/blob/main/docs/design/cli-native-h264.md). +Both CoreAV encode attempts share the original one-hour command limit. A native timeout is terminal rather than a reason to retry: an input that an external encoder could finish within the limit may time out with the native preference. + ### MP3 output In a build with the `transcode` feature, a `.mp3` output or `-f mp3` defaults to MP3 encoding. `-c:a libmp3lame`, `-codec:a libmp3lame` and `-acodec libmp3lame` select it explicitly. CoreAV runs the `ffmpeg` program named by `COREAV_FFMPEG`, else the one on `PATH`, with libmp3lame as a separate resource-limited encoder. CoreAV supplies decoded PCM, checks its output, and writes the MP3 file itself, including ID3v2.4 and Xing/Info gapless metadata. It never links or bundles libmp3lame. diff --git a/docs/agents/ci.md b/docs/agents/ci.md index 5de671440..6547c0b1f 100644 --- a/docs/agents/ci.md +++ b/docs/agents/ci.md @@ -84,6 +84,7 @@ List development caches with `docker volume ls --filter name=coreav-dev-`. After | Queue-detector failure diagnostics never echo bodies, tokens, exception arguments or arbitrary header/error fields; request metadata is conditional | `Api` owns a sanitized last-request trace, initially absent; `endpoint`, `response_facts`, `graphql_errors` and `diagnostic` emit closed labels, bounded digits and known query fields only. The `Diagnostics` regressions in `scripts/tests/test_queue_detector.py` inspect stdout, stderr, the job summary and persisted state with hostile bodies, headers and exception arguments; `test_raw_exception_arguments_are_never_rendered` requires a class-only diagnostic before request preparation. Types cannot classify untrusted HTTP strings | | Audio-source diagnostic preparation failures retain an incomplete report, and command failures expose a bounded stderr tail with their original cause | `main` owns report cleanup around preparation and comparison; `run` owns command failure details. Fixture, version and hash failures plus a real failed child are covered by `test_audio_source_diagnostic.py` (`deny.yml`). A return type cannot guarantee a file is written after an external command fails | | The generated-audio diagnostic runs after test failure only when its prerequisite build succeeded and the job was not cancelled | The named `decode-build` outcome and the diagnostic's prescribed status condition in `ci.yml`; `test_workflow_diagnostic_survives_test_failure_but_not_a_missing_build` requires both and names the condition to restore. A type cannot govern GitHub Actions scheduling | +| The native H.264 CLI job installs nasm before building its pinned x264 oracle and labels each installer stage | `test_cli_oracles_install_nasm_before_x264` checks provisioning order and stage labels in the job that runs the native and fallback comparisons. Rust types cannot establish a runner's system packages | | The WASM module workflow runs when a fixture helper or policy example changes | `test_wasm_triggers_cover_module_fixture_inputs` derives the helpers from the fixture generator and checks both trigger filters against them and the policy examples (`deny.yml`). A type cannot govern YAML filters | | The WASM workflow runs for every shared notice-record input | `test_wasm_triggers_cover_notice_record_inputs` reads `requiredInputs` from the record owner and checks both workflow trigger filters (`deny.yml`). Manifests outside the selected crate still enter Cargo metadata and the record, so a separate hand-written input list would drift | | An installed-package verification failure records a release blocker even when memory gates are skipped, but PR and merge-group failures do not file issues | `test_wasm_reports_installed_package_and_memory_failures` executes the actual reporter script with the workflow's real gate outputs and job condition for main, explicit verdict, PR, merge-group and non-main runs, including skipped checks, earlier failures and cancellation | diff --git a/docs/agents/transcode.md b/docs/agents/transcode.md index 4d98d5872..873b511dd 100644 --- a/docs/agents/transcode.md +++ b/docs/agents/transcode.md @@ -43,3 +43,4 @@ Encode path, sandboxed workers, and audio-encode selection. | AV1 CRF admission validates the original integer token before deriving a floating-point value | Every explicit AV1 route reparses with `parse_av1_crf`, which admits a bounded `u8` before converting exactly to `f32`; a successful legacy probe cannot authorize AV1. `av1_crf_admission_uses_the_exact_integer_token` rejects fractional tokens rounded to integers or zero, ordinary fractions and out-of-range integers before source/output access. Valid boundary controls cover both parser paths. The shared float field remains necessary for fractional H.264 CRF | | Native FFV1 refuses declared chroma siting that the Matroska writer cannot retain | `colour` owns admission for both stream and frame metadata; packet-admission tests reject all five non-left locations before encoder creation or packet publication. The required CLI gate verifies a real centered-chroma source and its specific refusal, while negative tests reject wrong source signalling, false success and unrelated errors. Frame types alone cannot express which metadata a selected container writer preserves | | The `coreav ffmpeg` video encode (H.264/AV1), audio encode and FFV1 jobs run under one command limit, never a library default such as the 60-second decoder timeout; the AVI MP3 encode keeps its own 120-second deadline | `ffmpeg::ENCODE_LIMIT` is the single owner that the encode request, the audio deadline and the FFV1 job options read. `the_encode_has_the_same_command_limit_as_other_encodes` failed on the FFV1 job's former 60-second default. A type cannot make a job builder read the constant, so the test covers the job that once missed it | +| Native H.264 refusal retries inherit the original command deadline, never a fresh encode limit | `EncodeAttempt` carries the same absolute deadline through both CLI attempts. `ffmpeg_native_retry_uses_only_the_original_deadline` checks reduced and expired budgets; `expired_external_retry_keeps_destination_and_removes_staging` executes the retry boundary and requires timeout without replacing the destination or leaving staging. Expiry is a runtime property, not a type invariant | diff --git a/docs/design/cli-native-h264.md b/docs/design/cli-native-h264.md index 753497051..a45d4e012 100644 --- a/docs/design/cli-native-h264.md +++ b/docs/design/cli-native-h264.md @@ -18,7 +18,7 @@ Native output uses Constrained Baseline, enabled deblocking, and an IDR interval Native admission and library failures are transactional. Before publication, an unsupported frame or library refusal discards the native staging and spool files, then reruns the unchanged command through today's external encoder or FFmpeg fallback. I/O, malformed-input, and deadline failures remain terminal. A default command never enters this retry path. -The native path checks its deadline before and after each bounded picture. It does not interrupt a picture already executing. +The native path checks its deadline before and after each bounded picture. It does not interrupt a picture already executing. One original command deadline covers both CoreAV encode attempts; a coding-limit retry receives only the remaining budget. A native timeout remains terminal, even if an external encoder could have completed the input within that limit. ## Verification diff --git a/scripts/tests/test_native_h264_cli.py b/scripts/tests/test_native_h264_cli.py index ead29f216..fd8c5a300 100644 --- a/scripts/tests/test_native_h264_cli.py +++ b/scripts/tests/test_native_h264_cli.py @@ -36,6 +36,15 @@ def test_workflow_builds_actual_base_and_candidate_modes(self): for path in ('crates/coreav-cli/**', 'crates/coreav-transcode/**', 'scripts/native_h264_cli.py'): self.assertIn(path, selected) + def test_cli_oracles_install_nasm_before_x264(self): + workflow = (ROOT / '.github/workflows/h264-encoder.yml').read_text() + job = workflow.split(' native-h264-cli:', 1)[1].split(' h264-encoder-conformance:', 1)[0] + self.assertLess(job.index('apt-get install -y --no-install-recommends nasm'), + job.index('scripts/install-x264.sh')) + for stage in ('Installing pinned FFmpeg', 'Installing pinned x264', + 'reason=pinned_ffmpeg_install_failed', 'reason=pinned_x264_install_failed'): + self.assertIn(stage, job) + if __name__ == '__main__': unittest.main() From 1fd6a57d49110d3ceabd3fab40f4bce9f30cf160 Mon Sep 17 00:00:00 2001 From: Gunwoo Kim Date: Sun, 8 Nov 2026 20:52:19 +0900 Subject: [PATCH 5/6] fix(transcode): keep native metadata defaults Signed-off-by: Gunwoo Kim --- crates/coreav-transcode/src/native_h264.rs | 2 ++ docs/design/cli-native-h264.md | 4 ++-- 2 files changed, 4 insertions(+), 2 deletions(-) diff --git a/crates/coreav-transcode/src/native_h264.rs b/crates/coreav-transcode/src/native_h264.rs index ff12da740..78ebe6208 100644 --- a/crates/coreav-transcode/src/native_h264.rs +++ b/crates/coreav-transcode/src/native_h264.rs @@ -48,6 +48,8 @@ impl Encoder { } let c = settings.color; let settings_native = Settings { + frame_rate: Default::default(), + sample_aspect_ratio: Default::default(), width: settings.width as usize, height: settings.height as usize, rate_mode: request.rate, diff --git a/docs/design/cli-native-h264.md b/docs/design/cli-native-h264.md index a45d4e012..746c9ccd5 100644 --- a/docs/design/cli-native-h264.md +++ b/docs/design/cli-native-h264.md @@ -12,7 +12,7 @@ The existing argument parser and transcode planner remain authoritative. Native Verification narrowed the initial MP4 and Matroska design to MP4. The existing Matroska cadence calculation omits `DefaultDuration` when rounded nanosecond intervals differ. Pinned ffprobe then reports 60 fps for the native stream instead of 30. Matroska therefore keeps the existing external path; correcting its shared cadence calculation is separate work. -Decoded pictures must be progressive planar 8-bit 4:2:0, with even dimensions from 16 through 1920 horizontally and 16 through 1080 vertically. The source must also fit those dimensions. The library currently signals exactly 30/1 with square pixels, so other clocks, variable frame rates, and non-square pixels keep the external path. Unsupported color descriptions also keep it. The adapter maps absent color code points to unspecified, without converting samples. +Decoded pictures must be progressive planar 8-bit 4:2:0, with even dimensions from 16 through 1920 horizontally and 16 through 1080 vertically. The source must also fit those dimensions. CLI admission remains exactly 30/1 with square pixels, so other clocks, variable frame rates, and non-square pixels keep the external path. The adapter uses the library's default 30/1 timing and absent sample-aspect signalling. Unsupported color descriptions also keep the external path. The adapter maps absent color code points to unspecified, without converting samples. Native output uses Constrained Baseline, enabled deblocking, and an IDR interval of 250. The native encoder receives the requested thread count, but its output cannot depend on that count. Existing decoding policy, staging, spool limits, timestamps, audio handling, and muxing are reused. Strict-safe selection retains today's path and policy rather than changing decoder or process authorization. @@ -28,4 +28,4 @@ The native path checks its deadline before and after each bounded picture. It do - Compare complete native MP4 bytes across thread counts, retain exact external Matroska output, and run the existing CLI limit and timeout tests. - Check default and release feature graphs, generated command documentation, public text, DCO, CI, and exact-head similarity. -Speed and quality comparisons are outside this change. Configurable VUI timing, sample aspect, and additional native encoder capabilities require separate library work. +Speed and quality comparisons are outside this change. Broader CLI frame-rate and sample-aspect admission remains separate work even when the library supports it. From 0fa92db77b1721e8f6e22b6133ec7190c3a7e9e6 Mon Sep 17 00:00:00 2001 From: Gunwoo Kim Date: Sun, 8 Nov 2026 21:03:27 +0900 Subject: [PATCH 6/6] docs(cli): clarify opt-in encoder dependencies Signed-off-by: Gunwoo Kim --- docs/agents/docs.md | 1 + docs/design/cli-native-h264.md | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/agents/docs.md b/docs/agents/docs.md index 0b75e89ed..73b1cd0c0 100644 --- a/docs/agents/docs.md +++ b/docs/agents/docs.md @@ -4,6 +4,7 @@ Documentation-site prose, schema wording, and design-note updates. | Rule | Enforced by | |---|---| +| Feature documentation distinguishes an optional encoder from its shared transcode dependency | The native H.264 CLI workflow checks that default and release normal-dependency graphs exclude `coreav-h264-encoder`; the meaning of broader dependency claims remains review judgment. Rust types cannot establish prose meaning | | Documentation checks read Unicode measurement units with an explicit UTF-8 encoding rather than the host locale | The coefficient-design check selects UTF-8 at every file read. `test_document_reads_do_not_use_locale_defaults` runs its real checks with implicit-encoding warnings fatal, rejecting the original omitted encoding. Choosing the format at the I/O boundary removes the locale dependency without a new reader wrapper | | The coefficient-level design describes the all-zero-block shortcut, including skipped DC trials, and records the accepted throughput tradeoff rather than leaving an unmet target as its status | `test_h264_g1_design.py` binds the stronger policy's documented result to `AC_COORDINATES`, requires full source identities, all five Y/U/V and one/four-worker rows, and the shared-host qualification; omission and changed-budget controls reject the former missing-result state. The design owns the result block. Types cannot bind external timings to prose; numerical accuracy and the shortcut explanation remain review against `Decision::refine_levels` and the retained observations | | An audio-option admission change updates both its explanatory prose and the option table | Not enforced: review judgment. The generated CLI tests establish effective codec and identity behavior, but types cannot establish whether separate prose statements agree | diff --git a/docs/design/cli-native-h264.md b/docs/design/cli-native-h264.md index 746c9ccd5..68e175e48 100644 --- a/docs/design/cli-native-h264.md +++ b/docs/design/cli-native-h264.md @@ -2,7 +2,7 @@ ## Selection -`coreav --native-h264-encoder ffmpeg -i in.mp4 -an -c:v libx264 -crf 23 out.mp4` prefers CoreAV's encoder. The `native-h264-encoder` Cargo feature enables the flag and the library wiring. Neither the feature nor its dependencies enter default, release-default, or release-full feature sets. A decoder backend remains a separate opt-in. +`coreav --native-h264-encoder ffmpeg -i in.mp4 -an -c:v libx264 -crf 23 out.mp4` prefers CoreAV's encoder. The `native-h264-encoder` Cargo feature enables the flag and the library wiring. The feature and the `coreav-h264-encoder` crate remain absent from default, release-default, and release-full builds. Both release feature sets already include the shared `transcode` dependency. A decoder backend remains a separate opt-in. The selector belongs to CoreAV, before the subcommand. A new codec name would require rewriting fallback arguments. A flag inside FFmpeg's arguments could collide with an option value or filename. An environment variable would make selection implicit. The outer flag avoids all three problems. Without the flag, the existing path remains unchanged, including in a feature-enabled build.