diff --git a/.claude/skills/forge-error-catalogue/SKILL.md b/.claude/skills/forge-error-catalogue/SKILL.md index 3bd30150..e8e19c71 100644 --- a/.claude/skills/forge-error-catalogue/SKILL.md +++ b/.claude/skills/forge-error-catalogue/SKILL.md @@ -14,7 +14,8 @@ description: >- schema errors, meson source-tree include / sanity-check breakage, duplicated patch hunks, iOS flatc MACOSX_BUNDLE, .dylib-vs-.so staging, libc++_shared, config.sub apple-ios, ctypes "Unable to find ... shared - library", PT_LOAD 16KB alignment, lazy_loader "non-existent stub" crashes + library", a green iOS build that silently ships another project's site-packages + (headerpad), PT_LOAD 16KB alignment, lazy_loader "non-existent stub" crashes (stripped *.pyi), the Flet 0.86 Android sitepackages.zip class (NotADirectoryError on a bundled data file -> extract_packages, untagged native .so ModuleNotFoundError -> forge ABI-tag, ctypes-by-__file__ loaders -> find_spec / @@ -120,7 +121,12 @@ instead of re-deriving it. of interdependent bundled dylibs (pyarrow, llama)** → **serious_python #223**: reconcile framework install-ids + `@rpath` deps to the dotted-framework paths (`reconcile_framework_install_names` in darwin scripts; needs an sp release for CI; - local sp fix cc28d13 verified pyarrow 4/4 on-sim). + local sp fix cc28d13 verified pyarrow 4/4 on-sim). **And its follow-on:** that same + reconcile pass FAILS on an extension linked without header padding + (`larger updated load commands do not fit`), which aborts the sync and leaves + `flet build` reporting success while shipping the shared pub-cache `dist_ios`'s + packages from some *other* project → **`-Wl,-headerpad_max_install_names`** on the + iOS lanes of any non-CMake recipe binding several bundled dylibs (av). - **Recipe-tester app failures** — host-build (`pg_config` etc.), pypi.flet.dev index precedence, "no matching distribution" (incl. **ios-simulator also resolving the iphoneos wheel**), **sdist-only pure-python dep → pip backtrack** diff --git a/.claude/skills/forge-error-catalogue/references/failure-catalogue.md b/.claude/skills/forge-error-catalogue/references/failure-catalogue.md index 60f89bd8..928ef3cd 100644 --- a/.claude/skills/forge-error-catalogue/references/failure-catalogue.md +++ b/.claude/skills/forge-error-catalogue/references/failure-catalogue.md @@ -330,6 +330,46 @@ flet build ios-simulator --python-version 3.12`) not `uvx` (else --- +### iOS: `flet build` reports SUCCESS but the `.app` carries **another project's** site-packages (or none) — `reconcile_framework_install_names … larger updated load commands do not fit` + +**Symptom:** the build prints "Successfully built your .app bundle" and exits 0, but +on device the app dies with `ModuleNotFoundError` for the recipe under test. Unpack +the bundle and its Python tree is from a *different* build entirely: + + .app/serious_python_darwin_serious_python_darwin.bundle/site-packages/ + -> bitarray/, msgpack/ … # a previous recipe's packages + -> no /, no _pytest/ + +`.app/Frameworks/` has the stdlib frameworks but none for your extension +modules. Found while validating `av` (49 extension modules × 7 FFmpeg dylibs). + +**Cause:** `install_name_tool` cannot grow a Mach-O load command in place. serious_python +rewrites every `@rpath/libavformat.dylib` to `@rpath/opt.lib.libavformat.framework/opt.lib.libavformat` +— **30 bytes longer** — and setuptools links extensions with no header padding, so it +fails with `changing install names or rpaths can't be redone … because larger updated +load commands do not fit`. `reconcile_framework_install_names` correctly treats that as +fatal (`|| exit 1` in `sync_site_packages.sh`), which aborts the sync **before** it +writes `dist_ios/site-packages`. That directory lives in the plugin's **shared pub +cache** (`~/.pub-cache/hosted/pub.dev/serious_python_darwin-/darwin/dist_ios`), so +SwiftPM then packages whatever the *last successful* build of any project left there. +`flet build` does not propagate the failure, so nothing is red until the app runs. + +**Fix (per-recipe):** link with `-Wl,-headerpad_max_install_names` on the iOS lanes. +For a PEP 517 / setuptools recipe that is `build.script_env.LDFLAGS` (forge appends to +LDFLAGS, and distutils' `customize_compiler` appends it to `LDSHARED`); for a `build.sh` +native lib it goes in the link flags the upstream build system takes (`--extra-ldflags` +for FFmpeg's configure). CMake recipes add it themselves on Apple platforms, which is +why pyarrow never hit this. Rule of thumb: **a non-CMake iOS recipe whose extensions +link more than a couple of bundled dylibs needs the flag.** + +**Confirming it is this and not something else:** grep the `-vv` build log for +`reconcile_framework_install_names` — the failure is printed there in full, tens of +thousands of lines above the cheerful success banner. And after any iOS build, check +`ls .app/serious_python_darwin_serious_python_darwin.bundle/site-packages` really +contains your package before trusting a green build. + +--- + ### iOS: app builds + launches but **SIGSEGV at launch** in `dyld … runInitializers`, 0-byte `console.log` — an extension runs the Python C-API in a C++ **static initializer** **Symptom:** same 0-byte `console.log` / crash-before-Python signature as the #223 diff --git a/.claude/skills/local-recipe-testing/SKILL.md b/.claude/skills/local-recipe-testing/SKILL.md index 790b4cf6..091ae095 100644 --- a/.claude/skills/local-recipe-testing/SKILL.md +++ b/.claude/skills/local-recipe-testing/SKILL.md @@ -1,6 +1,6 @@ --- name: local-recipe-testing -description: Run a mobile-forge recipe's wheel ON-DEVICE locally — Android emulator and/or iOS simulator — instead of waiting ~1 hour for a CI mobile-test cycle. Covers the recipe-tester app loop (build wheel → stage → flet build → install → read console.log), and the non-obvious gotchas that each cost a wasted cycle: use forge's stripped dist/ wheel, build the recipe against the SAME Python flet bundles, clear flet's build cache between rebuilds, use a rootable (google_apis, not playstore) arm64 AVD to read the app-private console.log (it's in the app's cache/ dir), give the emulator enough RAM/disk, build ALL THREE iOS slices before `flet build ios-simulator`, use explicit simulator UDIDs when more than one sim is booted, and verify the staged-test COUNT so a silently-failed staging can't replay stale tests as false passes. Also covers forge slice syntax, bundling model assets next to recipe tests, test-only deps via the meta.yaml test.requires field, desktop pre-validation via a sys.modules alias shim, and consumer verify-apps for beyond-pytest validation. USE THIS SKILL when iterating on a recipe's on-device behaviour (import works? functions run? crashes?), reproducing or debugging a CI mobile-test failure locally, or whenever someone says the CI mobile test is too slow to iterate on. Sibling of `new-mobile-recipe` (authoring), `forge-ci` (CI runs), `forge-error-catalogue` (build errors), and `native-recipe-bumps` (version bumps); this one is specifically the fast on-device validation loop. macOS + Apple Silicon assumed (the host this was developed on). +description: Run a mobile-forge recipe's wheel ON-DEVICE locally — Android emulator and/or iOS simulator — instead of waiting ~1 hour for a CI mobile-test cycle. Covers the recipe-tester app loop (build wheel → stage → flet build → install → read console.log), and the non-obvious gotchas that each cost a wasted cycle: use forge's stripped dist/ wheel, build the recipe against the SAME Python flet bundles, clear flet's build cache between rebuilds, use a rootable (google_apis, not playstore) arm64 AVD to read the app-private console.log (it's in the app's cache/ dir), give the emulator enough RAM/disk, build ALL THREE iOS slices before `flet build ios-simulator`, use explicit simulator UDIDs when more than one sim is booted, verify the staged-test COUNT so a silently-failed staging can't replay stale tests as false passes, and check the built iOS `.app` actually carries your package (a failed site-packages sync still exits 0). Also covers forge slice syntax, bundling model assets next to recipe tests, test-only deps via the meta.yaml test.requires field, desktop pre-validation via a sys.modules alias shim, and consumer verify-apps for beyond-pytest validation. USE THIS SKILL when iterating on a recipe's on-device behaviour (import works? functions run? crashes?), reproducing or debugging a CI mobile-test failure locally, or whenever someone says the CI mobile test is too slow to iterate on. Sibling of `new-mobile-recipe` (authoring), `forge-ci` (CI runs), `forge-error-catalogue` (build errors), and `native-recipe-bumps` (version bumps); this one is specifically the fast on-device validation loop. macOS + Apple Silicon assumed (the host this was developed on). --- # Testing a mobile-forge recipe locally @@ -37,8 +37,6 @@ cp dist/-*-android_24_arm64_v8a.whl /tmp/rt_dist/ # forge's dist/ whee ./tests/recipe-tester/stage_recipe.sh # 3. Clear flet's stale bundle (gotcha #3), then build the app. -# The recipe-tester targets Flet 0.86 (only there since flet#104), which is NOT -# on PyPI yet — pull it from pypi.flet.dev and pin the prerelease (gotcha #13). rm -rf tests/recipe-tester/build/site-packages tests/recipe-tester/build/.hash cd tests/recipe-tester PIP_FIND_LINKS=/tmp/rt_dist \ @@ -76,7 +74,7 @@ rm -rf tests/recipe-tester/build/site-packages tests/recipe-tester/build/.hash cd tests/recipe-tester PIP_FIND_LINKS="$(realpath ../../dist)" \ uvx --prerelease allow --with 'flet-cli' --with 'flet' \ - flet build ios-simulator --yes --python-version 3.12 # 0.86 pin — gotcha #13 + flet build ios-simulator --yes --python-version 3.12 # 0.86+ — gotcha #13 # 3. Boot any available iPhone sim, install, launch — ALWAYS by explicit UDID # (gotcha #11: `booted` is ambiguous the moment two sims are booted) @@ -87,7 +85,9 @@ xcrun simctl install "$UDID" build/ios-simulator/recipe-tester.app xcrun simctl launch "$UDID" com.flet.recipe-tester # NB bundle id: iOS uses a DASH (com.flet.recipe-tester); android package an UNDERSCORE (com.flet.recipe_tester) -# 4. Poll for the sentinel — the container is host-readable, no fixed sleep needed +# 4. Confirm the bundle really carries your package (gotcha #14), then poll for the +# sentinel — the container is host-readable, no fixed sleep needed +ls build/ios-simulator/recipe-tester.app/serious_python_darwin_serious_python_darwin.bundle/site-packages DATA=$(xcrun simctl get_app_container "$UDID" com.flet.recipe-tester data) for i in $(seq 1 30); do grep EXIT "$DATA/Library/Caches/console.log" 2>/dev/null && break; sleep 5; done ``` @@ -149,6 +149,14 @@ for i in $(seq 1 30); do grep EXIT "$DATA/Library/Caches/console.log" 2>/dev/nul rule: **a search that returns a surprising negative is a bug until proven otherwise** — spot-check one case you are certain about before reporting the absence as evidence. +13. **`--python-version` only exists in flet-cli 0.86+, and `uvx` can hand you 0.85.** + +14. **After an iOS build, check the `.app`'s bundled site-packages actually contains your package.** `flet build ios-simulator` reports success and exits 0 even when serious_python's site-packages sync **aborted**, because the failure is not propagated. The plugin's `dist_ios` lives in the shared pub cache, so the SwiftPM resource bundle then ships whatever the last *successful* build of any project left there — an app carrying a different recipe's packages entirely, which on device is an ordinary-looking `ModuleNotFoundError`. One line, worth it every time: + ```bash + ls build/ios-simulator/.app/serious_python_darwin_serious_python_darwin.bundle/site-packages + ``` + The known cause is an extension linked without `-Wl,-headerpad_max_install_names` (see the `forge-error-catalogue` skill), but the check is cheap and catches the whole class. The Android twin is gotcha #12's `unzip -l build/apk/…`. + ## Model assets & test-only deps `stage_recipe.sh` copies **every** file in `recipes//tests/` into the app (`cp -r tests/. recipe_tests/`), so a model dropped next to the test file becomes an app asset. Two tiers: diff --git a/.claude/skills/new-mobile-recipe/SKILL.md b/.claude/skills/new-mobile-recipe/SKILL.md index 715ee577..20cd0d95 100644 --- a/.claude/skills/new-mobile-recipe/SKILL.md +++ b/.claude/skills/new-mobile-recipe/SKILL.md @@ -82,6 +82,7 @@ Match the package to one of these shapes. Each maps to a template in `templates/ | C-ext that links a lib via a `*-config` tool | Compiled C-ext whose `setup.py` shells out to `pg_config`/`mysql_config`/… (psycopg2→libpq, mysqlclient→libmysqlclient) | A **static+PIC** `flet-lib*` (`build-flet-lib.sh` + `-fPIC`) shipping a config-shim, + consumer `script_env`/patch; see Pattern I | | **setup.py that drives CMake itself** for a vendored native lib | An sdist that vendors a C library and builds it with its own `subprocess` CMake call inside `build_ext`, then links the static result via `extra_objects` (pycares→c-ares). Not scikit-build-core — the arg list is hardcoded in `setup.py` | No template — copy `recipes/pycares/`: one patch appends `shlex.split(os.environ['FORGE_CMAKE_ARGS'])` to the arg list, `requirements.build: [cmake]`; see "vendored-CMake" deep-dive below | | CMake giant, **no sdist AND no setup.py/pyproject.toml** | Upstream's only wheel path is a host==target build script (onnxruntime's `ci_build/build.py`, TF's `build_pip_package_with_cmake.sh`) | No template — copy from `recipes/onnxruntime/` or `recipes/tflite-runtime/` (branches `machine/onnxruntime` / `machine/tflite-runtime`); see "PEP 517 shim" deep-dive below | +| C-ext linking a `flet-lib*`'s **shared** libs via `pkg-config` | Upstream's `setup.py` runs `pkg-config --cflags --libs …` and the package has too many extension modules to static-link into (PyAV: 49) | No template — copy `recipes/flet-libffmpeg/` + `recipes/av/`: the flet-lib ships SHARED libs plus **relocatable** `.pc` files (forge already has `opt/lib/pkgconfig` on `PKG_CONFIG_LIBDIR`, so the consumer needs no `script_env` for it), and the consumer adds `-Wl,-headerpad_max_install_names` on iOS. See "Cross-cutting conventions" in `references/recipe-patterns.md` | | **Prebuilt-repackage + host_build chain** | Upstream publishes official prebuilt mobile archives of the native lib AND the consumer's own cmake links + re-ships the `.so` (flet-libonnxruntime→sherpa-onnx) | `build.sh` repackager + consumer `requirements.host_build`; copy from `recipes/flet-libonnxruntime/` + `recipes/sherpa-onnx/` (branch `machine/sherpa-onnx`); see "prebuilt-repackage" deep-dive below | | **cffi/ctypes consumer of a prebuilt archive** | The package's own build DOWNLOADS a prebuilt static lib keyed by the build host's `uname` and statically links it (curl-cffi → curl-impersonate); breaks under cross because host≠target | `flet-lib*` prebuilt-repackage dep (`source.strip: 0` for root-level tarballs) + `requirements.host_build` + a `mobile.patch` opt-in env lever that steers arch/link off the forge target; copy from `recipes/flet-libcurl-impersonate/` + `recipes/curl-cffi/` (branch `curl-cffi`); see deep-dive below | diff --git a/.claude/skills/new-mobile-recipe/references/recipe-patterns.md b/.claude/skills/new-mobile-recipe/references/recipe-patterns.md index 42bba64d..5035ca25 100644 --- a/.claude/skills/new-mobile-recipe/references/recipe-patterns.md +++ b/.claude/skills/new-mobile-recipe/references/recipe-patterns.md @@ -692,6 +692,51 @@ and `scipy` are the fixed precedents. Getting it wrong the safe way costs a download; getting it wrong the unsafe way — moving a lib something actually `dlopen`s — is a runtime crash on device, so verify per consumer. +### A `flet-lib*` consumed through `pkg-config` needs relocatable `.pc` files + +forge already puts `/opt/lib/pkgconfig` on both `PKG_CONFIG_PATH` +and `PKG_CONFIG_LIBDIR` (`compile_env()` in `build.py`), so a consumer whose `setup.py` +shells out to `pkg-config` needs **no `script_env` at all** — provided the `flet-lib*` +ships its `.pc` files and they are relocatable. Most build systems bake absolute paths +from the staging prefix into all three of `prefix=`, `libdir=` and `includedir=` +(FFmpeg writes the latter two out in full rather than deriving them), and that prefix +no longer exists when the wheel is unpacked. Rewrite them in `build.sh`: + +```bash +for pc in "$PREFIX"/lib/pkgconfig/*.pc; do + sed -E -e 's|^prefix=.*|prefix=${pcfiledir}/../..|' \ + -e 's|^libdir=.*|libdir=${prefix}/lib|' \ + -e 's|^includedir=.*|includedir=${prefix}/include|' \ + "$pc" > "$pc.tmp" && mv "$pc.tmp" "$pc" +done +``` + +Note this contradicts the usual `flet-lib*` cleanup advice to delete `lib/pkgconfig` — +delete it only when nothing downstream reads it. **Real example:** +`recipes/flet-libffmpeg/` → `recipes/av/`. + +### iOS: extensions binding several bundled dylibs need `-Wl,-headerpad_max_install_names` + +serious_python relocates every bundled dylib into a framework and rewrites each +`@rpath/lib.dylib` reference to the much longer +`@rpath/opt.lib.lib.framework/opt.lib.lib`. `install_name_tool` cannot grow a +load command that has no padding, and the resulting failure aborts the sync **while +`flet build` still reports success** — see the `forge-error-catalogue` skill for the +full symptom. CMake adds the flag itself on Apple platforms; setuptools and plain +`configure` do not, so pass it explicitly on the iOS lanes: + +```yaml +build: +# {% if sdk != 'android' %} + script_env: + LDFLAGS: -Wl,-headerpad_max_install_names +# {% endif %} +``` + +and, for the `flet-lib*` underneath, in whatever the upstream build calls its link +flags (`--extra-ldflags` for FFmpeg). **Real example:** `recipes/av/` + +`recipes/flet-libffmpeg/`. + ### Version specifiers in requirements Forge parses `requirements.host` / `requirements.build` strings per `build.py:67-78`: diff --git a/recipes/av/README.md b/recipes/av/README.md new file mode 100644 index 00000000..2a77b6aa --- /dev/null +++ b/recipes/av/README.md @@ -0,0 +1,310 @@ +# av + +[`av`](https://pyav.basswood.io) (PyAV) is a Cython binding to FFmpeg's libraries — containers, +streams, packets, codecs and frames as Python objects, with the pixel and sample data reachable +in place. On mobile that is the only way to get at them: you cannot ship an `ffmpeg` binary in +an app bundle, and neither iOS nor Android will let you spawn one. The wheel carries its own +FFmpeg, so what your app can decode is fixed by the wheel and not by the OS underneath it. + +## Install + +```toml +# pyproject.toml +dependencies = [ + "flet", + "av", +] +``` + +**iOS needs Flet 0.86.0 or newer.** The FFmpeg libraries ship as dylibs that flet relocates +into the app's frameworks, and reconciling the `@rpath` references between them landed in +serious_python 4.3.2 — the version Flet 0.86.0 is the first to pin (0.85.2 pins 1.0.0, which +has no such pass). Below that the app dies at launch, before Python starts, with +`dyld: Library not loaded: @rpath/libavutil.dylib`. A bare `flet` resolves to the latest +release, so this only bites if something in your project holds Flet back. + +This is one of the larger wheels on the index — see [App size](#app-size) before adding it to +an app that has a download-size budget. + +## Examples + +See runnable Flet apps in [`examples/`](examples): + +- [`clip-roundtrip`](examples/clip-roundtrip) — writes an MP4, reopens it, and puts the frames + it reads back on screen. + +## Usage in a Flet app + +Open a file, decode a frame, and re-encode it as a JPEG for +[`ft.Image`](https://flet.dev/docs/controls/image/), which takes encoded bytes rather than raw +pixels: + +```python +import io +import os + +import av +import flet as ft + + +def first_frame(path: str, width: int = 320) -> bytes: + """Decode the first video frame of `path` and return it as JPEG bytes.""" + with av.open(path) as container: + frame = next(container.decode(video=0)) + + height = width * frame.height // frame.width + scaled = frame.reformat(width=width, height=height, format="yuvj420p") + + buffer = io.BytesIO() + with av.open(buffer, mode="w", format="mjpeg") as out: + stream = out.add_stream("mjpeg", rate=1) + stream.width, stream.height, stream.pix_fmt = width, height, "yuvj420p" + out.mux(stream.encode(scaled)) + out.mux(stream.encode(None)) + return buffer.getvalue() + + +clip = os.path.join(os.getenv("FLET_APP_STORAGE_DATA", "."), "clip.mp4") +page.add(ft.Image(src=first_frame(clip), width=320)) +``` + +### Storage + +Media files are large and slow to produce, so where you put them matters more than usual. +Anything the user would be upset to lose — a recording, an import, a rendered export — belongs +in [`FLET_APP_STORAGE_DATA`](https://flet.dev/docs/reference/environment-variables/#flet_app_storage_data), +which is app-private, never auto-deleted and included in backups. Put transcodes and thumbnails +you can regenerate in +[`FLET_APP_STORAGE_CACHE`](https://flet.dev/docs/reference/environment-variables/#flet_app_storage_cache) +instead — the OS may purge it under storage pressure, which for derived files is the behaviour +you want. Never write anything you intend to reopen to +[`FLET_APP_STORAGE_TEMP`](https://flet.dev/docs/reference/environment-variables/#flet_app_storage_temp); +it can vanish between launches. + +`av.open()` also takes any file-like object, so a clip you are only going to process once never +has to touch the filesystem at all. + +### Media over the network + +**There is no `https` protocol in this build**, and no `tls`, `rtmps` or encrypted HLS with it. +`av.open("https://…")` raises +[`av.error.ProtocolNotFoundError`](https://pyav.basswood.io/docs/stable/api/error.html). FFmpeg needs a TLS +backend for those, which would mean a second copy of OpenSSL inside the app. + +Fetch the bytes with a Python HTTP client and hand PyAV a file-like object instead — the same +`av.open` call, and it goes through your app's own TLS stack and certificate store. `httpx` is +already there, as one of Flet's own dependencies: + +```python +import io + +import httpx + +with av.open(io.BytesIO(httpx.get(url).content)) as container: + ... +``` + +A `BytesIO` is seekable, so `container.seek(...)` and formats that need to read a trailing +index (MP4 with the `moov` atom at the end) work normally. Wrapping a streaming response +directly gives you a non-seekable file object, which FFmpeg accepts but can only read +forwards — fine for a progressive download, not for a seek bar. + +Plain `http://`, `tcp`, `udp`, `rtp` and `rtmp` are compiled in, so an RTSP camera or an +`http://` stream on a local network opens directly. + +### Showing and playing what you decode + +PyAV decodes; it does not display. A decoded `VideoFrame` reaches the screen the way the +snippet above does it — re-encoded as JPEG or PNG bytes and handed to +[`ft.Image`](https://flet.dev/docs/controls/image/). That is fine for thumbnails, stills and +frame-by-frame inspection, and it is not a video player: every frame costs an encode, a +control update and a decode by the Flutter side. + +For actual playback, leave the file to Flet's own player controls — `Video` from +[`flet-video`](https://pypi.org/project/flet-video/) and `Audio` from +[`flet-audio`](https://pypi.org/project/flet-audio/), separate packages you add to +`dependencies` alongside `flet`. They wrap the platform's player, which is hardware-accelerated +where this build is not. The natural division of labour is PyAV for anything that reads or +rewrites the media — probing, trimming, remuxing, extracting frames or audio — and a Flet +control for playing the result. + +### Threading + +PyAV releases the GIL around demuxing, decoding and encoding, so this work genuinely runs in +parallel with the UI. Hand it to +[`page.run_thread(...)`](https://flet.dev/docs/controls/page/#flet.Page.run_thread) — a +half-second decode on the event-handler thread is a visibly frozen app. `run_thread` swallows +exceptions and does not carry an automatic update with it, so catch your own failures and +finish the handler with an explicit +[`page.update()`](https://flet.dev/docs/controls/page/#flet.Page.update). + +A single container is not safe to use from two threads at once. Give each thread its own +`av.open(...)`, or serialise access with a `threading.Lock`. + +### What this build can encode + +Decoding is broad: H.264, HEVC, AV1, VP8, VP9, ProRes, MJPEG, PNG, GIF, WebP, AAC, MP3, Opus, +Vorbis and FLAC are all present, in MP4/MOV, Matroska/WebM, MPEG-TS, HLS, WAV, OGG and the rest +of FFmpeg's demuxers. + +Encoding is narrower, and that is a licence consequence rather than a build oversight. The +video encoders for H.264, HEVC, VP8, VP9 and AV1 all come from external libraries — x264 and +x265 are GPL, and the BSD-licensed ones (libvpx, libaom, SVT-AV1) are not built here — so what +you get is FFmpeg's own: **`mpeg4`, `mjpeg`, `png`, `gif`, `prores` and `ffv1`** for video, and +**`aac`, `opus`, `vorbis`, `flac`, `alac` and PCM** for audio. There is no MP3 encoder either; +that one is LAME. + +In practice: remuxing an H.264 clip into another container is free (the packets are copied, not +re-encoded), extracting stills and audio works, and re-encoding video to H.264 does not. If you +need an H.264 file out of a Flet app, encode it on a server, or capture with the platform's own +recorder. + +No hardware acceleration is compiled in on either platform — no VideoToolbox, no MediaCodec — +so everything decodes on the CPU. A 1080p H.264 decode is comfortable on a modern phone; 4K is +not, and neither is real-time software encoding of anything large. + +`libavdevice` is linked and `av.device` works, but no capture devices are registered: +`av.enumerate_input_devices("avfoundation")` raises `ValueError`. Capture with +[`flet-camera`](https://pypi.org/project/flet-camera/) or +[`flet-audio-recorder`](https://pypi.org/project/flet-audio-recorder/) and hand the file they +produce to PyAV. + +### App size + +The wheels add approximately **12 MB compressed and 27 MB unpacked per architecture** — +roughly 21 MB of that is the FFmpeg libraries and 4 MB PyAV's own extension modules. On +Android that is per ABI, so ship an app bundle or split APKs, or narrow +[`target_arch`](https://flet.dev/docs/publish/android/#supported-target-architectures) to the +ABIs you actually support, rather than putting three copies in one universal APK. These are +package figures; what reaches a user's download depends on packaging and compression. + +### Other considerations + +A desktop `flet run` resolves PyPI's own `av` wheel, which is built against a **different** +FFmpeg: it has x264, x265, libvpx, SVT-AV1, dav1d, LAME, WebP and both Apple hardware +frameworks, plus TLS and capture devices. So `add_stream("libx264")`, `av.open("https://…")`, +`format="lavfi"` and camera enumeration all succeed on your Mac and fail on device. Anything +that names a codec, a protocol or a device has to be validated on a device or +emulator/simulator. + +## Things to know + +- **A decoded frame is much bigger than the file it came from.** One 1080p `yuv420p` frame is + about 3 MB; hold a few dozen and a phone will kill the app. Decode in a loop and let each + frame go, and if you need to keep frames, keep them re-encoded (JPEG/PNG bytes) rather than + as `VideoFrame` objects. +- **`seek` lands on a keyframe, not on your timestamp.** `container.seek(t)` positions to the + nearest preceding keyframe, so the first frame you decode after it can be a second or more + early. Decode forward to the frame you actually wanted, comparing `frame.time`. +- **`av.datasets` has nowhere to write.** Its fixture helpers cache downloads under + `/usr/local/share/pyav/datasets` and similar system paths, none of them writable from an app + sandbox, so they raise rather than download. Point `PYAV_TESTDATA_DIR` at app storage if you + want them, or bundle sample media in your app's `src/assets/` — which is what an app that + ships fixtures should do anyway. +- **`format="lavfi"` is unavailable.** The `lavfi` virtual input device is part of + `libavdevice`, which registers nothing here. Build the graph directly with + [`av.filter.Graph`](https://pyav.basswood.io/docs/stable/api/filter.html) instead; all 400-odd + filters are compiled in. +- **Subtitle character-set conversion is off.** `iconv` is unavailable on Android before API 28, + so it is disabled on both platforms for symmetry; the `sub_charenc` option has no effect. + UTF-8 subtitles are unaffected. +- **Licensing:** `av` itself is + [BSD-3-Clause](https://spdx.org/licenses/BSD-3-Clause.html), but the wheel is useless without + the FFmpeg libraries behind it, and those come from the separate + [`flet-libffmpeg`](../flet-libffmpeg) wheel, which declares + [LGPL-2.1-or-later](https://spdx.org/licenses/LGPL-2.1-or-later.html) `AND` + [IJG](https://spdx.org/licenses/IJG.html). Two things follow, and only the second asks + anything of you: + + - **The LGPL part.** No `--enable-gpl` and no `--enable-nonfree`, which is why the GPL-only + encoders above are missing. The libraries ship **shared** and are loaded dynamically, so + the LGPL's relinking condition is satisfied by the shape of the build; a closed-source app + carries the notice and says it uses FFmpeg under the LGPL. + - **The IJG part is an action.** libavcodec's JPEG DCT routines come from the Independent + JPEG Group, and that grant's condition (2) says that if you distribute **only executable + code** — which every published app does — the accompanying documentation must state that + the software is *"based in part on the work of the Independent JPEG Group"*. Put that + sentence wherever your app credits its third-party code: an about screen, a licences page, + the store listing. It applies whether or not you touch MJPEG, because the code is compiled + in either way. + + Both wheels ship their licence text; read it off the artifact rather than from a summary + elsewhere: + + ```bash + unzip -p '*.dist-info/METADATA' | grep -i license + unzip -l | grep licenses/ + ``` + + This is a flag, not legal advice. + +## Build notes (maintainers) + +### Recipe shape + +Two recipes: [`flet-libffmpeg`](../flet-libffmpeg) builds the seven `libav*` libraries shared, +and this one compiles PyAV's 49 Cython extension modules against them. It is the +`flet-libarrow` → `pyarrow` chain, with the same per-platform delivery — jniLibs on Android, +per-slice `*.fwork` frameworks on iOS plus an `__init__` preload shim. + +Shared is not a preference. PyAV's `setup.py` refuses a static FFmpeg outright ("Building PyAV +against static FFmpeg libraries is not supported"), and 49 extension modules each statically +linking a 13 MB `libavcodec` is not a wheel anyone would ship. It also means `setup.py` finds +FFmpeg *only* through `pkg-config`, which is why `flet-libffmpeg` ships relocatable `.pc` files +— forge puts `/opt/lib/pkgconfig` on `PKG_CONFIG_LIBDIR` and nothing else is +needed. + +The wheel is `abi3` (PyAV asks for `Py_LIMITED_API` on 3.12 and 3.13), so the modules are named +`*.abi3.so`. forge and serious_python both recognise that spelling; forge still re-tags the +wheel per Python version. + +**`-Wl,-headerpad_max_install_names` on iOS is load-bearing, and its absence fails silently.** +An iOS build creates 56 frameworks for this app — 49 extension modules and the 7 dylibs — and +serious_python then rewrites every `@rpath/libavformat.dylib` reference to +`@rpath/opt.lib.libavformat.framework/opt.lib.libavformat`, which is 30 bytes longer. With no +header padding `install_name_tool` refuses ("larger updated load commands do not fit"), +`reconcile_framework_install_names` aborts the sync, and `flet build` **still reports success** +— having produced an app whose site-packages are whatever the plugin's shared `dist_ios` +happened to hold from a previous build. Observed as an app carrying another recipe's packages +and no `av` at all. Both recipes therefore pass the flag: this one through `script_env` +`LDFLAGS`, `flet-libffmpeg` through `--extra-ldflags`. Any change that lengthens those +framework names needs it re-checked. + +### Upgrade hazards + +- **PyAV and FFmpeg move as a pair.** Each PyAV release supports one FFmpeg major line + (18.x is 8.x) and its `.pxd` headers track that ABI. Bumping one means bumping the other and + re-pinning `requirements.host` in `meta.yaml`; a floating `>=` there would let a future ABI + break resolve silently. +- **The encoder list is a licence boundary, not a snapshot.** If a future bump appears to add + H.264 encoding, something turned on `--enable-gpl` — check before believing it, because the + wheel's declared LGPL metadata would then be wrong. +- **The Flet floor in Install is about serious_python, not about PyAV.** It moves only if the + iOS framework-reconciliation contract changes. + +### Re-verification checklist + +- `tests/test_av.py::test_build_is_lgpl_only` asserts every library reports LGPL and that no + GPL/nonfree codec is linked; `test_no_capture_devices_registered` pins `--disable-devices`. + Both turn red on their own if the build configuration drifts, which is what they are for. +- The decoder/encoder names this page promises are asserted by + `test_expected_codecs_available`. Add to that list whenever the page starts promising more. +- `test_no_tls_protocols` pins the absence of `https`. The rest of the protocol list is not + asserted; read it off a device if the configure flags change, with + `av._core.library_meta["libavformat"]["configuration"]`. +- Re-measure the size figures rather than adjusting them by eye, and quote decimal MB. They + move with FFmpeg's codec tables on every bump. +- `flet-libffmpeg`'s `build.sh` extracts the IJG notice out of `libavcodec/jrevdct.c` at build + time and fails the build if it is not there, so an upstream move cannot silently drop it. + What that guard cannot see is the notice **spreading** — re-check on a bump that + `LICENSE.md` still carves out only those three files, since a new one would make the + `AND IJG` expression incomplete rather than wrong. + +### Coverage gaps + +The device tests exercise muxing, encoding, decoding, seeking through a `BytesIO`, swscale, +swresample, avfilter and the bitstream-filter registry — all on synthetic media generated in +the test. Nothing decodes a real-world H.264 or HEVC file, so a codec that is *present* but +broken on a given architecture would not be caught. Nothing touches the network protocols. +Nothing measures decode throughput, so the performance claims in **What this build can encode** +are judgement, not measurement. diff --git a/recipes/av/examples/clip-roundtrip/.gitignore b/recipes/av/examples/clip-roundtrip/.gitignore new file mode 100644 index 00000000..bf044c4f --- /dev/null +++ b/recipes/av/examples/clip-roundtrip/.gitignore @@ -0,0 +1,8 @@ +.venv/ +.flet/ +build/ +__pycache__/ +.pytest_cache/ +.ruff_cache/ +uv.lock +*.mp4 diff --git a/recipes/av/examples/clip-roundtrip/README.md b/recipes/av/examples/clip-roundtrip/README.md new file mode 100644 index 00000000..23de0252 --- /dev/null +++ b/recipes/av/examples/clip-roundtrip/README.md @@ -0,0 +1,39 @@ +# av clip roundtrip + +A one-screen app that writes a three-second MP4 into app storage with +[PyAV](https://pyav.basswood.io), reopens it to report what the container actually holds, and +seeks to four points to pull the frames back out as JPEGs. + +What it demonstrates: + +- **Muxing two streams** — synthetic video frames encoded with `mpeg4` and a 440 Hz tone + encoded with FFmpeg's own `aac`, written into one MP4 in + [`FLET_APP_STORAGE_DATA`](https://flet.dev/docs/reference/environment-variables/#flet_app_storage_data). +- **Building frames without numpy or Pillow** — `av.VideoFrame` planes are filled a row at a + time, padded to each plane's `line_size`, which is what + [`Plane.update`](https://pyav.basswood.io/docs/stable/api/video.html) expects. +- **Reading the file back** — container format, duration and per-stream codec, resolution and + sample rate, all read off the reopened container rather than remembered from the write. +- **Getting a decoded frame onto the screen** — [`ft.Image`](https://flet.dev/docs/controls/image/) + takes encoded bytes, not raw pixels, so each still is re-encoded as a JPEG through the + `mjpeg` muxer. The timestamps under the strip are the frames PyAV actually landed on: + `seek` moves to the nearest keyframe, so they are not evenly spaced. +- **Work off the UI thread** — the encode runs in + [`page.run_thread(...)`](https://flet.dev/docs/controls/page/#flet.Page.run_thread), which + needs an explicit [`page.update()`](https://flet.dev/docs/controls/page/#flet.Page.update) + at the end. + +## Try it + +[Build](https://flet.dev/docs/publish/) the app, then install it on a device or emulator/simulator: + +```bash +# Android +uv run flet build apk + +# iOS +uv run flet build ipa + +# iOS-Simulator +uv run flet build ios-simulator +``` diff --git a/recipes/av/examples/clip-roundtrip/pyproject.toml b/recipes/av/examples/clip-roundtrip/pyproject.toml new file mode 100644 index 00000000..43fd3013 --- /dev/null +++ b/recipes/av/examples/clip-roundtrip/pyproject.toml @@ -0,0 +1,16 @@ +[project] +name = "av-clip-roundtrip" +version = "1.0.0" +description = "Encode an MP4 with PyAV, read it back, and show the frames." +requires-python = ">=3.11" + +dependencies = [ + "flet==0.86.5", + "av==18.1.0", +] + +[dependency-groups] +dev = ["flet-cli", "flet-desktop", "flet-web"] + +[tool.flet.app] +path = "src" diff --git a/recipes/av/examples/clip-roundtrip/src/clip.py b/recipes/av/examples/clip-roundtrip/src/clip.py new file mode 100644 index 00000000..00693b96 --- /dev/null +++ b/recipes/av/examples/clip-roundtrip/src/clip.py @@ -0,0 +1,161 @@ +"""Writes a short MP4 with a video and an audio stream, reads it back to describe +what actually landed in the file, and pulls stills out of it as JPEG bytes an +`ft.Image` can display. +""" + +import io +import math +import os +import time +from fractions import Fraction + +import av + +WIDTH, HEIGHT = 320, 240 +FPS = 24 +SECONDS = 3 +SAMPLE_RATE = 44100 +TONE_HZ = 440 + + +def library_versions() -> str: + """One line naming the FFmpeg build the wheel is linked against.""" + major, minor, micro = av.library_versions["libavcodec"] + return f"PyAV {av.__version__} · FFmpeg {av.ffmpeg_version_info} · libavcodec {major}.{minor}.{micro}" + + +def clip_path() -> str: + """Where the clip lives: app-private storage that survives a restart.""" + return os.path.join(os.getenv("FLET_APP_STORAGE_DATA", "."), "clip.mp4") + + +def _video_frame(index: int): + """A yuv420p frame with a gradient background and a bar sweeping across it. + + Planes are filled a row at a time because a plane's `line_size` can exceed + the picture width — FFmpeg pads each row for alignment, and `update()` wants + exactly `buffer_size` bytes. + """ + frame = av.VideoFrame(WIDTH, HEIGHT, "yuv420p") + + bar = int((index / (FPS * SECONDS)) * WIDTH) + luma = bytearray(16 + (x * 180 // WIDTH) for x in range(WIDTH)) + for x in range(bar, min(bar + 24, WIDTH)): + luma[x] = 235 + + for plane_index, plane in enumerate(frame.planes): + rows = plane.buffer_size // plane.line_size + if plane_index == 0: + row = bytes(luma) + b"\x00" * (plane.line_size - WIDTH) + else: + # Chroma: a constant tint that drifts over the clip, so the frames + # are visibly different from one another and not just brighter. + value = 90 + (index * 3) % 80 + row = bytes([value]) * plane.line_size + plane.update(row * rows) + + return frame + + +def _audio_frames(stream): + """A 440 Hz tone, resampled into whatever layout the AAC encoder asked for.""" + resampler = av.AudioResampler( + format=stream.format, layout=stream.layout, rate=SAMPLE_RATE + ) + block = 1024 + for start in range(0, SAMPLE_RATE * SECONDS, block): + frame = av.AudioFrame(format="s16", layout="mono", samples=block) + payload = bytearray() + for n in range(block): + value = int( + 8000 * math.sin(2 * math.pi * TONE_HZ * (start + n) / SAMPLE_RATE) + ) + payload += value.to_bytes(2, "little", signed=True) + frame.planes[0].update(bytes(payload)) + frame.sample_rate = SAMPLE_RATE + yield from resampler.resample(frame) + yield from resampler.resample(None) + + +def write_clip(path: str) -> float: + """Mux a synthetic video + audio clip to `path`; return the seconds it took. + + mpeg4 rather than h264: this wheel's FFmpeg is LGPL, so it carries no x264 + and has no H.264 encoder at all. AAC is FFmpeg's own encoder. + """ + started = time.monotonic() + + with av.open(path, mode="w") as container: + video = container.add_stream("mpeg4", rate=FPS) + video.width, video.height = WIDTH, HEIGHT + video.pix_fmt = "yuv420p" + audio = container.add_stream("aac", rate=SAMPLE_RATE) + + for index in range(FPS * SECONDS): + container.mux(video.encode(_video_frame(index))) + for frame in _audio_frames(audio): + container.mux(audio.encode(frame)) + + container.mux(video.encode(None)) + container.mux(audio.encode(None)) + + return time.monotonic() - started + + +def probe(path: str) -> list[tuple[str, str]]: + """Describe the file as label/value pairs, read back off the container.""" + with av.open(path) as container: + rows = [ + ("container", container.format.long_name), + ("size", f"{os.path.getsize(path) / 1000:.0f} KB"), + ("duration", f"{container.duration / av.time_base:.2f} s"), + ] + for stream in container.streams.video: + rows.append( + ( + "video", + f"{stream.codec_context.name} · {stream.width}×{stream.height}" + f" · {float(stream.average_rate):.0f} fps", + ) + ) + for stream in container.streams.audio: + rows.append( + ( + "audio", + f"{stream.codec_context.name} · {stream.codec_context.sample_rate} Hz" + f" · {stream.codec_context.layout.name}", + ) + ) + return rows + + +def _jpeg(frame, width: int) -> bytes: + """Re-encode one decoded frame as a JPEG, scaled to `width`.""" + height = width * frame.height // frame.width + scaled = frame.reformat(width=width, height=height, format="yuvj420p") + + buffer = io.BytesIO() + with av.open(buffer, mode="w", format="mjpeg") as container: + stream = container.add_stream("mjpeg", rate=1) + stream.width, stream.height = width, height + stream.pix_fmt = "yuvj420p" + stream.time_base = Fraction(1, 1) + container.mux(stream.encode(scaled)) + container.mux(stream.encode(None)) + return buffer.getvalue() + + +def thumbnails(path: str, count: int = 4, width: int = 140) -> list[tuple[str, bytes]]: + """Seek to `count` evenly spaced points and return (timestamp, JPEG) pairs.""" + stills = [] + with av.open(path) as container: + stream = container.streams.video[0] + span = container.duration / av.time_base + + for index in range(count): + seconds = span * index / count + container.seek(int(seconds * av.time_base)) + for frame in container.decode(stream): + stills.append((f"{float(frame.time):.2f}s", _jpeg(frame, width))) + break + return stills diff --git a/recipes/av/examples/clip-roundtrip/src/main.py b/recipes/av/examples/clip-roundtrip/src/main.py new file mode 100644 index 00000000..dee89231 --- /dev/null +++ b/recipes/av/examples/clip-roundtrip/src/main.py @@ -0,0 +1,92 @@ +import flet as ft +from clip import clip_path, library_versions, probe, thumbnails, write_clip + + +def still(label, jpeg): + """One filmstrip cell: a decoded frame above the timestamp it was taken at.""" + return ft.Column( + horizontal_alignment=ft.CrossAxisAlignment.CENTER, + spacing=4, + controls=[ + ft.Image(src=jpeg, width=140, border_radius=6, gapless_playback=True), + ft.Text(label, size=11), + ], + ) + + +def row(label, value): + """One line of the probe readout: label on the left, what was read on the right.""" + return ft.Row( + alignment=ft.MainAxisAlignment.SPACE_BETWEEN, + controls=[ft.Text(label, size=12), ft.Text(value, size=12, selectable=True)], + ) + + +def main(page: ft.Page): + def run(): + """Lock the button, raise the spinner, and hand the work to a thread.""" + button.disabled = True + spinner.visible = True + page.update() + page.run_thread(compute) + + def compute(): + """Write the clip, probe it, extract stills, and update the page. + + run_thread swallows exceptions and does not carry an automatic update + with it, so this catches its own failures and ends with page.update(). + """ + try: + path = clip_path() + elapsed = write_clip(path) + readout.controls = [row(*pair) for pair in probe(path)] + readout.controls.append(row("encoded in", f"{elapsed:.2f} s")) + strip.controls = [still(label, jpeg) for label, jpeg in thumbnails(path)] + except Exception as exc: + readout.controls = [ft.Text(str(exc), size=12, selectable=True)] + button.disabled = False + spinner.visible = False + page.update() + + page.appbar = ft.AppBar(title=ft.Text("clip roundtrip"), center_title=True) + page.add( + ft.SafeArea( + expand=True, + content=ft.Column( + scroll=ft.ScrollMode.AUTO, + controls=[ + ft.Text(library_versions(), size=11), + ft.Text( + "Writes a 3-second MP4 into app storage, reopens it, and " + "seeks to four points to pull the frames below.", + size=11, + ), + strip := ft.Row( + wrap=True, + spacing=8, + run_spacing=8, + alignment=ft.MainAxisAlignment.CENTER, + ), + ft.Row( + controls=[ + button := ft.Button( + "Write a clip", + icon=ft.Icons.MOVIE_CREATION, + on_click=run, + ), + spinner := ft.ProgressRing( + width=20, height=20, visible=False + ), + ] + ), + readout := ft.Column(spacing=2), + ], + ), + ) + ) + + run() + + +if __name__ == "__main__": + ft.run(main) diff --git a/recipes/av/meta.yaml b/recipes/av/meta.yaml new file mode 100644 index 00000000..c995ba69 --- /dev/null +++ b/recipes/av/meta.yaml @@ -0,0 +1,40 @@ +package: + name: av + version: "18.1.0" + +build: + number: 1 +# {% if sdk != 'android' %} + script_env: + # iOS only. Each of the 49 extension modules records seven @rpath refs to the + # FFmpeg dylibs, and flet rewrites every one of them to the much longer + # @rpath/opt.lib.lib.framework/opt.lib.lib when it relocates the + # dylibs into frameworks. setuptools links with no header padding, so + # install_name_tool cannot grow the load commands ("larger updated load + # commands do not fit"). That aborts serious_python's reconcile pass — and + # `flet build` then produces an app with NO site-packages at all rather than + # failing, so the symptom is a silent ModuleNotFoundError on device. + # LDFLAGS is additive in forge's script_env handling, and distutils' + # customize_compiler appends it to LDSHARED. + LDFLAGS: -Wl,-headerpad_max_install_names +# {% endif %} + +patches: + - mobile.patch + +requirements: + host: + # PyAV compiles 49 Cython extension modules that link the seven libav* + # libraries; flet-libffmpeg ships them SHARED in opt/lib. A real runtime + # dep on both platforms, so requirements.host (promoted to Requires-Dist) + # rather than host_build — flet has to install the wheel for the libraries + # to reach the device at all. Android: serious_python copies opt/lib/*.so + # into the APK jniLibs and DT_NEEDED resolves them by basename. iOS: flet + # relocates each opt/lib dylib into its own *.framework, and the __init__ + # preload shim from mobile.patch CDLLs them so the extension modules' + # @rpath links bind by install-name. + # + # Pinned exactly: PyAV tracks the FFmpeg 8.x ABI closely and the two are + # released as a pair, so a floating >= would let a future ABI bump resolve + # into an `av` built against the old headers. + - flet-libffmpeg ==8.1.2 diff --git a/recipes/av/patches/mobile.patch b/recipes/av/patches/mobile.patch new file mode 100644 index 00000000..6bae43fd --- /dev/null +++ b/recipes/av/patches/mobile.patch @@ -0,0 +1,78 @@ +mobile.patch — iOS: preload the FFmpeg dylibs before the first extension import + +flet relocates PyAV's 49 extension modules (av/**/*.so) into per-module +*.framework bundles, but the seven FFmpeg libraries they link via @rpath ship as +plain dylibs in site-packages/opt/lib, and nothing on an extension module's rpath +resolves them. This shim preloads all seven with RTLD_GLOBAL (in dependency +order) before `from av._core import ...` on the first line of the package — +directly from opt/lib, or by following a ".fwork" marker where a +serious_python build has relocated one into a framework (serious-python#223) — +so dyld binds each @rpath reference by the already-loaded image's install-id. +No-op on Android (DT_NEEDED resolves the libs by basename from the APK jniLibs) +and on desktop. + +--- a/av/__init__.py ++++ b/av/__init__.py +@@ -1,3 +1,62 @@ ++# mobile-forge (iOS): the seven FFmpeg libraries this package's extension modules ++# link (@rpath/libavutil.dylib, @rpath/libavcodec.dylib, ...) ship as plain dylibs ++# in site-packages/opt/lib, and nothing on an extension module's rpath resolves ++# them -- so the `av._core` import below would die with "Library not loaded: ++# @rpath/libavutil.dylib". Preload them with RTLD_GLOBAL, in dependency order, ++# before the first extension import: dyld then satisfies every @rpath reference ++# from the already-loaded image's install-id. Where a serious_python build has ++# relocated a dylib into a *.framework it leaves a ".fwork" text pointer in ++# its place (serious-python#223) -- follow that instead. ++# No-op on Android (the libraries are .so in the APK's jniLibs, resolved by ++# DT_NEEDED basename) and on desktop (no opt/lib beside the package). ++def _preload_ffmpeg() -> None: ++ import ctypes ++ import os ++ ++ optlib = os.path.join( ++ os.path.dirname(os.path.dirname(os.path.abspath(__file__))), "opt", "lib" ++ ) ++ for base in ( ++ "libavutil", ++ "libswresample", ++ "libswscale", ++ "libavcodec", ++ "libavformat", ++ "libavfilter", ++ "libavdevice", ++ ): ++ dylib = os.path.join(optlib, base + ".dylib") ++ if os.path.exists(dylib): ++ try: ++ ctypes.CDLL(dylib, mode=ctypes.RTLD_GLOBAL) ++ except OSError: ++ pass ++ continue ++ fwork = os.path.join(optlib, base + ".fwork") ++ if not os.path.exists(fwork): ++ continue ++ with open(fwork) as fp: ++ relative = fp.read().strip() ++ # The pointer is relative to the app bundle, an unknown number of levels ++ # above site-packages; walk up until it resolves. ++ directory = optlib ++ for _ in range(12): ++ directory = os.path.dirname(directory) ++ candidate = os.path.join(directory, relative) ++ if os.path.exists(candidate): ++ try: ++ ctypes.CDLL(candidate, mode=ctypes.RTLD_GLOBAL) ++ except OSError: ++ pass ++ break ++ ++ ++try: ++ _preload_ffmpeg() ++except Exception: ++ pass ++del _preload_ffmpeg ++ + # MUST import the core before anything else in order to initialize the underlying + # library that is being wrapped. + from av._core import time_base, library_versions, ffmpeg_version_info diff --git a/recipes/av/tests/test_av.py b/recipes/av/tests/test_av.py new file mode 100644 index 00000000..ea14d7b3 --- /dev/null +++ b/recipes/av/tests/test_av.py @@ -0,0 +1,233 @@ +import io + + +def _solid_video_frame(av, width, height, luma): + """A yuv420p frame with every plane filled to a constant.""" + frame = av.VideoFrame(width, height, "yuv420p") + for index, plane in enumerate(frame.planes): + plane.update(bytes([luma if index == 0 else 128]) * plane.buffer_size) + return frame + + +def _sine_audio_frame(av, samples, sample_rate): + """A mono s16 frame holding a low-amplitude triangle wave.""" + frame = av.AudioFrame(format="s16", layout="mono", samples=samples) + payload = bytearray() + for n in range(samples): + value = (n % 64) * 256 - 8192 + payload += int(value).to_bytes(2, "little", signed=True) + frame.planes[0].update(bytes(payload)) + frame.sample_rate = sample_rate + return frame + + +def test_import_and_library_versions(): + """The package imports, which means all seven flet-libffmpeg libraries + loaded and every extension module bound to them.""" + import av + + for name in ( + "libavutil", + "libavcodec", + "libavformat", + "libavdevice", + "libavfilter", + "libswscale", + "libswresample", + ): + assert name in av.library_versions, name + assert av.library_versions[name][0] > 0, name + + assert av.ffmpeg_version_info + + +def test_build_is_lgpl_only(): + """flet-libffmpeg is configured without --enable-gpl / --enable-nonfree, so + every library reports an LGPL licence and no GPL-only codec is linked. + Turning either on flips this, and the wheel's declared LGPL-2.1-or-later + metadata would stop describing what it actually contains.""" + from av._core import library_meta + from av.codec.codec import codecs_available + + for name, meta in library_meta.items(): + assert meta["license"].startswith("LGPL"), (name, meta["license"]) + + for name in ("libx264", "libx265", "libxvid", "libfdk_aac"): + assert name not in codecs_available, name + + +def test_expected_codecs_available(): + """The codecs a Flet app is most likely to reach for are compiled in. These + are FFmpeg's own native implementations — no external codec libraries are + linked, so the set is fixed by the recipe rather than by the host.""" + from av.codec.codec import Codec + + for name in ("h264", "hevc", "vp8", "vp9", "aac", "mp3", "opus", "flac"): + assert Codec(name, "r").name == name, name + + for name in ("mpeg4", "mjpeg", "aac", "flac"): + assert Codec(name, "w").name == name, name + + +def test_container_formats_available(): + """libavformat carries the common container muxers/demuxers.""" + import av + + for name in ("mp4", "mov", "matroska", "webm", "wav", "mp3"): + assert name in av.formats_available, name + + +def test_video_encode_decode_roundtrip(): + """Encode synthetic frames into an in-memory MP4 and read them back. + Exercises the muxer, the encoder, the file-like I/O path (av.open on a + BytesIO — the documented way to play a network stream on mobile) and the + decoder in one pass.""" + import av + + width, height, count = 160, 120, 10 + + buffer = io.BytesIO() + with av.open(buffer, mode="w", format="mp4") as container: + stream = container.add_stream("mpeg4", rate=25) + stream.width = width + stream.height = height + stream.pix_fmt = "yuv420p" + for index in range(count): + frame = _solid_video_frame(av, width, height, 16 + index * 8) + container.mux(stream.encode(frame)) + container.mux(stream.encode(None)) + + assert buffer.tell() > 0 + buffer.seek(0) + + decoded = [] + with av.open(buffer) as container: + stream = container.streams.video[0] + assert stream.codec_context.name == "mpeg4" + for frame in container.decode(stream): + decoded.append(frame) + + assert len(decoded) == count + assert decoded[0].width == width + assert decoded[0].height == height + assert decoded[0].format.name == "yuv420p" + + +def test_audio_encode_decode_roundtrip(): + """Encode PCM to AAC in an in-memory MP4 and decode it back. FFmpeg's own + AAC encoder — no libfdk_aac (nonfree) in this build.""" + import av + + sample_rate = 44100 + + buffer = io.BytesIO() + with av.open(buffer, mode="w", format="mp4") as container: + stream = container.add_stream("aac", rate=sample_rate) + resampler = av.AudioResampler( + format=stream.format, layout=stream.layout, rate=sample_rate + ) + for _ in range(8): + frame = _sine_audio_frame(av, 1024, sample_rate) + for resampled in resampler.resample(frame): + container.mux(stream.encode(resampled)) + for resampled in resampler.resample(None): + container.mux(stream.encode(resampled)) + container.mux(stream.encode(None)) + + buffer.seek(0) + + total_samples = 0 + with av.open(buffer) as container: + stream = container.streams.audio[0] + assert stream.codec_context.name == "aac" + assert stream.codec_context.sample_rate == sample_rate + for frame in container.decode(stream): + total_samples += frame.samples + + assert total_samples > 0 + + +def test_video_reformat_uses_swscale(): + """VideoFrame.reformat is libswscale: convert pixel format and scale.""" + import av + + frame = _solid_video_frame(av, 64, 48, 200) + converted = frame.reformat(format="rgb24") + assert converted.format.name == "rgb24" + + scaled = frame.reformat(width=32, height=24, format="yuv420p") + assert (scaled.width, scaled.height) == (32, 24) + + +def test_audio_resampler_uses_swresample(): + """AudioResampler is libswresample: rate + format conversion.""" + import av + + resampler = av.AudioResampler(format="s16", layout="mono", rate=16000) + frame = _sine_audio_frame(av, 1024, 44100) + + out = resampler.resample(frame) + resampler.resample(None) + resampled_samples = sum(f.samples for f in out) + + assert resampled_samples > 0 + assert all(f.sample_rate == 16000 for f in out) + + +def test_filter_graph_uses_avfilter(): + """A buffer -> vflip -> buffersink graph exercises libavfilter end to end.""" + from fractions import Fraction + + import av + + graph = av.filter.Graph() + source = graph.add_buffer( + width=64, height=48, format="yuv420p", time_base=Fraction(1, 25) + ) + flip = graph.add("vflip") + sink = graph.add("buffersink") + source.link_to(flip) + flip.link_to(sink) + graph.configure() + + graph.push(_solid_video_frame(av, 64, 48, 90)) + out = graph.pull() + + assert (out.width, out.height) == (64, 48) + + +def test_bitstream_filters_available(): + """libavcodec's bitstream filters are compiled in — h264_mp4toannexb is the + one anything feeding a platform decoder needs. Instantiate the parameterless + `null` filter rather than that one, which errors without H.264 codec + parameters to work from.""" + import av + + assert "h264_mp4toannexb" in av.bitstream_filters_available + assert av.BitStreamFilterContext("null") is not None + + +def test_no_capture_devices_registered(): + """flet-libffmpeg is built --disable-devices: libavdevice is linked (PyAV + calls avdevice_register_all at import) but registers nothing, so there is no + camera, microphone or `lavfi` source to open. Pinned so that enabling + devices later surfaces as a test change rather than a surprise, and so the + README's account of what this wheel can and cannot do stays honest.""" + import av + import pytest + + for name in ("avfoundation", "android_camera", "v4l2", "lavfi"): + assert name not in av.formats_available, name + with pytest.raises(ValueError): + av.enumerate_input_devices(name) + + +def test_no_tls_protocols(): + """flet-libffmpeg links no TLS backend, so FFmpeg has no `https` protocol — + the reason the README tells consumers to fetch bytes themselves and hand + `av.open` a file-like object. Resolved before any socket is opened, so this + makes no network request.""" + import av + import pytest + + with pytest.raises(av.error.ProtocolNotFoundError): + av.open("https://example.invalid/clip.mp4") diff --git a/recipes/flet-libffmpeg/build.sh b/recipes/flet-libffmpeg/build.sh new file mode 100755 index 00000000..2d613b20 --- /dev/null +++ b/recipes/flet-libffmpeg/build.sh @@ -0,0 +1,186 @@ +#!/bin/bash +# flet-libffmpeg: FFmpeg's seven libav* libraries, built SHARED, for the `av` +# (PyAV) recipe. PyAV compiles 49 Cython extension modules against them and +# refuses a static FFmpeg outright ("Building PyAV against static FFmpeg +# libraries is not supported" — setup.py), so shared is the only shape that +# works; it is also the only one that doesn't fold a copy of libavcodec into +# each of those 49 modules. +# +# Delivery mirrors flet-libarrow -> pyarrow, the proven linked-consumer chain: +# Android: opt/lib/lib.so, unversioned, SONAME == filename. FFmpeg's +# own `android` target already emits exactly that (configure:6040). +# serious_python copies them into the APK's jniLibs, where the +# consumer's DT_NEEDED resolves them by basename. +# iOS: opt/lib/lib.dylib, unversioned, install-id @rpath/lib.dylib. +# flet turns each into a per-slice *.framework; a versioned dylib +# plus its symlinks breaks that conversion (three files match the +# lipo glob), hence the SLIBNAME_WITH_* overrides below. +# +# Licence-relevant: no --enable-gpl, no --enable-nonfree, no --enable-version3, +# and no external codec libraries. This is a plain LGPL-2.1-or-later FFmpeg. +set -eu + +# FFmpeg's configure takes its whole toolchain through its own flags. forge's +# CC/CFLAGS/LDFLAGS are composed for autotools and CMake consumers (they carry +# Python include/lib paths, an -F "..." framework path with literal quotes on +# iOS, ...) and configure would fold them into every probe. Drive it explicitly. +unset CC CXX CFLAGS CPPFLAGS LDFLAGS AR RANLIB STRIP NM || true + +COMMON_ARGS=" + --prefix=$PREFIX + --libdir=$PREFIX/lib + --incdir=$PREFIX/include + --pkgconfigdir=$PREFIX/lib/pkgconfig + --enable-cross-compile + --enable-shared + --disable-static + --enable-pic + --disable-programs + --disable-doc + --disable-debug + --disable-devices + --disable-autodetect + --disable-iconv + --enable-zlib +" +# --disable-autodetect is the deterministic switch: it turns off every library +# FFmpeg would otherwise probe for on the build host or in the SDK, so the +# same source produces the same feature set on a laptop and in CI. It also +# takes the hardware-accel list with it (videotoolbox/audiotoolbox on iOS, +# v4l2-m2m and vulkan on Android) — none of which a Flet app can drive today +# without a JavaVM handle or a Metal/VT session, and all of which would make +# the two platforms diverge. +# --enable-zlib is then re-requested explicitly: both the NDK sysroot and the +# iOS SDK ship libz, several codecs and the Matroska demuxer want it, and an +# explicit --enable- turns "not found" into a hard configure failure instead +# of a silently reduced build. +# --disable-iconv keeps the two platforms symmetric: Android's bionic has no +# iconv before API 28, iOS has one, and the only thing that turns on is +# subtitle charset conversion (`sub_charenc`). +# --disable-devices drops libavdevice's in/out devices (v4l2, AVFoundation). +# libavdevice itself still builds — PyAV links it and calls +# avdevice_register_all() — it just registers nothing. + +if [ "$CROSS_VENV_SDK" = "android" ]; then + TOOLCHAIN_BIN="$(echo "$NDK_ROOT"/toolchains/llvm/prebuilt/*/bin)" + [ -d "$TOOLCHAIN_BIN" ] || { echo "ERROR: no NDK toolchain under $NDK_ROOT"; exit 1; } + + # FFmpeg names architectures its own way, and the NDK's clang driver is + # named after the *compiler* triplet, which for 32-bit ARM is armv7a-... + # and not the armeabi/arm-linux-androideabi forms used elsewhere. + EXTRA_ARGS="" + case "$ANDROID_ABI" in + arm64-v8a) FF_ARCH=aarch64; FF_CPU=armv8-a; CC_TRIPLET=aarch64-linux-android ;; + armeabi-v7a) FF_ARCH=arm; FF_CPU=armv7-a; CC_TRIPLET=armv7a-linux-androideabi + EXTRA_ARGS="--enable-thumb" ;; + x86_64) FF_ARCH=x86_64; FF_CPU=x86-64; CC_TRIPLET=x86_64-linux-android + EXTRA_ARGS="--disable-x86asm" ;; + x86) FF_ARCH=x86; FF_CPU=i686; CC_TRIPLET=i686-linux-android + EXTRA_ARGS="--disable-x86asm" ;; + *) echo "ERROR: unhandled ANDROID_ABI $ANDROID_ABI"; exit 1 ;; + esac + + echo "=== configure (android $ANDROID_ABI, arch $FF_ARCH) ===" + ./configure $COMMON_ARGS $EXTRA_ARGS \ + --target-os=android \ + --arch="$FF_ARCH" \ + --cpu="$FF_CPU" \ + --sysroot="$TOOLCHAIN_BIN/../sysroot" \ + --cc="$TOOLCHAIN_BIN/${CC_TRIPLET}${ANDROID_API_LEVEL}-clang" \ + --cxx="$TOOLCHAIN_BIN/${CC_TRIPLET}${ANDROID_API_LEVEL}-clang++" \ + --ar="$TOOLCHAIN_BIN/llvm-ar" \ + --nm="$TOOLCHAIN_BIN/llvm-nm" \ + --ranlib="$TOOLCHAIN_BIN/llvm-ranlib" \ + --strip="$TOOLCHAIN_BIN/llvm-strip" \ + --extra-ldflags="-Wl,-z,max-page-size=16384" + + echo "=== build + install ===" + make -j "${CPU_COUNT:-4}" + make install +else + # One clang -target triple carries arch, platform, minimum version and the + # simulator marker; passing them as separate -arch/-mios-*-version-min flags + # is what gets an arm64 simulator slice built as a device slice. + case "$CROSS_VENV_SDK" in + iphoneos) TARGET_TRIPLET="${HOST_ARCH}-apple-ios${SDK_VERSION}" ;; + iphonesimulator) TARGET_TRIPLET="${HOST_ARCH}-apple-ios${SDK_VERSION}-simulator" ;; + *) echo "ERROR: unhandled SDK $CROSS_VENV_SDK"; exit 1 ;; + esac + + # No nasm/yasm is guaranteed on a macOS build host or CI runner, and the + # x86_64 slices only ever run in the Simulator. + EXTRA_ARGS="" + [ "$HOST_ARCH" = "x86_64" ] && EXTRA_ARGS="--disable-x86asm" + + echo "=== configure (iOS $HOST_ARCH, $TARGET_TRIPLET) ===" + ./configure $COMMON_ARGS $EXTRA_ARGS \ + --target-os=darwin \ + --arch="$HOST_ARCH" \ + --cc="$(xcrun -sdk "$CROSS_VENV_SDK" -f clang)" \ + --cxx="$(xcrun -sdk "$CROSS_VENV_SDK" -f clang++)" \ + --extra-cflags="-target $TARGET_TRIPLET -isysroot $SDK_ROOT" \ + --extra-ldflags="-target $TARGET_TRIPLET -isysroot $SDK_ROOT -Wl,-headerpad_max_install_names" \ + --install-name-dir=@rpath + + # -headerpad_max_install_names is load-bearing, not hygiene: flet renames each + # of these into a framework and rewrites every install name to the much longer + # @rpath/opt.lib.lib.framework/opt.lib.lib. Without the padding + # install_name_tool fails with "larger updated load commands do not fit", which + # aborts serious_python's reconcile pass — and the app is then built with NO + # site-packages at all rather than failing outright. + # + # Collapse Darwin's versioned dylib triplet (libavcodec.62.1.100.dylib plus + # two symlinks) to a single unversioned libavcodec.dylib. SHFLAGS embeds + # $(SLIBNAME_WITH_MAJOR) as the install name, so overriding it here also + # makes the install-id @rpath/libavcodec.dylib — which is what the `av` + # extension modules record, and what flet's framework relocation rewrites. + # Command-line make variables beat the config.mak assignments. + DEVERSION="SLIBNAME_WITH_VERSION=\$(SLIBNAME) SLIBNAME_WITH_MAJOR=\$(SLIBNAME)" + + echo "=== build + install ===" + make -j "${CPU_COUNT:-4}" $DEVERSION + make install $DEVERSION 'SLIB_INSTALL_NAME=$(SLIBNAME)' SLIB_INSTALL_LINKS= +fi + +# The .pc files land with absolute paths into this build's staging directory — +# and FFmpeg writes libdir/includedir out in full rather than deriving them from +# ${prefix}, so all three lines need rewriting. The wheel unpacks somewhere else +# entirely, so put them in pkg-config's relocatable form: forge adds +# /opt/lib/pkgconfig to PKG_CONFIG_LIBDIR for consumers, and +# PyAV's setup.py finds FFmpeg *only* through pkg-config. +echo "=== relocate pkg-config prefixes ===" +for pc in "$PREFIX"/lib/pkgconfig/*.pc; do + [ -f "$pc" ] || continue + sed -E -e 's|^prefix=.*|prefix=${pcfiledir}/../..|' \ + -e 's|^libdir=.*|libdir=${prefix}/lib|' \ + -e 's|^includedir=.*|includedir=${prefix}/include|' \ + "$pc" > "$pc.tmp" && mv "$pc.tmp" "$pc" +done + +# FFmpeg's LGPL notice does not cover everything in the binary: libavcodec's +# jrevdct.c, jfdctfst.c and jfdctint_template.c come from the Independent JPEG +# Group, carry their own grant in a header comment rather than a separate file, +# and are all compiled in (the MJPEG codec pulls them). Extract that notice so +# the wheel ships it, taking it from the source being built rather than a copy +# checked into the recipe — a copy would drift silently on the next bump. +echo "=== extract the IJG notice ===" +{ + echo "The notice below covers libavcodec/jrevdct.c, libavcodec/jfdctfst.c and" + echo "libavcodec/jfdctint_template.c, which FFmpeg takes from the Independent JPEG" + echo "Group's software and compiles into libavcodec. It is the verbatim header of" + echo "jrevdct.c; the other two carry the same terms under a different copyright year." + echo + echo "----------------------------------------------------------------------------" + echo + awk 'NR==1,/^ \*\/$/' libavcodec/jrevdct.c +} > COPYING.IJG +grep -q "Independent JPEG Group" COPYING.IJG || { + echo "ERROR: no IJG notice in libavcodec/jrevdct.c — has upstream moved it?"; exit 1; } + +# Nothing downstream consumes these, and .la files carry absolute build paths. +shopt -s nullglob +rm -rf "$PREFIX/share" +rm -f "$PREFIX"/lib/*.la + +echo "=== installed ===" +ls -la "$PREFIX/lib" diff --git a/recipes/flet-libffmpeg/meta.yaml b/recipes/flet-libffmpeg/meta.yaml new file mode 100644 index 00000000..33e38eb6 --- /dev/null +++ b/recipes/flet-libffmpeg/meta.yaml @@ -0,0 +1,35 @@ +# {% set version = "8.1.2" %} +package: + name: flet-libffmpeg + version: '{{ version }}' + +build: + number: 1 + +source: + # FFmpeg's libav* as SHARED libraries, for the `av` (PyAV) recipe. PyAV 18.x + # supports the FFmpeg 8.x ABI and upstream's own wheels are built against + # 8.1.2, so the pair is the combination upstream tests. + url: https://ffmpeg.org/releases/ffmpeg-{{ version }}.tar.xz + +about: + # Built WITHOUT --enable-gpl / --enable-nonfree / --enable-version3, so the bulk + # of this wheel is LGPL-2.1-or-later. Discovery would also bundle + # COPYING.GPLv2/GPLv3/LGPLv3 — notices for code this build does not contain — so + # the list is pinned: the LGPL text, LICENSE.md (upstream's own account of which + # parts carry which terms), and COPYING.IJG, which build.sh extracts from + # libavcodec/jrevdct.c. + # + # AND IJG, not plain LGPL: libavcodec's jrevdct.c / jfdctfst.c / + # jfdctint_template.c are Independent JPEG Group code and are genuinely compiled + # in (the MJPEG codec pulls them). LICENSE.md concludes "in combination the LGPL + # v2.1+ applies", but the IJG grant adds a condition that survives the + # combination and lands on whoever ships the binary — condition (2): an app that + # distributes only executables must say in its documentation that it is "based + # in part on the work of the Independent JPEG Group". recipes/av/README.md tells + # consumers so; the expression is what makes it discoverable from the wheel. + license: LGPL-2.1-or-later AND IJG + license_file: + - COPYING.LGPLv2.1 + - LICENSE.md + - COPYING.IJG