diff --git a/.claude/skills/forge-error-catalogue/SKILL.md b/.claude/skills/forge-error-catalogue/SKILL.md index d92a6e2c..3bd30150 100644 --- a/.claude/skills/forge-error-catalogue/SKILL.md +++ b/.claude/skills/forge-error-catalogue/SKILL.md @@ -88,6 +88,8 @@ instead of re-deriving it. transient/scattered in CI where a rerun clears it). **A vendored lib built by setup.py's OWN cmake call** (arg list hardcoded, so `CMAKE_ARGS` does nothing) → green host-configured library, fix by patching in a `FORGE_CMAKE_ARGS` extend. + **`conflicting types for 'fseek'` from a project's own `compat.h`, 32-bit Android only → + the NDK toolchain pins `CMAKE_SYSTEM_VERSION` to 1; read `ANDROID_PLATFORM_LEVEL`**. - **Runtime failures** (device/emulator/simulator) — **the Flet 0.86 Android `sitepackages.zip` class** (its umbrella entry explains "why only now"): `NotADirectoryError` on a bundled data file → **`extract_packages`** meta field; diff --git a/.claude/skills/forge-error-catalogue/references/failure-catalogue.md b/.claude/skills/forge-error-catalogue/references/failure-catalogue.md index 0be23b6d..60f89bd8 100644 --- a/.claude/skills/forge-error-catalogue/references/failure-catalogue.md +++ b/.claude/skills/forge-error-catalogue/references/failure-catalogue.md @@ -1732,6 +1732,48 @@ iOS doesn't need this — Apple's clang resolves the C++ runtime to system libc+ --- +### A **cffi** ABI-mode wrapper cannot open a `flet-lib*` on iOS — `ffi.dlopen` is not `.fwork`-aware + +**Symptom:** a pure-Python package that does `ffi.dlopen(...)` (cffi *ABI* mode — +`ffibuilder.set_source("_x", None)`, no compiled extension anywhere) fails at import on +iOS even though the `flet-lib*` wheel is bundled and the same recipe works on Android. +cffi's error helpfully appends `Additionally, ctypes.util.find_library() did not manage to +locate a library called '.fwork'`, which sends you chasing `find_library` instead of +the real cause. + +**Cause.** serious-python *moves* the real binary into an embedded framework and leaves a +`.fwork` ASCII **text** stub at the old `opt/lib/` path. The dereference of that stub lives +**only** in iOS CPython's patched `Lib/ctypes/__init__.py` — `CDLL.__init__` rewrites a +`*.fwork` name to `os.path.join(dirname(sys.executable), )` before +`_dlopen`, then stores it in `self._name`. Nothing below ctypes knows about `.fwork`: +`_ctypes` doesn't, dyld doesn't, and `cffi`'s `ffi.dlopen` is a raw `dlopen(3)`. +(`importlib`'s `AppleFrameworkLoader` maps `.so`→`.fwork` for module **imports** only, +never for data libraries.) So on iOS the old path is gone and the `.fwork` path is not +Mach-O — both fail. + +**Fix — let ctypes resolve, then hand cffi the path it settled on:** + +```python +path = _ctypes_find_library(name) +if path is None and _sys.platform in ('android', 'ios'): + import ctypes as _ctypes + optlib = os.path.join(os.path.dirname(os.path.abspath(__file__)), 'opt', 'lib') + for candidate in (f'lib{name}.so', # Android jniLibs, by soname + os.path.join(optlib, f'lib{name}.fwork'), # iOS embedded framework + os.path.join(optlib, f'lib{name}.so')): # not relocated + try: + return _ctypes.CDLL(candidate)._name # resolved absolute path + except OSError: + continue +``` + +Wrapping `ctypes.util.find_library` (rather than rewriting the package's loader) keeps +everything downstream upstream's. Worked example: `recipes/soundfile/patches/mobile.patch`. + +**Pure-ctypes wrappers need none of this** — `CDLL('lib.fwork')` just works +(pyzbar, pysodium, python-magic all rely on it). The problem is specific to cffi, and to +anything else calling `dlopen` below the Python level. + ### `ImportError: Unable to find shared library` (ctypes wrapper, at import) **Cause:** a pure-Python `ctypes` wrapper called `ctypes.util.find_library()`, @@ -2091,6 +2133,36 @@ face3d, still counts as pure.) --- +### `error: conflicting types for 'fseek'` / `'ftell'` from a project's own `compat.h` (Android **32-bit only**) + +``` +sysroot/usr/include/stdio.h:216:5: error: conflicting types for 'fseek' + 216 | int fseeko(FILE*, off_t, int) __RENAME(fseeko64) __INTRODUCED_IN(24); +compat.h:67:16: note: expanded from macro 'fseeko' + 67 | #define fseeko fseek +``` + +**Cause — a CMake project reading the Android API level from `CMAKE_SYSTEM_VERSION`.** +The NDK's own `android.toolchain.cmake` pins `CMAKE_SYSTEM_VERSION` to **1** and reports +the real level in **`ANDROID_PLATFORM_LEVEL`**; only CMake's *built-in* Android support +(no toolchain file) puts the API level in `CMAKE_SYSTEM_VERSION`. Every recipe here uses +the toolchain file, so a `CMAKE_SYSTEM_VERSION VERSION_LESS 24` test is **always true**. +The project then concludes fseeko is unavailable and defines `fseeko`→`fseek`, which +rewrites bionic's own LFS *declaration* — and `off_t` is 64-bit (forge compiles with +`_FILE_OFFSET_BITS=64`) while `long` is 32-bit, so the two prototypes disagree. + +**Why only armeabi-v7a (and x86):** 64-bit ABIs have `off_t == long`, and such guards are +usually also gated on `CMAKE_SYSTEM_PROCESSOR MATCHES "i686|armv7-a"`, so arm64/x86_64 +escape and the recipe looks fine until the 32-bit slice. + +**Fix:** patch the condition to prefer `ANDROID_PLATFORM_LEVEL` when defined. Passing +`-DCMAKE_SYSTEM_VERSION=$SDK_VERSION` does **not** work — the toolchain file overwrites +it with a normal (non-cache) `set()`, which shadows your cache entry. Worked example: +`recipes/flet-libflac/patches/android-api-level.patch`. + +**Generalise the tell:** any CMake project inspecting `CMAKE_SYSTEM_VERSION` for Android +is reading `1`. Grep a new native recipe for it before trusting a green 64-bit build. + ### `: no licence file found, so this wheel would ship the library's object code with no notice` **Cause:** a `build.sh` recipe's wheel is synthesised by forge, so nothing carries a diff --git a/.claude/skills/local-recipe-testing/SKILL.md b/.claude/skills/local-recipe-testing/SKILL.md index d446755a..790b4cf6 100644 --- a/.claude/skills/local-recipe-testing/SKILL.md +++ b/.claude/skills/local-recipe-testing/SKILL.md @@ -100,7 +100,9 @@ for i in $(seq 1 30); do grep EXIT "$DATA/Library/Caches/console.log" 2>/dev/nul 1. **Use forge's `dist/` wheel, NOT `build/.../target/wheels/`.** The latter is maturin's raw output — **unstripped**. For polars that meant a **1.27 GB** `.so` (vs 130 MB stripped); it blows up install space and may not load. forge strips + repacks into `dist/`. Always test the `dist/` wheel. -2. **Build the recipe against the SAME Python `flet build` bundles (3.12 for flet 0.85.x).** forge's Android Rust `.so` hard-links `libpythonX.Y.so` (`DT_NEEDED`) — so the **`abi3` wheel tag is misleading**; it still needs the matching `libpython` at `dlopen`. A 3.14-built wheel in a 3.12 app fails: `dlopen … libpython3.14.so` missing → the package reports its "binary missing" (e.g. polars `NameError: PySeries`). Verify with `llvm-readelf -d | grep NEEDED`. If you only have a different support tree, you can retag a wheel for flet's python with `uvx --from wheel wheel tags --python-tag cp312 --abi-tag abi3 --remove `, but the underlying `libpython` link still has to match — so really, build on the right python. +2. **Build the recipe against the SAME Python `flet build` bundles — and know which one that is.** `flet build` **0.86.5 defaults to Python 3.14** (measured 2026-09-14: the app's staged site-packages hold `_cffi_backend.cpython-314-*.so`); the snippets above pass `--python-version 3.12` to pin it, and CI's recipe-tester does the same. A **consumer** example app built the plain way (`flet build apk`, no flag) therefore gets 3.14, so build the recipe for 3.14 (`source ./setup.sh 3.14`) before that pass, or pass the flag. `flet-lib*` build.sh recipes are `py3-none-` and version-independent — only the Python package needs the extra build. + + Two different failure shapes if you get it wrong. For a **compiled** package: forge's Android Rust `.so` hard-links `libpythonX.Y.so` (`DT_NEEDED`) — so the **`abi3` wheel tag is misleading**; it still needs the matching `libpython` at `dlopen`. A 3.14-built wheel in a 3.12 app fails: `dlopen … libpython3.14.so` missing → the package reports its "binary missing" (e.g. polars `NameError: PySeries`). Verify with `llvm-readelf -d | grep NEEDED`. If you only have a different support tree, you can retag a wheel for flet's python with `uvx --from wheel wheel tags --python-tag cp312 --abi-tag abi3 --remove `, but the underlying `libpython` link still has to match — so really, build on the right python. For a **pure-Python** package with a mobile patch there is no link error at all — pip simply finds no `cp3XX` match, silently installs PyPI's `py3-none-any` wheel, and the app fails on device with the unpatched loader's own error. Tell: the app's `build/site-packages//-*.dist-info/METADATA` is missing the `flet-lib*` `Requires-Dist` the recipe promotes. 3. **Clear `tests/recipe-tester/build/site-packages` + `build/.hash` between rebuilds.** `flet build` keys its skip-site-packages cache on the requirement *string*, not wheel content — so swapping a same-version wheel is silently ignored and it re-bundles the old `.so`. Tell-tale: the APK size doesn't change after you changed the wheel. diff --git a/.claude/skills/new-mobile-recipe/SKILL.md b/.claude/skills/new-mobile-recipe/SKILL.md index cc57532a..715ee577 100644 --- a/.claude/skills/new-mobile-recipe/SKILL.md +++ b/.claude/skills/new-mobile-recipe/SKILL.md @@ -77,6 +77,7 @@ Match the package to one of these shapes. Each maps to a template in `templates/ | C-ext consuming an existing flet-lib | Already-built `flet-libX` covers the C dep (libxml2, libcurl, libssl via openssl, etc.) | Adapt `templates/meta-with-patches.yaml` + add `requirements.host` | | Native library itself (flet-lib*), **static** | New C library a Python C-extension links at build time (libxml2, libcurl, libgeos…) | `templates/meta-flet-lib.yaml` + `templates/build-flet-lib.sh` | | Native library, **ctypes-loaded (shared)** | A pure-Python wrapper `dlopen`s the lib at runtime via `ctypes` (pyzbar→libzbar, python-magic→libmagic) | `templates/meta-flet-lib.yaml` + `templates/build-flet-lib-shared.sh`; see Pattern H | +| Native library, **cffi-loaded (shared)** | Same, but the wrapper is **cffi ABI mode** (`ffibuilder.set_source(name, None)` → `ffi.dlopen`, no compiled extension anywhere) (soundfile→libsndfile) | As Pattern H, plus a loader patch that resolves through ctypes — `ffi.dlopen` is a raw `dlopen(3)` and cannot follow an iOS `.fwork`; copy `recipes/soundfile/` | | Cython-accelerated pure-Python (poetry-core build script) | `build-backend = "poetry.core.masonry.api"` + `[tool.poetry.build] script` that cythonizes the runtime `.py` files themselves (zeroconf; the Home-Assistant-ecosystem idiom). Forge's PEP 517 path handles poetry-core unchanged | No template — copy `recipes/zeroconf/` (branch `zeroconf`): `script_env REQUIRE_CYTHON: "1"` + a fail-loud patch (upstream swallows compile errors → silent pure-py wheel), test asserts the modules are real extensions | | C-ext that links a lib via a `*-config` tool | Compiled C-ext whose `setup.py` shells out to `pg_config`/`mysql_config`/… (psycopg2→libpq, mysqlclient→libmysqlclient) | A **static+PIC** `flet-lib*` (`build-flet-lib.sh` + `-fPIC`) shipping a config-shim, + consumer `script_env`/patch; see Pattern I | | **setup.py that drives CMake itself** for a vendored native lib | An sdist that vendors a C library and builds it with its own `subprocess` CMake call inside `build_ext`, then links the static result via `extra_objects` (pycares→c-ares). Not scikit-build-core — the arg list is hardcoded in `setup.py` | No template — copy `recipes/pycares/`: one patch appends `shlex.split(os.environ['FORGE_CMAKE_ARGS'])` to the arg list, `requirements.build: [cmake]`; see "vendored-CMake" deep-dive below | @@ -156,6 +157,38 @@ The shape that works — a `flet-lib*` prebuilt-repackage dep + a small opt-in p **Real example:** branch `curl-cffi` — `recipes/flet-libcurl-impersonate/` (build.sh, `source.strip: 0`) + `recipes/curl-cffi/` (`patches/mobile.patch`, 3 hunks). World-first curl-cffi iOS wheels; on-device 4/4 both platforms, and a real impersonated HTTPS request returns 200 on each. Needed one forge-core change: exposing `source.strip` in `src/forge/schema/meta-schema.yaml` (the code already honored it). +### Shape deep-dive: cffi ABI-mode wrapper of a shared flet-lib (soundfile -> libsndfile) + +**When:** the package has **no compiled extension at all** — `soundfile_build.py` does +`ffibuilder.set_source("_soundfile", None)`, so cffi emits a pure-Python `_soundfile.py` +and the library is reached entirely through `ffi.dlopen(...)`. PyPI ships a +`py2.py3-none-any` wheel plus platform wheels that differ only by a bundled `.so`. + +It looks like Pattern H and mostly is, with three differences that each cost a cycle: + +1. **`ffi.dlopen` is a raw `dlopen(3)`.** It cannot dereference the iOS `.fwork` stub — + that lives only in iOS CPython's patched `ctypes/__init__.py`. Resolve through ctypes + and hand cffi `CDLL(candidate)._name`. Full mechanism and patch shape in the + `forge-error-catalogue`, "A **cffi** ABI-mode wrapper cannot open a `flet-lib*` on iOS". +2. **A recipe is still required even though nothing compiles**, because the loader's + `sys.platform` dispatch (darwin/win32/linux) ends in a bare `raise` on `android`/`ios`. + forge's `fix_wheel` retags the pure wheel `cp3XX-cp3XX-`, which is what makes + the patched copy outrank PyPI's `any` wheel at the same version — do not "fix" upstream's + own tagging in a patch. +3. **Build it for the Python `flet build` actually bundles.** A cp312-only wheel loses to + PyPI's `py2.py3-none-any` on a 3.14 app with no resolution error at all; the tell is the + app's staged `METADATA` missing the `flet-lib*` `Requires-Dist`. See `local-recipe-testing`. + +**Optional sub-libraries:** where the native library has them (libsndfile's +FLAC/Ogg/Vorbis/Opus/mpg123/LAME), build each as a **static PIC** `flet-lib*` and declare +them `requirements.host_build` of the shared one, which absorbs them into a single `.so`. +`host` would promote six wheels into every consuming app whose contents already ride inside +that `.so`. Restrict the iOS export list to the public prefix +(`-Wl,-exported_symbol,'_sf_*'`) so the absorbed symbols stay hidden the way libsndfile's +own version script already keeps them on Android — jniLibs is a flat namespace shared with +every other native wheel. And keep `-Wl,-headerpad_max_install_names` on any hand-linked +iOS shared image: CMake adds it automatically, a hand `$CC -shared` does not. + ### Naming - Recipe directory name and `package.name` in meta.yaml must match the PyPI sdist filename exactly (case-sensitive). `MarkupSafe`, not `markupsafe`. `cffi`, not `CFFI`. Look at `-.tar.gz` filename on PyPI for the ground truth. diff --git a/recipes/flet-libflac/build.sh b/recipes/flet-libflac/build.sh new file mode 100755 index 00000000..0c92de32 --- /dev/null +++ b/recipes/flet-libflac/build.sh @@ -0,0 +1,43 @@ +#!/bin/bash +set -eu + +# ENABLE_MULTITHREADING puts Threads::Threads in libFLAC's exported CMake +# target, which libsndfile (no find_package(Threads)) then cannot resolve. +# libsndfile drives the single-threaded encoder API regardless. +common_args="\ + -DCMAKE_BUILD_TYPE=Release \ + -DCMAKE_POLICY_VERSION_MINIMUM=3.5 \ + -DCMAKE_INSTALL_PREFIX=$PREFIX \ + -DCMAKE_POSITION_INDEPENDENT_CODE=ON \ + -DCMAKE_PREFIX_PATH=$PLATLIB/opt \ + -DBUILD_SHARED_LIBS=OFF \ + -DBUILD_CXXLIBS=OFF \ + -DBUILD_PROGRAMS=OFF \ + -DBUILD_EXAMPLES=OFF \ + -DBUILD_TESTING=OFF \ + -DBUILD_DOCS=OFF \ + -DINSTALL_MANPAGES=OFF \ + -DWITH_OGG=ON \ + -DENABLE_MULTITHREADING=OFF \ + -DOGG_INCLUDE_DIR=$PLATLIB/opt/include \ + -DOGG_LIBRARY=$PLATLIB/opt/lib/libogg.a" + +if [ "$CROSS_VENV_SDK" = "android" ]; then + cmake -B build \ + -DCMAKE_SYSTEM_NAME=Android \ + -DANDROID_PLATFORM=$SDK_VERSION \ + -DANDROID_ABI=$ANDROID_ABI \ + -DCMAKE_TOOLCHAIN_FILE=$NDK_ROOT/build/cmake/android.toolchain.cmake \ + $common_args +else + cmake -B build \ + -DCMAKE_SYSTEM_NAME=iOS \ + -DCMAKE_OSX_SYSROOT=$SDK \ + -DCMAKE_OSX_ARCHITECTURES=$HOST_ARCH \ + $common_args +fi + +cmake --build build -j "$CPU_COUNT" +cmake --install build + +rm -rf "$PREFIX/share" diff --git a/recipes/flet-libflac/meta.yaml b/recipes/flet-libflac/meta.yaml new file mode 100644 index 00000000..3a553aff --- /dev/null +++ b/recipes/flet-libflac/meta.yaml @@ -0,0 +1,29 @@ +{% set version = "1.5.0" %} + +# Static PIC libFLAC for flet-libsndfile's FLAC support (native .flac and +# Ogg-FLAC). Link-time only -- nothing ships to the device. +package: + name: flet-libflac + version: '{{ version }}' + +build: + number: 1 + +source: + url: https://github.com/xiph/flac/releases/download/{{ version }}/flac-{{ version }}.tar.xz + +requirements: + build: + - cmake + host_build: + - flet-libogg 1.3.6 + +patches: + - android-api-level.patch + +about: + # Only libFLAC is built here. The GPL-2.0 arm of FLAC's split licence covers + # the command-line tools (BUILD_PROGRAMS=OFF), which this wheel does not + # contain; the library itself is BSD-3-Clause (COPYING.Xiph). + license_file: COPYING.Xiph + license: BSD-3-Clause diff --git a/recipes/flet-libflac/patches/android-api-level.patch b/recipes/flet-libflac/patches/android-api-level.patch new file mode 100644 index 00000000..36b89866 --- /dev/null +++ b/recipes/flet-libflac/patches/android-api-level.patch @@ -0,0 +1,37 @@ +Read the Android API level from the variable the NDK toolchain actually sets. + +FLAC disables fseeko/ftello on 32-bit Android "before API 24" and reads the API +level from CMAKE_SYSTEM_VERSION. That only works under CMake's built-in Android +support; the NDK's own android.toolchain.cmake -- which is what every recipe +here uses -- pins CMAKE_SYSTEM_VERSION to 1 and reports the level in +ANDROID_PLATFORM_LEVEL instead. So the test fires at every API level, and on +armeabi-v7a compat.h's `#define fseeko fseek` then collides with bionic's own +64-bit-off_t declaration (forge compiles with _FILE_OFFSET_BITS=64): + + stdio.h:216:5: error: conflicting types for 'fseek' + compat.h:67:16: note: expanded from macro 'fseeko' + +arm64-v8a and x86_64 escape only because the processor test does not match them. + +diff -ruN fa/CMakeLists.txt fb/CMakeLists.txt +--- fa/CMakeLists.txt 2025-02-10 15:20:26 ++++ fb/CMakeLists.txt 2026-09-14 21:42:44 +@@ -122,7 +122,17 @@ + endif() + + +-if(ANDROID AND CMAKE_SYSTEM_VERSION VERSION_LESS 24 AND (CMAKE_SYSTEM_PROCESSOR MATCHES "i686" OR CMAKE_SYSTEM_PROCESSOR MATCHES "armv7-a")) ++if(ANDROID AND DEFINED ANDROID_PLATFORM_LEVEL) ++ # The NDK's own android.toolchain.cmake pins CMAKE_SYSTEM_VERSION to 1 and ++ # reports the real API level in ANDROID_PLATFORM_LEVEL; only CMake's built-in ++ # Android support puts the API level in CMAKE_SYSTEM_VERSION. Read whichever ++ # is meaningful, or the test below disables fseeko at every API level. ++ set(FLAC_ANDROID_API_LEVEL ${ANDROID_PLATFORM_LEVEL}) ++else() ++ set(FLAC_ANDROID_API_LEVEL ${CMAKE_SYSTEM_VERSION}) ++endif() ++ ++if(ANDROID AND FLAC_ANDROID_API_LEVEL VERSION_LESS 24 AND (CMAKE_SYSTEM_PROCESSOR MATCHES "i686" OR CMAKE_SYSTEM_PROCESSOR MATCHES "armv7-a")) + # fseeko/ftello may link, but it's not usable before Android API 24 on 32-bit Android + # https://android.googlesource.com/platform/bionic/+/main/docs/32-bit-abi.md + message(STATUS "Disabling fseeko/ftello for 32-bit Android before API 24") diff --git a/recipes/flet-libmp3lame/build.sh b/recipes/flet-libmp3lame/build.sh new file mode 100755 index 00000000..6dd8018f --- /dev/null +++ b/recipes/flet-libmp3lame/build.sh @@ -0,0 +1,39 @@ +#!/bin/bash +set -eu + +# PIC: the archive is linked into the shared libsndfile.so. +export CFLAGS="${CFLAGS:-} -fPIC" + +# Only libmp3lame is wanted; the frontend drags in a terminal UI and, on 3.100, +# an frontend/get_audio.c that does not cross-compile. +common_args="\ + --disable-dependency-tracking \ + --enable-static --disable-shared \ + --disable-frontend --disable-decoder --disable-gtktest \ + --disable-analyzer-hooks --disable-nasm" + +if [ "$CROSS_VENV_SDK" = "android" ]; then + host=$HOST_TRIPLET +else + # lame 3.100's config.sub (2016) rejects arm64-apple-ios -- feed it the + # equivalent Darwin triplet; CC/CFLAGS do the real targeting. + case $HOST_TRIPLET in + arm64-apple-ios) host=aarch64-apple-darwin23 ;; + arm64-apple-ios-simulator) host=aarch64-apple-darwin23 ;; + x86_64-apple-ios-simulator) host=x86_64-apple-darwin23 ;; + *) echo "Unknown iOS host triplet: $HOST_TRIPLET"; exit 1 ;; + esac + # forge's iOS flags carry a quoted -F path; the quotes reach the compiler + # literally and break preprocessing. The forge paths have no spaces. + export CFLAGS="$(printf '%s' "$CFLAGS" | tr -d '"')" + export CPPFLAGS="$(printf '%s' "${CPPFLAGS:-}" | tr -d '"')" + export LDFLAGS="$(printf '%s' "${LDFLAGS:-}" | tr -d '"')" +fi + +./configure --host=$host --prefix=$PREFIX $common_args +make -j "$CPU_COUNT" +make install + +shopt -s nullglob +rm -rf "$PREFIX/share" "$PREFIX/bin" +rm -f "$PREFIX"/lib/*.la diff --git a/recipes/flet-libmp3lame/meta.yaml b/recipes/flet-libmp3lame/meta.yaml new file mode 100644 index 00000000..999bc2d3 --- /dev/null +++ b/recipes/flet-libmp3lame/meta.yaml @@ -0,0 +1,23 @@ +{% set version = "3.100" %} + +# Static PIC libmp3lame -- the MP3 *encoder* half of flet-libsndfile's MP3 +# support (mpg123 is the decoder half; libsndfile needs both or neither). +# Link-time only -- nothing ships to the device. +package: + name: flet-libmp3lame + version: '{{ version }}' + +build: + number: 1 + +source: + # Debian's pool copy, not SourceForge's. `downloads.sourceforge.net` bounces + # through a randomly chosen mirror and timed out its TLS handshake on two of + # six CI legs (run 34892127701); this host is stable and serves the same bytes. + # Upstream remains https://sourceforge.net/projects/lame/files/lame/{{ version }}/. + url: https://deb.debian.org/debian/pool/main/l/lame/lame_{{ version }}.orig.tar.gz + +about: + # COPYING is the LGPL v2 text; every source header adds "or (at your option) + # any later version". + license: LGPL-2.0-or-later diff --git a/recipes/flet-libmpg123/build.sh b/recipes/flet-libmpg123/build.sh new file mode 100755 index 00000000..b2a064e2 --- /dev/null +++ b/recipes/flet-libmpg123/build.sh @@ -0,0 +1,48 @@ +#!/bin/bash +set -eu + +# PIC: the archive is linked into the shared libsndfile.so. +export CFLAGS="${CFLAGS:-} -fPIC" + +# --disable-components + --enable-libmpg123 drops the CLI, libout123 and +# libsyn123; libsndfile only decodes through libmpg123. +common_args="\ + --disable-dependency-tracking \ + --enable-static --disable-shared \ + --disable-components --enable-libmpg123 \ + --disable-modules --disable-network" + +# The packed CPU sets carry a NEON/SSE decoder plus a generic fallback and pick +# at runtime, so one build serves every device of that ABI. +case $HOST_ARCH in + arm64-v8a|arm64) cpu=aarch64 ;; + armeabi-v7a) cpu=arm_fpu ;; + x86_64) cpu=x86-64 ;; + *) cpu=generic_fpu ;; +esac + +if [ "$CROSS_VENV_SDK" = "android" ]; then + host=$HOST_TRIPLET +else + # config.sub predates Apple's mobile triplets -- feed it the equivalent + # Darwin one; CC/CFLAGS do the real targeting. + case $HOST_TRIPLET in + arm64-apple-ios) host=aarch64-apple-darwin23 ;; + arm64-apple-ios-simulator) host=aarch64-apple-darwin23 ;; + x86_64-apple-ios-simulator) host=x86_64-apple-darwin23 ;; + *) echo "Unknown iOS host triplet: $HOST_TRIPLET"; exit 1 ;; + esac + # forge's iOS flags carry a quoted -F path; the quotes reach the compiler + # literally and break preprocessing. The forge paths have no spaces. + export CFLAGS="$(printf '%s' "$CFLAGS" | tr -d '"')" + export CPPFLAGS="$(printf '%s' "${CPPFLAGS:-}" | tr -d '"')" + export LDFLAGS="$(printf '%s' "${LDFLAGS:-}" | tr -d '"')" +fi + +./configure --host=$host --prefix=$PREFIX --with-cpu=$cpu $common_args +make -j "$CPU_COUNT" +make install + +shopt -s nullglob +rm -rf "$PREFIX/share" "$PREFIX/bin" +rm -f "$PREFIX"/lib/*.la diff --git a/recipes/flet-libmpg123/meta.yaml b/recipes/flet-libmpg123/meta.yaml new file mode 100644 index 00000000..f859bbe2 --- /dev/null +++ b/recipes/flet-libmpg123/meta.yaml @@ -0,0 +1,18 @@ +{% set version = "1.33.7" %} + +# Static PIC libmpg123 -- the MPEG Audio *decoder* half of flet-libsndfile's +# MP3 support (mp3lame is the encoder half; libsndfile needs both or neither). +# Link-time only -- nothing ships to the device. +package: + name: flet-libmpg123 + version: '{{ version }}' + +build: + number: 1 + +source: + url: https://www.mpg123.de/download/mpg123-{{ version }}.tar.bz2 + +about: + # No "or later": COPYING and every source header name LGPL 2.1 flat. + license: LGPL-2.1-only diff --git a/recipes/flet-libogg/build.sh b/recipes/flet-libogg/build.sh new file mode 100755 index 00000000..e939bd8d --- /dev/null +++ b/recipes/flet-libogg/build.sh @@ -0,0 +1,31 @@ +#!/bin/bash +set -eu + +# PIC: the archive is linked into the shared libsndfile.so. +common_args="\ + -DCMAKE_BUILD_TYPE=Release \ + -DCMAKE_INSTALL_PREFIX=$PREFIX \ + -DCMAKE_POSITION_INDEPENDENT_CODE=ON \ + -DBUILD_SHARED_LIBS=OFF \ + -DBUILD_TESTING=OFF \ + -DINSTALL_DOCS=OFF" + +if [ "$CROSS_VENV_SDK" = "android" ]; then + cmake -B build \ + -DCMAKE_SYSTEM_NAME=Android \ + -DANDROID_PLATFORM=$SDK_VERSION \ + -DANDROID_ABI=$ANDROID_ABI \ + -DCMAKE_TOOLCHAIN_FILE=$NDK_ROOT/build/cmake/android.toolchain.cmake \ + $common_args +else + cmake -B build \ + -DCMAKE_SYSTEM_NAME=iOS \ + -DCMAKE_OSX_SYSROOT=$SDK \ + -DCMAKE_OSX_ARCHITECTURES=$HOST_ARCH \ + $common_args +fi + +cmake --build build -j "$CPU_COUNT" +cmake --install build + +rm -rf "$PREFIX/share" diff --git a/recipes/flet-libogg/meta.yaml b/recipes/flet-libogg/meta.yaml new file mode 100644 index 00000000..5165dc89 --- /dev/null +++ b/recipes/flet-libogg/meta.yaml @@ -0,0 +1,21 @@ +{% set version = "1.3.6" %} + +# Static PIC archive: the Ogg container layer for flet-libsndfile's FLAC, +# Vorbis and Opus support. Folded into libsndfile.so at link time, so this +# is a host_build (link-time) dependency and nothing ships to the device. +package: + name: flet-libogg + version: '{{ version }}' + +build: + number: 1 + +source: + url: https://github.com/xiph/ogg/releases/download/v{{ version }}/libogg-{{ version }}.tar.gz + +requirements: + build: + - cmake + +about: + license: BSD-3-Clause diff --git a/recipes/flet-libopus/build.sh b/recipes/flet-libopus/build.sh new file mode 100755 index 00000000..8206794b --- /dev/null +++ b/recipes/flet-libopus/build.sh @@ -0,0 +1,32 @@ +#!/bin/bash +set -eu + +common_args="\ + -DCMAKE_BUILD_TYPE=Release \ + -DCMAKE_POLICY_VERSION_MINIMUM=3.5 \ + -DCMAKE_INSTALL_PREFIX=$PREFIX \ + -DCMAKE_POSITION_INDEPENDENT_CODE=ON \ + -DBUILD_SHARED_LIBS=OFF \ + -DBUILD_TESTING=OFF \ + -DOPUS_BUILD_PROGRAMS=OFF \ + -DOPUS_BUILD_TESTING=OFF" + +if [ "$CROSS_VENV_SDK" = "android" ]; then + cmake -B build \ + -DCMAKE_SYSTEM_NAME=Android \ + -DANDROID_PLATFORM=$SDK_VERSION \ + -DANDROID_ABI=$ANDROID_ABI \ + -DCMAKE_TOOLCHAIN_FILE=$NDK_ROOT/build/cmake/android.toolchain.cmake \ + $common_args +else + cmake -B build \ + -DCMAKE_SYSTEM_NAME=iOS \ + -DCMAKE_OSX_SYSROOT=$SDK \ + -DCMAKE_OSX_ARCHITECTURES=$HOST_ARCH \ + $common_args +fi + +cmake --build build -j "$CPU_COUNT" +cmake --install build + +rm -rf "$PREFIX/share" diff --git a/recipes/flet-libopus/meta.yaml b/recipes/flet-libopus/meta.yaml new file mode 100644 index 00000000..e8567549 --- /dev/null +++ b/recipes/flet-libopus/meta.yaml @@ -0,0 +1,20 @@ +{% set version = "1.5.2" %} + +package: + name: flet-libopus + version: '{{ version }}' + # Static PIC libopus for flet-libsndfile's Ogg Opus support. Link-time only + # -- nothing ships to the device. + +build: + number: 1 + +source: + url: https://github.com/xiph/opus/releases/download/v{{ version }}/opus-{{ version }}.tar.gz + +requirements: + build: + - cmake + +about: + license: BSD-3-Clause diff --git a/recipes/flet-libsndfile/build.sh b/recipes/flet-libsndfile/build.sh new file mode 100755 index 00000000..e42a305f --- /dev/null +++ b/recipes/flet-libsndfile/build.sh @@ -0,0 +1,83 @@ +#!/bin/bash +# flet-libsndfile: libsndfile as a SHARED library (Pattern H) for the pure-Python +# `soundfile` package, which dlopens it through cffi. Ships opt/lib/libsndfile.so +# on both platforms; serious-python surfaces it as Android jniLibs / an iOS +# embedded framework + libsndfile.fwork pointer. +set -eu + +NAME=sndfile + +common_args="\ + -DCMAKE_BUILD_TYPE=Release \ + -DCMAKE_POLICY_VERSION_MINIMUM=3.5 \ + -DCMAKE_INSTALL_PREFIX=$PREFIX \ + -DBUILD_PROGRAMS=OFF \ + -DBUILD_EXAMPLES=OFF \ + -DBUILD_TESTING=OFF \ + -DENABLE_CPACK=OFF \ + -DENABLE_PACKAGE_CONFIG=OFF \ + -DINSTALL_PKGCONFIG_MODULE=OFF \ + -DENABLE_EXTERNAL_LIBS=ON \ + -DENABLE_MPEG=ON \ + -DCMAKE_FIND_PACKAGE_PREFER_CONFIG=ON \ + -DCMAKE_PREFIX_PATH=$PLATLIB/opt \ + -DCMAKE_FIND_ROOT_PATH_MODE_PACKAGE=BOTH \ + -DCMAKE_FIND_ROOT_PATH_MODE_INCLUDE=BOTH \ + -DCMAKE_FIND_ROOT_PATH_MODE_LIBRARY=BOTH \ + -DCMAKE_FIND_USE_CMAKE_SYSTEM_PATH=NO" + +if [ "$CROSS_VENV_SDK" = "android" ]; then + cmake -B build \ + -DCMAKE_SYSTEM_NAME=Android \ + -DANDROID_PLATFORM=$SDK_VERSION \ + -DANDROID_ABI=$ANDROID_ABI \ + -DCMAKE_TOOLCHAIN_FILE=$NDK_ROOT/build/cmake/android.toolchain.cmake \ + -DCMAKE_SHARED_LINKER_FLAGS="$LDFLAGS" \ + -DBUILD_SHARED_LIBS=ON \ + $common_args + cmake --build build -j "$CPU_COUNT" + cmake --install build +else + # iOS: build static, then hand-link the shared image. CMake would emit the + # versioned dylib triplet, and serious-python's framework name is the relative + # path truncated at the first dot -- libsndfile.dylib and libsndfile.1.dylib + # would both become one framework. One unversioned file named .so also matches + # Android's jniLibs glob, so the loader patch needs a single candidate name. + # + # -headerpad_max_install_names is not optional: serious-python rewrites install + # names when it relocates the image into a framework, and without the padding + # that rewrite fails while the build still exits 0. + cmake -B build \ + -DCMAKE_SYSTEM_NAME=iOS \ + -DCMAKE_OSX_SYSROOT=$SDK \ + -DCMAKE_OSX_ARCHITECTURES=$HOST_ARCH \ + -DBUILD_SHARED_LIBS=OFF \ + $common_args + cmake --build build -j "$CPU_COUNT" + cmake --install build + + # force_load (not all_load) so only libsndfile's own objects are taken + # whole; the codec archives contribute just what libsndfile references. + # -exported_symbol hides the absorbed codec symbols, matching what the + # version script already does on Android. + cd "$PREFIX/lib" + $CC $CFLAGS -shared \ + -Wl,-headerpad_max_install_names \ + -Wl,-exported_symbol,'_sf_*' \ + -Wl,-force_load,"lib$NAME.a" \ + "$PLATLIB/opt/lib/libFLAC.a" \ + "$PLATLIB/opt/lib/libvorbisenc.a" \ + "$PLATLIB/opt/lib/libvorbis.a" \ + "$PLATLIB/opt/lib/libopus.a" \ + "$PLATLIB/opt/lib/libogg.a" \ + "$PLATLIB/opt/lib/libmp3lame.a" \ + "$PLATLIB/opt/lib/libmpg123.a" \ + -install_name "@rpath/lib$NAME.so" \ + -o "lib$NAME.so" + rm -f "lib$NAME.a" + cd - >/dev/null +fi + +shopt -s nullglob +rm -rf "$PREFIX/share" "$PREFIX/bin" "$PREFIX/include" +rm -rf "$PREFIX/lib/cmake" "$PREFIX/lib/pkgconfig" "$PREFIX"/lib/*.la diff --git a/recipes/flet-libsndfile/licenses/LICENSE.ALAC b/recipes/flet-libsndfile/licenses/LICENSE.ALAC new file mode 100644 index 00000000..d6456956 --- /dev/null +++ b/recipes/flet-libsndfile/licenses/LICENSE.ALAC @@ -0,0 +1,202 @@ + + Apache License + Version 2.0, January 2004 + http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + + 4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + + 5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + + 6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + + 7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + + 8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + + 9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + + END OF TERMS AND CONDITIONS + + APPENDIX: How to apply the Apache License to your work. + + To apply the Apache License to your work, attach the following + boilerplate notice, with the fields enclosed by brackets "[]" + replaced with your own identifying information. (Don't include + the brackets!) The text should be enclosed in the appropriate + comment syntax for the file format. We also recommend that a + file or class name and description of purpose be included on the + same "printed page" as the copyright notice for easier + identification within third-party archives. + + Copyright [yyyy] [name of copyright owner] + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. diff --git a/recipes/flet-libsndfile/meta.yaml b/recipes/flet-libsndfile/meta.yaml new file mode 100644 index 00000000..ba3d7458 --- /dev/null +++ b/recipes/flet-libsndfile/meta.yaml @@ -0,0 +1,42 @@ +{% set version = "1.2.2" %} + +package: + name: flet-libsndfile + version: '{{ version }}' + +build: + number: 1 + +source: + url: https://github.com/libsndfile/libsndfile/releases/download/{{ version }}/libsndfile-{{ version }}.tar.xz + +requirements: + build: + - cmake + host_build: + # The codec libraries, as static PIC archives folded into libsndfile.so. + # host_build, not host: they are link-time only, so consuming apps must not + # also download six wheels whose contents already live inside our .so. + - flet-libogg 1.3.6 + - flet-libvorbis 1.3.7 + - flet-libflac 1.5.0 + - flet-libopus 1.5.2 + - flet-libmpg123 1.33.7 + - flet-libmp3lame 3.100 + +patches: + - security-backports.patch + +about: + # libsndfile itself is LGPL-2.1-or-later and ships as a replaceable shared + # library, which is what keeps that workable for a closed app. The absorbed + # archives are all BSD-3-Clause, plus the bundled Apple ALAC codec + # (Apache-2.0, src/ALAC/) and the GSM 06.10 codec (src/GSM610/COPYRIGHT, + # a bespoke permissive notice with no SPDX identifier -- hence no `AND` + # clause for it; the file itself ships). ALAC carries its Apache notice only + # in per-file headers, so the licence text is vendored in licenses/. + license_file: + - COPYING + - src/GSM610/COPYRIGHT + - licenses/LICENSE.ALAC + license: LGPL-2.1-or-later AND BSD-3-Clause AND Apache-2.0 diff --git a/recipes/flet-libsndfile/patches/security-backports.patch b/recipes/flet-libsndfile/patches/security-backports.patch new file mode 100644 index 00000000..eaa1817d --- /dev/null +++ b/recipes/flet-libsndfile/patches/security-backports.patch @@ -0,0 +1,124 @@ +Backport upstream's post-1.2.2 memory-safety fixes. + +1.2.2 (August 2023) is still the only libsndfile release, and ~80 commits have +landed on master since — including several heap over-read/overflow fixes in +parsers this build enables. A mobile app hands libsndfile whatever audio file a +user picked, so these are reachable. Five upstream commits, cherry-picked onto +the 1.2.2 tag with no conflicts: + + daf79ae6 caf, aiff: fix heap buffer over-read in channel map parsing + a3827572 Fix heap-buffer-overflow in vox_read_block (vox_adpcm.c) + 5d165fb9 chunk: Fix heap overflow in psf_save_write_chunk + 90ef19a2 nms_adpcm: Fix out-of-bounds write on short read + b9103bdd alac: fix pakt_size + 4 overflow in alac_pakt_read_decode + +Drop this file the moment upstream tags a release that contains them, and check +`git log 1.2.2..` for anything newer rather than assuming this list is +still the whole set. + +diff --git a/src/aiff.c b/src/aiff.c +index a2bda8f..80b32f3 100644 +--- a/src/aiff.c ++++ b/src/aiff.c +@@ -1790,14 +1790,20 @@ aiff_read_chanmap (SF_PRIVATE * psf, unsigned dword) + psf_binheader_readf (psf, "j", dword - bytesread) ; + + if (map_info->channel_map != NULL) +- { size_t chanmap_size = SF_MIN (psf->sf.channels, layout_tag & 0xffff) * sizeof (psf->channel_map [0]) ; ++ { /* The channel map buffer must hold psf->sf.channels entries, because that ++ ** is how many SFC_GET_CHANNEL_MAP_INFO and the channel-layout matcher read ++ ** back. The layout tag may describe fewer channels than the file declares, ++ ** so size the allocation by the channel count and copy only the smaller of ++ ** the two to avoid a heap over-read. */ ++ size_t chanmap_size = (size_t) psf->sf.channels * sizeof (psf->channel_map [0]) ; ++ size_t copy_size = SF_MIN (psf->sf.channels, layout_tag & 0xffff) * sizeof (psf->channel_map [0]) ; + + free (psf->channel_map) ; + +- if ((psf->channel_map = malloc (chanmap_size)) == NULL) ++ if ((psf->channel_map = calloc (1, chanmap_size)) == NULL) + return SFE_MALLOC_FAILED ; + +- memcpy (psf->channel_map, map_info->channel_map, chanmap_size) ; ++ memcpy (psf->channel_map, map_info->channel_map, copy_size) ; + } ; + + return 0 ; +diff --git a/src/alac.c b/src/alac.c +index a6668f3..cea9fb8 100644 +--- a/src/alac.c ++++ b/src/alac.c +@@ -823,6 +823,8 @@ alac_pakt_read_decode (SF_PRIVATE * psf, uint32_t UNUSED (pakt_offset)) + psf->get_chunk_size (psf, chunk_iterator, &chunk_info) ; + + pakt_size = chunk_info.datalen ; ++ if (pakt_size > UINT32_MAX-5 ) ++ return NULL; + chunk_info.data = pakt_data = malloc (pakt_size + 5) ; + if (!chunk_info.data) + return NULL ; +diff --git a/src/caf.c b/src/caf.c +index a886bf5..9e4e723 100644 +--- a/src/caf.c ++++ b/src/caf.c +@@ -811,14 +811,20 @@ caf_read_chanmap (SF_PRIVATE * psf, sf_count_t chunk_size) + psf_binheader_readf (psf, "j", chunk_size - bytesread) ; + + if (map_info && map_info->channel_map != NULL) +- { size_t chanmap_size = SF_MIN (psf->sf.channels, layout_tag & 0xff) * sizeof (psf->channel_map [0]) ; ++ { /* The channel map buffer must hold psf->sf.channels entries, because that ++ ** is how many SFC_GET_CHANNEL_MAP_INFO and the channel-layout matcher read ++ ** back. The layout tag may describe fewer channels than the file declares, ++ ** so size the allocation by the channel count and copy only the smaller of ++ ** the two to avoid a heap over-read. */ ++ size_t chanmap_size = (size_t) psf->sf.channels * sizeof (psf->channel_map [0]) ; ++ size_t copy_size = SF_MIN (psf->sf.channels, layout_tag & 0xff) * sizeof (psf->channel_map [0]) ; + + free (psf->channel_map) ; + +- if ((psf->channel_map = malloc (chanmap_size)) == NULL) ++ if ((psf->channel_map = calloc (1, chanmap_size)) == NULL) + return SFE_MALLOC_FAILED ; + +- memcpy (psf->channel_map, map_info->channel_map, chanmap_size) ; ++ memcpy (psf->channel_map, map_info->channel_map, copy_size) ; + } ; + + return 0 ; +diff --git a/src/chunk.c b/src/chunk.c +index 9b181fc..d8daf04 100644 +--- a/src/chunk.c ++++ b/src/chunk.c +@@ -246,6 +246,7 @@ psf_save_write_chunk (WRITE_CHUNKS * pchk, const SF_CHUNK_INFO * chunk_info) + return SFE_MALLOC_FAILED ; + } else { + pchk->chunks = new_chunks; ++ pchk->count = new_count ; + } ; + } ; + +diff --git a/src/nms_adpcm.c b/src/nms_adpcm.c +index 96d6ad2..96ea9f8 100644 +--- a/src/nms_adpcm.c ++++ b/src/nms_adpcm.c +@@ -704,7 +704,7 @@ psf_nms_adpcm_decode_block (SF_PRIVATE *psf, NMS_ADPCM_PRIVATE *pnms) + + if ((k = (int) psf_fread (pnms->block, sizeof (short), pnms->shortsperblock, psf)) != pnms->shortsperblock) + { psf_log_printf (psf, "*** Warning : short read (%d != %d).\n", k, pnms->shortsperblock) ; +- memset (pnms->block + (k * sizeof (short)), 0, (pnms->shortsperblock - k) * sizeof (short)) ; ++ memset (pnms->block + k, 0, (pnms->shortsperblock - k) * sizeof (short)) ; + } ; + + if (CPU_IS_BIG_ENDIAN) +diff --git a/src/vox_adpcm.c b/src/vox_adpcm.c +index e206675..cfc83ee 100644 +--- a/src/vox_adpcm.c ++++ b/src/vox_adpcm.c +@@ -139,6 +139,7 @@ vox_read_block (SF_PRIVATE *psf, IMA_OKI_ADPCM *pvox, short *ptr, int len) + + ima_oki_adpcm_decode_block (pvox) ; + ++ pvox->pcm_count = SF_MIN (pvox->pcm_count, len - indx) ; + memcpy (&(ptr [indx]), pvox->pcm, pvox->pcm_count * sizeof (short)) ; + indx += pvox->pcm_count ; + } ; diff --git a/recipes/flet-libvorbis/build.sh b/recipes/flet-libvorbis/build.sh new file mode 100755 index 00000000..2e2dbf80 --- /dev/null +++ b/recipes/flet-libvorbis/build.sh @@ -0,0 +1,35 @@ +#!/bin/bash +set -eu + +# Hand libvorbis's bundled FindOgg the flet-libogg paths directly: its +# find_path/find_library calls are re-rooted at the SDK sysroot under both +# cross toolchains, so a pkg-config HINT alone never resolves. +common_args="\ + -DCMAKE_BUILD_TYPE=Release \ + -DCMAKE_POLICY_VERSION_MINIMUM=3.5 \ + -DCMAKE_INSTALL_PREFIX=$PREFIX \ + -DCMAKE_POSITION_INDEPENDENT_CODE=ON \ + -DBUILD_SHARED_LIBS=OFF \ + -DBUILD_TESTING=OFF \ + -DOGG_INCLUDE_DIR=$PLATLIB/opt/include \ + -DOGG_LIBRARY=$PLATLIB/opt/lib/libogg.a" + +if [ "$CROSS_VENV_SDK" = "android" ]; then + cmake -B build \ + -DCMAKE_SYSTEM_NAME=Android \ + -DANDROID_PLATFORM=$SDK_VERSION \ + -DANDROID_ABI=$ANDROID_ABI \ + -DCMAKE_TOOLCHAIN_FILE=$NDK_ROOT/build/cmake/android.toolchain.cmake \ + $common_args +else + cmake -B build \ + -DCMAKE_SYSTEM_NAME=iOS \ + -DCMAKE_OSX_SYSROOT=$SDK \ + -DCMAKE_OSX_ARCHITECTURES=$HOST_ARCH \ + $common_args +fi + +cmake --build build -j "$CPU_COUNT" +cmake --install build + +rm -rf "$PREFIX/share" diff --git a/recipes/flet-libvorbis/meta.yaml b/recipes/flet-libvorbis/meta.yaml new file mode 100644 index 00000000..399f6cb1 --- /dev/null +++ b/recipes/flet-libvorbis/meta.yaml @@ -0,0 +1,22 @@ +{% set version = "1.3.7" %} + +package: + name: flet-libvorbis + version: '{{ version }}' + # Static PIC archives (vorbis + vorbisenc + vorbisfile) for flet-libsndfile's + # Ogg Vorbis support. Link-time only -- nothing ships to the device. + +build: + number: 1 + +source: + url: https://github.com/xiph/vorbis/releases/download/v{{ version }}/libvorbis-{{ version }}.tar.gz + +requirements: + build: + - cmake + host_build: + - flet-libogg 1.3.6 + +about: + license: BSD-3-Clause diff --git a/recipes/soundfile/README.md b/recipes/soundfile/README.md new file mode 100644 index 00000000..bd042bb9 --- /dev/null +++ b/recipes/soundfile/README.md @@ -0,0 +1,250 @@ +# soundfile + +[`soundfile`](https://github.com/bastibe/python-soundfile) reads and writes audio files as +numpy arrays. It is a thin cffi wrapper over +[libsndfile](https://libsndfile.github.io/libsndfile/), the C library that most of the +scientific-Python audio stack sits on — librosa, wfdb and pywavelets pipelines all expect a +`sf.read()` at the front. + +The wheel here is pure Python; the audio work happens in +[`flet-libsndfile`](../flet-libsndfile), which pip pulls in automatically. That library is +built with FLAC, Ogg Vorbis, Opus and MP3 compiled in, so the container list on a phone is +the same one you get on a desktop. + +## Install + +```toml +dependencies = [ + "flet", + "soundfile", +] +``` + +## Examples + +See runnable Flet apps in [`examples/`](examples): + +- [`codec-roundtrip`](examples/codec-roundtrip) — encodes a generated chord into every + container and reports the size, time and error of each round trip. + +## Usage in a Flet app + +`read` and `write` cover most of it — a path or any binary file object, in and out: + +```python +import soundfile as sf + +data, samplerate = sf.read("clip.wav") # float64 (frames,) or (frames, channels) +sf.write("out.flac", data, samplerate) # format inferred from the extension +``` + +`dtype` decides what you get back, and it matters more on a phone than on a desktop: +`float64` is the default and doubles your memory for no accuracy you can hear. + +```python +data, sr = sf.read("clip.wav", dtype="float32") # half the RAM +data, sr = sf.read("clip.wav", dtype="int16") # what the file already holds +``` + +For anything longer than a few seconds, read it in blocks rather than whole: + +```python +with sf.SoundFile("long.wav") as f: + print(f.samplerate, f.channels, len(f)) # header only, nothing decoded yet + for block in f.blocks(blocksize=16000, dtype="float32"): + process(block) +``` + +`sf.info(path)` is the cheapest call in the library — it opens the header and closes the +file, so it is how you check a samplerate or duration before deciding to decode. + +### Reading audio the user picked + +[`FilePicker`](https://flet.dev/docs/controls/filepicker/) hands back a path on both +platforms, and `sf.read()` takes it directly. **A real path is the only form that works +everywhere** — see the iOS note below. + +Where you hold bytes instead (a download, a file read out of `assets/`), `soundfile` does +accept any file object, and on Android that works: + +```python +import io + +data, sr = sf.read(io.BytesIO(raw_bytes)) # Android and desktop only +``` + +**On iOS this raises `MemoryError`,** and no version of the package will fix it. That path +uses libsndfile's virtual I/O, which means handing the C library a `ffi.callback()`; cffi +writes that trampoline into memory at runtime, and iOS refuses write+execute pages to an +app without the JIT entitlement. The message names the cause: + + MemoryError: Cannot allocate write+execute memory for ffi.callback(). + +Write the bytes to a file and read that instead — it costs one write, needs no +per-platform branch, and is the recommended form on both platforms: + +```python +import os + +path = os.path.join(os.getenv("FLET_APP_STORAGE_TEMP"), "clip.wav") +with open(path, "wb") as f: + f.write(raw_bytes) +data, sr = sf.read(path) +``` + +Only the *file-object* form is affected. Paths, and integer file descriptors (which go +through `sf_open_fd`), use no callbacks and work normally on iOS. + +### Storage + +`soundfile` reads and writes nothing of its own — no config, no cache, no env vars. Files you +write need a real writable directory, which on both platforms means +[`FLET_APP_STORAGE_DATA`](https://flet.dev/docs/reference/environment-variables#flet_app_storage_data) +for anything the user should keep, or +[`FLET_APP_STORAGE_TEMP`](https://flet.dev/docs/reference/environment-variables#flet_app_storage_temp) +for scratch. The app's working directory is not writable. + +```python +import os + +out = os.path.join(os.getenv("FLET_APP_STORAGE_DATA"), "recording.flac") +sf.write(out, data, 44100) +``` + +### Threading + +Decoding is CPU-bound and libsndfile releases the GIL, so put it in +[`page.run_thread(...)`](https://flet.dev/docs/controls/page/#flet.Page.run_thread) and end +the worker with an explicit +[`page.update()`](https://flet.dev/docs/controls/page/#flet.Page.update) — a background +thread does not get the automatic one. A `SoundFile` object carries a file position; do not +share one across threads, open one per thread. + +### App size + +`flet-libsndfile` is about 2.4 MB per Android ABI and 3.0 MB per iOS slice, all of it the +one shared library, and the `soundfile` wheel itself is around 27 KB. That figure is the +price of the codecs: an MP3 decoder, an MP3 encoder, FLAC, Vorbis and Opus account for most +of it, and there is no build flag here to drop them per app. + +`numpy` is the bigger line item for an app that does not already carry it. + +### Other considerations + +A desktop `flet run` uses PyPI's own `soundfile` wheel, which bundles its own copy of +libsndfile 1.2.2 built with the same codec set — so the format list matches what you get on +device, which is not something you can assume for every package. + +## Things to know + +- **`sf.read()` defaults to `float64`.** A three-minute stereo 44.1 kHz track is 127 MB as + `float64` and 63 MB as `float32`. On a phone that is the difference between working and + being killed by the OS. Pass `dtype="float32"` unless you specifically need the precision, + and prefer `blocks()` over reading whole files. + +- **iOS cannot read audio from a file object.** Repeating it here because it is the one + thing that behaves differently between the two platforms, and it fails at the point of + use rather than at import. `io.BytesIO` in, `MemoryError` out; write a file first. + +- **Opus only encodes at 48 kHz.** libsndfile will not resample for you; `sf.write(..., + format="OGG", subtype="OPUS")` at any other rate raises. Resample first — [`soxr`](../soxr) + is on pypi.flet.dev for exactly this. + +- **Writing MP3 works, and is usually the wrong choice.** The encoder (LAME) is compiled in, + so `sf.write("x.mp3", ...)` succeeds. For storing audio your app will read back, FLAC is + lossless, decodes faster and is only a few times larger. MP3 is for handing a file to + something else. + +- **`sf.available_formats()` is the honest answer to "is X supported?"** It asks libsndfile's + own registry rather than a table in Python, so it reflects this build and not upstream's + documentation. Worth calling once in development if you are unsure about a container. + +- **Lossy round trips do not line up sample-for-sample.** Vorbis, Opus and MP3 all add + codec delay and pad the tail, so a decoded file is longer than what you encoded and + shifted. Anything comparing audio before and after a lossy hop needs to align first — this + surprises people writing tests far more often than it affects playback. + +- **This is a `.so` the app loads at runtime, not a compiled extension.** If `import + soundfile` fails on device with an `OSError` about finding the library, the cause is + `flet-libsndfile` not being bundled — check it is in the build's resolved dependencies + rather than looking for a Python-side problem. + +- **libsndfile is LGPL-2.1-or-later and ships here as a replaceable shared library**, which + is the arrangement that licence is written for; the notices ride in the wheel under + `dist-info/licenses/`. LAME additionally asks that apps using it acknowledge LAME and link + to its site. This is a flag, not legal advice. + +## Build notes (maintainers) + +### Recipe shape + +Pattern H — a pure-Python wrapper plus a shared `flet-lib*` — the +[`pyzbar`](../pyzbar)/[`python-magic`](../python-magic) archetype, with one twist: the +wrapper uses **cffi**, not ctypes. + +That twist is the whole reason for `patches/mobile.patch`. `ffi.dlopen()` is a raw +`dlopen(3)`, so unlike `ctypes.CDLL` it cannot dereference the `.fwork` text pointer +serious-python leaves behind when it relocates a library into an iOS framework (that +dereference lives in iOS CPython's patched `ctypes/__init__.py`, nowhere lower). The patch +therefore wraps `ctypes.util.find_library`, lets **ctypes** resolve the name, and hands cffi +the absolute path ctypes settled on (`CDLL(candidate)._name`). Candidates, in order: the bare +soname `libsndfile.so` (Android jniLibs), `opt/lib/libsndfile.fwork` (iOS framework), +`opt/lib/libsndfile.so` (not relocated). Everything downstream of `find_library` is +upstream's, untouched. + +The patch is mandatory, not an optimisation: upstream's loader branches on `sys.platform` +over `darwin`/`win32`/`linux` only, and both fallbacks end in a bare `raise`. On +`android`/`ios` `import soundfile` fails outright without it. + +The codec chain is six `flet-lib*` recipes — `flet-libogg`, `flet-libvorbis`, +`flet-libflac`, `flet-libopus`, `flet-libmpg123`, `flet-libmp3lame` — built as **static PIC +archives** and declared `requirements.host_build` of `flet-libsndfile`, which absorbs them +into its one shared library. They are deliberately not `requirements.host`: nothing of them +is loaded at runtime, so promoting them to `Requires-Dist` would make every consuming app +download six wheels whose contents already live inside `libsndfile.so`. + +### Upgrade hazards + +- **There has been no libsndfile release since 1.2.2 (August 2023).** `flet-libsndfile` + carries `patches/security-backports.patch`, five upstream memory-safety fixes cherry-picked + from master. When upstream finally tags a release, drop that patch and re-check + `git log 1.2.2..` rather than assuming the list was complete. +- **soundfile's loader has no anchor comments.** The patch keys on the + `from ctypes.util import find_library` import and on the `try: # packaged lib` line. A + bump that reworks the loader will need the patch rewritten, not rebased — read + `soundfile.py`'s platform dispatch before assuming a clean apply. +- **`numpy` became a hard dependency in soundfile 0.13.0.** There is no numpy-free build. +- **soundfile's own `bdist_wheel` mis-tags under cross-compilation** — it emits + `py2.py3-none-any` for an unknown `sys.platform`. forge's `fix_wheel` overwrites the tag, + which is what makes our wheel outrank PyPI's `any` wheel at the same version. Do not + "fix" the tag in a patch. +- **FLAC 1.5.0 needs `-DCMAKE_SYSTEM_VERSION` on Android** and `-DENABLE_MULTITHREADING=OFF`; + both have comments in `recipes/flet-libflac/build.sh` saying what breaks without them. + +### Re-verification checklist + +- **Codecs actually linked:** `sf.available_formats()` on device must list `FLAC`, `OGG` and + `MP3`, and `available_subtypes("OGG")` must be exactly `{VORBIS, OPUS}`. The CMake configure + summary is not sufficient — `ENABLE_EXTERNAL_LIBS`/`ENABLE_MPEG` silently downgrade to OFF. +- **Nothing leaked from the build host:** `flet-libsndfile` passes + `-DCMAKE_FIND_USE_CMAKE_SYSTEM_PATH=NO` so a Homebrew `flac`/`opus` cannot be found instead + of ours. If that is ever removed, a developer machine and CI will produce different wheels. +- **Exported symbols are `sf_*` only** — `nm -gU` on iOS (41 symbols, all `_sf_`), `llvm-nm + -D --defined-only` on Android (version-scripted `@@libsndfile.so.1.0`). The absorbed codec + symbols must stay hidden; jniLibs is a flat namespace shared with every other native wheel. +- **Wheel hygiene:** one unversioned `opt/lib/libsndfile.so` per slice; Android `DT_NEEDED` + limited to `libc`/`libm`/`libdl`, `SONAME` exactly `libsndfile.so`, every `LOAD` aligned + `0x4000`; iOS `otool -L` showing only `libSystem`, filetype `DYLIB`. +- **METADATA:** `soundfile` promotes `flet-libsndfile (==1.2.2)`; the six codec libraries + must NOT appear. + +### Coverage gaps + +The device tests cover loading the library, WAV round trips in three subtypes, every +container and codec, virtual I/O from `BytesIO` (Android) and its `MemoryError` on iOS, +the write-a-file workaround, block reads with seeking, and the format registry. They do not +cover: writing to `FLET_APP_STORAGE_*` (a path question, not a +libsndfile one), any real recorded audio file, `sf.info()`, the `RAW` format's manual +`samplerate`/`channels`/`subtype` arguments, or multi-threaded decoding. The security +backports are compile-verified only — there are no regression tests for the five fixes, since +reproducing them needs the malformed inputs from upstream's fuzz corpus. diff --git a/recipes/soundfile/examples/codec-roundtrip/.gitignore b/recipes/soundfile/examples/codec-roundtrip/.gitignore new file mode 100644 index 00000000..429a8307 --- /dev/null +++ b/recipes/soundfile/examples/codec-roundtrip/.gitignore @@ -0,0 +1,7 @@ +.venv/ +.flet/ +build/ +__pycache__/ +.pytest_cache/ +.ruff_cache/ +uv.lock diff --git a/recipes/soundfile/examples/codec-roundtrip/README.md b/recipes/soundfile/examples/codec-roundtrip/README.md new file mode 100644 index 00000000..e297fbb7 --- /dev/null +++ b/recipes/soundfile/examples/codec-roundtrip/README.md @@ -0,0 +1,55 @@ +# soundfile codec round-trip + +A two-second chord, generated in numpy when the app starts, then encoded into every container +this build of libsndfile supports and decoded straight back. Each row reports the encoded +size, how that compares with the raw float64 samples, how long the round trip took, and how +far the decoded audio drifted from the original. + +What it demonstrates: + +- **Which formats you actually get on a phone.** The rows are not a table copied from + documentation — each one is a real `sf.write()` followed by a real `sf.read()` on the + device. FLAC, Ogg Vorbis, Opus and MP3 appear because `flet-libsndfile` links libFLAC, + libvorbis, libopus, libmpg123 and LAME; if a codec were missing from a build, its row + would show the error libsndfile raised instead of a size. +- **The size decision, with numbers.** Two seconds of audio is 128 kB as raw float64 and + around 3 kB as MP3. The compression column is the argument for not storing WAV on a device + with a user's storage quota — and the error column is what that costs. +- **Lossless is not lossy-free.** FLAC and ALAC come back with a small error rather than + zero, because both are 16-bit integer formats and the source is float64. It is + quantisation, not codec loss — a distinction worth seeing once. +- **Writing into Flet's storage.** Each round trip writes a real file under + [`FLET_APP_STORAGE_TEMP`](https://flet.dev/docs/reference/environment-variables#flet_app_storage_temp) + and deletes it again. Files rather than `io.BytesIO` deliberately: a file object routes + through libsndfile's virtual I/O, which needs a cffi callback, which needs write+execute + memory that iOS refuses — so the file-object form works on Android and raises + `MemoryError` on iOS. A path works on both. +- **Compute off the UI thread.** The sweep runs in + [`page.run_thread(...)`](https://flet.dev/docs/controls/page/#flet.Page.run_thread) with a + spinner up, ending in the explicit + [`page.update()`](https://flet.dev/docs/controls/page/#flet.Page.update) a background + thread needs. libsndfile releases the GIL while coding, so this is real parallelism. + +The two waveform strips are peak envelopes: the generated source, and the audio recovered +from the MP3 — near enough to look identical at this scale, which is the point of a lossy +codec. + +The `time` column includes the file write and read, so it is a whole-round-trip figure +rather than a codec benchmark. + +The audio is generated rather than bundled, so the example ships no asset. + +## Try it + +[Build](https://flet.dev/docs/publish/) the app, then install it on a device or emulator/simulator: + +```bash +# Android +uv run flet build apk + +# iOS +uv run flet build ipa + +# iOS-Simulator +uv run flet build ios-simulator +``` diff --git a/recipes/soundfile/examples/codec-roundtrip/pyproject.toml b/recipes/soundfile/examples/codec-roundtrip/pyproject.toml new file mode 100644 index 00000000..552c1a27 --- /dev/null +++ b/recipes/soundfile/examples/codec-roundtrip/pyproject.toml @@ -0,0 +1,16 @@ +[project] +name = "soundfile-codec-roundtrip" +version = "1.0.0" +description = "Encodes a generated chord to every container libsndfile supports and decodes it back." +requires-python = ">=3.12" + +dependencies = [ + "flet==0.86.5", + "soundfile==0.14.0", +] + +[dependency-groups] +dev = ["flet-cli", "flet-desktop", "flet-web"] + +[tool.flet.app] +path = "src" diff --git a/recipes/soundfile/examples/codec-roundtrip/src/audio_codecs.py b/recipes/soundfile/examples/codec-roundtrip/src/audio_codecs.py new file mode 100644 index 00000000..83ef6fa5 --- /dev/null +++ b/recipes/soundfile/examples/codec-roundtrip/src/audio_codecs.py @@ -0,0 +1,91 @@ +"""Encode a generated signal to every container this wheel supports, decode it back, +and report what each one cost. No soundfile object escapes: callers get plain values.""" + +import os +import tempfile +import time + +import numpy as np +import soundfile as sf + +RATE = 16000 +SECONDS = 2.0 + +# (label, libsndfile format, subtype, samplerate). Opus only accepts 48 kHz, so it +# gets its own rate rather than being silently resampled by libsndfile. +CODECS = [ + ("WAV / PCM_16", "WAV", "PCM_16", RATE), + ("WAV / FLOAT", "WAV", "FLOAT", RATE), + ("FLAC", "FLAC", "PCM_16", RATE), + ("CAF / ALAC", "CAF", "ALAC_16", RATE), + ("OGG / Vorbis", "OGG", "VORBIS", RATE), + ("OGG / Opus", "OGG", "OPUS", 48000), + ("MP3", "MP3", "MPEG_LAYER_III", RATE), +] + + +def signal(rate=RATE, seconds=SECONDS): + """A deterministic three-partial chord — compresses like real audio, unlike a + pure sine, so the codec sizes below are not flattering.""" + t = np.arange(int(rate * seconds)) / rate + wave = sum(a * np.sin(2 * np.pi * f * t) for f, a in ((220, 0.5), (330, 0.3), (550, 0.2))) + envelope = np.minimum(1.0, 8 * np.minimum(t, seconds - t)) + return (wave * envelope).astype(np.float64) + + +def roundtrip(label, fmt, subtype, rate): + """Encode to a container on disk and decode it back. + + Returns the label, encoded size in bytes, encode+decode time in seconds, and the + RMS difference from the source — quantisation-limited for the integer formats, + audible-codec-sized for the lossy ones. Returns an `error` string instead if the + format is not compiled in. + + Files, not io.BytesIO: a file object would route through libsndfile's virtual I/O, + which needs a cffi callback, which needs write+execute memory that iOS refuses. + """ + source = signal(rate) + # FLET_APP_STORAGE_TEMP on device; tempfile's default elsewhere. + directory = os.getenv("FLET_APP_STORAGE_TEMP") or tempfile.gettempdir() + path = os.path.join(directory, f"roundtrip-{subtype.lower()}.{fmt.lower()}") + started = time.monotonic() + try: + sf.write(path, source, rate, format=fmt, subtype=subtype) + size = os.path.getsize(path) + decoded, out_rate = sf.read(path) + except Exception as exc: # unsupported subtype, missing codec, ... + return {"label": label, "error": str(exc)} + finally: + if os.path.exists(path): + os.remove(path) + elapsed = time.monotonic() - started + + # Lossy codecs add leading silence and pad the tail, so compare the overlap. + n = min(len(decoded), len(source)) + rms = float(np.sqrt(np.mean((decoded[:n] - source[:n]) ** 2))) + + return { + "label": label, + "bytes": size, + "ratio": (source.nbytes / size), + "seconds": elapsed, + "rms_error": rms, + "frames": len(decoded), + "rate": out_rate, + "waveform": decoded, + } + + +def run_all(): + """Round-trip every codec in CODECS, in order.""" + return [roundtrip(*entry) for entry in CODECS] + + +def envelope(samples, buckets=120): + """Peak amplitude per bucket, normalised to 0..1 — enough to draw a waveform.""" + if len(samples) == 0: + return [0.0] * buckets + edges = np.linspace(0, len(samples), buckets + 1).astype(int) + peaks = np.array([np.max(np.abs(samples[a:b]), initial=0.0) for a, b in zip(edges, edges[1:])]) + top = peaks.max() + return (peaks / top if top else peaks).tolist() diff --git a/recipes/soundfile/examples/codec-roundtrip/src/main.py b/recipes/soundfile/examples/codec-roundtrip/src/main.py new file mode 100644 index 00000000..70169b15 --- /dev/null +++ b/recipes/soundfile/examples/codec-roundtrip/src/main.py @@ -0,0 +1,128 @@ +import flet as ft +import soundfile as sf +from audio_codecs import RATE, SECONDS, envelope, run_all, signal + +BARS = 64 +WAVE_HEIGHT = 70 + + +def bars(levels, color): + """A waveform strip: one centred bar per bucket, height scaled to its peak.""" + return ft.Row( + controls=[ + ft.Container( + width=3, + height=max(2, level * WAVE_HEIGHT), + bgcolor=color, + border_radius=1, + ) + for level in levels + ], + spacing=1, + alignment=ft.MainAxisAlignment.CENTER, + vertical_alignment=ft.CrossAxisAlignment.CENTER, + height=WAVE_HEIGHT, + ) + + +def cell(text, width, color=None, mono=True): + """One fixed-width table cell.""" + return ft.Container( + width=width, + content=ft.Text( + text, + size=11, + color=color, + font_family="monospace" if mono else None, + ), + ) + + +def row_for(result): + """One result line: size, compression against raw float64, time, RMS error.""" + if "error" in result: + return ft.Row( + controls=[ + cell(result["label"], 110, mono=False), + cell(result["error"][:40], 200, color=ft.Colors.RED), + ] + ) + return ft.Row( + controls=[ + cell(result["label"], 110, mono=False), + cell(f"{result['bytes'] / 1000:.1f} kB", 62), + cell(f"{result['ratio']:.0f}x", 40, color=ft.Colors.BLUE), + cell(f"{result['seconds'] * 1e3:.0f} ms", 55), + cell(f"{result['rms_error']:.0e}", 55, color=ft.Colors.GREY), + ] + ) + + +def main(page: ft.Page): + def encode_all(): + """The whole sweep. Runs on a worker thread; soundfile releases the GIL.""" + button.disabled = True + spinner.visible = True + page.update() + + results = run_all() + table.controls = [ + ft.Row( + controls=[ + cell("", 110), + cell("size", 62), + cell("vs raw", 40), + cell("time", 55), + cell("rms err", 55), + ] + ), + ft.Divider(height=1), + *(row_for(r) for r in results), + ] + + lossy = next(r for r in results if r["label"] == "MP3" and "waveform" in r) + decoded.controls = [ + ft.Text("decoded MP3", size=11), + bars(envelope(lossy["waveform"], BARS), ft.Colors.ORANGE), + ] + + button.disabled = False + spinner.visible = False + page.update() # auto-update does not reach background threads + + page.appbar = ft.AppBar(title=ft.Text("soundfile round-trip"), center_title=True) + page.add( + ft.SafeArea( + expand=True, + content=ft.Column( + scroll=ft.ScrollMode.AUTO, + controls=[ + ft.Text( + f"{SECONDS:g} s chord, {RATE // 1000} kHz, generated in numpy", + size=11, + ), + bars(envelope(signal(), BARS), ft.Colors.BLUE), + ft.Row( + controls=[ + button := ft.Button( + "Re-encode", + on_click=lambda: page.run_thread(encode_all), + ), + spinner := ft.ProgressRing( + visible=False, width=18, height=18 + ), + ] + ), + table := ft.Column(spacing=2), + ft.Divider(), + decoded := ft.Column(spacing=2), + ft.Divider(), + ft.Text(f"libsndfile {sf.__libsndfile_version__}", size=11), + ], + ), + ) + ) + page.run_thread(encode_all) + + +ft.run(main) diff --git a/recipes/soundfile/meta.yaml b/recipes/soundfile/meta.yaml new file mode 100644 index 00000000..de510e0e --- /dev/null +++ b/recipes/soundfile/meta.yaml @@ -0,0 +1,24 @@ +{% set libsndfile_version = "1.2.2" %} + +package: + name: soundfile + version: "0.14.0" + +build: + number: 1 + +requirements: + build: + # setup.py-only sdist (no [build-system] table) -- seed the legacy + # setuptools backend, plus the cffi that setup.py's `cffi_modules` hook + # needs to generate the ABI-mode _soundfile.py. + - setuptools + - cffi + host: + # soundfile is pure Python: it dlopens libsndfile through cffi at import + # time (see mobile.patch). Ship the library -- fix_wheel promotes this to + # a Requires-Dist so the app bundles the platform .so. + - flet-libsndfile {{ libsndfile_version }} + +patches: + - mobile.patch diff --git a/recipes/soundfile/patches/mobile.patch b/recipes/soundfile/patches/mobile.patch new file mode 100644 index 00000000..3cacf71e --- /dev/null +++ b/recipes/soundfile/patches/mobile.patch @@ -0,0 +1,73 @@ +Make soundfile find libsndfile on Flet's mobile runtimes. + +soundfile is pure Python: it dlopens libsndfile through cffi at import time. +Its loader knows darwin/win32/linux only, so on Android (sys.platform +'android') and iOS ('ios') every branch misses, ctypes.util.find_library() +returns None -- there is no ld cache or compiler to consult -- and the chain +ends in a bare `raise`: `import soundfile` fails outright. + +Wrap find_library() so that, on those two platforms only, it also looks for the +library flet-libsndfile ships. serious-python surfaces it as an Android jniLibs +`libsndfile.so` (loadable by bare soname) or, on iOS, an embedded framework +plus a `libsndfile.fwork` text pointer beside the wheel's opt/lib. cffi's +dlopen is the raw dlopen(3) and cannot dereference a .fwork, so the lookup goes +through ctypes -- iOS CPython patches CDLL to follow the pointer -- and returns +the path ctypes settled on for cffi to open. Everything downstream of +find_library() is upstream's, unchanged. + +diff -ruN pa/soundfile.py pb/soundfile.py +--- pa/soundfile.py 2026-06-06 10:55:35 +diff -ruN pa/soundfile.py pb/soundfile.py +--- pa/soundfile.py 2026-06-06 10:55:35 ++++ pb/soundfile.py 2026-09-14 20:55:33 +@@ -14,7 +14,7 @@ + import sys as _sys + import threading as _threading + from collections.abc import Generator +-from ctypes.util import find_library as _find_library ++from ctypes.util import find_library as _ctypes_find_library + from os import SEEK_CUR, SEEK_END, SEEK_SET + from typing import Any, BinaryIO, Final, Literal, TypeAlias + +@@ -158,6 +158,41 @@ + 'AVERAGE': 1, + 'VARIABLE': 2, + } ++ ++ ++def _find_library(name: str) -> str | None: ++ """Locate a shared library by name, extended for Flet's mobile runtimes. ++ ++ `ctypes.util.find_library()` has no ld cache or compiler to consult on ++ Android or iOS and always returns None there, so the fallbacks below -- ++ which only know darwin/win32/linux -- end in a bare `raise` and ++ `import soundfile` dies. The flet-libsndfile wheel ships libsndfile, and ++ serious-python surfaces it as an Android jniLibs `libsndfile.so` or, on ++ iOS, an embedded framework plus a `libsndfile.fwork` text pointer beside ++ the wheel's opt/lib. cffi's dlopen is the raw dlopen(3) and cannot ++ dereference a .fwork, so resolution goes through ctypes -- iOS CPython ++ patches `CDLL` to follow the pointer -- and cffi is handed the path ctypes ++ settled on. ++ """ ++ path = _ctypes_find_library(name) ++ if path is not None or _sys.platform not in ('android', 'ios'): ++ return path ++ ++ import ctypes as _ctypes ++ ++ _optlib = _os.path.join( ++ _os.path.dirname(_os.path.abspath(__file__)), 'opt', 'lib') ++ for _candidate in ( ++ f'lib{name}.so', # Android jniLibs, by soname ++ _os.path.join(_optlib, f'lib{name}.fwork'), # iOS embedded framework ++ _os.path.join(_optlib, f'lib{name}.so'), # not relocated (either platform) ++ ): ++ try: ++ return _ctypes.CDLL(_candidate)._name ++ except OSError: ++ continue ++ return None ++ + + try: # packaged lib (in _soundfile_data which should be on python path) + if _sys.platform == 'darwin': diff --git a/recipes/soundfile/tests/test_soundfile.py b/recipes/soundfile/tests/test_soundfile.py new file mode 100644 index 00000000..bdf42a8a --- /dev/null +++ b/recipes/soundfile/tests/test_soundfile.py @@ -0,0 +1,165 @@ +import io +import sys + +import numpy as np +import pytest +import soundfile as sf + +SR = 16000 + + +def _tone(seconds=0.25, freq=440.0, sample_rate=SR): + """Deterministic mono float64 sine — no RNG, no assets, no network.""" + t = np.arange(int(sample_rate * seconds)) / sample_rate + return np.sin(2.0 * np.pi * freq * t) + + +def test_libsndfile_loaded(): + """The cffi loader found the flet-libsndfile .so and bound the C API — this is + the single claim the mobile patch exists to make true.""" + assert sf.__libsndfile_version__ + assert sf.SoundFile is not None + + +def test_wav_roundtrip(tmp_path): + """A float64 signal written as 32-bit float WAV reads back bit-identical, with + the samplerate and channel count preserved.""" + path = str(tmp_path / "tone.wav") + x = _tone() + sf.write(path, x, SR, subtype="DOUBLE") + + y, sr = sf.read(path) + assert sr == SR + assert y.shape == x.shape + np.testing.assert_array_equal(y, x) + + +def test_stereo_and_int_subtypes(tmp_path): + """Two-channel PCM_16 and PCM_24 round-trip within their quantisation step — + covers the integer conversion paths, not just the float passthrough.""" + x = np.stack([_tone(), 0.5 * _tone(freq=880.0)], axis=1) + for subtype, step in (("PCM_16", 2**-15), ("PCM_24", 2**-23)): + path = str(tmp_path / f"{subtype}.wav") + sf.write(path, x, SR, subtype=subtype) + y, sr = sf.read(path) + assert sr == SR + assert y.shape == x.shape + assert np.max(np.abs(y - x)) <= step + + +@pytest.mark.parametrize( + "fmt, ext, subtype, lossy", + [ + ("WAV", "wav", "PCM_16", False), + ("AIFF", "aiff", "PCM_16", False), + ("AU", "au", "PCM_16", False), + ("CAF", "caf", "PCM_16", False), + ("W64", "w64", "PCM_16", False), + ("RF64", "rf64", "PCM_16", False), + ("FLAC", "flac", "PCM_16", False), + ("OGG", "ogg", "VORBIS", True), + ("OGG", "opus", "OPUS", True), + ("MP3", "mp3", "MPEG_LAYER_III", True), + ], +) +def test_container_roundtrip(tmp_path, fmt, ext, subtype, lossy): + """Every container this wheel's libsndfile is built for encodes and decodes on + device — including the ones that only work because flet-libflac, flet-libogg, + flet-libvorbis, flet-libopus, flet-libmpg123 and flet-libmp3lame are linked in. + Opus resamples to 48 kHz internally, so its frame count is not compared.""" + path = str(tmp_path / f"tone.{ext}") + x = _tone(seconds=0.5) + sr = 48000 if subtype == "OPUS" else SR + sf.write(path, x, sr, format=fmt, subtype=subtype) + + y, out_sr = sf.read(path) + assert out_sr == sr + assert y.ndim == 1 + if lossy: + # Codec delay and framing make a sample-wise comparison meaningless; + # assert the decode produced a real signal of roughly the right length. + assert y.size >= x.size // 2 + assert np.sqrt(np.mean(y**2)) > 0.1 + else: + assert y.shape == x.shape + assert np.max(np.abs(y - x)) <= 2**-15 + + +@pytest.mark.skipif(sys.platform == "ios", reason="no W+X memory for ffi.callback()") +def test_read_from_file_object(): + """sf.read() works on an in-memory binary stream via libsndfile's virtual I/O — + the path an app takes for audio fetched over the network, where there may be + no real file to open. Android only; see the iOS counterpart below.""" + buf = io.BytesIO() + x = _tone() + sf.write(buf, x, SR, format="WAV", subtype="DOUBLE") + + buf.seek(0) + y, sr = sf.read(buf) + assert sr == SR + np.testing.assert_array_equal(y, x) + + +@pytest.mark.skipif(sys.platform != "ios", reason="iOS-only limitation") +def test_file_object_is_unavailable_on_ios(): + """Passing a file object on iOS raises MemoryError, and this is permanent: virtual + I/O hands libsndfile a `ffi.callback()`, cffi writes that trampoline at runtime, and + iOS refuses write+execute pages to an app without the JIT entitlement. Asserted + rather than skipped so the README's "write it to a file first" guidance is checked, + and so a future cffi or OS change that lifts it does not go unnoticed.""" + buf = io.BytesIO() + with pytest.raises(MemoryError, match="write.execute"): + sf.write(buf, _tone(), SR, format="WAV", subtype="DOUBLE") + + +def test_path_roundtrip_is_the_ios_workaround(tmp_path): + """The replacement for the file-object path: spill the bytes to a real file and read + it back. Works on every platform, so an app needs no per-platform branch.""" + raw = io.BytesIO() + x = _tone() + try: + sf.write(raw, x, SR, format="WAV", subtype="DOUBLE") + encoded = raw.getvalue() + except MemoryError: # iOS: produce the same bytes without virtual I/O + staged = str(tmp_path / "staged.wav") + sf.write(staged, x, SR, subtype="DOUBLE") + encoded = open(staged, "rb").read() + + path = tmp_path / "from_bytes.wav" + path.write_bytes(encoded) + y, sr = sf.read(str(path)) + assert sr == SR + np.testing.assert_array_equal(y, x) + + +def test_blockwise_read(tmp_path): + """SoundFile seeking and block reads reassemble the whole signal — the memory- + bounded path for files too large to hold in RAM on a phone.""" + path = str(tmp_path / "tone.wav") + x = _tone(seconds=1.0) + sf.write(path, x, SR, subtype="DOUBLE") + + with sf.SoundFile(path) as f: + assert len(f) == x.size + assert f.samplerate == SR + assert f.channels == 1 + + blocks = [f.read(1024) for _ in range(int(np.ceil(x.size / 1024)))] + np.testing.assert_array_equal(np.concatenate(blocks), x) + + f.seek(SR // 2) + np.testing.assert_array_equal(f.read(100), x[SR // 2 : SR // 2 + 100]) + + +def test_available_formats_advertises_the_linked_codecs(): + """libsndfile's own format registry lists the codec containers, proving they + were compiled in rather than merely present at link time.""" + formats = sf.available_formats() + for fmt in ("WAV", "AIFF", "AU", "CAF", "W64", "RF64", "FLAC", "OGG", "MP3"): + assert fmt in formats, f"{fmt} missing from {sorted(formats)}" + + assert set(sf.available_subtypes("FLAC")) >= {"PCM_16", "PCM_24"} + assert set(sf.available_subtypes("OGG")) == {"VORBIS", "OPUS"} + assert "MPEG_LAYER_III" in sf.available_subtypes("MP3") + # ALAC ships inside libsndfile itself, no external library involved. + assert "ALAC_16" in sf.available_subtypes("CAF")