Request a format Filee can't read, from the donut and the toast

.github/ISSUE_TEMPLATE/feature_request.md

@@ -1,6 +1,6 @@
 ---
 name: Feature request
-about: A new format, preset option or idea
+about: A preset option, engine or idea (a file type Filee can't open goes in "Format request")
 labels: enhancement
 ---
 

.github/ISSUE_TEMPLATE/format_request.yml

@@ -0,0 +1,49 @@
+name: Format request
+description: A file type Filee can't open yet
+title: "Format request: "
+labels: ["enhancement"]
+body:
+  - type: markdown
+    attributes:
+      value: |
+        > [!IMPORTANT]
+        > **Filee can't convert files that are encrypted or copy-protected (DRM), whatever their format.** Converting them means breaking that protection, so these requests are closed.
+        >
+        > Some examples, not a complete list:
+        > - music downloaded in a streaming app (NetEase Cloud Music `.ncm`, QQ Music `.qmc` / `.mflac` / `.mgg`, Kugou `.kgm`, Apple Music, Spotify)
+        > - e-books bought with DRM (Kindle `.azw` / `.kfx`, Adobe DRM EPUB or PDF)
+        > - videos saved by streaming apps or sites that don't allow downloading
+        > - files locked with a password you don't have
+        >
+        > If the app that made the file can export or save it in a common format, that is the way to go.
+  - type: input
+    id: extension
+    attributes:
+      label: File extension
+      placeholder: ".xyz"
+    validations:
+      required: true
+  - type: input
+    id: source
+    attributes:
+      label: What makes these files?
+      description: The program, device or website.
+    validations:
+      required: true
+  - type: input
+    id: target
+    attributes:
+      label: What should Filee convert them to?
+      placeholder: "PDF, MP3, PNG…"
+  - type: textarea
+    id: details
+    attributes:
+      label: Anything else?
+      description: A sample file if you can share one, a link to the format's documentation, or an open-source tool that reads it.
+  - type: checkboxes
+    id: not-protected
+    attributes:
+      label: Protection
+      options:
+        - label: These files aren't encrypted or copy-protected (DRM), and I can open them without a password.
+          required: true

.github/workflows/portable.yml

@@ -1,5 +1,11 @@
 name: Linux and macOS packages
 
+# Linux packages are built on Ubuntu 24.04 but must run on Ubuntu 22.04 (glibc 2.35). The rhwp CLI release needs a
+# newer glibc, so it is built from pinned source on Ubuntu 22.04 first (linux-native, build/build-linux-native.py) and
+# replaces the download in the package. SharpHook (>= 8.0.1) ships libuiohook builds for glibc 2.34 and is packaged as
+# is. Every ELF file in a Linux package is then checked against glibc 2.35, and the x64 and arm64 packages are tried on
+# Ubuntu 22.04, which also catches a SharpHook or engine update that raises the requirement again.
+
 on:
   workflow_call:
     inputs:
@@ -18,7 +24,80 @@ permissions:
   contents: read
 
 jobs:
+  linux-native:
+    name: Linux native ${{ matrix.rid }} (Ubuntu 22.04)
+    strategy:
+      fail-fast: false
+      matrix:
+        include:
+          - runner: ubuntu-22.04
+            rid: linux-x64
+          - runner: ubuntu-22.04-arm
+            rid: linux-arm64
+    runs-on: ${{ matrix.runner }}
+    timeout-minutes: 90
+    defaults:
+      run:
+        shell: bash
+    env:
+      NATIVE_OUT: ${{ github.workspace }}/build/.cache/linux-native/${{ matrix.rid }}
+    steps:
+      - uses: actions/checkout@v4
+
+      # Fails when engines.json moved to another rhwp version while build/build-linux-native.py still builds the old
+      # source (also when the build below comes from the cache).
+      - name: Check pinned sources
+        run: python3 build/build-linux-native.py check
+
+      - name: Restore native builds
+        id: native-cache
+        uses: actions/cache@v4
+        with:
+          path: build/.cache/linux-native/${{ matrix.rid }}
+          key: linux-native-${{ matrix.rid }}-ubuntu-22.04-${{ hashFiles('build/build-linux-native.py') }}
+
+      # rhwp's default features are pure Rust: a C toolchain for the linker and build scripts is enough.
+      - name: Build dependencies
+        if: steps.native-cache.outputs.cache-hit != 'true'
+        run: |
+          set -euo pipefail
+          sudo apt-get update
+          sudo apt-get install -y build-essential pkg-config git
+          if ! command -v rustup >/dev/null; then
+            curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --profile minimal --default-toolchain none
+            echo "$HOME/.cargo/bin" >> "$GITHUB_PATH"
+          fi
+
+      - name: Restore cargo downloads
+        if: steps.native-cache.outputs.cache-hit != 'true'
+        uses: actions/cache@v4
+        with:
+          path: |
+            ~/.cargo/registry/index
+            ~/.cargo/registry/cache
+            ~/.cargo/git/db
+          key: cargo-registry-${{ matrix.rid }}-${{ hashFiles('build/build-linux-native.py') }}
+          restore-keys: cargo-registry-${{ matrix.rid }}-
+
+      - name: Build rhwp
+        if: steps.native-cache.outputs.cache-hit != 'true'
+        run: python3 build/build-linux-native.py rhwp --output "$NATIVE_OUT/rhwp" --work "$RUNNER_TEMP/rhwp"
+
+      - name: Check glibc requirements
+        run: python3 build/check-elf-glibc.py --max 2.35 --needed "$NATIVE_OUT"
+
+      - name: Upload native builds
+        uses: actions/upload-artifact@v4
+        with:
+          name: linux-native-${{ matrix.rid }}
+          path: build/.cache/linux-native/${{ matrix.rid }}/
+          if-no-files-found: error
+
   portable:
+    needs: linux-native
+    # macOS packages don't use the Linux builds: they still run when those failed (the Linux legs then stop at the
+    # download).
+    if: ${{ !cancelled() }}
     strategy:
       fail-fast: false
       matrix:
@@ -39,6 +118,13 @@ jobs:
     steps:
       - uses: actions/checkout@v4
 
+      - name: Download Linux native builds
+        if: runner.os == 'Linux'
+        uses: actions/download-artifact@v4
+        with:
+          name: linux-native-${{ matrix.rid }}
+          path: ${{ runner.temp }}/linux-native
+
       - uses: actions/setup-dotnet@v4
         with:
           global-json-file: global.json
@@ -49,7 +135,7 @@ jobs:
         run: |
           sudo apt-get update
           sudo apt-get install -y ca-certificates libc6 libgcc-s1 libgssapi-krb5-2 libicu74 libssl3t64 libstdc++6 tzdata zlib1g \
-            libfontconfig1 libfreetype6 libx11-6 libice6 libsm6 libxrandr2 libxkbcommon0 libxi6 libxtst6 xvfb fonts-noto-cjk
+            libfontconfig1 libfreetype6 libx11-6 libice6 libsm6 libxrandr2 libxkbcommon0 libxi6 libxtst6 libxt6 xvfb fonts-noto-cjk
 
       - name: Fonts
         run: ./build/fetch-fonts.ps1
@@ -77,106 +163,28 @@ jobs:
             if ($LASTEXITCODE -ne 0) { exit $LASTEXITCODE }
           }
 
+      # Linux: rhwp from linux-native instead of the download (see the top).
       - name: Build portable package
         env:
           PACKAGE_VERSION: ${{ inputs.version }}
-        run: ./build/build-portable.ps1 -Runtime '${{ matrix.rid }}' -Version $env:PACKAGE_VERSION -SkipFonts
-
-      - name: Check archive and run native converters
         run: |
-          $archives = @(Get-ChildItem Releases -Filter '*-${{ matrix.rid }}.tar.gz')
-          if ($archives.Count -ne 1) { throw 'Expected one portable archive.' }
-          $archive = $archives[0]
-          $expected = (Get-Content (Join-Path Releases ($archive.Name.Replace('.tar.gz', '-SHA256SUMS.txt')))).Split(' ')[0]
-          if ((Get-FileHash $archive.FullName -Algorithm SHA256).Hash -ne $expected) { throw 'Archive checksum mismatch.' }
-          $smoke = Join-Path $env:RUNNER_TEMP 'filee-smoke'
-          New-Item -ItemType Directory -Path $smoke | Out-Null
-          tar -xzf $archive.FullName -C $smoke
-          if ($LASTEXITCODE -ne 0) { throw 'Archive extraction failed.' }
-          $package = Join-Path $smoke ($archive.Name.Replace('.tar.gz', ''))
-          if ($IsMacOS) {
-            codesign --verify --deep --strict (Join-Path $package 'Filee.app')
-            if ($LASTEXITCODE -ne 0) { throw 'Extracted app signature did not verify.' }
+          $native = @{}
+          if ($IsLinux) {
+            $native = @{ RhwpOverride = Join-Path $env:RUNNER_TEMP 'linux-native/rhwp' }
           }
-          $bin = if ($IsMacOS) { Join-Path $package 'Filee.app/Contents/MacOS' } else { $package }
-          $cli = Join-Path $bin 'filee-cli'
-          & $cli version
-          if ($LASTEXITCODE -ne 0) { throw 'CLI did not start.' }
-          & (Join-Path $bin 'engines/rhwp/rhwp') --version
-          if ($LASTEXITCODE -ne 0) { throw 'rhwp did not start.' }
-          & (Join-Path $bin 'engines/7zip/7zz') i | Out-Null
-          if ($LASTEXITCODE -ne 0) { throw '7-Zip did not start.' }
-          $inputFile = Join-Path $smoke 'sample.txt'
-          'Filee 한글 conversion smoke test' | Set-Content $inputFile -Encoding utf8NoBOM
-          & $cli convert $inputFile --to hwpx -o (Join-Path $smoke 'converted') --json
-          if ($LASTEXITCODE -ne 0 -or -not (Test-Path (Join-Path $smoke 'converted/sample.hwpx'))) { throw 'Published CLI conversion failed.' }
-          "FILEE_SMOKE_APP=$(Join-Path $bin 'Filee')" >> $env:GITHUB_ENV
+          ./build/build-portable.ps1 -Runtime '${{ matrix.rid }}' -Version $env:PACKAGE_VERSION -SkipFonts @native
+
+      - name: Check archive and run native converters
+        run: ./build/smoke-portable.ps1 -Runtime '${{ matrix.rid }}'
+
+      - name: Check glibc requirements
+        if: runner.os == 'Linux'
+        shell: bash
+        run: python3 build/check-elf-glibc.py --max 2.35 "$FILEE_SMOKE_PACKAGE"
 
       - name: Launch native desktop app
         shell: bash
-        run: |
-          set -euo pipefail
-          python3 - <<'PY'
-          import os
-          import pathlib
-          import signal
-          import subprocess
-          import sys
-          import time
-
-          app = os.environ['FILEE_SMOKE_APP']
-          shots = pathlib.Path(os.environ['RUNNER_TEMP']) / 'filee-screens'
-          shots.mkdir(exist_ok=True)
-          command = [app, '--settings', '--install-engines=', '--context-menu=off', '--start-with-system=off']
-          if os.environ['RUNNER_OS'] == 'Linux':
-              command = ['xvfb-run', '-a', 'env', '-u', 'WAYLAND_DISPLAY', 'XDG_SESSION_TYPE=x11', *command]
-          log_path = pathlib.Path(os.environ['RUNNER_TEMP']) / 'filee-startup.log'
-          with log_path.open('w') as log:
-              process = subprocess.Popen(command, stdout=log, stderr=subprocess.STDOUT, start_new_session=True)
-              try:
-                  time.sleep(8)
-                  if process.poll() is not None:
-                      raise RuntimeError('The desktop app exited before the startup check.')
-                  if os.environ['RUNNER_OS'] == 'macOS':
-                      # The real screen with Filee's window, as a person would see it (uploaded with the results).
-                      subprocess.run(['screencapture', '-x', str(shots / 'settings.png')], check=False, timeout=30)
-                      # A second start forwards to the running Filee, which brings its window to the front.
-                      try:
-                          subprocess.run([app], check=False, timeout=30)
-                      except subprocess.TimeoutExpired:
-                          raise RuntimeError('A second start did not hand over to the running Filee.')
-                      time.sleep(4)
-                      subprocess.run(['screencapture', '-x', str(shots / 'second-start.png')], check=False, timeout=30)
-                      if process.poll() is not None:
-                          raise RuntimeError('The desktop app exited after a second start.')
-                  quit_code = 'import os, sys; os.setsid(); os.execv(sys.argv[1], [sys.argv[1], "--quit"])'
-                  # Reap Filee as soon as it ends while --quit waits for it, as launchd or a shell would: a process
-                  # that has ended but was not reaped by its parent (this script) still counts as running.
-                  quitter = subprocess.Popen([sys.executable, '-c', quit_code, app])
-                  quit_deadline = time.monotonic() + 30
-                  while quitter.poll() is None:
-                      process.poll()
-                      if time.monotonic() > quit_deadline:
-                          quitter.kill()
-                          raise RuntimeError('filee --quit did not finish.')
-                      time.sleep(0.05)
-                  if quitter.returncode != 0:
-                      raise RuntimeError(f'filee --quit exited with {quitter.returncode}.')
-                  if process.wait(timeout=15) != 0:
-                      raise RuntimeError('The desktop app did not exit cleanly.')
-              except BaseException:
-                  log.flush()
-                  print(log_path.read_text(errors='replace'), file=sys.stderr)
-                  raise
-              finally:
-                  if process.poll() is None:
-                      os.killpg(process.pid, signal.SIGTERM)
-                      try:
-                          process.wait(timeout=10)
-                      except subprocess.TimeoutExpired:
-                          os.killpg(process.pid, signal.SIGKILL)
-                          process.wait()
-          PY
+        run: python3 build/smoke-launch-app.py
 
       - name: Collect Filee logs
         if: always()
@@ -207,3 +215,67 @@ jobs:
             ${{ runner.temp }}/filee-startup.log
             ${{ runner.temp }}/filee-screens/
           if-no-files-found: ignore
+
+  # The packages built above, unpacked and started on the oldest supported Ubuntu, with only the runtime libraries a
+  # user installs from installer/linux/INSTALL.txt.
+  ubuntu-22-04:
+    name: Package on Ubuntu 22.04 (${{ matrix.rid }})
+    needs: portable
+    # Runs when another platform's package failed; a missing Linux package fails the download.
+    if: ${{ !cancelled() }}
+    strategy:
+      fail-fast: false
+      matrix:
+        include:
+          - runner: ubuntu-22.04
+            rid: linux-x64
+          - runner: ubuntu-22.04-arm
+            rid: linux-arm64
+    runs-on: ${{ matrix.runner }}
+    timeout-minutes: 30
+    defaults:
+      run:
+        shell: pwsh
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Linux desktop dependencies (Ubuntu 22.04)
+        shell: bash
+        run: |
+          sudo apt-get update
+          sudo apt-get install -y ca-certificates libc6 libgcc-s1 libgssapi-krb5-2 libicu70 libssl3 libstdc++6 tzdata zlib1g \
+            libfontconfig1 libfreetype6 libx11-6 libice6 libsm6 libxrandr2 libxkbcommon0 libxi6 libxtst6 libxt6 xvfb fonts-noto-cjk
+
+      - name: Download package
+        uses: actions/download-artifact@v4
+        with:
+          name: portable-${{ matrix.rid }}
+          path: Releases
+
+      - name: Check archive and run native converters
+        run: ./build/smoke-portable.ps1 -Runtime '${{ matrix.rid }}'
+
+      - name: Check glibc requirements
+        shell: bash
+        run: python3 build/check-elf-glibc.py --max 2.35 "$FILEE_SMOKE_PACKAGE"
+
+      - name: Launch native desktop app
+        shell: bash
+        run: python3 build/smoke-launch-app.py
+
+      - name: Collect Filee logs
+        if: always()
+        shell: bash
+        run: |
+          mkdir -p "$RUNNER_TEMP/filee-screens"
+          if [ -d "$HOME/.config/Filee/logs" ]; then cp -R "$HOME/.config/Filee/logs" "$RUNNER_TEMP/filee-screens/"; fi
+
+      - name: Upload diagnostics
+        if: always()
+        uses: actions/upload-artifact@v4
+        with:
+          name: ubuntu-22-04-results-${{ matrix.rid }}
+          path: |
+            ${{ runner.temp }}/filee-startup.log
+            ${{ runner.temp }}/filee-screens/
+          if-no-files-found: ignore

CONTRIBUTING.md

@@ -35,8 +35,13 @@ install/uninstall scripts. `Filee-<version>-<rid>-SHA256SUMS.txt` verifies the a
 `com.filee.app` and a development ad-hoc signature by default; public distribution still needs Developer ID signing
 and notarization. `-SigningIdentity` can supply a signing identity, but does not perform notarization.
 
+Linux release packages must run on Ubuntu 22.04 (glibc 2.35). CI builds rhwp from source on Ubuntu 22.04
+(`build/build-linux-native.py`) and passes it with `-RhwpOverride`; a local package without it needs the glibc of the
+machine that built it. Check a package with
+`python3 build/check-elf-glibc.py --max 2.35 <unpacked package>`. See [docs/ENGINES.md](docs/ENGINES.md#linux-builds-for-older-glibc).
+
 - **Linux:** drag gestures require X11. Install the Avalonia desktop dependencies (fontconfig, FreeType, X11,
-  Xrandr, Xi, Xtst, XkbCommon, ICE and SM) and CJK fonts for document comparisons. File actions are installed for
+  Xrandr, Xi, Xtst, Xt, XkbCommon, ICE and SM) and CJK fonts for document comparisons. File actions are installed for
   Thunar, Nautilus, Nemo and Dolphin. The first enabled keyboard shortcut uses Thunar's native custom action;
   close Thunar before changing or pausing that shortcut and reopen it afterwards. Global selected-file queries
   and hold gestures are not supported. Wayland can use file actions, the drop zone, watch folders and CLI.
@@ -48,7 +53,8 @@ and notarization. `-SigningIdentity` can supply a signing identity, but does not
   The macOS LibreOffice package does not include H2Orestart; native rhwp/Unhwp provide the supported HWP routes.
   Use Settings > Engines to inspect actual availability before comparing formats.
 
-The reusable `portable.yml` workflow builds and tests all four targets. Native platform tests skip on other OSes.
+The reusable `portable.yml` workflow builds and tests all four targets and starts the Linux packages on Ubuntu 22.04.
+Native platform tests skip on other OSes.
 Before calling a port ready, verify the installed package, CLI conversion, Finder/Thunar selection with Unicode
 and spaced paths, drag and keyboard gestures, permissions denied/granted, Retina/scaling, tray, login startup,
 engine downloads, watch folders, and uninstall. CI GUI startup alone does not validate these interactions.

Directory.Packages.props

@@ -22,7 +22,7 @@
     <PackageVersion Include="Microsoft.Extensions.DependencyInjection" Version="10.0.12" />
     <PackageVersion Include="Microsoft.Extensions.Logging" Version="10.0.12" />
     <PackageVersion Include="Microsoft.Extensions.Logging.Abstractions" Version="10.0.12" />
-    <PackageVersion Include="SharpHook" Version="8.0.0" />
+    <PackageVersion Include="SharpHook" Version="8.0.1" />
   </ItemGroup>
 
   <ItemGroup Label="Conversion engines">