Skip to content

feat(mobile): on-device dictation models, voice filter and correction learning on iOS - #59

Merged
pseudoseed merged 1 commit into
mainfrom
feat/ios-local-dictation
Nov 10, 2026
Merged

pseudoseed merged 1 commit into
mainfrom
feat/ios-local-dictation

Conversation

@pseudoseed

Copy link
Copy Markdown
Owner

Problem

Dictation on iOS and iPadOS could only use Apple's recognizer. There was no choice of model, no way to keep other people's speech out, and the same mishearings had to be fixed by hand every time.

Fix

Fork-only feature. It is not meant for upstream.

  • Models. Settings > Microphone offers 3 downloadable Parakeet models (Fast, Balanced, Most accurate). Each is gated by the device's memory and free space. Transcription runs on the device; the network is used only to download a model.
  • Only my voice. Records a voice profile and removes other speakers before transcribing.
  • Corrections. Replacement rules can be written by hand or learned from the edits made to a dictation before it is sent.
  • Upstream merges. The code lives in a local Expo module (apps/mobile/modules/pseudo-dictation), a config plugin that adds the FluidAudio pod, and apps/mobile/src/features/dictation. Upstream files change in 6 small places. Android and builds without the module keep the existing behavior.
  • A cancelled download removes its files. A failed one keeps them so a retry resumes.

Verification

  • vp run check:changed exited 0.
  • Native tests with real models and LibriSpeech voices: 52 executed, 0 failures. Voice filter: speaker's words lost 46 of 1902 (2.4%), stranger's words kept 54 of 1126 (4.8%).
  • TypeScript tests for the feature: 124 passed.
  • iPad simulator (iOS 27): download, select, cancel, delete, sample transcription, voice filter with the sample voice, replacement rules, settings search.
  • Shipped as TestFlight build 49 (2.2.0) and tried on a device by Chris.

Not verified here: Apple's recognizer path (not available on the iOS 27 simulator). Screenshots from the simulator pass are kept locally and are not attached.

Known limits

  • Correction learning reads only messages sent from the thread composer.
  • A failed (not cancelled) download keeps its partial files until a retry completes.

Model: Claude Fable 5. Harness: Claude Code in PseudoCode.

🤖 Generated with Claude Code

… learning on iOS

Dictation on iOS and iPadOS could only use Apple's recognizer: no choice of
model, no way to keep other people's speech out, and the same mishearings
had to be fixed by hand every time.

Settings > Microphone now offers 3 downloadable Parakeet models (Fast,
Balanced, Most accurate), each gated by the device's memory and free
space. "Only my voice" records a voice profile and removes other
speakers before transcribing. Replacement rules can be written by hand
or learned from the edits made to a dictation before it is sent. All of
it runs on the device; the network is used only to download a model.

The feature lives in a fork-only Expo module (modules/pseudo-dictation),
a config plugin that adds the FluidAudio pod, and features/dictation, so
upstream files change in 6 small places. Android and builds without the
module keep the existing behavior.

A cancelled download removes its files. A failed one keeps them so a
retry resumes.

Model: Claude Fable 5. Harness: Claude Code in PseudoCode.

Co-Authored-By: Claude Code <noreply@anthropic.com>
@github-actions github-actions Bot added vouch:trusted PR author is trusted by repo permissions or the VOUCHED list. size:XXL 📱 Native Change Changes the native fingerprint; merging blocks production OTAs until a new store build ships. labels Nov 10, 2026
@github-actions

Copy link
Copy Markdown

Thread transfer impact

⚠️ The latest CI run did not produce a thread transfer result for dc8ce41.

This comment will update automatically after the next completed run.

@pseudoseed

Copy link
Copy Markdown
Owner Author

Merging with "Mobile Native Static Analysis" cancelled. GitHub did not assign a macOS runner in 2 attempts of 15 minutes each (annotation: "The job was not acquired by Runner of type hosted even after multiple attempts"), so the job never ran, and "Check" fails only because of it. The same lint (vp run lint:mobile) exits 0 on this commit locally, and its config covers no file this PR changes. Every job that ran passed.

@pseudoseed
pseudoseed merged commit 4fd9885 into main Nov 10, 2026
38 of 42 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

📱 Native Change Changes the native fingerprint; merging blocks production OTAs until a new store build ships. size:XXL vouch:trusted PR author is trusted by repo permissions or the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant