Skip to content

feat(audio): integrate MicYou DSP, capture-first startup and upstream sync - #1242

Open
aeoform wants to merge 8 commits into
Open-Less:betafrom
aeoform:fix/issue-1026-micyou-audio
Open

aeoform wants to merge 8 commits into
Open-Less:betafrom
aeoform:fix/issue-1026-micyou-audio

Conversation

@aeoform

@aeoform aeoform commented Oct 20, 2026

Copy link
Copy Markdown
Collaborator

The previous audio enhancement used an RMS gate and fixed gain, leaving noise mixed with speech largely untreated. This change imports MicYou's real DSP pipeline and exposes its controls across the shared core, React settings and Linux egui.

Shortcut recording now starts capture before provider preparation completes. Early PCM uses the existing bounded session buffer and is replayed to ASR in order, with cleanup on preparation failure or cancellation. This does not add an always-on microphone or enlarge the buffer.

Changes:

  • Pin MicYou DSP/inference sources and models with provenance, licenses and checksums; add RNNoise/PureVox, guarded AGC, gain, VAD, EQ, comb filtering and configurable stage order.
  • Preserve streaming resampling state and flush partial frames/model tails to both ASR and recordings. Add startup timing logs for native measurements.
  • Add a checked upstream importer and scheduled workflow that tests relevant updates and opens draft sync PRs. Conflicts and API changes still require review; updates are not automatically merged.
  • Package third-party notices and document behavior, maintenance and latency measurement.

Validation:

  • Shared core: 1,326 passed, 3 ignored; production recorder: 23 passed; DSP model tests: 16 passed.
  • Frontend: 145 test files passed, including importer verification; importer regression: 6 passed.
  • Linux egui and Harmony host checks passed; DSP checks passed for Windows, Android and OpenHarmony.
  • Offline WAV processing preserved sample count; callback partition, tail, cancellation and AGC noise regressions are covered.
  • git diff --check passed.

Native Windows/macOS/mobile microphone listening and startup P50/P95 measurements remain to be performed. Full Windows Tauri cross-compilation was blocked by the local MSVC C toolchain; native CI is still required. AEC7 reference processing is imported and tested, but application AEC is not enabled because synchronized speaker-reference capture is not connected.

Refs #1026, #1150.

@aeoform
aeoform requested a review from H-Chris233 October 21, 2026 16:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant