Skip to content

feat(transcription): choose a provider and model, with ElevenLabs as the default - #539

Open
chhoumann wants to merge 13 commits into
feat/transcription-enginefrom
feat/transcription-providers
Open

chhoumann wants to merge 13 commits into
feat/transcription-enginefrom
feat/transcription-providers

Conversation

@chhoumann

@chhoumann chhoumann commented Nov 14, 2026 •

Copy link
Copy Markdown
Owner

Summary

PodNotes is moving transcription to current models from several providers, so that a provider is a credential row, one provider file and a registry line, and no user stays on a model OpenAI retires. This PR is step 3 of that plan. A user now chooses the provider and the model in settings, and ElevenLabs Scribe v3 is the default. It stacks on #538.

The diff includes #537, which is merged into this branch, until #537 lands.

  • Settings. The Transcripts group reads Transcription provider, the chosen provider's API key, Model, Speaker labels, Speaker label format, then the path and template. Keys linked for other providers stay stored. Speaker labels is disabled for a model that cannot label speakers, and the row says so. Diarization provider is gone.
  • Model. Recommended (<name>) follows the provider's first model, so a later release can move everyone who never picked one.
  • ElevenLabs. scribe_v3 gets the address of an episode that a saved public feed lists, so PodNotes downloads and uploads nothing. Any other episode is uploaded in one request. The note has a timestamp on each paragraph and numbered speakers.
  • A provider or model this version does not have is kept as stored and shown under that name. An unknown provider refuses to transcribe with a notice. An unknown model runs the recommended model and the notice says so.
  • Notices that name the providers able to label speakers now list ElevenLabs first.

Not in this PR: OpenRouter, merged speaker labels across parts, and the docs, which describe the released version until the release.

New install: ElevenLabs Deepgram chosen
A 2.24.0 library that labelled speakers with OpenAI Speaker labels off
The notice that stays An ElevenLabs note

Verification

Run on Linux with Node 22.23.4, on the final tree:

  • npm run lint, npm run format:check, npm run typecheck, npm run build and npm run docs:build pass.
  • npm run test: 124 files, 2,857 tests pass.
  • npm run test:tooling: 8 files, 48 tests pass.
  • npm run test:e2e on Obsidian 1.14.4: 33 files, 255 tests pass.
  • PODNOTES_OBSIDIAN_APP=/opt/Obsidian/obsidian npm run test:e2e -- settings-tab journey-resilience on Obsidian 1.13.7: 2 files, 21 tests pass.
  • npm run plant: 52 planted defects, each caught by the unit test named below, and 2 more caught by npm run test:e2e.

Obsidian 1.14.4 on Linux under Xvfb, in this worktree's isolated vault, with the verify skill's fixture provider answering the three providers. No real key was used. The fixture feed is http://podnotes.test:28062/feed.xml and the private one is at priv3.supercast.com.

  • Settings rows. The screenshots above show each provider. The provider was chosen with the mouse and arrow keys. data.json then held the provider with model: "".
  • ElevenLabs, public episode. Transcribe current episode from audio sent one request: POST https://api.elevenlabs.io/v1/speech-to-text, 604 bytes, naming http://podnotes.test:28062/episode-2.wav. The fixture server saw no request during the command. The note has **1:** at 0 s and **2:** at 12 s.
  • ElevenLabs, private feed. The feed was saved with a urlSecretId. The command sent GET /p/episode-1.wav, then one POST of 960,649 bytes with no address in it.
  • Speaker labels off. The same public episode gave a note with timestamps and no speaker names.
  • A 2.24.0 library. The released 2.24.0 main.js wrote two libraries through its own settings tab. Its local build has the SHA-256 of the release asset, b7ba1dfe...0530484. This build loaded speaker labels with Deepgram, no Deepgram key and a linked OpenAI key as Deepgram, asked for a Deepgram key, and sent nothing. It loaded speaker labels with OpenAI as OpenAI, showed the disabled Speaker labels row, saved the transcript without labels, and the notice was still up 20 s later. data.json was byte-identical after each load.
  • 2.24.0 on a library saved by this build. 2.24.0 loaded it without a write. After a change in its settings tab, the sorted JSON differed in that setting and in transcript.diarization.provider: "openai", which 2.24.0 adds for itself. transcript.provider, transcript.model and elevenLabsSecretId were kept. This build then read its fields back unchanged.

dev:errors reported no error after each step. The lead verifies with real keys separately.

Planted defects, each caught by the named test through npm run plant
Planted defect In Caught by
migration: Deepgram without its key falls to the linked OpenAI key settingsMigrations.ts speaker labels with Deepgram and no Deepgram key, beside an OpenAI key gets its provider, on the recommended model, without a repair
migration: Deepgram counts only with its key linked settingsMigrations.ts, persistence/settingsCodec.ts speaker labels with Deepgram and no key at all gets its provider, on the recommended model, without a repair
migration: Deepgram picked without speaker labels counts as Deepgram settingsMigrations.ts Deepgram picked while speaker labels are off, with only its key gets its provider, on the recommended model, without a repair
import: Deepgram picked without speaker labels counts as Deepgram settingsMigrations.ts a 2.24.0 export with Deepgram picked while speaker labels are off keeps this library's provider and model
migration: the new default turns speaker labels on for a released library settingsMigrations.ts a library saved before speaker labels existed keeps them off
migration: an OpenAI key without speaker labels moves to the default settingsMigrations.ts no speaker labels and an OpenAI key linked gets its provider, on the recommended model, without a repair
migration: a key stored as text before 2.19.0 is not seen persistence/settingsCodec.ts a library from before 2.19.0 that holds its OpenAI key as text stays on OpenAI
migration: filling the provider adds a decode warning persistence/settingsCodec.ts speaker labels with OpenAI gets its provider, on the recommended model, without a repair
codec: writes diarization.provider persistence/settingsCodec.ts a provider chosen since stays, whatever the 2.24.0 fields say
codec: clamps diarization.provider as 2.24.0 did persistence/settingsCodec.ts a value 2.24.0 would have clamped in its provider field is left as found
codec: clamps an unknown provider to the default settingsMigrations.ts a provider and a model this version does not have are kept as stored, without a repair
codec: clamps an unknown model to the recommended one settingsMigrations.ts a provider and a model this version does not have are kept as stored, without a repair
defaults: a new install is on OpenAI constants.ts a new install starts on ElevenLabs with speaker labels on
defaults: a library with no key and no speaker labels stays on OpenAI constants.ts no speaker labels and no key gets its provider, on the recommended model, without a repair
defaults: a new install has speaker labels off constants.ts a new install starts on ElevenLabs with speaker labels on
export: includes diarization.provider settingsTransfer.ts exports the provider and model, without the provider field of versions up to 2.24.0
import: a 2.24.0 Deepgram export keeps the current provider settingsTransfer.ts imports a 2.24.0 export that labels speakers with Deepgram as Deepgram, on its recommended model
import: a 2.24.0 Deepgram export keeps the current model settingsTransfer.ts imports a 2.24.0 export that labels speakers with Deepgram as Deepgram, on its recommended model
import: replaces the stored speaker settings whole settingsTransfer.ts keeps the speaker settings an import leaves out
import: drops the stored diarization.provider settingsTransfer.ts importing this version's export leaves the provider field of versions up to 2.24.0 as the library stored it
import: takes a 2.24.0 file's diarization.provider settingsTransfer.ts importing a 2.24.0 export that names OpenAI there leaves the provider field of versions up to 2.24.0 as the library stored it
choice: an unknown provider runs the default provider services/transcription/registry.ts Transcribe current episode refuses a provider this version does not have, and sends nothing
service: an unknown provider is refused only once the job runs services/TranscriptionService.ts Transcribe current episode from audio refuses a provider this version does not have, and sends nothing
choice: an unknown model is sent to the provider services/transcription/registry.ts a model this version does not have runs the recommended model, and the notice says so
engine: an unknown model gives no warning services/transcription/transcribe.ts a model this version does not have runs the recommended model, and the notice says so
notice: ElevenLabs is not named among the providers that label speakers services/transcription/registry.ts with speaker labels on is saved without them, and the notice names the providers that label speakers and stays
deepgram: asks for speakers with speaker labels off services/transcription/providers/deepgram.ts with speaker labels off asks Deepgram for none, and the note names no speaker
engine: a provider's speakers reach the note with speaker labels off services/transcription/cues.ts names no speaker when none was asked for, whatever the provider answered
elevenlabs: always uploads services/transcription/providers/elevenlabs.ts a public episode goes by its address in one request without a file, and its audio is never loaded
engine: a private feed's address is passed through services/transcription/transcribe.ts a private feed's episode is uploaded to ElevenLabs, and no request holds its address or its feed's
elevenlabs: asks for speakers with speaker labels off services/transcription/providers/elevenlabs.ts with speaker labels off asks ElevenLabs for none, and the note names no speaker
elevenlabs: every 401 is a used-up quota services/transcription/providers/elevenlabs.ts a rejected key for a request by address is said in PodNotes' words and is not followed by an upload
elevenlabs: a used-up quota reads as a rejected key services/transcription/providers/elevenlabs.ts a used-up quota for a request by address is said in PodNotes' words and is not followed by an upload
elevenlabs: an unsupported model reads as rejected audio services/transcription/providers/elevenlabs.ts a model the key may not use for a request by address is said in PodNotes' words and is not followed by an upload
elevenlabs: a failed download is not followed by an upload services/transcription/providers/elevenlabs.ts an address ElevenLabs could not download is followed by one upload
elevenlabs: every bad request is followed by an upload services/transcription/providers/elevenlabs.ts a request ElevenLabs could not read for a request by address is said in PodNotes' words and is not followed by an upload
elevenlabs: drops the speaker of each word services/transcription/providers/elevenlabs.ts has a timestamp on each paragraph, numbers the speakers in the order they first speak, and leaves out sounds
elevenlabs: keeps sounds as words services/transcription/providers/elevenlabs.ts has a timestamp on each paragraph, numbers the speakers in the order they first speak, and leaves out sounds
elevenlabs: sends the key as a bearer token services/transcription/providers/elevenlabs.ts a public episode goes by its address in one request without a file, and its audio is never loaded
elevenlabs: asks for sounds to be tagged services/transcription/providers/elevenlabs.ts a public episode goes by its address in one request without a file, and its audio is never loaded
settings: every provider's key row shows ui/settings/PodNotesSettingsTab.ts offers each provider and shows the API key row of the one chosen, keeping the other keys
settings: the key row does not follow the provider ui/settings/PodNotesSettingsTab.ts offers each provider and shows the API key row of the one chosen, keeping the other keys
settings: the model stays when the provider changes ui/settings/PodNotesSettingsTab.ts another provider brings its own models, on the recommended one
settings: the model list is not refilled ui/settings/PodNotesSettingsTab.ts another provider brings its own models, on the recommended one
settings: no Recommended option ui/settings/PodNotesSettingsTab.ts Model offers the recommended model and each of the provider's, and saves the one chosen
settings: Speaker labels stays enabled for a model without them ui/settings/PodNotesSettingsTab.ts Speaker labels cannot be changed for a model without them, and the row says why
settings: Speaker label format always shows ui/settings/PodNotesSettingsTab.ts Speaker labels saves the choice, and Speaker label format shows only while they are on
settings: an unknown provider is not offered ui/settings/PodNotesSettingsTab.ts a provider this version does not have stays selected under its stored name, with no key or model row
settings: an unknown model is not offered ui/settings/PodNotesSettingsTab.ts a model this version does not have stays selected under its stored name
settings: Speaker labels unlocks during an import ui/settings/PodNotesSettingsTab.ts Speaker labels stays locked during import and unlocks once it ends
settings: a new install shows OpenAI constants.ts a new install is on ElevenLabs Scribe v3 with speaker labels on
migration: recomputes the provider on every load settingsMigrations.ts a provider chosen since stays, whatever the 2.24.0 fields say
e2e: ElevenLabs always uploads services/transcription/providers/elevenlabs.ts J-TR-08 transcribing a public episode with ElevenLabs sends its address in place of its audio, and the note has timestamps and speaker labels
e2e: every provider's key row shows ui/settings/PodNotesSettingsTab.ts names each field by its row's name where Tab takes focus

Renaming the ElevenLabs row's settingsKey or exportKey fails npm run typecheck, in constants.ts and in the tests that pass elevenLabs, so no plant covers those names.

Real ElevenLabs key (lead, after review)

Obsidian 1.14.4, Linux, build of 1eef3e6f, palette command "Transcribe current episode from audio", speaker labels on, a pass-through request log (no fixture):

  • NPR News Now, "7pm ET, November 13" (280 s), added through Add podcast from https://feeds.npr.org/500005/podcast.xml: one 929-byte POST to api.elevenlabs.io/v1/speech-to-text carrying source_url, no enclosure download. Saved in 5 s with timestamp links and speakers 1 to 4.
  • A 60 s local MP3 clip: one 688,921-byte POST with the audio, saved with timestamps.
  • dev:errors: none. Free-tier credits limited this to short audio; the lab measured a 54 min episode in about 30 s.

Review fixes (lead)

  • Export: diarization.provider is now "deepgram" exactly when the library uses Deepgram, else "openai", because 2.24.0 imports the object whole and fills a missing provider with OpenAI. Importing here copies only enabled and speakerTemplate, so the stored field stays as the library wrote it. Plants: omit the provider (caught by exports the provider and model, and gives versions up to 2.24.0 Deepgram only for a Deepgram library); spread the file's object again (caught by importing this version's export leaves the provider field of versions up to 2.24.0 as the library stored it).
  • Speaker labels on a model without them can be turned off, then stay disabled. Plant: disable whenever the model cannot label speakers (caught by Speaker labels can only be turned off for a model without them, and the row says why).
  • Checks after the fixes: lint, format, typecheck, build, size (104,706 of 104,866), 2,857 unit tests.

Release impact

Storage. schemaVersion stays 2. The first save adds transcript.provider, transcript.model and elevenLabsSecretId. Nothing else in a 2.24.0 data.json changes, and the load itself writes nothing.

Migration, when transcript.provider is absent. It reads only the stored document, adds no decode warning, and gives the same answer on every load until a save stores it.

2.24.0 library Provider What the user sees
Speaker labels on with Deepgram, key linked or not Deepgram Same provider. A missing key still asks for the Deepgram key.
Speaker labels on with OpenAI OpenAI Transcripts without speaker labels, and a notice that stays and names ElevenLabs and Deepgram.
Speaker labels off, OpenAI key linked OpenAI gpt-transcribe, as on #538.
Anything else ElevenLabs Asked for an ElevenLabs key on the first transcription.
New install ElevenLabs Scribe v3 with speaker labels on.

Every row gets model: "". A library that never stored the speaker labels choice keeps them off. A library from before 2.19.0 that holds its OpenAI key as text counts as holding an OpenAI key.

Sync with 2.24.0. A device on 2.24.0 keeps loading the library and keeps the three new fields on its own saves. It transcribes as before, from transcript.diarization, and writes transcript.diarization.provider there. This version reads that field only for the migration, never writes it, and keeps it as found. A provider chosen here does not reach a 2.24.0 device, and a change to Diarization provider there does not reach this version once transcript.provider is stored.

Settings export. The envelope stays at version 2. secrets gains elevenLabs, and settings.transcript carries provider and model without diarization.provider. Importing a 2.24.0 export that labels speakers with Deepgram selects Deepgram. Any other 2.24.0 export keeps the provider. An import leaves the stored diarization.provider alone.

Leaving diarization.provider out does not protect a 2.24.0 importer. Run against the 2.24.0 source, its import replaced the whole diarization object and filled the provider with "openai", with the field absent as with "openai" in the file. Only a file with no diarization object left its Deepgram choice alone, and that file would not carry the speaker labels setting either. This needs a decision.

Speaker labels with OpenAI. The row is disabled for OpenAI, so a library that arrives with speaker labels on cannot turn them off without first choosing another provider. Until then the notice shows after every transcription. This needs a decision too.

Bundle. main.js grows from 318,278 to 320,988 raw bytes and from 103,795 to 104,661 gzip bytes, measured against the merge of #538 and #537. That is 866 gzip bytes over the engine, and 661 over the 104,000 budget. The budget rises by the measured 866 to 104,866, which keeps the 205 bytes of headroom. scripts/size-baseline.md records it.

ElevenLabs errors. The provider reads detail.code and detail.message as the lead measured them. The tests build those bodies from that description, not from a captured response.

Note

Add transcription provider and model selection with ElevenLabs as default

  • Adds a transcription provider registry in registry.ts covering ElevenLabs, Deepgram, and OpenAI. New libraries default to ElevenLabs with speaker labels enabled, an empty model ID, and an empty ElevenLabs secret reference in constants.ts
  • Implements an ElevenLabs provider in elevenlabs.ts that sends a multipart request with the model, speaker-label setting, and word-level timestamps, using a source URL or uploaded audio. Adds quota, unsupported-model, and URL-fetch failure classification
  • Reworks the settings tab in PodNotesSettingsTab.ts with provider, provider-specific API-key, model, and Speaker-labels rows. Changing provider clears the model; unrecognized stored providers and models display as stored without triggering a settings write
  • Speaker labeling is now a transcript-level boolean. Deepgram sets diarize from this flag, and timed cues omit provider-supplied speaker names when labels are disabled in cues.ts
  • Migrates pre-2.25 settings in settingsMigrations.ts: legacy diarization state and OpenAI-key presence infer the prior provider; stored provider and model IDs are preserved. Settings export/import in settingsTransfer.ts handles all credential kinds and writes a legacy diarization provider for consumers up to 2.24.0
  • Behavioral Change: existing libraries with enabled Deepgram diarization migrate to the Deepgram provider; others with an OpenAI key migrate to OpenAI, and the rest to ElevenLabs. Settings-schema consumers reading diarization.provider after 2.24.0 must rely on the compatibility rewrite in the portable settings helper

Macroscope summarized 1eef3e6.


Note

Medium Risk
Touches API keys, SecretStorage migration, settings import/export across versions, and paid third-party transcription flows—well-tested but user-facing and billing-sensitive.

Overview
Adds ElevenLabs as the default transcription backend and replaces the old speaker-diarization provider split with explicit Transcription provider, Model, and Speaker labels settings (per-provider API key rows, recommended vs named models, unknown providers/models kept as stored).

The transcription stack gains an ElevenLabs Scribe v3 implementation (URL-based transcription for public episodes, upload fallback for private/local audio), a unified credential registry including elevenLabsSecretId, and load/save/import migration that maps pre-2.24.0 diarization choices into transcript.provider/model. TranscriptionService now blocks unsupported providers and missing keys up front; Deepgram/ElevenLabs honor the speaker-labels toggle.

E2E/verify harness stubs ElevenLabs alongside OpenAI/Deepgram; gzip bundle budget rises to 104,866 bytes for the new UI and provider code.

Reviewed by Cursor Bugbot for commit 1eef3e6. Bugbot is set up for automated code reviews on this repo. Configure here.

Each test passes on the hand-written code. Together they catch wrong
derivations that no existing test sees: the -1 SecretStorage ID suffix
that released versions accept, the provider an export error names, the
kind a settings key row adopts a secret for, and the key migration
notices.
CREDENTIALS in src/types/Credentials.ts holds each kind's settings
field, export key and label. The credential repository, settings
decoder, settings transfer, settings tab and key migration notice loop
over it. IPodNotesSettings takes its key fields from it, so a kind with
no default in DEFAULT_SETTINGS fails typecheck. The plaintext key map
from before 2.19.0 stays fixed.
A provider's name was written twice: once in its credential row and once in
its provider file. The row now owns it, so a provider file holds only its
models and its request. Each model gains the name the settings will show.
The provider was derived from the speaker settings: Deepgram with speaker
labels on, OpenAI otherwise. It is now stored as transcript.provider, with
transcript.model beside it, where "" follows the provider's recommended model.
schemaVersion stays 2 and both fields are additions, so a device on 2.24.0
keeps loading the library and keeps the fields on its own saves.

A library that has no transcript.provider gets the provider it ran with:
Deepgram for speaker labels with Deepgram, with or without its key, and OpenAI
otherwise. The migration reads only the stored document, adds no decode
warning, and gives the same answer on every load until a save stores it.

transcript.diarization.provider belongs to versions up to 2.24.0. This version
reads it for that migration alone, never writes it, keeps it in data.json as
found, and leaves it out of settings exports. Importing a 2.24.0 export that
labels speakers with Deepgram selects Deepgram.

A provider or model ID this version does not have is kept as stored and shown
under that name. An unknown provider refuses to transcribe with a notice. An
unknown model runs the recommended model and the notice says so.

The Transcripts settings now read: Transcription provider, the chosen
provider's API key, Model, Speaker labels, Speaker label format, then the path
and template. Speaker labels is disabled for a model that cannot label
speakers. The Diarization provider row is gone.
ElevenLabs is a credential row, one provider file and a registry line. Its key
reference is the new root field elevenLabsSecretId, stored in SecretStorage as
podnotes-elevenlabs-api-key, and a settings export carries the key as
secrets.elevenLabs. The export envelope stays at version 2.

A public episode goes to ElevenLabs by its address, so nothing is downloaded.
Any other episode is uploaded in one request. Word times and speakers reach
the note, and turning speaker labels off asks for none.

ElevenLabs answers 401 for a used-up quota as for a rejected key, and 400 for
a model the key may not use as for an address it could not download, so the
provider reads detail.code. Only the last of these is followed by an upload.
A request waits 10 minutes for its answer.

A new install starts on ElevenLabs with speaker labels on. A library saved up
to 2.24.0 that holds no OpenAI key and has speaker labels off moves to
ElevenLabs too, because it never set transcription up. A library that never
stored the speaker labels choice keeps them off.

The notice for a model without speaker labels now names ElevenLabs first.
A journey now seeds its transcription provider with its key, because a new
vault is on ElevenLabs. The OpenAI journeys choose OpenAI. A new journey runs
the default: a public episode goes to ElevenLabs by its address, no audio is
fetched, and the note has timestamps and speaker labels. The ElevenLabs stub
answers with the words ElevenLabs returns and records what each request named.

The settings tab test walks the new Transcripts rows, and the key recovery
test links the ElevenLabs key, the one a new vault shows.
The fixture provider answers api.elevenlabs.io with ElevenLabs' words, two
speakers with the second 12 s in, and logs the address a request named in
place of a file. The transcript guide follows the new settings rows and adds
the ElevenLabs runs and a 2.24.0 library check.
…choice

The provider row decided for itself whether the stored provider is one this
version has, and the Speaker label format row read the stored speaker setting.
Both now ask transcriptionChoice, as the key rows, the model row and the
Speaker labels row do. The engine no longer exports a type nothing imports.
The Transcripts tests looked each control up by a CSS selector inside its row
and asserted on a hidden row's inline style. They now query by role and by the
row's name, as a user or a screen reader finds a control, so a hidden row is
one the query does not find.
…ings measure

The transcription engine left main.js at 103,795 gzip bytes. The provider and
model settings, their migration and the ElevenLabs provider add 866, which
takes it to 104,661 and over the 104,000 budget. The budget rises by those 866
bytes to 104,866, so the 205 bytes of headroom stay as they were.
…m library

2.24.0 imports the diarization object whole and fills a missing provider
with OpenAI, so omitting the field moved Deepgram libraries to OpenAI.
Imports here keep the provider this library stored.
A library migrated with OpenAI speaker labels on could not turn them off,
so every transcription repeated the notice.
@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Nov 14, 2026 •

Copy link
Copy Markdown

Deploying podnotes with  Cloudflare Pages  Cloudflare Pages

Latest commit: 1eef3e6
Status: ✅  Deploy successful!
Preview URL: https://443218dd.podnotes.pages.dev
Branch Preview URL: https://feat-transcription-providers.podnotes.pages.dev

View logs

@chhoumann
chhoumann marked this pull request as ready for review November 14, 2026 14:49

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant