Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion .claude/skills/verify/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -225,7 +225,8 @@ its profile. It keeps the vault and `.obsidian-e2e-artifacts/`. Delete

- `.claude/skills/verify/net.mjs` runs as `watch`, `log` or `unwatch`. It logs
every `fetch` and `requestUrl` call PodNotes makes, without query strings,
and answers OpenAI and Deepgram with a fixture provider while it watches.
and answers ElevenLabs, Deepgram and OpenAI with a fixture provider while it
watches.

- `.claude/skills/verify/ui.mjs` sends trusted input through
`obsidian dev:cdp`. Its verbs are `click <css> [--text <s>] [--right]`,
Expand Down
50 changes: 30 additions & 20 deletions .claude/skills/verify/features/transcripts.md

Large diffs are not rendered by default.

29 changes: 22 additions & 7 deletions .claude/skills/verify/net.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,12 @@
// Network log for this worktree's Obsidian instance, with a fixture provider in
// place of the paid transcription APIs.
//
// node .claude/skills/verify/net.mjs watch start a fresh log; answer OpenAI and Deepgram locally
// node .claude/skills/verify/net.mjs watch start a fresh log; answer ElevenLabs, Deepgram and OpenAI locally
// node .claude/skills/verify/net.mjs log print each request as JSON, oldest first; a path only
// for the fixture server, OpenAI and Deepgram
// for the fixture server and the three providers
// node .claude/skills/verify/net.mjs unwatch restore fetch and requestUrl
//
// PodNotes sends its requests (feeds, audio, transcripts, OpenAI, Deepgram)
// PodNotes sends its requests (feeds, audio, transcripts, the providers)
// through requestUrl, which Obsidian forwards to its main process on the
// `request-url` IPC channel. The fetch log catches anything that bypasses it.
import { fileURLToPath } from "node:url";
Expand All @@ -33,7 +33,7 @@ const instance = resolveInstanceOptions(
const CALL_TIMEOUT_MS = 15000;
const FIXTURE_TRANSCRIPTION = "This text came from the fixture provider.";
const FIXTURE_REPLY = "A second speaker answers.";
const PATH_HOSTS = [FIXTURE_HOST, "api.openai.com", "api.deepgram.com"];
const PATH_HOSTS = [FIXTURE_HOST, "api.elevenlabs.io", "api.deepgram.com", "api.openai.com"];

/** @param {string} code */
async function evaluate(code) {
Expand Down Expand Up @@ -84,7 +84,20 @@ const WATCH = `(() => {
end: from + index * 0.4 + 0.3,
speaker,
}));
const deepgram = { results: { channels: [{ alternatives: [{ words: [...words(text, 0, 0), ...words(${jsLiteral(FIXTURE_REPLY)}, 1, 12)] }] }] } };
const reply = ${jsLiteral(FIXTURE_REPLY)};
const deepgram = { results: { channels: [{ alternatives: [{ words: [...words(text, 0, 0), ...words(reply, 1, 12)] }] }] } };
// ElevenLabs' words, at the same times, with a spacing item between each two.
const scribeWords = (sentence, speaker, from) => words(sentence, speaker, from).flatMap(({ punctuated_word, start, end }, index) => {
const word = { text: punctuated_word, type: "word", start, end, speaker_id: "speaker_" + speaker };
return index === 0 ? [word] : [{ text: " ", type: "spacing", start, end: start, speaker_id: word.speaker_id }, word];
});
const elevenlabs = { language_code: "eng", text: text + " " + reply, words: [...scribeWords(text, 0, 0), ...scribeWords(reply, 1, 12)] };
// A multipart request that names an address in place of a file is a few hundred bytes.
const namedAddress = (request) => {
if (!(request.body?.byteLength < 4096)) return undefined;
const match = /name="source_url"\\r\\n\\r\\n([^\\r]+)/.exec(new TextDecoder().decode(request.body));
return match ? logged(match[1]) : undefined;
};
const originalFetch = window.fetch;
window.fetch = async function (input, init) {
const url = typeof input === "string" ? input : input instanceof URL ? input.href : input.url;
Expand All @@ -101,9 +114,11 @@ const WATCH = `(() => {
? { text }
: request.url.startsWith("https://api.deepgram.com/")
? deepgram
: null;
: request.url.startsWith("https://api.elevenlabs.io/")
? elevenlabs
: null;
if (answer) {
entries.push({ via: "requestUrl", method, url: logged(request.url), answeredBy: "fixture provider", contentType: request.contentType, uploadBytes: request.body?.byteLength ?? 0 });
entries.push({ via: "requestUrl", method, url: logged(request.url), answeredBy: "fixture provider", contentType: request.contentType, uploadBytes: request.body?.byteLength ?? 0, sourceUrl: namedAddress(request) });
const body = new TextEncoder().encode(JSON.stringify(answer)).buffer;
queueMicrotask(() => ipc.emit(replyChannel, {}, { status: 200, headers: { "content-type": "application/json" }, body }));
return;
Expand Down
2 changes: 1 addition & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ svelte-check's native API path passes when a tsconfig matches no files, which
`check:a11y` in temporary workspaces and requires it to fail on that case and
when the native compiler cannot start.

After `npm run build`, `npm run size` enforces 330,000 raw bytes and 104,000 gzip bytes at level 9 for root `main.js` in the Test workflow's Build and sizes worker. `npm run size:data` reports a synthetic v2 stress library with 50 feeds, 2,000 history entries, and 200 saved episodes from frozen NPR metadata, including duplicated snapshots. Its 1.5 MB document goal is advisory. Pass an additive candidate JSON path to report optional-field overhead without changing existing data. In the isolated Obsidian instance, `npm run count:writes -- start`, `report`, and `stop` record writes to PodNotes' exact `data.json` path. Stop the recorder before stopping the instance, because recorder commands start a stopped instance; plugin reloads invalidate its counts.
After `npm run build`, `npm run size` enforces 330,000 raw bytes and 104,866 gzip bytes at level 9 for root `main.js` in the Test workflow's Build and sizes worker. `npm run size:data` reports a synthetic v2 stress library with 50 feeds, 2,000 history entries, and 200 saved episodes from frozen NPR metadata, including duplicated snapshots. Its 1.5 MB document goal is advisory. Pass an additive candidate JSON path to report optional-field overhead without changing existing data. In the isolated Obsidian instance, `npm run count:writes -- start`, `report`, and `stop` record writes to PodNotes' exact `data.json` path. Stop the recorder before stopping the instance, because recorder commands start a stopped instance; plugin reloads invalidate its counts.

Before opening a PR or cutting a release, run the CI-equivalent checks locally:

Expand Down
6 changes: 4 additions & 2 deletions obsidian-e2e.config.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -114,12 +114,14 @@ export default {
},
openAISecretId: "",
deepgramSecretId: "",
elevenLabsSecretId: "",
transcript: {
path: "transcripts/{{podcast}}/{{title}}.md",
template: "# {{title}}\n\nPodcast: {{podcast}}\nDate: {{date}}\n\n{{transcript}}",
provider: "elevenlabs",
model: "",
diarization: {
enabled: false,
provider: "openai",
enabled: true,
speakerTemplate: "**{{speaker}}:** ",
},
},
Expand Down
12 changes: 11 additions & 1 deletion scripts/size-baseline.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ main.js raw: 307131 bytes; budget 330000 bytes; 22869 bytes remaining
main.js gzip (level 9): 97161 bytes; budget 100000 bytes; 2839 bytes remaining
```

The gzip budget is now 104,000 bytes. On 2026-10-14 it rose from 100,000 for
The gzip budget is now 104,866 bytes. On 2026-10-14 it rose from 100,000 for
the full player's Transcript panel, which the owner approved at about 1.9 KB
and which measured 2,576 bytes. The coordinator set 103,000 to keep the
panel's Find and Follow chip and to leave room for fixes in flight. The
Expand All @@ -34,6 +34,16 @@ main.js raw: 318989 bytes; budget 330000 bytes; 11011 bytes remaining
main.js gzip (level 9): 103397 bytes; budget 104000 bytes; 603 bytes remaining
```

On 2026-11-14 it rose from 104,000 for the transcription provider and model
settings with ElevenLabs Scribe v3. The transcription engine had taken the
branch to 103,795 bytes, and the settings, their migration and the ElevenLabs
provider measured 866 more. The budget rose by those 866 bytes. With them:

```text
main.js raw: 320988 bytes; budget 330000 bytes; 9012 bytes remaining
main.js gzip (level 9): 104661 bytes; budget 104866 bytes; 205 bytes remaining
```

Gzip uses Node's `gzipSync` at level 9, without a filename or timestamp.
Budgets use decimal bytes. The brief's 96,533-byte reference is unexplained.
This report fixes the compressor instead of comparing compressed sizes across
Expand Down
2 changes: 1 addition & 1 deletion scripts/size.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ try {
if (!bundle.length) throw new Error(`${path} is empty. Run npm run build first.`);
const measurements = [
{ name: "raw", bytes: bundle.length, budget: 330_000 },
{ name: "gzip (level 9)", bytes: gzipSync(bundle, { level: 9 }).length, budget: 104_000 },
{ name: "gzip (level 9)", bytes: gzipSync(bundle, { level: 9 }).length, budget: 104_866 },
];
const rows = measurements.map(({ name, bytes, budget }) => {
const overage = Math.max(0, bytes - budget);
Expand Down
2 changes: 1 addition & 1 deletion scripts/size.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ describe("bundle budget CLI", () => {
const result = check("gzip.js", randomBytes(110_000));
expect(result.status).toBe(1);
expect(result.stdout).toMatch(
/gzip \(level 9\): \d+ bytes; budget 104000 bytes; OVER by \d+ bytes/,
/gzip \(level 9\): \d+ bytes; budget 104866 bytes; OVER by \d+ bytes/,
);
expect(result.stdout).toContain("budget 330000 bytes; 220000 bytes remaining");
});
Expand Down
6 changes: 4 additions & 2 deletions src/constants.ts
Original file line number Diff line number Diff line change
Expand Up @@ -158,12 +158,14 @@ export const DEFAULT_SETTINGS: IPodNotesSettings = {
},
openAISecretId: "",
deepgramSecretId: "",
elevenLabsSecretId: "",
transcript: {
path: "transcripts/{{podcast}}/{{title}}.md",
template: "# {{title}}\n\nPodcast: {{podcast}}\nDate: {{date}}\n\n{{transcript}}",
provider: "elevenlabs",
model: "",
diarization: {
enabled: false,
provider: "openai",
enabled: true,
speakerTemplate: "**{{speaker}}:** ",
},
},
Expand Down
56 changes: 55 additions & 1 deletion src/main.persistence.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ import { attachDataFileWriter, dataFileText, manualClock } from "../tests/mocks/
import { FakeDocument } from "../tests/mocks/fakeMedia";
import released2200 from "../tests/fixtures/persistence/released-2.20.0.json";
import released2211 from "../tests/fixtures/persistence/released-2.21.1.json";
import released2240 from "../tests/fixtures/persistence/released-2.24.0.json";

function memorySecretStorage(initial: Record<string, string> = {}) {
const values = new Map(Object.entries(initial));
Expand Down Expand Up @@ -401,11 +402,15 @@ it("takes in the file a save read back before one that lands after it", async ()
});
});

/** A released library with no API key and no speaker labels, as this version saves it. */
function savedByThisVersion(released: Record<string, unknown>): Record<string, unknown> {
const data: Record<string, unknown> = {
libraryId: "",
...structuredClone(released),
headsetPreviousTrack: "captureTimestamp",
headsetNextTrack: "skipForward",
elevenLabsSecretId: "",
transcript: { ...(released.transcript as object), provider: "elevenlabs", model: "" },
};
delete data.feedCache;
return data;
Expand Down Expand Up @@ -730,6 +735,55 @@ describe("PodNotes persistence integration", () => {
expect(persisted).not.toHaveProperty("diarizationApiKey");
});

it("names exactly the legacy keys it moved into SecretStorage", async () => {
const notice = vi.spyOn(obsidian, "Notice");
const migrate = async (legacy: Record<string, string>) => {
notice.mockClear();
const { storage } = memorySecretStorage();
const plugin = makePlugin();
Object.assign(plugin, {
app: { ...plugin.app, secretStorage: storage },
credentials: new CredentialRepository(storage),
loadData: vi.fn().mockResolvedValue({ schemaVersion: 1, ...legacy }),
});
await plugin.loadSettings();
return notice.mock.calls;
};

expect(await migrate({ openAIApiKey: "sk", diarizationApiKey: "dg" })).toEqual([
["Moved your OpenAI and Deepgram API keys into Obsidian SecretStorage."],
]);
expect(await migrate({ diarizationApiKey: "dg" })).toEqual([
["Moved your Deepgram API key into Obsidian SecretStorage."],
]);
});

it("says its API keys could not move when only a Deepgram key fails to migrate", async () => {
const notice = vi.spyOn(obsidian, "Notice");
const storage = {
getSecret: vi.fn(() => null),
setSecret: vi.fn(() => {
throw new Error("keychain unavailable");
}),
listSecrets: vi.fn(() => []),
} as unknown as SecretStorage;
const plugin = makePlugin();
Object.assign(plugin, {
app: { ...plugin.app, secretStorage: storage },
credentials: new CredentialRepository(storage),
loadData: vi.fn().mockResolvedValue({ schemaVersion: 1, diarizationApiKey: "dg" }),
});
vi.spyOn(console, "error").mockImplementation(() => undefined);

await expect(plugin.loadSettings()).rejects.toThrow("keychain unavailable");
expect(notice.mock.calls).toEqual([
[
"PodNotes could not securely migrate its API keys. Your existing data was kept unchanged. Restart PodNotes to retry.",
0,
],
]);
});

it("leaves legacy data untouched when SecretStorage migration fails", async () => {
const saveData = vi.fn().mockResolvedValue(undefined);
const raw = { schemaVersion: 1, openAIApiKey: "sk", defaultVolume: 0.3 };
Expand Down Expand Up @@ -915,6 +969,7 @@ describe("PodNotes persistence integration", () => {
it.each([
["2.20.0", released2200],
["2.21.1", released2211],
["2.24.0", released2240],
])("loads a %s library with a private feed unchanged", async (_version, released) => {
const id = "podnotes-feed-url-00000000-0000-4000-8000-000000000211";
const secretUrl = "http://podnotes.test:41627/private.xml?auth=released";
Expand All @@ -935,7 +990,6 @@ describe("PodNotes persistence integration", () => {
expect(saveData.mock.calls[0][0]).toEqual({
...savedByThisVersion(released),
defaultVolume: 0.4,
libraryId: "",
});
expect(values.get(id)).toBe(secretUrl);
expect(resolveFeedUrl(plugin.settings.savedFeeds["PRIV2 Both"], plugin.feedUrls)).toBe(
Expand Down
13 changes: 5 additions & 8 deletions src/main.ts
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,7 @@ import {
WriteRefusedError,
} from "./persistence/writer";
import { CredentialRepository } from "./services/CredentialRepository";
import { CREDENTIAL_KINDS, CREDENTIALS } from "./types/Credentials";
import { FeedUrlRepository } from "./services/FeedUrlRepository";
import { privateFeedAddresses, protectPrivateFeeds, resolveFeedUrl } from "./services/privateFeeds";
import { redactPrivateFeedAddresses } from "./utility/privateFeedUrl";
Expand Down Expand Up @@ -370,6 +371,9 @@ export default class PodNotes extends Plugin implements IPodNotes {
decoded.retiredPlaintextPresent ||
movedFeeds
) {
const migratedProviders = CREDENTIAL_KINDS.filter(
(kind) => decoded.legacySecrets[CREDENTIALS[kind].exportKey],
).map((kind) => CREDENTIALS[kind].label);
// SecretStorage writes are synchronous. Store every legacy value and
// verify it first, then write the v2 snapshot directly because the normal
// save queue intentionally stays dormant until onload finishes. A v2 file
Expand All @@ -388,15 +392,8 @@ export default class PodNotes extends Plugin implements IPodNotes {
);
if (written.kind !== "verified") throw this.writeFailure();
} catch (error) {
throw new PodNotesSchemaMigrationError(
error,
Boolean(decoded.legacySecrets.openAI || decoded.legacySecrets.deepgram),
);
throw new PodNotesSchemaMigrationError(error, migratedProviders.length > 0);
}
const migratedProviders = [
decoded.legacySecrets.openAI ? "OpenAI" : null,
decoded.legacySecrets.deepgram ? "Deepgram" : null,
].filter((provider): provider is string => provider !== null);
if (migratedProviders.length > 0) {
new Notice(
`Moved your ${migratedProviders.join(" and ")} API ${migratedProviders.length === 1 ? "key" : "keys"} into Obsidian SecretStorage.`,
Expand Down
Loading
Loading