Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
aca94c4
test(transcription): send OpenAI diarization no part over 1400 s
chhoumann Nov 13, 2026
217b040
fix(transcription): cap OpenAI diarization chunks at 1400 s
chhoumann Nov 13, 2026
61c8c52
test(transcription): send a rejected OpenAI request once
chhoumann Nov 13, 2026
44947b5
fix(transcription): send a rejected OpenAI request once
chhoumann Nov 13, 2026
a9a3b07
fix(transcription): keep OpenAI diarization requests inside the timeout
chhoumann Nov 13, 2026
50765eb
Merge branch 'master' into fix-issue-532
chhoumann Nov 14, 2026
f04c340
fix(transcription): count MP3 chunks in frames so no one-frame tail i…
chhoumann Nov 14, 2026
c93cad3
Merge branch 'master' into fix-issue-532
chhoumann Nov 14, 2026
910ef79
fix(transcription): split MP3 chunks equally even when the size limit…
chhoumann Nov 14, 2026
29b3a8b
Merge remote-tracking branch 'origin/fix-issue-532' into fix-issue-532
chhoumann Nov 14, 2026
f088015
fix(transcription): fill MP3 chunks up to the size limit before addin…
chhoumann Nov 14, 2026
e0c8679
fix(transcription): cut MP3 chunks as few as fit, moving cuts either way
chhoumann Nov 14, 2026
3568940
build(size): raise the gzip budget by the 136 bytes the #532 chunker …
chhoumann Nov 14, 2026
4227538
fix(transcription): sum each MP3 frame's duration when cutting chunks
chhoumann Nov 14, 2026
db2835d
fix(transcription): skip ID3v2 tags before and after MP3 frames
chhoumann Nov 14, 2026
c085a11
fix(transcription): skip an ID3v2 tag that follows another
chhoumann Nov 14, 2026
8a8da6b
fix(transcription): step past an ID3v2 tag's footer before the next tag
chhoumann Nov 14, 2026
7572d7d
build(size): set the gzip budget to the 104,145 bytes the #532 chunke…
chhoumann Nov 14, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ svelte-check's native API path passes when a tsconfig matches no files, which
`check:a11y` in temporary workspaces and requires it to fail on that case and
when the native compiler cannot start.

After `npm run build`, `npm run size` enforces 330,000 raw bytes and 104,000 gzip bytes at level 9 for root `main.js` in the Test workflow's Build and sizes worker. `npm run size:data` reports a synthetic v2 stress library with 50 feeds, 2,000 history entries, and 200 saved episodes from frozen NPR metadata, including duplicated snapshots. Its 1.5 MB document goal is advisory. Pass an additive candidate JSON path to report optional-field overhead without changing existing data. In the isolated Obsidian instance, `npm run count:writes -- start`, `report`, and `stop` record writes to PodNotes' exact `data.json` path. Stop the recorder before stopping the instance, because recorder commands start a stopped instance; plugin reloads invalidate its counts.
After `npm run build`, `npm run size` enforces 330,000 raw bytes and 104,145 gzip bytes at level 9 for root `main.js` in the Test workflow's Build and sizes worker. `npm run size:data` reports a synthetic v2 stress library with 50 feeds, 2,000 history entries, and 200 saved episodes from frozen NPR metadata, including duplicated snapshots. Its 1.5 MB document goal is advisory. Pass an additive candidate JSON path to report optional-field overhead without changing existing data. In the isolated Obsidian instance, `npm run count:writes -- start`, `report`, and `stop` record writes to PodNotes' exact `data.json` path. Stop the recorder before stopping the instance, because recorder commands start a stopped instance; plugin reloads invalidate its counts.

Before opening a PR or cutting a release, run the CI-equivalent checks locally:

Expand Down
11 changes: 10 additions & 1 deletion scripts/size-baseline.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ main.js raw: 307131 bytes; budget 330000 bytes; 22869 bytes remaining
main.js gzip (level 9): 97161 bytes; budget 100000 bytes; 2839 bytes remaining
```

The gzip budget is now 104,000 bytes. On 2026-10-14 it rose from 100,000 for
The gzip budget is now 104,145 bytes. On 2026-10-14 it rose from 100,000 for
the full player's Transcript panel, which the owner approved at about 1.9 KB
and which measured 2,576 bytes. The coordinator set 103,000 to keep the
panel's Find and Follow chip and to leave room for fixes in flight. The
Expand All @@ -34,6 +34,15 @@ main.js raw: 318989 bytes; budget 330000 bytes; 11011 bytes remaining
main.js gzip (level 9): 103397 bytes; budget 104000 bytes; 603 bytes remaining
```

On 2026-11-14 it rose from 104,000 for issue #532: OpenAI diarization cuts
MP3s into the fewest balanced chunks under 20 MB and 600 s, skipping ID3v2
tags. With it:

```text
main.js raw: 320267 bytes; budget 330000 bytes; 9733 bytes remaining
main.js gzip (level 9): 104145 bytes; budget 104145 bytes; 0 bytes remaining
```

Gzip uses Node's `gzipSync` at level 9, without a filename or timestamp.
Budgets use decimal bytes. The brief's 96,533-byte reference is unexplained.
This report fixes the compressor instead of comparing compressed sizes across
Expand Down
2 changes: 1 addition & 1 deletion scripts/size.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ try {
if (!bundle.length) throw new Error(`${path} is empty. Run npm run build first.`);
const measurements = [
{ name: "raw", bytes: bundle.length, budget: 330_000 },
{ name: "gzip (level 9)", bytes: gzipSync(bundle, { level: 9 }).length, budget: 104_000 },
{ name: "gzip (level 9)", bytes: gzipSync(bundle, { level: 9 }).length, budget: 104_145 },
];
const rows = measurements.map(({ name, bytes, budget }) => {
const overage = Math.max(0, bytes - budget);
Expand Down
2 changes: 1 addition & 1 deletion scripts/size.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ describe("bundle budget CLI", () => {
const result = check("gzip.js", randomBytes(110_000));
expect(result.status).toBe(1);
expect(result.stdout).toMatch(
/gzip \(level 9\): \d+ bytes; budget 104000 bytes; OVER by \d+ bytes/,
/gzip \(level 9\): \d+ bytes; budget 104145 bytes; OVER by \d+ bytes/,
);
expect(result.stdout).toContain("budget 330000 bytes; 220000 bytes remaining");
});
Expand Down
72 changes: 71 additions & 1 deletion src/services/TranscriptionService.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
import { describe, expect, test, vi } from "vitest";
import { Notice, type RequestUrlParam } from "obsidian";
import { deferred } from "../../tests/mocks/feeds";
import { OPENAI_FAILURES, rateLimited } from "../../tests/mocks/openaiFailures";
import { OPENAI_FAILURES, OPENAI_REJECTIONS, rateLimited } from "../../tests/mocks/openaiFailures";
import {
createMockPlugin,
diarizeWithDeepgramMock,
Expand All @@ -11,6 +11,7 @@ import {
mockEpisode,
noticeMessages,
} from "../../tests/mocks/transcriptionService";
import { MP3_STREAMS, mp3Bytes, mp3Seconds } from "../../tests/mocks/mp3";
import { openAIMock, sentForm } from "../../tests/mocks/openaiRequestUrl";
import { TranscriptionService } from "./TranscriptionService";
import type PodNotes from "src/main";
Expand Down Expand Up @@ -522,6 +523,18 @@ describe("TranscriptionService", () => {
},
);

test.each(OPENAI_REJECTIONS)(
"Whisper %s shows the API error after one send",
async (_name, respond, message) => {
openAIMock.mockImplementation(respond);

await expect(finalNotice(createMockPlugin())).resolves.toBe(
`Transcription failed: all 1 audio chunk(s) failed or returned no text. Last error: ${message}`,
);
expect(openAIMock).toHaveBeenCalledOnce();
},
);

test("Whisper answering without text fails the run after three sends", async () => {
openAIMock.mockImplementation(async () => Response.json({}));

Expand Down Expand Up @@ -627,5 +640,62 @@ describe("TranscriptionService", () => {
expect(openAIMock).toHaveBeenCalledTimes(3);
},
);

test.each(OPENAI_REJECTIONS)(
"OpenAI diarization %s shows the API error after one send",
async (_name, respond, message) => {
openAIMock.mockImplementation(respond);
const plugin = createMockPlugin();
plugin.settings.transcript.diarization = {
enabled: true,
provider: "openai",
speakerTemplate: "**{{speaker}}:** {{text}}",
};

await expect(finalNotice(plugin)).resolves.toBe(
`Transcription failed: OpenAI diarization failed for every chunk: ${message}`,
);
expect(openAIMock).toHaveBeenCalledOnce();
},
);

test("OpenAI diarization parts stay under the model's 1400 s limit and answer inside the 10 min request timeout (#532)", async () => {
// Measured on gpt-4o-transcribe-diarize: the slowest of 81 requests took 0.58 s per second of audio.
const slowestSecondsPerAudioSecond = 0.58;
const stream = MP3_STREAMS.mpeg25At8Kbps;
getEpisodeAudioBufferMock.mockResolvedValue({
buffer: mp3Bytes(stream, 44_667).buffer,
extension: "mp3",
basename: "episode",
});
const sentSeconds: number[] = [];
openAIMock.mockImplementation(async (request: RequestUrlParam) => {
const seconds = mp3Seconds(stream, (await sentFile(request)).size);
sentSeconds.push(seconds);
return seconds > 1400
? Response.json(
{
error: {
message: `audio duration ${seconds} seconds is longer than 1400 seconds which is the maximum for this model`,
},
},
{ status: 400 },
)
: Response.json({
segments: [{ speaker: "A", text: `${seconds} s.`, start: 0, end: 1 }],
});
});
const plugin = createMockPlugin();
plugin.settings.transcript.diarization = {
enabled: true,
provider: "openai",
speakerTemplate: "**{{speaker}}:** {{text}}",
};

await expect(finalNotice(plugin)).resolves.toBe("Transcription completed and saved.");
expect(Math.max(...sentSeconds)).toBeLessThanOrEqual(1400);
expect(Math.max(...sentSeconds) * slowestSecondsPerAudioSecond).toBeLessThan(10 * 60);
expect(sentSeconds.reduce((total, seconds) => total + seconds)).toBeCloseTo(3216.024);
});
});
});
18 changes: 11 additions & 7 deletions src/services/TranscriptionService.ts
Original file line number Diff line number Diff line change
Expand Up @@ -10,13 +10,14 @@ import type { IPodNotesSettings } from "src/types/IPodNotesSettings";
import { openNote } from "src/ui/openNote";
import { TimerNotice } from "src/ui/TimerNotice";
import { createChunkFiles, getMimeType } from "./audioChunker";
import { createTranscription, retryDelayMs } from "./openaiTranscription";
import { canRetry, createTranscription, retryDelayMs } from "./openaiTranscription";
import {
type DiarizationAudio,
type DiarizationProviderId,
type DiarizedSegment,
diarizeWithDeepgram,
diarizeWithOpenAI,
OPENAI_DIARIZE_PART_SECONDS,
renderDiarizedTranscript,
requiredTranscriptionKeyPresent,
} from "./diarization";
Expand Down Expand Up @@ -382,11 +383,13 @@ export class TranscriptionService {
return segments;
}

// OpenAI diarization shares Whisper's ~20 MB chunk limit (a conservative
// margin under OpenAI's 25 MB request cap), so reuse the same chunking.
// Speaker labels can differ across chunks on a long episode.
// OpenAI diarization takes Whisper's ~20 MB chunks, capped at 600 s so each request
// answers inside the timeout. Speaker labels can differ across chunks.
updateNotice("Creating audio chunks...");
const chunkFiles = await createChunkFiles(audio);
const chunkFiles = await createChunkFiles({
...audio,
maxSeconds: OPENAI_DIARIZE_PART_SECONDS,
});
this.assertActive();
const segments = await diarizeWithOpenAI({
apiKey: this.getApiKey(),
Expand Down Expand Up @@ -447,16 +450,17 @@ export class TranscriptionService {
} catch (error) {
this.assertActive();
retries++;
if (retries >= MAX_RETRIES) {
if (retries >= MAX_RETRIES || !canRetry(error)) {
console.error(
`Failed to transcribe chunk ${index} after ${MAX_RETRIES} attempts:`,
`Failed to transcribe chunk ${index} after ${retries} attempt(s):`,
error,
);
transcriptions[index] = chunkErrorPlaceholder(index);
failedChunks++;
lastError = error;
completedChunks++;
updateProgress();
break;
} else {
await this.waitForRetry(retryDelayMs(error, 1000 * retries));
}
Expand Down
Loading
Loading