Skip to content

Bill mock interviews by the minute and stop sessions at 0 credits - #138

Merged
alpha5611331 merged 10 commits into
mainfrom
feat/minute-billing
Oct 6, 2026
Merged

alpha5611331 merged 10 commits into
mainfrom
feat/minute-billing

Conversation

@alpha5611331

Copy link
Copy Markdown
Member

Closes #137
Backend: PowerInterviewAI/backend#67 (deploy that first)

What

Mock interviews are billed by the minute like live ones, and both stop when credits run out.

  1. Per-minute mock billing (5d82110):
    • Removes MockBilling/billing=per_turn, remaining_questions, metered=0, mockPricing and the price quote.
    • New shared start rule in lib/credit-gate.ts: at least 1 minute of credit. It's used by both home cards, the live control panel and the mock setup form, each with a Buy credits action.
    • Mock setup shows "about N min, about X credits", how far the balance goes, and that the session ends with its report when credits run out. Lengths are no longer disabled.
  2. Stop at 0 (7c164b1):
    • AudioWsStream never reconnects on close code 4402. It checks for the code in the bound close handler (ahead of active), before open, and in the retry loop.
    • start() throws OutOfCreditsError for a socket refused while starting.
    • Live gets one stop per session, through the normal Stop path, so the transcript can still be saved.
    • Mock ends through endSession(), keeping the answer in progress and going to the report. A 4402 during Scoring is ignored.
    • Sockets carry kind=mock and a per-session client_session_id for the backend ledger.
  3. Low-balance warnings (62a8d15): a toast at 5 minutes and at 1 minute left, live and mock.
  4. Docs (1c1b6d7, b38cac9).

The title bar already shows the remaining time on every page, including the mock session, so no extra display was added there.

Compatibility

  • Old backend: it ignores kind and client_session_id. This client sends no metered, so a mock is billed at 10/min on the old ASR meter, and no billing field is sent that could also charge per turn. Nothing is charged twice.
  • No 4402 from an old backend: the stop at 0 only happens once the backend from backend#67 is deployed.

Tests

  • pnpm test:main: all checks pass.
  • tsc (both configs) and eslint are clean.
  • New test/credit-gate.test.mjs runs the start rule at runtime.
  • New test/out-of-credits.test.mjs pins:
    • no reconnect on 4402, in every path
    • one stop per session
    • the mock Scoring exemption
    • the warnings
    • the URL tags
  • test/mock-billing-contract.test.mjs is rewritten for the new contract, with channels=1 still pinned.

Manual checks before release

  • Live with 70 credits: both warnings appear, then the stop at 0 with the save prompt, with no reconnect loop in the console. Start is disabled afterwards.
  • Mock with 70 credits: at 0 the session goes to the report and keeps the last answer.
  • With 9 credits: both Start paths are blocked with Buy credits.

🤖 Generated with Claude Code

alpha5611331 and others added 5 commits October 6, 2026 09:56
Drop per-turn mock pricing: MockBilling and billing=per_turn on the
three mock requests, remaining_questions, metered=0 on the mock socket,
mockPricing on the ping and the price quote in the setup form.

Live and mock now share one start rule (lib/credit-gate.ts): at least
one minute of credit. The home cards, the live control panel and the
mock setup form all use it and offer Buy credits. The setup form shows
an estimate in minutes and credits, how far the balance goes, and that
a session ends with its report when credits run out.

Refs #137

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The backend closes an ASR socket with 4402 when the balance reaches
zero, and refuses one opened at zero. AudioWsStream never reconnects on
that code (in the bound handler, before open, and in the retry loop),
and start() throws OutOfCreditsError for a socket refused while it was
starting.

- Live: one out-of-credits event per session (both channels close),
  which the control panel turns into the normal Stop path, so the
  transcript can still be saved, plus a Buy credits toast.
- Mock: ends through endSession(), keeping the answer in progress and
  going on to the report. Ignored once scoring has begun.
- Sockets carry kind=mock and a per-session client_session_id so the
  backend ledger can group one interview's sockets.

Refs #137

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A running live or mock session gets one toast when the balance falls to
five minutes and one at one minute, read off the ping balance. A session
that starts below a threshold gets only the lowest one it is under. Not
shown while a mock is being scored.

Refs #137

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Refs #137

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Refs #137

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@gitar-bot

gitar-bot Bot commented Oct 6, 2026 •

Copy link
Copy Markdown

Gitar is working

Gitar

@alpha5611331

Copy link
Copy Markdown
Member Author

Review notes

I found no blocking issues. These are worth knowing:

  1. Start race against a stale balance. If the ping balance is stale and the backend refuses the socket at 0, the 4402 usually arrives while start() is still building the audio graph, so start() throws OutOfCreditsError and the start fails cleanly. If it arrives after start() has finished, the out-of-credits stop fires while startAssistant is still completing. The 1-minute start gate makes this unlikely (it needs a stale balance at exactly 0), and the session still ends, but the start and the stop may overlap.
  2. A language switch after the session ran out also shows the language menu's error toast ("Out of credits..."), alongside the out-of-credits toast. The text is accurate, but there are two toasts.
  3. The 402 path in mock-interview.service.ts (lastQuestionUnaffordable) is now unreachable against the new backend, which no longer answers 402 on /question. It's kept because an old backend could still send it, and it does no harm.
  4. The warnings read the 5-second ping balance, so they can lag the real balance by one ping. The stop itself is enforced by the backend.
  5. Old backend (current prod): this client sends no metered and no billing, so a mock is billed at 10/min on the old ASR meter and is never also charged per turn. The stop at 0 and the ledger tags only take effect once PowerInterviewAI/backend#67 is deployed.
  6. Installed clients on the new backend are refused at the handshake at 0 and back off (fixed in backend a42829a). They show "disconnected" until stopped or updated.

Verified

  • pnpm test:main passes, including the new credit-gate and out-of-credits checks.
  • tsc (both configs) and eslint are clean.

Still to do by hand: the end-to-end runs listed in the PR description. The stop at 0 in particular needs the backend from #67 running locally.

Out of stealth, running out of credits ends through the Stop path with
the save prompt. In stealth a screen share is likely live, so it now
stops the way the stop hotkey does: no modal, no jump to the dashboard,
and the next Start still asks about the transcript.

The live service also reports out-of-credits only once both channels
have started. A channel refused while the other is still starting is
start() rejecting, and reporting it too ran a stop beside the start
path's own teardown.

Refs #137

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@alpha5611331

Copy link
Copy Markdown
Member Author

Review pass 2

Fixed in 1a44668

  1. Running out of credits in stealth mode no longer puts a modal on a screen share. The out-of-credits stop used to go through useEndLiveSession every time, which raises the save prompt and navigates to the dashboard. In stealth a screen share is likely live, and that's exactly why the stop hotkey deliberately avoids both. In stealth it now stops the way the hotkey does. The transcript is kept, and the next Start asks about it.
  2. No stop running beside a failed start. If one channel was refused for credits just after the other finished starting, the service reported out-of-credits while startAssistant was still running, so a stop ran alongside the start path's own teardown. It now reports only once both channels have started; a refusal during start is just start() rejecting.
  3. A comment in the control panel was moved back above the code it describes.

Checked and consistent with backend#67 and the merged admin and hero changes

  • Close code: 4402 on both sides.
  • Socket parameters: kind=mock only for mock. client_session_id is a crypto.randomUUID(), which fits the backend's id shape.
  • Thresholds: the backend refuses at 0 and the client requires 1 minute to start. That split is intended.
  • Every live start path (the home card, the router auto-start, the Start button) goes through checkCanStart. There is no hotkey that starts a session.
  • Copy: the admin help text and the hero copy match: 10/min, a 1-minute minimum, warnings at 5 and 1 minutes, the stop at 0 and the mock report.

Small leftover: hero's live-interview.md says the session "offers to save the transcript" at zero. That holds except in stealth, where it now just stops. It's a one-line doc tweak if you want it exact.

pnpm test:main, both tsc configs and eslint are clean.

startAssistant writes Running a few seconds after the sockets are up. A
4402 inside that window stopped the session and was then overwritten by
the start path writing Running, leaving a console that showed a live
session with no sockets behind it. A report during Starting is now held
and acted on once Running, or dropped if the start fails.

Also: a language switch refused for credits no longer warns that the
language only half applied (the session is ending), and a start refused
for credits says "not enough credits to start" instead of "the session
has ended".

Refs #137

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@alpha5611331

Copy link
Copy Markdown
Member Author

Review pass 3

Fixed in 2161d73

  1. The console could get stuck showing "Running" with nothing behind it. startAssistant writes Running about 3 seconds after the sockets are up. A 4402 inside that window stopped the session, and then the start path wrote Running over the stop. A report that arrives during Starting is now held: it's acted on once the session reaches Running, or dropped if the start fails. It needs a stale balance at the edge of zero, so it's rare, but it left the app in a broken state.
  2. Wrong toast on a language switch at zero. A switch refused for credits showed "language half-applied / suggestions only" on top of the out-of-credits toast. It now stays quiet, because the session is ending and the out-of-credits toast already says so.
  3. Wrong message on a start refused for credits. It said "the session has ended". It now says "Not enough credits to start".

Rechecked with no new findings

  • Every live start path goes through checkCanStart, and there's no start hotkey.
  • The mock out-of-credits handling (ignored during Scoring, otherwise endSession()).
  • The contract with backend#67: 4402, kind, the client_session_id shape, and the refusal at 0 versus the 1-minute start minimum.

pnpm test:main, both tsc configs and eslint are clean.

The mock socket opens before main is asked to start, so a 4402 could
land while the session still read Idle, and the listener ignored it.
The session then ran on a socket that never reconnects: questions
generated, no transcription. A refusal in that window is now held and
acted on once main's start resolves.

Also puts the unmount-cleanup comment back above the effect it
describes.

Refs #137

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@alpha5611331

Copy link
Copy Markdown
Member Author

Review pass 4. Fixed 049bc30: the mock socket opens before main is asked to start, so a 4402 could land while the session still read Idle. The listener ignored it, and the session ran on a socket that never reconnects: questions were generated, but nothing was transcribed. That refusal is now held and acted on once main's start resolves, the same way the live side was fixed in 2161d73. I also moved the unmount-cleanup comment back above its effect. Checked with no change needed: nothing in src/ or test/ still references the removed billing fields. pnpm test:main, tsc and eslint are clean.

Main's start resolves over one IPC message and the state saying the
session is running arrives over another, so this side can still read
Idle for a moment after start() returns. A 4402 in that gap was parked
as pending after the pending check had already run, and dropped. Once
main has started, a refusal reading Idle is now acted on at once.

Refs #137

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@alpha5611331

Copy link
Copy Markdown
Member Author

Review pass 5. Fixed 403257f: main's start resolves over one IPC message and the state saying the session is running arrives over another, so the renderer can still read Idle for a moment after start() returns. A 4402 in that gap was parked as pending after the pending check had already run, and was dropped. Once main has started, a refusal that still reads Idle is now acted on immediately. I re-checked the live hold-until-Running path from 2161d73 and found no gap there. Tests, tsc and eslint are clean.

@alpha5611331 alpha5611331 self-assigned this Oct 6, 2026
@alpha5611331

Copy link
Copy Markdown
Member Author

Review pass 6: termination, network loss, several clients on one account

I traced each scenario through the code in backend#67 and client#138. Nothing in either PR needed fixing. I added one test, backend bc2dfa4, for the concurrent-clients case.

App closed or killed during an interview

  • Normal quit (window button, Cmd+Q): the close guard still offers to save first. When the renderer goes away, its sockets close with 1001. The relay loop in _stream_client_to_provider sees the disconnect, and its finally calls close(). That charges the unpaid tail (to the nearest credit) and writes ASR_STOP with the total, reason_code and client_session_id. Nothing is charged after that.
  • Crash or kill: the OS closes the TCP connection, so the result is the same as above.
  • Power loss or laptop lid: no close reaches the server. The 10s/10s ping drops the socket within 20s. The meter, and then the tail, charge up to that point. That's at most about 3 credits per live session. It was 40s before this PR.
  • Mock killed mid-report: the report generation finishes on the backend and isn't delivered. The time up to the socket closing was billed. This hasn't changed.

Network lost during an interview

  • Server side: the socket is dropped within 20s and billing stops. Nothing is charged while offline.
  • Client side: the browser often doesn't notice until the network comes back. Audio is dropped once the send buffer is full. On reconnect it gets 1006 and opens a new socket with the same client_session_id, so the ledger groups the rows.
  • Brief overlap on reconnect: if the client reconnects before the server has timed out the old socket, both are billed for the overlap. That's at most 20s on one channel, about 2 credits.
  • A 4402 lost on the network: it arrives as 1006. The reconnect is accepted and closed with 4402, and the bound close handler then ends the session (active is true during a reconnect). This settles itself.
  • Flapping network: each short socket pays its tail, so frequent reconnects no longer mean free transcription.
  • Mock HTTP calls during an outage follow the existing paths: retry once and end, move on, or Retry on the report. No billing is involved, because nothing charges outside the socket.
  • Backend deploy or restart: sockets close with 1012, which isn't 4402, so clients reconnect with backoff. The charge up to the drop is already taken. ASR_STOP can be missing for a killed process; that's the ledger only.

Several clients on one account

  • Shared balance: deduct_credits is one atomic pipeline update that clamps at zero and reads the pre-image. Concurrent meters can neither take more than the balance between them nor charge any of it twice.
  • Both stop at 0: the device whose deduction empties the balance closes with 4402. The other device's next deduction reads 0 and closes within one interval (12s). The new test test_two_devices_on_one_account_drain_it_once_and_both_stop pins all of this: the totals add up to the starting balance, it ends at 0, and both sockets get 4402.
  • Starting a third session at 0: it's refused at the handshake, with 4402 for new clients and a rejected handshake for older ones.
  • Each device warns on its own, from its own ping. The ping reports the shared balance, so both see the combined drain.
  • Ledger: each device's sessions have their own client_session_id, so rows from two devices don't mix.
  • Same machine: the single-instance lock prevents two app instances, and live and mock can't run at once on one device.

Checks

  • Backend: 37 tests in test_asr_billing.py, ruff, mypy and the pre-push hook all pass.
  • Client: unchanged in this pass, with CI green at 403257f.

Refs #137

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@alpha5611331

Copy link
Copy Markdown
Member Author

Review pass 7: no functional findings. b8b978f rewrites a stale comment: the mock 402 handling said a refusal names the per-question price. No backend can send that 402 to this client any more (the current backend never answers 402; an older one did only for clients that declared billing=per_turn). It's kept as a guard, and the comment now says so. A grep of src/, test/ and CLAUDE.md for other per-turn pricing wording found nothing else.

@alpha5611331

Copy link
Copy Markdown
Member Author

End-to-end run against a local backend

I ran the backend from this branch against a local MongoDB and real Deepgram. A throwaway test user was used and removed afterwards.

Bug found and fixed: 2403855 (backend)

A client-initiated close never charged its tail or wrote ASR_STOP.

  • What happened: on a client disconnect, the relay task calls close(). Closing the provider socket inside it ends the provider task, which wakes run(), which cancels every pending task, including the relay, part-way through teardown.
  • Effect: on every Stop, killed app or dropped network, the ledger row was missing and, with this PR, the unpaid tail went uncharged. Only out-of-credits closes, which the meter starts, got their row.
  • Age: the missing ledger row predates this PR. The tail charge added here turned it into a billing gap as well.
  • Fix: close() now runs the teardown once, in its own task, and every caller awaits it shielded.
  • Test: the new unit test test_a_teardown_cancelled_part_way_still_charges_and_logs fails without the fix.

Corrected claim: a vanished client is dropped in about 20-40s, not 20s. After a missed pong, the legacy websockets protocol waits its own 10s close timeout, which uvicorn doesn't expose. Measured: 28-41s. CLAUDE.md and cfg/asr.py are updated.

Backend scenarios: raw protocol, after the fix

Scenario Result
New client opens at 0 credits closed with 4402
Old client (no client_session_id) at 0 handshake refused, HTTP 403
Reconnect with 3 credits accepted
Mock runs out (5 credits) 4402 after 38s, balance 0, ledger 5, mock_minutes
Live killed abruptly after 20s (two sockets) charged 4, two ASR_STOP rows, ledger 4, nothing charged after
Dead network (client stops reading and ponging) dropped after 28s, 5 credits for about 30s of mock (exactly 10/min)
Two devices, one account, 6 credits, four sockets all closed with 4402, balance 0, ledger total 6

Client socket code (AudioWsStream from client#138, transpiled and run against the same backend)

Scenario Result
Start at 0 one socket opened, refused, no retry; next connect throws OutOfCreditsError ("Not enough credits to start"); session not notified
Running out mid-session (2 credits) onOutOfCredits fired exactly once, after 14s; no reconnect
Ordinary drop with credits left reconnects (the 4402 guard doesn't block it)

Not covered here

The Electron UI itself: the save prompt, the stealth behaviour, the warning toasts and the Buy credits links. Those need someone to click through the app with a microphone and screen capture.

One Deepgram connect timeout came up during the run (timed out during opening handshake). The session never started and correctly wrote no row. It's external flakiness, unrelated to this change.

@alpha5611331
alpha5611331 merged commit 0fc4f34 into main Oct 6, 2026
1 check passed
@alpha5611331
alpha5611331 deleted the feat/minute-billing branch October 6, 2026 16:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bill mock interviews by the minute and stop sessions at 0 credits

1 participant