Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
91 changes: 43 additions & 48 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -547,7 +547,7 @@ tests are.
Idle it arrived at on its own, and nothing in the renderer read it - which did not show while that
branch was unreachable, and would have turned a blank screen into a silent bounce to the launch
cards the moment it was. The case it exists for is a dead or muted microphone: the silence backstop
skips its way through a session whose questions have already been billed, `finishToScoring` resets
skips its way through a session that has been on the meter the whole time, `finishToScoring` resets
with a message naming the microphone, and the candidate is owed it. `/mock-interview` reports it
deduplicated by the message rather than from one place, because both places see the same string and
neither can be dropped - `start()` writes it onto the session *and* throws, so the broadcast races
Expand Down Expand Up @@ -665,53 +665,48 @@ sheet written for the live report's shape. Picking levels by nesting depth inste
Answer", "Score" and "Stronger Answer" at H5, so three centred labels appeared over left-ranged
body text in every question.

### What a mock interview costs

A live interview is metered per minute; a mock one is priced per question, follow-up and report,
and its ASR socket is not metered at all. The reason is how the two spend their wall clock: every
minute of a real interview is a minute of value, while a mock session spends a large part of its
clock in `Generating`, `Speaking`, `Evaluating` and `Scoring` - none of which the candidate can act
during - and most of the rest on think-time, which is the behaviour the feature exists to train.
Billing that by the second charges for the app talking to itself, and leaves the candidate unable
to find out what a session costs before starting it.

**The prices arrive on the ping** (`AppState.mockPricing`, from `ClientPingResponse.mock_pricing`)
rather than being mirrored as constants, because the backend owns them and a stale copy here would
quote a number the user is not charged. `undefined` is not free and is not zero - it means the
backend predates per-turn pricing and is still metering a mock by the minute, so the client quotes
nothing and gates nothing, which is exactly what it did before any of this existed.

**Two numbers are quoted, and the smaller one is the promise.** `mockSessionPrice()` is every
question plus the report, and it is what the start gate reserves; `mockSessionCeiling()` adds the
maximum follow-ups and is the number you are never charged more than. Follow-ups are charged only
as they are asked, and the backend declines one rather than let it eat into the rest of the
session - so quoting the ceiling as the price would refuse a five-question mock to someone holding
200 credits for a session that will almost certainly cost 170.

**A length the balance cannot cover is disabled in the picker, not refused on Start.** The answer
to "not enough credits" is then a shorter interview the candidate can choose on the spot rather
than a dead end. `checkCanStart`'s own credit check is the backstop for the case where *every*
length is out of reach, and it carries the route out (a Buy credits action). The home screen's mock
card says the same thing one level earlier, against the shortest session there is, so the candidate
is not sent into a dialog in which nothing is selectable.

**A 402 is not retried.** `generateNextQuestion` retries once on failure, which is right for a
provider blip and pointless for a balance: a second attempt cannot succeed and only doubles the
wait before the candidate is told. It is also reported differently - the backend's message already
names the price and the balance, so it is passed through rather than prefixed with "could not
generate the first question", which describes a fault the candidate does not have.

**The client declares how it expects to be billed, and it is the only party that can.**
`MockBilling.PerTurn` goes on all three charged requests and `metered=0` goes on the mock socket,
because only this side knows whether that socket is asking to be metered. An older backend ignores
both and bills by the minute; an older client sends neither and is billed by the minute. No mixed
state charges twice, which is the only outcome that must not happen.

The mock socket sends **both** `channels=1` and `metered=0`, and the pairing is what a tidy-up
breaks. `channels=1` looks redundant once `metered=0` exists, but it is what keeps the older
behaviour correct against a backend that ignores `metered`: the default of two assumes the live
session's pair of sockets, so dropping it would halve every mock interview's bill on every
deployment not yet updated. `test/mock-billing-contract.test.mjs` pins both halves.
### What an interview costs

**Every interview, live or mock, is billed by the minute on its ASR socket, at the same rate, and
nothing else charges.** A mock used to be priced per question, follow-up and report, with
`billing=per_turn` on its requests and `metered=0` on its socket; both are gone. The mock socket is
open from Start until the session reaches Idle or Finished (`use-mock-interview.ts`), so metering it
already covers question generation, speaking, scoring and the report.

The mock socket still sends `channels=1`, and that is the one parameter a tidy-up must not drop:
the backend divides the per-minute price by the number of sockets a session holds, defaulting to
the live session's two, so without it a mock pays half. `test/mock-billing-contract.test.mjs`
pins it, and that nothing still declares per-turn billing - a backend that predates the switch
would honour such a declaration *and* run the clock.

**One start rule for both** (`lib/credit-gate.ts`): at least one minute of credit. The home cards,
the live control panel and the mock setup form all use it and send the user to Buy credits. The
backend refuses only at zero, because it cannot tell a new session from a reconnect inside one; this
side can, so the minimum lives here. An unknown balance (before the first ping) is let through
rather than greying both cards out at launch. The mock setup screen gives an estimate in minutes and
credits and says how far the balance goes, rather than disabling lengths.

**The backend ends a socket at zero with close code 4402** (`WS_CLOSE_INSUFFICIENT_CREDITS`), and
`AudioWsStream` must never reconnect on it - a reconnect is refused the same way, and the backoff
loop would retry it until the user stopped the session by hand. The code is checked in the bound
close handler *before* `active` (a socket refused during `start()` closes before the graph is
built, and `start()` throws `OutOfCreditsError`), before open, and in the retry loop. A 4402 lost on
the network arrives as a 1006, the reconnect is refused with 4402, and the session ends one round
trip later. Live reports it once per session (both channels close), and only once both channels have
started - a refusal during start is `start()` rejecting, not a stop. A report that lands while
`runningState` is still `Starting` is held until `Running`: `startAssistant` writes `Running` a few
seconds after the sockets are up, and a stop inside that window was overwritten by it. Out of stealth it ends through
`useEndLiveSession`, so the transcript can still be saved; in stealth it only stops, like the stop
hotkey, because a save prompt and a jump to the dashboard do not belong on a screen share. The
subscription is in the control panel because its hooks run in stealth mode too. Mock ends through `endSession()`, keeping the
answer in progress, and ignores it once scoring has begun. Its socket opens before main is asked
to start, so a refusal that lands while the session still reads Idle is held and acted on once
main's start resolves. `test/out-of-credits.test.mjs` pins all
of it.

`useLowBalanceWarning` toasts once at five minutes and once at one, off the ping balance. Sockets
also carry `kind=mock` and a per-session `client_session_id`, which the backend uses only to label
and group its ledger rows.

### Window and Stealth Mode

Expand Down
2 changes: 1 addition & 1 deletion src/main/ipc/app-state.ts
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ import { type AppState } from '../types/app-state.js';
* `FAILURE_INTERVAL`/`SUCCESS_INTERVAL` later) silently overwrote it - an unguarded path onto a
* financial field that happened to have no caller today.
*/
const SERVER_OWNED_KEYS = ['credits', 'creditsPerMinute', 'userRole', 'mockPricing'] as const;
const SERVER_OWNED_KEYS = ['credits', 'creditsPerMinute', 'userRole'] as const;

function stripServerOwnedFields(updates: Partial<AppState>): Partial<AppState> {
const sanitized = { ...updates };
Expand Down
11 changes: 3 additions & 8 deletions src/main/services/health-check.service.ts
Original file line number Diff line number Diff line change
Expand Up @@ -63,13 +63,9 @@ export class HealthCheckService {
credits: res.data?.credits,
userRole: res.data?.user_role,
providedLLMModel: res.data?.provided_llm_model,
// Undefined on a backend that predates per-turn pricing, and carried through as
// undefined rather than defaulted: the mock setup dialog reads the absence as "this
// deployment still meters a mock by the minute", and a zero would read as free.
mockPricing: res.data?.mock_pricing,
// Same reasoning: undefined here means "not answered yet", not free and not the
// compiled-in default - the renderer falls back to its own mirror for that case. An
// unusable rate is folded into that same absence; see `usableRate`.
// Undefined here means "not answered yet", not free and not the compiled-in default -
// the renderer falls back to its own mirror for that case. An unusable rate is folded
// into that same absence; see `usableRate`.
creditsPerMinute: usableRate(res.data?.credits_per_minute),
});
} catch (error) {
Expand Down Expand Up @@ -199,7 +195,6 @@ export class HealthCheckService {
credits: res.data?.credits,
providedLLMModel: res.data?.provided_llm_model,
userRole: res.data?.user_role,
mockPricing: res.data?.mock_pricing,
creditsPerMinute: usableRate(res.data?.credits_per_minute),
});
}
Expand Down
34 changes: 7 additions & 27 deletions src/main/services/mock-interview.service.ts
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,6 @@ import {
GenerateMockReportRequest,
isMockInterviewSessionActive,
MockAnswer,
MockBilling,
MockCurrentQuestion,
MockInterviewSessionState,
MockInterviewSetup,
Expand Down Expand Up @@ -63,7 +62,7 @@ function describeApiError(error: unknown): string {
return error instanceof Error ? error.message : String(error);
}

/** A charge the balance could not cover - see `generateNextQuestion`. */
/** A refusal for credits - no longer sent by any backend; see `generateNextQuestion`. */
const HTTP_PAYMENT_REQUIRED = 402;

function initialSession(): MockInterviewSessionState {
Expand Down Expand Up @@ -294,10 +293,6 @@ class MockInterviewService {
// The number this question will carry once installed. `installQuestion` advances the
// counter only for a new question, so a follow-up keeps the one on screen.
question_number: isFollowUp ? this.session.questionNumber : this.session.questionNumber + 1,
// This client pays per turn and its ASR socket asks not to be metered, so it says so on
// every request that is charged for. An older backend ignores the field and meters the
// socket as it always did; see `MockBilling`.
billing: MockBilling.PerTurn,
};

let attempt = 0;
Expand All @@ -309,11 +304,12 @@ class MockInterviewService {
const response = await this.api.generateQuestion(request);
if (seq !== this.sessionSeq) return;

// Not retried, and not lumped in with the failures below. A balance that cannot pay for
// this question cannot pay for it a second time either, so a retry only doubles the wait
// before the candidate is told - and what they are told is already the whole answer,
// naming the price and their balance, so it is passed through rather than prefixed with
// "could not generate the question", which describes a fault they do not have.
// Not expected from any backend now: a mock is billed by the minute on its ASR socket,
// the current backend never answers 402 here, and an older one did only for a client
// that declared per-turn billing, which this one does not. Kept as a guard: if a refusal
// for credits ever does arrive, it is not retried (a balance that cannot pay once cannot
// pay twice) and its message is passed through rather than prefixed with "could not
// generate the question", which describes a fault the candidate does not have.
if (response.status === HTTP_PAYMENT_REQUIRED) {
this.lastQuestionError =
response.error?.message || uiStrings().mockErrors.notEnoughCredits;
Expand Down Expand Up @@ -614,16 +610,6 @@ class MockInterviewService {
this.setState(MockInterviewState.Evaluating);
this.broadcast();

// What the rest of the session still owes, for the follow-up's billing test below. `setup` is
// non-null for any session that has reached `Listening`, but the type cannot know that, and
// reading a missing one as "nothing left to ask" is the harmless direction: the backend then
// tests the follow-up against the report alone, on a session that is already inconsistent.
const remainingQuestions = Math.max(
0,
(this.session.setup?.question_count ?? this.session.questionNumber) -
this.session.questionNumber
);

let action: MockTurnAction = MockTurnAction.Next;
let followUpQuestion = '';
try {
Expand All @@ -633,11 +619,6 @@ class MockInterviewService {
answer: answerText,
kind: question.kind,
follow_up_count: this.followUpCount,
// Lets the backend decline a follow-up that would leave the session unable to finish the
// questions it was quoted. Counted off `questionNumber`, which a follow-up deliberately
// does not advance.
remaining_questions: remainingQuestions,
billing: MockBilling.PerTurn,
};
const response = await this.api.evaluateTurn(request);
if (seq !== this.sessionSeq) return;
Expand Down Expand Up @@ -791,7 +772,6 @@ class MockInterviewService {
profile_data: interviewConfig.profileData,
context: interviewConfig.context,
questions: this.session.answers.map((a) => ({ question: a.question, answer: a.answer })),
billing: MockBilling.PerTurn,
};
const response = await this.api.generateReport(request);
if (seq !== this.sessionSeq) return;
Expand Down
13 changes: 2 additions & 11 deletions src/main/types/app-state.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
* Application State Types
*/

import { MockPricing, UserRole } from './health-check.js';
import { UserRole } from './health-check.js';
import { Language } from './language.js';
import { SuggestionMode } from './llm.js';
import { MockInterviewSessionState } from './mock-interview.js';
Expand Down Expand Up @@ -178,16 +178,7 @@ export interface AppState {
*/
mockInterviewSupported: boolean | null;
/**
* What a mock interview costs per unit of work, or `undefined` before the backend has said.
*
* `undefined` is not free and is not a price of zero - it means this backend predates per-turn
* pricing and is still metering a mock session by the minute on its ASR socket. The client
* reads it as "quote nothing and gate nothing", which is exactly the behaviour it had before
* any of this existed. See `MockBilling` on the backend for the whole compatibility story.
*/
mockPricing?: MockPricing;
/**
* The price of a live interview, in credits per minute, or `undefined` before the backend has
* The price of an interview, live or mock, in credits per minute, or `undefined` before the backend has
* said.
*
* `undefined` does not mean free and does not mean the shipped default - it means this ping
Expand Down
21 changes: 1 addition & 20 deletions src/main/types/health-check.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8,31 +8,12 @@ export enum UserRole {
Admin = 'admin',
}

/**
* What a mock interview costs, in credits per unit of work delivered.
*
* A mock is priced per question, follow-up and report rather than by the clock: most of a mock
* session's wall time is the product generating a question, speaking it, scoring the turn or
* writing the report, none of which the candidate can act during, and the rest is think-time,
* which is the behaviour the feature exists to train.
*
* Sent on the ping because the client needs it before the candidate commits - the setup dialog
* quotes the session and refuses one the balance cannot see through to its report.
*/
export interface MockPricing {
per_question: number;
per_follow_up: number;
per_report: number;
}

export interface ClientPingResponse {
credits: number;
provided_llm_model: string;
user_role: UserRole;
/** Absent on a backend that predates per-turn pricing - see `AppState.mockPricing`. */
mock_pricing?: MockPricing;
/**
* The price of a live interview, in credits per minute. Optional here even though the
* The price of an interview, live or mock, in credits per minute. Optional here even though the
* backend always sends it now: this client ships independently of the hand-deployed backend,
* so an older deployment still answers without it - see `AppState.creditsPerMinute` for the
* fallback that case reads as.
Expand Down
37 changes: 3 additions & 34 deletions src/main/types/mock-interview.ts
Original file line number Diff line number Diff line change
Expand Up @@ -110,29 +110,7 @@ interface MockQuestionHistoryEntry {
answer: string;
}

/**
* How this client expects the work it is asking for to be paid for.
*
* The switch that makes per-turn pricing safe across a hand-deployed backend and a client that
* ships on its own schedule. A mock interview used to be billed by the minute, on the wall clock
* of an ASR socket held open for the whole session; it is now billed per question, follow-up and
* report, and its socket asks not to be metered at all (`metered=0`).
*
* Only this side knows which of those two it is doing, because only this side opens the socket -
* so it says. An older client that says nothing is billed exactly as it was, and a newer client
* against an older backend has both halves ignored and is also billed exactly as it was. No
* combination charges twice, which is the only outcome that must not happen.
*/
export enum MockBilling {
PerTurn = 'per_turn',
}

/** The billing declaration shared by the three mock requests that are charged for. */
interface BilledMockRequest extends LLMRequest {
billing: MockBilling;
}

export interface GenerateMockQuestionRequest extends BilledMockRequest {
export interface GenerateMockQuestionRequest extends LLMRequest {
setup: MockInterviewSetup;
profile_data: string;
context: string;
Expand All @@ -149,23 +127,14 @@ export interface GenerateMockQuestionRequest extends BilledMockRequest {
question_number: number;
}

export interface EvaluateMockTurnRequest extends BilledMockRequest {
export interface EvaluateMockTurnRequest extends LLMRequest {
question: string;
answer: string;
kind: MockQuestionKind;
follow_up_count: number;
/**
* How many questions are still to be asked after this turn.
*
* Billing only. A follow-up is charged as an extra on top of the price the session was quoted,
* so the backend declines one when paying for it would leave the session unable to finish the
* questions it promised. It cannot work that out on its own: `history` counts turns, so it runs
* ahead of the question number wherever a follow-up was asked.
*/
remaining_questions: number;
}

export interface GenerateMockReportRequest extends BilledMockRequest {
export interface GenerateMockReportRequest extends LLMRequest {
setup: MockInterviewSetup;
profile_data: string;
context: string;
Expand Down
Loading
Loading