Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
56 changes: 43 additions & 13 deletions docs/avatar-contract.md
Original file line number Diff line number Diff line change
Expand Up @@ -318,22 +318,52 @@ in the cost column for a borrowed mascot.

### Captions

`#captions-bar` lives inside `#jim-stage`, under the avatar. It sizes to the
turn rather than clamping: a fixed `max-height` cut sentences in half behind a
scrollbar nobody reaches for mid-interview. `CAPTION_MAX_CHARS` already bounds
the text, so the remaining `min(40vh, 20rem)` cap is a backstop, not the normal
case.

It is a live subtitle, not a log. The transcript tab keeps every turn, so the
bar hides after `CAPTION_IDLE_HIDE_MS` (12 s) with nobody speaking and returns
on the next word. `init` arms the timer for the "Listening..." placeholder too,
or it would sit over the editor for the whole interview having said nothing.
`#captions-bar` lives inside `#jim-stage`, under the avatar. It grows to fit
its text, with `max-height: min(40vh, 20rem)` and scrolling as a backstop.

A completed Jim turn is displayed in full, without `CAPTION_MAX_CHARS`
truncation or an idle timeout. It stays until the candidate's next transcribed
speech or an edit that changes the code text. Speech replaces it with the
candidate's caption; a code text change hides the bar. Clicking or focusing
the editor and switching languages preserve Jim's caption. A new Jim turn
replaces the previous one and starts at the top of the caption bar.

Keeping the last turn lets the candidate reread a hint or follow-up question
while thinking, without switching away from the problem. The Transcript tab
still keeps every turn.

Jim captions reveal gradually, even after final text arrives, because
generation can finish while audio is still queued. An edit during the reveal
is remembered for that Jim segment: its final text finishes revealing, then
the bar hides without another edit. While revealing, Jim captions keep the
`CAPTION_MAX_CHARS` window and `CAPTION_IDLE_HIDE_MS` timeout, refreshed by
pacing updates. Candidate captions keep their 160-character window and 12 s
timeout. The "Listening..." placeholder also keeps its 12 s timeout.

A stream with `lk.transcription_final` set to `"true"` must be read
successfully to its end before its turn can be retained. Once the reveal
reaches the end, the full turn stays without an idle timeout. A broken text
stream leaves partial captions temporary. The reveal uses a fixed rate;
audio playout timing is independent. Updates for already dismissed or
replaced Jim segment IDs are ignored. Final copies of earlier candidate turns
do not dismiss Jim's current turn.

A longer interim update to the same candidate segment can pause Jim's
interim reveal without retiring it. Jim's next update can resume the reveal.
A new candidate segment or a candidate response after Jim's final retires
Jim's segment.

## Checks

`node --test tests/browser/avatar.test.js` covers the behavior with a fake
clock, a fake mount, and a fake model. It cannot see WebGL, a canvas, or a
layout, and no assertion in it should claim to.
`node --test tests/browser/avatar.test.js tests/browser/captions.test.js`
covers avatar behavior with a fake clock, a fake mount, and a fake model.
Caption tests use a fake clock and actual page functions to cover retention,
dismissal, final-stream handling, and editor input. These checks cannot see
WebGL, a canvas, or a layout, and no assertion in them should claim to.

`node --test tests/browser/editor-pipeline.test.js` runs Chromium against the
interview page. It checks caption retention during editor focus, caret
movement, and language switches, plus scroll reset between Jim turns.

`BROWSER_CHECK_FLOW=avatar ./scripts/browser-check.sh` is the only check that
runs the renderer. It asserts the mount leaves `loading` and then asserts
Expand Down
56 changes: 10 additions & 46 deletions tests/browser/avatar.test.js
Original file line number Diff line number Diff line change
Expand Up @@ -720,59 +720,23 @@ test("avatar blink and breath stay inside their envelope", () => {
assert.ok(Math.abs(breathOffset(1234)) <= BREATH_AMPLITUDE);
});

// What the executable tests cannot see: which track the analyser is pointed at.
// Getting this wrong is not a rendering bug, it is the avatar watching the
// candidate, so it is worth a text assertion.
// Captions are a live subtitle under Jim's face, not a log; the transcript tab
// keeps every turn. Two properties, both invisible to a node test at runtime,
// so they are pinned where they are decided.
test("captions size to the turn and retire when nobody is speaking", () => {
// Caption timing is covered by executable tests in captions.test.js.
// These source assertions pin CSS rules; Node does not render the layout.
test("caption CSS keeps a viewport cap and scrolling", () => {
const css = read("web/styles.css");
const bar = css.slice(css.indexOf(".jim-stage .captions-bar"));
const start = css.indexOf(".jim-stage .captions-bar");
assert.ok(start >= 0, "the caption sizing rule must exist");
const bar = css.slice(start);
const rule = bar.slice(0, bar.indexOf("}"));
// A fixed max-height cut sentences in half behind a scrollbar. The cap that
// remains is viewport-relative, a backstop rather than the normal case.

assert.doesNotMatch(
rule,
/max-height:\s*\d+(\.\d+)?rem;/,
"a fixed max-height truncates the turn instead of fitting it",
"captions must retain a viewport-relative height cap",
);
assert.match(rule, /max-height:\s*min\(40vh/);

// Shown on every new caption, hidden after an idle stretch, and the
// placeholder is armed too or it would sit there for the whole interview.
assert.match(script, /const CAPTION_IDLE_HIDE_MS = \d+;/);
const show = functionBody(script, "showCaptions");
assert.match(show, /nodes\.captionsBar\.hidden = false;/);
assert.match(
show,
/clearTimeout\(captionIdleTimer\)/,
"a new turn must reset the countdown, not stack timers",
);
assert.match(
show,
/setTimeout\([\s\S]*?nodes\.captionsBar\.hidden = true;[\s\S]*?CAPTION_IDLE_HIDE_MS\)/,
);
assert.match(
script,
/nodes\.captionsText\.textContent = `\[\$\{speaker[^`]*`;\n\s*showCaptions\(\);/,
"every caption update must un-hide the bar",
);

// The reveal has to keep asking for the next tick while there is more of the
// turn to show. Dropping the re-arm leaves Jim's line frozen part-way through
// a sentence, which reads as a stall rather than as a bug.
const pace = functionBody(script, "paceInterviewerCaption");
assert.match(
pace,
/caption\.timer = setTimeout\(paceInterviewerCaption, CAPTION_TICK_MS\);/,
"an unfinished turn must schedule its next tick",
);
assert.match(
pace,
/caption\.shown < caption\.text\.length/,
"and stop scheduling once the whole turn is on screen",
);
assert.match(rule, /overflow-y:\s*auto;/);
assert.match(rule, /pointer-events:\s*auto;/);
});

// A node test has no layout engine, so it cannot see whether the stage
Expand Down
Loading