FE-1795: Prepare Live voice briefs and summarize settled Brunch results

[kostandinang]

🌟 What is the purpose of this PR?

Let people speak naturally to Brunch, see their original words, and hear concise acknowledgements and settled-result summaries.

This PR prepares structured briefs without losing the finalized utterance, adds explicit retention consent, and supports continuing a stopped written response through Voice. Provider-audio behavior remains unverified.

🔗 Related links

• FE-1795 (internal)
• Stack: FE-1793: Unify Chat and Voice conversation presentation #9829 → FE-1793: Scope destructive model-edit approval to one conversation #9856 → FE-1793: Keep experiment runs and results in their originating conversation #9857 → this PR

🚫 Blocked by

• FE-1793: Keep experiment runs and results in their originating conversation #9857
• Record and inspect the complete provider-audio flow before marking ready

🔍 What does this change?

• Grow one display-only user bubble during partial speech, then replace it with one finalized message.
• Show brief preparation as Preparing → Sending → Sent and submit the complete utterance alongside extracted excerpts; display-only “Still open” values are omitted.
• Save finalized words, brief excerpts, and Brunch answers after renewed consent. Spoken-caption annotations remain browser-local.
• Generate short spoken acknowledgements and settled-result summaries instead of reading full written answers.
• Keep sentence-level caption grouping stable across result handoffs and preserve every spoken word and repetition.
• When a written response is stopped, give Live quiet context and let “Continue” request a new Brunch turn without replaying stopped tools or speech.
• Keep steering-specific Ledger, playback, and experiment-relay behavior outside this stack.

Pre-Merge Checklist 🚀

🚢 Has this modified a publishable library?

• Includes the required @hashintel/petrinaut patch changeset.

📜 Does this require a change to the docs?

• Updates the assistant guide and Voice consent wording.

🕸️ Does this require a change to the Turbo Graph?

• No execution-graph changes.

⚠️ Known issues

• Preserving short utterances prevents handoff data loss but does not prove Brunch interprets every reply correctly.
• GPT-Live provides neither stable turn identity nor playback-complete events, so caption grouping and acknowledgement timing remain approximate.
• Saved finalized words live in Brunch history; browser-local caption annotations are removed with browser storage and do not replay audio.
• Stop → Continue, revised prompts, summary quality, acoustic interruption, and provider timing have mocked coverage but no live-audio verification.

🐾 Next steps

Record a support-desk session, inspect how short choices and assumptions are interpreted, and verify acknowledgements, summaries, captions, and Stop → Continue against actual provider audio.

🛡 What tests cover this?

• All 18 build, type, and lint tasks passed. The complete suites passed separately: 946 website tests and 1,516 Petrinaut tests.
• 138 focused mediation tests cover preparation, lossless handoff, consent, captions, interruption, stopped responses, continuation, duplicate events, and error states.
• Controlled browser checks covered the real UI and mocked transport. No provider or microphone session was run, and no petrinaut-core behavior changed.

❓ How to test this?

1. Seed old consent, confirm the new retention disclosure is required, then start a fresh Live session.
2. Speak a support-desk request and confirm one bubble, no partial submission, and Preparing → Sending → Sent for the finalized message.
3. Give a short choice, correction, or permission to use defaults. Confirm the submitted request retains the complete words alongside excerpts and does not present assumptions as observed facts.
4. Compare spoken output with captions, then stop a written answer and say “Continue”. Confirm a new Brunch turn continues the answer without replaying stopped work or audio.

📹 Demo

Controlled screenshots show the Voice brief, separate spoken and written replies, and Voice-origin messages. They do not verify live speech or playback timing.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
hash	Ready	Preview	Oct 1, 2026 6:28am UTC
petrinaut	Ready	Preview	Oct 1, 2026 6:28am UTC
petrinaut-docs	Ready	Preview	Oct 1, 2026 6:28am UTC

1 Skipped Deployment

Project	Deployment	Actions	Updated
hashdotdesign-tokens	Ignored	Preview	Oct 1, 2026 6:28am UTC

[cursor]

PR Summary

Medium Risk
Touches the Live–Brunch admission pipeline, a new OpenAI-backed mediation API, and complex async interruption/continuation behavior, though canonical conversation authority and tool execution are explicitly preserved.

Overview
Adds a browser-local display projection for Live Brunch voice so chat can show verbatim user speech, brief-preparation state, and spoken captions while canonical Brunch transport and history stay unchanged. Brunch chat wires mapMessagesForDisplay through per-conversation VoiceMediationHistory (persisted finalized turns; captions stay local).

Live admission path now runs POST /api/voice/mediation for structured brief extraction before submit (utterance plus excerpts; failed prep falls back to raw text and marks display-only failure) and wrap-up summarization after settlement so GPT-Live gets short spoken lines instead of full written answers. LiveBrunchBridge gains superseded-transcript handling, richer stop/error/speech-interruption instructions, and quiet appendThinking context when a response is stopped so Continue admits a new turn without replaying partial speech.

Live session UX streams provisional input from provider transcript deltas (LiveSpeechCaptions), pins annotations to the conversation a turn started in, and bumps consent to v3 with clearer text-retention disclosure. Server Live prompts and shared request guarding are updated accordingly; docs and large test suites cover mediation, captions, and interruption flows.

^{Reviewed by Cursor Bugbot for commit 96a72ae. Bugbot is set up for automated code reviews on this repo. Configure here.}

[cursor]

Stale Bugbot comment from a previous run.

[cursor]

Stale Bugbot comment from a previous run.

[cursor]

Stale Bugbot comment from a previous run.

[cursor]

Stale Bugbot comment from a previous run.

[lunelson]

Thanks — all threads addressed.

One non-blocking follow-up: the "Still open" contract now crosses the package boundary. The website writes stillOpen as a comma-joined string or the literal "None identified in this turn", and voice-input-provenance.tsx in the published package splits and compares that string. A copy change on either side breaks the display. Passing the open field names as a list in the data part, like preparationFailed, would remove the string contract.

[cursor]

Stale Bugbot comment from a previous run.

[cursor]

Stale Bugbot comment from a previous run.

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 96a72ae. Configure here.}