[bgotthold-aai]

Builds on the HTTP client from #277 (merged).

What

aai.TTSSession and aai.AsyncTTSSession: a client for the Text-to-Speech WebSocket API (wss://tts.us.assemblyai.com/v1/ws, host from settings.tts_base_url). One connection, text in as it is written, audio out, for voice agents.

• with aai.TTSSession(config=TTSConfig(voice_id="aai-jane")) as session: connects and waits for Begin; a refusal raises TTSError there. Leaving the block sends Terminate, waits for Termination and sets session.termination.
• speak(text) sends Generate + Flush (splitting text longer than one frame) and returns an iterator of that request's audio bytes, ending at its FlushDone.
• generate(text), flush() -> flush_id, audio(flush_id) stream text in as an LLM produces it and pipeline requests; cancel() is the barge-in; keep_alive() for sparse sessions.
• on(TTSEvents.X, handler) delivers Audio, FlushDone, WordBoundaries, Cancelled, Termination and Error frames to handler(event) on the reader thread. With an Audio handler registered, audio is pushed instead of queued for pulling.
• token= for a temporary token (query parameter) instead of the API key header; word_boundaries and inactivity_timeout as keyword arguments. TTSConfig and TTSError are shared with the HTTP client (container/bit_rate are left out of the URI; an Error frame maps error_type to error_code with the frame in body).
• AsyncTTSSession: async with, coroutine sends, async for chunk in session.speak(text), handlers may be coroutines.

Out of scope: reconnect/backoff, the OpenAI route, voice management.

How

Mirrors the streaming STT client's shape (reader thread or task, lazy websockets import through its connect seams, websockets>=11) with a pull queue on top, since the HTTP client already teaches for chunk in .... Details that matter:

• flush_id is tracked client-side: every Flush and Cancel advances it, as the API does. A Cancel records a watermark so every pipelined request it dropped ends its audio() iteration instead of hanging.
• The pull queue is bounded (2,000 frames, about 160 s); when nothing reads it the oldest audio is dropped with one warning, so a push-style caller cannot grow memory without bound.
• Close codes come from the close frame (exc.rcvd; none means 1006), the non-deprecated reading on every supported websockets release; 1006, 1011, 1012 and bare 3005/3008/3009 closes are retryable. An unreadable frame ends the session with a TTSError rather than killing the reader. A send after a failure raises the stored error, and a failure nothing in the with block observed is raised when the block ends.
• Settings are copied (resolve_settings), so a session builds no HTTP pool and token= does not cache a keyless default client.
• A session object can be reused: connect() resets its per-connection state.

Tests

• tests/unit/test_tts_session.py and test_tts_session_async.py: 37 tests against a small fake server that answers Generate/Flush/Cancel/Terminate the way the API does: URI and headers, speak, pipelining, barge-in before and after a request, pipelined cancel, errors before and after Begin, abnormal close, unreadable frames, handlers (sync and coroutine), the pull/push rule, token auth, settings host and scheme, text limits, reuse, the bounded queue.
• Full suite (830) passes on pydantic 2 with and without pydantic-settings, pydantic 1.10, httpx 0.22 and websockets 11.0.
• Live against production: first audio 84 to 118 ms per sentence on a warm session, pipelined flush ids, barge-in with the next sentence back in ~300 ms, six word-boundary frames, the async twin over 8 kHz mu-law, session reuse, and a voice_not_found refusal raised from the with.
• ruff and mypy clean on the new files. A high-effort self-review ran before the first push; its ten findings are fixed in the first commit.
• Three independent first-time users (voice agent with token streaming and barge-in, telephony with mu-law re-framing, word boundaries, temporary tokens and idle timeouts, and a README copy-paster on Python 3.9 with mypy) installed the branch into fresh venvs and used it from the docs alone. Their findings are the second fix commit: a barge-in no longer plays in-transit audio, audio() cannot hang on an unissued id and can follow the request being written, misspelled event names and pulling under an Audio handler are refused, HTTP-only config options are refused on a session, session errors carry the session id, and a session opens cleanly under -W error (shared _audio._config_to_json now prefers model_dump, a one-line change that also removes the pydantic warning from the dictation and sync clients).

🤖 Generated with Claude Code

[aleks-mitov]

Thanks Ben, this is a really nice client. The pull and push split, the flush_id watermark and the fake server made it easy to review and to build on.

I pushed fixes to this branch as normal commits on top of 1e822d3 (no force push). Every code change comes with tests. At c6fbbbb the full unit suite, ruff and mypy pass, also on the websockets 11 and pydantic 1.10 floors and with the session tests under -W error. A live run against our sandbox (speak, pipelining, barge-in, a cancel from another thread, an idle close followed by a reconnect, the async twin and a cancelled async connect) passed too.

• c287dff: every audio() and speak() drain in the session tests has a deadline, so a regression fails the run instead of hanging it.
• 36b4b4a: the close code is read from the close frame (exc.rcvd, 1006 when none came) instead of the deprecated .code and .reason, so the sessions no longer hang under -W error. Any failure in the reader still ends the session with a TTSError. A bare close with 3005, 3008 or 3009 is retryable, and a missing code counts as 1006.
• 60b8e96: a full pull queue drops only the oldest audio frame and never a FlushDone, Cancelled, error or close marker, so audio(flush_id) always ends. The warning now also covers a reader that is just slow.
• 81bb42f: sync flush() and cancel() take their id and send under one lock, so ids follow wire order when a cancel comes from another thread (the usual barge-in).
• 0a0d8ed: a send that races the API's close raises the session's TTSError (or one built from the close code) instead of a raw ConnectionClosedError.
• 377107f: AsyncTTSSession is cancellation safe. A cancelled connect() closes what it opened, terminate() always closes, and close() lets the caller's CancelledError through.
• 2992e60: waiters are released before Error handlers run, and speak(), audio() and terminate() raise RuntimeError from a handler instead of deadlocking.
• 0210912: connect() reopens a session the API has ended (for example after an idle close) instead of raising "already connected", and refuses while a handler of the previous connection is still running. The docs say a token= session cannot reconnect, since the token is spent at Begin.
• 418feae: the inactivity_timeout and keep_alive() docs match the API: 5 to 120 s, 120 s by default, and playing audio already received counts as idle time.
• d36dcbe: a failed handshake keeps its HTTP status (legacy and current websockets clients) and is retryable for 408, 429, 500, 502, 503 and 504, a refused connection and a timeout.
• 8da63b1: a session error's close code is always in body["close_code"], and the TTSError docstring, README and CLAUDE.md describe session errors (status_code None, close codes 3006 and 1011, retryable from the frame).
• 9a19c69: generate() checks text against the limit Begin reports (limits.max_generate_text_length) and falls back to 2,000.
• 27b13b3: a Begin with a missing or null configuration still opens the session.
• 6c83204: TTSSession.audio() before the first connect raises RuntimeError, as the async session already did.
• 3775aeb: a path prefix in tts_base_url is kept, so https://proxy.example.com/tts becomes wss://proxy.example.com/tts/v1/ws.
• c6fbbbb: container=raw is accepted on a session, and the wav and bit_rate check runs when the session is built.

With these, the client matches two API changes that just merged on the server and go live with the next TTS release: voices are named by voice_id only (the shared TTSConfig from #277 already refuses voice), and every session closes after 120 s with no message either way, which inactivity_timeout can only shorten. That close reaches the app as a retryable inactivity_timeout error, and the same session object can then reconnect.

The remaining items are inline: a few design calls I left for you and one follow-up idea. One small thing outside the diff: the "How" section of the description still says close codes come from exc.code; after 36b4b4a they come from the close frame.

[bgotthold-aai]

Thanks for the fixes, Aleks. Pulled them as they are, and the close-code change was right: on websockets 17 .code / .reason warn, and my earlier reading hung the suite under -W error. The design calls are answered inline (four taken, two kept with reasons).

Reviewing the new code paths turned up four things, fixed in 1946d4e with tests: a stale audio() iterator kept reading a reconnected session's queue (it now ends when the connection changes), a send that met the API's close waited up to a second while holding the send lock so a handler that sends could block the reader (the wait now happens outside the lock), a handshake that failed on a reconnect left fresh state for audio() to poll forever (close() runs on that path), and a connection dropped during the handshake raises InvalidMessage, which was not retryable. The async dispatch also moved the cancel watermark a second time; removed.

Two smaller things I left alone: the code is None branch in error_from_close is dead now that close_code_and_reason always returns an int, and the wire-order test's final assertion holds whatever the order (it catches a regression only through the drain timeout). Happy to take either if you want them in this PR.

Full suite passes on Python 3.9, pydantic 1.10, pydantic 2 with and without pydantic-settings, and websockets 11 and 17; live against production for long-text speak() and a push-mode failure surfacing at the end of the block.