feat(tts): WebSocket session client

CLAUDE.md

@@ -54,9 +54,11 @@ aai.settings.api_key = "your-key"
 - `aai.DictationResponse` — Dictation result: `.text`, `.words` (`DictationWord` with `text` + `confidence`), `.confidence`, `.llm_response`, `.llm_error`, `.audio_duration_ms`, `.session_id`, `.request_time_ms`, `.sync_time_ms`, and `.final_text` (the LLM rewrite, falling back to `.text`)
 - `aai.TextToSpeech` — Text-to-Speech HTTP API: one text per request, audio out. Methods: `synthesize()` (complete audio as a `TTSResponse`), `stream()` (a `TTSStream` context manager yielding bytes as the API generates them), `close()` / `with` for a client built from `api_key=`. Host from `aai.settings.tts_base_url` (`https://tts.us.assemblyai.com`; set `https://tts.eu.assemblyai.com` for the EU zone)
 - `aai.AsyncTextToSpeech` — Asyncio counterpart of `TextToSpeech`; `synthesize()` is a coroutine and `stream()` returns an `AsyncTTSStream`. Owns an HTTP pool: use `async with` or `await aclose()`
+- `aai.TTSSession` — Text-to-Speech WebSocket session: one connection, text in as it is written, audio out. `with` connects and waits for `Begin` (`session.begin`); `speak(text)` yields one request's audio; `generate(text)` / `flush()` -> flush_id / `audio(flush_id)` for streamed-in text and pipelining (`audio()` with no id follows the request being written, from another thread); `cancel()` for barge-in (ends the iteration at once); `keep_alive()`; `on(TTSEvents.X, handler(event))` for `Begin`, `Audio` (switches audio from pull to push), `FlushDone`, `WordBoundaries`, `Cancelled`, `Termination`, `Error`. Exit sends `Terminate` and sets `session.termination` (`None` after a failure). Refusals and failures raise `TTSError` (`status_code` None, `request_id` = session id). Raw PCM/G.711 only; 2,000 chars per generate unless `Begin` reports another `limits.max_generate_text_length`; ~100 chars/s sustained. `token=` for temporary tokens
+- `aai.AsyncTTSSession` — Asyncio counterpart: `async with`, coroutine sends, `async for chunk in session.speak(text)`
 - `aai.TTSConfig` — Text-to-Speech options: `voice_id` (a library voice such as `aai-jane`, or a cloned voice by UUID or slug), `model_id` (`assemblyai-tts-1`), `language`, `sample_rate`, `encoding` (`aai.TTSEncoding`: `pcm_s16le` default | `pcm_mulaw` | `pcm_alaw` | `mp3`), `container` (`aai.TTSContainer`: `raw` default | `wav`), `bit_rate`, `extend_biometric_consent` (owner-only consent renewal of a cloned voice). Rejects unknown fields
 - `aai.TTSResponse` — Text-to-Speech result: `.audio` (bytes) plus the response headers: `.request_id`, `.region`, `.model`, `.voice_id`, `.language`, `.encoding`, `.sample_rate`, `.container`, `.bit_rate`, `.content_type`, `.voice_expires_at`, `.billed_characters`, `.audio_duration_ms`. `TTSStream.info` carries the same fields without `audio`; `audio_duration_ms` is `None` on a stream with audio (`0` when the text had nothing to speak)
-- `aai.TTSError` — Raised on any Text-to-Speech refusal: `.status_code`, `.error_code` (stable API code; `biometric_terms_not_accepted` 403 until the account accepts the biometric data terms in the dashboard, `auth_unavailable` 503 retryable), `.param`, `.request_id`, `.retryable`, `.retry_after`, `.errors`, `.body` (the whole envelope)
+- `aai.TTSError` — Raised on any Text-to-Speech refusal: `.status_code`, `.error_code` (stable API code; `biometric_terms_not_accepted` 403 until the account accepts the biometric data terms in the dashboard, `auth_unavailable` 503 retryable; on a session these arrive as close 3006 and 1011, with `status_code` None, the close code in `.body["close_code"]` and `.retryable` from the API's Error frame), `.param`, `.request_id`, `.retryable`, `.retry_after`, `.errors`, `.body` (the whole envelope)
 - `assemblyai.streaming.v3.RealTimeTranscriber` — Real-time streaming with event-based API (threaded)
 - `assemblyai.streaming.v3.AsyncRealTimeTranscriber` — Asyncio-native counterpart; same options/events
 - `aai.LLMGateway` — LLM Gateway client. Resources: `models.list()`, `chat.completions.create()` (incl. `stream=True`), `understanding.create()`/`.validate()`
@@ -479,6 +481,13 @@ with tts.stream("Hello.", telephony) as speech:
 
 # A cloned or library voice by id
 tts.synthesize("Hello.", aai.TTSConfig(voice_id="aai-alba"))
+
+# A WebSocket session for a voice agent: no handshake per sentence, barge-in with cancel()
+with aai.TTSSession(config=aai.TTSConfig(voice_id="aai-jane")) as session:
+    for sentence in llm_reply_sentences():
+        for chunk in session.speak(sentence):
+            play(chunk)
+# session.termination is the summary (None if the session failed); raw PCM/G.711 only, no mp3, wav or bit_rate
 ```
 
 ## LLM Gateway

README.md

@@ -1013,6 +1013,68 @@ asyncio.run(main())
 
 </details>
 
+<details>
+  <summary>Stream text in and audio out over a WebSocket session (voice agents)</summary>
+
+`aai.TTSSession` keeps one connection open and speaks text as it is written, so an LLM's reply can be voiced sentence by sentence with no per-request handshake. `speak()` sends one text and yields its audio; `generate()` / `flush()` do the same in two steps, so tokens can be sent as they arrive; `cancel()` is the barge-in. Leaving the `with` block ends the session cleanly and `session.termination` holds the summary (`None` if the session failed instead).
+
+A session streams raw `pcm_s16le` (the default), `pcm_mulaw` or `pcm_alaw` at the config's `sample_rate`; `mp3`, a `wav` container and `bit_rate` are HTTP-only and are refused. Chunk sizes vary and are not aligned to frames, so re-frame for a fixed-size sink (20 ms of 8 kHz mu-law is 160 bytes). `word_boundaries=True` and `inactivity_timeout=<seconds>` are arguments of the session, not of `TTSConfig`; word boundaries are off unless asked for. A session that fails raises `TTSError` wherever the app meets it first: the `audio()` iteration, the next send, or, for an app that pushes audio to a handler and never reads, the end of the `with` block.
+
+```python
+import assemblyai as aai
+
+aai.settings.api_key = "<YOUR_API_KEY>"
+
+with aai.TTSSession(config=aai.TTSConfig(voice_id="aai-jane")) as session:
+    for sentence in llm_reply_sentences():
+        for chunk in session.speak(sentence):  # raw 24 kHz 16-bit mono PCM
+            play(chunk)
+
+print("spoken:", session.termination)  # the session's summary
+```
+
+Streaming the text in as it is generated, with a barge-in:
+
+```python
+with aai.TTSSession(config=aai.TTSConfig(voice_id="aai-jane")) as session:
+    for token in llm_tokens():
+        session.generate(token)  # joined exactly as sent: include the spaces
+    request = session.flush()  # the request's flush_id; audio may start before this
+
+    for chunk in session.audio(request):
+        if user_started_talking():
+            session.cancel()  # ends this iteration at once; the server drops the rest
+            break
+        play(chunk)
+```
+
+Audio starts before `flush()` once whole sentences have arrived. To play it while the text is still being sent, read `session.audio()` with no id from another thread (or task): it follows the request being written, and ends at that request's `FlushDone`. Requests can be pipelined: call `flush()` again before reading the previous request's audio, then read each `audio(flush_id)` in order. Every `flush()` and every `cancel()` takes the next `flush_id`, so ids skip one after a cancel.
+
+Limits: a `Generate` frame takes at most 2,000 characters by default (`session.begin.configuration["limits"]["max_generate_text_length"]` has the limit in force; split longer text across calls), and the API accepts text at about 100 characters per second after an initial 1,500, holding at most 8,000 unspoken characters; sending far ahead of the speech ends the session with `input_rate_exceeded`, so pace text to the audio. The session ends after an hour, or after `inactivity_timeout` seconds (5 to 120, default 120) with no frame either way; send `keep_alive()` between sparse requests. Audio the SDK has already received does not hold the session open: the API counts idle time from the last frame it sent or received, so playing buffered audio, or waiting for the user, counts as idle. Send `keep_alive()` at least once per idle window during both. `session.begin` holds the `Begin` frame with the session id and the server's configuration and limits.
+
+Register handlers for the other frames with `session.on(event, handler)`, where `handler(event)` runs on the session's reader thread: `TTSEvents.Begin`, `TTSEvents.FlushDone`, `TTSEvents.WordBoundaries`, `TTSEvents.Cancelled`, `TTSEvents.Termination` and `TTSEvents.Error` (each delivers the matching `TTS...Event`, or the `TTSError`). Registering a `TTSEvents.Audio` handler switches audio from the pull iterators to the handler; `speak()` and `audio()` then raise. A handler may send (`generate()`, `flush()`, `cancel()`, `keep_alive()`, `close()`) but must not wait on the session: the reader that runs the handler is what `speak()`, `audio()` and `terminate()` wait for, so they raise `RuntimeError` there. Hand the work to another thread (or task) instead. Word timings (`word_boundaries=True`) arrive once per request after its `FlushDone`, usually after its `audio()` iteration has ended; their `start`/`end` count from the start of the session's audio, so subtract the event's `audio_start_ms` for a time within the request.
+
+A refusal of the session raises `TTSError` from the `with`; a failure mid-session raises it from the next `audio()` iteration or send, and reaches the `Error` handler. On a session `status_code` is `None` (there is no HTTP status), `request_id` is the session id and `body["close_code"]` is the WebSocket close code (`biometric_terms_not_accepted` is a `3006`, `auth_unavailable` a retryable `1011`); `retryable=True` means a new session can succeed, such as after an `inactivity_timeout`.
+
+Once the API has ended a session (an `inactivity_timeout`, say), the same `TTSSession` object opens a new one with `with session:` or `session.connect()`.
+
+For a session opened on a user's device, pass a temporary token instead of the key: `aai.TTSSession(token=token)` (mint it server-side on `https://token.assemblyai.com/v1/token?product=tts&expires_in_seconds=60`, which answers `{"token": ..., "expires_in_seconds": 60}`). The API spends the token when the session begins, so a token opens one session: to reconnect, mint a new token and build a new `TTSSession` with it.
+
+The asyncio twin is `aai.AsyncTTSSession`:
+
+```python
+async with aai.AsyncTTSSession(config=aai.TTSConfig(voice_id="aai-jane")) as session:
+    async for chunk in session.speak("Hello there."):
+        await play(chunk)
+    await session.generate("Streamed ")
+    await session.generate("in.")
+    request = await session.flush()
+    async for chunk in session.audio(request):
+        await play(chunk)
+```
+
+</details>
+
 <details>
   <summary>Renew a cloned voice's consent</summary>
 
@@ -2102,7 +2164,7 @@ All custom SDK exceptions inherit from `aai.AssemblyAIError`, except the streami
 | `aai.AssemblyAIError` | Base exception for all AssemblyAI errors. | `status_code: Optional[int]` |
 | `aai.TranscriptError` | Raised when a prerecorded transcript API call fails at the HTTP level or polling times out. | `status_code: Optional[int]` |
 | `aai.SyncTranscriptError` | Raised when synchronous transcription fails (`aai.SyncTranscriber`). | `status_code: Optional[int]`, `error_code: Optional[str]`, `retry_after: Optional[int]` |
-| `aai.TTSError` | Raised when a Text-to-Speech request fails (`aai.TextToSpeech`). | `status_code: Optional[int]`, `error_code: Optional[str]`, `param: Optional[str]`, `request_id: Optional[str]`, `retryable: Optional[bool]`, `retry_after: Optional[int]`, `errors: Optional[List[Dict[str, Optional[str]]]]`, `body: Optional[dict]` |
+| `aai.TTSError` | Raised when a Text-to-Speech request fails (`aai.TextToSpeech`) or a session is refused or fails (`aai.TTSSession`, where `status_code` is `None`, `request_id` is the session id and `body["close_code"]` is the WebSocket close code). | `status_code: Optional[int]`, `error_code: Optional[str]`, `param: Optional[str]`, `request_id: Optional[str]`, `retryable: Optional[bool]`, `retry_after: Optional[int]`, `errors: Optional[List[Dict[str, Optional[str]]]]`, `body: Optional[dict]` |
 | `assemblyai.types.RedactedAudioIncompleteError` | Raised when requesting redacted audio before processing completes. | `status_code: Optional[int]` |
 | `assemblyai.types.RedactedAudioExpiredError` | Raised when requesting redacted audio that has expired and is no longer available. | `status_code: Optional[int]` |
 | `assemblyai.types.RedactedAudioUnavailableError` | Raised when requesting redacted audio that is unavailable at the given URL. | `status_code: Optional[int]` |

assemblyai/__init__.py

@@ -31,15 +31,25 @@
 from .transcriber import Transcriber, Transcript, TranscriptGroup
 from .tts.v1 import (
     AsyncTextToSpeech,
+    AsyncTTSSession,
     AsyncTTSStream,
     TextToSpeech,
+    TTSAudioEvent,
+    TTSBeginEvent,
+    TTSCancelledEvent,
     TTSConfig,
     TTSContainer,
     TTSEncoding,
     TTSError,
+    TTSEvents,
+    TTSFlushDoneEvent,
     TTSResponse,
     TTSResponseInfo,
+    TTSSession,
     TTSStream,
+    TTSTerminationEvent,
+    TTSWord,
+    TTSWordBoundariesEvent,
 )
 from .types import (
     AssemblyAIError,
@@ -140,6 +150,7 @@ def __getattr__(name: str):
     "AsyncDictationTranscriber",
     "AsyncLLMGateway",
     "AsyncSyncTranscriber",
+    "AsyncTTSSession",
     "AsyncTTSStream",
     "AsyncTextToSpeech",
     "AsyncTranscriber",
@@ -211,13 +222,22 @@ def __getattr__(name: str):
     "SyncTranscriptResponse",
     "SyncWord",
     "Timestamp",
+    "TTSAudioEvent",
+    "TTSBeginEvent",
+    "TTSCancelledEvent",
     "TTSConfig",
     "TTSContainer",
     "TTSEncoding",
     "TTSError",
+    "TTSEvents",
+    "TTSFlushDoneEvent",
     "TTSResponse",
     "TTSResponseInfo",
+    "TTSSession",
     "TTSStream",
+    "TTSTerminationEvent",
+    "TTSWord",
+    "TTSWordBoundariesEvent",
     "TextToSpeech",
     "Transcriber",
     "TranscriptionConfig",

assemblyai/__version__.py

@@ -1 +1 @@
-__version__ = "1.7.0"
+__version__ = "1.8.0"

assemblyai/_audio.py

@@ -164,7 +164,10 @@ def _config_to_json(config: Any, *, exclude: Tuple[str, ...] = ()) -> Optional[d
 
     Returns: the JSON-ready dict, or `None` when nothing is set.
     """
-    data = config.dict(exclude_none=True)
+    if hasattr(config, "model_dump"):
+        data = config.model_dump(exclude_none=True)
+    else:
+        data = config.dict(exclude_none=True)
     for field in exclude:
         data.pop(field, None)
     return data or None

assemblyai/tts/__init__.py

@@ -7,21 +7,41 @@
 
 from .v1 import (
     AsyncTextToSpeech,
+    AsyncTTSSession,
     AsyncTTSStream,
     TextToSpeech,
+    TTSAudioEvent,
+    TTSBeginEvent,
+    TTSCancelledEvent,
     TTSConfig,
     TTSContainer,
     TTSEncoding,
     TTSError,
+    TTSEvents,
+    TTSFlushDoneEvent,
     TTSResponse,
     TTSResponseInfo,
+    TTSSession,
     TTSStream,
+    TTSTerminationEvent,
+    TTSWord,
+    TTSWordBoundariesEvent,
 )
 
 __all__ = [
+    "AsyncTTSSession",
     "AsyncTTSStream",
     "AsyncTextToSpeech",
+    "TTSAudioEvent",
+    "TTSBeginEvent",
+    "TTSCancelledEvent",
     "TTSConfig",
+    "TTSEvents",
+    "TTSFlushDoneEvent",
+    "TTSSession",
+    "TTSTerminationEvent",
+    "TTSWord",
+    "TTSWordBoundariesEvent",
     "TTSContainer",
     "TTSEncoding",
     "TTSError",